base on An excellent Weibo public opinion analysis and prediction system. <div align="center">
<!-- # π Weibo Public Opinion Analysis System -->
<img src="https://github.com/666ghj/Weibo_PublicOpinion_AnalysisSystem/blob/main/static/image/logo_compressed.png" alt="Weibo Public Opinion Analysis System Logo" width="800">
[](https://github.com/666ghj/Weibo_PublicOpinion_AnalysisSystem/stargazers)
[](https://github.com/666ghj/Weibo_PublicOpinion_AnalysisSystem/network)
[](https://github.com/666ghj/Weibo_PublicOpinion_AnalysisSystem/issues)
[](https://github.com/666ghj/Weibo_PublicOpinion_AnalysisSystem/graphs/contributors)
[](https://github.com/666ghj/Weibo_PublicOpinion_AnalysisSystem/blob/main/LICENSE)
[English](./README.md) | [δΈζζζ‘£](./README-CN.md)
</div>
**Weibo Public Opinion Analysis and Prediction System** is a **social network public opinion analysis system** designed to monitor, analyze, and predict public opinion trends on social media platforms such as Weibo. This system leverages deep learning, natural language processing (NLP), and machine learning technologies to extract valuable public opinion information from vast amounts of social media data, helping governments, enterprises, and other organizations promptly understand public attitudes, respond to emergencies, and optimize decision-making. π
Through powerful data collection and processing capabilities, the Weibo Public Opinion Analysis and Prediction System achieves real-time data collection, sentiment analysis, topic classification, and public opinion prediction, ensuring that users can obtain accurate and comprehensive insights into public opinion in the complex and changing social network environment. The system adopts a modular design, making it easy to maintain and expand, aiming to provide users with an efficient and reliable public opinion analysis tool, assisting various organizations in making informed decisions in the information age.
## β¨ Features
- **Real-time Data Collection**: Utilize web scraping technologies to obtain user-generated content from social platforms like Weibo in real-time.
- **Data Cleaning and Processing**: Preprocess collected data, including tokenization, removal of stop words, emojis, and URLs.
- **Topic Classification**: Automatically classify posts and comments into topics using machine learning and natural language processing techniques.
- **Sentiment Analysis**: Analyze the sentiment orientation (positive, neutral, negative) within texts to understand public emotions.
- **Public Opinion Monitoring and Prediction**: Monitor changes in public opinion in real-time and predict future trends based on historical data.
- **Data Visualization**: Display analysis results through charts and graphics for easy understanding and decision-making.
- **User Management**: Provide user registration, login, and session management features to ensure system security and personalized services.
## π Getting Started
Follow the steps below to run the project on your system.
### Prerequisites
- [Python](https://www.python.org/) 3.7 or higher
- [MySQL](https://www.mysql.com/) Database
- [Conda](https://docs.conda.io/en/latest/) (optional, for environment management)
- A valid Weibo account (for data collection)
### Installation Steps
1. Clone the repository:
```bash
git clone https://github.com/666ghj/Weibo_PublicOpinion_AnalysisSystem.git
cd Weibo-Public-Opinion-Analysis-System
2. Create and activate a virtual environment (optional):
```bash
conda create -n weibo_opinion_analysis python=3.8
conda activate weibo_opinion_analysis
```
3. Install dependencies:
```bash
pip install -r requirements.txt
```
4. Configure the MySQL database:
- Run `createTables.sql` to create the necessary database tables.
- Modify the database connection settings in `config.py` to match your MySQL configuration.
5. Start the Flask application:
```bash
python app.py
```
6. Access the application: Open your browser and navigate to http://localhost:5000 to use the system.
## π οΈ Technology Stack
The Weibo Public Opinion Analysis and Prediction System employs a range of modern technologies to ensure efficiency and scalability:
- **[Flask](https://flask.palletsprojects.com/en/stable/)** - A lightweight web application framework.
- **[MySQL](https://www.mysql.com/)** - A relational database used to store collected and processed data.
- **[Scrapy](https://scrapy.org/)** - A powerful web scraping framework used for data collection.
- **[Jieba](https://github.com/fxsjy/jieba)** - A Chinese text segmentation tool used for text preprocessing.
- **[SnowNLP](https://github.com/isnowfy/snownlp)** - A Chinese natural language processing library used for sentiment analysis.
- **[BERT](https://github.com/google-research/bert)** - A pre-trained language model used for topic classification.
- **[Pandas](https://pandas.pydata.org/)** - A data analysis and manipulation library.
- **[Matplotlib](https://matplotlib.org/)** - A data visualization library.
- **[Scikit-learn](https://scikit-learn.org/)** - A machine learning library used for model training and evaluation.
- **[TensorFlow](https://www.tensorflow.org/)** ζ **[PyTorch](https://pytorch.org/)** - Deep learning frameworks used for advanced model development.
## π€ Contribution
We welcome your contributions! Follow the steps below to participate in the project:
1. Fork this repository.
2. Create your feature branch (`git checkout -b feature/your-feature`).
3. Commit your changes (`git commit -m 'Add some feature'`).
4. Push to the branch (`git push origin feature/your-feature`).
5. Open a Pull Request.
Please ensure that all tests pass before submitting and follow the project's coding standards.
## π License
This project is licensed under the [GPL-2.0 License](https://github.com/666ghj/Weibo_PublicOpinion_AnalysisSystem/blob/main/LICENSE) - see the [LICENSE](https://github.com/666ghj/Weibo_PublicOpinion_AnalysisSystem/blob/main/LICENSE) file for details.
## π Show Your Support
If you like this project, please give it a star β on [GitHub](https://github.com/666ghj/Weibo_PublicOpinion_AnalysisSystem)!
## π« Contact Us
If you have any questions or suggestions, feel free to contact us through the following methods:
- GitHub Issues: [Create a new issue](https://github.com/666ghj/Weibo_PublicOpinion_AnalysisSystem/issues)
- Email:
[email protected]
## β¨ Contributors
Thanks to the following contributors:
[](https://github.com/666ghj/Weibo_PublicOpinion_AnalysisSystem/graphs/contributors)
", Assign "at most 3 tags" to the expected json: {"id":"12461","tags":[]} "only from the tags list I provide: [{"id":77,"name":"3d"},{"id":89,"name":"agent"},{"id":17,"name":"ai"},{"id":54,"name":"algorithm"},{"id":24,"name":"api"},{"id":44,"name":"authentication"},{"id":3,"name":"aws"},{"id":27,"name":"backend"},{"id":60,"name":"benchmark"},{"id":72,"name":"best-practices"},{"id":39,"name":"bitcoin"},{"id":37,"name":"blockchain"},{"id":1,"name":"blog"},{"id":45,"name":"bundler"},{"id":58,"name":"cache"},{"id":21,"name":"chat"},{"id":49,"name":"cicd"},{"id":4,"name":"cli"},{"id":64,"name":"cloud-native"},{"id":48,"name":"cms"},{"id":61,"name":"compiler"},{"id":68,"name":"containerization"},{"id":92,"name":"crm"},{"id":34,"name":"data"},{"id":47,"name":"database"},{"id":8,"name":"declarative-gui "},{"id":9,"name":"deploy-tool"},{"id":53,"name":"desktop-app"},{"id":6,"name":"dev-exp-lib"},{"id":59,"name":"dev-tool"},{"id":13,"name":"ecommerce"},{"id":26,"name":"editor"},{"id":66,"name":"emulator"},{"id":62,"name":"filesystem"},{"id":80,"name":"finance"},{"id":15,"name":"firmware"},{"id":73,"name":"for-fun"},{"id":2,"name":"framework"},{"id":11,"name":"frontend"},{"id":22,"name":"game"},{"id":81,"name":"game-engine "},{"id":23,"name":"graphql"},{"id":84,"name":"gui"},{"id":91,"name":"http"},{"id":5,"name":"http-client"},{"id":51,"name":"iac"},{"id":30,"name":"ide"},{"id":78,"name":"iot"},{"id":40,"name":"json"},{"id":83,"name":"julian"},{"id":38,"name":"k8s"},{"id":31,"name":"language"},{"id":10,"name":"learning-resource"},{"id":33,"name":"lib"},{"id":41,"name":"linter"},{"id":28,"name":"lms"},{"id":16,"name":"logging"},{"id":76,"name":"low-code"},{"id":90,"name":"message-queue"},{"id":42,"name":"mobile-app"},{"id":18,"name":"monitoring"},{"id":36,"name":"networking"},{"id":7,"name":"node-version"},{"id":55,"name":"nosql"},{"id":57,"name":"observability"},{"id":46,"name":"orm"},{"id":52,"name":"os"},{"id":14,"name":"parser"},{"id":74,"name":"react"},{"id":82,"name":"real-time"},{"id":56,"name":"robot"},{"id":65,"name":"runtime"},{"id":32,"name":"sdk"},{"id":71,"name":"search"},{"id":63,"name":"secrets"},{"id":25,"name":"security"},{"id":85,"name":"server"},{"id":86,"name":"serverless"},{"id":70,"name":"storage"},{"id":75,"name":"system-design"},{"id":79,"name":"terminal"},{"id":29,"name":"testing"},{"id":12,"name":"ui"},{"id":50,"name":"ux"},{"id":88,"name":"video"},{"id":20,"name":"web-app"},{"id":35,"name":"web-server"},{"id":43,"name":"webassembly"},{"id":69,"name":"workflow"},{"id":87,"name":"yaml"}]" returns me the "expected json"