SNS Data Analysis System
Dashboard screen
Brief information:
In this case study, BHSoft helps customers to build a SNS system to predict and tackle natural disasters.
The Client:
The client is a Japanese firm who provides manufacturing solutions to manufacturing industry. In particular, they develop various automatic design systems based on CAD, developing structural analysis and integration system for construction/ civil engineering fields, providing environmental evaluation simulation as well as GIS asset management solution.
This client has been loyal customer of BHSoft for a certain period of time and they plan to continue to co-operate with BHSoft.
Client business needs:
- In this project, BHSoft helps the client to build a SNS (social network service) data analysis system to forecast, prepare and forewarn possible occurrence of natural disasters based on data provided by social media channels. In this case, Twitter is the selected social media channel.
The project features:
BHSoft has years of experiences working with Geographical Information System (GIS). This time we build a geospatial data information system for the purpose of monitoring and evaluating information on natural disaster. This information will be showed on the dashboard screen.
The SNS system monitors predefined keywords and then list all essential statistics on the dashboard. The keywords are the name of natural disasters along with all related information.
Main features:
Dashboard menu
Dashboard screen of SNS system:
From the dashboard, users can have an overview of data keyword situation. Specifically,
- Keyword ranking to show the list of most frequently tweeted keywords.
- Keyword chart to show list of the most frequently tweeted keywords within the most recent hour.
- Map to show location of tweeted keywords along with their detailed information (time and location of the tweets, who tweets it and what information is included in those tweets).
Dashboard screen
Tweet Search Screen:
On Tweet Search Screen, users own the possibility to monitor the keyword situation directly through attaining data from Twitter. Users have the option to choose date and time, input keyword in ‘Keyword’ field to choose which disaster situation they would like to follow. In case users want to search multiple keywords, they can separate the keywords by using semicolon. The ‘Exclusion’ field allows users to exclude any keyword they are not interested in. In addition, users can also choose which language they would like to view the map and tweet information in. After filling all the required information, users are able to view Tweets information of the searched keyword: who tweets it, time and location of the tweet as well as detailed information in the tweet.
Users also have option to export data in CSV form.
Tweet Search Screen: Map
Tweet Search Screen: Tweet details
Statistic Screen:
Similar to Tweet Search, here users have access to choose date and time, keyword to search, exclusion keyword and language option.
From this screen, users can view which keyword is ranking first, number of recorded times along with the percentage.
Users also have the option to list output, create a graph and export data in CSV form.
Statistic Screen: Rank
Statistic Screen: Graph
Keyword Management Screen:
Maintenance menu
This screen is available for Admin User only. With the Standard User, this function will be disabled. With this function, Admin User can manage Keyword, Keyword threshold, Exclusion Keyword.
Keyword Management screen
In this screen, Admin User can manage all monitoring keywords. Admin User is also able to create a new Keyword, edit or remove added keyword and their keyword threshold. Defined keywords will be listed in the Keyword List.
The keyword list lists down all defined Keywords. Admin User can determine which keyword will be monitored on History chart (in Dashboard screen) by History checkbox. Admin User is also able to modify also the keyword by clicking in Edit button.
Keyword Exclusion Screen:
Next is Keyword Exclusion Management Screen. In this screen, Admin User can manage all exclusion keywords. That means Admin User can create a new Keyword, edit or remove added exclusion keyword. Defined exclusion keywords will list down in the Exclusion Keyword List.
The Keyword Exclusion list lists down all defined Exclusion Keywords. Admin User can modify also the exclusion keyword by clicking in Edit button.
Keyword Exclusion Screen
User Management Screen:
This screen is available for Admin User only. With the Standard User, this function will be disabled. With this function, Admin User can manage Standard User: create a new Standard User, edit or delete one or multiple users in the User List.
User Management Screen
In this screen, Admin User is able to manage all Users. That means Admin User can create a new user, edit or remove added users. User List lists down all defined users. Admin User can modify also the users by clicking in Edit button.
Add new user
Edit user
Challenges:
Twitter data is too big and overloaded so the system is slowed down.
Solution:
BHSoft has many years of experience working with GIS so we figured out the solution immediately. We utilized big data analysis to optimize Twitter data so the whole project went smoother and the system runs seamlessly.
Technology used:
- Angular 8 for front-end part of the system
- Node.JS 10.x for back-end part of the system. Application server runs on Node.JS 10.x
- A web server using Nginx to serve Frontend Application
- Nginx as a web server and proxy
- MariaDB to store system data (user, keywords and settings)
- MongoDB to store Twitter data
- Use Open Layer to display Geolocation data
- Use base map data from Open Street Map
Communication Channel:
- Communicating via Skype
- BHSoft members going on-site at partner company
Methodology used:
- Agile/ Scrum
Project size:
4 MMS
Duration:
4 months since mid-July till October 2019
