SENTIMENT ANALYSIS - DATA SCRAPPING: GENERATIVE AI

For some, Generative AI popularly known as AI is the best technological innovation yet, for others, AI is a scary Grinch that is here to steal our jobs and take over the world. These are two extremes, kindly explore my viz to see the in between, and what people generally think of 'AI'.

This project is important to me because it taught me data scrapping, sentiment analysis, “viz in tooltip” on Tableau, and new standardization techniques in Python.



Firstly, I scrapped the comment section of a Reddit post about general opinions of Generative AI, using Instant Data Scrapper.

I then imported the unstructured data into Jupyter Notebook for basic analysis like data structure, null values count, and data types. I standardized, wrangled, and transformed the data into a structured data. I did sentiment categorization but could only achieve less than 15% accuracy. So, I exported the cleaned data into Excel for manual categorization to ensure above 85% accuracy.

In excel, I created 2 new columns to drill properly into each opinions.

Column 1 : Reason: This column was created to give reasons for opinions.

Column 2 : Overall sentiment: This column categorizes each opinion into positive, negative, or neutral.

I proceeded to use tableau for visualization which gives insight into:

- Post engagement over time.

- ⁠Overall sentiment.

- ⁠Insight into categorized opinions with “viz in tooltip” implemented to drill into each category which helps to understand the underlying reasons for sentiment.

Interact with viz here: https://public.tableau.com/app/profile/deborah.david6433/viz/OpinionsonGenerativeAi/Dashboard12?publish=yes

I am happy with this project because I learnt new things, it helped to showcase my analytical and problem solving skills and it gave me a lovely reminder about how several truths can coexist, and that they are all valid in their unique ways.

I could be your Sentiment Analyst any day. Kindly connect with me for opportunities. Thank you.