For: DPG Media – CRM team
Project Goal
This project aimed to enable CRM colleagues to make sense of the abundance of feedback being collected via E-mail. Ultimately helping them spot potential e-mail (content/technical) issues and discover new opportunities. I extracted user feedback from Excel files, classify the feedback into relevant topics, determine the sentiment (positive, negative, or neutral) of the responses, and then save the processed data into a CSV file. This helps in quickly understanding the main themes and feelings within large sets of user comments.
How it was built
The process began by loading an Excel file, identifying columns containing user feedback, and then creating a ‘prompt’ column that combines questions and answers. Each row was then processed using OpenAI’s GPT-3.5-turbo model to classify the text into pre-defined topics and subtopics, and determine the sentiment of the user’s answer. The results, along with the original data, were then merged and cleaned. A custom function was used to ensure that each row had a corresponding topic and sub-topic, even if multiple were present, achieved through exploding the data into multiple rows. Finally, the data was saved to a CSV file.
Notable features include:
- Dynamic identification of feedback columns based on numeric prefixes.
- Integration with OpenAI’s GPT model for complex text analysis, accessed through their Python API.
- A Python Library used to remove any potential PII related data.
- Custom prompt engineering to ensure the model responds with the correct JSON format.
- Use of pandas to handle data manipulation and transformation.
- A custom function to split and explode the dataframe.
- A custom function to clean and format the topic and sub-topic columns.
Technologies used
- Python
- pandas
- OpenAI API
- json
- httpx
- asyncio