I then turned to another area that takes a lot of time for our team. We provide a service where we will take survey comments, attempt to find themes and then report back to clients the themes and comments associated with them. It’s time consuming and also subjective. What one person thinks applies to theme A, another may attribute to theme B. It’s hard to go through 5-10,000 comments and capture the themes correctly and consistently. Given that, I built a machine learning pipeline that attempts to cluster segments of texts and predict a name for the cluster in the form of a theme.
By taking that manual labor out, we use our time to review and correct the theme names that don’t really capture the spirit of the comments. The net result is a huge time savings and the ability to better relate our findings through visualizations our team has created—like word clouds. We can then provide this service to more clients who will be better informed of how to take action in support of their staff.
With regard to the surveys, people always ask if anybody is looking at their comments and if their comments are anonymous. I hope new AI-powered tools like those we have now and are creating will make people realize that we definitely want to hear and understand their feedback. We are committed to treating comments with the utmost respect and ensure that people’s anonymity is never jeopardized throughout the process of creating meaningful feedback for leadership. The new AI-powered tools and our humans-in-the-loop are helping all of that happen more seamlessly.
That was all pre-Chat GPT—what are you all working on now?
There’s a lot of momentum to use the models behind ChatGPT to build sophisticated custom applications. Here’s one example. Our clients always have questions about the data that may not be captured in the report we provide. It would be very time consuming—or maybe even impossible—to build a report with everything they could possibly ask for, so we hope to deploy in our analytics product the ability to use a chat bot that will allow clients to talk with their survey data. They’ll be able to ask questions like, What are the five things people are most unhappy with? What are the top three things we can do to improve? What are people most positive about? The chat bot will be able to provide answers on the basis of survey comments along with citations so it can be fact-checked. It is also being built so that clients can provide feedback on the chat bot responses that can in turn be used to monitor and improve the model as time goes on.
We are also working on a product to dynamically generate reports based on a user’s description of what they want to see. Currently, if someone from Tritonlytics wants to know how a specific survey did, the developers and the rest of the staff can create that, but we are looking at a way to create reports by simply having project managers talk to an application and describe what they want. The application will generate the code to create that report without having to go to a developer.
What do you think about the future of AI?
In my experience, AI works best when humans are in the loop. You automate the tedious tasks so people can focus on the important things. This makes us more efficient and improves the quality of our output.
I foresee a definite uptick in the number of AI-powered applications within our department and the university over the next three to five years. Right now, I’m convinced that most individuals don’t really understand what AI is, and don’t know whether to be afraid of it or even how it might be used. I hope to be part of clearing up those misconceptions and help others figure out how to work with AI, here at UC San Diego and beyond.
