AI-POWERED EXam Builder
World's First AI Exam Builder
The AI-powered Exam Generator saves hundreds of hours per customer, significantly improving operational efficiency and minimizing investment in manual workflows.
This exploration helped us invest in more AI initiatives which has produced significant value to our customers and added revenue to our SaaS business.
The goal was to explore AI/ML opportunities within our advanced products, focusing on using AI to help create exams for our customers' training documents.
My Contribution
Product strategy, product leadership, product design, vendor research, user research
The Team
Product design manager (me), VP of AI/ML, 1 product manager, 1 machine learning engineer, 2 sr. product designers, 1 backend architect, 1 front end developer, and 1 researcher
Background
MasterControl is a leading Quality Management Platform. It helps more than 1000 highly regulated companies ensure high quality product development, manufacturing and compliance with digital process, data management, documentation and analytics.
As the Design Manager, I collaborated with our VP of AI/ML, product management, and the AI/ML team to evaluate vendor options. I also led our product designers in crafting user journeys, wireflows, and prototypes, which we developed into a Beta product for testing. Less than a month later, we released to full fledged product.
Responsibilities
Collaborated on crafting Generative AI requirements and seeking out vendor options
Collaborated with the AI team to make a final decision on our best vendor for our needs
Lead quantitive and qualitative research with our user researcher to understand user concerns with AI and to understand the problems we needed to solve with creating exams
Managed work and provided direction for Sr. product designers
Problems to Solve
Select the best Generative AI solution that meet our product requirements and business goals.
Address the challenges faced by training coordinators in creating exams and alleviate any concerns they have about using Generative AI.
Design an intuitive and streamlined experience for uploading, reviewing AI-generated exam questions, and integrating the exam into the system for SOP document training.
Actions Taken
Engage in customer conversations about generative AI to understand their needs and concerns.
Explored and evaluated LLM technologies to ensure customer needs and concerned were addressed.
We discovered that we needed to create an AI Exam Generator that was simple, secure, and maintained a “human in the loop” approach.
Personalization
Customizable settings allow users to tailor scheduling preferences and priorities to their unique needs.
Calendar integration
Seamless integration with popular calendar platforms such as Google Calendar and Outlook.
The resulting AI-powered exam builder offered a seamless user experience, allowing individuals to effortlessly create training exams.
Results
Our AI Exam Generator was a major success:
We first showcased it at our annual Masters conference, where the room was packed, with many users standing in the back to see the product in action.
Within less than 24 hours, our initial beta program received sign-ups from over 500 users, showing strong interest in the product.
Additional highlights
To date, our AI Exam Generator has been used over 200,000 times by our users.
The AI Exam Generator enabled the creation of exams in minutes, a task that would typically take a training coordinator hours to complete for a 200-page SOP document.
Thanks to the dedication and passion of the team, we’ve successfully integrated many other AI/ML innovations into our product ecosystem.
What I learned
Over the three months of researching, leading, and contributing to the creation of the AI Exam Generator, I gained several key insights:
While there is enthusiasm for generative AI in SaaS products, many companies using MasterControl want assurance that their sensitive company data remains secure and isn’t exposed to other businesses.
Generative AI can be a powerful tool, especially for saving time on repetitive tasks like creating exam questions for training, but it’s crucial that AI-generated content undergoes human review (human-in-the-loop) to catch any potential inaccuracies or hallucinations.
AI and LLM technology are still in their early stages, but the future is promising for how these technologies can significantly enhance our processes.