On-demand mock interviews powered by AI
Project Overview
About Career Chat
Career Chat is a web application that allows users to practice job interviews and receive feedback, all facilitated by ChatGPT. I teamed up with two Product Managers and a Full-stack Engineer to redesign the beta version of the product in response to initial customer feedback.
Since the release of self-serve account creation, new user sign-ups on the landing page have increased.
Boost the conversion rate of new users to complete one or more practice interviews in the same session.
Problem
Solution
Research
Survey to shape the persona
A survey was distributed to 105 people on the waitlist for Career Chat, resulting in responses from 45 individuals.
Key findings from Career Chat users
User interviews to identify pain points about mock interviews
I conducted five 30-minute interviews with participants who have practiced interviewing in the past six months. Two of these participants are Career Chat users.
Affinity Map to identify patterns
Key findings about mock interviewing
Defining the problem
The problem
Job to be done
Problematic user flow from marketing site
Ideating solutions
Adaptive layout requirement
Device | Size | Why it’s important |
Laptop | 1512 x 982 | Majority of mock & real interviews |
Half-screen | 256 x 982 | Splitting screen between interview & notes |
Mobile | 393 x 852 | Completing interview after signing up or practicing on the go |
Constraints for redesign
- No updates to back-end prompts sent to ChatGPT based on the user’s input
- No updates to the interview questions or feedback generated by ChatGPT.
- Keep costs down by minimize number of API call made to ChatGPT
Sketching ideas to determine responsive layout
I drafted several layouts to identify the best format for both wide and narrow viewports.
Usability test reveals opportunity for quick iterations
I distributed 65 unmoderated usability tests using Maze, which resulted in 14 respondents. These tests were conducted to evaluate the usability of the redesigned product. The participants comprised both users and non-users of Career Chat.
Check out the Usability Test here
Information on four categories to help user understand rating
Nudge user to click primary button to answer interview question
Usability test success metrics
Usability Task 1: Start a new practice job interview
Usability Task 2: Review your practice interview feedback
Task | Metric | Detail | Results |
Task 1 | Task completion | Success or Failure | 87.5% |
Task 1 | CSAT
(1 = Strongly dissatisfied and 5 = Strongly satisfied) | How satisfied are you with the practice interview experience? | Avg. 4.9 |
Task 2 | Task completion | Success or Failure | 100% |
Task 2 | CSAT
(1 = Strongly dissatisfied and 5 = Strongly satisfied) | How satisfied are you with the type of interview feedback? | Avg. 4.9 |
Task 2 | Multiple Choice (Just right, too short, too generic, too long, too complex, other) | What are your thoughts on the type of feedback that was provided? | 6 = Just right
1 = Too complex |
Overall | Likert Rating
(1 = Strongly Disagree and 5 = Strongly Agree) | How would you rate the following statement:
After completing the practice interview, I feel more prepared for the real interview. | Avg. 4.4 |
High-Fidelity Designs
Responsive design implementation
Below are the key screens of the redesigned Career Chat web-app, along with highlights of my design decisions.
Updated Home Screen
Job to Be Done: I want to know what to expect during the mock interview
- Learnability: To orient user on what to expect during the mock interview, I redesigned the home page.
- Primary button hierarchy: On mobile, I decided to elevate the hierarchy of the “Get Started” button on the page, ensuring it appears above the fold and does not go unnoticed.
Completing the mock interview intake
Job to Be Done: I want to specify the role for the mock interview
- AI-based product design patterns: To specify the role and job description, I designed quick suggestions and a free form option. This design pattern, frequently used in other AI-based products, aids in preventing user input errors.
- Distinct User Experience for Setup: To distinguish the interview intake from the actual mock interview, I ensured the user experience and user interface were distinct, while maintaining a similar tone and style in the copy.
Answering the interview question
Job to Be Done: I want to answer a role specific interview question that feels realistic
- Emphasized through Simplified UI: I removed several unnecessary visual elements and controls to maintain the user's focus on answering the interview question.
- Replacing text-to-speech visual: I added a placeholder for the transcription and a recording animation. This provides users with a visual indication that the system is functioning. Before, they were shown the speech-to-text transcription, which was distracting.
- Users Adjust Screen During Interviews: User research revealed that users often resize their interview window on their laptops to accommodate notes. Therefore, I've designed a responsive version of the interview layout.
Practice interview feedback
Job to Be Done: I want to receive helpful interview feedback, so that I can improve my interviewing skills.
- Emphasis on one feedback at a time: Not in scope was altering the raw, text heavy ChatGPT output. Therefore, I leveraged negative space and proximity to display the interview feedback for one question at a time.
- Scannable feedback: To help users quickly understand their performance, I introduced a three-tier rating scale for their response clarity, structure, role fit, and differentiation. We were previously not making use of this information sent back by the ChatGPT API.
Setting user expectations through helpful loading screens
Job to Be Done: I want to be informed about what is about to happen, so that I feel in control.
- Setting Expectations: Career Chat users expressed surprise over the use of sound and speaking in the mock interview. They were also uncertain about the start and end times of the interview. To address this, I designed clear loading screens to help users prepare. An additional benefit is that this allows the system time to make API calls without compromising the user experience.
- Delightful Brand Moments: Despite keeping the branding minimal, I incorporated lively animations to motivate users during the mock interview process, which can be stressful for many.
- Open source component library optimized for fast development and easy maintenance
- UI components are flexibly designs and accessible
Project Outcome
Measuring success
Metric | Baseline | 90-day goal |
Increase users rating of preparedness for the real interview | Avg. 4.4 | Avg. 4.6 |
Decrease bounce rate of users signing up via mobile | 10% | 6% |
Increase number of successfully completed mock interviews in first session | instrument tracking | 60% |
Future iterations
- Simplify interview feedback further improve scalability of the information
- Combine Situation and Task section on feedback screen to reduce visual density
- Consider prompting user to complete new interview instead of redoing the interview
Learnings
- Responsive design means more than just designing for different devices. In this case, I learned that users change the screen size on their laptop to view notes while interviewing.
- Unmoderated usability tests increases participant reach but findings lack fidelity on the “why” behind answers and actions.
- Leveraging the Radix design system allowed me to have a solid, modular foundation that I could adapt for Career Chat’s UI kit.