The Classroom Product’s History

Classroom is Top Hat’s cornerstone product that helped catapult them towards product-market fit, high growth, and venture capital. Top Hat has displaced existing hardware clickers, in favour of using smartphones and web apps. To date, millions of students have used Top Hat in higher education institutions across the United States, Canada, and the UK. From the sound of it, everything in the product must be peachy, right?

When I joined in 2017, Classroom had already been built. The issue was that crucial parts of the user experience had been neglected in a mad scramble for growth. This lead to retention problems because the feature set had become complex, bloated, and basic expectations of usability weren’t being met. The same sad story has befell many products. We weren’t unique. We needed to get back to basics and focus on the core experience.

Top Hat Classroom is relatively simple. A professor asks students a question during their lecture, and students are prompted to answer it on their phone or laptop, while the results display live in class. Professors use this interaction to engage and assess students’ comprehension in real time, so they can pivot their teaching on-the-fly, and generate discussion in class.

Success for both users depends on a student’s ability to answer questions and receive grades for participating and getting correct answers.

As long as students are able to meet these basic expectations, both users are relatively happy, right? Well, if the hot water’s barely running in your hotel, and you’re not sure how to work the faucet, there’s still a problem. For us, our Classroom apps worked, but it was missing the mark on some basic expectations.

My Role

How my project teams were organized

At the time, I worked on Top Hat Classroom leading design on iOS, Android, and two web teams at a ratio of 1 designer : 2 product managers : 14 developers. I lead product design from ideation to implementation.

For this specific project, I was focused primarily on iOS.Over 50% of student users submit their answers on iOS devices, so we wanted to drive the biggest impact by starting on iOS. Many of the iterative changes were adopted in the web app after being rolled out on iOS.

We were able to get laser sharp focus on the problem space thanks to our super secret weapon Jeremy, our user researcher. He helped connect us with professors for direct observation in classrooms, so we weren’t making decisions based on assumptions. My product manager Blanca was hyper focused on finding problems, and steered our ship in the right direction.

The Classroom Experience

We started to attend classrooms and perform observational research. We’re lucky that for our use case, it’s relatively easy to sit in a classroom and observe our product in action. What’s more, is that there was a professor at University of Toronto who was using Top Hat only 10 minutes away from our office! There’s no better way to build empathy than to be a part of the experience.

Checkpoints & Quizzes

One of the most common use cases is for professors to present slides, and ask questions throughout their lecture. They do this to check comprehension and pivot what they need to teach to students in real time. Internally, we called these comprehension checks checkpoints.

How a professor checks comprehension throughout a lecture

Another common use case is for professors to present a 4-5 questions quiz at the beginning of class, to check if students did their homework, and understand their level of comprehension. This activity helps them choose which topics they need to go over again in class.

To enable these types of tasks, students need to be able to answer questions on their device without issues. If there are any problems while asking questions, it disrupts the learning flow, and interrupts a professor’s classroom goals.

Asking Questions in the Classroom

The user task flow for asking and answering in the classroom

In the “happy path” scenario, professors open a question to gather responses from students. Students then open the question on their device, choose a response, and submit their answer. It’s important that their submission is received and grades are recorded.

While this flow might seem simple, what types of issues cause an “unhappy path?” How might we handle cases when there is low network connectivity? What happens when multiple questions are asked at once? How can we encourage students to respond to questions faster, so that we don’t waste valuable class time? How can we make things work the way students expect them to?

Problems

Someone, somewhere, probably in a Medium article, said “Good designers fall in love with the problem, not their solutions.” Our team’s firm grounding in research made it easy to discover problems. From the observational research, event tracking data, and design heuristics we were able to have productive discussions to prioritize which problems we would tackle together.

This diagram helped align our team around what needs to focus on

For students in the classroom we discovered the following needs:

1. Open and answer questions faster

   • This keeps the teaching flow moving along inside of the classroom.
   • We noticed some students missed the submit button completely!
   • Some students lagged behind and missed questions

2. General usability improvements

   • Students didn’t understand when a question was “closed” or “paused” by a professor. The UI didn’t reinforce what was happening.
   • iOS navigation patterns didn’t feel natural, and caused usability issues. Users would view slides and expect to swipe from slide to slide. There was a lot of unnecessary context switching between views, and confusion while navigating.
   • The UI for questions was inconsistent, did not comply with Accessibility guidelines.
   • The language used to communicate states were different in the professor UI, student web UI, and student mobile UI
   • Style guidelines, and interaction patterns were not well defined or documented.

3. Technical problems caused reliability/perceived reliability issues

   • Reliability issues caused a lot of noise and confusion in classrooms. Students were flustered and confused when their answers didn’t submit properly.
   • We noticed sometimes students were frustrated and perceived that their answer hadn’t been submitted, when really our UI didn’t clearly communicate what was happening.
   • We save student responses when there is poor network connectivity, but we weren’t doing a great job of letting users know that we had done so.

4. Better assurance of answer submission

   • While submitting, students were unclear if their answer had been submitted. The UI didn’t instill high user confidence.
   • Answer resubmission was unclear. If users changed their answer, they had no idea which answer had been recorded. We noticed students mashing the submit button multiple times to ensure their answer went through.
   • Unsure if they got grades for their answer. We saw that students would immediately check the gradebook after a question was closed, sometimes to their detriment, as they would miss a follow up question as they were busy checking the gradebook.

Solutions

This is the exciting part of design.

The Product Dictionary

The first step was to start to align around language. Adam, another designer on my team, had lead a project on improving the professor experience in Classroom. We came up with a Product Dictionary the would help to define what language conventions we’d use in our designs.

The beginning of our Product Dictionary

We gathered common terms that we would use to describe actions inside of our product. For example, do you “send” a student’s answer, or do you “submit” a students answer? Aligning our language conventions kept our products in sync, and made the experience less confusing for users.

2. Improved Questions & Slides on iOS

I worked on 3-4 different prototypes to help solve some of these issues. To generate ideas, I planned a sketching session with the mobile development team, and a separate session with the design team. I presented some of the problems we identified, we looked at how slides and questions work inside of the app currently. After sketching, I gathered these ideas to put together initial prototypes using Invision to create video clips I could start to send to colleagues on Slack to help start prioritization discussions.

This is a small sample of some of the UI iterations we explored

Sometimes you need to dream big and look at the ideal situation, even if it won’t be fully implemented. This can help expand the realm of possibility, and help get better compromises that put greater focus on the user. If you think small, ideas can sometimes get squeezed and scoped down into nonexistence, and in the end the user suffers.

What We Built

After discussing, prioritizing, and thinking pragmatically about what we could build, we settled upon a design that was still user focused. We prioritized improving the questions experience, since it’s the primary task students need to accomplish. The ideas we looked at around optimizing the slides experience were great, but viewing slides is a secondary task, and is used less on mobile due to the screen size.

A walkthrough of the questions UI flow View the Flow made in Overflow

A piece was still missing. We couldn’t just forget about slides. After all, 50% of the professors that use Classroom present slides in Top Hat. Though, we didn’t have the resources to rebuild how slides worked, so we found a simple compromise to reuse the existing slide carousel functionality. I believe that this compromise actually made the design simpler and clearer to understand for users, and still keeps questions at the forefront.

Caption: Our implementation using the existing slides carousel

User Testing & Roll Out

Another advantage to having access to classrooms, was that we could mitigate product risk by testing new features in classrooms. After building, we loaded a build of the improved classroom experience onto our QA test devices, and brought them to a classroom.

We took videos of students using our app live in a real classroom. This helped us to scoop up some bugs and interaction design issues, allowing us to be more confident to roll it out. From a qualitative standpoint, the design was a success.

To date, we have released it to 100% of users, and it has landed successfully, without any major issues. As the semester progresses, we’re learning more and will have more measurable data.

This is one piece of the retention puzzle. There were other parts of the product that we improved, as well as worked on other people focused strategies to improve retention. Though our current retention rate on Classroom is much higher, it’s hard to attribute it to only one action!