Teddy Billings - Portfolio

Empathize

Picking up market research

I started off by looking previous research that showed social studies teachers are creating their own coursework to fit state standards. The requirement is that we need to fit this new standard and capitalize on the first-to-market advantage

Focus groups

Now that we knew there was quantitatively a need for a new product that would fit state standards, we wanted to know more about the specific problems they were facing with products today. We held a few focus groups to get some more insight

What we did:

8 Focus groups
56 Social studies educators
2 Prototypes presented
Remote, moderated, web cam

What we learned:

Teachers want to mix lecture and active learning
Students are coming in unprepared
Primary sources are cruical to History
Teachers vary in how the feel about content difficulty

Define

User Journey

Based off of the insights from the focus group and the new C3 standard, the team and I came up with a “Learning Journey” for the student (persona Adam) who could meet the learning objectives.

Competitive Analysis

There are a ton of products that inspired us. To get a better idea of our scope, we looked at the learning potential and engagement factor of other products.

GOOGLE MAPS

What We Like
Provides Real World Context, Gives connection to current day, 3d environments feel immersive and exploratory, students can move to different areas

What we Didn't Like
Its too exploratory, can go off track to easily, Students may have to leap to get a good connection

NEW YORK TIMES PARALLAX

What We Like
The immersiveness of the story telling, the amazing content, the bone-chilling story, the visual effect of the parallax

What we Didn't Like
There is a lot of text, we feel as if students may not read it. The parallax can be distracting, it might be difficult to load all the images and videos

NATIONAL GEOGRAPHIC MAP MAKER

What We Like
Customizable for the teachers to author, Tips & Tricks section for better usage, Provides real context in terms of the map, Large Images, Use of primary source as a main learning point (the map itself), Twitter Connections

What we Didn't Like
Linear Path is not exploratory, Sometimes lost the proximity of a map that was too close, didn't like the red text.

PEARSON MY WORLD INTERACTIVE

What We Like
Students were engaged in creating their own product (User generated content), Provides ownership, Students who engage properly demonstrate mastery of the content.

What we Didn't Like
Students often go off track and miss the point, Teachers find it a hassle to grade all of them, Doesn’t feel immersive, Can be stressful for students who don’t have the skills or equipment, Doesn't match C3 Framework.

COMPANY OF HEROES 2

What We Like
Gamification, students love the video game elements, has realistic timeline integrated, is engaging, student feel empathy for character, minimalist UI.

What we Didn't Like
Slightly violent, afraid that student wont make connection, minimalist UI, teachers might be turned off, harder to sell.

Information Architecture

We created a site map to map out the user flow. A requirement was that we had to use our native Learning Management System. This effort was to help match the instructional model with the design pattern we wanted to achieve.

Prototype

Paper Prototype

The team and I created a few sketches of how we thought the designs might work. We ideated and created 6 different prototypes

Axure Wireframes

The team and I created a few sketches of how we thought the designs might work. We ideated and created 6 different prototypes

Validate

User Testing

To validate our designs, we conducted a navigation based usability test, along with debrief interview questions.

What we did:

5 Social Studies Teachers and 7 high school students
4 Navigation based usability tasks
4 Interview questions
In-person moderated study

What we learned:

Teachers loved the idea of using primary sources
Teachers were not a huge fan of the visual design (they wanted photographs and “high school” level illustrations)
Teachers wanted more different kinds of activities. They feel that writing is necessary for a lot of cases, but sometimes they want to have more creative solutions.
Teachers feel as if the short answer questions are too open ended.
Students and teachers were unable to navigate back to an introduction video (hidden in a menu)
Students felt the material was at an appropriate level

Show statistics

	Students	Teachers
System Usability Score (Mean of 100)	79.0/100	85.7/100
Overall User-friendliness	4 Excellent 1 Good 0 Okay	5 Excellent 1 Good 1 Okay
Percieved Usefuleness	4.6/7	6.6/7
Net Promoter Score	not measured	3 Promoter 1 Passive 1 Detractor

Field Study

How will this product perform in the real world. Do social studies students and teachers use Project Imagine in a way that meets their needs? We conducted a field and daily journal study to see how users would use this product in the classroom. I lead the creation of the exit survey for students, and guided the general research goals.

What we did:

3 Participating teachers (5 classes)
90 Exit surveys
359 completed daily journal entries
5 Days of study
Asked list of general pros and cons for each day

What we learned:

Students felt the readings were too long
Students needed more guidance and instruction
Students liked being able to see how different people lived
Students enjoyed the videos, and wanted more
Students like the hands-on activities but thought they were too easy
Students had a hard time finding connections between what they did and the real world

Extremely engaging	5
Very engaging	56
Neutral engagement	23
Not Very engaging	13
Extremely unengaging	12

Pros	Cons
Many students noted they program was interesting and contained a lot of good information. Many students noted they enjoyed learning about different perspectives of individuals during the war. Students liked being able to see how different people lived and worked at that time. Some also noted it had a wide variety of activities. Some students mentioned they enjoyed the videos (non-specific). Some students noted they especially liked learning about the Role Play - Francisco	Students would like less reading/shorter reading passages. Some students mentioned they’d like more teacher guidance because they might need clarity/have confusion about the content or navigating the product. A few Orosi students expressed interest in working through the immersives in small groups. A few students noted they wanted the ability to walk around in the Japanese Incarceration 360 Immersive. Include more activities where students need to interact/engage with each other A few students thought hands-on activities as a part of the instruction would be helpful A few mentioned they wanted less talking within the videos.

Show statistics

Implement

Key Insights

From the user test and the field study, we came up with some solutions to relieve the pain points our users were facing. There were many minor usability issues, but below are some of the key problems and solutions.

Perspective Shift

From the usability study, we knew that perspective was key to feeling immersive. We switched the perspectives to make it feel like you are playing the character, rather than just watching them.

Animation & meaningful choices

There were some problems in the navigation, so we wanted to add some clairty in progression. We added animations to show movement from point A to point B. We also added "wrong choices" for some responses to add both guidance and challenge for students. I also added a progress indicator to provide orientation, to help students with timing.

Variation in activities

Many teachers wanted variation in activities, so we created more.

Photography visual direction

Finally, our users were complaining about the lack of sophistication in the illustrations and images. We changed from illustrations to photography, and in the field test, people generally liked the visuals.

Final Visuals

Below are some of the final visuals of the project. I worked closely with the visual designer to make sure we meet accessibility requirements and that the research was incorporated into the final designs.

Next steps & reflection

Post release of the product, we will capture quantitative analysis. Most of the preliminary research was qualitative in nature, and while it answers the “why” factor, it does not answer the “how.”

Additionally, there were many limitations of the platform that are currently being updated to better support different products. Since the teams work in parallel, we needed to release the product before some platform features could support what we needed. In future iterations we would like to take advantage of these updates to the platform to better fit our needs.

More projects

Assignment Viewer

Designing a scalable content viewer to let students and teachers view multi-part assignments.

Cloud Machine

Helping students learn fractions through gamification and visualization.

Puzzle Creator

Increasing engagement for Middle-grade students by allowing them to create and share their own content.