Puzzle Creator
Cloud machine is a puzzle game created for students in grades 3-8. The puzzle creator feature allows middle grade students to create and share their own puzzles with their fellow classmates.
The project started off with a meeting with the product owner to understand some requirements and define the problem. There was a market opportunity for the company to expand into Middle School territory. There was one relevant game that specifically focused on fractions, called Cloud Machine, that we decided to expand upon.
How might we expand upon Cloud Machine to better fit middle schoolers?
From there, the next step is to understand our target users: Middle School students. The research team and I worked closely to interview middle school students. The interviews were focused around how to engage middle school students. What are the motivational differences between elementary and middle school students?
After interviewing many students, we found 4 common themes, which helped guide our design decisions:
Using these 4 persona attributes, we have a small group brainstorm about how we can better service students through creativity, competence, autonomy and social. How do we engage middle schoolers through social features, by giving them the freedom to explore creativity, by allowing them to convey competence, and by giving them autonomy?
The team of 12 devs, designers, editors and product managers split into 4 groups of 3 to ideate.
There were many great ideas, but the one that kept appearing multiple times (and that was most compelling to the PM) was the ability for students to create and share their own levels.
This idea hit many of the 4 pillars: It allowed students to be social by sharing their puzzles, it gave students the freedom to express their creativity through puzzle design, it helped students show their competence by allowing them to make difficult and challenging puzzles, and it gave students autonomy by giving them control of their own content.
One huge advantage we had with the idea of creating and sharing levels is that we already had an internal "puzzle maker." This already allowed the staff at CA to make puzzles and share it with students. This meant that we could perform a product reaction user test at little-to-no development effort. The purpose was to validate our assumption: are middle grade students going to want to use the puzzle creator to make and share their own puzzles?
First off, we needed to make the puzzle creator look visually similar to the rest of the game. Many students complained about the wire-frame style of the prototype, and were confused by the lack of consistency.
Students were confused on why they were creating puzzles, so we wanted to provide more story. In other levels, you play the game to make rain, which will feed/animate plants and animals. The problem with having normal rewards in the puzzle creator is that we cannot account for how many puzzles the students will make. In theory, the students could make 1000s of puzzles. We needed a way for the rewards to be scalable, so we made rewards temporary for these levels.
Students have difficulty dragging with a trackpad (on top of being non WCAG compliant). Many students were also frustrated that the double-click would delete a previously created container. To account for this, we changed the way that you create puzzle components on the screen. Rather than dragging across a grid, you select a tool and then click on the grid with the tool selected. This acts as a brush, and “paints” the selected component onto the grid.
Finally, we added a banner to the game that lets students know that they are able to share the puzzles they create.
During the development process, we quickly learned there were many edge-case scenarios that students could potentially encounter. Product suggested that we do a limited release to about 5% of our users to get more data. The goals of the limited release were:
Now that we knew “how” students were using the puzzle creator, we wanted to investigate the “why” and “what.”
Why weren’t students publishing their puzzles? Why are students missing the puzzle creator button? Why are students not understanding the difference between a normal class category and a shared class category?
We also dramatically changed the look and feel of the UI, so we wanted to validate all of our changes since the last user test.
Show more research
Many students did not have the vocabulary or knowledge of fractions to play this game effectively. We created a short tutorial to help the student understand the basics fractions, and to better get at the some more complicated terms, such as Numerator and Denominator.
Players that have experience with the game do better. We decided to only introduce this feature after the user has demonstrated their ability to play, by completing 5 of the other levels. Students will only unlock the puzzle creator after they have completed this requirement.
Students did not understand the difference between the class category button and the normal category button. We updated the design to give a different look and feel, and to distinguish the two types of categories.
Students were confused by the names on the class categories, so we decided to let teachers re-name their classes so that students aren’t confused by the strange class names in the systems.
Students took a very long time to create puzzles. We suspect this is why many students were starting to create puzzles, but not finishing them. To account for this, we decided to add a “save” feature so that students who are not finished can save their puzzles.
After the full release of the puzzle creator, we observed that active student playtime for middle grades increased by 12%. This means we were successful in our goal of increasing student play time for middle grade students.
One part of the project that did not go smoothly is the user recruitment during the pandemic. Because of this, we were unable to visit schools, which often paints a different picture. This may affect how students performed in the usability tests, and also may have an impact on the quantitative data. Additionally, we used “Friends and family” testing, which means that we tested with the students of internal colleagues. I suspect that students of parents who work for an educational company may behave differently than students whose parents do not.
