Personalizing with technology isn’t easy—especially when the user has to adjust to the product, rather than the other way around.
Take KIPP Bay Area math teacher Tricia Dong. A few years ago, she could see that some of her fifth graders struggled with second grade math skills, while others were capable of eighth grade content on the NWEA Measures of Academic Progress (MAP) assessment. She wanted to create unique content for each of them—but it wasn’t easy. “The first time I created Khan Academy playlists for each of my 100 fifth graders,” says Dong, “it took me 6 hours.”
Individualizing lessons for students is a still a manual process for most teachers—Dong’s experience is not unusual—which means customization can happen sporadically or not at all. But for good learning to happen, customization is a non-negotiable.
Four KIPP teachers decided to take matters into their own hands, responding to this frustration by building something better. Today, KIPP Bay Area teachers can generate personalized Khan Academy playlists for their students in under 10 minutes using an automated tool. The tool was designed by KIPP Bay Area teachers and, unsurprisingly, solves a real problem while saving time.
How’d they do it? Here’s their story.
Building Tools That Teachers Want To Use
KIPP Bay Area is a public charter school network operating 11 schools serving 4,700 K-12 students. Seventy-seven percent of the network's students are low-income, and 95% are students of color. And like in many districts and schools, teachers are surrounded by “helpful” tools that are not very helpful.
For example, KIPP students are assessed three times a year on the NWEA MAP test. MAP is unique because it measures students’ academic growth throughout the year. It can tell a teacher that their 7th grade student started the year with the math skills of a typical 4th grader and ended the year performing like the average 6th grader. Traditional assessments do not capture that type of student growth data nor do end-of-year state tests. But there’s a problem: the data is not actionable for teachers. What does a teacher do now that they know Carlos has a 174 scale score in “Measurement and Data”?
Prior to the 2014-2015 school year, teachers also had access to Khan Academy, where math content is organized by grade level. But organizing content by grade level is not helpful when students are all over place in their knowledge and skills. “Data that is not actionable plus online content that is not customized is a recipe for disappointment,” laments Jennie Dougherty, KIPP Bay Area’s Associate Director of Innovation.
Teachers, by default, began creating their own solutions. In fall 2014, Khan Academy quietly released a PDF document that told teachers which Khan Academy exercises are most relevant to students based on their MAP scores. Kamal Pannu, who was a 5th grade math teacher back then, began manually assigning Khan Academy exercises to students using this guidance. “I prototyped an early version of a tool that created Khan Academy playlists based on students’ individual MAP scores, but the initial setup took me weeks, and it wasn’t very practical,” said Pannu.
Dougherty, who manages innovation for the region, noticed that Pannu and other teachers were making different hacks to the same problem. She worked with them to build an automated tool. The resulting 1.0 version was a Google spreadsheet (shown below) that allows teachers to copy-and-paste their students’ MAP scores and generate an individualized Khan Academy playlist for each student.
All of a sudden, the MAP assessment data was actionable and the Khan Academy content was personalized. KIPP teachers understood that, in order for Khan Academy to be more useful in the classroom, students couldn’t just roam free in the software; the classroom experience needed coherence. The tool (pictured above) is a Rosetta Stone of sorts that takes a student’s individual MAP data, uses this data to help students move through Khan Academy in targeted ways—moving up or down grade levels as appropriate—and allows teachers to link the work on Khan Academy to the classroom.
Let’s dig into how the playlists are used.
These Teachers Have Moves Like Jagger
The customized Khan Academy playlists have completely transformed KIPP’s classrooms! Ok, just kidding. But in all seriousness, the playlists became an important part of these teachers’ classrooms, helping struggling students close skill gaps and advanced students access new content.
During the start of the 2015-2016 school year, KIPP Bay Area began implementing the Eureka Math curriculum while adopting the Common Core standards. In Ms. Dong’s classes, she grouped her 5th graders into low, medium and high groups, and each group did three rotations during a 70 minute period—using a combination of Khan Academy and Eureka Math. See below.
| Group | Rotation 1 | Rotation 2 | Rotation 3 |
|---|---|---|---|
| Low | Small group Eureka lesson with teacher | Independent practice on Eureka | Khan Academy exercises to fill skill gaps |
| Medium | Students try Eureka lesson on their own | Small group Eureka lesson with teacher | Independent practice on Eureka |
| High | Students try Eureka lesson on their own | Khan Academy extension exercises | Small group instruction with teacher to correct Eureka misperceptions or support extension activities |
The structure allows Dong to spend her time pushing students’ conceptual understanding on Eureka math while giving students customized practice opportunities on Khan Academy.
Four teachers—Neil Davis, Tricia Dong, Katie Hogan, and Kamal Pannu—stood out among their colleagues as having the best methods to combine small group instruction, individual support and customized practice opportunities for students.Their students’ growth on the MAP test that year ranged from promising to earth-shattering. Hogan and Pannu, for example, finished the year with two-thirds of their students scoring in the top quartile nationally.
This success led KIPP Bay Area to refine the tool and release it to all math teachers in the fall of 2015. And here’s a crazy stat: Dougherty’s team generated individualized playlists for over 3,600 students during September alone.
Hungry for More
Ultimately, these KIPP teachers wanted tighter alignment between their classroom lessons and what students were doing online. They appreciated online learning’s potential to provide personalized practice, but it needed to be part of coherent classroom experience.
Aligning MAP to Khan Academy was a first step. But a little alignment created hunger for more—and these teachers aren’t stopping the design process anytime soon.
“Let’s break the mystery of ‘black box’ online content and align Eureka Math to Khan Academy,” says Pannu. “For each Eureka lesson, let’s find the foundational math skills on Khan Academy for students who are 1-2 years behind grade level.”
Sounds like another tool to build. Take your pencils and notebooks out, Silicon Valley. Class is in session.
