Not all feedback is equal--especially when it come from surveys where the respondents don’t understand the question. Just ask Panorama Education, a Y Combinator and Imagine K12 graduate that has helped over 5,000 schools in 30 states over the past two and a half years develop, deploy and analyze student surveys.
“Our idea at the beginning was that schools would come to us with existing surveys and questions to get the data that they want to collect [from students]” Aaron Feuer, CEO and co-founder of Panorama Education, tells EdSurge. But schools weren’t sure about what they wanted or how to phrase their needs. “Many clients were asking the same questions in different ways. Everyone was re-creating the wheel.”
Today, the Boston, MA-based startup is making the Panorama Student Survey available to all educators for free. The questions, targeted at different grade levels (3rd to 5th grade and 6th to 12th grade) cover nine topics: five focused directly on teaching and learning and four covering classroom climate, engagement, grit and sense of belonging.
Designing a quality survey, especially one for students, is no easy task. And it comes with a hefty price tag: Schools and districts sometimes spend “up to six figures” to consultants to write survey questions, according to Feuer. Compounding the problem is the fact that, due to strict copyright and licensing terms, schools are asked to pay an extra fee for the rights to modify and use these surveys.
Last year, the company decided to spend six figures of its own and approached Hunter Gehlbach, an Associate Professor of Education at Harvard University, to lead a team of graduate students in designing a standardized student survey.
Survey design is notoriously deceptive, says Gehlbach. “We’ve all taken surveys, and so it may seem very intuitive,” says Gehlbach. “But a little decision in language can make a big difference.” He points to the classic “opt-in” case study about the dramatic difference in organ donation consent rates across European countries as an example of how “one word can make a powerful suggestion about what’s normal and how people should respond.”
Surveys also often reflect certain agendas and biases--sometimes unintentionally. Many of the school surveys that Feuer has seen “simply ask, ‘How much do you agree or disagree with the following statement?’” Such questions generally don’t offer useful insights, he says, because “people are generally nice and tend to agree.”
The seven-step design process for the Panorama Student Survey, which included literature review, interviews with academics and students, and pilots in 13 school districts, took over a year. It may seem like a long time for a set of 90-odd questions, but the precise wording around every question and response warrants careful attention. Here’s one question from the “Supportive Relationships” topic:
If you walked into class upset, how concerned would your teacher be?
- Not at all concerned
- Slightly concerned
- Somewhat concerned
- Quite concerned
- Extremely Concerned
Each answer choice repeats a keyword used in the question to “reinforce to the respondent what it is you are really asking when they search their memories,” says Gehlbach. “Feedback can be a really powerful tool, but you have to ask the right questions in the right way.”
Panorama will integrate the Student Survey into its technology platform, which includes tools that help schools and educators deliver surveys and analyze results. The team says it is “open sourcing” these surveys by inviting others to customize the questions and share them with other districts and the Panorama research team.
While many edtech tools focus on improving student learning outcomes, not many offer students a voice. With the Panorama Student Survey, the company aims to provide schools with a reliable way to capture feedback from the very people they serve. With enough usage, the company says it can create a baseline for schools to see how they measure up with others.
