At a time when qualified teachers are in high demand, there’s another role that districts are looking to fill: data experts.
Charter management organizations, which operate publicly-funded schools, are making million-dollar investments on data teams and shaking up staffing norms in the process. Today, education job boards feature openings such as strategic data facilitators, data analysts, and data coordinators.
Schools are hiring for these positions with hopes that the data they collect from online educational tools will inform decisions about classroom instruction, staff development, operational efficacy, funding, community engagement—and almost any other question public school stakeholders care to ask.
Yet how these data teams and systems are built and funded vary significantly. Some charter management organizations, such as New Visions for Public Schools, have spent millions of dollars on data projects and staff, who build their systems completely in-house. Others with more modest budgets rely on small teams that connect and collect data from a patchwork of tools created by third-party vendors.
These efforts have also raised questions over the extent to which data informs the work of educators. Does more data always lead to better decisions and outcomes? And is all this investment of time and money worth it?
A ‘New Vision’ for Data Application in Schools
New Visions for Public Schools, founded in 1989, currently runs 10 charter high schools serving 3,600 students. The nonprofit grew from a reform organization focused on policy, programming and professional development for New York City’s public schools to a charter management network with a sizable data team.
To date, New Visions has invested approximately $8 million into its School Systems and Data Analytics team, which has 26 full-time staff who are headquartered close to the heart of Manhattan on the east side of 42nd street. This commitment to data stems from the organization’s belief in making data-driven decisions. “Data define[s] our strategies and allow[s] us to personalize and customize supports to schools and students,” reads the company website.
Their team has developed a variety of in-house educational tools, from Google plugins that help teachers manage documents to student data warehouses.
Last fall, New Visions unveiled one of its biggest projects: the Data Portal, a platform that helps administrators make decisions about students’ schedules to ensure they graduate on time. Now used in all of New Visions’ schools, the portal allows administrators to plan for graduation and regent exams, view credit gaps, and monitor students’ attendance and performance on an individual, class and school level. Students with credit gaps are flagged, and educators can go into the system and see which classes students need to be on track for graduation.
“The complexity of management at the high school level has really been underestimated,” says Mark Dunetz, president of New Visions, in an interview with EdSurge. “One of the barriers to school improvement is a set of effective systems at the administrative level that allows groups of people and schools to make decisions that are informed.”
Dunetz notes that making scheduling decisions is incredibly complex for school officials juggling hundreds of students, each with different needs, goals and graduation requirements. This is particularly difficult for educators in high-needs populations where students may be repeating classes or in need of remedial supports. Missing a particular course requirement can put students in summer school, force them to take remedial courses in college or even impede their ability to graduate.
While New Visions is investing significantly in developing backend technologies that help schools operate, the network is not as keen on building tools that change the way teachers interact directly with students.
Their approach aligns closely with the vision of other prominent educators in New York City such as Success Academy founder Eva Moskowitz, who noted last December that technology should not attempt to replace classroom instruction, but can “digitize and automate” the back office or administrative work that educators do.
That’s a sentiment Duntez reiterated. “We want to give [educators] all the information that is relevant to them making decisions.” But, he adds, “we don’t want to supplant educators making decisions about students that they have relationships with because, even in a best case, what we see in a data system or from a distance is a fraction of the total picture.”
The data portal promises to be an upgrade over how New Visions' staff used to manage data: by cobbling together different Excel spreadsheets.
Staff members credit their commitment to data with the improvements they see in student outcomes, among them:
- student graduation rate increased from 74 percent for the class of 2013 to 83 percent for 2017;
- sixty-two percent of schools today graduate at least 80 percent of their students, up from 35 percent in 2013;
- college readiness rates (the number of students who do not have to take remedial courses when entering college) have increased, from 35 percent for the class of 2013 to 49 percent for 2017.
Yet there’s a cost to get all the data needed to make New Visions’ system work, one that sometimes falls on students: a steady flow of assessments. A former New Visions teacher, who spoke to EdSurge on condition of anonymity, noted feeling uncomfortable with the number of assessments given to students. She also said the task of analyzing that data often fell on teachers.
Multiple anonymous reviews on Glassdoor, a website for company reviews, also critiqued New Visions’ “heavy emphasis on fast data analysis and reporting everything for record keeping.” The comments also questioned the amount of funds invested in the warehouse project that could have been spent on personnel.
In response, New Visions officials noted that though they have created tools (such as a Google addon called quizBanker and mock state exams) to support schools offering assessments, the focus has been to help schools in managing the data they have, not to create new assessments.
When asked why not invest the funds into more staff and personnel, Duntez was strongly against the idea. He notes that the problem he is trying to solve is straightforward, high-volume, time-sensitive work that is better handled by a machine.
“The solution you are suggesting is the one that has failed over and over, which is, ‘let’s add another guidance counselor and hope a set of systems problems resolves themselves,’” explains Duntez. “It’s not to make a case for or against any particular staffing. There are places where we could use additional staffing in schools, but this [data] infrastructure is parallel to the modern infrastructure that we have built in every other industry.”
Stringing Together Data Systems Like Mardi Gras Beads
Like New Visions, ReNEW Schools also professes a belief in helping educators make data-informed decisions. The New Orleans-based charter network identifies low-performing public schools and assuming control of the institutions—makes staffing, culture, operational and academic changes with hopes to improve student performance. They currently control five pre-K-8 schools and a high school that altogether serve about 4,100 students.
Yet ReNEW has far fewer resources. With a data team of just three people, the network has invested $250,000 into its data projects. And unlike New Visions, which has invested millions in creating their own platforms and building teams, ReNEW relies on tools created by outside vendors, only dabbling minimally with the creation of their own add-ons.
When asked why his team opted for using vendors as opposed to creating their own systems, Sumeet Goil, the executive director of data and assessment at ReNEW Schools, noted that such activities were not a network priority at this time.
“What I have found is that undertaking that type of project is very complex and resource intensive,” explains Goil. “If that type of project is not a network priority, it is not going to be money well spent.”
The data team currently uses PowerSchool, a widely-used student system platform for enrollment and state reporting. They also use Schoolrunner for logging attendance, grades and managing the school systems. The team also has various online instructional tools such as ST Math that an instructor, outside of the data team, uses to collect and analyze information.
Since adopting the data platforms, Goil notes that it has been easier for his team to communicate with educators, but he admits that the network still struggles with synchronizing data from multiple sources.
“One of the challenges of data in education is that a lot of it is siloed, so our schools are generating lots of different kinds of data all the time. We often have data that is living in different systems, and it has been a challenge to connect all those different systems together in terms of analyzing it in a very effective way,” says Goil.
Several districts share the problem of interoperability that Goil points out. As more schools adopt separate tools for separate needs, the desire for information to flow more seamlessly between products grows.
Yet, interoperability is not the only thing that has troubled ReNEW. Some ReNEW schools have not shown strong academic results, despite the implementation of new data tools. This past December, according to local reports, the state board chose not to renew charters for three schools in Louisiana, including ReNEW’s Cultural Arts Academy, because the school fell below acceptable performance standards in the state.
State standards for Louisiana charter schools take into account test scores, credit accumulation, attendance and enrollment. On an A through F evaluation scale, charters must maintain a ‘C’ or above for contract renewals. ReNEW’s Cultural Arts Academy fell from a ‘C’ to a ‘D’ last year.
How Much Should Data Be Trusted?
As charter networks ramp up their spending on data work, skeptics have raised ethical questions about such efforts.
Using data to make decisions about struggling schools can be a misguided effort, notes Roderic Crooks, a postdoctoral fellow at the University of California at Irvine. His continuing research, following the role of informatics in urban education, warns against misconceptions caused by perceived objectivity in data.
“The promise of data science is that we could make better decisions, and [that] better decisions are objective decisions. But none of this works without human interpretation and therefore human subjectivity,” says Crooks. “Data science applied in public education just moves the points of interpretation and subjectivity. It doesn’t eliminate them.”
Where subjectivity in decision-making may have been in the hands of teachers and administrators in the past, Crooks notes that now it also lies in the hands of school data scientists. And although data is assumed to offer a reliable, objective way of analyzing a given situation, oftentimes scientists remove outliers or what is assumed to be errors. They remove these data points so that the information can tell a single story that decision makers can act on.
Crooks calls this process of redacting data “a subjective act” since data scientists, charged with cleaning information, do so based on assumptions of what the data means.
He also notes that the way data scientists present data in visualizations can draw on emotional responses from decision-makers, biasing them towards particular narratives.
“Remember, most people aren’t looking at data. They’re looking at a visualization,” says Crooks “Visualizations also appear objective, and they hide the viewpoints that produce them.”
He notes that for data to be used effectively in public schools, it is important for educators to know how the data is produced, and what information is omitted.
Despite the problem of objectivity, Crooks is optimistic that schools who use data to make decisions or promote themselves as data-driven will have a competitive advantage in areas such as funding proposals, as many grant opportunities increasingly ask for data. Yet he is still concerned about the lack of training that new data teams in public schools require.
“We are at a really important point where we would need to demand ethical training for data scientists since data science work comes with ethical commitments,” says Crooks. “But there is not a lot of ethical training for data scientists, and that’s something I think is drastically important in this space.”
