Discussions about the future of school curricula tend to be cannibalized by the sensationalized debate over the Common Core, but for those able to glimpse the long-term future, a key issue is open educational resources (OER). For many, the question is not so much if or when OER will become the standard, but how we can maximize the strength and sustainability of their impact.
Yet for many others--specifically those making the policy decisions that affect thousands of kids--the big question remains whether or not OER work. To date, there has been little empirical evidence demonstrating its effect in classrooms. And with the phrase “evidence-based” now being thrown around like champagne in a celebratory locker room, more is needed before OER make large inroads in schools.
In a new study published in the Educational Researcher, however, a big leap toward that point has been taken. The study, which examined a school district in Utah, provides one of the largest, methodologically rigorous evaluations of OER to date. The research was led by Jared Robinson, a PhD student at BYU, and Lumen Learning co-founder David Wiley was among three other BYU researchers who were co-authors on the paper.
The initial sample consisted of 43 teachers and over 4,000 high school students. A technique called propensity score matching was used to match each student in an OER classroom with a student in a non-OER classroom who was similar in terms of demographics and prior academic achievement. Because some students did not have matches, the final matched sample included slightly more than 2,500 students, with nearly all of them in grades 9-11.
OER materials came from a group of 18 teachers who created their own textbooks using OER originally created by the CK-12 Foundation. Students were given printed textbooks, which cost $5 each, as well as access to the textbooks in digital form. During the 2011-12 school year--the year in which the study took place--43% of all students in the district who were enrolled in the three science courses used OER.
Student performance was examined in three science subjects: Earth systems, biology, and chemistry. The effect of the textbooks was measured using a criterion-referenced test in each of the three subjects. In addition to controlling for demographics and prior academic performance, the researchers controlled for teacher effects by including the past performance of each teacher’s students in the analysis.
What did the researchers find? Across all three subjects students in OER classrooms performed better than students in normal classrooms--a difference that was statistically significant. When each subject was analyzed individually, the difference was only statistically significant for chemistry. It’s worth noting that although the effects were statistically significant, the effect sizes were extremely small. (Specifically, the effect of OER textbooks on a student’s performance was about a quarter of the size of the estimated influence of his or her prior GPA.) Nevertheless, the impact of classroom materials on performance is generally fairly limited, and so the small effect sizes should be viewed within that context.
More importantly, given the other benefits of OER--chiefly, enormous cost savings--the key finding may simply be that the students who used OER didn’t perform any worse. As long as OER textbooks are just as good (or better), schools can use them to save money, then put the funds toward increasing achievement in other areas.
One limitation of the study is that it doesn’t shed light on exactly why OER are effective. It could be that the specific content in the materials isn’t superior per se, but that the larger teacher-created structure of OER better aligns with how teachers envision their lessons. Alternatively, improved performance may be the result of lower prices providing students with better access to course materials. More research is needed to tease apart these potential effects.
From a policy standpoint, additional evaluations on this scale are needed before more districts throw their hats into the OER ring. Yet even a single study can create a positive feedback loop where more districts use OER, thereby creating more opportunities to evaluate its impact, thereby creating more potential evidence that can convince districts to transition to OER.
Things tend to move slowly in the world of educational materials, but there is likely a tipping point where the quantity of positive OER evaluations makes the potential cost savings impossible to ignore. Once that point is crossed the long-term issues OER advocates have been debating--such as whether or not all OER should remain free--could quickly find their way onto everybody’s agenda.
