Many students struggle with early college courses—whether developmental courses preparing them for college-level math and English or introductory courses in subjects like biology, psychology and business. Colleges and universities concerned with high failure rates in these courses are exploring how new learning technologies, like courseware that delivers and personalizes instructional content, can help faculty adapt the learning experience to the needs of individual students.
In 2013, the Bill & Melinda Gates Foundation started the Adaptive Learning Market Acceleration Program (ALMAP) to generate evidence regarding the impact of adaptive courseware on outcomes, especially for low-income college students, and to examine associated changes in costs for institutions. The foundation awarded grants to 14 colleges and universities to support their use of adaptive courseware in a range of courses and contexts. In tandem, the foundation funded SRI Education to conduct a rigorous evaluation of this program. Led by Dr. Louise Yarnall, our team at SRI analyzed data collected from over 19,500 students taught by 280 instructors at these 14 colleges. This evaluation studied the impact of adaptive courseware on outcomes, institutional costs, and on the experiences and reactions of faculty and students. Here’s what we’ve learned so far:
Impacts on student outcomes: Across the diverse courses and colleges studied, SRI’s evaluation found evidence that adaptive courseware could improve some student outcomes. For example, when compared to lecture courses, blended courses with the adaptive courseware produced higher scores on learning assessments. Students and instructors in developmental courses and at two two-year colleges had more positive perceptions of the adaptive courseware than did students at four-year colleges.
Skeptics believe learning technologies can perpetuate inequities and the “digital divide” that exists between affluent and low-income and minority student populations. SRI specifically analyzed the impact of adaptive courseware on low-income populations. Overall the evaluation found that compared to lecture courses, blended courses with the adaptive software produced higher scores on learning assessments, with no indication that adaptive courseware had an adverse impact on low-income students.
Impacts on costs: Because instructors needed to become familiar with new courseware or adapt their content for it, the first year of implementing adaptive software led to higher costs on average. Over time, the adaptive courseware sections usually had lower costs than business-as-usual versions of the same course.
Implications: SRI’s findings underscore the need to understand how faculty and students are using the courseware and what instruction looks like in classes without adaptive courseware. In some cases, outcomes varied for the same product when used at the same institution for multiple terms. More research is needed to fully understand how different types of colleges can achieve consistently improved outcomes, and how faculty can best design courses and use adaptive courseware.
In reviewing these findings, one should keep in mind the youth of the adaptive courseware market and the products that were evaluated. The institutions that participated in the evaluation should be applauded for their important role in furthering our understanding of the impacts of adaptive courseware. Further details on the ALMAP evaluation are available in the final report on the SRI Education website.
