When edtech companies approach Kara Monroe, they all tout how their product is different from everybody else’s. But without proof, Monroe, vice president of academic innovation and support at Ivy Tech Community College, and her colleagues are searching in the dark to find the right tools that faculty can use to improve learning outcomes for the more than 200,000 students in the Indiana community college system.
“There’s huge amount of money coming into the edtech market for adaptive learning, personalized learning, competency-based materials, open materials. So much is coming at us,” Monroe says. That’s why she’s excited about recent efforts to make sense of the latest digital courseware products. A new free tool called the Courseware in Context (CWiC) framework aims to categorize and compare the range of emerging products that faculty are using to teach their online and blended courses. Tyton Partners, a Boston-based investment banking and strategy consulting firm focused on education, developed the framework in collaboration with the Online Learning Consortium and research firm SRI International, with funding from the Bill & Melinda Gates Foundation.
The CWiC Framework is geared toward helping faculty, administrators and instructional designers become savvy consumers of courseware products, in hopes that they’ll make better-informed decisions that improve outcomes for students. “It’s a way to look at the commonalities and start a bigger conversation around what quality really looks like,” says Monroe, who served on an advisory committee that provided feedback during the year-long process of developing the framework.
A pilot version of the framework is available today, and its creators stress that it’s a work in progress.
When You Say Courseware...
Digital courseware is an ambiguous term mired in hype. Does it mean adaptive software? (Often.) Does it allow faculty to customize content? (Sometimes.) The CWiC framework defines it as “instructional content that is scoped and sequenced to support delivery of an entire course through purpose-built software.”
Users can try the tool to evaluate courseware products that meet the framework’s definition. These include products that come from big publishers like Pearson, Cengage and McGraw Hill and smaller organizations like OpenStax, Junction Education and Soomo Learning. The framework also links to research about different product features, and an “implementation scorecard” to help users select tools that meet their specific needs.
“The framework provides a rigorous way for institutions to evaluate courseware products in light of their context and to do that in a way that’s aligned with efficacy research,” says Gates Bryant, a partner at Tyton who managed the framework’s development. By context, he’s referring to the fact that users can input their specific needs and find courseware products that best meet their criteria.
Prove It
Researchers at SRI International were tasked with finding studies related to how the courseware products work. Rather than study the efficacy of particular products, they looked at features that are in courseware, such as gaming components or how soon students receive feedback, and found learning science research related to those features.
When users evaluate a product they’ll access a list of studies—and a brief synopsis—related to that tool’s specific features. If a product includes socio-emotional interventions, for example, someone looking at the framework would see a list of studies that pertain to how interventions have affected student outcomes.
Barbara Means, director of the Center for Technology in Learning at SRI, says that she and her colleagues have “just scratched the surface,” by looking at close to 150 studies so far—a number the firm hopes to quadruple by October. She acknowledges where the research is weak, too. “We were looking for illustrative studies that show impact of features. There’s also research showing no impact,” she says, adding that the framework needs to include more than positive results.
Means also stresses that institutions need do their own research once they decide to try a tool. “You want to make a smart decision in trying courseware, but then you also want to look at outcomes. The need for research doesn’t stop by looking at framework.”
What’s Next?
Monroe of Ivy Tech is hardly alone in feeling overwhelmed with selecting and approving digital tools. UNC recently built its “Learning Technology Commons”—an app store to help faculty find and implement technology solutions to use in their classes.
The framework is available today for free under a Creative Commons license and its creators say they hope users will download it, customize it and share their feedback. It will formally launch out of beta in Fall 2016 and be updated annually.
