Stanford, Harvard Scholars Dissect Big Data

GOOD DATA, BAD DATA: After blended learning, Big Data, and MOOCs, another edtech term is gaining steam in 2013: learning analytics. The phrase (which refers to finding meaningful data patterns that inform effective learning) is presumably where the Big Data movement in education is placing all bets. The only problem is mining data for meaningful patterns is a bit difficult when there's no strong definition of effective learning.

Just ask Stanford GSE Professor Roy Pea.

In this recent keynote address delivered at the Educause Learning Initiative, Pea cautions that learning sciences are "largely missing" from MOOCs and expose "a great chasm" in their design. In short, while the technology has turned the corner, online pedagogy is still held captive to policy and tradition. Online courses largely copy the brick-and-mortar style of lecture and quizzing even as research-grounded notions of student-centric learning have been circulating for some time -- Pea and co. have already provided such a foundation in the 2010 National Education Technology Plan.

Across the country at Harvard's Berkman Center for Internet and Society, Reynol Junco is equally unforgiving. In this NPR interview, he describes "learning analytics as the ultimate formative assessment." But when pressed to provide current examples of this type of assessment, Junco comes across more reserved:

"Generally, what you see is learning analytics applied to course management systems and course management systems is basically just a fancy way of saying an online discussion board and they are looking at how many times a student will respond, how - to a discussion - how quickly they respond to a discussion, how often they log on. So just very, very basic levels of data that they're looking at, and so the predictive models are not as accurate as they could be."

His call for more learning sciences is a bit more nuanced than Pea's. In place of (or in addition to) more integrated learning sciences, Junco would like to see the use of trace data -- "data that [students] leave behind just by the nature of what they do every day" -- to better predict student outcomes.

Still any type of data can be difficult to handle in the context of school environments.

The data points that Junco advocates for -- Facebook, Twitter, and other cookie-enabled web impressions -- lose a bit of luster under the magnifying glass of FERPA and COPPA legislation.

And even Junyo, which boasts a who's who list of data-crunching wizards among its founding team members, was forced to pivot away from their original blended learning vision, instead choosing to work with content providers.

What's clear is that K-16 education is only experiencing the tip of the iceberg when it comes to online learning. Before anyone makes any further claims of "revolutionary" platforms and advanced "adaptive" algorithms, let's take a deep breath and think about what we're measuring and why.