Archive for month: October, 2019

“Assessment automation is a key step to – finally – unlocking the real potential of e-learning”

There’s growing recognition that automation, artificial intelligence, and electronic assessment tools have an important role in the future of e-learning. But e-learning company Enrolo sees assessment automation as a vital step towards transforming that future.

“Assessment automation is a key step to – finally – unlocking the real potential of e-learning,” says Enrolo CEO Adam Menary. “Like many companies, we’ve been exploring a range of new innovations and approaches to e-learning. Though we’ve also been approaching e-learning in a way that draws upon the best thinking and best practices from other industries.”

Beyond e-learning, Mr Menary has extensive experience in HACCP, risk and quality management systems. “These involve a systematic approach to assessing performance and achieving continuous improvement”, Mr Menary said. “This approach has been missing from e-learning because most data and systems are focussed on the learner. This can be useful, but ultimately, it’s a bit like trying to understand a tennis match when you’re seeing only one of the players and one side of the tennis court.”

So why does assessment automation matter? “It’s about advancing the assessment process with new technology-based tools, data and innovative approaches,” said Mr Menary. “Through our research projects, we’re starting to see that automation tools can transform the role of assessors, amplifying their ability to deliver value, both for students, and across the entire e-learning process.”

Enrolo CEO Adam Menary talked with -e-learning analyst Paul Wilson over coffees and Skype.

Question: What is it about assessment that has been missing from e-learning?

Enrolo CEO Adam Menary:                                             

Two things. Firstly, we’ve missed the opportunity to leverage the knowledge, value and insights of that assessors can provide.

In the drive to get e-learning content online and to get student numbers, the actual assessment process has received little attention. Without new assessment tools to match the pace and scale of work generated by learner-side technology advances, the ability of assessors to contribute value has been gradually diminishing. This can lead to a cost-focussed view of assessment, and with that, risks to learning outcomes – for example, through oversimplification of questions to reduce assessment costs, or reduced quality of assessment and feedback.

More broadly, in a competitive online sales environment this situation risks an unhealthy low cost and low value “race to the bottom” scenario for the future of e-learning.

However, it does not need to be this way.

The assessor is the person on the other side of the screen in a student’s e-learning experience. They can be the drivers of e-learning value and continuous improvement. They have front-line contact with students; they provide assessments and feedback that helps students confirm and improve their understanding; they also gain important insights into the strengths and weaknesses of e-learning content. They have the knowledge and insights to make a difference – and that’s usually the reason they’ve got into education in the first place.

Secondly, capturing rich data in the assessment process is essential to an integrated and scientific approach to continuous improvement across the entire e-learning process and organization. While other data is also important, data from the assessment process complements learner data, and helps provide a more complete view of overall e-learning performance.

Question: What do you mean by an “integrated and scientific approach”?

Enrolo CEO Adam Menary:

HACCP (food safety) systems use continuously-collected data to help ensure that food is safe to eat and free from contamination. Historically, e-learning has lacked that grade of data. For example, while SCORM involved collecting data that a student had successfully completed an assessment, SCORM might not be configured to keep track of which questions they answered, which they failed, where they got help or feedback, and whether that made a useful difference. This meant the data didn’t support efforts to statistically or scientifically track down areas where issues were occurring, or to identify whether changes were actually improvements.

The limitations of SCORM have been widely recognised, and xAPI emerged to address some of these. However, the focus remains on the learner and capturing learning-related data. Complementing this with data specifically about the assessment process can do two things: support work to streamline and improve assessment activities, and also provide insights about the overall performance of e-learning.

Question: Do you have any thoughts on how this might affect assessment from a learner’s perspective?

Enrolo CEO Adam Menary:

Yes. Multiple choice questions, true/false, checklists and the like are basic online forms of traditional assessment approaches but today they should be viewed by educators as simple check your progress tools rather than actual assessments. And sure, like others, Enrolo has done some interesting and innovative things with interactive check your progress. However, across the industry, e-learning has been constrained by a particular horizon of assessment possibilities, based on the costs, abilities, and practicalities of an assessor working in somewhat traditional manner. Automation, data and tools can help expand that horizon of assessment possibilities – and so offers further opportunities to transform e-learning.

And really, this is essential. Some of e-learning’s traditional assessment approaches are not that great. They can be blunt and ineffective tools for confirming a learner’s understanding.

We’re starting to explore ways to completely change assessment tasks, and learning content, based on the new possibilities of automation-enabled assessment. For example, what might it mean to have more free-form styles of assessment task, or much faster cycles of submission and feedback. Or imagine having assessment feedback going beyond comments – for example, using it the assessment feedback to dynamically reconfigure e-learning content to leverage a student’s strengths, or provide extra examples and explanations to address areas of weaknesses.

As the traditional true/false, pick from a list, or drag and drop type questions are relegated to check your progress aids I think we’re going to see some really important innovations and powerful new approaches to assessment and meaningful learning outcomes.