Could technology render external assessment irrelevant?

18/02/2014John Ingram, Managing Director, RM Assessment & Data

“If I had asked people what they wanted, they would have said faster horses.”  So, reputedly, said Henry Ford on the topic of innovation. Regardless of the quote’s authenticity, it’s a useful reminder to step outside the norm from time to time and wonder what a bolt from the blue would do to our day-to-day existence.

Technology has already streamlined our assessment processes. According to Ofqual, onscreen marking is now the main type of marking for general qualifications in the UK. Onscreen marking involves scanning exam papers and digitally distributing them to examiners to mark using specialist software. In 2012 66% of nearly 16 million exam scripts were marked this way in England, Wales and Northern Ireland. Onscreen marking is also gaining in popularity in other territories: RM’s onscreen marking system has been used by awarding organisations in Eastern Europe, North America, Asia and Australasia.

As well as reducing the time and risk involved in transporting exam papers to and fro, onscreen marking improves reliability by automatically adding up the marks. Teams of examiners can be monitored in real time, with the system stopping under-performing markers from marking further questions.

On the whole, however, onscreen marking is just a smarter way of assessing hand-written exams. The fact that it can also be used to mark computer-based tests, coursework and audio-visual files is becoming less relevant in a country such as England where the emphasis is on linear assessment and paper-based exams, at least where school exams are concerned.

Let’s call onscreen marking of exams ‘faster horses’, then; it’s better than marking by hand but it doesn’t revolutionise the way we evaluate learning. So what’s the ‘motorcar’? Tests taken on computer? Countries such as Denmark and Norway have introduced computer-based testing for national exams. The next round of PISA tests in 2015 will be taken on computer. Moving from paper to computers does feel like progress – until you look around you.

The world has moved on to tablets, smartphones and – those clunky phrases – the ‘internet of things’ and ‘the internet of customers’. Which could mean that while we polish our current system to its highest possible sparkle, waiting in the wings is a disruptor which will render it irrelevant.

It’s perhaps natural that in education, where the stakes are so high, there can be fear of technology. There’s a worry that hi-tech can mean low quality – quicker, shorter, and more superficial assessment. But that needn’t be the case.

We’re already seeing glimmers of new ways of experiencing and demonstrating learning. Open badges add context to academic achievement. MOOCs offer access to expertise from all over the world. There will always be a place for face-to-face teaching and core subjects, but the way we learn is becoming broader, more granular, more accessible. With digitisation comes the expectation of immediacy: on-demand exams, instant results, instant certificates to share online.

For education to exploit technology for our children’s benefit, we need to learn from other fields. So far this year we’ve seen babygrows that monitor temperature and breathing. Contact lenses that measure glucose levels. Even toothbrushes that tell tales to your dentist when you’ve been less than thorough. It isn’t too much of a stretch to imagine multiple data streams which continually monitor a student’s development and trigger a feedback loop to help them gain the required level of attainment. Meaning a one-off, external exam is rendered unnecessary. Will it happen by 2025?  To answer that with any certainty I’d need to ditch my smartphone and dig out the crystal ball.

What can we learn from other uses of technology like flight simulators?

28/01/2014Gareth Mills, Trustee, Futurelab, and Member, 21st Century Learning Alliance

Technology enhances human capability. It always has done. The telescope allowed us to see further and the microscope helped us to look closer. Coupled with our incredible human capacity to imagine, technological tools have helped to unlock the wonders of the universe and the secrets of our genetic make-up. The history of mankind is a story of ingenuity in the use of tools to solve problems and create new possibilities.

It is surprising, given the transformations seen in many other professions, that so little of genuine significance has been done to exploit technology in the field of educational assessment. What has happened is the automation of many of the easy-to-automate processes of traditional assessment. This includes the marking of multiple-choice questions and the crunching and analysis of big data. The application of technology has tended to serve the needs of administrative efficiency rather than trigger genuine transformation.

Without undermining what has been achieved to date we might, by 2025, seek to harness technology to do more significant things.

So how might we use technology more imaginatively to see further and look closer? Let’s consider just three examples.

Even traditionalists tend to agree that sitting students in a hall to take pencil and paper tests is, at best, a proxy for something else we value much more. Whether students head for university or the world of work, employers and lecturers will value their capacity to manage themselves, show initiative, undertake research, think critically and creatively, work collaboratively and have good interpersonal skills. Employers also say that they look for qualities such as determination, optimism and emotional intelligence alongside competency in literacy and numeracy.

Modern conceptions of competency for future success in life include a wider set of attributes than can generally be found in the mark schemes of most GCSEs. Being fit for the future goes way beyond what can be captured adequately within three hours in an exam hall.

By 2025, one thing we should have explored is the use of scenarios and immersive environments in assessment. No doubt, some traditionalists will baulk at the suggestion; however, most of us feel reassured that the pilot flying our holiday jet has made good use of a flight simulator.  It is reassuring to know that the person at the controls has learned about the handling characteristics of the aircraft, practised how to deal with unusual weather conditions or mechanical failures and rehearsed landing at the world’s most difficult airports in a virtual environment. Immersive environments help to strengthen the authenticity of learning, they are dynamic enough to respond to the user and are able to test capability in many different contexts.

In medicine, the military and the health and safety industries we are seeing a growth in the use of virtual environments to support learning. We can find examples in education too, however, nothing has yet made it into the mainstream or challenged the hegemony of traditional tests.

Is it too far fetched to imagine that by 2025 education assessment might be making use of rich on-screen scenarios to support learning and assessment? Shouldn’t we be using our ingenuity to make assessment more authentic, dynamic and contextually situated? As I write, however, policymakers seem to be marching in the opposite direction.

By 2025 we should also have made significant progress in the use of existing technology in assessment situations. How about, for example, the use of internet-enabled laptops in the exam hall? In Denmark they were piloting such initiatives years ago.  With a set of challenging tasks and tracking software the skills of searching, selection, synthesis, analyses, argument and presentation can all be evaluated alongside the application of knowledge. Such an approach would better reflect the way many will be expected to work in real life. We use tools, not to cheat, but as a way to increase our capacity for critical and creative thought.

By 2025 we will have also taken some technology-enabled assessments to scale. When and how did you take the theory section of your driving test? Since the early 2000’s candidates have taken an online test and a screen-based hazard perception test, involving video clips and touch sensitive surfaces. Of course, a hands-on practical driving test is also required before successful candidates are let loose on the roads.  It seems like a well-balanced assessment to me – knowledge recall, perception testing and practical applied skills. Importantly, no one feels cheated because everyone doesn’t sit the on-line test nor drive along the same roads on the same day.

Perhaps in 2025 we might have more well-balanced, when-ready assessments rather than the set piece, once-a-year, no re-sits culture that drives assessment at the moment. If we can get technology assessment to scale in an important arena like driving, why not in others?

Despite media reports to the contrary, the UK has for many years been highly regarded for the quality of its public education and it is, consequently, a major exporter of educational services and assessments. I fear that by allowing our system to ossify, by not keeping pace with innovation we are in danger of missing a golden opportunity. As a country we need to be investing far more in R&D and developing new products and services to support high quality learning and assessment. We should seek to become the ‘silicon valley’ of technology-enabled learning.

Technology itself, of course, is not a silver bullet. Like all tools it is neutral. We can use a hammer to build or destroy. It is how we choose to use the tool that matters. We need to be at the leading edge in nurturing young people to develop the capacities they will need to flourish in life and work in the future. One way to do this will be through the use of technology coupled with, of course, that enduring human attribute… ingenuity.