eScape - Controlled Assessment using mobile technologies
Despite the fact that learning activities in Design and Technology (‘D&T’) and other areas of the curriculum are increasingly influenced by digital technology, the final presentation of work for GCSE assessment is almost entirely paper-based, relying on print-outs of any digital work. Most teachers agree that the present assessment system at GCSE for D&T is seriously flawed and often rewards the wrong students, notably the ones that meticulously follow the rules rather than break them and come up with really exciting ideas.
Project e-scape is a new approach to the assessment of creativity and design innovation at GCSE level where students use mobile digital devices to evidence their on-going work during carefully structured design tasks. Their photos, sketches, notes and audio comments are instantly uploaded into a multimedia web-based portfolio that teachers and examiners can then use to assess individual performance.
The e-scape project originated from a paper based assessment system which took place over six hours on two consecutive mornings. The tightly structured format was designed to stimulate pupil creativity and group work in studio-based GCSE D&T activities. Digital photos of each student’s 3D models were taken every hour, as their designing progressed. This had significant effects on both performance and behaviour, motivating students to work more purposefully and to take more risks. It also helped the assessors to see a more complete picture of each project. This led the e-Scape team to consider how the collection and recording of other aspects of performance, such as discussion in the form of short audio clips, might help to present an even more complete picture of D&T capability.
e-Scape Phase 1 (Proof of concept)
The DfES and QCA were also keen to explore multimedia evidence trails and agreed to support the development of a digital version of the TERU paper based system and, in January 2005 TERU started work on a proof of concept. TERU were determined to avoid the twin traps of forcing pupils out of workshops and studios into second-hand, virtual experiences in computer suits, or overburdening teachers with data capture and scanning tasks. As technology changes so rapidly TERU wanted to make sure they had a broad perspective of what was possible and explored a wide range of web based portfolios and small scale, handheld systems that could be linked to allow students to collect their own evidence and trialled a series of single media systems including:
- digital cameras (still and video)
- digital memo/voice recorders
- digital pens.
Of these, the digital pens seemed to offer the most effective route to capturing students’ notes and sketches but the students went further and suggested the digital pen would be better if it had a camera and microphone built in. They were essentially describing a PDA, and when we subsequently trialled with Palm Zire 72 devices, we began to see significant motivational and inclusive benefits during, as well as after the activities.
e-scape Phase 2 (Working prototype)
With sufficient evidence from the proof of concept trials, the project moved on to develop the e-scape process into a working prototype with a system that managed the dynamic dataflow from a web server to and from the PDAs, using the Red Halo PDA Platform from Handheld Learning (http://www.redhalo.com/) to synchronise document flow and a web-based e-portfolio system based on the three times award winning e-portfolio assessment system ‘MAPS’ from TAG Learning (www.mapsassessment.com).
During the sessions, activity instructions were delivered to the PDAs at timed intervals, students’ work was beamed around their PDAs to facilitate group support and review. Throughout, all the media created and collected by the students was uploaded to individual secure web portfolios as the activity progressed. In this way students recorded the progress of their designing on their own PDA using photos, sketches, notes and audio comments.
With a sufficiently robust system completed, a national trial in 14 schools was undertaken across the country through June and July 2006. As a result 250 e-portfolios of year 10 learners’ performance were gathered on a e-scape website, together with a class set of year five work (who had no problems with the technology and produced some really exciting work).
The combination of the activity structure, supported by the handheld technology had a profound effect on all the students TERU worked with. Teachers TV videoed one of the early trial sessions, together with a discussion panel and a broader look at e-assessment.
e-scape Phase 3 (National scalability)
This phase of the project extend the activities beyond D&T to investigate the extent to which this approach to structured, monitored, coursework assessment can be applied to other subjects. This work includes an authoring tool to enable teachers to build and modify their own coursework activities for themselves and to scale the system up to understand the issues awarding bodies may face if they were to implement it nationally at GCSE. A number of successful pilots were conducted over the summer 2008 to test this phase of the project.