- Case Study -

Diagnosing gaps in learning with CEM assessments



Moving forward after COVID-19

Kolej Tuanku Ja’afar (KTJ) is an age 3-19 not-for-profit boarding school about 40km south of Kuala Lumpur, in Negeri Sembilan state.

The student body comprises about 77% Malaysian students with about 800 students in the secondary school, most of whom are boarders, along with smaller numbers from Korea, Indonesia, Thailand and other ASEAN countries.

Alastair Farquharson, Assistant Head of School (Student Outcomes), explains how KTJ is using CEM assessments as part of their approach to support students' return to school after the recent disruption to classroom-based teaching.

‘KTJ has high academic standards - 82.4% A*/A at A Level in 2020 - which has brought quite specific challenges during the period of online learning in terms of managing different levels of motivation for example. The Kolej moved to online learning on 17th March and had initially hoped to keep boarding facilities open for overseas students and those who were due to sit exams.

‘However, the situation rapidly changed, and the boarding department had to scramble to source flights to various countries - there were a few cases of children being turned back from the airport because of last minute flight cancellations and other travel uncertainties. The anticipated full return is 1st September 2020 - after about 166 days away from school for most students.’



Tracking progress against targets with CEM assessments


‘There are two key elements that will be useful post-home learning when KTJ returns on 1st September. The elements which will really be able to enhance their provision and support for all students are the Individual Pupil Records and the Chances Graphs.

‘I will admit readily that none of these ideas is entirely original but are all gleaned from other CEM and Cambridge Assessment blog posts and webinars - for those who are not using CEM products at the moment, this is, for me, a really strong feature of CEM, as well as the fact they have a track record of being linked to first class universities, initially at the University of Durham’s Faculty of Education, and more recently with Cambridge University through Cambridge Assessment.’


‘KTJ is very aware that the process of nurturing students once they return to school is going to be crucial.

‘The traditional post summer holiday learning dip will be proportionately amplified both by time and the isolation that everyone has had to endure. Focused mentor or tutor sessions will be a perfect opportunity to share and discuss IPRs with students, but they are also something that can be effectively used within departments and by class teachers to spot students who may have struggled, fallen behind or developed gaps in their knowledge.

‘What will be really important though, in making sure that every student catches up to where they are meant to be, is a much closer integration between analyses of target data, standardised scores within the CEM assessments and ongoing class data. We believe that this integration of different data sets is really essential to how the available information is used - by building a framework around triangulated data, it helps us understand students in comparison to where they actually are, and where they would have ordinarily been.’



Neglected areas of study


‘Given standardised assessments examine a student’s innate ability, more effective analysis of IPRs and standardised assessments can highlight areas which a student may have neglected during their time away from school.

‘Of course, it might not be the case, but it is important to compare to class data to see if it is true, and then provide additional support where necessary.

‘For example, Andrea’s IPR shows lower ability scores for Maths, so there may be a risk that he will have subconsciously neglected STEM areas. Of course, his performance is not in itself weak - but it is weaker than other areas.’

The challenge for English language learners

case-study-graph2-diagnosing-gaps‘In this particular situation post online learning, many of KTJ’s students will have been spending 4 ½ months in an environment where opportunities for the social practice of speaking and reading English are limited, and EAL students will almost be doubly impacted.

‘For example a Year 9 Japanese boy, Shinsuke, was initially identified (based on baseline data and predicted grades) as benefitting from a transferable skills programme run for weaker students in place of one IGCSE. After looking at attainment data from Term 1 and 2, it was felt he had made sufficient progress not to need this intervention after all, but he really struggled during home-based learning and so will be following the transferable skills course.

‘Two important reasons he said he was struggling was because of language understanding and isolation, as quite a sociable boy he talked and practised English constantly, whereas back in Japan with a family who did not speak strong English his opportunities to speak were lessened.

‘This example is quite common for a number of KTJ students, and indeed any school around the world where the Vocabulary score is very much lower than the maths and patterns scores.'

Making predictions

‘KTJ currently uses Yellis and Alis to set Subject Specific Predicted Grades for IGCSE, AS and A Level.

‘These are used by Heads of Department and teachers as the basis for their targets, but these are seen as best guesses, rather than a locked in prediction.

‘The other important element in target-setting is teacher judgement - for many reasons, there are a few cases where students’ Yellis predictions vary from alternative data, so the lived experience, as it were, of teacher judgement is seen to be essential.

‘In addition to the use of predicted grades, we also use the value-added analysis post-exams to monitor student performance, evaluate successful strategies and provide support where necessary to staff and students. We also compare ongoing assessment data against progress made towards their target in this way, and we will continue to do so after we return to school.’


Chances Graphs


‘The other element of CEM assessment that will be particularly useful in school in September are the Chances Graphs. By the time KTJ returns to face-to-face teaching on 1st September the need to mentor and nurture them back into the routine of class, homework and independent study is greater than ever. A useful tool for these conversations will be the Chances Graphs.

‘One argument against sharing predicted grades is that it can act as a limiter as well as a target, and so the use of the Chances Graphs is a great way to promote a growth mindset at any time, but particularly after a period of inertia.

‘In this example, the Predicted Grade for this student is a C, but whilst it is the highest specific chance, there is also a 43% combined chance of getting a B or higher - so when carefully used in a mentor/tutor conversation this can be a powerful tool to promote positivity.’

A bigger picture: moderation and triangulation

‘Finally, with regard to IPRs, we take the view that all data must always be seen as part of a bigger picture. Standardised scores and predicted grades are indicators, but teachers need to combine this with accurate assessment.

‘No data, whether CEM, class assessments or more formal summative assessment should be taken as a stand-alone data set, but should be moderated and discussed within departments, year groups and between class teachers. So, at KTJ, we triangulate data in a few ways: CEM data - both Standardised Scores, Predicted Grades and Chances are shared with teachers and Heads of Departments so they can actually evaluate the data, and compare to their own data sets.’




Want to know how CEM data can help in your school?


Find out more