Over the past couple of years, NAPLAN has been a major talking point in Australia and around the world, and just this week ACARA released the 2018 preliminary NAPLAN results.1 This comes after weeks of delay due to concerns from the teachers’ unions over whether the online and written format test results could accurately be compared.2
The American authors of the Perelman Report shared similar concerns, and have called for the 2018 NAPLAN results to be scrapped entirely3. Professor Perelman’s argument is that the results between the online and pen-to-paper formats are statistically unable to be compared against one another. Among other factors, Perelman believes the adaptive testing design of the online version, which adjusts the level of difficulty of the questions asked in accordance to the students’ ability, can skew results and therefore cannot be compared like-for-like.
Chair of the Measurement Advisory Groups, Ray Adams, released a statement rejecting Perelman’s claims saying the report was “under-researched and lacks the technical understanding,” and further backed NAPLAN Online stating, “there are numerous examples of international assessments, similar to NAPLAN, that have clearly demonstrated comparability between online and paper versions of a test.”
Around 20% of Australian students completed the online format of the NAPLAN test this year, however by 2020, all Australian states and territories will make the move to online. “NAPLAN testing will become more effective when it moves to online testing” according to Federal Education Minister, Simon Birmingham, “as schools will receive almost immediate feedback and have the ability to adjust programs and implement changes.”
As for the results, since NAPLAN was introduced in 2008, they reveal that there has been gradual improvement across most of the domains tested. However, writing results have declined. On average, one fifth of all students are failing to meet the minimum standards in writing, and this year reports the worst results since NAPLAN began.4
Interesting findings did emerge when comparing the writing results from the two test formats. On average, students who completed the test online received noticeably better scores than those who completed it with pen and paper.5 This could be attributed to the fact that students who completed it online had the ability to edit their answers and are more accustomed to typing on a computer or device. ACARA Chief executive, Rob Randall, also admitted that this may have impacted student results, “Students who sat the online writing exam may have done better because they could more easily edit their work. Kids who have paper and pencil haven’t got that, [and] we have observed there is a difference”. 5
When looking at the 2017 versus the 2018 results, there was relatively little change in all domains across the Australian states and territories.6 Here are some of the key preliminary findings:
- NSW, Victoria and the ACT continue to perform well, with their scores either reaching or exceeding the national average across all year levels of the domains tested.
- The Northern Territory under performed across all year levels of the domains tested.7
- Western Australia’s year nine’s saw positive results, with all students meeting the average or exceeding the national average. However, for years three, five and seven, most students fell below the national average.
- There was improved grammar and punctuation results for year nine students across all Australian states and territories8
Now that ACARA has released the results to the state and territory curriculum authorities, schools will begin to receive their individual data, and reports will be forwarded to parents in the next two weeks. For teachers, these outcomes provide insights into potential learning gaps at a student or cohort level, and can be used, within reason, as a guide to determine how well Australian students are learning core English and Mathematics skills.
Finally, teachers may use this information to adjust their teaching plans, but it’s also important to remember that NAPLAN is not the “be all and end all,” as there are many measures throughout the year to determine students’ ability.