Aligning of English marks (1 Viewer)

[~ ReNcH ~]

Are individual questions aligned separately or is the entire exam aligned as a whole?
E.g. from what I can gather, this year's Module B questions was difficult for students who studied Citizen Kane but not for the rest of the students - are they aligned separately? If not, how is English aligned?

 

[Lazarus]

The entire exam is aligned as a whole.

However, where a choice of questions is available, statistical adjustments may be made to bring the results of an anomalous question in line with the others.

 

[Xayma]

Would this statistical adjustment be made with scaling as well?

Or would (in this case) the citizen kane students suffer as a result of their text choice?

 

[~ ReNcH ~]

The entire exam is aligned as a whole.

However, where a choice of questions is available, statistical adjustments may be made to bring the results of an anomalous question in line with the others.

Are statistical adjustments only made when one question is significantly more difficult than the rest? So, in that sense, is the process of adjusting the marks similar to the aligning process except within the question itself?

Btw. if the exam is aligned as a whole, then how do they publish Band 5/6 and Exemplar etc. responses in the Standards Packages? Coz one person might get 19/20 for one response, but only 12/20 for another response and end up with a final aligned mark of 98, in which case publishing the 12/20 response would be misleading.

 

[Lazarus]

Would this statistical adjustment be made with scaling as well?

Or would (in this case) the citizen kane students suffer as a result of their text choice?

The adjustments are made to the raw examination marks for particular questions, and these marks are simply incorporated into the student's total raw examination mark. This is then scaled for inclusion in students' UAIs. No students are disadvantaged.

Are statistical adjustments only made when one question is significantly more difficult than the rest? So, in that sense, is the process of adjusting the marks similar to the aligning process except within the question itself?

The exact criteria that are used to determine whether an adjustment should be made haven't been published by the Board. It isn't the same as the aligning process, because it doesn't involve the same sort of multi-linear mapping - as far as I know, the mean and standard deviation would just be altered to produce a more appropriate distribution.

See this publication from the Board:

Adjusting for variation in difficulty level of optional HSC exam questions

The inclusion of optional topics in HSC syllabuses allows students to focus on areas of greatest interest to them and provides topic choice for both students and teachers.

During the 2001 HSC, concerns were expressed that students would be disadvantaged by the level of difficulty of some option questions in examinations compared with others.

The Board has procedures in place to ensure that students are not disadvantaged for having chosen a particular option.

In setting exam papers all examination committees strive to make sure that optional questions are of similar intellectual demand.

They design related optional questions to assess a similar mix of skills at a comparable level of rigour. The demands of each optional question are then set down in the marking guidelines for that question.

During the pilot-marking phase minor adjustments can be made to the marking guidelines, making it more likely that comparable marks will be allocated to students of similar ability across different optional questions.

Despite these checks and adjustments, it is sometimes the case that an optional question turns out to be easier or more difficult for students than the others. In this situation a statistical adjustment is made so that the final distribution of marks for the question compares fairly with the mark distribution for the others.

As noted in the Masters report, the issue is not so much whether optional questions are equally difficult but whether students choosing different options are treated fairly in the marking if it is found that those options are not of the same difficulty.

The Board has a routine ‘marks monitoring and adjustment’ process that looks for any differences in the difficulties of optional questions and identifies cases requiring necessary statistical adjustment to marks. The process takes into account the performance of each group on the common sections of the paper. Where appropriate, statistical adjustments are then made to the spread of marks for each question.

This procedure is standard practice.

http://www.boardofstudies.nsw.edu.au/hsc/hsc_update_2002_02.html

That's about all we know.

Btw. if the exam is aligned as a whole, then how do they publish Band 5/6 and Exemplar etc. responses in the Standards Packages? Coz one person might get 19/20 for one response, but only 12/20 for another response and end up with a final aligned mark of 98, in which case publishing the 12/20 response would be misleading.

Good question.

Whilst the exam is aligned as a whole, raw band cut-off marks are still determined for each question (and the individual question cut-offs are aggregated to produce the final exam cut-off). I imagine these cut-offs would be used to select responses for the Standards Packages.

 

[Will_Sparky]

Okay, is this applicable for subjects such as Modern History? Because this years Germany question was tougher than most of the others.

 

[Lazarus]

Yes, it's applicable to every examination where students can choose between questions.