"Board of Studies lashed over HSC" and other stories

[lyounamu]

You can hide this advertisement by registering.

Who reckons we should have a party over this great news?

[cem]

Originally Posted by ajdlinux

Well yes, we do know the method, but we don't know enough to actually be able to do the alignment ourselves. (Same with ATAR scaling, except that's a whole lot more complicated anyway, and we can do a fairly good estimate with past UAC stats.)

I don't understand what you need to know to be able to do the aligning yourselves.

All the judges have to make their determination is the marking guidelines, the question and the performance descriptor bands.


The only thing that the judges have that a student wouldn't have is 'professional judgement' and that only comes from years of experience of teaching and marking the course.

What more do you need to be able to determine the aligning yourself?

[helper]

Originally Posted by cem

As a marker I don't get given sample answers to use when marking.


All I get is a marking criteria so I don't know what he is getting at.

I never see a 'correct' answer - I suppose marking Modern History where there is no such thing might explain it but what is in the Notes is all that I have ever seen at marking - the marking criteria, which in Modern History hasn't really changed over the last 3 - 4 years.

Sciences may be different. There is a marking scheme and marking guidelines. There also is a sample answer given.

Both the scheme and guidelines are signed off by the exam committee before they can be used. The scheme is a fleshing out of the guidelines, which is done by the senior marker and pilot markers during 1-2 days before the rest of the markers arrive. It clarifies any ambiguities but the guidelines, is always referred back to if there is a dispute over marking. The guidelines overrule the scheme.

Eg the scheme might include a overhead sheet to place onto the diagram, that shows the range of accepted answers for a graph, with the line of best fit or plotting points.

The guideline, would only say "trend line reflects data"

The sample answer given, is often written up at the last minute and often isn't a perfect answer or a complete answer. They don't reflect all the aspects of the question or all possible answers. They are basically ignored by the markers. They were published in this years notes. Looking back on them earlier this year, the amount they lack is obvious to a marker. I think they are going to cause more confusion than clarification because often they are not answers that would obtain full marks because they are only written to reflect part of the question.

[ajdlinux]

Originally Posted by cem

I don't understand what you need to know to be able to do the aligning yourselves.

All the judges have to make their determination is the marking guidelines, the question and the performance descriptor bands.


The only thing that the judges have that a student wouldn't have is 'professional judgement' and that only comes from years of experience of teaching and marking the course.

What more do you need to be able to determine the aligning yourself?

The professional judgement, access to marking schemes (NOT the published marking guidelines), an advisory committee...

Yes, teachers would be able to estimate the cutoffs using their knowledge, standards packages and all that, but most students can't do that, and most teachers are too busy.

edit: Also, even if we could align it ourselves, we could still get a different cutoff from what the Board of Studies actually used, which is all we really care about. Even a difference of 2-3 marks in the cutoff could potentially be significant.

[helper]

Originally Posted by ajdlinux

Even a difference of 2-3 marks in the cutoff could potentially be significant.

Significant in terms of what?

[ajdlinux]

Originally Posted by helper

Significant in terms of what?

Significant in terms of I prefer getting 90 to getting 88 :P

[helper]

Originally Posted by ajdlinux

Significant in terms of I prefer getting 90 to getting 88 :P

Knowing the cut-offs is not going to earn you any more marks.

[ajdlinux]

Yes, that's true. But when doing past papers, I'd like to know with more certainty what I would have gotten.

[helper]

Originally Posted by ajdlinux

Yes, that's true. But when doing past papers, I'd like to know with more certainty what I would have gotten.

When it takes days of training to understand to read the guidelines and schemes, I wonder how much better you will be estimate the mark just on reading them.

[cem]

Originally Posted by helper

When it takes days of training to understand to read the guidelines and schemes, I wonder how much better you will be estimate the mark just on reading them.

There is a full day of training on how to do the aligning and even then there is a BOS adviser with the group to explain what they are to do if they are heading in the wrong direction (based on my experience of actually doing it).

[helper]

Originally Posted by cem

There is a full day of training on how to do the aligning and even then there is a BOS adviser with the group to explain what they are to do if they are heading in the wrong direction (based on my experience of actually doing it).

In terms of aligning I would agree. You look at how long the aligning process takes, I wonder how many people will be better off knowing what the cut offs are or how.

I was more refering to the scheme and guidelines, which in science:
Senior Marker comes in for 2 days to check the guidelines and come up with a preliminary scheme.
Pilots come in for 2 days to go over guidelines and scheme to check if it will work against sample papers and refine it.
Markers come in for 1 day and are introduced to guidelines and scheme. Start looking at sample papers.
Second and third day, double or triple mark papers together, to ensure understanding, which are also checked by senior markers and pilots.
Fourth day and later, you would normally be single marking with check marking and random samples sent round to confirm consistency.

When refering to a day, I mean a night session

Obtaining guidelines and schemes are not going enable people to mark the papers consistent with markers, without the training. You see when markers change groups, how long it takes them to come up to speed on marking and that is with support.

A better push and more helpful, would be to try and make the board publish standards packages each year for each subject, like they did for the first few years of the HSC.

[Lazarus]

If you put aside consideration of the mathematics courses and certain parts of the science courses, it is true that there is little practical benefit to be gained from knowing past cut-offs.

There is, however, a great deal more transparency.

The Ombudsman observed: "No student currently has enough information available to them to fully understand how the final results are produced ... Students [will] benefit from having a better understanding of the process. The Board benefits from students being more confident in the reliability of the process and being able to appreciate how much care and thought has gone into its development." (p24)

It is this understanding as to how the final results are produced that is important. In the past, the primary reason for refusing access to raw marks has been the confidentiality of the cut-offs. Once the cut-offs are in the public domain, there will no longer be any reason to deny students access to their own raw marks.

Now, students might finally be able to obtain statements of their raw and scaled marks and catch a glimpse of how their UAI or ATAR has been calculated.

That is the real benefit, in my opinion.

The next challenge is to equip students with the understanding necessary to read those statements and appreciate how the various marks have been arrived at.

[shady145]

Originally Posted by lyounamu

Who reckons we should have a party over this great news?

sure, ill be the Dj XD

[helper]

Originally Posted by Lazarus

If you put aside consideration of the mathematics courses and certain parts of the science courses, it is true that there is little practical benefit to be gained from knowing past cut-offs.

Are you after the banding levels of the questions as well because I can't see how the subject effects the use of cut offs.

There is, however, a great deal more transparency.

Agree

Now, students might finally be able to obtain statements of their raw and scaled marks and catch a glimpse of how their UAI or ATAR has been calculated.

That is the real benefit, in my opinion..

I can't see the harm in publishing your raw marks raw but I don't think they are really going to understand anymore about how their ATARs are calculated. I think its just going to make more confusion because people will look at their own marks and start questioning why their marks were manipulated like they were because they don't have the whole cohort's marks

The next challenge is to equip students with the understanding necessary to read those statements and appreciate how the various marks have been arrived at.

Fair enough but when it takes days to train a teacher to do it for about 7-10 marks of an exam, do you really think it is going to be feasible to train students to correctly interpret every guideline?

[Lazarus]

Originally Posted by helper

Are you after the banding levels of the questions as well because I can't see how the subject effects the use of cut offs.

No - I referred to mathematics because I think it's relatively simple and straightforward for a student to mark their own maths paper, thereby obtaining a total raw mark, and then to align the raw mark using the cut-offs previously determined for the paper.

It's obviously much harder to do this with most other courses as they involve answers that are not strictly right or wrong (e.g. English). There is litte practical benefit gained from having the cut-offs for those courses.

Originally Posted by helper

I can't see the harm in publishing your raw marks raw but I don't think they are really going to understand anymore about how their ATARs are calculated. I think its just going to make more confusion because people will look at their own marks and start questioning why their marks were manipulated like they were because they don't have the whole cohort's marks

I don't think publishing raw marks will give students an understanding of how ATARs are calculated. I do think that suppressing raw marks will prevent students from understanding how ATARs are calculated.

In my mind, publishing the marks opens the door to understanding - it's necessary but not sufficient - it's just the first step.

Originally Posted by helper

Fair enough but when it takes days to train a teacher to do it for about 7-10 marks of an exam, do you really think it is going to be feasible to train students to correctly interpret every guideline?

I didn't mean to suggest that students should learn how to act in a capacity similar to the Board's teacher-judges who set the cut-offs. I was intending to refer to the tertiary entrance side of the system.

I meant that students might now be able to obtain statements of their raw and scaled marks and see how their scaled aggregate, cohort percentile and ATAR have been calculated from those marks.

I maintain the belief that it is possible to educate students about scaling, ATARs and how their marks are adjusted. I think the truth of that proposition can clearly be seen from the discussions which take place on this website.

There is no doubt that there are students out there who have no idea what they're talking about when it comes to scaling. But there are also those who have put some effort into understanding the system and are capable of explaining it to others.

The number of intelligent - and, more importantly, accurate - discussions concerning scaling etc which have taken place on this website since we launched in 2002 has increased dramatically. People still get it wrong. But there is much greater awareness of the technical points now and many more are getting it right.

In my mind the next challenge is to bring this understanding to the greater student population, either by educating them directly or equipping a sufficient number of students with the knowledge to go out and explain it to others themselves.