I'll tell you my story:
Back in Year 11... my worst subject was definitely Physics... (despite going tutoring for Physics, Economics and Business Studies were definitely better because I just liked Social Science more)
So yeah... I just didn't put enough effort into Physics...
At the end of the year, I was contemplating on whether I should drop it or not.
Then I was like ''I probably can handle 12 units'' and basically I didn't want to waste all that tutoring money... (basically too late to back off now)
Then up until now for Year 12.... I still don't put enough effort but surprisingly, my Physics ranks have improved...
For the first 2 assessments, I ranked really well (approximately Band 6 'rank')...
For my recent assessment, I did alright I guess (high Band 5 'rank')...
Right now, my worst subject is probably either MX1 or MX2.
MORAL OF THE STORY:
Just keep it for awhile.... cause you never know what can happen when you hit Year 12.