Well. It depends on:
1). How well you do at school naturally.
2). How much effort you have to put in to do well. It could be different for different people. Someone who has a natural aptitude for maths could do a lot less study than someone who struggles to understand it and still do just as well.
There isn't really a set amount of hours you should do each night to ensure that you get a 95-99 ATAR. The best thing to do is to work steadily throughout the year, ensure all your assignments are well done, organise well and get notes done, and practice. You should have a rough idea of how much you need to study to get the ATAR you want. But I'm aiming for above 95...and do about an hour and a half - two hours a night, probably. One day of each weekend is generally study, and the holidays are mostly studying. Hope this helps!