Speaking from experience, a band 4 will affect your ATAR quite substantially (igotb4ineconomics), however it does depend on the mix of subjects, strength of your cohort, as well how well you perform in your b5/6 subjects.
A tip however to try and maximise your ATAR, is to focus on the subjects...