Depends on the subject. For maths, I'd definitely go with exercise books because per page, they're cheaper than reinforced loose leaf paper. For content heavy subjects/subjects that get a lot of handouts of stuff, I'd use a binder so that you don't loose the handouts. But again, if you use loose leaf paper aka binders, and you're not careful, once you loose one sheet of paper, you'll miss whatever content was on that sheet.
I've been using exercise books for every subject this year and now my room is filled with handouts and regret
For subjects such as Society and Culture, Legal Studies, Visual Arts (basically humanities and creative arts) I definitely recommend using binders or folders (except for a visual diary for Vis Arts obviously). Otherwise, for sciences and maths use exercise books.