I was in the same position with Japanese. It used to be my best subject, but then I started losing confidence because I was losing a lot more marks than I was used to losing, and the content became harder. However, I definitely don't regret continuing :)
I'm a little biased towards studying...