Physics? Finance?
If you're interested in the mathematical portion of finance - then you take mathematics, not physics, I'd say.
If you're interested in the mathematical portion of physics - then I believe there is a major called "Mathematical Physics". I know someone doing that.