This is the way it is for me.
Cantonese - Speak slowly or If I can read the peng yum, I'd be able to understand most of it. Speak fast or normally I can't understand jack all.
Mandarin - I can read and say a lot of words. I have absolutely no clue wtf im reading. It makes no sense to me...