Let me paraphrase a Feynman quote once more (although this quote is maybe a bit too "expressive" — hopefully I don't get banned for it!): "Physics is like sex: it has some practical use, but that's not the reason why we do it."
Anyway, arithmetic isn't really math, it's just number-pushing, and I totally understand why you don't like it. That's even the cause of my love/hate relationship with algebra, although I'd say I'm really good at it. However, people (myself included) sometimes tend to get a bit upset when they spend 2 hours calculating something, only to see that 1½ hour ago they have committed a sign error and most of their work is only good for throwing into the trash can .