"Without mathematics we cannot penetrate deeply into philosophy.
Without philosophy we cannot penetrate deeply into mathematics.
Without both we cannot penetrate deeply into anything." _ Leibniz
There is mathematics in it, but it's not just mathematics; it was originated, and is still used in areas like AI, as an attempt to formalize thought, i.e. a kind of formal philosophy. A good portion of the foundational work was done by philosophers (e.g. Frege, Russell), and a lot of current work (especially in modal logic) is done by philosophers as well.
Usually logic is considered part of mathematics. However, in an important sense, logic precedes mathematics. You can't do mathematics without using logic.
Many have, take the sort of mathematics done in order to understand physics. Logic is the study of formal reasoning. Mathematics is the study of form and structure. It took R&W's Principia Mathematica to attempt to show that the entirety of mathematics was rested on a logical foundation. That's pretty recent on the scene if you think about the history of mathematics! To me, the idea you expressed, seems like a very 20th century thing.
> Math is a symbol game
So is logic.
> philosophy is a discussion of human problems.
No. It's not.