Empiricism is often involves in mathematical discoveries but the key difference is math isn't reliant on empiricism and at least in theory the current body of mathematical knowledge can reasoned entirely from simple axioms.
You recognize that "empirical science" has a clear meaning to most scientists, right? This idea is quite different than formal reasoning (e.g. deduction over theorems).