"Numbers is to mathematics as letters is to language."
Not all mathematics is concerned with numbers.
It's not?
No. Geometry is the easiest example,
actually. Sure, you can express it numerically,
but that's not necessary. Formal logics are
also a branch of mathematics, and they don't
deal in numbers (unless you want to consider
True and False numbers - but that's just a matter
of convention).