Because computing science (see below), and even computer programming, is applied mathematics.
Consider what mathematics is, at the most general level. You choose or create a language in which you can express certain ideas and then do symbolic manipulation according to a set of rules you've also chosen or created to come up with to create more valid statements in that language according to those rules. If you're not careful to do this correctly, you may come out with invalid statements. The results you come up with may have some sort of application in the "real world" (e.g., I can use the language and rules of "integers" to help keep track of what people owe me and I owe them) or may be just work to help you better understand how you can use the language and rules and how they can be helpful to you in further use of them.
In many parts of mathematics we use particular symbols called "numbers" and have a large library of oft-shared rules and languages related to this, but there are other areas of mathematics that don't use numbers at all (e.g., category theory), or, though they can be applied to numbers, are not really about numbers per se (group theory, algebraic structures, many more).
Even before you get into the study or use of particular algorithms and the like, writing a computer program is basically what I described above. Many of the "simplest" concepts in computer programming that we use every day, such as the idea of a function, are purely mathematical concepts.
Now as you've seen, it's perfectly possible to attack real-world problems with these mathematical tools in a non-rigorous way and get useful results. Typically the results will not be truly correct (i.e., your programs will have bugs), but they will be "correct enough" to do the job. (For a well written program in industry, you may never even encounter the situations that would demonstrate that it's incorrect.) That's what the discipline of engineering is: getting results that work well enough in the real world at acceptable cost.
But even when you're doing engineering, much of what you do works well only because someone has gone and done enough mathematical heavy lifting to give you concepts and tools that you can use to do this. You may not have a really good understanding of what a function or a relation is, but your programming language or database system works because somebody did figure those out.
And the people who did that work are the computing scientists.
All this has been known and seriously contemplated for a long time. I think it's particularly well demonstrated by a comment in Peter Landin's classic 1966 paper ["The Next 700 Programming Languages"][landin66]:
The most important contribution of LISP was not in list processing or storage allocation or notation, but in the logical properties lying behind the notation. here ISWIM makes little improvement because, except for a few minor details, LISP left none to make. There are two equivalant ways of stating these properties.
(a) LISP simplified the equivalence relations that determine the extent to which pieces of a program can be interchanged without affecting the outcome.
(b) LISP brought the class of entities that are denoted by expressions a programmer can write nearer to those that arise in models of physical systems and in mathematical and logical systems.
If you understand this (which probably requires some at least intuitive understanding of the of the lambda calculus or similar), you probably realize that a lot of the problems we deal with today are still the same basically mathematical problems that were being investigated back in the '60s when we were first seriously investigating what a "programming language" really is and means.
On Working Programs
One can also look at this from the more narrow viewpoint of, "I just want to write a program and make sure it works." Even here this becomes math if you take as a constraint "I really do want to, as best I can, make sure it works." Dijkstra's EWD303, "On the Reliability of Programs," makes this argument in detail. His summary:
Reliability concerns force us to restrict ourselves to intellectually manageable programs. This faces us with the questions "But how do we manage complex structure intellectually? What mental aids do we have, what patterns of thought are efficient? What are the intrinsic limitations of the human mind that we had better respect?" Without knowledge and experience, such questions would be very hard to answer, but luckily enough, our culture harbours with a tradition of centuries an intellectual discipline whose main purpose it is to apply efficient structuring to otherwise intellectually unmanageable complexity. This discipline is called "Mathematics". If we take the existence of the impressive body of Mathematics as the experimental evidence for the opinion that for the human mind the mathematical method is, indeed, the most effective way to come to grips with complexity, we have no choice any longer: we should reshape our field of programming in such a way that their methods of understanding become equally applicable, for there are no other means.
On "Computing Science" versus "Computer Science"
Some amongst us, including the University of Alberta, find the more common name of the discipline slightly misleading and instead prefer to call it Computing Science. As Keith Smillie said in "Computing Science at the University of Alberta, 1957 - 1993":
The choice of the name "computing science" instead of the more common "computer science" was deliberate in order to indicate that computing rather than computers was to be the foundation of the discipline.
Thinking about what we are wrangling with as "computing" rather than "computers" way may help you remember that all the software running the world today is much more dependent on the mathematical tools we use to be able to effectively and accurately model our problems and the world than on the hardware on which it runs.