I wrote my statement of interests where one paragraph is dedicated to discuss my past works. But I wonder if that is necessary because it feels like repeating abstracts from the papers. And probably it will not make much sense to those who do not specialize in those topics. Should I just refer to the paper and make no formal introduction of what I did? Since there is a word limit, it is really hard for me to explain all those ideas in just a few words without assuming any technical prerequisites, and I feel it is not really stating my interests and my visions of my doctoral studies anyways, because I'm open to other related topics as well and don't want people to think "oh, this is a XXX-guy and our faculties do not work on XXX so he is a mismatch".
I don't think Ben's answer says this strongly enough:
Your statement must include a description of your past work, if you have any.
The primary feature that PhD admissions committees looking for in applicants is evidence of potential for high-quality independent research. The strongest possible evidence of your potential to do high-quality independent research is the fact that you have actually published high-quality independent research.
It doesn't matter whether your past research is on the same topics that you want to pursue in the future. It's easy to communicate a change in interests in your statement with a single sentence: "More recently, however, my interests have shifted to quantum tropical M-theory." (Obviously this sentence should be followed by a description of your specific interests in quantum tropical M-theory, written with enough maturity to convince experts in quantum tropical M-theory that you have the necessary knowledge and skills to be successful in quantum tropical M-theory, but that's what you were going to write anyway.)
Crucially, if your previous papers have coauthors, your statement (and your recommendation letters!) should focus on your specific technical contributions to those papers. Remember that you are not trying to sell the work as good mathematics; you are trying to sell yourself as a good mathematician.
In particular, you are not trying to sell your past work to experts in your subsubfield of (past) interest, so you should not merely quote the abstracts. You are trying to sell yourself to a general audience of trained mathematicians, some of whom are experts in your subsubfield of (past) interest. So your description needs to be understandable to that broader audience, but still interesting enough to experts that they will want to read your papers for the actual technical details.
Obviously you should provide direct links to arXiv versions of your papers, in both your CV and your statement, so that interested committee members can read them without having to search.
[I'm an American theoretical computer scientist, which means I'm like a mathematician, except that I write less code.]
I wrote my statement of interests where one paragraph is dedicated to discuss my past works. But I wonder if that is necessary because it feels like repeating abstracts from the papers.
If your past work relates to the research interests you are raising in your statement then it would not be out of place to mention these works. Indeed, in many cases this will be useful context, since it sets out what you have already done, and frames your interests in the context of existing work. Mentioning these works would not necessarily entail repeating information in the abstracts to those works. Bear in mind that even if you do not mention your past papers, they will be present in your CV, so the academics reviewing your application will know that you have written previous papers. Ultimately it is up to you to decide if these papers add useful context or information about your research interests.
...probably it will not make much sense to those who do not specialize in those topics. ...
Then it is badly written and you need to write it better. Your statement should be for a general audience of academics in your broad field, without necessarily having specialist knowledge in your particular topic. These people are experienced researchers with PhDs and probably decades of experience in your degree area; if what you write does not make sense to them then you have written it badly, and you need to smack yourself on the wrists and try again. Learning to write about technical research in a way that does not get bogged down in minutiae is an acquired skill, but it is one you need to practice for entry into a PhD program. It should not be impossible to explain your past research in an way that makes sense to experienced researchers, even if your past research falls in an area that is not their particular specialty. Keep trying to write out a good explanation, and explain the essence of the work while shedding any unnecessary technical information, until you have something that makes sense.
Should I just refer to the paper and make no formal introduction of what I did?
There would be no point to that, so no, you should not do that. If you mention your past papers at all, it should be to provide context for your research interests and the present context of that research.
...I feel it is not really stating my interests and my visions of my doctoral studies anyways, because I'm open to other related topics as well...
That will take about one sentence in your application to explain.