I contacted several ones who have even cited my papers, but they simply replied "they cannot be my reference as they do not know me", meaning that why they bother to recommend someone they do not know.
This is to be expected. The purpose of letters of reference is to provide a personal perspective on a candidate from an eminent figure who knows the candidate well and is able to compare her/him with peers. The simple fact that they cited you means close to nothing. I have probably cited hundreds of researchers, maybe thousands, some of which I don't even remember the name. There is also the question what these references are actually supposed to write in their letters if they don't know you personally - "Mia wrote this pretty good paper in 2012, but I have never met her or talked to her outside of when she asked me for this reference." isn't a great letter of reference.
This gave me a bad impression that no matter how hard or good you work, career success depends where you are and how many people know/care about you.
It is a bit of a depressing way of putting it, but at the end you are not incorrect. "Knowing people", or more correctly, "making people know you" is indeed very important for a researcher who wants to progress her/his career.
How to find professional references?
This depends a lot on your level. As a student, you typically want your professors to act as references. As a more advanced researcher, which you seem to be, it is very important that you:
- Find and occupy a niche (that is, if people think about subsubfield A in area B, yours should be the first or one of the first names that pop into people's mind). This works only if you repeatedly annoy people about a subject in papers, keynotes, lectures, and tutorials.
- Go to conferences, write joint papers, and work on joint proposals related to said niche
- Visit other universities, or invite researchers from other universities working in your or a related niche
- In this way build a substantial network of collaborators and co-authors
- Follow-up on on your network, make sure that people that you once knew are still aware of your recent work
- Maintain reasonable standing with the important players in your larger area
If you end up with a reasonable network, which clearly takes time and dedication, you typically want to select the most eminent members of it suitable for a given application to act as references.