Does a low score in the range of 60th percentile rule one out of top 20 programs?
I am already confused by "a low score in the range of the 60th percentile." A score in the 60th percentile is, by definition, high rather than low. A math PhD student should know that. :)
In terms of whether that score would "rule you out": again, every program and even every member of every admissions committee has to decide how to weigh the various factors. But that is why you apply to more than one program. I think that if your application is otherwise magnificent, you are a very likely admit at several top 20 programs.
As an international student, I have no idea how terrible such a score is and I am considering stopping the application process to US universities since I doubt if I stand a chance.
Yes, you stand a chance, so please don't stop your application for this reason. The smart thing to do is to divide the programs of interest to you into tiers and apply to a few schools in each tier. For instance, my program (at UGA) is about the 50th best in the US, and for us a 60th percentile score would in all likelihood not hurt your application at all. So it would be smart to apply to some schools in the UGA tier. (Perhaps even UGA itself...)