The revelations arising from your colleague's discoveries raise several causes for concern, which may lead your supervisor to believe that your application was misleading, made in bad faith, or even fraudulent. The issues as I see them (writing from a UK perspective) are as follows.
Not one, but two references from "friends"
It is possible that a reference/recommendation from a "friend" may be deemed inappropriate, especially a family friend. Having two references/recommendations from friends is even more problematic, even if it were permissible (because it would make me wonder why you are struggling to find anyone else to vouch for you professionally). If permissible, you should declare that the referee is a "friend" (and the reference/recommendation should also declare this). If the referee is a close friend, failure to declare the fact might be deemed fraudulent.
Might the reference/recommendation have exaggerated the extent to which the referee knew the subject?
"we reached out to my parents' friend and also got a strong recommendation"
"reached out" implies (to my British-English ears) that your "parents' friend" was not very familiar with your work, but wrote a strong reference/recommendation as a personal favour. To be honest, he/she should not have written the reference/recommendation in such circumstances, or he/she should have made it clear that he/she had never worked with you professionally. A reference/recommendation should make clear the capacity in which the subject is known to the referee, and not mislead by omission. Although it is the referee's responsibility to get this right, a lapse will, rightly or wrongly, still undermine the subject's credibility.
Irresponsible attitude that may perpetuate nepotism (or perceptions thereof, and thus incite self-fulfilling prophecies)
"My friend and my parents' friend both gave me a strong recommendation to my advisor. And of course, I excelled in the interview. So the connection is very important."
...And of course, people who go into academia with this attitude that "the connection is very important" will only perpetuate nepotism in the future (especially if/when the time comes for them to make hiring/admissions decisions). This comment is irresponsible not only because it impugns your supervisor's capacity to select candidates on merit, but, more generally, because it might discourage people without friends well established in the same academic discipline from applying. In case anybody reading this is thinking of applying for a PhD studentship, I would like to reiterate that you do NOT need to already have friends or family connections in the same academic discipline.
Having said all this, your supervisor is acting irresponsibly in asking you to leave in the manner you describe. If your conduct is deemed so inappropriate as to make your position untenable, he/she should initiate formal disciplinary action. It is possible that your supervisor does engage in nepotism (the fact that he/she had dinner with your parents is slightly incriminating… then again, that happened after you were admitted), and, now that the rumour is abroad, is hoping to get rid of you quietly to avoid getting into trouble himself/herself.