This kind of advice is often given to people, but in my view it's fundamentally misguided.
It's true to some extent that many of the top conferences have a particular way of doing things, and that people in the community associated with a conference will often have more experience with that way of doing things and find it easier to get papers accepted there. However, it's untrue that the communities associated with particular conferences are a closed shop: it's entirely possible to adapt your way of doing things to what you know different conferences expect, and that will improve your acceptance rate at those conferences. Moreover, it's absolutely worth your while to do this, because publishing at the top conferences massively improves your chances of having a successful academic career (not to mention that it puts you in contact with a large number of good researchers in your field).
In that sense, the people who are suggesting that you eschew the top conferences and simply send your work to second-tier venues are doing you a serious disservice, because they're basically ensuring that you're not even on the pitch where you could compete with your peers (indeed, the heavy focus that top groups put on publishing at top conferences tends to mean that they are underwhelmed by people who don't compete). If you're good and you put in sufficient effort, you can definitely play at the right level, whereas counting yourself out of the game before you've even tried to succeed is deeply unwise in my view.
TL;DR The people giving you this advice are leading you astray in my view. It's not wise to spend your time and energy submitting papers to venues that many researchers will view as being less than top-flight, when you could be refining your papers to make sure that they get into the top venues.
More generally, people unfortunately tend to give similar advice about all kinds of things, not just academic conferences. A classic example would be applying to university, or applying for top jobs. The general form of such advice tends to be:
- This venue is biased against you specifically, or against people like you (whatever that means) in general.
- This venue wouldn't suit you if you were accepted -- you'd enjoy it way less than a lower-tier venue.
- This venue isn't all it's cracked up to be anyway -- they just have a high opinion of themselves.
There are certainly situations in which people might be a bit biased, or in which you wouldn't enjoy some place, or in which the place isn't as good as it thinks it is, but these things are by no means universally true. If people tell you "don't apply to X because I have specific, verifiable evidence of Y", then that may be one thing (check the "verifiable evidence"), but if they tell you not to apply to top venues in general, then you need to seriously question that.