If you are new to research or new to a particular field, you should discuss this question with a colleague who has experience in that field (for students, this is your advisor). They know which conferences are the most important/useful/well-attended. After some time you will know this too and can decide based on personal experience.
Outside of expert advice or personal experience, here are my criteria, in order of importance:
- It is sponsored by one of the major relevant professional societies. Which are relevant depends on your field. In mine, this means SIAM.
- The work I would like to present is relevant to the focus of the conference.
- The timing and location make sense. This can depend on many things, but it is less convenient to travel when you need to be in class (as a teacher or a student). You may have limited funds, or limited tolerance for long flights and layovers. Choosing a conference because you have some particular desire to visit that location is not generally a good idea unless the other factors here align too.
- The size of the conference, in number of attendees and also breadth of scope. After many years I have come to the conclusion that smaller, more focused meetings are more useful to me, but I know people who feel just the opposite.
In fields where conference submission acceptance rates are relatively low (i.e., the process is highly competitive), conference prestige or rankings should indeed be an important factor. In others (like mine, applied math) conference submissions are not really competitive and there is not a hierarchy of conferences in terms of prestige.
For small thematic workshops, criterion #1 above is often irrelevant; but for anything larger it is essential.
Don't choose a conference based on the food.