Even before the pandemic, not only in upper-tier "research" universities, but also smaller colleges, in the U.S., in math, "visiting assistant professor" positions were/are mostly for very junior people who somehow did not have any better job offer in the current cycle, but/and will come and teach. Prompted by @Alexander Woo to be more precise: in some cases these positions do not have such an onerous teaching load but that maybe they can enhance their CV in a year or two, and then go back on the job market. In others, the teaching load is "normal" or worse. In fact, this ambiguity is surely worth tracking for people contemplating taking such a job: they're not all the same. Get details.
More senior people, who do not need any sort of temporary job to keep a paycheck while looking for something better, would usually just be "visitor", or not even have any title. This might include people, with tenure elsewhere, on sabbatical from their home institution, who may or may not teach at the host institution. If they are given a title, it might well be "visiting assoc prof" or "visiting prof"... but/and, often, by the time people get to such positions, they are full professors at their home institutions, so they'd just be "visiting professor".
Hence, few visible "visiting assoc profs".