I noticed that many ads say "Review of applications will begin on Day X, and continue until the position is filled". I wonder how this kind of search works. Assume the search committee received a certain number of applications on Day X. Usually none of the applications is from a superstar that you definitively want to hire, but let's say between five and ten are good enough for you to hire one of them. What would the search committee do? Would they contact the candidates, arrange interviews and hire the best one there? Or would they still wait for a month or two just to see if a superstar would apply after all?
"Review of applications will begin on Day X, and continue until the position is filled" is also translated as:
We want you to get your application in on time, but if you're totes amazeballs and late we'll overlook it.
The motivation is the same as what was mentioned. If you say that there's a hard deadline for submissions, and then Dr. Awesome Sauce applies a day later and gets hired, you're opening yourself up to a lawsuit from all the Dr. Not-So-Awesomes.
Practically, academic hiring works on a calendar, and universities compete with each other. For example, in computer science it's rare to be making a first offer well into May unless you're at the University of I'm-So-Cool and know that you can afford to be last. And even then it doesn't always work. So practically speaking there's a deadline of sometime in March to make decisions on offers and start wooing your preferred candidates.
This means that you can't realistically process a new application any later than early-mid February (because it takes time to collect references and organize a trip)