As others have noted, in most fields postdocs are very competitive. For the lab director and/or hiring committee, there is also a lot at stake. Choosing poorly can mean the postdoc "flakes out". At best, a slot and resources are wasted. At worst, a promising stream of research goes nowhere. The topic may become stale, experiments or analysis may be flawed, or completing the work carries baggage (an unproductive, unsatisfied (co)author to cart along) so harder to get someone else to finish it off.
What this means is that directors or committees are pretty risk averse, and pretty likely to shy away from candidates with something strange in their file, like silence or a weak reference from their advisor. That's expecially the case if someone else is in the pool whose file is more normally complete.
Therefore, without knowing specifics of the situation, my Occam's Razor hypothesis is not that your advisor is actively torpedoing you in some way. Merely that the absence of their strong recommendation, with no countervailing clear evidence of your brilliance obviously visible in your file, indirectly turns you into an also-ran in the post-interview deliberation.
Made-up but plausible committee dialogue:
As you know, X's file is a bit strange. There's no letter from their advisor, though their other references are OK. We brought them in, and their interview went quite well. Personable. Pretty good talk. But you know, it's weird with that advisor. I wonder if they'd fit in? Then we had Y. A more normal file. Their advisor, Z, wrote a very nice letter. Other references good. Personable too. OK talk; probably not quite as good as X. But maybe they were nervous. Z's letter explains how significant their results are and that they're very independent. ... ... ... You know, Y's just a safer bet.
If this is true, you're not sunk, you just have to work really hard to make your file stand out. Have great publications/conference presentations. Be a super presenter. Get one of your other references to really advocate on your behalf, i.e. be your advisor stand-in. You just need to be super on other dimensions to make others overcome their risk aversion.
If my Occam's Razor is dull, i.e. your unphoned reference's speculation that your supervisor was called anyway is true, it is also unnecessary to assume your supervisor actively torpedoed you by saying something bad. They merely may have damned you with a factual (i.e. not at all "defamatory") statement like:
Yes, X was my student. They completed the degree requirements. Yes, it is true I did not send in a letter of reference. X and I discussed it and I felt others could probably write a more enthusiastic reference letter. No, there was no significant problem, they completed the degree requirements satisfactorily. I don't really have anything else to say.
The solution for the future is still the same. Excel elsewhere in your file, and find a reference who can be a supervisor stand-in. Proactively (writing a letter, calling up the committee, whatever...more than waiting for a call that doesn't come.)