First a bit of chronological background:
PhD in pure math (2 publications, 1 preprint), 1 year of postdoc in pure math (unproductive) 2 years of postdoc in computational medical imaging (1 publication each, and one of these postdocs were in application of differential geometry and machine learning for medical imaging), 1 year of postdoc in computer vision (unproductive, as I was partially spending my time to move to industry - I accept the blame here, mostly as a data scientist or machine learning researcher), 2 years of failed industry experience (4 jobs, all permanent, all let go within the first 4-5 months in the respective trial periods). So in essence, 4 years of postdoc, partially productive and 2 years of failed industry experience after PhD.
What's going on now:
I've a strong realization that the traditional industrial positions are not from me, and I'm planning to go back to academia, as well as on my way to obtain my freelance status, so I'd be able to consult on the side. But since I worked 2 years in industry immediate prior, I feel I need to get a long (at least a year and a half) project to get back to the full research mode, find a line of projects and use that to get a permanent position. These projects wouldn't be in pure math, but rather in theoretical machine learning, statistics or applications of differential geometry/topology to solve data analysis problem.
Immediate next - a potential answer to a question I might be facing during my next hiring:
I'll be having a few interviews, and I'm sure some postdoctoral advisors would ask me why I'm not looking for a permanent position, since I'm already 6 years past my PhD. My honest answer would be: "Initially I thought that the academia wasn't for me, that the stakes of getting a permanent position are pretty low, and even then, I thought it wasn't rewarding enough, me being based in Europe and not planning to move. So I wanted to move to industry but in R&D and I thought I'd be doing almost fundamental research; but I was wrong - the industry mindset and environment I found to be very different than the academic one, and hence I'm realizing my mistake only now, and trace my way back into academia."
But if I do give that honest answer as I wrote above, will it hurt my chance to get into postdoc positions that I'd want to get into? I don't want to paint a wrong image of myself to my future academic employers, but I fear that if I tell them that I moved into industry as I thought I wasn't good enough for academia and it was not rewarding enough, then my potential academic employers might think that I'm coming back to academia as a second choice and not out of passion. But the thing is that I had to go through a two year period of self-realization, and yes I did make a career mistake, which I'm planning to rectify. So how do I convey this truth in a manner that doesn't make me look like I wanted to run away from fundamental research (in fact I didn't, I planned to do it, but outside the scope of academia, which did fail and seemed almost non-existent) and work against me in an academic hiring process?
If you want more details about the reasons for being let go:
They're mentioned as comments to the answer by @jerlich.