I'll have to guess a bit on this, never having heard of the situation before. I think you are wise to be worried, though. The intention of post docs is to provide research (sometimes also teaching) experience to people who have just finished their doctorates. Some positions will be explicit about that.
But even if you are technically eligible, I think a lot of people would wonder about you if you took that path. "Why is he leaving a regular position..." They might even question your qualifications since leaving an Assistant Professor position (in the US) often means that tenure was denied or the candidate assumed it would be. That isn't a good impression to leave, even if it is inaccurate.
Assistant professor may mean something different in your country, of course. But consider the "expected" career path of such a person in your decision.
On the other hand, it is also (US, again) possible to move from an Assistant Professor position at one university to a similar, regular, position at another. The new position might enable more research possibilities, especially if there is a larger, more active, faculty.