I worked on editorial boards of 3 journals (for the record, my journals do not give referee awards). Academic/business affiliation is completely irrelevant for us when choosing a referee,referee; what matters is the relevance of person's prior work (typically in the form of publications) to the paper. That's all.
Regarding specific questions:
"How will it look if the editors, in effect, publicly admit to sending manuscripts for review to an individual who is outside academia and not affiliated with any institution or even a business company?" - it will look just fine. A journal is judged by the quality of its publications. The community knows all too well that sometimes referees/editors make mistakes and accept faulty papers that have to be subsequently withdrawn. As long as your journal does not have this problem, everything is OK.
"It raises questions and thereby somewhat undermines the reputation of the journal." - no, it does not in the slightest.
"To make things worse, I haven't published anything since I left academia years ago." - this is irrelevant.
"The editors know about my expertise from my reviews, but how can the scientific community know?" - scientific community judges the quality of journals by other parameters. Typically, the community trusttrusts editors to do the right thing. This trust is undermined by faulty publications, but that's a separate issue.