I worked on editorial boards of 3 journals. Academic/business affiliation is completely  irrelevant for us choosing a referee, what matters is the relevance of person's prior work (typically in the form of publications) to the paper. That's all.  

Regarding specific questions: 

1. "How will it look if the editors, in effect, publicly admit to sending manuscripts for review to an individual who is outside academia and not affiliated with any institution or even a business company?" - it will look just fine. A journal is judged by the quality of its publications. The community knows all too well that sometimes referees/editors make mistakes and accept faulty papers that have to be subsequently withdrawn. As long as your journal does not have this problem, everything is OK. 

2. "It raises questions and thereby somewhat undermines the reputation of the journal." - no, it does not in the slightest. 

3. "To make things worse, I haven't published anything since I left academia years ago." - this is irrelevant. 

4. "The editors know about my expertise from my reviews, but how can the scientific community know?" - scientific community judges quality of journals by other parameters. Typically, community trust editors to do the right thing. This trust is undermined by faulty publications, but that's a separate issue.