I disagree pretty strenuously with what you have done.
First of all, being an alumnus is not an "affiliation". If you weren't a student or employee of the university at the time you did the work, it's not your affiliation and should not be listed as such. You're welcome to list your alumni email address if you like, but that's your contact information, not your affiliation.
I would say your affiliation is your employer, the publisher. Or, if you feel strongly that the work was done "on your own time", then you may omit any affiliation.
However, independently of any of this, you had a duty to disclose your employment by the publisher, as a potential conflict of interest. Conflicts of interest may exist regardless of your affiliation and must be disclosed. You can certainly add additional information that you think may help the reader evaluate the issue (e.g. "I was not involved in the review").
I have to say I'm pretty concerned with the notion that you would "get out of this" by concealing the potential conflict. That's exactly the wrong instinct, and contrary to every principle of academic ethics. You address conflicts of interest by disclosing them. The fact that "nobody has complained" isn't dispositive; it's not their job to find and disclose your conflicts for you. What you've done can be grounds for retraction all by itself (see for instance https://retractionwatch.com/category/by-reason-for-retraction/failure-to-disclose-coi/), though if the omission seems to be accidental, a published correction can suffice.
If the letter hadn't yet been published, I would advise you to contact the editor and ask for a conflict-of-interest disclosure to be added ("the author is an employee of XXX but did not participate in the review of the cited paper"). Or if that is not possible, to withdraw the letter.
In this case, you say the letter was published long ago, so it's less clear what to do. Probably the safest course of action would be to write to the editors, apologize, and ask that a correction be published. I don't think it's necessary to correct the affiliation per se, but rather to correct the fact that your potential conflict was not disclosed. It's possible they will decide that retraction is more appropriate, but that's their call.