I have a submitted a paper to a IEEE letter. Unfortunately, at a later stage I noticed that letter is still in process to have an impact factor. I already received reviews from reviewers and the journal is willing to accept the paper after minor revision. Is it ethical to withdraw the paper at this stage?
It is fine (and ethical) to retract your paper at any stage. Also, you have no obligation to justify the retraction (but the editor will be curious, of course). The reviewers have put in work so it may not be NICE, but that is another story. In the end it is YOUR work and your paper, and you can decide what to do with it. If it deserves to be published in a (much) better journal: go for it.
On the other hand, if the journal is not a "predatory journal", your paper is close to being accepted, and the possible increase in impact factor is minor, it may not be worth the extra work to change the formatting and go through the whole review process again.
There is nothing to do with ethics as question asks. This is simple professionalism. The journal has devoted enough time for reviewing the manuscript. The reviewers have put in their efforts.
If you are really fixed about not go ahead with submission of revised manuscript, then do the following:
Don't submit the revised manuscript to the journal.
If the journal editor asks about the revision, then say that you are extending the article in various ways and would not submit the manuscript. But, thank them for their efforts.
But, I would suggest that you had made a mistake not looking at the journal statistics, you should go ahead with this journal with the publication.