First of all, I assume that you do not mean "colleague" in the narrow sense of someone at your institution? That would make a (nearly?) universally inappropriate choice of reviewer, right?
Regarding your question: I must admit that I don't have much direct experience with "suggested excluded reviewers". I am almost never asked that question on papers I submit; I think I might have done so once: when I did it, the idea was to help out the editors by pointing to someone that they might have thought had a lot of directly relevant expertise but that I knew would not actually be so interested in reviewing the work in question. The point is that I interpreted this as a (ever so slightly pathetic, I might add: I think this occurred at, by far, the lowest quality journal I have ever submitted to) request from the editors for me to help them out in finding a good person for the job, not excluding potential reviewers for "conflict of interest" reasons.
Because of this I am not totally clear on what constitutes a "conflict of interest" for a reviewer: in doing any referee work one volunteers to give one's professional opinion. I don't have any "academic enemies", but moreover I once recommended for rejection a paper by one of my closest friends: and it was a very good paper; I just thought that it was not good enough for the journal to which it was submitted. (Because I knew the person I also felt that he could get a better publication by working on it a little longer, and in fact that is what happened: about a year later he published a magnificent paper in a higher quality journal.) Oh, back to you: is it clear that your colleague would actually be an inappropriate referee for the paper? Were his opinions about doing the alternate method actually fully professional opinions, or is that just his more personal reaction to the paper, reading it in terms of his own interests? Moreover, does his alternate perspective really lack validity in some sense? If so, then wouldn't a good editor not choose him anyway?
To summarize the above: it is not clear to me that "X already read my paper, and he didn't like it so much" is a sufficient reason to exclude X as a potential referee. You wonder what the language of exclusion should be, and I agree: you'll have to work a little harder to paint this as a conflict of interest: I'm not seeing that.
But okay, now a direct answer: well, they asked you if there's anyone you'd like to exclude, so it's really up to them to evaluate the reasons. If you don't want this person to review your paper I would say so using exactly the reasons you've told us: then it is up to the editors to decide whether to grant your request.
Let me give one other piece of advice which may seem a little shady at first but which I claim is mostly practical:
Consider not acknowledging your colleague in the first version of the paper you submit, knowing that you will put this acknowledgment back in the published version.
Concerning this, let me first say that you cannot do it if his contributions were beyond a certain point: it would then simply be academic dishonesty. But many acknowledgments are subjective largesse: you have not just won an Academy Award, so you don't thank every single person that helped you.
Anyway, the reason why I recommend this if it is possible is to short-circuit the dopey editorial practice -- which I have seen happen several times -- that the editors send a paper to a certain reviewer because she is explicitly acknowledged. What a frustratingly lazy practice this is.
The editors who ask the author's opinion on referees tend (in my direct experience) to be lazy types, who just might choose someone in the acknowledgments...and might do so thinking they are doing you a favor! So the idea of delaying the acknowledgment to forestall a bad editorial choice seems ethically defensible to me. Finally, if the editors are asking for the author's opinion perhaps they will also ask you for especially plausible referees? If so you should think hard and suggest really good choices: i.e., the most qualified people, not necessarily the ones who would like your paper the best.
Added: After airing out my advice to unacknowledge your former collaborator X, I am having trouble standing by it. Based on what you say there is a good chance that X actually would be the referee, and if he can then see that your paper has been modified according to his advice but that you have not mentioned him, then it is possible that he might be personally hurt by it and that this might come out in the referee report. I think the "unacknowledgment" suggested above is only feasible if the version of the paper you're submitting does not bear any mark of X's helpful suggestions. As I said, whether X is an appropriate referee for your paper really is up to the editors to decide. You can help them out by giving them all the appropriate information. Doing much more than that could be ethically problematic...
Added: Upon further reflection I can only clearly remember the practice of being asked to referee a paper in which I was acknowledged happening once. I'm sorry for the inaccuracy. However that one time was from a leading journal, and I made sure to ask whether the editor was aware that I had been acknowledged in the paper, and he was.