In fields (like pure math and TCS) in which the convention is to list authors in alphabetical order, it makes a very jarring, striking statement to put authors non-alphabetically. The most common instance of this in my experience is when one author is very senior, enough to perform a power play on their very junior coauthor. Unless the reasons for the non-alphabetical order are carefully explained and seem logical to the readers, this could make a widely negative impression.
Non-alphabetical ordering is almost never used in pure math and TCS to signal importance of contribution: the convention, for better and for worse, is that the authorial contributions are not disclosed to the reader. As a result of this, unlike many other academic fields in which being a fifth author may be understood to mean very little contribution, in pure math and TCS there is a cutoff phenomenon: you have to decide what level of contribution merits coauthorship, and there will certainly be many cases in which a positive contribution does not result in an authorship. People who are not coauthors but made intellectual contributions to the work should be acknowledged for this. If you read acknowledgments on papers in this area, you will find that people are sometimes acknowledged for things that are clearly significant contributions: e.g. for providing the key idea of the project, or for providing correct proofs of key technical results needed by the authors.
The general conventions are to be generous and modest. Namely, the coauthors should err on the side of generously offering coauthorship to those whose contributions were positive, even if they were lesser in magnitude than the other coauthors. On the other hand, one should be modest in responding to coauthorship requests: if what you did feels like helping out a colleague rather than doing substantial, original work, then probably you do not want to be listed as a coauthor.
Of course in practice there is a lot of subjectivity here. In the case at hand, another answerer says that because your colleague's contributions "did not actually make it into your paper", then they are not sufficient for coauthorship. I don't completely agree. Telling people what not to do and setting them back on the right track can be an invaluable contribution.
Recently a junior colleague found a key mistake in a draft of a work of mine in which she was not an author. The coauthor and I withdrew the paper immediately and were able to fairly quickly recover the main result of the paper by arguing in a different way. The paper was then accepted. I think it is quite likely that if not for my colleague's contribution, the original, erroneous version of the paper would have been published. In principle we would have fixed it eventually, but in practice: yikes -- she deflected a bullet. I offered my colleague coauthorship...and she declined. In this case the offer was essentially mandatory, whereas both accepting and declining it seem reasonable.
If the OP feels that the contribution to the work was sufficient to make it plausible for the contributor to be listed as an author, I would recommend that the coauthorship offer be extended. If you know your colleague pretty well, I might follow the offer with an offer to discuss the situation: that's a good way
to make sure that you reach an outcome which you are both comfortable with. I don't think your colleague's last name has any role to play in this discussion.
Added: Here is the American Mathematical Society's 2004 "culture statement" on coauthorship. It says that more than 75% of coauthored math papers with at least one American author use alphabetical order, and that this percentage is over 90% in pure mathematics. I think that for publications in more mainstream and prestigious journals, the percentage is even higher. For instance, I checked the last 100 multiply authored papers in the Journal of the American Mathematical Society and found that alphabetical order was preserved 99 out of 100 times.
The cultural statement also says: "Joint work in mathematics almost always involves a small number of researchers contributing equally to a research project." Unfortunately I do not completely agree with this statement. For one thing, the number of coauthors on papers has been rising so sharply that even a statement from 2004 seems a little out of date: I remember that 10 years ago, seeing a math paper with more than three or four authors was worth a raised eyebrow. That is really not true anymore: a substantial minority of papers has a long enough list of authors -- with such widely varying expertise -- that it is not plausible that all are contributing equally. Moreover, unlike alphabetical order, I don't know how one can gather statistics about whether authors contribute equally. To me this statement reads like an aspiration rather than a description of reality. In my experience, there is a cultural push for authors to contribute equally, but there are other cultural pushes as well. Listing authors alphabetically essentially always in confluence with the reality that sometimes the contributions are not equal seems to me to an underacknowledged problem of our profession.
