1) Comparing with results that other authors report is in all likelihood acceptable.
2) However, published results shouldn't by uncritically believed, so it is better (in the sense of being a better service to science) to replicate other authors' results. Also, who knows, in case reported good results do not replicate you have another argument against a competitor and in favour of your method.
If information to do that is not sufficient in the paper you're citing (which reviewers of that paper should have criticised in my opinion), the most reliable way of doing that is to ask the authors for their code (if it isn't available anywhere anyway, that is).
3) If the authors don't share their code it is a good thing in my view to try to replicate their results using your own code (although in case you can use published results it is not mandatory for your publication, see point 1). If you find differences, it's best to contact the original authors about this, but you are also well within your rights to say in your paper that these results deviate from the original ones despite making your best attempt to replicate them. In any case you should acknowledge that this is your implementation, and list any decisions made by you for the implementation that are not obvious from the original paper. Once more, this is more work than just using their reported results and in all likelihood not required for publication, however the reward is a certain chance to find something that you can use against a competitor (I of course mean this in a purely peaceful way, but implicitly assuming that you want to write a paper and readers and reviewers may want to know what makes your method worthwhile compared with XXX).
I add (belatedly) that apart from the value for the specific publication you may have in mind, I learned a few valuable things trying to replicate other people's work.
4) The other respondents are right about only comparing what's comparable, see their answers.