I sent many emails to prospective PhD advisors looking for a position in their research groups. Recently, I got two offers from professors in different universities and I choose one of them as his research interests fit more with my experience. Now, I have to apologize to the second one that I won't to be able to work with him. How to do that without leaving a bad impression ?
There is no need for an apology - there may only be a need for an explanation (and maybe even not that, as dimpol said, this happens all the time). The other professor will not be offended in any way if you take another offer, but he may be interested in an explanation. So you could write something like "Thank you very much for you offer. However, I have chosen to accept another offer for a position which does fit my experience better." and you should be fine.
Also note that "I won't be able to work with him." is not necessarily true - in principle you could still start working with him remotely if both are interested and if time permits (e.g. it does not conflict with your work at the new workgroup).