I am writing a paper jointly with coauthors from another research group. In their contributions to the paper they include a lot of self-citations, many of which are (in my opinion) irrelevant.
How do I convince them that this is bad style and that they should reduce the number of self-citations, without creating an antagonistic situation between our groups?
While you can give your best arguments to the others they probably won't be convincing. On the other hand, I wouldn't worry about it too much since the most powerful advocates for your position will be those who review the paper after submission. Reviewers don't especially like to go through "crufty" papers with a lot of irrelevant stuff. If your co-authors are, indeed, over-the-top on self-citation, it will likely be noted and need to be cleaned up after reviews come in.
Rather than cause too much difficulty, let the reviewers be your allies if it is really needed.
I would, however, make the argument to your collaborators that a lot of the citations are irrelevant and detract from the main ideas of the paper. You could even send them back a draft in which you have removed the citations you think are unneeded, just so you have a "working document" without those citations.
(Due to reputation<50, I cannot comment) What is the constellation in your group and the other group (PhD student, post-doc, professor, ...) and how well do you know each other?
First of all, you have to have a consistent opinion within your group about whether the references are indeed irrelevant self-citations only to pump up some citation counts or not. By discussing this issue with your group leader (assuming you are the PhD student/post-doc in your group), you will get a feeling whether confrontation must be avoided at all cost or your group leader will back you up on this issue.
Then, consider politely asking the co-authors to reconsider the references list. Point out to the guidelines of the journal and/or of our university that state address citations. While this answer is not creative and you surely have thought of this one yourself, it is the honest approach which will define the framework for the future publications this collaboration will hopefully yield.
