In my experience (which, beware, is in pure mathematics) a faculty member has very little incentive to describe undergraduate research done under their supervision in negative terms: it is hard for the reader of such a letter not to view the student's failure as their failure. Also in my experience (which, beware...) it is quite common for an undergraduate working over a short period (like the summer) not to end up contributing anything much to the larger research project, so that even this outcome need not be described in negative terms: after all, the student still got the experience. I am aware that in the laboratory sciences a student is working on a piece of "the real project" much more so than for student research in pure mathematics, but still....would an experienced PI really give a critical piece of their project to an all but unknown person? Again, if they do so and it works out badly, it seems to reflect badly on them.
All this is to say that I find it unlikely that you will get a negative recommendation. What you risk getting is a recommendation that is not positive enough to help you with your graduate school applications. If that happens, and if as you say you did good work over the summer (both your grad student supervisor and your PI are much more experienced in this regard than you, so without further information we have to at least contemplate the possibility that their judgment, rather than yours, is correct), then yes -- you probably did end up in a bad group.
What can you do? Well, you booked an appointment with your PI, and I think that's wise. During that appointment your goals are (i) to the extent that seems reasonable and reasonably polite, make your case that you did good work over the summer and (ii) suss out your PI's opinion of the work to see what kind of letter you will get. Here I will observe that if you are overly aggressive or defensive on point (i), that might hurt you on point (ii): the PI may not want to give their honest opinion if they have already had to argue with you too much. If it turns out that the PI has a not very positive opinion of your work, then indeed you may not want to get a letter from them. That's a bit of a shame, but the lack of it should not really hurt your application, and an equivalent way of saying that the PI's opinion of your work is unreasonably low is that all your other academic work will be evaluated much more positively by others...so go with these other, better recommendations.
Anyway, all this is just one summer: if you stick around, there will be others...Good luck.