Many of the responses here give correct answers. They also explain that the view of this quote is not the norm, and that one shouldn't necessarily trust Grothendieck's advice based anyway. I agree with all of this, BUT with caveats so significant I believe it deserves a separate response.
Caveat 1: He is not implying that one should reject other people in the field and not cooperate with them.
As a matter of fact, in the same text he talks about how almost all his first ideas in algebraic geometry were stimulated by Serre. He was certainly no stranger to collaboration, and this is completely compatible with this quote. Rather than explaining abstractly this compatibility abstractly, it may be more explain to learn how he allegedly worked. In his IHES days, he would present his seminar once a week and follow it up with discussions. Sometimes he would also often invite others to his home to do math. But on other days, he would work alone, often late into the night, to come up with the notions he would discuss with other mathematicians.
The most well-known example of this is the notion of a scheme. This idea, or at least close variants of it, were up in the air at the time. But how did Grothendieck know that the very foundations of algebraic geometry should be rewritten with them? (a truly gargantuan task) According to this same text, he had to work alone, plumbing this concept himself to see this was the way to do it all. Another concept, the "Grothendieck topology" generalized the very notion of a topology in a seemingly naive way. But this was what was necessary to define etale cohomology, which led to the proof of the Weil conjectures.
For a more detailed discussion, see here. It seems to me that whenever he did math, he had a grand vision. This vision was certainly stimulated and enhanced by others, but it was something he could "see" on his own that was remarkably unconstrained.
Caveat 2: This quote is less a judgement of others than an expression of something personally important.
While this quote does indeed mention the impact of other mathematicians, I don't think that is the point of the paragraph. If you read on, you will find that this section is about the "interior adventure", especially about his experience of mathematics. He explains that for him, innocence and his ability to listen to the nature of things were the most essential traits to his success in math. While this formulation may sound romanticized, the content seems sound to me.
The notion that other mathematicians would have gained from what he describes is, in my opinion, a valid one. I believe most of the pushback against it comes from the fact that he is saying about himself, which may lead some people to associate it with arrogance. As a matter of fact, I have heard people talk about how various fields could have used a Grothendieck without any controversy.
Caveat 3: Just as one should not accept advice just because it is from Grothendieck, one should not reject it just because it is from Grothendieck.
There are some rather disparaging comments in the other answers about Grothendieck as a person, and how we should not listen to his advice as a result. I find this attitude very troubling for two reasons.
First, it is largely based on negative stereotypes that are absolutely false in this case. I've even seen many of the concrete facts about him be twisted or even made up to promote this sensationalist archetype of a recluse mathematician who understands nothing but math. Having read some of his later writings, I can confirm that such a depiction of Grothendieck is completely wrong. For sure, some of the events in his life did superficially resemble the stereotype, but his philosophy and understanding of life are absolutely different.
Second, one should in principle not discount somebody else's opinion based on the person, but judge it on its own merit. Just because someone was "troubled" doesn't mean that their view is worthless. (Especially when the cause of much of the trouble was a perception of the coming environmental crisis, leading him to be one of the first ecological activists.) In fact, as Grothendieck possessed both a unique and a supremely educated perspective, one should not be surprised if he has interesting things to say. At the very least, they will not be the same clichés that add no value.
Even in his writing outside of math, time and time again I have been amazed by how his perspective on a topic already accounted for my own. Experiencing this is certainly startling when you expect to be the one assessing the supposedly crazy views of another. His work is one of the very few that I have learned from upon reading multiple times, which is why I am very disappointed when people dismiss him based off of a stereotype.