For a PhD in computer science or informatics, are there good reasons not to choose an area of research in which very few (3 or 4) known (small- to middle-sized) research groups work in the world?
are there good reasons not to choose an area of research in which very few people work worldwide?
There might be.
- a research clique might be at work: sometimes, often unintentionally, you can observe a bunch of researchers starting to push some topic and ending up absorbed in their own little world proposing extensions of extensions of extensions and not noticing that they became irrelevant. Usually because something is wrong with the problem they work on. But sometimes such a niche might be actually very healthy, except the topic did not caught up more popularity in the relevant mainstream community.
- something might be wrong with the problem they tackle: sometimes a group of absorbed (as above) researchers looses connection with reality and pushes towards solving a problem which does not exist anymore because somebody else solved it just by the way along a route to some other problem and everybody else recognized.
- something might be wrong with the method the small group applies to the problem they tackle: sometimes the problem is good (see below), but the bunch tries to hopelessly push a method which simply does not yield anything. Often this is accompanied with use of a highly specialized terminology so that it is difficult to see that the method they use and the problem they tackle is in fact very much related to something else what yields more healthy research in the relevant mainstream community.
More generally, don't worry about the size of the field. What matters is whether the problem at the core of the niche is real and sound. Real means that solving it can have an impact on the society at large. You need to believe in your cause. For the sake of this blob, let sound mean that you see a solid way to articulate it and get a grip on it, most preferably including a method to measure progress. If you have the two, you are set even if the community tackling the problem is relatively small. All the rest is up to your capabilities and the difficulty level of the problem.
Trust your judgement.
One key thing is that there might be no open position for your to continue your line of research, if you wish to.
But here it is an important question, whether there are growing, or are in decline. And, more importantly, if you like and believe in this field.
Otherwise importance of the problem they are solving lies on a different axis than number of groups. Not every important topic is popular (and vice versa).