It is referring to practices such as citation rings (e.g. this account), excessive self citations within a journal or other malpractices to boost the impact factor of a journal or a group of journals.
These three articles will give you some more background and examples:
I don't know if the exact algorith used my Thomson-Reuters to detect “Anomalous citation patterns” has been officially published. However, they look at things like change in impact factor between years (which can lead to false-positives) and the citation networks between journals. As you note, the practice is very similar to link farming.
This quote, from another Nature article on a Brazilian citation cartel, offers some information on how Thomson-Reuters work to detect malpractices:
Marie McVeigh, who leads Journal Citation Reports, Thomson Reuters’ annual report of journal citation patterns and impact factors, says that the firm does not assume intent in the patterns it sees. “We just analyse data, and see where the impact factor, due to this unusual concentration of citations, is not a reflection of the journal’s citation in the broader literature. Until we can algorithmically measure motive, we are just going to measure citations.” She says that because citation stacking is only a problem if it excessively distorts journal rank, only four Brazilian journals, and not all of those that Rocha-e-Silva says participated in the arrangement, have had their impact factors suspended.