Speaking as a math professor in Germany, a suit is definitely not necessary, and might even be seen as overdoing it. (Caveat: things might be different in financial mathematics.) Most of my colleagues never wear a suit except for a few very formal occasions (meetings with the top administration, hearings for grants or evaluations, festive affairs).
In particular, if you are visiting for a job interview, they want to get a feel of how it would be to have you as a colleague -- if you come across as too stiff, that might not be in your interest. If you want to, you can wear a suit for your talk; nobody would complain, but you might get some amused smirks ("He's really keen on this job, isn't he?"). A research seminar of a group is usually a rather informal event (compared to, say, a faculty colloquium).
Of course, the main thing is that you feel comfortable, but if you're unsure what to wear, go "smart casual":
neutral jeans (no holes, not too baggy/tight) in blue or black (or chinos, if you prefer; jeans seem to be more common in Germany, though),
a nice dress shirt (i.e., with a collar, ideally ironed),
decent shoes (i.e., no dirty worn-out sneakers); for extra-smart, wear leather shoes,
This will be appropriate for any occasion during your visit, and be seen (by those who look for such things) as making enough of an effort to avoid distracting (by being either under- or over-dressed).