A few thoughts in addition to the existing answers:
As MadScientist says, social insurance in Germany mostly works along the assumption that everyone is employee. However, some parts allow volountary contributions in order to get some insurance also if you are not in the obligatory social insurance.
- Health insurance: you have to have one. It is your choice whether you go for a private one (if you come from a foreign country, this includes the possibility to have a health insurance from there which covers Germany) or if you go for the Freiwillige Pflichtversicherung (literally volountary obligatory insurance :-> ) of one of the compulsory health insurers (gesetzliche Krankenkasse)
- Pension cass: accepts volountary contributions, which will earn you a) contribution years (Beitragsjahre) and b) pension in proportion to your contribution.
- unemployment insurance: you may be able to enter Freiwillige Weiterversicherung (volountary continuation of the unemployment insurance). Relevant court decision about a research scholarships. Note though, that the scholarship question is subject to income tax.
However, you need to have been obligatorily insured before.
- Accident insurance: it is obligatory for an employer to have work accident insurance for their employees. I.e., your new university will have it (and thus have to comply with the prevention rules), but it will not cover you since your are not an employee.
Again, you may be able to enter one yourself the way freelancers can (along the lines of the court decision I linked above).
If you do not have work accident insurance, your health insurance will cover medical costs. However, work accident insurance may get you preferential treatment compared to a compulsory health insurance (waiting times, rehabilitation programs, ... Private health insurance will get you even more preferential treatment in terms of waiting time at the doctor's, but I don't know their general reputation in terms of rehabilitation etc.)
I'll also second MadScientists advise to talk to the tax office. It is their legal duty to explain to you whether you have file a tax declaration, and also where in their forms to fill in the stipend iff it turns out to be taxable.
Here's a highly relevant post by a tax advisor. Roughly speaking, the "home" tax office of the funding agency is the one responsible for finding out whether the scholarship is taxable income or not. They'll issue a binding decision to you or to your tax office (binding means: if they write that your scholarship is not subject to income tax, your local tax office has to accept this decision. You may still have to file a tax declaration depending on other circumstances).
This means: they will very likely be able to tell you immediately.
The post also says that scholarships from funders in the EU or EWR can also be exempt from income tax. (Your "local" tax office in spe can tell you what tax office you need to ask in that case.)
In general, it is your legal duty to sign up for tax numbers and submit declarations in time if they are obligatory.
My experience is that German tax offices are very fair and reliable in their explanations.
They will not give advise like a tax advisor (e.g. telling you what kind of business decisions will optimize your tax payload by shifting income and costs between years) - and they'll tell you so if they think you ask such advise of them - as opposed to how to correctly declare taxes for your given situation.
You can certainly also ask a tax advisor. However, if you just ask the same questions that you'd otherwise ask the tax office, the tax advisor is a rather expensive alternative. They are most useful if you want to argue with the tax office that they are wrong about your taxes. And of course, if you think this likely to happen, you'd talk to them first before talking to the tax office.
For employees, there are also the Lohnsteuerhilfevereine who help with income tax declaration and are much less expensive than tax advisors/tax lawyers - but their services are only for employees.
(I've had a scholarship as PhD student, and my local tax office on looking through my paperwork told me it is not subject to income tax, that I can volountarily file a declaration - which would get me some withholdings refunded [wage tax withholding calculations are done on the assumption that the wage goes on throughout the year] - and if I do file, the stipend needs to go into line so-and-so as income that is not subject to income tax. All in a very friendly and professional manner. I've since had various kinds of complicated tax declarations that a scientist moving temporarily into other countries, or a freelancer with part-time employment in science encounters. The tax officers I've dealt with have all been very professional. On several occasions they did not immediately know the answer: in that case, they told me they'd have to look it up and/or ask a colleague and would I please call again next week. When I did so, they did have the answer ready.)