As a new university professor, I have been informally tasked with making sure the MSc students in my program are prepared for their dissertations, by prepping them in my research methods course.
There is a lot going on in this first sentence!
First of all I see that you are new, which makes things especially difficult for you: you don't really know how the department or the MSc program works yet, but you are trying to guide students through it. Trying to do everything exactly right the first time around seems unrealistic: I think you should concentrate on making a good effort and showing the students and your colleagues that you take the job seriously, bring a lot of skills to the table, but are also open to advice and adjustment in terms of this particular service situation. This may end up resulting in your spending more time and effort the first time around than you will in the long run, un/fortunately.
(Let me also say that, based only on this sentence, I am not thrilled with your department for placing you in this situation. If the newest member of the pantheon is immediately signed up for something so critical and complex, one has to wonder how functional the pantheon was before the new arrival.)
Second of all you mentioned an informal task but then also a formal task: you are teaching a research methods course. The latter sounds good to me, and you should certainly be having office hours in this course, and setting more as needed if the students seem to be struggling as a group. But the task of being single-handedly responsible for the preparedness of all the MSc students in your department for their dissertations sounds like a lot to hang on your head, if by that you mean much more than successfully teaching the research methods course. Expecting every student to be prepared to write a dissertation after completing one course doesn't sound very realistic to me. Also I hope that each student has, or will have, an advisor for their dissertation other than you (at least in most cases), and that these advisors will take on more of the responsibility of their preparedness/progress than you. You simply can't carry an entire master's program on your back in your first semester. So I would try to create a distinction between the amorphous informal task and the reasonable formal task, and set yourself up for success in the latter.
In terms of helping the students more versus coming to a point where you can't help them: unfortunately the setting discussed above makes this an especially open-ended and intense version of that. I think the other answers have given you good advice nevertheless.
You do need to limit your total amount of involvement: set aside a certain number of office hours per week, including extra office hours for drop-in appointments. You can help the students a great deal, but not infinitely, and not "as much as they need".
If the students seem far from the mark, have an assessment meeting with each student (yes, this is very time-consuming...). At that meeting you can learn from the student their background, their progress and their current position, and then you can make an individualized plan to help the student move forward. It is not realistic for the goal for every student to be the same, but it is a reasonable (though still ambitious) goal to help every student make clear forward progress during the course of the semester.
Make sure that your plans for the student give them plenty of things to do, and things that they can do: i.e., you want them to be doing most of the work, and you can help them with it at key moments. If you ask them to do something that they simply don't know how to do, then you're either going to fail them or spend way too much time doing it for them. But in a research methods class every student should be able to start somewhere. Make sure that the ground that they start on is relatively solid: I think that's much more important than how far they get by the end of the course.