There's nothing bad about this; it's pretty common, especially for advanced graduate courses. You don't have to be secretive about it. But here are a few things to consider.
First, time. As a new assistant professor you are going to be busier than you ever imagined. Are you sure this is the best use of three hours a week? Keep in mind that taking this class isn't something that "counts" towards tenure, so the only career benefit for you would be if it helps improve your research productivity. Of course, if it's just something generally interesting to you that will make your life happier, that's a consideration too - but think about whether you can afford it.
Most people do this by "sitting in on" a class - you'd plan to attend the lectures, but normally you wouldn't turn in homework or take exams like a real student would, since that would create extra work for the instructor.
In all cases you should ask the course instructor for permission, by email or in person. "Hi, I'm Lupocci, I'm a new assistant professor in department X. I was wondering if it would be all right if I were to sit in on your course ABC 321. I have been meaning to learn more about this topic and this seems like a great opportunity to do it." Make sure there is available space and that you won't displace actual students.
You should ask the instructor how they would like you to participate in the class. Should you ask questions, participate in class discussions or activities, etc? Sit in the front or the back? There's a possibility for your presence to be distracting to the students, so you want to minimize that.
Some institutions might require you to formally register for the course as an auditing student, so check your regulations. Tuition would usually be waived, but you still might have to pay lab fees or something. At other places this sort of thing is done "off the books" as a professional courtesy between faculty. At still others, rules require you to register, but everyone ignores the rules and just does it off the books. So you might also ask your department chair or a senior colleague what the norms are.
Finally, as you take the class, be mindful that you don't interfere with the learning of the students. It might be tempting to ask lots of questions in class, bring up connections to things you are interested in, etc, but this may tend to derail the class, especially if the questions are too advanced to be useful to the students. This might also have the effect of intimidating students who worry that their questions are not as smart as yours. So you may have to stifle yourself in class. You could discuss things with the instructor outside of class, but try not to take up too much of their time either (see second paragraph).
