I've seen many students and postdocs get requests from admins for their loaned computers when they leave. Now I'm not saying this is what you should do, but as a matter of fact as to what happened in these incidents. The people who ended up with the best outcome were the students and postdocs who just ignored the admin email requests for the laptop. They didn't ask anyone. They figured if the computer was really important, their supervisor or head of the department would contact them or the admin would call them on the phone or physically visit them in person. But guess what? That never happened. These students were able to just return the computer a few months later, and everyone was happy. Of course this is only for a short period of time. What happened to the students who asked permision to keep the computer? Exactly what you are experiencing. It drew attention to themselves and created work from everyone else. After seeing multiple types of responses across 10 or so incidents at 2 universities, this is just the reality. I'm not saying you should ignore the admins, I'm just saying doing so may be practical for everyone. It depends on your comfort not following the rules. Sometimes, the admin team is just following bureaucratic procedures and doesn't have any use for the laptop at all. Other times they have a reason they want the laptop back, other than just following the motions. If the former is the case, the admin is unlikely to bother your supervisor or department heads. At least not for a while. If the latter is the case, they will call you, visit you in person, or contact your supervisor. If that happens you can and should return it immediately. But also if this happens, you weren't going to be able to keep the laptop with permission anyways.
So here is a potential option, you return the laptop immediately if you receive a phone call, text, or in person visit from the admin who sent the first email. You also return the laptop immediately if anyone outside the initial email sender, asks for it back in any medium, without you bringing it up. But if you don't hear from anyone again, don't return it until you do. Note because you've already asked about the computer, your experience may be different from others. It is possible that your supervisor does not care about it. Your supervisor might have only said something because you asked about it, in which case they can't tell you to break the rules, even if they believe you should or could.
Are you breaking the rules? Yes. Are you committing a crime? Maybe, it depends on your jurisdiction and the context, best to consult a legal professional. Before it is actually a crime, they do have to make their "best endeavours" to contact you, in the USA or Aus. If they can reasonably contact you using a means besides email, they must do so, as emails could just be going to your spam folder, and wouldn't suffice as "best endeavours". I worked for a car rental service for a bit. People kept the car past the due date more frequently than you would imagine. We only contacted them if the car was booked by someone else, or if the renter was several days past the due date, and hadn't reached out. Usually, they returned the car eventually, and we charged them for the extra days, with no additional penalty and no drama at all. We only contacted the police after trying to get in contact with the renter via phone, email, and text, and the renter didn't respond for more than 30 days. The point is, you were legally loaned the laptop. This situation is not the same as you breaking into a store or bank and stealing it, as some of the comments make it out to be. It will be obvious when the situation is gradually escalating. You will not be charged or punished suddenly without warning (in the USA or Australia). You will have plenty of opportunities to return it quickly, without penalty, if the university really wants it.