There is no legal reason to hold on to old reviews, it would be the responsibility of the journal in that case. I have kept a record of my reviews just as you describe you have done, mostly because I like to save my work. I sometimes curse this because I can see reviews I would like to share with students but cannot since I perceive them as confidential unless agreed otherwise with the author(s), and the reasons I would like to share them usually does not reflect well on the paper.
Unless the reviews are made under a contract of some sort, I would argue that they fall under immaterial rights; it is your intellectual work which is provided to (1) the author and (2) the journal. I do not think anyone else can claim rights to a review.
The review work is generally made under the assumption that it is a communication between reviewer and author, albeit filtered through an editor. I have not seen any instance where someone has argued ownership of a review in my field, either in general or in the case of the journal where I am an editor. As a reviewer one must always consider the fact that all help that is provided is practically given away and should of course be seen in the greater perspecite of both giving and receiving in some form of balance.
Instances where reviews may be contracted are common when reviewing reports for government agencies or commercial enterprises. I have not seen, but cannot discount the possibility, that a journal or publisher could have such agreements. But I cannot imagine such an agreement would be hidden, it would be communicated very clearly.
As for keeping your reviews safe from being spread, I would say that the journals electronic manuscript systems are far more likely to be hacked into than that of an individual researcher. To some extent I would compare having a review on a hard drive to having it in a ring-binder. It IS possible to get hold of it but I doubt more security than that is required (i.e. ordinary computer "security" measures, e.g. provided by a university).