As both an answer to this question and as a PSA, most major universities in the US are signatories to the "April 15th resolution," which gives students until April 15th to accept offers of admission/financial aid. A full list is here: http://cgsnet.org/ckfinder/userfiles/files/CGSResolution_RevisedOct2017.pdf
As part of that policy, most relevant sections bolded:
In those instances in which a student accepts an offer before April 15, and subsequently desires to withdraw that acceptance, the student may submit in writing a resignation of the appointment at any time through April 15. However, an acceptance given or left in force after April 15 commits the student not to accept another offer without first obtaining a written release from the institution to which a commitment has been made.
Assuming the universities you are interested in are on that list, then, you can still change your mind according to policy. What remains to consider is the personal relationship with the institution/professor whom you have already accepted an offer to.
The institution itself is unlikely to care all that much since you are likely one of many graduate students - but do inform them quickly once a decision is made. Ideally it is most polite to give them as much time before April 15th to offer your funds to another prospective student. This might be different if it's a smaller program. Some professors at the institution might be a bit offended, but many will also understand the attraction of a fellowship over an assistantship, especially if the fellowship comes with any prestige. If you have accepted a position with an individual professor, that person is the person most likely to be offended, but this will depend entirely on their personality and how much stake they had in bringing you to their lab.
The ideal course of action would have been to wait to accept anything until you had all possible offers in-hand. It's unfortunate that no one gave you this advice while you were applying, but somewhat understandable that you would make this mistake simply because people don't typically apply to graduate programs more than once or twice in their lifetime.
Personally, I would probably not inform the school where you accepted simply that you are interviewing at the other school to avoid causing bad blood if you end up staying with the initial offer after all, but if you change your mind you must inform them immediately.
For your particular situation, consider also the duration of the package offered and the field you are in - a fellowship is nice, but in some fields a research assistantship effectively pays you to do your own research, and is only marginally better than a fellowship (besides any prestige of the fellowship). A one-year fellowship may be hardly worth changing your university choice. In other fields, a fellowship is a huge deal because RAs are not typically part of your personal research and degree progress.