If you haven't signed anything, or at least been given a written offer, then you may not have an enforceable contract. Changing your mind within a few days will cause only minimal, if any, disruption at the other end. 

If they believe that you do have a binding contract, then they will let you know immediately and you will have to deal with that. You could ask to be released from it, of course. But speed is essential. 

Just send them something in writing along with an email declining the offer and offering apologies for your earlier statements. You don't really need to say more than that. 

They might be unhappy, but would be less happy to have you join them grudgingly. 

I suppose there are some places where a verbal agreement can hold legal weight, but I doubt that even then, they would think it wise to enforce it. They likely have other options.