Getting started on a postdoc in a different environment, with a different supervisor after a successful PhD can feel overwhelming and intimidating and it is certainly not uncommon to doubt yourself. I think many of us have been there. That being said, this shouldn't be so all consuming that you walk around with a knot in your stomach. That is a sign that other things are going on.
Don't give up and call it quits before you take a breath and try to break things down to see where your struggles are and try some things to improve the situation.
Clearly there's your situation at home which seems to be adding to the stresses of this new job in a new town. Try to craft some time in your life where you really are off work for real without feeling guilty for not working. Even if it's two evenings and a weekend morning or afternoon where you can be physically and mentally present with your partner 100%.
Then there's the job itself. You probably need to adjust to how your supervisor is communicating. In an ideal world they would also communicate with you in a way that works best for you, but even people who are really good at doing this may need some time to figure that out - and perhaps this is just what your supervisor is like. Can you talk to some other lab members to ask what the best approach is to communicate with him? Do you feel comfortable to tell your supervisor directly that you need to adjust to his direct way of communication? Can you ask what he thinks you need to focus on first to be successful in this project?
One way or another, you need to find a way to handle this. As long as you can hold on to the view that he has good intentions and both of your best interests in mind, that's a good point of departure and a place you can always return to. If you don't understand him, simply rephrase it at the end of a conversation: so if I can summarize then I need to do X? This is the best way to prevent misunderstandings.
Then there's you and this project. Good for you for changing topics. This can make you feel like a beginner all over again - not the best feeling in the world. Remind yourself of why you made this choice (hopefully there was a really good reason for picking this postdoc as this may be the incentive you need to keep going).
Ask yourself what you could do more or differently. I don't just mean 'work harder' but also 'work smarter' (i.e. are you putting your energy where it matters). It sounds like you are happy behind the bench, but make sure that you indeed take the time to focus on the larger theory and concepts - otherwise you may not be doing the right experiments.
Are you reading the right papers? Are you taking initiative to find out where the difficulties are that your PI things you are underestimating (maybe not the best motivational speech but useful information)? Are you asking enough questions and talking to other people who may have bits of useful information? Use the weekly meetings with your PI to ask questions, bounce ideas off of him and try to see where he sees this project going.
Hopefully in time, your confidence and mastery of the subject will grow and you can take this project your own direction. You have a PhD, so you should have the skills to master this topic as well. If not, well, then maybe this particular postdoc or an academic career is really not for you. But let's not jump to that conclusion just yet.