I moved from France to the US to do a 1 year-post-doc in Computer Science, with my wife and our two children (3mo and 2yo at the time), so I guess my situation is very close to yours.
I was paid ~$50.000 / year, didn't pay taxes the first year (because of the tax treaty between France and the US), didn't have a car, and lived in a reasonably cheap community (cost of living index ~ 105, for instance, the rent was $800 / month for a very small flat, furnished and including internet, water, some other utilities. It didn't had a washing machine, for instance, so it was a bit rough with two children).
We were not rich, but it was completely doable. It was a bit risky, but we managed to live entirely on my salary, something we couldn't have done in France, so I would think that our situation was rather correct.
Here's what we learned:
- Don't expect your partner to bring an income. Day care is rare and costly (it really varies, but $500 / child / month seems to be a minimum for a "full day" care), and your partner probably won't have the right to work before a couple of months anyway. Getting a work permit while on a J2 requires to fill a I-765, pay ~$500, be patient (up to 3 months), and it comes with restrictions (you can't leave the country while your record is examined, if I remember correctly).
- Moving is super expensive. Plane ticket + buying basic housewares + rent deposit, etc. is really costly, especially if the dollar is high when you move. The exchange rate can vary greatly, so that's difficult to assess. Of course, you'll be paid at the end of your first month of employment, so you'll have to have some provisions on top of that. Also, you won't be able to bring everything in three suitcase, so there will be some buying to do, and clothes, if the weather isn't the same as where you currently live.
- People can be really supportive. We were moving as a family, and everyone, from the PI to my future colleagues (that I never met) to our landlords and even neighbours, brought something, gave us basic silverware, etc. You have to be lucky, but Americans often are willing to help, and have a couple of extra stuff that they'd be willing to share with you.
- Check your health care. Your children might be sick due to the travel, the time change, etc. In fact, you should plan that they will be sick, and check how well you're covered before being employed / during the first month of employment. Our health care took a month to kick in, we had to pay for a basic care for our son, bim, $1.000 (yes, health is expensive here).
- Papers everywhere. Be organized, a new start requires a lot of papers. Moving in an another country is pretty much starting from scratch: you'll have to open a bank account, get a Social Security Number, get a phone, subscribe to Internet, fill in tons of paperwork for the embassy, your new employer, etc. It is doable, but can be really complex (since your situation will be more complex that the average American), so you have to be super organized.
Of course, it makes more sense to start such an adventure for a 2 years (or more) post-doc.
But moving with kids (in young age) is fun: they'll grasp the new culture super easily, they'll make friends, they'll help you to make friends, they'll make people chatty, and they will break any sense of isolation you could have by moving to a foreign country.
Actual experience may vary: the USA is a vast country, and the salary range, as well as the cost of living, vary greatly. I'm not a super hero, but with the support of your family, you can make it. Tons of people did it in the past