If a non tenure assistant professor moves to a different University inside the United States, would his two years of experience count or would it be as if he just started in terms of promotion to associate
I can only speak to common practice in the US. Other places may differ.
That is something to be negotiated as part of the process of accepting the new position. I doubt that many universities have fixed rules, other than being open to considering it.
For some people it is an advantage to get "credit", shortening the probationary period. This would be someone with a strong record; research and/or teaching as appropriate to the position.
For some people it would be a disadvantage. It might be useful to have a longer probationary period if your record is relatively weak and you need the time to build up that record of accomplishment.
I was once able to get a 2 year probationary period as I was moving into a senior position. Yes, even a Full Professor might need a probationary period and a tenure decision. Obtaining tenure-at-hire can happen, but I doubt that it is the norm.