The question boils down how to effectively choose the associate editor and assign my paper to him?
The associate editor (AE) has to first decide if the paper is worth reviewing for the journal (they can often do a summary reject if the quality of the work is low). If it is, they then have to select the reviewers for your paper, get the decision from the reviewers, and then use that to make a recommendation. So, ideally you'd prefer someone who is:
- familiar with your area and work,
- familiar with people who would be good reviewers for your paper,
- someone that you or your co-authors get along with.
Getting #1 is hardest - most journals don't have AEs in every sub-area of a field. As an AE I often handle paper that are in related fields so I know of possible reviewers, even if I'm not an expert in the field myself.
For #3 you can't have someone too close to you (e.g. someone that a co-author regularly publishes with), and you also don't want someone with whom one of your co-authors has very poor relations (e.g. fundamental disagreements about the field).
Note that while you may recommend an AE to the editor when submitting a paper, the editor has the final say on the AE that is selected for the paper.
With regard to your other questions:
Is it possible that wrong choice of associate editor can lead to rejection of paper?
It is always possible (eg if the AE doesn't like the line of research or chooses poor reviewers), but there are many factors that can lead to the rejection as well, and it is almost impossible to control for all of them.
Or is it possible that even if I choose an associate editor who is
not an expert in my field he may send my paper to the appropriate
The editor may do this if the AE is busy or if the editor thinks that someone else can handle the paper well. It would be more rare for an AE to ask for a paper to be sent to someone else.