Will it be really bad for my graduate school application?
You need to be clear about the policy of the program under which you undertook the student exchange. These programs differ widely, and so do their effects on your transcript. In my university alone, one exchange program allows students to undertake courses with NO EFFECT on their transcripts while another program requires students to undertake courses for credit equivalent purposes consistent with the original degree programs in their home university. You need to be clear, because the same two universities may be operating a number of exchange programs simultaneously at the level of the faculty, school of department.
Is there any way I can avoid the bad influence or can I not submit the transcript of the exchange university?
If you've determined that there is a credit-bearing grade policy in effect, then there's little you can do about this, because the credit exchange occurs automatically between the Registrars of both universities. If you are in a position to lodge a mitigating circumstances claim, then you might do so under the policy of the exchange university. However, once the grades have been approved by the moderating body, then only procedural irregularities may be the only reason for a review of the grade. In this case, you will need to be clear about the assessment policies of the exchange university, which are probably an order of magnitude more complex than the policies for exchange.
To be clear, under particular policies, you can't simply choose to not submit the transcript of the exchange university for two reasons: (1) the results may be incorporated within your transcript or (2) your transcript will contain fewer credits than is usual. Either way, a graduate program admissions officer will spot the discrepancy easily.
If you're really worried, you might choose to mitigate the effect by turning to your statement of purpose. You could use that to explain under what circumstances the 3.3 GPA was received. Be careful, though, because placing it on the SOP will call attention to it and, as @ff524 states, 3.3 really isn't all that bad.
Good luck to you.
