If you are aiming for a high-impact general-interest journal, then the goal of the cover letter is to avoid editorial rejection.
When you are aiming at a broad journal that takes lots of submissions (e.g., PLOS ONE) or a field-specific journal that sends pretty much everything out for review, a bare-bones cover letter as described in some of the other answers is probably fine.
If you're aiming for something where there is strong editorial selection, however, it's a different matter. A good indicator you are dealing with such a journal is if they make a point of inviting pre-submission inquiries (though some don't). In this case, most papers are never sent out for review: they are rejected by an editor as "not being of sufficiently broad interest" or "not likely to have sufficient significance" or something of the sort. In this case, the cover letter is a big part of the decision, because the cover letter is the first impression you will make on the editor, and is where you can explain your work and its significance more informally.
The form that I have seen used for cover letters of this type is approximately as follows:
- Short summary of the paper and its key results
- Explanation of the significance of the paper
- Explanation of the community who will find the paper interesting (which should be one of the key communities served by the journal)