Ideally, you'll discuss the first few letters you write with a mentor, who can offer feedback and keep you from writing an ineffective letter (getting advice certainly helped me tremendously). The same situation can come up again later in your career, for example the first time you write a letter for a liberal arts college if you are used to recommending people for jobs at research universities. If you aren't confident that you can write an effective letter in certain circumstances, then you should seek advice from someone with more experience, even if you are used to writing other sorts of letters.
Serving on hiring and admissions committees is extremely valuable, both in figuring out what people reading letters want to know, and in calibrating how strong the competition is and how much enthusiasm one should express in any given case.
Trial and error does not seem useful, at least in my experience. There's just too much randomness in the decision-making progress and too many unknown factors that differ between students.
Writing good letters also gets easier with age, as you develop more experience and have a broader basis for comparison. As an incoming faculty member, you haven't seen how careers progress over time, and you've never had the experience of finding out which of the smart young people you know will live up to or exceed their early promise and which will not. Of course you'll never be able to predict this with certainty, but over time you'll start to see patterns, and you'll be able to make more nuanced and illuminating comparisons in letters.