Peter Jansson's answer tackles a good part of the process, but let me add the question of the publisher.
You can go through an existing publisher, in which case you will have to choose one (and then convince it that it is a good idea to start your journal, but if you managed steps 1-5 in Peter Jansson's answer, it should not be difficult). There are several criteria to be considered:
What service does the publisher provide to the editorial board? Discuss with editorial boards of journals published there to find out. Also, a publisher which is well-known in your field can have an easier time making your colleagues become aware of the existence of your journal, but it seems that this task is largely up to the editorial board anyway.
What service does the publisher provide to authors? For example some publishers ask authors to format their papers in their style prior to submission, which can be a pain, or may have painful electronic system that authors must use to submit or to contact the editorial board. Your past experience can give you a good idea. Also, the question of usual cost the journal charges for extra pages or for open-access is crucial. Publishers can also have very different policy with respect to green open access, see the Sherpa/Romeo site for info about this.
What service does the publisher provide to readers? For example some commercial publishers provide very poor copy-editing; you can care or not. Also, the question of the usual cost of subscriptions for non-open access journal is crucial.
You could also want to go with your university's press if it exists, so as to be close to the publisher and interact more easily with it.
If you plan on act as publisher as well, things are more complicated. Let's assume that you do not want to manage subscription payments nor article processing charges; you will still deal with quite some stuff.
Your journal should have an owner; it may be you, or your university, or another institution, or you could create a legal body for this (e.g. a foundation), notably if you want some money to flow. For example, I know a journal run by department, with money and one full-time staff from a university and a national research institution.
Your journal should have an identification, most notably an electronic ISSN. This is not difficult to get as far as I know.
Your journal should be indexed in the databases used in your field (e.g. Zentralblatt and MSN in mathematics). Asking the publisher of the database would be the obvious way to go.
You will need a software to track submissions (which need to be assigned to an editor, which need to be assigned to a referee, which await for a decision, etc.) and to host the website of the journal. Most probably you know how to do this part since you did it for your conferences.
If you want a paper version, you will have to deal with a printer and with mailing issues, but this is very XXth-century and very likely for a journal that does not charge subscriptions.
You may need to use an anti-plagiarism software, depending on your field (sometimes in small fields plagiarism is easily detected, but this is not universally true).
I certainly miss some points, please free to add in comment and I'll try to keep the list up-to-date.