I am applying for grad school and a bit confused about taking references. Should I take a reference from my employer where I had worked on an unrelated field for one and half year? Or Should I prefer taking it from academia from a Prof. with whom I have worked on unrelated field again.
You need a recommendation that will convince a committee that you will be able to rise to the occasion (i.e. be successful in your coursework, exams, research, thesis and thesis defense) if they admit you to graduate study in their department. Choose letters that will have as close an effect to that as possible.
You haven't given us enough information about your employment to enable us to provide an opinion about that potential letter.
Note that if someone asks for one reference, and you have two references which, combined, give the desired picture of you, then the thing to do is submit both letters.
Also note that there are threads here that provide advice to those who don't currently have any suitable references (e.g. take another class, do an internship -- something to garner a new reference).
After you've read some of the related Q&A threads, feel free to edit your question, giving additional information about your situation, and pointing out how yours is different from the others you read about.
In the U.S., you'd almost surely need at least 3 letters, so you'd want both of those. Much as @aparente001 says, the point of the letters (in the U.S.) is to convince the admissions committee (in the department to which you're applying) that your future potential is sufficient to get you through that grad program, and to be a contributing member of your professional group thereafter (or else possibly be a blot on their reputation). Pointedly, it's about the future. Relevant past accomplishments are a very good predictor. Dubiously relevant past accomplishments less so. What's the dividing line? Not so clear, perhaps, but it is amenable to rational reflection.