After finishing my studies in Electrical and Computer engineering, I worked as a researcher for a few months in the field of my master thesis (developing scientific software for applied physics). I was not quite satisfied with the field, as it was becoming more focused in applied physics rather than computer science, a fact which was highlighted by my former supervisor. This position could lead to a PhD position in applied physics with good salary for 2.5 years, but I didn't want to proceed at that point, because I was interested to continue in computer science and not physics.
So, I tried to transition to industry. In my first job I was working in R&D as junior engineer and I quite liked it. Later on, I moved to pure software engineering and I hated it. Even though I was receiving a good salary for my experience. I couldn't stand the lack of innovation, I couldn't bring my ideas and I didn't even need to solve an algorithmic issue, which is something that I liked from my first year of studies. I wasn't even good enough in my job (this is my personal opinion), I was always bored and after a while the dissatisfaction from my job made me unhappy for my life as a whole. I couldn't imagine myself doing this for the next 40 years. I quit my job and this is my last week, after that I have nothing scheduled regarding my career.
I am really very confused about my next steps and I have been thinking about finally doing a PhD. The reasons behind this is that I want to feel that I can really make contributions, bring my ideas, solve problems and not just write code lines. I would prefer intellectual and time flexibility, as I believe that I am functioning better this way. I had never thought of following an academic career before, now it may not be my ultimate goal, but I could consider it. I know that becoming a professor is very hard, and I am not targeting such a position, but I really want to do research and not just software engineering. I would be mostly interested in doing an R&D job later, either at a company or at a research center. The field I would like to focus on is artificial intelligence and machine learning, and I am planning to go back to the university where I received my degree (in another department and with another professor than the one I did my master thesis with).
I have been reading here that a PhD can be a waste of time for people that do not seek a purely academic position. I have already applied a couple of times for research positions in companies and research centers without success. Thus, I thought that a PhD could make the difference here. What is your opinion regarding my case? Is it a good idea to go for a PhD or can it be a burnout after some bad career decisions until now?
PS: I don't know what's going on with fundings in my university in the AI field