Yes, you can develop your earlier ideas. If others have contributed to them you need to, at least, acknowledge those contributions to avoid any charge of plagiarism. But no one can stop you from developing an idea (patent law aside).
If an idea is "out there" then anyone can work on it. It is only necessary to acknowledge others who contribute. Sufficient contribution earns others the right of authorship, of course.
I think it would be a mistake, for lots of reasons, to hide your future development from others you've worked with in the past. Collaborations are usually valuable to early career researchers. But those discussions haven't given ownership or control over the concept to anyone. If someone would object, it would be improper. If someone would ask to be part of your research it would probably be an advantage all around.
And, beyond this proposal, I suggest that you keep a notebook of research ideas that might bear fruit someday, but which you don't have time at the moment to work on. These commonly arise while doing other research. Put separate ideas on separate pages/documents and review them periodically. Some of them might be developed while you seem stuck on your current project. Some might be developed later when you are done and waiting for word from publishers.