As a frequent long-term (more than a year) visitor (with official visitor status) to a different nearby institution, would you list it as an additional affiliation? Much of the experimental work (the field is physics) on a project has been carried out at the other institution by myself.
I don't think that this question has a general answer as there might be guidelines applying to specific institutions.
I know of some institutions that explicitly require long-term visitors to state their affiliation. You should probably inquire about this with your institution. The institutions that I know that have rules about this usually require you to state your affiliation, and I have never seen a rule where you would be disallowed from doing that. So in any case, if a lot of the work has been carried out at the other institution and thus been taking advantage of their resources, an acknowledgement of that should probably be given in some way, whether it is in the form of a full affiliation, an affiliation with a footnote, or mentioning them in the acknowledgement section of the publication.
I suggest that you do so, but specify that your position is visitor. This will avoid confusion, especially as you seem to have a normal position at your home institution. "University of the Sublime, 2018-20, visitor"
Or "visiting researcher". But the other university might have a say in what you call yourself there. You might ask a department head or dean.
For a paper/talk it might be important to do so if the research was carried out there.