My current boss suggested that our institution could issue an official letter explaining the lack of teaching duties or opportunities.
This seems like a bad idea to me; these letters aren't a place to make an excuse that you don't have an opportunity to teach, they're a place to highlight your qualification for teaching.
the only course I taught as the main instructor was during the lockdown, where no faculty oversaw me. The instructor of record on paper for the latter offered to write me a letter that basically just says that I indeed taught in my graduate school, but nothing more
Someone is responsible for the course you taught: that's oversight, even if they weren't literally in the room observing you. In some cases that might be the leadership of a department that assigns courses to particular instructors and would manage any complaints if they arose; however, for you it sounds like the most relevant person is this "instructor of record on paper" who is clearly the person overseeing your teaching (again, whether or not they actually stood in a room or whether there was a room at all).
However, they should also know that within the US if you write "I can confirm that Pteromys taught a course" and nothing else, this could easily be taken as a negative letter (a colleague of mine likes to use the joking phrasing: "of all the students I've met, this applicant was one of them") unless the person reading the letter recognizes this as a style from a different cultural context.
Recommendation letters do not necessarily need to be based entirely on first-hand experience. If I was the instructor of record for a course you taught in, I would ask you to give me your resume that includes all your teaching experiences, and I would write a prose letter about these experiences, mentioning the specific subjects of the courses and scope of your duties and punctuating it with how I personally knew you and vouched for your work. Critical to this letter would be your independence, which you phrase here as "no faculty oversaw me" when an appropriate positive spin would be that no faculty needed to oversee you, or rather that you taught the course independently and did not need require day-to-day supervision. It would be further worth mentioning the circumstances under which you taught and any special preparations you needed for that teaching: for example, if you had materials designed for in-person instruction and you successfully adapted these for online instruction, congratulations: you've successfully passed a pandemic test that many more experienced instructors struggled with mightily, and I'd sure hope a recommendation letter would highlight that.