my supervisor signed us both up to this series of conferences that are running over the next few weeks.
Whether it's real-life or online conferences - unless you both have extremely high stamina, or you absolutely must attend (e.g. since you're presenting) - attending multiple consecutive conferences back-to-back with a full(ish) schedule is not a good idea, and few people would be able to maintain proper focus after, oh, a week of that I would think.
Having already participated in a couple I kind of know what I'm in for and wondered how you guys tackle online conferences?
Not all that well frankly. Not only is it tiring, but after enough days staring at a monitor I can get a migraine.
Is it bad that I sometimes don't/can't pay full attention to certain talks?
Your having physical and psychological limits is not "bad". Also, even if we had told you: "Piece of cake, no problem" - you can't take others as your measuring-stick. It's not as though you've made some wrong moral choice or something. Every one of us has their limits.
Anyway, I would take the time to go over the schedule carefully - even if the first session of the day has already started - and to actively avoid diving head-in to session after session after session. Instead, make sure you identify some cant-miss ones. Beyond that - don't be too "adventurous" with adding sessions that are close to your must-attends. Allow for some time to take a stroll, a jog, or even a nap (Yeah, I said it. A proper actual nap.) between sessions if you can no longer concentrate, or are getting annoyed, or having a headache etc. Also remember you don't fully recharge after a night's sleep; nor fully recharge after a couple of days of rest between conferences.
Finally, find the time to absorb and reflect on sessions you've attended, during and after the conference. Possibly with colleagues or even with your supervisor if your relationship is pleasant enough.