I accidentally included a source in my bib, that I didn't end up using in my paper. So basically it included a pointless source that has no info or quote included in the paper. Will this look bad or will I get docked for it?
Some students actually try to last-minute pad the bibliography that way. If it doesn't look like that (i.e. noticeable many unused references to sources, which you were expected to read&know), you probably will be ok.
In general, it is better practice to include a reference to a source pertaining to the topic of your paper, even if you are not directly quoting from your reference, than to arouse suspicion of concealing to having "used" it. Depending on the context and regulations, you might even be prompted to add references to non-written material (like a conversation) to the bibliography.
It is even rumored some reviewers expect their recent work to be referenced, no matter if it's of any real use to your work. On a funnier note, some writers try to sneak in a fun reference (use with consideration!, I'm just reporting what I have seen...).
The goal of the bibliography is to inform readers where they can find more info. Practically, almost all references will be cited in the document itself, simply because you used that reference when writing a particular section. That said, it is not uncommon to include references that are simply "related reading". The only real issue here is that no one reads the bibliography like that; they go there go find a reference, not to browse and see what you think is useful.
To that extent, if you want to list sources that are relevant but not directly cited, I would simply add a "related reading" section at the end of the document and list your extra citations there. This is a very common pattern in book chapters. Personally, I've never seen a "related reading" section in an academic journal, but I don't see any reason why not to include it.