I would definitely not cite it without a strong reason to include it.
The scientific community used to rely on "personal communication" as a method to cite truly useful or pivotal information that came from a source that had a meaningful influence on the development of the work, but was not publicly available. However, it was generally discouraged for use only in extreme cases because it undermines the nature of peer review.
The two primary purposes of peer review are to check validity and to provide feedback to improve the paper for the benefit of the intended audience. If the reviewers (which are part of the audience) find the paper not valuable enough to accept and the authors don't find the work valuable enough to improve, then why cite it? Just because somebody wrote it, doesn't mean it is valuable and if it is duplicitous, then, by definition, it lacks value from a scientific perspective. It is the author's responsibility to demonstrate the worth of the contents, not the other way around. It is also the responsibility of authors of subsequent works to respect their audience and provide the audience value. Citing a paper that the community found lacking, solely for the purposes of "completeness", is not a service to your audience. I would argue it is a waste of the reader's time and dilutes the value of other works. Would you list a paper you knew was deceptive or falsified, just to provide completeness? (I hope not!)
If an unpublished work truly influenced your work, then, by all means cite it and explain why you are doing so. However, citing a paper that the authors, themselves, do not feel is worth their time to improve has a negative reflection on your own efforts, which I would avoid. Rejected papers on Arxiv are little different than an unreviewed work on a personal website or an advertising site. Some communities use them for feedback, which is great! But failure to respect the feedback and improve and eventually publish the paper should not be encouraged.
Scientists need to be cautious or the entire enterprise is jeopardized. Once, in my own naivete, I "invented" the Hough Transform. But just because I thought of it independently doesn't mean I deserve equal credit. In fact, it demonstrated how little I knew!