This question is specifically in the context of an American public institution and a generic faculty member, specifically not someone working in religious studies or a similar department. I have noticed over the years that many faculty members have religious items on display in their office or on their office doors, and I was wondering on the one hand whether they are legally allowed, and on the other hand how wise it is to display them while being sensitive to a diverse student population. Here are a few examples from my own experience (all in mathematics).
Several years ago I had a colleague post the ten commandments outside his office door.
I have another colleague who has Bible study books prominently displayed on his bookshelf.
I myself have a poster of the Buddha and a Buddha statue in my office, both given to me by a student.
Somewhat less on topic: a now-retired faculty member used to put ads for a Bible study that he held in his home in graduate student mailboxes. (I know many foreign graduate students found this off-putting.)