I feel like none of the existing answers are doing a good job answering this question.
If I simply don't use my office will I look bad to other grad students or professors? Will I maybe miss out on productive discussions? I can see if my office mates are in the same classes; we can work on the problem sets together.
Assuming that:
- Your advisor is not co-located with you
- You are working 100% by yourself, without engaging other graduate students
You can work wherever you want with no likely no problems.
However, consider that a majority of graduate work involves other people - whether a research group, collaborating on papers, or simply sharing knowledge and brainstorming ideas (do NOT underestimate how useful it is to discuss research proposals or topics with peers/faculty). In these cases, you run into two problems.
First, how it looks to others, which is the first of your questions. It is easy to say, "people are only evaluated based on their results - you can get results regardless of whether you are in the office or not!" Unfortunately people aren't this rational and objective, and particularly in research (where it is harder to have "clear results") you will suffer from perception problems.
This is unrelated to whether it's true or not. The unfortunate situation is that an overwhelming majority of people correlate butt-in-seat time with productivity. Fair? True? Absolutely not. But just because something isn't fair doesn't mean you won't be perceived this way.
In particular, other graduate students will see you not at your desk. They won't have insight into your time at a coffee shop, working from home, or whatever. This means their impressions of you will be fairly negative. Unless you are producing crazy high quality work (which as a new grad student is unlikely), you're going to be starting from a negative place in gaining peer-status with your fellow grad students.
It will be harder to take spontaneous breaks with peers. This might be coffee, meals, games, or other silly things. While initially unrelated to research, those activities significantly contribute to how we form relationships. Working remote from peers makes it much harder to form these relationships.
This leads you to be detached from your peers (and potentially even your advisor). People quip, "out of sight, out of mind" but there's a reason it's a quip. Many advisors are really busy and their ability to drop by and talk may require you to have more flexibility. Basically, the busier your advisor is, the more important your availability becomes - if you are consistently working in a desk near your advisor's office, this helps.
What are the potential pitfalls of not being in my office other than for office hours?
All of the above plays into the second problem, which is your overall, long term efficiency and "status" as a graduate student. If your goal is future research, whether as a professor, post-doc, industry, or a PhD (in the case of you being a masters student), developing and fostering relationships with your colleagues is quite important. These colleagues are those you will want to gain the following from:
- Collaboration/assistance with research
- Networking for collaboration, guidance, and/or project collaboration
- Finding future jobs
People often forget how important their peer relationships are. But consider that if you are pursuing a PhD, it is likely in 5-10 years some of your current peers will have positions where they can dramatically influence your career. This might be because they have a large grant they need more researchers on, because they are on a hiring board at a university, or know of a position that comes up. Ultimately, these relationships are invaluable to an academic. Realistically this applies to everyone, but if you are pursuing a PhD it is likely the number of stakeholders worldwide in your field is small enough that you could over the course of a career meet most of them (between conferences, collaboration, etc).
If your goal long term is research, you want to know, foster, and develop relationships. Alienating yourself from the first group of people who make this the easiest (peers and other faculty at your institution) is not a good move longterm.
Now... if none of the above applies to you for some reason, then it won't really matter. But I suspect a ridiculously small percentage of graduate students can realistically claim none of the above applies to them.
Graduate school efficiency
Ignoring the above problems entirely, another consideration is the fact that when you enter graduate school you often have no clue how graduate school works. The best way to learn is interactions and informal conversations with your peers, particularly the senior students.
Spontaneous conversation is nearly impossible if you are working remote the entire time. The lack of relationship with peers means suddenly you will be in your last few years with no working peer relationships.
If you ultimately do decide to avoid your desk, I would recommend you consider ways to avoid the above problems entirely. This will depend on your specific circumstances, so I can't really give good guidance, but it would be good to consider the entirety of this post as you begin your graduate career.