I believe my department is failing to enforce standards in doctoral examinations. How to proceed?

Question

I have witnessed multiple doctoral examinations at my (top-10 rated) university in which the student did not, in my opinion, demonstrate the required level of knowledge and understanding under questioning but was awarded the degree anyway. The committee consists of the advising professor, usually one or two additional professors (one of whom may have been a co-advisor), and the professor that chairs the doctoral examination committee in the department.

I suspect that the degrees are awarded at least partially out of expediency because failing the student after admitting him/her to the examination would be unfair to the student and indicate a failure by the advisor and be embarrassing to both, especially because the examination is open to the public (so other students as well as friends and family are present).

In addition, I believe that choosing one examiner to be a co-advising professor is a blatant conflict of interest, since failing the student would also amount to a failure of the co-advisor and is therefore not in his/her interest. The third examiner is sometimes selected by the advisor strategically from a pool of current or former collaborators.

Finally, I get the impression that the chair of the doctoral examination committee defers to the recommendation of the advisor and does not serve as an additional arbiter of standards for the department (as I assume would be his/her duty), again out of expediency, because if he/she did refuse to sign the departmental forms, it would create a conflict and presumably a lot of additional work.

I know I'm not the only one who feels this way about the situation, but so far nobody, to my knowledge, has spoken out. I'm inclined to speak out anonymously because I find the lack of enforcing standards to be embarrassing and sickening, especially given the rating of my university. Speaking out, even anonymously, could create problems for me, which is why the best way forward is unclear to me.

Does anybody have any recommendations about how best to proceed? Are my expectations unrealistic?

Edit: One commenter asked me to be more specific about what I thought they lack. Briefly put, they all struggled to answer questions about the problems they investigated. These questions required nothing more than a fundamental understanding of the relevant physical processes. Some of the required understanding is part of MSc courses and would not have been acquired as part of the PhD research itself. What troubles me is that rather than continuing to question the student, the committee members seemed to be going out of their way to give clues and avoiding pushing the student too far in the sense of making the deficiencies too obvious.

Answer 1

The observation that the level of knowledge, skills and achievements shown by someone finishing their PhD varies wildly both among institutions and also among students at the same institution is a very widely made one, but nevertheless is still striking to those who observe it for themselves. At my current institution, we have some graduates that I really wonder how they could possibly have gotten through the program, and we have other graduates whose superior work shocked me: once every few years a graduating student in my current department (UGA math: about the 50th best in the country) has a thesis that would have been regarded as distinctly strong at the department where I received my PhD (Harvard math: always in the top three). Is the variation lesser at a place like Harvard? No, I think it's greater. In 2003 I graduated from that department with a respectable thesis. One of the 2004 graduates had a thesis that was, suffice it to say, orders of magnitude better. Do Harvard math PhDs have an understanding of the relevant mathematical processes? On average, yes, to a reasonable degree. All of the time? No, probably not.

I don't think it is helpful to cast the problem in "whistle-blowing" terms. That suggests some kind of clandestine nefarious activity, but so far as I know most interested parties are acutely aware of the issues raised by the OP and described above. If the OP is a tenure-track faculty member at the institution he describes, it is (to say the least) entirely his right to take action to enforce higher standards in the PhD program. I don't understand the need to do so anonymously -- you don't need to do your job anonymously, and doing so sounds difficult.

Here are some thoughts about how to get involved with this.

1) Pace yourself.

The cliche is that young faculty members come in wanting to fix the new department. In my experience these desires are overall quite positive, but they need to be tempered with an awareness of how the department actually works (which will not be exactly the same way as the few departments they have previously been involved with) and also need to take time to get to know the personnel.

While you are an assistant professor you should start serving on thesis committees of other students. You should be conscientious but less vocal than you plan to be in the future: you have a lot to take in. In particular I think it's best if you spend at least a year observing PhD students and the PhD program before taking on students of your own. So long as faculty are invited to exams and thesis defenses, make a point of showing up, even if you are not on the committee. If such events are not typically open to faculty members, consider asking if you can show up "as an observer".

2) As long as you are confident that you will not be making real trouble for other students and faculty, don't be afraid to enforce your personal standards. You need not be held to the lowest common denominator.

As a faculty member in your department, some portion of the task of setting the standard for the students falls to you. You can be stricter or more exacting than some of your colleagues, so long as you are not making bitter enemies or ruining people's lives. In my department, the oral exams run by my research area are well known to be longer and more grueling than anyone else's oral exams. Maybe this dissuades a few people, but not too many since more students study in my research area than any other area. A faculty member who has a reputation for high standards and working with students to meet them often becomes very popular and influential in the department.

3) Realize that the thesis defense is too late to "hold the line" for any standard other than the requirements of the thesis work and the written thesis itself.

Academic traditions must differ a bit, but I have never seen a thesis committee test for basic knowledge during a thesis defense. The job of the thesis committee is to make sure that the thesis is an acceptable written document and of course to convince ourselves that the student was responsible for it. That is not a formality: it often involves hard work down to the wire by the student, the advisor and some of the committee members to get the thesis in acceptable shape. The thesis defense itself really is a formality: in my experience whatever issues may remain with the written thesis are not even much engaged at the defense itself, though they may still be dealt with later. If the committee doesn't think the student is ready to get a PhD, the thesis defense should not be scheduled. Period.

If a student exhibits a lack of basic knowledge at the thesis defense: yes, that's a distressing failure worth doing something about. But the failure took place years before. Most PhD programs have several rungs of exams, both written and oral, extending over several years. A student should not be able to pass these exams without a knowledge of the fundamentals of the field. Moreover the typical culture of academia is that not everyone passes these exams the first time they take them. Generally speaking it's quite culturally acceptable as a faculty member to say "Sorry, Student A doesn't know Subject X well enough to continue on with his studies." In my experience, these kind of exams are often done relatively quietly in the department. Some faculty -- maybe those who are directly affiliated with the most relevant coursework -- are involved in administering the exams. The majority of the faculty may not be. If you're crusading for higher standards (not a bad thing!!), this is your place to get involved, get active, and draw your lines in the sand. Take time to figure out exactly what the problem is. Are the exams not being graded seriously enough? Are the syllabi not testing what you want or not testing it to a high enough level? Are the advisors of students given an undue influence over the result of these exams? (E.g. "It is a waste of my Student A's time to know Subject X. I want him to work on Subject Y only.") Once you've identified the problem(s) you can get to work on the amelioration process, patiently but implacably. Because it is in fact not a criminal conspiracy, if you are passionate, smart and keep your eye on the ball over a period of years, you will have more of an influence than other faculty members who are a bit too lazy, too "nice" or too whatever to run things in the best way.

Answer 2

Yes, I have seen some similar things in my life and this is indeed a very tricky question. There is a strong tension between our ideals regarding the quality of PhD's (and other degrees) we feel our institutions should be awarding and the practical realities on the ground, which involve messy and awkward decisions that have a direct influence on people's lives. This reality is such that sometimes we feel we have no choice but to compromise our lofty ideals to some extent. As long as the compromises are small, this is probably as it should be, but when the compromising reaches alarming levels, it is reasonable to start becoming seriously concerned.

What I would suggest is this. First of all, it may be worth remembering that the behavior you are describing is not your doing nor your responsibility. The chair and the members of the doctoral examination committee are the ones who are making those decisions and putting their own reputation and credibility behind them. So, although you find the behavior concerning and perhaps for a good reason, you cannot reasonably be expected to fight all instances of poor decision-making at your university. Thus, one option is to simply do nothing except to resolve privately that if you ever preside over a doctoral examination yourself, you will follow your conscience and let the chips fall where they may, and thus play at least a small role in making your department and university better.

If you want to be more proactive, an honorable course of action would be to actively pursue the role of chair or member of the doctoral examination committee so you can put the goal of raising standards directly on the agenda. Or you could strive to be appointed to the departmental committee that decides the makeup of the doctoral examination committee, and thereby help get people with high standards be placed on that committee and help prevent the kind of conflicts of interest you are describing. Through such behavior you will be setting a personal example and showing that you are not just criticizing others but willing to take action yourself for the goal you believe in. Since you mention that you believe others in your department feel the same as you, you may find that you have a good number of allies who will support you in your mission, and even your opponents will respect that you are acting on your convictions. Of course, it will take time and effort to bring about change, so be prepared for a long struggle ahead.

Finally, if your position within the department does not make it realistic for you to seek the kind of active, visible role I suggested above, or you are unwilling to make such a large effort to see change happening, I don't have much to suggest. I don't think there are easy fixes; someone will have to do the work to change the culture of your department. If it's not you, maybe you can work behind the scenes to identify more senior allies willing to take on that role. You might get lucky and find someone passionate enough about the subject, but probably you won't.

Answer 3

I have observed the same problem in a number of high-ranked universities in the North America and Europe. In fact, I would say that I have only observed two or three defenses where the student exhibited true competence and control over the subject matter (factual and logical) -- all were my own students. Coupled with student-progress related discussions at faculty meetings, this led me to reexamine my assumptions about what the degree Ph D "means". Rather than consider only what I believe the standards should be, I concentrated on what they actually are. Consequently, I saw that none of these dissertation defenses were shockingly below the existing standard, even though they were sometimes disappointingly low. I came to understand that the students' shortcomings were largely a failure to consider alternative and a failure to know facts, but generally not a failure to have any idea how to answer the question.

As a member of the examining committee, you (may) be in a position to clearly indicate that the student is not ready yet, and if that is the case, you should not shirk your obligation to maintain appropriate academic standards. If you are not in that position, you have two realistic options. One is to focus on asking relevant questions from the floor. Institutions vary in terms of how "outsiders" can interact with the defense, ranging from completely closed to open to the public and anyone can ask questions. The goal should not be to prevent the student from getting the degree, but rather to make it publicly known that there is a problematic failure to enforce proper standards regarding core knowledge in the discipline. The second is to make this be an agenda item for discussion at faculty meetings, where policies regarding graduate programs are (possibly) set. It may be that requiring a specific course at the PhD level would address the problem.

Answer 4

Nobody knows everything. My experience with defenses is that the questions keep getting more and more difficult, until the student says "I don't know", at which point the faculty make an effort to walk the student through the issues. After the committee is convinced that the basics are really in place, this process repeats itself. AT many universities, this process takes place behind closed doors with the committee, AFTER the public presentation.

So, what should you do? Assuming, for the moment, that you're faculty, if there has been a closed session after the defenses you were not privy to, you really have no idea how the student performed in the closed session, thus you have little to say officially about the matter. That said, if you believe that the student does not merit the degree, and that releasing said student to the world would harm your department's reputation, you might mention your concerns to the chair of your DEPARTMENT'S grad program or your chair (not the Dean's office, university officials, etc.). Keep it in the house.

If this resonates, this might rise to the level of a committee or general faculty meeting, in which case you can offer your suggestions on how your department might better groom your candidates for defense -- Perhaps a more rigorous proposal process or more frequent committee meetings or even create a system where the student gets more practice and feedback on presentations and QA sessions (we have a summer presentation series of grad students presenting to other grad students with no faculty presence that seems to help). If this conversation doesn't happen, and you still feel strongly about it, talk to your chair about placing you on departmental committees that have some oversight, like your graduate committee, and gently try to sell your point there. As others have said, this becomes an issue of changing the departmental culture. My own opinion is that this difficult task is best approached through a "what can I do to help make this better?" strategy, rather than a dig in your heels, "We're doing it wrong" stance.

One thing that I've seen that puts students in bad situations like you describe is late formation of doctoral committees. Committees should form early, and meet with regularity. If you don't do this, you can wind up in a situation where a faculty member gets involved too late to do any good.

The one approach I WOULD NOT recommend is playing pariah on a students committee at defense. If you want to make your statement, the time is actually WELL BEFORE the defense takes place. You can nudge a committee you're on into laying out a careful schedule of benchmark reviews with the student, and whip him or her into shape if things aren't looking good, and even hold up the defense date until the student is ready.

Interestingly, at our university, the policy following distribution of the dissertation used to be that the committee chair (who is external to the department) would informally ask other committee members if the defense should not happen. This has changed to a more anonymous electronic mechanism where each committee member needs to affirm that he or she believes that based on the dissertation, the student is ready to defend.

If you're a student observing the sort of pattern you describe, you might mention it to your adviser or chair, but I wouldn't expect this to be well received. I'd put this in my "stuff I've observed that I can do better when I launch my career" pocket, and try not to forget it later.

Answer 5

An issue not yet raised is that of expectation. If the defense is sold as a "thesis defense", as it frequently is in North America, then it is unreasonable to expect the student to have prepared answers to questions outside of that scope. In the defenses I have been party to, it was strictly about the thesis itself. The outcome of deliberations had to fall into one of four narrowly defined categories:

1. Accept thesis as is
2. Accept with minor corrections
3. Major revisions required
4. Reject

(I have seen categories 2-4 used; 1 I take to be exceedingly rare.) All of these categories have to do with the thesis. Nowhere does it state that the candidate must exhibit sufficient expertise in the general subject area; this is supposed to have taken place during preliminary and qualifying examinations. (With the continuing specialization and narrowing of focus it is not even clear that the subject area can be properly defined.)

As Dan Romik states, changing this state of affairs requires a deep change of culture and lots of time and energy.