My research supervisor wanted to produce a cost-benefit analysis for an infrastructure project (civil engineering). First he had someone else work on it, then he involved me. When I got involved, I noticed that the benefits were inflated and the costs downplayed to convince the reader of a massive benefit-cost ratio. I do not know if he stands to personally gain from this. I discussed it with him a few times, he pushed back with "I am experienced and I know I am right", and eventually in the final revision of the paper he changed my numbers. The way he changed the numbers is not even smart, he just changed the text, he didn't redo the calculations. Now I fear that if I bend under pressure, these bad numbers will be out there, forever! I am sure he will take the published paper and send it to every politician in the country trying to get this project approved. What can I do? I am still his PhD student so I can't get too confrontational.
Make absolutely sure you understand the numbers correctly before suspecting, or even worse, accusing your advisor of dishonesty.
The following text only applies if you have satisfied yourself that there is no way these numbers could have been reached in an honest way.
If you are absolutely convinced of this, get out of there, under whatever pretext. You do not need to confront your supervisor, for which you have little power at this stage; but get out.
You are not only in danger of damaging your academic reputation. If the study enters policy, you may be indirectly or directly made responsible for wastage of public money and could face potential legal consequences stemming from that.
If your supervisor is really into shady practices, he may even deflect any potential fallout that might transpire to his "dishonest" student coauthor; i.e. you.
Do not accuse him of that. You have no proof and it is almost impossible to establish facts in the constellation you are in. Just get out of the paper, and ideally, out of the PhD relation. Find an excuse.
You have my sympathies for a situation which no one should be ever be in.
First off, I'm sorry you were put in this position. I know you don't want to get confrontational but your supervisor crossed a line. I don't know how far along you are in your PhD but I think it's time to look for another supervisor (could be a different university). Here's what I would do:
- Stall this publication to give you some time
- Quietly look for another supervisor
- Once you have a back-up plan, confront your current supervisor
- If he agrees to keep the real numbers, great, go ahead and submit and then change supervisor
- If he refuses, demand not to be listed as an author, and then change supervisor. DO NOT ACCEPT TO BE A PART OF THIS
In my opinion, you have to cut your ties with this guy as soon as possible (even if he agrees not to change the numbers). He might already have done the same before and his publications could be a ticking time bomb until someone figures it out and his reputation collapses. He might also already have a shady reputation to begin with.
You need to make sure your understanding of cost-benefit analyses is on solid ground before anything else
So this is this is my first time answering a question on Academia, but given this situation is based within my area of expertise as a Professional Civil Engineer, I think it's appropriate to provide input on how difficult it is to 'accurately' put together a cost/benefit analysis.
Costs - From your question, you had indicated that your advisor had utilized the costs of a local contractor while you were depending upon published numbers from similar papers/reports. Consider a few things with the sources you're using, if you're relying upon RSMeans for example, you're going to need to consider that the costs of all work will vary considerably between Oregon, Texas, and New Jersey. Furthermore, you must further break those costs down to consider the cost of material, labor, and overhead/profit. Greater union participation in the northeast results in labor costs being higher than the national average.
In addition, you need to consider the local economic climate. Estimates I prepared prior to Covid are completely worthless today due to massive supply chain disruptions which have invalidated my previously reliable material costs.
What I can tell you from my work experience handling asset management planning for municipalities is that I don't rely upon published numbers for asset valuation and maintenance if I can help it. I'd much rather get my information from the people that work in the area and adjusting for CPI because when it comes time for that maintenance to be done, they'll be the ones performing it.
Additionally, you should be aware that there are definitely some costs which are very difficult to account for and it's not unreasonable for a civil engineer to just throw a dollar amount and justify it with 'professional judgement'. I don't think I've ever issued a formal estimate that I didn't slap an additional 20% onto the back-end for 'contingency', because the amount of times we open the ground and it's not identical to the survey is just about 100%.
Benefits - So this is where things get a little bit hazy. You have to consider who the audience is. From your question it doesn't sound like the target audience is another engineer, rather it's political in nature and to that end they may've hired your advisor to put together an analysis which depicts this project in an extremely favorable way. I want to make clear that this is not automatically unethical, there are a lot of gray areas in civil engineering and if you elect to pursue this as a career you will find yourself in them often.
ESPECIALLY for infrastructure projects. They are costly to perform, they result in traffic, they often require tax increases, and a bunch of other politically unpopular things. But the reality is, very often municipalities aren't setting money aside to maintain infrastructure because otherwise that political victory might happen in a different administration and the current politician loses their re-election bid because they're 'wasting' money on infrastructure projects and maintenance that 'definitely isn't needed'.
As for accounting for benefits due to increased economic activity not being 'actual dollars related to the project', are you sure? Would that increased activity have occurred without it? If you are trying to justify installing a new tide gate to mitigate flooding issues caused by worsening climate change, you may need to include in the benefits all the flood damage you don't have to account for. As a whole, the general public doesn't appreciate money that wasn't spent on things that didn't happen, so your advisor might be trying to show this ratio to help them see that benefit in a different way.
- Ensure you understand the goals of the client on this. Bear in mind that as a civil engineer, you will occasionally be in ethically ambiguous situations, but you do have an ethical obligation to represent your client's interest. This doesn't mean that you lie, but there are lots of ways to present technical information that makes something look more appealing. If you're unsure if you're going too far, consider the key tenets from ASCE's Code of Ethics, which require you to hold paramount the health and safety of the public. As a reminder, the public often won't understand why they need to pay more in taxes, so it can sometimes be your job to present data in a way to persuade them.
- Have a discussion with a colleague (not your advisor) about the ethics of the situation. Ethics is always hazy and having another person to discuss things with can help a great deal. If your colleague considers the matter to be unethical, or bordering along there, then request a meeting with your advisor and ask for their input on the ethical implications. It may be that the paper should have some kind of disclosure included; this is akin to a City Engineer disclosing to the City Council about some kind of conflict of interest they may have relating to a project they're responsible for inspecting. The Council can elect to replace the engineer for that project or acknowledge the conflict and elect to allow things to proceed based on their trust of the engineer.
- Document everything. Hopefully your advisor's word-of-mouth cost estimate is backed up in writing in some fashion, because that's going to need to be an appendix. For most projects, it's very common a ratio on the order of a 20 page narrative and 200 pages of appendices, so don't be shy on providing them.
Every cost benefit analysis includes assumptions. There will always be pieces that are a bit ambiguous and can be done several ways. Based on your edit, the items you are disagreeing about would not be a case of fraud, but of judgement (and the supervisor’s judgment does not seem unreasonable on the face of it here).
The correct thing to do is to be very clear about the assumptions and where the numbers you are using are coming from. Ideally, you would also be transparent about the other options you did not choose and how they would affect your results in the paper, but that does not always happen.
At this stage, I would suggest you do two things. First, add text around assumptions made to the paper. This may be an appendix. This will not change the overall cost-benefit “answer”, but will at least allow a careful reader to ascertain what is being included and left out. It should also be okay with the supervisor since it is not affecting the paper’s takeaway “sound bite.”
Second, if there is really no justifiable way to defend some of the calculations, suggest an alternative with the justification that “readers may find this number to be a problem” or “people will attack our overall conclusion as being biased if we include this number.” The key is to put your objections “in the mouths of the reviewers” and therefore not make it about your inexperience vs. your supervisor’s experience. If the supervisor won’t budge, just continue to document heavily in the appendix.
I do not understand why none of the other answers may consider the case that there is no ulterior motives here. Especially in the case of engineering, academic calculations are based on ideal situations (none of which are in the real world) modeled after conditions which are not well understood. Actual estimates from industrial sources (the alleged source of the professor) are closer to reality when looking good data to use. The professor is presumably more experienced in the field than a PhD student and likely has a better idea than a PhD student on the reliability of sources iif information in the real world.
In addition, the subject matter is a cost/benefit scenario, a question of economics; riddled with assumptions based on an economic model that doesn't fit reality that requires predictions of the future (something which is somewhat of an art) so I wouldn't even take a subject matter expert's likely facetious statement like "Just trust me I'm right" offensively as their best guess (a core engineering skill) is likely better than a student's just ftom experience.
Just to offer a different perspective, if the proffesor says his information is more reliable, just try to understand the underlying reason for determining the reliability of the data. That said, you should absolutely point out that the calculations are inaccurate if they aren't rerun with the new data presented.
If I want to get a new roof next week, which cost is more relevant?
A) A published average for roof replacement, or
B) The estimate a local contractor gives me to do the work next week.
Surely, you can find pros and cons to each: published numbers aren't for my roof, so they're biased if my roof isn't a typical one. Published numbers might be based on costs for materials and labor when the previous study was done, possibly a year or decade ago, and not apply to next week (even straightforward adjustments like accounting for inflation may not be representative, since inflation doesn't affect prices of everything equally).
On the other hand, the contractor I got an estimate for may be cheaper or more expensive than average. Their estimate may not include cost overruns that are typical for a project like this.
Neither A nor B is fake, and probably most people would be offended and insulted if they used either estimate A or estimate B and you called them a fraud, or told other people they were a fraud. It seems your advisor is leaning towards using an estimate like B, and they feel their expertise gives them a better judgment than you in that decision. Maybe they're right, maybe not; we don't have enough information here to know.
It's fine for you to argue for the benefits of (A) over (B), but I would keep out any accusations of serious misconduct until you have strong evidence of them. The most important thing for your paper is to be clear in your methodology. If your cost estimate is based on an estimate given by a local contractor, you need to write that in your paper and explain how that estimate was reached.
If your benefits estimate includes secondary economic benefits, then explain clearly what those secondary economic benefits are: what are they based on, how does the methodology used to reach those numbers compare to other literature, etc.
You certainly should not be basing your final judgment of the estimate on whether the number "seems reasonable". Sure, it's a bit of a red flag to investigate if your numbers are far from what you expect, but ultimately the best answer is the answer you get with the methodology you justify. If that number is cost:benefit of 20:1, then that's the answer. It doesn't matter if 20:1 is "reasonable" itself or not, if all your assumptions to get there are reasonable then the answer is, too.
I'd also echo a comment by @a.t. - when there are multiple ways to answer an estimation problem, one solution is to provide a range of answers. If you can demonstrate how different assumptions lead to different answers, your reader can be better informed, can better understand the levels of uncertainty, and can possibly use their own judgment of what circumstances are most likely closest to the truth.
Cost Benefit Analysis is not an exact science and all benefits cannot be accurately quantified. The key is to understand if the revised calculations are transparent and defendable. Let your supervisor know you still don't understand how he has come up with his numbers and what his assumptions are. Also ask him who the audience for the analysis and what he expects they are looking for.
It's likely your supervisor has one foot in industry and one foot in academia. Large infrastructure programs, where CBA is used, generally do not show strong returns when initially calculated on base assumptions. Some of the benefits need to be padded a little, and this is where your supervisor will be relying on his experience. His padding will need to be supported by his assumptions.
Politics are also heavily intertwined with infrastructure in the public space. Politicians will expect, because they think something is a good idea, that it will be supported regardless of the result of the analysis. There are surely cases where the benefits will never stack up, such as a monorail, and others that, although the benefit can never be shown on paper are generally considered good ideas with significant public benefit.
I am not a civil engineer, so it's totally impossible for me to judge what you are saying based on the merits. So, this is some general advice.
I think, as others have said, a cost-benefit analysis is not something that has a right answer. I think the main requirement should be that the numbers are justifiable and source-able, and there is a reproducible way to get from your assumptions to the results of your analysis.
The differences of opinion in the "Note" part of your answer, to me, are difficult to parse for correctness. For example:
he wants to rely on word-of-mouth cost estimates by a local contractor instead of my idea which is to use published numbers in similar papers/reports.
The details really matter here. How "similar" are the papers and reports to the actual costs the project will incur? How reliable are the local contractors? Based only on what you have written, the truth could really be on either side... it could be that you are estimating costs for some very specialized task that local contractors don't have experience with, and that you found some papers/reports covering essentially the same project that are very reliable. Or, it could be that the papers/reports are similar but different in some crucial aspects and include some extra elements that are not necessary for the project you are working on, and the contractors can give a better estimate for the specific tasks you need.
Given the murkiness, I would generally advise against thinking of this situation in black and white ethical terms. If you were very experienced in this field and could prove your advisor's estimates were going to miss the mark by a factor of 20, then it might be a different situation, but I suspect this isn't the case as you are a research student and were pulled onto this project at the last minute.
Instead, I would focus on trying to understand what assumptions your advisor is making and how they derives their results from their assumptions. Unless your advisor is totally incompetent or a fraud, both of which are unlikely, there is some path that you can follow to derive the results. Even if you don't agree with the assumptions, if the assumptions are clearly stated and the main results can be reproduced, then I would classify this as a difference of opinion, and not an ethical dilemma. You may indeed learn something from understanding your advisor's reasoning or why they trust their source of information over the sources you are using. Again, you might learn something, even if you don't agree with the methods in the end. And it won't ruin your career to be associated with this analysis. You will gain experience, have a paper, and as you progress you'll have more opportunities to do things your way.
In the event that you are 100% sure there really is no way to justify the analysis, either because the assumptions are objectively wrong or the analysis is bogus, and after discussing your concerns with your advisor they aren't willing to correct demonstrably false statements, then I agree you need to get out. To be clear, I think being 100% sure is a pretty high bar here, because a cost-benefit analysis is going to be murky and you are not an expert with a lot of experience. But if you are really sure that this is what is happening, talk to someone senior in your department that you trust (like a grad chair), explain the situation, and try to get away from this advisor.
Look, you have to get your good name out of this thing. You are being used as a frontman at best and a patsy at worst by this professor for a dodgy study. Your professor is an old dog who knows how the dark trade-offs work. If he gets a "result" from this "study" there will be more and more "consultancy".
I assume you have had a look at similar studies during your initial reading into the programme. It might be helpful to find some past projects that were over optimistic in their B/C and the consequences to that country's exchequer from all that goof-up. Spain has done a lot of infrastructure projects in the last 40 years and their construction cost-control processes are well-regarded. I'd expect that their vision of B/C would be pretty clear by now, especially w.r.t. local economy benefits.
Beyond that I can only suggest gently sounding out the postgraduate studies dean or HoD - or any sympathetic professor - to get ideas on all this. In talking to these people, do not adopt a pitiful oh-my-career or aggressive damn-your-corruption manner. Just be quite cool and matter-of-fact about it all and clarify the long-term danger to a department's reputation by association with a dubious study in support of a huge public investment. Impartiality and objectivity are all the more essential when glory-hole projects are touted. It's one thing to support a local authority in a relatively small but socially beneficial project for the city: a department without a heart is one without a head. But it's quite another thing to put the department's stamp (not just yours) on a study potentially instrumental in guiding a decision on massive public spending. Rehearse your lines with someone trustworthy at home before going to speak to anyone and note their comments.
If this fails, you have to extricate yourself academically from your supervisor even at the risk of a career detour out of academia for a while. You can plead "personal reasons" (e.g. need extra income, desire to be closer to ill parent, wife's career/personal choice, etc) for this detour.