The arcane biomedical research funding system inhibits innovation because its bureaucratic complexity perverts meritocracy and stifles imagination, making it difficult for research dollars to reach the most innovative ideas. Can the Cancer Moonshot program change that? Observations from the trenches of the war on cancer.
When we fail in a big task, there are two possibilities: either it is because the task was inherently difficult and thus beyond our reach no matter how hard we try, or alternatively, it is because we did not try hard enough and made mistakes. Sometimes we can distinguish between these two: Did I fail to solve that math problem because there exists no simple solution, or because I am poor at math and did not study hard enough? A quick comparison with classmates will answer this question.
In cancer research, such a comparison of our performance is not possible, for there is only one planet with human beings on it, whose unfortunate biology makes them universally prone to develop cancer. More than 20% of people will die of cancer which now is overtaking heart diseases as the most common cause of death. After almost 45 years of the “War on Cancer”, declared by Richard Nixon in 1971, and more than $100 billions spent by the National Cancer Institute (NCI), the largest of the institute within the National Institutes of Health (NIH), we have not yet cured cancer. Despite the regular reports of “breakthroughs” and of declining (relative) cancer mortality, still nearly 600,000 people die of cancer in the United States every year and the absolute numbers are increasing due to aging population. Thus, as widely noted, we are not winning the war on cancer.
But why haven’t we by now cured cancer? We are about to spend another $1 billion in president Obama’s Cancer Moonshot program, led by Vice President Joe Biden. This is just 1% of what the government has spent in this entire war so far. It is therefore clear: if the Cancer Moonshot is to change the trajectory of failure, it has to use this relatively tiny infusion of money not to double-down on existing efforts but to catalyze a major shift of this culture of failure. We cannot stay the course.
Let us think big and examine cancer research in the scheme of the two types of failure. We can’t tell for sure which one is true:
(A) Could it be that cancer is inherently hard to cure –perhaps impossible– due to some fundamental constraints of biological physics, much as there is a theoretical limit to fuel efficiency of combustion engines or to energy efficiency of incandescent light bulbs?
(B) Or could it simply be that the cancer research community, involving Governments, Academia and Industry, is failing big time in how it funds, steers and manages cancer research?
The first question (A) is rather academic and an answer will be very technical: one can argue that for some profound theoretical reasons, controlling all the trillions of highly diverse and dynamic cancer cells of a tumor, which all embody the evolved resilience of nature, might be akin to herding cats in a multi-dimensional space. But it would clearly suggest that doubling down, that is, pushing the limits of the current celebrated but “partially working” approaches, is not the path to a cure. The tech industry has long realized that the future lies not in squeezing out the remaining potential for efficiency improvement of traditional combustion engines or incandescent light but instead, in new, outside-the-box concepts: hybrid or electric cars, LED lights. But our cancer research funding system is simply not structured to promote the equivalent of such revolutionary advances.
Thus, the more pressing and practical problem is the possibility of insufficiency of our own system (B). We need to identify the shortcomings in managing cancer research. If we correct them this may open hidden doors to outside-the-box ideas, revealing new room for improvement. Currently we are just doubling-down on existing approaches. Responses to the Cancer Moonshot announcement are all about: more clinical trials, faster paths to approval for new drugs — which are only “variations on a theme”, not new themes. More targeted therapy, more immunotherapy.
So, what fundamental features of our collective cancer research efforts prevents us from departing from this culture of “stay-the-course but just with more troops” ? What prevents us from changing strategies? When hearing about the Cancer Moonshot program and the dollars that it will spend, the public needs to know that our system to direct research funding to those who could change the paradigm is highly ineffective. Here is the central problem.
Animals do it, athletes do it, companies, civilizations do it: They all have evolved systems in which the strongest, the best, the most talented rise to the top of some hierarchy of influence and recognition. The overarching problem in cancer research is that it is not meritocratic. True meritocracy, the principle that the most talented in an organization is rewarded with recognition, responsibility and resources, so that she can steer its course to the benefit of all, is nearly non-existent in cancer research. (Don’t be fooled by the apparently impressive bios of its celebrated leaders –more on this self-fulfilling prophecy later). Unlike most academic fields, the way forward in biomedical research, and specifically, in cancer research, is bumpy and messy. It is too complex to provide a level playing field for open competition that is the underpinning of a meritocratic system. The rugged and convoluted path to the top creates many loopholes that afford those researchers with little imagination, courage, dedication or intellectual potential to innovate, but with keen interest to be a leader (or just in need of a salary as a researcher), ample opportunity to game the system and rise to a leadership position and to stay there. Unlike in athletics, arts, business and engineering, or most other domains of civilization, academic biomedical research offers too many hidden shortcuts to bypass competition with respect to domain-relevant abilities. This creates a fertile breeding ground for “careerists”. A system incapable to put its most talented at its top will go nowhere.
Below I discuss five mechanisms, rarely articulated but tacitly acknowledged by many, which have perverted meritocracy in cancer research:
- THOSE THAT BUMP INTO LOW-HANGING FRUITS ARE TOMORROWS’ LEADERS
A first mechanism that undermines meritocracy lies in the fundamental way in which discovery in molecular biology works. Let us demonstrate this using two fictive cancer researchers, Peter and Mary, both hardworking, rather skilled scientists. In his graduate studies, Peter, despite using simple, standard tools, stumbles onto a gene that is mutated in 50% of all cancers. What a discovery! By contrast, Mary discovered, after years of tedious research employing most creative techniques, a gene that is mutated only in 1 of 100 cancers. Peter will be able to publish his finding in a prestigious journal, for a gene so frequently mutated in tumors must play a very important role in cancer development — it could be a worthwhile drug target. In fact, in the real world such type of discoveries have made a number of young researchers famous overnight; and Peter would eventually move on to lead a major academic institution. Mary, with her solid but non-spectacular result achieved with sweat and elegance, and careful thought, may secure a faculty position at a smaller university.
Now, let me ask, what is easier: Finding a mutation that occurs in 50% of cancer patients, as Peter did, or one that is present in only 1% of all cancer cases, as the one Mary discovered? Also, please ask: What requires more skill? And: who of these two will more likely to become a leader? There are many Peters and Marys in cancer research. While the Peters are almost guaranteed to become (perceived as) leading experts, they are not necessarily the most creative and innovative. (Of course the Peters might, with small probability be exceptional, but that probability is further diminished by the small probability of being the first to find the low hanging-fruits). Nor are the Marys, of which there are many more, automatically the more creative researchers. But if one of them has a brilliant idea, our system will not be able to single her out.
So the field of biomedical research tends to favor the luckiest (from among all the hard working and to some minimal extent talented). We could live with that. But here is the bad news: it is even worse. Our system selectively promotes precisely those with a penchant for picking up the brightest, lowest-hanging fruits, while disfavoring the most persistent and smartest researchers who are inclined to seek the hidden fruits, undeterred from the risk of failing to find any, who see meaning in abstract details that no one sees, in places where no one searches. We do not simply operate a lottery to promote leaders, we are biased against the very best and brightest.
Answering a question that everybody asks is much better for a career in cancer research than asking a question that no one asks. But what if the cure of cancer lies in the answer of that one question that no one asks?
Now, if Peter was just lucky , or worse, benefited from some bias — don’t think that chance will eventually “even out” the career fortunes of the Peter’s and Mary’s. There is another problem: the hierarchy of academic haves and have-nots is stabilized by a self-sustaining vicious cycle. Worse, the gap in their research funding will grow, just like income inequality. This is because of the rich-get-richer principle: publishing in prestigious journals and being at prestigious institutions have perception consequences. And such perception affords power and influence, making publishing and obtaining research dollars even easier, further enhancing the perception, thus fueling a self-fulfilling prophecy that is detached from true merit and is so robust that even the equalizing hand of chance events cannot slow down.
2. CROWD WISDOM AND DEMOCRATIZATION LEAD TO MEDIOCRATIZATION.
A second mechanism to which meritocratic principles fall victim lies in the fundamental nature of the peer-review system through which the NIH selects the “principal investigators”, the Peters and Marys, who will receive research funds. First let me explain how the system works (very roughly).
The NIH awards research grants (of roughly, 1–5 M $ per investigator and his/her laboratory over five years for a defined project) from a total annual budget of more than $30 billion dollars, of which the NCI controls $ 5 billion. Academic investigators apply for funding by proposing particular research projects. A grant application contains typically a total of 100–400 pages (of which 15% is the actual scientific proposal), depending on the type of grant (from a huge zoo of programs and award types). They require time, sweat, anxiety, and countless phone calls with collaborators and suppliers and budget managers to write. And yes, a pinch of scientific creativity –but not too much– may also help. In other industries, such proposal writing would be a full-time job, and represented in the job description of entire staffs dedicated to it. In academic research scientists do this “on the side”. Repeated submissions are almost always necessary to obtain funding, such that the time span from inception of an idea to receiving the research money can be two or more years — a major distraction from creative work. Therefore, the somewhat unfair perception that NIH stifles rather than promotes innovation contains a proverbial grain of truth, and is captured in the suspicion that ‘NIH’ may actually stand for ‘Not Invented Here’. (Similarly, some say, NCI means “No Cure Intended” because of its lack of courage to embrace new ideas. More about that later.)
Much has been written about how peer-review kills innovation. To be fair, predicting the future creativity of a professional, let alone evaluating an overtly detailed plan to plow unknown territories, is inherently difficult. Grant applications submitted to the NIH by researchers around the country are evaluated and scored by peer-review panels, each consisting of 20 or more “peer-reviewers” — biomedical researchers like you and me. The panels, organized by subject matter, read and discuss the applications and give each application a score that then NIH officials at the various institutes, such as the NCI, use to make decisions on which proposed project to fund. In doing so, they almost blindly trust the reviewer’s evaluations (with little leeway to override them). It is hence important to realize that the collective of peer-reviewers, and not the NIH officials, essentially determine the specific direction of cancer research. Thus, the latter are off the hook. The problem is in the system. Of course, consensus-constrained evaluation by panels of peers only lets through a tiny, highly streamlined range of type of research that pleases everyone. Worse, it kills ingenuity — for, the genius has no peers, as computer science pioneer Edsger Dijkstra once noted. But let’s look at the details.
At NIH, realistically only the top scoring 10% or less of the applications end up “getting funded” in each round of application. But the scoring process is so capricious that success in the grant application process is widely considered to approximate the mathematical randomness of lotteries. Yet the NIH must be commended for its awareness of this fact and its continuous effort in conducting statistical self-assessments and for the continuous attempts to improve the evaluation of proposals.
Unfortunately, an untouchable and unquestioned tradition suppresses meritocracy: the peer-reviewers are anonymous to the applicants and their technical expertise to evaluate a given proposal is based on self-assessment and never thoroughly scrutinized.
The practice of anonymity in scientific review goes back more than 150 years to the early days of the British Royal Society and was well-intended: chiefly, to protect a peer-reviewer from retaliation by, well, her peers whose application she evaluates. Anonymity would allow for frank, possibly negative critique to surface, so the idea. But it has unintended consequences. For, anonymity also frees reviewers from accountability, that is, from sanctions if they do not perform with due care. No other system in our society absolve its critical decision makers from accountability. Worse, since serving on review panels is an onerous task, only attracting junior investigator in need for decorating their CVs with such service, few experienced and seasoned (and more busy) scientists participate in review panels.
Just imagine a corporate world in which companies would make decisions on products, mergers or CEO hires based on averaging the opinions provided by a group of personnel crudely selected from the lower echelons of the organization, without ever having been tested for their expertise and judgement, let alone intellectual facility, yet given the authority of the incontestable sage. And imagine that there is no consequence for making incorrect evaluations that can affect the companies’ well-being.
So here is the problem: The NIH’s funding system comes close to such cartoonish dysfunctionality. The opinion, let alone the expertise of the reviewers, cannot be questioned. The reviewers themselves are never reviewed. Instead of granting authority to the wisest and most experienced and visionary NIH program officials, who could then single-handedly pick the most promising and unorthodox ideas to fund because they have seen the entire landscape of cancer research and could be neutral arbiters, the NIH, perhaps for fear of nepotism, abnegate the power to steer research directions (except at the very high level through program announcements). Decision making is handed down to the poorly vetted peer reviewers whose opinions the NIH officials slavishly follow. The pope would pale of envy if he saw the infallibility that NIH officials project into the peer-review panels.
3. IS THERE MORALITY IN ANONYMITY?
The third mechanism that subverts meritocracy is a further down consequence of the afore-discussed anonymity of peer review system: Worse than lack of accountability and untested expertise, anonymity also opens a backdoor for an elementary human weakness. How much can a human being be trusted to value a foreign idea that contradicts her own as more worthy than her own, if one can trash it with great effect and impunity? And even less flattering for the human character: How many of us can, while being incognito, really “do unto others as you would have them do unto you”? Only the most idealistic, perhaps the saints among us, will possess the grace and selflessness to exert the necessary utmost due diligence in reading the abstract ideas that spring from a competitor’s mind, all written down in a complicated dense document. Who will display the self-denial required to achieve objectivity in proposal evaluation when such acts remain unseen and unpraised? Greatness of heart and mind does not come for free. The good that human do happens because subliminally, one expects reciprocity.
Protected by anonymity, a review panel member of average moral standard and intellect, who necessarily is in the majority, will hardly take the immense intellectual effort to comprehend a new idea that is beyond her intellectual grasp. It is extra hard work that is not rewarded. But lack of understanding always evokes a negative opinion. Would you vote for a candidate for presidential office if you do not understand her tax plan, let alone vision?
Moreover, an innovative idea will, by its nature, be at odds with existing orthodoxy and hence, will be feared if not fought, a tendency that human mind is heir to. Thus, an anonymous peer-review system is the most natural and effective apparatus for protecting orthodoxy. It is a mechanism that automatically filters out outside-the-box ideas –those in the realm of all the unthinkable concepts in which deep insights that could lead to a cure for cancer might lie. Yes, of course the peer review system prevents subpar, sloppy research from being funded (and the majority of outside-the-box ideas will be such — see Figure above) but it also chains research to mediocrity.
The extent of human ability to take a high road when no one is watching is poorly understood. Certainly, review panels do by far not live up to the ideal that motivated the Chair of the Royal Society when in 1832 he first proposed a referee system for publications, stating that reviewers are “men who are elevated by their character and reputation above the influence of personal feelings of rivalry or petty jealousy possess an authority sufficient to establish at once the full importance of a discovery, (…) and to define the probable effect upon future progress of science”.
4. THE STAYING POWER OF “GOOD” IDEAS
The fourth mechanism that prevents the best ideas from winning is not linked to human moral but more mundanely, pertains to psychology and pragmatism. Shakespeare taught us that good ideas must give place to better ones. If followed, this simple principle would drive scientific progress. But cognitive psychologists have found that good ideas actively suppress better ones.
For instance, using precisely tailored “target-selective drugs” (“smart drugs”) to block distinct cogs in the cell’s molecular machinery which are crucial for the cancer cell to survive is obviously a good idea. Immunotherapy of cancer is a good idea. But good ideas stifle progress: The notoriously expensive target-selective drugs extend patient life (in good cases) on average by just a few months over existing treatments, at a cost of $50,000 or more per year. The top targeted therapies generate enormous annual revenues of over billions of dollars for their manufacturers. If so, how can there be any incentive to do better? Despite the modest benefits for patients, the financial success stimulates many “me too” efforts in academia and industry: If minimal, risk-free efforts, such as chemically modifying a drug to create a new (patentable) version that works on the very same principle can give a company a small slice of a huge cake, why then innovate?
Proposals to study the biological principles behind the observation that the majority of patients do not respond to a given targeted drug despite a full mechanistic rationale that the drug should work, do not excite the NIH reviewers. Their habitual notion of novelty is not epitomized by new abstract biological principles but by novel material entities that obey the same principle: yet another cog in cancer cells’ machinery that can be blocked by a drug. Most peer-reviewers, drawn from the province of mediocrity, succumb to a narrow herd-mentality and see novelty only in a “variation on a theme” but are unable to enjoy new unfamiliar themes. They are blinded from seeing approaches outside-the-box of molecular targeting.
Cancer immunotherapy is also a “good”, even a great idea, holding robust promise for some cancer types in some patients. But the current rage about the “good” idea of immunotherapeutics also suppresses “better” ideas: an embrace of more encompassing categories of biological thinking that may help overcome the increasingly obvious inherent limitations of immunotherapy. Exceptional patient responses have so far only prompted a doubling-down, stay-the-course mentality, with the goal to replicate such success in more patients by more powerful stimulation of the immune system. It might work, but there also could be intrinsic limits — we don’t know but do not even ask.
5. THE PLAYING FIELD FOR COMPETITION IS TOO SMALL
Finally, meritocracy in cancer research is sidelined not only because the playing field is uneven but also because it is too narrow: There is a lack of opportunity for direct competition of skills and ideas of cancer researchers that are directly relevant to understanding the biology of cancer under the auspices of a fair arbiter. We simply do not put the very best and devoted at the front in this war on cancer. Competition takes place in the side arena of fund-raising, politics and PR. The arcane and complicated grant application process and the abysmally low rates of getting funded, along with its stochasticity, have created an intense competition circus that sucks up all the energy and oxygen from active researchers.
To partake, researcher have to develop a new type of skills. I estimate that the vast majority of researchers over the age of 45 spends 50–90% of their time in one way of another in applying for research funding or in activities that increase the odds of getting funded but are not related to enhance our knowledge of cancer. As said, this is essentially a full-time job — but not recognized as such. The more researchers realize that getting funded is a lottery, the more grant applications they submit, which in turn lowers the quality both of the proposals and of the overwhelmed review process, which further reduces meritocracy. This in turn invites more opportunistic, subpar researchers who need grants for their salary and academic promotion, to take a shot in this game. A self-propelling downward spiral.
Hardly a laboratory head enters her wet-lab anymore, let alone touches the benches. Most of the actual work of cancer research, the experiments and computational data analysis, is carried out by “first timers” –trainees who work with minimal supervision and minimal salary on tasks that they perform for the first time in life (and perhaps, for the last time because they move on or quit science). There is no continuity: no cultivation of technical skills, no accumulation of specific, detailed expertise. Work in biomedical research is almost exclusively an endeavor of learning on the job by inexperienced temporary workers on their way to medical school or in need of publications to find faculty positions. New hires often start almost from scratch in setting up routine experiments, thus reinventing many wheels. No wonder we find this notoriously high rate of irreproducible results in the publications. One recent estimate has it that 50% of the billions of dollar spent on pre-clinical research is wasted in non-reproducible research that produces potentially false results.
Cancer is complex and the war on cancer even more so. We extract the most experienced warriors in this war from the actual research at its frontline and put them into offices. Just fancy for a moment military officers at the frontline in an active war would be asked to spend up to 90% of their time to do paper and PR work to raise money to pay for weapon and ammunition that their units need to complete their mission. Those commanders that are more versed in such tasks are not necessarily better warriors, yet they are the most likely to ascend the ranks.
C O N C L U S I O N
All this reminds me of a U.S. Presidential candidate in this election who recently said in defense of some goofs in her campaign that she is not good at running for president but will be a good president. This is a fundamental dilemma that cancer researchers face: Spend more time and effort to improve the skills in gaming a complicated funding system, or learn about what really makes tumors tick and how to conquer them? Both in politics and science it now seems that the former is more effective for one’s own rise. A system that has become so competitive that it creates the demand for a new set of secondary skills that is different in nature from those required to accomplish its primary mission, defeats itself. It is almost shameful that in academia the art of “grantsmanship” has become a desirable skill. Research dollars go to the “wrong” talents, and the result is that some without great ideas swim in research money, and some with great ideas are stuck.
It is in this sense that the Cancer Moonshot program, despite criticism and ridicule, is aptly named: It captures the concept of true merit and the boldness of imagination that we have lost in the war on cancer. Landing on the moon owes its success to a climate of thought that not only must unconditionally submit to the principles of science but also had allowed for the embrace of unorthodox ideas, such as the use of the lunar orbiter rendez-vous, which was initially considered insane. But the real moonshot was easier –perhaps not simply because going to the moon was largely an engineering and managerial and less of a scientific challenge, but because there was no inertia of legacy from past attempts. We have been trying to cure cancer for 40 years but we had not been trying to fly to the moon for 40 years before we did it.
In the enduring war on cancer we have created a huge administrative apparatus that fosters many inapt leaders and has erected a rigid intellectual structure in which myriads of concepts exist, each with its ideological stakeholder defending it, thus preventing progress. But we simply do not know how to cure cancer –so we better learn how to deal with our collective ignorance, for science is, as Richard Feyman said, “the believe in the ignorance of experts”. This requires trust, courage and epistemic awareness. Instead, we fetishize anonymous peer-reviewers who are not even experts and clearly, “are not that into innovation”.
Going through the list of members of the Blue Ribbon panel that advises the vice president Biden in “mission control” one encounters an impressive list names, the crème-de-la-crème of cancer research. But this also means: same old names, same old ideas. Many of them by necessity epitomize the aforementioned Peter-types who have risen through the ranks of a cancer research system that is, well, not very meritocratic. Could this explain WHY there is no innovation in cancer research beyond technically obvious incrementalism? What if cancer is indeed not inherently incurable but our collective effort is inadequate because we do not know how to a manage a competition to identify the very best ideas?
The spirit of a moonshot at least stimulates imagination. Just imagine: What if we put true meritocracy back into cancer research? What if we make peer-reviewers accountable for their evaluation of proposals? What if NIH officials stop nominating scientifically-illiterate peer-reviewers to the review panels, but instead, recruit more visionary and epistemologically versed and dedicated scientists? What if the NIH program directors have the courage to free themselves from the worship of peer-review panels and instead, take the lead? What if we promote expertise in cancer biology and punish pure grantsmanship? What if …
Now is the moment to allow “good ideas to give place to better ones”.