The answer is 17 years, what is the question: understanding time lags in translational research
- 1Institute of Public Health, University of Cambridge, Cambridge CB2 0SR, UK
- 2RAND Europe, Cambridge CB4 1YG, UK
- Correspondence to: Jonathan Grant. Email: jgrant{at}rand.org
Abstract
This study aimed to review the literature describing and quantifying time lags in the health research translation process. Papers were included in the review if they quantified time lags in the development of health interventions. The study identified 23 papers. Few were comparable as different studies use different measures, of different things, at different time points. We concluded that the current state of knowledge of time lags is of limited use to those responsible for R&D and knowledge transfer who face difficulties in knowing what they should or can do to reduce time lags. This effectively ‘blindfolds’ investment decisions and risks wasting effort. The study concludes that understanding lags first requires agreeing models, definitions and measures, which can be applied in practice. A second task would be to develop a process by which to gather these data.
Introduction
Timely realization of the benefits of expensive medical research is an international concern attracting considerable policy effort around ‘translation’.1,2 Policy interventions to improve translation respond to a vast empirical literature on the difficulties of getting research across research phases and into practice.3–11
Both literature and policy tend to assume that speedy translation of research into practice is a good thing. Delays are seen as a waste of scarce resources and a sacrifice of potential patient benefit.12 Although some lag will be necessary to ensure the safety and efficacy of new interventions or advances, in essence we should aim to optimize lags. One recent study (of which JG and SW were co-authors) estimating the economic benefit of cardiovascular disease (CVD) research in the UK between 1975 and 2005, found an internal rate of return (IRR) of CVD research of 39%.13 In other words, a £1.00 investment in public/charitable CVD research produced a stream of benefits equivalent to earning £0.39 per year in perpetuity. Of this, 9% was attributable to the benefit from health improvements, which is the focus of this paper. (The remaining 30% arise from ‘spillovers’ benefiting the wider economy.) This level of benefit was calculated using an estimated lag of 17 years. Varying the lag time from 10 to 25 years produced rates of return of 13% and 6%, respectively, illustrating that shortening the lag between bench and bedside improves the overall benefit of cardiovascular research. What is notable is that all the above calculations depended upon an estimated time lag; estimated because, despite longstanding concerns about them,14 time lags in health research are little understood.
It is frequently stated that it takes an average of 17 years for research evidence to reach clinical practice.1,3,15 Balas and Bohen,16 Grant17 and Wratschko18 all estimated a time lag of 17 years measuring different points of the process. Such convergence around an ‘average’ time lag of 17 years hides complexities that are relevant to policy and practice which would benefit from greater understanding.13
Despite longstanding concerns about delays in getting research into practice, the literature on time lags seems surprisingly under-developed. To help address this gap, this paper aims to synthesize existing knowledge and to offer a conceptual model that can be used to standardize measurement and thus help to quantify lags in future. This would allow efforts to reduce lags to be focused on areas of particular concern or value, or on areas where interventions might be expected to have best effect. It would also provide the potential for evaluating the cost-effectiveness of translation interventions if their impact on lags can be measured. The aim was to overlay empirical lag data onto the conceptual model of translational research to provide an overview of estimated time lags and where they occur. The first part of the paper explores conceptual models of the translation pipeline in order to provide context. The second part of the paper presents a review of the literature on time lags to present current estimates and issues. This leads to a discussion on the current state of understanding about time lags and considers the implications for future practice and policy.
Methods
For the first part of the study we identified literature that described conceptual models of translation. Our search was not intended to be exhaustive, but included key policy documents and searches of Google Scholar, Web of Science, PubMed and EBSCO. Key words used to retrieve relevant studies included ‘valley of death’, ‘bench to bedside’, ‘translational research’ and ‘commercialisation’. In general, ‘grey’ literature was not included in the search, but the HERG study19 was included because of the authors' involvement in it. The models in the literature found by these methods were summarized into a simple conceptual model.
For the second part of the study we reviewed the literature on time lags in health research. We used the same methods and literature as for the first part but included additional search terms such as ‘time lag’ or ‘time-lag’, ‘delays’, ‘time factors’ (PubMed MESH term) and ‘publication bias’. We found a formal search yielded few relevant papers so combined a number of approaches to increase our confidence that relevant papers had been identified. We undertook backward and forward citation tracking to identify related work and used searches within targeted journals – e.g. Scientometrics and Journal of Translational Medicine. To analyse the lag data, we used a data extraction template with the following fields: start and end dates for measurement period, range, mean, median, dates used, topic, country of study. In addition, the start point and endpoint of the time lag measured in each study were mapped onto specific stages in the conceptual model developed in the first part of the study.
Findings
Conceptualising translational research
Understanding time lags requires a conceptual model of how research in science is converted to patient benefit so that the durations of activities and waits can be measured. This process of conversion of basic science to patient benefit is often called ‘translation’.1,2,18,20–22 Woolf has argued that ‘translation research means different things to different people’23 and this is reflected in the various models and definitions found in the literature. However, as translational research also ‘seems important to almost everyone’23 there would seem to be benefit in trying to unify models and definitions.
We have attempted to synthesize these models to identify key features of the translation process and to offer a tentative unified model. This was intended to help stakeholders agree a model which could be used to support future data gathering and better guide policy-making. We recognized that drug development, public health, devices and broader aspects of healthcare practice will vary in nature. The translation process is summarized briefly in Figure 1. Clearly this model can be critiqued for being linear and we acknowledge the considerable literature that challenges this notion and accept that research translation is a messy, iterative and complex process (see Balaconi et al. for a good review of the liner models critiques and their partial rebuttal24). At the same time, we would argue that for the purposes of understanding and conceptualising time lags the model is appropriate in showing common steps found in the literature.
A conceptual model of the journey of health (biomedical) research from research into benefit, as derived from the literature
‘Translational research’ is typically separated into two phases of research. Type 1 translation, also somewhat confusingly called ‘bench to bedside’, refers to the conversion of knowledge from basic science research into a potential clinical product for testing on human subjects. Type 2 translation, ‘research into practice’, tends to refer to the process of converting promising interventions in clinical research into healthcare practice (thus is closer to the notion of the ‘bedside’).2,20,21,25,26 Each phase of translational research is associated with a set of research activities which contribute to lags.27 These include processes around grant awards, ethical approvals, publication, phase I, II, III trials, approvals for drugs, post-marketing testing, guideline preparation and so forth. Some of these activities are repeated in different phases – grants and publications most particularly. Each activity involves a lag, either because the effort required for carrying out the task or as a result of non-value adding waits. The activities are used as ‘markers’ in studies of lags.
Conceptual models typically include ‘translational gaps’, which describe the movement from one phase of research to another. Each of these is also associated with delays, although precisely what and where these gaps are, and how long they are, is again not consistent in the literature. Policy measures to expedite the translation process typically focus on these gaps.
Estimating time lags in the translation process
Table 1 shows a summary of estimates derived from empirical studies of lags.
Summary of studies of time lags in health research
Figure 2 shows these time-lag estimates by research phase. Some additional ‘averaging’ has been necessary to provide single figures where ranges only were used in the original paper. The source data are presented in Appendix A (see http://jrsm.rsmjournals.com/content/104/12/510/suppl/DC1).
Chart showing the approximate range and average time lag reported in studies of time lags in health research. NB – HERG is the Health Economics Research Group at Brunel University
Issues with measurement and estimation
As is shown in Table 1, studies of time lags in translation of research to practice often measure different points in the process. For example, Decullier et al.28 and Stern and Simes29 measure time between ethical approval and date of first publication; Grant et al.30 and HERG et al.13 look at publication to guideline; DiMasi calculated the length of time within and between phases in US drug development to calculate the costs associated with the phases.31 Sternitzke looked at commercialization of pharmaceutical innovations from ‘chemical synthesis’ to FDA approval.32 Ioannidis attempted to estimate the time lag between date of trial registration and several milestones to publication.33 Grant et al., Mansfield and Comroe and Dripps work backwards from practice to publication.17,34–36
Not surprisingly given they are measuring different lags, Figure 2 helps show that data are generally sparse and estimates vary.37 Some studies report longer lags for publication to guideline17,38 than others do for development to commercialization.18,32,35 Table 1 also points to two substantive gaps in knowledge: the time lag involved in and between discovery and development (T1), and the time lag between publication to practice. Only one study has ‘implementation’ into practice as its endpoint.
Measurement and reporting is often poor. For example, Decullier et al. report ‘mean’ lags,28 and Dwan, in reporting Decullier et al., in their review refer to ‘median’ lags.36 Neither reports distributions. Ranges – or even interquartile ranges as large as 221 years38 – are seldom reported. Furthermore, where it was possible, further investigation of the average revealed wide variation; variation which is not highlighted or discussed in the papers. For example, Hopewell et al. in their review of publication bias conclude that clinical trials with null or negative results ‘on average’ took ‘just over a year longer to be published than those with positive results’.39 This average is associated with a range of six to eight years for studies with a negative or null result, compared with four to five years for those with positive results. Comparing the slowest negative publication with the fastest positive publication makes a potential difference of four years – half of the maximum lag.
Some studies aggregate data from earlier studies without critical reflection or recognition of this.16,25 For example, Balas and Bohen calculate an average of 17 years from original research to practice formed from adding together a number of single studies of different phases including one that estimates a lag of 6–13 years.16 Accounting for this changes their estimate of the time lag between journal submission to use in practice from between 17 years to 23 years.
Not surprisingly, studies also show variation in time lags by domain38 and even intervention within a single domain. For example, examining research relating to advances in neonatal care, Grant et al. traced research papers back through four ‘generations’ of publication. They found ‘the overall time between generations 1 to 4 ranges from 13 years (for artificial surfactant) to 21 years (for parenteral nutrition). The other three advances took 17 years to develop through four generations of citations’. Atman et al.'s study of treatment for myocardial infarction yielded similar results: it took six years for a review of evidence supporting the use of thrombolytic drugs to result in a standard recommendation, whereas prophylactic lidocaine was used widely in practice for 25 years based on no evidence of effectiveness.40
Content also appears to influence time lags. A common theme found in the literature concerns publication bias, and their implications for judging effectiveness.28,29,33,37–39,41–47 Altman looked at citations of new statistical techniques applied to health and found that it took 4–6 years for a paper to receive 25 citations if the technique was new. An ‘expository article’ could achieve 500 citations over the same period.48 Contopoulos-Ioannidis found different publication trajectories for different types of invention.38
Studies also show that time lags are not stable over time. For example, Pulido noted a difference of 0.9 years or 0.3 years from acceptance to publication in 1992 and 1982, respectively.49 DiMasi reported a slight shortening of the approval process between 1991 and 2003.31 Tsuiji and Tsutani reported reduced lags in the drug approval process in Japan following a change in policy to try and expedite it.50
Single papers raise issues that are not generally discussed but do seem relevant to measuring time lags from publications in particular. These issues include ‘generations’ of research19 and overlaps in research publications.19,33 For example, Ducullier, of the 649 studies they included, five years later 59% had published research findings but most (84%) had more than one paper from the same study.28
Discussion
This paper aimed to synthesize existing knowledge to offer a conceptual model that can be used to standardize measurement and thus help to quantify lags in future. The strengths of the study are that, to our knowledge, this provides the first attempt to review lags comprehensively, both in terms of using multiple approaches to find studies, but also in attempting to quantify time lags along the translation continuum. The review exposed a number of weaknesses in the literature and gaps in knowledge, which are not often discussed. Despite our attempts to be comprehensive, however, we are aware that studies of time lags in health research are widely distributed and not easily identified using formal literature searches and we may have failed to capture relevant studies. We struggled to find research quantifying lags in basic research and the first translation gap in particular.
Our aim to understand lags has been limited by the weaknesses of existing data. Limitations of the literature examined include the use of proxy measures. Much of the literature on lags focuses on dissemination and publication in peer-review journals in particular as these are the most measureable. If there are significant lags in, say, the grant or ethics process, this is less likely to be reflected in current total lag estimations. Moreover, the variation in choice of proxy measures means that studies are almost never measuring the same thing, making valid aggregation and generalization difficult.
There is a clear trend in the literature to seek a single answer to a single question through the calculation of an average. The variation found in the literature suggests that this is not possible (or even desirable), and variation matters. Moreover, many of the published ‘averages’ are derived from adding an empirically derived mean duration for one section of journey from one point in time, in one topic, and adding it to other parts without reflection. Thus any poor estimates are transferred forward into later analysis, and also hide a complexity which is highly relevant to research policy.
There also appears to be a mismatch between conceptual models of the translation process, and the measuring of lags. For example, the gap between guideline publication and translation into actual practice is often ignored, suggesting an under-estimation of the time lags in some cases. On the other hand, interventions may come into use before guidelines outlining them have been published – suggesting an over-estimation of time lags in other cases.
Using different endpoints, different domains and different approaches, Balas and Bohen16 and Grant et al.30 both estimate the time lag in health research being 17 years. Wratschko also suggested 17 years as the highest limit for the time taken from drug discovery to commercialization.18 It is surprising that 17 is the answer to several related but differing questions. Is this coincidence or not? One possible reason for the convergence is the difficulty of measuring longer lags – because of limitations of citation indexes, other records and recollections – which provides a ceiling to such estimates and leads to a convergence of average lags.
While not able to adequately quantify time lags in health research, this study provides lessons for future research policy and practice. Concerns about lags are not new14 but are unresolved. Based on the review, and our own work on lags,13,17,19,30,51 we would argue that an essential step to being able to quantify time lags, and thereby make improvements, requires stakeholders to agree definitions, key stages and measures. It also perhaps requires stakeholders to develop a more nuanced understanding of when time lags are good or bad, linked to policy choices around ethics and governance for example,52 or reflect workforce issues.52,53 Indeed, a recent paper by Trochmin et al.54 proposes a ‘process maker model’ whereby they identify a set of operational and measureable markers along a generalized pathway like that illustrated in Figure 1. It seems to us that this provides an excellent framework to support future data gathering and analysis and thus provide a more informed base from which to develop policy to address time lags.
Currently much of the complexity, and therefore the potential for improvement, are hidden in this preference for ‘averages’. No attention is given to understanding distributions and variations. This effectively ‘blindfolds’ investment decisions and risks wasting efforts to reduce lags. As noted in the introduction, some lags are necessary to ensure the safety and efficacy of implementing new research into practice. The discussion in the literature fails to consider what is necessary or desirable, tending to assume that all lags are unwelcome. A key question for policy is to identify which lags are beneficial and which are unnecessary, but to answer this question it is necessary to have an accurate and comparable estimate of the lags.
Conclusion
Translating scientific discoveries into patient benefit more quickly is a policy priority of many health research systems. Despite their policy salience, little is known about time lags and how they should be managed. This lack of knowledge puts those responsible for enabling translational research at a disadvantage. An ambitious reason for being able to accurately measure lags is that it would be possible to look at their distribution to identify research that is both slow and fast in its translation. Further investigation of the characteristics of research at both ends of a distribution could help identify actionable policy interventions that could speed up the translation process, where appropriate, and thus increase the return on research investment.
DECLARATIONS
Competing interests
None declared
Funding
This is an independent paper funded by the Policy Research Programme in the Department of Health. The views expressed are not necessarily those of the Department
Ethical approval
Not applicable
Guarantor
JG
Contributorship
ZSM designed, conducted and analysed the literature review, and drafted and revised the paper; JG initiated the project, drafted and revised the paper, and has led a number of studies cited that attempted to measure lags; SW revised the paper
Acknowledgements
This paper derives from work undertaken by RAND Europe within Centre for Policy Research in Science and Medicine (PRISM), funded by the UK Department of Health. The authors thank NIHR for their support
- © 2011 SAGE Publications
This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Other Related Journals