Paper #1: Using clinically stable disease (NEDA 2) as a proxy for disease impact and employment in MS paints an incomplete picture.
Roman S.
J Neurol Neurosurg Psychiatry. 2020;91:5.
Paper #2: When Can Intermediate Outcomes Be Used as Surrogate Outcomes?
DeMets DL, Psaty BM, Fleming TR.
JAMA. 2020;323:1184-5.
Bottom Line: The purpose of a clinical trial is to determine if a particular intervention, (a drug or a procedure) is of benefit. The challenge in all trials is deciding what is a “benefit.” For some diseases defining a benefit is relatively easy. In cancer trials or trials for heart disease the benefit or “primary endpoint” can be prevention of death or disease recurrence. In a disease as complex as MS choosing a primary endpoint is more difficult. The obvious desired benefits would be to prevent disease progression and disability both for the short term and long term, to promote healing and to restore central nervous system function. Unfortunately, none of the usual primary endpoints used in recent clinical trials are able to predict such benefits. Many endpoints have been used (discussed below), each intending to define a result that predicts both short-term and long-term benefits. Recently a new primary endpoint has been suggested. The endpoint is called NEDA-3. NEDA-3 stands for “no evidence of disease activity” based on three criteria: stable history, stable neurologic exam, and no worsening of lesions on central nervous system MRIs. Achieving NEDA-3 may sound like a reasonable goal, but a critical question is does NEDA-3 truly identify all the effects of an intervention on the clinical status of a person with MS and does it predict long-term benefits? Additionally, if NEDA-3 is not achieved or does not persist, is that sufficient reason to consider the intervention ineffective? Put another way, is NEDA-3 a true surrogate or indicator that accurately reflects all major effects of an intervention on persons with MS both short term and long term? The above two papers cast doubt on these assumptions. Since currently used primary endpoints in MS trials do not predict all benefits and risks of an intervention, one should consider these endpoints “intermediate outcomes.” As explained in Paper #2, intermediate outcomes may correlate with short-term benefits, but may not predict long-term benefits. Indeed this has been shown in MS trials. What does all this mean for persons with MS? It suggests that prior to choosing treatment with a disease-modifying therapy, the person with MS should discuss in detail with his/her healthcare provider what the primary endpoints were for that particular agent, whether there was sufficient long-term follow-up to assess both safety and effectiveness, and whether the positive results of the trial were sufficiently robust and persuasive to accept the risks and adverse events that may result from taking the drug.
Key Points:
1. There are three phases to most clinical trials. A Phase I trial determines if a particular intervention (drug or procedure) is safe. Such trials are usually small and often involve disease-free individuals.
2. A Phase II trial is often a “proof of concept” trial. Can the proposed intervention show any sign of benefit in individuals with a particular illness? Such trials also are often relatively small, fairly short in duration (weeks to months) and may be performed at only one or two centers. Phase II clinical trials can have as primary endpoints a laboratory result (changes in blood sugar or blood pressure), changes on imaging studies (an absence or reduction of new central nervous system lesions), or measure a short term clinical benefit (e.g. shortened duration of relapse symptoms or a lengthened time between relapses).
3. If a Phase II trial shows “benefit,” as defined by the Phase II primary endpoint, a Phase III trial may be started. The primary endpoint of Phase III trials must be a clinical endpoint (for MS, a change in clinical course). This chosen endpoint must be met for the intervention to be considered of benefit.
4. Most trials also have secondary endpoints. These are other measures of presumed effectiveness, such as reduction in numbers of new MRI lesions, a slower rate of loss of brain tissue (atrophy), or changes in blood or spinal fluid levels of disease markers such neurofilament light chains.
5. Phase III trials usually involve large numbers of individuals, at multiple institutions, involving at least two groups of participants. One group receives the experimental intervention. The other group either is given a placebo or does not receive the experimental intervention.
6. For many disease, especially chronic diseases such MS, Phase III trials must be relatively long, often for years, to determine a change in clinical course. This requires multiple intervening assessments to monitor the safety of the drug or procedure, requiring not only face-to-face health provider contact, but also laboratory studies and imaging procedures.
7. Because of their complexity, Phase III trials are very expensive, require the recruitment of large numbers of individuals, and must be sufficiently long to detect changes in an illness with a highly variable course. In spite of this need, very few Phase III trials in MS are longer than two years in duration, making long-term effects uncertain. Given these factors, there has been a great effort to find an accurate primary endpoint that is a true indicator or surrogate for an intervention’s immediate and as well as long-term effects.
8. Most recent Phase III clinical trials in MS dealt with relapsing forms of multiple sclerosis. The most common primary endpoints in these trials have been changes in numbers of attacks or relapses per year (annualized relapse rate or ARR), changes in time between relapses, and changes in disability as measured by the Expanded Disability Status Score (EDSS).
9. Reducing numbers of relapses is certainly desirable, but is that an indicator for decreased long term disability? Does it result in less brain atrophy? Does it result in less cognitive impairment? Less fatigue? Less depression? In other words, are the benefits of taking a particular medication sufficient to warrant the possible long-term risks of infection and malignancy?
10. Secondary endpoints address some of these issues, but don’t have the impact of meeting the primary endpoint. The main thrust of a Phase III trial is to meet the primary endpoint and if not met, all the participants’ efforts and trial costs could be for naught.
11. Some trials undergo “post-hoc” or “after the fact” analyses, trying to find evidence of benefit even if the primary endpoint wasn’t met. Results of such post-hoc analyses should be viewed with caution since they involve manipulation of data not originally included in the trial, and thus subject to biases.
12. The outcomes of most clinical trials are analyzed mathematically. The intent is to look for “statistical significance.” In other words, did the results occur as a matter of chance or was the outcome highly unlikely to be happenstance?
13. Finding that an outcome is statistically significant is important, but there is a big difference between a statistically significant change in a primary endpoint and one that is clinically meaningful. Such is the case in some recent trials involving progressive forms of MS, where changes in accumulation of disability were significantly delayed in persons on treatment, but benefits were only noted in a subgroup of individuals and only to a relatively minor degree. Whether such small but significant benefits outweigh the potential side effects and risks of the treatment need to be discussed in detail.
14. In an effort to find a better surrogate for an intervention’s effects researchers have suggested NEDA-3 (“no evidence of new disease activity by history, on exam or on central nervous system MRIs”). Different variations of NEDA-3 are proposed. There is NEDA-4, which adds the assessment of brain shrinkage or atrophy, and NEDA-5, which adds testing for atrophy and cognitive impairment. These additions are important and address important clinical changes. Unfortunately, most studies in which NEDA-3 was an endpoint showed a decline in the ability to maintain this endpoint over time. Why?
15. Multiple pathologic processes occur in MS. In relapsing forms of multiple sclerosis there is a great deal of acute inflammation, with loss of the nerve cell’s myelin insulation (demyelination), loss of nerve cell fibers or axons, and irreversible formation of scar tissues or plaques. In about half of persons with relapsing forms of multiple sclerosis there is a gradual decline in both active inflammation and numbers of relapses, only to be followed by a slow, insidious loss of brain tissue with gradual progression of disability (secondary progressive MS).
16. Since there is this continuing change is the pathology and course of MS, is achieving NEDA-3, NEDA-4 or NEDA-5 an accurate predictor of benefit in preventing progressive disability and progression to secondary progressive MS? A large study followed individuals with relapsing forms of multiple sclerosis for 10 years. While 18% of these individuals had NEDA-3 after 2 years of study, by 10 years of follow-up these individuals had no benefit in terms of delayed disability as measured by EDSS. If anything, those with NEDA-3 at two years fared slightly worse 10 years later. Another smaller study with 7 years follow-up showed 46% of the study participants had NEDA-3 at year 2 but only about 8% had NEDA at year 7. Nevertheless, those with NEDA earlier in the study had less disability by year 7. Again, what does all this mean for persons with MS?
17. There is no question that early treatment of active relapsing forms of multiple sclerosis is of benefit in altering the course of disease and reducing the risk of long-term disability and progression to secondary progressive MS. It also appears that some disease-modifying therapies, those with more potent immune suppressive properties, are better in this regard, albeit with more risks.
18. However, all Phase III clinical trial results are based on responses of large groups of individuals, with large differences in responses between individuals. Thus, one cannot predict how an individual will respond to a particular disease-modifying therapy prior to starting a drug.
19. Since none of the endpoints currently used in Phase III clinical trials of relapsing forms of multiple sclerosis can fully predict their ability to achieve the ultimate long-term benefits of stopping disease progression, maximizing healing, with minimal long-term risks, choosing one should be based on the degree of change induced by the drug on primary and secondary endpoints, on the importance of these endpoints as markers that correlate with disease control, and on the short-term and long-term risks of taking the drug.
20. There are no “right” or “wrong” decisions in this regard. Knowing the details of a drug’s trial results and discussing benefits and risks of a drug with an MS-treatment experienced healthcare provider is the best way to make a choice that is acceptable to the person with MS and his/her family and to ensure long-term compliance with taking the drug.
Discussion: A challenge in any Phase III clinical trial is to define a clinical meaningful endpoint, one that is sufficiently beneficial to justify any side-effects of toxicities of the drug or procedure. As noted above, trials dealing with some diseases, such as cancer or heart disease, can have clear, non-controversial endpoints such as preventing death. The issues are much more complex in the case of MS. Ideally, the goal of an intervention in MS should result in a cure for all phases of the disease, both the relapsing phase and the progressive phase, with enhanced healing and no progression of any disability, both physical and emotional. Since that is not possible at this time, researchers have searched for primary endpoints they hope will strongly correlate with these optimal goals. For relapsing forms of multiple sclerosis, the main primary outcome in many trials was a reduction in numbers of annual relapses. This is clearly a desirable goal, but does achieving this result in less accumulation of long-term physical and cognitive disability? One study found no benefit. The same can be said for changes on central nervous system MRIs. Many trials have as secondary endpoints a reduction in numbers of new lesions or reductions in brain shrinkage or atrophy. Again, these are desirable goals, but how well do they correlate with prevention of disease progression and transformation to secondary progressive MS? With these issues in mind, researchers decided to use a composite of end point called NEDA-3, for “no evidence of disease activity.” Three parameters were monitored: no new attacks, no progression of disability, and no new lesions on central nervous system MRIs. During the early years of trials using NEDA-3 as their primary endpoint, more than a third of participants met criteria for NEDA-3. However, as the trials continued, fewer and fewer individuals met these criteria. Importantly, the effects of many MS disabilities were not addressed, such as fatigue, cognitive difficulties and changes on central nervous system MRIs not related to new lesions. In addition, since decreasing numbers of trial participants were able to maintain NEDA-3, did that mean the treatment was ineffective and shouldn’t be considered? What about long term benefits? Did achieving NEDA-3 initially define a population of individuals that had less long-term disability, and were some treatments better at achieving NEDA-3? Again, as noted above, results are conflicting. One large study with ten-year follow-up showed no, and possibly less, long term benefit from having initially achieved NEDA-3. Another study with 7-year follow-up did find that patients achieving NEDA-3 had less disability after 7 years. To complicate things further, a study with 16- and 21-year follow-ups after treatment with a high-dose interferon-beta found that initial or baseline findings of disability, relapse rate and lesion volume on brain MRIs were better predictors of long term disability than NEDA-3 at Year 21 but NEDA-3 was a good predictor of disability at Year 16. Researchers are trying to further expand the concept of NEDA to include measures of brain shrinkage or atrophy (NEDA-4) along with measures of cognitive function (NEDA-5). Results of trials using these outcomes are pending.
What does all this mean for persons with MS? Since there are no generally accepted true surrogates or predictors of long-term disability in MS, current primary endpoints can be considered “intermediate outcomes,” that is outcomes that strongly correlate with reductions in certain aspects of MS-related disability but may not define all aspects of it. Indeed, in a paper cited in Paper #2, correlation with a particular outcome may not necessarily predict it. As cautioned in Paper #2, “Correlation does not a surrogate make.”
There are several implications these observations for persons with MS. First, when choosing a disease-modifying therapy it is important to evaluate the nature of the drug trial’s primary endpoint and the strength of the effect on that endpoint. Second, one should consider how these changes compare to the control group. Was the control group an appropriate one? Did the control group receive an inactive placebo, a disease-modifying therapy with a similar mechanism of action, or a disease-modifying therapy with a totally different mechanism of action, making comparisons more difficult? Thirdly, one should assess the duration of the trial, (ideally at least two years) and the duration of any stated benefits during the trial. Was the degree of benefit noted throughout the trial, or did it gradually diminish? Finally, one should evaluate all data on potential short-term and long-term adverse events and toxicities. Are the described benefits of the drug sufficient to justify the risk of taking the drug? These are very difficult questions and much depends on an individual’s past history of treatment, on any other illnesses they may have, and on their willingness to accept known and unknown risks. Discussion with a health care provider experienced in the treatment of persons with MS is essential to allow one to help make the best decision for them.