The risks of converting post-hoc findings into primary outcomes in subsequent trials

Huntington’s disease (HD) is a devastating neurodegenerative condition caused by a triplet repeat expansion of the huntingtin gene (1). Although rare it is amongst the most frequent autosomal dominant causes of dementia, frequently affecting individuals in the most productive decades of their lives. Clinically, it is characterized by a classic triad of fluctuating neuropsychiatric symptoms, and progressive movement and cognitive disorders, accompanied by other symptoms such as weight loss and sleep impairment. It is severely debilitating, has a huge impact on quality of life and is fatal, with a median survival after motor onset of around a quarter of a century (2).

The culprit genetic defect was first assigned to chromosome 4 in 1983 (3) and sequenced in 1993 (4). Whilst great hope had been placed on this discovery, nearly three decades later there is still no cure or any intervention that delays or stops disease progression (5). Nevertheless, several promising compounds are currently active in the drug development pipeline (6).

With regards to symptom management the therapeutic armamentarium is broad, although few interventions are supported by high quality evidence (7). As an example, for the hyperkinetic movement disorder (i.e., chorea) characteristic of HD, there are currently two FDA-approved medications—tetrabenazine and deutetrabenazine—although unclarity still exists about how dissimilar their efficacy and safety profiles are (8,9).

HD is a promising disease model to study neurodegeneration, due to several characteristics, including: a precise pathogenic agent (4), a natural history that comprises a long presymptomatic phase (10,11) followed by an extended diseased survival (12), a broad symptomatic spectrum including most of the clinical features present on neurodegenerative diseases (motor, behavioural and cognitive) (1), well-validated assessment tools (13-16), pioneering imaging and biofluid biomarkers (10,17), and well-organized research and patient networks (18). These factors have encouraged drug developers to invest in HD as shown by the proportionately high number of clinical trials conducted in this rare disease population. Unfortunately, the success rate of the development pipeline underperforms when comparing with other disorders (5).

Genetic interventions are only now coming to age due to recent breakthroughs in DNA and RNA manipulation techniques, and optimization of drug stability, immunogenicity and delivery. There is optimism that these new tools may help change the fate of diseases like Huntington’s. The ideal drug development program needs to happen with minimal human and financial burden and maximal efficiency. This includes informative preclinical data and early phase trial results, but also early “go/no go” decision timings and criteria.

Pridopidine is an interesting molecule from the pharmacological perspective (Figure 1). Acting on the dopaminergic system—which is primarily involved in the genesis of the disease phenotype—it is classified as a modulator as it can have both an agonistic and antagonistic effects through a state-dependent effect on dopamine receptors. When there is overactivity it acts as an antagonist, under underactivity conditions as an agonist, and has little to no influence under physiological circumstances. In addition, it also seems to interact with N-Methyl-D-aspartate (NMDA) and sigma-1 receptors.

Figure 1 Pridopidine molecule.

Thus far pridopidine has been tested in four successive randomized controlled trials (RCTs) in HD, enrolling a grand total of over 1,000 participants (Table 1) (19,20,22,24). Unfortunately, in all of them the primary outcome was not met.

Table 1 Clinical trials’ characteristics
Full table

At first, a small RCT tested 50 mg of pridopidine a day against placebo in 58 people with manifest HD, recruited from 6 centres in Sweden and Norway. The trial followed participants for 4 weeks and was not able to show an effect on its primary outcome, a composite cognitive score. Nevertheless, secondary and exploratory analyses revealed a nominal improvement in the modified motor score (mMS), especially in a subgroup of more severely affected participants. This subscore of the Unified Huntington’s Disease Rating Scale (UHDRS) total motor score (TMS) comprises items related with voluntary motor control (i.e., items 4–10 and 13–15) (19).

Building up on these results, two follow-up randomized controlled studies ensued: the MermaiHD study in 32 European centres and the HART study in 27 North American centres. Based on the assumption that pridopidine could have an effect on voluntary movement control, both were designed and powered to show a difference in the mMS and recruited people with manifest HD and a mMS of 10 or more.

The MermaiHD study recruited 437 participants to investigate 45 and 90 mg of pridopidine daily compared with placebo over 26 weeks. Alas, this trial did not show an effect for its primary and secondary outcomes. Nonetheless, the analyses were statistically significant for the comparison 90 mg versus placebo in the per-protocol sample (i.e., 70% or greater compliance with treatment and completed the study) for the mMS and in the intention-to-treat sample for the UHDRS TMS (20).

The HART study had a similar design and tested 3 dosages of pridopidine (20, 45 or 90 mg daily) against placebo in 227 participants. The primary outcome evaluated at 12 weeks depicted no differences between the different dosages and the placebo arm, however secondary analyses showed a significant difference for UHDRS TMS for the comparison 90 mg versus placebo (22).

Lastly, the PRIDE-HD study recruited 408 participants with manifest HD, at least 25 points on the UHDRS TMS and 90% or less in the UHDRS independence score (IS), from 53 sites across 12 countries in Europe, North America and Australia. Follow-up was 53 weeks, and four doses of pridopidine (45, 67.5, 90 and 112.5 mg daily) were tested against placebo. The primary outcome was changed in the UHDRS TMS at 26 weeks and failed to be achieved. Exploratory analyses for all tested dosages showed similar results at 52 weeks. Exploratory analyses revealed an effect on the UHDRS total functional capacity (TFC) in the 45 mg arm. Subgroup post-hoc tests found this effect to be more evident in participants in earlier disease stages (24).

To summarise, the clinical development pipeline of pridopidine began with a negative, relatively small and short-lasting trial aimed at cognition but with interesting findings on a voluntary movements’ secondary outcome. It was followed by two well-powered but also negative trials designed to investigate the effects on voluntary movements. Both showed differences in a semi-structured neurological exam scale at the highest tested dosage (90 mg daily). A forth well-powered trial was deployed to investigate the effects on motor signs across a range of dosages. The trial was also negative, and none of the dosages shaped compelling differences after 6 months and 1 year on motor signs, but exploratory investigations disclosed an effect on functional capacity with low-dose pridopidine in early disease.

The cumulative evidence from these four trials seems to support that pridopidine is relatively safe and well-tolerated. The a priori hypothesis that it has an effect on the cognitive features of HD has been proven false. The findings on motor effects learnt from the earliest trials were not replicated in a large built-for-purpose trial. Were the first ones spurious positive results stemming from exploratory analyses, or were the results of the latter trials just unfortunate? The investigators blame an unexpectedly high response by the placebo group in the primary outcome. Indeed by the end of the study period, this was the arm with the largest effect size. The sponsor, Teva Pharmaceutical Industries Ltd., focused their attention on exploratory and post-hoc subgroup analyses concluding that “pridopidine demonstrates slowing of progression of Huntington disease in PRIDE-HD study as measured by Total Functional Capacity”. After sparked criticism from the clinical and scientific community, aware of the potential damages caused by such interpretations, the European Huntington’s Disease Network toned down the sponsor’s statement explaining that “this should not be misunderstood as a demonstration of disease modification or of neuroprotection”.

While there are multiple possible justifications for these results, one should always consider the possibility that pridopidine, as any other compound in development, may not induce the hypothesised clinical effect. The central nervous system therapeutic area has a low success rate from first-in-man to registration of around 7–8% comparing with the grand mean of 11%, and other therapeutic areas such as cardiovascular where the success rate is around 20% (25). In neurology, the drug development pipeline performs appreciably poorly in the phase III and registration phases (25). In HD only 2 molecules survived these phases, and overall the success rate is even lower than that of other therapeutic areas (5). Many reasons have been hypothesized to explain such phenomenon: incomplete understanding of the disease physiopathology; weak association between the therapeutic target, and the disease pathogenesis and natural history; limited animal models; incomplete evaluation of preclinical effects; suboptimal pharmacokinetic and pharmacodynamic characteristics; limitations of current study designs (no biomarkers, short study durations); unenthusiastic commercial interests, among others.

Pridopidine is an example where several of these factors came into play, halving the already low chances of first-in-man to registration success. It is well stablished that only a small proportion of science generates positive results, including rigorous and well-report clinical trials (26). While the scientific milieu should reward progress, industry and researchers are often motivated by other factors. As tempting as it may sound to regard these after-the-event announcements relevant, history has thought us that hypotheses and new trials generated by post-hoc positive results are legitimate but may be wrong.

Understandably and in the absence of better approaches, this strategy is frequently used across medicine. Pridopidine seems unlikely to be clinically helpful for people with HD but we hope this story will teach us about the design of drug development programmes and what to avoid in the future in order to deliver efficacious medicines and optimise drug development.

Acknowledgments

None.

Footnote

Conflicts of Interest: The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

References

1. McColgan P, Tabrizi SJ. Huntington's disease: a clinical review. Eur J Neurol 2018;25:24-34. [Crossref] [PubMed]
2. Rodrigues FB, Abreu D, Damásio J, et al. Survival, mortality, causes and places of death in a European Huntington's disease prospective cohort. Mov Disord Clin Pract 2017;4:737-42. [Crossref] [PubMed]
3. Gusella JF, Wexler NS, Conneally PM, et al. A polymorphic DNA marker genetically linked to Huntington's disease. Nature 1983;306:234-8. [Crossref] [PubMed]
4. A novel gene containing a trinucleotide repeat that is expanded and unstable on Huntington's disease chromosomes. The Huntington's Disease Collaborative Research Group. Cell 1993;72:971-83. [Crossref] [PubMed]
5. Travessa AM, Rodrigues FB, Mestre TA, et al. Fifteen years of clinical trials in Huntington's disease: a very low clinical drug development success rate. J Huntingtons Dis 2017;6:157-63. [Crossref] [PubMed]
6. Rodrigues FB, Ferreira JJ, Wild EJ. Huntington's disease clinical trials corner: June 2019. J Huntingtons Dis 2019;8:363-71. [Crossref] [PubMed]
7. Bachoud-Lévi AC, Ferreira J, Massart R, et al. International guidelines for the treatment of Huntington's disease. Front Neurol 2019;10:710. [Crossref] [PubMed]
8. Rodrigues FB, Duarte GS, Costa J, et al. Tetrabenazine versus deutetrabenazine for Huntington's disease: twins or distant cousins? Mov Disord Clin Pract 2017;4:582-5. [Crossref] [PubMed]
9. Rodrigues FB, Duarte GS, Costa J, et al. Meta-research metrics matter: letter regarding article "indirect tolerability comparison of deutetrabenazine and tetrabenazine for Huntington disease". J Clin Mov Disord 2017;4:19. [Crossref] [PubMed]
10. Tabrizi SJ, Scahill RI, Owen G, et al. Predictors of phenotypic progression and disease onset in premanifest and early-stage Huntington's disease in the TRACK-HD study: analysis of 36-month observational data. Lancet Neurol 2013;12:637-49. [Crossref] [PubMed]
11. Paulsen JS, Hayden M, Stout JC, et al. Preparing for preventive clinical trials: the Predict-HD study. Arch Neurol 2006;63:883-90. [Crossref] [PubMed]
12. Rodrigues FB, Ferreira JJ. Overall survival and causes of death in neurodegeneration-an overlooked and underreported theme. JAMA Neurol 2017;74:1379. [Crossref] [PubMed]
13. Mestre TA, Busse M, Davis AM, et al. Rating scales and performance-based measures for assessment of functional ability in Huntington's disease: critique and recommendations. Mov Disord Clin Pract 2018;5:361-72. [Crossref] [PubMed]
14. Mestre TA, Carlozzi NE, Ho AK, et al. Quality of life in Huntington's disease: critique and recommendations for measures assessing patient health-related quality of life and caregiver quality of life. Mov Disord 2018;33:742-9. [Crossref] [PubMed]
15. Mestre TA, Forjaz MJ, Mahlknecht P, et al. Rating scales for motor symptoms and signs in Huntington's disease: critique and recommendations. Mov Disord Clin Pract 2018;5:111-7. [Crossref] [PubMed]
16. Mestre TA, van Duijn E, Davis AM, et al. Rating scales for behavioral symptoms in Huntington's disease: critique and recommendations. Mov Disord 2016;31:1466-78. [Crossref] [PubMed]
17. Byrne LM, Rodrigues FB, Johnson EB, et al. Evaluation of mutant huntingtin and neurofilament proteins as potential markers in Huntington's disease. Sci Transl Med 2018. [Crossref] [PubMed]
18. Landwehrmeyer GB, Fitzer-Attas CJ, Giuliano JD, et al. Data analytics from enroll-HD, a global clinical research platform for Huntington's disease. Mov Disord Clin Pract 2016;4:212-24. [Crossref] [PubMed]
19. Lundin A, Dietrichs E, Haghighi S, et al. Efficacy and safety of the dopaminergic stabilizer Pridopidine (ACR16) in patients with Huntington's disease. Clin Neuropharmacol 2010;33:260-4. [Crossref] [PubMed]
20. de Yebenes JG, Landwehrmeyer B, Squitieri F, et al. Pridopidine for the treatment of motor function in patients with Huntington's disease (MermaiHD): a phase 3, randomised, double-blind, placebo-controlled trial. Lancet Neurol 2011;10:1049-57. [Crossref] [PubMed]
21. Squitieri F, Landwehrmeyer B, Reilmann R, et al. One-year safety and tolerability profile of pridopidine in patients with Huntington disease. Neurology 2013;80:1086-94. [Crossref] [PubMed]
22. Huntington Study Group HART Investigators. A randomized, double-blind, placebo-controlled trial of pridopidine in Huntington's disease. Mov Disord 2013;28:1407-15. [Crossref] [PubMed]
23. McGarry A, Kieburtz K, Abler V, et al. Safety and exploratory efficacy at 36 months in open-HART, an open-label extension study of pridopidine in Huntington's disease. J Huntingtons Dis 2017;6:189-99. [Crossref] [PubMed]
24. Reilmann R, McGarry A, Grachev ID, et al. Safety and efficacy of pridopidine in patients with Huntington's disease (PRIDE-HD): a phase 2, randomised, placebo-controlled, multicentre, dose-ranging study. Lancet Neurol 2019;18:165-76. [Crossref] [PubMed]
25. Kola I, Landis J. Can the pharmaceutical industry reduce attrition rates? Nat Rev Drug Discov 2004;3:711-5. [Crossref] [PubMed]
26. Munafò MR, Nosek BA, Bishop DVM, et al. A manifesto for reproducible science. Nature Human Behaviour 2017;1:0021.