Randomized controlled trials on acupuncture for migraine: research problems and coping strategies

Introduction

Acupuncture has been effectively used in clinical practice for the treatment of migraine. Specific clinical pathway and practical guideline were released by State Administration of Traditional Chinese Medicine of the People’s Republic of China (SATCM) and China Association of Acupuncture-Moxibustion (CAAM), which may promote the widely use of acupuncture treatment for migraine (1). To summarize the current clinical evidence, we did a systematic review and meta-analysis searching for the published randomized controlled trials (RCTs) evaluating the effect of acupuncture on migraine (2). In the previous works, 394 articles were initially identified and 62 eligible articles were eventually included. In the process of screening and quality assessment, we found that the quality of this kind of research is generally low which may obstruct the objective evaluation of the effect of acupuncture on migraine. The problems can be divided into two aspects: unstandardized trial design and inappropriate statistical analysis. This article intended to carry out an example analysis of these issues, and propose targeted solutions, hoping to provide reference for clinical researchers.

Methods

Inclusion and exclusion criteria

In previous searches, randomized control trials (RCTs) on acupuncture for migraine were included. We included participants with migraine and excluded patients who had migraine caused by other organic disorders. The intervention we focused on was acupuncture in the meta-analysis. But studies based on ‘microsystems’ theory were excluded, like eye-acupuncture and ear-acupuncture. For the control groups, studies including no treatment, sham acupuncture and medication were all included.

We searched seven databases from inception to Oct 19^th, 2017, including PubMed, the Cochrane Library, Web of Science, EMBASE, China Biology Medicine disc (CBM), China National Knowledge Infrastructure (CNKI) and the Wanfang Database. Details of the search strategies are available in the published papers.

Meta-analysis

Through reading the titles and abstracts, we initially identified the potential papers. These articles were obtained and the included studies were selected after reading the full papers. For the included papers, two reviewers independently extracted the data, which were next used to assess the risk of bias and perform meta-analysis. More details of the meta-analysis can be found in the published articles.

Literature arrangement

During the study, some shortages on acupuncture clinical researchers were found. We collected and synthesized the common problems and then summarized the main shortcomings.

Results and discussion

Problems and analysis

Trial design

The normative methodological design is of vital importance in the whole process of an acupuncture clinical trial. However, more than 70% of the studies we included in this area has certain deficiencies. The details are as follows:

Inclusion and exclusion criteria

Of the 62 articles included, 30.6% had some problems in inclusion and exclusion criteria: (I) 9 articles only provided diagnostic criteria instead of specific inclusion and exclusion criteria; (II) 5 articles took the opposite of inclusion criteria as exclusion criteria. For example, “The exclusion criteria are those that do not meet the inclusion criteria”; (III) 5 articles described the inclusion or exclusion criteria inadequately.

There’s no doubt that the inclusion and exclusion criteria should be clearly and sufficiently defined beforehand. The inclusion criteria are the general characteristics of the prospective subjects, such as meeting the diagnostic criteria of the disease and TCM syndrome; within the range of age, gender and race; having signed on informed consent, etc. Exclusion criteria are those characteristics that disqualify prospective subjects from inclusion in the study, which take more concern of the subject safety and may exclude the factors that interfere with the effect (3).

Control group

Of the 62 articles included, 66.1% took drug therapies as controls. However, the controlled treatment of Chinese patent medicines, traditional Chinese medicine decoctions, and combined therapeutic regimens are not the first-line recommended therapies in the guidelines, which have not yet been widely recognized, not to mention their actual efficacy. Therefore, in the selection of positive drugs as a control, we should try to choose the guideline recommended drugs, such as flunarizine hydrochloride, topiramate, triptans and so on (4).

In the studies with sham acupuncture as a control, 4.6% took shallow stab at the same acupoint; 40.9% took shallow stab in non-acupoint area; 54.5% took deep stab in non-acupoint area. It should be aware that shallow stab as a common acupuncture manipulation, its specific effects cannot be ignored. Therefore, it is still in controversial to take shallow stab at the same acupoint as placebo control (5). What’s more, it is hard to define non-acupoint area. Besides 361 meridian points and more than 2,200 extra-points, there are numerous ashi acupuncture points which are difficult to count and locate (6). Therefore, the best way of performing sham acupuncture control needs further study. In the studies of pain disorders like migraine, the sham acupuncture control group could select acupoints at the distal end of the non-identical nerve segment or local anatomy keeping away from the site of the pain source (7).

Random allocation

It was found in the screening of 394 related articles that 59 studies had obvious problems in random allocation: (I) 15 studies didn’t specify the exact randomization method. For example, “60 patients were randomly and equally divided into two groups.”; (II) 44 studies used pseudo-random allocation methods, such as random grouping based on date of birth, visit order, or ID number. Researchers should use the randomization methods correctly and report the method of random distribution sequence generation in the article.

Allocation concealment is the key to successful implementation of randomization, while only 18 studies used sealed opaque envelopes or central random methods to implement allocation concealment. None of the rest of studies mentioned allocation concealment. It was reported that studies that didn’t implement allocation concealment or that had insufficient allocation concealment tended to exaggerate the efficacy by 40% more than fully concealed ones (8). Therefore, there should be a clear division of labor in generating random assignment sequences, recruiting subjects, and allocating interventions. Doing the best to avoid the subjectively change of the subject’s enrollment or intervention received, due to the exposure to the randomized protocol.

Blinding

Of the 62 articles included, only 25.8% mentioned the use of evaluator-blind. 70.1% didn’t explain the specific process of blinding which cannot meet the Consolidated Standards of Reporting Trials (CONSORT) (9). Indeed, it is difficult to double-blind the operator and the subject in acupuncture RCT studies, due to the peculiarity of acupuncture and the general recognition of it in patients. Therefore, it is preferable to at least maintain the blindness of the outcome appraisers and reduce the measurement bias as much as possible to obtain an objective and true assessment.

Interventions

The Standards for Reporting Interventions in Clinical Trials of Acupuncture (STRICTA) (10) mentioned that research design should fully demonstrate the details of acupuncture intervention. But 46.8% of the included articles described it incompletely. The most often missing descriptions were the depth of inserting the needles, the method of lifting, thrusting and rotating the needles, intervals between single treatments and treatment courses as well as combined interventions. Such inattention to details may skew the result. For example, the retention time of needle is an important parameter for acupuncture treatment, but it is not the longer the better. The optimal needle retention time should be specified based on the purpose of the study with extensive references to relevant literatures (11).

Meanwhile, 27.4% of the interventions in the control group also have incomplete descriptions, such as the brand, manufacturer and specification of the drug, the depth and stimulus intensity of sham acupuncture. The investigators should fully consider the parameters of the acupuncture and control interventions during the trial design, as well as the operator’s medical background and practitioner experience. Try to make uniform request and report in the article.

Outcomes

In the included studies, the outcome indicators of migraine mainly include the efficiency, Visual Analogue Scale (VAS), Migraine Specific Quality of Life Questionnaire (MSQ), Self-rating Depression Scale (SDS), etc. Such indicators are self-rated by the patient and susceptible to subjectivity. Besides, 11.3% of the studies used only efficiency or VAS as a single outcome, making it more difficult to evaluate the efficacy objectively.

Therefore, according to the research purpose, the researcher can select 1–2 indicators with objectivity as the main outcome indicators, and define their definition and measurement method in advance, such as VAS. Indicators such as pain duration and seizure frequency can be selected as secondary outcome indicators, which would help to obtain a more comprehensive assessment.

Statistical analysis

Statistical analysis runs through the whole process of acupuncture clinical trials, from trial design to data analysis and results reporting. Problems found in the literature screening are as follows:

Sample size estimation

Estimating the appropriate sample size is critical for obtaining credible results. However, none of the included 62 articles provided the basis of calculation or pilot data for estimating the sample size. Only two articles enlarged the sample size considering the conditions of shedding, lost to follow-up, and withdrawal events. Most of the studies had a small sample size (sample size was positively skewed, with a median of 64.5 and an average of 80.39).

It is wise for researchers to select the appropriate sample size calculation method according to the purpose of the study and the data type of the main outcome indicators. The parameters determination can consort to the relevant documents, pilot data, and the clinical meaningful difference. After achieving the minimum number of evaluable subjects required for statistical significance, the drop-out rate should be considered to estimate an applicable sample size.

Statistical methods

Only 14 articles distinguished whether the data conformed to a normal distribution in statistical method section. One study used two independent sample t-tests instead of paired t-test to analyze the difference before and after treatment within the same group. 6.5% articles described the methods simply like “count data using ꭓ² test and measurement data suing t-test”. 4.8% articles didn’t specify the statistical method used, just claimed to use SPSS software for data processing. It reminded us that the description of statistical methods should be correct, and the data processing and analysis software used is supposed to be described.

Data report

Inaccurate data reporting will seriously affect the reliability of the results. In the 394 articles initially identified, 27 articles were excluded due to data reporting. The problems found are as follows: (I) small sample, non-normally distributed studies didn’t use medians and quartiles to describe their concentration and dispersion trends, but described the results with x±s instead; (II) the efficacy was divided into significant efficiency, efficiency and inefficiency, but the sum of the three was not 100%; (II) the data presented in text and in chart were inconsistent; (IV) some data were missing and the indication of a certain number was unclear. It reminded the researchers to maintain a rigorous scientific attitude, carefully analyze the data and present it faithfully.

Article structure and content

Of the 62 articles included, 35% articles had problems in structure and content. 4 articles presented methodological design and results together. Two articles directly took the result as a conclusion. More than 25% articles lacked of diagrams to demonstrate the participant flow. The researchers should make interpretation consistent with the results, provide information of registration and funding source. In addition, the combination of appropriate charts and necessary instructions can enhance the scientificity and readability of the article.

Coping strategies

Targeting the problems found in RCTs evaluating acupuncture treatment for migraine, our team put forward the following suggestions for the development of acupuncture RCT design.

Develop systemized top-level design

First of all, one should fully consider the research question and objectives to select the appropriate type of trial design. Crossover design, factorial design and other types of design can be selected according to actual needs. After selecting the type of research, the researcher should further clarify the scope of the research object, making specific inclusion and exclusion criteria. As for the control selection, medication, sham acupuncture, and other physical treatments are potential candidates. When it is difficult to achieve double blindness, the third-party blind assessment is highly recommended to enhance the reliability of the trial.

As for conducting random allocation, the researcher should use simple randomization, block randomization, stratified randomization or other appropriate randomization methods instead of pseudo-random allocation, and allocation concealment must be included. It is better to give interventions according to standard operating procedure, especially for manipulations like acupuncture. In addition, consulting to modern medicine knowledge could help to select local points more precisely, determine the direction and depth of acupuncture more accurately, and controlling needling sensation and comfortability (12). Finally, objective outcome indicators that can fully reflect the research objectives are preferred.

Conduct statistical analysis correctly

The primary cause of errors in statistical analysis is that researchers have insufficient knowledge of statistical methods. In this regard, researchers should systematically study statistical knowledge, and choose the appropriate statistical method according to the purpose of the study, the data types of variables. The consideration of statistical methods should be throughout the trial design and implementation process, not only in the analysis of results. Researchers may invite experts in the field of statistics to participate in trial design, implementation process, and data analysis. In addition, two-person independent analysis and cross-checking can be used to improve data accuracy.

Present the study clearly and precisely

Scientific research articles are the presentation of research results. They not only reflect the researchers’ ideas and study results, but also show their research attitudes. Writing should pay attention to: (I) the structure is clear, which should include research background, purpose, plan, results, discussions and conclusions. (II) The content should be properly detailed and concise. The research background should be refined and summarized, and the purpose of the research should be concise. The research plan should be concise and complete; the reference method should indicate the source; the results section can be fully displayed in conjunction with the necessary diagrams; and the discussion should not duplicate results but focus on analyzing the potential causes or related factors and exploring the disadvantages of researches. The analysis can be combined with disease and clinical practice; the conclusion should be concise. (III) Writing meticulously. Scientific research articles should avoid typos, missing data, and unclear labels.

Conclusions

The development of high-quality clinical research on acupuncture is an important way to objectively evaluate the therapeutic effect of acupuncture in traditional Chinese medicine and make it internationally accepted. It is also an important way to promote the development of acupuncture treatment standards (13). In recent years, some high-level clinical acupuncture studies with clear research objectives, rigorous design, and standardized statistical reporting have been carried out in China (14,15). However, most studies still have problems in experimental design and statistical analysis, which need all the researchers’ attention. In this regard, researchers should increase scientific awareness, improve trial design, and apply research methods appropriately. With the deepening of interdisciplinary cooperation, acupuncture clinical research should also actively expand research ideas and methods, enlarge their advantages, combine multi-disciplinary advantages to enable rapid and orderly development, promote steady improvement of evidence quality, and eventually serve the clinic.

Acknowledgements

Funding: This work was supported by the National Natural Science Foundation of China (grant number 81674050).

Footnote

Conflicts of Interest: The authors have no conflicts of interest to declare.

References

1. Jiao Y, Wu ZC, Hu J, et al. Interpretation of Evidence-based Guidelines of Clinical Practice with Acupuncture and Moxibustion: Migraine of the version 2014. Zhongguo Zhen Jiu 2016;36:751-6. [PubMed]
2. Jiang Y, Bai P, Chen H, et al. The Effect of Acupuncture on the Quality of Life in Patients with Migraine: A Systematic Review and Meta-analysis. Front Pharmacol 2018;9:1190. [Crossref] [PubMed]
3. Zhang Q, Meng P, Shan AL. Some opinion of inclusion and exclusion criteria in the drug clinical trial. Chin J Clin Pharmacol 2017;33:99-101.
4. Tfelt-Hansen P, Pascual J, Ramadan N, et al. Guidelines for controlled trials of drugs in migraine: third edition. A guide for investigators. Cephalalgia 2012;32:6-38.
5. Guo QL, Liu QG, Zhan H, et al. Application and consideration of placebo-acupuncture control in clinical research. J Clin Acupunct Moxibustion 2017;33:1-3.
6. Liu Y. A complete collection of extra-points in China. 1th ed. Shanghai: Shanghai Scientific and Technological Literature Press, 2001.
7. Mao WC, Liu BY, He LY, et al. Discussion on existing problems of placebo acupuncture design based on acupuncture analgesia. Zhen Ci Yan Jiu 2013;38:163-7. [PubMed]
8. Schulz KF, Altman DG, Moher D, et al. CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. BMJ 2010;340:698-702. [Crossref] [PubMed]
9. Zeng XT, Shen K, Luo J. Meta-analysis series XII: assigning hidden evaluations. Chin J Evid Based Cardiovasc Med 2013;5:219-21.
10. Macpherson H, White A, Cummings M, et al. Standards for reporting interventions in controlled trials of acupuncture: The STRICTA Recommendations. Acupunct Med 2002;20:22-5. [Crossref] [PubMed]
11. Han MJ, Zhao H, Jing XH, et al. Literature analysis of influence of different needle-retaining duration on acupuncture efficacy. J Tradit Chin Med 2017;58:334-9.
12. Zhou QC, Wang L, Hou XS, et al. Influence of modern medicine knowledge on diagnosis and treatment of acupuncture for chronic low back pain. J Tradit Chin Med 2017;58:833-5.
13. He P, Li LD, Li YK. Top ten problems in Chinese medicine. J Tradit Chin Med 2017;58:1808-10.
14. Liu Z, Liu Y, Xu H, et al. Effect of Electroacupuncture on Urinary Leakage Among Women With Stress Urinary Incontinence: A Randomized Clinical Trial. JAMA 2017;317:2493-501. [Crossref] [PubMed]
15. Zhao L, Chen J, Li Y, et al. The Long-term Effect of Acupuncture for Migraine Prophylaxis: A Randomized Clinical Trial. JAMA Intern Med 2017;177:508-15. [Crossref] [PubMed]