Skip to main content

Extrapolation errors in Liu et al.’s CAM integrative review of health care professionals in New Zealand

Matters Arising to this article was published on 13 May 2024

The Original Article was published on 13 February 2021

Abstract

This letter is to highlight errors made by Liu et al. in their 2020 paper in BMC Complementary Medicine and Therapies, “Complementary and alternative medicine—practice, attitudes, and knowledge among healthcare professionals in New Zealand: an integrative review”. Substantial errors in their citation of the recent research and methodology by McDowell, Kohut & Betts (2019) pertaining to the practice of acupuncture in New Zealand by physiotherapists are presented. The actual results of McDowell et al.’s work and the true state of acupuncture use by their sample group is reported.

Peer Review reports

Background

This letter is to highlight errors made by Liu et al. in their 2020 paper in BMC Complementary Medicine and Therapies, “Complementary and alternative medicine—practice, attitudes, and knowledge among healthcare professionals in New Zealand: an integrative review” [1], when citing the paper “Safe acupuncture and dry needling during pregnancy: A survey of New Zealand physiotherapists’ practice”, published in 2019 in Integrative Medicine Research [2]. Extrapolation errors are evident in their abstract, results and discussion’s statistics. Furthermore, the methodology has been mis-cited, as has interpretation of postgraduate physiotherapy acupuncture training in New Zealand (NZ).

Extrapolation errors – statistical

In Liu et al.’s abstract on complementary and alternative medicine (CAM) they state:

“When treating pregnant women, 48.4% of physiotherapists practise acupuncture, and 37.3% of midwives recommend CAM” page 1 [1].

These statistics are repeated in their results section for prevalence of practice, use and referral for CAM:

“Findings indicated that around 25% of GPs practise CAM, and 82.3% refer patients to CAM practitioners. When treating pregnant women, 48.4% of physiotherapists practise acupuncture, and 37.3% of midwives recommend CAM” page 4 [1],

and again, in their discussion:

“Findings indicated that CAM use is common among New Zealand healthcare professionals: around 25% of GPs practise CAM, and 82.3% refer patients to CAM practitioners; when treating pregnant women, 48.4% of physiotherapists practise acupuncture, and 37.3% of midwives recommend CAM” page 7 [1].

More accurately McDowell et al. [2] reported that 60 physiotherapists (48.4% of the 124 total respondents) practise acupuncture during pregnancy. This represented 0.8% of the 7337 registered physiotherapists in NZ. Liu et al.’s citation would infer that 48.4% of all registered physiotherapists in NZ (3551/7337) practice acupuncture for pregnant women; a 60-fold overestimation. McDowell et al. very clearly reported the limitations of their study;

“The results of this purposive study may not be indicative of the opinion of all NZ physiotherapy acupuncturists and dry needlers” page 37 [2] ,

and Liu et al.’s interpretative flaws may lead to future publication errors if it is cited further.

Extrapolation error—Post graduate training

The following statement in Liu et al.’s paper is inaccurate and misleading.

“Findings indicated that approximately one quarter of GPs have received formal training in CAM, while 44.4% of physiotherapist received (postgraduate) training in acupuncture” page 6 [1].

More correctly 53% of physiotherapists (n = 32) of those who answered in the affirmative to McDowell et al.’s survey, (i.e., who would provide acupuncture during pregnancy), had undergone post-graduate training with a resulting university-based qualification. It is important to note that physiotherapists choose to study and practice acupuncture in NZ as a postgraduate competency. Yet again Liu et al. have committed a statistical error of extrapolating a very small survey finding to the entire NZ physiotherapy population.

Methodology citation error

In their paper Liu et al. also state:

“Instruments used in the eight surveys varied, without psychometric properties reported; four studies used instruments that were developed and adapted from previous studies”. page 4 [1].

McDowell et al.’s paper clearly describes the development of the electronic survey questionnaire, expert peer review, and piloting prior to the main survey. It was not adapted from a previous study as cited by Liu et al. above.

Discussion

The current state of acupuncture practice in NZ by physiotherapists

The practice of acupuncture is not currently regulated for physiotherapists in NZ, being considered “a modality within the practice of physiotherapy” [3]. Registered physiotherapists may practice within a defined field of practice as autonomous practitioners but must “complete or be currently undertaking, relevant, sufficient, and appropriate education and training and continuing professional development to maintain such knowledge and competence” [4]. NZ registered physiotherapists who have gained post graduate qualifications in acupuncture or have trained in dry needing are termed “physiotherapy acupuncturists” to identify as a separate profession to acupuncturists.

At the time of the survey 7337 physiotherapists were registered to practice in NZ [5]. Sixty nine percent of those were members of their national society, Physiotherapy New Zealand (PNZ) [6], and only 310 were members of PNZ’s special interest group the Physiotherapy Acupuncture Association of New Zealand (PAANZ). Further demographic information on the total number of NZ physiotherapists who have undertaken relevant education or who are currently practicing needling was not available.

Acupuncture has been taught and practiced by physiotherapists in NZ since 1992 [7,8,9]. However, the total number of physiotherapists utilising this complementary and alternative medicine in NZ is unknown. This has been a limiting factor for several research surveys to date [10, 11]. The paper “Safe acupuncture and dry needling during pregnancy: New Zealand physiotherapists’ opinion and practice”, was published in 2019 in Integrative Medicine Research [2]. It is important to emphasise that the paper’s results did not have access to the entire population of potential respondents. The Privacy Act of New Zealand [12] prohibits the direct use of Physiotherapy Board of New Zealand (PBNZ) membership lists to target the entire survey population. McDowell et al.’s survey recruitment was via invitation only by PNZ e-mail, with additional links available through PAANZ and PNZ webpages, closed acupuncture group Facebook pages and physiotherapy newsletters within NZ.

Whilst McDowell et al.’s primary aim was to gather basic demographics and examine the opinions, practice and level of understanding held by NZ physiotherapy acupuncturists providing acupuncture and dry needling for pregnant women, their results were never intended to be reflective of the entire population of physiotherapists in New Zealand.

Regrettably, this fact seems to have been lost by Liu et al. when extrapolating McDowell et al.’s data as verbatim statistics for the entire NZ physiotherapy population.

Survey response rates have been on the decline for the past decade in the field of health related research [13, 14], with response rates consistently lower than those of surveys of the general population [15]. Field et al. [16] reported response rates of between 13 and 39% are common in healthcare research. Cunningham et al. [13] found an overall response rate of 35% amongst physicians in Canada on a sensitive topic (medical billing practices) which was comparable to 39.6% response rate reported in the meta-analysis by Cook [17]. The McDowell et al.’s survey noted the existence of a low response rate (particularly from physiotherapy acupuncturists with only dry needling training), and a high number of missing responses within individual responses. This sample may not represent any definable population larger than NZ physiotherapy acupuncturists, a concept that appears to have been neglected by Liu et al.’s integrative review [1].

Further research

Research-based practice is central to improving the quality of health care by obtaining information about the knowledge, attitudes, practice patterns, the needs of practitioners [18] and to evaluate the impact of clinical research on practice [13]. Soliciting practitioner input is also essential when existing healthcare policies are being updated or to inform new policies [19].

Liu et al. have failed to recognize the limitations of McDowell et al.’s research sample, thus committing a statistical error of extrapolating a small survey finding to the entire NZ physiotherapy population. This raises concern that others may read their paper and cite it further (especially with the error prominent in the abstract), compounding errors in perceived practice if they are compared to the inflated NZ data. To date four papers have cited Liu et al.’s paper [20,21,22,23], fortunately without specifically referring to the erroneous sections. We ask that Lui et al.’s statements be corrected via publication of this correspondence, that is indexed and bidirectionally linked to the original article to prevent further and future errors.

Availability of data and materials

Not applicable.

References

  1. Liu L, Tang Y, Baxter GD, Yin H, Tumilty S. Complementary and alternative medicine - practice, attitudes, and knowledge among healthcare professionals in New Zealand: an integrative review. BMC Complement Med Ther. 2021;21(1):63.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  2. McDowell JM, Kohut SH, Betts D. Safe acupuncture and dry needling during pregnancy: New Zealand physiotherapists’ opinion and practice. J Integrative Med. 2019;17(1):30–7.

    Article  Google Scholar 

  3. Physiotherapy Board of New Zealand. Acupuncture, a modality within the practice of physiotherapy. In: Touch The Newsletter of the Physiotherapy Board of New Zealand. 2004:9.

  4. Physiotherapy Board of New Zealand. Physiotherapists practising in a defined field standard. Wellington: Physiotherapy Board of New Zealand; 2021.

    Google Scholar 

  5. Physiotherapy Board of New Zealand. Annual Report 2020/21. Wellington; 2021.

  6. Kirby S. PNZ Leadership day. Wellington. 2021.

  7. Kohut SH, Larmer PJ, Johnson G. Western acupuncture education for New Zealand physiotherapists. PTR. 2013;16(2):106–12.

    Article  Google Scholar 

  8. Scrymgeour J. Moving on. A History of the New Zealand Society of Physiotherapists Inc. 1973–1999. Wellington: New Zealand Society of Physiotherapists; 2000.

    Google Scholar 

  9. Rapson L, Ellis N, Turnball D, Madzokere H, Haker E, Carballo AM. Introduction of acupuncture into some countries. In: Hopwood V, Lovesey M, Mokone S, editors. Acupuncture and Related Techniques in Physical Therapy. New York: Churchill Livingstone; 1997. p. 163–73.

    Google Scholar 

  10. McDowell JM, Johnson GM. Acupuncture needling styles and reports of associated adverse reactions to acupuncture. Med Acupunct. 2014;26(5):271–8.

    Article  Google Scholar 

  11. McDowell JM, Johnson GM, Hale L. Adverse reactions to acupuncture: policy recommendations based on practitioner opinion in New Zealand. NZ J Physiother. 2013;41(3):94–101.

    Google Scholar 

  12. New Zealand Government. Privacy Act, (30 June 2020).

  13. Cunningham CT, Quan H, Hemmelgarn B, Noseworthy T, Beck CA, Dixon E, et al. Exploring physician specialist response rates to web-based surveys. BMC Med Res Methodol. 2015;15(1):32.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Burke M, Hodgins M. Is ‘Dear colleague’ enough? Improving response rates in surveys of healthcare professionals. Nurse Res. 2015;23(1):8–15.

    Article  PubMed  Google Scholar 

  15. Martins Y, Lederman RI, Lowenstein CL, Joffe S, Neville BA, Hastings BT, et al. Increasing response rates from physicians in oncology research: a structured literature review and data from a recent physician survey. Br J Cancer. 2012;106(6):1021–6.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  16. Field TS, Cadoret CA, Brown ML, Ford M, Greene SM, Hill D, et al. Surveying physicians: do components of the “Total Design Approach” to optimizing survey response rates apply to physicians? Med Care. 2002;40(7):596–605.

    Article  PubMed  Google Scholar 

  17. Cook C, Heath F, Thompson RL. A meta-analysis of response rates in web- or internet-based surveys. Educ Psychol Measur. 2016;60(6):821–36.

    Article  Google Scholar 

  18. Bonevski B, Magin P, Horton G, Foster M, Girgis A. Response rates in GP surveys - trialling two recruitment strategies. Aust Fam Physician. 2011;40(6):427–30.

    PubMed  Google Scholar 

  19. Cushman DM, Scholl LV, Ludlow M, Cunningham S, Teramoto M. Accuracy of rib palpation for dry needling of deep periscapular musculature, measured with ultrasound. J Bodyw Mov Ther. 2021;26:7–11.

    Article  PubMed  Google Scholar 

  20. Graca S, Betts D, Roberts K, Smith CA, Armour M. The changing clinical landscape in acupuncture for women’s health: a cross-sectional online survey in New Zealand and Australia. BMC Complement Med Ther. 2022;22(1):94.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Yingying P, Zang L, Wang X, Yang X. Effect of continuous care combined with constraint-induced movement therapy based on a continuing care health platform on MBI and FMA scores of acute stroke patients. J Healthc Eng. 2022;2022:5299969.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Yue T, Li J, Yang J, Fan D. Spinal manipulative therapy and acupuncture for chronic low back pain: a systematic review and meta-analysis of RCTs. Research Square. 2021;PREPRINT (Version 1). https://0-doi-org.brum.beds.ac.uk/10.21203/rs.3.rs-1192409/v1.

  23. Lesniewska M, Koziol I, Budzynska J, Kopystecka A, Patryn R. Opinions and experiences of physicians regarding complementary and alternative medicine (CAM). A research report from Poland. Research Square. 2022;PREPRINT (Version1). https://0-doi-org.brum.beds.ac.uk/10.21203/rs.3.rs-1138578/v1

Download references

Acknowledgements

Not applicable.

Funding

No funding was received for the original survey “Safe acupuncture and dry needling during pregnancy: New Zealand physiotherapists’ opinion and practice”. A prize to encourage participant completion of the survey was donated by AcuFuture, NZ.

Author information

Authors and Affiliations

Authors

Contributions

JM, SK and DB were the authors of the original paper. JM drafted the letter. All authors read and approved the final version.

Corresponding author

Correspondence to Jillian Marie McDowell.

Ethics declarations

Ethics approval and consent to participate

The survey referred to in this correspondence was undertaken with an electronic questionnaire. Ethics approval for this study was granted by the Auckland University of Technology Ethics Committee, application number 17/100. Participants were provided with an explanation of the study, consent details and researcher contact details.

Consent for publication

Not applicable (No identifying data).

Competing interests

JM and SK are executive members of, and tutors for the Physiotherapy Acupuncture Association of New Zealand (PAANZ). They are responsible for updating the PAANZ Guidelines for Safe Acupuncture practice biannually. SK is on the executive of International Acupuncture Association of Physical Therapists. DB is employed by the New Zealand School of Acupuncture and Traditional Chinese Medicine as a supervisor for a maternity acupuncture clinic at Hutt Valley Hospital, New Zealand.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

McDowell, J.M., Kohut, S.H. & Betts, D. Extrapolation errors in Liu et al.’s CAM integrative review of health care professionals in New Zealand. BMC Complement Med Ther 24, 187 (2024). https://0-doi-org.brum.beds.ac.uk/10.1186/s12906-023-04259-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://0-doi-org.brum.beds.ac.uk/10.1186/s12906-023-04259-3

Keywords