A Description of the Methodology Used in an Overview of Reviews to Evaluate Evidence on the Treatment, Harms, Diagnosis/Classification, Prognosis and Outcomes Used in the Management of Neck Pain

Types of Participants in the Reviews and CPG

Articles had to include populations that had any type of NP to be eligible. The anatomical boundaries of the neck are defined as commencing from the occiput (base of the skull) to the upper thoracic spine (T1-T6, mid upper back) and can include the upper regions of the torso or shoulder area. The upper shoulder region is included within the definition because muscles attached to the neck are also attached as far as the lateral ends of the scapulae. Some articles included populations with general musculoskeletal or chronic pain that could potentially include NP; from these studies, only those that reported stratified data for NP, or some primary studies with NP or that included some primary studies with greater than 50% of NP patients in the sample, were eligible. Studies were limited to those that included adult participants (18 years or older) who complained of NP.

All types of non-malignant NP were eligible including both non-specific (no specific identifiable etiology) and specific groups. Non-specific NP is considered to have a postural or mechanical basis and includes whiplash associated disorders (WAD I to II), myofascial neck pain, degenerative changes, and cervicogenic headache (corresponding to diagnostic classification 11.2.1 proposed by the International Headache Society classification and diagnostic criterion for headache disorders) [6] Different types of non-specific NP are subdivided on the basis of etiology into non-traumatic and traumatic (e.g. whiplash). Fracture, dislocation, neurological disorders and fibromyalgia associated with the neck were excluded from non-specific neck pain. Specific NP includes radicular pain (including WAD III), cervical disc prolapse pain, and facet joint pain.

Exclusions: Articles were excluded if they investigated NP with the following: a) trauma associated with fracture or head injury; b) definite or possible long tract neurological signs (e.g., myelopathies); c) NP caused by other pathological entities (e.g., tumours, infections); d) headache not of cervical origin but associated with the neck (e.g., migraine, tension-type headache).

Types of Interventions in the Reviews and CPG

For SR evaluating the efficacy of treatment or harms associated with the treatment, the interventions were limited to manual therapy, physical medicine modalities, drug therapies, patient education and communication, ergonomics, orthoses, prevention, and psychological interventions. For SR evaluating treatment and harms, only reviews that had searched at least two bibliographic databases and had undertaken some form of assessment of risk of bias of the included studies were eligible; this restriction was not applied to the other clinical areas (prognosis, diagnosis, or outcomes). This approach is consistent with methods for OvR to ensure only the best evidence is included in the findings and to minimize duplication [5]. Exclusions: treatment SR published prior to 2005 that searched a single database, or did not undertake a risk of bias assessment.

Diagnostic tests considered in the OvR included clinical examinations, radiologic tests (i.e., X-ray, mylography, electro-diagnostics, CT, MRI, provocation discography), specialized clinical tests (i.e., ROM, muscle endurance and strength, palpation tests, provocation tests for nerve tissue, functional tests, non-organic signs), injections, laboratory tests, blood tests; no citation was excluded based on the types of diagnostic tests. Articles that attempted to evaluate different systems of NP classification systems, consensus processes or models, definitions, terminologies, or frameworks aimed at categorization of NP diagnoses or disorders were also eligible. Exclusions: None for diagnostic tests or classification systems of the neck.

For reviews evaluating the evidence for prognostic factors, any prognostic factor or predictor (i.e., accident parameters, biological factors, psychological factors, behavioral factors, symptoms or interference factors, medico-legal context factors, other social and demographic factors) of outcomes were eligible; studies evaluating primary risk factors for the onset of neck-related problems were excluded. Articles were not restricted by the type of outcome associated with the prognostic factor in patients with NP. Where described, outcomes were grouped according to those types that could be included within the general International Classification of Function (ICF) domains of body structure and function, activity limitation, restricted participation, and environmental factors. Exclusions: None for prognostic factors; SR of risk factors for developing NP were excluded.

For reviews evaluating outcome measures, publications that evaluated at least one of the psychometric/clinimetric properties (reliability, validity, responsiveness, cross-cultural validation, floor-ceiling effect, interpretability) of self-reported or observed outcomes were eligible. There were no restrictions on the type of outcome measure evaluated within NP populations. Exclusions: None for type of outcome measure.

Types of Publications and Language Restrictions for Reviews and CPG

Eligible publications included systematic or narrative reviews [7], other OvR, CPG or consensus statements or commentaries associated with NP. Treatment, harms, and prognosis areas were limited to SR only; diagnosis and outcomes allowed narrative reviews and commentaries related to the area being evaluated. SR are characterized by comprehensive methods to identify and synthesize all the literature on a given topic. A publication was considered to be a narrative review if it labeled itself as a review but did not clearly identify the methods for selecting relevant studies or synthesizing the evidence. All CPG and consensus statements that included recommendations or algorithms for the management of NP were eligible. Note that CPGs are defined as systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances; they can be developed by local, regional, national or international groups or affiliated governmental organizations. Consensus statements are similar; but reflect a different methodology for deriving recommendations. Algorithms are variable in how they present guidance and may reflect recommended pathways for treatment or diagnosis.

Search Methods for Identification of Studies

A research librarian searched the following computerized bibliographic databases of the medical, chiropractic, and allied health literature, without language restrictions from January 2000 to March 2012: MEDLINE, EMBASE, CINAHL, ILC, and CENTRAL, and LILACS. References within eligible articles were screened for any additional references, as well as relevant publications from personal files of the investigative team. APPENDIX shows the search terms used for the OvR in MEDLINE.

Data Collection and Analysis

Qualitative Analysis of the Strength of Evidence

Currently, no methods exist to evaluate the strength of evidence (SOE) for findings across different SRs. Approaches to assess SOE, such as GRADE, are currently used with primary studies from a single systematic review [12-18]. In general, GRADE criteria used to judge the primary studies within a SR include the study design, risk of bias; imprecision (sample size); inconsistency (direction and magnitude of results); indirectness (applicability to patient populations with NP); and reporting bias.

As such, our team developed an approach using the GRADE but adapted for SR (rather than primary studies) evaluating the efficacy of treatments and diagnostic tests. Where available the rating of SOE within each review for specific interventions were extracted; when a SOE rating was not reported, an attempt was made by investigators to rate this evidence using the GRADE approach. If more than one SR was eligible for one type of intervention (e.g. three SR on exercise and NP)) and only one provided a rating of SOE using the GRADE approach, then that SOE reported was the one that was extracted and reported (for all three SR in this example). When SR reported SOE using different systems, these were compared to those SR that did use the GRADE approach; any similarities or differences between SOE ratings in these reviews were compared and a final judgement was made for the specific intervention. When there were two or more SR evaluating the same intervention and that provided GRADE SOE ratings, then all ratings were extracted and reported. Attempts were made to reach a consensus with respect to a final rating for the SOE for each intervention. A qualitative summary of the SOE using the same domains (risk of bias, inconsistency, imprecision, indirectness, and publication bias) is presented for evaluating the overall quality of OvR for prognostic or outcome measurement studies; currently no guidelines are provided for prognostic or outcome type studies using the GRADE system. Magnitude and consistency of synthesized results across different SR teams provided the final SOE statement in these areas. For the CPG, the proportion achieving greater than 50% in each of the six domains within the AGREE II is used as the threshold to characterize the overall quality of the guidelines.

Relative Distribution of Literature on NP

Fig. (1) shows that the majority of eligible articles were for the clinical area of treatment (n=117) and the smallest number for harms and prognosis (n=16 each). Diagnostic tests also yielded a relatively large number of articles of which the majority were SR (44 from 54). When considering the reasons for exclusion at full text, the majority of excluded articles were treatment reviews published prior to 2005 (n=103). Similarly, a large number of articles that evaluated chronic pain management for musculoskeletal disorders did not have sufficient (greater than 50% studies on NP) or stratified data for NP and were excluded (n=73). Fig. (1) shows other reasons for exclusion at full text. It was anticipated that some areas of NP management would not be well indexed or have few SR. As such, narrative reviews and commentaries were eligible for diagnosis/classification, prognosis and outcomes. Although a large number of articles for diagnosis and classification (n=109) were eligible given the intentionally broad criteria during screening, the greatest proportion of articles were narrative reviews or commentaries on classification of neck disorders (n=55). However, a preliminary analysis of the articles for classification revealed a disparate set of publications that were not primarily focused on summaries or evidence for disease taxonomy. There was often overlap with other clinical areas and very little data to extract for classification. As such, these 55 articles are now excluded from this OvR (see Fig. 1). Table 1 shows that the OvR of diagnosis and outcomes included a small proportion of narrative reviews in their final inclusion that provided summary information on a specific diagnostic test or outcome measures.

Risk of Bias

Figs. (2, 3) compare the criteria for AMSTAR across the areas of NP management. AMSTAR scores for each individual area of NP management are shown in web APPENDIX (Figs. 1-5). There were three criteria where generally all clinical areas were at high risk of bias (Fig. 2). A specific conflict of interest statement (AMSTAR Q11) was predominately absent (88% to 100% of SRs) in all areas of NP management.

Assessment of publication bias (AMSTAR Q10) was also absent in most areas of NP management (96% to 100% of reviews). Similarly, the majority of SR did not report the list of excluded studies (AMSTAR Q5) (61% to 94%) or used the publication status as an inclusion criteria (AMSTAR Q4) (43% to 92%). With the exception of treatment articles (56%), SR in the other clinical areas employed “duplicate” selection and data extraction of eligible studies (AMSTAR Q2) infrequently (25% to 46%).

In contrast, all areas of NP management scored well on a priori specification of the protocol (AMSTAR Q1) (88% to 100%) (Fig. 3). With the exception of the areas of prognosis (57%), all other areas generally undertook a comprehensive search (AMSTAR Q3) (80% to 100%). Generally, assessment of the risk of bias (AMSTAR Q7) and appropriate use of this in the formulation of conclusions (AMSTAR Q8) was achieved fairly well in the treatment and prognosis SR (89/85% to 93/86%); this was less consistently undertaken in the areas of diagnosis, outcomes, or harms (54% to 57%). Note that the AMSTAR does not assess the merits of the various methods used to assess risk of bias of the primary studies included in each SR. With the exception of diagnostic and outcome reviews (46% to 69%), the SR in the other three clinical areas adequately described the characteristics of included studies (AMSTAR Q6) (81% to 86%).

Meta-analyses were rarely undertaken in (AMSTAR Q9) the SR and as such most areas (68% to 100%) received a ‘not applicable’ rating for methods used to combine studies (even in SR of treatment). The reporting within the reviews was sufficiently ambiguous that it could not be determined if the studies undertook duplicate study selection and extraction (AMSTAR Q2) from 10% to 12% for treatment and harms and 20% to 23 % in the other areas.

Yield and Overlap Among the Areas of NP

Our yield of articles across the different clinical areas shows a significant gap with respect to areas of inquiry on the management of NP. This search of reviews showed a primary focus of all publications to be on treatment of neck pain and to a lesser extent for diagnostic tests; the numerous treatments and clinical tests may in part account for this. The majority of CPG also focused on treatment recommendations, with a lesser emphasis on diagnostic tests. In contrast, relatively few SR evaluated the potential for harms, prognostic factors, or the use or properties of outcome measures. Previous research [25, 26]. would suggest a different distribution of literature across the clinical areas, with studies on risk factors and treatment for neck pain being most frequent; the area most infrequently assessed in this previous OvR was articles assessing economic costs. This previous research included predominately primary studies rather than SR. Owing to our purpose to use the results of these OvRs to inform clinical practice in all areas of NP management, we were not concerned with SRs of primary risk factors for the onset of new neck pain, rather our focus was on management, measurement and prognosis (secondary risk) of existing neck pain. Harms, outcome measures, and classification studies are novel additions of the current series of OvRs, having not been evaluated in the previous OvR.

When considering duplication of articles across different bibliographic databases, it was surprising that the SR on outcome measures yielded the smallest proportion of overlap; this suggests that searching multiple databases are important for this clinical area of NP management. Additionally, there was a fair amount of overlap in articles across the different clinical areas. This may reflect a trend in assembling more comprehensive summaries addressing multiple areas, especially combined guidelines and best evidence syntheses.

The screening yielded a large number of articles for diagnosis and classification (n=109), and the greatest proportion were narrative reviews on classification of single or multiple NP disorders or syndromes (n=55). Our criteria for screening in classification-related articles were very broad and may have been a factor in this large number. However, a preliminary analysis of the articles for classification revealed a disparate set of publications that were not primarily focused on summaries or evidence for disease taxonomy. There was overlap with other clinical areas and very little data to extract for classification. This suggests that our search terms, which were limited to systematic and narrative reviews, were not sufficiently comprehensive. A review of primary studies may be required to adequately address the evidence for classification of NP.

Diagnostic Tests versus Outcome Measures

There were a number of methodological issues that ensued while undertaking this OvR. One issue that required some a posteriori consensus concerned the definition of an outcome measure versus a clinical or diagnostic test. A traditional conceptualization of a diagnostic or clinical test was one that would assist in confirming the presence or absence of disease. However, recent conceptualizations of clinical tests have been expanded to broaden this scope, such that any process that is used to inform or change patient management can be regarded as a clinical test [27]. Clinical tests can be used to monitor disease changes, assess prognosis, and screen for presence of disease. Parallel to this is more recent definitions of an “outcome measure”, which defines the variety of outcomes as ones that provide clinical “endpoints” (that may include morbidity, health or functional status, mortality, and even laboratory values) [5]. The distinction between a diagnostic or clinical test and an outcome measure appears to be less clear. This is also evident in clinical practice where a clinical test can be used several times for different purposes. As an illustrative example, one might consider range of motion testing, where this is typically used to assess current mobility status at initial assessment (diagnose movement problems), it can also be used to monitor the impact of treatment, as part of a final assessment to indicate the end of treatment (outcome), or as a prognostic factor following traumatic neck injury. Similarly, the Neck Disability Index can be used to assess severity of functional limitations and be used by a clinician to stratify patients into prognostic groups or to inform selection of treatment options (based on these groupings), but it can also be used to assess the outcome of the treatment. Thus, the same clinical test/outcome measure can be used for assessment (or diagnostic) purposes, as well as for prognosis, classification, or measurement of treatment outcome (outcome measure).

From a practical perspective distinguishing where the eligible reviews should be categorized with respect to placement within the diagnostic, prognostic or outcome groups in some cases proved challenging. One attempt was to distinguish the aim of the review by the types of the analyses undertaken, but this too proved problematic. Often the eligible citation would evaluate a clinical test using traditional accuracy performance measures (i.e., sensitivity and specificity) and also clinometric characteristics (reliability, validity, responsiveness). Our attempt to resolve this classification dilemma was to include the articles in several of the clinical areas in consultation with the expert panel, many of whom had representation on the included SRs. Nonetheless, this overlap may serve to inflate the relative proportion of literature in these three areas and report findings in duplicate across the different clinical areas.