Clinical Correlates of False Positive Assignment in Bipolar Screening Measures Across Psychiatric Diagnoses among Patients without Bipolar Disorder
Article information
Abstract
Objective
In this study, we aimed to determine clinical correlates of false positive assignment (FPA) on commonly used bipolar screening questionnaires.
Methods
A retrospective chart review was conducted to a total of 3885 psychiatric outpatients. After excluding patients who have bipolar spectrum illnesses, patients who were assigned as having hypomania on the mood disorder questionnaire (MDQ) or the hypomania checklist-32 (HCL-32) were identified as patients who had FPA. Psychiatric diagnoses and severity of emotional symptoms were compared between patients with and without FPA.
Results
Patients with FPA on the MDQ showed significant associations with presence of major depressive disorder, generalized anxiety disorder, and alcohol-use disorder, while patients with FPA on the HCL-32 showed associations with presence of panic disorder and agoraphobia. FPA on the MDQ was also associated with greater emotional symptoms and lifetime history of suicide attempts. Logistic regression analysis showed that male sex, younger age, presence of alcohol-use disorder, and severity of depression and obsessive-compulsive symptoms were significantly associated with FPA on the MDQ.
Conclusion
The FPA for the MDQ was associated with clinical factors linked to trait impulsivity, and the FPA for both the MDQ and the HCL-32 could be related to increased anxiety.
INTRODUCTION
Bipolar disorder (BD) is considered an under-recognized disease. Under-diagnosis can lead to delayed diagnosis and mistreatment, which eventually results in a worse clinical course and poorer treatment outcome. However, it is still difficult to differentiate BD from unipolar depressive disorder in clinical settings.
Diagnosis of BD depends on a patient’s history of hypomanic/manic episodes, indicating the importance of capturing hypomania through retrospective examination. To facilitate early recognition of bipolar disorder, several self-reported screening questionnaires have been widely applied in clinical settings. The Mood Disorder Questionnaire (MDQ) [1] and the Hypomania Symptom Checklist-32 (HCL-32) [2] are two of the most commonly used questionnaires. Their abilities to detect bipolarity initially appear comparable [3], but several studies showed that the HCL-32 may be better at detecting hypomania than the MDQ [4].
Although the MDQ and the HCL-32 are designed to be used for patients with major depression to detect (hypo)mania, they both are commonly used to define so-called bipolarity in diverse populations, both in clinical and research settings [5-9]. But this can lead to different problems related to overscreening. These screening measures generally have low positive predictive values, so additional interviews with trained professionals are necessary to confirm the presence of mania/hypomania [10]. To use these questionnaires properly, we need to identify the clinical correlates of false positive assignment (FPA) from these questionnaires in patients whose clinical diagnoses are not bipolar disorders.
A few studies have reported on the clinical correlates of FPA for bipolar screening questionnaires. Two studies [11,12] reported that FPA from the MDQ was associated with elevated rates of anxiety, impulse control, substance use, and attention-deficit disorder. However, clinical correlates of FPA from the HCL-32 have not been studied.
This study aimed to identify the clinical correlates of FPA for the MDQ and the HCL-32 in Korean psychiatric outpatients with diverse mood and anxiety disorders. Among patients who were diagnosed with non-bipolar mood and anxiety disorders via standard psychiatric interviews, we explored clinical features associated with false positive assignment on the two most commonly used bipolar screening questionnaires.
METHODS
Study populations
Data were obtained from a retrospective chart review of patients who visited the Depression Center of the Department of Psychiatry, Samsung Medical Center, between January 2011 and December 2017. After an initial interview with a boardcertified psychiatrist, patients between 18 and 65 years old who were suspected to have mood and anxiety problems were referred to a psychologist for a comprehensive psychological evaluation. The psychologist who conducted the evaluation had more than two years of clinical experience. The psychological evaluation was part of routine care and designed to confirm subject clinical diagnosis based on the DSM-IV-TR and to evaluate symptom severity. After the psychologist finalized patients’ psychiatric diagnoses, the result was reviewed and confirmed by a psychiatrist who order the test. Patients suspected of psychosis, intellectual disability, cognitive impairment, organic brain disease or who had difficulty in understanding overall evaluation process were excluded. Additionally, those who required an emergency admission were not referred for evaluation. This study used test results from comprehensive psychological evaluations.
The study protocol was approved by the Institutional Review Board of Samsung Medical Center (IRB no. 2018-11-019). All identifying data were removed from the clinical database prior to analyses. Because this study was a retrospective chart review, no consent was needed from participants.
Applied measures
Evaluation of FPA on the bipolar screening measures The Mood Disorder Questionnaire (MDQ) [1] and the Hypomania Symptom Checklist-32 (HCL-32) [2] were used to evaluate subthreshold hypomania. The MDQ evaluates experiences of (hypo)manic episodes using 13 questions derived from items of DSM-IV manic episode. The HCL-32 asks an individual their experiences when they were in a ‘high’ mood. Each question asks specific behavior, thoughts and emotions that can be experienced during (hypo)manic state. Furthermore, the HCL-32 evaluates whether the episode impact on their daily living. Widely used cut-off scores showed relatively low specificity for BD [10], so we adopted cut-off scores previously shown to have >80% specificity in Korean populations: the cut-off score for the MDQ was 9 [13], and the cut-off score for the HCL-32 was 20 [9]. We defined the MDQ (+) group as patients with an MDQ score ≥9 and the HCL (+) group as patients with HCL-32 score ≥20.
The psychological assessment
Subject diagnoses were evaluated using the Korean version of the Mini International Neuropsychiatric Interviews (MINI) [14]. If patients have dual diagnoses based on the MINI, they were counted twice in comparing frequencies of psychiatric diagnoses. Of the MINI modules, the suicide module was used to evaluate suicide risk. Suicidality was measured using the MINI suicide-item modules and the HAMD suicide-item scores. The MINI suicide module is a six-question, yes-or-no, interviewer-administered questionnaire used to evaluate suicidality and current risk of suicide. The MINI suicide module asks about experiences of recurrent thoughts of death, idea of self-harm, presence of suicide ideation, plans and (current and past) suicide attempts. A single score is generated after summing the weighted scores. Suicide risk scores are categorized as low risk (<6), medium risk (6–9), or high risk (≥10) for a future suicide attempt [15].
The Hamilton Depression Rating Scale (HAM-D) (Hamilton, 1967) and the Beck Depression Inventory II (BDI-II) were used to evaluate current depressive symptoms. The Beck’s hopelessness scale (BHS) [16] was used to evaluate severity of pessimism. The Hamilton Anxiety Rating Scale (HAM-A) [17] and the Beck Anxiety Inventory (BAI) [18] were used to evaluate anxiety symptoms. The Anxiety Sensitivity Index-3 (ASI-3) [19] was used to evaluate anxiety sensitivity. The Albany Panic and Phobia Questionnaire (APPQ) [20] was used to assess fear of situations and activities that are commonly avoided by individuals with agoraphobia and social anxiety disorder. The Penn State Worry Questionnaire (PSWQ) [21] was used to evaluate severity of worry. The Liebowitz social anxiety scale [22] was used to measure severity of social anxiety. The Obsessive Compulsive Inventory-Revised (OCI-R) [23] was applied to obsessive compulsive symptoms.
Statistical analyses
Chi-square tests were used to compare rates of sex and independent Student’s t-test was used to compare age between the MDQ (+) and (-) groups, and the same analyses were conducted between the HCL-32 (+) and (-) groups. Multivariate logistic regression analyses was conducted to determine the association between DSM diagnoses and FPA after adjusting for age and sex. Each DSM diagnosis was entered as a dependent variable, and MDQ (+) group and the HCL-32 (+) group memberships were entered as independent variables. Two-way analyses of covariance (ANCOVA) were conducted to compare emotional symptom severity depending on the FPA membership. MDQ (+) group and the HCL-32 (+) group memberships were entered as independent variables, and age and sex were entered as covariates. We also compared suicide risk using multivariate logistic regression analyses. We additionally conducted logistic regression analyses to determine factors that discriminated FPA on each screening measure. Only significant variables in aforementioned analyses were included as independent variables in the logistic regression models.
All analyses were performed with SPSS 24.0 (IBM Corp., Armonk, NY, USA). Considering the exploratory nature of this study, statistical significance was evaluated using a twosided design with alpha set a priori at 0.01.
RESULTS
A total of 4162 subjects underwent the standard evaluation. After excluding patients with BD [bipolar I, bipolar II and bipolar disorder, not otherwise specified (NOS)], a total of 3,885 subjects was included in the final analyses. Among them, 815 (21.0%) received an FPA on the MDQ score [MDQ (+) group], and 404 (10.4%) received an FPA on the HCL-32 score [HCL-32 (+) group].
Both the MDQ (+) and HCL-32 (+) groups were younger [MDQ (+) group mean age=38.43 (standard deviation (SD)= 12.28] years old, MDQ (-) group mean age=44.19 (SD=13.30) years old, t=8.84, p<0.001; HCL-32 (+) group mean age=37.77 (SD=12.31) years old, HCL-32 (-) group mean age=43.56 (SD= 13.50) years old, t=7.04, p<0.001) and included more males [MDQ (+) group number of male=402 (49.3%), MDQ (-) group number of male=1065 (34.7%), χ2=58.69, p<0.001; HCL-32 (+) group number of male=200 (49.5%), HCL-32 (-) group number of male=1267 (36.4%), χ2=26.42, p<0.001] compared to (-) groups.
Table 1 shows the comparisons of psychiatric diagnoses of patients included in the study. The MDQ (+) group showed significant associations with major depressive disorder (adjusted odds ratio (aOR)=1.34, 95% confidence interval (CI): 1.13–1.59), generalized anxiety disorder (aOR=1.73, 95% CI: 1.38–2.18), and alcohol-use disorder (aOR=2.00, 95% CI: 1.47–2.73). The HCL (+) group showed significant associations with panic disorder (aOR=1.45, 95% CI: 1.12–1.88) and agoraphobia (aOR=1.85, 95% CI: 1.35–2.55).
Psychiatric diagnoses of patients; comparison between patients with and without false positive assignment (FPA)
Table 2 shows the comparisons of emotional symptom severities between groups. We conducted 2-way ANCOVA analyses with the MDQ membership and the HCL-32 memberships as independent variables after adjusting for age and sex. No significant interaction effects between the MDQ and HCL32 groups were detected. The MDQ (+) group showed significant associations with higher scores except the HAMD. In particular, the effect sizes of the association was small (η2>0.01) in terms of the ASI-3, BDI-II, BAI, and OCIR scores. In contrast, the HCL-32 (+) group showed significant associations with the ASI-3 and PSWQ scores, but the effect sizes were almost negligible (η2<0.01).
Comparisons of emotional symptoms depending on false positive assignment on the mood disorder questionnaire (MDQ) and the hypomania checklist-32 (HCL-32)
We also compared suicide risk between the two groups (Table 3). After adjusting for age, sex, and presence of major depressive disorder, the MDQ (+) group showed significant associations with lifetime history of suicide attempt (aOR=1.70, 95% CI: 1.21–2.38). The HCL-32 (+) group did not show significant associations with suicide risk.
Comparisons of suicide risk evaluated using the MINI suicide module depending on false positive assignment on the mood disorder questionnaire (MDQ) and the hypomania checklist-32 (HCL-32)
We conducted logistic regression analyses to determine which variables contributed to the discrimination of FPA (Table 4). Variables shown significant associations with FPA were entered as independent variables. When the MDQ (+) group was entered as a dependent variable, the model χ2 was 312.49 (df=9, p<0.001). Using this model, 78.3% of subjects were classified correctly. Male sex, younger age, presence of alcohol-use disorder, and higher scores of BDI-II and OCIR were significantly associated with the MDQ (+) group. When HCL-32 (+) group was entered as a dependent variable, the model χ2 was 110.30 (df=8, p<0.001). Using this model, 89.6 % of subjects were classified correctly. But only male sex and younger age were significantly associated with the HCL-32 (+) group.
Logistic regression analyses to determine variables contributing to the discrimination of false positive assignment on the mood disorder questionnaire (MDQ) and the hypomania checklist-32 (HCL-32)
DISCUSSION
Although the MDQ and HCL-32 are widely used to define so called “bipolarity,” the FPA rate from these scales is substantial. The cut-off scores that are generally used for screening in clinical practice and in prior studies [3,10] are lower than those we used in this study. We adopted higher cut-off scores that yielded >80% specificity in previous studies [9,13]. Even with these cutoff scores, about 10–20% of our subjects had an FPA (MDQ: 21.0%, HCL-32: 10.4%). Previous studies argued that hidden or undiagnosed bipolar disorders are substantial among nonbipolar illnesses based on positive assignments from the questionnaires [24-26]. But, as observed from our study findings, a considerable number of subjects who were positively assigned based on the questionnaires may not have had actual mania or hypomania.
The FPAs observed in our study could be associated with subthreshold hypomania that only partially meets the DSM criteria. But in a prior study that compared HCL-32 scores and clinically significant subthreshold hypomanic symptoms that partially meet DSM criteria, the HCL-32 was not strongly correlated with subthreshold hypomanic symptoms [27]. In this regard, FPA from the questionnaires does not fully reflect true subthreshold hypomania based on DSM criteria.
Consistent with previous study findings conducted at academic hospitals [11,12], the MDQ (+) group showed significant associations with major depressive disorder, generalized anxiety disorder, and alcohol-use disorder. Additionally, the MDQ (+) group showed significant associations with severe depressive and anxiety symptoms. Cyprien et al. [28] reported that psychiatric comorbidity and symptom severity decreased the positive predictive value of MDQ in participants recruited at academic hospitals, which corroborates our study findings. Consistent with Cyprien et al., several studies reported that positive assignment from the MDQ was significantly associated with more severe forms of depression [29-31].
It is particularly notable that the MDQ (+) group showed associations with clinical factors that could be related to trait impulsivity, including alcohol-use disorder [32], obsessive compulsive symptoms (OCIR) [33], and lifetime suicide attempts [34]. Both bipolar disorder and unipolar depression had greater impulsivity compared with healthy controls [35], and impulsivity predicted increased risk for developing hypomania [36]. Prior study results reported that association between bipolarity and suicide risk [29,37] might be moderated by impulsivity.
Clinical correlates of FPA on the HCL-32 were different from those on the MDQ in our study. The HCL-32 (+) group did not show significant associations with variables related to trait impulsivity. Furthermore, the HCL-32 (+) group showed negligible associations with depressive and anxiety symptom severity. Since its development, the HCL-32 was intended to reflect the bright side of hypomania [2], and a meta-analysis suggested that the HCL-32 was better able to detect hypomania than the MDQ [4]. It is unclear if these well-known characteristics of the HCL-32 affect the differences in clinical FPA correlates for the HCL-32 and MDQ. Additional research is needed to examine clinical factors associated with FPA on the HCL-32.
One common factor associated with both the MDQ and the HCL-32 in our study was anxiety disorder. FPA was associated with increased anxiety. Although diagnosis and differentiation of unipolar depression and bipolar disorder have focused on mania and depression symptoms, anxiety is also tightly associated with BD. Anxiety disorder is very common both in unipolar depression [38] and bipolar disorder [39]. Anxiety is also a part of diagnostic criteria for mood disorder, and presence of anxiety is associated with poorer clinical courses. Recent epidemiologic studies showed that subjects who have manic episodes have an approximately equivalent risk of developing depressive episodes and anxiety disorders [40]. From this perspective, anxiety may not exist as a separate disease entity from mood disorder.
A prior study reported an association between FPA and anxiety disorder on the MDQ [11], and they suggested the association might reflect underlying emotional dysregulation. Emotional dysregulation is considered to be a core psychopathology that exists under diverse psychiatric conditions ranging from mood disorder to anxiety disorders [41]. A recent study reported that emotional instability showed a significant genetic overlap with bipolar disorder, unipolar depression, and anxiety [42]. Additional studies are needed to determine the biological correlates of positive assignment by bipolar screening questionnaires.
Another explanation of our study findings could be underlying bipolar spectrum that exists dimensionally. Classic definition of hypomania are too strict, and previous studies showed that softer hypomania that did not meet the full criteria of DSM were also clinically significant [43]. Prior studies using the bipolar screening questionnaires in defining hypomania also showed distinct neurobiological characteristics [5]. Further studies are needed to interpret true meaning of FPA observed in our study.
Our findings need to be interpreted within the context of our study design. First, this study used cross-sectional data. We relied on a one-time evaluation to determine DSM diagnoses and questionnaire responses. Second, we could not examine the effect of sub-factors often adopted in previous studies within the MDQ or HCL-32 [8,9], but several studies have confirmed that a single factor is enough to justify use of these measures in a clinical setting [44]. Third, we did not evaluate comorbid personality disorders that might have overlapping manifestations with those described in the MDQ or HCL-32 items. Last, we could not adjust patients’ education level which might impact on understanding the exact content of the questionnaire. We tried to exclude any patient who had difficulty in understanding the overall evaluation process.
Despite these limitations, this study explored the clinical correlates of FPA from commonly used bipolar screening questionnaires. The rate of FPAs on the MDQ and HCL-32 was substantial in patients without bipolar spectrum disorders. FPAs on the MDQ and the HCL-32 were both associated with anxiety disorder, which may link to underlying emotional dysregulation. The MDQ (+) group was associated with clinical conditions linked to trait impulsivity and greater symptom severity. The HCL-32 (+) group showed negligible association with depression and anxiety severity. Future studies using these questionnaires should be cautious in interpreting clinical meanings of the positive assignment.
Acknowledgements
This study was supported by a National Research Foundation of Korea (NRF) grant funded by the Korean government (MSIT) (No. 2019R1G1A1091679).
Notes
The authors have no potential conflicts of interest to disclose.
Author Contributions
Conceptualization: Ji Hyun Baek. Data curation: Ji Hyun Baek. Formal analysis: Ji Hyun Baek. Funding acquisition: Ji Hyun Baek. Investigation: Ji Hyun Baek. Methodology: Ji Hyun Baek, Ji Sun Kim. Supervision: Kyung Sue Hong, Andrew A. Nierenberg. Validation: Ji Hyun Baek. Visualization: Ji Hyun Baek. Writing—original draft: Ji Hyun Baek. Writing—review & editing: all authors.