Cut-Off Scores of an Olfactory Function Test for Mild Cognitive Impairment and Dementia
Article information
Abstract
Objective
We aimed to find the optimal cut-off scores for screening of odor detection threshold, odor discrimination, and odor identification tests for detection of mild cognitive impairment (MCI) and dementia in Korean elderly.
Methods
A total of 195 elderly people were divided into three groups: the normal cognition (NC), MCI, and dementia groups. All participants underwent neurocognitive and olfactory function tests. We used k-means cluster analysis and receiver operating characteristic (ROC) analysis to identify the most appropriate cut-off value.
Results
To distinguish the MCI from NC groups, odor identification [area under the curve (AUC)=0.670, p<0.007] with a cut-off point of 7 showed greater validity for screening (sensitivity/specificity=0.462/0.837) than did other olfactory function tests. To distinguish the MCI and dementia from NC as well, odor identification (AUC=0.817, p=0.002) with a cut-off point of 7 showed the highest validity for screening (0.785/0.654). To distinguish MCI from AD, an odor detection threshold (AUC=0.722, p=0.001) with a cut-off point of 2 showed the highest validity for screening (0.785/0.654).
Conclusion
Olfactory function tests may be a useful screening tool for cognitive decline before clinical symptoms of dementia have completely developed. This tool can be used as a supplementary tool to enhance the sensitivity of traditional cognitive tests to screen for dementia.
INTRODUCTION
As the number of patients with dementia increases rapidly, the need for early detection and management of mild cognitive impairment (MCI) and Alzheimer’s dementia (AD) is being emphasized [1]. Early detection of MCI is also important because people with MCI are known to have a 6.7 times higher risk of developing AD than the normal cognitive function group [2]. MCI, defined as the transitional state between the normal aging process and dementia, can be diagnosed if there are cognitive complaints by patients or caregivers, objective cognitive impairment on neuropsychological examination less severe than dementia, and preserved independence in daily activities [3].
Previously, tests for cognitive dysfunction were mainly used as screening tools for MCI and AD [4,5]. The Mini-Mental State Examination (MMSE) is the most widely used screening method, and it shows good sensitivity (0.89) and specificity (0.89) for dementia [6] but lower sensitivity (0.63) and specificity (0.65) for MCI [7]. There has been controversy over the usefulness of cognitive screening for AD because physiological and pathological abnormalities occur 20–30 years before cognitive symptoms in AD [4,8,9]. Therefore, there has been active research on biomarkers to detect early AD before cognitive impairments. On the basis of AD neuropathology, which is characterized by neural loss, intracellular neurofibrillary tangle deposition composed of the tau protein, and amyloid-β (Aβ) plaques in the brain [10], biomarkers related to amyloid beta deposition and tau aggregation have been investigated. Biomarkers that reflect amyloid beta deposition include cerebrospinal fluid (CSF) Aβ 42, the Aβ 42/Aβ 40 ratio, and amyloid positron emission tomography (PET) [11]. Biomarkers reflecting tau aggregation include CSF phosphorylated tau and Tau PET [11]. Biomarkers of neurodegeneration or neuronal injury include CSF total tau, Fluorodeoxyglucose (FDG)-PET, and brain structural magnetic resonance imaging (MRI) [11]. However, CSF and blood biomarkers require invasive methods and brain imaging such as PET and MRI is expensive; thus, they are not suitable for screening.
Olfactory dysfunction has attracted attention as a new biomarker that can predict MCI and AD progression [12-14]. Olfactory function is divided into three odor categories: 1) odor detection threshold: the molecular concentration that can be detected by an individual; 2) discrimination: ability to distinguish one odor from other odors; and 3) identification: to associate odor molecules with related words or images [15-17]. All of these three olfactory functional domains are impaired in MCI and AD patients compared to healthy controls [16,18]. Moreover, olfactory dysfunction is known to appear prior to cognitive decline in AD [18]. The degree of olfactory impairment is greater in AD than in MCI [16,19]. Among these, the impairment in odor identification is more pronounced than in other olfactory domains in both MCI and AD patients [16,20,21].
According to a large longitudinal study, the elderly with normal cognitive function had a 2.18 times higher risk of progressing to MCI if they had abnormalities in odor identification [12]. In patients with MCI, the risk of developing AD was 5.20 times higher in patients with odor identification dysfunction [12]. This is thought to be due to the deposition of neurofibrillary tangles in the olfactory bulb and entorhinal cortex in the early stages of AD [22].
Several studies have evaluated the diagnostic validity of olfactory function tests as a screening tool for MCI and AD [16,23-27]. Most previous studies used the odor identification test to distinguish MCI or AD from the healthy control group. However, the cut-off scores of the odor identification test to screen for MCI and AD in Koreans are still unknown. The odor identification and discrimination test is difficult to generalize internationally because each region and culture has different familiar smells. Therefore, it is also necessary to evaluate the usefulness of olfactory function tests, specifically in Koreans. In addition, so far, few studies have evaluated the diagnostic usefulness of distinguishing cognitively healthy individuals, MCI, and AD using all the odor threshold, discrimination, and identification tests. Since olfactory functional domains may affect each other but are different, testing them together and comparing their usefulness for screening tool and their cutoff points for cognitive decline will be meaningful.
In this study, we aim to find optimal cut-off scores for the screening of MCI and dementia using odor detection threshold, discrimination, and identification tests, and evaluate the usefulness of the test in Korean elderly people. Based on the results of previous studies, we hypothesized that odor identification tests would be most useful to screen for MCI and dementia with the highest sensitivity and specificity.
METHODS
Study participants
This cross-sectional study was approved by the Institutional Review Board of Chung-Ang University Hospital (approval number: 1811-001-309). Written informed consent was obtained from all study participants and their caregivers. Study participants were recruited at the Department of Psychiatry of Chung-Ang University Hospital in Seoul, Korea, from August 2018 to August 2019. Through a hospital bulletin board, we recruited patients diagnosed with MCI or AD, patients who visited the hospital to check for cognitive dysfunction, and healthy individuals who wished to participate in this study. The inclusion criteria were as follows: 1) participants had to be aged between 50 and 90 years; and 2) they or their caregivers had to have understood the study objective and process and signed the informed consent form. The exclusion criteria were 1) past or current neurologic diseases (such as brain tumor, epilepsy, and Parkinson’s disease) or psychiatric disorders (such as major depressive disorder, bipolar disorder, and schizophrenia) other than MCI and dementia (major neurocognitive disorder) based on the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) [28]; 2) past or current diagnosis of dementia other than AD (such as vascular dementia, Parkinson’s disease dementia, and Lewy body dementia); 3) history of head injury, cerebral hemorrhage, or cerebral infarction; 4) current nasal and sinus disorders, asthma, history of obstructive nasal disease, or history of nasal-sinus surgery; 5) communication difficulty due to severe hearing loss or aphasia; and 6) participants or their caregivers being unable to understand the study objective and process.
A total of 224 individuals were screened for this study. Of them, two were excluded because they could not understand the study objectives and processes and five because of communication difficulty; four were excluded because of a history of obstructive nasal disease or nasal-sinus surgery, six because of neurologic or psychiatric diseases, and five because of a history of head injury or stroke. Therefore, 202 individuals were enrolled in this study. As seven participants did not complete the cognitive and olfactory function tests, the final data from 195 participants were statistically analyzed.
Measurement
YSK olfactory function test kit
To evaluate the olfactory function of the study participants, the YSK olfactory function test kit (RHICO Medical Co., Seoul, Korea) was used [29]. This is a newly developed olfactory function test that is an improvement over the Korean Version of the Sniffin’ Sticks test (KVSS). It consists of three subtests: odor detection threshold, odor discrimination, and odor identification. The odor stimuli are administered by a felt-tip pen filled with liquid odorants. To administer the odor, the examiner removes the cap and places the tip of the pen approximately 2 cm in front of both nostrils of the participant. We selected the YSK Olfactory Function Test Kit for this study because it contains odorants that are culturally well-known to Koreans.
The odor detection threshold is measured using 12 pens with phenyl-ethyl-alcohol (PEA) odorants diluted in 12 steps. The participants are asked to smell gradually from the lowest concentration to the highest concentration and answer whether they can smell the odorants. The score ranges from 0 to 12 points. The odor discrimination test is performed using triplets of pens, two of which have the same odor while the third has a different one. Participants are asked to pick the pen with a different odor. The tester performs 12 sets of tests and scores the number of correct answers. The total score ranges from 0 to 12 points. In the odor identification test, participants are given 12 types of odors, including both universal odorants (spearmint, baby powder, cinnamon, chocolate, medicated patch, ashes, peach, and naphthalene) and familiar Korean odorants (oriental medicine, grilled beef, Korean red ginseng, and burnt rice). Participants are asked to select the identified odor from four alternatives. The number of correct answers is recorded as the score, which ranges from 0 to 12 points.
Neurocognitive test
To evaluate the neurocognitive function of the participants, the Korean Version of the Consortium to Establish a Registry for Alzheimer’s Disease Assessment Packet (CERAD-K) was used [30]. It consists of subtests such as the Korean version of the Mini-Mental State Examination (MMSE-KC), verbal fluency, modified Boston naming, word list memory, word list recall, word list recognition, constructional praxis, constructional praxis recall, trail making test, and Stroop test. These tests can assess language, memory, visuospatial skills, attention, and executive function.
Diagnosis of MCI and AD was made according to the DSM5 [28]. Participants were divided into three groups based on a clinical interview by psychiatrists, and cognitive function was assessed by the CERAD-K: The Normal cognition (NC) group (n=104), the Mild cognitive impairment (MCI) group (n=26), and the Dementia group (n=65).
Statistical analysis
Differences in socio-demographic characteristics between the NC group, the MCI group, and the Dementia group were analyzed using a chi-square test and analysis of variance (ANOVA) with the Bonferroni correction for post hoc tests. Next, k-means cluster analyses were performed three times to segregate all the participants into either 1) the NC group or the Dementia group; 2) the NC group or the MCI group; and 3) the MCI group or the Dementia group. Receiver operating characteristic (ROC) curves were used to calculate the sensitivity, specificity, and area under the ROC curve (AUC) and determine the cut-off points for dividing. The cut-off point with the highest values of sensitivity+specificity was considered the best. For comparison with the screening sensitivity and specificity on using the olfactory function subtests, the sensitivity and specificity values of the MMSE-KC were also analyzed. All data analyses were carried out using SPSS ver. 19.0 (IBM Corp., Armonk, NY, USA).
RESULTS
Socio-demographic characteristics of study participants
The socio-demographic characteristics of the 195 study participants (age mean±SD: 73.05±8.10, 48 males and 147 females) are summarized in Table 1. Differences in socio-demographic characteristics between the NC group (n=104), MCI group (n=26), and Dementia group (n=65) are presented in Table 2. The MCI (74.96±9.58) and Dementia groups (78.28±7.07) were significantly older than the NC group (69.30±6.16; F=34.044, p<0.001). Regarding the level of education, significantly more participants in the Dementia group reported below middle school graduation, while more participants in the NC and MCI groups reported high school graduation or higher (χ2=9.108, p=0.011). The MMSE-KC scores were highest in the order of the NC group>MCI group>Dementia group (F=199.869, p<0.001). The CERAD-K scores also were highest in the order of the NC group>MCI group>Dementia group (F=333.650, p<0.001). The odor detection threshold scores (F=12.731, p<0.001) and odor discrimination scores (F=20.552, p<0.001) were lower in the Dementia group than the NC and MCI groups. The odor identification score (F=58.324, p<0.001) and the total score (F=53.769, p<0.001) were highest in the order NC group>MCI group>Dementia group. There were no significant differences in sex between the three groups.
Socio-demographic characteristics of the study population (N=195)
Differences in socio-demographic characteristics between the normal cognition group, the mild cognitive impairment group, and the dementia group (total N=195)
Results of cluster analysis and ROC curve analysis: NC group vs MCI group
Odor detection threshold
Cluster analysis and ROC curve analysis for odor detection threshold between the NC and MCI groups found was no significant cut-off score.
Odor discrimination
In the k-means cluster analysis of the NC and MCI groups (n=130), the highest (12) and lowest odor discrimination scores (0) were selected as initial seeds (centroids of respective groups; Table 3A). The final centroid and standard deviation (mean±SD) of the odor discrimination scores of NC and MCI were 4.75±1.29 and 8.28±1.39, respectively. The final Euclidean distance between the two groups was 3.526. After k-means cluster analysis, the 130 participants were classified into the NC group (72) and the MCI group (58). The cut-off odor discrimination score of the MCI group was set at 6 with an AUC of 0.624 at trend level (p=0.052) (Figure 1A, Table 4A). As shown in Table 5A, the sum of sensitivity and specificity values (sensitivity+ specificity) was the highest (1.221) for the odor discrimination score of 6.
Results of k-means cluster analysis of the study sample
Receiver operating characteristics (ROC) curves for the olfactory function subtest scores. A: NC group (N=104) vs. MCI group (N=26); area under the curve (AUC) for threshold=0.476, AUC for discrimination=0.624, and AUC for identification=0.670. B: MCI group (N=26) vs. Dementia group (N=63); AUC for threshold=0.722, AUC for discrimination=0.660, and AUC for identification=0.711. C: NC group (N=104) vs. MCI+Dementia group (N=91); AUC for threshold=0.642, AUC for discrimination=0.714, and AUC for identification=0.817. NC: normal cognition, MCI: mild cognitive impairment.
Area under the curve (AUC) for the olfactory function test scores
Sensitivity and specificity of the olfactory function test scores
Odor identification
In the k-means cluster analysis of the NC and MCI groups (n=130), the highest (12) and lowest odor identification scores (0) were selected as initial seeds (centroids of the respective groups) (Table 3A). The final centroid and standard deviation (mean±SD) of the odor identification scores of the NC and MCI were 4.28±2.03 and 10.94±1.29, respectively. The final Euclidean distance between the two groups was 6.665. After the k-means cluster analysis, the 130 participants were classified into the NC group (29) and the MCI group (101). The cutoff odor identification score of the MCI group was set at 6 with an AUC of 0.670 at trend level (p<0.007) (Table 4A). As shown in Table 5A, the sum of sensitivity and specificity values (sensitivity+ specificity) was the highest (1.298) for the odor identification score of 7.
Results of cluster analysis and ROC curve analysis: MCI group vs. Dementia group
Odor detection threshold
The results of the k-means cluster analysis of the MCI and Dementia groups (n=89) are presented in Table 3B. The cut-off odor detection threshold score of the Dementia group was set at 2 with an AUC of 0.722 (p=0.001) (Figure 1B, Table 4B). As shown in Table 5B, the sum of sensitivity and specificity values (sensitivity+specificity) was the highest (1.438) for the odor detection threshold score of 2.
Odor discrimination
The cut-off odor discrimination score of the Dementia group was set at 4 with an AUC of 0.660 (p=0.018) (Figure 1B, Table 4B). As shown in Table 5B, the sum of sensitivity and specificity values (sensitivity+specificity) was the highest (1.246) for the odor discrimination score of 4.
Odor identification
The cut-off odor identification score of the Dementia group was set at 7 with an AUC of 0.711 (p=0.002) (Figure 1B, Table 4B). As shown in Table 5B, the sum of sensitivity and specificity values (sensitivity+specificity) was the highest (1.369) for the odor identification score of 7.
Results of cluster analysis and ROC curve analysis: NC group vs. MCI+Dementia group
Odor detection threshold
The results of the k-means cluster analysis of the NC group (n=104) and the MCI+Dementia group (n=91) are presented in Table 3C. The cut-off odor detection threshold score of the MCI+Dementia group was set at 1 with an AUC of 0.642 (p=0.001) (Figure 1C, Table 4C). As shown in Table 5C, the sum of sensitivity and specificity values (sensitivity+specificity) was the highest (1.275) for the odor detection threshold score of 1.
Odor discrimination
The cut-off odor discrimination score of the MCI+Dementia group was set at 6 with an AUC of 0.714 (p<0.001) (Figure 1C, Table 4C). As shown in Table 5C, the sum of sensitivity and specificity values (sensitivity+specificity) was the highest (1.326) for the odor discrimination score of 6.
Odor identification
The cut-off odor identification score of the MCI+Dementia group was set at 7 with an AUC of 0.817 (p=0.002) (Figure 1C, Table 4C). As shown in Table 5C, the sum of sensitivity and specificity values (sensitivity+specificity) was the highest (1.562) for the odor identification score of 7.
Sensitivity and specificity values of the MMSE-KC
The sensitivity and specificity values of the MMSE-KC are presented in Table 6. The sum of sensitivity and specificity values (sensitivity+specificity) of the MMSE-KC was highest in the order MCI group vs. Dementia group (1.737)>NC group vs MCI+Dementia group (1.709)>NC group vs MCI group (1.337).
Sensitivity value and specificity value of the MMSE-KC
DISCUSSION
We found the optimal cut-off scores and evaluated the validity of the odor detection threshold, odor discrimination, and odor identification tests as screening tools for MCI and AD. To distinguish MCI from the normal control group, odor identification with a cut-off point of 7 showed the highest validity for screening. To distinguish MCI and dementia from the normal cognition group as well, odor identification with a cut-off point of 7 showed the highest validity for screening. To distinguish MCI from AD, an odor detection threshold with a cut-off point of 2 showed the highest validity for screening. The sum of sensitivity and specificity values (sensitivity+specificity) of the MMSE-KC was higher than those of the olfactory function test domains.
In this study, the odor identification score was significantly lower in the MCI group than in the NC group, whereas the discrimination and threshold scores were not significantly different between the MCI group and the NC group. This is consistent with the findings of previous studies characterizing the impaired odor identification of patients with MCI and prodromal AD [19,21]. AD is known to show a marked decline in odor identification, which requires higher-level cognitive processes [20]. Since odor identification demands more complex cognitive processes than odor discrimination and odor detection threshold, it is thought that a deficit in odor identification is already evident in the early stages of AD [19-21]. Odor identification can be achieved not only by sensory functions that detect and discriminate odors, but also through an interaction with cognitive functions that recall and relate to memories of appropriate words for the odors [31]. In previous studies, odor identification was associated with higher-order cognitive functions such as semantic memory, verbal memory, and executive functions, while the odor detection threshold was least dependent on cognitive function and was more associated with basic sensory perception processes [31-33].
In this study, the MCI and dementia groups were significantly older than the NC group; therefore, the effect of age on olfactory function should be considered as well. According to a previous study on normative data for olfactory function tests in 3,282 subjects, odor detection threshold, odor discrimination, and odor identification scores decreased with age [34]. The decrease in the odor detection threshold score in particular, was much greater compared to the discrimination and identification scores [34] In the pattern of olfactory decline seen in old age, the odor detection threshold was mainly impaired as an age-related change, whereas in the MCI and AD, odor identification was more impaired than other olfactory domains [19,21]. In this study, there was no difference in the odor detection threshold score between the MCI group and the NC group, but the odor identification score was significantly lower in the MCI group than in the NC group. Thus, we consider the odor identification dysfunction in the MCI group shown in this study to reflect changes due to AD pathology rather than age.
In this study, odor identification showed higher discriminant validity than other olfactory domains when discriminating the MCI group from the NC group, and when discriminating the MCI+Dementia group from the NC group. Odor identification has been consistently recommended as an appropriate test for early screening of AD [20,21]. Previous studies using the odor identification test also showed good sensitivity and specificity in distinguishing MCI or AD from healthy controls [16,23-26]. In this study, the odor identification scores showed consistent optimal cut-off scores of 7, regardless of NC vs. MCI, MCI vs. Dementia, or NC vs MCI+Dementia. This may mean that cut-off scores of 7 in the odor identification test can divide individuals into two groups, i.e., better cognitive function vs. worse cognitive function, regardless of the severity of the cognitive decline of individuals as a whole. The sum of the sensitivity and specificity value at the cut-off score of 7 was the largest when distinguishing the MCI+ Dementia group from the NC group. Therefore, it can be recommended for use to distinguish cognitive disorders including MCI and dementia from normal cognition in the elderly.
Interestingly, in distinguishing the MCI from the Dementia group, the odor detection threshold score showed higher screening validity than the odor identification score. This is inconsistent with the results of a meta-analysis showing a significant difference in odor identification compared to other olfactory domains between MCI and AD [19]. The reasons for the discrepancies in these findings may include the variety of procedures or materials used to measure olfactory functions in various studies in different countries, small sample sizes, and so on. Since the odor detection threshold test takes less time than other identification or discrimination tests, it has an advantage as a screening tool. In addition, since the distinguishing tests or cut-off scores are different between the groups, we can also increase the reliability through a combination of test scores.
In this study, the sensitivity+specificity values of MMSEKC were higher than those of olfactory function tests for all group comparisons: NC vs. MCI, MCI vs. Dementia, and NC vs. MCI+Dementia. However, the sensitivity of odor discrimination (cut-off point of 6) and identification (cut-off point of 7) in clustering NC vs. MCI, as well as the sensitivity of odor discrimination (cut-off point of 6) in clustering NC vs. MCI+Dementia, was higher than the sensitivity of the MMSE-KC. Dementia screening tools are mainly used for the general elderly population at primary care sites, and people who screen positive can be diagnosed for dementia by undergoing more expensive and precise confirmatory tests. Thus, for screening tests, it is important to use tests with good sensitivity so that people with dementia risk do not miss the opportunity to perform a diagnostic test. For confirmatory tests, good specificity is necessary to ensure that non-dementia people do not receive unnecessary medication and treatment. Therefore, the olfactory function test can be used as a supplementary tool to enhance the sensitivity of MMSE-KC, which is widely implemented.
This study has several limitations. First, as the number of MCI groups was small, this might have contributed to the low AUC, sensitivity, and specificity in distinguishing MCI from the control group. Second, the results can be difficult to generalize because the participants were all from Seoul, Korea, and most of them were women. Third, the NC group was significantly younger than the MCI and AD groups. Thus, there might have been an effect on olfactory function and cognitive function by age. Fourth, we have not collected information on other risk factors that may affect cognitive and olfactory functions, such as the ApoE gene, serum vitamin B12, serum folate, and severity of depression.
In conclusion, olfactory function tests are a useful screening tool for cognitive decline among elderly people. This tool can be used to screen MCI and dementia patients. Moreover, this tool is expected to screen cognitive decline before clinical symptoms of dementia have completely developed. In addition, the olfactory function test can be used as a supplementary tool to enhance the sensitivity of traditional cognitive tests to screen for dementia. Since existing olfactory function tests take a long time to use as screening tools, which is a potential disadvantage, follow-up research to develop a more abbreviated and faster olfactory function test is necessary.
Acknowledgements
This study was supported by a grant from the Biomedical Research Institute of the Chung-Ang University Hospital in 2018 and another grant from the Chung-Ang University Research Grants in 2019.
Notes
The authors have no potential conflicts of interest to disclose.
Author Contributions
Conceptualization: Sun Mi Kim, Doug Hyun Han. Data curation: Hye Ri Kim, Wonshik Seong, Hyun Jin Min, Kyung Soo Kim. Formal analysis: Sun Mi Kim, Hye Ri Kim. Funding acquisition: Sun Mi Kim. Investigation: Hye Ri Kim, Wonshik Seong, Hyun Jin Min, Hyuk Ga. Methodology: Sun Mi Kim, Hye Ri Kim. Supervision: Doug Hyun Han, Kyung Soo Kim. Writing—original draft: Sun Mi Kim, Hye Ri Kim. Writing—review & editing: Doug Hyun Han.