Application of Stress and Anxiety to Viral Epidemics-6 to Measure the Anxiety Response of Cold Chain Practitioners During the COVID-19 Post-Pandemic Era in China

Article information

Psychiatry Investig. 2023;20(2):75-83
Publication date (electronic) : 2023 February 23
doi : https://doi.org/10.30773/pi.2022.0197
1Department of Nursing, Taiyuan Central Hospital, Shanxi Medical University, Taiyuan, Shanxi, China
2Department of Health Statistics, School of Public Health, Shanxi Medical University, Taiyuan, Shanxi, China
3Department of Psychology, University of Chittagong, Chattogram, Bangladesh
4National Centre for Epidemiology and Population Health, Australian National University, Canberra, Australia
5Department of Psychiatry, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
Correspondence: He Runlian, MA Department of Nursing, Taiyuan Central Hospital, Shanxi Medical University, 1 Dongsandao-alley, Jiefang-road, Xinghualing-district, Taiyuan, Shanxi, China Tel: +86-139-0351-6863, Fax: +86-0351-5656-25, E-mail: 2604695837@qq.com
Correspondence: Seockhoon Chung, MD, PhD Department of Psychiatry, Asan Medical Center, University of Ulsan College of Medicine, 86 Olympic-ro 43-gil, Songpa-gu, Seoul 05505, Republic of Korea Tel: +82-2-3010-3411, Fax: +82-2-485-8381, E-mail: schung@amc.seoul.kr
Received 2022 July 8; Revised 2022 October 18; Accepted 2022 October 31.

Abstract

Objective

This study explored the psychometric properties of the Chinese version of the Stress and Anxiety to Viral Epidemics-6 Items (SAVE-6) scale for cold chain practitioners exposed to moderate-to-high risk of infection.

Methods

A total of 233 cold chain practitioners participated in an anonymous online survey, conducted from October to November 2021. The questionnaire comprised participant demographic characteristics, the Chinese version of SAVE-6, the Generalized Anxiety Disorders-7 (GAD-7), and the Patient Health Questionnaire-9 (PHQ-9) scales.

Results

Based on the results of the parallel analysis, the single-structure model of the Chinese version of SAVE-6 was adopted. The scale showed satisfactory internal consistency (Cronbach’s alpha=0.930) and good convergent validity based on Spearman’s correlation coefficient with the GAD-7 (rho=0.616, p<0.001) and PHQ-9 (rho=0.540, p<0.001) scale scores. The optimal cutoff score for Chinese Stress and Anxiety to Viral Epidemics-9 Items was identified as ≥12 (area under the curve=0.797, Sensitivity=0.76, Specificity=0.66) for cold chain practitioners.

Conclusion

The Chinese version of the SAVE-6 scale has good psychometric properties and can be applied as a reliable and valid rating scale to assess the anxiety response of cold chain practitioners in the post-pandemic era.

INTRODUCTION

With the continued spread of the coronavirus disease 2019 (COVID-19) pandemic worldwide, it remains a public health emergency that is seriously threatening individuals’ physical and mental health [1,2]. As of April 2022, COVID-19 had infected over 465 million people worldwide, causing more than six million deaths according to the World Health Organization [3]. Studies have found that during the early or late phase of the COVID-19 outbreak, diverse populations, especially healthcare workers, have experienced varying degrees of psychiatric issues [4]. Healthcare workers have suffered from severe psychological distress, including depression, anxiety, insomnia, or post-traumatic stress symptoms during the pandemic [5,6]. As frontline healthcare workers care for infected patients directly, they worry about their own safety and that of their family, neighbors, and patients [7]. According to a survey on the psychological impact of the pandemic on healthcare workers in China, 54.2% and 58% of respondents reported symptoms of anxiety and depression, respectively [8]. Compared with other occupational groups, healthcare workers are more likely to suffer from poor sleep quality during the pandemic [9]. Therefore, there is an urgent need to investigate COVID-19-related anxiety and other psychological issues among the general population.

There are numerous internationally recognized scales to assess individual psychological anxiety, such as the Generalized Anxiety Disorders-7 items scale (GAD-7) [10], the State and Trait Anxiety Inventory [11], and the Hamilton Anxiety Scale [12] among others. However, these scales lack specificity in assessing psychological states during the COVID-19 pandemic. To bridge this gap, Chung et al. [13] developed the Stress and Anxiety to Viral Epidemics-9 items scale (SAVE-9). Compared with other instruments developed to assess anxiety surrounding COVID-19, such as the Coronavirus Anxiety Scale (CAS) [14] and the Coronavirus Pandemic Anxiety Scale (CPAS) [15], SAVE-9 has many distinct advantages. First, the CAS is a 5-item scale validated for only one dimension, thus limiting its reliability and validity, whereas the CPAS is an 11-item scale with two dimensions. In contrast, SAVE-9 is designed with two factors and nine items, and with fewer items compared to the CPAS. Second, SAVE-9 has very good psychometric properties, consisting of items that probe into a person’s fears or thoughts about the risk of infection, the resulting impact on his or her physical health, and avoiding others. The SAVE-9 scale has been translated and analyzed in several countries, including Korea [13], Russia [16], Italy [17], Japan [18], Turkey [19], and Germany [20]. Third, it can be used to assess the stress caused not only caused by COVID-19 but also other possible epidemics in the future.

Originally, Chung et al. [13] developed SAVE-9 to measure the work-related stress and anxiety responses of healthcare workers specific to the COVID-19 pandemic. The scale is clustered into two factors: Factor I, anxiety about the epidemic (items 1, 2, 3, 4, 5, and 8) and Factor II, work-related stress about the epidemic (items 6, 7, and 9). The Stress and Anxiety to Viral Epidemics-6 items (SAVE-6) scale [21] was derived from Factor I of the SAVE-9 scale (six items), and it was demonstrated that SAVE-6 could be applied to the general population in samples from Korea [21], Lebanon [22], Bangladesh [23], Canada [24], and the United States [25]. SAVE-6 has also been validated among medical students [26,27], public workers [28], healthcare workers [29], and cancer patients [30].

However, the SAVE-6 scale has thus far only been applied to high-risk groups exposed to COVID-19, such as medical personnel and public workers, and not to high-risk populations vulnerable to exposure via viral shipments. The Chinese Center for Disease Control and Prevention isolated SARS-CoV-2 directly from a surface swab of the outer packaging of frozen cod in the Qingdao outbreak [31]. The COVID-19 outbreak in Qingdao was traced to two infected stevedores, and sporadic outbreaks associated with cold-chain food were reported in several parts of China [32,33]. This indicates that the cold chain industry is a medium-to-high-risk industry [34,35], similar to the medical industry. The transmission of COVID-19 is possible via cold chain products as carriers [31,36,37]. A study on sporadic COVID-19 cases associated with the imported cold food supply chain in China showed that all confirmed cases were cold chain workers who had direct or indirect contact with contaminated outer packaging. The patients consisted mainly of stevedores and drivers at seafood ports. For individuals working long hours in medium-to-high-risk environments [38,39], the degree of pandemicrelated anxiety is worthy of study. However, to the best of our knowledge, no such study has been conducted on cold chain practitioners.

Therefore, this study focused on investigating the COVID-19-related anxiety of cold chain practitioners who are required to maintain daily contact with frozen foods that may transmit the virus [40,41]. These individuals must frequently undergo the COVID-19 nucleic acid amplification testing (NAAT) to protect their health [34]. While frequent testing provides reassurance that they are healthy, it can also amplify their anxiety due to the constant face-to-face contact with medical staff. In addition, the vulnerability to infection, fear of COVID-19, overwhelming workload, and financial burden places cold chain practitioners at a higher risk of psychological distress [42]. Currently, there is no psychometric evidence of SAVE-6 in these groups. Therefore, it is pertinent to analyze such a unique population to expand the future use of the SAVE-6 tool for such at-risk populations.

This study hypothesized that the SAVE-6 scale can be successfully applied to measure pandemic-related anxiety among cold chain workers. Thus, the reliability and validity of the Chinese version of the SAVE-6 scale among cold chain workers in China was assessed.

METHODS

Study design

A cross-sectional survey was conducted from October to November 2021, using the Wenjuanxing platform (https://www.wjx.cn/), a widely used web-based survey tool in China. The study link was shared within the messenger group (We-Chat) of cold chain food practitioners by researchers who also participated in cold chain food nucleic acid testing. The questionnaire was anonymous, and the participants were informed about the study’s purpose and the intended scientific use of the data. The first entry was set as “whether or not to voluntarily participate in this study.” If the participant clicked “no,” the questionnaire would directly end. Participants could terminate taking the survey at any time, and personal information was kept confidential.

A minimum of 60 samples were required to validate the scale in this study as the SAVE-6 has six items. The survey form was developed according to the Checklist for Reporting Results of Internet E-Surveys guidelines [43], and investigators checked the technical functionality of the survey form before implementation. The sample size was estimated based on the 10:1 rule—the ideal ratio of respondents to items [44]. However, a target sample size of 300 cold chain practitioners was estimated to validate the SAVE-6 scale based on the recommended range of 200–300 for factor analysis [45,46]. After fulfilling all criteria, a total of 233 responses were collected. The study protocol was approved by the Institutional Review Board (IRB) of Taiyuan Central Hospital (2022008), and written informed consent was waived by the IRB.

Participants

A total of 233 individuals responded to the questionnaire, and data was also collected from a market in Taiyuan, Shanxi Province, China, which is responsible for nucleic acid testing and outer packaging disinfection of imported cold chain foods in Taiyuan at the request of the government. The inclusion criteria were age ≥18 years, engaged in work related to the cold chain process, skilled in phone operation, and voluntary participation in the study. The exclusion criterion was non-willingness to participate in the study. For the purpose of this study, cold chain food practitioners have been defined as individuals who work in close contact with frozen food, including frozen food sellers, sterilization testers, delivery workers, and managers. Participants’ demographic information (age, gender, marital status, education level, and job duties) were collected.

The cold chain food practitioners who participated in this study were responsible for disinfecting outer packaging, sampling the internal and external packaging and food surfaces of goods, undergoing NAAT, collecting and reporting information about positive food tested, disinfecting and testing the environment and objects, and selling cold chain food with qualified tests, among others.

Symptom rating scales

SAVE-6

The SAVE-6 scale was developed to measure an individual’s anxiety during viral epidemics [21]. The SAVE-6 scale was derived from Factor I of the SAVE-9 scale, a rating scale for assessing the work-related stress and viral anxiety of healthcare workers [13]. SAVE-6 consists of six items rated on a 5-point Likert scale: 0 (never), 1 (rarely), 2 (sometimes), 3 (often), and 4 (always). The total score ranges from 0 to 24, and a higher total score reflects a higher level of viral anxiety. To ensure “linguistic and conceptual equivalence,” a Chinese (Mandarin) version (Supplementary Material in the online-only Data Supplement) was developed in this study, using translation and back-translation methods. Specifically, the original English scale was translated into Chinese by two bilingual experts (Chinese and English), and the Chinese translation was backtranslated into English by a different pair of bilingual experts. The back-translated and original versions were then compared by the two authors of this study. In addition, the scale was sent to several colleagues for their feedback to ensure the best fit to the Chinese cultural background and language habits. Consequently, the final Chinese version of the SAVE-6 tool was confirmed.

GAD-7

The GAD-7 is a self-rating scale, measuring an individual’s severity of generalized anxiety [10]. The scale consists of seven items scored on a 4-point Likert scale, from 0 (not at all) to 3 (nearly every day). The total score can range from 0 to 21, and a higher score reflects a higher level of anxiety. In this study, the Chinese version of the GAD-7 [47] was used, and its Cronbach’s alpha was 0.939.

Patient Health Questionnaire-9

The Patient Health Questionnaire (PHQ)-9 is a rating scale, measuring an individual’s severity of depression [48]. The scale consists of nine items scored on a 4-point Likert scale, from 0 (not at all) to 3 (nearly every day). The total score can range from 0 to 27, and a higher score reflects a more severe level of depression. The Chinese version of the PHQ-9 scale [49] was applied in this study, and its Cronbach’s alpha was 0.930.

Statistical analysis

The normality assumption of all the items of the Chinese SAVE-6 was examined based on skewness and kurtosis of values within the range of ±2 [50]. To conduct factor analysis, sampling adequacy was determined using the Kaiser–Meyer– Olkin (KMO) value, and data suitability was confirmed using Bartlett’s test of sphericity. Parallel analysis was conducted with a reduced correlation matrix to explore the dimensional factor of the Chinese SAVE-6 scale [51]. Confirmatory factor analysis (CFA) was conducted to explore the SAVE-6’s construct validity among cold chain practitioners. A satisfactory model fit was defined as a standardized root-mean-square residual (SRMS) value of ≤0.05, a root-mean-square-error of approximation (RMSEA) value of ≤0.10, and comparative fit index (CFI) and Tucker Lewis index (TLI) values ≥0.90 [52,53]. Multi-group CFAs were conducted to examine whether the Chinese version of the SAVE-6 can assess viral anxiety among cold chain practitioners similarly across sex (male vs. female), considering generalized anxiety (GAD-7 ≥10) or depression (PHQ-9 ≥10).

Psychometric properties were assessed via the Item Response Theory (IRT) approach (graded response model [GRM]) and the Rasch model. In the IRT approach, the GRM was used as it is suitable for polytomous data. In the GRM, item fit was assessed through S-χ2 and its p values (adjusted for false discovery rate) and RMSEA values. The GRM provides the slope/discriminating parameters and threshold/difficulty parameters of items. The GRM outputs were estimated using the R mirt package version 1.34 (https://cran.r-project.org/web/packages/mirt/index.html). In addition, the IRT reliability and rho coefficient were estimated. In the Rasch model, infit mean square (MnSQ), outfit MnSQ, item difficulty, item and person separation index, and item and person reliability were estimated.

Finally, the reliability and internal consistency of the scales were assessed using Cronbach’s alpha and McDonald’s omega. To explore convergent validity, a Spearman’s correlation analysis between SAVE-6 scales and the depression and anxiety subscale of the PHQ-4 was conducted, since the distribution of the PHQ-9 and GAD-7 was not within normal limits. SPSS version 21.0, AMOS version 27 (IBM Corp., Armonk, NY, USA), JASP version 0.14.1.0 (JASP Team, Amsterdam, Netherlands), jMetrik version 4.1.1 (https://itemanalysis.com/jmetrik-download/), and RStudio software (https://posit.co) were used for statistical analysis.

RESULTS

Demographic characteristics

All 233 participant responses were collected (Table 1). Among the participants, 54.5% were male, 63.5% were frozen food sellers, and 37.8% were high school educated. Among the 200 participants (Table 1), 14.0% were living alone, and 57.5% responded that transmission of the virus via cold chain products was possible. Regarding questions related to COVID-19, 13.0% answered that they were exposed to frozen products infected with COVID-19, and 15.5% experienced being quarantined. None of the participants had a confirmed COVID-19 diagnosis.

Demographic characteristics of participants

Initial exploratory factor analysis

Based on a skewness and kurtosis within the range of ±2 (Table 2), the normality assumption for all items of the SAVE-6 scale was evaluated. Sampling adequacy and data suitability for factor analysis were confirmed based on the KMO measure (0.90) and Bartlett’s test of sphericity (p<0.001). A singlefactor model of the Chinese version of the SAVE-6 scale among cold chain practitioners was advised based on an eigenvalue plot over 1 (eigenvalue=4.156) in scree analysis, and the result of the parallel analysis with Minimum Rank Factor Analysis (MRFA) extraction (reduced eigenvalue=4.126 and 95 percentile of random reduced eigenvalue=0.341) (Table 3).

Item properties of the Chinese version of SAVE-6 items scale among cold chain practitioners (N=233)

Scale level psychometric properties of the Chinese version of SAVE-6 (N=233)

CFA

The CFA results showed good model fit for the Chinese version of the SAVE-6 scale among cold chain practitioners (CFI=1.000, TLI=1.003, RMSEA=0.000, SRMR=0.030) (Table 3). The multi-group CFA with configural invariance (Supplementary Table 1 in the online-only Data Supplement) indicated that the single-factor Chinese version of the SAVE-6 can measure the viral anxiety of cold chain practitioners in the same way across sex (male vs. female, CFI=1.000, RMSEA=0.000), considering the pre-existence of generalized anxiety (GAD-7 ≥10, CFI=1.000, RMSEA=0.000) or depression (PHQ-9 ≥10, CFI=1.000, RMSEA=0.000).

Graded response model

Supplementary Table 2 (in the online-only Data Supplement) presents the GRM outputs. Non significant p-values (≥0.01, adjusted for false discovery rate) of the S-χ2 and RMSEA values (≤0.08) suggest that all the items belong to the same latent construct. As shown in Supplementary Table 2 (in the online-only Data Supplement), all the slope parameters (α) were very high, ranging between 2.378 and 4.081 (mean α=3.273). For the threshold/difficulty parameters (b), the slope parameters ranged between 0.799 and 3.032 (mean=1.690). Concerning threshold coefficients (b), a higher latent trait, or theta, was required to endorse the response options of “often” and “always” in all items. Item characteristic curves (Figures 1 and 2) depict the above information graphically.

Figure 1.

Item response category characteristic curve of the Chinese version of the SAVE-6 scale among cold chain practitioners. SAVE-6, Stress and Anxiety to Viral Epidemics-6.

Figure 2.

Scale information curve of the Chinese version of SAVE-6 among cold chain practitioners. I, information; SE, standard error; SAVE-6, Stress and Anxiety to Viral Epidemics-6.

Rasch outputs

Supplementary Table 3 (in the online-only Data Supplement) presents the Rasch model outputs. Infit and mean squares of all items were between the recommended range (0.50–1.50). Regarding item difficulty, item 1 was the least difficult and item 5 the most difficult. Item and person reliability and separation indices were also above the recommended cutoffs (>2 and ≥0.7, respectively) (Table 3).

Reliability, evidence, and cutoff scores

The Chinese version of the SAVE-6 scale showed good internal consistency (Cronbach’s alpha=0.930, McDonald’s omega=0.932), and the Cronbach’s alpha if the item dropped was between 0.911–0.928. This scale also indicated good IRT reliability (0.918) and rho coefficient (0.936). The Chinese version of the SAVE-6 score was significantly correlated with the GAD-7 (rho=0.616, p<0.001) and PHQ-9 scores (rho=0.540, p<0.001). The Chinese version of the SAVE-6 total score was significantly higher among participants who had generalized anxiety (GAD-7 ≥10, t(231)=8.40, p<0.001) or depression (PHQ-9 ≥10, t(231)=7.02, p<0.001) (Supplementary Table 4 in the online-only Data Supplement).

A receiver operating characteristic (ROC) analysis was conducted to examine the appropriate cutoff score of the Chinese version of the SAVE-6 scale in accordance with a mild degree of generalized anxiety symptoms (GAD-7 ≥5), and the cutoff point was calculated to be 12 (area under the curve [AUC]=0.797, sensitivity=0.76, specificity=0.66).

To determine whether there were differences between the groups regarding general information and anxiety scores, independent t-tests and one-way ANOVAs were performed. The results indicate that differences in job duties, perceived risk of infection at work, the perceived probability of virus transmission via the cold chain, exposure to frozen products infected with COVID-19, and quarantine experience had an impact on the SAVE-6 scores, and sleep duration per night impacted the differences in GAD-7 scores.

DISCUSSION

This study explored the reliability and validity of the Chinese version of the SAVE-6 scale among cold chain practitioners prone to psychological anxiety due to frequent daily contact with frozen food with a high probability of transmitting viruses. A good model fit was observed for the single-structure of the SAVE-6 scale, showing good convergent validity with the GAD-7 and PHQ-9 scales. The Chinese version of the SAVE-6 successfully measured cold chain practitioners’ viral anxiety regardless of gender, pre-existing anxiety, or pre-existing depression. The optimal cutoff point was identified as 12 in accordance with a mild degree of generalized anxiety symptoms. Therefore, the SAVE-6 scale is a reliable and valid rating scale to measure viral anxiety among cold chain practitioners.

This study is the first to explore the reliability and validity of the Chinese version of the SAVE-6 scale for cold chain practitioners to assess their viral anxiety during the COVID-19 pandemic. Limited studies have evaluated the impact of the pandemic on the mental health of non-healthcare workers, and its psychological impact on those working in the cold chain industry has yet to be explored. Notably, the first COVID-19 outbreak in late 2019 was associated with a seafood market in Wuhan, China, while the second, in June 2020, was also related to a seafood market in Beijing [54,55].

The demographic characteristics of cold chain practitioners may influence their understanding of the scale and responses. They tend to have a lower level of education, and their economic situation has been impacted negatively by the COVID-19 pandemic. Specifically, sellers of imported cold chain foods not only bear some of the cost of testing frozen foods but also face detention at port, destruction of positive goods, and other unexpected situations.

In the present sample, the factor loading of item 5 (“Are you worried that others might avoid you even after the infection risk has been minimized?”) was high (0.801), contrary to previous studies [23,27,29]. There are several possible factors for such an outcome. First, item 5 may not be useful to assess an individual’s viral anxiety. Although the SAVE-9 scale was originally clustered into two factors in South Korea, viral anxiety (six items, including item 5) and work-related stress (three items) [13], item 5 was clustered into work-related stress in Russia [16] and Germany [20]. Second, people may slowly adjust to the long period of the pandemic, and stigmatization would thus no longer a serious problem anymore. Third, cultural differences or differing participant samples might influence the low factor loading value. In this study, 42.5% of study participants were still unaware that the virus can be transmitted through food in the cold chain, and 57.5% believed that their risk of infection at work was low, which may result in cold chain practitioners not worrying about others avoiding them even after the infection risk is minimized. In addition, none of the participants had been diagnosed with COVID-19, weakening the notion that others would avoid them.

Compared with previous studies, the level of anxiety among the sample population in this study was lower. This finding may be explained by the following: data collection was in the second year of the outbreak (the period of pandemic normalization of the epidemic); people were more accepting of the existence of the virus; many focused on providing for their families under economic pressure and ignored or downplayed its threat; strict control by the Chinese government and the development of the COVID-19 vaccine also helped reduce fear of the virus and alleviate anxiety.

The ROC analysis indicated that the appropriate cutoff score of the SAVE-6 is 12 among the participants of this study. Previously, a cutoff score of 15 was reported among the general population [21] and medical students [26] and 16 among public workers [28] in South Korea, despite the low score (12 point) among the general population in Lebanon [22]. The 12-point cutoff score of the Chinese version of the SAVE-6 among cold chain practitioners may be influenced by differences in the COVID-19 situation or cultural factors. A relatively low cutoff score in accordance with generalized anxiety symptoms may reflect that the level of generalized anxiety symptoms was higher in this sample than that in previous studies. Previously, the mean GAD-7 score was reported as 3.2±3.9 in males and 3.4±3.7 in females among the general population sample [21], but a higher GAD-7 score of 8.9±5.3 was reported among the general population in Lebanon [22]. A mean score of 3.9±4.7 among the present sample might influence the low cutoff point of 12 compared to the original scale [21].

In addition, differences in job duties, perceived risk of infection at work, perceived probability of virus transmission via the cold chain, and exposure to frozen products infected with COVID-on the SAVE-6 were observed, with some possible explanations presented as follows. Participants with a history of quarantine were more likely to suffer psychological problems. Social isolation is generally associated with physical and mental health problems such as, anxiety and depression [56]. Participants who had been exposed to frozen virusinfected products were more likely to be anxious, similar to medical personnel exposed to patients with COVID-19 who were more likely to have stress and anxiety [57]. Higher levels of anxiety also were found among the study participants who perceived that viruses can be transmitted through the cold chain and who believed that they currently work in high-risk occupations. These populations may actively search for relevant information on the Internet, thus leading to information overload and information anxiety [58]. Anxiety scores in this study were higher among disinfection testers and stevedores and lower among sellers and managers. This could be because disinfection testers and stevedores were exposed to uninspected commodities, whereas sellers and managers were exposed to inspected and approved commodities.

Despite this study’s important findings, it has some limitations. First, the results do not justify that the Chinese version of the SAVE-6 scale can be applied to the Chinese general population with good validity and reliability. We have conducted several studies exploring the reliability and validity of the SAVE-9 or SAVE-6 scales among special populations at a high risk of infection in this pandemic, such as schoolteachers [59], public workers [28], or cancer patients [30]. This study was conducted among cold chain practitioners as a special population. Second, the survey was conducted anonymously online, without face-to-face contact with the participants. Although cross-cultural adaptation and expert reviews have made the scale more compatible with Chinese cultural habits, the probability of biased data due to differential understanding of the items by participants cannot be dismissed. Third, the COVID-19 pandemic is still evolving even though it has been more than two years since the first reported case, and the psychological perception of the population is changing rapidly. The results of this study represent only the psychological assessment of cold chain personnel in the postpandemic era. Fourth, it is difficult to classify whether cold chain personnel are sellers of imported or domestic frozen food because the products for sale vary according to the situation. Finally, this was a cross-sectional study and did not examine the stability of the SAVE-6 scale over time; therefore, longitudinal studies should be conducted in the future for a more comprehensive analysis of the topic.

In conclusion, the results indicate that the Chinese version of the SAVE-6 scale is a valid and reliable rating scale to measure the anxiety level of cold chain workers in response to the COVID-19 pandemic. It further shows that it is stable in measuring the psychological anxiety of Chinese cold chain practitioners regardless of sex and level of anxiety or depression.

Supplementary Materials

The online-only Data Supplement is available with this article at https://doi.org/10.30773/pi.2022.0197.

Supplementary Material

Stress and Anxiety to Viral Epidemics - 9 items (SAVE-9) for Healthcare workers

pi-2022-0197-suppl1.pdf
Supplementary Table 1.

Measurement invariance

pi-2022-0197-suppl2.pdf
Supplementary Table 2.

Graded response model

pi-2022-0197-suppl3.pdf
Supplementary Table 3.

Rasch output

pi-2022-0197-suppl4.pdf
Supplementary Table 4.

Demographic differences between the SAVE-6 and GAD-7

pi-2022-0197-suppl5.pdf

Notes

Availability of Data and Material

The datasets generated or analyzed during the study are available from the corresponding author on reasonable request.

Conflicts of Interest

Seockhoon Chung, a contributing editor of the Psychiatry Investigation, was not involved in the editorial evaluation or decision to publish this article. All remaining authors have declared no conflicts of interest.

Author Contributions

Conceptualization: He Runlian, Seockhoon Chung. Data curation: He Runlian, Du Xinjie. Formal analysis: He Runlian, Du Xinjie, Oli Ahmed, Seockhoon Chung. Investigation: Seockhoon Chung. Methodology: Oli Ahmed, Eulah Cho. Project administration: Du Xinjie, Eulah Cho. Supervision: He Runlian. Writing—original draft: all authors. Writing—review & editing: all authors.

Funding Statement

None

References

1. Bao Y, Sun Y, Meng S, Shi J, Lu L. 2019-nCoV epidemic: address mental health care to empower society. Lancet 2020;395:e37–e38.
2. Vindegaard N, Benros ME. COVID-19 pandemic and mental health consequences: systematic review of the current evidence. Brain Behav Immun 2020;89:531–542.
3. World Health Organization. WHO Coronavirus (COVID-19) Dashboard [Internet]. Avaliable at: https://covid19.who.int. Accessed May 1, 2022.
4. Evanoff BA, Strickland JR, Dale AM, Hayibor L, Page E, Duncan JG, et al. Work-related and personal factors associated with mental well-being during the COVID-19 response: survey of health care and other workers. J Med Internet Res 2020;22e21366.
5. Qiu Y, Wu Q, Chen R, Guan C. Research on psychological stress and mental health of medical staff in COVID-19 prevention and control. Int J Disaster Risk Reduct 2021;65:102524.
6. Bahadirli S, Sagaltici E. Post-traumatic stress disorder in healthcare workers of emergency departments during the pandemic: a cross-sectional study. Am J Emerg Med 2021;50:251–255.
7. Ahn MH, Shin YW, Suh S, Kim JH, Kim HJ, Lee KU, et al. High workrelated stress and anxiety as a response to COVID-19 among health care workers in South Korea: cross-sectional online survey study. JMIR Public Health Surveill 2021;7e25489.
8. Xiao X, Zhu X, Fu S, Hu Y, Li X, Xiao J. Psychological impact of healthcare workers in China during COVID-19 pneumonia epidemic: a multicenter cross-sectional survey investigation. J Affect Disord 2020;274:405–410.
9. Huang Y, Zhao N. Generalized anxiety disorder, depressive symptoms and sleep quality during COVID-19 outbreak in China: a web-based cross-sectional survey. Psychiatry Res 2020;288:112954.
10. Spitzer RL, Kroenke K, Williams JB, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med 2006;166:1092–1097.
11. Spielberger CD, Gorsuch RL. AManual for the state-trait anxiety inventory (form Y): “self-evaluation questionnaire.” Palo Alto, CA: Consulting Psychologists Press; 1983.
12. Hamilton M. The assessment of anxiety states by rating. Br J Med Psychol 1959;32:50–55.
13. Chung S, Kim HJ, Ahn MH, Yeo S, Lee J, Kim K, et al. Development of the Stress and Anxiety to Viral Epidemics-9 (SAVE-9) scale for assessing work-related stress and anxiety in healthcare workers in response to viral epidemics. J Korean Med Sci 2021;36e319.
14. Lee SA. Coronavirus anxiety scale: a brief mental health screener for COVID-19 related anxiety. Death Stud 2020;44:393–401.
15. Bernardo ABI, Mendoza NB, Simon PD, Cunanan ALP, Dizon JIWT, Tarroja MCH, et al. Coronavirus Pandemic Anxiety Scale (CPAS-11): development and initial validation. Curr Psychol 2022;41:5703–5711.
16. Mosolova E, Chung S, Sosin D, Mosolov S. Stress and anxiety among healthcare workers associated with COVID-19 pandemic in Russia. Psychiatr Danub 2020;32:549–556.
17. Tavormina G, Tavormina MGM, Franza F, Aldi G, Amici P, Amorosi M, et al. A new rating scale (SAVE-9) to demonstrate the stress and anxiety in the healthcare workers during the COVID-19 viral epidemic. Psychiatr Danub 2020;32(Suppl 1):5–9.
18. Okajima I, Chung S, Suh S. Validation of the Japanese version of Stress and Anxiety to Viral Epidemics-9 (SAVE-9) and relationship among stress, insomnia, anxiety, and depression in healthcare workers exposed to coronavirus disease 2019. Sleep Med 2021;84:397–402.
19. Uzun N, Akça ÖF, Bilgiç A, Chung S. The validity and reliability of the Stress and Anxiety to Viral Epidemics-9 items scale in Turkish health care professionals. J Community Psychol 2022;50:797–805.
20. König J, Chung S, Ertl V, Doering BK, Comtesse H, Unterhitzenberger J, et al. The German translation of the Stress and Anxiety to Viral Epidemics-9 (SAVE-9) scale: results from healthcare workers during the second wave of COVID-19. Int J Environ Res Public Health 2021;18:9377.
21. Chung S, Ahn MH, Lee S, Kang S, Suh S, Shin YW. The Stress and Anxiety to Viral Epidemics-6 items (SAVE-6) scale: a new instrument for assessing the anxiety response of general population to the viral epidemic during the COVID-19 pandemic. Front Psychol 2021;12:669606.
22. Hong Y, Yoo S, Mreydem HW, Abou Ali BT, Saleh NO, Hammoudi SF, et al. Factorial validity of the arabic version of the Stress and Anxiety to Viral Epidemics-6 items (SAVE-6) scale among the general population in Lebanon. J Korean Med Sci 2021;36e168.
23. Ahmed O, Hossain KN, Hiramoni FA, Siddique RF, Chung S. Psychometric properties of the Bangla version of the Stress and Anxiety to Viral Epidemics-6 items scale among the general population in Bangladesh. Front Psychiatry 2022;13:804162.
24. Park CHK, Ahmed O, Lee S, Suh S, Chung S, Gouin JP. The psychometric properties of the French-Canadian Stress and Anxiety to Viral Epidemics-6 scale for measuring the viral anxiety of the general population during the COVID-19 pandemic. Front Psychiatry 2022;13:807312.
25. Lee S, Lee J, Yoo S, Suh S, Chung S, Lee SA. The psychometric properties of the Stress and Anxiety to Viral Epidemics-6 items: a test in the U.S. general population. Front Psychiatry 2021;12:746244.
26. Ahn J, Lee J, Hong Y, Park J, Chung S. Stress and Anxiety to Viral Epidemics-6 for medical students: psychometric properties of the anxiety measure for the COVID-19 pandemic. Front Psychiatry 2021;12:705805.
27. Lapeyre-Rivera A, Javier-Murillo N, Perea-Flórez F, Gamonal B, Velásquez-Rimachi V, Alva-Díaz C, et al. Validation of the Peruvian Spanish version of the Stress and Anxiety to Viral Epidemics-6 scale to measure viral anxiety of medical students during COVID-19. Front Psychiatry 2022;13:876379.
28. Park CHK, Ju G, Yi K, Lee S, Suh S, Chung S. Application of Stress and Anxiety to Viral Epidemics-6 items (SAVE-6) to public workers for measuring their anxiety response during the COVID-19 pandemic. Front Psychiatry 2021;12:701543.
29. Moraleda-Cibrián M, Ahmed O, Albares-Tendero J, Chung S. Validity and reliability of the Stress and Anxiety to Viral Epidemics-6 (SAVE-6) scale to measure viral anxiety of healthcare workers in Spain during the COVID-19 pandemic. Front Psychiatry 2022;12:796225.
30. Kim H, Kim H, Lee HJ, Cho E, Koh SJ, Ahmed O, et al. The validation study of the Stress and Anxiety to Viral Epidemics-6 scale among patients with cancer in the COVID-19 pandemic. Front Psychiatry 2022;13:811083.
31. Liu P, Yang M, Zhao X, Guo Y, Wang L, Zhang J, et al. Cold-chain transportation in the frozen food industry may have caused a recurrence of COVID-19 cases in destination: successful isolation of SARS-CoV-2 virus from the imported frozen cod package surface. Biosaf Health 2020;2:199–201.
32. Ma H, Wang Z, Zhao X, Han J, Zhang Y, Wang H, et al. Long distance transmission of SARS-CoV-2 from contaminated cold chain products to humans—Qingdao City, Shandong Province, China, September 2020. China CDC Wkly 2021;3:637–644.
33. Ma H, Zhang J, Wang J, Qin Y, Chen C, Song Y, et al. COVID-19 outbreak caused by contaminated packaging of imported cold-chain products—Liaoning Province, China, July 2020. China CDC Wkly 2021;3:441–447.
34. Han S, Liu X. Can imported cold food cause COVID-19 recurrent outbreaks? A review. Environ Chem Lett 2022;20:119–129.
35. Love DC, Allison EH, Asche F, Belton B, Cottrell RS, Froehlich HE, et al. Emerging COVID-19 impacts, responses, and lessons for building resilience in the seafood system. Glob Food Sec 2021;28:100494.
36. Godoy MG, Kibenge MJT, Kibenge FSB. SARS-CoV-2 transmission via aquatic food animal species or their products: a review. Aquaculture 2021;536:736460.
37. Han J, Zhang X, He S, Jia P. Can the coronavirus disease be transmitted from food? A review of evidence, risks, policies and knowledge gaps. Environ Chem Lett 2021;19:5–16.
38. Kumar S, Singh R, Kumari N, Karmakar S, Behera M, Siddiqui AJ, et al. Current understanding of the influence of environmental factors on SARS-CoV-2 transmission, persistence, and infectivity. Environ Sci Pollut Res Int 2021;28:6267–6288.
39. Sajadi MM, Habibzadeh P, Vintzileos A, Shokouhi S, Miralles-Wilhelm F, Amoroso A. Temperature, humidity, and latitude analysis to estimate potential spread and seasonality of coronavirus disease 2019 (COVID-19). JAMA Netw Open 2020;3e2011834.
40. Chi Y, Wang Q, Chen G, Zheng S. The long-term presence of SARS-CoV-2 on cold-chain food packaging surfaces indicates a new COVID-19 winter outbreak: a mini review. Front Public Health 2021;9:650493.
41. González N, Marquès M, Domingo JL. Respiratory viruses in foods and their potential transmission through the diet: a review of the literature. Environ Res 2021;195:110826.
42. Chi Y, Zheng S, Liu C, Wang Q. Transmission of SARS-CoV-2 on coldchain food overpacks: a new challenge. J Glob Health 2021;11:03071.
43. Eysenbach G. Improving the quality of web surveys: the checklist for reporting results of internet E-surveys (CHERRIES). J Med Internet Res 2004;6e34.
44. Nunnally JC. Psychometric theory New York: McGraw-Hill; 1978.
45. Guadagnoli E, Velicer WF. Relation of sample size to the stability of component patterns. Psychol Bull 1988;103:265–275.
46. Comrey AL. Factor-analytic methods of scale development in personality and clinical psychology. J Consult Clin Psychol 1988;56:754–761.
47. He XY, Li CB, Qian J, Cui HS, Wu WY. Reliability and validity of a generalized anxiety scale in general hospital outpatients. Shanghai Arch Psychiatry 2010;22:200–203.
48. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 2001;16:606–613.
49. Wang W, Bian Q, Zhao Y, Li X, Wang W, Du J, et al. Reliability and validity of the Chinese version of the Patient Health Questionnaire (PHQ-9) in the general population. Gen Hosp Psychiatry 2014;36:539–544.
50. Gravetter F, Wallnau L. Essentials of statistics for the behavioral sciences (8th ed) Belmont, CA: Wadsworth; 2014.
51. Glorfeld LW. An improvement on Horn’s parallel analysis methodology for selecting the correct number of factors to retain. Educ Psychol Meas 1995;55:377–393.
52. Brown TA. Confirmatory factor analysis for applied research New York: Guilford Press; 2006.
53. Byrne BM. Structural equation modeling with AMOS: basic concepts, applications, and programming Mahwah, NJ: Lawrence Erlbaum Associates; 2001.
54. Pang X, Ren L, Wu S, Ma W, Yang J, Di L, et al. Cold-chain food contamination as the possible origin of COVID-19 resurgence in Beijing. Natl Sci Rev 2020;7:1861–1864.
55. Dominic Welling. China finds more imported frozen seafood products tainted with COVID-19 [Internet]. Bergen: Intrafish; 2020. [cited 2022 May 1]. Available from: https://www.intrafish.com/trade/chinafinds-more-imported-frozen-seafood-products-tainted-with-covid-19/2-1-855829.
56. Holt-Lunstad J, Smith TB, Baker M, Harris T, Stephenson D. Loneliness and social isolation as risk factors for mortality: a meta-analytic review. Perspect Psychol Sci 2015;10:227–237.
57. Kannampallil TG, Goss CW, Evanoff BA, Strickland JR, McAlister RP, Duncan J. Exposure to COVID-19 patients increases physician trainee stress and burnout. PLoS One 2020;15e0237301.
58. Soroya SH, Farooq A, Mahmood K, Isoaho J, Zara SE. From information seeking to information avoidance: understanding the health information behavior during a global health crisis. Inf Process Manag 2021;58:102440.
59. Yoo S, Lee J, Ju G, Lee S, Suh S, Chung S. The schoolteachers’ version of the Stress and Anxiety to Viral Epidemics-9 (SAVE-9) scale for assessing stress and anxiety during the COVID-19 pandemic. Front Psychiatry 2021;12:712670.

Article information Continued

Figure 1.

Item response category characteristic curve of the Chinese version of the SAVE-6 scale among cold chain practitioners. SAVE-6, Stress and Anxiety to Viral Epidemics-6.

Figure 2.

Scale information curve of the Chinese version of SAVE-6 among cold chain practitioners. I, information; SE, standard error; SAVE-6, Stress and Anxiety to Viral Epidemics-6.

Table 1.

Demographic characteristics of participants

Variables Value (N=233)
Sex (male) 127 (54.5)
Age
 18–25 yr 18 (7.7)
 26–35 yr 77 (33.0)
 36–45 yr 82 (35.2)
 46–55 yr 49 (21.0)
 >56 yr 7 (3.0)
Educational level
 Primary school or below 10 (4.3)
 Middle school 61 (26.2)
 High school 88 (37.8)
 University or above 74 (31.8)
Marital status
 Single 35 (15.0)
 Married 193 (82.8)
 Divorced or widowed 5 (2.1)
Job duties
 Frozen food sellers 148 (63.5)
 Frozen food sterilization testers 9 (3.9)
 Frozen food stevedores 21 (9.0)
 Frozen food managers 55 (26.3)
Time engaged in work*
 <5 yr 98 (49.0)
 5–10 yr 49 (24.5)
 >10 yr 53 (26.5)
Daily working hours*
 <7 hr 13 (6.5)
 7–9 hr 150 (75.0)
 >9 hr 37 (18.5)
Sleep duration per night*
 <7 hr 45 (22.5)
 7–8 hr 133 (66.5)
 >8 hr 22 (11)
Living with family*
 Yes 172 (86.0)
 No 28 (14.0)
Perceived risk of infection at work*
 Low risk 114 (57.0)
 Medium risk 62 (31.0)
 High risk 24 (12.0)
Perceived probability of virus transmission via cold chain*
 Possible 115 (57.5)
 Uncertain 63 (31.5)
 Unlikely 22 (11.0)
COVID-19 questions*
 Have you been diagnosed with COVID-19? (yes) 0
 Have you been exposed to frozen products infected with COVID-19? (yes) 26 (13.0)
 Have you experienced quarantine? (yes) 31 (15.5)
Rating scale scores
 Stress and Anxiety to Viral Epidemics-6 items 11.0±7.2
 Generalized Anxiety Disorders-7 items 3.9±4.7
 Patient Health Questionnaire-9 items 3.9±5.1

Values are presented as mean±standard deviation or number (%), unless otherwise indicated. Percentages may not total 100% due to rounding.

*

There is some missing data (N=200). COVID-19, coronavirus disease 2019

Table 2.

Item properties of the Chinese version of SAVE-6 items scale among cold chain practitioners (N=233)

Items Response scale*
Descriptive
CITC CID Factor loading
0 1 2 3 4 M±SD Skewness Kurtosis EFA CFA
Item 1 18.0 11.6 28.3 21.9 20.2 2.15±1.36 -0.21 -1.09 0.712 0.928 0.749 0.741
Item 2 32.2 14.6 21.0 15.5 16.7 1.70±1.48 0.23 -1.34 0.847 0.911 0.884 0.885
Item 3 20.6 18.9 23.6 18.5 18.5 1.95±1.39 0.04 -1.25 0.843 0.911 0.881 0.882
Item 4 24.9 24.9 24.0 13.7 12.4 1.64±1.33 0.36 -1.00 0.796 0.918 0.828 0.830
Item 5 28.8 19.7 25.3 15.9 10.3 1.59±1.33 0.30 -1.07 0.768 0.921 0.800 0.801
Item 6 23.2 15.9 23.2 20.2 17.6 1.93±1.41 0.00 -1.28 0.810 0.916 0.844 0.846
*

Response scale: 0, never; 1, rarely; 2, sometimes; 3, often; 4, always.

SAVE-6, Stress and Anxiety to Viral Epidemics-6 items; M, mean; SD, standard deviation; CITC, corrected item-total correlation; CID, Cronbach’s alpha if item deleted; EFA, exploratory factor analysis; CFA, confirmatory factor analysis

Table 3.

Scale level psychometric properties of the Chinese version of SAVE-6 (N=233)

Psychometric properties Scores Suggested cutoff
 Cronbach’s alpha 0.930 ≥0.7
 McDonald’s omega 0.932 ≥0.7
 Standard error of measurement 1.891 Smaller than SD 4.85/2
 Rho coefficient 0.936 ≥0.7
 IRT reliability 0.918 ≥0.7
 Item separation index 3.457 ≥2
 Person separation index 2.859 ≥2
 Item reliability 0.921 ≥0.7
 Person reliability 0.891 ≥0.7
Statistics from exploratory factor analysis
 KMO measure of sample adequacy 0.90 0.50
 Bartlett’s test of sphericity 1077.048 (p<0.001) Significant
 Determinant 0.0091
 Eigenvalue 4.156 1 or above
 Variance 69.3
 SRMR 0.04
 TLI 0.945
Outputs from parallel analysis
 Reduced eigenvalue 4.126 1 or above
 95 percentile of random reduced eigenvalue 0.341
Confirmatory factor analysis model fits
 χ2/df 5.128/9 (p=0.823) Nonsignificant
 CFI 1.000 >0.95
 TLI 1.003 >0.95
 RMSEA 0.000 <0.08
 SRMR 0.030 <0.08

SAVE-6, Stress and Anxiety to Viral Epidemics-6 items; SD, standard deviation; IRT, Item Response Theory; KMO, Kaiser-Meyer-Olkin; TLI, Tucker Lewis index; CFI, comparative fit index; RMSEA, a root-mean-square-error of approximation; SRMR, standardized root-mean-square residual