EJIHPE | Free Full-Text | The Vaccination Fear Scale (VFS-6): Adaptation, Cross-Cultural Validation, and Invariance among Genders and Six Different Cultures, Applying Classical Test Theory (CTT) and Item Response Theory (IRT)

According to the World Health Organization (WHO), vaccination prevents 2 to 3 million deaths annually, being one of the most highly cost-effective disease prevention methods [1]. Despite the compelling evidence of its benefits, fear and distrust of vaccines have led to declining vaccination rates in recent decades, emerging as one of the top ten major threats to global health [1]. Consequently, increasing vaccination rates have become a crucial topic integrated into the work plans that support the Sustainable Development Goals (SDGs) outlined in the 2030 Agenda, which has been adopted by all United Nations member states [2].

As documented in the literature, COVID-19 profoundly affected the well-being and mental health of individuals [3,4]. Although researchers unanimously recommended vaccination as one of the most efficient and cost-effective ways to curb the pandemic, people’s reluctance toward vaccination represented a critical issue in many countries [3,4,5]. Fear has been recognized as a significant factor which contributes to vaccination reluctance [6,7]. Additionally, belief in conspiracy theories, distrust in medical information, and mistrust in medicine and science have been noted [4]. In any case, fear is identified as the most powerful predictor to date concerning vaccination intention [8], particularly regarding the fear of the vaccine’s adverse effects [9,10].

Fear is commonly defined as an unpleasant emotion that is triggered in response to a stimulus perceived as potentially dangerous and can lead to protective behaviours such as avoidance [11,12]. Vaccination fear can arise from various phenomena and psychological processes related to factors such as fear of vaccine side effects and needles [13,14] and beliefs that vaccines may have no beneficial effects or could even harm individuals [13].

Research on this topic often provides the relevant information needed for the development of interventions that are aimed at reducing fear, particularly given that fear symptoms are relatively identifiable and manageable [15]. However, it is noteworthy that there is a notable lack of reliable tools available for measuring the impact of fear, which might explain the scarcity of studies in this area. However, previous studies have revealed the availability of some tools tailored to measure different aspects related to vaccination, such as worry [16,17], confidence [18], hesitancy [19], or reluctance [20,21], and other tools specifically aimed at assessing psychological barriers to vaccination [22].

Recently, the Fear of COVID-19 Scale (FoCV-19S) developed by Ahorsu et al. [23], was adapted and validated to assess fear of vaccination (6-items Vaccination Fear Scale: VFS-6) within a large Spanish sample by Malas and Tolsá [9]. Despite several attempts to adequately measure vaccination fear using unique or ad hoc items [3,13], only the VFS-6 [9] could represent the complexity of the vaccination fear construct in a valid and reliable manner by condensing the quite extensive literature related to vaccination fear [8].

Subsequently, Ochnik et al. [15] developed and validated the Coronavirus Vaccination Fear Scale (FoCVVS) concerning the German, Polish, Slovenian, and Hebrew languages, following the same methodology used by Malas and Tolsá [9] in creating the VFS-6. Despite both scales starting from the same methodology, there are certain differences between them. In the VFS-6 scale, item number 5 was excluded because it loaded almost equally on the two factors with an unacceptable loading weight (15] scale. It is worth mentioning that the VFS-6 was adapted by Malas and Tolsá [9] to measure fear of any vaccine, including the COVID-19 vaccine, thereby resulting in more generalized items. This difference may explain the variations between the two scales.

The VFS-6 was successfully replicated by Duradoni et al. [8] with an Italian sample yielding results that are consistent with those observed in the Spanish sample. Both studies found a clear and significant association between fear of vaccination and vaccination intention, with the fear of vaccination emerging as a stronger predictor compared to hesitancy [8] or fear of the targeted disease [9]. A clear relationship has also been described between fear of vaccination and the self-report of conspiracy beliefs [24,25].

In cross-cultural research, ensuring the widespread applicability of measurement scales requires examining the equivalence of the instrument in different populations with diverse cultural backgrounds [26]. However, the development of an appropriate instrument for different cultures requires meticulous translation, content and semantic accuracy, as well as technical precision. Equally important is an accurate interpretation to avoid any construct, item-related, or method-related biases that may arise due to problematic instrument administration [27,28].

To facilitate a robust cross-cultural adaptation, Hinkle [26] provided a study which includes relevant validation aspects to ensure a robust fit. Also, a statistical test of the invariance of measurements between different populations is recommended. To analyse the equivalence of the measure, Lacko et al. [29] proposed a stepwise methodology based on the measurement of four different levels of invariance (configural, metric, scalar, and strict) with increasing strength. Configural invariance provides the evidence of qualitative similarity of the factor structure in different groups. Metric invariance enables the assumption that the metric remains consistent across all groups. Scalar invariance allows the assumption that the differences in item measurement between groups are due solely to differences in the latent construct being measured and not to the presence of item biases or differential item functioning. Finally, the strict invariance indicates the equality of the residual variance which implies that the measurement error is uniform across all groups.

There remains a dearth of studies regarding the invariance of the VFS-6. Duradoni et al. [8] obtained evidence of configural and metric invariance between sexes basically at the limit of metric–scalar invariance (ΔCFI = 0.005, ΔRMSEA = 0.01). On the other hand, Ochnik et al. [15] observed configurational, metric, scalar, and strict invariance between gender for the FoCVVS but only configurational invariance between countries or languages analysed. In any case, researchers in this field have emphasised that complete measurement invariance is often not met and represents too stringent a requirement for group comparison [30]. Consequently, the attainment of metric invariance has been considered sufficient to proceed with inferential analysis [31].

This current study successfully confirmed the two correlated factors model (cognitive symptoms and somatic symptoms) using CFA in the total sample and replicated it across sex and for the samples from the different countries. These findings align with the findings of previous studies conducted by Malas and Tolsá [9] and Duradoni et al. [8]. Additionally, the model with second-order factors proposed by Ochnik et al. [15] also showed acceptable results. However, the fit indices were slightly less robust compared to the two correlated factors model, which led us to select the two correlated factors model as the most appropriate.

No studies have been found using IRT analysis for VFS-6, nor for similar scales, so there are no reference data. But, based on the guidelines established by Becker [37] and others such as Stavropoulos et al. [42] or Wilson [43], IRT analysis suggested an appropriate level of discrimination and difficulty of the test with the somatic dimension characterised by higher difficulty compared to the cognitive dimension. These results add great value to those obtained through the CTT, since unlike this, the IRT parameters estimate the relationships between the levels of a latent trait θ and the elements and, therefore, are independent of the sample [37,43].

The internal consistency (α > 0.80; ω > 0.75) and other validity indices were highly appropriate for the scale and its subscales, both for the total sample and by countries, confirming its convergent validity (AVE ≥ 0.50), divergent validity (HTMT 9] and Duradoni et al. [8], confirms that vaccination fear is a predictor of vaccination. In this case, the percentage of explained variance was 45.8%. Sex invariance observed in this study indicates that both sexes understood the content of the VFS-6 equally. This result is consistent with the information provided by Ochnik et al. [15] but differs from those reported by Duradoni et al. [8], which only obtained configural and metric invariance. Across the participating countries, configurational invariance was obtained which suggests that the structure of the two correlated factors was the most appropriate, regardless of the country and language used.

The country/country invariance also give rise to two well-differentiated groups of countries which show configural and metric invariance between them, which, according to Hsiao and Lai [31], would allow inferential analysis between them. These comprise Spain, Italy, and Ukraine on one hand and Nigeria, Lebanon, and Turkey on the other, as well as, Ukraine and Nigeria. On the contrary, Spain and Italy, in relation to Lebanon, Nigeria, and Turkey, only present configurational invariance. Metric invariance was not observed, which implies potential differences in the meaning attributed by the two groups of countries to the underlying construct being studied. This could explain the absence of scalar invariance (scores of latent variables not comparable between groups) and consequently, strict invariance, as the latent construct is not being measured identically in the two groups of countries.

As detailed in the methodology section, data collection was conducted across the six countries during the initial months of 2023, prior to the WHO declaration of the end of the health crisis and their recommendation to sustain seasonal vaccinations against COVID-19. Consequently, vaccination programs are in their final phase worldwide; it can be inferred that the pandemic phase likely has not substantially impacted the levels of fear and anxiety associated with vaccination. Previous research has suggested that vaccination plans are influenced by a combination of socioeconomic and psychological factors including belief, perception, and attitude towards health which vary between countries [44]. In this study, the countries with the lowest vaccination rates were Nigeria and Ukraine. The latter country went to war with Russia in 2022, which explains the referenced rate. As can be seen in Table 1, Nigeria presents an unusual situation. Only 37.61% of the participants declare having been vaccinated at the time this study was carried out, despite the fact that 61.50% were health sciences students. The percentage of vaccinated people is far below that of participants from other countries and, at the same time, the percentage in health sciences was far above participants from other countries, which is paradoxical. In 2021, Nigeria already had vaccines against COVID-19 [45]. Despite this, on those dates, in a survey of students (67.9% in health sciences), only 40% were in favour of getting vaccinated; 15.5% were willing to pay for the vaccine, and 37.3% were willing to recommend it to others [46]. Studies carried out at that time in Spain [24] and other European countries [47] showed similar results regarding vaccination intention. But, finally, as can be seen in Table 1, in 2023 the vaccination percentages exceeded 96%. On the contrary, in Nigeria, the results are very similar to those reported in 2021 [46]. Although Mustafa et al. [46] describe sociodemographic causes, such as age, gender, and ethnicity, the main reasons cited by students for not getting vaccinated were distrust in the government regarding vaccines (73.6%) and the need to pay for them. Students also indicated that they were willing to be vaccinated if ordered by the directors of their educational institutions. It can be assumed that this educational authority did not order vaccination, which would allow us to hypothesize a distrust of vaccines beyond the university students themselves. Hence, it is undeniable that culture will influence the emotional experience of fear, particularly with respect to the secondary processes of communication and coping. Therefore, further study is needed to assess the relationship between the fear of vaccination and country-specific social variables. This will provide further understanding of how the heterogeneity of countries is reflected in the analysis and how the scale is sensitive to cross-cultural contexts.