EJIHPE | Free Full-Text | Development and Initial Validation of the Russian Version of the RAADS-14: A Self-Report Questionnaire to Assess Autistic Traits

Autism spectrum disorder (ASD) is a range of neurodevelopmental conditions characterized by persisting deficits in social communication and interaction, repetitive and stereotyped behavior, and restricted interests [1]. The prevalence of ASD in the adult population is steadily increasing, it greatly affects the quality of life of adults with ASD and impairs a significant burden on health and social care [2]. Autism in adults with no intellectual impairments is also quite frequent [3], and often underdiagnosed [4], particularly in women [5]. Late identification and general underdiagnosis and misdiagnosis of ASD leads to a problem of ‘a lost generation’ of autistic adults with no intellectual impairments [6]. The comorbidity of ASD and other neuropsychiatric conditions is quite high [7], and differential diagnostics can also pose a problem; therefore, proper assessment of autistic traits and related problems can be crucial for increasing the quality of life of people with ASD [8]. There is a range of self-report questionnaires available for screening of ASD in adults with no intellectual impairments, with the most widely used being the following: the Autism-Spectrum Quotient (AQ, 50 items) [9] and its shorter versions, AQ-10, AQ-20, AQ-J-21, and AQ-S [10]; the Ritvo Asperger and Autism Diagnostic Scale—Revised (RAADS-R, 80 items) [11] and its short version RAADS-14 [8]; the Social Responsiveness Scale, 2nd edition—Adult form (SRS-A or SRS2-A, 65 items) [12]; and the Adult Social Behavior Questionnaire (ASBQ, 44 items) [13]. A systematic review found that stronger evidence of good or satisfactory measurement and diagnostic properties exists for AQ-50, AQ-S, RAADS-R, and RAADS-14 [10].

The RAADS-14 is a 14-item self-report questionnaire specifically designed to be used as a screening tool for ASD and distributed under the CC BY 2.0 Creative Commons License [8]. It was developed on the basis of the longer and more comprehensive RAADS-R, which has demonstrated good overall psychometric properties, test–retest reliability, discriminant and convergent validity, sensitivity, and specificity [11,14,15]. The RAADS-14 was developed using the Swedish version of the RAADS-R [8] and validated on Swedish clinical and non-clinical samples. According to the reported factor structure, the RAADS-14 consists of three factors corresponding to domains of mentalizing deficits, social anxiety, and sensory reactivity. All the items are scored on a four-point Likert scale (ranging from 0 to 3) indicating the duration of each symptom (3 = ‘true now and when I was young’, 2 = ‘true only now’, 1 = ‘true only when I was younger than 16’, and 0 = ‘never true’). Item scores are summed to produce the total score and the subscale scores. The recommended cut-off of 14 for the original English version yielded an excellent sensitivity of 0.97 and specificity of 0.95 over the non-psychiatric population. The gender differences in the original validation study were significant but of moderate effect size: neurotypical males scored somewhat higher than females in the mentalizing deficits and social anxiety domains, while females scored higher than males in the sensory reactivity domain.

The situation with clinical diagnostics of ASD in the adult population in Russia remains very problematic but is changing for the better, with underdiagnosis being the major issue. According to the official statistical data by Rosstat (Russian statistical agency), in 2016 there were only 96 adults officially diagnosed as having ASD (less than 0.001% of the adult population) [16], which is at least 1000 times less than the estimates based on European cohort statistics. This huge disagreement reveals the fact that it was virtually impossible to obtain an official diagnosis of ASD for high-functioning adolescents and adults until recently for two major reasons. The first reason is related to the underdiagnosis of ASD in childhood from 1970 to the 2010s, which was less pronounced than for adults but still quite prominent: as of 1999, the officially recognized ratio was 1 in 385 [17]. For children without intellectual impairments, the diagnosis was almost never officially established, often in order to help the parents avoid the stigma [18]. The second reason is an almost routine obligatory change in the official clinical diagnosis from ASD to some other neuropsychiatry condition performed when a person reaches adolescence or adulthood, most commonly, to schizoaffective disorder and/or mental retardation; this clinical practice was officially reproved only on 30 June 2014 [19]. Due to the efforts of governmental and non-governmental institutions, autism professionals, the parental community, and the autism community, it is now widely recognized that ASD in adults is currently underdiagnosed in Russia. Implementing the diagnostic routines will require some time and is hindered by the lack of diagnostic tools and trained professionals. The Russian version of ADOS-2 was published only in 2016 [20,21] and its adult module is still undergoing validation.

In this situation, the need for self-assessment tools in the autistic community has long been perceived as high. Despite that, to the best of our knowledge, no self-assessment tools for evaluating autistic traits and for screening ASD in the adult population or adolescents have been officially developed, adapted, and validated on Russian-speaking samples. Several attempts were made by members of the autistic community to provide Russian versions of internationally recognized questionnaires, even if just for a rough reference; the largest collection of unofficial translations (including AQ, RAADS-R, Aspie Quiz, and The Broad Autism Phenotype Test) is presented on a website developed and maintained by members of the Russian autism community that provides information related to autism and the problems of people with ASD [22]. There are multiple anecdotal pieces of evidence that indicate that most of these questionnaires used with cut-offs established for European samples tend to overestimate the prominence of autistic traits, but no data on this issue have been officially reported.

The complete lack of ASD diagnostic tools and inventories assessing autistic traits validated on Russian-speaking samples significantly limits the options to assess the criterion validity of any emerging ASD screening questionnaire. In order to obtain additional circumstantial evidence, special efforts could be made to evaluate the questionnaire’s discriminant validity comparing it to other personality trait inventories. The Big Five personality model is probably the most well-established general personality construct [23], which was found to strongly relate to autism traits in a large number of studies, consistently revealing moderate positive correlations of ASD scores to neuroticism, and weak to moderate negative correlations to all the other Big Five factors [24]. A study utilizing multivariate approaches to explore the relations between IPIP-NEO-120 and RAADS-R scores found that Big Five personality factors accounted for 70% of the variance in autism trait scores [25]. Thus, as the Big Five relationship to the original RAADS-R is known, replication of those findings with a validated Russian Big Five questionnaire and RAADS-14 translation appropriately would be another evidence of a close relationship between the English and the Russian versions of RAADS-14.

The above-mentioned study was based on the pilot study by the same group presented at the INSAR virtual conference in 2020 [26]. The results of the current study differ in the number of participants (in particular, the ASD group was expanded); it minorly affected the factor composition and slightly increased the discriminative power.

• The RAADS-14 Rus factor structure would be equivalent or similar to the factor structure of the original RAADS-14;

• The distribution of the RAADS-14 scores could be different for the general Russian-speaking population, producing greater mean total score values and yielding potentially higher cut-off scores to provide reasonable specificity of the questionnaire;

• The RAADS-14 Rus total scores and subscale scores will have weak to moderate correlations to the scores of the Big Five personality factors, in particular, positive to neuroticism and negative to extraversion, consistent with the previous studies.

The aim of this study was to develop a Russian version of the RAADS-14 (the RAADS-14 Rus) and to assess its reliability, factor structure, validity, and discriminative properties. The RAADS-14 Rus has exactly the same item composition as the original version; confirmatory factor analysis (CFA) revealed that all the model fit indices were reasonably good, suggesting that the factor structure of the Russian version is equivalent to the original version [8], confirming our first research hypothesis. There was a very robust difference in the scores of all the RAADS-14 Rus screening domains between the non-psychiatric population and the ASD sample, further confirming the construct validity of the RAADS-14 Rus.

The original RAADS-14 was composed of the items yielding the highest discriminative properties for ASD diagnosis; owing to this approach, the questionnaire was characterized by excellent discriminative properties at distinguishing between ASD participants and non-psychiatric controls (AUC = 0.99) and by good discrimination between ASD and other neuropsychiatric conditions. The total RAADS-14 Rus score has good discriminative properties for distinguishing between ASD participants and the general population control sample (AUC of 0.92), which is still not nearly as good as in the original study. This difference can be best attributed to the fact that mean RAADS-14 scores were much higher for the Russian sample than for the original Swedish sample, particularly for non-psychiatric controls. At the moment we would recommend using a provisional cut-off of 22 points and above. This cut-off yields a sensitivity of 85.7% and a specificity of 85.4%. While taken per se these discriminative properties may seem operational, it should be noted that using the RAADS-14 Rus as a screening tool would yield a low positive predictive value, given the estimated prevalence of high-functioning forms of ASD. Using an official prevalence estimate of high-functioning autism of 1/200 for Russian samples [17], the positive predictive value would be as low as 0.029 (one correctly identified autistic person for 32 non-autistic individuals erroneously identified as autistic). Still, at the moment, with no other validated ASD self-assessment tools available for the Russian-speaking population, we suggest that one can use RAADS-14 Rus for screening if all the above-mentioned considerations are taken into account. Further research is sought to develop more sensitive autism screening tools specifically for Russian-speaking samples.

While this paper was in preparation, the results of another study assessing the RAADS-14 properties for a New Zealand national sample were published [31]. The two studies are similar in their purpose and design, and there are notable similarities in the results as well. In both studies, the mean RAADS-14 scores for non-ASD groups were much higher than in the original study (Russia 13.9, New Zealand 9.85 vs. Sweden 3.9); this, combined with other possible factors, led to substantial worsening of the discriminative properties of the questionnaire. The mean RAADS-14 scores were the highest for the Russian study; this confirms our second research hypothesis originated from prior anecdotal evidence that pilot non-validated versions of autism assessment self-report questionnaires produce higher scores for Russian samples than for Western European and American samples. The underlying reasons for this effect are yet unclear; several factors may contribute but at this moment we would suggest that this difference is to be largely attributed to culture-specific biases in self-reporting and speaking about one’s problems and social competence rather than to elevated autistic traits in the Russian neurotypical population. Despite that, the RAADS-14 Rus performed somewhat better than the New Zealand version of the RAADS-14, producing better model fit indices and somewhat better sensitivity and specificity for the author-suggested cut-offs.

The RAADS-14 Rus total score revealed a weak but significant negative correlation with age for the non-psychiatric young adult sample, consistent with the previous data indicating that different facets of autistic traits in adults tend to somewhat decrease with age [36,37]. The female-to-male differences in the RAADS-14 Rus scores were quite minor and generally similar to data from the original RAADS-14 validation study collected for Swedish samples [8]. In our study, a significant part of subjects in Samples 2 and 3 indicated their gender as ‘other’ or non-binary (total n = 36). The non-binary participants scored significantly higher than the males and females; this also corresponds to the prior findings in other national samples [38,39,40].

The correlations of the Big Five personality traits with the total RAADS-14 Rus score, taken as a general measure of autistic traits, were highly consistent with the earlier findings [24], revealing moderate positive correlations with neuroticism, moderate negative correlation with extraversion, and weaker negative correlations with agreeableness, openness to experience, and conscientiousness, similar to a study using RAADS-R and IPIP-NEO-120 [25]. These results support good discriminative validity of the RAADS-14 Rus and confirm our third research hypothesis that the character of the relation between autistic traits, as measured using the RAADS-14 Rus, and the Big Five personality traits for the Russian sample is similar to such relation for other national populations. It provides evidence of criterion validity of the RAADS-14 Rus, as it correlates with a Big Five version validated for the Russian population in the same manner as the original RAADS-14 does with an English version, which means they measure similar traits.