Scarcity of research on psychological or psychiatric states using validated questionnaires in low- and middle-income countries: A ChatGPT-assisted bibliometric analysis and national case study on some psychometric properties

Background It is vital to assess whether research on psychological or psychiatric states using validated questionnaires is still lagging in low- and middle-income countries and to what degree, and to continue to assess the psychometric properties of the most informative questionnaires. Methods We performed a bibliometric analysis of Web of Science Core Collection for all years to determine the number of studies performed in each country that used an inventory or a questionnaire on aggression, anxiety, depression, borderline personality, narcissism, self-harm, shame, or childhood trauma. We conducted a simple observational analysis of distributions by countries to derive the main overall conclusions, assisted by ChatGPT to test its ability to summarise and interpret this type of information. We also carried out a study in Croatia to examine some psychometric properties of five commonly used questionnaires, using Cronbach’s α coefficient and zero-order correlations. Results We observed a concentration of research activity in a few high-income countries, primarily the United States and several European nations, suggesting a robust research infrastructure and a strong emphasis on studying psychological and psychiatric states within their population. In contrast, low- and middle-income countries were notably under-represented in research on psychological and psychiatric states, although the gap seems to be closing in some countries. Turkey, Iran, Brazil, South Africa, Mexico, India, Malaysia and Pakistan have been consistently contributing an increasing number of studies and catching up with the most research-intensive high-income countries. The national case study in Croatia confirmed adequate psychometric properties of the most frequently used questionnaires. Conclusions Addressing research gaps in low- and middle-income countries is crucial, because relying solely on research from high-income countries may not fully capture the nuances of psychological and psychiatric states within diverse populations. To bridge this gap, it is essential to prioritise mental health research in low-resource settings, provide training and resources to local researchers, and establish international collaborations. Such efforts can lead to the development of culturally valid questionnaires, an improved understanding of psychological and psychiatric states in diverse contexts, and the creation of effective interventions to promote mental well-being on a global scale.

The study of psychological and psychiatric states is crucial for understanding and addressing the mental health challenges faced by individuals worldwide.Recent systematic reviews have shown that mental health disorders are still one of the leading causes of global disability, with depressive and anxiety disorders causing 49.4 and 44.5 million disability-adjusted life-years (DALYs) worldwide, respectively [1,2].However, despite the growing recognition of mental health as a global concern, there remains a significant scarcity of research focused on low-and middle-income countries (LMICs).These regions are home to a substantial proportion of the world's population, and the burden of mental health disorders is particularly pronounced [1,3].Inadequate attention to mental health research in LMICs perpetuates disparities in access to evidence-based interventions and contributes to a lack of culturally sensitive approaches for mental health promotion and treatment [4].
Here we present a bibliometric analysis of the scarcity of research on psychological and psychiatric states using validated inventories or questionnaires in LMICs.We provide an overview of the research output in this field, enabling us to examine the geographical distribution of studies and national-level overall productivity across eight inventories and questionnaires, aiming to highlight the urgent need for increased attention and investment in mental health research in LMICs [5].
We also present a national case study in Croatia that further examines the psychometric properties of these inventories and questionnaires.In the case of mental health research in Croatia, previous empirical studies utilising such self-report instruments have assessed levels and correlates of depression and anxiety [6,7], pathological narcissism [8,9], shame [8], borderline personality [10], childhood trauma [11], and aggression [12].For example, certain cross-cultural differences were obtained between post-war transitional Croatian society and some Western countries in regard to depression [13] and pathological narcissism [14].This part of our study aims to provide a contextualised understanding of the challenges and opportunities that influence the validity of research on psychological and psychiatric states, taking into account cultural factors.

Global bibliometric analysis of the differences between high-income countries and low-and middle-income countries
We searched Web of Science Core Collection from inception to 6 July 2023 (day of search), using the following search terms: aggression, anxiety, depression, borderline personality, narcissism, self-harm, shame, and childhood trauma.We paired each term with "AND (inventory or questionnaire)" to return all studies, distributed by country.We selected these terms because they relate to very common types of mental health disorders and personality dysfunctions and because we could provide national data in the context of broader literature.The classification of countries into high-income and low-and middle-income was based on most recent World Bank data [15].The search results showing the top 20 high-income countries and top 20 lowand middle-income countries are presented in Table 1, Table 2, Table 3, and Table 4.Each table was then entered in the ChatGPT with a prompt to interpret and discuss the results.

National case study in Croatia
Croatia's income classification by the World Bank's data [15] has changed over time.Before 2008, Croatia was classified as an upper-middle-income country.From 2008 to the present, which was also reclassified in 2019, it has been considered a high-income country.
The research was conducted on 111 participants aged between 19 and 62 years, diagnosed with borderline personality disorder (BPD) (ICD-10 code = F60.3 [16]).Twenty-eight (25.2%) of the participants were male, with an average age of 41.2 years (standard deviation (SD) = 10.3), while the remaining 83 (74.8%) were female, with an average age of 36.5 years (SD = 12.9).We categorised age into three groups based on the first and third quartiles of the age distribution within the sample: the youngest group consisted of the first 25% (18-26 years), the adult group included the next 50% (27-49 years), and the oldest group comprised the remaining 25% (50-62 years).The average age was 23.1 (n = 30 (SD = 2.76)) years for the youngest participants, 38.1 (N = 57, SD = 7.2) years for adults, and 55.1 (N = 24, SD = 3.2) years for the oldest group.
In this sample, we tested the reliability of five commonly used questionnaires or inventories (The Childhood Trauma Questionnaire-Short Form (CTQ-SF) [17,18], Pathological Narcissism Inventory (PNI) [19], Experience of Shame Scale (ESS) [20], The Aggression Questionnaire (AQ) [21], and Borderline Personality Questionnaire (BPQ) [22]) using Cronbach's alpha, which is a statistical measure used to assess the internal consistency or reliability of a scale or a set of items in a questionnaire or test.It ranges between 0 and 1, with a higher value indicating greater internal consistency, meaning that the items in the scale are more closely related to each other and are measuring the same concept.Cronbach's alpha was calculated using the standard procedure, taking a score from each scale item and correlating it with the total score for each observation, after which the resulting correlations are compared with the variance for all individual item scores [23].
PAPERS 2023 • Vol.    4).Interestingly, for depression, although the USA was clearly ahead of the rest of the world with 35 019 studies, which is nearly a third of the total global output, the number of studies in some of the LMIC was comparable to the most productive high-income countries (HICs): China (1 st among LMICs) was more productive than Germany (3 rd among HICs), Turkey (2 nd among LMICs) than Spain (8 th among HICs), and Iran and Brazil (3 rd and 4 th among LMICs) than France (11 th in HICs).These results are somewhat surprising.Even Jordan and Bangladesh (19 th and 20 th in LMICs) accounted for 285 and 245 studies, respectively.This shows that inventories and questionnaires for depression have spread all over the world and there is sufficient information to assess it, even in most middle-income countries.Only low-income countries in Africa and South-East Asia showed a real scarcity of output.
We observed a similar situation with anxiety inventories and questionnaires.Again, the USA was first in overall productivity, as its researchers published nearly a quarter of the global output.However, China was again ahead of the 3 rd ranked country among HICs (Australia), while Turkey was ahead of 7 th -ranked HIC (Spain), and Iran and Brazil were comparable to Sweden, France, and Japan.We again found a surprisingly large number of studies in large low-income countries, such as Pakistan (n = 608), Ethiopia (n = 248), and Bangladesh (n = 195).This suggests that depression and anxiety are very well-researched in many places, and although the number of studies per capita is certainly much greater in HICs, as is the number of citations per paper compared LMICs (data not shown), the difference in overall intensity in research activity is not striking -it is clear that research on depression and anxiety in populations living in LMIC is growing.
However, the situation seemed to be much different regarding the six remaining inventories or questionnaires.In all years, there were 9551 studies on aggression, 3185 on borderline personality, 1939 on narcissism, 1815 on self-harm, 1658 on shame, and 5118 on childhood trauma.The early adoption of these questionnaires was much greater in HICs than in LMICs, with a real scarcity among the latter.China, Turkey, Iran, Brazil, Russia, and India were the only LMIC countries that consistently had a number of studies comparable to second-tier productivity among HICs, while other LMICs countries were still lagging behind and were only beginning to adopt these questionnaires.Therefore, there is real scarcity of information on aggression, borderline personality, narcissism, self-harm, shame, and childhood trauma for most of the world's population, particularly in low-resource settings.
There are some specific findings that deserve mention.The United States emerged as the country with the highest number of publications in this area, and was typically followed by the UK.Spain, Germany, Canada, Australia, Netherlands, and Italy are also consistently highly productive countries in this area of research, while France and especially Japan and South Korea were less productive than could be expected by their population sizes and level of development of their research infrastructure.However, Poland had surprisingly high productivity among high-income countries, as the only consistently prominent example from Eastern Europe.Turkey was a positive example of productivity among LMICs; it was the 2 nd most productive LMIC behind China for six out of eight inventories and questionnaires, and was ranked 1 st (ahead of China) for the number of studies on borderline personality.Iran (ranked 2 nd once and 3 rd six more times among LMICs), as well as Ethiopia and Bangladesh (consistently among the top 20 LMICs) were also found to be good examples of high productivity and output.
We found wide variation in the number of research studies utilising validated questionnaires to detect anxiety in different countries.The United States emerged as the country with the highest number of publications.

The use of ChatGPT 3.5 to interpret the results
We presented Table 1, Table 2, Table 3, and Table 4, to ChatGPT 3.5, accompanied with explanations provided in table legends, and asked it to summarise and interpret the results (Box 1).We wanted to explore ChatGPT's ability to analyse and interpret this kind of data, based on a recent demonstration on its capacity to assist in writing a research article [24].We also tested it table-by-table and questionnaire-by-questionnaire (data not shown).ChatGPT wrote many very general sentences typical of scientific papers' results and discussion sections, but it proved rather unreliable.It had difficulties with a correct understanding of the meaning of two separate columns.It did not always classify individual countries in HIC vs. LMIC group correctly, making misclassification errors in both directions.Also, it focused mainly on the HIC situation and was unable to adequately describe the LMIC context.This reflects the shortcomings of its current version in being used to assist in writing scientific papers.Furthermore, the references it provided mainly did not exist, which is another interesting characteristic of its output.Consequently, we recommend caution Box 1.Output from the ChatGPT 3.5 Prompt: "Here are four tables that show the results of bibliometric analysis of the use of the inventory or questionnaire on aggression, anxiety, depression, borderline personality, narcissism, self-harm, shame and childhood trauma.For each of the eight inventories and questionnaires, the tables show the top 20 high-income countries and the top 20 low-and middle-income countries, ranked by the number of published papers.Please analyse and discuss the results."ChatGPT 3.5: "The provided tables present the results of bibliometric analysis on the use of various inventories and questionnaires related to aggression, anxiety, depression, borderline personality, narcissism, self-harm, shame, and childhood trauma.The tables are divided into two sections: one for high-income countries and the other for low-and middle-income countries.The results are based on the number of published papers related to each inventory or questionnaire.

Let's analyze and discuss the key findings from each table: Table 1: Availability of studies on aggression and anxiety The USA and China are the top two countries with the highest number of published papers on both aggression and anxiety. High-income countries dominate the list of top contributors for both aggression and anxiety, with only a few low-and middle-income countries making it to the top 20. Table 2: Availability of studies on depression and borderline personality The USA and China lead in the number of published papers related to depression, while Turkey and Iran follow closely. Similar to the aggression and anxiety inventories, high-income countries are more represented in the top 20 contributors for depression and borderline personality, though Turkey and Brazil from low-and middle-income countries also appear. Table 3: Availability of studies on narcissism and self-harm The USA and Germany are the top contributors to research on narcissism, while the USA and China lead in the self-harm category. There is a relatively more balanced representation of low-and middle-income countries in the top 20 contributors for narcissism and self-harm, with countries like Turkey, Iran, and Malaysia making appearances. Table 4: Availability of studies on shame and childhood trauma
The USA is the leading contributor to research on both shame and childhood trauma, followed by China for shame and Turkey for childhood trauma.The representation of low-and middle-income countries is slightly better in the top 20 contributors for shame and childhood trauma, with countries like Turkey, Brazil, South Africa, and India making appearances.Overall, the analysis reveals a significant concentration of research in high-income countries across all inventories and questionnaires.These countries, particularly the USA and China, dominate the research landscape in almost every category.However, there is evidence of some contribution from low-and middle-income countries as well, particularly in areas like self-harm, narcissism, and childhood trauma.The findings suggest that there is a need to encourage and support more research on mental health and related issues in lowand middle-income countries to gain a comprehensive understanding of these phenomena across diverse populations.

Additionally, collaborative efforts between high-income and low-and middle-income countries could enhance research outcomes and promote global mental health initiatives."
in using ChatGPT to summarise and interpret large amounts of data in a research paper, especially if this requires interpreting complex tables, a need for a link to a third source (e.g. the World Bank's categorisation of countries into HIC vs LMIC), or appropriate and correct referencing.These are areas where we expect ChatGPT to improve in its future iterations.

National case study on the psychometric properties in Croatia
We evaluated five of these inventories and questionnaires in Croatia, to explore if translations are still culturally appropriate and measure the studied traits reasonably well, thus making them applicable in a broad global context.Internal consistency of the CTQ-SF [17,18], with its 25 items, measured by a Cronbach's α coefficient of 0.908, means it could be considered a reliable instrument for assessing overall childhood trauma (Table 5).The Cronbach's α coefficients for the subscales were determined based on their components.All subscales exhibited high reliability, except for physical neglect, which fell slightly below the threshold of reliability.Notably, the CTQ-SF subscales do not follow a normal distribution.Items CTQ10, CTQ16, and CTQ22 were excluded in the process.The subscales of the CTQ-SF can be considered reliable in Croatian context (Table 5).
The PNI self-assessment questionnaire for pathological narcissism [19] consists of 52 items, with responses coded from 0 to 5. The high Cronbach's α coefficient value of 0.963 indicates the questionnaire's high reliability in assessing self-assessment of pathological narcissism.To assess the components of the self-assessment questionnaire for pathological narcissism, 10 subscales were formed (Table 6).The reliability of these aggregated items was slightly reduced, but still remained above the very high threshold of 0.9, as measured by Cronbach's α coefficient.Omitting any of the subscales did not affect the reliability of the self-assessment questionnaire for pathological narcissism.
In assessing the ESS [20], the high Cronbach's α coefficient value of 0.957 indicates its high reliability (Table 7).Three subscales were formed to assess the components of the self-assessment questionnaire for shame experience.The reliability of these aggregated items was slightly reduced, but remained above the very high threshold of 0.920, as measured by Cronbach's α coefficient (Table 7).Omitting any of the items in any of the cases did not alter the reliability of the shame experience scale.The subscale values for the AQ [21] had a Cronbach's α coefficient of 0.922, indicating the scale's high reliability in assessing aggression.To assess the components of the aggression questionnaire and their reliability, four subscales were formed (Table 8).The reliability of these aggregated items has been significantly reduced, especially in the case of verbal aggression (AQ-VA), as its reliability was only 0.658.The reliability of the other subscales had also been reduced, but they remained at an acceptable level.
According to the BPQ [22], the assessment procedure involves summing selected items into nine subscales (Table 9).The items that define each subscale are provided as well.Some of the items are included in the sum in an inverted form, marked additionally with "-I".The reliability of these subscales, measured by Cronbach's α coefficients, ranged from 0.708 to 0.869, indicating a satisfactory level of reliability, from sufficiently to highly reliable.The total score (BPQ-Total) is the sum of all nine subscales.Its reliability, determined by Cronbach's α coefficient of all 80 items, was slightly higher than the reliability of the components, with a value of 0.901 (Table 9).Omitting individual items only slightly affected the reliability.

Exploitativeness Entitlement Rage
We also carried out zero-order correlations between the investigated psychological/psychiatric variables, with an emphasis on the associations of CTQ-SF scales with the rest of the used questionnaires (i.e.PNI, ESS, AQ, BPQ) (Online Supplementary Document).

DISCUSSION
Despite the growing interest in bibliometric analyses of the progress in the fields of psychology and psychiatry, there is still a lack of topical and focused analyses [25][26][27][28][29][30][31][32], particularly those assessing the situation in less developed populations.To our knowledge, this is one of the first studies to compare research intensity in this field between HICs and LMICs.
The distribution of research studies on all the validated inventories and questionnaires, apart from the two most commonly used -i.e. on anxiety and depression, demonstrates a concentration of research activity in high-income countries, primarily the USA and several European nations (Table 1, Table 2, Table 3, and Table 4).The predominance of the USA in terms of publication output suggests a robust research infrastructure and a strong emphasis on studying psychological and psychiatric traits within their population.This may be attributed to many underlying factors, such as the availability of research funding, established research institutions, and a culture of prioritising mental health research.The high number of research studies in the UK, Germany, Spain, Australia, Canada, and the People's Republic of China also indicates significant attention given to studying those traits using validated questionnaires.All of those countries possess supportive research environments, well-established academic networks, and research initiatives that prioritize mental health-related issues.Their sizable research output may also be influenced by their population sizes.
Notably, several low-and middle-income countries, such as Turkey, Spain, Brazil, and China, also contributed a substantial number of research studies.This indicates a growing recognition of the importance of mental health research and the need to understand and address these challenges in LMICs.The distribution of these studies suggests efforts to bridge the research gap between HICs and LMICs in studying psychological and psychiatric traits.However, most LMICs are notably underrepresented in research on the six validated questionnaires that do not include anxiety and depression.The limited number of studies from these countries raises concerns about the lack of research capacity, funding constraints, and potential cultural or language barriers that hinder the adoption of standardised questionnaires in measuring psychological or psychiatric traits.This scarcity of research in low-and middle-income countries emphasises the urgent need for greater investment in mental health research and the development of culturally sensitive tools and frameworks to understand aggression in diverse populations.
Addressing the research gaps in low-and middle-income countries is crucial for several reasons.First, the prevalence and manifestation of psychological and psychiatric issues may differ across cultures and socioeconomic contexts.Therefore, relying solely on research conducted in high-income countries may not fully capture the nuances of psychological and psychiatric issues within diverse populations.Second, a lack of research in these countries hampers the development and implementation of evidence-based interventions and policies tailored to their specific needs.Finally, by increasing research capacity in low-and middle-income countries, it becomes possible to foster international collaborations and exchange knowledge and best practices in addressing aggression globally.
To bridge the research gap, it is essential to prioritise mental health research in underrepresented regions, provide training and resources to local researchers, and establish collaborations between high-income and low-and middle-income countries.It is also crucial to acknowledge that the number of research studies alone does not determine the quality or impact of the research conducted.However, the variation in the number of studies across countries provides valuable insights into the research landscape and highlights the areas where more attention and resources are required.Overall, this analysis underscores the need for a more balanced and inclusive approach to mental health research.

CONCLUSIONS
The findings from this analysis emphasise the need for a comprehensive and global approach to researching narcissism, borderline personality disorder, childhood trauma, aggression, self-harm and shame.By expanding research efforts to a wider range of countries, it becomes possible to gain a more nuanced understanding of these complex traits and develop culturally sensitive interventions to address and mitigate its negative impacts.
To the best of our knowledge, we conducted the first analysis of ChatGPT's potential to summarise the results of bibliometric evaluation of the inventories and questionnaires in psychology and psychiatry.We showed that it is not quite ready yet for the full-scale implementation with this particular aim, but it is possible that it might become more useful in time, with further iterations.
Finally, the addition of a national-level case study in Croatia, where five of the eight inventories and questionnaires were evaluated, showed that they were highly reliable when translated and applied in a different cultural context.Therefore, alongside the instruments detecting anxiety and depression, which have already achieved widespread global use, there is a future for increased and expanded use of the six instruments narcissism, borderline personality disorder, childhood trauma, aggression, self-harm and shame in LMICs, and we would like to encourage the roll-out of this research in the coming years.

Table 1 .
Availability of studies on aggression (left) and anxiety (right) in high-income vs low-and middle-income countries (according to Web of Science Core Collection, all years, search conducted on July 6, 2023)*

Aggression and (inventory or questionnaire) (n = 9551) Anxiety and (inventory or questionnaire) (n = 93 983) High income Low-and middle-income High income Low-and middle-incomeTable 2 .
Availability of studies on depression (left) and borderline personality (right) in high-income vs low-and middle-income countries (according to Web of Science Core Collection, all years, search conducted on July 6, 2023)* Depression and (inventory or questionnaire) (n = 121 136) Borderline personality and (inventory or questionnaire) (n = 3185)

doi: 10.7189/jogh.13.04102 RESULTS Bibliometric analysis of the global use of the inventory or questionnaire on aggression, anxiety, depression, borderline personality, narcissism, self-harm, shame, and childhood trauma The
results of the analysis showed a clear disparity in the number of research studies utilising validated questionnaires to detect various psychological or psychiatric states in different countries.Depression and anxiety were by far the most researched states, with 121 136 and 93 983 studies published across all years,

Table 3 .
Availability of studies on narcissism (left) and self-harm (right) in high-income vs low-and middle-income countries (according to Web of Science Core Collection, all years, search conducted on July 6, 2023)* Narcissism and (inventory or questionnaire) (n = 1939)Self-harm and (inventory or questionnaire) (n = 1815) *Data presented as country: n (%).

Table 4 .
Availability of studies on shame (left) and childhood trauma (right) in high-income vs low-and middle-income countries (according to Web of Science Core Collection, all years, search conducted on July 6, 2023)* Shame and (inventory or questionnaire) (n = 1658) Childhood trauma and (inventory or questionnaire) (n = 511) High income Low-and middle-income High income Low-and middle-income USA: 484 (29.19)People's Republic of China: 70 (4.22)USA: 1722 (33.65)People's Republic of China: 457 (8.

Table 5 .
Subscales of the Childhood Trauma Questionnaire -Short Form

Table 6 .
Subscales of the Pathological Narcissism Inventory

Table 7 .
Subscales of the Experience of Shame Scale

Table 8 .
Subscales of the Aggression Questionnaire

Table 9 .
Subscales of the Borderline Personality Questionnaire