Impact factor (WEB OF SCIENCE - Clarivate)

2 year: 4.413 | 5 year: 5.197

Articles

Conditional cash transfers and adolescent mental health in Brazil: Evidence from the 2004 Pelotas Birth Cohort

Carolina Ziebold1, Cristiane Silvestre Paula1, Iná S Santos2,3, Fernando C Barros4, Tiago N Munhoz5, Crick Lund6,7, David McDaid8, Ricardo Araya6, Annette Bauer8, Emily Garman7, A-La Park8, Annie Zimmerman6,9, Philipp Hessel10, Mauricio Avendaño10,11, Sara Evans-Lacko8, Alicia Matijasevich3,12

1 Programa de Pós-graduação em Distúrbios do Desenvolvimento. Universidade Presbiteriana Mackenzie, São Paulo, Brazil
2 Postgraduate Program in Pediatrics and Child Health, School of Medicine, Pontifical Catholic University of Rio Grande do Sul, Porto Alegre, Brazil
3 Postgraduate Program in Epidemiology, Federal University of Pelotas, Pelotas, Brazil
4 Postgraduate Program in Health and Behavior, Universidade Católica de Pelotas, Pelotas, Rio Grande do Sul, Brazil
5 Faculty of Psychology, Federal University of Pelotas, Pelotas, Brazil
6 Centre for Global Mental Health, Health Services and Population Research Department, Institute of Psychiatry, Psychology and Neuroscience, King’s College London.
7 Alan J Flisher Centre for Public Mental Health, Department of Psychiatry and Mental Health, University of Cape Town, Cape Town, South Africa
8 Care Policy and Evaluation Centre, Department of Health Policy, London School of Economics and Political Science, London, UK
9 Department of Global Health & Social Medicine, King’s College London, London, UK
10 Escuela de Gobierno Alberto Lleras Camargo, Universidad de Los Andes, Bogotá, Colombia
11 Center for Primary Care and Public Health (Unisanté), Department of Epidemiology and Health Systems, University of Lausanne, Lausanne, Switzerland
12 Departamento de Medicina Preventiva, Faculdade de Medicina FMUSP, Universidade de São Paulo, SP, Brazil

Share:

Share on facebook
Facebook
Share on twitter
Twitter
Share on linkedin
LinkedIn
Abstract

Background

Young people living in poverty are at higher risk of mental disorders, but whether interventions aimed to reduce poverty have lasting effects on mental health has not been well established. We examined whether exposure to Brazil’s conditional cash transfers programme (CCT), Bolsa Família (BFP), during childhood reduces the risk of mental health problems in early adolescence.

Methods

We used data from 2063 participants in the 2004 Pelotas Birth Cohort study. Propensity score matching (PSM) estimated the association between BFP participation at age 6 and externalising problems (Strengths and Difficulties Questionnaire – SDQ and violent behaviour) and socio-emotional competencies (Development and Well-Being Assessment questionnaire, and the Nowick-Strickland Internal-External Scale) at age 11.

Results

PSM results suggest that programme participation at age of six was not significantly associated with externalising problems (P = 0.433), prosocial behaviour (P = 0.654), violent behaviour (P = 0.342), social aptitudes (P = 0.281), positive attributes (P = 0.439), or locus of control (P = 0.148) at the age of 11 years.

Conclusions

Participation in BFP during childhood was not associated with improved or worsened mental health in early adolescence. While we cannot fully discard that findings may be due to adverse selection, results suggest that CCTs alone may not be sufficient to improve mental health outcomes and would be prudent to assess whether mental health interventions as an addition to CCTs may be helpful.

Print Friendly, PDF & Email

Poverty is a complex multidimensional concept that goes beyond lack of income, and involves health, education and living standard deprivations [1], among others. Poverty is closely linked to mental health problems during childhood and adolescence [2]. In particular, poverty during childhood and adolescence has been linked to higher risk of externalising behavioural problems, including attention-deficit/hyperactivity, oppositional defiant and conduct disorders [3]. Similarly, externalising problems experienced early in life can reduce educational and employment opportunities and increase the risk of lifetime poverty [4,5]. Mechanisms underlying this association include food insecurity [6], exposure to violence [7], diminished social support [8], dysfunctional coping strategies, and diminished social competencies [9,10]. There is also growing evidence suggesting that poverty may exacerbate caregiver’s depression which in turn may increase offspring’s mental health problems [9,11,12]. An important, yet untested hypothesis is that interventions that address poverty during childhood and adolescence may help improve the mental health of young people living in poverty [13].

Conditional cash transfers programmes (CCTs) have emerged as one of the most widely used anti-poverty policies in Low- and Middle-Income Countries (LMICs). CCTs provide cash payments on the condition that families meet specific requirements related to children´s school attendance, vaccinations or health check-ups [14]. The majority of evidence in this area has focused on evaluating CCTs impact on poverty reduction [15], child health, nutrition [16], and educational outcomes [17], finding overall positive effects of CCTs on these outcomes. Although mental health problems are not directly targeted by CCTs, some studies have examined their potential impact on the mental health of young people living in poverty [1820]. The rationale of these studies is that because poverty may be linked to mental health problems, CCTs provide an opportunity to test whether improvements in living and social conditions associated with poverty (food security, parental supervision, access to health and education services) may exert a positive effect on mental health [13,1921]. Prior studies have examined the effect of CCTs on depression and anxiety among youth (15 to 24 years) [22], and on behavioural problems and socio-emotional competencies, among children (<10 years) [1921]. Whilst some find modest improvements [1922] others show no effect [21]. To our knowledge, there has been no research on the association between CCTs, externalising problems and socio-emotional competencies among adolescents living in LMICs. Examining this issue in a LMIC context is important. On the one hand, externalising problems are more common among groups with the lowest socioeconomic status and are associated with considerable social and economic consequences over the lifetime due to, for example, higher violent offending, drug use, teenage pregnancy and school dropout [2325]. On the other hand, it has been suggested that improved socio-emotional competencies (eg, interpersonal skills, social aptitudes and prosocial behaviour) may prevent the development of externalising behaviours [26].

Brazil’s Bolsa Família programme (BFP) began in 2003 and is nowadays the CCT with the largest number of beneficiaries in the world, providing financial assistance to approximately 14 million families [27]. BFP aims to both alleviate poverty by providing a cash benefit to families and reduce the inter-generational transmission of poverty by conditioning receipt of these transfers on human capital investments (school attendance, immunization, pre-natal care) [27]. Programme eligibility is based on per capita household income and household composition (pregnant/breast feeding mothers; children aged 0-17). Monthly payments are made to mothers (average US$35 per month in 2020) [27].

Most BFP`s evaluation studies have examined impacts of the programme on poverty [28,29], food security and nutrition [30,31], infant and child mortality [32,33], as well as health [34] and education services [35,36]. Only one study has examined impacts on mental health, finding a positive association of BFP with psychosocial functioning among children and adolescents (7-17 years) in an urban slum community in Northeast Brazil [34]. Evidence also suggests that an increase in BFP coverage was associated with a reduction in suicide rates, especially among women in municipalities where high coverage of the programme was maintained for 3 years or more [37].

To advance our understanding of the association between CCTs and mental health outcomes among children and adolescents, we analysed the association of BFP participation with externalising problems and socio-emotional competencies (social aptitudes, positive attributes, and locus of control). Our study is the first in analysing this relationship and unique by linking exposure to the programme relatively early in life (age 6 years) to mental health outcomes in early adolescence (age 11 years). We hypothesized that BFP at the age of six may reduce adolescent`s externalising problems and increase socio-emotional competencies at the age of 11.

METHODS

Study site

We used data from the 2004 Pelotas Birth Cohort study. Pelotas is a city in the south of Brazil, with an estimated population of 342 405 inhabitants in 2019. Major economic activities are commerce, services and industry [38]. In 1982, the city’s gross domestic product was 9% above the national mean, but by 2004 it was 39% lower, reflecting the relative impoverishment of the city [39]. A previous study from the 2004 Pelotas Birth Cohort found that BFP coverage – the percentage of poor people in the cohort receiving the benefit – increased from 29% in 2004 to 63% in 2010, whereas the targeting of the programme – the percentage of eligible people among the beneficiaries – remained constant at about 37% [40]. Nowadays, BFP coverage in Pelotas city is 54% [41].

Participants

The 2004 Pelotas Birth Cohort is a population-based birth cohort of children born in Pelotas from January 1 to December 31, 2004. All live births of women residing in the urban area of the city during that year were recruited for the study (n = 4231, nonresponse rate at recruitment <1%). A detailed description of the methodology can be found elsewhere [42]. After perinatal assessment, the cohort was followed up at the ages of 3, 12, and 24 months, as well as at 4, 6 and 11 years, with follow-up rates between 87% and 96%. In the present study we included cohort participants who: 1) had complete data on sociodemographic characteristics gathered during the perinatal and 4-year assessments; 2) were living in households from the four first poorest quintiles of wealth at the age of 4; we excluded participants from the richest quintile as these participants were in principle not eligible for the BFP and only a small fraction (7.3%) reported receiving it; 3) had complete data on both BFP participation status at the age of 6 and mental health outcomes at age 11 (6.3% lost to follow-up). Study participants’ flowchart (Figure S1 in the Online Supplementary Document) and characteristics related with attrition are presented in Table S1 in the Online Supplementary Document.

Exposure: Bolsa Família benefit at the age of 6

During follow-up when children reached the age of 6, mothers or other main caregivers were asked: “Do you receive the BFP?”. We defined BFP beneficiaries as those who responded “yes” to this question.

Eligibility: BFP eligibility is based on per capita household income and household composition, according to the information provided by families registered in the Unified Registry for Social Programmes of the Federal Government. In 2010, families with a monthly per capita income of up to R$70 (about US$42 at 2010 values) were classified as extremely poor and were eligible for the programme [27]. Families with per capita incomes between R$70.01 and R$140.00 were classified as poor and could be beneficiaries if they contained children between 0-6 years of age, or children and adolescents enrolled in school (7-17 years), or pregnant or breastfeeding woman (14-44 years) [27]. The number of families participating in BFP is also limited by quotas per municipality that are calculated on the basis of estimates of the number of families living in poverty in each municipality [43]. The entry of families into the programme follows the following order: first, families considered to be priorities (quilombola [ethnic-racial groups]; indigenous; recyclable material collectors; families in situations of child labour; and with members freed from slave-like labour); second, families with the lowest monthly income per capita; third, families with the highest number of children and adolescents from 0 to 17 years old [43].

Benefit: In 2010, extremely poor families received an unconditional amount of R$68 (about US$40 in 2010). Poor families received only benefits according to household composition. Both extremely poor and poor families received additional benefits dependent on the number of children in the family (0-15 years) and whether the mother was pregnant or breastfeeding (up to five additional benefits of R$22 [about US$14] per family: R$110 in 2010) [27]. An additional amount of R$33 (about US$20) was received by families with adolescents enrolled in school aged 16-17 years (up to two per family: R$66). Monthly payments are made to mothers using an electronic payment card [27].

Conditionalities: For both extremely poor and poor families, additional benefits are dependent on compliance with health and education conditionalities: 1) Children under the age of 7 are expected to comply with Brazil’s childhood immunization schedule and to make growth and development monitoring visits with the frequency recommended by the Ministry of Health [27], 2) Children between the ages of 6-17 years are expected to be enrolled in school and maintain a minimum daily school attendance of 85% (75% for ages 16-17), 3) Pregnant women have to comply with prenatal care [27], 4) Breastfeeding mothers (14-44 years) have to comply with health care controls. Schools and health centres are responsible for reporting adherence [27].

Outcome measures

Externalising problems at the age of 11 years

At subsequent follow-up when children reached 11 years, mothers/main caregivers were interviewed by trained psychologists using the Strengths and Difficulties Questionnaire (SDQ) [44], an instrument designed to assess mental health problems in children and adolescents, validated in Brazil [45]. The conduct problems, hyperactivity and prosocial behaviour subscales of the SDQ were used to measure adolescent externalising problems [46]. Each subscale contains 5 items asking about the child’s behavioural problems or strengths, as for example ‘Steals from home, school or elsewhere’ (conduct), ‘Restless, overactive, cannot stay still for long’ (hyperactivity), or ‘Considerate of other people’s feelings’ (prosocial). Items can be answered as “Not True”, “Somewhat True” and “Certainly True”, receiving scores from 0 to 2 for each answer. The SDQ externalising score ranges from 0 to 20 and is the sum of the conduct and hyperactivity scales, where greater scores are indicative of greater externalising problems [46]. The prosocial behaviour score ranges from 0 to 10. Reversed scores were computed, where greater scores are indicative of less prosocial behaviour [46].

Violent behaviour was evaluated using a self-completed confidential questionnaire developed by the 1993 Pelotas Birth Cohort authors [47] where adolescents responded to a yes or no question: “During the last year, did you participate in a fight in which someone was injured?”.

Socio-emotional competencies at age 11

Three socio-emotional competencies were tested: Social aptitudes were assessed by the Social Aptitude Scale (SAS) [48], a 10-item scale designed to evaluate individual social abilities that form part of the Development and Well Being Assessment (DAWBA) [49] that has been validated in Brazil [50]. Parents were asked to rate the ability of their child, when compared with children of the same age, to read social and emotional cues rapidly in complex situations in order to guide socially skilled behaviour in ten different social situations. Total scores range between 0 and 40, where higher scores represent more social aptitudes.

Positive attributes were assessed using the Youth Strengths Inventory (YSI) version for parents, which also forms part of the DAWBA [4951]. The first part of the YSI asks how applicable various descriptions are to the child (for example being generous, affectionate, caring), while the second part asks about positive behaviours (for example being good with friends, helpful at home, and polite). Each part has 12 items scored on a three-point Likert scale, with scores ranging from 0 to 48, where higher scores represent greater positive attributes.

Locus of Control (LoC) describes the degree to which individuals perceive that outcomes of events in their lives result from their own behaviours (internal LoC), or from forces that are external to themselves (external LoC) [52]. Adolescents’ LoC were evaluated using the 12-item version of the Nowick-Strickland Internal-External Scale (CNSIE) [52]. Each item is dichotomous, with yes-no responses, and a total score is derived by summing scores for all items. Higher scores indicate a more external LoC, while lower scores indicate greater internal LoC. The Portuguese version has shown good psychometric properties [53].

Covariates

Maternal characteristics at childbirth: maternal schooling (number of completed years of formal education); maternal age; marital status (single mother or living with a partner); and number of live children, all gathered at perinatal interview.

Child characteristics at childbirth: sex (male and female); gestational age; weight; and skin colour (white, black, and mixed-race).

Household wealth score at 4-year follow-up: Household characteristics were evaluated according to the National Wealth Index questionnaire [54]. Principal component analysis was used to consolidate the following household characteristics into one measure representing the household’s wealth [55]: household head’s education; number of bedrooms; number of bathrooms (with shower and toilet); number of television sets; number of vehicles; ownership (yes/no) of assets: radio, refrigerator, DVD or video tape, freezer/duplex refrigerator; washing machine, microwave, telephone line, computer and air conditioner [55]. The wealth index allows classification of households into five wealth strata according to reference cut-off values for each municipality. Using Pelotas-specific reference values, wealth indexes were stratified as follows: 20-280 = first strata (poorest households), 281-367 = second strata, 368-475 = third strata, 476-618 = fourth strata, and 619-1478 = fifth strata (wealthiest households, who were excluded from the analysis). Even though BFP eligibility is based on per capita household income, it has been suggested [40] that the first strata of the wealth index constructed as explained above is a good proxy of BFP eligibility. It represents the poorest population, those that would be potential beneficiaries of the programme in the municipality, being less subject to temporal variability and errors of information than household income [40]. Figure S2 in the Online Supplementary Document shows the rates of receipt of BFP into each wealth strata.

Maternal depressive symptoms at 6-year follow-up: Maternal depressive symptoms during the past 7 days were assessed using the Edinburgh Postnatal Depression Scale (EPDS) [56]. The EPDS is a 10-item scale originally created for the identification of postpartum depression. Total scores range from 0 to 30, with higher scores indicating more severe depressive symptoms. Maternal depressive symptoms was defined using a cut-off of ≥13 that was validated to screen major depressive episodes among adults from the general population in Brazil [57].

Ethical considerations

All procedures were approved by the Research Ethics Committees of the Universidade Federal de Pelotas and the Universidade de São Paulo, School of Medicine. Mothers or legal guardians of all subjects gave written informed consent. At 11 years of age, the adolescents also gave written informed assent.

Data analysis

We first analysed the frequency and factors related to attrition between follow up at 6 and 11 years. Due to the low attrition rate (6.3%) and its lack of relationship with factors under study (Table S1 in the Online Supplementary Document); we performed a complete-case analysis. For main tests we adopted a level of significance of 5%.

Treatment effects estimation

To assess the effect of BFP on adolescents’ externalising problems and socio-emotional competencies, we used propensity score matching (PSM) (Stata 16, College Station, TX) [58,59]. PSM allows mitigation of bias associated with differences in the distributions of observed covariates in BFP beneficiaries (treatment group) and non-beneficiaries (comparison group). PSM matches treatment and comparison groups according to the subject’s similarity on estimated treatment probabilities, known as propensity scores [5860].

Propensity score (PS): For this study, the PS is the probability of being a BFP beneficiary given observed covariates for the child, mother, and household. We selected BFP participation covariates using probit regression models and we also estimated covariates of our outcomes of interest using generalised linear models (Tables S2 and S3 in the Online Supplementary Document). The PS was estimated according to variables strongly associated to both BFP participation and outcomes [59]: household wealth index, maternal characteristics (age at childbirth, number of living children, maternal schooling, and maternal depressive symptoms), and child characteristics (sex and skin colour).

Matching: To reduce bias, we used 1:1 nearest-neighbour matching (NNM), where each beneficiary is matched with the non-beneficiary who has the closest PS, with replacement and a caliper (specification of the maximum distance at which two observations are a potential match) of 0.005 [58,59]. NNM is considered the most effective method for selecting individuals for follow-up [61]. After each model, we performed tests of balance to check that each covariate did not significantly differ between the treated and comparison groups (non-significant mean differences) [5860]. In addition, we present density-distribution plots of propensity scores for BFP and non-BFP participants. These plots allow us to visually check the overlap or common support condition which ensures that respondents with the same values on the PS have a positive probability of being both BFP participants and nonparticipants.

Average treatment effect on the treated: PSM allows estimation of the average treatment effect on the treated (ATT), where the treatment effect is computed by taking the average of the difference between the outcome of BFP beneficiaries and the outcome of eligible non-beneficiaries [58,60].

Sensitivity analysis: We tested two alternative matching methods: kernel matching (Epanechnikov type), where all BFP beneficiaries were matched with a weighted average of all non-beneficiaries with weights that were inversely proportional to the distance between the PS of beneficiaries and non-beneficiaries, and radius matching (with caliper of 0.005) which uses as many comparison cases (not only the nearest-neighbour) as are available within the caliper [62]. We also performed subgroup analyses to test whether the results of the PSM would vary among participants in the first (poorest) and fourth (richest) wealth strata. Finally, we also present the results of simple regression models testing the association of BFP and adolescent mental health adjusting by the same covariates included in the PSM.

RESULTS

Sample differences before and after propensity score matching (PSM)

The total sample included 2063 participants, where 873 (42.3%) were BFP beneficiaries at the age of 6 years and 1190 (57.7%) were non-beneficiaries. Table 1 illustrates sample characteristics. 20.4% of beneficiaries fell within the first wealth strata (poorest), with most concentrated in the second (28.1%) and third (33.1%) wealth strata, whereas 18.4% were in the fourth wealthiest strata. In terms of BFP coverage, 61.2% of cohort participants from the first wealth strata were beneficiaries (Figure S2 in the Online Supplementary Document). Before PSM, beneficiaries presented higher mean scores on SDQ externalising problems (β = 0.14, SE = 0.04, P < 0.001) and external locus of control (β = 0.06, SE = 0.01, P < 0.001) compared with non-beneficiaries.

Table 1.  Sample description: Sociodemographic characteristics and means of externalising problems and socio-emotional competences, Pelotas 2004 Birth Cohort. (N = 2063)

BFP – Bolsa Família Programme, N – number of observations, SD – standard deviation, SDQ – Strengths and Difficulties Questionnaire

*Conduct plus hyperactivity items. Higher scores indicate greater externalising problems.

†Prosocial subscale is reverse scored. Higher scores indicate less prosocial behaviour.

‡Any 12-month physical aggression that caused injuries, N = 2007 adolescent report.

Table 2 presents the probit model with the covariates predicting BFP participation. Beneficiaries were more likely than non-beneficiaries to be non-white, to reside in a household with lower wealth, to have mothers with higher odds of presenting depressive symptoms, lower education, lower age at childbirth, and with more children. We did not find differences according to child sex (β = 0.04, SE = 0.06, P = 0.474), weight at birth (β = 0.000003, SE = 0.0001, P = 0.961), and maternal marital status (β = -0.001, SE = 0.08, P = 0.986) between the two groups. However, as child sex was significantly associated with our outcomes of interest (Tables S2-S3 in the Online Supplementary Document) we included this variable in the PSM.

Table 2.  Probit regression model of participation in the BFP

BFP – Bolsa Família Programme, Coef. – coefficient in the Probit regression model, SE – standard deviation, CI – confidence interval, PseudoR2 – pseudo-R squared statistics

Table 3 shows covariates balance before and after PSM using the NNM technique. After PSM the mean differences between groups were in all cases non-significant, meaning that PSM succeeded in removing differences in the covariates’ distributions. Figure S3 in the Online Supplementary Document shows the overlap of the PS between the matched beneficiaries and non-beneficiaries, meaning that, according to the observed covariates, each matched pair had equal probability of being eligible for BFP.

Table 3.  Covariates balance before and after nearest-neighbour propensity score matching

BFP – Bolsa Família Programme

Table 4 presents the number of cases included (on support) and excluded (off support) in the PSM and the percentage of bias reduced after PSM. The matched sample included more than 96% of beneficiaries and all non-beneficiaries. Bias reduction ranged between 63.3% to 80.9%.

Table 4.  Nearest-neighbour propensity score matching: common support and bias reduction*

BFP – Bolsa Família Programme, N – number of observations, SDQ – Strengths and Difficulties Questionnaire

*Matched by sex, ethnicity, maternal schooling, household wealth score, maternal age at childbirth, maternal depressive symptoms, and number of live children at birth.

†Conduct plus hyperactivity items.

‡Prosocial subscale is reverse scored.

§Any 12-month physical aggression that caused injuries, N = 2007 adolescent report.

‖N = 1959 adolescent report.

BFP participation, externalising problems and socio-emotional competencies

Using the nearest-neighbour matched sample we calculated the ATT of BFP participation at age 6 on externalising problems and socio-emotional competencies at age of 11. Table 5 shows that BFP was not significantly associated with SDQ externalising problems at age of 11 (P = 0.433), or with prosocial (P = 0.654) or violent (P = 0.342) behaviour.

Table 5.  Effect of BFP participation at 6 years on mental health at 11 years among 2004 Pelotas Birth Cohort participants*

BFP – Bolsa Família Programme, SDQ – Strengths and Difficulties Questionnaire, SE – standard error

*Treatment effects estimator: Nearest-neighbor Propensity score matching. Matched by sex, ethnicity, maternal schooling, household wealth score, maternal age at childbirth, maternal depressive symptoms, and number of live children at birth.

†Conduct plus hyperactivity items. Higher scores indicate greater externalising problems.

‡Prosocial subscale is reverse scored. Higher scores indicate less prosocial behaviour.

§Any 12-month physical aggression that caused injuries, N = 2007 adolescent report.

‖N = 1959 adolescent report.

As shown in Table 5, we did not find any significant association of BFP with socio-emotional competencies – social aptitudes (P = 0.281), positive attributes (P = 0.439), or locus of control (P = 0.148) – at the age of 11.

Sensitivity analysis

Tables S5-S7 and Figures S4-S5 in the Online Supplementary Document show that kernel and radius matching methods had similar results as those presented with the NNM in terms of ATT (non-significant association between BFP and all outcomes assessed) and overlap assumption, even showing higher bias reduction. Because NNM is considered the most effective method for selecting individuals for follow-up [61], we decided to present its results in the main analysis nonetheless. Stratified analysis also showed a non-significant association between BFP and all outcomes assessed among those in the poorest and wealthiest strata (Tables S8-S9 in the Online Supplementary Document). Finally, simple regression models also showed non-significant associations between BFP and adolescent mental health after adjusting for the covariates included in the PSM (Table S10 in the Online Supplementary Document).

DISCUSSION

We analysed the longitudinal association between BFP at age 6 and externalising problems and socio-emotional competencies at age 11 among a population-based birth cohort of young people in Brazil. We found that BFP participation at the age of 6 was not associated with externalising problems, prosocial and violent behaviour or socio-emotional competencies in early adolescence (age 11 years).

Our findings differ from the positive effect of BFP on psychosocial functioning that Shei et al [34] found among young people in Northeast Brazil. It could be argued that their cross-sectional study design, measures (the SF-10 Health Status questionnaire), data analysis (propensity score weighting) and population (children and adolescents (7-17 years) living in an urban slum in Salvador de Bahía) significantly differed from our study. Further studies are needed to verify if the impact of BFP on mental health outcomes varies according to the time frame (short-term vs longer-term), type of outcome (externalising, and internalising disorders or positive well-being) and the age group. It is particularly important to replicate these studies in different regions of the country to understand if cultural and socioeconomic backgrounds, quality of services and implementation features of the programme, can exert different impacts of BFP on mental health in young people.

Our results also differ from those found by Ozer et al [20] who examined the cross-sectional effect of the Mexico’s Oportunidades CCT on behaviour problems among children (4-6 years) living in poor rural areas. They found a 10% decrement in aggressive and oppositional symptoms (assessed by the Behaviour Problems Index – BPI). Although, like our study, they used propensity score matching to define comparison groups, our longitudinal design, measures, and population differ considerably from theirs. In Nicaragua, Macours et al [21] performed a randomised controlled trial (RCT) to evaluate the effect of the Atención a Crisis CCT on behavioural problems (assessed by the BPI) and social competencies (assessed by the Denver Developmental Screening Test) among children (3-7 years) nine months after the programme began and two years after its end. They found a significant positive effect of the programme on social competencies but, as in our study, they found non-significant effect on behavioural problems.

In spite of crude associations showing that beneficiaries would have higher levels of externalising problems and external locus of control, our results using PSM suggest that BFP is not associated with worsened or improved externalising mental health problems and socio-emotional competencies among low-income adolescents living in Pelotas. Our interpretation of this null association between BFP and adolescent mental health is consistent with the fact that young people living in poverty are exposed to risk factors for externalising problems, as community and domestic violence [7], diminished social support [8], and caregiver’s depression [9,12], those may not be properly addressed by the programme. On the other hand, BFP includes no specific mental health interventions, such as preventive interventions aiming to develop socio-emotional competencies, or screening and referral for mothers and children presenting with mental disorders. It is also possible that the pressures of conditionalities for the BFP may bring stress to families as observed in another CCT [63], reducing the potential positive benefit that poverty reduction may have on mental health.

A secondary finding of this study was the association between BFP receipt and maternal depressive symptoms. As in other studies [11,12,64], we also found that these depressive symptoms were related with child behavioural problems. These results highlight the need for further research to evaluate potential complementary mental health interventions that can be delivered among families participating in the programme. For instance, an RCT in Liberia [65] compared the effect of CTs vs CTs plus Cognitive Behaviour Therapy (CBT) on antisocial behaviour among vulnerable young males. The combined arm of cash plus CBT showed a dramatic reduction in antisocial behaviour one year later, while CTs on their own had no effect.

As stated earlier, mental health problems are not only more likely among people living in poverty [2], they can also reduce the life-chances and increase the risk of lifetime poverty [4,5]. In consequence, it would be prudent to assess whether mental health interventions may be included as part of poverty reduction strategies [66].

Strengths and limitations

The present study contributes to understanding the association between CCTs and early adolescent mental health in LMICs, using individual longitudinal data. We analysed data from a large population-based birth cohort with low attrition rates, using validated instruments and exploiting longitudinal assessments. In addition, we used PSM to address potential selection and confounding. However, our findings need to be interpreted with caution, and a number of important limitations need to be considered. First, the interpretation of our treatment effect estimates from PSM analyses relies on the assumption that there are no unmeasured confounding variables producing biased effect estimates. For instance, many factors correlated with poor mental health may also be associated with participation into BFP, biasing our estimates against positive mental health effects. Second, our study was limited to evaluate outcomes at only one follow-up and future cohort waves are needed to evaluate the stability of the association between BFP participation and mental health during youth. Third, we did not have data for many variables that could allow for an improved understanding of the BFP effects as amount of benefit, compliance with conditionalities, use and quality of health, educational and social services, and years of receipt of BFP. Nevertheless, we were able to verify (Table S11 in the Online Supplementary Document) that the results did not change when we excluded from the analyses 149 controls (13%) those became beneficiaries at age of 11. We also found that ‘losing’ (46%) vs ‘maintaining’ (54%) the benefit at the age of 11 was not associated with outcomes assessed (Table S12 in the Online Supplementary Document).

Implications for research and policy

The results of the current study reinforce the need to further evaluate mental health outcomes of CCT in LMICs [13,67], and to identify which aspects of the programmes (eg, conditionalities, amount of money, length of benefit, and complementary health, social and educational interventions) can help to prevent or reduce the severity of mental health problems among young people. Our study also raises the need for replication in other Brazilian regions and to systematically use mixed-methods evaluations to verify if the BFP is improving mental health outcomes in young people.

Given that households living in poverty are exposed to preventable risks to mental health, greater attention needs to be paid to the mental health of programme beneficiaries. Our research provides an opportunity to advise policymakers to also consider incorporating mental health interventions into their poverty reduction strategies. Furthermore, evaluation of evidence-based complementary tailored interventions that can be delivered to prevent mental health problems among BFP beneficiaries, specifically mothers, children and adolescents, would be recommended [65].

Additional material

Online Supplementary Document

Acknowledgements

All procedures were approved by the Research Ethics Committee of the Universidade Federal de Pelotas and the Research Committee of the Universidade de São Paulo, School of Medicine. Mothers or legal guardians of all subjects gave written informed consent. At 11 years of age, the adolescents also gave written informed assent. The authors would like to thank the participating families and staff who collaborated in the various stages of the 2004 Pelotas Birth Cohort study.

[1] Funding: CZ, CP, CL, RA, MA, AB, EG, PH, AP, DM, AZ, and SE-L received a grant from the Economic and Social Research Council (ESRC, ES/S001050/1) for the project: “Poverty reduction, mental health and the chances of young people: understanding mechanisms through analyses from 6 low- and middle-income countries” (UK Research and Innovation Global Challenges Research Fund). This article is based on data from the 2004 Pelotas Birth Cohort study, conducted by the Postgraduate Programme in Epidemiology of the Federal University of Pelotas, Brazil, with the support of the Brazilian Public Health Association (ABRASCO). Previous phases of the study were supported by the World Health Organization (Grant number 03014HNI), National Support Programme for Centres of Excellence (PRONEX) (Grant number 04/0882.7), Brazilian National Research Council (CNPq) (Grant number 481012-2009-5; 484077-2010-4; 470965-2010-0; 481141-2007-3), Brazilian Ministry of Health (Grant number 25000.105293/2004-83) and Children’s Pastorate. The 11-year follow-up was also funded by the Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP; grant number 2014/13864-6). IS, FCB and AM are supported by the CNPq.

[2] Authorship contributions: CZ, CP, and AM participated in the conception and analytic strategies of the current study. CZ did the analysis and drafted the manuscript. AM, IS, FB, and TM, developed and conducted the 2004 Pelotas Birth-Cohort data collection. AM, CP, CL, RA, MA, AB, EG, PH, AP, DM, AZ, and SE-L made substantial contributions to the interpretation of the data and revision of the text. All authors approved the final version of the manuscript.

[3] Competing interests: The authors completed the ICJME Declaration of Interest Form (available upon request from the corresponding author) and declare no conflicts of interest.

references

[1] United Nations. Ending Poverty. Available: https://www.un.org/en/sections/issues-depth/poverty/index.html. Accessed: 31 July 2020.

[2] C Lund, A Breen, AJ Flisher, R Kakuma, J Corrigall, and JA Joska. Poverty and common mental disorders in low- and middle- income countries: A systematic review. Soc Sci Med. 2010;71:517-28. DOI: 10.1016/j.socscimed.2010.04.027. [PMID:20621748]

[3] F Reiss. Socioeconomic inequalities and mental health problems in children and adolescents: A systematic review. Soc Sci Med. 2013;90:24-31. DOI: 10.1016/j.socscimed.2013.04.026. [PMID:23746605]

[4] M Knapp, D King, A Healey, and C Thomas. Economic outcomes in adulthood and their associations with antisocial conduct, attention deficit and anxiety problems in childhood. J Ment Health Policy Econ. 2011;14:137-47. [PMID:22116171]

[5] Richards M, Abbott R. Childhood mental health and life chances in post-war Britain. Insights from three national birth cohort studies. London: The Smith Institute, Sainsbury Centre for Mental Health & Unison; 2009.

[6] N Slopen, G Fitzmaurice, DR Williams, and SE Gilman. Poverty, Food Insecurity, and the Behavior for Childhood Internalizing and Externalizing Disorders. J Am Acad Child Adolesc Psychiatry. 2010;49:444-52. [PMID:20431464]

[7] J Murray, DR de C Cerqueira, and T Kahn. Crime and violence in Brazil: Systematic review of time trends, prevalence rates and risk factors. Aggress Violent Behav. 2013;18:471-83. DOI: 10.1016/j.avb.2013.07.003. [PMID:24027422]

[8] RG Wight, AL Botticello, and CS Aneshensel. Socioeconomic Context, Social Support, and Adolescent Mental Health: A Multilevel Investigation. J Youth Adolesc. 2006;35:109-20. DOI: 10.1007/s10964-005-9009-2

[9] B Devenish, M Hooley, and D Mellor. The Pathways Between Socioeconomic Status and Adolescent Outcomes: A Systematic Review. Am J Community Psychol. 2017;59:219-38. DOI: 10.1002/ajcp.12115. [PMID:28127777]

[10] P Kim, C Neuendorf, H Bianco, and GW Evans. Exposure to Childhood Poverty and Mental Health Symptomatology in Adolescence: A Role of Coping Strategies. Stress Health. 2016;32:494-502. DOI: 10.1002/smi.2646. [PMID:26234956]

[11] E Raposa, C Hammen, P Brennan, and J Najman. The Long-Term Effects of Maternal Depression: Early Childhood Physical Health as a Pathway to Offspring Depression. J Adolesc Health. 2014;54:88-93. DOI: 10.1016/j.jadohealth.2013.07.038. [PMID:24060574]

[12] AC Bernard-Bonnin. Maternal depression and child development. Paediatr Child Health. 2004;9:575-98. DOI: 10.1093/pch/9.8.575. [PMID:19680490]

[13] C Lund, M De Silva, S Plagerson, S Cooper, D Chisholm, and J Das. Poverty and mental disorders: breaking the cycle in low-income and middle-income countries. Lancet. 2011;378:1502-14. DOI: 10.1016/S0140-6736(11)60754-X. [PMID:22008425]

[14] Fizbein A, Schady N. The Economic Rationale for Conditional Cash Transfers. Conditional Cash Transfers: reducing present and future poverty. 2009.

[15] F Bastagli, J Hagen-Zanker, L Harman, V Barca, G Sturge, and T Schmidt. The Impact of Cash Transfers: A Review of the Evidence from Low- and Middle-income Countries. J Soc Policy. 2019;48:569-94. DOI: 10.1017/S0047279418000715

[16] JE Cooper, T Benmarhnia, A Koski, and NB King. Cash transfer programs have differential effects on health: A review of the literature from low and middle-income countries. Soc Sci Med. 2020;247:112806. DOI: 10.1016/j.socscimed.2020.112806. [PMID:32086171]

[17] S García and JE Saavedra. Educational Impacts and Cost-Effectiveness of Conditional Cash Transfer Programs in Developing Countries: A Meta-Analysis. Rev Educ Res. 2017;87:921-65. DOI: 10.3102/0034654317723008

[18] LC Fernald, PJ Gertler, and LM Neufeld. Role of cash in conditional cash transfer programmes for child health, growth, and development: an analysis of Mexico’s Oportunidades. Lancet. 2008;371:828-37. DOI: 10.1016/S0140-6736(08)60382-7. [PMID:18328930]

[19] LC Fernald, PJ Gertler, and LM Neufeld. 10-year effect of Oportunidades, Mexico’s conditional cash transfer programme, on child growth, cognition, language, and behaviour: a longitudinal follow-up study. Lancet. 2009;374:1997-2005. DOI: 10.1016/S0140-6736(09)61676-7. [PMID:19892392]

[20] EJ Ozer, LCH Fernald, JG Manley, and PJ Gertler. Effects of a Conditional Cash Transfer Program on Children’s Behavior Problems. Pediatrics. 2009;123:e630-7. DOI: 10.1542/peds.2008-2882. [PMID:19336354]

[21] K Macours, N Schady, and R Vakis. Cash Transfers, Behavioral Changes, and Cognitive Development in Early Childhood: Evidence from a Randomized Experiment. Am Econ J Appl Econ. 2012;4:247-73. DOI: 10.1257/app.4.2.247

[22] K Kilburn, L Ferrone, A Pettifor, R Wagner, FX Gómez-Olivé, and K Kahn. The Impact of a Conditional Cash Transfer on Multidimensional Deprivation of Young Women: Evidence from South Africa’s HTPN 068. Soc Indic Res. 2020;151:865-95. DOI: 10.1007/s11205-020-02367-y. [PMID:33029038]

[23] MT Greenberg and MA Lippold. Promoting Healthy Outcomes Among Youth with Multiple Risks: Innovative Approaches. Annu Rev Public Health. 2013;34:253-70. DOI: 10.1146/annurev-publhealth-031811-124619. [PMID:23297659]

[24] Scott S. Oppositional and conduct disorders. 6th Editio. Rutter’s Child and Adolescent Psychiatry. 6th Editio. Chichester: John Wiley & Sons, Ltd; 2015.

[25] PJ Piotrowska, CB Stride, SE Croft, and R Rowe. Socioeconomic status and antisocial behaviour among children and adolescents: A systematic review and meta-analysis. Clin Psychol Rev. 2015;35:47-55. DOI: 10.1016/j.cpr.2014.11.003. [PMID:25483561]

[26] MH Bornstein, C-S Hahn, and OM Haynes. Social competence, externalizing, and internalizing behavioral adjustment from early childhood through early adolescence: Developmental cascades. Dev Psychopathol. 2010;22:717-35. DOI: 10.1017/S0954579410000416. [PMID:20883577]

[27] Ministério da Cidadania. Secretaría Especial do Desenvolvimento Social. Brasil. Bolsa Família. Available: http://mds.gov.br/assuntos/bolsa-familia. Accessed: 12 April 2020.

[28] CR Peña, DS Pinheiro, PHM Albuquerque, and LM Fernandes. A eficácia das transferências de renda: as tendências da desigualdade antes e depois do Programa Bolsa Família. Rev Adm Pública. 2015;49:889-914. DOI: 10.1590/0034-7612135532

[29] JM de Araujo. Alves J do A, Besarria C da N. O impacto dos gastos sociais sobre os indicadores de desigualdade e pobreza nos estados brasileiros no período de 2004 a 2009. Rev Econ Contemp. 2013;17:249-75. DOI: 10.1590/S1415-98482013000200003

[30] F de CC Oliveira and RMM Cotta. Sant’Ana LF da R, Priore SE, Franceschini S do CC. Programa Bolsa Família e estado nutricional infantil: desafios estratégicos. Cien Saude Colet. 2011;16:3307-16. DOI: 10.1590/S1413-81232011000800030

[31] R Paes-Sousa, LMP Santos, and ÉS Miazaki. Effects of a conditional cash transfer programme on child nutrition in Brazil. Bull World Health Organ. 2011;89:496-503. DOI: 10.2471/BLT.10.084202. [PMID:21734763]

[32] D Rasella, R Aquino, CA Santos, R Paes-Sousa, and ML Barreto. Effect of a conditional cash transfer programme on childhood mortality: a nationwide analysis of Brazilian municipalities. Lancet. 2013;382:57-64. DOI: 10.1016/S0140-6736(13)60715-1. [PMID:23683599]

[33] A Shei. Brazil’s Conditional Cash Transfer Program Associated With Declines In Infant Mortality Rates. Health Aff (Millwood). 2013;32:1274-81. DOI: 10.1377/hlthaff.2012.0827. [PMID:23836744]

[34] A Shei, F Costa, MG Reis, and AI Ko. The impact of Brazil’s Bolsa Família conditional cash transfer program on children’s health care utilization and health outcomes. BMC Int Health Hum Rights. 2014;14:10 DOI: 10.1186/1472-698X-14-10. [PMID:24690131]

[35] MCS Santos and LR Delatorre. Ceccato M das GB, Bonolo P de F. Programa Bolsa Família e indicadores educacionais em crianças, adolescentes e escolas no Brasil: revisão sistemática. Cien Saude Colet. 2019;24:2233-47. DOI: 10.1590/1413-81232018246.19582017. [PMID:31269182]

[36] AA Simões and R Sabates. The contribution of Bolsa Família to the educational achievement of economically disadvantaged children in Brazil. Int J Educ Dev. 2014;39:141-56. DOI: 10.1016/j.ijedudev.2014.07.012

[37] FJO Alves, DB Machado, and ML Barreto. Effect of the Brazilian cash transfer programme on suicide rates: a longitudinal analysis of the Brazilian municipalities. Soc Psychiatry Psychiatr Epidemiol. 2019;54:599-606. DOI: 10.1007/s00127-018-1627-6. [PMID:30456426]

[38] Instituto Brasileiro de Geografia e Estatística – IBGE. Cidades. Pelotas. Available: https://cidades.ibge.gov.br/brasil/rs/pelotas/panorama. Accessed: 4 August 2020.

[39] AD Bertoldi, FC Barros, PRC Hallal, GI Mielke, PD Oliveira, and MFS Maia. Trends and inequalities in maternal and child health in a Brazilian city: methodology and sociodemographic description of four population-based birth cohort studies, 1982–2015. Int J Epidemiol. 2019;48:i4-15. DOI: 10.1093/ije/dyy170. [PMID:30883654]

[40] KH Schmidt, J Labrecque, IS Santos, A Matijasevich, FC Barros, and AJD Barros. Focus and coverage of Bolsa Família Program in the Pelotas 2004 birth cohort. Rev Saude Publica. 2017;51:22 DOI: 10.1590/s1518-8787.2017051006792. [PMID:28380211]

[41] Ministério da Cidadania. Bolsa Família e Cadastro Único no seu Município. Available: https://aplicacoes.mds.gov.br/sagirmps/bolsafamilia/relatorio-completo.html. Accessed: 19 November 2020.

[42] IS Santos, AJ Barros, A Matijasevich, MR Domingues, FC Barros, and CG Victora. Cohort Profile: The 2004 Pelotas (Brazil) Birth Cohort Study. Int J Epidemiol. 2011;40:1461-8. DOI: 10.1093/ije/dyq130. [PMID:20702597]

[43] Ministério do Desenvolvimento Social e Combate à Fome. Portaria GM/MDS No 341. 2008. Available: http://www.mds.gov.br/webarquivos/licitacao/organismos_internacionais/anexo13_5_portaria341_procedimentos_operacionais.pdf. Accessed: 19 November 2020.

[44] R Goodman. The Strengths and Difficulties Questionnaire: A Research Note. J Child Psychol Psychiatry. 1997;38:581-6. DOI: 10.1111/j.1469-7610.1997.tb01545.x. [PMID:9255702]

[45] B Fleitlich, P Cortázar, and R Goodman. Questionário de capacidades e dificuldades (SDQ). Infanto Rev Neuropsiquiatr Infanc Adolesc. 2000;8:44-50.

[46] A Goodman, DL Lamping, and GB Ploubidis. When to Use Broader Internalising and Externalising Subscales Instead of the Hypothesised Five Subscales on the Strengths and Difficulties Questionnaire (SDQ): Data from British Parents, Teachers and Children. J Abnorm Child Psychol. 2010;38:1179-91. DOI: 10.1007/s10802-010-9434-x. [PMID:20623175]

[47] CL Araújo, AMB Menezes, M de FA Vieira, MB Neutzling, H Gonçalves, and L Anselmi. The 11-year follow-up of the 1993 Pelotas (Brazil) birth cohort study: methods. Cad Saude Publica. 2010;26:1875-86. DOI: 10.1590/S0102-311X2010001000003. [PMID:20963284]

[48] EB Liddle, MJ Batty, and R Goodman. The Social Aptitudes Scale: an initial validation. Soc Psychiatry Psychiatr Epidemiol. 2009;44:508-13. DOI: 10.1007/s00127-008-0456-4. [PMID:18979221]

[49] R Goodman, T Ford, H Richards, R Gatward, and H Meltzer. The Development and Well-Being Assessment: Description and initial validation of an integrated assessement of child and adolescent psychopathology. J Child Psychol Psychiatry. 2000;41:645-55. DOI: 10.1111/j.1469-7610.2000.tb02345.x. [PMID:10946756]

[50] B Fleitlich-Bilyk and R Goodman. Prevalence of Child and Adolescent Psychiatric Disorders in Southeast Brazil. J Am Acad Child Adolesc Psychiatry. 2004;43:727-34. DOI: 10.1097/01.chi.0000120021.14101.ca. [PMID:15167089]

[51] P Vidal-Ribas, R Goodman, and A Stringaris. Positive attributes in children and reduced risk of future psychopathology. Br J Psychiatry. 2015;206:17-25. DOI: 10.1192/bjp.bp.114.144519. [PMID:25359925]

[52] S Nowicki and BR Strickland. A locus of control scale for children. J Consult Clin Psychol. 1973;40:148-54. DOI: 10.1037/h0033978

[53] A Barros, F Neto, and J Barros. Avaliação do locus de controlo e do locus de causalidade em crianças e adolescentes. Rev Port Educ. 1992;5:55-64.

[54] AJD Barros and CG Victora. Indicador econômico para o Brasil baseado no censo demográfico de 2000. Rev Saude Publica. 2005;39:523-9. DOI: 10.1590/S0034-89102005000400002. [PMID:16113899]

[55] F Ewerling and AJD Barros. After 10 years, how do changes in asset ownership affect the Indicador Econômico Nacional? Rev Saude Publica. 2017;51:10 DOI: 10.1590/s1518-8787.2017051006517. [PMID:28273231]

[56] JL Cox, JM Holden, and R Sagovsky. Detection of Postnatal Depression. Br J Psychiatry. 1987;150:782-6. DOI: 10.1192/bjp.150.6.782. [PMID:3651732]

[57] A Matijasevich, TN Munhoz, BF Tavares, APPN Barbosa, DM da Silva, and MS Abitante. Validation of the Edinburgh postnatal depression scale (EPDS) for screening of major depressive episode among adults from the general population. BMC Psychiatry. 2014;14:284 DOI: 10.1186/s12888-014-0284-x. [PMID:25293375]

[58] StataCorp. STATA Base Reference Manual Release 16 ®. StataCorp LLC. 2019.

[59] JL Adelson, DB McCoach, HJ Rogers, JA Adelson, and TM Sauer. Developing and Applying the Propensity Score to Make Causal Inferences: Variable Selection and Stratification. Front Psychol. 2017;8:1413 DOI: 10.3389/fpsyg.2017.01413. [PMID:28861028]

[60] PC Austin. The use of propensity score methods with survival or time-to-event outcomes: reporting measures of effect similar to those used in randomized experiments. Stat Med. 2014;33:1242-58. DOI: 10.1002/sim.5984. [PMID:24122911]

[61] EA Stuart. Matching Methods for Causal Inference: A Review and a Look Forward. Stat Sci. 2010;25:1-21. DOI: 10.1214/09-STS313. [PMID:20871802]

[62] SO Becker and A Ichino. Estimation of Average Treatment Effects Based on Propensity Scores. Stata J Promot Commun Stat Stata. 2002;2:358-77. DOI: 10.1177/1536867X0200200403

[63] S Baird, J de Hoop, and B Özler. Income shocks and adolescent mental health. J Hum Resour. 2013;48:370-403.

[64] SH Goodman, MH Rouse, AM Connell, MR Broth, CM Hall, and D Heyward. Maternal Depression and Child Psychopathology: A Meta-Analytic Review. Clin Child Fam Psychol Rev. 2011;14:1-27. DOI: 10.1007/s10567-010-0080-1. [PMID:21052833]

[65] C Blattman, JC Jamison, and M Sheridan. Reducing Crime and Violence: Experimental Evidence from Cognitive Behavioral Therapy in Liberia. Am Econ Rev. 2017;107:1165-206. DOI: 10.1257/aer.20150503. [PMID:29553237]

[66] A Bauer, E Garman, D McDaid, M Avendano, P Hessel, and Y Díaz. Integrating youth mental health into cash transfer programmes in response to the COVID-19 crisis in low-income and middle-income countries. Lancet Psychiatry. 2021;8:340-6. DOI: 10.1016/S2215-0366(20)30382-5. [PMID:33549174]

[67] A Zimmerman, E Garman, M Avendano-Pabon, R Araya, S Evans-Lacko, and D McDaid. The impact of cash transfers on mental health in children and young people in low-income and middle-income countries: a systematic review and meta-analysis. BMJ Glob Health. 2021;6:e004661. DOI: 10.1136/bmjgh-2020-004661. [PMID:33906845]

Correspondence to:
Carolina Ziebold
Programa de Pós-graduação em Distúrbios do Desenvolvimento
Universidade Presbiteriana Mackenzie
Rua da Consolação
896 – prédio 28, 1° andar – Consolação
São Paulo – CEP 01302-907
São Paulo
carolaziebold@gmail.com