Mailman School Methods in program evaluation questions

Anonymous
timer Asked: Mar 15th, 2019
account_balance_wallet $40

Question Description

Review the following article (ATTACHED):

Ager, A., Akesson, B., Stark, L., Flouri, E., Okot, B., McCollister, F., & Boothby, N. (2011). The impact of the school‐based Psychosocial Structured Activities (PSSA) program on conflict‐affected children in northern Uganda. Journal of child psychology and psychiatry, 52(11), 1124-1133.

Answer the following questions:

  1. What were the original program goals for the intervention?
  2. Sketch out a bare-bones logic model summarizing the inputs, activities, and outcomes. For the purposes of this exercise, skip the outputs and impacts.
  3. How did the authors measure and operationalize their primary outcome? What were the indicators that were used?
  4. What is the study design used?
  5. What are the threats to internal validity common in this design?


Read the case study:

WVI Child-Friendly Spaces (CFS) Intervention at Buramino Camp, Dolo Ado, Ethiopia:

Buramino refugee camp, located along the southern Somali-Ethiopian border, was selected in cooperation with World Vision Ethiopia as the first evaluation site. At the time of the evaluation, the Ethiopian Administration for Refugee and Returnee Affairs (ARRA), the main implementing partner of UNHCR, had organised five camps within two hours of the Dollo Ado transit centre to address the growing needs of Somali refugees who had migrated in response to drought and ongoing conflict in the region. In January 2012, World Vision Ethiopia’s first Child Learning Centre (subsequently referred to through this report as a CFS site) was established, offering a range of services to children aged six to 17 years. The second CFS site started activities later that year, in May 2012. Within the wide continuum of CFS programming approaches, these two sites emphasised functional literacy and numeracy skills. Based on discussions with ARRA and parents of children attending these centres, psychosocial activities, such as drawing, singing, and recreational play, were included, but to a modest degree. On-site counselling and a feeding programme were also offered. To help meet the overwhelming demand in the camp, each centre provided two three-hour sessions for children: a morning session for younger children aged six to 11 and an afternoon session for older children aged 12 to 17. The first CFS provided services for 784 children (500 aged between six and 11 and 284 aged between 12 and 17). The second CFS site provided services for 787 children (506 aged between six and 11 and 281 aged between 12 and 17).

Source: http://www.cpcnetwork.org/wp-content/uploads/2015/04/WV-CU-CFS-Ethiopia-Field-Study-Summary-Report.pdf

Answer the following questions:

  • Develop one evaluation question
  • Consider what study design would best answer your question
  • Decide what data you need to collect

4.Recommend one random and one non-random sampling scheme. (circle your choices)

Types of random sampling

Types of non-random sampling

Simple random sample

Systematic

Proportionate stratified

Disproportionate stratified (Cluster sampling)

Quota sampling

Extreme case sampling

Maximum variation sampling

Homogeneous sampling

Typical case sampling

Snowball sampling

Criterion sampling

5.Discuss the advantages and disadvantages of each scheme that you choose.

6.Why did you choose the random sampling scheme you did?

7.What aspect(s) of the evaluation question would this scheme help you answer?

8.Why did you choose the non-random sampling scheme?

9.What aspect(s) of the evaluation question would this scheme help you answer?

Unformatted Attachment Preview

Journal of Child Psychology and Psychiatry 52:11 (2011), pp 1124–1133 doi:10.1111/j.1469-7610.2011.02407.x The impact of the school-based Psychosocial Structured Activities (PSSA) program on conflict-affected children in northern Uganda Alastair Ager,1 Bree Akesson,2 Lindsay Stark,3 Eirini Flouri,4 Braxton Okot,5 Faith McCollister,3 and Neil Boothby1 1 Mailman School of Public Health, Columbia University, USA; 2Care and Protection of Children in Crisis-Affected Countries Initiative; 3Program on Forced Migration & Health, Mailman School of Public Health, Columbia University, USA; 4Department of Psychology and Human Development, Institute of Education, University of London, UK; 5Child Resilience Program, Save the Children in Uganda Background: Children in northern Uganda have undergone significant psychosocial stress during the region’s lengthy conflict. A Psychosocial Structured Activities (PSSA) program was implemented in 21 schools identified as amongst those most severely affected by conflict-induced displacement across Gulu and Amuru Districts. The PSSA intervention comprised a series of 15 class sessions designed to progressively increase children’s resilience through structured activities involving drama, movement, music and art (with additional components addressing parental support and community involvement). Method: Eight schools were selected by random quota sampling from those schools receiving the PSSA intervention. Two hundred and three children were identified in these schools as being scheduled to receive intervention, and were followed up 12 months later following engagement with PSSA activities. A comparison group comprised 200 children selected from schools that had met inclusion criteria for receipt of intervention, but were not scheduled for intervention coverage until later. Preliminary research used participatory focus group methodology to determine local indicators of child well-being as viewed by parents, teachers, and children respectively. Pre- and post- assessments focused on ratings for each child – by parents, teachers and children – with respect to these indicators. Results: Significant increases in ratings of child well-being were observed in both intervention and comparison groups over a 12-month period. However, the well-being of children who had received the PSSA intervention increased significantly more than for children in the comparison group, as judged by child and parent (but not teacher) report. This effect was evident despite considerable loss-to-follow-up at post-testing as a result of return of many households to communities of origin. Conclusion: General improvement in child well-being over a 12-month period suggests that recovery and reconstruction efforts in Northern Uganda following the onset of peace had a substantive impact on the lives of children. However, exposure to the PSSA program had an additional positive impact on child well-being, suggesting its value in post-conflict recovery contexts. Keywords: Psychosocial, structured activities, children, schools, Northern Uganda, evaluation. Abbreviations: PSSA: Psychosocial Structured Activities. Northern Uganda is in a state of transition after 20 years of conflict between the Lord’s Resistance Army (LRA) and Ugandan government forces. Despite continuing LRA activity in the region and General Joseph Kony’s refusal to sign a Final Peace Agreement in March 2008, the war drew to an effective close in northern Uganda with an ‘Agreement on Cessation of Hostilities’ signed in 2006 between the Government of Uganda and the LRA (Government of Uganda & Lord’s Resistance Army/ Movement, 2006; Machar, 2008). During the conflict between 1.8 and 2 million people were displaced to over 200 camps, located primarily in Gulu, Amuru, Kitgum and Pader districts. These Internally Displaced Persons (IDP) camps were often cramped and lacking in sanitation, and left most families unable Conflict of interest statement: At the time of the study Braxton Okot was serving as Program Manager of the Child Resilience Program. Other authors declare no conflict of interest with respect to the reported study. to support themselves through traditional subsistence agricultural practices (Civil Society Organizations for Peace in Northern Uganda (CSOPNU), 2006). Excess mortality rates exceeded emergency thresholds with estimates as high as 1,000 deaths per week in 2005 among IDPs in the region (World Health Organization, 2005). The conflict disrupted the social and economic development of northern Uganda for a generation. Children were particularly harshly affected by this conflict. One of the defining tactics of the LRA has been its use of children as combatants, porters, and sex slaves, often abducting them from their homes and schools. While many of these formerly abducted children eventually returned to their communities, many returned with significant physical and emotional after-effects of their experience. Amongst abducted youth in the region Annan, Blattman, and Horton (2006) showed that ‘high levels of violence committed and violence experienced are associated  2011 The Authors. Journal of Child Psychology and Psychiatry  2011 Association for Child and Adolescent Mental Health. Published by Blackwell Publishing, 9600 Garsington Road, Oxford OX4 2DQ, UK and 350 Main St, Malden, MA 02148, USA Psychosocial structured activities with conflict-affected children in northern Uganda with substantial increases in emotional distress’ (p. 67). Those who were not abducted were also found to have experienced significant violence, with male youths between 14 and 30 typically reporting having experienced nine traumatic events over their lifetime. The educational system – and thus children’s schooling – has also been significantly affected by the conflict. A 2006 survey indicated that 60% of schools in the affected regions were not functioning, depriving over 250,000 children of an education (CSOPNU, 2006). The Uganda Ministry of Education, in cooperation with a number of international nongovernmental agencies, responded to this situation by expanding places at schools adjacent to IDP camps (and deploying displaced teachers across such schools). In Gulu and Amuru Districts, for example, 40 schools were identified as having major increases in student enrollment, and received assistance in infrastructure development to assist them to respond to such demands (Wilson, 2007). Accommodating large numbers of displaced children, many of whom had been abducted or experienced other extreme conflict-related trauma and virtually all of whom had experienced personal and familial threat and loss linked to the conflict, was acknowledged to have significant impact on the learning environment of the schools. It was in this context that Save the Children proposed to work in a number of these schools with the ‘Child Resilience’ curriculum project, which involved deployment of the Psychosocial Structured Activities (PSSA) intervention. The PSSA intervention builds upon the work of Robert Macy’s Classroom-Based Intervention (CBI) Program, which has been used in a number of settings, including Palestine (Khamis, Macy, & Coignez, 2004) and Sri Lanka (Somasundaram, 2007).1 It is a school-based, multi-phased approach designed to use children’s natural resilience to help them recover from trauma. Its core focus is 15 progressively structured sessions leading from themes of safety and control, through those of awareness and selfesteem, to personal narratives, coping skills, and future planning. These sessions incorporate play therapy, drama, art and movement in an effort to enhance children’s resilience and feelings of stability and security after trauma as they progress emotionally and cognitively over the course of the program. Each session opens with an activity designed to prepare children for the session and then continues into a central activity and cooperative game designed around the session’s theme. The PSSA 1 In some contexts the CBI program is designated as a CampBased or Community-Based Intervention. The CBI Manual was adapted for use in the Uganda context by Susan Garland. Copies of the CBI manual can be obtained by contacting Robert Macy. http://www.traumacenter.org/initiatives/psychosocial. php. 1125 program also seeks to incorporate a community service component, and encourages parental involvement through periodic meetings during which issues related to the children’s experience and the needs of the local community are discussed. There is considerable value in determining a robust methodology to establish the impact of interventions such as PSSA. The PSSA program was implemented by Save the Children in the United States on the Gulf Coast after Hurricane Katrina and in tsunami-affected areas of Indonesia. Few published, rigorous evaluations exist measuring the impact of the program, though reports from the field suggested that it was a popular intervention that garnered the support of parents and school officials (see Jaycox, Morse, Tanielian, & Stein, 2006; Lauten & Lietz, 2008). Encouraged by such reports, Save the Children in Uganda implemented the PSSA program among children attending government schools in northern Uganda beginning in the 2006/2007 school year. Positive anecdotal reports led to the commissioning of an evaluation (Wilson, 2007) that suggested that the program had indeed a positive impact on the lives of children and their caregivers. The intervention was reported to have had a positive impact on attendance rates, student grades, and teacher–student interactions in the classroom. Increases in children’s resilience (defined through consolidation of items elicited through a free-listing exercise with participants in a fashion built upon in the current study) were reported by both children and parents. However, in the absence of baseline measures, the study relied on retrospective judgments of changes since receiving the intervention. It also had no form of control condition, only collecting data from individuals exposed to the intervention. The current study represents an attempt to establish a more rigorous evaluation of the impact of the PSSA program on children’s well-being in this context. In general, there have been few rigorous studies of the implementation of school-based psychosocial activities in conflict and post-conflict environments. An evaluation of a secondary-school-based mental health intervention for children exposed to political violence in Indonesia found some improvement in post-traumatic stress disorder symptoms and hope but did not reduce a variety of other traumatic-stress symptoms (Tol et al., 2008). A study of structured activities in child-focused interventions in Palestine designed to increase children’s resilience found improved emotional and behavioral well-being and parental support, but no increase in hopefulness (Loughry et al., 2006). A cluster randomized trial of a version of the CBI program in Nepal found no main effect of the intervention on mental health but significant impact on social well-being indicators among sub-groups (such as girls’ prosocial behavior and hope amongst older children; Jordans et al., 2010). A recent literature review by Peltonen and Punamaki (2010) analyzed the effectiveness and  2011 The Authors. Journal of Child Psychology and Psychiatry  2011 Association for Child and Adolescent Mental Health. 1126 Alastair Ager et al. theoretical basis of 16 interventions for children exposed to trauma associated with armed conflict, a number of which involved school-based programming on conflict or post-conflict settings (Berger, Pat-Horenczyk, & Gelkopf, 2007; Chase et al., 1999; Layne et al., 2001, 2008; Woodside, Santa Barbara, & Benner, 1999). The studies by Berger et al. (2007) and Layne et al. (2008) were two of only four studies identified as meeting formal criteria for meta-analysis of intervention effectiveness (an analysis that was suggestive of the effectiveness of cognitive-behavioural and resilience-enhancing approaches on relief of post-traumatic symptoms). Otherwise, although reporting some promising findings, Peltonen and Punamaki (2010) suggest that little can be concluded overall from these studies in terms of intervention effectiveness as a result of methodological weaknesses, including the lack of randomization and control conditions. It is clear that implementation in conflict and postconflict settings means that such studies have had to deal with significant methodological challenges in relation to design and inference. For example, although the validity of deploying psychometric measures of well-being developed in other cultures and circumstances is frequently questioned, there are clear challenges for developing rigorous measures de novo in such settings (Ager, Stark, Akesson, & Boothby, 2010a). Promising approaches appear to be characterized by robust specification of replicable data collection and analytical methods which, through effective fieldwork engagement, nonetheless flexibly accommodate to local construction of concepts (e.g., Annan et al., 2006; Bolton et al., 2007). Study designs that provide some form of control or comparison condition – generally utilizing a population ‘waited listed’ for receipt of an intervention as a result of resource constraints – are also now increasingly seen as both feasible and appropriate in such settings (Ager, Ager, Stavrou, & Boothby, 2011). The study reported here sought to build upon such principles in constructing a rigorous methodology for evaluating the impact of the PSSA intervention as its implementation was scaled up in Gulu and Amuru Districts through 2007 to 2009. Methods Participants and setting As noted earlier, education authorities in Gulu and Amuru districts, in collaboration with Save the Children, had identified 40 schools significantly impacted by displacement of children following the conflict as eligible for physical infrastructure upgrading and potential curriculum support through programs such as PSSA. All 40 schools were physically upgraded, but funding and logistical concerns limited initial implementation of PSSA to 21 schools located across Gulu and Amuru’s four sub-districts. Eight intervention schools were selected for inclusion in the study – with a quota of two schools per sub-district – by toss of a coin. Comparison schools were selected from the remaining 19 schools meeting eligibility criteria for support (for all of which PSSA implementation was provisionally scheduled for the following year). Comparison schools were selected and matched with intervention schools on the basis of geographical location, which was seen as appropriate proxy for both exposure to conflict and local resource conditions. On this basis, one school was selected to serve as a comparison for two intervention schools (though different grades were drawn upon, see below). School names are not reported in this article to ensure confidentiality of study participants. From each of the eight intervention schools, the first group of 25 students selected to undergo the PSSA program in the school year 2007/08 was the focus of the evaluation. The program was targeted toward all children who had grown up in the context of stress and conflict, not only formerly abducted children or those with identified psychological difficulties. At each school a ‘rolling program’ of implementation by successive ‘batches’ of student groups was conceived, with all students potentially benefiting. However, in selecting the initial grade level for intervention, teachers were encouraged to consider the number of children in that year group who met certain criteria, such as those who isolated themselves, were frequently absent, seemed particularly stressed, had low self-esteem and/or a violent family background. These criteria were additionally used by teachers to select from the chosen grade the first ‘batch’ of students to receive the intervention. In the matched schools, children from the same grades were chosen for the comparison group, with similar criteria adopted to select specific participants (i.e., following intervention group selection criteria). In total, 403 primary school students took part in the study, 202 girls and 201 boys, with 203 in the intervention group and 200 in the comparison group. At baseline children ranged in age from 7 to 12 years with a mean age of 10.23 (sd = 1.61). Students from primary school year groups 1, 3, 4, 5, and 6 had been selected by schools to receive the intervention, which contributed to the wide range in ages participating in the evaluation. Intervention The intervention was structured according to the three PSSA components noted in the introduction. The classroom component involved 15 highly structured one-hour sessions delivered over the course of five weeks by two regular school teachers trained in the methodology in a residential workshop before the start of the school year. Sessions followed a regular structure of opening, thematic and closing activities, with thematic activities addressing issues of safety and control, self-esteem, thoughts and reactions during danger, resource identification and coping skills. Methods included didactic presentations, reflective exercises, drama and games. The community service component was facilitated by the same teachers, and included helping the sick and elderly, digging boreholes and planting trees. The parental engagement component  2011 The Authors. Journal of Child Psychology and Psychiatry  2011 Association for Child and Adolescent Mental Health. Psychosocial structured activities with conflict-affected children in northern Uganda was principally implemented through periodic discussion meetings with parents facilitated by Save the Children in Uganda (SCiUg) Child Resilience Project staff who made regular supervisory visits to implementation schools. In original program documentation these components had related to distinct program goals of ‘enhancing school performance’, ‘increasing participation in social activities and self-help community initiatives’ and ‘improving care and emotional support received in families’. However, through inception the program elements were increasingly seen as connected to the core goal of enhancing child well-being and ‘resilience’. With this construct the central focus of change, the study focused on operationalization of this concept. Measures Researchers adopted a modified form of brief ethnographic interviewing (Hubbard, 2008) to determine local understandings of child-well-being and resilience. Prior to the evaluation, each of the eight intervention schools was visited, and discussions held with children, parents (and other main caregivers) and teachers. In each school, children who had engaged in the first ‘batch’ of PSSA activities at the school (in academic year 2006/7) engaged in a ‘free-listing’ exercise, identifying characteristics of a resilient child who demonstrated good well-being (after the methodology outlined in Ager et al., 2011). Responses from the 120 children engaged in this manner were pooled and independently grouped by seven raters with local language and cultural knowledge. The preliminary categories of response were then, through discussion, consolidated into the six indicators listed in Table 1. An analogous procedure was followed with 136 parents from across the eight schools – and with 42 teachers during the course of their training and supervision in implementation of PSSA – to yield the list of indicators listed in Table 1. Data collection 1127 day for data collection at each school, the research team gathered the selected children outside or in an empty classroom. The purpose of the evaluation was discussed and the meaning of each of the indicators reviewed in Acholi. Chi ...
Purchase answer to see full attachment

Tutor Answer

EngDuke1993
School: Rice University

Attached.

Running head: PSYCHOLOGY

1

PSYCHOLOGY
Name:
Institution affiliation:
Date:

PSYCHOLOGY

2

PSYCHOLOGY
What were the original program goals for the intervention?
The program aimed to progressively enhance the resilience of children through structured
activities such as drama, movement, music and art. The PSSA intervention aimed at promoting
children’s feeling of stability and security after experiences of trauma as the children improve
emotionally and cognitively as the program continues. The program’s goal is also to introduce a
community service aspect and promote parental contribution through periodic forums where there
are discussions of significant matters related to the experience of children.
Sketch out a bare-bones logic model summarizing the inputs, activities, and outcomes. For
the purposes of this exercise, skip the outputs and impacts.

Paren
ts

Teac
hers

Inputs

Children

Drama

Music

aa

c

Activities

Movement

Area
Recovery &
Reconstruct
ion
Outcomes

Child-Well
Being

How did the authors measure and operationalize their primary outcome? What were the
indicators that were used?

PSYCHOLOGY

3

The researchers randomly sampled eight schools under the PSSA intervention in which
two hundred and three children received intervention. After twelve months, there was a follow-up
on the children involved in PSSA activities. The parents, teachers, and children gave their ratings
on each child before and after the evaluation The authors used indicators of child well-being from
the perspective of parents, teachers, and children.
What is the study design used?
The researchers used a randomized controlled trial
What are the threats to internal validity common in this design?
Validity may require multiple sites which may become difficult to manage. Trials that
require long run period may become irrelevant by the time of publication of the results. The
allocation of participants may sometimes be predictable which introduces the problem of bias.
Results may not also be an indicator of real life situation. There is also a question of ethics with
regard to randomizing the participants.
Develop one evaluation question
How do the conditions of refugees affect their functional literacy and numeracy skills in
the refugee camps?
Consider what study des...

flag Report DMCA
Review

Anonymous
Top quality work from this guy! I'll be back!

Brown University





1271 Tutors

California Institute of Technology




2131 Tutors

Carnegie Mellon University




982 Tutors

Columbia University





1256 Tutors

Dartmouth University





2113 Tutors

Emory University





2279 Tutors

Harvard University





599 Tutors

Massachusetts Institute of Technology



2319 Tutors

New York University





1645 Tutors

Notre Dam University





1911 Tutors

Oklahoma University





2122 Tutors

Pennsylvania State University





932 Tutors

Princeton University





1211 Tutors

Stanford University





983 Tutors

University of California





1282 Tutors

Oxford University





123 Tutors

Yale University





2325 Tutors