High-Risk Youth (HRY)
Study Design
The National Cross-site Evaluation is a five-year study that examines selected high risk youth, female adolescent and replication initiatives funded in 1994 and 1995. A total of approximately 50 programs will be evaluated, with approximately 6000 participants and 4000 comparison youth. Data will be collected by EMT from both treatment and comparison youth (ages 9 to 18) at four points:
Data Analysis will focus on assessing the effectiveness of High Risk Youth interventions in attaining intended outcomes generally, and in establishing effectiveness of alternative intervention strategies across programs. Specific comparisons of individual programs will not be conducted, nor will programs be specifically identified in the cross-site findings. Models and program designs will be reported on in the findings.
The Process Component
The process component of the Cross-Site Evaluation is designed to provide focused information of clear relevance to assessing the effectiveness of alternative intervention strategies and the impacts of program implementation factors. Unlike many previous cross-site studies, process data collection is specifically targeted on providing program-level measures of service type, intensity and duration for statistical analysis, and on identifying contextual factors important to interpreting study findings. Process information is being collected through annual three-day site visits, and continuous collection of detailed attendance data to support measures of program dosage.
The Outcome Component
The outcome component of the Cross-site Evaluation is designed to determine the degree to which the High Risk Youth programs reduce risk, increase resiliency, and reduce ATOD use in participating youth as compared to similar youth who did not receive CSAP-funded High Risk Youth services. Outcome findings will distinguish between immediate outcomes (at program exit), short-term outcomes (6 months after program exit) and longer term outcomes (18 months after program exit). The outcome analysis will contribute to prevention policy and program design by identifying differences in effectiveness between program strategies and service mixes, by testing the relation between prevention dosage (duration and intensity of program services) and outcomes, and by specifying the role of risk and resiliency factors in producing positive ATOD outcomes. Click here to jump to the latest update on the High Risk Youth project and information on the latest outcomes information.
Working With Local Programs
The Cross-site Evaluator has collaborated with each of the local programs. A local data collection consultant has been hired by the evaluator to collect data at each program site. All program directors and local evaluators have been invited to attend annual workshops on research and evaluation in order to strengthen the collaboration between local sites and national evaluation, and to inform sites of emerging evaluation issues. Youth survey data collected in collaboration with local evaluators will be made available to respective programs for use in their own local evaluations. Click here to find out when the next annual meeting will occur.
The Data Collection Instruments
Individual level data will be collected using the National Youth Survey, an instrument that has been carefully designed to measure program impacts ATOD use and risk/resiliency factors that have been shown to affect ATOD use. Most importantly, the instrument includes measures of:
The TEG
A Technical Expert Group (TEG) has been formed to provide input on all facets of evaluation design, instrumentation, and implementation. The eleven-member TEG includes nationally known researchers and practitioners who have made strong contributions in prevention program design and implementation, the evaluation of prevention programs, and evaluation research methods.
Overview
The Cross-Site Evaluation of the High Risk Youth Demonstration Program
is being conducted by EMT Associates under to the Office of Scientific
Analysis at the Center for Substance Abuse Prevention (CSAP). This
evaluation assesses the effectiveness of the two cohorts funded by CSAP
in 1994 and 1995. The High Risk Youth demonstration grantees have been
funded to implement initiatives to prevent and reduce the use of
alcohol, tobacco, and other drugs among at-risk youth. These programs
provide a wide array of culturally sensitive interventions that are
often specifically designed for youth in certain age and gender groups
and settings (e.g. schools or housing projects). Approximately 50 High
Risk Youth programs have been selected to participate in the cross-site
evaluation. This evaluation is designed to assess the implementation and
effectiveness of common program interventions implemented across the
selected programs. More specifically, there are three objectives for the
evaluation:
- Assess the impact of the programs in preventing and/or reducing ATOD use among youth at high risk for such behavior;
- Assess the effectiveness of specific intervention strategies in reducing selected risk factors or enhancing protective factors on which the specific programs are focused; and
- Document the process of service delivery and program implementation.
- This is the first of CSAP's cross-site studies of high risk youth programs that will use a common core data collection instrument.
Study Design
The National Cross-site Evaluation is a five-year study that examines selected high risk youth, female adolescent and replication initiatives funded in 1994 and 1995. A total of approximately 50 programs will be evaluated, with approximately 6000 participants and 4000 comparison youth. Data will be collected by EMT from both treatment and comparison youth (ages 9 to 18) at four points:
- Program Entry (baseline)
- Program Exit
- 6 Month Follow-up
- 18 Month Follow-up
Data Analysis will focus on assessing the effectiveness of High Risk Youth interventions in attaining intended outcomes generally, and in establishing effectiveness of alternative intervention strategies across programs. Specific comparisons of individual programs will not be conducted, nor will programs be specifically identified in the cross-site findings. Models and program designs will be reported on in the findings.
The Process Component
The process component of the Cross-Site Evaluation is designed to provide focused information of clear relevance to assessing the effectiveness of alternative intervention strategies and the impacts of program implementation factors. Unlike many previous cross-site studies, process data collection is specifically targeted on providing program-level measures of service type, intensity and duration for statistical analysis, and on identifying contextual factors important to interpreting study findings. Process information is being collected through annual three-day site visits, and continuous collection of detailed attendance data to support measures of program dosage.
The Outcome Component
The outcome component of the Cross-site Evaluation is designed to determine the degree to which the High Risk Youth programs reduce risk, increase resiliency, and reduce ATOD use in participating youth as compared to similar youth who did not receive CSAP-funded High Risk Youth services. Outcome findings will distinguish between immediate outcomes (at program exit), short-term outcomes (6 months after program exit) and longer term outcomes (18 months after program exit). The outcome analysis will contribute to prevention policy and program design by identifying differences in effectiveness between program strategies and service mixes, by testing the relation between prevention dosage (duration and intensity of program services) and outcomes, and by specifying the role of risk and resiliency factors in producing positive ATOD outcomes. Click here to jump to the latest update on the High Risk Youth project and information on the latest outcomes information.
Working With Local Programs
The Cross-site Evaluator has collaborated with each of the local programs. A local data collection consultant has been hired by the evaluator to collect data at each program site. All program directors and local evaluators have been invited to attend annual workshops on research and evaluation in order to strengthen the collaboration between local sites and national evaluation, and to inform sites of emerging evaluation issues. Youth survey data collected in collaboration with local evaluators will be made available to respective programs for use in their own local evaluations. Click here to find out when the next annual meeting will occur.
The Data Collection Instruments
Individual level data will be collected using the National Youth Survey, an instrument that has been carefully designed to measure program impacts ATOD use and risk/resiliency factors that have been shown to affect ATOD use. Most importantly, the instrument includes measures of:
- Risk Factors (Personal Factors, Family Environment, Neighborhood Environment, Social Influences)
- Resiliency Factors (School and Family Bonding, Self-Concept, Self-Control, Self-Efficacy, Positive Outlook, Confidence and Cooperation)
- ATOD Attitudes and Use (Tobacco, Alcohol, Marijuana, Inhalants, Other Drugs)
The TEG
A Technical Expert Group (TEG) has been formed to provide input on all facets of evaluation design, instrumentation, and implementation. The eleven-member TEG includes nationally known researchers and practitioners who have made strong contributions in prevention program design and implementation, the evaluation of prevention programs, and evaluation research methods.
HRY Study Publications
http://www.health.org/govpubs/FO36/