SAMSHA National Registry of Evidence-Based Programs and Practices (NREPP)

The National Registry of Evidence-based Programs and Practices (NREPP) is an online, searchable database of interventions designed to promote mental health or to prevent or treat substance abuse and mental disorders. The registry is funded and administered by the Substance Abuse and Mental Health Services Administration (SAMHSA), part of the U.S. Department of Health and Human Services. The goal of the Registry is to encourage wider adoption of evidence-based interventions and to help those interested in implementing an evidence-based intervention to select one that best meets their needs.

Overview

Logo: SAMHSA's National Registry of Evidence-based Programs and Practices (NREPP), www.nrepp.samhsa.gov

In the behavioral health field, there is an ongoing need for researchers, developers, evaluators, and practitioners to share information about what works to improve outcomes among individuals coping with, or at risk for, mental disorders and substance abuse. Discussing how this need led to the development of NREPP, Brounstein, Gardner, and Backer (2006)^[1] write:

It is important to note that not all prevention programs work. Still other programs have no empirically based support regarding their effectiveness. […] Many others have empirical support, but the methods used to generate that support are suspect. This is another reason to highlight the need for and use of scientifically defensible, effective prevention programs. These are programs that clearly demonstrate that the program was well implemented, well evaluated, and produced a consistent pattern of positive results.

The focus of NREPP is on delivering an array of standardized, comparable information on interventions that are evidence based, as opposed to identifying programs that are “effective” or ranking them in effectiveness. Its peer reviewers use specific criteria to rate the quality of an intervention’s evidence base as well as the intervention’s suitability for broad adoption. In addition, NREPP provides contextual information about the intervention, such as the population served, implementation history, and cost data to encourage a realistic and holistic approach to selecting prevention interventions.^[2]

As of 2010, the interventions reviewed by NREPP have been implemented successfully in more than 229,000 sites, in all 50 States and more than 70 countries, and with more than 107 million clients.^[3] Versions of ura review process and rating criteria have been adopted by the National Cancer Institute^[4] and the Administration on Aging.

The information NREPP provides is subject to certain limitations.^[5] It is not an exhaustive repository of all tested mental health interventions; submission is a voluntary process, and limited resources may preclude the review of some interventions even though they meet minimum requirements for acceptance.^[6] The NREPP home page prominently states that “inclusion in the registry does not constitute an endorsement.”

Submission process

NREPP holds an open submission period that runs November 1 through February 1. For an intervention to be eligible for a review, it must meet four minimum criteria:^[7]

1. The intervention has produced one or more positive behavioral outcomes (p ≤ .05) in mental health, mental disorders, substance abuse, or substance use disorders use among individuals, communities, or populations.
2. Evidence of these outcomes has been demonstrated in at least one study using an experimental or quasi-experimental design.
3. The results of these studies have been published in a peer-reviewed journal or other professional publication, or documented in a comprehensive evaluation report.
4. Implementation materials, training and support resources, and quality assurance procedures have been developed and are ready for use by the public.

Once reviewed and added to the Registry, interventions are invited to undergo a new review 4 or 5 years after their initial review.

Review process

The NREPP review process consists of two parallel and simultaneous review tracks, one that looks at the intervention’s Quality of Research (QOR) and another that looks at the intervention’s Readiness for Dissemination (RFD). The materials used in a QOR review are generally published research articles, although unpublished final evaluation reports can also be included. The materials used in an RFD review include implementation materials and process documentation, such as manuals, curricula, training materials, and written quality assurance procedures.

The reviews are conducted by expert consultants who have received training on NREPP's review process and rating criteria. Two QOR and two RFD reviewers are assigned to each review. Reviewers work independently, rating the same materials. Their ratings are averaged to generate final scores.

While the review process is ongoing, NREPP staff work with the intervention’s representatives to collect descriptive information about the intervention, such as the program goals, types of populations served, and implementation history.

The QOR ratings, given on a scale of 0.0 to 4.0, indicate the strength of the evidence supporting the outcomes of the intervention. Higher scores indicate stronger, more compelling evidence. Each outcome is rated separately because interventions may target multiple outcomes (e.g., alcohol use, marijuana use, behavior problems in school), and the evidence supporting the different outcomes may vary. The QOR rating criteria are:

1. Reliability of measures
2. Validity of measures
3. Intervention fidelity
4. Missing data and attrition
5. Potential confounding variables
6. Appropriateness of analysis

The RFD ratings, also given on a scale of 0.0 to 4.0, indicate the amount and quality of the resources available to support the use of the intervention. Higher scores indicate that resources are readily available and of high quality. These ratings apply to the intervention as a whole. The RFD criteria are:

1. Availability of implementation materials
2. Availability of training and support resources
3. Availability of quality assurance procedures

Reviewers

QOR reviewers are required to have a doctoral-level degree and a strong background and understanding of current methods of evaluating prevention and treatment interventions. RFD reviewers are selected from two categories: direct services experts (including both providers and consumers of services), or experts in the field of implementation. Direct services experts must have previous experience evaluating prevention or treatment interventions and knowledge of mental health or substance abuse prevention or treatment content areas.

Products and publications

NREPP publishes an intervention summary for each intervention it adds to the Registry. The summaries, which are accessed through the Registry’s search engine,^[8] contain the following standardized information:

• A brief description of the reviewed intervention, including targeted goals and theoretical basis
• Study populations (age, gender, race/ethnicity)
• Study settings and geographical locations
• Implementation history
• Funding information
• Comparative evaluation research conducted with the intervention
• Adaptations
• Adverse effects
• List of studies and materials reviewed
• List of outcomes
• Description of measures and key findings for each outcome
• Research design of the studies reviewed
• Quality of Research and Readiness for Dissemination ratings
• Reviewer comments (Strengths and Weaknesses)
• Costs
• Replication studies
• Contact information

NREPP also maintains an online Learning Center. Offerings include learning modules on implementation and preparing for NREPP submission; a research paper on evidence-based therapy relationships; and links to screening and assessment tools for mental health and substance use.

Predecessor system

The registry originated in 1997^[9] and has gone through several changes since then. The predecessor to today's NREPP was the National Registry of Effective Prevention Programs (later renamed the National Registry of Effective Programs and Practices), which was developed by SAMHSA's Center for Substance Abuse Prevention as part of the Model Programs initiative. Procedures under this earlier registry were developed to review, rate, and designate programs as Model, Effective, or Promising.^[10] Based on extensive input from scientific communities, service providers, expert panels, and the public, the procedures were revised. Reviews using the new NREPP system began in 2006, and the redesigned Web site debuted in March 2007.^[11]

Further reading

• Hennessy, K., Finkbiner, R., & Hill, G. (2006). “The National Registry of Evidence-Based Programs and Practices: A decision-support tool to advance the use of evidence-based services”. International Journal of Mental Health, 35(2), 21–34. doi: 10.2753/IMH0020-7411350202.
• Brounstein, P. J., Gardner, S. E., & Backer, T. (2006). “Research to practice: Efforts to bring effective prevention to every community”. Journal of Primary Prevention, 27(1), 91-109. doi: 10.1007/s10935-005-0024-6. PMID 16421654. These criteria and the accompanying rating anchors are unique to NREPP but share common elements with the types of standards used by other Federal agencies to assess evidence-based programs.^[12]

External links

• Official Web site
• Substance Abuse and Mental Health Services (SAMHSA)

References