Format

Individual Presentation

First Presenter's Institution

Regional Educational Laboratory Central

Second Presenter's Institution

Regional Educational Laboratory Central

Third Presenter's Institution

Regional Educational Laboratory Central

Fourth Presenter's Institution

Communities in Schools

Fifth Presenter's Institution

North Dakota Department of Public Instruction

Location

Session 4 (Plimsoll)

Strand #1

Family & Community

Relevance

This session directly connects to the fifth conference strand, Home, which focuses on cultivating and strengthening supports for children in the family and community. The work highlighted in this session involves a collaboration among staff at the Regional Educational Laboratory (REL) Central, the North Dakota Department of Public Instruction (NDDPI), the Communities in Schools (CIS) program, and the Full-Service Community Schools (FSCS) program. REL Central, one of ten federally funded Regional Educational Laboratories, supports educators to use data and research to improve academic outcomes for students. REL Central is partnering with NDDPI to support the growth of school-community connection using the CIS and FSCS programs. CIS is a model that schools and districts can implement to meet the nonacademic needs of their students (e.g., social and emotional skills) to help them stay in school and succeed academically. FSCS offers wide-ranging services for students, including academic, health, and mental supports and resources, to improve educational outcomes (e.g., student attendance). Both the CIS and FSCS programs are intended to increase student outcomes, such as engagement, attendance, achievement, and graduation, and to decrease dropout. This session shares the initial evaluation process and lessons learned when trying to build supports for youth at risk, which can inform or inspire similar efforts in the audience’s community.

Brief Program Description

Community school programs respond to students’ nonacademic needs so that they can focus on their learning. This session highlights a collaboration of organizations across North Dakota as they implement two community school programs. A shared evaluation plan supports the implementation of the two programs and will provide data to improve services provided to students.

Summary

This session shares the work of the Communities in Schools (CIS) and Full-Service Community Schools (FSCS) programs in North Dakota. These programs respond to a high-leverage need for youth at risk whose outcomes are less positive than the outcomes of youth across the state. Because these students lack access to services such as healthcare or mental health providers, they are at risk for academic outcomes and beyond.

To combat this lack of support, schools have the opportunity to become hubs for needed services. The North Dakota Department of Public Instruction has provided funding to school districts to implement the CIS program, a model to help those districts meet the nonacademic needs of their students (e.g., social and emotional skills). Additionally, the U.S. Department of Education has provided funding to implement the FSCS program in urban, rural, and high-poverty areas. FSCS provides services for students, including academic, health, and mental health supports and resources. With the availability of these services, educators can focus on academic needs while a sense of community grows, building a strong, educational ecosystem.

In spring 2019, the Regional Educational Laboratory (REL) Central began supporting CIS and FSCS staff to design a plan to evaluate successes, identify challenges, and inform continuous improvement in program implementation. Together, REL Central and program staff have developed a comprehensive evaluation plan that includes (1) CIS and FSCS individual logic models and a shared logic model, (2) quality evaluation questions that align with the CIS and FSCS logic models, (3) sources to help answer the evaluation questions through existing data and new data collection instruments as needed; (4) an analysis plan to answer the evaluation questions; and (5) a timeline to implement the evaluation plan. Additionally, REL Central has offered strategies for data use to ensure that CIS and FSCS implementation is completed as planned and is progressing toward the intended outcomes. This session shares this collaborative process, the current status of program implementation, successes, challenges that the programs have overcome, and lessons learned to inform similar efforts in other communities.

Evidence

Evaluation is a critical skill to document the intended implementation of activities and assess outcomes of programs. A logic model is the foundation of a sound program evaluation (Kekahio, Cicchinelli, Lawton, & Brandon, 2014; Lawton, Brandon, Cicchinelli, & Kekahio, 2014; Shakman & Rodriguez, 2015). It can be used to identify barriers to program successes, implementation, and attainment of milestones leading to anticipated outcomes. The creation of a logic model is an iterative process as it can change throughout a program evaluation, depending on variances in implementation of the program.

A logic model then informs the creation of evaluation questions. Evaluation questions should address logic model components, with process questions focusing on understanding and evaluating implementation while outcome questions focus on short-, mid-, and long-term outcomes. High-quality evaluation questions are pertinent, answerable, reasonable, specific, evaluative, and complete (Wingate & Schroeter, 2016). Evaluation questions are the essential element to beginning a program evaluation, followed by data sources to answer evaluation questions, a data analysis plan, and the dissemination of evaluation results to key stakeholders.

The Communities in Schools (CIS) program has been evaluated in various contexts across the United States and has shown some evidence of success in improving student outcomes, making a comprehensive evaluation plan critical for the implementation of CIS and Full-Service Community Schools (FSCS) in North Dakota. Recent CIS (2017, 2018) reports show that 88 percent of case-managed students across schools implementing CIS saw improved academics: 93–94 percent were promoted in K–11; 91–93 percent graduated; and 78–80 percent had higher attendance rates. Students who were case-managed through CIS said that they had enhanced relationships with adults and peers, were more engaged in school, had more positive attitudes toward their education, and saw value in education for their futures (CIS, 2018; Parise et al., 2017). A research-based evaluation plan is needed for both the CIS and FSCS programs in North Dakota contexts to examine whether their effectiveness will reach urban, rural, and high-poverty students.

References:

Communities in Schools. (2017). All in for students: 2017 national impact report. Arlington, VA: Author. Retrieved from https://www.communitiesinschools.org/our-data/publications/publication/2017-national-impact-report

Communities in Schools. (2018). Community matters: Focus on chronic absenteeism. Arlington, VA: Author. Retrieved from https://www.communitiesinschools.org/our-data/publications/publication/community-matters-report-2018

Kekahio, W., Cicchinelli, L., Lawton, B., & Brandon, P. R. (2014). Logic models: A tool for effective program planning, collaboration, and monitoring (REL 2014–025). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Pacific. Retrieved from https://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=REL2014025

Lawton, B., Brandon, P. R., Cicchinelli, L., & Kekahio, W. (2014). Logic models: A tool for designing and monitoring program evaluations (REL 2014–007). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Pacific. Retrieved from https://ies.ed.gov/pubsearch/pubsinfo.asp?pubid=REL2014007

Parise, L. M., Corrin, W., Granito, K., Haider, Z., Somers, M.-A., & Cerna, O. (2017). Two years of case management: Final findings from the Communities in Schools random assignment evaluation. New York, NY: MDRC. Retrieved from https://www.mdrc.org/publication/two-years-case-management

Shakman, K., & Rodriguez, S. M. (2015). Logic models for program design, implementation, and evaluation: Workshop toolkit (REL 2015–057). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Northeast & Islands. Retrieved from https://ies.ed.gov/ncee/edlabs/projects/project.asp?ProjectID=401

Wingate, L., & Schroeter, D. (2016). Evaluation questions checklist for program evaluation. Kalamazoo, MI: The Evaluation Center, Western Michigan University. Retrieved from https://wmich.edu/evaluation/checklists

Biographical Sketch

Lucy Fredericks is an enrolled member of the Standing Rock Sioux Tribe. She has worked in education for the past 29 years in various positions. She received her Bachelor and Master’s degrees from the University of North Dakota in Elementary Education and Educational Leadership. Prior to being employed as Director of Indian/Multicultural Education for NDDPI, Lucy worked for Standing Rock Elementary School in Fort Yates, ND as the elementary school principal for seven years. She enjoys working with all schools in ND and is strongly committed to improving education for all our students. Lucy currently resides in Mandan, ND and also has a home in Twin Buttes, ND where her husband Lyndon is employed. Lucy and Lyndon have two children; Jonas and Chanda and one Grandson Easton.

Betty Li Simpson joined the CIS national office in June of 2016 and currently serves as a Research and Evaluation Associate based out of Washington, DC. In her role, Betty supports the execution of the CIS research agenda and evaluation of various strategic investments in CIS’ network of affiliate organizations and licensed partners. Prior to joining CIS, Betty assisted on research projects related to summer slide and inquiry-based science pedagogy at the University of Virginia’s Curry School of Education and Human Development. She also taught life and physical sciences at a public middle school in Central Virginia. Betty has a Master’s of Teaching in Secondary Science and a Bachelor’s degree in biology from the University of Virginia.

Nathan Anderson, PhD, is the proprietor of Anderson Inquiry, LLC. His work focuses on research, evaluation, and curriculum development in PK-12 and Higher Education. He is interested in identifying and developing strategies to help ensure important decisions in educational settings are evidence-based and that all collected data are utilized for intended purposes. Anderson provides evaluation services to the North Dakota FSCS Consortium.

Lyndsi Engstrom, MS, currently serves at the Director of Research, Design and Value of the Central Region Education Association (CREA), one of the 7 Regional Education Associations in North Dakota. She provides technical assistance, designs and conducts professional development, and establishes direct student service programming for North Dakota schools, specifically the 50-member school districts of the CREA. Engstrom serves as one of three regional coordinators for FSCS which involves coordinating activities and organizational responsibility.

Mckenzie Haines, BA, is a research associate at Marzano Research. Haines provides research assistance and technical support for REL Central projects as well as education research studies focused on educator effectiveness and program improvement. She also contributes to technical research reports. Prior to her role at Marzano Research, Mckenzie gained experience in research and evaluation by conducting survey-based research at Indiana University and serving as an intern in the department of research and evaluation at the Children’s Museum of Indianapolis.

Shelby Hubach, PhD, is a senior researcher at Marzano Research. Hubach has direct experience with community-based mental health organizations and more than 10 years of experience managing and executing research and evaluation projects. During Hubach’s time in graduate school, she gained direct experience with community-based mental health services at Larimer Center for Mental Health Summit Stone Health partners, a community-based mental health organization that provides services to individuals, youth, and families in Larimer County. Her doctoral research focused on the physical, emotional, and intellectual safety of students.

Jeanette Joyce, PhD, is a researcher at Marzano Research. Her primary research interests involve the combination of theory and practice to consider how to best support learning in an inclusive setting. She has presented her work nationally and internationally, and has published research in peer-reviewed journals, including the International Journal of Inclusive Education. In addition to research and program evaluation, Joyce has more than 15 years of experience in K–Post Secondary classroom instruction, special education services, and language/learning evaluation.

Keyword Descriptors

community schools, school-community connection, nonacademic needs, supports, youth at risk, community, access to services, student outcomes

Presentation Year

March 2020

Start Date

3-10-2020 8:30 AM

End Date

3-10-2020 9:45 AM

Download

Share

COinS
 
Mar 10th, 8:30 AM Mar 10th, 9:45 AM

Building an Ecosystem of Support: An Evaluation of North Dakota’s Community School Programs

Session 4 (Plimsoll)

Community school programs respond to students’ nonacademic needs so that they can focus on their learning. This session highlights a collaboration of organizations across North Dakota as they implement two community school programs. A shared evaluation plan supports the implementation of the two programs and will provide data to improve services provided to students.