Format
Individual Presentation
First Presenter's Institution
Armstrong State University
Second Presenter's Institution
N/A
Third Presenter's Institution
N/A
Fourth Presenter's Institution
N/A
Fifth Presenter's Institution
N/A
Location
Westbrook
Strand #1
Academic Achievement & School Leadership
Strand #2
Mental & Physical Health
Relevance
- Is using the individual norm referenced standardized test (NRST) score effectively using data to answer the questions: Did I teach? Did they learn? Conference Strand 1. “Head”: Academic Achievement & Leadership. Can the individual norm referenced standardized test score harm a student? Conference Strand IV. “Health”: Mental and Physical Health.
- If it is state law that students must attend school or be home schooled until they are a certain age, then must the state protect them while at public schools? The answer seems self-evident. Indeed, the United States Constitution’s 14th Amendment guarantees equal protection under the laws to its citizens. But do school practices unintentionally violate this protection?
Brief Program Description
- : If it is state law that students must attend school or be home schooled until they are a certain age, then must the state protect them while at public schools? The answer seems self-evident. Indeed, the United States Constitution’s 14th Amendment guarantees equal protection under the laws to its citizens. But do school practices unintentionally violate this protection?
Summary
- : I will focus on two possibilities. Norm referenced standardized tests (NRSTs) may fail accurately explaining student competence. The instructional alignment construct explains if we align instruction and assessment conditions, our assessments accurately explain if we taught and the students learned. The NRST insults the instructional alignment construct.
Commercially prepared NRSTs, such as those given students throughout the country, are unreliable, in other words, if the student takes the test more than once, the scores may vary.
First, here are some elementary psychometric principles found in any introductory text about testing:
- A test score is made up of two components, (1) true score (2) scoring error.
- The error component represents measurement error, or chance (random) factors that reduce the score’s reliability (accuracy).
The Standard Error of Measurement (SEM) formula determines the error component. The SEM statistical formula explains how much we can trust the individual’s NRST score. Or, in other words, how much will the scores vary if we give the test more than once to a student. Cohen explains in “Tests Marked for Life?” that on a widely used NRST, the standard error of measurement explains that if an individual score is at the 75th percentile, the true score lies in a range between the 69th and 85th percentile. This means, if we administered the test a second time, the score may be higher or lower than the 75th percentile, somewhere between the 69th to the 85th percentile range.
To me, this seems like a lot of slippage. Because, what if we are judging student competence with a test cut off score and the person misses the cut off by one point, say they need the 75th percentile score and they score at the 74th percentile. The true score may be higher than the 74th percentile. Nevertheless, the third grade student may be kept from the fourth grade. Or for teacher education students, the pass score is 154. But if they score 153, they are kept from certification or student teaching.
The NRST insults the instructional alignment construct. The NRST is correlated with an existing NRST for validity. But the correlation formula demands a normal distribution of scores. Should we be using a test for judging teaching and individual student competence that demands a normal distribution of scores?
Evidence
- The reported score, true score, and Standard Error of Measurement are accepted statistical standards used in data analysis.
“The relationship with the standard deviation is defined such that, for a given sample size, the standard error equals the standard deviation divided by the square root of the sample size. As the sample size increases, the dispersion of the sample means cluster more closely around the population mean and the standard error decreases.”
The true score resides within the range created by the standard error of measurement.
Instructional Alignment
“Objective Design and Instructional Alignment
Instructional alignment is an essential characteristic of any effective learning experience. The presence of instructional alignment positively impacts learning. Instructional alignment is also a significant and necessary prerequisite condition for both the pursuit of efficacy and the application of more complex aspects of learning design, such as those related to designing for motivation and adaptive learning.
Instructional alignment refers to a high degree of agreement among the objectives, assessments, and the content in a learning experience (Cohen, 1987). It also includes deriving objectives from and aligning them to sets of relevant external frameworks or standards. Objective design refers to the practice of articulating the objectives, based on relevant sets of standards or external frameworks, for a learning experience such that the objectives reflect the specified measurability attributes while explicitly stating the learning goals. The objectives form the central framework for a learning experience and function as the critical point of reference when designing aligned assessments and content.
The main components of this principle are Pearson’s research-based points-of-view on objectives, including their definitions, attributes that support measurability (Anderson, 2001; Dick, Carey, & Carey, 2015; Hattie, 2009; Marzano, 1998; Messick, 1989 ) and their function and the framework they form (Anderson, 2001; Dick, Carey, & Carey, 2015; Fullan & Langworthy, 2014; Hattie, 2008; Mayer, 2011). The principle also
covers the role of standards alignment in the design process as well as an overview of assessment alignment and content alignment.”
Retrieved from: https://www.pearson.com/content/dam/one-dot-com/one-dot-com/global/Files/efficacy-and research/methods/LDP_Cards_V4b_References_120516.pdf
Anderson, L. W. (2001). A taxonomy for learning, teaching, and assessing: a revision of Bloom’s taxonomy of educational objectives (Complete ed.). New York: Longman.
Cohen, S. (1987). Instructional alignment: Searching for a magic bullet.
Educational Researcher, 16(8), 16–20.
Cohen, S. (1988). Tests: Marked for life? New York: Scholastic.
Dick, W., Carey, L., & Carey, J. (2015). The Systematic Design of Instruction
(8 ed.). New York: Pearson.
Fullan, M. & Langworthy, M. (2014). A Rich Seam: How New Pedagogies Find Deep
Learning. London: Pearson.
Hattie, J. (2008). Visible learning: A synthesis of over 800 meta-analyses relating to
achievement. New York, NY, USA: Routledge.
Marzano, R. A. Theory-Based Meta-Analysis of Research on Instruction, 127-128.
Retrieved May 5, 2014, from http://www.peecworks.org/peec/peec_research/I01795EFA.2/Marzano%20Instruction%20Meta_An.pdf
Mayer, R. E. (2011). Applying the science of learning. Boston, MA: Pearson/Allyn & Bacon.
Messick, S. Meaning and Values in Test Validation: The Science and Ethics of Assessment. Educational Researcher, 18, 5-11.
Biographical Sketch
- 23 years public school teaching experience, 25 years higher education experience, 42 years’ experience explaining and analyzing United States Department of Education programs.
Keyword Descriptors
Mandated Testing, Assessment, Academic Achievement, Marking Students With Test Scores
Presentation Year
2018
Start Date
3-5-2018 1:15 PM
End Date
3-5-2018 4:15 PM
Recommended Citation
Hobe, John, "1. Why Use Norm Referenced Standardized Tests to Answer: Did I Teach? Did They Learn?" (2018). National Youth Advocacy and Resilience Conference. 33.
https://digitalcommons.georgiasouthern.edu/nyar_savannah/2018/2018/33
1. Why Use Norm Referenced Standardized Tests to Answer: Did I Teach? Did They Learn?
Westbrook
- : If it is state law that students must attend school or be home schooled until they are a certain age, then must the state protect them while at public schools? The answer seems self-evident. Indeed, the United States Constitution’s 14th Amendment guarantees equal protection under the laws to its citizens. But do school practices unintentionally violate this protection?
