Proposal Title

Using Item Response Theory and Post Test Interviews to Improve Assessment in Large General Chemistry Classes

Proposal Abstract

In the University of Georgia's Freshman Chemistry program we have assessed students for seven years using computerized tests. All of the students' responses are stored in our server's database. In 2005, we began analyzing student performance on our assessments using Item Response Theory (IRT). From our IRT analysis we have learned which chemical topics are the most difficult for our students. Based upon that knowledge we have modified our instruction slightly and detected an increase in student performance data. Concurrently, we have performed post test interviews of approximately 80 students. In these interviews we asked students with varying IRT ability levels to show us how they approached certain problems. From the interviews, we have gleaned stumbling blocks that impede performance, particularly for lower ability students. We will present IRT and interview data as well as student performance data in our talk.

Full Proposal

1.

In this session we will describe how we have used computerized testing in general chemistry at UGA for the last seven years and what our analysis of the acquired data has told us about student performance.

2.

We intend to show that using Item Response Theory analysis and posttest interviews with students that it is possible to learn what particular topics in general chemistry are the most troubling for students and to some small degree why they are troubling.

3.

We can show a typical freshman chemistry test at UGA and describe how we design questions to tell us if the students understand the concepts or just can manipulate equations.

4.

Attendees can see how a modern psychometric analysis tool (IRT) coupled with posttest interviews is an effective method of determining those large topics which bedevil student learning. We will also discuss some of the techniques we are using to improve our students' abilities and scores.

Location

Room 2903

Publication Type and Release Option

Presentation (Open Access)

Share

COinS
 
Nov 1st, 3:00 PM Nov 1st, 3:45 PM

Using Item Response Theory and Post Test Interviews to Improve Assessment in Large General Chemistry Classes

Room 2903

In the University of Georgia's Freshman Chemistry program we have assessed students for seven years using computerized tests. All of the students' responses are stored in our server's database. In 2005, we began analyzing student performance on our assessments using Item Response Theory (IRT). From our IRT analysis we have learned which chemical topics are the most difficult for our students. Based upon that knowledge we have modified our instruction slightly and detected an increase in student performance data. Concurrently, we have performed post test interviews of approximately 80 students. In these interviews we asked students with varying IRT ability levels to show us how they approached certain problems. From the interviews, we have gleaned stumbling blocks that impede performance, particularly for lower ability students. We will present IRT and interview data as well as student performance data in our talk.