Psychometric Testing of a New Instrument to Measure Workplace Mental Health and Well-Being
Abstract
Workplace well-being and mental health among the US workforce is an ongoing concern. The pandemic both highlighted and worsened this issue, particularly among healthcare workers, who more frequently experience burnout and highly stressful workplace conditions. To address these issues, the US Surgeon General’s office proposed a five-domain framework around Worker Voice and Equity in October 2022. To our knowledge, no psychometrically developed measure is available to map the framework's components on a scale. We developed the Augusta Scale from the framework and administered it to Georgia Area Health Education Centers (AHEC) medical preceptors. Using this pilot data, we assessed the scale’s psychometric properties.
We psychometrically assessed the Augusta Scale under the Classical Test Theory and Item Response Theory (IRT) paradigms. The instrument’s reliability was assessed using Cronbach’s α and omega with reference value >0.70. Convergent validity was checked by conducting logistic regression on the association between the total score on the Augusta Scale and a one-item emotional burnout scale measuring a similar construct. A Confirmatory Factor Analysis (CFA) was performed using the five-domain structure. Various fit indices were estimated to check the plausibility of the five-factor solution.
Domain-specific Cronbach’s α ranged from 0.71 (0.67-0.75) to 0.90 (0.87-0.92), while the overall scale α was 0.94 (0.93-0.95), suggesting strong reliability. The omega score was 0.91, confirming items measured the latent construct. Convergent validity analysis confirmed the inverse relationship between total scale score and perception of burnout. The model fit statistics revealed good fit for the model, validating the five-factor solution for the CFA. The high-order model indicated positive correlations between domains and the second-order factor “well-being.” On the IRT paradigm, the item discrimination properties mostly ranged between “Moderate” and “Very High”.
The Augusta Scale is a valid and reliable measure that can assess workplace mental health and well-being of medical workers.
Keywords
Confirmatory Factor Analysis, Item Response Theory, Psychometric Analysis, Workplace Well-being, Mental Health, Burnout
Psychometric Testing of a New Instrument to Measure Workplace Mental Health and Well-Being
Workplace well-being and mental health among the US workforce is an ongoing concern. The pandemic both highlighted and worsened this issue, particularly among healthcare workers, who more frequently experience burnout and highly stressful workplace conditions. To address these issues, the US Surgeon General’s office proposed a five-domain framework around Worker Voice and Equity in October 2022. To our knowledge, no psychometrically developed measure is available to map the framework's components on a scale. We developed the Augusta Scale from the framework and administered it to Georgia Area Health Education Centers (AHEC) medical preceptors. Using this pilot data, we assessed the scale’s psychometric properties.
We psychometrically assessed the Augusta Scale under the Classical Test Theory and Item Response Theory (IRT) paradigms. The instrument’s reliability was assessed using Cronbach’s α and omega with reference value >0.70. Convergent validity was checked by conducting logistic regression on the association between the total score on the Augusta Scale and a one-item emotional burnout scale measuring a similar construct. A Confirmatory Factor Analysis (CFA) was performed using the five-domain structure. Various fit indices were estimated to check the plausibility of the five-factor solution.
Domain-specific Cronbach’s α ranged from 0.71 (0.67-0.75) to 0.90 (0.87-0.92), while the overall scale α was 0.94 (0.93-0.95), suggesting strong reliability. The omega score was 0.91, confirming items measured the latent construct. Convergent validity analysis confirmed the inverse relationship between total scale score and perception of burnout. The model fit statistics revealed good fit for the model, validating the five-factor solution for the CFA. The high-order model indicated positive correlations between domains and the second-order factor “well-being.” On the IRT paradigm, the item discrimination properties mostly ranged between “Moderate” and “Very High”.
The Augusta Scale is a valid and reliable measure that can assess workplace mental health and well-being of medical workers.
