Stabilized Optimization Methods for Multinomial Logistic Regression in Statistical Learning
Faculty Mentor
Dr. Divine F. Wanduku
Location
Russell Union Ballroom
If Other was choses above, please indicate your topic area here:
Statistics
Type of Research
On-going
Session Format
Poster Presentation
College
College of Science & Mathematics
Department
Mathematical Sciences
Abstract
Multinomial logistic regression is a fundamental model in statistical learning for multi-class classification. Maximum likelihood estimation relies on iterative second-order optimization methods whose performance depends critically on the conditioning of the Hessian matrix. In practice, near-singularity, rank deficiency, or flat likelihood curvature can lead to unstable updates, slow convergence, or algorithmic failure. This study presents a systematic computational comparison of likelihood-based optimization algorithms for multinomial logistic models, including Ordinary Newton–Raphson, Weighted and Generalized Newton–Raphson variants, Gauss–Newton, and Levenberg–Marquardt methods. Special attention is given to performance under ill-conditioned or singular design matrices, where classical Newton updates may diverge. Stabilized approaches incorporating weighting, damping, and curvature regularization are shown to improve numerical robustness while maintaining convergence. Model performance is evaluated using likelihood-based criteria (AIC) and classification accuracy measures, while convergence behavior is examined through profile plots and curvature diagnostics. The framework is extended to include LASSO-penalized likelihood to support variable selection and stability in higher-dimensional settings. Simulation studies and application to health data involving classification of disease states (recovered, deceased, asymptomatic carrier) demonstrate that stabilization methods enhance computational reliability without sacrificing predictive accuracy. These findings underscore the importance of stabilized second-order optimization in modern computational statistics and statistical learning, particularly for classification problems with ill-conditioned likelihood surfaces.
Program Description
.
Start Date
4-23-2026 2:00 PM
End Date
4-23-2026 4:00 PM
Recommended Citation
Wambua, Josphat and Chukwu, Chidozie Williams, "Stabilized Optimization Methods for Multinomial Logistic Regression in Statistical Learning" (2026). GS4 Student Scholars Symposium. 188.
https://digitalcommons.georgiasouthern.edu/research_symposium/2026/2026/188
Stabilized Optimization Methods for Multinomial Logistic Regression in Statistical Learning
Russell Union Ballroom
Multinomial logistic regression is a fundamental model in statistical learning for multi-class classification. Maximum likelihood estimation relies on iterative second-order optimization methods whose performance depends critically on the conditioning of the Hessian matrix. In practice, near-singularity, rank deficiency, or flat likelihood curvature can lead to unstable updates, slow convergence, or algorithmic failure. This study presents a systematic computational comparison of likelihood-based optimization algorithms for multinomial logistic models, including Ordinary Newton–Raphson, Weighted and Generalized Newton–Raphson variants, Gauss–Newton, and Levenberg–Marquardt methods. Special attention is given to performance under ill-conditioned or singular design matrices, where classical Newton updates may diverge. Stabilized approaches incorporating weighting, damping, and curvature regularization are shown to improve numerical robustness while maintaining convergence. Model performance is evaluated using likelihood-based criteria (AIC) and classification accuracy measures, while convergence behavior is examined through profile plots and curvature diagnostics. The framework is extended to include LASSO-penalized likelihood to support variable selection and stability in higher-dimensional settings. Simulation studies and application to health data involving classification of disease states (recovered, deceased, asymptomatic carrier) demonstrate that stabilization methods enhance computational reliability without sacrificing predictive accuracy. These findings underscore the importance of stabilized second-order optimization in modern computational statistics and statistical learning, particularly for classification problems with ill-conditioned likelihood surfaces.
