Robust Regularization Strategies Under Data Scarcity

Abstract

Machine learning models trained under data scarcity often suffer from unstable representations, poor
generalization, and memorization-driven failure modes. This article investigates the effectiveness of
different categories of regularization strategies structural, feature-space, and learning-dynamic in
mitigating these challenges. A multi-phase evaluation approach is used to examine model behavior
across varying levels of training data availability and incremental learning conditions. Structural
regularization methods such as weight sharing and low-rank factorization produced the most consistent
stability, while feature-space constraints enhanced representational coherence and transferability.
Learning-dynamic strategies provided partial benefits but required adaptive control to avoid suppressing
meaningful learning signals. The results indicate that robust generalization under data scarcity is best
supported by regularization approaches that shape internal feature geometry rather than simply
constraining parameter magnitudes. This study provides practical insights for deploying models in real
world conditions where data availability is inherently limited.