Dynamic Loss Rebalancing for Sequential Curriculum Learning

Abstract

Sequential curriculum learning improves training efficiency by introducing examples from simple to complex,
but fixed loss weighting across curriculum stages often leads to overemphasis on early learning phases and
insufficient adaptation to later complexities. This work proposes a dynamic loss rebalancing framework that
continuously adjusts loss contributions in response to real-time learning state indicators such as representation
drift, gradient variance, and prediction entropy. By aligning the emphasis of training with the model’s evolving
internal structure, the framework ensures smoother learning progression, improved generalization under
distribution shift, and greater stability during incremental retraining. Experimental results show that dynamic
rebalancing reduces early-stage dominance, enhances representational flexibility, and produces more consistent
model behavior across complexity transitions. The approach is particularly suited for deployment contexts
where models must maintain continuity and reliability while adapting to new patterns over time.