Interference-Gated Dynamic Activation for Task-Agnostic Continual Learning: A Formal-Empirical Audit of Stability, Forgetting, and Failure Regimes

Abstract

Continual learning methods frequently reduce forgetting by adding replay, regularization, or routing constraints, yet activation functions are usually treated as fixed nonlinearities rather than adaptive components of the retention mechanism. We study a task-agnostic setting where task identities are unavailable and dynamic activation updates must operate under matched replay and optimization budgets. In this setting, we define an interference-gated projected activation update, formalize a one-step forgetting-proxy comparison against static-gate baselines, and introduce a fairness-normalized attribution predicate that blocks invalid comparisons when memory or optimization controls differ. The empirical study executes seed-aggregated runs on reduced Split CIFAR-100, reduced Split TinyImageNet, and a third real fallback online digits stream, with static GELU, ReLU, Mish, A-GEM, EWC, and ablated dynamic variants as comparators. Results show that the full dynamic gate is indistinguishable from static GELU in the executed matrix, while replay-family changes can worsen forgetting and destabilize backward transfer, yielding mixed support for retention-improvement claims. Symbolic checks confirm local first-order identities and fairness-accounting completeness but expose a failed simplification in one projection-bound audit, limiting formal conclusions to local validity windows. These findings sharpen the boundary between plausible mechanism intuition and supported claim scope: activation-state adaptation can be audited rigorously, but global forgetting guarantees remain unresolved without stronger alignment conditions and broader benchmark execution.