paper
arXiv cs.LG
November 11th, 2026 at 5:00 AM

On Compatible Generalization Bounds of Deep Residual Networks From a Dynamical System Perspective

arXiv:2602.20921v2 Announce Type: replace Abstract: Deep neural networks (DNNs) have significantly advanced machine learning, with model depth playing a central role in their successes. Deep residual networks (ResNets) fall into one of the most important classes of DNNs. In this paper, we study the compatible generalization behavior of deep residual networks with a broad class of activation functions in the deep-layer limit, from a dynamical system perspective. We first show a refined contraction inequality for Rademacher complexity of function classes composed by that type of activation functions. We then conduct an equivalence analysis of component hypothesis sets in Rademacher complexity and forward convergence analysis of network state variables, which enable us to establish compatible uniform generalization bounds that simultaneously achieve the optimal \(O(1/\sqrt{S})\) sample rate for both discrete- and continuous-time ResNets. The bounds also share a common layer-number independent leading term and non-positive refinement terms that capture the decomposition structure of the activation function, revealing a consistent generalization behavior in the deep-layer limit. Compared to previous works considering both discrete- and continuous-time ResNets simultaneously, our theoretical results not only have better sample rate, but also are established under weaker assumptions on the learnable parameters. We also support our theoretical findings with numerical experiments.

#research

Get stories like this every Monday

The week's most important AI news in one email. Free, unsubscribe anytime.

Score: 3.15

Engagement proxy: 0

Canonical link: https://arxiv.org/abs/2602.20921