Overfitting has a limitation: a model-independent generalization gap bound based on R\'enyi entropy

ArXi:2506.00182v3 Announce Type: replace-cross Will further scaling up of machine learning models continue to bring success? A significant challenge in answering this question lies in understanding generalization gap, which is the impact of overfitting. Understanding generalization gap behavior of increasingly large-scale machine learning models remains a significant area of investigation, as conventional analyses often link error bounds to model complexity, failing to fully explain the success of extremely large architectures. This research.