Sample-Conditioned Hypothesis Stability Sharpens Information-Theoretic Generalization Bounds

Ziqiao Wang; Yongyi Mao

NeurIPS 2023poster7 citations

Sample-Conditioned Hypothesis Stability Sharpens Information-Theoretic Generalization Bounds

Abstract

We present new information-theoretic generalization guarantees through the a novel construction of the "neighboring-hypothesis" matrix and a new family of stability notions termed sample-conditioned hypothesis (SCH) stability. Our approach yields sharper bounds that improve upon previous information-theoretic bounds in various learning scenarios. Notably, these bounds address the limitations of existing information-theoretic bounds in the context of stochastic convex optimization (SCO) problems, as explored in the recent work by Haghifam et al. (2023).

generalizationinformation-theoretic boundsstability

BibTeX

@inproceedings{
wang2023sampleconditioned,
title={Sample-Conditioned Hypothesis Stability Sharpens Information-Theoretic Generalization Bounds},
author={Ziqiao Wang and Yongyi Mao},
booktitle={Thirty-seventh Conference on Neural Information Processing Systems},
year={2023},
url={https://openreview.net/forum?id=oqDSDKLd3S}
}