AUTHOR=Wang Xiaoyu , Benning Martin 

TITLE=A lifted Bregman formulation for the inversion of deep neural networks

JOURNAL=Frontiers in Applied Mathematics and Statistics

VOLUME=Volume 9 - 2023

YEAR=2023

URL=https://www.frontiersin.org/journals/applied-mathematics-and-statistics/articles/10.3389/fams.2023.1176850

DOI=10.3389/fams.2023.1176850

ISSN=2297-4687

ABSTRACT=We propose a novel framework for the regularised inversion of deep neural networks. The framework is based on the authors' recent work on training feed-forward neural networks without the differentiation of activation functions. The framework envisages lifting the parameter space into a higher dimensional space by introducing auxiliary variables and penalising these variables with tailored Bregman distances. We propose a family of variational regularisations based on these Bregman distances, present theoretical results and support their practical application with numerical examples. In particular, we present the first convergence result (to the best of our knowledge) for the regularised inversion of a single-layer perceptron that only assumes that the solution of the inverse problem is in the range of the regularisation operator and that shows that the regularised inverse provably converges to the true inverse if measurement errors converge to zero.