AUTHOR=Tyukin Ivan Y. , Gorban Alexander N. , Sofeykov Konstantin I. , Romanenko Ilya 

TITLE=Knowledge Transfer Between Artificial Intelligence Systems

JOURNAL=Frontiers in Neurorobotics

VOLUME=Volume 12 - 2018

YEAR=2018

URL=https://www.frontiersin.org/journals/neurorobotics/articles/10.3389/fnbot.2018.00049

DOI=10.3389/fnbot.2018.00049

ISSN=1662-5218

ABSTRACT=We consider the fundamental question: how a legacy ``student'' Artificial Intelligent (AI) system could learn from a legacy ``teacher'' AI system or a human expert without re-training and, most importantly, without requiring significant computational resources.  Here ``learning'' is broadly understood as an ability of one system to mimic responses of the other to an incoming stimulation and vice-versa. We call such learning an Artificial Intelligence knowledge transfer. We show that if internal variables of the  ``student'' Artificial Intelligent system have the structure of an $n$-dimensional topological vector space and $n$ is sufficiently high then, with probability close to one, the required knowledge transfer can be implemented by simple cascades of linear functionals. In particular, for $n$ sufficiently large, with probability close to one, the ``student'' system can successfully and non-iteratively learn $k\ll n$ new examples from the ``teacher'' (or correct the same number of mistakes) at the cost of two additional inner products. The concept is illustrated with an example of knowledge transfer from one pre-trained convolutional neural network to another.