Tensor Methods for Deep Learning

Deep learning has revolutionized a large number of areas of machine learning and data science. The advances in new architectures and their unparalleled performance in novel application domains, ranging from images to text, and from graphs to recommendation system, have established deep learning as the state of the art in machine learning and data analytics alike.

At the core of deep learning models lies the concept of a tensor, a multi-dimensional matrix that represents data flowing through the deep network and deep network parameters alike. The versatility of this tensor representation within deep learning has brought about advances in the development of deep learning toolboxes (e.g., TensorFlow) that have contributed to the wide applicability of deep learning.

Tensor methods have, on the other hand, enjoyed a growth which, for a long time was largely independent of the rise of deep learning, focusing on unsupervised exploratory extensions of matrix factorization and PCA to higher-order tensors. However, tensor methods, in addition to tensor representations, have recently been introduced in deep learning, with notable success stories including the compression of deep models by leveraging low-rank tensor factorization of the tensor weights in the model. Subsequently, novel methods and frameworks that leverage tensor factorization for training deep networks and theoretically analyzing various properties such as generalizability and expressivity.

The objective of this Research Topic is to bring together researchers from a diverse set of fields, including deep learning, computer vision, and tensor algorithms and applications, provide a venue for cross-pollination of ideas, and foster the emerging field of tensor methods for deep learning.

In this call for contributions, we solicit work that lies in the intersection of tensor methods and deep learning, focusing, but not limited to:
● Tensor factorization for deep model compression and efficiency
● Tensor factorization for training deep models
● Tensor factorization for analyzing deep models theoretically
● Tensor factorization for deep model interpretability