Structured Representation Learning
Download Structured Representation Learning PDF/ePub or read online books in Mobi eBooks. Click Download or Read Online button to get Structured Representation Learning book now. This website allows unlimited access to, at the time of writing, more than 1.5 million titles, including hundreds of thousands of titles in various foreign languages.
Structured Representation Learning
This book introduces approaches to generalize the benefits of equivariant deep learning to a broader set of learned structures through learned homomorphisms. In the field of machine learning, the idea of incorporating knowledge of data symmetries into artificial neural networks is known as equivariant deep learning and has led to the development of cutting edge architectures for image and physical data processing. The power of these models originates from data-specific structures ingrained in them through careful engineering. To-date however, the ability for practitioners to build such a structure into models is limited to situations where the data must exactly obey specific mathematical symmetries. The authors discuss naturally inspired inductive biases, specifically those which may provide types of efficiency and generalization benefits through what are known as homomorphic representations, a new general type of structured representation inspired from techniques in physics and neuroscience. A review of some of the first attempts at building models with learned homomorphic representations are introduced. The authors demonstrate that these inductive biases improve the ability of models to represent natural transformations and ultimately pave the way to the future of efficient and effective artificial neural networks.
Bayesian Structured Representation Learning
Bayesian methods offer the flexibility to both model uncertainty and incorporate domain knowledge into the modeling process. Deep generative modeling and Bayesian deep learning methods, such as the variational autoencoder (VAE), have expanded the scope of Bayesian methods, enabling them to scale to large, high-dimensional datasets. Incorporating prior knowledge or domain expertise into deep generative modeling is still a challenge, often resulting in models where Bayesian inference is prohibitively slow or even intractable. In this thesis, I first motivate using structured priors, presenting a contribution in the space of interactive structure learning. I then define Bayesian structured representation learning (BSRL) models, which combine structured priors with the VAE, and present foundational work along with applications of BSRL models.
Deep Representation Learning with Induced Structural Priors
With the support of big-data and big-compute, deep learning has reshaped the landscape of research and applications in artificial intelligence. Whilst traditional hand-guided feature engineering in many cases is simplified, the deep network architectures become increasingly more complex. A central question is whether we can distill the minimal set of structural priors that can provide us the maximal flexibility, and lead us to richer sets of structural primitives. Those structural priors will make the learning process more effective, and potentially lay the foundations towards the ultimate goal of building general intelligent systems. This dissertation focuses on how we can tackle different real world problems in computer vision and machine learning with carefully designed neural network architectures, guided by simple yet effective structural priors. In particular, this thesis focuses on two structural priors that have proven to be useful and generalizable in many different scenarios: the multi-scale prior, with an application in edge detection, and the sparse-connectivity prior implemented for generic visual recognition. Examples will be presented in the last part, on how to learn meaningful structures directly from data, rather than hard-wiring them by, for example, learning a convolutional pseudo-prior in the label space, or adopting a dynamic self-attention mechanism.