PCA vs Laplacian Eigenmaps

At first glance, PCA and Laplacian Eigenmaps seem both very similar. We can view both algorithms as constructing a graph from our data, choosing a matrix to represent this graph, computing the eigenvectors of this matrix, and then using these eigenvectors to determine low-dimensionality embeddings of our data. However, the... [Read More]
Tags: Dimensionality Reduction, PCA, Laplacian Eigenmaps

Learning Complexity and Generalization Bounds

In a typical supervised learning setting, we are given access to a dataset of samples which we assume are drawn from a distribution over . For simplicity, we will assume that is either the space or and that is either the space or . Given a set of functions that... [Read More]
Tags: Learning, Complexity, Generalization, VC Dimension, Vapnik, Chervonenkis, Rademacher

Models of Learning

Machine Learning researchers have a tough time agreeing on the best formulations for the problems they face. Even within the relatively well-defined setting of supervised learning, there are lots of ways to express the nature of the problem. At a very high level, we can express supervised learning as a... [Read More]
Tags: PAC, Computational, Learning, Theory

White Noise is Pretty Weird

I recently went off on a tangent trying to figure out how white noise works, and I found that there is a lot of strangeness to it that may not be apparent at a first glance. The content in this post is primarily from: This stackexchange answer This stackexchange answer... [Read More]
Tags: White Noise, Probability, Random Variables, Stochastic Process

What are you modeling?

In this post, we will explore how Discriminative/Generative and Frequentist/Bayesian algorithms make different decisions about what variables to model probabilistically. There are many ways to characterize Machine Learning algorithms. This is a direct consequence of the rich history, broad applicability and interdisciplinary nature of the field. One of the clearest... [Read More]
Tags: Machine Learning, Probability, Discriminative, Generative, Frequentist, Bayesian