## PCA vs Laplacian Eigenmaps

Posted on May 9, 2020

At first glance, PCA and Laplacian Eigenmaps seem both very similar. We can view both algorithms as constructing a graph from our data, choosing a matrix to represent this graph, computing the eigenvectors of this matrix, and then using these eigenvectors to determine low-dimensionality embeddings of our data. However, the...

[Read More]
Tags:
Dimensionality Reduction, PCA, Laplacian Eigenmaps

## Learning Complexity and Generalization Bounds

Posted on January 30, 2020

In a typical supervised learning setting, we are given access to a dataset of samples \(S = (X_1, y_1), (X_2, y_2), ..., (X_n, y_n)\) which we assume are drawn from a distribution \(\mathcal{D}\) over \(\textbf{X} \times \textbf{y}\). For simplicity, we will assume that \(\mathbf{X}\) is either the space \(\{0,1\}^n\) or...

[Read More]
Tags:
Learning, Complexity, Generalization, VC Dimension, Vapnik, Chervonenkis, Rademacher

## Models of Learning

Posted on January 23, 2020

Machine Learning researchers have a tough time agreeing on the best formulations for the problems they face. Even within the relatively well-defined setting of supervised learning, there are lots of ways to express the nature of the problem. At a very high level, we can express supervised learning as a...

[Read More]
Tags:
PAC, Computational, Learning, Theory

## White Noise is Pretty Weird

Posted on January 9, 2020

I recently went off on a tangent trying to figure out how white noise works, and I found that there is a lot of strangeness to it that may not be apparent at a first glance. The content in this post is primarily from: This stackexchange answer This stackexchange answer...

[Read More]
Tags:
White Noise, Probability, Random Variables, Stochastic Process

## What are you modeling?

Posted on January 3, 2020

In this post, we will explore how Discriminative/Generative and Frequentist/Bayesian algorithms make different decisions about what variables to model probabilistically. There are many ways to characterize Machine Learning algorithms. This is a direct consequence of the rich history, broad applicability and interdisciplinary nature of the field. One of the clearest...

[Read More]
Tags:
Machine Learning, Probability, Discriminative, Generative, Frequentist, Bayesian