PCA vs Laplacian Eigenmaps
Posted on May 9, 2020
At first glance, PCA and Laplacian Eigenmaps seem both very similar. We can view both algorithms as constructing a graph from our data, choosing a matrix to represent this graph, computing the eigenvectors of this matrix, and then using these eigenvectors to determine low-dimensionality embeddings of our data. However, the...
[Read More]
Tags:
Dimensionality Reduction, PCA, Laplacian Eigenmaps
Learning Complexity and Generalization Bounds
Posted on January 30, 2020
In a typical supervised learning setting, we are given access to a dataset of samples \(S = (X_1, y_1), (X_2, y_2), ..., (X_n, y_n)\) which we assume are drawn from a distribution \(\mathcal{D}\) over \(\textbf{X} \times \textbf{y}\). For simplicity, we will assume that \(\mathbf{X}\) is either the space \(\{0,1\}^n\) or...
[Read More]
Tags:
Learning, Complexity, Generalization, VC Dimension, Vapnik, Chervonenkis, Rademacher
Models of Learning
Posted on January 23, 2020
Machine Learning researchers have a tough time agreeing on the best formulations for the problems they face. Even within the relatively well-defined setting of supervised learning, there are lots of ways to express the nature of the problem. At a very high level, we can express supervised learning as a...
[Read More]
Tags:
PAC, Computational, Learning, Theory
White Noise is Pretty Weird
Posted on January 9, 2020
I recently went off on a tangent trying to figure out how white noise works, and I found that there is a lot of strangeness to it that may not be apparent at a first glance. The content in this post is primarily from: This stackexchange answer This stackexchange answer...
[Read More]
Tags:
White Noise, Probability, Random Variables, Stochastic Process
What are you modeling?
Posted on January 3, 2020
In this post, we will explore how Discriminative/Generative and Frequentist/Bayesian algorithms make different decisions about what variables to model probabilistically. There are many ways to characterize Machine Learning algorithms. This is a direct consequence of the rich history, broad applicability and interdisciplinary nature of the field. One of the clearest...
[Read More]
Tags:
Machine Learning, Probability, Discriminative, Generative, Frequentist, Bayesian