Improving a Machine Learning System (Part 3 - A/B testing)

This post is part three in a three part series on the challenges of improving a production machine learning system. Find part one here and part two here. A/B Testing When engineers and data scientists optimize machine learning systems they often focus on improving offline metrics like cross entropy, ROC-AUC,... [Read More]
Tags: Machine Learning, Machine Learning Systems, A/B, A/B testing

Improving a Machine Learning System (Part 2 - Features)

This post is part two in a three part series on the challenges of improving a production machine learning system. Find part one here and part three here. Adding New Features or Improving Existing Features A machine learning model is only as powerful as the features it is trained with.... [Read More]
Tags: Machine Learning, Machine Learning Systems, Features

Improving a Machine Learning System (Part 1 - Broken Abstractions)

This post is part one in a three part series on the challenges of improving a production machine learning system. Find part two here and part three here. Suppose you have been hired to apply state of the art machine learning technology to improve the Foo vs Bar classifier at... [Read More]
Tags: Machine Learning, Machine Learning Systems, Abstractions

Optimizers as Dynamical Systems

The ideas in this post were hashed out during a series of discussions between myself and Bruno Gavranović Consider a system for forecasting a time series in \(\mathbb{R}\) based on a vector of features in \(\mathbb{R}^a\). At each time \(t\) this system will use the state of the world (represented... [Read More]
Tags: Machine Learning, Category Theory, Lens, Dynamical System, Gradient Descent

Supervised Clustering With Kan Extensions

Clustering algorithms allow us to group points in a dataset together based on some notion of similarity between them. Formally, we can consider a clustering algorithm as mapping a metric space \((X, d_X)\) (representing data) to a partitioning of \(X\). In most applications of clustering the points in the metric... [Read More]
Tags: Clustering, Machine Learning, Extrapolation, Kan Extension, Category Theory, Functorial