Large Mistakes in Regression Problems

Many real world problems can be framed as regression: use a collection of features \(X\) to predict a real-valued quantity \(y\). However, this framing can obfuscate a very important detail: which kinds of mistakes are most important to avoid? Many factors can influence this, including how the predictions will be... [Read More]
Tags: Machine Learning, Regression, ML

Predicting a Quantity with Categorical Features

Introduction Many prediction problems can be framed as “given the knowledge that this sample belongs to categories \(A,B,C,\cdots,D\), predict something about this sample.” As a concrete example, suppose we would like to use linear regression to predict the value of a transaction based on a small set of categorical features... [Read More]
Tags: Machine Learning, Machine Learning Systems, ML, Features

Improving a Machine Learning System (Part 3 - A/B testing)

This post is part three in a three part series on the challenges of improving a production machine learning system. Find part one here and part two here. A/B Testing When engineers and data scientists optimize machine learning systems they often focus on improving offline metrics like cross entropy, ROC-AUC,... [Read More]
Tags: Machine Learning, Machine Learning Systems, A/B, A/B testing

Improving a Machine Learning System (Part 2 - Features)

This post is part two in a three part series on the challenges of improving a production machine learning system. Find part one here and part three here. Adding New Features or Improving Existing Features A machine learning model is only as powerful as the features it is trained with.... [Read More]
Tags: Machine Learning, Machine Learning Systems, Features

Improving a Machine Learning System (Part 1 - Broken Abstractions)

This post is part one in a three part series on the challenges of improving a production machine learning system. Find part two here and part three here. Suppose you have been hired to apply state of the art machine learning technology to improve the Foo vs Bar classifier at... [Read More]
Tags: Machine Learning, Machine Learning Systems, Abstractions