Crate pav_regression[−][src]

Pair Adjacent Violators algorithm for isotonic regression

What is “Isotonic Regression” and why should I care?

Imagine you have two variables, x and y, and you don’t know the relationship between them, but you know that if x increases then y will increase, and if x decreases then y will decrease. Alternatively it may be the opposite, if x increases then y decreases, and if x decreases then y increases.

Examples of such isotonic or monotonic relationships include:

  • x is the pressure applied to the accelerator in a car, y is the acceleration of the car (acceleration increases as more pressure is applied)
  • x is the rate at which a web server is receiving HTTP requests, y is the CPU usage of the web server (server CPU usage will increase as the request rate increases)
  • x is the price of an item, and y is the probability that someone will buy it (this would be a decreasing relationship, as x increases y decreases)

These are all examples of an isotonic relationship between two variables, where the relationship is likely to be more complex than linear.

So we know the relationship between x and y is isotonic, and let’s also say that we’ve been able to collect data about actual x and y values that occur in practice.

What we’d really like to be able to do is estimate, for any given x, what y will be, or alternatively for any given y, what x would be required.

But of course real-world data is noisy, and is unlikely to be strictly isotonic, so we want something that allows us to feed in this raw noisy data, figure out the actual relationship between x and y, and then use this to allow us to predict y given x, or to predict what value of x will give us a particular value of y.
This is the purpose of the pair-adjacent-violators algorithm.

…and why should I care?

Using the examples I provide above:

  • A self-driving car could use it to learn how much pressure to apply to the accelerator to give a desired amount of acceleration
  • An autoscaling system could use it to help predict how many web servers they need to handle a given amount of web traffic
  • A retailer could use it to choose a price for an item that maximizes their profit (aka “yield optimization”)

Isotonic regression in online advertising

If you have an hour to spare, and are interested in learning more about how online advertising works - you should check out this lecture that I gave in 2015 where I explain how we were able to use pair adjacent violators to solve some fun problems.

Modules

pav