This is one of the most interesting experiments ever.
A simple text change ➜ $100M/ year.
Here's the full story from Ronny Kohavi:
In 2012, a Bing employee had a simple idea.
What if ad headlines were longer...
By combining them with the first line of text below?
Nobody thought much of it.
The feature sat in the backlog for six months.
Until one developer decided to test it.
Within hours of launching the A/B test...
Alerts started firing!
Revenue was too high.
Something had to be wrong.
Except… nothing was wrong.
The change ended up generating $100M annual revenue.
The Single.
Biggest.
Revenue-driving feature in Bing’s history.
And yet, it almost never saw the light of day.
What an experiment, eh?