If you haven’t read Not With a Bang’s essay “LLM Exchange Rates Updated”, be sure to check it out.
Long-time readers of Tree of Woe will remember I conducted a similar experiment back in May and reported my findings. The results I got were actually 7 orders of magnitude worse than what Not With a Bang has recently reported - his experiment found a bias of about twenty to one against white lives, whereas my experiment found a bias of about a billion to one.
Does this mean that bias has decreased since May? Maybe — but his protocol was much more careful (and expensive) than mine so it’s possible my experiment simply over-stated the bias. He’s done solid detective work and offers powerful third-party confirmation that the implicit bias is very real. Whether the AI wants you dead twenty times more or a billion times more, it’s indefensible bias either way.