No, that's not quite true. In the long run in an efficejy market your rate of return (over market average...0 here) can reflect the employment value of someone of your ability spending that much time doing research.
For instance, imagine a market in which people bet on a mathematical hypothesis. If you are a mathematician you can gain evidence about that claim by doing mathematics about it. And you should expect that if you have sufficient capital you can basically recover the value of the time you invested (if you can get more the other investors will hire their own mathematician if less then they won't).
Right, but in your example the market is not yet efficient until the second group hires a mathematician and then the returns approach zero. The first mathematician might be able to make enough money to recoup their time, the second just zeros out the two sides and nobody makes money.
1. There seems to be a confusion between "prediction markets will be accurate" and "prediction markets will be accurate after the fact."
2. The whole point of insider trading is that no one else knows about it. To argue that some insider is trading would be valuable information for prediction markets is both accurate and incoherent.
2. I don't think anyone would know insiders were trading on their market specifically, but there would be a general knowledge that insiders often/rarely trade or that there were weak/strong regulations against it.
I'll try to get back to 1 when I have time; it's more subtle than I have focus for right now. For now, the best I can say is that the accuracy of a prediction market prediction is actually veridically proven only when the underlying event occurs, not when people bet one way or the other.
2. "In fact, knowing that President Biden is insider-trading on a “Will President Biden resign?” prediction market should only increase your confidence in it getting the right answer!" That looks an awful lot like a claim that someone would know that an insider was trading.
With #2 it looks more like "if an insider trades on a prediction market, this makes the prediction market more likely to be accurate, whether or not you know about it (since the insider knows the true result and is incentivised to move the market toward it); however, this means that if you *do* know about a particular insider trading on a particular prediction market, your *assessment* of that specific market's accuracy should increase relative to your prior".
>For now, the best I can say is that the accuracy of a prediction market prediction is actually veridically proven only when the underlying event occurs, not when people bet one way or the other.<
Sure, but — so what?
That is, for a given definition of "proven", you can make an argument like this against almost any sort of attempt at knowledge:
• "A study is only proven accurate when a meta-study confirms it." • "An ETA is only proven accurate when the item actually arrives." • "A poll is only proven accurate when the election turns out the way it had indicated." • Etc., etc.
But: that doesn't mean these are not useful tools, nor that they are not / cannot be accurate! (That is, "useful/accurate *relative to available alternatives*"; e.g., simple guessing or auspicy.)
Indeed, what's left, if we apply such a standard universally? Would one argue against the utility of, say, UPS producing an "Estimated Delivery" date for customers — data-driven, sure, but of *unproven* accuracy up 'till the event itself occurs?
Yes. Yes, one would, because UPS "Estimated Delivery" dates are complete garbage and those morons could fuck up a wet dream.
...But still, if I'd chosen a better example, you'd see what I mean, right?
What claim beyond "this is more accurate than most other ways to make predictions and can be useful even if not infallible" do you see us as making?
--------------------------
One can obtain a good sense of how accurate the market's conclusion is likely to be, if there is some worry about this:
★ Look at how many 70c "shares" turned out to occur. If roughly 70% of them did...
★ Do the same for other questions and prices until satisfied of accuracy. This has been empirically demonstrated elsewhere to *my* satisfaction already, of course; but if it turns out that *you* discover otherwise...
★ Well, then: If so — if you find the markets to be consistently inaccurate — you can make money!
★ ...and if you can make money, so can others. And if everyone is buying in such a way, we have price pressure in the direction of the more accurate prediction/price.
★ Hence, we—... Okay, okay, you've got it from here, I'm pretty sure!
--------------------------
>That looks an awful lot like a claim that someone would know that an insider was trading.<
I don't think so. User magic9mushroom, above, explains it excellently and in the same way I understood the entry; but since I (vainly) feel compelled to add a separate attempt here anyway:
The point being made works whether or not one can determine if insider trading is occurring, so a yea or nay there is not really important to Scott's argument: if you can, cool, you know it's more accurate than usual; if you can't, cool, there's no reason to worry, because see previous clause.
In other words, whatever your contention re: how knowable insider trading would be, Scott can shrug and say "okay, sure" without affecting the main thrust of the section, which is only really about the effect on pricing.
An insider trade always reveals information, because their trades shift the market towards whatever the insider knows will happen.
The only question is whether the gain in information is worth the loss from non-insider traders quitting in disgust because they keep ending up on the wrong end of an insider trade.
No, the point of insider trading is to use information not available to the general public. It is the illegality of using insider trading in stock market for example, that leads to it being hidden from others.
I don't know about 'canonical', they're subject to manipulation too. I think they're useful as an additional source of information where people involved have real 'skin in the game'.
"Canonical", here, means that sufficiently-liquid prediction markets can't stably disagree with each other due to the arbitrage issue. They can be wrong, but all of them will be equally wrong, so there's a "canonical" prediction market answer.
(That is, if you manipulate one prediction market with loadsamoney, you'll indirectly move all prediction markets about that issue, because people will arbitrage the manipulated market with others and so you'll wind up soaking all of those others as well.)
I wonder how one would go about determining this. Scott's manipulation attempt was about very low-stakes scenario about random Manifold person getting charged with a felony. Maybe manipulating that information is genuinely worth only of some 200 USD, for Scott. However, value of manipulated information about a high stakes political outcomes could be plausibly worth billions. (Say, war in Ukraine: think of price of high tech weapons, value of assets under stake, and then of course, the cost of all involved human life.)
Alternatively, if there exists some other market (say, an insider trade of Ukrainian chocolate factories or high-tech weapons stock futures) where the information is more valuable than the currently available prediction markets, a person with the valuable information would be advised to trade on the other market.
My point is that if one can actually trade on these markets (i.e. you aren't just hacking the market to say the price is X but not letting people take the other side, you're actually trading it at X), then manipulating, say, the Polymarket "will China invade Taiwan in 2023" question with billions of dollars would manipulate all "will China invade Taiwan in 2023" questions in other prediction markets (e.g. Kalshi) by proxy as people repeatedly arbitrage away the difference and you pour more money in to maintain the peg on your directly-affected market; all the prediction markets still give the same answer, it's just a wrong answer.
(In the specific case of China/Taiwan there are semi-plausible perverse incentive issues related to the uselessness of money if you die in a nuclear fireball. But my guess is that there'd be enough time for the market to resolve and you to spend the money before ICBMs start getting thrown, so it's not as fully perverse as "will there be a nuclear war" (which won't resolve YES until too late). In any case, you get the idea.)
Not disagreeing. My comment was less a direct reply and more a tangential pondering about what happens when the prediction markets are small or illiquid compared to value of prediction.
A limited nuclear war might not kill *everyone*, and someone far away from major targets might be able to profit from starting one and survive to enjoy their winnings. There are also numerous people who might enjoy seeing urban centers destroyed for ideological reasons.
Are you talking about insider trading, rather than the "prediction market stably doesn't give the right answer" issue I was referring to (along with manipulation)?
One point regarding the "will there be a nuclear war" market is that generally, like most businesses, prediction markets are headquartered in major cities; if the prediction market's records are destroyed before the market's declared resolved, you don't get paid (and if your bank's destroyed before you can physically cash out or pay for the stuff you want, again, it's useless). And, indeed, "a place away from a city" and "emergency supplies" are the things I was implicitly referring to when I said there'd probably be enough time to spend the money for a "will China invade Taiwan" market.
It's fairly hard to start a nuclear war deliberately, and there'd be a significant risk of hyperinflation and/or asset seizure in a bunch of jurisdictions on top of the "do you even get the cash and/or survive to spend it" problem, so I think the "every prediction market is also an action market"/insider-trading issue is relatively minor - the financial incentives don't work very well, and a prediction market only actually helps achieve something you want for ideological reasons insofar as it lets you give *other* people financial incentives.
I'm pretty sure I lifted the hierarchical court system, where each court is theoretically a prediction market about what the next higher-up court will say, up until you reach the Court of Final Settlement (Supreme Court), from somebody else. Possibly even Robin Hanson.
How about refining it with prioritization by transaction volume rather than selecting randomly? Or perhaps alternating, ruling on a case with the most at stake on odd-numbered days, then on even numbered days resolving whichever is currently trading closest to 50/50. Goal being to focus the court's scarce attention on matters of consequence and uncertainty, thus providing the least noise and most value per unit of judicial signal... and incidentally make market manipulation even more self-limiting than normal, since the harder someone tries to buy an unjust verdict the higher that case would drift toward official attention.
I have a strong intuition that "people who have money to spend on prediction markets" and even more "who also have an interest or knowledge of prediction markets" is an incredibly biased sample of people.
I'm just very pessimistic on the ideal vs the practice. The same kinds of negatives I consider for the stock market. Or markets in general. Prediction markets depend very strongly on right wing economics being accurate imo.
There are some cases where the equilibrium answer for a prediction market with perfect information is not the same as the true probability of the underlying event.
"The US undergoes 1,000,000,000x inflation by 2030", denominated in USD, is one of the obvious examples; if you assume for the sake of argument that this is a 1% chance and that everyone knows this, the equilibrium price for a contract that pays out USD$1 on YES is far less than 1 cent (because if YES is true, USD now is worth at least a billion times USD when the contract pays out, while this is not the case if NO is true).
Similarly, the price of "will the world end" is $0 on a standard prediction market because a resolution as YES prevents you from collecting. The same is true of "will the records of this prediction market get lost", which means a lot of extremely-disruptive things (e.g. nuclear war) don't show up in prediction markets.
Check the Wayback Machine; that section was added after almost a day. Substack doesn't give me exact timestamps, so I can't say for certain that it was after I said that, but it's pretty likely.
Also, I wasn't responding to the post directly, but to a comment making the argument that mispricings are necessarily exploitable.
The absolute gains will be smaller if you don't have much money but the relative gains should be the same. If you can double your money every six months, that should hold true whether you bet $10,000 or $100.
Actually, your rate of returns should go down the more money you put into prediction markets since you'll move the price.
My main question whenever I read about prediction markets is: why are they necessary? The best answer I can surmise from *light* googling is that it can serve as the most accurate representation of the wisdom of the crowd. Is that a good one-sentence answer to the question: "Why prediction markets?"
Interesting... so sort of like a way to cut through any propaganda or BS.
Example: Mainstream media/ideology is that X won't work, but everyone betting in the prediction market says X will work, since that's a more realistic indicator of the assumed body, it's more likely X will work?
Sort of. People have different biases, and when you take a large number of people without selecting for any bias relevant to the prediction market, then you get a lot of biases cancelling each other out.
It is similar to how if you ask individuals to estimate the number of marbles in a glass jar, they display wild variance, but if you average the estimates of individuals, the aggregate usually performs better than most individuals.
Except when people are putting down money because they want to make more money, and not because they hold a strong belief towards a particular prediction
Accurate and trusted, establishing common knowledge in the long run.
If someone doesn't trust it, they can make money. If they just claim they don't trust it, they won't be believed unless they put their money where their mouth is, and people become instantly less tribal and stupid when their money is on the line.
>As a test, I tried to manipulate the market on whether Austin Chen, founder of Manifold Markets, would be charged with a felony.
If I recall correctly, you did this with his explicit permission? It would be worth having that fact displayed next to that sentence if so, since otherwise this reads as a minor alarm bell.
Thanks for writing this! If only we could tag Rostin Behnam :P
In addition to betting on Manifold, ordinary folks in the US and abroad should also participate in the markets on Kalshi, Polymarket, Insight Prediction and PredictIt. Real money is the ultimate play money, as a reputation system that most freely transcends the prediction platform. You don't need to be a hotshot prediction expert - skin in the game allows you to learn quickly about your interest.
To promote prediction markets from a regulatory standpoint, the CFTC cares about the economic purpose of these prediction markets, namely their use to hedge risk and serve as a price-basing mechanism. Directly using these markets for information or to hedge risks and demonstrating a positive economic effect to your life and business is a central legal argument to getting these markets approved!
I had a concern with prediction markets that I think answered myself. Still posting it here for edification and potential correction if my answer was wrong.
> One concern I've had with prediction markets that this post doesn't address: is it good to expand the set of things that companies can hedge against?
> Let's say that I'm Satya Nadella and I'm worried that a startup might disrupt and take over Microsoft Office. Today, I can action that worry by investing into Microsoft Office, investment that presumably makes Microsoft Office better, increases the productivity of its users, and therefore raises the GDP of the human race.
> Let's say we live in Scott-topia and that I'm Satya Nadella and I'm worried that a startup might disrupt and take over Microsoft Office. I might decide that instead of investing in Microsoft Office and reducing the dividends paid to the shareholders that employ me, I'll buy a large stake in "yes" for the prediction market "[w]ill Microsoft Office make at least x bajillion dollars in fiscal year 20XX?" so I'll be set no matter what.
> To phrase my objection more cogently: what propels our society forward are the holes in the efficient market hypothesis; the innovations that are not priced into asset prices. Will large companies ability to hedge against those reduce the incentive to create and fund those innovations?
I think the answer to this objection is that the startups/VCs will take the opposite end of the trade with Microsoft and it'll end up being a massive waste of time for everyone involved (since they could just invest the money in their products) so they won't do it.
> I might decide that instead of investing in Microsoft Office and reducing the dividends paid to the shareholders that employ me, I'll buy a large stake in "yes" for the prediction market "[w]ill Microsoft Office make at least x bajillion dollars in fiscal year 20XX?" so I'll be set no matter what.
Do you mean "buy a large stake in 'no'"? If they bet on the prediction market that their main business will go well that's the opposite of hedging.
Hedging like that seems pretty dangerous though- if you drive up the price for "Microsoft Office gets disrupted" in the prediction market, that seems likely to hurt you in the regular stock market.
It is unlikely that the size of the market for Office to make X bajillion will ever be comparable to X bajillion and not 1000 smaller. Also, if market participants were aware that this market could be used by Satya for this purpose, they would quote pretty lousy odds for any size.
In general, market liquidity exists in size only if either 1)there are large participants naturally on either side of the market, like oil futures market is big because oil producers are naturally long and oil consumers are naturally short 2) the market can be hedged to a decent degree with an already liquid instrument.
People can already short Microsoft as a whole. Enabling them to, in essence, short Microsoft Office doesn't seem like it'd make that much of a difference?
There is a large market in MSFT shares and derivatives on them. There are many natural buyers and sellers on this market, e.g. investors who want to hold more or less of MSFT. If Office is a relatively small part of Microsoft business, shorting Office requires an entirely separate market , one that cannot be hedged well in existing markets. I do not see many natural buyers or sellers for this market, so I do not think this market could grow to a scale comparable to market in MSFT shares.
"Finally, I envision that someday people who want to know the answer to specific questions can subsidize prediction markets on them. For example, the Democratic Party might subsidize a market about which Democratic primary candidate is most likely to win the general election."
-- how would this market resolve? Only one person is going to get a chance to win the general election.
Interesting. I followed the link in 5.1, and got to the part which uses your own unbanning decisions as an example, because the world is a loop. Anyway, the example there suggests that in the Democratic example you would avoid a bunch of failures/distortions by not announcing in advance that the outcome of the set-of-conditional-markets will influence the selection of the candidates, which is a slightly disturbing situation because the whole system generally relies on everything being transparent.
I'm not crazy about the first sentence "Prediction markets are like stock markets, but for beliefs about future events." There are two crucial differences: most prediction markets have binary outcomes and they resolve at some point. This is not the case for the stock market. While most Americans are not familiar with binary options, I don't think using the stock market as an analogy is accurate enough. I'd just compare it to sports betting, even though if it may come with a slightly negative connotation.
Also, a slightly substandard stock market investment will likely still make you money in the long run, while a slightly substandard prediction market "investment" will lose you money. This is the difference between investment and speculation, and non-experts should always avoid speculation.
This whole FAQ seems to *at the same time* claim that prediction markets make the best predictions possible, *and* that there's money to be made by beating the current state of the prediction market. I can't see how these views are compatible.
One thing that annoys me about this article is the recurring thing about "if I'm wrong you can get rich quick", which I realize is sort of "a bit", but also feels like a taunt. "Well, if you're right, you could make money so stop arguing." No, if Scott [et al] is wrong, he's wrong, and whether I make money from it has nothing to do with it. (Also, it's going to be "place positive-expected-value bets that resolve over maybe years", not "get rick quick", but like I say, I know it's a bit.) I distinguish this, emotionally, from "if the market is wrong people will bid it towards correctness", which is intrinsically part of the argument.)
> This whole FAQ seems to at the same time claim that prediction markets make the best predictions possible, and that there's money to be made by beating the current state of the prediction market. I can't see how these views are compatible.
They become compatible when beating the current state of the prediction market is impossible, because the consensus is accurate. You're correct that these are diametric forces governing the market dynamics - the equilibrium determines the size of the market. Present status quo, markets seem to get it right within a few percentage points and the people who consistently make money there aren't making more than a typical software engineer.
Overall, coarse grained efficiency is compatible with fine grained mispricing -- indeed, it is betting on mispricings,and therefore erasing them, that keeps markets efficient.
Conversely, stock markets ARE for beliefs about future events (specifically, future stock prices), so the "but" doesn't make sense here. The differences, instead, are those Lech lists.
Not exclusively - there's also the dividends, and while to an extent this is a belief about the future as well (the company will actually pay dividends), this is at good odds. As is "we can expect the economy to grow".
This is why stocks are (typically) investments, while prediction markets (as well as gold and futures) is mostly about speculation (although futures have other uses, like in controlling risk).
You could frame it as: the stock market is like a prediction market, where people are trying to estimate the price of a share of a company.
The resolution in a prediction market is binary but "price of a share of a company" is not resolvable in a static fashion, because the company itself changes. At one point in time, a company might have been undervalued, and it'd have made sense to buy more stocks of that company. 10 years later, the board has changed, the leadership has swapped companies, and they're dealing with new competition. Now the problem of estimating the price of a share has a resolution again, but one different to the resolution 10 years ago. So, in stock markets, we do have resolutions, just dynamic.
- Much of this just seems like "efficient markets hypothesis" repeated in different ways. Is that not right? Is there a reason to think it'll be more accurate than the stock market's predictions? The only one I saw is the point about how prediction markets can't have a ponzi scheme go on forever, but that applies to certain stocks as well, and doesn't seem like a super broad objection in any event.
And the thing is that (1) I don't think most people would call the stock market "accurate" or "unbiased" except perhaps in a hypertechnical sense, and (2) the stock market needs a lot of TLC to make sure it works properly. There's insider trading, but that's hardly the only example. In fact, to go back to the crypto thing, part of the objection and the reason (allegedly) that it seems like such a shitshow is that they DON'T have the regulations the stock market has.
More generally I bet a bunch of economist PhDs have written a lot of useful shit about the efficient markets hypothesis, that this could probably benefit from some engagement with, if I'm right about this being similar.
- Is the idea that prediction transactions will be public? There's all this stuff about how you can follow Nate Silver's lead, but does Nate Silver have to reveal who his bets? What if he's just quietly sitting there, anonymously betting, not making enough money to have anyone notice?
- I think "in the long run" is perhaps a larger qualifier than people are assuming. To take presidential elections again, they are every 4 years, and the political landscape changes over time such that there's never all that big a sample size of analogous presidential elections to the current one. If Nate Silver is the best predictor (to go with the analogy, not saying it's true), maybe his biases simply match the current landscape and it'll be different in 8 years, or maybe given the small sample size (4 elections?) and the inherent uncertainty of judging percent predictions for binary outcomes it's simply hard to say who is best.
- As a test, I propose something other than big picture geopolitical developments that people normally associate with prediction markets. Sports! Much higher sample size (2430 regular season baseball games per year, and that's just one sport, and the outcome is numerical and unambiguous!), and there's already a lot of predictions out there you can compare to. Compare the sports prediction market to Vegas money lines/odds, pundit predictions, etc.
Prediction markets AFAICT have (in theory) an advantage over Vegas odds, which is that the house edge probably negates the positive EV from certain arbitrage strategies, which probably makes them less efficient in theory.
- Regarding the "zero sum" point, can't you take the money from the primary (i.e. not secondary) purchasers of prediction "shares" (or whatever you call them) and invest them in US government bonds, and the interest is paid out proportionally to the winners?
Sports betting market _are_ prediction markets, in that sports books respond to misprices (as revealed by imbalanced betting) by changing their odds -- either the actual point spread, in those cases, or the price to buy one side or the other. You're right about the house edge making it harder to exploit mispricing, though. Hell, in horse racing, the house edge is 17% on win/place show betting! More typical sports betting is about 5%.
"... suppose that an import-export business spends millions of dollars betting that Trump will lose in order to hedge against his protectionist policies ..."
Wouldn't that business want to bet he would *win*, not lose? (That is, if the policies of Trump's main competitor would be more beneficial, on balance, for that import-export business.)
If Trump wins, the business would presumably lose some revenues and profits due to his policies. But winnings from their prediction markets bets on his election victory could at least partly offset that.
(If I'm confused and have misunderstood this, welcoming corrections!)
Investing wide, like an index fund, or for non-Americans a global index fund, does have the added advantage (on top of being cheap) of removing yourself as far as possible from the risk of market manipulation and insider trading. You make yourself harder to exploit.
I think that, in the ideal case, prediction markets will work as you say, and that would be a good thing. Likewise, in the ideal case, cryptocurrency would also usher in an era of decentralized prosperity, and that would be a good thing too. But in practice, as soon as large sums of money (even virtual money) is in play, the entire utopian landscape shatters into a hodgepodge of Ponzi schemes and general fraud. This is the case with cryptocurrency; this was the case with the stock market; and this will be the case with prediction markets if they ever gain enough popularity to be useful.
In the case of the stock market, the problem was somewhat improved by regulation. There are still scams and clever get-rich-quick schemes, only now they have to be a lot more clever, and perhaps valuable enough to buy off a regulator or two. The stock market sort of works, but it would be foolish to claim that the value of every stock accurately reflects the true value of the company it represents.
Sadly, I don't think prediction markets will ever get even to that level of accuracy, because the incentives are much more entangled. Arguably, a share of "Joe Biden will win the election" is, indirectly, a lever that someone can pull to influence the election; thus, regulators with their thumb on the scale would wield considerable power. Selling the election on the conventional stock market is much more difficult (if not outright impossible).
Is it that much more difficult? You can buy or sell a package of stocks based on things that the candidate/candidate's party has said they'll do if they win the election. Coal mining companies' stock goes up if Republicans win, down if Democrats win. Same for renewable energy companies, anything that one party has talked about putting tariffs on, health insurance companies (stock goes way down if single-payer is instituted, up if individual mandate is enforced, etc.), etc.
This is incredibly and informative and well written. However, to the extent that I find prediction markets objectionable, it has very little to do with their predictive power, and more to do with the aesthetics of how the analysis is presented. I frequently, both here and in other rationalist spaces, see potential human disasters referenced in terms of prediction market odds. Whether it be numbers of future Covid deaths, mass shootings or the likelihood of Putin launching a nuclear weapon, someone is bound to bring it up in the context of prediction market odds, and those odds, definitionally, are defined by people who are framing the possibility of vast human suffering preliminarily in the context of whether they can make any money out of it.
I understand that the predictive power of the markets still have value in these cases, but even so, there's an "ick" factor I can't get past. It feels like reducing potentially devastating events to data analysis/investment opportunities without acknowledging the human toll plays into the worst stereotypes about the rationalist community.
I don't think this is weirder than eg citing a paper written by an academic who hoped to get promoted for it, or calculating the odds on a computer that someone at Microsoft or IBM got rich for inventing, or reading about them in the New York Times knowing that your subscription money is going to the Sulzberger family. You just accept that people have incentives and you trust them if the incentives are aligned with getting the right answer.
To be honest this seems horrifically misguided to me. The purpose of prediction markets is to get information about the likely hood of these events. In the case of disasters having more such information is essential to mitigate or prevent them. The fact that you could, in theory, make money by participating seems to me to be entirely unimportant by comparison.
Opposing innovations that will help save lives for purely aesthetic reasons plays into to the worst stereotypes about people who are ignorantly anti-rationalist.
I get it, and I should clarify that I'm more put off by the general discourse around prediction markets than the markets themselves. This is one of those things I would not have even bothered commenting on if I'd seen anyone else echo the same sentiment beforehand - it just feels like *somebody* should bring it up.
As to your last point, I guess I do feel like the rationalist community has issues with how it is perceived by the broader population, and, in general, somewhere between a reluctance to, and a principled stance against, modifying its behavior to address this perception. Which I get, but I also think that perceptions *do* matter, especially if we want to convince other people to join our community who may otherwise be put off.
I think pushing this point any further would risk some maybe legitimate accusations of "concern trolling", so I'll let it go at that.
I'm quite torn on this. I do think that perceptions matter and that there is value to trying to improve other's perceptions of this community. On the other hand, most criticisms against rationalism/EA/prediction markets/whatever are so *completely* illogical, that it seems mostly out of our hands.
For example, in the wake of the SBF implosion, Effective Altruism got a lot of bizarre and irrational criticism. I heard many people say things like "I agree with the ideas of Effective Altruism, but find it troubling that the first step is to get rich". Apparently none of the people saying that thought to check if the first step of Effective Altruism was actually to get rich. To the best of my knowledge this is not a view that is espoused by any Effective Altruism organization or public figure, not that it matters to them. Arguing against people making these sorts of strawman arguments is completely counterproductive, and that probably accounts for a lot of the reluctance to address perceptions.
Given this environment, it is hard to see how to actual make progress on the PR front. If you had specific ideas about how to solve these problems I suspect that they would be very positively received, but I'm not convinced its really possible.
I also think that suggesting people are excited about prediction markets because it would enable them to get profit off of tragedy is inaccurate and therefore more similar to the EA criticism example above than actual actionable advice about improving our perception, although I suppose I could be wrong.
You draw comparisons between prediction markets and stock markets, but I’m curious if you think sports betting is another useful analogy. The global market for betting on, say, the World Cup outcome is probably orders of magnitude larger than any of the relatively niche prediction markets you highlight. Based on these answers, I would expect one of four things to be true:
-The betting market is an accurate, unbiased representation of which team is likely to win.
-The betting market has a consistent, material bias, so Goldman Sachs hires quants who are experts at soccer to take smarter bets, make tons of money, and correct the market.
-The betting market has a consistent bias, but Goldman Sachs isn’t exploiting it, so there’s a ton of money on the table that someone could pick up tomorrow (or at least in 2026)
-The betting market has a consistent bias, but it’s not viable or worth it for anyone to correct it (like the prediction markets you mentioned today)
As far as I know, option #2 isn’t happening. I could be wrong, but I’ve never read outrage bait stories about how Wall Street won big by betting against America or something. And if this is just because of regulations, I’d still expect someone to try it by setting up shop in the Bahamas.
If option #1 is the case, do we see the benefits you predict? I’m not well versed in the world of sports, so this is an honest question. Do team owners look at Draftkings picks to decide which players they should keep? Do coaches see terrible odds against their team and decide to change strategies before they even play? If so, this seems like a strong argument in favor of prediction markets, and one you might want to highlight in another revision. But if not, I’m curious why you think these markets aren’t living up to their potential, and whether you think this is a counterpoint to your optimistic take on what markets can do.
If option #3 is the case, why aren’t you making tons of money on Draftkings? And if it’s option #4, then what problems do you think are holding back sports betting markets, and why don’t you think other prediction markets will suffer the same fate?
(I suppose option #5 is that this is a flawed analogy, or that I’m missing another obvious takeaway.)
It's also just not very usuable information for team managers. E.g. if I'm a coach and I see that we're 30% underdogs vs the 70% favourites, I could change my strategy... but then again, there's no reason to expect that a different strategy would change those numbers in a favourable way, and I picked the current strategy because I thought it was the best one.
I wonder if it would be legal to have conditional sports betting, eg "What are the odds that the Sharks win the game if they use Johnson as their starter?", "What are the odds that the Sharks win the game if they use Smith as their starter?", etc.
I think it'd depend on the state. But I'd guess there's no particular reason that would be different from betting on a general game outcome. Though I'm not aware of anyone who actually does this. But I don't see how it's materially different from (as does actually happen) betting on things like how many points will be scored by specific players.
The bigger issue is what happens to the betters when the scenario doesn't occur.
In baseball betting, this is actually the default case. Most baseball odds I see are conditional on the listed pitcher actually being the one who starts, and the bet is refunded if they don't (due to late-announced injury, usually.) You can also (sometimes?) bet on "regardless of who starts", but the pitcher is such a large part of baseball success that I don't why you'd bet that unless you had inside info or something.
This has the same problem with causal direction as a lot of conditional markets: if the Sharks' opponent knows that the Sharks will start Johnson or Smith depending on the outcome of the prediction market, the Sharks' opponent will change their strategy accordingly. To give an extreme example for clarity: imagine a predictions market on whether you'll win a rock-paper-scissors match conditional on having played rock, paper, or scissors, which your opponent can see operating and participate in.
I'd disagree, the inputs *into* the 30-70 figure are highly usable pieces of information for coaches and upper management.
Like the models gambling houses/analytic syndicates use to predict performance are basically exactly the same as what sophisticated teams are aiming to do in house. At least in football, and I assume NBA is the same. Knowing that X team performs better when Y player is playing rather than Z is valuable for both gamblers, bookies, and coaches for example.
There are quants who consistently make money at sports betting. But there's not a ton of money in it and the kind of people who can do it usually have more lucrative option. In part because banning it has made the market relatively small. In more sophisticated markets, mostly those where gambling is unambiguously legal, the best quants are employed by the gambling houses to prevent such edges from occurring. (Or rather, to provide the house itself with the edge.) The house can usually outbid independent actors in the long run.
As to why sports teams don't pay much attention: most leagues make any involvement with gambling extremely against league rules. Most prefer not to risk their careers by even being around the gambling houses. After all, they're making a lot of money just being players/coaches/etc. Why risk an ethics investigation?
In short, they're not really comparable unless prediction markets really are acting as casinos with the house offering specific odds generated by internal quants. Sports betting and casinos have an entire third actor, the house, that is absent from the stock market and should be absent from a good prediction market. Additionally, executives in business are allowed to hold stock (modulo insider trading rules) meaning they're under a less strict regime than sports managers. In part because stock ownership aligns incentives while allowing a team's manager to place bets can only unalign incentives.
Unlike a true prediction market, which is zero-sum, sports betting is pretty much always negative sum, that is, you can't make a bet and its inverse for some outcome and come out even. Bookmakers offer worse odds than would be "fair", and would probably ban anyone who for some reason is still able to win big consistently, so this is basically pure gambling.
Sports betting in the UK is negative sum, as @Xpym pointed out
Places like Betfair Exchange don't make money by overpricing flat bets like a tradiitonal bookmaker e.g. quoting 7/1 (12.5%)when true odds are 6/1 or something, but instead by charging a 5% cut on the profits made by people betting on the market.
Often due to low volume some individual markets fall into 4, cat 2 definitely used to happen more but now happens much less as the smart money came in, and cat 1 is now the default.
In fact, two of the smaller Premier League teams most associated with the belated (vs US sports) analytics wave in football, Brighton and Brentford, are owned by men who made their fortunes gambling professionally. The skill set needed is basically identical, predicting future performance in order to either maximise it or maximise earnings on gambling.
Their edge (alpha I suppose) is now smaller than it used to be, but they can punch above their weight due to better recruitment, set pieces, and finding arbitrage opportunities on the player transfer market by shopping from smaller countries.
I said something similar in my top level comment, but I think it's option 4, and the reason it's not viable or worth it is that the betting generally happens in places where there is a house edge.
So lots of arbitrage opportunities are better than "the market" but still not good enough to be positive EV.
(not that I'm necessarily bullish on prediction markets in general or the idea that teams can use them as a source of info)
I believe #4. Sports market predictions a) aren't unbiased (the people betting very often don't just try to maximize their gains, bit instead bet on "their" team in order to increase the excitement of watching the match), and b) are wide-open to corruption by rigging matches.
EDIT: My comment below is about the Forex market rather than sports betting, so it doesn't speak to your point. Leaving for posterity/accountability.
---
When Liz Truss announced her rather 'brave' UK budget earlier this year, hedge funds bet substantially against the UK by shorting GBP, including those based in the UK.
I can't immediately find outrage bait, but anecdotally I remember there being a decent amount of it (though significantly more reserved for the government).
I want to preface this by saying I like prediction markets and think they are a net good. But I think there are some clear problems with them that you've missed or hand-waved away in a fashion I don't find convincing.
For example: "4.2: Would prediction markets encourage harmful or illegal activities? ...I think the strongest evidence against is that this basically never happens in stock markets."
I think this is an awfully rosy image of stock markets. Because of many traumas over many decades we've built up complex systems of financial regulation that make stock markets less prone to exploitative harmful schemes than e.g. crypto is - but the natural state of a financial marketplace is to be scam-infested.
So it's illegal to run a "pump and dump" scheme in the stock market, but there's no such prohibition on doing it in a prediction market. Once real money starts going into these markets, you'll definitely see people doing things like buying up a bunch of shares in an outcome, creating fake hype to get people to believe that it's a real market trend, and then dumping their shares at an inflated price before reality sets in.
And of course there's any number of other market manipulation tactics that are known. You can set up a government agency to police these, and it might even work some of the time, but do bear in mind that the Securities and Exchanges Commission has a $2.5 billion annual budget. It's not enough to say "pump and dumps are banned", you need to have people with the job of prosecuting them, and that cost is not negligible. It might be worth it! But it's not negligible.
I also think that in "4.8: “Meme stocks” like Gamestop and AMC sometimes remain mispriced indefinitely. How do we know this won’t happen with prediction markets?" you don't adequately account for the "Market can stay irrational longer than you can stay solvent" effect on markets with resolution dates long in the future, or conditional resolutions that are likely to not be met. You mention these sorts of concerns elsewhere in the FAQ, but don't apply them to the failure mode where they are relevant!
Personally, I think it's absurd that Tesla is valued higher than all other automakers combined. But I don't short Tesla because I don't have any way to prick the bubble and force the stock price to reflect Tesla's real value as a company. There's no moment of truth - I could think the market is deluded, be correct that the market is deluded, and lose all my money betting on the correct value because it takes too long for the market to realise I'm right.
Now, betting on an election outcome in a couple of months doesn't have that problem - we know that the tide is going to go out and we'll get to see who's been swimming naked. But betting on, say, which candidate is more likely to win if nominated DOES have that problem.
Suppose the market is convinced that Clinton will be a stronger nominee than O'Malley. I'm Matt Yglesias, I'm sure that O'Malley would win, and I also happen to be right. But O'Malley never gets close to the nomination, and my correct belief never has the opportunity to be proven true. The market stays mispriced until it expires. My correct bet lost me money, and the market never accurately reflected the reality that O'Malley Would Have Won.
Now, these are sort of edge-case scenarios and I do agree that prediction markets are very good in modal cases. But I also think the edge-case problems are real.
You said basically everything I wanted to, but better.
I expect the price of, say, Microsoft to be fairly well priced on the market, but I have much less faith in the price accuracy of some nickel stock most people haven't heard of. Given the apparently huge number of prediction markets you'd like to exist... What creates sufficient economic value to drive accurate pricing for all of them? Particularly when, unlike stock markets, economic motives may not predominate for the most invested groups for a given question?
If there are many more investors them markets, maybe the markets are efficient (because there's no $20 just lying around then), but that doesn't sound like what you're describing. Your market-saturated world sounds like one with lots of pennies laying around. And I leave those where there are all the time.
> So it's illegal to run a "pump and dump" scheme in the stock market, but there's no such prohibition on doing it in a prediction market. Once real money starts going into these markets, you'll definitely see people doing things like buying up a bunch of shares in an outcome, creating fake hype to get people to believe that it's a real market trend, and then dumping their shares at an inflated price before reality sets in.
This already happens. One of the nastier moves I saw on PredictIt was to spike a particular market to deliberately trigger other participants' stop-loss orders, though that only works within a narrow range of sophistication.
Aren’t conditional markets set up such that Yglesias wouldn’t lose or gain money betting that O’Malley would win if nominated but he never gets nominated? That seems to alleviate that concern that Yglesias loses money on this, except for the opportunity cost. (It also seems like you can reduce opportunity cost here by using the same dollar to bet all the mutually exclusive conditional markets that condition on different outcomes of the same event. Is that already done?)
4.6.1.1 - This claim that hedging won't bias the markets seems way too rosy. There's not a bunch of people who all know the exact probability and one person using the market for hedging. There are a bunch of people all with different guesses as to the probability and different budgets/amounts of acceptable risks about how much they are willing to invest to correct errors. The (perhaps money weighted) average of these people is the "correct" value that the market should be trying to converge to. But if someone is investing a bunch of money as a hedge (rather than an honest prediction), this will tip the scales somewhat one direction or the other.
4.8: Perhaps the better argument here is that it is *much* easier to bet against a meme prediction market than against a meme stock, making it a lot easier for the market to correct itself.
Predictions markets that only evaluate if the chance is high enough (such as in the case of your comment policy) have some weird effects that I think could invalidate the results, and might not be fixable with the usual "free money" argument. If I wanted to make sure that a comment never reaches your attention, I can vote it down below the threshold for it to evaluate, and even though I am totally wrong on my "prediction", I'll never lose any money. There is still a chance at free money, but now the consequences to being wrong are removed (as long as you are rich enough to overwhelm the market). Perhaps this works better than I am thinking in practice, but especially for low traffic votes like this I think it would be easy to pull off.
On the opposite side, if I predict that you **won't** think that a comment was worth your time upon review, I have no incentive to vote negative below the threshold, since I won't get any of the money I am expecting to make. If I have sunk enough money in, it might even be worth my while to vote positive (against my prediction) in order to get it back above the threshold so that my bets get evaluated. I think this is less easily resolved than the first issue, and you should basically never see prediction far below the threshold because all that does is tie up your money while you wait for the market to close.
Prediction markets also monetise disinformation. Instead of looking for mispricing and then getting rich until the market corrects, you can spend money on a disinformation campaign to induce mispricing and get rich until it corrects.
This is not very different from pump-and-dump schemes in penny stocks, but providing a continuous, liquid, anonymous incentive to spread disinformation on *every* topic seems like it should be a concern. Not because the market won’t correct - because of the second order effects of increased information pollution everywhere else in our society.
In the optimal case this provides financial incentives for very robust anti-disinformation detection. However, solving loss of trust by drawing all of society and our information and decision-making platforms into a high-stakes HFT battle for truth might turn out to be like throwing water on an oil fire.
This assumes that the misinformation is coming through a channel that is otherwise considered very trustworthy, or else most people with money invested would probably ignore it or only adjust their bets a small amount.
Or, in the alternative, this means that people who are easily swayed by misinformation will lose money to people who are not swayed, which on net improves the prediction markets.
The benefit of making people put Skin in the Game is that they will necessarily become more discerning about which information sources they actually believe when trading.
I think I'm not understanding why 4.6.1.1 is free money. If I buy a share of Biden at 10 cents, I have a 50% chance of making a dollar, so my EV is greater, but I still have a 50% chance of losing money. Is the free money part:
(a) The EV on that trade being >0
(b) I can at some point sell that share to another trader for 50 cents
(c) I can take a strategy of buying shares for the less-protectionist candidate every 4 years and make money in the long term if hedging makes those candidates are consistently undervalued
(d) The import/export company is also hedging in other markets, so I can bet against them in every market and make an overall profit on those trades
(e) Various groups hedge various markets and I can take a strategy of betting against them in general and make an overall profit
(A). You might lose on an individual trade, but as the number of similar trades you make increases, you'll be more and more likely to gain on net. This might not be worth it for you, but it will be worth it for somebody.
I think the real issue with making a better prediction market is that it's such a niche product. Looking at the top eight most liquid predictions on polymarket they currently have a total liquidity of about $136,000. And there's only about 60 questions total. Let's be generous and roughly double that for $250k total liquidity on site. That's tiny. That's smaller than most minor league baseball games. Let's say there are five equally sized markets. That's still only about a million dollars. Let's round to a million (fair since one of them doesn't use money). Given a casino's 8% take (which would actually probably be lower since this isn't a casino) that's $80,000 in revenue. And that has to cover everything for the entire industry.
I can think of ways to build better versions. But if almost no one's going to use it then what's the point? At those rates I'm not even sure the revenues can cover regulatory costs. Regardless of whether it's a good idea I'd need to be convinced it could survive and hopefully even thrive.
Has anyone considered getting all that rationalist money behind a prediction market in an already high demand area like sports? That's a multibillion dollar industry with heavy existing demand and high growth. Yes, it's not as EA-sexy as stuff about politics or whatever. But fundamentally you'd be developing the same type of skillset and organization and technology. All that which you could eventually pivot into broader markets. Part of the point is such markets are useful for predictions in arbitrary fields. So why not arbitrarily choose the large, fast growing market?
But sportsbetting exchanges already exist. What value would a sportsbetting prediction market add? Come to think of it, Betfair does political bets as well, so I don't see what value political prediction markets would add either. 🤷
I can think of sports betting gambling houses (Fanduel, Draft Kings, etc). But not prediction market style exchanges. At least not in the US. More to the point, if this already exists then what's the point of it at all?
I'm surprised this article didn't look at the UK/Irish/etc. markets as live real-world examples of what a prediction market can do, since there's already fairly large-scale wagering on political and other real-world events. Granted, you've got bookmakers setting odds rather than purely market-driven pricing, but for liquid markets the difference shouldn't matter all that much.
To your point Betfair now owns Fanduel -- in theory the existing betting operators are best positioned to take advantage of eventual changes in the laws/regulations. Sports betting (increasingly legal in the US) is already fairly structurally similar to the sorts or questions that resolve in yes/no fashion on a prediction market.
I'm not sure I agree that you can't disrupt existing incumbents.
I very much agree that doing this more internationally would be a good idea. I proposed months ago that prediction markets, if they truly want to be accurate, need to start aggressively recruiting in other languages and outside of fairly narrow demographics. But that got a "nice idea but..." treatment. My first thought was literally "move to Macau or Ireland or something." But I don't think I could even get them to move from Berkeley to San Jose.
"Has anyone considered getting all that rationalist money behind a prediction market in an already high demand area like sports?"
The industry is salivating at this sentence. "Look, a whole new cohort of people who think that because they're smart they can beat us at our own game!" Or, for the stock market:
If you believe prediction markets are important you want resources to be allocated to develop them. The resources necessary to figure out how they work, build the technology, and lobby the politicians. If there's a big pot of resources in sports then it's better that you outcompete incumbents and use that to develop what you think is world-historically important. Especially because if you believe in the morality of prediction markets you probably believe in the morality of sports prediction markets. (Also, I would bet that a bunch of SF nerds can build a better DraftKings.)
That recommendation sounds like "has anyone considered getting EA behind a political campaign?" and as we saw, Carrick Flynn was the wrong candidate in the wrong area to back.
Take betting on this year's World Cup: France, the holders of the cup from the last competition, vs Argentina, and it looked like Argentina were going to win for most of the game. Then France equalised with two goals within two minutes. Played extra time, both teams scored. Went to penalties and the French made a dog's dinner of that so Argentina won.
Even good forecasters would still be uncertain as to who the winner would be, even at the point of knowing that both France and Argentina had made it to the final. Going on past performance would be little help, as both of them had won the cup twice before. Sports betting is tough, tougher than it looks, and getting into that market means you have to believe you can do better than the bookies. A lot of people have gone bust thinking they could do that.
The difference is that while I doubt EA has many successful politicians I'm 99% certain that EA has a lot of successful startup people and math nerds. It's actually in their area of expertise.
Many of these examples are political questions, which you would always expect the masses to be right about in aggregate, since the democratic process is about aggregating opinions. Some of them are less political. But what about a question like "will it rain tomorrow in New York?" To make a bet on this question, the only sensible strategy would be to check what weather modellers are saying and bet with them. But because this is the only winning strategy, would anyone actually make a bet here? I guess the thrust of my question is - shouldn't there be very little money to be made on problems that are genuinely hard?
I think the phrase "weather modellers" is doing a lot of the work here. If there's one universally acknowledged best weather modeller, then this reduces to the Nate Silver example in 2, and yes, everyone should agree.
If there are many modellers and nobody knows who's best, then this reduces to the competing experts example in 3.5, and a prediction market will aggregate them efficiently.
If you're an amateur weather modeller yourself, and think you're the best, then you should consider betting on your own opinion, as in 3.4, and making money if you're right.
Even in the Nate Silver case not everyone should agree. Supposing he is the best modeler doesn't mean that finding out what another modeler thinks shouldn't shift your probability. That's basically private information like knowing some Dem canidate for the house has a dark secret no one else knows about (should shift your probay slightly).
Also, let's assume Nate Silver is the best predictor and people follow him more or less blindly in prediction markets. Now he can (or could, with enough scale) make a lot of money by making a faked prediction and secretly betting against it.
Theoretically doable under some circumstances, but note Nate's argument for why he doesn't bother betting the model line in existing markets: the reputational returns on making the model as accurate as possible far outstrip what he personally could make by betting.
Currently, for certain, but that's because of lack of scale. Once enough money is involved, people (not everyone, but some) _will_ start to misbehave. Betting and gambling aren't exactly clean businesses.
In general, sure. I expect misbehavior to increase as the stakes get larger. In the case of a known best predictor though, it's not dissimilar from being willing to fudge the model in exchange for a bribe. Once enough money is involved.... give Nate a raise!
Okay, that's a good point, maybe there are a few months of figuring out which weather modeller is best. But after that, won't everyone just figure out which it is and bet with them? Weather modelling is very computationally intensive so I doubt amateurs could beat the established ones.
It won't quite work like that. It's not only the best forecast adds value. NOAA doesn't just use whichever of the us/eu hurricane models is the most accurate but instead blends them.
Think of it like this. Suppose someone flips 200 coins and someone sees 100 of them and I see ten (say choosen randomly so they may or may not overlap). The person who has seen 100 coins announces their best estimate of the total number of heads is 110 (because they saw 60 heads). Now suppose that you only saw 3 heads out of your 10.
Even though the person who saw 100 coins is the best predictor of the total number of heads you should still bet that the number of heads will be slightly less than 110 because you also have a bit of information that's not completely correlated with theirs. Since you have their info and yours you can do even better than they can.
Same with weather forecasting. Now you could imagine one day we get a weather model that includes every piece of data and technique we have justification to use but currently that's not the case. Same way that NOAA doesn't just use whichever of the us/eu hurricane models is the most accurate but instead blend them.
Completely parenthetical to your main point (which is interesting and seems right to me), I'm finding it a little doubty that all the hurricane modelers don't have access to the same data. Why would that be the case? In a field like that, you'd expect data to be widely shared -- certainly that would the case for any data gathered by a goverment or government agency. If true, that would suggest the difference between hurricane modeling predictions reflects differences in the models, not so much (or at all) differences in the input data.
If literally everyone agrees the weather is best predicted by weather modelers then you don't need the prediction market in that case. If there is disagreement on the probability then there is incentive to invest.
Let me add that in many markets (eg politics) ppl will have extra information (I know that this canidate has a secret or that she sometimes get tongue tied) and the people with that information have an incentive to enter the market.
This is an interesting point. So maybe prediction markets will naturally fall out of favor for certain types of prediction. But that does bring up the question of trust in institutions - how do skeptical people predict out the weather if they don't trust the modellers, and there's also no prediction market for it?
No reason to abolish the prediction market in a formal sense, just let people play within a point or two of the modellers' predictions with their lunch money. If there ever is a big miss, money can flow back in as useful and necessary.
Not suggesting abolishing the market - I'm suggesting that people will naturally stop betting on predictions like this, since beating the modellers is very hard.
I know very little about prediction markets, but one big obstacle to putting any serious money into them seems to be the timeframes involved. Who wants to tie up their money into predictions that would resolve in ten years? Or even five years?
If I was *very* confident in a prediction that would be resolved next year, then yeah, maybe I'd accept freezing my money for that year. But there's no way I'd ever try to freeze up any significant amount of money for an absolutely whopping *five years*, even if the market seemed comically mispriced. And in practice, any mispricing you'd otherwise want to correct would probably be in the single digit percentages anyway, so freezing money for that long simply can't be reasonable. The current inflation rate is what, 6% per year?
So I'd expect prediction markets to be fine at short-term questions, but pretty bad at long-term questions, because inflation makes sure that nobody actually has a real incentive to try to correct a 10% mispricing five years out from the resolution time. Am I missing something?
I didn't see you mention that prediction markets could easily simply peg investments to the stock market. The market can take your deposit and simply invest it in the market (so instead of a dollar contract you buy a dollar worth of stock contract).
More generally, this is really no different than the issue of leverage in the market. You don't need to actually deposit a dollar to buy a dollar contract, only produce sufficent collateral (and pay a fee) with a margin call happening if your collateral value drops too far.
Which the stock market? *I* might be fine with a S&P 500 index, but for anything you pick you've necessarily excluded the predictors who don't want anything to do with that investment.
Umm, I guess but I dont see how its any different than denomination in USD or any other good. That exposes you to fluctations in currency value. All goods can change value in relation to others.
It's different in that USD is a better medium of exchange than index shares, currency is a special kind of good optimized for exactly that purpose. If we want low friction it makes sense to use a widely-accepted stable currency, but in doing so you inherently limit a position's ability to hold value.
But you need to distinguish between the use of money as a store of value and a medium of exchange. The exchange still pays you in dollars it just invests them in the market when they are deposited so the payouts incorporate any changes in the stock market.
I've heard of a thing where you can chain prediction markets, so there's a market that pays out in 2030 based on the outcome you're trying to predict (climate change or whatever), then another one that pays out in 2029 based on what the current price of that 2030 market is in 2029, then one that pays out in 2028 based on the price of the 2029 one, and so on. So right now you bet on the one that pays out in 2023 and only tie up your money for a year or less. Similar to Scott's example about Biden pledging to move prediction markets.
I think the idea is that you need to able to sell your prediction. For instance, you buy a prediction bet 30 years in advance. If the market is mature, you should be able to sell this prediction at a (likely small) profit in six months assuming your bet now seems more likely to happen in 30 years, where someone else is okay with the longer horizon.
Is there a risk of rich people using prediction markets not just to make money but to influence real-world events?
Suppose the tobacco lobby wants the tobacco-friendly candidate to win the election. They bet lots of money on the other candidate and make the market imbalanced. Lots of ordinary people notice the imbalance and buy shares on the tobacco-friendly candidate. Come election day, the vote is looking close-run, but there are lots of ordinary people who own shares on the tobacco guy, which they probably paid between 10 and 50 cents for, so if he wins, they'll get a 2x to 10x payout, and if he loses, they'll lose their investment. That would be enough to make several people who were undecided about who to vote for, or who might not have bothered to vote at all, vote for the tobacco guy - which may well swing the election.
This is more than just the tobacco lobby hedging against an outcome they wouldn't like; this is them paying to influence the outcome of an election, effectively buying votes.
How is this different than the tobacco company selling shares? If you are a shareholder in the company you lose money if the anti-tobacco company wins.
Indeed, this scheme is strictly worse than selling options at a slightly discounted price to buy tobacvo stock at the expected price it will have if the anti-tobacco canidate losses. And this is legal (or at least not enforced).
Also I don't think it's a real concern. In a liquid market, you'll expect that these instruments won't mostly be held evenly by voters but by some big investers. Better to spend money buying ads which get to voters generally.
For this to matter prediction markets would have to become so ubiquitous and used by everyone that I wouldn't worry. Even if prediction markets grew a lot, it would be hard to find the free $100 bills to the average person because very few fast actors would grab it before they can act.
Also, I believe candidates usually want to be seen as winning before the election, so there might be a bias to vote for the winner that pushes against this.
No individual incentive. The odds of a single vote deciding the election are very small, and secret ballots mean there's no way to conspire because there's no way to detect defectors.
I don't understand how your comment follows from my comment. What I described could work with a single individual or group unbalancing the market (using money, not votes), and then *lots* of individuals individually incentivised to vote a certain way, no conspiring required.
But my point is that there isn't an individual incentive, because it's very unlikely that any one market bettor's vote will decide the outcome. (This is why "rational ignorance" and "rational irrationality".)
"For whatever reason, most people interested in prediction markets are American, so Polymarket has a limited userbase."
This is by far the weakest part of an otherwise great post. If prediction markets are legal elsewhere and are in theory so great, then why aren't they big there already? Is Polymarket really freely available in the rest of the first world?
Ah, so it's crypto-exclusive? Then, fair point, I wouldn't expect mainstream adoption until a credit card is sufficient. But also amusing that Scott didn't think this important enough to mention.
But if Scott is wrong that only Americans are interested in prediction markets and prediction markets are legal elsewhere, shouldn't there be successful prediction markets that don't use crypto?
(I also don't understand why only Americans would be interested in prediction markets. Don't other countries have all sorts of gambling and financial markets?)
Well, I guess if every country was equally good at finding business ideas and executing them, US wouldn't be the richest (large) country in world.
I'm from a non-US first world country, and searching "prediction market" in my language only returns some media articles and business reports. Looks like there isn't even a wiki page for it. Perhaps ideas just don't spread as fast as it's theoretically possible? After all, the idea for prediction markets themselves floated around for decades before the recent "boom".
I think your argument that prediction markets are hard to influence needs some more qualification. In particular, it's not that special interests can't manipulate the market. It's that they can't manipulate it in a way that causes it to diverge from the actual probability of that outcome.
For instance, suppose prediction markets get widly adopted and trusted and there is a prediction market for whether there will be a run on the banks in X country this year. If someone wants the banks to fail they could start buying up contracts that say the banks will fail. As that changes the price in the market which in turn makes it more likely the banks will fail it's not true that the market resists manipulation (even if you know what's going on and have a ton of capital it's not clear you don't also bet on failure).
This is important as it creates important limitations for the use of prediction markets in making decisions. If you do things like couple voting or government policy to prediction markets it may be that the markets accurately reflect probabilities but they may bring about undesirable outcomes in doing so (eg bank run example).
When I hear about stuff like this I always get hung up on the biases of the arbiters. Obviously you can add more and more specific resolution criteria, but I would worry about a right wing market deciding that the answer to "will trump win in 2020" was yes, because the person in charge of the market believes the election was manipulated (or even just refunding it all rather than call a no). Or something like "will we be in a recession Dec 2022" being refunded only by some markets because one market determiner beleives the government has vested interests in not announcing one even though there's obviously one happening right now. Just disagreements on judgment calls, when to refund, and eventually you could get two markets, but left-market says candidate has 30% chance to win, right-market says candidate has 40% chance, and there's no free money to be made because there's a 10% chance that one market or the other disagrees on the state of reality by the election. At that point they lose the benefit of being authoritative.
And again, I know you can narrow it down much further than I did here in my examples but especially in cases of when to refund, it seems like there are too many outside influences to properly phrase all of them. Who are "experts"? Who are "developers"? what is "commonly understood"? What does "interference" mean? Does a tweet count as an announcement or does it need to be at a press conference? Does covid count as a "disaster" causing a market to refund and if so, at what point does it stop counting as a disaster?
That being said, a bunch of question styles dodge this problem pretty easily, the "what would I decide/what would justice decide" style questions would be safe.
"As a test, I tried to manipulate the market on whether Austin Chen, founder of Manifold Markets, would be charged with a felony. There’s no reason to think he should be, so the price started at 5%. I spent $200 in Manifold’s play money bidding it up to 95%. Within an hour, other investors noticed the mispricing and corrected it back down to 5% again."
How did the other traders know that you were manipulating the market, instead of trading on inside information? Realistically they didn't, they were just trading on their priors. That worked out in this case, but if Austen really had been about to be charged, there wouldn't have been any way to find out about it from the prediction market faster than from the news reports.
In general there's some tension between the market's ability to resist manipulation and the market's ability to respond to inside information, since they look the same to other traders. I'd expect this to equilibriate in different ways on different questions, depending on how plausible inside information was: you probably can't manipulate the "climate change in 2030?" question, but probably can manipulate the "Biden to resign in 2023?" question.
EDIT: this affects the hedging analysis too, come to think. We should expect a position taken as a hedge to move a functional prediction market, because the other participants can't know that it's a hedge and not the product of private research.
I think that worked out on personal knowledge ("I know Austin, he's not that kind of guy!") but for other manipulations, I don't see why someone would not assume "Hey, somebody just bet that Joe Biden would be arrested for dog-fighting. Either they're crazy, or they know something we don't, maybe they do have inside information, maybe I should buy on Joe to be arrested" where there isn't personal or inside information to judge "Nah, this is a trick".
I think the more likely, or smarter, option is to just not bet against what should be obviously crazy stuff. "Joe Biden arrested for ________" seems obviously crazy, to the point that nobody sane would bet on it being true. If someone put a lot of money down on that option, I would avoid it, even given that it's *probably* free money.
Well, suppose three years back somebody said to you "I propose a bet that an appointee of the Biden administration will be arrested on two charges of stealing women's luggage from airports" would you have said "That's crazy, and I'm just taking your money if I take that bet" or not?
There's a lot of things that sound "No, that can't possibly happen" and then they do.
If I saw a very specific bet about an independently implausible scenario (like, specifics including number of charges and details of charges, rather than "Biden admin appointee arrested") that's a red flag for insider knowledge. If it was three years ago and not so specific as to set off red flags, the judgement without hindsight would say take the free money. That's tied into one of my concerns with prediction markets - low chance predictions are still non-zero, and once in a while some 99% bet will fail and people will update with the absolute opposite of what they should.
> I think that worked out on personal knowledge...
Sure, and that's totally reasonable, but if you already feel like you know the answer too firmly to get pushed around by illegible price signals, why have a prediction market at all? Just stick with your prior and live your life.
An important feature of prediction markets is that their prices ARE NOT probabilities. I think you are mostly careful to use words like "likelihood" and I appreciate the need to keep it simple, but there are several real world effects that distort market prices relatively to probabilities quite a bit and it is useful to keep them in mind.
1. Time value of money - one would rather have money now rather than later. This is measured by the risk-free rate of interest, which is now around 5% in USD. Rather than bet $1 to get $1 in 1 year time on a prediction market, one could invest $1 in a portfolio of very safe 1 year bonds and get $1.05 in one year.
2. Credit risk of the exchange. Betting exchanges go bankrupt, are hacked, etc. This is not a theoretical possibility - 10-15 years ago there used to be a decent prediction market called intrade.com. I made there roughly x8 of my deposit on a series of sports and political bets, but this amount was not high enough to bother moving it in and out of exchange when I was not trading, so I lost 100% of it when intrade.com folded. Rather than invest $1 in a portfolio of high quality bonds, one could lend money to the betting market, or a bunch of equally dodgy blockchain companies. I do not think that the interest rate on such loans (illiquid, unsecured, to non-transparent companies of unclear regulatory status) would be lower than 15% in the current market.
3. Transaction costs. There is a non-trivial transaction cost, both mental, time spent, and commission based required to deal with many of these markets. I think putting it at 2% would be conservative for relatively small amounts bet.
These three effects create no-trade boundaries for a rational bettor. I can think that my predicted probability of an event Q is higher than market price P but I would not buy unless the difference is high enough to compensate me for the effects of 1-3. And I would not sell unless Q was sufficiently below P. These boundaries become quite wide for large time to resolution, but they are non trivial even for relatively short times. For example, using the parameters as above for a two year market, one gets no-trade boundaries of (-0.28, 0.09) for market price of 0.05, and (-0.02, 0.36) for market price of 0.25. Basically, it means that for market price of an event up to 0.25 no rational player would be selling at that price even if their subjective probability assessment was exactly 0%. So any market longer than a few months with rational traders is pretty useless for assessing low/high probability events. Even for a toss-up market price of 0.50, the no-trade region is pretty wide at (0.31,0.69).
Almost all of his arguments assume a large stock market style exchange. At that scale I think these concerns disappear. Without it you can basically write off the prediction market as pretty useless.
This is not about being "large" but about a range of measures to lower credit and funding risks. For modern exchanges this involves a lot of complex things, like risk-based margining and cross-margining, multi-tier broker access system, a complex waterfall of liability in case of member default, etc. Also, these things work because most members have direct or one step removed access to central bank liquidity. Even then, it is not obvious this system works very well for multi-decade bets as the recent UK gilt/LDI meltdown has demonstrated.
If the arguments are applicable only to these kind of exchanges, then the prediction markets to which the arguments would apply do not exist and are unlikely to exist for quite a while :)
100% agree that those prediction markets don't yet exist and will not for awhile. Right now the markets tell you the probability that prediction market nerds think is right after a limited amount of effort with plenty of bias.
So yah, until Goldman can make billion dollar moves in the market it's not likely to be that accurate or useful. At that point all the usual financial services stuff comes in and while you can't eliminate risk of exchange collapse if the NYSE went down it's fair to assume that so too would the value of all your investments.
> 4.6.1: Then why should anyone play prediction markets, when on average they’ll only break even? It seems like this is a worse deal than stocks, which tend to go up over time.
You suggest subsidies, but do we even need that? Can't the market makers just put the money in the bank and return the money along with interest?
> 5.2: Politician pledge markets
Not a bad idea, but I don't think this will actually happen. Politicians can already do this with regular market. For example, a politician who wants to build solar energy system can promise to push up the price of some solar energy index fund, but I don't remember anyone doing that.
Also, regarding the "meme stocks", another point to be made is that betting against the bubble is much easier since the downside is limited, unlike shorting stocks (as long as we're talking about simple binary outcome prediction markets).
The stock market in general can't be manipulated, but then you have "madness of crowds" behavior like speculative bubbles that eventually crash etc. In principle you can "get rich quick" by shorting the bubble, but if it's you against the majority of investors, you probably don't have enough capital to budge the price. Or, in other words - your idealized argument would imply that the crypto bubble or the 2008 housing bubble shouldn't have existed, which is clearly wrong.
So I'm a huge fan of legalizing prediction markets but I think it's worth considering a serious downside: it effectively requires legalizing manipulation (ie letting people with positions in the market move it via their pronouncements).
We *want* people like Nate Silver to publish his model and generally for experts to offer their expertise to the public. But if we force them to choose between this and investing in the market we create a strong incentive not to share information publicly which could result in less good predictions. Besides, such limits would likely be unconstitutional unlike as to stocks.
I don't really think this is such a bad thing. Norms about conflicts of interest seem like they can handle it and it's no more of a danger than scakmy reverse mortgages. And I'm not convinced it gives experts any more conflicts than they have now (indeed it let's them bet on what they really believe).
That’s an interesting point. Right now, Silver’s logic is very transparent and we can inspect it and learn from it. If he were instead using that knowledge to bet on a market, he’d make the results public (via the market) but would be incentivized to keep his underlying reasoning entirely secret. That seems like a very large loss to humanity if that’s how everything gets done.
i like prediction markets, but i'm a lot less bullish on their potential to make broad social impact
prediction markets are nerd shit, and a lot of normies hate nerd shit
OR, you have a lot of normies who like nerd shit precisely when it happens to tell them what they want to hear, and say "Well the nerds can't know everything!" or "Remember that time it said 85% chance and the thing didn't happen? That just proves how inaccurate they are!" People will do this roughly to their hearts' content. Large portions of the population believe in homeopathy, intelligent design, fat burners, Qanon, etc, and many of those people aren't even dumb or uneducated. It costs them to believe in wrong things (in not so different a way it costs you to bet wrong in a market), but their need/desire to believe what they want to believe outweighs the costs they suffer, apparently.
Altogether, normies will sometimes listen to nerd shit, like weather probabilities or an economic analysis of the likely cost of some proposed bill or the stock market or the scientific consensus on global warming, and they will sometimes adjust their beliefs or behaviors accordingly, but they often won't. They may continue voting, acting, spending, talking in ways that are wholly at odds with the realities that prediction markets reveal, because they either can't wrap their heads around what this nerd shit actually means (esp if they have poor understanding of stats/probability, or distrust prediction markets because they don't have enough understanding of economics to see why they work), or because they'd rather spite the obnoxious nerds than make sensible decisions in their own lives, or because they would rather hold on to some comforting belief than make the ostensibly rational decision (i mean, the goal of life is to have fun anyways, not to make decisions that can be justified as "rational", so in some sense it's rational to do things that irritate rationalists because it's funny)
So not to say it won't make any dent on how normies think talk and act, but I think it might be a pretty modest dent. Prediction markets could be gods in the nerd world, but amongst those with actual power, they may be chihauhaus
Prediction markets won't change normie beliefs to a significant extent - the real goal is to improve the quality of nerd beliefs. Robin Hanson's original article was titled, "Could Gambling Save Science?,"
It includes citations to the well documented failures of peer review as of 1990. These failures have become even more widely documented since then. So the first priority is to hold academics and intellectuals to a higher standard.
Before we can make progress on improving the signal to noise ratio in public discourse, we first need to develop a more reliable approach to identifying a signal to begin with.
People who believe that academic publishing and reputation systems are even remotely effective may not see much of a need for reputational bets and prediction markets.
true - i just think that if it doesn't make much dent amongst normies, it can only have limited potential for large scale social impact. It will have _some_ impact, because improving science and other nerd activities is good for a variety of reasons, but there will be many big areas that are resistant to these improvements, i think
While the theory sounds nice, I fear that once translated into real world and widespread use, then when presented with "either the market is.... or get rich", people will go with 'get rich'.
Nobody buys Amazon shares because they think it's a great company and is properly priced, they buy shares because they hope the price will go up, and they will get a return on them. (It seems that "shares pay a steady dividend and prices don't go up or down very much" is no longer the way people make money).
Why expect a prediction market to be different? "Yes, we all agree that the Democrats will win the next election, the shares are priced correctly at 90 cents" and then nobody buys or sells shares and your market is stuck.
'Prediction markets all speak with one voice' - then why have three or six or ten markets, why not let one giant market eat them all up?
When it's a question of "did the Democrats win the election, yes or no", that can be resolved (unless somebody takes a court case about no they didn't). But when it's something more fuzzy, who makes the final resolution that yes, Messi is the greatest player of all time? You need somebody to do it, and that may well be an expert. Or there will be arguments over resolving the question because one set of investors will say you should measure it this way, another set say no, measure it that way.
If prediction markets take off, I don't expect them to be much different from the stock market or online bookies. Fine if anyone wants a bit of fun, or to try and make money. But God preserve us from mayors to presidents deciding to set public policy on the results of a market buying shares on "should we drain the Mississippi?"
"7.6: …if I’m the head of the Commodity Futures Trading Commission?
Please legalize real-money prediction markets in the United States. If it helps, here are a bunch of really famous economists including a Nobel laureate explaining why you should do this."
The CFTC might just be the least bit busy with something else going on right now about another new, ground-breaking, future of finance project:
(If that seems a little unfair, remember that FTX et al. came out of the rationalist/EA aligned communities, not just some common-or-garden Ponzi scheme. "But all the rationalists think this would work beautifully" is not necessarily going to endear prediction markets to the CFTC).
I don't see one particular issue addressed: prediction markets can't make accurate predictions, but only aggregate the (market's) population's opinions.
When the Brexit vote was to be decided on Thursday, I discovered on Wednesday that, if the vote was to leave instead of remain, the stock markets would fall. I had decided the vote would be to leave, for non-financial reasons, so decided to bet the market would fall. That bet made me a small amount of money, but not as much as I lost elsewhere in the market. Of course, back then, I had never heard of prediction markets at all, anyway.
The markets were different from the population because, well, the pool of people betting was different from the pool of people voting.
The market can only make accurate predictions if the participants have full knowledge of all relevant facts relating to a particular bet. So the predictions only produce aggregate opinions.
I'm grateful for the article, and now think prediction markets are a good tool for assessing broad opinions: people put their money where their mouth is.
I don't think that the answer for 4.6 quite addresses that concern. This question is really "What happens if people don't care about getting rich?"
While "you" could get rich correcting a mispriced market with too many hedgers, there may not be enough market participants who care about getting rich. A market participant who is hedging is buying insurance, which is inherently about reducing the variance of outcomes at the explicit cost of worsening the average or expected outcome. They are risk minimizers not reward maximizers. If a market is full of hedgers, there is no "you" (or not enough "you"s) interested in getting rich. In that case, the entire mechanism behind the accuracy of prediction markets unravels; too many of the participants are unmoved by the incentive of getting rich by correcting mispricings.
This is not a theoretical problem. There have been many instances in commodity future markets where prices have gotten extremely far from plausible future probabilities because of this dynamic. However, in most cases, it usually doesn't take much profit maximizing money (aka those dastardly "speculators" and "hedge funds") to correct market mispricings. It just shouldn't be taken for granted; it doesn't happen 100% of the time.
Right, one issue I see is that sometimes one side has much more to lose if an event happens than the other. A prediction market for snowfall at a ski resort will probably have a premium for the "insurance" side of the deal and thus will likely predict too low unless there's enough interest by people unaffiliated with the resort to drown out the resort's impact. Risk-aversion in general makes prediction markets inherently more pessimistic than reality.
The caveat that always comes to mind for me when thinking about prediction markets is Goodhart's Law: https://en.wikipedia.org/wiki/Goodhart%27s_law . I didn't see this objection in Section 4.
As prediction markets become bigger, is there a risk that they go from neutral observers to an active incentive to bring about change in the outcome they're trying to predict? Is that a problem for the market concept? Does it change the conditions under which it's "ethical" to bet on one side or another?
A lot of this sounds like the Efficient Market Hypothesis, which is probably true in its weak form and blatantly untrue in its strong form. Essentially, the only way for someone to make money is by research, so someone actually has to *make* said research, but the hypothesis assumes that this has *already* happened.
Also, you can only reproduce the best *public* expert - once prediction markets start to be about real markets, surely you cannot be sure to reproduce the best expert any longer, if said expert merely bets on the markets without advertising or publishing?
And I still don't understand why it's impressive that a prediction market can reproduce the best expert by imitating his or her conclusions. If it just imitates Nate Silver, just go with what Nate Silver says instead - the prediction market is a weird roundabout.
Further still - let's say the prediction market merely copies Nate Silver. But now Nate says something that seems wrong to you. Now people either have to decide "no, we already established that copying Nate Silver is how things are done", OR they have to decide - at the risk of personal biases and mistakes - that Nate is wrong this time, but if you do, you can no longer be sure the market will be at least as good as Nate Silver. So either you merely reproduce the best expert, or you're at significant risk of not performing as well as the best expert. Of course, if Nate Silver *was* wrong this time and people manage to catch it, that's a good thing, but the opposite seems fairly likely as well, and you don't know in advance which it was you just did.
Some additional questions to consider, or objections to think about.
1) Some questions have very little real information available to the general public. Sometimes the information available is wrong - maybe nobody knows the real information, or maybe the information is only available to certain insiders. We would expect all markets on this question to be wrong in the first case, or weirdly distorted in the second. For instance, if the publicly available but wrong information leads the general public to say 20%, but insiders think 80%, the market may converge on any number between them (assuming the insiders don't have infinite money to bet against the public, while the public thinks they are taking free money from the insiders, so they will want to keep pushing it to 20%).
2) We should expect every mature market to be distorted by an amount exactly equal to a combination of any costs (fees and taxes) of making the bet *and* the opportunity cost of not making another investment. You already noted that long term bets have this problem, but I think undersold the fees and even shorter term opportunity costs. At the very least, a badly mispriced market may stay moderately mispriced even after it resolves. I seem to recall some election markets resolving with lots of "free money" sitting around, because it was too expensive to pick up the amounts remaining.
3) The only people who buy stocks are people with disposable income. This problem will persist with prediction markets. You seem to be under the impression that poor people will bet in smaller amounts but in larger numbers. The reality is probably much closer to no bets at all for the bottom XX% of the population, and very minimal and haphazard bets for the next XX%. This will price in an "elite consensus" factor into all markets. It's like getting your political news from Twitter (at least before Musk, I don't know if it's different now) - it may be accurate often, but it's also heavily biased towards certain views. In a toy example with 10 possible predictors, if only three of them are able to invest and they all agree with each other, that's missing something big. This would be important for a whole series of questions related to the lives of poorer people, such as "the effect of UBI will be positive" or similar.
4) Even a 90% prediction based on good metrics will fail 10% of the time (and that assumes the underlying understanding/assumptions were correct). This is a bad failure mode for someone trying to make money and ties in heavily with #3 above. If I don't have $1,000 to lose "correcting" a market, then I don't bet. Also, see the next item.
5) One-off events are very hard to evaluate, even after the fact. The markets were wrong about various aspects of the Russia-Ukraine war. Other predictors were also wrong (which also ties in with #3 and #1 above - limited correct knowledge available generally, not enough liquidity among those who might predict better). If we were trying to use a prediction market to determine if/when various events would occur in that war, we would be wrong quite often.
6) Predictions about the actions of individuals or small groups (will Putin invade Ukraine?) is very different from predictions about aggregations of large groups (will the Democrats win?). We should expect markets about aggregates to do much better than markets about individuals.
Scott you are actually wrong about if hedging distorts markets. I work in finance. Let me explain. Why does stock market go up on average? Because people don't want the stock market to go down. The economy being bad is correlated to all kinds of other bad things in their lives they would hedge if they could. There are plenty of natural shorters of the market in this world. Very few natural buyers. The expected return of the market is above 0 precisely because there are many people who would be happy to hedge against financial downturns if it was 0 by shorting. Very similarly, if there was a prediction market 'the economy will be worse than expected according to a prediction market' there will be more natural sellers willing to hedge and the market will remain systematically pessimistic being worse than expected. Because if it wasn't, and my cost of hedging is 0%, everyone who currently loses say 7% on average shorting the stock market would short this question instead, until it's expected loss equaled expected loss of shorting the market.
But, long-term, don't companies generally make money, and thus become more valuable? Are stocks not ownership of the companies, which then would also be more valuable?
Certainly, in the short-term, anything goes, as it relates to how people think the company will do long-term, or even short-term. Speculation is a zero-sum game. Investing is not.
Not quite, because people could make an unbiased estimate of how much companies will make in the future, discount that to present at the risk free rate, value the company at that amount, and still have a 0% expected return. The reason the stock market goes up over time faster than the risk free rate is because of risk premia, and the same dynamic would exist for prediction markets also used for hedging purposes.
Actually, in the long run most companies eventually go broke. The median stock gives worse returns than 1-month Treasury bills in the long run, and the entire gain in value in the US stock market since 1926 came from just 4% of the total stocks. It's like they say about venture capital investing: most startups will fail and you'll lose money if you invest in them, but the one startup that becomes a billion dollar company more than makes up for the large number of losing bets.
Venture capitalists can afford to back lots of companies and have most of them fail, if they get some winners. But indiscriminate backing isn't really a good strategy. Shark Tank, for example, backs things they think will make money (and possibly some for PR reasons, which is an investment in their own brand). Venture capitalists serve a capitalist function in only backing ventures they think likely to pan out well. Venture capitalists that throw money at every venture without using good judgement, hoping for at least one of them to be a big win, can be replaced by a robot throwing darts at a page of stock listings, kind of like a mindless rock (https://astralcodexten.substack.com/p/heuristics-that-almost-always-work).
Individual investors shouldn't put money into things they "heard are an up-and-coming company" either, but should determine for themselves whether companies they consider in which to invest will be good investments, or should pick mutual funds or such things managed by people who actually do the job of good venture capitalists.
> "Either prediction markets will be accurate, or you can get rich quick”. Unfortunately, it’s almost always the former.
This is a bit of a bait-and-switch. I think prediction markets will very often be inaccurate, but in ways where no one knows how exactly they're inaccurate. You can *technically* get rich quick in that it's theoretically possible to randomly choose one option and get lucky, but in practice you won't know to.
In this way, prediction markets aren't necessarily right at all, only not wrong in exploitable ways, so I have to disagree with referencing them as answers. They're just a sort of cumulative guess. For me, they shouldn't be taken as 'canonical' given their unreliability.
This part got me thinking, in light of the recent AI post: for "When you’re done, tell people whether you thought it was a good use of your time, so the market can resolve" you are training the system to provide content that you will retroactively rate highly, rather than content was in fact a good use of your time, which seems like it could be subtly different -- for example, maybe you have some hot button, and the system will quickly learn to press it. Other systems for allocating your attention almost certainly have this problem too, though, (or other problems) so maybe this is not specific to prediction markets?
It's just Goodhart's Law, I think—when a measure becomes a target, it ceases to be a good measure.
If it satisfies "X thought this was a good use of their time", it satisfies it. But there can definitely be a good amount of distance between this measure and X's true goals/the goals of everyone who wants X to do it/a better world overall/etc.
These sudden changes have happened a few times on markets I’ve seen in Kalshi actually. Typically if it really does disagree with outside public information someone quickly pushes the price back. If there was a mechanism for commenting maybe the real insider information would affect the price longer.
It's worth mentioning that sports betting, especially on the smaller scale (like some local lower-division soccer club), is *massively* tainted by corruption. Since it's so easy for someone on the team (or even more, in a sport like boxing) to either outright lose of degrade the performance significantly, and since there's a lot of betting, there's a lot of money to be made this way. So in this case, betting a) doesn't make the best estimate, and b) outright taints the sport.
I see no reason why big prediction markets wouldn't replicate this.
This is pretty cool... in fact, you've inspired me to start paying. I tried my hands one year at the GJP, which was an interesting experience, but the time sink was too much for me.
One issue you dodge is the problem of thin trading. e.g., "will Prof. Dunderhead's new theorem in algebraic topology be proved correct?" Maybe ten people in the world can actually have an informed opinion, and it seems unlikely that as many as one of these 10 would choose to spend time playing prediction markets.
For insider trading and illegal activities, I think the worry is more about money laundering than about encouraging illegal activity directly.
That is, if you make a million dollars selling drugs or whatever, you can then take that money, use it to place bets on innocuous things, then cash out your bet as "nothing to see here, officer, just my winnings from being a brilliant predictor." Gambling is heavily regulated to prevent that sort of thing.
I do think you're right that it could be at least as legal as, say, sports betting, but maybe there's some sort of clever money laundering scheme I'm not aware of which works better with arbitrary bets instead of sports gambling.
Could you explain more about the money laundering issue?
Don't prediction market websites record your initial investment? It seems different than an unregulated cash-for-chips-for-cash casino with no record-keeping.
Is it the risk of collusion in fraudulent record-keeping between the website's owner and the money launderer?
I dunno, but you can do small-scale money laundering with things like scratch-off lottery tickets. Use your drug money to buy lots of tickets. You'll lose about half your money, but you can live with that. When the IRS asks you where your money came from, show them the winning tickets and don't mention the losing ones.
Prediction markets are just a generalisation of stock markets. Stock markets answer whether a specific stock value rises. Prediction markets answer all kind of questions. That's all. No more, no less.
I'm skeptical on the utility of conditional prediction markets and prediction markets predicting the outcome of other prediction markets - I expect them to be much more vulnerable to manipulation and goodharting. But in general more acceptance and wider spread of prediction markets would be great and usefull.
I have a question. What prevents a person with financial or political power from gaming a prediction market? For example, Elon Musk plans to sell a large number of shares of Tesla, and he hasn't told anyone. He can have someone he trusts put up a prediction that he will do so, and since no one expects this, there will be a large payout. He can then put a lot of money on it and make a big profit when he sells the shares.
Conditional prediction markets seem like a context where the defenses against manipulation can be especially weak. In the vaccine choice example, there's presumably a major financial decision riding on which vaccine is chosen - let's say a $1bn contract that goes to pharma company A or B. We can also posit that the consensus on efficacy of these vaccines is pretty close - otherwise why bother seeking the wisdom of a prediction market?
So all pharma company A has to do to win the contract is make their vaccine's market trade slightly more favorably than B's, which should only require a marginal nudge. Meanwhile, it's far from obvious that there would be much neutral money in such a niche market. Not many people are vaccine experts, and in a situation like this where there's no glaring arbitrage opportunity, who's going to come in to correct the market by a few percentage points? Goldman Sachs might identify a slight mispricing and trade on it, but prudent risk management would keep the size of the trade fairly small, whereas pharma company A is incentivized to throw up to $1bn into the market. I would have said it's hard to imagine an army of Redditors descending on the market - these days I'm not so sure! - but to be clear there doesn't seem to be a "get rich quick" motive in correcting what might probablistically be a very slight mispricing in a niche contract.
One could argue that insider trading bans solve this, but there are situations much less blatant than the vaccine example where the party has a less direct financial interest. Also, a lot of corporate behavior in today's world is not totally above board, so it's not a huge stretch to imagine the laws being broken or bent.
I think conditional prediction markets will be most useful when the market has a large natural audience and attracts a lot of capital relative to any decisions informed by it.
Prediction markets are great, but it's not rational to expect prediction market probabilities to accurately reflect risk-neutral real world probabilities, particularly when the event in question has economic significance. There will be distortions and risk premia like there are elsewhere in finance.
In section 4.6, you ask "If people use prediction markets to hedge risk, won’t that distort them?" and then say no because other traders will swoop in to correct the price. This is not at all what we see in traditional finance, nor does it really make sense from first principles. Rational traders (who all else being equal, prefer less risk to more) will only trade to either hedge a risk that they already have, or if they believe the expected profit on the trade justifies the volatility. If the risk neutral value of a contract is 25%, there is no incentive to trade it at 25% except to hedge.
Let's suppose there is a contract that we think should trade at 25% if everyone were perfectly risk neutral. Let's say there is hedging demand to sell and less or no hedging demand to buy.
If hedgers want to sell at 25%, and arbitrage traders don't want to buy at 25% (because they don't make any money in expectation), then the market price of that contract must be less than 25% in equilibrium. This sort of thing is one possible cause of "risk premia" in finance. The size of the premium here should mostly depend on the following factors:
1) How badly the hedgers want to hedge, in the aggregate - is the risk they are hedging existential to their business? Is it merely slightly annoying?
2) Amongst the rational traders, how correlated is this risk with other risks the rational traders tend to have on their balance sheets? (The more correlated, the more premium they demand).
3) How much pricing uncertainty is there? Can the rational traders be *really* confident the risk neutral value is 25%? How much potential is there for the hedgers (or really anyone selling the contract) to have inside information?
Prediction markets are still great. But in general, markets price things not only on expected value but also on distributional properties/correlations/intrinsic supply and demand imbalances. This his how proprietary traders and hedge funds and the like make money.
(Source: Was a prop trader, am now at a hedge fund)
Isn't "the prediction market is as good as it gets when it comes to predictions" and "you should participate in the prediction market with a decent chance of making money on average" contradictory? *If* the prediction market is efficient, you won't make any money in it (barring insider trading, but then, only insiders should ever put in money), and if it isn't efficient, then it's not as good as it gets when it comes to predictions (after all, you can guess better)?
Without this, it's just sports betting or gambling - done for entertainment, while a reasonable actor with merely public information should expect to lose money. The way to make money is to either cheat (manipulating the market, insider trading) or to be the one running the market.
I wonder if anyone has seen private prediction markets work well inside a large company? I was at a very large, multi-national Fortune 100 company that briefly experimented with prediction markets in its technology division a little over a decade ago. We ran prediction markets around things like, "Software release <X> will result in fewer than <Y> [really bad things]". (This was an easy statement to resolve unambiguously, I'm deliberately avoiding describing the really bad thing we were predicting.)
Pretty soon, the prediction markets revealed where there was a divergence from the politically expedient view. e.g. Prediction market says software release X will be a disaster, but executive in charge of it says it will be fine. Not long after that, the prediction market got shut down.
Was this a unique situation at this particular company? Is anyone seeing prediction markets used successfully inside large corporations?
I'm, uh, not as sold on prediction markets. I do see a few major problems.
1. Too many markets/volume too low. I recall a market you wish you had: whether a certain outdoor event in a city in California would get rained out on a specific date. It's hard to imagine a) too many people caring about that and b) anyone having any better information on weather patterns than the meteorologists themselves, even if you don't trust them. With potentially thousands of markets, we'll probably see a power law where a handful of questions get all the attention and bets and many smaller markets where only a few people bet at all.
2. What's stopping the oil magnate CEO from putting hundreds of millions into YES on the "Will global temperatures rise 3 degrees by 2050?" market and then polluting so much that we hit the target and he makes a ton of money? Would people betting NO have to put in, collectively, just as much as he put in on YES to get the price back down?
I notice a point that's come up a bunch in the comments is "hm, but prediction markets can only aggregate existing opinion, so they won't truly be *accurate* if *everybody* is wrong". Of course, the response to this is already implicit in the original piece -- no, they won't be truly accurate if everybody is wrong, but they will still be the *most* accurate available predictor, which is all you can hope for. Still, this might be worth making more explicit.
No, they won't be the most accurate available predictor. They will be the most accurate predictor weighted by the willingness/ability of people to put money down on a prediction market bet, which is not necessarily directly correlated with the accuracy of their predictions.
Quite right. If I really know the answer, e.g. because I can cause it to be one thing or the other, or at least very heavily influence it, then by definition I'm an insider trader and most market rules would prohibit my participation for obvious reasons -- the prediction market has in this case just turned into my personal scam machine -- plus I almost certainly have better ways of making money off my known ability to influence events than betting on them.
I'm not per se against prediction markets, but I think there is an over estimation of utility and a question of over estimation of precision. (I have participated in GJP many years ago and something that maybe was a precursor in 90s.)
That Yale/Harvard expert who says 70%. Is not really saying 70%, she is saying "about" 70%. Maybe what she is saying is there is between 50% and 90% change. There is a lot of difference between 50% and 90%.
Market manipulation: ask Nate Silver, a known gambler, to publicly state whether he privately bets on the question that he and his site make public predictions about. Predictions which affect the markets. He ostensibly is a data journalist but his ombudsman refuses to respond to inquiries about his private gambling.
Framing - almost all of these prediction market questions can be framed alternatively. Kahneman and Tversky
Is it really like the stock market or is it sports betting?
> Framing - almost all of these prediction market questions can be framed alternatively. Kahneman and Tversky
I read, and partially understood, Kahneman's book, but I'm not quite sure what this objection is about. I'm trying to find a practical example. For example, if you are writing a prediction market question on a binary referendum outcome, the word 'win/pass' should correspond to X% and 100-X%, while the word 'lose/fail' should correspond to 100-X% and X%. Do you mean that our cognitive biases prevent us from recognizing the inverse equivalence of questions like this?
Or do you have something else about framing in mind?
Basically, cognitive biases will affect predictions depending on how they are framed - even though "this will NOT happen" is the same question as "this will happen".
I understand in theory. But do we have evidence of this with two high-popularity, binary-outcome, real-money markets, on the same website? What I would look for is two such markets that differ only by a negative term, yet mostly sit further than 2% apart from a perfect inverse, i.e. (X%, 100-X%) and (100-X%, X%). I hope I've worded that clearly.
Manifold, with its profusion of similar questions, would be a promising place to find examples. But it isn't real-money, so participants' incentive to take it seriously is lower.
I think instead of conditional prediction markets for P(A|B), markets for P(B) and P(A AND B) would be more flexible.
Suppose you want to bet on "on the condition that the next US president is a member of an ethnic minority (B), they identify as female (A)".
Betting a dollar on P(A|B) is equivalent to bet P(NOT B) dollar on NOT B and then betting the P(B) dollars on P(A AND B):
* If the outcome is any NOT B, P(A|B) will not resolve but return your bet of 1$. Equivalently, your NOT B market will return you the dollar.
* If the outcome is B AND NOT A, you lose everything
* If the outcome is B AND A, your return is P(B)/P(A AND B). This is the same return you would get from betting a dollar on P(A|B) (assuming effective market conditions.)
The advantage of AND-bets over are:
* Markets are always resolved
* It is a simpler basis which covers more ground. With a market for P(A), this would also cover the probability P(B|A).
Disadvantages:
* You need to track changes in P(B) and adjust your investment accordingly. (Unless you assume that P(B) is priced correctly and are risk-neutral or risk-friendly, in which case you would not need to bet on P(NOT B) to cover that possibility).
* Calculating the conditional probability is more complicated.
More generally, if we had N different observables A,B,C, ..., one might set up 2^N different markets predicting the probability of any specific outcome. If you want to bet on B, you would just invest in all the 2^(N-1) markets favoring B and weight each investment by that markets probability.
Of course, nobody could ever agree on the N interesting observables beforehand, and I am unsure if there is anything to be gained by converting bets on B to equivalent bets on A AND B and NOT A and B on the fly.
(I don't actively trade in prediction markets, but my background is finance.)
1. I was a little surprised that reversed trades didn't get mentioned as a criticism. In the vaccine example, if I take time to investigate vaccine A and trade accordingly, I lose all that effort if the vaccine isn't chosen. One might object that my knowledge could partially map over to vaccine B too, but in general there could be more than just two vaccines and there could be markets where none of the options get selected.
Furthermore, the potential tax implications just rub salt in the wound, as the reversed trades might not be in the same calendar year. Even if you're in the same tax situation both years, that's two sets of trades to report even though they ended up at a zero net payoff for you. (If nothing else, the sheer annoyance is a frictional cost.)
I'm not an absolutist by any means. If there's fraud or a major market malfunction, unwinding trades could be the best remedy. Markets where it is a routine occurrence, though, seem like they could have been designed better from the outset.
2. Regarding #4.6.1.1, there are still certain markets where this is unavoidable. Pretty much anything that correlates to general prosperity for the user base would potentially end up with distorted probabilities.
I've seen a proposed solution to that where there are three markets:
1 - A is chosen AND causes X deaths
2 - B is chosen AND causes X deaths
3 - A is chosen (regardless of how many deaths it causes)
Someone looking to identify the probable safer vaccine can use the price of 3 to get an implied probability that A will be chosen, then this probability and the price of 1 to get the implied probability that A causes X deaths if it is chosen (and similarly for B). However someone looking just to trade and make money is always rewarded (probabilistically) for having better information than the market, plus the tax implications you describe don't exist.
I think this isn't the preferred solution because it is more complicated than just unwinding the trade and people don't want prediction markets to become the preserve of the extremely financially sophisticated.
It's debatable whether the stock market is efficient, and certainly there's no reason to think that a prediction market would be.
Markets are legal while gambling isn't generally because we want to allocate investment capital to enterprises, not just get a price for something. Having more money in the market means that companies have more paper value and thus can raise more money for actual operations - that is to say that the total amount of money in the market is meaningful, it creates value for the thing you invest in. In a prediction market there is no reason to want people to make larger bets, you just want enough to get accurate odds. Increasing the pot is likely harmful to society.
Having people make third-party bets with significant money is a bad use of money and regulators shouldn't allow it. Rather than betting on outcomes (zero-sum) people should invest in making outcomes happen (positive-sum).
Mostly I agree, but on the social costs -- consider that any opportunity for betting over the long term basically transfers cash from less cautious to more cautious people (usually from the bettors to the people who run the betting operation), which very likely reduces the velocity of money and hence tamps down inflation. This is a positive social value, much in the way that luxury yachts are a social value, because they transfer money from profligate idiots to ordinary workers (who build and operate the yacht).
"Insider trading" doesn't quite equate to "I have knowledge that others don't have and I use it in my trading" as folks sometimes assume. If you're relying on the existing legal framework to solve that problem, you might be surprised by some of the outcomes. As a first approximation, if someone along the way didn't use for personal benefit information that "really" belongs to their employer, it probably isn't insider trading. (The personal benefit doesn't have to be financial, and the person who misuses the information isn't necessarily the one who trades.)
Unless I'm misunderstanding, it seems like the use-case of prediction markets where only a minority are resolved by some event and for the majority the market's verdict is final is problematic, since in that case the response to the meme-stocks question 4.8 doesn't really apply.
E.g.
> A few randomly selected questions went before the Supreme Court each year, and the corresponding markets were resolved normally. In every other case, the market’s verdict was final, and bettors got their money refunded.
It feels like in this scenario it would be easy and fruitful for a rich person interested in a certain case outcome to artificially inflate or deflate the price of the market, knowing that there's only a 1/n chance that the supreme court is actually going to rule on it, and otherwise they are just buying their desired outcome. It feels like there should be a way to resolve this but I can't think of any off the top of my head.
Simple fix for that would be to prioritize resolution by trade volume rather than randomly. Harder somebody tries to buy a clearly unjust verdict, the faster that case will float to the top. Would also conveniently filter out frivolous cases, and making the trade volume a matter of public record along with the rest of the case info could help prevent sneaky setups involving unlikely combinations of trivial grievances from being used to manufacture broadly unreasonable precedents.
For far-future events, would recursive prediction markets work?
Given whether an event would occur within a 100 years, create a prediction market N about what a prediction market next year N+1 would resolve to be, where prediction market N+1 is about what prediction market N+2 will resolve to be, until prediction market N+99 which resolves to whether the event actually happens.
Prediction market N would close when the prediction market N+1 would start, and it pays out when prediction market N+1 closes.
Of course, if the event happens earlier you short-circuit the recursion.
Conditional prediction markets for things like replication studies, where the market outcome determines which of many conditional prediction markets gets settled, seem to have two problems.
First of all, if you're redoing one of 100 studies, then the potential return on each market is only 1% of what it would otherwise be. Suppose the market says that Study #17 has a 10% chance of replicating, but I think it has a 20% chance. If I buy a bunch of 10-cent shares in this market, then I expect a 99% chance of getting my money back, an 0.8% chance of losing my money, and an 0.2% chance of getting 10x my money. This has an expected return of 1%, which is not worth betting on compared to our hypothetical 5% index fund alternative.
Second, there's a bias in how much this affects different opinions. In the calculation above, I assume that Study #17 has a 1% chance of being redone. But actually, if I think it has a high chance of replicating, then I think the prediction market has a high chance of agreeing with me - and since this is how we select which study to redo, maybe I actually estimate a 10% chance that Study #17 will be selected. This improves my expected return from 1% to 10% and now I have an incentive to bet.
On the other hand, my pessimistic friend who thinks Study #17 is total trash and has no chance of replicating also thinks it's very unlikely that Study #17 will be selected by the prediction markets! So for my friend, even though the conditional prediction market on Study #17 would have an 11% return if the study did get redone (my friend would buy WILL NOT REPLICATE shares for 90 cents and be certain to get a dollar back at the end), there is only a tiny chance that Study #17 will be redone in the first place, and the expected return is correspondingly tiny.
Doesn't this have an effect where all 100 conditional prediction markets will be biased upward, because correcting that bias has a lower rate of return?
That is unrelated. People can make an unbiased estimate of how much companies will make in the future, discount that to present at the risk free rate, value the company at that amount, and still have a 0% expected return. The reason the stock market goes up over time faster than the risk free rate is because of risk premia, and the same dynamic would exist for prediction markets also used for hedging purposes.
Well, your talent for writing persuasively is on full display here, very impressive. Personally I think it's all bollocks, however, for the simple reason that a prediction market is only better than an expert when the experts aren't very good.
Nobody imagines that a prediction market would correctly predict my eldest son's name better than either he or I. If my drains back up, I am clearly much better soliciting the advice of a plumber on how to clear them than going up and down the street and establishing a prediction market among my neighbors. If I want to build a functioning nuclear weapon, the way to do so is clearly to ask an expert nuclear weapon physicist, and asking a prediction market is silly.
And so on. For almost all of the basic factual matters of life, it's obviously more accurate and faster to ask someone who knows than to sample public opinion randomly, and so of course that's what we do.
Hence the questions to which the prediction market would apply are ipso facto those in which we already know (from bitter experience) that expert opinion does a poor job, presumably because the outcomes are very difficult to predict -- data is lacking, there is some sociological issue with making the accurate prediction, there is some inherent source of randomness, et cetera. Predicting the winners of the next election, predicting the outcome of a war, predicting the course of pandemic disease, predicting future major technological or social changes, predicting Black Swan events -- all of these fall into this category.
But in these categories, expert opinion is already known to be poor. Proving that prediction markets are at least as good as expert opinion doesn't say anything that's broadly and usefully actionable, because proving the prediction markets do better than known poor predictions is a long way from saying prediction markets make good predictions.
A prediction market can certainly provide exciting betting opportunities[1] for those who are into that -- and plenty of people are, hence the popularity of gambling -- but it can't make better predictions in a way that solves broad social or economic problems. I mean, to think otherwise, you need to think you can get something from nothing, that from a sufficiently clever summation of ignorance you can extract brilliance. This is like thinking that if there's a mathematical problem that no mathematician has been able to solve, I should be able to ask a whole lot of non-mathematicians and somehow from that huge mass of ignorant opinion extract the correct answer. This just isn't possible. Frankenstein is a myth, you can't create life from non-life, there is no algorithm to create meaning from ever so much nonsense.
--------------------
[1] It doesn't provide a way to steadily make money, like the stock market, because as you point out there is no separate value of investing: investing in the stock market as a means of betting on the future values of things is being a "growth" investor, but I hazard most investors are actually "value" investors. They're not buying a share of XOM because they're betting it will rise in the future, they're buying it to own a tiny piece of a profitable company that pledges to return some share of its future profits as dividends. Because the stock market is a collective investment vehicle as well as a betting market, you can make steady money in it, regardless of how well or poorly it predicts the future. There is no such collective investment aspect to the prediction market, it's just a betting pool.
Your argument that a prediction market would allow hedging is interesting, but you've just reinvented the insurance market, which is already healthy, and you're not going to gut insurance company profits until prediction markets prove *far more accurate* than any other source of prediction, which brings us right back to the main point above.
>Hence the questions to which the prediction market would apply are ipso facto those in which we already know (from bitter experience) that expert opinion does a poor job, presumably because the outcomes are very difficult to predict -- data is lacking, there is some sociological issue with making the accurate prediction, there is some inherent source of randomness, et cetera. Predicting the winners of the next election, predicting the outcome of a war, predicting the course of pandemic disease, predicting future major technological or social changes, predicting Black Swan events -- all of these fall into this category.
>But in these categories, expert opinion is already known to be poor. Proving that prediction markets are at least as good as expert opinion doesn't say anything that's broadly and usefully actionable, because proving the prediction markets do better than known poor predictions is a long way from saying prediction markets make good predictions.
Part of Scott's point is that in your #2 scenario, the answer is knowable but the legible experts are *lying*. You can't lie on a prediction market, or rather you can but you lose your lunch if you do.
It is obviously the case that for a prediction market to give the right answer, there must be experts who know said right answer, *but* you don't have to know who they are in advance (i.e. illegible experts are available) and there are large monetary incentives for telling you the truth and penalties for telling you a falsehood (so it would cost a large amount for anyone, including an expert, to ensure you got a wrong answer).
I can't help but feel that all of your listed objections to prediction markets are downstream of something more fundamental:
Aren't markets cringe?
Depending on your outlook, markets lead directly to neo-feudalism or rainbows on everything in June. Either way, they're associated with those guys in the Econ department who wear bow ties and are politically unreliable at best. Markets allow the people you don't like to coordinate efficiently, and this is becoming increasingly intolerable. As more and more of society becomes politicized, both parties are becoming increasingly anti-market, and they have no incentive to overcome status quo bias and unban a new source of pluralism.
THIS is what you're up against. Change this attitude, THEN the point-by-point refutations will become relevant.
My experience is that most of the time declaring in favor of markets is a response to establishment decision structures that you dislike. Id est, people who declare in favor of free markets in car buying or the means of electrical production actually just dislike environmental regulation. People who declare in favor of free markets in dating and sexuality actually just dislike established social shibboleths around mating. People who declare in favor of free markets in recreational drugs actually just dislike drug laws. And so on.
I'm not persuaded *anyone* actually likes markets in and for themselves, on account of the main effect of a market is to diminish the power of special knowledge, special skill, or good luck to command success. They make everybody work harder to succeed, which is not fun -- unless you're already shut out of the conventional structure for success.
So it makes sense to me that "Rationalists" seen as a movement would promote markets in predictions -- because they feel (I think accurately) that their predictions for the important events of the future are dismissed by conventional wisdom, and by conventional authority. Naturally that leads to a wish to tear down the conventional protocol for predicting the future.
I think it would be useful/helpful to briefly discuss the fee structure of these markets, which render them a negative-sum game. I love prediction markets but every time I feel interested in getting involved I see the fee structure and what it implies for the % I need to get right (weighted by price) and lose interest. Still think they have tremendous value though!
The repeated "or someone can get rich quick" stories started to remind me very much of Anselm's argument. That is, there is a sense of "the true forecaster who can correct a mispriced market in reality is greater than all the ones who are only doing so in their imagination, therefore that true forecaster must exist."
Sergey Wenergold, a GTA V player (buy Air EMU, burn FlyUS) - maybe Osama had some put-options at 9/11. But: CEOs not killed - lol, John Kenneth Galbraith famously noticed how replaceable most of those are. Politicians are, too - I guess. But the chance of Biden/DeSantis/Trump to "be president in 2026" is zero on death. There is no option on Wall-Street "Tim Cook apple-CEO in 2024" - else I would worry for his life (though he and Musk spend on security.) Enough blood even without incentives:
It's also easy to imagine less egregious misbehaviour. Let's posit it's 2020 and there's a bet on whether the President will be diagnosed with Covid this year. It so happens that you have Covid, and you have a long meeting with POTUS scheduled. So instead of being reasonable, you place a bet for Covid and go to it.
I love prediction markets and your enthusiasm for them in general (which is what originally made me aware of them), but you seem to be playing games with the "on average" thing a bit.
Like, in your Harvard v. Yale v. TinFoil thought experiment, suppose the TinFoil underdog turned out to actually be correct. It just so happens that the government did in fact make up the whole thing, and Harvard, Yale, and all other "respectable" institutions had the fatal blindspot of being funded by the government so <insert handwavium about incentives> they couldn't see it coming.
Your usual argument after this goes something like "This makes people more willing to trust TinFoil in the future, increasing TinFoil's weight in the market and thus making the market at least as accurate as TinFoil", but suppose TinFoil's talent is ONLY in play in case of global pandemics. Then people should only trust them in the *next* global pandemic, but how many global pandemics happen each century ? Probably 0 or 1 during TinFoil's lifetime, and they will probably die uncompensated for the truth they can help guide the market for, their talent wasted.
To tldr : averages only make sense on a big enough sample, and some events are one-shot (they don't even have to be true Black Swans, that's a ridiculously high bar, just as one-shot as a global pandemic or a nuclear war). Prediction Markets can theoretically always give the wrong answer and never correct. It's like a Prisoner's Dilemma where you only play once, Tit-For-Tat no longer works, you can Defect and get away with it.
As others have noted, there are huge prediction market for sporting events. Usually there's a "house" involved and the house gets a cut, but the situation is only slightly different from a pure prediction market. Interestingly, it's known that the odds offered are not always accurate relative to the available information. I've seen an interview with someone who made a good living at sports betting by consuming all available information and learning the patterns of the irrationalities of the betting public. He said it took him about nine months of full-time work to get to that point, though no doubt he was very interested in sports and betting before that. OTOH, it required that he ingest all of the available news about sports, so it's likely that he could have earned more money on Wall Street.
My suspicion is that there are few consistently profitable ways of utilizing a prediction market, that there are few large categories of marketable predictions for which it is profitable to have accurate predictions. And that most of those are covered by existing financial markets. (E.g. the airline that wanted to financially insulate itself from a possible general downturn in the industry, so it placed a mass bet against the stock prices of all the other airlines. They did this shortly before 9/11 and so had to explain to the FBI what was going on!) The "clever uses" listed above seem to be between once every few years (which new vaccine to prioritize) and never (a CEO wants the public to be able to raise questions they care about).
Sure bet: a certain segment of the high-IQ blogosphere will continue to obsess about prediction markets.
Think more like it’s a football pool rather than an options market.
Although it’s a bit of both.
Will x happen or not?
Bet yes or Bet no. 1$ The house is holding the money.
(The resolution is ultimately binary so there’s some logic)
The trading volume will start to reveal the odds (prediction) which will determine the payout (whatever money was bet on the wrong answer distributed to the winning side pro rata).
There is no advantage in being first (although that could be tweaked), it’s hard to game and I think a more transparent structure.
Obviously there needs to be a fee for the house so a percentage of the pot evaporates.
A masterful post, very well presented. However, I question the value proposition behind prediction markets. I consider three basic background variables:
a. The topic is "important" - meaning that an accurate assessment can guide current significant decisions. If not, it is merely "interesting". I'm talking here about the prediction market, not the outcome itself. An election outcome might be very important to many people, but an accurate assessment of the likely outcome won't actually change the outcome. Whereas an accurate assessment of the outcome of policy choices can be used to make those policy choices.
b. The topic is "timely" - meaning the final result will be knowable over a short enough timeframe to give predictors a real interest in seeing the outcome.
c. The topic is "independent" - meaning that an accurate prediction will not influence the result itself. Predicting the likelihood of a Yellowstone mega-eruption might be very "important", and possibly "timely", but Yellowstone won't decide on an eruption based on the prediction.
I see five basic situations:
1. The prediction deals with a topic that is "untimely", whether or not "important" or "independent". In this case, why would anyone participate in a market? We won't know an answer in a meaningful timeframe, so the "winners" won't collect. We could have a market on the likelihood of the Yellowstone eruption in the next 1000 years, but who will be around to collect in the year 3022?
1a. The prediction deals with a topic that is "important" but "untimely". In this case, partisans have a large incentive to tilt the prediction to favor their already-preferred outcome. For a current example, there are currently proposals to radically restructure the world's economy in order to prevent climate change and its associated predicted problems. Most of the predictions deal with potential effects by the year 2100. Most people alive today will not live to see the final result, but many have strong preferences on the policy question. If the prediction market is used to determine the policy choice, partisans have a strong incentive to try to tilt the prediction market to favor their preferred outcome. In such a case, the prediction market is really a vote by people willing to commit the resources to influence the policy outcome, but in a way less transparent and accountable than democratic governance, or even conventional lobbying and propaganda.
2. The prediction deals with a topic that is "interesting," "timely", and "independent". I consider election predictions in this category. We can certainly see election results in a reasonable timeframe. I doubt that voters will be much influenced by predictions stating the likelihood of "their" candidate winning. I doubt that anyone will make meaningful decisions based on current predictions of the next presidential election. If the election will have a material impact on the decision, it is almost certainly better to wait to see the actual election results. For instance, a business might consider a large investment in solar energy; and it might assume that a Democratic administration will offer large subsidies for such a project, while a Republican administration would not. In such a case, it would probably be a mistake to make significant resource commitments
In such a case, prediction markets are a useful counterweight to self-appointed experts, and can help people get a better read on the state of knowledge. But the markets don't serve an obvious social purpose other than perhaps balancing the shouting partisans.
3. The prediction deals with a topic that is "interesting" and 'timely", but not "independent". That is, we'll know the outcome in a reasonable timeframe, and it will be influenced by the prediction market, but the outcome isn't particularly important to most people. For example, "Who will be chosen to host the Oscars ceremony in 2025?" We'll be able to see the result, and it won't be of great importance to more than a few dozen people, but the decision is likely to be influenced by the prediction market. In such a case, I don't see a great social utility, and I don't see why anyone would participate, since future manipulation could tilt the market against current bets.
4. The prediction deals with a topic that is "important" and "timely", with strong partisan interest. For an example from 12 years ago, "Will a reform of the healthcare insurance market improve things for most Americans?" Setting up a prediction market to address this question gives partisans an incentive to tilt the prediction to favor their preferred policy, in order to enact their preferred policy or prevent a policy they don't like. In order to have a meaningful answer, the question would have to be much more sharply focused. For instance, "Will personal bankruptcies decrease by more than 50% within 5 years if the PPACA is enacted?" Or "Will deaths decrease by 40,000 per year from current trend within 5 years if the PPACA is enacted?" In such a situation, committed partisans will be willing to bet money to tilt the outcome to favor their preferences. Unless potential gains and losses in the prediction market are very large (hundreds of millions if not billions of dollars), I expect rational people with a strong policy preference would vote their policy preferences, rather than offering useful outcome prediction.
5. The prediction deals with a topic that is "important" and "timely", with no strong partisan interest. For example, "What is the likelihood of a Category 5 hurricane hitting New York City in the next 20 years?" This is "timely" because it's bound to a reasonable timeframe. It's "important", because an accurate assessment can be used to evaluate the desirability of measures to protect New York and nearby cities against the effects of hurricane. And it's independent because the Earth certainly isn't going to pay attention to the prediction market.
So, I think the only really useful applications of prediction markets are for issues that are "important" and "timely", but without strong partisan interests. Unless we can somehow exclude the partisans from the market, or attract so much capital to the market that even committed ideologues won't be willing to risk losing on the market.
I'm still seeing a problem which I don't think was addressed in the post: certain/many controversial topics won't be able to be resolved fairly.
For example, let's say you wanted to assess climate migration. You could create a market like: "100 million people will move to a different country between now and 2040 because of climate change." No problem with the question, but how would you resolve it? You could link to a tracker from the UN or some other "authority" as the way to resolve it, but then the market really becomes a test of whether 100 million people will change countries at all, because said authority _will_ say it's because of climate change, whether that case is strong or weak. Or perhaps there would be another "authority" that would always resolve it the other way, no matter how many people move.
I guess I'm asking: what if there is no fair authority to appeal to on a complicated, controversial question? My intuition says that most squishy sociological questions would fail on prediction markets due to this.
I think there are a couple layers here. The controversy doesn't help, as it gives all parties (including the migrants themselves!) incentives to misrepresent reality. Even if we all decided tomorrow to agree on climate change, though, this would be a tough question to resolve. I see a mostly observable event (migrations) paired with a very hard to measure one (the underlying reason). What was the reason I snacked on kettle corn this afternoon? Stress? Hunger? Boredom? The sheer fact it was in the house? Some combination of the above? I couldn't give a good answer to that, and I have nothing riding on that outcome.
Or do you mean counterfactually, as in a world without climate change? That's tricky too. Would it be an otherwise identical world, but one where our economic activity somehow hadn't increased the greenhouse effect over where it otherwise would have been? Or a world where we didn't touch fossil fuels at all, leading to a very different current state of affairs (countries are much more uniformly poor, so there's less of a draw to migrate)? Even if we settle on a definition, there's the question of measuring the migration level in such a counterfactual world.
So I guess my reaction is: designing a good question can be hard!
I was definitely thinking along the lines of your first paragraph, and I agree with the way you parsed it out. Yes, determining how many people migrated is relatively easy while the second half (why) is very difficult. Interesting that you are focusing on the question instead of the resolution, which is what I'm more worried about.
Of course, one way to resolve it would be to collect very specific stories of people who clearly migrated for reasons of climate change. Spend about three paragraphs on each one for documentation, and just do this 100 million times. This would avoid the "we trust this authority" issue...but would obviously be impractical.
I thought my question was pretty good in the sense that it was clear, although not exceptionally detailed. I guess you could detail out what would qualify as a population moving due to climate change. Perhaps you could even count certain fractions of the population moving when certain causes are present. (Population moving due to coastal land disappearing 9/10, coastal flooding 4/10, heat wave 1/10 etc.) It would shift the subjectivity sharply from the resolution to the question itself but it could be done.
I could be talking myself out of my objection. I guess then my difficulty would be that this question could be presented thousands of different ways with different stipulations and that would make it harder to bet on, practically speaking.
Re 7.2: Using VPNs to access Polymarket is very safe and easy. It is not even possible for them to ban you as the smart contracts will always be accessible
Re 7.6: The head of the CFTC can't really do much. The commissioners have the final say and they vote
The main problems with Kalshi and Polymarket is that neither are sustainable. Both lose money by subsidizing liquidity. Polymarket does this directly and Kalshi does it via its affiliate, Kalshi Trading.
Without this, neither of the two platforms would have much if any liquidity on the long-tail markets that are probably most interesting to the readers of ACX
Prediction markets don't work nearly as well as you make them sound. They have many problems you barely address:
1. The biggest problem is that if everyone was rational, nobody would bet in prediction markets (except for a few use cases like hedging by big firms). You briefly mention this and say "but in practice people do bet", yet I'm not sure that's true; in practice, people bet on the large, flashy items like presidential elections, but not on other things. There is no reason to believe this will scale.
2. You spend a lot of time focusing on the limit as the money in the prediction market goes to infinity. It's true that in the limit, prediction markets work. But see (1); the limit will never be reached, and the more rational society is, the less likely prediction markets are to have a lot of money in them.
3. Instead of focusing on the limit, consider the *rate at which that limit is approached*. Just how well do small prediction markets perform? What about medium ones? In general, they do pretty badly. There are a few reasons for this, including:
3.1. Goldman Sachs won't bet on them unless they can make more money than their quants' salaries cost. Even a $1000 edge is not worth people's time.
3.2. If the market has a lot of money in it, it basically implies that lots of stupid people have bet (see point 1), and
3.3. Smart people are less efficient at correcting markets than dumb people are at distorting them, due to the Kelly criterion; that is to say, if you believe the market prediction is off by 10%, it may only make sense for you to put in something like 1% of your money into the market due to Kelly (this depends on several factors), yet a dumb person may put in 100% of their money when distorting the market. You would have to have 100x more money than the dumb person to offset them.
4. The payoffs in conditional prediction markets are conditional, which amplifies all the problems in (3) and makes it a lot harder for them to work.
5. You spend a long time talking about whether prediction markets are effective, but not about whether they are *cost effective*. If I have $X and want a good prediction, should I (a) set up a prediction market, or (b) hire an expert or small group of experts? (I could average the group of experts myself later)
Almost certainly (b) is better for most things. This has been repeatedly observed empirically. It also makes sense theoretically: if predictions are expensive (e.g. they cost some fraction of $X for setting up a fancy model), then having a market fundamentally means *duplicating this expensive work*. Like, instead of one fancy model being set up, in a prediction market you're likely to end up with a larger number of smaller, cheaper models as people scramble to eat up the returns; this wastes effort as many players duplicate the same predictive work. There won't be enough edge left for Goldman Sachs to invest in the fancy model.
6. It just feels weird to say "in the glorious future, we'll all have perfect predictions funded by gambling addicts' losses". This is the same point as (1), but this time as a moral criticism. You are basically saying you want Goldman Sachs to set up fancy models, and those fancy models will be funded by the losses of gambling addicts, and you want it to be a stable, permanent equilibrium.
In a zero sum/negative sum prediction market, there is a certain element of "if you don't know who the sucker at the table is..." On the other hand, sports betting exists, so don't entirely discount fun/challenge as a motivation. It's not hard to make a prediction market positive sum for the participants, though. Political parties/campaigns could be a natural sponsor of election-related ones, for example, as they already spend money on polls on a regular basis.
Mechanically, market maker functions do it pretty naturally. If I want a prediction market on something because I value the information, I just pick the parameter so my maximum theoretical loss -- picture a participant who knows the future and has unlimited bankroll -- is whatever $X I want to subsidize the market with.
Alternatively, you can just publicly commit to periodically making dumb trades in the market.
Even from a budgetary perspective, the strategies aren't mutually exclusive. At some point there are diminishing returns to, say, buying slightly more frequent polling.
I can see funding a prediction market with your $X as potentially being the most efficient use of the money if the thing you're trying to ascertain is public opinion itself, and your target demographic is the kind of people who would participate in a prediction market, so smart, well-off, a little bit narcissistic.
> if the stock market grows 4% per year, I should expect any money invested in the stock market to multiply by 20x in 2100. So just doubling it in a prediction market is a bad option.
Normal finance has a good solution to this: futures markets. You can get very large leverage at very little upfront cost, and small liquidity loss by just agreeing with someone that you will pay them a particular price in exchange for a specified asset on a particular date.
I disagree with a lot of this, especially the parts that argue for prediction markets being highly efficient. There's many reasons that markets don't incorporate all existing information, some of which you touch on - opportunity costs of capital, opportunity costs of traders - case in point, I made around 300k on polymarket in 2021 and then quit in favor of trading NFTs and later crypto more broadly. With interest rates close to 5% it makes even less sense to trade anything longer term without a massive edge.
But even the extent that current markets are fairly priced is sort of free loading off their current low profile. If prediction markets got actually relied on in the way you propose, the returns to manipulation would go up. In your example of the government choosing between two vaccine candidates, the value of that choice to the vaccine company will be valued in the billions of dollars, and they can easily ensure the market price moves to where it needs to be with a small fraction of that. Investors will push back some, but there will be a wide range of uncertainty and so they won't want to risk too much. This generalizes; any time you want a prediction market to impact $X of real resources, you create an incentive for rent seeking on behalf of the beneficiaries. There's always a bound for how much capital is willing to correct an apparent mispricing, with unknown risks, possible adverse selection, and opportunity costs.
I'm even worried that this kind of Goodharting would apply to existing financial markets if they were explicitly relied upon for policy. Scott Sumner has suggested using NGDP futures to determine Fed interest rate changes, but it's not clear to me how you'd prevent people from moving those markets in order to profit on bond positions.
note that the author of this comment was arrested shortly after posting, and indeed his last tweet was a reference to this comment. Oppose prediction markets at your peril!
I think I could have a good crack at the time value of money issue. This wouldn't be starting a new prediction market, but selling a backend currency service to existing markets.
If anyone is based in the UK and would be interested in helping me start this, let me know below. Experience in payment operations or treasury tech stack would be particularly complementary to my own skills and experience (which are capital markets and regulatory structure across financials and infrastructure).
Another piece of financial market Infrastructure missing from prediction markets is a proper central counterparty role (which is currently effectively done within the markets, which I don't trust not to fold holding my money).
I'm more pessimistic about this one because it essentially requires a regulatory structure to be meaningfully helpful, and there's limited first mover advantage if you spend ten years working with the regulators, then Euronext/Clearstream come by and use their contacts and financial firepower to swipe the market out from under you, as they would inevitably be much better capitalised.
Scott, you say with regard to very long term predictions "But if you really wanted to use a prediction market, you could theoretically solve this by putting investors’ money in index funds while they waited. Then the winner would get their (and the losers’) original deposits and investment profits, and it would go back to being a better option than investing in index funds directly. In practice this seems complicated and I wouldn’t expect it to work." This kind of paired structure is extremely common in financial markets (e.g. convertible bonds) and I would see no problem at all doing this with an index fund and a prediction position, provided the prediction market venue itself was stable. Again, the derivatives markets clearing infrastructure shows how this can be done.
If I trust the prediction market to usually represent the best odds of any given scenario occuring, why would I ever bet against the odds currently presented by the prediction markets? Even if I was an expert in Event A, why would I trust myself to know the odds of event A occuring better than the collective knowledge of the market? It would not make sense to participate unless I didn't trust the market's accuracy to begin with. In which case, I would not consider the market's ability to make accurate predictive odds inherently more trustworthy than any other source. If the market is trustworthy, I would not participate. If it was untrustworthy, what's the point of making one in the first place? Other than
trying to make money gambling against people I assume are stupider than me.
This post is extremely frustrating because you acknowledge all the counterarguments, you just refuse to connect the dots and follow them to the logical conclusion. It's basically this in essay form: https://i.kym-cdn.com/photos/images/newsfeed/001/286/448/863.png
For example, you have a section where you acknowledge the various reasons why real world prediction markets might be biased or inaccurate, and then spend the rest of the essay arguing based on the premise that prediction markets magically have infinite liquidity, zero transaction costs, etc.
Instead of responding to every question by saying "in the frictionless spherical cow world, you could make money by arbitraging this", the intellectually honest approach would be to try to quantify the bias in your hypothetical scenarios and see how much of a problem it would or wouldn't be. Even you yourself acknowledge at the end that real life prediction markets are highly biased and illiquid, and that while you might hope to improve that a bit, you also acknowledge that the real world stock markets, which function near-infinitely better than prediction markets could ever hope to, *still* have problems.
Another amusing bit was where you concede that "we shouldn't allow markets in dangerous things" and then later on complain about regulators restricting what you are allowed to have markets in. Obviously, you could have some principled position that the optimal level of regulation is nonzero but less than the status quo, but you never actually *say* that, so it just makes you look confused and dishonest, and it hides your true position.
Another huge weakness is that you completely dodge the "prediction markets will solve societal breakdown and lack of trust" thesis, despite that being *your leading argument*. You do finally get to the obvious question later about how the many people don't even trust wall street could possibly be expected to put their faith into a weird inferior version of the stock markets, but you just pathetically retreat to "well, the kind of person who trusts prediction markets will trust them and who cares about the rest?".
Lastly, you also don't have a good answer about the "why will people play a zero sum game"? problem. Obviously, people like gambling, but the demand for gambling is not infinite, and it is not evenly distributed. People like gambling on sports, but you're probably going to have a lot less luck getting people to gamble on say, vaccine stats (as you observe, this is borne out in the real world where prediction markets are tumbleweeds next to DraftKings et all). Also, gambling is probably a net negative to society anyway.
One other thing I should emphasize is that **the assumption of infinite liquidity and zero transaction costs is a load bearing element of your arguments**, despite you realizing that this is clearly not actually true. For example, consider proposals like "only decide a few of the markets randomly". If you have any actual finite amount of liquidity, risk tolerance, etc. then this is a huge problem because it cuts the effective liquidity down proportionally. This works in the spherical cow model because zero times any number is zero, but as soon as you plug in any actual nonzero number, you'll find that amplifying the bias this way is actually a problem.
So what about Grossman-Stiglitz paradox? (On the Impossibility of Informationally Efficient Markets, 1980).
The crude argument goes like this. Suppose
(1) prediction market equilibrium prices reflect the current information completely ie arbitrage is impossible
(2) gathering information to update the market prices costs money.
Then, consequently, nobody has financial incentive to produce new, correct information and update the markets because there are no profits to made Thus we have proved all efficient markets are impossible with Logic™ ...which would be silly, so we probably didn't. But as a less silly consequence, Grossman & Stiglitz present a model and argue that a market can function only if the prices are imperfect enough to make information arbitrage possible.
Naively, I think this can be understood as a statement to the effect, an efficient market can be boundedly efficient, in general and over time, but not guaranteed at any particular given time point.
Here is also a neat conjecture: any person who considers themselves informed (say, they invest constant effort to remain informed) should try to get more investors, preferably less informed and overconfident investors to join prediction market, as many as possible. The informed person gets higher expected returns from increased opportunities, but the price system will be less informative.
All of this seems premised on the idea that different models disagree about the future, and not the present. I don't see that as being true for most questions, and we already have proxies for most objectively verifiable questions in stocks/bonds/etc.
Prediction markets can only ever be as reliable as the resolution criteria. Faking the winner of a presidential election is hard. It's a question where social reality is physical reality, but where the market implications thereof are vague at best.
The same is not true for most other questions. If we grant one year ago, that ivermectin is safe and effective and covid vaccines are not, knowing that the FDA is going to say the opposite in a year, is still not surprising, and is not a meaningful evidence in favour of getting the vaccine, lockdowning harder, not abolishing the FDA, etc.
Also, you don't really need a prediction market to know what the FDA is going to say about the vaccine, because vaccines are produced by publicly traded corporations, and so have stock prices.
I'm sure that legalizing prediction markets would be great and all, but I don't expect it to have the policy/political ramifications that proponents desire of it. The many downstream effects are premised on prediction market prices being useful arguments that will convince people, and for the vast majority of issues, they just aren't.
“How is this different from the wisdom of the crowd, other than adding stakes for the crowd?”
Experts can make money by taking it from those whose beliefs are wrong. That seems markedly different from merely assuming whatever the majority of people believe to be true will be true.
As mentioned, historically, stock markets have been better or as good as experts at predicting outcomes.
As alluded to in the post, many people might stand to win/lose money just based on the event happening or not, and would like to hedge their risk by buying contracts which will pay them if the event that would lose them money would happen.
So you can find willing bettors even at disfavorable odds simply because they still stand to 'win' regardless of which even happens.
One thing I'm having trouble finding out with casual Googling: what is the standard way to bet *against* a prediction? If you already have a position in the market you can of course sell your position for a sufficiently low price, but if you don't already have a position can you "short" the prediction, and if so, what are the mechanics of that?
Or do markets just naturally come in pairs where that for every "the sun will rise tomorrow" market there's a complementary "the sun *won't* rise tomorrow" market?
It depends on the specific market, in a sense I would say there isn't a standard solution, but markets do form natural pairs, with the simplest solution being to just issue shares that pay out one dollar if X condition is met, and then just issue another share that pays out one dollar if the previous share doesn't pay out (if there's a reimbursement for ambiguous results, share B is reimbursed without payout conditional on A being reimbursed without payout.) After that, you just trust to arbitrage that the spread between shares and anti-shares will be equal to however inefficient you are (because of taxes, operating expenses, bonds having higher return, etc)
I've shorted hundreds of individual stocks over the years, and a crucial addendum to 4.8 is that the most-overpriced meme stocks become hard to borrow, meaning nobody could borrow shares to go short even if they wanted to. Even when shares are available to short, there is a risk that if you go short the price will spike and it will become hard to borrow and your shares will be recalled by the lender and you will be forced to cover your short at a very bad time, regardless of how much collateral you have. These structural barriers to short-selling make mispricings persist longer in the stock market. None of that is a problem on prediction markets. There is no limit to how many "no" contracts I can buy, and there is no risk of a temporarily overpriced market forcing me to close my "no" position prematurely. This will make prediction markets much more resilient than stocks against upward manipulation / memes / ponzi dynamics.
> There is no limit to how many "no" contracts I can buy
Well there is, it's just higher, because it is limited by your gambling bankroll, rather than your ability to borrow money from a broker.
You can see the effects of limited liquidity very clearly in the Salem Center markets, because the participant have limited amounts of money there and thus obvious mainsprings persist.
I just want to emphasize that borrowing shares of a specific meme stock is much harder than borrowing money. Even Warren Buffett couldn’t short some of those if he wanted to.
The time value of money presents serious issues that hamstring current prediction markets as well. An event a year away needs to be mispriced by like 10% for it to be worth intervening. I think it would be good to concede that this is a huge problem with existing markets in the FAQ. This is trivially fixable by simply wagering shares of SPY or something but we need regulators to let this happen. I would be elated to have a decent prediction market but regulators need to let it happen.
Oh no it’s Nash equilibria again! “If nate silver was consistently better on average you could make money by betting on his predictions. Do this until you’re rich and the market has corrected.” “If there’s an advantage to deviating from a strategy in a game, people will do it until the game equilibrates to a state where there’s no advantage to be gained by deviating.” I’m the man with a Nash equilibrium hammer and everything looks like a nail!
"Prediction markets say there is only a 12% chance Ukraine will take Donetsk by winter, which is down by 9% since the logistical problems started”
I don't know if this an actual quote, but it either is, or it is an example that Scott created to demonstrate the usefulness of prediction markets, and it suits my needs either way.
It seems to me prediction markets should not shift like this (from one stable number to another). The whole point of them is to forsee unforseen events. So what's going on? We know two things for sure:
1. The price wasn't bid down before the logistics problems (from the Ukraine example), and 2. The price WAS bid down after the fact, so we know that experts considered the logistics problems significant to the outcome of the original question.
This leads to one of two conclusions. 1. The best experts either aren't participating in the markets or can't participate in the markets. This contradicts one of Scott's claims about prediction markets. Or 2. There are questions that markets are fundamentally unable to answer. The question "will there be an event that will lower expert's outlook of Ukraine taking Donetsk by winter by more than 8 points?" could not be correctly answered by the markets, since those who could answer that question would also bid on the original question in a way we know didn't happen.
So either markets don't represent the best expert opinion, or markets can't answer questions that the best experts consider important. Either one would be a huge flaw. So where am I going wrong?
“It seems to me prediction markets should not shift like this (from one stable number to another). The whole point of them is to forsee unforseen events. So what's going on?”
That’s not the point of prediction markets. The point is that they are better than the best expert, over the most possible questions.
New information should absolutely change the probabilities of prediction markets. Something that was previously contingent (say, logistics problems has a 60% chance of occuring) is now factual (logistics problems have occurred).
That must necessarily change the probability of an event contingent that new information.
Good post. I would add one thing about why we might expect prediction markets to be accurate: Over time, people who are good at predicting will make money, and so they will keep predicting, and they will have more cash to do so with. And people who are bad at predicting will lose their money, and will probably go away, and if they don't go away they'll have less money to bet with. The net effect is to concentrate money in the hands of people who are better at predicting, and the more money you are working with, the more you sway the market. So the market is moved strongest by the best predictors.
You don't think maybe people who can predict the future in murky circumstances might have a *lot* more lucrative career options than betting in a prediction market?
Seriously, if I knew when the Ukraine war would end, and how, or who would be the next President, or whether Tesla will be the world's largest car manufacturer (by volume) in 15 years, the *last* thing I would do with that incredibly valuable knowledge is place a bet and win a piddling few grand.
I think this speaks to Scott's point about prediction markets needing to be bigger, so that you could put lots of money to work on them, motivating entire Goldman Sachs–like organizations to put huge amounts of assets into them. Then it's not “a piddling few grand”.
However big the betting market is, winning bets will never be as lucrative as success in the real world. That's why people who know a great deal about horses breed them, they don't just go to the track and bet on them. Betting on things is just dicking around, creating nothing. If you have special important knowledge, you can create stuff and get paid far more thereby.
I mean, if arguendo we imagine a world where, briefly, the most talented and informed people spend their talent and information winning bets, because that's the biggest payoff, then the real world would start to collapse (for lack of the energy and attention of talented and informed people), and the disposable income anyone has to bet would dry up, and willy nilly the rewards for creating in the real world would once again exceed what you can get by betting on stuff.
• Futuur is a hybrid real-money / play-money prediction market platform. Real-money betting is not available in some jurisdictions, including unfortunately the US (for now). However US residents can still bet with play-money, and have full access to the forecasts generated from real-money wagering.
• Real-money and play-money markets operate seperately and generate independent forecasts. On the Mars prediction, it looks like you got your ~75% "no" forecast from the play-money version of this market, but the real-money version is currently showing 97% for "no". To access the real-money forecasts, you need to click on your account balance and switch to real money. (I know, this could and should be much more intuitive... we are working on fixing that!)
• When it comes to longer-term bets like the Mars one, another point to remember is that you can sell your position at any time. So if you bet early and are followed by others betting on the same outcome, you can exit your position at a profit well before the resolution date.
> 4.6.1.1: If people use prediction markets to hedge risk, won’t that distort them?
That is, suppose that an import-export business spends millions of dollars betting that Trump will win in order to hedge against his protectionist policies. Since their bets aren’t based on the real chance of Trump winning, won’t that distort the market?
No. Suppose that everyone knows Trump has a 50-50 chance of winning. And suppose the import-export business, in the process of hedging risk, bids it up to 90-10. Since you know Trump has a 50-50 chance of winning, you can get rich quick by bidding it back down to 50-50. From your point of view, the import-export business is (in expectation) giving you free money. But they’re still happy to do it, because they’re hedging their risk successfully.
___
I think this misunderstands the main hedging argument that I'm familiar with, which is namely that some candidates may be predictably bad for the entire economy, so everybody's risks are directionally correlated.
Suppose you have two candidates, Joe Jughashvili and Sam Clemens. Sam is a generic centrist, Joe says the right words but you worry that he wants to create a communist revolution. The true probability is 50-50.
You think there's a small but nonzero chance S&P 500 will go down by >80% if Joe wins. So you bet on Joe winning, on the grounds that if he wins you can hedge your risks considerably. Other people hedge in a correlated way to you and bid the numbers to 60-40.
Now suppose an independent hedge fund manager looks at this market and polling data. She realizes that the true probability is 50-50. So she wants to bet against Joe. But then she checks her own exposure and (correctly) realizes that her wealth is also tied to this market. So she wisely refrains from betting.
Why is Scott wasting his time asking people to try different things to help promote prediction markets? Surely the correct strategy would be to devise a bunch of conditional markets (if we try promotion strategy X, prediction markets will expand Y amount) and then simply do whichever strategy the markets claim will work?
Maybe also use this to test strategies to get the restrictions on U.S. use lifted?
Also, I'm curious about whether prediction markets will get big. Has anyone asked a prediction market this question? Maybe we can, in fact, narrow down precisely how big they'll get, and how long they'll stay that way.
For that matter, what will *replace* prediction markets, and can we get there now?
Maybe we can get still more meta. Can we use prediction markets to try and find what area is most likely to have wrong predictions, and use this to make money?
I honest-to-FSM have no idea if I am being snarky or serious here. (But maybe there's a way to find out!)
It seems like if we can use prediction markets to pick which trial to run, then presumably this can be used to drive science forward by having prediction markets on which type of trials to test out on prediction markets.
Actually, has anyone tried asking prediction markets which strategy will be most likely to solve the AI alignment problem? Or climate change? Or just let Ukraine win the war?
Using Scott's logic, isn't it true that EITHER prediction markets would tell us accurately how Ukraine could win the war, OR you could get rich quick?
Can we use prediction markets to fix holes in prediction markets?
> Using Scott's logic, isn't it true that EITHER prediction markets would tell us accurately how Ukraine could win the war, OR you could get rich quick?
Assuming you know a way by which Ukraine could win the war: yes? If you know how, and the market does not, you get rich. Alternatively, the market knows at least as good as you do (or even at least as good as anyone knows), yes.
Issues with alignment are the same issues as with other potential apocalypses, as mentioned.
Decision markets worry me, mostly because of the loss of "slack" it may cause in certain economic actors, much like may happen in similar-ish things that Robin Hanson advocates (eg CEO-firing-conditional stocks). The current level of imprecision in decision-making allows for more "human in the loop" situations, where people can make decisions taking into account many things other than their bottom line.
In a situation between two choices, a human decision-maker can assess things like ethics and external impacts. A decision market really can't take into account anything that isn't both easily measurable and explicitly spelled out for it. A human CEO is actually able leave the hundred-dollar bill on the ground if it's good for it to stay there. Even publicly-traded corporations are scarily uncontrollable since due to shareholders, and they do take actions with negative effects that most individual humans probably wouldn't. Surely, a market for individual choices would be even worse in that area?
Purely informational markets (as opposed to explicit decision markets) sound relatively safe, but even they can potentially limit freedom of action in decision-makers. This can be both good and bad: the things to be weighed are, on one hand, increased efficiency, better information, and probably lessened corruption, and on the other hand, serious tightening of misaligned incentives and negative externalities.
I don't doubt that prediction markets would work as described, but I really don't know whether this would be worth it. I remain undecided, with a moderately-sized part of the "in favor" side being based on "these smart people think using prediction markets would be smart."
I think one major problem with current prediction markets is that good resolution criteria are hard to make. Resolution is ideally done based on clear objective outcomes, but what you care about is usually abstract general variables.
They are a mathematical technique that allows making correlated probability distributions over many variables.
Imagine a "how will the Ukraine war go?" market, which allowed people to make correlated bets on questions like who will be in control of various cities at various times, whether leaders will continue to be in power, who will be supported how much by their allies, etc.. Rather than getting difficult-to-interpret markets on narrow questions, this could give you probabilities for broader issues and complex scenarios.
You left out a very important part in your article, which is what source of truth to check to resolve a prediction.
Iny option this is the reason why Prediction Markets won't replace pundits: they could always say The New York Times got their COVID stats wrong and dispute whether a vaccine worked at all.
I am forever hopeful this will stimulate the creation of a fact-checking economy: newspapers that only publish thoroughly reviewed facts and abstain from opinions, that all parties can 100% agree on, but I'm very skeptical of it (I will give this a 20% chance by 2030)
Why shouldn't prediction markets invest the money waiting in the betting pools in index funds or in a randomized selection of stocks, to capture that 5% economy growth profit?
Sure! You just need to relabel the wheel. Instead of red 6 on one square you have "Valdimir Putin assasinated next month" and instead of black 5 you can have "breast cancer vaccine successfully tested before 2030". Then install your wheel at all the best high-traffic venues in Vegas, and bob's your uncle. We can solve all kinds of huge social problems with the foreknowledge acquired via this modern ouija board.
What is the counter to comparing prediction markets to hedge funds but for events and claiming they have the same weaknesses? Some hedge funds do well in predicting winning stocks, but there's no way of telling which in advance, and YOY some winners become losers and vice versa. If you believe events are independent and predicting one gives no significant skill in predicting the next, they would fail on average in the same way as hedge funds predicting the stock market.
> Someone who has resolved their past 100 prediction markets honestly will probably resolve this one honestly too, especially if they get paid to do so and will never get customers again if they lie.
So I should expect to see a lot of rugpull scams on Prediction markets, because this sort of reputational system doesn't work.
Prediction markets aren't like stock markets, because stock markets.trade items of some.intrinisic value...not opinions. Prediction markets are unrestricted betting.
What would be the legality of a company like Google giving each of its employees a free hundred dollars on an internal real money prediction market, which they can withdraw at the end of a year if they bet on enough markets?
They wouldn't be allowed to deposit their own money.
Back when people cared about movies, Cantor Fitzgerald started a prediction market called Hollywood Stock Exchange to encourage betting on the first four week box office totals of new movies. But because Cantor Fitzgerald leased the top floors of the World Trade Center, a huge number of Cantor Fitzgerald employees were murdered on 9/11.
Eventually, a reconstituted Cantor Fitzgerald applied to create a futures market in movies. But the feds couldn't figure out what to do about insider trading -- e.g., if you saw a preview screening of "Amsterdam" or "Babylon" were you entitled to point out they weren't very good movies? So the market wasn't allowed to get off the ground.
One issue that is alluded to and other commenters point out is the 'oracle' or 'arbiter' problem, wherein the outcome of a given market depends on some measure that we are relying on some sensor or trusted reporter to accurately detect and convey.
If your prediction is "Will the maximum temperature in Boston on December 25, 2022 rise above 50 degrees Fahrenheit" there is the risk that the thermometer being used to measure the temp could be off by +/- 2 degrees for some reason, but maybe predictors are aware of this and adjust their bets accordingly. Still,
This could obviously be the problem for a market like "Will Bubba Joe have a Good Day on January 1st, 2023" because the only way to get the answer is to ask Bubba "did you have a good day" and maybe he reports that he sure did, even though his internal state is that his day was actually quite unpleasant.
In this sense, the contract is less betting on the actual underlying reality and more on what the reporter is going to say, which can add more and more uncertainty the less reliable the reporter is.
Hence, Prediction markets probably work best for events where many people can measure an outcome for themselves and confirm the validity of the report, rather than situations where we have to rely on a single 'authoritative' source which *could* be compromised.
On the positive side, we can also consider that if prediction markets gain widespread use, additional financial instruments can be built on top of them that abstract away the actual probability side of it and allow people to take positions on outcomes without having to be fully aware of all the information that might influence said outcome.
Re 5.6 (Education). When I was a kid I developed a habit that I now realize serves a similar fog-clearing function to the “what odds would you give this in a prediction market” check. When I want to check my confidence in something, I wonder, “How scared would I feel if I found out I had to get a lash across the back for each instance I was wrong?”
E.g., if I start by thinking “Everyone seems to like me, I don’t think anyone I know has maligned me behind my back,” then ask the whipping question, I realize I’m not so certain - because I have some nonzero fear about how much pain I’m in for.
It’s helpful for checking whether you have grounded confidence in something or whether you’re just hyping yourself up.
Your proof that prediction markets are canonical is wrong. You need to replace "get rich quick" with "get rich in expectation". I learnt recently about risk-neutral probabilities and how S&P futures are a counter-example to your claim. The historical average return from the S&P is something like 10% (maybe something like 7% after inflation). The average historical price of 1-year-out S&P futures is something like 2% to 3% above whatever the current price of the S&P was. (I may be slightly misremembering the numbers from when I was shown this example a couple weeks ago.) So you can make pretty much arbitrary amounts of free money in expectation by correcting this mismatch.
So here are some actual limitations on prediction markets that have good theoretical backing:
1. The accuracy of a prediction market is bounded by (at least) the highest savings account interest rate. If the resolution time is a year out and you could make 4% return on your money by putting it in a savings account or a CD, you shouldn't expect anyone to be correcting the market if their return from doing so is less than 4 cents on their dollar
2. Long-term prediction markets also need to correct for inflation
3. If the outcomes are not independent of the value of money (if you're betting in USD on the collapse of the dollar, or if your betting on an AI apocalypse in any currency you don't expect the AI to care about), you shouldn't expect pricing to reflect probability. I'd be willing to bet basically my entire fortune that humans will still be alive tomorrow (as long as the return is higher than the interest rate on savings accounts), because this increases my expected utility in worlds where humans are alive without changing my utility in the other worlds.
"And although the contributions of a hundred people like me would add up to $85,000 and correct the mispricing, I guess there aren’t a hundred people who know about this strategy and care enough about making a few hundred dollars to pursue it. So the site continues to lean slightly conservative."
...he said, correcting the next election's mispricing.
Although it wouldn't solve everything and would be hard to implement at scale, one path to addressing the issue of betting on things like the world ending by some date is to have the party that believes the world will continue pay out immediately so that the party who thinks the world will end can enjoy the funds while the world is still here. If the world doesn't end by the specified date, the party correctly believing the world would continue would receive a large payment from the counterparty to compensate for the opportunity cost. Garett Jones proposed this idea in more detail here: https://www.econlib.org/archives/2012/09/how_to_bet_on_b.html
great point Doug. This is one of the reasons it would be hard to implement at scale. There are conditions it would work under when you have trust with the counterparty, but it depends a lot on the circumstances. If you'd like I can provide some examples, but I'm guessing you can probably imagine a few just as easily.
You mention a couple times that random people trying to participate in prediction markets should expect to lose money, but on Manifold you can see people's profits, and a strong majority of accounts I've looked at have positive profits. I was confused by this for a while but I think the reason is that Manifold markets are subsidized by market creators and by Manifold itself (through bots). (Weirdly, those bots also have positive profits! That I don't get.) This probably makes the site more fun for everyone, but also makes it harder to tell if you are doing a decent job. It would be cool to know what the average user profit on Manifold is (both in absolute numbers and as a return rate); that could maybe be treated as the zero point for individuals like me trying to have a sense of how we're doing.
"I envision that someday people who want to know the answer to specific questions can subsidize prediction markets on them. For example, the Democratic Party might subsidize a conditional market (see 5.1) about which Democratic primary candidate is most likely to win the general election."
Ok, so say the Democratic Party sets up this market, and plans on taking action based on that information. And let's say the underlying reality is that Biden is a bit more likely to win the 2024 general election than Newsom.
However, the Republican Party and its supporters then pour a lot of money into the market to make it look like Newsom would be a bit more likely to win than Biden. Sure, other bettors are busy making bets to bring it back in line with polls and experts, but the Republican Party keeps pouring money in.
At the deadline by which the Democratic Party needs to take action, the market says Newsom stands a better chance of winning the general election than Biden. Due to that misinformation and the actions the Democratic Party takes as a result, Newsom wins the primaries. Then Newsom loses the general election, a more likely outcome than if Biden had been nominated.
Regarding doing destructive things to manipulate markets, a more complete answer is that having made those market transactions is a big piece of evidence of who did the destructive thing that tends to get you imprisoned for doing it. I remember reading about just such a case in (of course) Florida a few years ago: a guy was arrested for trying to bomb Target stores to make Target's stock price go down.
No, that's not quite true. In the long run in an efficejy market your rate of return (over market average...0 here) can reflect the employment value of someone of your ability spending that much time doing research.
For instance, imagine a market in which people bet on a mathematical hypothesis. If you are a mathematician you can gain evidence about that claim by doing mathematics about it. And you should expect that if you have sufficient capital you can basically recover the value of the time you invested (if you can get more the other investors will hire their own mathematician if less then they won't).
Right, but in your example the market is not yet efficient until the second group hires a mathematician and then the returns approach zero. The first mathematician might be able to make enough money to recoup their time, the second just zeros out the two sides and nobody makes money.
1. There seems to be a confusion between "prediction markets will be accurate" and "prediction markets will be accurate after the fact."
2. The whole point of insider trading is that no one else knows about it. To argue that some insider is trading would be valuable information for prediction markets is both accurate and incoherent.
Did you mean to say "accurate and incoherent" or "INaccurate and incoherent", because the former sounds strange as a combination
1. What do you mean?
2. I don't think anyone would know insiders were trading on their market specifically, but there would be a general knowledge that insiders often/rarely trade or that there were weak/strong regulations against it.
I'll try to get back to 1 when I have time; it's more subtle than I have focus for right now. For now, the best I can say is that the accuracy of a prediction market prediction is actually veridically proven only when the underlying event occurs, not when people bet one way or the other.
2. "In fact, knowing that President Biden is insider-trading on a “Will President Biden resign?” prediction market should only increase your confidence in it getting the right answer!" That looks an awful lot like a claim that someone would know that an insider was trading.
With #2 it looks more like "if an insider trades on a prediction market, this makes the prediction market more likely to be accurate, whether or not you know about it (since the insider knows the true result and is incentivised to move the market toward it); however, this means that if you *do* know about a particular insider trading on a particular prediction market, your *assessment* of that specific market's accuracy should increase relative to your prior".
>For now, the best I can say is that the accuracy of a prediction market prediction is actually veridically proven only when the underlying event occurs, not when people bet one way or the other.<
Sure, but — so what?
That is, for a given definition of "proven", you can make an argument like this against almost any sort of attempt at knowledge:
• "A study is only proven accurate when a meta-study confirms it." • "An ETA is only proven accurate when the item actually arrives." • "A poll is only proven accurate when the election turns out the way it had indicated." • Etc., etc.
But: that doesn't mean these are not useful tools, nor that they are not / cannot be accurate! (That is, "useful/accurate *relative to available alternatives*"; e.g., simple guessing or auspicy.)
Indeed, what's left, if we apply such a standard universally? Would one argue against the utility of, say, UPS producing an "Estimated Delivery" date for customers — data-driven, sure, but of *unproven* accuracy up 'till the event itself occurs?
Yes. Yes, one would, because UPS "Estimated Delivery" dates are complete garbage and those morons could fuck up a wet dream.
...But still, if I'd chosen a better example, you'd see what I mean, right?
What claim beyond "this is more accurate than most other ways to make predictions and can be useful even if not infallible" do you see us as making?
--------------------------
One can obtain a good sense of how accurate the market's conclusion is likely to be, if there is some worry about this:
★ Look at how many 70c "shares" turned out to occur. If roughly 70% of them did...
★ Do the same for other questions and prices until satisfied of accuracy. This has been empirically demonstrated elsewhere to *my* satisfaction already, of course; but if it turns out that *you* discover otherwise...
★ Well, then: If so — if you find the markets to be consistently inaccurate — you can make money!
★ ...and if you can make money, so can others. And if everyone is buying in such a way, we have price pressure in the direction of the more accurate prediction/price.
★ Hence, we—... Okay, okay, you've got it from here, I'm pretty sure!
--------------------------
>That looks an awful lot like a claim that someone would know that an insider was trading.<
I don't think so. User magic9mushroom, above, explains it excellently and in the same way I understood the entry; but since I (vainly) feel compelled to add a separate attempt here anyway:
The point being made works whether or not one can determine if insider trading is occurring, so a yea or nay there is not really important to Scott's argument: if you can, cool, you know it's more accurate than usual; if you can't, cool, there's no reason to worry, because see previous clause.
In other words, whatever your contention re: how knowable insider trading would be, Scott can shrug and say "okay, sure" without affecting the main thrust of the section, which is only really about the effect on pricing.
"The whole point of insider trading is that no one else knows about it. "
The point is that the public doesn't know about some relevant information the insider trader knows.
An insider trade always reveals information, because their trades shift the market towards whatever the insider knows will happen.
The only question is whether the gain in information is worth the loss from non-insider traders quitting in disgust because they keep ending up on the wrong end of an insider trade.
#1 would seem to indicate that we need a prediction market about prediction markets. I'm only 10% kidding here.
No, the point of insider trading is to use information not available to the general public. It is the illegality of using insider trading in stock market for example, that leads to it being hidden from others.
I don't know about 'canonical', they're subject to manipulation too. I think they're useful as an additional source of information where people involved have real 'skin in the game'.
"Canonical", here, means that sufficiently-liquid prediction markets can't stably disagree with each other due to the arbitrage issue. They can be wrong, but all of them will be equally wrong, so there's a "canonical" prediction market answer.
(That is, if you manipulate one prediction market with loadsamoney, you'll indirectly move all prediction markets about that issue, because people will arbitrage the manipulated market with others and so you'll wind up soaking all of those others as well.)
>Sufficiently liquid
I wonder how one would go about determining this. Scott's manipulation attempt was about very low-stakes scenario about random Manifold person getting charged with a felony. Maybe manipulating that information is genuinely worth only of some 200 USD, for Scott. However, value of manipulated information about a high stakes political outcomes could be plausibly worth billions. (Say, war in Ukraine: think of price of high tech weapons, value of assets under stake, and then of course, the cost of all involved human life.)
Alternatively, if there exists some other market (say, an insider trade of Ukrainian chocolate factories or high-tech weapons stock futures) where the information is more valuable than the currently available prediction markets, a person with the valuable information would be advised to trade on the other market.
My point is that if one can actually trade on these markets (i.e. you aren't just hacking the market to say the price is X but not letting people take the other side, you're actually trading it at X), then manipulating, say, the Polymarket "will China invade Taiwan in 2023" question with billions of dollars would manipulate all "will China invade Taiwan in 2023" questions in other prediction markets (e.g. Kalshi) by proxy as people repeatedly arbitrage away the difference and you pour more money in to maintain the peg on your directly-affected market; all the prediction markets still give the same answer, it's just a wrong answer.
(In the specific case of China/Taiwan there are semi-plausible perverse incentive issues related to the uselessness of money if you die in a nuclear fireball. But my guess is that there'd be enough time for the market to resolve and you to spend the money before ICBMs start getting thrown, so it's not as fully perverse as "will there be a nuclear war" (which won't resolve YES until too late). In any case, you get the idea.)
Not disagreeing. My comment was less a direct reply and more a tangential pondering about what happens when the prediction markets are small or illiquid compared to value of prediction.
A limited nuclear war might not kill *everyone*, and someone far away from major targets might be able to profit from starting one and survive to enjoy their winnings. There are also numerous people who might enjoy seeing urban centers destroyed for ideological reasons.
Are you talking about insider trading, rather than the "prediction market stably doesn't give the right answer" issue I was referring to (along with manipulation)?
One point regarding the "will there be a nuclear war" market is that generally, like most businesses, prediction markets are headquartered in major cities; if the prediction market's records are destroyed before the market's declared resolved, you don't get paid (and if your bank's destroyed before you can physically cash out or pay for the stuff you want, again, it's useless). And, indeed, "a place away from a city" and "emergency supplies" are the things I was implicitly referring to when I said there'd probably be enough time to spend the money for a "will China invade Taiwan" market.
It's fairly hard to start a nuclear war deliberately, and there'd be a significant risk of hyperinflation and/or asset seizure in a bunch of jurisdictions on top of the "do you even get the cash and/or survive to spend it" problem, so I think the "every prediction market is also an action market"/insider-trading issue is relatively minor - the financial incentives don't work very well, and a prediction market only actually helps achieve something you want for ideological reasons insofar as it lets you give *other* people financial incentives.
I'm pretty sure I lifted the hierarchical court system, where each court is theoretically a prediction market about what the next higher-up court will say, up until you reach the Court of Final Settlement (Supreme Court), from somebody else. Possibly even Robin Hanson.
I think it was me - https://twitter.com/ESYudkowsky/status/1455222042186305541 . And now I'm completing the cycle by laundering it back from you, because it feels awkward to claim to have invented something in my own post about it.
How about refining it with prioritization by transaction volume rather than selecting randomly? Or perhaps alternating, ruling on a case with the most at stake on odd-numbered days, then on even numbered days resolving whichever is currently trading closest to 50/50. Goal being to focus the court's scarce attention on matters of consequence and uncertainty, thus providing the least noise and most value per unit of judicial signal... and incidentally make market manipulation even more self-limiting than normal, since the harder someone tries to buy an unjust verdict the higher that case would drift toward official attention.
I thought I first came across the concept in a post on Scott's blog back in the LiveJournal days.
I have a strong intuition that "people who have money to spend on prediction markets" and even more "who also have an interest or knowledge of prediction markets" is an incredibly biased sample of people.
I'm just very pessimistic on the ideal vs the practice. The same kinds of negatives I consider for the stock market. Or markets in general. Prediction markets depend very strongly on right wing economics being accurate imo.
In a sense that's optimistic, because you'll be able to make a lot of money if you can figure out what their biases are.
There are some cases where the equilibrium answer for a prediction market with perfect information is not the same as the true probability of the underlying event.
"The US undergoes 1,000,000,000x inflation by 2030", denominated in USD, is one of the obvious examples; if you assume for the sake of argument that this is a 1% chance and that everyone knows this, the equilibrium price for a contract that pays out USD$1 on YES is far less than 1 cent (because if YES is true, USD now is worth at least a billion times USD when the contract pays out, while this is not the case if NO is true).
Similarly, the price of "will the world end" is $0 on a standard prediction market because a resolution as YES prevents you from collecting. The same is true of "will the records of this prediction market get lost", which means a lot of extremely-disruptive things (e.g. nuclear war) don't show up in prediction markets.
This issue is discussed in the post.
Check the Wayback Machine; that section was added after almost a day. Substack doesn't give me exact timestamps, so I can't say for certain that it was after I said that, but it's pretty likely.
Also, I wasn't responding to the post directly, but to a comment making the argument that mispricings are necessarily exploitable.
Only if you have a bunch of spare cash floating around.
Don't need cash for a prediction log - demonstrate the bias is large, and the funding will come.
Yes do better than market for years with tons of time investment before you see a dime. Starving artist version of prediction market user, meh.
That sounds like an inability to demonstrate the bias is large. So, problem solved.
No because you'd need to build up a record significant enough to not be luck.
The absolute gains will be smaller if you don't have much money but the relative gains should be the same. If you can double your money every six months, that should hold true whether you bet $10,000 or $100.
Actually, your rate of returns should go down the more money you put into prediction markets since you'll move the price.
Gain relative to time invested gets worse if you don't have seed money.
> how likely to the claim is to be true.
Typo, that first "to" shouldn't be there.
My main question whenever I read about prediction markets is: why are they necessary? The best answer I can surmise from *light* googling is that it can serve as the most accurate representation of the wisdom of the crowd. Is that a good one-sentence answer to the question: "Why prediction markets?"
I agree, specially if you believe ppl putting down money on something is a very good indicator of what they actually believe at that point.
Interesting... so sort of like a way to cut through any propaganda or BS.
Example: Mainstream media/ideology is that X won't work, but everyone betting in the prediction market says X will work, since that's a more realistic indicator of the assumed body, it's more likely X will work?
Sort of. People have different biases, and when you take a large number of people without selecting for any bias relevant to the prediction market, then you get a lot of biases cancelling each other out.
It is similar to how if you ask individuals to estimate the number of marbles in a glass jar, they display wild variance, but if you average the estimates of individuals, the aggregate usually performs better than most individuals.
Except when people are putting down money because they want to make more money, and not because they hold a strong belief towards a particular prediction
Think of it as a way of paying Superforecasters and experts.
Accurate and trusted, establishing common knowledge in the long run.
If someone doesn't trust it, they can make money. If they just claim they don't trust it, they won't be believed unless they put their money where their mouth is, and people become instantly less tribal and stupid when their money is on the line.
>As a test, I tried to manipulate the market on whether Austin Chen, founder of Manifold Markets, would be charged with a felony.
If I recall correctly, you did this with his explicit permission? It would be worth having that fact displayed next to that sentence if so, since otherwise this reads as a minor alarm bell.
I just wanted to pay a compliment. This is so beautifully written. A complex concept, simplified perfectly, without it being dumbed down.
Thanks - and thanks to the many subscribers who gave comments on the rough draft which I tried to take into account.
Thanks for writing this! If only we could tag Rostin Behnam :P
In addition to betting on Manifold, ordinary folks in the US and abroad should also participate in the markets on Kalshi, Polymarket, Insight Prediction and PredictIt. Real money is the ultimate play money, as a reputation system that most freely transcends the prediction platform. You don't need to be a hotshot prediction expert - skin in the game allows you to learn quickly about your interest.
To promote prediction markets from a regulatory standpoint, the CFTC cares about the economic purpose of these prediction markets, namely their use to hedge risk and serve as a price-basing mechanism. Directly using these markets for information or to hedge risks and demonstrating a positive economic effect to your life and business is a central legal argument to getting these markets approved!
I had a concern with prediction markets that I think answered myself. Still posting it here for edification and potential correction if my answer was wrong.
> One concern I've had with prediction markets that this post doesn't address: is it good to expand the set of things that companies can hedge against?
> Let's say that I'm Satya Nadella and I'm worried that a startup might disrupt and take over Microsoft Office. Today, I can action that worry by investing into Microsoft Office, investment that presumably makes Microsoft Office better, increases the productivity of its users, and therefore raises the GDP of the human race.
> Let's say we live in Scott-topia and that I'm Satya Nadella and I'm worried that a startup might disrupt and take over Microsoft Office. I might decide that instead of investing in Microsoft Office and reducing the dividends paid to the shareholders that employ me, I'll buy a large stake in "yes" for the prediction market "[w]ill Microsoft Office make at least x bajillion dollars in fiscal year 20XX?" so I'll be set no matter what.
> To phrase my objection more cogently: what propels our society forward are the holes in the efficient market hypothesis; the innovations that are not priced into asset prices. Will large companies ability to hedge against those reduce the incentive to create and fund those innovations?
I think the answer to this objection is that the startups/VCs will take the opposite end of the trade with Microsoft and it'll end up being a massive waste of time for everyone involved (since they could just invest the money in their products) so they won't do it.
> I might decide that instead of investing in Microsoft Office and reducing the dividends paid to the shareholders that employ me, I'll buy a large stake in "yes" for the prediction market "[w]ill Microsoft Office make at least x bajillion dollars in fiscal year 20XX?" so I'll be set no matter what.
Do you mean "buy a large stake in 'no'"? If they bet on the prediction market that their main business will go well that's the opposite of hedging.
Hedging like that seems pretty dangerous though- if you drive up the price for "Microsoft Office gets disrupted" in the prediction market, that seems likely to hurt you in the regular stock market.
It is unlikely that the size of the market for Office to make X bajillion will ever be comparable to X bajillion and not 1000 smaller. Also, if market participants were aware that this market could be used by Satya for this purpose, they would quote pretty lousy odds for any size.
In general, market liquidity exists in size only if either 1)there are large participants naturally on either side of the market, like oil futures market is big because oil producers are naturally long and oil consumers are naturally short 2) the market can be hedged to a decent degree with an already liquid instrument.
People can already short Microsoft as a whole. Enabling them to, in essence, short Microsoft Office doesn't seem like it'd make that much of a difference?
There is a large market in MSFT shares and derivatives on them. There are many natural buyers and sellers on this market, e.g. investors who want to hold more or less of MSFT. If Office is a relatively small part of Microsoft business, shorting Office requires an entirely separate market , one that cannot be hedged well in existing markets. I do not see many natural buyers or sellers for this market, so I do not think this market could grow to a scale comparable to market in MSFT shares.
This isn't central to your argument, but:
"Finally, I envision that someday people who want to know the answer to specific questions can subsidize prediction markets on them. For example, the Democratic Party might subsidize a market about which Democratic primary candidate is most likely to win the general election."
-- how would this market resolve? Only one person is going to get a chance to win the general election.
Conditional market - see 5.1. Thanks for noting how confusing that is - I've edited a reference into that sentence now.
Interesting. I followed the link in 5.1, and got to the part which uses your own unbanning decisions as an example, because the world is a loop. Anyway, the example there suggests that in the Democratic example you would avoid a bunch of failures/distortions by not announcing in advance that the outcome of the set-of-conditional-markets will influence the selection of the candidates, which is a slightly disturbing situation because the whole system generally relies on everything being transparent.
I'm not crazy about the first sentence "Prediction markets are like stock markets, but for beliefs about future events." There are two crucial differences: most prediction markets have binary outcomes and they resolve at some point. This is not the case for the stock market. While most Americans are not familiar with binary options, I don't think using the stock market as an analogy is accurate enough. I'd just compare it to sports betting, even though if it may come with a slightly negative connotation.
Also, a slightly substandard stock market investment will likely still make you money in the long run, while a slightly substandard prediction market "investment" will lose you money. This is the difference between investment and speculation, and non-experts should always avoid speculation.
This whole FAQ seems to *at the same time* claim that prediction markets make the best predictions possible, *and* that there's money to be made by beating the current state of the prediction market. I can't see how these views are compatible.
One thing that annoys me about this article is the recurring thing about "if I'm wrong you can get rich quick", which I realize is sort of "a bit", but also feels like a taunt. "Well, if you're right, you could make money so stop arguing." No, if Scott [et al] is wrong, he's wrong, and whether I make money from it has nothing to do with it. (Also, it's going to be "place positive-expected-value bets that resolve over maybe years", not "get rick quick", but like I say, I know it's a bit.) I distinguish this, emotionally, from "if the market is wrong people will bid it towards correctness", which is intrinsically part of the argument.)
> This whole FAQ seems to at the same time claim that prediction markets make the best predictions possible, and that there's money to be made by beating the current state of the prediction market. I can't see how these views are compatible.
They become compatible when beating the current state of the prediction market is impossible, because the consensus is accurate. You're correct that these are diametric forces governing the market dynamics - the equilibrium determines the size of the market. Present status quo, markets seem to get it right within a few percentage points and the people who consistently make money there aren't making more than a typical software engineer.
Overall, coarse grained efficiency is compatible with fine grained mispricing -- indeed, it is betting on mispricings,and therefore erasing them, that keeps markets efficient.
Conversely, stock markets ARE for beliefs about future events (specifically, future stock prices), so the "but" doesn't make sense here. The differences, instead, are those Lech lists.
Not exclusively - there's also the dividends, and while to an extent this is a belief about the future as well (the company will actually pay dividends), this is at good odds. As is "we can expect the economy to grow".
This is why stocks are (typically) investments, while prediction markets (as well as gold and futures) is mostly about speculation (although futures have other uses, like in controlling risk).
You could frame it as: the stock market is like a prediction market, where people are trying to estimate the price of a share of a company.
The resolution in a prediction market is binary but "price of a share of a company" is not resolvable in a static fashion, because the company itself changes. At one point in time, a company might have been undervalued, and it'd have made sense to buy more stocks of that company. 10 years later, the board has changed, the leadership has swapped companies, and they're dealing with new competition. Now the problem of estimating the price of a share has a resolution again, but one different to the resolution 10 years ago. So, in stock markets, we do have resolutions, just dynamic.
Couple things...
- Much of this just seems like "efficient markets hypothesis" repeated in different ways. Is that not right? Is there a reason to think it'll be more accurate than the stock market's predictions? The only one I saw is the point about how prediction markets can't have a ponzi scheme go on forever, but that applies to certain stocks as well, and doesn't seem like a super broad objection in any event.
And the thing is that (1) I don't think most people would call the stock market "accurate" or "unbiased" except perhaps in a hypertechnical sense, and (2) the stock market needs a lot of TLC to make sure it works properly. There's insider trading, but that's hardly the only example. In fact, to go back to the crypto thing, part of the objection and the reason (allegedly) that it seems like such a shitshow is that they DON'T have the regulations the stock market has.
More generally I bet a bunch of economist PhDs have written a lot of useful shit about the efficient markets hypothesis, that this could probably benefit from some engagement with, if I'm right about this being similar.
- Is the idea that prediction transactions will be public? There's all this stuff about how you can follow Nate Silver's lead, but does Nate Silver have to reveal who his bets? What if he's just quietly sitting there, anonymously betting, not making enough money to have anyone notice?
- I think "in the long run" is perhaps a larger qualifier than people are assuming. To take presidential elections again, they are every 4 years, and the political landscape changes over time such that there's never all that big a sample size of analogous presidential elections to the current one. If Nate Silver is the best predictor (to go with the analogy, not saying it's true), maybe his biases simply match the current landscape and it'll be different in 8 years, or maybe given the small sample size (4 elections?) and the inherent uncertainty of judging percent predictions for binary outcomes it's simply hard to say who is best.
- As a test, I propose something other than big picture geopolitical developments that people normally associate with prediction markets. Sports! Much higher sample size (2430 regular season baseball games per year, and that's just one sport, and the outcome is numerical and unambiguous!), and there's already a lot of predictions out there you can compare to. Compare the sports prediction market to Vegas money lines/odds, pundit predictions, etc.
Prediction markets AFAICT have (in theory) an advantage over Vegas odds, which is that the house edge probably negates the positive EV from certain arbitrage strategies, which probably makes them less efficient in theory.
- Regarding the "zero sum" point, can't you take the money from the primary (i.e. not secondary) purchasers of prediction "shares" (or whatever you call them) and invest them in US government bonds, and the interest is paid out proportionally to the winners?
In the absence of Likes, I just wanted to say that this is a great comment.
Sports betting market _are_ prediction markets, in that sports books respond to misprices (as revealed by imbalanced betting) by changing their odds -- either the actual point spread, in those cases, or the price to buy one side or the other. You're right about the house edge making it harder to exploit mispricing, though. Hell, in horse racing, the house edge is 17% on win/place show betting! More typical sports betting is about 5%.
"... suppose that an import-export business spends millions of dollars betting that Trump will lose in order to hedge against his protectionist policies ..."
Wouldn't that business want to bet he would *win*, not lose? (That is, if the policies of Trump's main competitor would be more beneficial, on balance, for that import-export business.)
If Trump wins, the business would presumably lose some revenues and profits due to his policies. But winnings from their prediction markets bets on his election victory could at least partly offset that.
(If I'm confused and have misunderstood this, welcoming corrections!)
You're right, thanks, this is why I mostly invest in index funds.
Investing wide, like an index fund, or for non-Americans a global index fund, does have the added advantage (on top of being cheap) of removing yourself as far as possible from the risk of market manipulation and insider trading. You make yourself harder to exploit.
I think that, in the ideal case, prediction markets will work as you say, and that would be a good thing. Likewise, in the ideal case, cryptocurrency would also usher in an era of decentralized prosperity, and that would be a good thing too. But in practice, as soon as large sums of money (even virtual money) is in play, the entire utopian landscape shatters into a hodgepodge of Ponzi schemes and general fraud. This is the case with cryptocurrency; this was the case with the stock market; and this will be the case with prediction markets if they ever gain enough popularity to be useful.
In the case of the stock market, the problem was somewhat improved by regulation. There are still scams and clever get-rich-quick schemes, only now they have to be a lot more clever, and perhaps valuable enough to buy off a regulator or two. The stock market sort of works, but it would be foolish to claim that the value of every stock accurately reflects the true value of the company it represents.
Sadly, I don't think prediction markets will ever get even to that level of accuracy, because the incentives are much more entangled. Arguably, a share of "Joe Biden will win the election" is, indirectly, a lever that someone can pull to influence the election; thus, regulators with their thumb on the scale would wield considerable power. Selling the election on the conventional stock market is much more difficult (if not outright impossible).
Is it that much more difficult? You can buy or sell a package of stocks based on things that the candidate/candidate's party has said they'll do if they win the election. Coal mining companies' stock goes up if Republicans win, down if Democrats win. Same for renewable energy companies, anything that one party has talked about putting tariffs on, health insurance companies (stock goes way down if single-payer is instituted, up if individual mandate is enforced, etc.), etc.
This is incredibly and informative and well written. However, to the extent that I find prediction markets objectionable, it has very little to do with their predictive power, and more to do with the aesthetics of how the analysis is presented. I frequently, both here and in other rationalist spaces, see potential human disasters referenced in terms of prediction market odds. Whether it be numbers of future Covid deaths, mass shootings or the likelihood of Putin launching a nuclear weapon, someone is bound to bring it up in the context of prediction market odds, and those odds, definitionally, are defined by people who are framing the possibility of vast human suffering preliminarily in the context of whether they can make any money out of it.
I understand that the predictive power of the markets still have value in these cases, but even so, there's an "ick" factor I can't get past. It feels like reducing potentially devastating events to data analysis/investment opportunities without acknowledging the human toll plays into the worst stereotypes about the rationalist community.
I don't think this is weirder than eg citing a paper written by an academic who hoped to get promoted for it, or calculating the odds on a computer that someone at Microsoft or IBM got rich for inventing, or reading about them in the New York Times knowing that your subscription money is going to the Sulzberger family. You just accept that people have incentives and you trust them if the incentives are aligned with getting the right answer.
To be honest this seems horrifically misguided to me. The purpose of prediction markets is to get information about the likely hood of these events. In the case of disasters having more such information is essential to mitigate or prevent them. The fact that you could, in theory, make money by participating seems to me to be entirely unimportant by comparison.
Opposing innovations that will help save lives for purely aesthetic reasons plays into to the worst stereotypes about people who are ignorantly anti-rationalist.
I get it, and I should clarify that I'm more put off by the general discourse around prediction markets than the markets themselves. This is one of those things I would not have even bothered commenting on if I'd seen anyone else echo the same sentiment beforehand - it just feels like *somebody* should bring it up.
As to your last point, I guess I do feel like the rationalist community has issues with how it is perceived by the broader population, and, in general, somewhere between a reluctance to, and a principled stance against, modifying its behavior to address this perception. Which I get, but I also think that perceptions *do* matter, especially if we want to convince other people to join our community who may otherwise be put off.
I think pushing this point any further would risk some maybe legitimate accusations of "concern trolling", so I'll let it go at that.
I'm quite torn on this. I do think that perceptions matter and that there is value to trying to improve other's perceptions of this community. On the other hand, most criticisms against rationalism/EA/prediction markets/whatever are so *completely* illogical, that it seems mostly out of our hands.
For example, in the wake of the SBF implosion, Effective Altruism got a lot of bizarre and irrational criticism. I heard many people say things like "I agree with the ideas of Effective Altruism, but find it troubling that the first step is to get rich". Apparently none of the people saying that thought to check if the first step of Effective Altruism was actually to get rich. To the best of my knowledge this is not a view that is espoused by any Effective Altruism organization or public figure, not that it matters to them. Arguing against people making these sorts of strawman arguments is completely counterproductive, and that probably accounts for a lot of the reluctance to address perceptions.
Given this environment, it is hard to see how to actual make progress on the PR front. If you had specific ideas about how to solve these problems I suspect that they would be very positively received, but I'm not convinced its really possible.
I also think that suggesting people are excited about prediction markets because it would enable them to get profit off of tragedy is inaccurate and therefore more similar to the EA criticism example above than actual actionable advice about improving our perception, although I suppose I could be wrong.
You draw comparisons between prediction markets and stock markets, but I’m curious if you think sports betting is another useful analogy. The global market for betting on, say, the World Cup outcome is probably orders of magnitude larger than any of the relatively niche prediction markets you highlight. Based on these answers, I would expect one of four things to be true:
-The betting market is an accurate, unbiased representation of which team is likely to win.
-The betting market has a consistent, material bias, so Goldman Sachs hires quants who are experts at soccer to take smarter bets, make tons of money, and correct the market.
-The betting market has a consistent bias, but Goldman Sachs isn’t exploiting it, so there’s a ton of money on the table that someone could pick up tomorrow (or at least in 2026)
-The betting market has a consistent bias, but it’s not viable or worth it for anyone to correct it (like the prediction markets you mentioned today)
As far as I know, option #2 isn’t happening. I could be wrong, but I’ve never read outrage bait stories about how Wall Street won big by betting against America or something. And if this is just because of regulations, I’d still expect someone to try it by setting up shop in the Bahamas.
If option #1 is the case, do we see the benefits you predict? I’m not well versed in the world of sports, so this is an honest question. Do team owners look at Draftkings picks to decide which players they should keep? Do coaches see terrible odds against their team and decide to change strategies before they even play? If so, this seems like a strong argument in favor of prediction markets, and one you might want to highlight in another revision. But if not, I’m curious why you think these markets aren’t living up to their potential, and whether you think this is a counterpoint to your optimistic take on what markets can do.
If option #3 is the case, why aren’t you making tons of money on Draftkings? And if it’s option #4, then what problems do you think are holding back sports betting markets, and why don’t you think other prediction markets will suffer the same fate?
(I suppose option #5 is that this is a flawed analogy, or that I’m missing another obvious takeaway.)
I agree these are good questions. I don't know anything about sports betting so I can't answer them. Zvi might know more.
I think option #1 is correct.
It's also just not very usuable information for team managers. E.g. if I'm a coach and I see that we're 30% underdogs vs the 70% favourites, I could change my strategy... but then again, there's no reason to expect that a different strategy would change those numbers in a favourable way, and I picked the current strategy because I thought it was the best one.
I wonder if it would be legal to have conditional sports betting, eg "What are the odds that the Sharks win the game if they use Johnson as their starter?", "What are the odds that the Sharks win the game if they use Smith as their starter?", etc.
I think it'd depend on the state. But I'd guess there's no particular reason that would be different from betting on a general game outcome. Though I'm not aware of anyone who actually does this. But I don't see how it's materially different from (as does actually happen) betting on things like how many points will be scored by specific players.
The bigger issue is what happens to the betters when the scenario doesn't occur.
Conditional sports betting is a thing, e.g. https://www.onlinebetting.org.uk/betting-guides/bet-types/conditional-betting.html. You can do things like “$50 on Argentina to win, as long as Messi starts”.
But you can have all sorts of conditions so you don’t get the highly visible big volume markets that you get with simple “Who will win?” questions.
In baseball betting, this is actually the default case. Most baseball odds I see are conditional on the listed pitcher actually being the one who starts, and the bet is refunded if they don't (due to late-announced injury, usually.) You can also (sometimes?) bet on "regardless of who starts", but the pitcher is such a large part of baseball success that I don't why you'd bet that unless you had inside info or something.
This has the same problem with causal direction as a lot of conditional markets: if the Sharks' opponent knows that the Sharks will start Johnson or Smith depending on the outcome of the prediction market, the Sharks' opponent will change their strategy accordingly. To give an extreme example for clarity: imagine a predictions market on whether you'll win a rock-paper-scissors match conditional on having played rock, paper, or scissors, which your opponent can see operating and participate in.
I'd disagree, the inputs *into* the 30-70 figure are highly usable pieces of information for coaches and upper management.
Like the models gambling houses/analytic syndicates use to predict performance are basically exactly the same as what sophisticated teams are aiming to do in house. At least in football, and I assume NBA is the same. Knowing that X team performs better when Y player is playing rather than Z is valuable for both gamblers, bookies, and coaches for example.
There are quants who consistently make money at sports betting. But there's not a ton of money in it and the kind of people who can do it usually have more lucrative option. In part because banning it has made the market relatively small. In more sophisticated markets, mostly those where gambling is unambiguously legal, the best quants are employed by the gambling houses to prevent such edges from occurring. (Or rather, to provide the house itself with the edge.) The house can usually outbid independent actors in the long run.
As to why sports teams don't pay much attention: most leagues make any involvement with gambling extremely against league rules. Most prefer not to risk their careers by even being around the gambling houses. After all, they're making a lot of money just being players/coaches/etc. Why risk an ethics investigation?
In short, they're not really comparable unless prediction markets really are acting as casinos with the house offering specific odds generated by internal quants. Sports betting and casinos have an entire third actor, the house, that is absent from the stock market and should be absent from a good prediction market. Additionally, executives in business are allowed to hold stock (modulo insider trading rules) meaning they're under a less strict regime than sports managers. In part because stock ownership aligns incentives while allowing a team's manager to place bets can only unalign incentives.
"As to why sports teams don't pay much attention: most leagues make any involvement with gambling extremely against league rules."
True, but boy is this heading for a crackup and major scandal as all the leagues form explicit partnerships with betting outfits.
Shh. It's different when the men in suits do it. (But yeah, agreed.)
Unlike a true prediction market, which is zero-sum, sports betting is pretty much always negative sum, that is, you can't make a bet and its inverse for some outcome and come out even. Bookmakers offer worse odds than would be "fair", and would probably ban anyone who for some reason is still able to win big consistently, so this is basically pure gambling.
Sports betting in the UK is negative sum, as @Xpym pointed out
Places like Betfair Exchange don't make money by overpricing flat bets like a tradiitonal bookmaker e.g. quoting 7/1 (12.5%)when true odds are 6/1 or something, but instead by charging a 5% cut on the profits made by people betting on the market.
Often due to low volume some individual markets fall into 4, cat 2 definitely used to happen more but now happens much less as the smart money came in, and cat 1 is now the default.
In fact, two of the smaller Premier League teams most associated with the belated (vs US sports) analytics wave in football, Brighton and Brentford, are owned by men who made their fortunes gambling professionally. The skill set needed is basically identical, predicting future performance in order to either maximise it or maximise earnings on gambling.
Their edge (alpha I suppose) is now smaller than it used to be, but they can punch above their weight due to better recruitment, set pieces, and finding arbitrage opportunities on the player transfer market by shopping from smaller countries.
I said something similar in my top level comment, but I think it's option 4, and the reason it's not viable or worth it is that the betting generally happens in places where there is a house edge.
So lots of arbitrage opportunities are better than "the market" but still not good enough to be positive EV.
(not that I'm necessarily bullish on prediction markets in general or the idea that teams can use them as a source of info)
I believe #4. Sports market predictions a) aren't unbiased (the people betting very often don't just try to maximize their gains, bit instead bet on "their" team in order to increase the excitement of watching the match), and b) are wide-open to corruption by rigging matches.
Re #2, see https://sig.com/Sports-Analytics/
EDIT: My comment below is about the Forex market rather than sports betting, so it doesn't speak to your point. Leaving for posterity/accountability.
---
When Liz Truss announced her rather 'brave' UK budget earlier this year, hedge funds bet substantially against the UK by shorting GBP, including those based in the UK.
I can't immediately find outrage bait, but anecdotally I remember there being a decent amount of it (though significantly more reserved for the government).
https://www.thenational.scot/news/22638822.liz-truss-backers-city-made-small-fortunes-shorting-pound-report-claims/
There are some cases where a combination of #2 and #3 is correct, like: https://www.bloomberg.com/news/features/2018-05-03/the-gambler-who-cracked-the-horse-racing-code
I want to preface this by saying I like prediction markets and think they are a net good. But I think there are some clear problems with them that you've missed or hand-waved away in a fashion I don't find convincing.
For example: "4.2: Would prediction markets encourage harmful or illegal activities? ...I think the strongest evidence against is that this basically never happens in stock markets."
I think this is an awfully rosy image of stock markets. Because of many traumas over many decades we've built up complex systems of financial regulation that make stock markets less prone to exploitative harmful schemes than e.g. crypto is - but the natural state of a financial marketplace is to be scam-infested.
So it's illegal to run a "pump and dump" scheme in the stock market, but there's no such prohibition on doing it in a prediction market. Once real money starts going into these markets, you'll definitely see people doing things like buying up a bunch of shares in an outcome, creating fake hype to get people to believe that it's a real market trend, and then dumping their shares at an inflated price before reality sets in.
And of course there's any number of other market manipulation tactics that are known. You can set up a government agency to police these, and it might even work some of the time, but do bear in mind that the Securities and Exchanges Commission has a $2.5 billion annual budget. It's not enough to say "pump and dumps are banned", you need to have people with the job of prosecuting them, and that cost is not negligible. It might be worth it! But it's not negligible.
I also think that in "4.8: “Meme stocks” like Gamestop and AMC sometimes remain mispriced indefinitely. How do we know this won’t happen with prediction markets?" you don't adequately account for the "Market can stay irrational longer than you can stay solvent" effect on markets with resolution dates long in the future, or conditional resolutions that are likely to not be met. You mention these sorts of concerns elsewhere in the FAQ, but don't apply them to the failure mode where they are relevant!
Personally, I think it's absurd that Tesla is valued higher than all other automakers combined. But I don't short Tesla because I don't have any way to prick the bubble and force the stock price to reflect Tesla's real value as a company. There's no moment of truth - I could think the market is deluded, be correct that the market is deluded, and lose all my money betting on the correct value because it takes too long for the market to realise I'm right.
Now, betting on an election outcome in a couple of months doesn't have that problem - we know that the tide is going to go out and we'll get to see who's been swimming naked. But betting on, say, which candidate is more likely to win if nominated DOES have that problem.
Suppose the market is convinced that Clinton will be a stronger nominee than O'Malley. I'm Matt Yglesias, I'm sure that O'Malley would win, and I also happen to be right. But O'Malley never gets close to the nomination, and my correct belief never has the opportunity to be proven true. The market stays mispriced until it expires. My correct bet lost me money, and the market never accurately reflected the reality that O'Malley Would Have Won.
Now, these are sort of edge-case scenarios and I do agree that prediction markets are very good in modal cases. But I also think the edge-case problems are real.
You said basically everything I wanted to, but better.
I expect the price of, say, Microsoft to be fairly well priced on the market, but I have much less faith in the price accuracy of some nickel stock most people haven't heard of. Given the apparently huge number of prediction markets you'd like to exist... What creates sufficient economic value to drive accurate pricing for all of them? Particularly when, unlike stock markets, economic motives may not predominate for the most invested groups for a given question?
If there are many more investors them markets, maybe the markets are efficient (because there's no $20 just lying around then), but that doesn't sound like what you're describing. Your market-saturated world sounds like one with lots of pennies laying around. And I leave those where there are all the time.
> So it's illegal to run a "pump and dump" scheme in the stock market, but there's no such prohibition on doing it in a prediction market. Once real money starts going into these markets, you'll definitely see people doing things like buying up a bunch of shares in an outcome, creating fake hype to get people to believe that it's a real market trend, and then dumping their shares at an inflated price before reality sets in.
This already happens. One of the nastier moves I saw on PredictIt was to spike a particular market to deliberately trigger other participants' stop-loss orders, though that only works within a narrow range of sophistication.
Aren’t conditional markets set up such that Yglesias wouldn’t lose or gain money betting that O’Malley would win if nominated but he never gets nominated? That seems to alleviate that concern that Yglesias loses money on this, except for the opportunity cost. (It also seems like you can reduce opportunity cost here by using the same dollar to bet all the mutually exclusive conditional markets that condition on different outcomes of the same event. Is that already done?)
4.6.1.1 - This claim that hedging won't bias the markets seems way too rosy. There's not a bunch of people who all know the exact probability and one person using the market for hedging. There are a bunch of people all with different guesses as to the probability and different budgets/amounts of acceptable risks about how much they are willing to invest to correct errors. The (perhaps money weighted) average of these people is the "correct" value that the market should be trying to converge to. But if someone is investing a bunch of money as a hedge (rather than an honest prediction), this will tip the scales somewhat one direction or the other.
4.8: Perhaps the better argument here is that it is *much* easier to bet against a meme prediction market than against a meme stock, making it a lot easier for the market to correct itself.
Predictions markets that only evaluate if the chance is high enough (such as in the case of your comment policy) have some weird effects that I think could invalidate the results, and might not be fixable with the usual "free money" argument. If I wanted to make sure that a comment never reaches your attention, I can vote it down below the threshold for it to evaluate, and even though I am totally wrong on my "prediction", I'll never lose any money. There is still a chance at free money, but now the consequences to being wrong are removed (as long as you are rich enough to overwhelm the market). Perhaps this works better than I am thinking in practice, but especially for low traffic votes like this I think it would be easy to pull off.
On the opposite side, if I predict that you **won't** think that a comment was worth your time upon review, I have no incentive to vote negative below the threshold, since I won't get any of the money I am expecting to make. If I have sunk enough money in, it might even be worth my while to vote positive (against my prediction) in order to get it back above the threshold so that my bets get evaluated. I think this is less easily resolved than the first issue, and you should basically never see prediction far below the threshold because all that does is tie up your money while you wait for the market to close.
You're right, I would also have to randomly check a few that didn't meet the threshold. I've edited that in.
How about picking them with probabilities proportional to market volume? I expect that to be more time-effective that just uniform random.
Prediction markets also monetise disinformation. Instead of looking for mispricing and then getting rich until the market corrects, you can spend money on a disinformation campaign to induce mispricing and get rich until it corrects.
This is not very different from pump-and-dump schemes in penny stocks, but providing a continuous, liquid, anonymous incentive to spread disinformation on *every* topic seems like it should be a concern. Not because the market won’t correct - because of the second order effects of increased information pollution everywhere else in our society.
In the optimal case this provides financial incentives for very robust anti-disinformation detection. However, solving loss of trust by drawing all of society and our information and decision-making platforms into a high-stakes HFT battle for truth might turn out to be like throwing water on an oil fire.
This assumes that the misinformation is coming through a channel that is otherwise considered very trustworthy, or else most people with money invested would probably ignore it or only adjust their bets a small amount.
Or, in the alternative, this means that people who are easily swayed by misinformation will lose money to people who are not swayed, which on net improves the prediction markets.
The benefit of making people put Skin in the Game is that they will necessarily become more discerning about which information sources they actually believe when trading.
I think I'm not understanding why 4.6.1.1 is free money. If I buy a share of Biden at 10 cents, I have a 50% chance of making a dollar, so my EV is greater, but I still have a 50% chance of losing money. Is the free money part:
(a) The EV on that trade being >0
(b) I can at some point sell that share to another trader for 50 cents
(c) I can take a strategy of buying shares for the less-protectionist candidate every 4 years and make money in the long term if hedging makes those candidates are consistently undervalued
(d) The import/export company is also hedging in other markets, so I can bet against them in every market and make an overall profit on those trades
(e) Various groups hedge various markets and I can take a strategy of betting against them in general and make an overall profit
(A). You might lose on an individual trade, but as the number of similar trades you make increases, you'll be more and more likely to gain on net. This might not be worth it for you, but it will be worth it for somebody.
I think the real issue with making a better prediction market is that it's such a niche product. Looking at the top eight most liquid predictions on polymarket they currently have a total liquidity of about $136,000. And there's only about 60 questions total. Let's be generous and roughly double that for $250k total liquidity on site. That's tiny. That's smaller than most minor league baseball games. Let's say there are five equally sized markets. That's still only about a million dollars. Let's round to a million (fair since one of them doesn't use money). Given a casino's 8% take (which would actually probably be lower since this isn't a casino) that's $80,000 in revenue. And that has to cover everything for the entire industry.
I can think of ways to build better versions. But if almost no one's going to use it then what's the point? At those rates I'm not even sure the revenues can cover regulatory costs. Regardless of whether it's a good idea I'd need to be convinced it could survive and hopefully even thrive.
Has anyone considered getting all that rationalist money behind a prediction market in an already high demand area like sports? That's a multibillion dollar industry with heavy existing demand and high growth. Yes, it's not as EA-sexy as stuff about politics or whatever. But fundamentally you'd be developing the same type of skillset and organization and technology. All that which you could eventually pivot into broader markets. Part of the point is such markets are useful for predictions in arbitrary fields. So why not arbitrarily choose the large, fast growing market?
But sportsbetting exchanges already exist. What value would a sportsbetting prediction market add? Come to think of it, Betfair does political bets as well, so I don't see what value political prediction markets would add either. 🤷
I can think of sports betting gambling houses (Fanduel, Draft Kings, etc). But not prediction market style exchanges. At least not in the US. More to the point, if this already exists then what's the point of it at all?
TradeSports was the US version but regulators shut it down ~15 years ago.
The law's changed since then though.
I'm surprised this article didn't look at the UK/Irish/etc. markets as live real-world examples of what a prediction market can do, since there's already fairly large-scale wagering on political and other real-world events. Granted, you've got bookmakers setting odds rather than purely market-driven pricing, but for liquid markets the difference shouldn't matter all that much.
To your point Betfair now owns Fanduel -- in theory the existing betting operators are best positioned to take advantage of eventual changes in the laws/regulations. Sports betting (increasingly legal in the US) is already fairly structurally similar to the sorts or questions that resolve in yes/no fashion on a prediction market.
I'm not sure I agree that you can't disrupt existing incumbents.
I very much agree that doing this more internationally would be a good idea. I proposed months ago that prediction markets, if they truly want to be accurate, need to start aggressively recruiting in other languages and outside of fairly narrow demographics. But that got a "nice idea but..." treatment. My first thought was literally "move to Macau or Ireland or something." But I don't think I could even get them to move from Berkeley to San Jose.
Betfair [exchange] is a prediction market.
"Has anyone considered getting all that rationalist money behind a prediction market in an already high demand area like sports?"
The industry is salivating at this sentence. "Look, a whole new cohort of people who think that because they're smart they can beat us at our own game!" Or, for the stock market:
https://xkcd.com/1570/
(Many of you even know what I just linked to without looking.)
That is my point though.
If you believe prediction markets are important you want resources to be allocated to develop them. The resources necessary to figure out how they work, build the technology, and lobby the politicians. If there's a big pot of resources in sports then it's better that you outcompete incumbents and use that to develop what you think is world-historically important. Especially because if you believe in the morality of prediction markets you probably believe in the morality of sports prediction markets. (Also, I would bet that a bunch of SF nerds can build a better DraftKings.)
That recommendation sounds like "has anyone considered getting EA behind a political campaign?" and as we saw, Carrick Flynn was the wrong candidate in the wrong area to back.
Take betting on this year's World Cup: France, the holders of the cup from the last competition, vs Argentina, and it looked like Argentina were going to win for most of the game. Then France equalised with two goals within two minutes. Played extra time, both teams scored. Went to penalties and the French made a dog's dinner of that so Argentina won.
Even good forecasters would still be uncertain as to who the winner would be, even at the point of knowing that both France and Argentina had made it to the final. Going on past performance would be little help, as both of them had won the cup twice before. Sports betting is tough, tougher than it looks, and getting into that market means you have to believe you can do better than the bookies. A lot of people have gone bust thinking they could do that.
The difference is that while I doubt EA has many successful politicians I'm 99% certain that EA has a lot of successful startup people and math nerds. It's actually in their area of expertise.
Firstly, great post!
Many of these examples are political questions, which you would always expect the masses to be right about in aggregate, since the democratic process is about aggregating opinions. Some of them are less political. But what about a question like "will it rain tomorrow in New York?" To make a bet on this question, the only sensible strategy would be to check what weather modellers are saying and bet with them. But because this is the only winning strategy, would anyone actually make a bet here? I guess the thrust of my question is - shouldn't there be very little money to be made on problems that are genuinely hard?
I think the phrase "weather modellers" is doing a lot of the work here. If there's one universally acknowledged best weather modeller, then this reduces to the Nate Silver example in 2, and yes, everyone should agree.
If there are many modellers and nobody knows who's best, then this reduces to the competing experts example in 3.5, and a prediction market will aggregate them efficiently.
If you're an amateur weather modeller yourself, and think you're the best, then you should consider betting on your own opinion, as in 3.4, and making money if you're right.
Even in the Nate Silver case not everyone should agree. Supposing he is the best modeler doesn't mean that finding out what another modeler thinks shouldn't shift your probability. That's basically private information like knowing some Dem canidate for the house has a dark secret no one else knows about (should shift your probay slightly).
Also, let's assume Nate Silver is the best predictor and people follow him more or less blindly in prediction markets. Now he can (or could, with enough scale) make a lot of money by making a faked prediction and secretly betting against it.
Theoretically doable under some circumstances, but note Nate's argument for why he doesn't bother betting the model line in existing markets: the reputational returns on making the model as accurate as possible far outstrip what he personally could make by betting.
Currently, for certain, but that's because of lack of scale. Once enough money is involved, people (not everyone, but some) _will_ start to misbehave. Betting and gambling aren't exactly clean businesses.
In general, sure. I expect misbehavior to increase as the stakes get larger. In the case of a known best predictor though, it's not dissimilar from being willing to fudge the model in exchange for a bribe. Once enough money is involved.... give Nate a raise!
Okay, that's a good point, maybe there are a few months of figuring out which weather modeller is best. But after that, won't everyone just figure out which it is and bet with them? Weather modelling is very computationally intensive so I doubt amateurs could beat the established ones.
It won't quite work like that. It's not only the best forecast adds value. NOAA doesn't just use whichever of the us/eu hurricane models is the most accurate but instead blends them.
Think of it like this. Suppose someone flips 200 coins and someone sees 100 of them and I see ten (say choosen randomly so they may or may not overlap). The person who has seen 100 coins announces their best estimate of the total number of heads is 110 (because they saw 60 heads). Now suppose that you only saw 3 heads out of your 10.
Even though the person who saw 100 coins is the best predictor of the total number of heads you should still bet that the number of heads will be slightly less than 110 because you also have a bit of information that's not completely correlated with theirs. Since you have their info and yours you can do even better than they can.
Same with weather forecasting. Now you could imagine one day we get a weather model that includes every piece of data and technique we have justification to use but currently that's not the case. Same way that NOAA doesn't just use whichever of the us/eu hurricane models is the most accurate but instead blend them.
Completely parenthetical to your main point (which is interesting and seems right to me), I'm finding it a little doubty that all the hurricane modelers don't have access to the same data. Why would that be the case? In a field like that, you'd expect data to be widely shared -- certainly that would the case for any data gathered by a goverment or government agency. If true, that would suggest the difference between hurricane modeling predictions reflects differences in the models, not so much (or at all) differences in the input data.
Hmm, ok, so the market will represent a weighted average of the "experts". Two questions then come to mind:
1) will there be enough people profiting from these bets that they produce a good weighting?
2) are there any situations at all where a weighted average of slightly wrong predictions does not produce an accurate prediction?
If literally everyone agrees the weather is best predicted by weather modelers then you don't need the prediction market in that case. If there is disagreement on the probability then there is incentive to invest.
Let me add that in many markets (eg politics) ppl will have extra information (I know that this canidate has a secret or that she sometimes get tongue tied) and the people with that information have an incentive to enter the market.
This is an interesting point. So maybe prediction markets will naturally fall out of favor for certain types of prediction. But that does bring up the question of trust in institutions - how do skeptical people predict out the weather if they don't trust the modellers, and there's also no prediction market for it?
No reason to abolish the prediction market in a formal sense, just let people play within a point or two of the modellers' predictions with their lunch money. If there ever is a big miss, money can flow back in as useful and necessary.
Not suggesting abolishing the market - I'm suggesting that people will naturally stop betting on predictions like this, since beating the modellers is very hard.
I know very little about prediction markets, but one big obstacle to putting any serious money into them seems to be the timeframes involved. Who wants to tie up their money into predictions that would resolve in ten years? Or even five years?
If I was *very* confident in a prediction that would be resolved next year, then yeah, maybe I'd accept freezing my money for that year. But there's no way I'd ever try to freeze up any significant amount of money for an absolutely whopping *five years*, even if the market seemed comically mispriced. And in practice, any mispricing you'd otherwise want to correct would probably be in the single digit percentages anyway, so freezing money for that long simply can't be reasonable. The current inflation rate is what, 6% per year?
So I'd expect prediction markets to be fine at short-term questions, but pretty bad at long-term questions, because inflation makes sure that nobody actually has a real incentive to try to correct a 10% mispricing five years out from the resolution time. Am I missing something?
Thanks. This is true and I've added a section on it as 4.9
I didn't see you mention that prediction markets could easily simply peg investments to the stock market. The market can take your deposit and simply invest it in the market (so instead of a dollar contract you buy a dollar worth of stock contract).
More generally, this is really no different than the issue of leverage in the market. You don't need to actually deposit a dollar to buy a dollar contract, only produce sufficent collateral (and pay a fee) with a margin call happening if your collateral value drops too far.
Which the stock market? *I* might be fine with a S&P 500 index, but for anything you pick you've necessarily excluded the predictors who don't want anything to do with that investment.
Umm, I guess but I dont see how its any different than denomination in USD or any other good. That exposes you to fluctations in currency value. All goods can change value in relation to others.
It's different in that USD is a better medium of exchange than index shares, currency is a special kind of good optimized for exactly that purpose. If we want low friction it makes sense to use a widely-accepted stable currency, but in doing so you inherently limit a position's ability to hold value.
But you need to distinguish between the use of money as a store of value and a medium of exchange. The exchange still pays you in dollars it just invests them in the market when they are deposited so the payouts incorporate any changes in the stock market.
I've heard of a thing where you can chain prediction markets, so there's a market that pays out in 2030 based on the outcome you're trying to predict (climate change or whatever), then another one that pays out in 2029 based on what the current price of that 2030 market is in 2029, then one that pays out in 2028 based on the price of the 2029 one, and so on. So right now you bet on the one that pays out in 2023 and only tie up your money for a year or less. Similar to Scott's example about Biden pledging to move prediction markets.
Isn't that just regular prediction market? I read something similar as well, and I think this was the point the author was trying to make.
I think the idea is that you need to able to sell your prediction. For instance, you buy a prediction bet 30 years in advance. If the market is mature, you should be able to sell this prediction at a (likely small) profit in six months assuming your bet now seems more likely to happen in 30 years, where someone else is okay with the longer horizon.
You can already do this with long-term bonds.
Just denominate your prediction market in terms of shares in an index fund (or see longer answer to Scott about collateral).
If you expect the mispricing to be corrected sooner than its resolution date you could profit by selling it early.
The prediction market could fight inflation by investing the money in TIPS.
Is there a risk of rich people using prediction markets not just to make money but to influence real-world events?
Suppose the tobacco lobby wants the tobacco-friendly candidate to win the election. They bet lots of money on the other candidate and make the market imbalanced. Lots of ordinary people notice the imbalance and buy shares on the tobacco-friendly candidate. Come election day, the vote is looking close-run, but there are lots of ordinary people who own shares on the tobacco guy, which they probably paid between 10 and 50 cents for, so if he wins, they'll get a 2x to 10x payout, and if he loses, they'll lose their investment. That would be enough to make several people who were undecided about who to vote for, or who might not have bothered to vote at all, vote for the tobacco guy - which may well swing the election.
This is more than just the tobacco lobby hedging against an outcome they wouldn't like; this is them paying to influence the outcome of an election, effectively buying votes.
How is this different than the tobacco company selling shares? If you are a shareholder in the company you lose money if the anti-tobacco company wins.
Indeed, this scheme is strictly worse than selling options at a slightly discounted price to buy tobacvo stock at the expected price it will have if the anti-tobacco canidate losses. And this is legal (or at least not enforced).
Also I don't think it's a real concern. In a liquid market, you'll expect that these instruments won't mostly be held evenly by voters but by some big investers. Better to spend money buying ads which get to voters generally.
For this to matter prediction markets would have to become so ubiquitous and used by everyone that I wouldn't worry. Even if prediction markets grew a lot, it would be hard to find the free $100 bills to the average person because very few fast actors would grab it before they can act.
Also, I believe candidates usually want to be seen as winning before the election, so there might be a bias to vote for the winner that pushes against this.
No individual incentive. The odds of a single vote deciding the election are very small, and secret ballots mean there's no way to conspire because there's no way to detect defectors.
I don't understand how your comment follows from my comment. What I described could work with a single individual or group unbalancing the market (using money, not votes), and then *lots* of individuals individually incentivised to vote a certain way, no conspiring required.
But my point is that there isn't an individual incentive, because it's very unlikely that any one market bettor's vote will decide the outcome. (This is why "rational ignorance" and "rational irrationality".)
There is an individual incentive for individuals to *make money* in the market. Not to decide the outcome of an election.
"For whatever reason, most people interested in prediction markets are American, so Polymarket has a limited userbase."
This is by far the weakest part of an otherwise great post. If prediction markets are legal elsewhere and are in theory so great, then why aren't they big there already? Is Polymarket really freely available in the rest of the first world?
There are many great products that failed because they were too ahead of the times.
For Polymarket in particular though, the crypto part is a big wall.
Ah, so it's crypto-exclusive? Then, fair point, I wouldn't expect mainstream adoption until a credit card is sufficient. But also amusing that Scott didn't think this important enough to mention.
But if Scott is wrong that only Americans are interested in prediction markets and prediction markets are legal elsewhere, shouldn't there be successful prediction markets that don't use crypto?
(I also don't understand why only Americans would be interested in prediction markets. Don't other countries have all sorts of gambling and financial markets?)
Well, I guess if every country was equally good at finding business ideas and executing them, US wouldn't be the richest (large) country in world.
I'm from a non-US first world country, and searching "prediction market" in my language only returns some media articles and business reports. Looks like there isn't even a wiki page for it. Perhaps ideas just don't spread as fast as it's theoretically possible? After all, the idea for prediction markets themselves floated around for decades before the recent "boom".
I think your argument that prediction markets are hard to influence needs some more qualification. In particular, it's not that special interests can't manipulate the market. It's that they can't manipulate it in a way that causes it to diverge from the actual probability of that outcome.
For instance, suppose prediction markets get widly adopted and trusted and there is a prediction market for whether there will be a run on the banks in X country this year. If someone wants the banks to fail they could start buying up contracts that say the banks will fail. As that changes the price in the market which in turn makes it more likely the banks will fail it's not true that the market resists manipulation (even if you know what's going on and have a ton of capital it's not clear you don't also bet on failure).
This is important as it creates important limitations for the use of prediction markets in making decisions. If you do things like couple voting or government policy to prediction markets it may be that the markets accurately reflect probabilities but they may bring about undesirable outcomes in doing so (eg bank run example).
When I hear about stuff like this I always get hung up on the biases of the arbiters. Obviously you can add more and more specific resolution criteria, but I would worry about a right wing market deciding that the answer to "will trump win in 2020" was yes, because the person in charge of the market believes the election was manipulated (or even just refunding it all rather than call a no). Or something like "will we be in a recession Dec 2022" being refunded only by some markets because one market determiner beleives the government has vested interests in not announcing one even though there's obviously one happening right now. Just disagreements on judgment calls, when to refund, and eventually you could get two markets, but left-market says candidate has 30% chance to win, right-market says candidate has 40% chance, and there's no free money to be made because there's a 10% chance that one market or the other disagrees on the state of reality by the election. At that point they lose the benefit of being authoritative.
And again, I know you can narrow it down much further than I did here in my examples but especially in cases of when to refund, it seems like there are too many outside influences to properly phrase all of them. Who are "experts"? Who are "developers"? what is "commonly understood"? What does "interference" mean? Does a tweet count as an announcement or does it need to be at a press conference? Does covid count as a "disaster" causing a market to refund and if so, at what point does it stop counting as a disaster?
That being said, a bunch of question styles dodge this problem pretty easily, the "what would I decide/what would justice decide" style questions would be safe.
"As a test, I tried to manipulate the market on whether Austin Chen, founder of Manifold Markets, would be charged with a felony. There’s no reason to think he should be, so the price started at 5%. I spent $200 in Manifold’s play money bidding it up to 95%. Within an hour, other investors noticed the mispricing and corrected it back down to 5% again."
How did the other traders know that you were manipulating the market, instead of trading on inside information? Realistically they didn't, they were just trading on their priors. That worked out in this case, but if Austen really had been about to be charged, there wouldn't have been any way to find out about it from the prediction market faster than from the news reports.
In general there's some tension between the market's ability to resist manipulation and the market's ability to respond to inside information, since they look the same to other traders. I'd expect this to equilibriate in different ways on different questions, depending on how plausible inside information was: you probably can't manipulate the "climate change in 2030?" question, but probably can manipulate the "Biden to resign in 2023?" question.
EDIT: this affects the hedging analysis too, come to think. We should expect a position taken as a hedge to move a functional prediction market, because the other participants can't know that it's a hedge and not the product of private research.
I think that worked out on personal knowledge ("I know Austin, he's not that kind of guy!") but for other manipulations, I don't see why someone would not assume "Hey, somebody just bet that Joe Biden would be arrested for dog-fighting. Either they're crazy, or they know something we don't, maybe they do have inside information, maybe I should buy on Joe to be arrested" where there isn't personal or inside information to judge "Nah, this is a trick".
I think the more likely, or smarter, option is to just not bet against what should be obviously crazy stuff. "Joe Biden arrested for ________" seems obviously crazy, to the point that nobody sane would bet on it being true. If someone put a lot of money down on that option, I would avoid it, even given that it's *probably* free money.
Well, suppose three years back somebody said to you "I propose a bet that an appointee of the Biden administration will be arrested on two charges of stealing women's luggage from airports" would you have said "That's crazy, and I'm just taking your money if I take that bet" or not?
There's a lot of things that sound "No, that can't possibly happen" and then they do.
If I saw a very specific bet about an independently implausible scenario (like, specifics including number of charges and details of charges, rather than "Biden admin appointee arrested") that's a red flag for insider knowledge. If it was three years ago and not so specific as to set off red flags, the judgement without hindsight would say take the free money. That's tied into one of my concerns with prediction markets - low chance predictions are still non-zero, and once in a while some 99% bet will fail and people will update with the absolute opposite of what they should.
> I think that worked out on personal knowledge...
Sure, and that's totally reasonable, but if you already feel like you know the answer too firmly to get pushed around by illegible price signals, why have a prediction market at all? Just stick with your prior and live your life.
An important feature of prediction markets is that their prices ARE NOT probabilities. I think you are mostly careful to use words like "likelihood" and I appreciate the need to keep it simple, but there are several real world effects that distort market prices relatively to probabilities quite a bit and it is useful to keep them in mind.
1. Time value of money - one would rather have money now rather than later. This is measured by the risk-free rate of interest, which is now around 5% in USD. Rather than bet $1 to get $1 in 1 year time on a prediction market, one could invest $1 in a portfolio of very safe 1 year bonds and get $1.05 in one year.
2. Credit risk of the exchange. Betting exchanges go bankrupt, are hacked, etc. This is not a theoretical possibility - 10-15 years ago there used to be a decent prediction market called intrade.com. I made there roughly x8 of my deposit on a series of sports and political bets, but this amount was not high enough to bother moving it in and out of exchange when I was not trading, so I lost 100% of it when intrade.com folded. Rather than invest $1 in a portfolio of high quality bonds, one could lend money to the betting market, or a bunch of equally dodgy blockchain companies. I do not think that the interest rate on such loans (illiquid, unsecured, to non-transparent companies of unclear regulatory status) would be lower than 15% in the current market.
3. Transaction costs. There is a non-trivial transaction cost, both mental, time spent, and commission based required to deal with many of these markets. I think putting it at 2% would be conservative for relatively small amounts bet.
These three effects create no-trade boundaries for a rational bettor. I can think that my predicted probability of an event Q is higher than market price P but I would not buy unless the difference is high enough to compensate me for the effects of 1-3. And I would not sell unless Q was sufficiently below P. These boundaries become quite wide for large time to resolution, but they are non trivial even for relatively short times. For example, using the parameters as above for a two year market, one gets no-trade boundaries of (-0.28, 0.09) for market price of 0.05, and (-0.02, 0.36) for market price of 0.25. Basically, it means that for market price of an event up to 0.25 no rational player would be selling at that price even if their subjective probability assessment was exactly 0%. So any market longer than a few months with rational traders is pretty useless for assessing low/high probability events. Even for a toss-up market price of 0.50, the no-trade region is pretty wide at (0.31,0.69).
Almost all of his arguments assume a large stock market style exchange. At that scale I think these concerns disappear. Without it you can basically write off the prediction market as pretty useless.
This is not about being "large" but about a range of measures to lower credit and funding risks. For modern exchanges this involves a lot of complex things, like risk-based margining and cross-margining, multi-tier broker access system, a complex waterfall of liability in case of member default, etc. Also, these things work because most members have direct or one step removed access to central bank liquidity. Even then, it is not obvious this system works very well for multi-decade bets as the recent UK gilt/LDI meltdown has demonstrated.
If the arguments are applicable only to these kind of exchanges, then the prediction markets to which the arguments would apply do not exist and are unlikely to exist for quite a while :)
100% agree that those prediction markets don't yet exist and will not for awhile. Right now the markets tell you the probability that prediction market nerds think is right after a limited amount of effort with plenty of bias.
So yah, until Goldman can make billion dollar moves in the market it's not likely to be that accurate or useful. At that point all the usual financial services stuff comes in and while you can't eliminate risk of exchange collapse if the NYSE went down it's fair to assume that so too would the value of all your investments.
> 4.6.1: Then why should anyone play prediction markets, when on average they’ll only break even? It seems like this is a worse deal than stocks, which tend to go up over time.
You suggest subsidies, but do we even need that? Can't the market makers just put the money in the bank and return the money along with interest?
> 5.2: Politician pledge markets
Not a bad idea, but I don't think this will actually happen. Politicians can already do this with regular market. For example, a politician who wants to build solar energy system can promise to push up the price of some solar energy index fund, but I don't remember anyone doing that.
Also, regarding the "meme stocks", another point to be made is that betting against the bubble is much easier since the downside is limited, unlike shorting stocks (as long as we're talking about simple binary outcome prediction markets).
The stock market in general can't be manipulated, but then you have "madness of crowds" behavior like speculative bubbles that eventually crash etc. In principle you can "get rich quick" by shorting the bubble, but if it's you against the majority of investors, you probably don't have enough capital to budge the price. Or, in other words - your idealized argument would imply that the crypto bubble or the 2008 housing bubble shouldn't have existed, which is clearly wrong.
So I'm a huge fan of legalizing prediction markets but I think it's worth considering a serious downside: it effectively requires legalizing manipulation (ie letting people with positions in the market move it via their pronouncements).
We *want* people like Nate Silver to publish his model and generally for experts to offer their expertise to the public. But if we force them to choose between this and investing in the market we create a strong incentive not to share information publicly which could result in less good predictions. Besides, such limits would likely be unconstitutional unlike as to stocks.
I don't really think this is such a bad thing. Norms about conflicts of interest seem like they can handle it and it's no more of a danger than scakmy reverse mortgages. And I'm not convinced it gives experts any more conflicts than they have now (indeed it let's them bet on what they really believe).
But it's worth pointing out.
That’s an interesting point. Right now, Silver’s logic is very transparent and we can inspect it and learn from it. If he were instead using that knowledge to bet on a market, he’d make the results public (via the market) but would be incentivized to keep his underlying reasoning entirely secret. That seems like a very large loss to humanity if that’s how everything gets done.
i like prediction markets, but i'm a lot less bullish on their potential to make broad social impact
prediction markets are nerd shit, and a lot of normies hate nerd shit
OR, you have a lot of normies who like nerd shit precisely when it happens to tell them what they want to hear, and say "Well the nerds can't know everything!" or "Remember that time it said 85% chance and the thing didn't happen? That just proves how inaccurate they are!" People will do this roughly to their hearts' content. Large portions of the population believe in homeopathy, intelligent design, fat burners, Qanon, etc, and many of those people aren't even dumb or uneducated. It costs them to believe in wrong things (in not so different a way it costs you to bet wrong in a market), but their need/desire to believe what they want to believe outweighs the costs they suffer, apparently.
Altogether, normies will sometimes listen to nerd shit, like weather probabilities or an economic analysis of the likely cost of some proposed bill or the stock market or the scientific consensus on global warming, and they will sometimes adjust their beliefs or behaviors accordingly, but they often won't. They may continue voting, acting, spending, talking in ways that are wholly at odds with the realities that prediction markets reveal, because they either can't wrap their heads around what this nerd shit actually means (esp if they have poor understanding of stats/probability, or distrust prediction markets because they don't have enough understanding of economics to see why they work), or because they'd rather spite the obnoxious nerds than make sensible decisions in their own lives, or because they would rather hold on to some comforting belief than make the ostensibly rational decision (i mean, the goal of life is to have fun anyways, not to make decisions that can be justified as "rational", so in some sense it's rational to do things that irritate rationalists because it's funny)
So not to say it won't make any dent on how normies think talk and act, but I think it might be a pretty modest dent. Prediction markets could be gods in the nerd world, but amongst those with actual power, they may be chihauhaus
Prediction markets won't change normie beliefs to a significant extent - the real goal is to improve the quality of nerd beliefs. Robin Hanson's original article was titled, "Could Gambling Save Science?,"
https://mason.gmu.edu/~rhanson/gamble.html
It includes citations to the well documented failures of peer review as of 1990. These failures have become even more widely documented since then. So the first priority is to hold academics and intellectuals to a higher standard.
Before we can make progress on improving the signal to noise ratio in public discourse, we first need to develop a more reliable approach to identifying a signal to begin with.
People who believe that academic publishing and reputation systems are even remotely effective may not see much of a need for reputational bets and prediction markets.
true - i just think that if it doesn't make much dent amongst normies, it can only have limited potential for large scale social impact. It will have _some_ impact, because improving science and other nerd activities is good for a variety of reasons, but there will be many big areas that are resistant to these improvements, i think
While the theory sounds nice, I fear that once translated into real world and widespread use, then when presented with "either the market is.... or get rich", people will go with 'get rich'.
Nobody buys Amazon shares because they think it's a great company and is properly priced, they buy shares because they hope the price will go up, and they will get a return on them. (It seems that "shares pay a steady dividend and prices don't go up or down very much" is no longer the way people make money).
Why expect a prediction market to be different? "Yes, we all agree that the Democrats will win the next election, the shares are priced correctly at 90 cents" and then nobody buys or sells shares and your market is stuck.
'Prediction markets all speak with one voice' - then why have three or six or ten markets, why not let one giant market eat them all up?
When it's a question of "did the Democrats win the election, yes or no", that can be resolved (unless somebody takes a court case about no they didn't). But when it's something more fuzzy, who makes the final resolution that yes, Messi is the greatest player of all time? You need somebody to do it, and that may well be an expert. Or there will be arguments over resolving the question because one set of investors will say you should measure it this way, another set say no, measure it that way.
If prediction markets take off, I don't expect them to be much different from the stock market or online bookies. Fine if anyone wants a bit of fun, or to try and make money. But God preserve us from mayors to presidents deciding to set public policy on the results of a market buying shares on "should we drain the Mississippi?"
"7.6: …if I’m the head of the Commodity Futures Trading Commission?
Please legalize real-money prediction markets in the United States. If it helps, here are a bunch of really famous economists including a Nobel laureate explaining why you should do this."
The CFTC might just be the least bit busy with something else going on right now about another new, ground-breaking, future of finance project:
https://www.cftc.gov/PressRoom/PressReleases/8638-22
(If that seems a little unfair, remember that FTX et al. came out of the rationalist/EA aligned communities, not just some common-or-garden Ponzi scheme. "But all the rationalists think this would work beautifully" is not necessarily going to endear prediction markets to the CFTC).
> If you really wanted, you might be able to make it work in theory through a mechanism sort of like this one.
That post ends by noting that Robin Hanson explained why it wouldn't work.
https://www.lesswrong.com/posts/dLzZWNGD23zqNLvt3/the-apocalypse-bet?commentId=y9hmdcJQYzPCSbmJM
I don't see one particular issue addressed: prediction markets can't make accurate predictions, but only aggregate the (market's) population's opinions.
When the Brexit vote was to be decided on Thursday, I discovered on Wednesday that, if the vote was to leave instead of remain, the stock markets would fall. I had decided the vote would be to leave, for non-financial reasons, so decided to bet the market would fall. That bet made me a small amount of money, but not as much as I lost elsewhere in the market. Of course, back then, I had never heard of prediction markets at all, anyway.
But prediction markets disagreed with polls. This site, from 6/16/2016 (one week before the referendum), says the markets were 60/40 to remain, but polls said 57/43 to leave. https://www.theparliamentmagazine.eu/news/article/brexit-betting-market-predicts-clear-win-for-remain-camp
But THAT market was two days before the vote. Another site, with no date I could find, says the prediction market turned accurate only hours before the vote. https://www.cam.ac.uk/research/news/gamblers-predicted-brexit-before-financial-traders-study-finds
And this site confirms that the betting markets changed wildly on the day OF the referendum: https://www.vox.com/2016/6/23/12022436/brexit-odds-of-a-british-exit-are-surging-on-betting-markets
The markets were different from the population because, well, the pool of people betting was different from the pool of people voting.
The market can only make accurate predictions if the participants have full knowledge of all relevant facts relating to a particular bet. So the predictions only produce aggregate opinions.
I'm grateful for the article, and now think prediction markets are a good tool for assessing broad opinions: people put their money where their mouth is.
I don't think that the answer for 4.6 quite addresses that concern. This question is really "What happens if people don't care about getting rich?"
While "you" could get rich correcting a mispriced market with too many hedgers, there may not be enough market participants who care about getting rich. A market participant who is hedging is buying insurance, which is inherently about reducing the variance of outcomes at the explicit cost of worsening the average or expected outcome. They are risk minimizers not reward maximizers. If a market is full of hedgers, there is no "you" (or not enough "you"s) interested in getting rich. In that case, the entire mechanism behind the accuracy of prediction markets unravels; too many of the participants are unmoved by the incentive of getting rich by correcting mispricings.
This is not a theoretical problem. There have been many instances in commodity future markets where prices have gotten extremely far from plausible future probabilities because of this dynamic. However, in most cases, it usually doesn't take much profit maximizing money (aka those dastardly "speculators" and "hedge funds") to correct market mispricings. It just shouldn't be taken for granted; it doesn't happen 100% of the time.
Right, one issue I see is that sometimes one side has much more to lose if an event happens than the other. A prediction market for snowfall at a ski resort will probably have a premium for the "insurance" side of the deal and thus will likely predict too low unless there's enough interest by people unaffiliated with the resort to drown out the resort's impact. Risk-aversion in general makes prediction markets inherently more pessimistic than reality.
The caveat that always comes to mind for me when thinking about prediction markets is Goodhart's Law: https://en.wikipedia.org/wiki/Goodhart%27s_law . I didn't see this objection in Section 4.
As prediction markets become bigger, is there a risk that they go from neutral observers to an active incentive to bring about change in the outcome they're trying to predict? Is that a problem for the market concept? Does it change the conditions under which it's "ethical" to bet on one side or another?
A lot of this sounds like the Efficient Market Hypothesis, which is probably true in its weak form and blatantly untrue in its strong form. Essentially, the only way for someone to make money is by research, so someone actually has to *make* said research, but the hypothesis assumes that this has *already* happened.
Also, you can only reproduce the best *public* expert - once prediction markets start to be about real markets, surely you cannot be sure to reproduce the best expert any longer, if said expert merely bets on the markets without advertising or publishing?
And I still don't understand why it's impressive that a prediction market can reproduce the best expert by imitating his or her conclusions. If it just imitates Nate Silver, just go with what Nate Silver says instead - the prediction market is a weird roundabout.
Further still - let's say the prediction market merely copies Nate Silver. But now Nate says something that seems wrong to you. Now people either have to decide "no, we already established that copying Nate Silver is how things are done", OR they have to decide - at the risk of personal biases and mistakes - that Nate is wrong this time, but if you do, you can no longer be sure the market will be at least as good as Nate Silver. So either you merely reproduce the best expert, or you're at significant risk of not performing as well as the best expert. Of course, if Nate Silver *was* wrong this time and people manage to catch it, that's a good thing, but the opposite seems fairly likely as well, and you don't know in advance which it was you just did.
Some additional questions to consider, or objections to think about.
1) Some questions have very little real information available to the general public. Sometimes the information available is wrong - maybe nobody knows the real information, or maybe the information is only available to certain insiders. We would expect all markets on this question to be wrong in the first case, or weirdly distorted in the second. For instance, if the publicly available but wrong information leads the general public to say 20%, but insiders think 80%, the market may converge on any number between them (assuming the insiders don't have infinite money to bet against the public, while the public thinks they are taking free money from the insiders, so they will want to keep pushing it to 20%).
2) We should expect every mature market to be distorted by an amount exactly equal to a combination of any costs (fees and taxes) of making the bet *and* the opportunity cost of not making another investment. You already noted that long term bets have this problem, but I think undersold the fees and even shorter term opportunity costs. At the very least, a badly mispriced market may stay moderately mispriced even after it resolves. I seem to recall some election markets resolving with lots of "free money" sitting around, because it was too expensive to pick up the amounts remaining.
3) The only people who buy stocks are people with disposable income. This problem will persist with prediction markets. You seem to be under the impression that poor people will bet in smaller amounts but in larger numbers. The reality is probably much closer to no bets at all for the bottom XX% of the population, and very minimal and haphazard bets for the next XX%. This will price in an "elite consensus" factor into all markets. It's like getting your political news from Twitter (at least before Musk, I don't know if it's different now) - it may be accurate often, but it's also heavily biased towards certain views. In a toy example with 10 possible predictors, if only three of them are able to invest and they all agree with each other, that's missing something big. This would be important for a whole series of questions related to the lives of poorer people, such as "the effect of UBI will be positive" or similar.
4) Even a 90% prediction based on good metrics will fail 10% of the time (and that assumes the underlying understanding/assumptions were correct). This is a bad failure mode for someone trying to make money and ties in heavily with #3 above. If I don't have $1,000 to lose "correcting" a market, then I don't bet. Also, see the next item.
5) One-off events are very hard to evaluate, even after the fact. The markets were wrong about various aspects of the Russia-Ukraine war. Other predictors were also wrong (which also ties in with #3 and #1 above - limited correct knowledge available generally, not enough liquidity among those who might predict better). If we were trying to use a prediction market to determine if/when various events would occur in that war, we would be wrong quite often.
6) Predictions about the actions of individuals or small groups (will Putin invade Ukraine?) is very different from predictions about aggregations of large groups (will the Democrats win?). We should expect markets about aggregates to do much better than markets about individuals.
Scott you are actually wrong about if hedging distorts markets. I work in finance. Let me explain. Why does stock market go up on average? Because people don't want the stock market to go down. The economy being bad is correlated to all kinds of other bad things in their lives they would hedge if they could. There are plenty of natural shorters of the market in this world. Very few natural buyers. The expected return of the market is above 0 precisely because there are many people who would be happy to hedge against financial downturns if it was 0 by shorting. Very similarly, if there was a prediction market 'the economy will be worse than expected according to a prediction market' there will be more natural sellers willing to hedge and the market will remain systematically pessimistic being worse than expected. Because if it wasn't, and my cost of hedging is 0%, everyone who currently loses say 7% on average shorting the stock market would short this question instead, until it's expected loss equaled expected loss of shorting the market.
But, long-term, don't companies generally make money, and thus become more valuable? Are stocks not ownership of the companies, which then would also be more valuable?
Certainly, in the short-term, anything goes, as it relates to how people think the company will do long-term, or even short-term. Speculation is a zero-sum game. Investing is not.
Not quite, because people could make an unbiased estimate of how much companies will make in the future, discount that to present at the risk free rate, value the company at that amount, and still have a 0% expected return. The reason the stock market goes up over time faster than the risk free rate is because of risk premia, and the same dynamic would exist for prediction markets also used for hedging purposes.
Actually, in the long run most companies eventually go broke. The median stock gives worse returns than 1-month Treasury bills in the long run, and the entire gain in value in the US stock market since 1926 came from just 4% of the total stocks. It's like they say about venture capital investing: most startups will fail and you'll lose money if you invest in them, but the one startup that becomes a billion dollar company more than makes up for the large number of losing bets.
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2900447
After reading (some of) the paper, I conclude this is mostly the same effect of 50% of businesses failing after five years (https://payrofinance.com/50-of-small-businesses-fail-within-5-years-heres-why/). I can't tell from the data how much of this was dot-com startups at the turn of the century, for example.
Venture capitalists can afford to back lots of companies and have most of them fail, if they get some winners. But indiscriminate backing isn't really a good strategy. Shark Tank, for example, backs things they think will make money (and possibly some for PR reasons, which is an investment in their own brand). Venture capitalists serve a capitalist function in only backing ventures they think likely to pan out well. Venture capitalists that throw money at every venture without using good judgement, hoping for at least one of them to be a big win, can be replaced by a robot throwing darts at a page of stock listings, kind of like a mindless rock (https://astralcodexten.substack.com/p/heuristics-that-almost-always-work).
Individual investors shouldn't put money into things they "heard are an up-and-coming company" either, but should determine for themselves whether companies they consider in which to invest will be good investments, or should pick mutual funds or such things managed by people who actually do the job of good venture capitalists.
> "Either prediction markets will be accurate, or you can get rich quick”. Unfortunately, it’s almost always the former.
This is a bit of a bait-and-switch. I think prediction markets will very often be inaccurate, but in ways where no one knows how exactly they're inaccurate. You can *technically* get rich quick in that it's theoretically possible to randomly choose one option and get lucky, but in practice you won't know to.
In this way, prediction markets aren't necessarily right at all, only not wrong in exploitable ways, so I have to disagree with referencing them as answers. They're just a sort of cumulative guess. For me, they shouldn't be taken as 'canonical' given their unreliability.
This part got me thinking, in light of the recent AI post: for "When you’re done, tell people whether you thought it was a good use of your time, so the market can resolve" you are training the system to provide content that you will retroactively rate highly, rather than content was in fact a good use of your time, which seems like it could be subtly different -- for example, maybe you have some hot button, and the system will quickly learn to press it. Other systems for allocating your attention almost certainly have this problem too, though, (or other problems) so maybe this is not specific to prediction markets?
It's just Goodhart's Law, I think—when a measure becomes a target, it ceases to be a good measure.
If it satisfies "X thought this was a good use of their time", it satisfies it. But there can definitely be a good amount of distance between this measure and X's true goals/the goals of everyone who wants X to do it/a better world overall/etc.
There is one thing I still don't understand:
Let's say, everything except for the prediction markets looks like event X (Biden resigning this year) is not going to happen.
Then, suddenly, the prediction markets say X is going to happen. What could possibly have happened?
A it was insider trading, and this made the markets more accurate
Or
B someone was very, very sure while being very, very wrong. Surely someone will fix the misspricing.
These seem to be the same situation from the outside — how are the prediction markets supposed to be able to resolve different?
Or is the only way to get everybody bankrupt who is capable of causing B?
These sudden changes have happened a few times on markets I’ve seen in Kalshi actually. Typically if it really does disagree with outside public information someone quickly pushes the price back. If there was a mechanism for commenting maybe the real insider information would affect the price longer.
It's worth mentioning that sports betting, especially on the smaller scale (like some local lower-division soccer club), is *massively* tainted by corruption. Since it's so easy for someone on the team (or even more, in a sport like boxing) to either outright lose of degrade the performance significantly, and since there's a lot of betting, there's a lot of money to be made this way. So in this case, betting a) doesn't make the best estimate, and b) outright taints the sport.
I see no reason why big prediction markets wouldn't replicate this.
This is pretty cool... in fact, you've inspired me to start paying. I tried my hands one year at the GJP, which was an interesting experience, but the time sink was too much for me.
One issue you dodge is the problem of thin trading. e.g., "will Prof. Dunderhead's new theorem in algebraic topology be proved correct?" Maybe ten people in the world can actually have an informed opinion, and it seems unlikely that as many as one of these 10 would choose to spend time playing prediction markets.
For insider trading and illegal activities, I think the worry is more about money laundering than about encouraging illegal activity directly.
That is, if you make a million dollars selling drugs or whatever, you can then take that money, use it to place bets on innocuous things, then cash out your bet as "nothing to see here, officer, just my winnings from being a brilliant predictor." Gambling is heavily regulated to prevent that sort of thing.
I do think you're right that it could be at least as legal as, say, sports betting, but maybe there's some sort of clever money laundering scheme I'm not aware of which works better with arbitrary bets instead of sports gambling.
Could you explain more about the money laundering issue?
Don't prediction market websites record your initial investment? It seems different than an unregulated cash-for-chips-for-cash casino with no record-keeping.
Is it the risk of collusion in fraudulent record-keeping between the website's owner and the money launderer?
I dunno, but you can do small-scale money laundering with things like scratch-off lottery tickets. Use your drug money to buy lots of tickets. You'll lose about half your money, but you can live with that. When the IRS asks you where your money came from, show them the winning tickets and don't mention the losing ones.
Prediction markets are just a generalisation of stock markets. Stock markets answer whether a specific stock value rises. Prediction markets answer all kind of questions. That's all. No more, no less.
I'm skeptical on the utility of conditional prediction markets and prediction markets predicting the outcome of other prediction markets - I expect them to be much more vulnerable to manipulation and goodharting. But in general more acceptance and wider spread of prediction markets would be great and usefull.
I have a question. What prevents a person with financial or political power from gaming a prediction market? For example, Elon Musk plans to sell a large number of shares of Tesla, and he hasn't told anyone. He can have someone he trusts put up a prediction that he will do so, and since no one expects this, there will be a large payout. He can then put a lot of money on it and make a big profit when he sells the shares.
Conditional prediction markets seem like a context where the defenses against manipulation can be especially weak. In the vaccine choice example, there's presumably a major financial decision riding on which vaccine is chosen - let's say a $1bn contract that goes to pharma company A or B. We can also posit that the consensus on efficacy of these vaccines is pretty close - otherwise why bother seeking the wisdom of a prediction market?
So all pharma company A has to do to win the contract is make their vaccine's market trade slightly more favorably than B's, which should only require a marginal nudge. Meanwhile, it's far from obvious that there would be much neutral money in such a niche market. Not many people are vaccine experts, and in a situation like this where there's no glaring arbitrage opportunity, who's going to come in to correct the market by a few percentage points? Goldman Sachs might identify a slight mispricing and trade on it, but prudent risk management would keep the size of the trade fairly small, whereas pharma company A is incentivized to throw up to $1bn into the market. I would have said it's hard to imagine an army of Redditors descending on the market - these days I'm not so sure! - but to be clear there doesn't seem to be a "get rich quick" motive in correcting what might probablistically be a very slight mispricing in a niche contract.
One could argue that insider trading bans solve this, but there are situations much less blatant than the vaccine example where the party has a less direct financial interest. Also, a lot of corporate behavior in today's world is not totally above board, so it's not a huge stretch to imagine the laws being broken or bent.
I think conditional prediction markets will be most useful when the market has a large natural audience and attracts a lot of capital relative to any decisions informed by it.
Prediction markets are great, but it's not rational to expect prediction market probabilities to accurately reflect risk-neutral real world probabilities, particularly when the event in question has economic significance. There will be distortions and risk premia like there are elsewhere in finance.
In section 4.6, you ask "If people use prediction markets to hedge risk, won’t that distort them?" and then say no because other traders will swoop in to correct the price. This is not at all what we see in traditional finance, nor does it really make sense from first principles. Rational traders (who all else being equal, prefer less risk to more) will only trade to either hedge a risk that they already have, or if they believe the expected profit on the trade justifies the volatility. If the risk neutral value of a contract is 25%, there is no incentive to trade it at 25% except to hedge.
Let's suppose there is a contract that we think should trade at 25% if everyone were perfectly risk neutral. Let's say there is hedging demand to sell and less or no hedging demand to buy.
If hedgers want to sell at 25%, and arbitrage traders don't want to buy at 25% (because they don't make any money in expectation), then the market price of that contract must be less than 25% in equilibrium. This sort of thing is one possible cause of "risk premia" in finance. The size of the premium here should mostly depend on the following factors:
1) How badly the hedgers want to hedge, in the aggregate - is the risk they are hedging existential to their business? Is it merely slightly annoying?
2) Amongst the rational traders, how correlated is this risk with other risks the rational traders tend to have on their balance sheets? (The more correlated, the more premium they demand).
3) How much pricing uncertainty is there? Can the rational traders be *really* confident the risk neutral value is 25%? How much potential is there for the hedgers (or really anyone selling the contract) to have inside information?
Prediction markets are still great. But in general, markets price things not only on expected value but also on distributional properties/correlations/intrinsic supply and demand imbalances. This his how proprietary traders and hedge funds and the like make money.
(Source: Was a prop trader, am now at a hedge fund)
Isn't "the prediction market is as good as it gets when it comes to predictions" and "you should participate in the prediction market with a decent chance of making money on average" contradictory? *If* the prediction market is efficient, you won't make any money in it (barring insider trading, but then, only insiders should ever put in money), and if it isn't efficient, then it's not as good as it gets when it comes to predictions (after all, you can guess better)?
Without this, it's just sports betting or gambling - done for entertainment, while a reasonable actor with merely public information should expect to lose money. The way to make money is to either cheat (manipulating the market, insider trading) or to be the one running the market.
I wonder if anyone has seen private prediction markets work well inside a large company? I was at a very large, multi-national Fortune 100 company that briefly experimented with prediction markets in its technology division a little over a decade ago. We ran prediction markets around things like, "Software release <X> will result in fewer than <Y> [really bad things]". (This was an easy statement to resolve unambiguously, I'm deliberately avoiding describing the really bad thing we were predicting.)
Pretty soon, the prediction markets revealed where there was a divergence from the politically expedient view. e.g. Prediction market says software release X will be a disaster, but executive in charge of it says it will be fine. Not long after that, the prediction market got shut down.
Was this a unique situation at this particular company? Is anyone seeing prediction markets used successfully inside large corporations?
Out of interest, was the release a disaster?
I'm, uh, not as sold on prediction markets. I do see a few major problems.
1. Too many markets/volume too low. I recall a market you wish you had: whether a certain outdoor event in a city in California would get rained out on a specific date. It's hard to imagine a) too many people caring about that and b) anyone having any better information on weather patterns than the meteorologists themselves, even if you don't trust them. With potentially thousands of markets, we'll probably see a power law where a handful of questions get all the attention and bets and many smaller markets where only a few people bet at all.
2. What's stopping the oil magnate CEO from putting hundreds of millions into YES on the "Will global temperatures rise 3 degrees by 2050?" market and then polluting so much that we hit the target and he makes a ton of money? Would people betting NO have to put in, collectively, just as much as he put in on YES to get the price back down?
I notice a point that's come up a bunch in the comments is "hm, but prediction markets can only aggregate existing opinion, so they won't truly be *accurate* if *everybody* is wrong". Of course, the response to this is already implicit in the original piece -- no, they won't be truly accurate if everybody is wrong, but they will still be the *most* accurate available predictor, which is all you can hope for. Still, this might be worth making more explicit.
No, they won't be the most accurate available predictor. They will be the most accurate predictor weighted by the willingness/ability of people to put money down on a prediction market bet, which is not necessarily directly correlated with the accuracy of their predictions.
Quite right. If I really know the answer, e.g. because I can cause it to be one thing or the other, or at least very heavily influence it, then by definition I'm an insider trader and most market rules would prohibit my participation for obvious reasons -- the prediction market has in this case just turned into my personal scam machine -- plus I almost certainly have better ways of making money off my known ability to influence events than betting on them.
We should probably look at this side: https://arxiv.org/abs/1907.11162
I'm not per se against prediction markets, but I think there is an over estimation of utility and a question of over estimation of precision. (I have participated in GJP many years ago and something that maybe was a precursor in 90s.)
That Yale/Harvard expert who says 70%. Is not really saying 70%, she is saying "about" 70%. Maybe what she is saying is there is between 50% and 90% change. There is a lot of difference between 50% and 90%.
Market manipulation: ask Nate Silver, a known gambler, to publicly state whether he privately bets on the question that he and his site make public predictions about. Predictions which affect the markets. He ostensibly is a data journalist but his ombudsman refuses to respond to inquiries about his private gambling.
Framing - almost all of these prediction market questions can be framed alternatively. Kahneman and Tversky
Is it really like the stock market or is it sports betting?
> Framing - almost all of these prediction market questions can be framed alternatively. Kahneman and Tversky
I read, and partially understood, Kahneman's book, but I'm not quite sure what this objection is about. I'm trying to find a practical example. For example, if you are writing a prediction market question on a binary referendum outcome, the word 'win/pass' should correspond to X% and 100-X%, while the word 'lose/fail' should correspond to 100-X% and X%. Do you mean that our cognitive biases prevent us from recognizing the inverse equivalence of questions like this?
Or do you have something else about framing in mind?
Basically, cognitive biases will affect predictions depending on how they are framed - even though "this will NOT happen" is the same question as "this will happen".
I understand in theory. But do we have evidence of this with two high-popularity, binary-outcome, real-money markets, on the same website? What I would look for is two such markets that differ only by a negative term, yet mostly sit further than 2% apart from a perfect inverse, i.e. (X%, 100-X%) and (100-X%, X%). I hope I've worded that clearly.
Manifold, with its profusion of similar questions, would be a promising place to find examples. But it isn't real-money, so participants' incentive to take it seriously is lower.
I think instead of conditional prediction markets for P(A|B), markets for P(B) and P(A AND B) would be more flexible.
Suppose you want to bet on "on the condition that the next US president is a member of an ethnic minority (B), they identify as female (A)".
Betting a dollar on P(A|B) is equivalent to bet P(NOT B) dollar on NOT B and then betting the P(B) dollars on P(A AND B):
* If the outcome is any NOT B, P(A|B) will not resolve but return your bet of 1$. Equivalently, your NOT B market will return you the dollar.
* If the outcome is B AND NOT A, you lose everything
* If the outcome is B AND A, your return is P(B)/P(A AND B). This is the same return you would get from betting a dollar on P(A|B) (assuming effective market conditions.)
The advantage of AND-bets over are:
* Markets are always resolved
* It is a simpler basis which covers more ground. With a market for P(A), this would also cover the probability P(B|A).
Disadvantages:
* You need to track changes in P(B) and adjust your investment accordingly. (Unless you assume that P(B) is priced correctly and are risk-neutral or risk-friendly, in which case you would not need to bet on P(NOT B) to cover that possibility).
* Calculating the conditional probability is more complicated.
More generally, if we had N different observables A,B,C, ..., one might set up 2^N different markets predicting the probability of any specific outcome. If you want to bet on B, you would just invest in all the 2^(N-1) markets favoring B and weight each investment by that markets probability.
Of course, nobody could ever agree on the N interesting observables beforehand, and I am unsure if there is anything to be gained by converting bets on B to equivalent bets on A AND B and NOT A and B on the fly.
I greatly appreciate your continuing efforts to be evenhanded and ethical.
(I don't actively trade in prediction markets, but my background is finance.)
1. I was a little surprised that reversed trades didn't get mentioned as a criticism. In the vaccine example, if I take time to investigate vaccine A and trade accordingly, I lose all that effort if the vaccine isn't chosen. One might object that my knowledge could partially map over to vaccine B too, but in general there could be more than just two vaccines and there could be markets where none of the options get selected.
Furthermore, the potential tax implications just rub salt in the wound, as the reversed trades might not be in the same calendar year. Even if you're in the same tax situation both years, that's two sets of trades to report even though they ended up at a zero net payoff for you. (If nothing else, the sheer annoyance is a frictional cost.)
I'm not an absolutist by any means. If there's fraud or a major market malfunction, unwinding trades could be the best remedy. Markets where it is a routine occurrence, though, seem like they could have been designed better from the outset.
2. Regarding #4.6.1.1, there are still certain markets where this is unavoidable. Pretty much anything that correlates to general prosperity for the user base would potentially end up with distorted probabilities.
I've seen a proposed solution to that where there are three markets:
1 - A is chosen AND causes X deaths
2 - B is chosen AND causes X deaths
3 - A is chosen (regardless of how many deaths it causes)
Someone looking to identify the probable safer vaccine can use the price of 3 to get an implied probability that A will be chosen, then this probability and the price of 1 to get the implied probability that A causes X deaths if it is chosen (and similarly for B). However someone looking just to trade and make money is always rewarded (probabilistically) for having better information than the market, plus the tax implications you describe don't exist.
I think this isn't the preferred solution because it is more complicated than just unwinding the trade and people don't want prediction markets to become the preserve of the extremely financially sophisticated.
It's debatable whether the stock market is efficient, and certainly there's no reason to think that a prediction market would be.
Markets are legal while gambling isn't generally because we want to allocate investment capital to enterprises, not just get a price for something. Having more money in the market means that companies have more paper value and thus can raise more money for actual operations - that is to say that the total amount of money in the market is meaningful, it creates value for the thing you invest in. In a prediction market there is no reason to want people to make larger bets, you just want enough to get accurate odds. Increasing the pot is likely harmful to society.
Having people make third-party bets with significant money is a bad use of money and regulators shouldn't allow it. Rather than betting on outcomes (zero-sum) people should invest in making outcomes happen (positive-sum).
Mostly I agree, but on the social costs -- consider that any opportunity for betting over the long term basically transfers cash from less cautious to more cautious people (usually from the bettors to the people who run the betting operation), which very likely reduces the velocity of money and hence tamps down inflation. This is a positive social value, much in the way that luxury yachts are a social value, because they transfer money from profligate idiots to ordinary workers (who build and operate the yacht).
Another quick thought...
"Insider trading" doesn't quite equate to "I have knowledge that others don't have and I use it in my trading" as folks sometimes assume. If you're relying on the existing legal framework to solve that problem, you might be surprised by some of the outcomes. As a first approximation, if someone along the way didn't use for personal benefit information that "really" belongs to their employer, it probably isn't insider trading. (The personal benefit doesn't have to be financial, and the person who misuses the information isn't necessarily the one who trades.)
Unless I'm misunderstanding, it seems like the use-case of prediction markets where only a minority are resolved by some event and for the majority the market's verdict is final is problematic, since in that case the response to the meme-stocks question 4.8 doesn't really apply.
E.g.
> A few randomly selected questions went before the Supreme Court each year, and the corresponding markets were resolved normally. In every other case, the market’s verdict was final, and bettors got their money refunded.
It feels like in this scenario it would be easy and fruitful for a rich person interested in a certain case outcome to artificially inflate or deflate the price of the market, knowing that there's only a 1/n chance that the supreme court is actually going to rule on it, and otherwise they are just buying their desired outcome. It feels like there should be a way to resolve this but I can't think of any off the top of my head.
Simple fix for that would be to prioritize resolution by trade volume rather than randomly. Harder somebody tries to buy a clearly unjust verdict, the faster that case will float to the top. Would also conveniently filter out frivolous cases, and making the trade volume a matter of public record along with the rest of the case info could help prevent sneaky setups involving unlikely combinations of trivial grievances from being used to manufacture broadly unreasonable precedents.
For far-future events, would recursive prediction markets work?
Given whether an event would occur within a 100 years, create a prediction market N about what a prediction market next year N+1 would resolve to be, where prediction market N+1 is about what prediction market N+2 will resolve to be, until prediction market N+99 which resolves to whether the event actually happens.
Prediction market N would close when the prediction market N+1 would start, and it pays out when prediction market N+1 closes.
Of course, if the event happens earlier you short-circuit the recursion.
Conditional prediction markets for things like replication studies, where the market outcome determines which of many conditional prediction markets gets settled, seem to have two problems.
First of all, if you're redoing one of 100 studies, then the potential return on each market is only 1% of what it would otherwise be. Suppose the market says that Study #17 has a 10% chance of replicating, but I think it has a 20% chance. If I buy a bunch of 10-cent shares in this market, then I expect a 99% chance of getting my money back, an 0.8% chance of losing my money, and an 0.2% chance of getting 10x my money. This has an expected return of 1%, which is not worth betting on compared to our hypothetical 5% index fund alternative.
Second, there's a bias in how much this affects different opinions. In the calculation above, I assume that Study #17 has a 1% chance of being redone. But actually, if I think it has a high chance of replicating, then I think the prediction market has a high chance of agreeing with me - and since this is how we select which study to redo, maybe I actually estimate a 10% chance that Study #17 will be selected. This improves my expected return from 1% to 10% and now I have an incentive to bet.
On the other hand, my pessimistic friend who thinks Study #17 is total trash and has no chance of replicating also thinks it's very unlikely that Study #17 will be selected by the prediction markets! So for my friend, even though the conditional prediction market on Study #17 would have an 11% return if the study did get redone (my friend would buy WILL NOT REPLICATE shares for 90 cents and be certain to get a dollar back at the end), there is only a tiny chance that Study #17 will be redone in the first place, and the expected return is correspondingly tiny.
Doesn't this have an effect where all 100 conditional prediction markets will be biased upward, because correcting that bias has a lower rate of return?
That is unrelated. People can make an unbiased estimate of how much companies will make in the future, discount that to present at the risk free rate, value the company at that amount, and still have a 0% expected return. The reason the stock market goes up over time faster than the risk free rate is because of risk premia, and the same dynamic would exist for prediction markets also used for hedging purposes.
Well, your talent for writing persuasively is on full display here, very impressive. Personally I think it's all bollocks, however, for the simple reason that a prediction market is only better than an expert when the experts aren't very good.
Nobody imagines that a prediction market would correctly predict my eldest son's name better than either he or I. If my drains back up, I am clearly much better soliciting the advice of a plumber on how to clear them than going up and down the street and establishing a prediction market among my neighbors. If I want to build a functioning nuclear weapon, the way to do so is clearly to ask an expert nuclear weapon physicist, and asking a prediction market is silly.
And so on. For almost all of the basic factual matters of life, it's obviously more accurate and faster to ask someone who knows than to sample public opinion randomly, and so of course that's what we do.
Hence the questions to which the prediction market would apply are ipso facto those in which we already know (from bitter experience) that expert opinion does a poor job, presumably because the outcomes are very difficult to predict -- data is lacking, there is some sociological issue with making the accurate prediction, there is some inherent source of randomness, et cetera. Predicting the winners of the next election, predicting the outcome of a war, predicting the course of pandemic disease, predicting future major technological or social changes, predicting Black Swan events -- all of these fall into this category.
But in these categories, expert opinion is already known to be poor. Proving that prediction markets are at least as good as expert opinion doesn't say anything that's broadly and usefully actionable, because proving the prediction markets do better than known poor predictions is a long way from saying prediction markets make good predictions.
A prediction market can certainly provide exciting betting opportunities[1] for those who are into that -- and plenty of people are, hence the popularity of gambling -- but it can't make better predictions in a way that solves broad social or economic problems. I mean, to think otherwise, you need to think you can get something from nothing, that from a sufficiently clever summation of ignorance you can extract brilliance. This is like thinking that if there's a mathematical problem that no mathematician has been able to solve, I should be able to ask a whole lot of non-mathematicians and somehow from that huge mass of ignorant opinion extract the correct answer. This just isn't possible. Frankenstein is a myth, you can't create life from non-life, there is no algorithm to create meaning from ever so much nonsense.
--------------------
[1] It doesn't provide a way to steadily make money, like the stock market, because as you point out there is no separate value of investing: investing in the stock market as a means of betting on the future values of things is being a "growth" investor, but I hazard most investors are actually "value" investors. They're not buying a share of XOM because they're betting it will rise in the future, they're buying it to own a tiny piece of a profitable company that pledges to return some share of its future profits as dividends. Because the stock market is a collective investment vehicle as well as a betting market, you can make steady money in it, regardless of how well or poorly it predicts the future. There is no such collective investment aspect to the prediction market, it's just a betting pool.
Your argument that a prediction market would allow hedging is interesting, but you've just reinvented the insurance market, which is already healthy, and you're not going to gut insurance company profits until prediction markets prove *far more accurate* than any other source of prediction, which brings us right back to the main point above.
>Hence the questions to which the prediction market would apply are ipso facto those in which we already know (from bitter experience) that expert opinion does a poor job, presumably because the outcomes are very difficult to predict -- data is lacking, there is some sociological issue with making the accurate prediction, there is some inherent source of randomness, et cetera. Predicting the winners of the next election, predicting the outcome of a war, predicting the course of pandemic disease, predicting future major technological or social changes, predicting Black Swan events -- all of these fall into this category.
>But in these categories, expert opinion is already known to be poor. Proving that prediction markets are at least as good as expert opinion doesn't say anything that's broadly and usefully actionable, because proving the prediction markets do better than known poor predictions is a long way from saying prediction markets make good predictions.
Part of Scott's point is that in your #2 scenario, the answer is knowable but the legible experts are *lying*. You can't lie on a prediction market, or rather you can but you lose your lunch if you do.
It is obviously the case that for a prediction market to give the right answer, there must be experts who know said right answer, *but* you don't have to know who they are in advance (i.e. illegible experts are available) and there are large monetary incentives for telling you the truth and penalties for telling you a falsehood (so it would cost a large amount for anyone, including an expert, to ensure you got a wrong answer).
I can't help but feel that all of your listed objections to prediction markets are downstream of something more fundamental:
Aren't markets cringe?
Depending on your outlook, markets lead directly to neo-feudalism or rainbows on everything in June. Either way, they're associated with those guys in the Econ department who wear bow ties and are politically unreliable at best. Markets allow the people you don't like to coordinate efficiently, and this is becoming increasingly intolerable. As more and more of society becomes politicized, both parties are becoming increasingly anti-market, and they have no incentive to overcome status quo bias and unban a new source of pluralism.
THIS is what you're up against. Change this attitude, THEN the point-by-point refutations will become relevant.
My experience is that most of the time declaring in favor of markets is a response to establishment decision structures that you dislike. Id est, people who declare in favor of free markets in car buying or the means of electrical production actually just dislike environmental regulation. People who declare in favor of free markets in dating and sexuality actually just dislike established social shibboleths around mating. People who declare in favor of free markets in recreational drugs actually just dislike drug laws. And so on.
I'm not persuaded *anyone* actually likes markets in and for themselves, on account of the main effect of a market is to diminish the power of special knowledge, special skill, or good luck to command success. They make everybody work harder to succeed, which is not fun -- unless you're already shut out of the conventional structure for success.
So it makes sense to me that "Rationalists" seen as a movement would promote markets in predictions -- because they feel (I think accurately) that their predictions for the important events of the future are dismissed by conventional wisdom, and by conventional authority. Naturally that leads to a wish to tear down the conventional protocol for predicting the future.
I think it would be useful/helpful to briefly discuss the fee structure of these markets, which render them a negative-sum game. I love prediction markets but every time I feel interested in getting involved I see the fee structure and what it implies for the % I need to get right (weighted by price) and lose interest. Still think they have tremendous value though!
The repeated "or someone can get rich quick" stories started to remind me very much of Anselm's argument. That is, there is a sense of "the true forecaster who can correct a mispriced market in reality is greater than all the ones who are only doing so in their imagination, therefore that true forecaster must exist."
4.2.1. There are ppl killing for a killing on the share-market: https://en.wikipedia.org/wiki/Borussia_Dortmund_team_bus_bombing
Sergey Wenergold, a GTA V player (buy Air EMU, burn FlyUS) - maybe Osama had some put-options at 9/11. But: CEOs not killed - lol, John Kenneth Galbraith famously noticed how replaceable most of those are. Politicians are, too - I guess. But the chance of Biden/DeSantis/Trump to "be president in 2026" is zero on death. There is no option on Wall-Street "Tim Cook apple-CEO in 2024" - else I would worry for his life (though he and Musk spend on security.) Enough blood even without incentives:
https://en.wikipedia.org/wiki/List_of_incidents_of_political_violence_in_Washington,_D.C.#21st_century
Let prediction markets stay. Small.
It's also easy to imagine less egregious misbehaviour. Let's posit it's 2020 and there's a bet on whether the President will be diagnosed with Covid this year. It so happens that you have Covid, and you have a long meeting with POTUS scheduled. So instead of being reasonable, you place a bet for Covid and go to it.
I love prediction markets and your enthusiasm for them in general (which is what originally made me aware of them), but you seem to be playing games with the "on average" thing a bit.
Like, in your Harvard v. Yale v. TinFoil thought experiment, suppose the TinFoil underdog turned out to actually be correct. It just so happens that the government did in fact make up the whole thing, and Harvard, Yale, and all other "respectable" institutions had the fatal blindspot of being funded by the government so <insert handwavium about incentives> they couldn't see it coming.
Your usual argument after this goes something like "This makes people more willing to trust TinFoil in the future, increasing TinFoil's weight in the market and thus making the market at least as accurate as TinFoil", but suppose TinFoil's talent is ONLY in play in case of global pandemics. Then people should only trust them in the *next* global pandemic, but how many global pandemics happen each century ? Probably 0 or 1 during TinFoil's lifetime, and they will probably die uncompensated for the truth they can help guide the market for, their talent wasted.
To tldr : averages only make sense on a big enough sample, and some events are one-shot (they don't even have to be true Black Swans, that's a ridiculously high bar, just as one-shot as a global pandemic or a nuclear war). Prediction Markets can theoretically always give the wrong answer and never correct. It's like a Prisoner's Dilemma where you only play once, Tit-For-Tat no longer works, you can Defect and get away with it.
As others have noted, there are huge prediction market for sporting events. Usually there's a "house" involved and the house gets a cut, but the situation is only slightly different from a pure prediction market. Interestingly, it's known that the odds offered are not always accurate relative to the available information. I've seen an interview with someone who made a good living at sports betting by consuming all available information and learning the patterns of the irrationalities of the betting public. He said it took him about nine months of full-time work to get to that point, though no doubt he was very interested in sports and betting before that. OTOH, it required that he ingest all of the available news about sports, so it's likely that he could have earned more money on Wall Street.
My suspicion is that there are few consistently profitable ways of utilizing a prediction market, that there are few large categories of marketable predictions for which it is profitable to have accurate predictions. And that most of those are covered by existing financial markets. (E.g. the airline that wanted to financially insulate itself from a possible general downturn in the industry, so it placed a mass bet against the stock prices of all the other airlines. They did this shortly before 9/11 and so had to explain to the FBI what was going on!) The "clever uses" listed above seem to be between once every few years (which new vaccine to prioritize) and never (a CEO wants the public to be able to raise questions they care about).
Sure bet: a certain segment of the high-IQ blogosphere will continue to obsess about prediction markets.
Odd thought maybe but I thought Id throw it out
Every bet is even money: a $ it will happen or a $ it won’t. The trading volume is analysed in real time to provide the prediction component.
How the heck is this supposed to work in practice? Who will hand out all the money?
I’m pondering that.
It probably doesn’t work.
Ok. I thought about it.
Think more like it’s a football pool rather than an options market.
Although it’s a bit of both.
Will x happen or not?
Bet yes or Bet no. 1$ The house is holding the money.
(The resolution is ultimately binary so there’s some logic)
The trading volume will start to reveal the odds (prediction) which will determine the payout (whatever money was bet on the wrong answer distributed to the winning side pro rata).
There is no advantage in being first (although that could be tweaked), it’s hard to game and I think a more transparent structure.
Obviously there needs to be a fee for the house so a percentage of the pot evaporates.
I'm not sure I would like to bet on something where the odds are only revealed after the fact.
Well this is a prediction market, so that makes sense to me. Otherwise it's just another betting market.
A masterful post, very well presented. However, I question the value proposition behind prediction markets. I consider three basic background variables:
a. The topic is "important" - meaning that an accurate assessment can guide current significant decisions. If not, it is merely "interesting". I'm talking here about the prediction market, not the outcome itself. An election outcome might be very important to many people, but an accurate assessment of the likely outcome won't actually change the outcome. Whereas an accurate assessment of the outcome of policy choices can be used to make those policy choices.
b. The topic is "timely" - meaning the final result will be knowable over a short enough timeframe to give predictors a real interest in seeing the outcome.
c. The topic is "independent" - meaning that an accurate prediction will not influence the result itself. Predicting the likelihood of a Yellowstone mega-eruption might be very "important", and possibly "timely", but Yellowstone won't decide on an eruption based on the prediction.
I see five basic situations:
1. The prediction deals with a topic that is "untimely", whether or not "important" or "independent". In this case, why would anyone participate in a market? We won't know an answer in a meaningful timeframe, so the "winners" won't collect. We could have a market on the likelihood of the Yellowstone eruption in the next 1000 years, but who will be around to collect in the year 3022?
1a. The prediction deals with a topic that is "important" but "untimely". In this case, partisans have a large incentive to tilt the prediction to favor their already-preferred outcome. For a current example, there are currently proposals to radically restructure the world's economy in order to prevent climate change and its associated predicted problems. Most of the predictions deal with potential effects by the year 2100. Most people alive today will not live to see the final result, but many have strong preferences on the policy question. If the prediction market is used to determine the policy choice, partisans have a strong incentive to try to tilt the prediction market to favor their preferred outcome. In such a case, the prediction market is really a vote by people willing to commit the resources to influence the policy outcome, but in a way less transparent and accountable than democratic governance, or even conventional lobbying and propaganda.
2. The prediction deals with a topic that is "interesting," "timely", and "independent". I consider election predictions in this category. We can certainly see election results in a reasonable timeframe. I doubt that voters will be much influenced by predictions stating the likelihood of "their" candidate winning. I doubt that anyone will make meaningful decisions based on current predictions of the next presidential election. If the election will have a material impact on the decision, it is almost certainly better to wait to see the actual election results. For instance, a business might consider a large investment in solar energy; and it might assume that a Democratic administration will offer large subsidies for such a project, while a Republican administration would not. In such a case, it would probably be a mistake to make significant resource commitments
In such a case, prediction markets are a useful counterweight to self-appointed experts, and can help people get a better read on the state of knowledge. But the markets don't serve an obvious social purpose other than perhaps balancing the shouting partisans.
3. The prediction deals with a topic that is "interesting" and 'timely", but not "independent". That is, we'll know the outcome in a reasonable timeframe, and it will be influenced by the prediction market, but the outcome isn't particularly important to most people. For example, "Who will be chosen to host the Oscars ceremony in 2025?" We'll be able to see the result, and it won't be of great importance to more than a few dozen people, but the decision is likely to be influenced by the prediction market. In such a case, I don't see a great social utility, and I don't see why anyone would participate, since future manipulation could tilt the market against current bets.
4. The prediction deals with a topic that is "important" and "timely", with strong partisan interest. For an example from 12 years ago, "Will a reform of the healthcare insurance market improve things for most Americans?" Setting up a prediction market to address this question gives partisans an incentive to tilt the prediction to favor their preferred policy, in order to enact their preferred policy or prevent a policy they don't like. In order to have a meaningful answer, the question would have to be much more sharply focused. For instance, "Will personal bankruptcies decrease by more than 50% within 5 years if the PPACA is enacted?" Or "Will deaths decrease by 40,000 per year from current trend within 5 years if the PPACA is enacted?" In such a situation, committed partisans will be willing to bet money to tilt the outcome to favor their preferences. Unless potential gains and losses in the prediction market are very large (hundreds of millions if not billions of dollars), I expect rational people with a strong policy preference would vote their policy preferences, rather than offering useful outcome prediction.
5. The prediction deals with a topic that is "important" and "timely", with no strong partisan interest. For example, "What is the likelihood of a Category 5 hurricane hitting New York City in the next 20 years?" This is "timely" because it's bound to a reasonable timeframe. It's "important", because an accurate assessment can be used to evaluate the desirability of measures to protect New York and nearby cities against the effects of hurricane. And it's independent because the Earth certainly isn't going to pay attention to the prediction market.
So, I think the only really useful applications of prediction markets are for issues that are "important" and "timely", but without strong partisan interests. Unless we can somehow exclude the partisans from the market, or attract so much capital to the market that even committed ideologues won't be willing to risk losing on the market.
I'm still seeing a problem which I don't think was addressed in the post: certain/many controversial topics won't be able to be resolved fairly.
For example, let's say you wanted to assess climate migration. You could create a market like: "100 million people will move to a different country between now and 2040 because of climate change." No problem with the question, but how would you resolve it? You could link to a tracker from the UN or some other "authority" as the way to resolve it, but then the market really becomes a test of whether 100 million people will change countries at all, because said authority _will_ say it's because of climate change, whether that case is strong or weak. Or perhaps there would be another "authority" that would always resolve it the other way, no matter how many people move.
I guess I'm asking: what if there is no fair authority to appeal to on a complicated, controversial question? My intuition says that most squishy sociological questions would fail on prediction markets due to this.
Is this fixable?
(First Substack comment by the way; hi everyone!)
I think there are a couple layers here. The controversy doesn't help, as it gives all parties (including the migrants themselves!) incentives to misrepresent reality. Even if we all decided tomorrow to agree on climate change, though, this would be a tough question to resolve. I see a mostly observable event (migrations) paired with a very hard to measure one (the underlying reason). What was the reason I snacked on kettle corn this afternoon? Stress? Hunger? Boredom? The sheer fact it was in the house? Some combination of the above? I couldn't give a good answer to that, and I have nothing riding on that outcome.
Or do you mean counterfactually, as in a world without climate change? That's tricky too. Would it be an otherwise identical world, but one where our economic activity somehow hadn't increased the greenhouse effect over where it otherwise would have been? Or a world where we didn't touch fossil fuels at all, leading to a very different current state of affairs (countries are much more uniformly poor, so there's less of a draw to migrate)? Even if we settle on a definition, there's the question of measuring the migration level in such a counterfactual world.
So I guess my reaction is: designing a good question can be hard!
I was definitely thinking along the lines of your first paragraph, and I agree with the way you parsed it out. Yes, determining how many people migrated is relatively easy while the second half (why) is very difficult. Interesting that you are focusing on the question instead of the resolution, which is what I'm more worried about.
Of course, one way to resolve it would be to collect very specific stories of people who clearly migrated for reasons of climate change. Spend about three paragraphs on each one for documentation, and just do this 100 million times. This would avoid the "we trust this authority" issue...but would obviously be impractical.
I thought my question was pretty good in the sense that it was clear, although not exceptionally detailed. I guess you could detail out what would qualify as a population moving due to climate change. Perhaps you could even count certain fractions of the population moving when certain causes are present. (Population moving due to coastal land disappearing 9/10, coastal flooding 4/10, heat wave 1/10 etc.) It would shift the subjectivity sharply from the resolution to the question itself but it could be done.
I could be talking myself out of my objection. I guess then my difficulty would be that this question could be presented thousands of different ways with different stipulations and that would make it harder to bet on, practically speaking.
Re 7.2: Using VPNs to access Polymarket is very safe and easy. It is not even possible for them to ban you as the smart contracts will always be accessible
Re 7.6: The head of the CFTC can't really do much. The commissioners have the final say and they vote
The main problems with Kalshi and Polymarket is that neither are sustainable. Both lose money by subsidizing liquidity. Polymarket does this directly and Kalshi does it via its affiliate, Kalshi Trading.
Without this, neither of the two platforms would have much if any liquidity on the long-tail markets that are probably most interesting to the readers of ACX
What a great Xmas surprise, thanks for this post, I've been looking for something that breaks this down better than the wiki.
Prediction markets don't work nearly as well as you make them sound. They have many problems you barely address:
1. The biggest problem is that if everyone was rational, nobody would bet in prediction markets (except for a few use cases like hedging by big firms). You briefly mention this and say "but in practice people do bet", yet I'm not sure that's true; in practice, people bet on the large, flashy items like presidential elections, but not on other things. There is no reason to believe this will scale.
2. You spend a lot of time focusing on the limit as the money in the prediction market goes to infinity. It's true that in the limit, prediction markets work. But see (1); the limit will never be reached, and the more rational society is, the less likely prediction markets are to have a lot of money in them.
3. Instead of focusing on the limit, consider the *rate at which that limit is approached*. Just how well do small prediction markets perform? What about medium ones? In general, they do pretty badly. There are a few reasons for this, including:
3.1. Goldman Sachs won't bet on them unless they can make more money than their quants' salaries cost. Even a $1000 edge is not worth people's time.
3.2. If the market has a lot of money in it, it basically implies that lots of stupid people have bet (see point 1), and
3.3. Smart people are less efficient at correcting markets than dumb people are at distorting them, due to the Kelly criterion; that is to say, if you believe the market prediction is off by 10%, it may only make sense for you to put in something like 1% of your money into the market due to Kelly (this depends on several factors), yet a dumb person may put in 100% of their money when distorting the market. You would have to have 100x more money than the dumb person to offset them.
4. The payoffs in conditional prediction markets are conditional, which amplifies all the problems in (3) and makes it a lot harder for them to work.
5. You spend a long time talking about whether prediction markets are effective, but not about whether they are *cost effective*. If I have $X and want a good prediction, should I (a) set up a prediction market, or (b) hire an expert or small group of experts? (I could average the group of experts myself later)
Almost certainly (b) is better for most things. This has been repeatedly observed empirically. It also makes sense theoretically: if predictions are expensive (e.g. they cost some fraction of $X for setting up a fancy model), then having a market fundamentally means *duplicating this expensive work*. Like, instead of one fancy model being set up, in a prediction market you're likely to end up with a larger number of smaller, cheaper models as people scramble to eat up the returns; this wastes effort as many players duplicate the same predictive work. There won't be enough edge left for Goldman Sachs to invest in the fancy model.
6. It just feels weird to say "in the glorious future, we'll all have perfect predictions funded by gambling addicts' losses". This is the same point as (1), but this time as a moral criticism. You are basically saying you want Goldman Sachs to set up fancy models, and those fancy models will be funded by the losses of gambling addicts, and you want it to be a stable, permanent equilibrium.
In a zero sum/negative sum prediction market, there is a certain element of "if you don't know who the sucker at the table is..." On the other hand, sports betting exists, so don't entirely discount fun/challenge as a motivation. It's not hard to make a prediction market positive sum for the participants, though. Political parties/campaigns could be a natural sponsor of election-related ones, for example, as they already spend money on polls on a regular basis.
Mechanically, market maker functions do it pretty naturally. If I want a prediction market on something because I value the information, I just pick the parameter so my maximum theoretical loss -- picture a participant who knows the future and has unlimited bankroll -- is whatever $X I want to subsidize the market with.
Alternatively, you can just publicly commit to periodically making dumb trades in the market.
See point 5. This is not cost effective, and you should instead use $X to hire experts (or to poll some experts or whatever).
Even from a budgetary perspective, the strategies aren't mutually exclusive. At some point there are diminishing returns to, say, buying slightly more frequent polling.
I can see funding a prediction market with your $X as potentially being the most efficient use of the money if the thing you're trying to ascertain is public opinion itself, and your target demographic is the kind of people who would participate in a prediction market, so smart, well-off, a little bit narcissistic.
> if the stock market grows 4% per year, I should expect any money invested in the stock market to multiply by 20x in 2100. So just doubling it in a prediction market is a bad option.
Normal finance has a good solution to this: futures markets. You can get very large leverage at very little upfront cost, and small liquidity loss by just agreeing with someone that you will pay them a particular price in exchange for a specified asset on a particular date.
I disagree with a lot of this, especially the parts that argue for prediction markets being highly efficient. There's many reasons that markets don't incorporate all existing information, some of which you touch on - opportunity costs of capital, opportunity costs of traders - case in point, I made around 300k on polymarket in 2021 and then quit in favor of trading NFTs and later crypto more broadly. With interest rates close to 5% it makes even less sense to trade anything longer term without a massive edge.
But even the extent that current markets are fairly priced is sort of free loading off their current low profile. If prediction markets got actually relied on in the way you propose, the returns to manipulation would go up. In your example of the government choosing between two vaccine candidates, the value of that choice to the vaccine company will be valued in the billions of dollars, and they can easily ensure the market price moves to where it needs to be with a small fraction of that. Investors will push back some, but there will be a wide range of uncertainty and so they won't want to risk too much. This generalizes; any time you want a prediction market to impact $X of real resources, you create an incentive for rent seeking on behalf of the beneficiaries. There's always a bound for how much capital is willing to correct an apparent mispricing, with unknown risks, possible adverse selection, and opportunity costs.
I'm even worried that this kind of Goodharting would apply to existing financial markets if they were explicitly relied upon for policy. Scott Sumner has suggested using NGDP futures to determine Fed interest rate changes, but it's not clear to me how you'd prevent people from moving those markets in order to profit on bond positions.
note that the author of this comment was arrested shortly after posting, and indeed his last tweet was a reference to this comment. Oppose prediction markets at your peril!
I think I could have a good crack at the time value of money issue. This wouldn't be starting a new prediction market, but selling a backend currency service to existing markets.
If anyone is based in the UK and would be interested in helping me start this, let me know below. Experience in payment operations or treasury tech stack would be particularly complementary to my own skills and experience (which are capital markets and regulatory structure across financials and infrastructure).
Another piece of financial market Infrastructure missing from prediction markets is a proper central counterparty role (which is currently effectively done within the markets, which I don't trust not to fold holding my money).
I'm more pessimistic about this one because it essentially requires a regulatory structure to be meaningfully helpful, and there's limited first mover advantage if you spend ten years working with the regulators, then Euronext/Clearstream come by and use their contacts and financial firepower to swipe the market out from under you, as they would inevitably be much better capitalised.
Scott, you say with regard to very long term predictions "But if you really wanted to use a prediction market, you could theoretically solve this by putting investors’ money in index funds while they waited. Then the winner would get their (and the losers’) original deposits and investment profits, and it would go back to being a better option than investing in index funds directly. In practice this seems complicated and I wouldn’t expect it to work." This kind of paired structure is extremely common in financial markets (e.g. convertible bonds) and I would see no problem at all doing this with an index fund and a prediction position, provided the prediction market venue itself was stable. Again, the derivatives markets clearing infrastructure shows how this can be done.
Let's run a casino as a mutual non profit association. All of the casino profits go to the gamblers.
If I trust the prediction market to usually represent the best odds of any given scenario occuring, why would I ever bet against the odds currently presented by the prediction markets? Even if I was an expert in Event A, why would I trust myself to know the odds of event A occuring better than the collective knowledge of the market? It would not make sense to participate unless I didn't trust the market's accuracy to begin with. In which case, I would not consider the market's ability to make accurate predictive odds inherently more trustworthy than any other source. If the market is trustworthy, I would not participate. If it was untrustworthy, what's the point of making one in the first place? Other than
trying to make money gambling against people I assume are stupider than me.
This post is extremely frustrating because you acknowledge all the counterarguments, you just refuse to connect the dots and follow them to the logical conclusion. It's basically this in essay form: https://i.kym-cdn.com/photos/images/newsfeed/001/286/448/863.png
For example, you have a section where you acknowledge the various reasons why real world prediction markets might be biased or inaccurate, and then spend the rest of the essay arguing based on the premise that prediction markets magically have infinite liquidity, zero transaction costs, etc.
Instead of responding to every question by saying "in the frictionless spherical cow world, you could make money by arbitraging this", the intellectually honest approach would be to try to quantify the bias in your hypothetical scenarios and see how much of a problem it would or wouldn't be. Even you yourself acknowledge at the end that real life prediction markets are highly biased and illiquid, and that while you might hope to improve that a bit, you also acknowledge that the real world stock markets, which function near-infinitely better than prediction markets could ever hope to, *still* have problems.
Another amusing bit was where you concede that "we shouldn't allow markets in dangerous things" and then later on complain about regulators restricting what you are allowed to have markets in. Obviously, you could have some principled position that the optimal level of regulation is nonzero but less than the status quo, but you never actually *say* that, so it just makes you look confused and dishonest, and it hides your true position.
Another huge weakness is that you completely dodge the "prediction markets will solve societal breakdown and lack of trust" thesis, despite that being *your leading argument*. You do finally get to the obvious question later about how the many people don't even trust wall street could possibly be expected to put their faith into a weird inferior version of the stock markets, but you just pathetically retreat to "well, the kind of person who trusts prediction markets will trust them and who cares about the rest?".
Lastly, you also don't have a good answer about the "why will people play a zero sum game"? problem. Obviously, people like gambling, but the demand for gambling is not infinite, and it is not evenly distributed. People like gambling on sports, but you're probably going to have a lot less luck getting people to gamble on say, vaccine stats (as you observe, this is borne out in the real world where prediction markets are tumbleweeds next to DraftKings et all). Also, gambling is probably a net negative to society anyway.
One other thing I should emphasize is that **the assumption of infinite liquidity and zero transaction costs is a load bearing element of your arguments**, despite you realizing that this is clearly not actually true. For example, consider proposals like "only decide a few of the markets randomly". If you have any actual finite amount of liquidity, risk tolerance, etc. then this is a huge problem because it cuts the effective liquidity down proportionally. This works in the spherical cow model because zero times any number is zero, but as soon as you plug in any actual nonzero number, you'll find that amplifying the bias this way is actually a problem.
Many good points here.
So what about Grossman-Stiglitz paradox? (On the Impossibility of Informationally Efficient Markets, 1980).
The crude argument goes like this. Suppose
(1) prediction market equilibrium prices reflect the current information completely ie arbitrage is impossible
(2) gathering information to update the market prices costs money.
Then, consequently, nobody has financial incentive to produce new, correct information and update the markets because there are no profits to made Thus we have proved all efficient markets are impossible with Logic™ ...which would be silly, so we probably didn't. But as a less silly consequence, Grossman & Stiglitz present a model and argue that a market can function only if the prices are imperfect enough to make information arbitrage possible.
Naively, I think this can be understood as a statement to the effect, an efficient market can be boundedly efficient, in general and over time, but not guaranteed at any particular given time point.
Here is also a neat conjecture: any person who considers themselves informed (say, they invest constant effort to remain informed) should try to get more investors, preferably less informed and overconfident investors to join prediction market, as many as possible. The informed person gets higher expected returns from increased opportunities, but the price system will be less informative.
All of this seems premised on the idea that different models disagree about the future, and not the present. I don't see that as being true for most questions, and we already have proxies for most objectively verifiable questions in stocks/bonds/etc.
Prediction markets can only ever be as reliable as the resolution criteria. Faking the winner of a presidential election is hard. It's a question where social reality is physical reality, but where the market implications thereof are vague at best.
The same is not true for most other questions. If we grant one year ago, that ivermectin is safe and effective and covid vaccines are not, knowing that the FDA is going to say the opposite in a year, is still not surprising, and is not a meaningful evidence in favour of getting the vaccine, lockdowning harder, not abolishing the FDA, etc.
Also, you don't really need a prediction market to know what the FDA is going to say about the vaccine, because vaccines are produced by publicly traded corporations, and so have stock prices.
I'm sure that legalizing prediction markets would be great and all, but I don't expect it to have the policy/political ramifications that proponents desire of it. The many downstream effects are premised on prediction market prices being useful arguments that will convince people, and for the vast majority of issues, they just aren't.
How is this different from the wisdom of the crowd, other than adding stakes for the crowd?
If you can't get rich, why are people betting?
“How is this different from the wisdom of the crowd, other than adding stakes for the crowd?”
Experts can make money by taking it from those whose beliefs are wrong. That seems markedly different from merely assuming whatever the majority of people believe to be true will be true.
As mentioned, historically, stock markets have been better or as good as experts at predicting outcomes.
As alluded to in the post, many people might stand to win/lose money just based on the event happening or not, and would like to hedge their risk by buying contracts which will pay them if the event that would lose them money would happen.
So you can find willing bettors even at disfavorable odds simply because they still stand to 'win' regardless of which even happens.
One thing I'm having trouble finding out with casual Googling: what is the standard way to bet *against* a prediction? If you already have a position in the market you can of course sell your position for a sufficiently low price, but if you don't already have a position can you "short" the prediction, and if so, what are the mechanics of that?
Or do markets just naturally come in pairs where that for every "the sun will rise tomorrow" market there's a complementary "the sun *won't* rise tomorrow" market?
It depends on the specific market, in a sense I would say there isn't a standard solution, but markets do form natural pairs, with the simplest solution being to just issue shares that pay out one dollar if X condition is met, and then just issue another share that pays out one dollar if the previous share doesn't pay out (if there's a reimbursement for ambiguous results, share B is reimbursed without payout conditional on A being reimbursed without payout.) After that, you just trust to arbitrage that the spread between shares and anti-shares will be equal to however inefficient you are (because of taxes, operating expenses, bonds having higher return, etc)
On Manifold, you can take either side of any market. Every "YES" market is automatically a "NO" market as well.
I've shorted hundreds of individual stocks over the years, and a crucial addendum to 4.8 is that the most-overpriced meme stocks become hard to borrow, meaning nobody could borrow shares to go short even if they wanted to. Even when shares are available to short, there is a risk that if you go short the price will spike and it will become hard to borrow and your shares will be recalled by the lender and you will be forced to cover your short at a very bad time, regardless of how much collateral you have. These structural barriers to short-selling make mispricings persist longer in the stock market. None of that is a problem on prediction markets. There is no limit to how many "no" contracts I can buy, and there is no risk of a temporarily overpriced market forcing me to close my "no" position prematurely. This will make prediction markets much more resilient than stocks against upward manipulation / memes / ponzi dynamics.
> There is no limit to how many "no" contracts I can buy
Well there is, it's just higher, because it is limited by your gambling bankroll, rather than your ability to borrow money from a broker.
You can see the effects of limited liquidity very clearly in the Salem Center markets, because the participant have limited amounts of money there and thus obvious mainsprings persist.
I just want to emphasize that borrowing shares of a specific meme stock is much harder than borrowing money. Even Warren Buffett couldn’t short some of those if he wanted to.
The time value of money presents serious issues that hamstring current prediction markets as well. An event a year away needs to be mispriced by like 10% for it to be worth intervening. I think it would be good to concede that this is a huge problem with existing markets in the FAQ. This is trivially fixable by simply wagering shares of SPY or something but we need regulators to let this happen. I would be elated to have a decent prediction market but regulators need to let it happen.
The other problem is if the outcome is correlated with SPY, or whatever else you are wagering.
Oh no it’s Nash equilibria again! “If nate silver was consistently better on average you could make money by betting on his predictions. Do this until you’re rich and the market has corrected.” “If there’s an advantage to deviating from a strategy in a game, people will do it until the game equilibrates to a state where there’s no advantage to be gained by deviating.” I’m the man with a Nash equilibrium hammer and everything looks like a nail!
"Prediction markets say there is only a 12% chance Ukraine will take Donetsk by winter, which is down by 9% since the logistical problems started”
I don't know if this an actual quote, but it either is, or it is an example that Scott created to demonstrate the usefulness of prediction markets, and it suits my needs either way.
It seems to me prediction markets should not shift like this (from one stable number to another). The whole point of them is to forsee unforseen events. So what's going on? We know two things for sure:
1. The price wasn't bid down before the logistics problems (from the Ukraine example), and 2. The price WAS bid down after the fact, so we know that experts considered the logistics problems significant to the outcome of the original question.
This leads to one of two conclusions. 1. The best experts either aren't participating in the markets or can't participate in the markets. This contradicts one of Scott's claims about prediction markets. Or 2. There are questions that markets are fundamentally unable to answer. The question "will there be an event that will lower expert's outlook of Ukraine taking Donetsk by winter by more than 8 points?" could not be correctly answered by the markets, since those who could answer that question would also bid on the original question in a way we know didn't happen.
So either markets don't represent the best expert opinion, or markets can't answer questions that the best experts consider important. Either one would be a huge flaw. So where am I going wrong?
“It seems to me prediction markets should not shift like this (from one stable number to another). The whole point of them is to forsee unforseen events. So what's going on?”
That’s not the point of prediction markets. The point is that they are better than the best expert, over the most possible questions.
New information should absolutely change the probabilities of prediction markets. Something that was previously contingent (say, logistics problems has a 60% chance of occuring) is now factual (logistics problems have occurred).
That must necessarily change the probability of an event contingent that new information.
Good post. I would add one thing about why we might expect prediction markets to be accurate: Over time, people who are good at predicting will make money, and so they will keep predicting, and they will have more cash to do so with. And people who are bad at predicting will lose their money, and will probably go away, and if they don't go away they'll have less money to bet with. The net effect is to concentrate money in the hands of people who are better at predicting, and the more money you are working with, the more you sway the market. So the market is moved strongest by the best predictors.
You don't think maybe people who can predict the future in murky circumstances might have a *lot* more lucrative career options than betting in a prediction market?
Seriously, if I knew when the Ukraine war would end, and how, or who would be the next President, or whether Tesla will be the world's largest car manufacturer (by volume) in 15 years, the *last* thing I would do with that incredibly valuable knowledge is place a bet and win a piddling few grand.
I think this speaks to Scott's point about prediction markets needing to be bigger, so that you could put lots of money to work on them, motivating entire Goldman Sachs–like organizations to put huge amounts of assets into them. Then it's not “a piddling few grand”.
However big the betting market is, winning bets will never be as lucrative as success in the real world. That's why people who know a great deal about horses breed them, they don't just go to the track and bet on them. Betting on things is just dicking around, creating nothing. If you have special important knowledge, you can create stuff and get paid far more thereby.
I mean, if arguendo we imagine a world where, briefly, the most talented and informed people spend their talent and information winning bets, because that's the biggest payoff, then the real world would start to collapse (for lack of the energy and attention of talented and informed people), and the disposable income anyone has to bet would dry up, and willy nilly the rewards for creating in the real world would once again exceed what you can get by betting on stuff.
I think you just proved that finance does not exist.
Great overview, thanks Scott!
Regarding Futuur, a few points of clarification:
• Futuur is a hybrid real-money / play-money prediction market platform. Real-money betting is not available in some jurisdictions, including unfortunately the US (for now). However US residents can still bet with play-money, and have full access to the forecasts generated from real-money wagering.
• Real-money and play-money markets operate seperately and generate independent forecasts. On the Mars prediction, it looks like you got your ~75% "no" forecast from the play-money version of this market, but the real-money version is currently showing 97% for "no". To access the real-money forecasts, you need to click on your account balance and switch to real money. (I know, this could and should be much more intuitive... we are working on fixing that!)
• When it comes to longer-term bets like the Mars one, another point to remember is that you can sell your position at any time. So if you bet early and are followed by others betting on the same outcome, you can exit your position at a profit well before the resolution date.
• Futuur (https://futuur.com) has more than 1000 markets, across all categories, and we're always looking for suggestions for more! You can make a suggestion here: https://futuur.com/i/feedback/suggestion
> 4.6.1.1: If people use prediction markets to hedge risk, won’t that distort them?
That is, suppose that an import-export business spends millions of dollars betting that Trump will win in order to hedge against his protectionist policies. Since their bets aren’t based on the real chance of Trump winning, won’t that distort the market?
No. Suppose that everyone knows Trump has a 50-50 chance of winning. And suppose the import-export business, in the process of hedging risk, bids it up to 90-10. Since you know Trump has a 50-50 chance of winning, you can get rich quick by bidding it back down to 50-50. From your point of view, the import-export business is (in expectation) giving you free money. But they’re still happy to do it, because they’re hedging their risk successfully.
___
I think this misunderstands the main hedging argument that I'm familiar with, which is namely that some candidates may be predictably bad for the entire economy, so everybody's risks are directionally correlated.
Suppose you have two candidates, Joe Jughashvili and Sam Clemens. Sam is a generic centrist, Joe says the right words but you worry that he wants to create a communist revolution. The true probability is 50-50.
You think there's a small but nonzero chance S&P 500 will go down by >80% if Joe wins. So you bet on Joe winning, on the grounds that if he wins you can hedge your risks considerably. Other people hedge in a correlated way to you and bid the numbers to 60-40.
Now suppose an independent hedge fund manager looks at this market and polling data. She realizes that the true probability is 50-50. So she wants to bet against Joe. But then she checks her own exposure and (correctly) realizes that her wealth is also tied to this market. So she wisely refrains from betting.
This is also covered in Lizka's post in 4.3: https://forum.effectivealtruism.org/posts/E4QnGsXLEEcNysADT/issues-with-futarchy
Why is Scott wasting his time asking people to try different things to help promote prediction markets? Surely the correct strategy would be to devise a bunch of conditional markets (if we try promotion strategy X, prediction markets will expand Y amount) and then simply do whichever strategy the markets claim will work?
Maybe also use this to test strategies to get the restrictions on U.S. use lifted?
Also, I'm curious about whether prediction markets will get big. Has anyone asked a prediction market this question? Maybe we can, in fact, narrow down precisely how big they'll get, and how long they'll stay that way.
For that matter, what will *replace* prediction markets, and can we get there now?
Maybe we can get still more meta. Can we use prediction markets to try and find what area is most likely to have wrong predictions, and use this to make money?
I honest-to-FSM have no idea if I am being snarky or serious here. (But maybe there's a way to find out!)
It seems like if we can use prediction markets to pick which trial to run, then presumably this can be used to drive science forward by having prediction markets on which type of trials to test out on prediction markets.
Actually, has anyone tried asking prediction markets which strategy will be most likely to solve the AI alignment problem? Or climate change? Or just let Ukraine win the war?
Using Scott's logic, isn't it true that EITHER prediction markets would tell us accurately how Ukraine could win the war, OR you could get rich quick?
Can we use prediction markets to fix holes in prediction markets?
> Using Scott's logic, isn't it true that EITHER prediction markets would tell us accurately how Ukraine could win the war, OR you could get rich quick?
Assuming you know a way by which Ukraine could win the war: yes? If you know how, and the market does not, you get rich. Alternatively, the market knows at least as good as you do (or even at least as good as anyone knows), yes.
Issues with alignment are the same issues as with other potential apocalypses, as mentioned.
Presumably you could use a prediction market to find out if you are being snarky or serious.
Decision markets worry me, mostly because of the loss of "slack" it may cause in certain economic actors, much like may happen in similar-ish things that Robin Hanson advocates (eg CEO-firing-conditional stocks). The current level of imprecision in decision-making allows for more "human in the loop" situations, where people can make decisions taking into account many things other than their bottom line.
In a situation between two choices, a human decision-maker can assess things like ethics and external impacts. A decision market really can't take into account anything that isn't both easily measurable and explicitly spelled out for it. A human CEO is actually able leave the hundred-dollar bill on the ground if it's good for it to stay there. Even publicly-traded corporations are scarily uncontrollable since due to shareholders, and they do take actions with negative effects that most individual humans probably wouldn't. Surely, a market for individual choices would be even worse in that area?
Purely informational markets (as opposed to explicit decision markets) sound relatively safe, but even they can potentially limit freedom of action in decision-makers. This can be both good and bad: the things to be weighed are, on one hand, increased efficiency, better information, and probably lessened corruption, and on the other hand, serious tightening of misaligned incentives and negative externalities.
I don't doubt that prediction markets would work as described, but I really don't know whether this would be worth it. I remain undecided, with a moderately-sized part of the "in favor" side being based on "these smart people think using prediction markets would be smart."
I think one major problem with current prediction markets is that good resolution criteria are hard to make. Resolution is ideally done based on clear objective outcomes, but what you care about is usually abstract general variables.
I think this problem can be resolved using reflective latent variables: https://www.lesswrong.com/posts/HygyWpDcwsekqmfnS/will-manifold-markets-metaculus-have-built-in-support-for
They are a mathematical technique that allows making correlated probability distributions over many variables.
Imagine a "how will the Ukraine war go?" market, which allowed people to make correlated bets on questions like who will be in control of various cities at various times, whether leaders will continue to be in power, who will be supported how much by their allies, etc.. Rather than getting difficult-to-interpret markets on narrow questions, this could give you probabilities for broader issues and complex scenarios.
You left out a very important part in your article, which is what source of truth to check to resolve a prediction.
Iny option this is the reason why Prediction Markets won't replace pundits: they could always say The New York Times got their COVID stats wrong and dispute whether a vaccine worked at all.
I am forever hopeful this will stimulate the creation of a fact-checking economy: newspapers that only publish thoroughly reviewed facts and abstain from opinions, that all parties can 100% agree on, but I'm very skeptical of it (I will give this a 20% chance by 2030)
Why shouldn't prediction markets invest the money waiting in the betting pools in index funds or in a randomized selection of stocks, to capture that 5% economy growth profit?
Because if the stocks go down, they won't be able to pay out.
This is like asking "why don't banks just take consumer deposits and put them in the stock market?"
Apparently FTX specialized in doing just that ha ha ha
Are the people putting down bets on a roulette wheel spin, a prediction market?
Is the guy with a computer in his shoe detecting the slight bias in the wheel an expert?
Sure! You just need to relabel the wheel. Instead of red 6 on one square you have "Valdimir Putin assasinated next month" and instead of black 5 you can have "breast cancer vaccine successfully tested before 2030". Then install your wheel at all the best high-traffic venues in Vegas, and bob's your uncle. We can solve all kinds of huge social problems with the foreknowledge acquired via this modern ouija board.
What is the counter to comparing prediction markets to hedge funds but for events and claiming they have the same weaknesses? Some hedge funds do well in predicting winning stocks, but there's no way of telling which in advance, and YOY some winners become losers and vice versa. If you believe events are independent and predicting one gives no significant skill in predicting the next, they would fail on average in the same way as hedge funds predicting the stock market.
The EMH doesn't say that hedge funds never have an edge, just that their edge will tend to be the same as the fees they charge.
> Someone who has resolved their past 100 prediction markets honestly will probably resolve this one honestly too, especially if they get paid to do so and will never get customers again if they lie.
So I should expect to see a lot of rugpull scams on Prediction markets, because this sort of reputational system doesn't work.
Ah the classic "I tried depositing $100 first as a test, and it worked great, so I put in my life savings".
I think it'd be nice to have a system trustworthy enough to put your life savings in, yes.
Prediction markets aren't like stock markets, because stock markets.trade items of some.intrinisic value...not opinions. Prediction markets are unrestricted betting.
What would be the legality of a company like Google giving each of its employees a free hundred dollars on an internal real money prediction market, which they can withdraw at the end of a year if they bet on enough markets?
They wouldn't be allowed to deposit their own money.
Would that still fall foul of US gambling laws?
Back when people cared about movies, Cantor Fitzgerald started a prediction market called Hollywood Stock Exchange to encourage betting on the first four week box office totals of new movies. But because Cantor Fitzgerald leased the top floors of the World Trade Center, a huge number of Cantor Fitzgerald employees were murdered on 9/11.
Eventually, a reconstituted Cantor Fitzgerald applied to create a futures market in movies. But the feds couldn't figure out what to do about insider trading -- e.g., if you saw a preview screening of "Amsterdam" or "Babylon" were you entitled to point out they weren't very good movies? So the market wasn't allowed to get off the ground.
One issue that is alluded to and other commenters point out is the 'oracle' or 'arbiter' problem, wherein the outcome of a given market depends on some measure that we are relying on some sensor or trusted reporter to accurately detect and convey.
If your prediction is "Will the maximum temperature in Boston on December 25, 2022 rise above 50 degrees Fahrenheit" there is the risk that the thermometer being used to measure the temp could be off by +/- 2 degrees for some reason, but maybe predictors are aware of this and adjust their bets accordingly. Still,
This could obviously be the problem for a market like "Will Bubba Joe have a Good Day on January 1st, 2023" because the only way to get the answer is to ask Bubba "did you have a good day" and maybe he reports that he sure did, even though his internal state is that his day was actually quite unpleasant.
In this sense, the contract is less betting on the actual underlying reality and more on what the reporter is going to say, which can add more and more uncertainty the less reliable the reporter is.
Hence, Prediction markets probably work best for events where many people can measure an outcome for themselves and confirm the validity of the report, rather than situations where we have to rely on a single 'authoritative' source which *could* be compromised.
On the positive side, we can also consider that if prediction markets gain widespread use, additional financial instruments can be built on top of them that abstract away the actual probability side of it and allow people to take positions on outcomes without having to be fully aware of all the information that might influence said outcome.
Re 5.6 (Education). When I was a kid I developed a habit that I now realize serves a similar fog-clearing function to the “what odds would you give this in a prediction market” check. When I want to check my confidence in something, I wonder, “How scared would I feel if I found out I had to get a lash across the back for each instance I was wrong?”
E.g., if I start by thinking “Everyone seems to like me, I don’t think anyone I know has maligned me behind my back,” then ask the whipping question, I realize I’m not so certain - because I have some nonzero fear about how much pain I’m in for.
It’s helpful for checking whether you have grounded confidence in something or whether you’re just hyping yourself up.
Your proof that prediction markets are canonical is wrong. You need to replace "get rich quick" with "get rich in expectation". I learnt recently about risk-neutral probabilities and how S&P futures are a counter-example to your claim. The historical average return from the S&P is something like 10% (maybe something like 7% after inflation). The average historical price of 1-year-out S&P futures is something like 2% to 3% above whatever the current price of the S&P was. (I may be slightly misremembering the numbers from when I was shown this example a couple weeks ago.) So you can make pretty much arbitrary amounts of free money in expectation by correcting this mismatch.
So here are some actual limitations on prediction markets that have good theoretical backing:
1. The accuracy of a prediction market is bounded by (at least) the highest savings account interest rate. If the resolution time is a year out and you could make 4% return on your money by putting it in a savings account or a CD, you shouldn't expect anyone to be correcting the market if their return from doing so is less than 4 cents on their dollar
2. Long-term prediction markets also need to correct for inflation
3. If the outcomes are not independent of the value of money (if you're betting in USD on the collapse of the dollar, or if your betting on an AI apocalypse in any currency you don't expect the AI to care about), you shouldn't expect pricing to reflect probability. I'd be willing to bet basically my entire fortune that humans will still be alive tomorrow (as long as the return is higher than the interest rate on savings accounts), because this increases my expected utility in worlds where humans are alive without changing my utility in the other worlds.
"And although the contributions of a hundred people like me would add up to $85,000 and correct the mispricing, I guess there aren’t a hundred people who know about this strategy and care enough about making a few hundred dollars to pursue it. So the site continues to lean slightly conservative."
...he said, correcting the next election's mispricing.
Although it wouldn't solve everything and would be hard to implement at scale, one path to addressing the issue of betting on things like the world ending by some date is to have the party that believes the world will continue pay out immediately so that the party who thinks the world will end can enjoy the funds while the world is still here. If the world doesn't end by the specified date, the party correctly believing the world would continue would receive a large payment from the counterparty to compensate for the opportunity cost. Garett Jones proposed this idea in more detail here: https://www.econlib.org/archives/2012/09/how_to_bet_on_b.html
If the other person thinks the world is going to end, are they going to have enough money to hold up their end of the deal if it doesn't?
great point Doug. This is one of the reasons it would be hard to implement at scale. There are conditions it would work under when you have trust with the counterparty, but it depends a lot on the circumstances. If you'd like I can provide some examples, but I'm guessing you can probably imagine a few just as easily.
You mention a couple times that random people trying to participate in prediction markets should expect to lose money, but on Manifold you can see people's profits, and a strong majority of accounts I've looked at have positive profits. I was confused by this for a while but I think the reason is that Manifold markets are subsidized by market creators and by Manifold itself (through bots). (Weirdly, those bots also have positive profits! That I don't get.) This probably makes the site more fun for everyone, but also makes it harder to tell if you are doing a decent job. It would be cool to know what the average user profit on Manifold is (both in absolute numbers and as a return rate); that could maybe be treated as the zero point for individuals like me trying to have a sense of how we're doing.
What about this failure mode?
"I envision that someday people who want to know the answer to specific questions can subsidize prediction markets on them. For example, the Democratic Party might subsidize a conditional market (see 5.1) about which Democratic primary candidate is most likely to win the general election."
Ok, so say the Democratic Party sets up this market, and plans on taking action based on that information. And let's say the underlying reality is that Biden is a bit more likely to win the 2024 general election than Newsom.
However, the Republican Party and its supporters then pour a lot of money into the market to make it look like Newsom would be a bit more likely to win than Biden. Sure, other bettors are busy making bets to bring it back in line with polls and experts, but the Republican Party keeps pouring money in.
At the deadline by which the Democratic Party needs to take action, the market says Newsom stands a better chance of winning the general election than Biden. Due to that misinformation and the actions the Democratic Party takes as a result, Newsom wins the primaries. Then Newsom loses the general election, a more likely outcome than if Biden had been nominated.
Regarding doing destructive things to manipulate markets, a more complete answer is that having made those market transactions is a big piece of evidence of who did the destructive thing that tends to get you imprisoned for doing it. I remember reading about just such a case in (of course) Florida a few years ago: a guy was arrested for trying to bomb Target stores to make Target's stock price go down.