Free markets take advantage of the wisdom of crowds

The recent near-collapse of the financial markets has lead to a call for tighter regulation; and, a higher proportion of news pieces mock of free markets.  There is comfort in the thought of a kind and wise government that protects its citizens from harm.   As a result, support is growing for the belief that free markets have soured and that government should take a more active role in controlling the economy.  Paradoxically, the root cause of our problems may be that our markets are not free enough.

The free market operates on the principle that complex information can be better processed by large numbers of people, each person being a tiny social neuron, that when combined into a crowd, can out think even the brightest individual or think tank.    The wisdom of crowds is not just an abstract idea.

Michael Shermer’s essay in Scientific American ( gives compelling evidence for the wisdom of crowds.  The gist of the thesis is that when a large group of diverse and independent individuals are given a problem, their solution will on average be optimal and superior to a consensus view of experts.  The free market, be it a market of ideas or a system of trade, works most efficiently when individuals make decisions by applying their unique expertise to the available information.

For a crowd to be smart, it must include a large number of members that have a diverse range of ideas, all pondering the data in hand without collaboration.  As a trivial example, Shermer describes how a group of 56 individuals were asked to estimate the number of jelly beans in a jar.  The group average of 871 was within 2.5% of the correct number (850), with only one individual guessing within this range.  The principle is that individual errors cancel.  Had the group been permitted to interact, a dominant individual could have swayed others, compromising error cancellation.  Consensus is not always the best way to settle issues – though it makes us feel good.

On the TV show,  “Who Wants to Be a Millionaire,” the audience is right 91% of the time while the “experts” are only correct 65% of the time.

A more interesting case is described in James Surowiecki’s book, “The Wisdom of Crowds.”  To locate the missing U.S. submarine Scorpion, which disappeared in May 1968, John Craven – a naval scientist – asked a diverse group of submarine experts, salvage divers, and mathematicians to to give their best estimate of the location, steepness of decent, as well as other variables based solely on the last-known speed and position of the sub.  Craven applied Bayes’s theorem to the experts’ guesses, which assigns probabilities to each variable, to determine the submarine’s location.  The actual location of the Scorpion was 220 yards from the calculated position.

The University of Iowa’s Henry B. Tippie College of Business runs the Iowa Electronic Markets (IEM) (, where one can buy futures in a variety of markets.  Most notable was the IEM’s 2008 presidential election markets, which I followed during the primaries and throughout the general elections.

Back in June 2006, one could bet on the outcome of the presidential election by buying a contract.  For example, in the winner takes all market, the Democrat contract pays only if the democratic candidate wins the popular vote and similarly, the Republican contract pays out only with a republican wins.  The price of a contract is determined by supply and demand.  An individual is not likely to sell a contract if he/she believes it is undervalued, thus limiting the availability of contracts.  Investors buy up contracts that they feel are undervalued.  Both of these actions drive up the price.  The equilibrium price (the price at which the number of buyers matches the number of sellers) reflects the crowd’s prediction of the outcome.

Throughout the primaries and general election, the winner takes all contract prices meandered or spiked day to day as investors evaluated news stories, gossip, and whatever other information they deemed important.  Amazingly, since the contracts first became available, the Democratic ticket never trailed in the winner takes all market.

The “Vote Share Market” contract, on the other hand, is a bet on the percentage of the popular vote cast for a given party.  For example, if the “average” investor believes that the republican candidate will receive 52% of the vote, the market price of a Vote Share Market republican contract will sell for 52 cents.

Prices on the IEM’s Vote Share Market predicted that Barack Obama would receive 53.55 percent of the two-party presidential popular vote, and John McCain would receive 46.45.  The actual numbers were 53.2% for Obama and 46.8% for McCain – an average error per contract of only 0.3 percent – compared with poling data that was off by 2%.

These examples illustrate how an independent analysis by a large number of individuals can lead to accurate predictions.  For a group to be smart, it must meet three criteria.  (1) The group must be cognitively diverse so that a broad range of expertise is represented; (2) It must be autonomous so that each individual acts independently; and, (3) The group must contain a large enough number of members so that errors cancel.  Free markets are large groups of individuals using information to make decisions, which in aggregate, are near optimal.  A successful market economy rests on these same three pillars.

In times of crisis, people naturally turn to leaders and experts.  With the near-collapse of financial markets and job insecurity, the idea of legislation to fix the problem is seductive.  With the prospect of huge unemployment, government bailouts are comforting.  But how can one individual find intelligent solutions that target the problems when the available information is so complex?

A top-down government fails to optimize results when a small group of individuals make decisions, especially when rulers have similar ideology, thus lacking information capacity and cognitive diversity.  The two-party system limits diversity and the added political pressure by special interest groups biases the decision-making process.

What is the solution to the current dilemma?  Being an individual, I don’t know; but, the crowd is likely wiser than a small group of elected leaders working behind closed doors.   The trick is to set up a system, such as the IEM, where the autonomous individuals who are acting on the data have a financial stake in their best guesses.  In contrast, we got into this mess because everyone was jumping on the sub-prime-loan bandwagon.  The problem was not greed, but stupidity.  The players did not act independently and the ideas where homogeneous.  Additionally, the packaging of mortgages degraded the quality of available information, making risks more difficult to judge.  This led to a market failure.  Legislation should focus on unshackling the free market and preventing market failures rather than regulating and micromanaging outcomes.

To support a successful free market, government needs to provide an institutional framework that insures fairness and transparency.  Government also serves an important role in legislation to prevent market failures.  For example, externalities (such as pollution) are usually not accurately reflected in the price of the finished good.

Finally, government can play an important role in making investments in areas where the profit motive fails.  For example, companies have no incentive to throw cash into research for alternate uses of inventions or drugs that are no longer protected by patents.  Existing medicines, vitamins, or common herbs might cure a particular type of cancer; but to determine the efficacy of any such treatment requires studies by scientists – costing millions of dollars.  Carefully targeted funding of research by government can clearly lead to large benefits to society.  This is particularly true of fundamental research.

With the right governmental institutions in place, truly free markets could increase our wealth and quality of life.

Imagine an unfettered free market that leads to a pool of employers and employees that operate on a contractual basis rather than on the present model of long-term employment.  If one employer goes bankrupt, the employee contractor has many other options.  Similarly, employers can draw on the expertise of a diverse pool of contractors.

The role of employers and contractors can be interchanged.  Contractors become employers when they need additional expertise on  a project and an employer sells services to a costumer who uses these services as a contractor.  Instead of insiders and incestuous board members rewarding each other, all work is done by independent individuals at a contract price that is set by supply and demand.  No longer is job security an issue as long as the pool of available jobs and contractors remain large and diverse.

Many successful business that are home-based and rely on the internet for advertising and contracting already follow such a model.

Protectionism and bailouts of large industries rewards inefficiency at the expense of successful companies and prolongs the misery of long-term unemployment, promotes geographic stagnation, and stifles innovation.  While the pain of the present crisis may require a step backwards, we should not loose site of the long-term advantages of a decentralized free market system where the crowds digest large amounts of information and optimize outcomes.  Where work contracts are determined by supply and demand so that people are paid fairly, and the availability of a broad range of jobs from competing companies brings the best kind of job security to the individual, who works as much or as little as he or she desires.

I believe that the answer to our ills does not lie in the experts or in the government, but in the genius us – large numbers of independently-acting individuals.

Mark G. Kuzyk

Leave a Reply

You must be logged in to post a comment.