The voting system is perhaps the most important aspect of any collaborative policy-building tool. For the past year, I have been working on an experimental side-project, code-named Commons, to build an online debate and voting platform to enable people to collaboratively build political policy. The aim is to build an online, distributed alternative to centralised parliaments, so that the general public can truly have political influence.
Voting in Commons works differently to the strategies used in actual parliaments. The traditional methods of voting are not appropriate when trying to build a debating environment for, theoretically, millions of people. This post details the voting system that I built for Commons.
This article does get technical fairly quickly, and it might be necessary to have some grasp of statistics to understand the latter parts. I have tried to keep the first half, which details the key ideas behind the Commons voting system, accessible to everybody.
The problem with a traditional voting system
To create new political policy or modify existing political policy in Commons, a user must create a “motion”, which is a proposal for the changes that should take place to the main corpus of political policy. Once a motion is published, the community can begin voting. If the community decides to accept a motion, it will be incorporated into the main corpus permanently, or at least until somebody else changes it with a future motion.
The first thing that must be realised is that, for each motion, it would be impractical to ask millions of people their opinion on it. We must use some method of sampling, which leads to two obvious voting systems. We could either stop voting after a definite number of votes have been counted, say 1,000 votes, or we can stop after a definite number of hours have passed, say 24 hours. Then we would simply total up the number of votes for, and the number of votes against, a motion and use the majority vote to either accept or reject it.
But there is an obvious problem with this traditional voting system. For a motion that everybody agrees should be accepted, we want that motion to be accepted as quickly as possible. And if everybody agrees it should be rejected, we similarly want to reject it quickly. If a motion divides opinion, we would like to keep it open for longer to enable more debate and to gather more votes.
Suppose we have a motion that runs for 24 hours and, in the first hour, it gathers 900 votes against it and 100 votes in support of it. You can intuitively infer the consensus opinion of people being against the motion. In this traditional voting system, however, the motion stays open for another 23 hours. It becomes a distraction from the other motions that are vying to be accepted.
Alternative voting system for Commons
The Commons voting system addresses this problem by accepting or rejecting a motion as soon as it possibly can. It removes the motions that become a distraction, encouraging debate and voting on the outstanding issues, where it actually matters.
The system looks at the difference between votes over time, and has two thresholds representing the vote difference where it is safe to assume that a motion is accepted or rejected. As soon as the vote difference hits one of these thresholds, it is closed automatically and, if it was accepted, the motion gets incorporated into the main corpus.
The motion page contains a chart which shows whether the motion is on course to be accepted or rejected. The chart looks rather simple, and is purposely devoid of any scale. The horizontal line is the ‘0 line’ which, for now, you can think of meaning the vote is split 50-50, i.e. there is no reason to believe the motion should be accepted or rejected. The upper dashed line is the ‘acceptance threshold’. If voting reaches this point, it is automatically accepted. Similarly, the lower dashed line is the ‘rejection threshold’, and a motion is rejected if it reaches this threshold.
The x-axis represents time, so we can see how the vote changed through the lifetime of a motion. The width of the entire chart shows the approximate voting duration which, in this voting system, is not fixed so must be estimated.
So what does the y-axis represent? Your instinct might be that it represents the number of votes against subtracted from the number of votes for a motion. It is safe to think this way in Commons, although it might not be true in the general scheme.
The statistics behind the voting system
Voting in Commons is driven by a statistical tool called a ‘sequential probability ratio test‘ or SPRT for short. SPRT is used to test a null hypothesis against an alternative hypothesis. The basic idea is to observe votes one at a time until it is ‘safe’ to stop and then to accept one of the hypotheses.
In Commons, we are interested in what the entire population thinks of a motion. We call the true vote p. This is what the outcome would be if we asked every member of Commons to vote. For example, if p = 0.70, and there were a million people on Commons, then 700,000 people would vote ‘Yes’ and 300,000 would vote ‘No’. However, we cannot ask a million people to vote, so the actual value of p is unobservable — we can only estimate it.
Our basic aim is to test whether more people are in favour than are against a motion. We restrict ourselves to two hypotheses for the values that p could take:
- H0: p = p0 = 0.49
- H1: p = p1 = 0.51
The null hypothesis H0 basically says that most people are against a motion, while the alternative hypothesis H1 says that most people are for it.
You may have noticed that there is a gap between the values of p. If the true value lies in this range, we do not care whether the motion is accepted or rejected. More specifically, we only want to have statistical control when the value of p is less that 0.49 or greater than 0.51. To use SPRT, it is necessary to have this gap; if the true value is p=0.50 exactly, should the motion be accepted?
I am not going to go into the precise details of SPRT (see Wald’s paper below if you are interested), but it basically works according to the following rules:
- Let Sn be the cumulative sum of the log-likelihood ratio. The likelihood ratio is the probability of the data being observed under H1: p = 0.51, divided by the probability of the data being observed under the H0: p = 0.49.
- If Sn > tA, then the motion is accepted.
- If Sn < tR, then the motion is rejected.
- Otherwise, observe the next datum n+1 and repeat.
The tA and tR are acceptance and rejection threholds, respectively. We take tA = log((1 – β) / α) and tR = log(β / (1 – α)). These are thresholds which achieve approximate type I and type II error rates of α and β. See Wald’s paper to see where these values come from.
We assume that votes come from a Bernoulli(p) distribution, iid, where p is the true unknown value. The cumulative-sum log-likelihood ratio can be written as:
Sn = NA(n) * log(p1 / p0) + NR(n) * log((1 – p1) / (1 – p0));
where NA(n) is the number of accepts after n votes, and NR(n) is the number of rejects after n votes.
That brings us back to the y-axis of the chart above. The y-axis scale is actually the cumulative sum of the log-likelihood ratio, and the blue line is Sn plotted over time. The two dashed lines are the thresholds tA and tR.
Remember that I said you could think of the y-axis as the number of votes against a motion subtracted from the number of votes for it. You can see why this is true, by looking at the individual contributions to the cumulative sum for an acceptance vote and a rejection vote:
- Accept: sA = log(p1 / p0) = log(0.51 / 0.49)
- Reject: sR = log((1 – p1) / (1 – p0)) = log(0.49 / 0.51) = – log(0.51 / 0.49)
So sA = – sR. This means the y-axis is simply a scaled version of NA(n) – NR(n). This is also why the zero line does indeed represent a ’50-50′ vote split. However, in a more general scheme, we could choose p0 and p1 differently, and this property might not hold.
So far, we have dealt with every aspect of the chart, except for one: the width of the chart. The width of the entire chart is a useful feature as it gives you some understanding of how close a motion is to completion. But, since we do not close motions after a set amount of time, this width also has a statistical basis. After some derivation, it turns out you can estimate the remaining votes according to the following formula:
Expected remaining votes = max(a* / C, b* / C) + 1
where C = p̂ log(p1 / p0) + (1 – p̂) log((1 – p1) / (1 – p0)), a* = tA – Sn, b* = tR – Sn and Sn is known. The only thing needed is a plug-in estimate of p̂.
Dynamic and relevant
The Commons voting system does not rely on a fixed number of votes or a fixed duration of voting. This system makes the Commons platform more dynamic, and makes the content more relevant to users. I see this system as a critical feature of Commons, which will hopefully lead to an engaging experience for users. There is still a lot of work to be done before the application is ready for release, but stay tuned to this blog for further updates.
 A. Wald. Sequential tests of statistical hypotheses. The Annals of Mathemat- ical Statistics, 16(2):117–186, 06 1945. doi: 10.1214/aoms/1177731118. URL http://dx.doi.org/10.1214/aoms/1177731118.