Research

Last updated on February 3, 2020

Vote.org makes every attempt to quantify our impact. Over the past two years, we have run a number of large-scale controlled experiments in partnership with the Analyst Institute, Pantheon Analytics, and the Harvard Kennedy School of Government. Read more below.

2018 Research

How can we best leverage “cold” SMS to increase voter turnout?

For the 2018 general election, Vote.org conducted SMS voter mobilization programs covering 12,681,951 people of color and unmarried women in 33 states. These programs used “cold” text messaging to registered voters who have no prior relationship to Vote.org. Key takeaways from Vote.org’s 2018 P2P SMS voter mobilization program are below.

In person voting

  • Vote.org’s SMS treatments increased turnout in the 2018 election, generating an average increase in turnout of 0.26 percentage points at a cost per net vote of $85.69 (11.7 net votes/$1000).
  • The program caused 17,586 people to vote who otherwise would have missed participating in the 2018 General Election.
  • If the weak confidence cell phone matches and unsuccessful attempt to mobilize people for early in person voting only are excluded, the average effect is 0.42 percentage points at a cost per net vote of $53.14 (18.8 net votes/$1000).
  • Vote.org’s “cold” SMS voter mobilization program identified several important lessons about targeting and messages to improve future effectiveness, including two messages that were the most effective: Adopt a Voter (encouraging recipients to get their friends and family to vote) and Social Pressure (thanking recipients for being a registered voter and reminding them that their voting record is public).

Postal voting

  • Vote.org’s SMS messages increased voter turnout among people who had requested a ballot. On average, the treatments generated a 0.21 percentage point increase in turnout at a cost per net vote of $75.14 (13.3 net votes/$1000).
  • The Social Pressure treatment was the most effective message for increasing turnout in an experiment comparing Social Pressure vs. Political Efficacy vs. Standard Practice), generating a statistically significant increase in turnout of 0.28 percentage points at a cost per net vote of $56.36 (17.7 net votes/$1000).

Early Voting vs Election Day In Person Voting

  • SMS mobilization is most cost effective for Election Day voting, generating an increase in turnout of 0.28 percentage points at a cost per net vote of $51.21 (19.5 net votes/$1000).
  • SMS mobilization for only Early In Person voting did not significantly increase turnout.
  • The impact of SMS mobilization for Early In Person voting and Election Day voting was not significantly or substantively larger than Election Day only (0.32 percentage points; cost per net vote of $113.34, 8.8 net votes/$1000).
  • In future “cold” SMS voter mobilization programs, Vote.org should focus on mobilization for Election Day voting.

—–

Which messages are most effective for turning out Vote.org tool users?

For the 2018 general election, Vote.org conducted SMS voter mobilization programs covering 1,790,781 people who had used one of Vote.org’s tools (“subscribers”). Since these individuals had previously interacted with Vote.org SMS messages could be delivered via the Mobile Commons platform (rather than peer-to-peer), but this relationship was “warm” only in a very limited sense.

Vote.org’s program compared four versions of the message sequence used to encourage turnout: Direct Link (provides URL to look up polling place info), Request Info (offers information about polling place location when a recipient responds with a request), Adopt-a-voter (encourages recipient to get friends and family members to vote), Recommend-a-friend (asks the recipient to provide the phone number of a friend so Vote.org can mobilize this individual).

Among the 47 states plus DC that offer in-person voting, the average effect was a 0.24 percentage point increase in turnout ($11.47/net vote; 87 votes/$1000).

However, there are wide differences in the effectiveness of the treatments, with the Request Information treatment being the most effective message for increasing turnout in this experiment, generating an increase in turnout of 0.49 percentage points at a cost per net vote of $5.62 (178 net votes/$1000).

The Direct Link treatment (generated increase in turnout of 0.30 percentage points) and Recommend-a-friend treatment (generated an increase in turnout of 0.21 percentage points) were statistically significant; the Adopt-a-voter treatment did not generate a statistically significant effect (0.11 percentage points).

These results mitigate concerns about using the Request Information approach in “cold” SMS programs when it is required (since you can’t send a link in the first SMS message to a contact). The results also suggest providing information is more effective than trying to increase motivation to vote in person among subscribers.

In the three states where all registered voters were mailed a ballot in 2018, the average effect was a 0.75 percentage point increase in turnout ($2.93/net vote; 342 votes/$1000). Again, there were notable differences between the two treatment tests, although the pattern was different than the in- person voting states.

  • The Adopt-a-voter treatment generated an increase in turnout of 1.02 percentage points at a cost per net vote of $2.15 (465 net votes/$1000).
  • The Direct Link treatment generated an increase in turnout of 0.47 percentage points at a cost per net vote of $4.67 (214 net votes/$1000).

In future “warm” SMS mobilization programs to subscribers, Vote.org will consider the Request Information treatment to be a best practice in in-person voting states, and Adopt-a-voter message to be a best practice in postal voting states.

—–

Can you use billboards to increase voter turnout?

In the lead up to the 2018 midterm elections, Vote.org ran a billboard program in 10 states. This report covers Scott Minkoff’s turnout analysis of Vote.org’s 2018 billboard programs in Mississippi and Maricopa County, AZ. Billboards were coded based on their location so as to establish which ones were most likely to have an identifiable effect. Two complementary approaches were then taken to estimate the effect of billboard proximity on voter turnout: a distance analysis that compared turnout among registered voters within a radius of a billboard and a ring analysis that compared people that live within a specified radius of a billboard with those that live far away from any billboard. There is evidence that the billboards had a small positive effect on low- and mid-propensity voter turnout in Mississippi. Evidence of such an effect is weaker for Vote.org’s Maricopa county program. The results offer preliminary support for the idea that billboards are an effective GOTV strategy in some locations, but additional research is necessary to reach more definitive conclusions.

—–

Does the ability to use an electronic signature to submit a form affect registration rates and turnout?

In 2018, Vote.org implemented an option on its absentee signup and registration pages that allowed users to sign and submit their forms electronically. The analysis by Pantheon Analytics finds that absentee tool users who chose E-Sign were much more likely to successfully vote absentee — and to vote overall — than those who chose to print their forms, despite the fact that E-Signers had lower pre-existing vote propensity scores. Users of the registration tool’s E-Sign option also had greater success in registering than those who printed their forms, though less success than those who opted to be redirected to their state’s online registration system. These improvements could be attributed to E-Sign, or they could be due to unknown self-selection factors. Indeed, analyses of Vote.org’s users in the weeks just before and just after the E-Sign implementation do not show higher overall success rates after E-Sign went live. Yet, there are also indications that success rates were bound to decline as Election Day drew closer, and that E-Sign may have mitigated these effects in a positive way. Overall, there are more positive indicators than negative ones for E-Sign’s efficacy at increasing registration rates, turnout, and absentee voting behavior.

—–

Can you use email and SMS for absentee ballot signup to increase voter turnout?

In October of 2018, Vote.org ran a randomized control trial to determine whether digital messaging (email and SMS) pushing voters to sign up for absentee ballots could increase overall turnout. The experiment was conducted in eight states that allowed no-excuse absentee voting, with a total audience size (post voter file match) of 212,000 voters. The overall results of this trial were inconclusive. In the full sample, no statistically significant effects were seen on turnout. Among states without permanent absentee voting, however, a significant half-point increase in absentee voting was observed in the test group. In Florida (the only state in the sample allowing online absentee signup), there was an even larger increase in absentee voting. In non-permanent absentee states, low-propensity voters saw a significant turnout effect of seven tenths of a percentage point. Among high-propensity voters, the treatment had a small but significant negative effect on turnout (just under half a point), while mid-propensity voters displayed marginally significant method-switching behavior and non-significantly higher turnout.

—–

Can a “Make a Plan to Vote” tool increase voter turnout?

In the lead up to the 2018 midterm elections, BallotReady partnered with Vote.org to create a co-branded website with voter information, including a vote-planning “Make a Plan to Vote” tool or MAPTV. BallotReady is a non-partisan social enterprise that runs a website, BallotReady.org; the website provides information about every candidate in upcoming elections. In this study, we aim to experimentally test the effectiveness of the Vote.org/BallotReady information and turnout tool on electoral participation; the study was a clustered randomized control trial (RCT) of BallotReady users in the 2018 US midterm elections. Users were randomly allocated to visiting the MAPTV version of the site (treatment) or the default, information-only BallotReady page (control). While users of both tools turned out at high rates well above the national average (74%), matching users to voter files based on address, we find no statistically significant effect of the MAPTV tool on turnout. Point estimates of the effect are generally positive although small in magnitude. The report considers several reasons why the intervention may have had only small observable effects.

2017 Research

How do you optimize get-out-the-vote SMS programs?

In the lead-up to the November 2017 gubernatorial election in Virginia, Vote.org ran a cold SMS get-out-the-vote program targeting unmarried women and people of color. After discovering in 2016 research that SMS is a cost-effective method to get people to vote, we began looking at ways to optimize this method. We tested whether it was more effective to send four election-related SMS messages in the month leading up to elections or to send one text message the week before the election. We also looked at which text messages were most impactful – those that provided polling place information, ballot information, or an encouragement to vote based on social norms.

Our results revealed an important learning — sending election reminder text messages in the week before the election can be as effective and cheaper than sending messages over a month-long period leading up to the elections.

We also learned that polling place location messages alone can increase turnout, while messages with ballot information alongside polling place location information may generate further boosts to voter engagement (at least in lower-salience election environments, like an off-year election, where voters may have less information about items on the ballot).

—–

What happens when you use both mail and SMS to get people to vote?

In the lead up to the December 2017 US Senate special election in Alabama, Vote.org, and Voter Participation Center (VPC) collaborated with Analyst Institute to test the effectiveness of SMS messages and social pressure mail on getting people to go vote. The test targeted African-American voters to determine which tactic most effectively mobilized the group — SMS alone, mail alone, or SMS and mail combined. The test also considered two different types of SMS messages – informational messages with logistical voting details and rationale messages that prompted recipients to consider reasons to vote.

We learned that social pressure mail – when combined with SMS or sent alone – is the most effective in getting African-American voters to turn out for a special election. Additionally, informational messages performed better than rationalization messages via SMS. Overall, combined mail and SMS messages were the most effective at driving turnout

2016 Research

Can you use one-to-one SMS to register people to vote?

In 2016, Vote.org ran the country’s first-ever voter registration drive done via text message. We purchased the phone numbers of just under 1 million unregistered voters, and hired actual humans to send them text messages on at a time encouraging them to registered to vote. We were genuinely curious if this would work, and entered the experiment with no expectations. We were pleasantly surprised to find that this outreach method not only worked, but that it was also cost effective. This method proved far more cost-effective than door-to-door registration; slightly less expensive than site-based registration, and almost as affordable as mail-based registration.

—–

Can you use one-to-one text messages to increase voter turnout?

Immediately after running the SMS voter registration drive, we turned our attention to GOTV. This time we purchased the cell phone numbers of registered voters, and hired humans to send them SMS reminders that the election was coming. We also proactively provided polling place location. We contacted a total of 3.9 million potential voters in this fashion, including just under 1 million potential voters on Election Day alone. Of this cohort, 1.2 million were part the experiment group. The experiment showed that texts providing polling locations increased voter turnout by 0.2 percentage points while plan-making texts were ultimately ineffective. Polling location text messages sent cold to young voters targeted from the voter file are an effective turnout tool, generating an effect on par with that observed in an academic meta-analysis of conventional nonpartisan GOTV mail programs.

—–

Will sending text and email reminders to your supporter list increase voter turnout?

Over 2 million people opted in to email election reminders from Vote.org in 2016, and another 1.1 million people opted in to SMS reminders. We were curious if these reminders would have any impact. Our experiment group included 325,000 potential voters who received aggressive SMS reminders, and 510,000 who received a series of email reminders. The email reminders seemed to slightly decrease turnout, but the result wasn’t statistically significant. Voters who received SMS reminders, however, were .65% more likely to vote than those who did not. This effect was three times as large as the effect of sending unsolicited SMS election reminders to registered voters, although both tactics have a statistically significant positive effect on turnout. A cold contact program, moreover, is easier and cheaper to scale given the low acquisition costs associated with purchasing phone numbers compared to building an opted-in list.

—–

Can you increase voter turnout by allowing electronic signatures on absentee ballot applications?

This one is a bit nerdy, so bear with us. Only a handful of states let you apply for your absentee ballot online. This means that the vast majority of potential absentee voters need to print and mail their forms. Only home printer ownership is so low that Hewlett Packard stopped publishing statistics on it in 2011, when home printer ownership had fallen into the single digits. Moreover post office branches have been closed and hours have been cut, and physical mailboxes are increasingly hard to find. What this means is that printing, stamping, and mailing a physical form is a clear roadblock to getting your absentee ballot. That’s where electronic signatures come in. Most states will let you fax your absentee ballot application. Vote.org worked with HelloSign to build a version of our absentee ballot tool that let voters “sign” their forms by taking a photograph of a manual signature. We then faxed the completed form to the appropriate Local Election Official using the HelloFax API. For the voter, the process was entirely “online” in that they didn’t need to print the form. For the Local Election Official, nothing changed: they still received a signed paper form, only the form was typed instead of handwritten, and therefore easier to read.

Voters were then given a choice of how they would like to submit their form: either they could print and mail their form themselves, or they could sign electronically, and have Vote.org submit the form for them via fax. Not surprisingly, younger voters were more likely to choose the electronic signature route. Shockingly, they then went on to vote at higher rates. We like to think of this as election alchemy: Vote.org turned low propensity voters into high propensity voters simply by making it easier for them to sign and submit their applications.