Measuring Campaign Efficiency

Courtesy: Jim Young /REUTERS

Courtesy: Jim Young /REUTERS

As you may have seen last month, we took a look at the results of our PAAR model scores for the 2016 U.S. Senate races. As was the case in 2014, we were surprised to see how the PAAR model stacked up to the final electoral results. As a quick reminder, the PAAR model is based on several factors including past election results, demographic changes, and whether an incumbent is running to predict the most likely outcome of race between a generic “Democrat” and a generic “Republican.” We developed this model as another way to judge the successes and failures of a campaign that moves us past just talking in terms of whether a campaign won or lost. This cycle, we ran the model in the middle of 2015 and it predicted that the Democrats would end up with 49 seats, which was just 1 off from the actual results and more accurate than many aggregators and prognosticators on the day before the election.

After analyzing the results, we wanted to include a larger perspective to these scores and add an additional measurement to judge the outcomes of the campaigns. Given the vast amounts of money candidates for U.S. Senate spend every cycle (not including outside PACs, Independent Expenditures, etc.), we realized there may be a way to use PAAR to measure campaign effectiveness. The questions we set out to answer were: How does one measure an “efficient” campaign? And how does PAAR fit into that measurement?

Using just the money candidates reported spending on their own[1], we crafted a formula based on this amount, the voting population in a candidate’s state to attempt to level the playing field between states (since $1 million means a lot more in North Dakota than it does in California), and our PAAR score. Once we put the data together, we realized this is more of a ranking than a score, since there is technically no upper limit on spending. That being said, it does function similar to a score in the sense that the closer you get to 0, the less efficient your campaign was. This year, our scale ranged from 0.38 to 45.75 points. For this analysis, we’re going to leave out campaigns that spend less than $0.10 per eligible voter since anything lower than this threshold tends to mean that the candidate did not really run a competitive race by any definition.

As we’ve been arguing for a while now, we need to move past wins and losses when it comes to measuring campaigns because that viewpoint only provides a one-dimensional screenshot and does not allow us to learn for the future. This Campaign Efficiency Ranking isn’t entirely about winning either, as you’ll see – some candidates with higher efficiency rankings lost badly in November, while some inefficient campaigns won. We tend to give accolades to the campaigns that raise the most money; however, we feel it is time to start recognizing who is spending the money well, and who is not.

Most Efficient Campaigns: Democrats

The most efficient Democratic campaign was also one of the biggest crowd favorites for this election cycle: Kamala Harris, who we also should remember ran against another Democrat last November rather than a Republican - but we’ll talk about that later. Harris spent about $14.5 million in a state with a voting-age population of over 29.6 million people. She did overperform with PAAR, scoring 3.9 points, putting her in 3rd place among successful Democratic campaigns on the PAAR scale, compared to Rep. Loretta Sanchez, who spent about $4 million to receive a PAAR score of -3.8 points. In the end, Harris won the election easily (62% to 38%) and has already positioned herself as a vocal new member of the Senate. Spending just $0.49 per eligible voter, she received the highest PAAR Efficiency score among Democrats of 21.96 points.

Among the top five most efficient Democratic campaigns, the only other Democrat to win besides Harris on November 8 was incumbent Senator Richard Blumenthal of Connecticut, who also won his reelection easily (63% to 35%). Spending approximately $5.9 million in a state with a little more than 2.8 million eligible voters, Blumenthal ranked 4th on the PAAR scale, also achieving 3.9 points. He received the 3rd highest PAAR Efficiency score of 15.87 points, spending $2.09 per eligible voter.

Interestingly enough, two of the three remaining holders of top five spots are candidates from extremely red states. Jim Gray from Kentucky was the 2nd most efficient Democratic campaign, despite losing to incumbent Senator Rand Paul handedly (57% to 43%). However, he scored the second highest PAAR score (5.6 points), and spent just over $6.3 million in a state with about 3.4 million possible voters. With all this in mind, he received a PAAR efficiency score of 17.00 points. In Louisiana, Democrat Foster Campbell certainly faced an uphill battle considering what happened to the last Democrat the state sent to the Senate. However, he overperformed with PAAR (scoring 1.5 points) and spent about $5.7 million in a state with a voting-age population of slightly more than 3.5 million – meaning he spent $1.63 per possible voter. This allowed him to achieve the 5th highest PAAR Efficiency score, 14.92 points.

Rounding up the top five most efficient Democratic campaigns is crowd favorite Deborah Ross, who also couldn’t manage to pull off a win on Election Night. Even though PAAR predicted she would lose, she still overperformed, scoring 1.8 points in our PAAR analysis. She also spent about $14 million in a state with 7.6 million voting-age people – figuring out to $1.84 per possible voter. She received the 4th highest PAAR Efficiency score of 14.98 points.

Harris and Blumenthal are the true winners here, as Democratic candidates in blue states who didn’t overspend to end up where they did on Election Night, especially considering they have some of the highest regular PAAR scores overall. Jim Gray also consistently scores well with our analyses; he ranked 2nd in both the overall PAAR scores and the PAAR Efficiency scores. Unfortunately, his fate may have been sealed by Kentucky’s history as a solid red state at the federal level.

Again, remember that efficiency doesn’t always translate to winning; it just means that with these outcomes, these candidates ran relatively resourceful campaigns.

Most Efficient Campaigns: Republicans

Given the outcome of the elections this year, it is not too surprising that three of the five most efficient Republican campaigns were successful on November 8 and that these campaigns were run by incumbents. Conversely, the two campaigns in the top five who lost on Election Night were running against Democrats in solid blue states. It’s important to note that each campaign in the top five earned a PAAR score that was 4.7 points or higher, meaning the money they spent had a positive outcome for them on November 8.

Interestingly, the two most efficient Republican campaigns were ones that also lost pretty badly. Scott Milne was bound to have a tough time running against Patrick Leahy in Vermont; Leahy not only enjoys one of the highest favorability ratings among his colleagues, but he is also the most senior member of the Senate, period – he’s been in office since 1975. Despite losing 61% to 33%, Milne scored much better than Leahy on our PAAR scale (6.7 points and -9.0 points, respectively), and spent only $0.21 per voter in a state with a voting-age population of just 504,976 people. He received a PAAR Efficiency score of 45.78 points, the highest across the board. Kathy Szeliga also faced a tough race in Maryland, where Chris Van Hollen easily took retiring Barbara Mikulski’s long-held seat (60% to 31%). However, like Milne in Vermont, Szeliga scored much higher with PAAR than Van Hollen did (5.1 points vs. -1.6 points), and only spent $0.33 per voter in a state with about 4.6 million people eligible to vote; her Efficiency score was 29.55 points.

While one could argue that these two campaigns, like Gray’s and Campbell’s campaigns, weren’t exactly effective in that they didn’t accomplish their desired outcome on November 8, the Efficiency Score shows that the campaigns were effective irrespective of its actual outcome when compared to the expected result.

In North Dakota, where no one really expected incumbent John Hoeven to lose reelection, the Senator outperformed beyond what pundits were already expecting. While he probably spent a little more than he probably should have to simply achieve reelection (he spent $4.88 per vote in a state with a voting-age population of just 570,955 people), he received a PAAR score of 17.6 points – one of the highest overall – and beat Eliot Glassheim by a 62-point margin (79% to 17%). He ranked 3rd among the most efficient Republican campaigns, with a PAAR Efficiency score of 17.61 points.

The last two most efficient Republican campaigns were in states considered to be battlegrounds this election cycle. Senators Marco Rubio and Rob Portman were both considered to be vulnerable incumbents, with the distraction of Rubio’s presidential run presumably hurting his image and Portman’s favorability ratings falling far behind his opponent’s, former Governor of Ohio Ted Strickland. Rubio spent $1.40 per voter in a state with more than 15.8 million eligible voters, earning a PAAR score of 4.7 points and a PAAR Efficiency score of 17.36 points. In Ohio, Portman spent $2.97 per voter with more than 8.9 eligible voters up for grabs and received a PAAR score of 8.8 points; thus, he earned a PAAR Efficiency score of 16.96 points, rounding out the top five most efficient campaigns.

Least Efficient Campaigns: Democrats

Not everyone can run an efficient campaign, right? Well, unfortunately for Democrats, only one out of the five least efficient Democratic campaigns was successful and all five candidates received negative PAAR scores. The one candidate who won was Chris Van Hollen in Maryland; even though he ran against Kathy Szeliga, who ran the 2nd most efficient campaign among Republicans, he won by nearly 30 points on Election Night. With a PAAR score of -1.6 points and basically spending $0.57 per eligible voter in the general election, he received a PAAR Efficiency score of 11.22 points, the 5th least efficient campaign among Democrats this cycle. So why did he win? Well, Democrat Barbara Mikulski held that seat for 30 years, there is only one Republican in Maryland’s congressional delegation, and the state hasn’t voted for a Republican in a Presidential or Senate election since the 1980s; it’s safe to say that Maryland is a solidly blue state, albeit one with a Republican Governor. With all this in mind, and noting that Maryland Democrats out-register state Republicans 2:1, Van Hollen would have had to have done a lot worse in order to lose in November. That being said, the money he spent in the general election was not spent as wisely as most.

In the red states of Idaho and Iowa, it didn’t come as much of a surprise that Democrats Jerry Sturgill and Patty Judge lost out to Republican incumbents by margins of 24 points or more. Sturgill, who spent $0.41 per voter in a state with about 1.2 million eligible voters, underperformed with PAAR, scoring -1.5 points. He received a PAAR Efficiency score of 10.33 points. Judge, who even received an endorsement from President Obama, fared even worse: spending $0.94 per voter in a state with a voting-age population of about 2.3 million people, she received one of the worst PAAR scores of the night, -8.1 points, giving her the 2nd worst PAAR Efficiency score of 5.41 points. These two races are examples of Democrats facing uphill battles that are nearly impossible to overcome in red states, and the money they spent didn’t help – and you could even argue that it hurt – their campaigns.

The other two candidates scoring some of the lowest Efficiency scores among Democrats were in battleground states. Katie McGinty, who was a crowd favorite to beat incumbent Senator Pat Toomey in Pennsylvania, spent $1.25 per voter during the general election in a state with a voting-age population of over 10 million only to underperform with PAAR, receiving a negative score of -4.6 points. Indeed, she received the 3rd worst PAAR Efficiency score of 10.32 points. In Ohio, Ted Strickland was long favored to win against incumbent Rob Portman at the beginning stages of the campaign, and ended up as one of the biggest losers on Election Night. He received one of the worst PAAR scores (-10.9 points) and spent $0.80 during the general election per voter in a state with a voting-age population of almost 9 million, earning the lowest PAAR Efficiency score among Democrats by far - 0.38 points. Both races show that it is far more important to be an incumbent than it is to be a crowd favorite.

Least Efficient Campaigns: Republicans

The story is very, very different when it comes to the least efficient campaigns on the Republican side. All five – yes, every single one – won on Election Night. They also all received negative PAAR scores, none better than -1.3 points, and each spent at least $1.47 per voter. The other common theme between these five candidates is perhaps also the most important: they are all incumbents. In fact, the only candidate of the five running for his second term is Rand Paul; everyone else has been in the Senate for even longer – in two cases, since 1987.

Let’s start with Rand Paul in Kentucky, who ranked at 2nd among least efficient Republican campaigns. Perhaps his inefficiency had something to do with his presidential bid this cycle or having an opponent who was running what would become the 2nd most efficient Democratic campaign, as we outlined above. Either way, in a state with just about 3.4 million eligible voters, Paul spent just over $11 million - $3.23 per voter – and still underperformed in PAAR with a score of -3.8 points. Because of all this, his PAAR Efficiency score was 12.83 points. However, he still won by 14 points (57% to 43%), but given his inefficient sending of money, and underperforming where he should have been we should probably hold off on the congratulations for his campaign team.

Senators Johnny Isakson and Mike Crapo were seeking their third and fourth terms in the US Senate, respectively. Isakson, who also beat his Democratic opponent by 14 points, spent $11.2 million in a state with 7.6 million eligible voters and earned a PAAR score of -1.3 points; this got him the 4th worst PAAR Efficiency score among Republicans, 13.12 points. Crapo spent about $5.8 million in a state with a voting-age population of about 1.2 million people (yes, that’s $4.83 per voter) to receive a PAAR score of -2.7 points, earning the 5th worst PAAR Efficiency score among Republicans of 13.44 points. With this and their incumbency in mind, their opponents also underperformed with PAAR which possibly mitigated the inefficiency of their campaigns.

John McCain was one Senator that the press hounded throughout the election cycle: was his disapproval of the GOP’s presidential candidate hurting his chances, or was he not speaking out enough? Pundits thought this race may turn out to be a close one; it wasn’t (McCain won 53% to 41%), but it sure could explain the amount of money the incumbent from Arizona cashed out this cycle. With a voting-age population of about 5.1 million people, McCain spent $2.81 per voter and still underperformed with PAAR, scoring -2.9 points. This got him the 3rd worst Efficiency score among Republicans: 12.97 points.

And then there’s Richard Shelby. Shelby has represented Alabama in the US Senate since 1987 (just as long as McCain), which was rated the second most conservative state in 2015 only behind Mississippi by Gallup. He won this year’s election by 28 points. Seems like a slam dunk for a Republican incumbent – so why did he spend over $12 million this cycle? Not only does this feel like a large amount of money on its own – he spent $3.22 per voter, with only 3.7 million possible voters to cater to – but it’s an even larger amount considering how his race turned out. Yes, he won by a solid margin, but he also received a poor PAAR score of -9.9 points and the worst PAAR Efficiency score of 10.93 points. By comparison, his Democratic opponent Ron Crumpton received a PAAR score of 11.4 points and only spent $29,419, which was such a small amount of money that we didn’t even analyze it for our Efficiency rankings.

We’re not sure why Senator Shelby felt the need to spend that much money, but it certainly didn’t help his case whether he was predetermined to win by his electorate and incumbency or not.  

Races of Note

One race we should discuss that includes a candidate we haven’t spoken about is in California. This Senate race was unique in that it was two Democrats running against each other, which is why we haven’t included Rep. Loretta Sanchez in our analysis yet. It isn’t fair to compare her situation with that of other Democrats, who, apart from her opponent Kamala Harris, all faced Republicans. However, had we included Sanchez in the general group of Democrats, she would have had not only the lowest PAAR Efficiency score among Dems but also the lowest score overall.  Only spending about $4.1 million, she received a PAAR score of -1.7 points, coming to an Efficiency score of 1.76 points (for some reference, the only score technically lower than this is that of Ted Strickland, 0.38 points). There are many arguments for why this would occur, including that she didn’t spend enough money to win over California voters, but this race is unique enough that the actual reason is hard to pinpoint. Grouping all Democrats together, this race contained the most efficient Democrat versus the least efficient Democrat, which may at least partially explain why Harris won so decidedly in November. 

Predictably, there are quite a few races where the more efficient campaign won against the more inefficient campaign. Ohio is the most glaring example, where Rob Portman (the 5th most efficient Republican campaign) easily defeated Ted Strickland (the least efficient Democratic campaign). Portman spent $19.4 million more than Strickland, but his money went a lot farther than Strickland’s did when it came to performance; Portman received a high PAAR score and Strickland received one of the worst Democratic scores. While it seems contradictory that the more efficient campaign of the two spent more money in the same environment, the money was obviously more well spent considering the outcome on Election Night.

In addition to Ohio, Iowa and Pennsylvania experienced similar situations, albeit not as drastic. In Iowa, incumbent Senator Chuck Grassley won against Democratic candidate Patty Judge by 24 points. While this was a narrower margin that Senator Grassley is normally used to, it was still a solid win. He also ranked 7th among Republicans concerning efficient campaigns, receiving a PAAR score of 5.6 points and an Efficiency score of 15.21 points. On the other hand, as we’ve already discussed, Patty Judge ran one of the most inefficient campaigns and underperformed vastly on Election Night. This, coupled with Iowa’s status as a red state and Grassley’s mostly stable incumbency, makes the outcome of Election Night unsurprising.

In Pennsylvania, where we’ve mentioned Katie McGinty ran the 3rd least efficient campaign among Democrats, incumbent Senator Pat Toomey ran a more efficient campaign in a contentious race. He ranked 10th in efficiency among Republicans, receiving a PAAR score of 1.7 points and a PAAR Efficiency score of 14.55 points. The amount he spent was more than double the amount spent by McGinty, but his spending certainly meant more on Election Night.

There were also many races where inefficient campaigns beat efficient ones, emphasizing the fact that the Efficiency score isn’t always about winning but is instead more about spending the right amount of money for the eventual outcome on Election Night. The first example is one we’ve already discussed at length and won’t spend too much time rehashing: Rand Paul won easily in Kentucky, even though he ran an awfully inefficient campaign compared to Jim Gray’s incredibly efficient one.

This also happened in Arizona. Despite Ann Kirkpatrick running the 6th most efficient Democratic campaign and receiving a positive PAAR score and John McCain running the 3rd least efficient Republican campaign and receiving a negative PAAR score, McCain was reelected on November 8.

In North Carolina, Richard Burr also enjoyed the benefits of incumbency by beating Deborah Ross despite his inefficient campaign squaring off against her efficient one. Ross ran the 4th most efficient Democratic campaign, while Burr ran the 7th most inefficient campaign among Republicans. She also received a positive PAAR score, while he scored in the negatives. And, atypically, she actually spent more than he did – about $0.52 more per voter. However, even though this was one of the closer races, Burr still won by 6 points.

This dynamic also existed in Maryland, but with the opposite outcome. Now-Senator Chris Van Hollen ran a vastly inefficient campaign, while Kathy Szeliga is ranked 2nd among Republicans. She also scored much better with PAAR than he did, and by spending much less. Still, Van Hollen won by 14 points.

imageedit_23_4658985093.png

So, what do these races that seem ironic and contradictory considering the less efficient candidate won tell us about federal campaigns? They certainly show the power of incumbency (or in Van Hollen’s case, his somewhat-inheritance of Mikulski’s incumbency) and the general makeup of the electorate. In a solidly red state, a Republican has a good chance of winning despite how much money they spend and where they spend it, and vice versa. Subsequently, this is our note to these candidates in those kinds of states: reign in your spending, and spend wisely – you’re probably going to be just fine either way.

There were two other races we noticed that could also give us some insight into why the polls were so neck-in-neck up until the last minute. In Missouri, both Republican Senator Roy Blunt and Democratic candidate Jason Kander received Efficiency scores that ranked them in similar spots, with Kander ranking better than his Republican counterpart despite Blunt spending nearly double; Blunt scored 15th among Republicans and Kander ranked 11th among Democrats. This was always going to be a close race (Blunt received a 0.1 point PAAR score and Kander got 0.2 points), but the comparative Efficiency scores reinforce this aspect.

Similarly, in New Hampshire, Maggie Hassan and Kelly Ayotte ranked somewhat in the middle with Hassan slightly higher (Hassan ranked 12th and Ayotte ranked 18th). Spending about $1 million less than her opponent and achieving a positive PAAR score, Hassan triumphed in one of only two races where Democrats could unseat Republican incumbents. It was a close and high-profile race from the beginning, perhaps explained by the proximity of their Efficiency scores and explaining why so much money was spent overall, but Hassan managed to pull it off and provide a silver lining for Democrats.

Conclusion

There are many factors swirling around a campaign, including the climate at both the national and state levels, campaign finances, and so on. You can’t just say a candidate won because they spent the most money, and you can’t just blame a candidate’s loss on a presidential wave. Everything is intertwined, so it is imperative that we look at these candidates and their campaigns through multiple lenses.

The Efficiency Score isn’t a predictor of a successful campaign, as candidates like Jim Gray prove. But it does give insight into what happened along the campaign trail and the situation we’re all in right now. With all the talk about campaign financing, it’s important to note where people spent a lot of money, how efficiently it was used, and how much it influenced the outcome of a particular race especially when we see people spending money they don’t need to spend, like incumbents over spending in safe states and challengers overspending in races that are beyond feasible. Democrats have a long road ahead, even beyond 2018 and 2020, so effectively allocating resources is going to be a key to success.

What the Efficiency score also proves is that money is easily wasted in states where the outcome is all but predetermined; it’s hard for Democrats to win in states like Kentucky or Louisiana, as it is for Republicans in states like Maryland or New York. Through this analysis, we’ve seen candidates waste a lot of money that had little to no effect on the outcome of their race.

We also need to start measuring our campaigns by more factors than simply who won, and we certainly need to start holding campaigns and consultants to task for running wildly inefficient campaigns, even if they did win. As is always the case, we don’t view these scores as the only way to look at the data, but we hope these numbers start a conversation on a new way to look at the campaigns that we run.


[1] We used spending amounts reported by OpenSecrets.org updated as of February 27, 2017. Additionally, for five candidates who dealt with significant primary races (Ted Strickland [OH], Katie McGinty [PA], Patrick Murphy [FL], Chris Van Hollen [MD], Tammy Duckworth [IL], and Todd Young [IN]), we took the amount of money spent during the “Primary” phase as noted in FEC reports and subtracted that amount from the OpenSecrets’ totals.

To see a PDF version of this memo, please click here.


Appendix I: States Ranked by Dem Efficiency Score

Appendix I: States Ranked by GOP Efficiency Score