If Slow and Steady Wins the Race Hillary May be on Track to 2016 Democratic Presidential Nomination

If you don’t follow politics, you might not have noticed that there are actually two parties with candidates vying to become their party’s nominee for the general election.

Well, the Democrats have just held their first debate, and like the Republicans, we’ve been monitoring and tracking the performance of each candidate’s web site home page. Luck for us, there are only five significant democratic candidates.

The diagram below shows the web end user response time for each of the candidate’s web sites for the week leading up to the start of the debate:

Screen Shot 2015-10-13 at 3.54.14 PM.png

As you can see from the chart, while there is some variation in the response time of each of the candidates’ websites (along with an occasional spike now and then) the average is fairly consistent. Bernie Sanders generally had the fastest site, though he made a change the day before the debate between 4 and 6 pm that somewhat increased his response time such that he dropped to second place Lincoln Chafee. Meanwhile, Hillary Clinton had the slowest responding web site.

For each web site, we measure over 10 different performance metrics of the home page, and we measure them for different network speeds including 3G, 4G, and Cable. The image below shows a table, for example, of eight of the key performance metrics measured at 4G network speeds just after the debate for the time period from 4-9 pm:

Screen Shot 2015-10-16 at 5.35.04 PM.png

We sort the sites according to the Speed Index metric which is a measure of how quickly the user will perceive the site to be, as it is computed as a ratio of the percentage of content rendered over time, which basically means that if two sites have similar Visually Complete times, the sight that renders more content sooner with be perceived to be faster than a site that renders a greater portion of its content later.

Some other interesting data points are that Martin O’Malley’s site has by far the biggest data load at 3.68 MB, but is the most efficient in the number of Resources Loaded per page view yet has the second fastest Speed Index and Visually Complete time, which is what really matters from an end user perspective. Meanwhile Hillary Clinton had the second highest data load at 2.78 MB, but was second slowest with a Speed Index of 8.5 and a Visually Complete time of 10.2 seconds. She also has the most in-efficient site coming in at 101 resources loaded. Jim Webb at the slowest performance despite having the least data at 0.71 MB and the most efficient resources loaded at 40, yet had a Speed Index of 11.2 seconds and a Visually Complete time 11.6 seconds.

The graphs below show the differences in performance at 3G and 4G network speeds between the period of the week leading up to the debate vs. the period of the debate:

3G Performance Dem Debate.png4G Performance Dem Debate.png

A journalist we spoke with could not believe that Hillary’s web site could be so slow and proposed that the data must be because she was supposedly getting so much traffic to her web site that it caused it to slow down and that if we looked at the data 3 am on a non-debate night that it would be much better.

Well, that’s a load of bunk and explains why he’s a political journalist and not a data scientist or web performance engineer. While it is certainly possible that a site could receive so much traffic that the infrastructure couldn’t handle it and would therefore slow down, the fact of the matter is that performance is more a function of the design of the web site and the architecture of the application serving it.

The fact of the matter is that we are measuring the web site performance of the candidates continuously at 30 minute intervals. As a result, it is easy to see that the performance over time is consistent (with minor variations) and is not so much affected by traffic, but perhaps occasionally when changes are being made to the content or other system factors. This is shown by the follow graph which shows several performance metrics of Hillary’s web site over the course of an entire week:

Screen Shot 2015-10-16 at 12.05.03 PM.png

The metrics shown in this graph (from bottom to top) are Hilliary’s First Byte time (blue), First Render time (aquamarine), Speed Index time (red), Visually Complete time (purple), and Complete Load time (green).

As is clearly shown by the graphs, while there is some variation, the general level of each of the metrics is quite consistent over time with little or no difference between peak traffic times and dead of night.

All of the candidates have many competing goals and objectives for their web sites. It’s a way of communication to supports and attracting new ones, it’s also a vehicle for raising funds, responding to new developments and other candidates, and dealing with the media. If a site doesn’t perform well, then the success of the site in achieving those goals could be affected. Regardless, it doesn’t help to have a speedy site if the content and features aren’t there for the intended audiences.

This is the delicate balance that all business and content providers must manage with web sites: understanding what the key business performance indicators for your site are and making sure that the technical performance of your site is helping you to achieve those objectives.

In order to accomplish that, you need web site performance monitoring and management technology that helps you to keep an eye on the technical performance, resolve issues, quickly, and correlate technical performance to your business performance via analytics and application intelligence.

AppDynamics offers a complete Application Intelligence platform to help you manage that balancing act. Let us know how we can help you, too.

Trump Still on Top of GOP Candidate Website Performance after Second Debate

Over the course of the 2016 presidential election, we are synthetically monitoring the performance of the major candidates’ website home pages. Each month and for specific occasions like debates, we are producing an index from those measurements to see how performance changes over time and for some of these specific events like last week’s second GOP debate hosted by CNN.

We look at the data over the course of the week leading up to the day before the event, and then again immediately before and after the debate to see if there are any significant changes from the longer-term trend. These differences could be the result of changes the candidate may have made specifically for the event (as Bush and Christie appear to have done for both debates so far), or perhaps as a result of changes in load that may occur during these high-profile events, possibly driving more traffic to the sites due to the attention the event receives from the media and the public (22 million viewers).

We measure both desktop performance from our cloud-based agents and mobile performance via 3G and 4G network speed profile emulation, since many viewers will frequently use their laptops or mobile devices at the same time as they are watching the event live on TV or streamed to one device or another.

For the second GOP debate, Donald Trump’s website performance continued to lead the pack both on desktop and mobile, and performance mirrored other trends that we observed during the first debate. For example, for the week prior to the debate, Trump’s desktop web site averaged an end-user response (EUR) time  of 2.9 seconds, and a fully loaded response time of 4.6 seconds, whereas John Kasich, the next closest website, had EUR of 5.9 seconds and a fully loaded time of 6.1 seconds. Meanwhile, towards the bottom of the pack, Carly Fiorina’s website had an EUR of 14.5 seconds and a fully loaded time of 16.1 seconds, and in last place was the Jeb! Bush website with an EUR of 17.9 and a fully loaded time of 18.2 seconds.

Screen Shot 2015-09-23 at 4.59.11 PM.png

Of course, these results are not surprising when we consider the other factors of a website’s design, such as the fact that Trump’s website is comparatively lightweight, coming in at only 0.89 MB of data downloaded for 36 elements, compared to 6.02 MB and 118 elements for Fiorina, and Bush’s website weighing in at a whooping 8.3 MB and 126 elements.

This data is further nuanced when you also look at the first-render times, in which case Bush actually was fastest at 1.9 seconds. So at least some content is starting to render quickly (faster than Trump at 2.8 and Carly at 3.4), even though it takes a very long time for the complete page to load.

These parameters reflect very conscientious design decisions and tradeoffs that are being made by the respective campaigns and their web design and content teams, given the many conflicting and contradictory goals and objectives for the candidate’s sites. For example, the websites are not just vehicles for educating prospective voters about a candidate’s positions and policies, but also about creating an emotional bond to the voter and also, and in some cases most importantly, about raising funds and campaign contributions, as witnessed by Scott Walker’s recent withdrawal from the campaign due to lack of cash to pay his extensive staff.

So the campaign websites have to balance the need to provide a content- and media-rich experience (that will be large in data and could take a long time to load), against the need for snappy performance to quickly engage the voter and encourage them to contribute. This dichotomy applies equally to political sites, as it does to retail or content sites (as I recently discussed in another blog on the catch-22 between content and performance https://blog.appdynamics.com/mobile/catch-22-revenue-content-performance/).

For example, according to Pew Research (http://www.pewinternet.org/2012/10/25/presidential-campaign-donations-in-the-digital-age/), 50 percent of 2012 campaign contributions were done online, and that number is only likely to increase as as the population has become increasingly comfortable with online transactions in the intervening years, and are spending more time both online and especially on their mobile devices (see Mary Meeker’s KPCB 2015 Internet Trends report). Furthermore, companies like Twitter have just released a one-click contribution widget, to send contributions to candidates (think one-click checkout for politics).  The websites (and experience) are not just about engagement; they’re also about raising a significant amount of cash. So, it’s no wonder that candidates pay a lot of attention to their online assets  today.

In order to address this conundrum, some candidates are actively changing their web sites at key times such as the recent debate to take advantage of the increased exposure at those times to vastly simplify and speed up their websites.

This strategy was pursued aggressively by Bush, Christie, and Jindal, as the data from immediately after the debate for the previous three hours shows. Bush decreased his EUR time by over 10 seconds, from 17.9 to 7.3 seconds, while slimming down his homepage by over 4X from 8.3 MB to 2.0 MB. Christie cut five seconds, while reducing his homepage by nearly almost 3X from 5.7 MB to 2.4 MB. And Jindal sped up by two seconds, also reducing his home page to a third of its pre-debate size, from 3.45 MB to 1.1 MB.

Screen Shot 2015-09-23 at 5.49.22 PM.png

Some candidates, however, didn’t quite get the message, and their website performance actually went in the opposite direction prior to and during the debates. In particular, Marco Rubio’s performance degraded by three seconds, as his EUR time went from 6.7 to 9.7 seconds, when he nearly doubled the data on the homepage from 2.45 MB to 4.4 MB. But Huckabee was the biggest performance offender, adding ten seconds to his EUR, going from 8.1 to 18.4 seconds, and nearly tripling the size of his homepage from 2.6 MB to 6.25 MB.

The graph below shows the Huckabee increase a little more than a day before the debate and the Christie decrease right before the debate started.

Screen Shot 2015-09-16 at 10.16.46 PM.png

Similarly, in this graph below we can see the Bush decrease and the Rubio increase as a result of changes they made to their websites just two hours before the start of the debate.

Screen Shot 2015-09-16 at 10.15.34 PM.png

In conclusion, whatever the goals of your website are, you need to understand how performance is inextricably linked to your design and content strategy, and you need tools to help you have insight into and measurement of how performance drives your KPIs.

At AppDynamics, we can help.

How We Synthetically Monitored the Republican Presidential Debate

Last week marked the first (of many) Republican presidential debates leading up to the 2016 election. All politics aside, we at AppDynamics were eager to see the investment these candidates made in their digital platforms, and how response time would be affected throughout the debate. As we saw with President Obama’s campaign efforts, modern-day candidates will substantially use their digital web presence for fundraising and branding efforts. A seamless website with quick performance becomes vital during peak periods.

To test these candidates’ websites, we put the new AppDynamics Synthetic Monitoring tool to gauge their loading speed times, and ultimately, the end-user experience.

The Test Methodology

We measured the main homepage of the websites for each Republican candidate every 30 minutes from two locations (VA and OR) using the Chrome browser for the full week leading up to the debate, ultimately averaging the response times to get a baseline. This initial controlled baseline would serve as the benchmark to show how exactly the debate affected the websites’ response times.

Starting at the beginning of the debate on August 6th and continuing every five minutes throughout the duration of the debate, our synthetic agents tested the end-user response time, first render time, fully loaded time, number of elements loaded, and the “page weight” (size in MB) of the fully loaded website.

Since we didn’t know exactly which candidates would be participating in the debate until about a day prior, we measured all of the typical suspects, even the ones who ended up not participating.

Here are the websites we monitored:

The Results

To save you from an eyesore dashboard graphic, we broke candidates into groups of five so the results become more visible.

Here are the results from Jeb Bush, Marco Rubio, Donald Trump, Scott Walker, and Ted Cruz:

Here are the results from Lindsey Graham, Ben Carson, Chris Christie, Mike Huckabee, and Rand Paul:

Here are the results from Carly Fiorina, John Kasich, Rick Perry, Bobby Jindal, George Pataki, and Rick Santorum:

Analysis

For quick comparisons sake, we created this easily-digestible graphic comparing the participating candidates’ websites before and after the debate, using end-user response time as the main metric.

As you can see, Donald Trump’s website fared the best both before the debate and after. Jeb’s website launched some major changes just prior to the debate, resulting in the biggest mover and success story. Our original hypothesis was the traffic would spike, resulting in higher response times. However, Jeb was able to actually lower his response time with the changes. After analyzing the data a bit more, we can see the decrease in response time was because the new page design decreased the total load from 7.8 MB to just over 1 MB.

Chris Christie was also able to significantly lower his response time, dropping from 9.5 to 6 due to changes in the page design. Analyzing the data further shows he actually reduced the entire page load from 6 MB to 2.2 MB. Our resident mobile expert, Peter Kacandes, states that on average each MB of page weight costs about 6 additional seconds in response time.

In the diagram below, you can see the time series data of the Bush (blue line) and Christie (green line) homepages showing the size of the homepage. While Christie’s team changed their homepage to reduce the size a few days before the debate, the Bush web team made the change to their homepage right before the debate and then changed it back to the regular design shortly after the debate was over.

Screen Shot 2015-08-07 at 12.25.15 PM.png

What This Means Moving Forward

It’s tough to make sweeping predictions or allegations from a basic test of website loading times, however, it’s important to note a few growing trends where digital performance plays a major role.

Fundraising

What Obama pioneered during the 2008 election is that by correctly using social media, Facebook in particular, and maintaining a viral web presence, you can raise a lot more money than the traditional methods. I won’t say Donald Trump will lead in fundraising simply because his website loads the fastest, but I will say that website performance is key to gaining money. If a page takes too long to load, stalls, or crashes, a would-be donor will likely become annoyed and possibly change their mind. In this respect, campaign fundraising could actually learn a lot from e-commerce best practices where their application performance is directly tied to their revenue.

Release Cycles and Preparedness

As we saw with Chris Christie and Jeb Bush altering their website to be more lightweight in times of heavier traffic, other candidates should take notice and do the same — looking at you Scott Walker and Ben Carson.

To appease user experience and satisfaction, a faster website could provide dividends during peak times such as future debates or even primary election nights. For candidates, brand value is of the utmost importance, and it’s important to understand the balance between providing a lot of information and providing a good user experience without risk it with slow website load times.

Full Results

Before the Debate

Candidate Web Site

(Ranked by End User Response Time)

End User Response Time (s)

First Render (s)

Fully Loaded (s)

Elements Loaded

Complete Load (MB)

www.donaldjtrump.com

2.2

2.1

14.8

95

2.10

www.marcorubio.com

3.9

3.8

5.6

75

2.30

www.johnkasich.com

5.8

4.4

6.7

62

2.70

www.mikehuckabee.com

6.2

3.7

6.3

105

1.02

www.randpaul.com

8.3

2.0

12.4

117

5.10

www.tedcruz.com

8.7

3.4

9.8

107

2.20

www.chrischristie.com

9.5

2.3

14.2

96

6.00

www.bencarson.com

11.2

2.9

12.5

144

1.50

www.jeb2016.com

12.7

1.3

14.1

81

7.80

www.scottwalker.com

14.1

6.0

17.2

134

3.50

           
           

OUT:

         

www.ricksantorum.com

5.5

3.9

6.1

56

1.05

www.georgepataki.com

5.5

2.5

5.7

68

1.80

www.bobbyjindal.com

5.7

4.6

11.1

88

1.70

landing.lindseygraham.com

8.2

4.7

10.3

97

2.20

www.rickperry.org

11.7

4.2

19.5

105

9.30

www.carlyforamerica.com

14.7

2.9

15.9

120

5.90

After the Debate

Candidate Web Site

(Ranked by End User Response Time)

End User Response Time (s)

First Render (s)

Fully Loaded (s)

Elements Loaded

Complete Load (MB)

www.donaldjtrump.com

2.3

2.2

14.7

96

2.10

www.georgepataki.com

5.1

2.6

5.2

66

1.81

www.ricksantorum.com

5.2

3.8

5.6

53

1.03

www.jeb2016.com

5.9

1.6

6.1

62

1.04

www.chrischristie.com

6.0

2.0

8.8

80

2.20

www.marcorubio.com

6.3

3.8

8.2

82

3.80

www.johnkasich.com

6.5

4.8

7.9

73

2.70

www.mikehuckabee.com

6.6

3.2

6.9

113

1.61

www.bobbyjindal.com

6.7

4.6

13.2

127

3.00

landing.lindseygraham.com

7.7

4.7

9.6

95

2.11

www.tedcruz.com

8.3

3.8

9.4

106

2.20

www.randpaul.com

8.4

2.6

10.6

131

3.40

www.rickperry.org

10.6

4.1

17.9

107

8.65

www.bencarson.com

12.8

2.9

14.4

146

1.50

www.scottwalker.com

13.5

6.3

16.2

131

2.74

www.carlyforamerica.com

14.9

3.0

15.9

119

5.94

Interested to see how AppDynamics Synthetic Monitoring can improve your website appearance? Check out a FREE trial today!