Internal data: Cannes archive
Billions of data points from the Cannes Lions archive were analyzed from the past 15 years, covering the 2001-2015 timeframe, consisting of:
- 403,528 total submissions
- 294,252,513 words describing the submissions
- 261,697 distinct individuals credited for contributing to the entries
- 39,804 agencies/companies (that have contributed to entries)
- 123 distinct countries represented
- 14,934 prize-winning submissions
External data: macroeconomics, industry revenue and media-spend measures
- We also added several external data items to the scope of analysis to broaden our understanding of creative performance, including:
- GDP data by country and by year from the World Bank
- Cross-referenced Ad Age Data Center for media spending by top Global Marketer
- Cross-referenced Ad Age Data Center for agency rankings and revenue data
- A Python package that predicts gender probability based on first name and country
The Cannes archive site is built using Angular JS for dynamic content delivery; hence the site structure is not conducive to standard site content-scraping approaches. We needed a custom method to extract data from the website. After some reverse engineering of the site structure, including structure of the site’s JSON data, we developed a programmatic approach to extract the data associated with each entry, leveraging a back-end API used to present the site content when visitors view an individual entry submission.
Execution of this approach required hundreds of lines of custom Python code written to find, extract and save approximately 100 million lines of entry meta-data from the archive into text files for analysis.
Several commercially available tools were used for the analysis, including:
- Web monitoring tools
- Chrome web developer tools
- iPython with various external Python libraries
- Tableau for data discovery and visualization during analysis
- SQL server for data transformation
- Google BigQuery for analysis
No assumptions going into the study
This task was conducted purely as an experiment, hence no preconceived assumptions were made going into the exercise. Though we did have a hypothesis, we weren’t sure the data would reveal anything significant. The final analysis however did indeed uncover the patterns, attributes, and secrets behind work that has been recognized over the past 15 years for its superior creative quality.
Win rates and primary measure of creative performance
Win rates (number of wins/number of entries) was established as our primary measure of creative performance. Win rates were also used to derive other, supporting creative performance measures. At the aggregate level, win rate data over the past 15 years revealed:
- The statistical probability of winning a Cannes Lion Grand Prix was 0.07 percent; Gold 0.78 percent; Silver 1.13 percent; Bronze 1.7 percent. The overall probability of winning a Cannes Lion is just 4%.
- The countries with the highest win rates (in order) for the 15-year period analyzed were New Zealand, Argentina, and France.
- The most awarded competitions: radio/entertainment, press/travel, outdoor/toiletries.
Influence of cost and budget
Do bigger budgets produce higher creative performance? It’s a question we often hear from creative teams who claim: “A lack of budget negatively impacts the ability to deliver great creative work.”
To study this assertion we compared an external dataset from AdAge (detailing the media expenditures of the world’s top 100 spenders) against the win rates of these same organizations. We found no correlation between the highest media spenders and the likelihood of winning an award.
Do the highest creative performers reside in the world’s wealthiest countries? We plotted the GDP per capita (a widely accepted measure of a country’s wealth) of the top 100 richest countries against the award win rate (our measure of creative performance).
No correlation was found between wealth on a national level and creative performance. When the top 10 countries are analyzed in terms of win rate, we find that only three of these are in the top 10 for GDP per capita. In fact, 70 percent of the most creative countries are those with lower levels of wealth.
People, relationships, and time
If big budgets and access to rich resources don’t play a significant role in creative performance, what does? Our hypothesis was that superior creative performance comes down to people, specifically how people work. We used keyword search and metadata to sort through nearly half a million submissions over the last 15 years to spot where a client and agency appear together. This let us map all the work they’d submitted to let us look at its creative performance. The results reveal a compelling pattern relative to client/agency relationships and creative teams.
The one-year climb; the three-year high. A study of the first few years of a client/agency relationship reveals a short-term climb, followed by a drop in the pair’s number of submissions (versus win rates). This doesn’t suggest the two organizations are no longer working together, rather they are not producing winning work. But we also found that creative performance from a client/agency relationship peaks at years two and three. This makes sense, given the time required for an agency to learn a client’s business to get a sense of its nuances and challenges, but also how to get great, brave work signed off by the right people. It takes clients time to trust the agency and give them the freedom and creative license to exert its best creative work.
The 10-year payoff. The data clearly shows win rates generally fall after year three with joint client/agency submissions. Again this is win rate; hence they are still submitting, but not winning, suggestingthat winning one year does not necessarily help you understand what it takes to win again. This does not mean that the client and agency go their separate ways and aim for a repeat of another three-year high with someone new. What the data does reveal is that client-agency partnerships not only perform better after the 10-year mark, they perform higher throughout the entire duration of the partnership. In fact, those client/agency relationships that last past the 10-year mark actually have a win rate that is twice that of the average (See Graph 1).
The role of team makeup. Finally, we looked at team composition to see what the data might reveal in terms of the best combination of people working on an engagement. This required us to examine who was credited in wins, being careful where and how we extracted the data, given that Cannes constantly evolves its policies on crediting.
Larger teams deliver higher creative performance. Winning entries had +26 percent more people credited than non-winning entries. This suggests the importance and role of collaboration in creative performance. It could also support a broader point about culture, which is that, regardless of how many people worked on a campaign, organizations that credit, champion, and value their teams are more successful.
Cross-discipline teams outperform teams of less diversity. When comparing winning entries to the rest, we found significantly higher levels of involvement from several key, supporting disciplines, including production, planning, technology, PR, and strategy. The rate of representation from these disciplines was up to 50 percent higher in winning entries than in entries that did not win. Supporting disciplines can be key in ensuring high-quality execution of creative ideas. Representation here means at least one team member from the discipline was involved.
Empowered people deliver higher creative performance. We also found that those submissions that had a larger share of credited staff—below director level—were more successful. Overall, winning submissions have 4 percent lower share of senior staff credited when compared to the rest. This finding is consistent with observations made by Contagious during 2016, where executives found higher creative performance in environments that made moves to dilute, even kill, the classic “rock star” culture.
Agency/client relationships that last past the 10-year mark have win rates twice the average.