Cracking the Creative Code

Experience innovation

A Standard of Creative Excellence

In the first study of its kind, debuted at the Cannes Lions Festival of Creativity 2016, Razorfish data scientists and Contagious analyzed 15 years of entries and awards from the industry’s most prestigious festival of international creativity to reveal the patterns, attributes and secrets behind the world’s top creative performers. While some findings are what you’d expect, many will surprise you.

The Cannes dataset reveals a high standard of creative excellence. In our 15-year analysis of the Cannes archive, the overall probability of winning a Lion at its International Festival of Creativity is just 4 percent, demonstrating how high the standard of creative excellence is at this event. (See detailed findings for a further breakdown of the statistical probability of winning at each level: Bronze, Silver, Gold, and Grand Prix). 

Key findings

  • High creative performance is not budget-dependent. Entries with the largest budgets performed no better than brands that lack such advantages.
  • Award winning creative is not location-dependent. The world’s wealthiest countries had no measurable advantage in the ability to deliver award-winning work.
  • High creative performance trends toward multidiscipline teams. Larger, collaborative, multi-disciplinary teams tend to outperform smaller ones.
  • Long-term relationships pay off. Long-enduring client-agency relationships have a win rate twice that of the average. 
  • A distinct code of conduct is consistently revealed. There are distinct behaviors behind the world’s top, award-winning creative work (that any organization can adopt).

Introduction

The challenge of measuring creative performance

When marketing executives set out to select an agency, they typically build a scorecard to quantify an agency’s financial performance, size, client base, and other attributes—to objectively align its capabilities to the marketing organization’s requirements. But when it comes to measuring creative performance, the exercise turns far more subjective. 

To bring objectivity to the task of gaining greater insight into the nature of creativity, we looked at the characteristics of high-performing creative work, through the lens of data science to see what data could tell us empirically about the nature of creativity. Such a study had never been conducted, and the capabilities have never been available, until now.

Is there a code for creativity, and can we crack it? Not exactly. But data science does reveal a consistent code of conduct behind top creative performers.

Scope of Analysis

Internal data: Cannes archive

Billions of data points from the Cannes Lions archive were analyzed from the past 15 years, covering the 2001-2015 timeframe, consisting of:  

  • 403,528 total submissions
  • 294,252,513 words describing the submissions
  • 261,697 distinct individuals credited for contributing to the entries 
  • 39,804 agencies/companies (that have contributed to entries)  
  • 123 distinct countries represented
  • 14,934 prize-winning submissions

External data: macroeconomics, industry revenue and media-spend measures

  • We also added several external data items to the scope of analysis to broaden our understanding of creative performance, including: 
  • GDP data by country and by year from the World Bank
  • Cross-referenced Ad Age Data Center for media spending by top Global Marketer
  • Cross-referenced Ad Age Data Center for agency rankings and revenue data 
  • A Python package that predicts gender probability based on first name and country

Data extraction

The Cannes archive site is built using Angular JS for dynamic content delivery; hence the site structure is not conducive to standard site content-scraping approaches. We needed a custom method to extract data from the website. After some reverse engineering of the site structure, including structure of the site’s JSON data, we developed a programmatic approach to extract the data associated with each entry, leveraging a back-end API used to present the site content when visitors view an individual entry submission.

Execution of this approach required hundreds of lines of custom Python code written to find, extract and save approximately 100 million lines of entry meta-data from the archive into text files for analysis.  

Tools

Several commercially available tools were used for the analysis, including:

  • Web monitoring tools
  • Chrome web developer tools
  • iPython with various external Python libraries
  • Tableau for data discovery and visualization during analysis
  • SQL server for data transformation
  • Google BigQuery for analysis

No assumptions going into the study

This task was conducted purely as an experiment, hence no preconceived assumptions were made going into the exercise. Though we did have a hypothesis, we weren’t sure the data would reveal anything significant. The final analysis however did indeed uncover the patterns, attributes, and secrets behind work that has been recognized over the past 15 years for its superior creative quality.

Win rates and primary measure of creative performance

Win rates (number of wins/number of entries) was established as our primary measure of creative performance. Win rates were also used to derive other, supporting creative performance measures. At the aggregate level, win rate data over the past 15 years revealed:

  • The statistical probability of winning a Cannes Lion Grand Prix was 0.07 percent; Gold 0.78 percent; Silver 1.13 percent; Bronze 1.7 percent. The overall probability of winning a Cannes Lion is just 4%.
  • The countries with the highest win rates (in order) for the 15-year period analyzed were New Zealand, Argentina, and France.
  • The most awarded competitions: radio/entertainment, press/travel, outdoor/toiletries.

Influence of cost and budget

Do bigger budgets produce higher creative performance? It’s a question we often hear from creative teams who claim: “A lack of budget negatively impacts the ability to deliver great creative work.”

To study this assertion we compared an external dataset from AdAge (detailing the media expenditures of the world’s top 100 spenders) against the win rates of these same organizations. We found no correlation between the highest media spenders and the likelihood of winning an award.

Do the highest creative performers reside in the world’s wealthiest countries? We plotted the GDP per capita (a widely accepted measure of a country’s wealth) of the top 100 richest countries against the award win rate (our measure of creative performance).  

No correlation was found between wealth on a national level and creative performance. When the top 10 countries are analyzed in terms of win rate, we find that only three of these are in the top 10 for GDP per capita. In fact, 70 percent of the most creative countries are those with lower levels of wealth. 

People, relationships, and time

If big budgets and access to rich resources don’t play a significant role in creative performance, what does? Our hypothesis was that superior creative performance comes down to people, specifically how people work. We used keyword search and metadata to sort through nearly half a million submissions over the last 15 years to spot where a client and agency appear together. This let us map all the work they’d submitted to let us look at its creative performance. The results reveal a compelling pattern relative to client/agency relationships and creative teams.

The one-year climb; the three-year high. A study of the first few years of a client/agency relationship reveals a short-term climb, followed by a drop in the pair’s number of submissions (versus win rates). This doesn’t suggest the two organizations are no longer working together, rather they are not producing winning work. But we also found that creative performance from a client/agency relationship peaks at years two and three. This makes sense, given the time required for an agency to learn a client’s business to get a sense of its nuances and challenges, but also how to get great, brave work signed off by the right people. It takes clients time to trust the agency and give them the freedom and creative license to exert its best creative work. 

The 10-year payoff. The data clearly shows win rates generally fall after year three with joint client/agency submissions. Again this is win rate; hence they are still submitting, but not winning, suggestingthat winning one year does not necessarily help you understand what it takes to win again. This does not mean that the client and agency go their separate ways and aim for a repeat of another three-year high with someone new. What the data does reveal is that client-agency partnerships not only perform better after the 10-year mark, they perform higher throughout the entire duration of the partnership. In fact, those client/agency relationships that last past the 10-year mark actually have a win rate that is twice that of the average (See Graph 1). 

Team dynamics 

The role of team makeup. Finally, we looked at team composition to see what the data might reveal in terms of the best combination of people working on an engagement. This required us to examine who was credited in wins, being careful where and how we extracted the data, given that Cannes constantly evolves its policies on crediting.

Larger teams deliver higher creative performance. Winning entries had +26 percent more people credited than non-winning entries. This suggests the importance and role of collaboration in creative performance. It could also support a broader point about culture, which is that, regardless of how many people worked on a campaign, organizations that credit, champion, and value their teams are more successful.  

Cross-discipline teams outperform teams of less diversity. When comparing winning entries to the rest, we found significantly higher levels of involvement from several key, supporting disciplines, including production, planning, technology, PR, and strategy. The rate of representation from these disciplines was up to 50 percent higher in winning entries than in entries that did not win. Supporting disciplines can be key in ensuring high-quality execution of creative ideas. Representation here means at least one team member from the  discipline was involved.

Empowered people deliver higher creative performance. We also found that those submissions that had a larger share of credited staff—below director level—were more successful. Overall, winning submissions have 4 percent lower share of senior staff credited when compared to the rest. This finding is consistent with observations made by Contagious during 2016, where executives found higher creative performance in environments that made moves to dilute, even kill, the classic “rock star” culture.

Agency/client relationships that last past the 10-year mark have win rates twice the average.

Cracking the Creative Code Graph.

Conclusions and 
Recommendations

Data science converts subjectivity to objectivity 

Given the extraordinary advances in computing power, the extreme availability of information, and the maturation of advanced analytics and algorithms, data science techniques have extraordinary potential to unearth truths that could never be found through manual (human) computation. Perhaps the greatest contribution of data science is the ability to replace hunch with fact. 

But recent breakthroughs in the underlying technology of data-driven marketing should not suggest that science has triumphed over art. What these advances do signal, is the ability to apply fact-based analysis to the secrets behind great creative work. When data science was applied to 15 years of award winning work from Cannes Lion, we discovered that high creative performance

  • Is not budget-dependent.
  • Is not bound by geography or location
  • Trends toward engaging multi-disciplinary teams
  • Illustrates the effectiveness of long-term relationships
  • Is underscored by a distinct code of conduct which any organization can adopt

Invest in long-term relationships

Don’t expect your best creative work to be delivered instantly. Client teams and their agencies should be aware of the three-year cliff of complacency and keep pushing each other to do great work. They should also recognize: it’s the long, established creative relationships that are proven (by data science) to be the highest performing over time. 

Adopt a code of conduct for creativity

While the code to creativity was not cracked in our study, our findings (informed by win rates and other observations) reveal a set of behaviors, or code of conduct, behind high creative performers. One of our strongest observations reveals that modern creativity demands collaboration, especially as we enter a connected economy where brand experiences are dependent upon business strategists, artists, technologists, and storytellers.

Empower Your Teams

Our code of conduct also revealed that multidiscipline teams, empowered by senior management to make their own creative decisions, also trend toward higher performance. Empower those who have demonstrated the capacity to handle responsibility. Create an environment where people are encouraged to grow their skills. Finally, give people autonomy and discretion over their assignments and resources.

Interesting facts 

  • Top three countries with highest win rates that have entered a handful of times: Kosovo, Macedonia, Kuwait.
  • Top three countries with highest win rates that have entered thousands of times: New Zealand, Argentina, France.
  • The most awarded names: Marcello (male); Veronica (female; which outperformed Marcello in the aggregate analysis).

Most highly awarded sections:

Outdoor (for Toiletries)

Press (for Travel)

Radio (for Entertainment)

  • For every submission entered, your chance of winning a Bronze is 1.70%; Silver 1.13%; Gold 0.78%; Grand Prix 0.07%. 
  • There is an overall 4% probability of winning a Lion for every submission entered.

Power in numbers

  • Award-winning work has 26% more team members credited.
  • 70% of the world’s most creative countries are not the wealthiest.

Creativity is not dependent upon:

  • Big budgets

  • Big agencies

  • Strong economies

But — superior creative demands a serious investment. 10-year+ client/agency relationships experience 2x the average win rate.

Women creative executives are underrepresented

  • 11% of creative directors are women
  • 9% of executive creative directors are women
  • 8% of chief creative officers are women

Related Content

Locations

34 Offices. 5 Continents.
1 Vision.