Mixpanel: Social Game Developers Use Tutorials to Get Crucial Early Retention

[Editor's Note: Facebook may be making significant changes to the platform, but developers aren't standing still -- they're busy continuing to find new ways to engage users. Here's a guest post by Tim Trefren, cofounder of real-time analytics company Mixpanel, looking at the latest trends his company is tracking among developers.]

Because a range of our customers are social game developers, we can get a high-level look at trends they’re seeing in their Facebook applications. One of the big trends we’re seeing is that games are using tutorials to generate strong retention among new users. A related trend is that this initial retention is critical to the health of your game, in the weeks following launch. Here’s a closer look.

Impressive Results From Tutorials

One thing we’re seeing succeed is the tutorial-based signup process. A well-crafted tutorial removes all the ambiguity out of getting started and helps teach a new user how to play the game.

If you’re not familiar with this technique, the FarmVille signup process is a good example. FarmVille explicitly teaches you how to harvest, plow, and plant seeds with a 3-step tutorial.

Now that you’re familiar with the concept, let’s take a look at the data I’ve compiled from a number of games.

By The Numbers

The most impressive finding of this analysis is that individual steps in a tutorial convert at over 90% on average. Meaning, once a user has started a tutorial, they have a greater than 90% chance of continuing at each step.

This doesn’t include the first step, however – as you might expect, it’s harder to get users to start a tutorial than it is to get them to complete additional steps.

First step conversion rate: 71.4%
Additional step conversion rate: 95.06%
Overall completion rate: 37.9%

Many companies are now utilizing the tutorial technique, and it clearly deserves its popularity. Conversion rates of 95% are practically unheard of, but tutorials appear to be delivering these results.

An Interesting Trend in Visitor Retention

Another thing I noticed was a strong trend in retention behavior. There are some remarkable similarities in the *pattern* of visitor retention across games, despite the differences in the actual numbers.

Before I go any further, here’s a quick overview of the concept: Visitor retention is the percentage of visitors who come back and interact with an application after their first visit.

Visitors are chunked into groups—also known as ‘cohorts’—and then analyzed based on the the behavior of the group as a whole. The most common method is to group by visit date. For example, one group might consist of all the visitors who were first seen in the week starting May 3rd.

Once you have grouped your visitors, you can track them over the following weeks and see how many from each cohort return to the site.

Now let’s look at some actual retention numbers for a variety of different games. To compile this data, I first took a sample of the different social games using our service. Then I looked at the average week-over-week retention for each game.

Here’s a graph of the average weekly retention rates for the different games:

You can see that on the surface, the retention numbers are pretty different – some of these games have long-term retention rates close to 50%, while others rapidly approach 0%.

However, the interesting thing to note is that while the absolute retention rates are different, the pattern of retention is very similar across games. They all have a massive dropoff after the first week, with relatively flat retention in the following weeks. If you take a closer look, the ‘flat’ parts of the graph run nearly parallel, meaning they have very similar weekly conversion rates.

We can take a closer look by calculating the “conversion rate” – (e.g. week 3 divided by week 2, etc) between adjacent weeks. Here’s a graph with this transformation:

See a pattern? At the first point on the x-axis (Week 0-1), we can see that the initial conversion rate ranged from 1.76% on the low end to 62.83% on the high end. The interesting part comes later, though – no matter what the initial conversion rate between weeks 0 and 1, the following weeks convert at close to 80% across all of the games.

Basically, this means that once you’ve had a user for at least a week, they have an 80% chance of coming back each following week.

This suggests that your initial retention rate is critical, because once you’ve retained users for a week you are likely to keep them for quite a while. This behavior also raises another question: why do almost all of the games in our sample exhibit this behavior? Is it possible that this is just how social games work – retained users have an 80 – 95% chance of returning each week? If so, this could mean that the only thing you have control over is the initial retention rate. Time to write and polish your tutorials.

Tim Trefren is the cofounder of real-time analytics company Mixpanel.

Inside Bitcoins

Mediabistro Event

Explore the Future of Virtual Currency

Discover why countless investors and businessmen, including the Winklevoss twins, are becoming big supporters of virtual currencies at Inside Bitcoins on July 30 in New York. You’ll hear from speakers like Charlie Shrem, Vice Chairman at Bitcoin Foundation, who runs one of the largest alternative payment companies. Every paid registrant will receive a Bitcoin paper wallet with 0.01 Bitcoin. Register early and save.

Leave a Reply

13 Responses to “Mixpanel: Social Game Developers Use Tutorials to Get Crucial Early Retention”

  1. Peter says:

    125%+ conversation rate? What is that?!?!

  2. Brian says:

    Peter, that can mean double + 25% the previous conversion rate.

  3. Aki says:

    Useful post – for a related topic, please check out my feature on social game tutorials at Gamasutra:

    http://www.gamasutra.com/view/feature/4327/first_five_minutes_how_tutorials_.php

  4. Kenny says:

    Excellent article. Just excellent

  5. The Early Game « Online Alchemy says:

    [...] get in the way of getting to enjoy a game quickly, they appear to work: Inside Social Games recently reported on some results from Mixpanel, saying that if you can get someone started on a tutorial, [...]

  6. This Week’s Headlines on Inside Social Games says:

    [...] Mixpanel: Social Game Developers Use Tutorials to Get Crucial Early Retention [...]

  7. Social Game Developers Use Tutorials to Get Crucial Early Retention « Mixpanel says:

    [...] Comment! This post was originally written by Tim Trefren on InsideSocialGames. [...]

  8. ngmoco:) | Games › Touch Pets Cats › Tutorial Focus says:

    [...] to Touch Pets Cats.  The science of tutorials is evolving, and the evidence is in – a good tutorial helps more people enjoy the game. So, I play the first five minutes of the game over and over and over and over again.  Whew! [...]

  9. Heidi Wolpert says:

    I really love Touch Pets Cats but I don,t know how many times I managed to log on and after I accumulate almost 800 points I am “out of the game” and have to start allover again from scratch. To make matters worse, now I am always told: My emailaddress is already taken. How do I ever enjoy my little virtual kitties again?…Heidi

  10. Financial Forecasting for Social Games (pt 2) « PlotLuck Games says:

    [...] this blog post from InsideSocialGames, Tim Trefren, from the analytics company Mixpanel, shows that after the initial drop-off, most [...]

  11. Quora says:

    What are good Day 2 and Day 7 retention rates for a Social Game on Facebook?…

    Check out the following article for some interesting insights. http://www.insidesocialgames.com/2010/06/29/mixpanel-social-game-developers/

  12. Why I’ll quit playing your mobile game | Sophia Perl of Wisdom says:

    [...] beginning.  I get frustrated when I do not know what to do next and will stop playing instantly.  Some say that tutorials will convert users to the next step 90% of the time.  Don’t waste your [...]

  13. Quora says:

    What’s the retention rate for a game like Farmville?…

    I think this blog post does a good job of describing what the retention rate is of a game like this is. http://www.insidesocialgames.com/2010/06/29/mixpanel-social-game-developers/

interested in advertising with inside social games?

Social Media Jobs
of the Day

Digital Media Production Assistant

Comedy Central
Santa Monica, CA

Marketing/Social Media Assistant

Education Social/Learning Network
Princeton, NJ

Social Media Manager

Comedy Central
Santa Monica, CA

Featured Company

Join leading companies like this one and recruit from the nation's top media job seekers on the Mediabistro Job Board. Every job post comes with our satisfaction guarantee. Learn More
 

Our Sponsors

Also from Inside Network:   AppData - Facebook & iOS Application Stats   PageData - Engagement Data on Facebook Pages   Facebook Marketing Bible   Inside Network Research
 
home | site map | advertising/sponsorships | about us | careers | contact us | help courses | browse jobs | freelancers | events | forums | content | member benefits | reprints & permissions about | terms of use | privacy policy Mediabistro Inc. call (212) 389-2000 or email us