Autopilot demo on sample data

Why new players on Demo network leave in their first session, what to change, and which join address is slipping.

New players1,512

First joined Sep 9 to Oct 9

7-day return38.0%

574 of 1,512 came back

Biggest drop-off37.7%

Left within 10 minutes

Fixes to look at3

1 join address alert in the last 7 days

Drop-off

When new players leave in their first session.

Where new players drop off

First sessions of new players who first joined Sep 9 to Oct 9, so each had 7 days to come back. A player can sit in more than one group.

All new players

Left within 10 minutes

Their first session ended less than 10 minutes after they joined.

37.7% of 1,471 players judged, 554 in this group

This group13.7%
Everyone else53.0%

-39.3 pp 7-day return against everyone else

Never left spawn

Stayed within 16 blocks of where they started, or never left the spawn region, and made no move off the hub server in their first session. Older connectors cannot say how far, so there it means no advancement.

The hub is lobby, the lobby server set below.

15.3% of 1,490 players judged, 228 in this group

This group18.4%
Everyone else43.0%

-24.6 pp 7-day return against everyone else

Quit after dying

Died in their first session and quit within 5 minutes of that death.

8.8% of 1,460 players judged, 129 in this group

This group26.4%
Everyone else41.5%

-15.2 pp 7-day return against everyone else

Kicked

Kicked in their first session for being idle or by a plugin. Ban and whitelist kicks are left out.

3.4% of 1,490 players judged, 51 in this group

This group23.5%
Everyone else41.3%

-17.8 pp 7-day return against everyone else

By join address

Share of judged new players in each drop-off group, by join address, and their 7-day return against the rest.
Join addressNew playersLeft within 10 minutesNever left spawnQuit after dyingKicked
list.example.net49846.1%11.2% came back, 44.1% of the rest18.9%14.0% came back, 32.9% of the rest10.0%20.8% came back, 33.0% of the rest4.1%Not enough players yet
play.example.net33130.1%17.5% came back, 58.2% of the rest12.0%23.1% came back, 49.1% of the rest8.2%Not enough players yet3.1%Not enough players yet
tiktok.example.net26242.0%12.1% came back, 50.0% of the rest17.1%15.9% came back, 40.2% of the rest9.1%Not enough players yet3.5%Not enough players yet
yt.example.net14924.1%22.9% came back, 66.4% of the rest10.2%Not enough players yet6.9%Not enough players yet2.0%Not enough players yet
Discord11827.8%18.8% came back, 61.4% of the rest11.2%Not enough players yet7.9%Not enough players yet2.6%Not enough players yet

Join addresses

Which address players use is slipping.

Join address alerts

Each join address against its own previous 4 weeks, checked once a day. A drop that passes the rule is logged here and posted to Discord.

This week against the 4 before

7-day return of players who first joined Oct 2 to Oct 9, against Sep 4 to Oct 2.

7-day return by join address this week and in the 4 weeks before.
Join addressThis week4 weeks beforeChangeStatus
tiktok.example.net23.9% of 8845.0% of 402-21.2 ppDropped
list.example.net39.5% of 12441.4% of 498-1.8 ppSteady
play.example.net53.2% of 7953.8% of 301-0.7 ppSteady
yt.example.net65.9% of 4164.4% of 160+1.5 ppSteady
Discord54.5% of 2253.7% of 95Too few playersNot enough players yet

Alert log

Alerts sent for join addresses, newest first.
Join addressDayThat week4 weeks beforeDropDiscord
tiktok.example.netOct 15, 202623.9% of 8845.0% of 402-21.2 ppPosted
list.example.netSep 27, 202629.0% of 13141.8% of 512-12.8 ppPosted

An alert fires when the 7-day return of players from one join address falls at least 5 points under its own previous 4 weeks, each side holds 30 players, and a two-proportion z test puts the drop at p under 0.05. At most one alert per address a week. Checked once a day.

Fixes

What to change first, from the two groups above.

Suggested fixes

One per pattern that passed the rule, with the numbers behind it. Rule based and worded in advance.

  1. Shorten the tutorial or the walk from spawn

    Pattern: Left within 10 minutes

    37.7% of judged new players left within 10 minutes (554 players of 1,471). 13.7% of them came back within 7 days, against 53.0% of the rest. Read from first sessions and 7-day return.

  2. Make the way out of spawn obvious

    Pattern: Never left spawn

    15.3% of judged new players never left spawn (228 players of 1,490). 18.4% of them came back within 7 days, against 43.0% of the rest. Read from first sessions and 7-day return.

  3. Check what changed on tiktok.example.net: its server or its ad

    Pattern: Join address return fell

    23.9% of 88 players who first joined on tiktok.example.net last week came back within 7 days, against 45.0% of 402 players in the 4 weeks before. Read from first joins by address and 7-day return.

Rules

How Autopilot judges a drop and where it tells you.

Autopilot settings

Where join address alerts are posted, and which server counts as spawn.

Posting to DiscordDocs
Post alerts to Discord
On
Discord webhook
Saved, webhook ending 6152
Lobby server
lobby

7-day return is the rule Growth and Overview use: a later qualifying session within 7 days of the first join. Groups under 30 players read not enough players yet.

Sample data