I was made the technical lead for Niantic’s anti-cheat initiative. Before I describe the experience, it might be useful for me to describe Niantic’s anti-cheat problem.

At the time that I joined Niantic, they only had two real games (Pokémon Go and Ingress), but they had additional games under development (including Harry Potter: Wizards Unite) and they really wanted Niantic to become a platform on which 3rd parties can develop their own games (the goal was to scale to support up to 100 games).

Niantic games are based on where you are in the real world (i.e. to move in the game, you have to move in real life), so most cheating in Niantic games involves location spooking (making the game think you are in one location when you are not).

Cheating behavior is different in Pokémon Go and Ingress.

Pokémon Go cheating patterns

Pokémon Go has regional and rare Pokémon which encourages spoofing across very long differences (e.g. if a Pokemon only exists in a specific region, you have an incentive to spoof to that region). While a few players may spoof just because they are too lazy to walk a few blocks to their local gym, many players will spoof to the other side of the world in an effort to catch a rare 100 IV Pokémon.

There are other forms of cheating as well:

• Some apps will intercept the calls made between the phone and the server.
• Players create multiple accounts so they can more easily win raids, trade Pokémon, win raids, and take over gyms.

• You get bonuses for walking, so people buy devices to make your phone think that you are waking when you are not.

As Pokémon Go was an extremely successful game, there is/was a large community of hackers dedicated to it.

Some of these hacks were fairly innocuous (and some would argue helped the game succeed). For example, the server security was initially easy to bypass such that bots could make server calls that would populate online maps showing the active Pokémon throughout the city. By the time this was fixed by Niantic, such maps accounted for about half of the Pokémon Go server traffic.

Other hackers were a little more brazen. For example, PokéGo++ was a modified app that required a $5/month subscription and provided various ways to cheat:

• It intercepted all communication between the hacked app and the server and populated their own database of valuable Pokémon throughout the world, allowing you to search for the Pokémon characteristics you want and then immediately jump to where that Pokémon is so you can catch it.
• It had a bunch of other features as well, such as the ability to make the game think that every throw is a perfect throw.

Niantic sued them and eventually made them stop.

Reactions to cheating in Pokémon Go are mixed. While people can play in groups, each individual creates their own collection of Pokémon and if one player expands their collection via cheating, it usually doesn’t directly impact other players.

• Some players are completely fine with all forms of cheating (even if they themselves don’t actively cheat).
• Some players don’t like it, but they also don’t care that much either way.
• Other players are absolutely incensed about any form of cheating.

Ingress cheating patterns

I’m not an Ingress player so I don’t understood the game as well, but Ingress has two teams (blue and green) who compete by taking over different parts of the world. While Pokémon Go is more of an individual game, Ingress is much more collaborative. When a team takes over three portals, they can create links between them that forms a triangle (and the team now owns everything under that triangle). 

In some cases, the three portals can be very far apart.

The three portals can be taken over by a single person (usually via spoofing), or they can be taken over by different players cooperating (which is usually the case).

Spoofing in Ingress very much impacts the fun of the game. Niantic had a magazine article posted in their lobby about a woman who flew to a remote area of Alaska for the sole purpose of blocking the other team.

On my first day of orientation, I met Brandon Downey (an avid Ingress player) who showed me how he had spent the previous weekend. There was a big green triangle over all of San Francisco, and he said that it took 200 players working collaboratively between 2am and 4am to accomplish it. One of the portals was the Farallon Islands, which I knew from Mythbusters is a wildlife sanctuary that doesn’t allow the general public, and he said that a friend had to volunteer with a marine expedition for a week to pull it off.

So spoofing can really kill the fun on Ingress.

Why is it so hard to catch cheaters?

It seems very intuitive that it would be easy to catch cheaters – especially those who spoof long distances. If they circumvent the world at superhuman speed, they are obviously cheating. In fact, if they jump between two nearby points at impossible speeds, they must also be cheating, right?

Unfortunately, it’s not that straightforward.

• Phone location services are not always exact – a number of things can throw them off. For example:
  • Atmospheric conditions can make the player jump around randomly – often between two points up to a mile apart.
  • They use visible wifi connections to help determine locations, and sometimes this information can be wrong.
  • Phones can sometimes cache a location, so if you fly to a new place, it might think that you started in the old palace and then immediately jumped to the new place.
• There is always a concern that a new phone will hit the market that behaves in such a way that we incorrectly detect cheating.
• It is impossible to know the ground truth. Pretty much all cheaters that get punished will loudly claim that they were punished for no reason. Hence, auditing our actions by following up with real players doesn’t usually result in the ground truth.
• The game teams have deemed that some actions are less egregious than other actions. For example, location spoofing is bad, but account sharing is less bad. Account sharing is when one user shares there account information with other users (probably close friends or family members) so they can log into your account to catch Pokémon or to make trades. But if somebody in Europe logs into an account where the owner lives in the USA, how do we differentiate this from spoofing?
  • Sometimes it is obvious, such as when both players simultaneously play on the same account (in this case it looks like the player moved between the USA and Europe every 15 seconds).

Different people have different ideas on how aggressively to punish players suspected.

The old anti-cheat system

Niantic had an existing anti-cheat system, but it was very expensive and didn’t work very well. The specific issues were:

• It was very opaque – it had no dashboards and was very hard to debug. If a bunch of people started getting punished on a new phone or OS version, we didn’t have an easy way to see that.
• It tried to re-create player state rather than store player state.
  • For example, Niantic uses the three strike system.
    • The first time you are caught you get a warning.
    • The second time you are caught, you get a 30 day suspension.
    • The third time you are caught, you get banned.
  • By not storing state:
    • It was possible that we’d issue a suspension before the player actually saw the first warning.
    • Often, we’d catch people cheating up to 10 times, but we kept issuing warnings or suspensions and never moved towards an actual ban.