SmartSnake

A deep Q-learning agent that teaches itself to play Snake. It starts from random weights and trains live in your browser; this is a TensorFlow.js port of the original PyTorch agent.

Figure 1. The agent playing.Press Start training to begin. The weights start out random, so the early games are short.

Figure 2. The network on the current move: the 11 inputs it sees, its 256 hidden activations, and the value it predicts for each action. It takes the action in orange.

Figure 3. Score in each game (blue) and the running mean (orange). The first point appears when the first game ends.

How it works

What it sees: Eleven yes-or-no inputs: whether there is danger straight ahead, to the right, or to the left; which way it is moving; and which way the food is.
The network: Eleven inputs, one hidden layer of 256 ReLU units, and three outputs: the predicted value (Q) of going straight, turning right, or turning left. It takes the highest.
How it learns: After every move it trains on that step, pushing the chosen action's Q-value toward the reward plus 0.9 times the best Q-value of the next state. After each game it also trains on 1,000 moves sampled from a memory of the last 100,000. Food is worth +10 and dying costs 10.
Exploring: To find out what works, it moves at random 40% of the time in its first game, falling to never by game 80.