↑↓ move · ←→ turn · Space/Enter photo · R restart · Q quit
A large language model is, at its core, a machine that predicts the next word — over and over. Feed it a prompt, it continues the text most plausibly, one token at a time. The trick that makes this useful for robots: if the prompt describes a task, the "most plausible continuation" is often the right action — and with tool calling, the model can ask your code to execute that action in the real world.
Text is chopped into small pieces called tokens. The model reads them inside a fixed-size context window and assigns a probability to every possible next token:
\[ P(w_{t+1} \mid w_1, \dots, w_t) \]
Everything the model "knows" comes from its training data — it has no senses, no memory beyond the context, and no idea that its words have consequences. That is exactly why we add the tools and the feedback loop.
The model can't move a drone; it can only emit text. Tool calling bridges the gap: the system prompt declares available functions (
move(), turn(), take_picture()
), the model responds with a structured "call", and your code executes it and feeds the result back into the conversation. The model becomes the planner; the code becomes the hands.
An agent is the loop that connects all of this:
\[ \text{perceive} \rightarrow \text{decide} \rightarrow \text{act} \rightarrow \text{observe} \rightarrow \cdots \]
In the workshop's drone safari, each round the model reads the state (drone position, camera view), decides the next command, the code flies the drone, and the new state goes back in — until the mission completes.
The system prompt is the agent's job description: who it is, what tools it has, what rules it must follow. Small wording changes measurably change behaviour — the workshop experiments with this directly, and the game lets you watch the consequences in real time.
Fly the drone across the safari, take pictures of the animals, and complete the mission. The game is the playground of the Wild Drone LLM Workshop — a hands-on course teaching students to build AI agents that understand natural language and control robotic systems.
WASD / arrows to fly, camera view in the corner, collect the photo targets before fuel runs out.
github.com/alejp1998/wilddrone-llm-workshop — notebooks, agents, game.