Control a robot from an Android app and ESP32

A phone is the fastest way to add a control panel to a Wi-Fi robot. The hard part is not the app; it is the protocol, the watchdog and the Wi-Fi setup.

This guide compares the three realistic build paths, explains which protocol to use for driving versus telemetry, shows the ESP32 side of the link, and lists the failures that make a robot lurch, lag or ignore the phone entirely.

Choose the build path first

For a first build, App Inventor is genuinely enough. A joystick, four buttons and a connection status light can be built in an afternoon, and the resulting app talks plain HTTP or Bluetooth to the ESP32.

Pick the transport before writing code

HTTP is easy to debug because every command is a URL you can test in a browser. The cost is that each command opens a request, and a stream of tiny requests creates lag and connection churn. For a robot that must react within 100 ms, use a WebSocket or Bluetooth.

The ESP32 side of an HTTP link

Start with a small set of endpoints that map to motor commands. Keep the handler short and never block inside it.

server.on("/drive", []() {
  int left  = server.arg("l").toInt();   // -255..255
  int right = server.arg("r").toInt();
  setMotors(constrain(left, -255, 255),
            constrain(right, -255, 255));
  lastCommandMs = millis();
  server.send(200, "text/plain", "ok");
});

A single /drive?l=..&r=.. endpoint is easier to reason about than separate forward, back, left and right endpoints, because it expresses the actual motor command and lets the app mix a joystick directly.

The watchdog is not optional

Wi-Fi drops, the app crashes, the phone sleeps. If the robot keeps the last motor command, it drives into a wall at full speed. Every remote-controlled robot needs a command timeout that stops the motors.

void loop() {
  server.handleClient();
  if (millis() - lastCommandMs > 500) {
    setMotors(0, 0);
  }
}

Five hundred milliseconds is a reasonable starting point for a small robot. A faster robot needs a shorter timeout. The app should also send a stop on disconnect, but never rely on that alone; the robot must stop itself.

Mapping a joystick to motor values

A joystick gives an x and y offset from centre. For a differential drive, convert them into left and right speeds with a mix, then scale to the PWM range.

left  = constrain(y + x, -1.0, 1.0) * 255;
right = constrain(y - x, -1.0, 1.0) * 255;

Normalise the joystick to −1 to 1 first, and apply a dead band near the centre so the robot does not creep when the thumb is slightly off. Apply a speed limit for the first tests: half power is enough to learn whether the wiring is right.

WebSocket for real driving

A WebSocket keeps one connection open, so each command is a small frame rather than a new HTTP request. The ESP32 can also push status back without polling, which makes battery voltage and sensor readings cheap.

• Send commands at 10–20 Hz, not as fast as the loop runs.
• Send the latest state, not a queue of stale commands.
• Include a sequence number so the robot can ignore out-of-order frames.
• Reconnect automatically on the app side, and keep the watchdog running.
• Keep payloads tiny: a short JSON or even a fixed text format.

Wi-Fi setup problems

Running the ESP32 as its own access point removes the router from the equation for a demo. You lose internet on the phone while connected, and Android may route traffic over mobile data anyway, so test that path before relying on it.

Security that matters even for a toy

• Never forward a port from the internet to the ESP32.
• Use WPA2 on the network, not an open hotspot.
• Add a shared token header so a random device on the network cannot drive the robot.
• Change default credentials on any cloud dashboard you use.
• Keep firmware updated, and test updates before a demo.

An ESP32 has no meaningful protection against a determined attacker. Treat it as a device on a trusted local network, and keep it off public IP addresses entirely.

Latency and control feel

HTTP commands over a clean 2.4 GHz network typically add 20–80 ms per request on top of the app’s own overhead. WebSocket frames are closer to 5–20 ms. Video adds far more, which is why a camera stream feels laggy even when the controls respond instantly.

For a robot driven by looking at it directly, 100 ms is fine. For first-person driving through a camera, plan for 200–400 ms and drive slowly. The control loop should also smooth commands: sending a new speed 20 times per second and ramping toward it feels much better than jumping between full-speed values.

A test checklist before the demo

• Motors stop within the watchdog timeout when the phone is switched off.
• The app reconnects after the robot reboots.
• Full battery, charged phone, and a spare power bank.
• Tested on the same Wi-Fi network and in the same room as the demo.
• Joystick dead band set so the robot does not creep.
• Speed limit set so a mistake does not break the chassis.

Related: Bluetooth robot car, ESP32 pinout, MQTT telemetry, ESP32-CAM streaming, how to power a robot.

Incoming app commands belong in a queue, as shown in the ESP32 FreeRTOS robot guide.

If you would rather use a browser than an app, this Wi-Fi robot build needs no installation at all.

Sources and further reading

• Arduino ESP32 documentation — Wi-Fi, WebServer and WebSocket examples.
• Android developer documentation — networking and cleartext policy.
• MIT App Inventor — block-based Android app building.
• Arduino language reference — timing and serial basics.