Overview
Fleet Orchestrator manages jobs and services across a mixed collection of repurposed computers, phones, and tablets. I built its Rust controller, device agent, and command-line interface to make these machines useful as remotely managed compute nodes.
The controller uses axum and SQLite to track devices, work, and results. Agents poll for jobs, report capabilities and health metrics, and buffer undelivered output locally. Supported workflows include probes, Unix scripts, output artifacts, supervised services, and staged rollouts with health checks and persistent rollback actions.
Recovery behavior is a central part of the design. Jobs use renewable leases, and results are checked against the owning device and attempt. Delivery can be retried after an interruption, so scripts must tolerate repeated execution. Device credentials are separate from administrator access, and script isolation depends on the kernel features available on each target.
Hardware testing on a General Mobile GM8 running Alpine Linux verified queued result delivery after a temporary network interruption and successful retry after a mid-job reboot. Those drills also exposed and helped resolve clock/TLS recovery and BusyBox sandbox compatibility problems. The device's missing namespace support remains an explicit isolation limitation.
The project includes Linux builds for x86-64, AArch64, and ARMv7, plus an Android foreground-service wrapper. Sustained hardware soak testing and broader recovery acceptance remain ongoing. The linked blog post explains the architecture through the failures that shaped it.
Technologies: Rust, axum, SQLite, Linux, Kotlin, Android, Prometheus
Project Tags: