Dec 2024 - Present
Event-Driven Telemetry Fan-Out for Fleet Operations
Built Kafka and Redis Pub/Sub fan-out paths for live fleet telemetry, enabling sub-second operator dashboard updates and horizontally scalable real-time processing.
MQTT Volume
1M+/day
Dashboard Freshness
Sub-second
Query Latency Reduction
42%
Telemetry Fan-Out Reliability
Normalized operator-facing freshness after separating ingestion, fan-out, and query-serving paths.
Kafka and Redis Pub/Sub separated live operator updates from ingestion pressure while MySQL tuning reduced serving latency.
Telemetry Fan-Out Pipeline
Ingest MQTT
100%Vehicle telemetry enters ECS-backed services
Fan out events
84%Kafka and Redis Pub/Sub decouple consumers
Serve dashboards
82%Operators receive sub-second updates
Tune hot queries
72%Indexes and schema changes cut MySQL latency
Problem
Fleet operations needed live telemetry updates without coupling every dashboard and service directly to the ingestion path.
Architecture
I implemented event-driven fan-out over production IoT and fleet backend services.
- Shipped TypeScript and NestJS services on AWS ECS across MySQL, MongoDB, Redis, SNS/SQS, and REST APIs.
- Used Kafka and Redis Pub/Sub to fan out telemetry updates for sub-second operator dashboard refreshes.
- Tuned high-traffic MySQL query paths with composite indexes and schema-level optimization.
Results
The fan-out path supported 1M+ MQTT messages/day, enabled horizontally scalable real-time processing, and reduced average MySQL query latency by 42%.
Engineering Notes
The key reliability win came from separating ingestion, fan-out, and query-serving responsibilities so operator-facing latency could improve without overloading the write path.