Work/Enterprise-Grade Kafka Producer
Configuration-driven publishing service.
Publishing data out to Kafka needed to be reliable in the face of partial failure: writing to the database and publishing to a topic are two operations, and a crash between them either loses an event or emits one for a write that never committed.
Implemented the Transactional Outbox pattern with a polling publisher, enabling reliable event delivery, configurable topic routing, and database-to-Kafka synchronization. All topics, mappings, and publishing behaviour are defined through a single JSON configuration, and the service publishes to multiple topics on a configurable schedule.
Next project
ETL Monitoring & Recovery Tool