Role

• Build, maintain, and optimize real-time data pipelines to process bid requests, impressions, clicks, and user engagement data.
• Develop scalable solutions using tools like Apache Flink, Spark Structured Streaming, or similar stream processing frameworks.
• Collaborate with backend engineers to integrate OpenRTB signals into our data pipelines and ensure smooth data flow across systems.
• Ensure data pipelines handle high-throughput, low-latency, and fault-tolerant processing in real-time.
• Write clean, well-documented code in Java, Scala, or Python for distributed systems.
• Work with cloud-native messaging and event platforms such as GCP Pub/Sub, AWS Kinesis, Apache Pulsar, or Kafka to ensure reliable message delivery.
• Assist in the management and evolution of event schemas (Protobuf, Avro), including data consistency and versioning.
• Implement monitoring, logging, and alerting for streaming workloads to ensure data integrity and system health.
• Continuously improve data infrastructure for better performance, cost-efficiency, and scalability.

Nice to have

• Familiarity with real-time analytics platforms (e.g., ClickHouse, Pinot, Druid) for querying large volumes of event data.
• Exposure to service mesh, auto-scaling, or cost optimization strategies in containerized environments.
• Contributions to open-source projects related to data engineering or stream processing.