Apache Druid

Position: Apache Druid

Location: Sunnyvale, CA (Onsite) pls local profiles only

Job Description:

Reliability and Availability:

• Ensure high availability and reliability of production systems.
• Implement and maintain robust monitoring and alerting systems.
• Participate in on-call rotations to respond to incidents and outages.
• Conduct post-incident reviews and implement preventative measures.
• Automation and Infrastructure as Code (IaC):
• Automate infrastructure provisioning, configuration, and deployment using IaC tools (e.g., Terraform, Ansible).
• Develop and maintain CI/CD pipelines to streamline software releases.
• Optimize and automate data pipelines and workflows.
• Apache Druid Management:
• Manage and optimize Apache Druid clusters for high performance and scalability.
• Troubleshoot Druid performance issues and implement solutions.
• Design and implement Druid data ingestion and query optimization strategies.
• Apache Airflow Orchestration:
• Design, develop, and maintain Airflow DAGs for data orchestration and workflow automation.
• Monitor Airflow performance and troubleshoot issues.
• Optimize Airflow workflows for efficiency and reliability.
• Monitoring and Logging:
• Implement and maintain comprehensive monitoring and logging solutions (e.g., Prometheus, Grafana, ELK stack).
• Analyze metrics and logs to identify performance bottlenecks and potential issues.
• Create and maintain dashboards for visualizing system health and performance.
• Collaboration and Communication:
• Collaborate with development, data, and operations teams to ensure smooth operations.
• Communicate effectively with stakeholders regarding system status and incidents.
• Document processes and procedures