Design, develop, and maintain scalable and reliable ETL pipelines using Apache Spark and Databricks.
Write high-performance, production-grade Scala code for data processing and transformation.
Implement and manage observability, monitoring, and alerting frameworks to ensure system health and data integrity.
Develop and maintain CI/CD pipelines for data applications and infrastructure using tools such as GitHub Actions or Azure DevOps.
Ensure adherence to data governance, data quality, and security best practices across all data systems.
Collaborate closely with cross-functional teams including platform, analytics, and product teams to deliver robust data solutions.
Continuously improve system performance, scalability, and operational excellence through best engineering practices.
Stay updated with emerging technologies, including advancements in Generative AI (GenAI), and evaluate their applicability to data platforms.