Senior Data Platform Engineer

Tasks

• Design, deploy, and operate data platform services on Kubernetes in Microsoft Azure.
• Build and manage lakehouse infrastructure on Apache Iceberg backed by ADLS, including catalog configuration, table maintenance, schema evolution, and partitioning strategy.
• Operate and tune Trino and Spark clusters for performance, reliability, and cost efficiency.
• Develop and maintain data transformation pipelines using DBT, with sound modeling, testing, and documentation practices.
• Define and enforce standards for data access, security, governance, and multi-tenant resource management across ADLS and query engines.
• Implement observability for the platform, including monitoring, alerting, capacity planning, and incident response.
• Partner with data engineers, analysts, and platform teams to onboard workloads and resolve performance bottlenecks.
• Automate infrastructure and deployments using infrastructure-as-code and CI/CD.

Requirements

• Azure, ADLS, Apache Iceberg, Azure Kubernetes (AKS), Spark, DPT, CI/CD
• Strong hands-on experience with **Kubernetes**, including deploying and operating stateful and distributed workloads.
• Strong hands-on experience with **Microsoft Azure Cloud**, including **ADLS**
• Hands-on experience with **Apache Iceberg** and Iceberg catalogs REST
• Expertise in distributed query and compute technologies, specifically **Trino** and **Spark**.
• Hands-on experience with **DBT** for data transformation and analytics engineering.
• Solid understanding of data lake and lakehouse architectures, file formats (Parquet, ORC), and query optimization.

How to apply

• Share your resume by email.

Location

• San Jose, CA - hybrid