Senior Azure Databricks Architect/Data amp; AI Architect

About the role

Lead enterprise Data and AI platform modernization.

Tasks

• Design scalable Lakehouse architectures, engineer real-time pipelines, establish data governance, and deploy modern AI solutions natively on Microsoft Azure.
• Design scalable Azure Databricks Lakehouse, streaming, and data engineering architectures using Delta Lake and Apache Iceberg interoperability.
• Build declarative data pipelines utilizing Lakeflow Designer, Lakeflow Connect, and Lakeflow Declarative Pipelines.
• Implement enterprise governance, automated lineage, and RBAC/ABAC access controls via Unity Catalog across Azure storage.
• Architect GenAI, RAG, and agentic workflows using Unity AI Gateway, Vector Search, Databricks Genie / Genie Code, and Agent Bricks.
• Lead technical design reviews, optimize Azure compute/storage costs (FinOps), and establish engineering standards.

Requirements

• Experience: 10+ Years
• Databricks Platform: Lakeflow (Connect / Designer / Declarative Pipelines), Unity Catalog, Delta Lake / Iceberg, Genie Code, Agent Bricks, Managed Agent Memory, Unity AI Gateway, Vector Search, MLflow.
• Azure Cloud: Azure Databricks, Azure Data Lake Storage Gen2 (ADLS Gen2), Azure Data Factory (ADF), Microsoft Entra ID (SCIM), Azure Key Vault, VNet injection / Private Link.
• Engineering amp; Streaming: PySpark, Apache Spark Structured Streaming, Python, SQL, Databricks Asset Bundles (DABs), CI/CD.
• 10+ years in Data Engineering, Cloud Architecture, or Enterprise Solution Architecture.
• 5+ years of hands-on Databricks architectur

Schedule

• Type: contract-to-hire – 6 months
• Hybrid

Location

• Houston, TX