Senior Data Engineer/ML Platform Engineer

About the role

Join an internal data platform team and build lakehouse pipelines, curated enterprise datasets and GenAI ready data products on Azure Databricks.

Tasks

• Build production pipelines on Databricks with PySpark, SQL, Delta Lake, Unity Catalog and Workflows, with testing, monitoring and clear lineage
• Model bronze, silver and gold layers and data contracts with schemas, SLAs and quality checks
• Create ML data products: feature tables, embedding generation, chunking and indexing for RAG, batch classification and enrichment
• Integrate with Azure services such as ADLS Gen2, Key Vault and managed identities following enterprise security patterns
• Partner with application and ML teams to expose stable tables, volumes and APIs for agentic workflows
• Document data dictionaries and runbooks, and take part in code review and agile ceremonies

Requirements

• 5+ years in data engineering, including 2+ years of production Databricks on Azure
• Strong Python (PySpark, pandas) and SQL, with experience running jobs at scale
• Hands on Delta Lake and Unity Catalog experience
• Pipelines built for ML and GenAI use cases: embeddings, vector or hybrid search inputs, labeled datasets, batch inference outputs
• Git, CI friendly repo layout and unit and integration testing for data pipelines
• Clear communication and comfort working alongside internal leads
• Nice to have: RAG design and retrieval evaluation, MLOps on Databricks, Structured Streaming, Terraform or Bicep, manufacturing data experience.

Schedule

• Onsite 4 days/week

How to apply

• Apply directly through the link
• Send your resume

Location

• Warren, MI, Austin, TX