Sr Data Engineer (Data Bricks)

Job Title: Senior Data Engineer (Databricks)Primary Location: Chicago, IL (Hybrid – 3 days onsite / 2 days remote)Position Type: Full-Time, Direct HireMust Be Citizen or Green Card
OVERVIEWA Senior Data Engineer. This is a full-time, direct hire role in Chicago, IL, working a hybrid schedule with a minimum of three days onsite per week. This role exists as part of a multi-year initiative to transform our client's Data & AI function, building a modern data platform that enables speed-to-insight and makes the organization truly data-driven.
You'll be one of the core builders of a brand-new Azure Databricks platform, designing and delivering the pipelines, transformations, and models that everything else runs on. The environment is greenfield: Databricks is stood up, but no production pipelines exist yet, so you'll help set the standards from day one. This is a hands-on senior individual contributor role, not a people management, architecture, AI/LLM, or BI/reporting role. The architecture is defined; your job is to build it well and raise the technical bar of a growing team (expanding to 25–30) that is largely BI-focused today.
The work carries strong executive sponsorship and enterprise-wide visibility, with the chance to influence engineering practices across the organization. The interview process includes a live technical assessment covering Python, SQL, and practical data pipeline logic.
WHAT YOU BRING TO THE ROLE. (IDEAL EXPERIENCE)Bachelor's degree in Computer Science, Engineering, Data Science, or a related field6+ years of hands-on data engineering experience, with a track record of owning pipelines end to end: ingestion, validation, transformation, and governanceReal, hands-on Databricks experience (Azure preferred; AWS or GCP acceptable), including Delta Lake, Unity Catalog, Databricks SQL, Spark clusters, and Workflows/JobsStrong coding skills in Python, SQL, and Apache Spark that go well beyond basic scriptingProven experience designing and building scalable ETL/ELT pipelines, including reusable, metadata-driven pipelinesExperience with object storage, lakehouse engines, orchestration tools, and streaming or CDC pipelinesSolid data modeling, schema design, and performance tuning experienceWorking knowledge of Git-based workflows, CI/CD, test automation, and monitoring and alertingStrong troubleshooting skills and a solid grasp of engineering fundamentals: scalability, reliability, and maintainabilityThe ability to clearly explain what you built, why it mattered, and how it solved a business or platform problem
Preferred Experience building or migrating a data platform from the ground upInfrastructure as Code (Terraform, Bicep, ARM) and Governance as Code experienceMulti-cloud background with depth in AzureLegacy data warehousing and/or Hadoop experienceDeep Databricks exposure beyond basic pipelines, such as enterprise-scale work and optimizationA track record of mentoring and upskilling peersAgile delivery experience
WHAT YOU'LL DO. (SKILLS USED IN THIS POSITION)Lead the design and build of scalable data pipelines and models in Azure Databricks on a brand-new platform with no production code yetOwn pipelines end to end, from ingesting source systems to transforming and modeling data to producing outputs for downstream analytics, BI, and AI/ML use casesRefactor and migrate multiple legacy data warehouses into the centralized Databricks lakehouseWork at the code level every day in Python, SQL, and Spark, building pipelines with Databricks Asset Bundles, Spark workflows, and API-driven jobsEstablish engineering standards for CI/CD (GitHub/Azure DevOps), automated data testing, monitoring, and alertingHelp shape and enforce standards for data modeling and governanceTroubleshoot and optimize performance across the platformMentor and coach junior engineers, conduct code reviews, and lead by example in design, coding, and testing practicesPartner with Data Engineering, BI, AI, Finance, and product owners, and help translate technical work into business impact