Azure Databricks Tech Lead

About the role

Lead the end-to-end migration of enterprise data integration workloads from Informatica PowerCenter to Azure Databricks.

Tasks

• Lead end-to-end migration of Informatica PowerCenter workflows and ETL pipelines to Azure Databricks.
• Assess existing Informatica mappings, transformations, and workflows and develop equivalent PySpark-based pipelines.
• Design and implement scalable ETL/ELT solutions using Azure Databricks, PySpark, Spark SQL, and Python.
• Analyze Oracle stored procedures, functions, and packages and determine migration or refactoring strategies.
• Work with Unity Catalog, Delta Live Tables (DLT), ADLS Gen2, and Delta Lake.
• Define target architecture, migration strategy, coding standards, testing, and deployment practices.
• Ensure data quality, reconciliation, performance, reliability, and production readiness.
• Collaborate with business and technical stakeholders to manage risks, dependencies, and delivery timelines.
• Lead cutover, post-migration stabilization, and performance optimization.
• Mentor engineers and promote Databricks best practices and reusable data engineering frameworks.

Requirements

• 8–10 Years experience.
• Strong experience with Azure Databricks, PySpark, Python, and Spark SQL.
• Hands-on experience migrating workloads from Informatica PowerCenter to Databricks.
• Expertise in Informatica mappings, transformations, lookups, joins, filters, routers, and expressions.
• Experience with Oracle databases, stored procedures, functions, and packages.
• Knowledge of Unity Catalog, Delta Live Tables (DLT), ADLS Gen2, and Delta Lake.

Schedule

• 5 Days Onsite

Location

• Dallas, TX / Atlanta, GA