Data Engineer - Databricks

Job OverviewWe are looking for an experienced Data Engineer with 3–5 years of hands-on experience in data engineering and strong expertise in Databricks. The ideal candidate should have solid experience in building scalable data pipelines, data transformation, and working with modern cloud data platforms.Key ResponsibilitiesDesign, develop, and maintain scalable data pipelines using Databricks.Develop data ingestion and transformation processes using PySpark and SQL.Work with Delta Lake and implement reliable data processing solutions.Build and optimize ETL/ELT pipelines for large-scale datasets.Implement data quality, validation, monitoring, and error-handling processes.Work with different data sources and integrate data into centralized data platforms.Optimize Spark jobs and Databricks workloads for performance and cost efficiency.Collaborate with Data Scientists, Analysts, and other engineering teams.Follow best practices for data security, governance, and pipeline development.Required Skills3–5 years of experience in Data Engineering.Strong hands-on experience with Databricks.Strong proficiency in PySpark and SQL.Experience with Delta Lake.Good understanding of ETL/ELT concepts and data pipeline development.Experience working with large datasets and distributed data processing.Good understanding of data warehousing and data modeling concepts.Experience with cloud platforms such as Azure, AWS, or GCP.Strong problem-solving and analytical skills.Preferred SkillsExperience with Medallion Architecture (Bronze, Silver, Gold).Experience with Databricks Workflows/Jobs.Knowledge of Auto Loader and Structured Streaming.Experience with Git and CI/CD practices.Knowledge of data governance and security.Experience with Azure Data Lake, AWS S3, or similar cloud storage platforms.