Senior Data Engineer

Key ResponsibilitiesData Engineering & Platform DevelopmentDesign, develop, and maintain scalable batch and streaming data pipelines using Databricks and Apache Spark.Build and optimize ingestion frameworks to ingest data from ERP, MES, IoT, CRM, supply chain, and external partner systems.Develop robust ELT/ETL pipelines leveraging Delta Lake, Spark SQL, and PySpark.Implement framework-driven engineering practices to improve reusability, scalability, and maintainability.Enterprise Data ModelingDesign and implement enterprise data models supporting reporting, analytics, data science, and AI use cases.Build and maintain Bronze, Silver, and Gold layers following Medallion Architecture principles.Create dimensional models, semantic layers, and curated business-ready datasets.Partner with business stakeholders to translate requirements into scalable data structures.Data Foundation TransformationContribute to the redesign of enterprise-wide data architecture and governance standards.Help establish data quality, lineage, metadata management, and monitoring frameworks.Drive modernization efforts by migrating legacy pipelines and data warehouses onto Databricks.Performance & ReliabilityOptimize Spark jobs and data models for performance, scalability, and cost efficiency.Implement automated testing, CI/CD deployment processes, and observability solutions.Ensure data reliability through validation, reconciliation, and monitoring controls.Cross-functional CollaborationWork closely with Product Managers, Data Architects, Analytics teams, Manufacturing SMEs, and Business stakeholders.Support analytics, reporting, machine learning, and operational decision-making use cases.Mentor junior engineers and promote engineering best practices across the team.Required QualificationsEducationBachelor's or Master's degree in Computer Science, Engineering, Information Systems, or a related field.Experience5-8+ years of experience in Data Engineering, Data Warehousing, or Big Data development.3+ years of hands-on experience with Databricks and Apache Spark.Experience building enterprise-scale data platforms in cloud environments.Technical SkillsDatabricks EcosystemStrong hands-on experience with:Databricks Lakehouse PlatformDelta LakeUnity CatalogDatabricks WorkflowsStructured StreamingDelta Live Tables (preferred)Programming & Data ProcessingAdvanced proficiency in:PySparkSpark SQLPythonSQLCloud PlatformsExperience with one or more:Azure (preferred)AWSGCPData ModelingStrong understanding of:Dimensional ModelingStar/Snowflake SchemasData Vault (preferred)Lakehouse ArchitectureMedallion Data Design PatternsDevOps & Engineering PracticesGit-based source controlCI/CD pipelinesInfrastructure as Code conceptsData Quality and Monitoring frameworksPreferred Industry ExperienceConsumer Packaged Goods (CPG)ManufacturingFood & BeverageBottled Water / Beverage ProductionSupply Chain & Logistics AnalyticsExperience working with data from:SAP ECC/S4HANAManufacturing Execution Systems (MES)Warehouse Management Systems (WMS)Transportation Management Systems (TMS)IoT and production telemetry sourcesis highly desirable.Desired CompetenciesStrong analytical and problem-solving skills.Ability to navigate ambiguity in large-scale transformation programs.Excellent stakeholder management and communication skills.Passion for building scalable enterprise data platforms.Ownership mindset with focus on quality and operational excellence.Ability to influence architecture and engineering decisions.
Mandatory: Databricks, data modelling,