Senior Data Engineer Python Snowflake Synapse Analytics

About the role

Join Akurey as a Senior Data Engineer to build and optimize scalable data pipelines and ETL processes using Python, Snowflake, and Azure Synapse Analytics. The role transforms and integrates large-scale data while ensuring performance, reliability, and data quality.

Responsibilities

• Design, develop, and maintain robust Python data pipelines to extract data from Snowflake and Azure Synapse Analytics and transform it for target systems.
• Implement efficient ETL processes for accurate, reliable, and scalable data movement across platforms.
• Work with stakeholders to understand source systems, target systems, and business requirements, and translate them into data pipeline solutions.
• Write clean, optimized, scalable code for large data volumes while maintaining performance and reliability.
• Monitor, troubleshoot, and optimize pipeline performance, resolving bottlenecks, failures, and data quality issues.
• Define transformation logic for cleansing, filtering, aggregation, normalization, and standardization.
• Develop data mapping and schema conversion processes for consistency and compatibility between systems.
• Establish and maintain connectivity with Snowflake and Azure Synapse Analytics through APIs, database connectors, and other integration methods.
• Integrate and synchronize data from multiple sources while maintaining consistency, accuracy, and data integrity.
• Partner with IT and platform teams on secure data transfer mechanisms aligned with governance and compliance requirements.
• Develop error handling and exception management processes for resilient and reliable data integrations.
• Document pipeline architecture, transformation rules, source specifications, and target system requirements.
• Collaborate with data analysts, data scientists, and business stakeholders.

Datos clave

• The role uses Python, Snowflake, and Azure Synapse Analytics.
• The work includes building and optimizing scalable data pipelines and ETL processes.
• The role focuses on transforming and integrating large-scale data while ensuring performance, reliability, and data quality.