Senior Data Engineer AWS

We are seeking a highly skilled Senior Data Engineer to join a mission-critical client project, acting as a bridge between complex business requirements and scalable technical execution. In this role, you will collaborate closely with Data Architects, Engineering leads, and Stakeholders to design and maintain sophisticated data models that enhance accessibility and governance. Your expertise will be the foundation of a high-performance data mesh environment, driving measurable business impact and ensuring data quality across the entire enterprise ecosystem.
About First FactoryWe are a software development company with over two decades of experience, boasting a dynamic team of 175+ professionals actively engaged in diverse projects across various industries. We invite you to join us on this journey as we thrive and embrace fresh challenges.
Key ResponsibilitiesDesign and develop efficient and scalable data pipelines between enterprise transactional systems, third-party, and analytics platformsMust be a crack coder in one of the following (go-lang, Node.js or Python)Must be proficient in writing and refactoring efficient SQL queries.Must be able to explain features of good data model design.Build and maintain a data environment for speed, accuracy, consistency and ‘up’ timeSupport analytics and data science by building a world-class data mesh environment that empowers analysts to determine insights into revenue and power products across the organization.Integrate third-party data sources and API’s into the Bright data mesh ecosystemWork closely with Data Science team and participate in development of feature engineering pipelines.Design and develop data products with modern AWS cloud technologies such as S3, Redshift, EMR, Hive, Presto, Flink and SparkWork with the machine learning engineering team to build a data eco system that supports AI products at scale.Design and deploy an enterprise data warehouse that supports internal and market facing analytics products at scaleEnsure data governance principles adopted, data quality checks and data lineageimplemented in each hop of the dataPartner with adjacent organizations to ensure proper integration and adherence to standardsBe in tune with emerging trends in data management and cloud technologies and participate in the evaluation of new technologiesEnsure compliance through the adoption of enterprise standards and promotion of best practice / guiding principles aligned with organization standard
Requirements5+ years of experience as a data engineer at an innovative organization4+ years of hands-on experience in implementing data lake systems using AWS cloud technologies such as S3, Redshift, EMR, Hive, Kafka and SparkExpert managing AWS services (EC2, S3, Route 53, ELB, VPC, cloudwatch, Lambda) in a multi-account production environmentExperience with development frameworks as well as data and integration technologies such as Python and ScalaCreate new ETLs in AWS Glue with Python or Node.Js as the scripting languageCreate AWS Lambdas using Python or Node.Js as the scripting languageModify existing ETLs to fix issues where approach is appropriateUse Glue for ETLs inside of AWS to and from all AWS types of data sourcesSupport the migration of data into S3, Redshift, DynamoDB, AWS RDSExperience With Machine Learning Libraries and Frameworks (TensorFlow , MLlib) is an added advantageExposure to R , SparklyR , and Other R packages is a PlusExpert knowledge of Agile approaches to software development and able to put key Agile principles into practice to deliver solutions incrementally.Monitors industry trends and directions; develops and presents substantive technical recommendations to senior managementExcellent analytical thinking, interpersonal, oral and written communication skills with strong ability to influence both IT and business partnersAbility to prioritize and manage work to critical project timelines in a fast-paced environmentAdvanced knowledge of Microsoft SQL Server for future migration to an AWS Database Platform
Nice to havePrevious experience with cloud development (AWS, GCP)Previous experience in design and deployment of data lakes, data mesh, data warehouses, and streaming platformsPrevious experience with data quality projects and public recordsPrevious experience with: AWS DynamoDB, AWS Elastic Map Reduce, AWS Lambda, AWS Step Functions, AWS Redshift, AWS RDS, Terraform, or CloudFormationAWS Architect Certification is a plus