Senior Data Engineer

Tasks

• Design, develop, and optimize scalable ETL/ELT pipelines using Python, SQL, PySpark, and Apache Spark.
• Build and manage cloud data platforms using Microsoft Azure and AWS.
• Develop solutions using Azure Data Factory, Azure Databricks, Azure Synapse Analytics, and Azure Data Lake Storage.
• Implement AWS data solutions using S3, Glue, Redshift, Lambda, and Athena.
• Design modern data lakehouse architectures, data warehouses, and data modeling solutions.
• Integrate AI/ML and Generative AI capabilities into data engineering workflows.
• Develop RAG pipelines, work with LLM APIs, embeddings, and vector databases.
• Implement data governance, security, quality checks, monitoring, and performance optimization.
• Collaborate with data scientists, architects, analysts, and cross-functional engineering teams.
• Mentor team members and contribute to cloud data architecture and engineering best practices.

Requirements

• Programming: Python, SQL, PySpark.
• Cloud Platforms: Microsoft Azure and AWS.
• Azure Services: Azure Data Factory, Databricks, Synapse Analytics, ADLS.
• AWS Services: S3, Glue, Redshift, Lambda, Athena.
• Big Data: Apache Spark, distributed processing, batch and streaming pipelines.
• AI & GenAI: LLMs, RAG, embeddings, vector databases, and AI integration.
• DevOps: Git, CI/CD, Docker, and infrastructure automation.
• Strong experience in data architecture, troubleshooting, and performance tuning.
• Work Authorization: OPT / STEM OPT

Schedule

• W-2 Contract

Pay

• $30/hour on W-2

Location

• Tampa, FL, USA