GCP Data Engineer
External job listingat Arkhyatech
We are #Hiring GCP Data Engineer: Charlotte NC Charlotte, NC (Hybrid) Pipeline Development & Optimization: Design and implement robust batch and real-time streaming pipelines using PySpar...
Salary
Not provided
CazVid estimate: $41 to $68 an hour. Not set by the employer.
Location
Charlotte, United States
Employment type
Full time
Workplace
Hybrid
GCP Data Engineer
Charlotte, United States
Job description
We are #Hiring GCP Data Engineer: Charlotte NC
Charlotte, NC (Hybrid)
Pipeline Development & Optimization: Design and implement robust batch and real-time streaming pipelines using PySpark, Hadoop, Hive, and Apache Kafka to process large volumes of security vulnerability findings.
Data Cleansing & Deduplication: Implement advanced data quality, enrichment (severity, criticality scoring), and deduplication logic comparing multiple scanning feeds (e.g., LLM scanners vs. Checkmarx/Snow AR) to eliminate duplicate findings.
Cloud Migration & Data Lake Integration: Ingest and process data within Google Cloud Platform (GCP) leveraging GCP Data Lake, BigQuery, and modern cloud storage/compute frameworks, facilitating the transition from on-premise processing to GCP.