Cloud Infrastructure Engineer

About the role

Pyramid Consulting, Inc is seeking Engineers to join its On-Premises to AWS S3 Migration Platform team. The team designs, builds, and operates a secure, scalable migration platform to move object storage data from on-premises, S3-compatible object storage to AWS S3. The role partners with application owners, storage engineering, cloud platform, and networking teams to support reliable, governed, and auditable migration execution at enterprise scale.

Responsibilities

• Own the end-to-end migration workflow from on-premises source buckets to AWS S3 destination buckets, covering manifest-based and non-manifest migration paths.
• Build and operate AWS-based orchestration using Step Functions, Lambda, Fargate, AWS Batch, DynamoDB, SQS, and rclone.
• Implement batching, sharding, task sizing, and retry handling based on documented compute-tier thresholds: DC node, Bare Metal, and manifest-required.
• Support express-route (like-for-like) and exception-route migrations, using data-detectable criteria to auto-classify buckets where possible.
• Automate pre-flight dry-run validation of credentials, connectivity, readability and writability, and size and object-count limits before transfers.
• Implement and verify source freeze by applying deny-write policy centrally and confirming enforcement with synthetic write-attempt checks.
• Operate the rclone-based migration runtime with credential handling via Kerberos keytab and IAM role assumption via STS, KMS-encrypted destinations, and manifest generation for large migrations.
• Build and run post-migration reconciliation checks comparing source and destination, including object count and size, per-object checksums, drift detection, version completeness, and configuration parity.
• Track and enforce the point-of-no-return cutover step: owner written acknowledgment, followed by centrally sealing the source against reads and writes.

Datos clave

• The migration platform moves on-premises, S3-compatible object storage data to AWS S3.
• The role owns end-to-end migration workflows, including manifest-based and non-manifest paths.
• The orchestration uses Step Functions, Lambda, Fargate, AWS Batch, DynamoDB, SQS, and rclone.
• The role includes pre-flight validation and post-migration reconciliation checks.