Lead Data Engineer

About SHR Consulting Group SHR is a premier technology integrator solving our nation’s most complex modernization and readiness challenges across the defense, federal civilian, and intelligence markets. Our robust portfolio of offerings includes high-end solutions in systems engineering and integration, enterprise IT, including cloud services: cyber, software, advanced analytics, and AI. With an intimate understanding of our customers’ challenges and deep expertise in existing and emerging technologies, we integrate the best components from our own portfolio and our partner ecosystem to deliver innovative, effective, and efficient solutions. Position Summary SHR Consulting Group is seeking a Senior Lead Data Engineer as a Key Personnel for a bid opportunity to lead data pipeline engineering, operations, performance, migration, and integration activities supporting an enterprise FEMA effort. This position is CONTINGENT UPON CONTRACT AWARD. Candidates selected for consideration must be willing to authorize the submission of their resume and qualifications in support of our proposal response. Key Responsibilities Lead technical delivery for FEMA pipeline engineering, operations, performance, and migration. Design, construct, test, deploy, and maintain scalable batch and streaming data pipelines. Build and maintain metadata-driven ingestion pipelines . Implement robust schema evolution and data-processing patterns. Diagnose and resolve production pipeline failures and complex data-storage issues. Design pipelines consistent with FEMA's multi-zone/Medallion architecture. Develop modular, parameterized, and reusable pipeline patterns. Implement scalable ingestion and transformation solutions supporting increasing data volume, velocity, and concurrency. Implement engineering practices for reliability and cost efficiency, including: Incremental processing Idempotent and restartable jobs Partitioning Storage/file-layout optimization Compute right-sizing Develop automated testing, data-quality validation, and deployment automation. Diagnose schema drift, missing or late data, checkpoint/watermark failures, orchestration faults, and performance degradation. Conduct root-cause analysis of significant or recurring production issues. Support migration of legacy data, pipelines, reporting workloads, and dependencies. Support source-system onboarding and integration using APIs, connectors, batch/file transfers, CDC, and custom integration patterns. Provide technical leadership and mentoring to Senior Data Engineers and other engineering staff. Required Qualifications Extensive hands-on experience designing and developing enterprise data pipelines. Demonstrated experience with Databricks or comparable cloud-based Lakehouse/data engineering platforms . Strong experience with metadata-driven data ingestion frameworks. Strong knowledge of schema evolution and enterprise data-processing patterns. Demonstrated ability to…