Learn Beyond Consulting LLCRemote

Lead Data Engineer (Google Cloud Platform)

Project-Based

Description

Lead Data Engineer (Google Cloud Platform) Location: Remote (U.S. Preferred) 10+ years Exp required The Role This is the most senior individual contributor on the platform a true technical leader and XXXX XXXX s right hand. You will own architecture decisions, lead requirements intake across business and analytics teams, and set the technical direction for the data platform build. This is not a people management role, but it requires strong technical leadership, confidence, and the ability to operate effectively with both engineers and non-technical stakeholders. What You ll Do Own Data Architecture: Design and evolve staging and curated data layers, including partitioning strategies, schema evolution, and pipeline architecture across 25+ heterogeneous data sources. Lead Requirements Intake: Partner directly with Analytics, Finance, and Label Operations to translate business needs into scalable data models and pipeline designs. Drive Technical Direction: Mentor and guide a team of up to 8 Data Engineers (contractors) through code reviews, architecture reviews, and enforcement of engineering best practices. Partner on Key Decisions: Work closely with XXXX XXXX to define and document critical architectural choices, including ingestion patterns, data quality frameworks, and platform standards. Hands-On Contribution: Write production-grade code, especially during the foundational build phase. What We re Looking For Must-Have Databricks & PySpark: Deep, production-level experience building scalable data pipelines (not PoC-level). Unity Catalog experience strongly preferred. Google Cloud Platform Expertise: Strong experience with BigQuery, Dataproc, Airflow/Cloud Composer, and Cloud Storage. Experience with GCS as a system of record is a plus. Advanced Data Modeling: Expertise in multi-source canonical modeling, medallion/layered architectures, and entity resolution design. Stakeholder Engagement: Proven ability to lead workshops and requirements sessions with business stakeholders and translate needs into technical solutions. Python Engineering: Production-grade development mindset writing reliable, testable, and maintainable code. Experience Level: 6+ years in data engineering, including at least 2 years operating at a lead or senior-lead level. Nice to Have dbt: Experience working with dbt within a medallion/lakehouse architecture. Domain Experience: Background in music, media rights, or royalties data. Additional Context This is a production-critical engineering role, not a research or experimental position. The platform you build will be heavily relied upon by Finance, M&A, and AI/ML teams from day one. We are looking for engineers who have delivered and supported real-world, business-critical data systems at scale.

Skills

BigQueryAirflowAIMachine LearningGCPData EngineeringMLdbtPythonDatabricks

Want AI to find more roles like this?

Upload your CV once. Get matched to relevant assignments automatically.

Try personalized matching