Data/Analytics Lead

Hiring for a next-generation technology company building sovereign, secure, and scalable digital platforms across AI, Web3.0, cloud, and health-tech.

Lead the design and implementation of a modern analytics platform using 100% open-source technologies. Build scalable data pipelines, lake house architecture, and BI infrastructure from scratch. Manage 4-6 data engineers.

Key Responsibilities

Leadership & Strategy

•Lead team of 4-6 data engineers •Define analytics platform architecture and roadmap •Build data platform from scratch using open-source tools •Establish data governance and quality frameworks

Platform Architecture •Design end-to-end analytics platform on open-source stack •Build data lake house on Cloudian object storage •Architect ClickHouse for OLAP (sub-second queries) •Design multi-source ingestion (APIs, databases, SaaS) • Implement real-time and batch pipelines

Data Ingestion & ETL •Build pipelines using Airbyte (CDC, connectors, validation) •Orchestrate workflows with Apache Airflow (DAGs, monitoring, error handling) •Decremental/full-load strategies,•Data Streaming using Kafka

Data Warehouse & Analytics •Optimize ClickHouse (materialized views, distributed tables) •Design dimensional models (star/snowflake schemas) •Build semantic layers for consistent metrics

BI Platform •Deploy Apache Superset for self-service analytics • Integrate Power BI (optional) • Implement Redis caching for performance

Data Quality & Governance • Implement quality frameworks (Great Expectations, Soda) •Build data reconciliation automation •Establish data lineage, catag, compliance ()

Performance & Automation •Write Python automation tools and custom connectors,• Implement CI/CD for data pipelines •Optimize costs and performance