Lead Data Engineer

Builds and operates the self-serve data platform that lets domain teams produce, publish, and consume data products without bespoke infrastructure work. They do not own data products or pipelines — they own the paved road that makes data products and pipelines cheap to ship.

Core skills

• Cloud data platforms Snowflake, Databricks, BigQuery, Microsoft Fabric, Synapse
• Infra-as-code Terraform, Pulumi, Bicep; Kubernetes, Helm, ArgoCD
• Catalog & discovery Unity Catalog, Purview, DataHub, Collibra, Atlan — these are how data products become findable and addressable in the mesh
• Mesh-platform primitives data product templates (cookiecutter / blueprints), automated provisioning of storage + compute + access + monitoring per product, output-port standards, contract registry
• Identity, access, encryption Entra ID, OAuth/OIDC, attribute-based access control, dynamic data masking, key management
• Observability Prometheus, OpenTelemetry, lineage propagation, cost telemetry per data product
• DevEx & FinOps developer portals (Backstage), chargeback/showback per domain, autoscaling and workload isolation
• AI platform extensions feature stores, model registries (MLflow), vector DBs, governed LLM gateways