Data Observability Architect

Architect and implement a comprehensive data observability framework leveraging Open Metadata, Great Expectations, and Open Lineage. Customize and extend open-source observability tools using Python/TypeScript/Java to meet enterprise-specific needs. Integrate data observability pipelines with existing data platforms, data warehouses, and workflow orchestration tools (e.g., Airflow, dbt, Spark). Design lineage tracking and metadata collection strategies across ingestion, transformation, and consumption layers. Establish data quality checks and automated validation suites using Great Expectations. Collaborate with data engineering and DevOps teams to integrate observability metrics into Prometheus, Grafana, and Loki dashboards for unified visibility. Define and enforce data reliability SLAs and SLOs, establishing proactive alerting and incident management workflows. Provide technical leadership and mentoring to engineering teams on best practices for data monitoring, lineage, and governance. ________________________________________ Strong hands-on experience with Open Metadata, Great Expectations, and Open Lineage. Proven ability to extend and customize these frameworks using Python/TypeScript/Java. Solid understanding of data architecture, ETL/ELT pipelines, and metadata-driven design. Proficiency in API integrations. Experience integrating observability into modern data stacks (e.g., Snowflake, Databricks, Kafka, dbt, Airflow). Knowledge of OpenTelemetry ecosystem (Prometheus, Grafana, Loki) for infrastructure and application monitoring. Familiarity with CI/CD, Kubernetes, and cloud platforms (AWS/Azure/GCP). Experience in implementing data governance and cataloging frameworks.