Tagged data-pipelines

Run fast lazy DataFrame queries in Polars for ETL, analytics and pandas migration

Watch CDP destinations, batch exports and hog flows for delivery failures and stalls

Build CocoIndex ETL flows that embed documents into vector databases

Design ETL pipelines, data models and quality checks with Airflow, dbt and Spark

Build ETL and ELT pipelines with Airflow, dbt, Spark and Kafka

Build Windmill data pipelines from scripts wired together with on and materialize

Build ETL pipelines with Spark, Airflow and dbt, plus data quality checks

Draw flowcharts, model structure and data pipeline diagrams as PDFs for a paper

Build data pipelines with Airflow, Dagster, Kafka, Flink and dbt with quality checks

Build fine-tuning datasets from production logs and frontier model outputs

Design and debug data pipelines with quality checks and orchestration

Design ETL and streaming data pipelines with dbt, Airflow, Spark and Kafka

Build data pipelines and warehouses with Spark, dbt, Airflow and streaming

Design scalable batch and streaming data pipelines with cost and reliability in mind

Create validated architecture and sequence diagrams as explorable HTML with exports

Set up end-to-end data sync pipelines on the TIS data integration platform

Process large datasets for ML with streaming Ray Data pipelines across CPUs and GPUs