SQL & Pipelines

Build and debug Apache Flink apps on Amazon Managed Service for Apache Flink

Inventory and audit Glue Data Catalog databases, tables and remote catalogs

Export RDS and Aurora snapshots to S3 as Parquet for analytics and backup

Locate tables and datasets across Glue, S3 Tables and Redshift catalogs

Load S3 files, databases and warehouses into an AWS data lake on Iceberg

Operate Kinesis Data Streams and deliver them to S3 Tables or S3 buckets

Grant, request and audit access to data assets through catalog subscriptions

Write and curate business metadata in SageMaker Catalog or Glue Data Catalog

Migrate Teradata warehouses to Amazon Redshift with schema and SQL conversion

Query VPC Flow, WAF and CloudFront logs with SQL over Iceberg in S3 Tables

Publish Redshift system logs to S3 Tables and query them with Athena

Query S3 object metadata, inventory and Storage Lens data with Athena SQL

Run SQL audits on SageMaker Catalog metadata exported as Iceberg tables

Run Athena SQL queries across Glue, S3 Tables and Redshift catalogs

Write correct Amazon Redshift SQL and avoid PostgreSQL-only habits

Test Amazon MWAA workflows by triggering runs and polling them to completion

Plan Amazon MWAA Airflow upgrades across 2.x and 3.x version jumps

Write correct, efficient KQL queries with joins, datetime handling and graph functions

Query a Slackdump SQLite database directly with the sqlite3 command line

Create, query and manage DataChain datasets over S3, GCS and Azure buckets

Use Feishu Bitable tables as a SQL data source and write results back

Run read-only SQL queries against the tables you have selected for analysis

Ingest GeoJSON, Shapefile or KML files into a PortalJS portal as map tiles

Own data governance, the warehouse, the semantic layer and analytics capability

Design and debug data pipelines with quality checks and orchestration

Move data between systems and reconcile counts so nothing is lost or corrupted

Design warehouse models, grain, dimensions and the semantic metric layer

Write and run SQL queries with multi-dimensional analysis and chart suggestions

Explore database tables, columns and relationships and export M-Schema format

Inspect Airbyte workspaces, connections, jobs and connector definitions from the terminal

Inspect Airflow DAGs, runs, task instances, pools and variables from the terminal

Manage ClickHouse Cloud services and run analytics queries from the command line

Explore an unfamiliar database schema before writing SQL or answering data questions

Write analytical SQL over time-series with TimescaleDB Toolkit hyperfunctions

Design ETL and streaming data pipelines with dbt, Airflow, Spark and Kafka

Parse unknown SQLite schemas and extract table data as structured JSON

Build data pipelines and warehouses with Spark, dbt, Airflow and streaming

Design scalable batch and streaming data pipelines with cost and reliability in mind

Build streaming SQL with windows, aggregates, CDC and CEP rules

Map dbt schema and source YAML into semantic layer models and column notes

Find recurring analytical SQL patterns across tables in historic query logs

Turn historic SQL table usage into typed evidence for schema documentation

Capture semantic layer updates from a live database schema snapshot

Extract metrics and semantic layer proposals from exported Looker dashboards

Convert LookML views, models and explores into semantic layer sources

Convert Metabase questions, models and metrics into semantic layer definitions

Map MetricFlow semantic models and metrics into semantic layer sources

Extract wiki knowledge and metric definitions from Sigma data models and workbooks

Query semantic layer metrics like ARR, churn and retention from YAML sources