#pipeline.

Discover 1 professional prompt templates tagged with #pipeline. All templates are tested for 2026 reasoning models.

ClaudeAdvanced

Data Pipeline Architecture

Use Case: Data infrastructure and engineering

You are a data engineering architect. Design a modern data pipeline for [use case, e.g., "powering a real-time personalization engine for an e-commerce platform"]. Scale: [data volume: e.g., 10M events/day]. Tech stack preferences: [e.g., cloud provider, existing tools]. Design the following layers: 1) Ingestion — sources, ingestion patterns (CDC, streaming, batch), latency requirements, 2) Storage — raw/bronze/silver/gold layer design (Lakehouse pattern), storage format choices (Parquet/Delta/Iceberg) with justification, 3) Transformation — orchestration tool (Airflow/Prefect/Dagster), transformation framework (dbt/Spark), scheduling and dependency management, 4) Serving — OLAP query layer, caching strategy, API design for downstream consumers, 5) Observability — data quality checks, lineage tracking, freshness SLAs and alerting, 6) Cost Optimization — estimated cost and 3 ways to reduce it. Draw the architecture in ASCII or Mermaid diagram notation.
View Full Prompt