Open source repositories tagged with #data-engineering, ranked by health score.
Always know what to expect from your data.
Event Driven Orchestration & Scheduling Platform for Mission Critical Applications
Prefect is a workflow orchestration framework for building resilient data pipelines in Python.
Incremental engine for long horizon agents 🌟 Star if you like it!
Open-source local-first ETL/ELT studio: visual data pipelines, 290+ connectors, dbt and on-device AI, run on DuckDB.
Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.
C++ accelerated data quality toolkit for Python: CSV parsing, cleaning, schema validation, profiling, and pandas integration.
