An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Kafka, Apache Zookeeper, Apache Spark, and Cassandra. All components are containerized with Docker for easy deployment and scalability.
active 2023-09-06 → 2026-05-24 (UTC)
Activity over time
Daily event counts in the loaded window
Line chart, 992 days from 2023-09-06 to 2026-05-24. Pushes: 3 total, peak 2 in a day. Pull requests: 4 total, peak 2 in a day. Issues: 10 total, peak 1 in a day. Comments: 12 total, peak 2 in a day. Stars: 290 total, peak 4 in a day.
- Pushes
- Pull requests
- Issues
- Comments
- Stars
Stars, PRs, issues and forks are under-captured in the later part of this window. GH Archive progressively stopped capturing non-push events during 2026 — −95% or worse by the end of the window. Every series here except Pushes fades for that reason, so a decline above reflects the archive, not this repository. Pushes stay reliable throughout, so read them, and the contributor counts derived from them, as the real signal. Data health has the measurements.
Top contributors
Pushes, PRs, issues, reviews and comments — stars and forks excluded, so this is contribution rather than popularity
| Contributor | Contributions | Pushes | PRs | Comments |
|---|---|---|---|---|
| airscholar | 9 | 3 | 2 | 2 |
| arrismo | 2 | 0 | 0 | 0 |
| oavioz | 2 | 0 | 0 | 1 |
| prikshit-2000 | 2 | 0 | 0 | 1 |
| andrewcaoo | 2 | 0 | 0 | 2 |
| vraj-apto | 2 | 0 | 0 | 0 |
| Ram443 | 1 | 0 | 0 | 0 |
| omursnck | 1 | 0 | 0 | 1 |
| TNQT14 | 1 | 0 | 0 | 0 |
| VuKhoiGVM | 1 | 0 | 0 | 1 |
| maidirepelle | 1 | 0 | 1 | 0 |
| aviravipati | 1 | 0 | 1 | 0 |
| kushagr538 | 1 | 0 | 0 | 1 |
| Kanangnut | 1 | 0 | 0 | 0 |
| ElNino9495 | 1 | 0 | 0 | 1 |
| NikhilLIv | 1 | 0 | 0 | 1 |
| teddythinh | 1 | 0 | 0 | 1 |
| choonhongyeoh0241 | 1 | 0 | 0 | 0 |
Recent activity
Latest issues, pull requests and releases
- Issue comment#8kushagr5382025-03-01 20:02Data is not streamed from Kafka to Cassandra
- Issue comment#4andrewcaoo2025-02-17 09:26Connect with Kafka not able to form
- Issue comment#4andrewcaoo2025-02-17 09:25Connect with Kafka not able to form
- Pull request#10airscholar2025-02-14 13:17
- Pull request#10maidirepelle2025-02-14 05:58
- Issue#9TNQT142025-01-19 06:05Error stream data Kafka to Spark
- Issue#8arrismo2024-11-17 21:47Data is not streamed from Kafka to Cassandra
- Issue#8arrismo2024-11-12 19:38Data is not streamed from Kafka to Cassandra
- Issue comment#6NikhilLIv2024-07-27 18:56Error : During spark-submit I get cassandra module not found error
- Issue comment#7oavioz2024-06-03 16:03ython dags/kafka_stream.py not working
- Issue#7oavioz2024-06-03 13:54ython dags/kafka_stream.py not working
- Issue comment#4omursnck2024-06-01 11:16Connect with Kafka not able to form
- Issue#6prikshit-20002024-05-07 04:58Error : During spark-submit I get cassandra module not found error
- Issue comment#5prikshit-20002024-05-05 05:54Exception has occurred: AirflowConfigException
- Issue comment#4ElNino94952024-04-18 13:18Connect with Kafka not able to form
- Issue comment#4teddythinh2024-03-06 11:23Connect with Kafka not able to form
- Issue comment#4VuKhoiGVM2024-03-03 08:35Connect with Kafka not able to form
- Issue#5Kanangnut2024-02-20 11:41Exception has occurred: AirflowConfigException
- Issue#4choonhongyeoh02412024-02-04 01:24Connect with Kafka not able to form
- Issue#3vraj-apto2023-12-13 17:47Container start error
- Issue comment#3airscholar2023-12-12 19:06Container start error
- Issue#3vraj-apto2023-12-12 18:57Container start error
- Issue#2Ram4432023-12-11 19:03web server is not working,Which dependency.
- Pull request#1airscholar2023-11-08 12:46
- Pull request#1aviravipati2023-10-05 23:33
Totals cover only the window loaded into ClickHouse and count events, not GitHub's lifetime totals — 290 stars here means stars gained during the window, not the repo's star count.