Skip to content

airscholar/E2EDataEngineering

View on GitHub ↗Related repositories →

This project serves as a comprehensive guide to building an end-to-end data engineering pipeline using TCP/IP Socket, Apache Spark, OpenAI LLM, Kafka and Elasticsearch. It covers each stage from data acquisition, processing, sentiment analysis with ChatGPT, production to kafka topic and connection to elasticsearch.

active 2023-10-28 → 2023-10-28 (UTC)

Complete coverage28,032 / 28,032 hourly files (100%) · 2 absent upstream2023-08-15 → 2026-10-25 (UTC)
Events
3
Pushes
1
Pull requests
0
Issues
0
Stars
0
Forks
0

Activity over time

Daily event counts in the loaded window

Line chart, 1 days from 2023-10-28 to 2023-10-28. Pushes: 1 total, peak 1 in a day. Pull requests: 0 total, peak 0 in a day. Issues: 0 total, peak 0 in a day. Comments: 0 total, peak 0 in a day. Stars: 0 total, peak 0 in a day.

  • Pushes
  • Pull requests
  • Issues
  • Comments
  • Stars

Top contributors

Pushes, PRs, issues, reviews and comments — stars and forks excluded, so this is contribution rather than popularity

ContributorContributionsPushesPRsComments
airscholar1100

Recent activity

Latest issues, pull requests and releases

No issue or PR events — this repo's activity is pushes only.

Totals cover only the window loaded into ClickHouse and count events, not GitHub's lifetime totals — 0 stars here means stars gained during the window, not the repo's star count.