Skip to content

EC528-Fall-2024/async-batch-cloud-llms

View on GitHub ↗Related repositories →

The vision of this project is to create an automated, scalable, and efficient asynchronous batch processing solution for cloud-based Large Language Models (LLMs) at Two Sigma.

active 2024-09-162025-06-19 (UTC)

Complete coverage26,714 / 26,714 hourly files (100%) · 2 absent upstream2023-08-152026-09-01 (UTC)
Events
537
Pushes
247
Pull requests
4
Issues
274
Stars
5
Forks
1

Activity over time

Daily event counts in the loaded window

Line chart, 277 days from 2024-09-16 to 2025-06-19. Pushes: 247 total, peak 19 in a day. Pull requests: 4 total, peak 2 in a day. Issues: 274 total, peak 38 in a day. Comments: 2 total, peak 1 in a day. Stars: 5 total, peak 2 in a day.

  • Pushes
  • Pull requests
  • Issues
  • Comments
  • Stars

Stars, PRs, issues and forks are under-captured in the later part of this window. GH Archive progressively stopped capturing non-push events during 2026 — −95% or worse by the end of the window. Every series here except Pushes fades for that reason, so a decline above reflects the archive, not this repository. Pushes stay reliable throughout, so read them, and the contributor counts derived from them, as the real signal. Data health has the measurements.

Top contributors

Pushes, PRs, issues, reviews and comments — stars and forks excluded, so this is contribution rather than popularity

ContributorContributionsPushesPRsComments
rayan-syed23711400
NoahRobit721466240
BennettTaylor521200
sergio-er493002
ASasamori261700
Ch1nyzzz171200

Recent activity

Latest issues, pull requests and releases

  • Issue#154rayan-syed2024-12-07 06:24
    Operator Dashboard
  • Issue#160rayan-syed2024-12-07 06:24
    UI Notebook fetches errors
  • Issue#145rayan-syed2024-12-05 05:04
    Fix Notebook and Helper Functions
  • Issue#162rayan-syed2024-12-05 04:00
    BigQuery to CSV for user
  • Issue#162rayan-syed2024-12-05 03:59
    BigQuery to CSV for user
  • Issue#161rayan-syed2024-12-04 07:18
    Re-write Reverse Batch Processor for Free-tier Users
  • Issue#160rayan-syed2024-12-04 03:15
    UI Notebook fetches errors
  • Issue#159rayan-syed2024-12-03 19:55
    Calculate and store LLM cost per job
  • Issue#159rayan-syed2024-12-03 06:20
    Calculate and store LLM cost per job
  • Issue#158rayan-syed2024-12-03 03:11
    Query input table in batches & Redis locks for status-writer
  • Issue#157ASasamori2024-12-02 22:20
    Integration with Outside BQT
  • Issue#157ASasamori2024-12-02 22:20
    Integration with Outside BQT
  • Issue#156rayan-syed2024-11-29 06:32
    Store all rows dropped and error messages in Errors doc in Firestore
  • Issue#156rayan-syed2024-11-29 06:32
    Store all rows dropped and error messages in Errors doc in Firestore
  • Issue#155rayan-syed2024-11-29 06:25
    End-time writes in reverse batch processor
  • Issue#155rayan-syed2024-11-29 06:25
    End-time writes in reverse batch processor
  • Issue#154rayan-syed2024-11-29 06:25
    Operator Dashboard
  • Issue#142rayan-syed2024-11-29 06:15
    create a function to get total_count and current_row from firebase
  • Issue#153rayan-syed2024-11-29 06:15
    Debug Firestore Writes
  • Issue#153rayan-syed2024-11-29 03:41
    Debug Firestore Writes
  • Issue#152rayan-syed2024-11-29 03:13
    Run, inspect, and debug large test of 1000 rows
  • Issue#152rayan-syed2024-11-29 03:13
    Run, inspect, and debug large test of 1000 rows
  • Issue#150rayan-syed2024-11-28 08:59
    All microservices send stats to stats-collector
  • Issue#151rayan-syed2024-11-28 01:25
    Add Redis distributed locks to all Firestore writing scripts
  • Issue#150rayan-syed2024-11-28 01:24
    All microservices send stats to stats-collector

Totals cover only the window loaded into ClickHouse and count events, not GitHub's lifetime totals — 5 stars here means stars gained during the window, not the repo's star count.