Free, student-built AI coding help for the Open Coding Society, served from donated GTX 1070 GPUs at https://ai.opencodingsociety.com.
Students point GitHub Copilot (or any OpenAI client) at /v1. An admission gateway on EC2 holds extra requests in a fair wait line, so the GPUs only run what they can actually serve. Open WebUI stays at /, with models from both rigs in one dropdown.
Copilot / curl ─HTTPS─▶ EC2 Nginx ─▶ admission gateway ─NetBird─▶ rig orchestrator ─▶ llama.cpp workers (GTX 1070s)
Start at docs/docs.md. Common entry points:
- Getting started: for new team members
- Using the API: curl, Python, Copilot
- Architecture overview: how it all fits together
- Status: what's running right now
- Decisions: why it's built this way
- Conventions: how to write and add docs
Coding agents: read AGENTS.md.
cp .env.example .env # set OCS_API_KEY (ask an operator; .env is gitignored)
python3 scripts/demo.py --pause # live walkthrough: health, auth, a completion, the wait line
./scripts/request.sh "Explain recursion in one sentence."Models: qwen2.5:0.5b (fast) and qwen3.8:27b (quality, reasoning, slower).
| Path | What |
|---|---|
gateway/ |
EC2 admission gateway (FastAPI) |
orchestrator/ |
Rig-side model router (FastAPI) |
infra/ |
systemd units, env templates, and the Nginx site for ec2/ and rig/; NetBird access policies in netbird/ |
benchmark/ |
Speed benchmark and Pareto chart (guide) |
scripts/ |
demo.py, request.sh, check_docs.py |
tests/ |
Offline tests: uv run pytest -q |
docs/ |
Documentation |
evidence/ |
Dated, append-only proof: captures, benchmark runs, weekly reports |