bash — ~/projects/prediction-engine

$cd ~/projects/prediction-engine && cat summary.txt

Predicts questions for Udvash weekly exams: a 12-stage NLP pipeline that ingests exam PDFs and Bengali board-book scans, builds a knowledge graph and vector index, then ranks and generates predicted questions with an evaluation scorecard.

PREDICTION ENGINE

ML Pipeline Engineer · #6e4b2f9 research

Python · PyMuPDF · easyocr · ChromaDB · sentence-transformers · pytest github.com/sleepyhead000/prediction-engine

~/projects/prediction-engine $

cat README.md

README.md

$cat pipeline.txt

Twelve disk-backed stages: ingest exams → OCR books → chunk + anchor → entities → concepts → graph → index (ChromaDB) → classify → features → score → generate → evaluate. Every stage writes its own artifact, so a run can restart from any point without redoing the work before it.

$cat data.txt

Sources are read-only: English exam PDFs parsed with PyMuPDF and Bengali board-book scans read by easyocr. Pydantic models define the JSON contract between stages; ground_truth/ holds labels and test data; data/, cache/ and out/ are regenerated and git-ignored.

$cat llm.txt

LLM calls go through a local API proxy with a cache keyed on sha256(prompt + model). temperature=0 for classification and extraction, and only generation uses temp>0. Model caches are pushed off the C: drive by the setup script so they never eat the system partition.

$cat rules.txt

Non-negotiable rules: the parser must pass its gate before any prediction-model work begins, raw data is never deleted, every stage writes to disk independently, and design decisions are documented in files/ — architecture, data contracts, evaluation protocol, prompts and a task backlog.

~/projects/prediction-engine $

ls -l features/

ingest

Exam ingest · PyMuPDF text extraction → data/01_questions.json

ocr

Bengali OCR · easyocr over board-book scans → book_pages.jsonl

graph

Knowledge graph · entities, concepts and book graph as separate stages

index

Vector index · ChromaDB over anchored book chunks

classify

Labelling · classification stage writes 03_labelled.json

score

Ranking · feature extraction + scoring → 05_ranked.json

generate

Prediction · LLM generation → out/prediction_wN.json

eval

Evaluation · scorecard per run → out/scorecard_wN.json

testing

pytest + ruff · setup_env.ps1 bootstraps a new machine end-to-end

~/projects/prediction-engine $

cat stack.yaml

language

Python 3.11+ · Pydantic schemas · httpx

text

PyMuPDF (PDF) · easyocr (scans) · selectolax · lxml · rapidfuzz

retrieval

ChromaDB · sentence-transformers · chunk + anchor stages

llm

LocalAPI proxy · sha256 prompt cache · temperature=0 for extraction

quality

pytest · ruff · gate-before-model-work rule · disk-backed stages

docs

files/ — architecture · data contracts · evaluation protocol · prompts · backlog

~/projects/prediction-engine $

git log --oneline

#2f7c9a5stage/ingest

Ingest & OCR stages

PyMuPDF · easyocr · read-only sources

English exam PDFs are parsed to text and written to disk as the first pipeline artifact — everything downstream reads files, not the raw PDFs.
Bengali board-book scans go through OCR into JSONL, then chunk + anchor stages prepare them for retrieval.
Raw sources are read-only and never deleted, per the project's non-negotiable rules.

#a83d1e6stage/graph

Entities, concepts & knowledge graph

Pydantic · JSON contracts · ChromaDB

Entity, concept and book-graph stages each write their own JSON, with Pydantic models guarding every contract between them.
Anchored chunks are indexed into ChromaDB for retrieval during ranking.
Because stages are independent, a schema change regenerates one file instead of the whole run.

#5c0b7f2stage/score

Classify, features & ranking

classification · feature extraction · scoring

Classification writes labelled data; feature extraction and scoring produce a ranked list per exam.
Every intermediate file stays inspectable — no hidden in-memory state carries between stages.
Temperature stays at 0 for classification and extraction so runs are reproducible.

#e71a4d8stage/eval

Generation & evaluation scorecard

LLM proxy · cached prompts · pytest

Generation writes out/prediction_wN.json and evaluation writes a scorecard alongside it for comparison across runs.
LLM calls are cached on sha256(prompt + model), so re-running an evaluation costs almost nothing.
Eleven design docs in files/ (START-HERE, architecture, data contracts, evaluation protocol, prompts, backlog) keep the pipeline reviewable.