bash — ~/projects/paper-pen-tablet

$cd ~/projects/paper-pen-tablet && cat summary.txt

Point a camera at a sheet of white paper, hold a normal pen, and use it as a graphics tablet — no digitizer, no modified pen, no extra hardware beyond a camera. Your Android phone works fine.

PAPER-PEN TABLET

Computer Vision Developer · #3c9f614 vision

Python · OpenCV · PySide6 · pytest github.com/sleepyhead000/paper-pen-tablet

~/projects/paper-pen-tablet $

cat README.md

README.md

$cat pipeline.txt

camera frame → paper detection → homography warp (top-down A4 view) → pen tip detection (classic CV: contours + width-taper) → pen-down signals (shadow gap / fresh ink, experimental) → One Euro filter + coasting through occlusions → canvas strokes / mouse events / CSV log.

$cat capture.txt

One FrameSource abstraction covers a local webcam, a camera index, a phone streamed over HTTP via IP Webcam, or an offline video file for repeatable re-runs. paperpen bench reports capture FPS and end-to-end glass-to-glass latency against any of them.

$cat output.txt

The same normalized coordinates drive three outputs: a PySide6 drawing canvas (Space as the pen key), system-wide mouse/pointer control for other apps, and a per-frame CSV log under data/logs. Vision code is decoupled from the UI so it can move to ONNX/native later.

$cat evaluation.txt

Printable ground-truth shapes plus scorers measure trace error in millimetres and pen-down detection against key presses; paperpen label collects tip keypoints for the detector milestone. 125 unit/integration tests exercise the pipeline on synthetic scenes.

~/projects/paper-pen-tablet $

ls -l features/

paper

Paper tracking · corner detection + homography warp, flow-tracked between redetects

tip

Pen-tip detection · classic CV — contours + width taper · any ordinary pen

pen-down

Pen-down signals · shadow gap / fresh ink heuristics (experimental)

filter

One Euro filter · smooths jitter · coasts through occlusions

outputs

Three outputs · PySide6 canvas · system mouse · per-frame CSV log

source

Any camera · webcam · camera index · phone (IP Webcam) · video file

bench

Benchmarking · capture FPS + glass-to-glass latency readout

evaluate

Ground truth · printable shapes · mm trace scoring · pen-down scoring

testing

125 tests · pytest unit/integration on synthetic scenes

~/projects/paper-pen-tablet $

cat stack.yaml

language

Python 3.12+ · typed config dataclasses · JSON config file

vision

OpenCV · paper corner detection · homography · contour + taper tip detection

PySide6 · worker thread · stroke canvas widget

input

FrameSource — webcam · camera index · IP Webcam HTTP stream · video file

filtering

One Euro filter · coasting through occlusions · pen-down signals

testing

pytest (125 tests) · printable ground-truth shapes · CSV scorers

~/projects/paper-pen-tablet $

git log --oneline

#6ad2f80calibrate

Paper detection & homography warp

OpenCV · corner detection · optical flow

Detects the paper's four corners and warps to a canonical top-down A4 view (1240×1754 at ~150 DPI), so tracked coordinates are resolution-independent.
PaperTracker reuses optical flow between redetects instead of searching every frame, keeping the loop cheap enough for live use.
paperpen view shows the live camera with an FPS readout and debug overlay for setup and framing.

#b41c9e7track

Tip detection, pen-down & filtering

classic CV · One Euro filter · pen-down heuristics

The pen tip is found with contours and width-taper heuristics — no digitizer, no modified pen, no trained model required.
Experimental pen-down signals (shadow gap / fresh ink) are scored against key ground truth with a dedicated scorer.
A One Euro filter smooths jitter while keeping fast strokes responsive, and coasting keeps a stroke alive through short occlusions.

#f08e5d3output

Canvas, mouse control & CSV logging

PySide6 · input capture · CSV logger

One set of normalized coordinates drives a drawing canvas, system mouse control (Space = pen key), and a per-frame CSV log.
--mouse mode turns the setup into a real pointing device usable in other applications.
The vision core is decoupled from the UI, so it can be ported later without touching detection code.

#19ba7c2evaluate

Ground-truth scoring & test suite

pytest · printable shapes · benchmark CLI

Printable test shapes plus scorers report trace error in millimetres; a second scorer grades contact signals against key presses.
125 unit/integration tests run the pipeline over synthetic scenes — milestones M0–M8 are code-complete.
Still pending on hardware: phone latency, real-paper tracing accuracy, and the pen-down method shootout.