2.3 KiB
Pokedex recognition server
Self-hosted VLM (Qwen2.5-VL-3B-Instruct) that identifies a Pokemon directly
from a photo -- no reference-image index needed. See
spikes/image_recognition/ for the de-risking spikes that led here
(embedding/nearest-neighbor matching topped out at 8/12 on our test set;
this approach got 10/12 cold).
Running it
One-time setup (already done on this machine, keeping this here for reference/reinstall):
py -3.13 -m venv .venv
.venv\Scripts\python.exe -m pip install --index-url https://download.pytorch.org/whl/cu128 torch torchvision
.venv\Scripts\python.exe -m pip install "transformers>=4.49" accelerate qwen-vl-utils pillow fastapi "uvicorn[standard]" python-multipart
Start the server (manual, current setup -- not auto-starting on boot/login yet):
.venv\Scripts\python.exe -m uvicorn server:app --host 0.0.0.0 --port 8420
First request after startup is slow-ish (model load, ~a few seconds); subsequent requests are fast since the model stays resident on the GPU.
One-time machine setup still needed
Run this once, as admin, to let the phone reach the server (not run automatically -- it's a firewall/security change):
New-NetFirewallRule -DisplayName "Pokedex recognition server" -Direction Inbound -Protocol TCP -LocalPort 8420 -Action Allow -Profile Private
Scoped to the Private network profile only.
Endpoint
POST /identify -- multipart form field file = the photo.
curl -X POST http://<this-machine-LAN-ip>:8420/identify -F "file=@photo.jpg"
Response:
{"recognized": true, "species": "Charizard", "raw_response": "Charizard"}
recognized: false / species: null means the model couldn't confidently
identify a Pokemon -- treat as "not recognized," not an error.
GET /health -- liveness/readiness check.
LAN address
At last check, this machine's Wi-Fi LAN IP was 192.168.68.53. This can
change (DHCP) -- if the app stops reaching the server, re-check with
ipconfig before assuming something else is broken. A static IP
reservation on the router, or mDNS/hostname, would avoid this long-term but
hasn't been set up.
Known gotcha
This machine also runs NordVPN (NordLynx interface). Shouldn't affect LAN traffic, but if the phone can't reach the server, check the VPN client's local-network-access/LAN-sharing setting first.