71 lines
2.3 KiB
Markdown
71 lines
2.3 KiB
Markdown
# Pokedex recognition server
|
|
|
|
Self-hosted VLM (Qwen2.5-VL-3B-Instruct) that identifies a Pokemon directly
|
|
from a photo -- no reference-image index needed. See
|
|
`spikes/image_recognition/` for the de-risking spikes that led here
|
|
(embedding/nearest-neighbor matching topped out at 8/12 on our test set;
|
|
this approach got 10/12 cold).
|
|
|
|
## Running it
|
|
|
|
One-time setup (already done on this machine, keeping this here for
|
|
reference/reinstall):
|
|
|
|
```
|
|
py -3.13 -m venv .venv
|
|
.venv\Scripts\python.exe -m pip install --index-url https://download.pytorch.org/whl/cu128 torch torchvision
|
|
.venv\Scripts\python.exe -m pip install "transformers>=4.49" accelerate qwen-vl-utils pillow fastapi "uvicorn[standard]" python-multipart
|
|
```
|
|
|
|
Start the server (manual, current setup -- not auto-starting on boot/login yet):
|
|
|
|
```
|
|
.venv\Scripts\python.exe -m uvicorn server:app --host 0.0.0.0 --port 8420
|
|
```
|
|
|
|
First request after startup is slow-ish (model load, ~a few seconds);
|
|
subsequent requests are fast since the model stays resident on the GPU.
|
|
|
|
## One-time machine setup still needed
|
|
|
|
Run this once, as admin, to let the phone reach the server (not run
|
|
automatically -- it's a firewall/security change):
|
|
|
|
```powershell
|
|
New-NetFirewallRule -DisplayName "Pokedex recognition server" -Direction Inbound -Protocol TCP -LocalPort 8420 -Action Allow -Profile Private
|
|
```
|
|
|
|
Scoped to the `Private` network profile only.
|
|
|
|
## Endpoint
|
|
|
|
`POST /identify` -- multipart form field `file` = the photo.
|
|
|
|
```
|
|
curl -X POST http://<this-machine-LAN-ip>:8420/identify -F "file=@photo.jpg"
|
|
```
|
|
|
|
Response:
|
|
|
|
```json
|
|
{"recognized": true, "species": "Charizard", "raw_response": "Charizard"}
|
|
```
|
|
|
|
`recognized: false` / `species: null` means the model couldn't confidently
|
|
identify a Pokemon -- treat as "not recognized," not an error.
|
|
|
|
`GET /health` -- liveness/readiness check.
|
|
|
|
## LAN address
|
|
|
|
At last check, this machine's Wi-Fi LAN IP was `192.168.68.53`. This can
|
|
change (DHCP) -- if the app stops reaching the server, re-check with
|
|
`ipconfig` before assuming something else is broken. A static IP
|
|
reservation on the router, or mDNS/hostname, would avoid this long-term but
|
|
hasn't been set up.
|
|
|
|
## Known gotcha
|
|
|
|
This machine also runs NordVPN (NordLynx interface). Shouldn't affect LAN
|
|
traffic, but if the phone can't reach the server, check the VPN client's
|
|
local-network-access/LAN-sharing setting first.
|