dsh-design-qa

Why I built it

I wanted a product-design capability on DeepSeek Harness: start from a design mock, write the implementation, judge how close it looks, fix and re-judge until it converges. The deadliest link in that loop is judgment — and DeepSeek is a text-only model. It writes the code but can see neither the mock nor the page it just built.

So I built this plugin: images never enter the model's context. The model sees a one-line text pointer, composes its own questions, and calls the deepseek_vision tool, where a real vision model (default qwen3.8-max, roughly 0.037 CNY per judgment) looks on its behalf and brings back words.

Why the model holds the right to ask

The eval forced this design. Same model, same images, only the phrasing changed: open-ended "find the differences yourself" recalled 0/2 hard defects, while "what is the value in each" reached 2/2. How much detail an image gets seen at depends on what you ask it — which is also a death sentence for auto-describing images on arrival: the description happens before the question exists, so it can only ever ask the worst-scoring kind of question.

The tool only lends the eye; the discipline of what makes a trustworthy question lives in the skill text. And one account that keeps getting conflated: caching what was seen is a liability, at worst a source of hallucination; caching what was asked and answered is an asset.

Why I dared to patch the DSH core

DSH flat-out rejects images for text-only models: the whole image pipeline exists, minus one admission check on whether the selected model declares image support — pipe laid, valve shut. Installation patches the core in three places instead of registering a new model option, on two product judgments: users think "one key gets me one model", not "a plugin grows a model option", and wedging a capability into the account list is a category error; and this core will be changed anyway — I only connect capability that is written but never opened up.

I know it is a short-term solution, so it ships with backups, one-click rollback and a daily dry-run check. The day DeepSeek officially opens image input, the tool finds its caller can see, refuses itself and steps aside, and the three patches restore in one command. Patches are consumables; the tool and the eval are assets.

The other half: is this eye fit to judge

Able to see is not usable for judgment: a model may read every word on an image yet fail to look, confabulate, wobble, or mumble. So the other half of the project is an eval: four fixture sets — landing page, dashboard, list, form — with 23 known defects (9 hard / 8 medium / 6 easy) and four pass lines matching those four failure modes. Only a clean sweep qualifies the eye for the production judging loop.

The eval holds itself to a method too: "passed" does not mean "verified" — first prove the test was able to fail. A known-blank image lives permanently in the set as a negative control; two-way directional questions score only after 3 full runs; the installer's dry-run check fails when every target reports "already patched". Raw model outputs live in eval/runs/ for cell-by-cell rechecking, and the gaps not yet closed are stated plainly in the README.

The story behind it

Building this eval, I thought I had verified something three times and had verified nothing — two rounds run on blank images, a single hit logged as a capability, an installer tested where it was already installed. The full crash-and-correction record is in the essay Borrowed Eyes.

dsh plugin --profile web add dsh-design-qa
cd "${DSH_HOME:-$HOME/.dsh}/profiles/web/node_modules/dsh-design-qa"
export BAILIAN_API_KEY=<your-key>
node install.mjs --route-only
node install.mjs --restart