Skip to content
All work

Scanned engineering drawings to an auditable process graph

Deterministic P&ID Extraction

A nine-stage computer-vision and spatial-database pipeline that converts scanned piping & instrumentation diagrams into a queryable process graph — components, pipes, tags, and connectivity. The core extraction path uses no machine learning at all. That is the design decision, not a limitation.

Status
In active development
Context
2026 · Independent work
Role
Sole designer & engineer
Code
Private — available on request
01The problem

Every valve, instrument, and line in a process plant is documented on a P&ID, and in most operating companies those drawings are scans. The knowledge is authoritative, it is complete, and it is entirely unavailable to software.

The obvious approach is to train a symbol classifier. For this particular problem the obvious approach is also the wrong one: a trained classifier gives you an answer you cannot explain, cannot reproduce exactly, and cannot trace back to the pixels that produced it.

For engineering drawings that is disqualifying. If a topology is going to inform an isolation decision or a management-of-change review, someone eventually asks why the system believes a particular valve exists — and “the model said so” does not survive that conversation. Auditability here matters as much as accuracy.

02How it works
  1. 01

    Deterministic by construction

    The same PDF always produces the same graph. Nine stages — primitive detection, symbol-group matching, OCR, OCR fusion, page regions, network assembly, cleanup, text binding, confidence scoring — each backed by geometric and statistical method rather than a trained model, and every threshold documented against measured evidence rather than tuned by feel.

  2. 02

    A shape classifier built from contour geometry

    Written from scratch using contour and hole-geometry analysis rather than Hough transforms, identifying valve bodies, tank domes, flanges, and flow arrows directly from raster contours. Hole topology does the work a convolutional filter would otherwise do, and unlike the filter it can be inspected and argued with.

  3. 03

    Four OCR engines, cross-validated

    pdfplumber, Tesseract, Google Vision, and PaddleOCR each read the drawing; results are clustered and cross-checked, and the confidence score comes from engine agreement rather than any single engine's self-report. Models are unreliable narrators about their own certainty — agreement between independent readers is not.

  4. 04

    Opposite-evidence cross-suppression

    Confirmed text deletes false-positive symbol detections, and confirmed symbol geometry deletes false-positive OCR reads sitting inside it. This resolves the genuinely annoying ambiguity class where a valve bowtie gets read as the letters “PED” — each evidence type is used to disprove the other, rather than both being kept and reconciled downstream.

  5. 05

    The graph lives in PostGIS

    Pipe and signal lines are traced and snapped into a proper node-and-edge graph with a topological cycle check. A Postgres generated column computes confidence and routes each result to auto-accept, human review, or reject. Data access is raw parameterized SQL with no ORM — deliberately, so every query that produced a result can be read and re-run.

  6. 06

    Evidence is immutable; humans outrank everything

    Raw OCR tokens and primitive detections persist immutably. A fused layer merges automated results, VLM arbitration, and human-verified corrections in explicit priority order — human above VLM above automated — so no downstream stage can silently overwrite a correction someone made on purpose.

  7. 07

    Where the models are allowed to sit

    Two narrow, fenced roles. A tool-calling agent exposes exactly four functions — graph query, revision diff, SVG overlay, document export — and only ever queries the finished graph; it never sees pixels. An optional vision-language oracle arbitrates only the lowest-confidence OCR disagreements, behind a sha256-keyed replay cache so paid calls are never repeated, and every verdict it renders is exported as a labeled training sample. A small weak-supervised U-Net trains on a remote GPU box over MCP as an auxiliary signal — to assist the deterministic pipeline, not replace it.

Design principle

“The same drawing must always produce the same graph, and every element of that graph must trace back to the pixels that produced it. In engineering work, an answer you cannot audit is not an answer.”
03Why it matters

  • It treats auditability as a first-class requirement rather than something to bolt on, which is the difference between a tool an engineering organization can adopt and one it cannot.

  • Every output is re-derivable from stored raw evidence. Three months later you can still answer why the system believed what it believed.

  • The boundary around the language models is drawn deliberately and enforced structurally — the agent queries structure, the VLM arbitrates only where independent readers already disagree.

  • The output is what downstream industrial work actually needs: isolation and consequence analysis, alarm rationalization, MOC review, and structural ground truth for any model reasoning about the facility.

04Stack
Core
Python 3.11FastAPIOpenCVShapelypypdfium2
Spatial data
PostgreSQLPostGISpgRoutingpgvectorpsycopg3
OCR
pdfplumberTesseractGoogle VisionPaddleOCR
Models
Local VLM (LM Studio)Azure OpenAIMCP tool interfaceU-Net (2.1M)
Orchestration
Docker ComposeIn-process DAG engine

Next project

UNSHelper

Want the architecture in more depth, or a walk through the code? I'm glad to go there.

Get in touch