Prolo — RFQ Matching AI Strategy (Deliverable 4, matching side)

↳ RFQ data flow ↳ Ingestion AI strategy

Matching is not one LLM call — a three-model retrieve-then-rerank stack: an embedding model (recall), a cross-encoder (precision), and an LLM (disambiguate the hard tail). All three off-the-shelf, prompt/config-driven — not fine-tuned.

flowchart LR
    N["SQL narrow
(no AI)"]
    K["kNN recall
embeddings · pgvector"]
    R["cross-encoder rerank
precision"]
    L["LLM disambiguate
Bedrock · hard tail only"]
    V["validate
schema + unit/qty checks"]
    P[("persist line_item_matches
roll up → MATCHED · submit")]
    H["HUMAN REVIEW"]

    N --> K --> R
    R -- clear winner --> V
    R -. unclear .-> L --> V
    V -- ok --> P
    V -.fail / low conf.-> H

    classDef ai fill:#fde047,stroke:#ca8a04,color:#000
    class K,R,L ai

    style N fill:#eef4ff,stroke:#3b7ddd,stroke-width:1.5px,color:#1f2328
    style V fill:#fff8e6,stroke:#bf8700,stroke-width:1.5px,color:#1f2328
    style P fill:#eef1f4,stroke:#57606a,stroke-width:1.5px,color:#1f2328
    style H fill:#fdeef0,stroke:#d1242f,stroke-width:1.5px,color:#1f2328

Why a funnel

Cheap/broad first (kNN over the corpus), expensive/narrow last (LLM on 2–3 items) → the priciest model runs least.
Off-the-shelf everything: managed embeddings + pretrained cross-encoder + stock Claude.
Defer fine-tuning — but if any, the cross-encoder first (confirmed/rejected matches are free labeled pairs).

Why three, not one

Embeddings — good recall, weak precision (near-synonyms rank alike).
Cross-encoder — precise but O(N); needs a kNN shortlist to reorder.
LLM — best at reasoning (bundles, units, quantity) but can't scan and is pricey → runs only on the ambiguous tail.

The three model roles

Layer	Model	Job	Trained / prompted
Recall	Embedding (Titan / Cohere)	vectorize → `pgvector` kNN shortlist	pretrained, no prompt
Precision	Cross-encoder (jina-style)	jointly score each (quote, RFQ) pair, reorder top-K	pretrained (fine-tune candidate)
Disambiguation	Bedrock Claude	resolve bundles / units / ambiguity, emit confidence	prompt-engineered

LLM-layer techniques

Schema-constrained → returns a MatchDecision, never prose.
Grounding — choose only among candidates; a confident no-match beats a forced one.
Confidence rubric (0–1) on description + unit + quantity alignment.

Show the reranked shortlist, not the corpus — tiny tokens.
Unit rules — never silently match per m ↔ per box.
Deterministic checks override — bad unit / out-of-range qty forces low confidence.

Templated prompts

Same discipline as ingestion — versioned YAML, placeholder interpolation, assistant prefill. A small family by scope:

Template	Used when	Returns
`matcher/disambiguate_line.yaml`	shortlist ambiguous (top pair not clearly best)	one RFQ line (or none) + confidence
`matcher/split_bundle.yaml`	one quote line covers several RFQ lines	multiple RFQ lines + per-edge confidence

matcher/disambiguate_line.yaml

prompt: |
  You match ONE supplier-quote line item to the customer's requested RFQ lines
  for construction materials. You are given the quote line and a SHORTLIST of
  candidate RFQ lines already retrieved and reranked by relevance.

  RULES
  - Choose the SINGLE best-matching RFQ line ONLY IF it is genuinely the same
    product/spec. If none truly matches, return an empty match with a reason —
    a confident no-match is better than a forced wrong one.
  - Respect units: do not match incompatible units of measure (e.g. "per m" to
    "per box") unless a stated conversion makes them equivalent; if unsure, lower
    confidence and explain.
  - Rate confidence 0.0–1.0 by how well DESCRIPTION + UNIT + QUANTITY align.

  QUOTE LINE:
  {{QUOTE_LINE}}

  CANDIDATE RFQ LINES (id · description · unit · qty · rerank_score):
  {{CANDIDATES}}

  OUTPUT FORMAT — valid JSON only, no prose:
  {{OUTPUT_SCHEMA}}

# response prefill: force straight into the JSON object
assistant_prompt: |
  {"quoteLineId":

{{CANDIDATES}} is the post-rerank top-K (not the whole corpus); {{QUOTE_LINE}} is the one line being resolved.

Schema Design — MatchDecision

The LLM returns a decision object (distinct from the ingestion SupplierQuoteObject) that maps 1:1 onto the match rows the matcher persists.

MatchDecision — JSON Schema (draft-07)

{
  "$id": "…/match-decision.json",
  "title": "MatchDecision",
  "type": "object",
  "required": ["quoteLineId", "matches"],
  "properties": {
    "quoteLineId": { "type": "string" },   // line being resolved
    "matches": {                            // 0 = no match; >1 = bundle
      "type": "array",
      "items": { "type": "object",
        "required": ["rfqLineId", "confidence", "reason"],
        "properties": {
          "rfqLineId":  { "type": "string" },
          "confidence": { "type": "number", "minimum": 0, "maximum": 1 },
          "reason":     { "type": "string" }
        }
      }
    }
  }
}

MatchDecision	match row
`quoteLineId`	`line_item_id`
`matches[].rfqLineId`	`rfq_id` + `rfq_line_id`
`matches[].confidence`	`match_confidence`
`matches[].reason` (+ scores)	`match_evidence` (jsonb)
empty `matches`	no row → Human Review

Empty matches = first-class confident no-match.
Many matches = a bundle → several edges (many-to-many).

The two retrieval models

Embedding — shared space (mandatory)

Reuse the same model + dimension ingestion uses — quote lines embedded at Stage 4, RFQ lines at sync; different models → kNN meaningless.
Pin once; changing it means re-embedding both corpora.
Normalize before embedding so vectors capture the product, not layout.

Cross-encoder — pretrained precision

Local CrossEncoder on the Fargate worker; scores (quote, RFQ) pairs jointly to separate near-synonyms.
The one model worth fine-tuning first (labeled pairs are free from Human Review) — but ships stock day one.
Local → no Bedrock quota; only embeddings + LLM count against it.