Methodology

# Methodology

One store (`www.wandrd.com`), one product family (PRVKE V4), one target

(PRVKE 31L, Black / Bag Only where a variant is required). Sibling colourways and

sizes are captured only to prove identity scope and fragmentation; they are never

merged into target facts. No reseller is treated as merchant truth.

The apex host redirects to `www` (`x-redirect-reason: canonical_host_redirection`);

the capture records the full redirect chain.

• Plain HTTP only (Python stdlib); **no page JavaScript is executed**.
• Requests are paced at ≥0.9 s; 429/503 and "Verifying your connection" challenge

pages are retried with exponential backoff. Challenge pages that remain are kept

as observed failures, never parsed as product data.

• Headers are normalised to lower-case when stored in `raw/http-meta/`.
• Raw captures are immutable: identical re-captures are recorded in the manifest

without rewriting; non-identical re-captures create numbered duplicates.

• The task's shorthand `raw/pdp.html` is provided as a symlink to

`raw/pdp/pdp.html`; the structured layout in `raw/` is authoritative.

Shopify documentation is preserved under `raw/docs/` and is the basis for the inferred

classification. Inferred metadata is never described as merchant-authored.

`normalized/facts.jsonl` carries one fact per line with:

`fact_id`, `subject`, `predicate`, `value`, `unit`, `conditions`, `source`,

`source_field`, `source_url`, `evidence`, `provenance`, `status`, `conflict_group`,

`retrieved_at`.

• Subjects are scoped: `PRVKE 31L V4` (model), `… / Black / Bag Only` (variant),

`… / Photography Bundle (bundle)` (bundle).

• `status` is `SUPPORTED`, `CONTRADICTED`, `CONFLICTED` or `UNKNOWN`; the ledger

generator only emits `SUPPORTED`/`CONFLICTED`, and conflicts are detected by

comparing semantically-normalised values inside a `conflict_group`

(`true`≡`InStock`, numeric strings, cube-model sets). Conflicts are never resolved

silently: every member keeps its own value, scope and source.

• Facts receive a `source_url` and/or `source_field`; `evidence` quotes the field or

value actually observed.

• `analysis/source_matrix.{csv,md}` — 42 semantic fields × 9 source columns with

`EXACT`/`PRESENT`/`PARTIAL`/`TRANSFORMED`/`ABSENT`/`CONFLICT`/`NOT_APPLICABLE` and

back-pointers to ledger fact IDs.

• `analysis/diff.md` — nine-section field-level diff (the primary artifact).
• `analysis/conflicts.{json,md}` — conflict groups with scopes and sources.
• `analysis/provenance.md` — label counts and application rules.
• `analysis/field_coverage.json` — per-field provenance coverage and merchant gaps.
• `data/verification.md` — 14 automated PASS/FAIL checks.

**Shopify / Cloudflare.** WANDRD is behind Cloudflare with Shopify's bot mitigation.

Observed during development: bursty sequential requests (and especially the large

`/products.json?limit=250` listing) can answer `429`/`503` with a

`Verifying your connection…` HTML body and no `Retry-After`. The capture layer therefore:

• sends a current desktop-browser User-Agent (the research token was removed),
• persists a per-host cookie jar in `data/cookies/` so the session established by the

PDP visit is reused (cookies are git-ignored, never committed),

• enforces a per-host minimum interval (default 1.2 s) that *tightens* on any challenge

(×1.8, capped at 30 s) and relaxes after success,

• detects challenges by status (403/429/503), `cf-mitigated: challenge`, and body marks,
• retries up to 5 times with jittered exponential backoff, honouring `Retry-After`

(capped at 180 s),

• records every attempt in `data/capture_manifest.jsonl` and the probe tables,
• keeps any persistent challenge page as a preserved source failure and never parses it

as product data; `/products.json` additionally has a documented fallback to

`/collections/all/products.json` if the listing endpoint stays blocked.

**GitHub.** Authenticated REST quota observed: 5,000 requests/hour, 30 search

requests/minute, plus secondary burst limits (403 + `retry-after`). `scripts/gh.py`

paces calls, reads `x-ratelimit-remaining`/`x-ratelimit-reset` and pauses below a

25-request watermark, backs off on 403/429/5xx, and creates the repository only after a

`GET` confirms it is absent. `scripts/push.sh` retries SSH pushes (not part of the REST

quota) with linear backoff.

• Observations are point-in-time (2026-09-28, US market, USD); prices, availability

and inferred metadata can change without notice.

• The merchant is on Shopify where `/products.json` is capped at 250 records per page;

only page 1 was inspected (sufficient to confirm the target's presence and sibling

records) — the store was not fully enumerated by design.

• Global Catalog search captures are resolver evidence; the persistent evidence set is

the exact `lookup_catalog`/`get_product` captures.

• The UCP `search_catalog` responses are point-in-time and are not intended to be

reused as a permanent search cache.

• Some embedded state (Okendo reviews, web pixels) is merchant-app data served on the

public page; it is classified `PUBLIC_SHOPIFY` as an observed platform surface, not

as a documented product API.

• "Customers Also Purchased" recommendations are rendered from the theme's own state;

their ordering/pricing can vary by traffic and time.

• Image comparison is metadata-only; images are not downloaded or rehosted.