Merchandising Diagnostics

Use this when a storefront/search result looks wrong because merchandising did not apply, applied stale rules, applied the wrong profile/global rules, or caused a request failure. This guide codifies the Muji recency investigation from the muji-fixes Claude team (recency-trace@muji-fixes) and folds in the current code paths investigators should inspect before proposing a fix.

Safety Gates

• Treat production merchandising data as customer-impacting. Reads are fine for investigation; writes to DDB, Cloudflare KV, storefront settings, or merchant configuration require explicit human confirmation.
• Prefer Console/Admin UI or controller APIs for approved data fixes. Direct DDB edits are last resort because they bypass validation and audit surfaces.
• Preserve the failing request, response body, and trigger inputs before making any change. A 422/500 can turn into a silent no-op after a partial fix.

Data Path

Key repo files:

Layer	Files to inspect
Admin proxy	components/shopify/admin_server/admin_server/routes/merchandise_routes.py, components/shopify/admin_server/admin_server/services/merchandising_proxy_service.py
Index settings / fields	components/shopify/admin_server/admin_server/routes/ecom_routes.py, components/shopify/admin_server/admin_server/services/index_settings_initialization_service.py, components/ecom_utils/ecom_utils/index_settings_service/index_settings_repository.py
Controller API/storage	components/controller/merchandise/views.py, components/controller/merchandise/services/merchandising_service.py, components/controller/merchandise/services/models.py
Exporter transform	components/merchandising_exporter/src/dynamodb_client.py, components/merchandising_exporter/src/cloudflare_kv_store_client.py, components/merchandising_exporter/src/merchandising_exporter.py, components/merchandising_exporter/src/models.py
Runtime proxy	components/search_proxy/src/merchandising_overrides.ts, components/search_proxy/src/search.ts
Indexed documents	components/ecom_indexer/ecom_indexer/lambda_function.py, components/ecom_indexer/ecom_indexer/document_operations.py, components/ecom_indexer/ecom_indexer/document_operations_test.py
Existing docs	docs/merchandising/scheduled-rules-design.md, docs/runbooks/diagnostics/components/controller.md, docs/runbooks/diagnostics/flows/settings-sync.md

Triage Checklist

1. Identify the surface.

   • Is it search, collection/page, subcollection, featured products, instant search, or an admin preview?
   • Capture merchandisingTriggerContext, merchandisingTrigger, profileId, index alias, storefront hostname, request body, and response.
   • Do not assume every storefront surface calls the same endpoint or carries the same market/profile/filter parameters.

2. Establish whether the failure is "rule missing", "rule wrong", "rule stale", or "rule rejected".

   • Missing/wrong result order usually points to DDB rule shape, KV export, proxy merge, document fields, or backend interpretation.
   • 4xx from Marqo often means the exporter/proxy produced a payload Marqo's request models reject. Preserve the validation detail.
   • Stale behavior usually points to #static.updated_at, exporter cadence, KV contents, or search_proxy cache.

3. Compare a broken trigger with a known-good trigger.

   • Same index and context is ideal, for example page#skirt-pants versus page#shoes.
   • Compare rules, recency, scoreModifiers, filterString, inheritGlobalScoreModifiers, inheritGlobalFilters, reinforcementLearning, and updatedAt.

4. Verify the DDB source of truth.

   • The static export marker is pk = "#static", sk = "{system_account_id}-{index_name}".
   • The per-index merchandising config row uses pk = "{system_account_id}", sk = "CONFIG#{index_name}".
   • Default profile trigger rows use pk = "{system_account_id}#INDEX#{index_name}", sk = "RULES#{context}#{trigger}".
   • Profile rows use sk = "PROFILE_RULES#{context}#{profile_id}#{trigger}".
   • Global context rows use sk = "GLOBAL_RULES#{context}".
   • Rule execution settings use sk = "RULE_SETTINGS"; recency field selection lives here as recencyField, not on each trigger.

5. Verify the exporter transform.

   • dynamodb_client.py parses DDB rows into exporter models.
   • _parse_recency_rule() reads rules.recency.weight, after, and before, then combines them with RULE_SETTINGS.recencyField and CONFIG.contexts[context].recency_settings.
   • cloudflare_kv_store_client.py serializes trigger entries with pin_rules, exclude_rules, filter_string, score_modifiers, recency, reinforcement_learning, override_relevance, profile rules, A/B tests, and scheduled overrides.

6. Verify proxy resolution.

   • search_proxy/src/merchandising_overrides.ts reads KV, normalizes context and trigger, resolves profile-specific entries, merges globals where inheritance is enabled, and returns a ResolvedMerchOverride.
   • search_proxy/src/search.ts attaches the resolved object as _merch_override_rules for profile-aware requests.
   • setRecencyApplyInRankingPhase() only changes an existing recencyParameters.applyInRankingPhase; it does not create recency parameters by itself.

7. Verify document fields when score modifiers or filters appear correct but do nothing.

   • Merchandising fields are tracked in add_docs_config.merchandising_fields.
   • POST /indexes/{index}/config/merchandise/fields updates config but its docstring warns it does not backfill existing documents.
   • Some settings update paths trigger index_service.reindex_fields() when net-new merchandising fields are added. Confirm which API path was used before assuming the document maps were populated.

Read-Only DDB Queries

Use placeholders and the correct AWS profile for the environment. These commands are examples of what to inspect; do not mutate data without approval.

aws dynamodb get-item \
  --region us-east-1 \
  --table-name prod-MerchandisingTable \
  --key '{"pk":{"S":"<system_account_id>#INDEX#<index_name>"},"sk":{"S":"RULES#page#<trigger>"}}'

aws dynamodb get-item \
  --region us-east-1 \
  --table-name prod-MerchandisingTable \
  --key '{"pk":{"S":"<system_account_id>"},"sk":{"S":"CONFIG#<index_name>"}}'

aws dynamodb get-item \
  --region us-east-1 \
  --table-name prod-MerchandisingTable \
  --key '{"pk":{"S":"<system_account_id>#INDEX#<index_name>"},"sk":{"S":"RULE_SETTINGS"}}'

aws dynamodb get-item \
  --region us-east-1 \
  --table-name prod-MerchandisingTable \
  --key '{"pk":{"S":"#static"},"sk":{"S":"<system_account_id>-<index_name>"}}'

If a profile is involved, inspect both the default and profile rows. A missing profile row can intentionally fall back to default/global rules in the worker.

Common Failure Modes

Symptom	Likely place to inspect	Notes
Marqo returns 422 mentioning recencyParameters.addToScoreWeight	DDB rules.recency, exporter _parse_recency_rule()	Stored recency weight must be positive for Marqo. A zero slider should usually omit recency rather than export weight 0.
Recency rule saved but no recency applies	RULE_SETTINGS.recencyField, CONFIG.contexts[context].recency_settings	Missing recencyField causes exporter recency parsing to return None. Missing context settings falls back to defaults rather than dropping the rule.
Boost/bury rule exists but ranking does not move	add_docs_config.merchandising_fields, indexed docs, reindex path	Adding a merchandising field after documents exist requires a reindex/backfill of internal merchandising maps.
Correct default rules but wrong profile behavior	PROFILE_RULES#..., profileId, inherit flags, resolveProfileOverride()	Profile entries are embedded into trigger KV entries; global inheritance is pre-merged by search_proxy for profile requests.
Global filters/modifiers missing	GLOBAL_RULES#{context}, inherit flags, exporter global parse	Globals are context-specific. Search and page globals are separate rows.
Rule saved but storefront still uses old behavior	#static.updated_at, exporter run, KV entry, search_proxy cache	The exporter discovers changed indexes through the #static row and writes KV on its schedule. search_proxy also caches KV reads.
Trigger works in search but not collection page, or vice versa	Request URL/body and trigger context	Context and surface matter. search and page rows are different keys and may pass through different frontend request builders.
Trigger key cannot be found	Normalization/hash, alias/index mapping	KV trigger keys use normalized context/trigger and a hash suffix. DDB stores the readable RULES#{context}#{trigger} key. Also distinguish storefront alias from internal index name.

Muji Case Study: skirt-pants Recency 422

The muji-fixes Claude team had a member named recency-trace@muji-fixes assigned to investigate why the Muji US skirt-pants collection failed. The root cause was a stored recency block with weight = 0:

{
  "recency": {
    "weight": 0,
    "after": 14,
    "before": 0,
    "sliderValue": 0
  }
}

The failing DDB row was:

table: prod-MerchandisingTable
pk: 9z6rkf1k#INDEX#shopify-muji-us
sk: RULES#page#skirt-pants

RULE_SETTINGS for the same index had recencyField = "productPublishedAtUnix", so the exporter built a recency payload and mapped the stored weight directly to addToScoreWeight = 0. That value propagated through Cloudflare KV and the search proxy into the Marqo backend, where request validation rejected it with number.not_gt.

Useful comparisons from the same incident:

• page#shoes and page#womens-tops had recency weight 262144 and worked.
• page#bags had no recency block and worked.
• The broken row also had pins and score modifiers, so deleting the whole row would have removed unrelated merchandising behavior.

Preferred remediation pattern:

1. If slider value 0 means recency disabled, remove the rules.recency block through Console/Admin UI or an approved controller/API path.
2. Add code protection at the authoring/validation layer so zero-weight recency is not stored as an active rule.
3. Add exporter-side protection so legacy/bad rows do not poison runtime KV.
4. Scan for other rows with rules.recency.weight <= 0 before calling the fix complete.

Fix Ownership

Finding	Likely owner/change
Bad stored row only	Approved merchant/config data fix via UI/controller; direct DDB only with explicit approval.
UI sends disabled controls as active rules	Console/Admin UI should omit inactive recency or score-modifier blocks.
Controller accepts invalid rule shape	components/controller/merchandise/services/models.py validation and controller tests.
Exporter turns legacy bad data into invalid runtime payload	components/merchandising_exporter/src/dynamodb_client.py guards and exporter tests.
Worker/profile/global merge is wrong	components/search_proxy/src/merchandising_overrides.ts and search_proxy tests.
Boost/filter references a field not indexed for merchandising	Admin settings path plus reindex/backfill, usually touching ecom_routes.py, index settings models, and ecom indexer tests.

Handoff Template

When handing a merchandising issue to another agent or dev, include:

• Merchant, index alias, internal index name, system account ID, environment.
• Exact failing surface and request: path, query params, body, headers that affect context/profile/market, and response body.
• DDB rows inspected: #static, CONFIG, RULE_SETTINGS, default trigger, profile trigger, and global context rows.
• KV evidence or exporter evidence, including timestamps.
• Broken trigger versus working trigger diff.
• Whether fields involved in score/filter rules are present in add_docs_config.merchandising_fields and whether affected docs were reindexed after the field was added.
• Proposed fix layer and the approval needed before any production write.