Enable relevance trigger for an index

Background

The relevance trigger predicts how useful a search result set is, returning a relevanceScore (0-10) in the search response when the client opts in via "relevanceScore": true. The score is produced by a per-index Gradient Boosting Regressor model stored as JSON in Cloudflare KV and walked at request time by the search proxy.

Two things must be in place for an index to receive scoring:

1. The trained model JSON must be uploaded to Cloudflare KV under key relevance_model:<model_id>.
2. The index settings must reference that model via relevance_trigger_config.model_id.

If either is missing, scoring is silently skipped and the response is returned without relevanceScore.

See the design doc for architecture and model details.

Prerequisites

• A trained model exported as JSON via tools/relevance_trigger_demo/export_model.py (in the rohits-solution-investigations repo). Output looks like model-<index>.json, ~70KB.
• The system account ID and index name for the target index.
• AWS console access via Escalator for prod DDB edits.
• Cloudflare console access for KV writes via wrangler, or the API token from secrets manager for scripted uploads.

Process

There are two parts: (1) upload the model JSON to KV, and (2) point the index at the model. They are independent — uploading a model that no index references is a safe no-op, and configuring model_id for an index without a corresponding KV entry simply skips scoring with a warning log.

Part 1: Upload model JSON to KV

The KV key format is relevance_model:<model_id>. The <model_id> convention is <system_account_id>:<index_name>:v<n> — self-describing so you can trace any model in KV back to which index owns it. Do not overwrite an existing key; bump the version and update model_id in settings (see "Updating an existing model" below).

KV namespace IDs (from components/search_proxy/wrangler.toml):

Env	KV namespace ID
Dev	9be217ff35744e68a0b0b756177ecfb2
Staging	b89e4485d04b480fb3a6e1cfc9920c1b
Preprod	b8aae886566f40f1b8441b071ea532ff
Prod	955f4fa05b004210b22a93b9662e85f5

Cloudflare account IDs:

Env	Account ID
Dev/Staging/Preprod	3a8e992c9f607dcb3b401878264df92e
Prod	96d9ae8ad2fe71a3c6ead54f3a775c05

Upload command (per environment)

From the directory containing model-<index>.json:

npx wrangler kv key put \
  --namespace-id "<KV_NAMESPACE_ID>" \
  "relevance_model:<model_id>" \
  --path model-<index>.json

Example for staging:

npx wrangler kv key put \
  --namespace-id "b89e4485d04b480fb3a6e1cfc9920c1b" \
  "relevance_model:<system_account_id>:<index_name>:v1" \
  --path model-photos.json

For prod, set the Cloudflare account by exporting CLOUDFLARE_ACCOUNT_ID=96d9ae8ad2fe71a3c6ead54f3a775c05 first, or pass it via --account-id.

Verify upload

npx wrangler kv key get \
  --namespace-id "<KV_NAMESPACE_ID>" \
  "relevance_model:<model_id>" | head -c 200

Should print the start of the JSON model artifact.

Part 2: Configure the index

Set relevance_trigger_config on the index settings record. There are two paths.

Configuration schema

Field	Type	Required	Description
modelId	string	yes	KV key suffix of the GBR model artifact. Convention: <system_account_id>:<index_name>:v<n>.
titleField	string | null	yes	Field name on each hit to read titles from. Pass null to skip title-derived features (jaccard, title coverage, query-words-in-titles, topic coherence).
tagsField	string | null	yes	Field name on each hit to read tags from. Pass null to skip tag-derived features (frac tag match, avg tag count).

All three fields are required — there is no implicit fallback. If you don't have a tags field on your index, pass "tagsField": null to skip those features.

Field names must match what the model was trained on. Mismatches collapse the corresponding features to 0 and produce meaningless scores.

Option A: Internal admin API (recommended)

relevance_trigger_config is an internal-only knob — model IDs reference Marqo-trained GBR models, not anything the customer can reason about — so it lives behind the internal admin lambda ({env}-AdminInternalApi), not the customer ecom API. Attempting to PATCH relevanceTriggerConfig through the customer /api/v1/indexes/{name}/settings endpoint is rejected with a 422.

The endpoint is PUT /api/v1/admin/accounts/{system_account_id}/indexes/{index_name}/relevance-trigger-config:

# Envato-style index (uses `title` and `tags` fields)
curl -X PUT ".../api/v1/admin/accounts/<system_account_id>/indexes/<index_name>/relevance-trigger-config" \
  -H "Content-Type: application/json" \
  -d '{
    "relevanceTriggerConfig": {
      "modelId": "<system_account_id>:<index_name>:v1",
      "titleField": "title",
      "tagsField": "tags"
    }
  }'

# Shopify-style index (uses `productTitle`; no tags field)
curl -X PUT ".../api/v1/admin/accounts/<system_account_id>/indexes/<index_name>/relevance-trigger-config" \
  -H "Content-Type: application/json" \
  -d '{
    "relevanceTriggerConfig": {
      "modelId": "<system_account_id>:<index_name>:v1",
      "titleField": "productTitle",
      "tagsField": null
    }
  }'

The body uses camelCase (relevanceTriggerConfig, modelId, titleField, tagsField); DDB stores snake_case. All three sub-fields are required — omitting titleField or tagsField returns 422. Unknown sub-fields are rejected with 422 (extra="forbid"). A 409 means another process updated the index since you last read it — retry. To read the current value, GET the same URL.

Option B: Direct DDB edit (prod, via Escalator)

1. Get admin access to the prod Controller account via Escalator.
2. Navigate to the prod-EcomIndexSettingsTable item explorer.
3. Find the record with pk = system account ID, sk = INDEX#<index_name>.
4. Add or update the relevance_trigger_config field (type: Map):

   {
     "model_id": "<system_account_id>:<index_name>:v1",
     "title_field": "title",
     "tags_field": "tags"
   }

   Use snake_case keys here (DDB stores them this way). All three fields must be present. Use null (DDB type NULL) for title_field or tags_field to skip those features.
5. Click Save.

For staging/preprod, use the corresponding staging-EcomIndexSettingsTable / preprod-EcomIndexSettingsTable tables.

Verify the change propagated

Settings are synced from DynamoDB to Cloudflare KV via the ecom_settings_exporter Lambda (triggered by DDB Streams).

1. Open the relevant KV namespace in the Cloudflare console:
   • staging-search-proxy-kv
   • prod-search-proxy-kv
2. Search for the key <system_account_id>-<index_name> (or the shop ID if applicable). Confirm relevance_trigger_config is present in the JSON.
3. Send a test search request with "relevanceScore": true and confirm relevanceScore is in the response:

   curl -X POST "https://<env-search-url>/api/v1/indexes/<index_name>/search" \
     -H "x-api-key: <API_KEY>" \
     -H "Content-Type: application/json" \
     -d '{ "q": "sunset beach", "limit": 10, "relevanceScore": true }'

Aliased indexes

Some indexes use aliases (e.g. envato-photos aliasing envato-photos-20260101). Settings are read from the alias index first, with retrieval happening from the actual index.

Set relevance_trigger_config on both the alias record and the underlying index record. The model JSON in KV is shared — only one upload per model_id regardless of how many index records reference it.

Updating an existing model

The search proxy caches loaded models in worker memory until the worker instance recycles, so always version the model_id rather than overwriting in place:

1. Upload the new model JSON under a new key, bumping the version suffix — e.g. relevance_model:<system_account_id>:<index_name>:v2.
2. Update relevance_trigger_config.model_id on the index settings to the new version (e.g. <system_account_id>:<index_name>:v2).
3. Once you've confirmed the new model is serving, the old v1 key can be deleted (give it ~24h for any lingering worker instances to recycle).

Rollback

To disable scoring for an index without removing it from the system:

• Via internal admin API: PUT .../api/v1/admin/accounts/{system_account_id}/indexes/{index_name}/relevance-trigger-config with body {"relevanceTriggerConfig": null}.
• Via DDB: remove the relevance_trigger_config attribute from the index record.

This causes the scorer to skip silently — search responses no longer include relevanceScore. The model JSON in KV can remain (it's harmless if no index references it).

Troubleshooting

See search-proxy diagnostics for the full troubleshooting checklist.