Ecommerce Platform

• Infra code: infra/ecom/
• Component code: components/shopify/, components/ecom_indexer/, components/ecom_settings_exporter/, components/ecom_metrics_consumer/, components/ecom_monitoring_service/

Product search for ecommerce platforms (primarily Shopify). Includes indexing pipeline, search proxy, and admin tools.

Architecture

graph TD
    A["Shopify Store"]

    subgraph proxy["Cloudflare Worker (search_proxy)"]
        B["Marqo (via service binding)"]
        C["KV (settings cache)"]
        D["SQS (metrics)"]
    end

    A --> B
    A --> C
    A --> D

    E["Shopify Admin"] --> F["API Gateway (Ecom API)"] --> G["Admin Lambda"] --> H["DDB tables"]
    I["Shopify Webhooks"] --> G --> J["Webhook Worker Lambda"] --> K["Indexer Lambda"] --> L["SQS"] --> M["Marqo"]

    N["DDB Stream (IndexSettingsTable)"] --> O["Settings Exporter Lambda"] --> P["Cloudflare KV"]
    Q["EventBridge (10 min)"] --> R["Monitoring Service Lambda"] --> S["checks alarms exist"]

AWS Resources

Resource	Name Pattern	How to Inspect
DynamoDB	{env}-EcomIndexSettingsTable	DynamoDB
DynamoDB	{env}-EcomIndexerJobsTable	DynamoDB
DynamoDB	{env}-EcomIndexQueryConfigsTable	DynamoDB
DynamoDB	{env}-EcomCollectionsTable	DynamoDB
DynamoDB	{env}-AgenticCachedQueriesTable	DynamoDB
Lambda	{env}-EcomIndexerFunction	Lambda
Lambda	{env}-EcomSettingsExporterLambda	Lambda
Lambda	{env}-EcomMetricsWorker	Lambda
Lambda	{env}-EcomMonitoringServiceLambda	Lambda
Lambda	{env}-ShopifyAppAdminFunction	Lambda
Lambda	{env}-ShopifyWebhookWorker	Lambda
SQS	{env}-EcomMetricsQueue	SQS
SQS	{env}-EcomMetricsQueueDLQ	SQS
S3	{env}-ecom-product-data-bucket	S3
S3	{env}-shopify-app-assets	S3
API Gateway	{env}-EcomApi (HTTP v2)	API Gateway

DynamoDB Table Schemas

EcomIndexSettingsTable

• pk (S): {system_account_id}
• sk (S): INDEX#{index_name}
• GSI_IndexRootsByName: pk=pk, sk=index_name
• Stream: NEW_IMAGE (triggers settings exporter)
• Stores Marqo index configs: create_index settings, search settings, collection configs, aliases.

EcomIndexerJobsTable

• pk (S): PLATFORM#{platform}#SHOP#{shop_id}
• sk (S): JOB#{created_at}#{job_id}
• GSI_JobLookup: pk=job_id, sk=created_at
• GSI_JobsByStatus: pk=shop_id, sk=status_created_at
• TTL: ttl (30-day auto-cleanup)

EcomIndexQueryConfigsTable

• pk (S): {system_account_id}
• sk (S): INDEX#{index_name}#QUERY#{normalized_query}

AgenticCachedQueriesTable

• pk (S): {account_id}#{index_name}#{normalized_query}
• GSI (accountId-indexName-index): pk=gsi_pk ({account_id}#{index_name})

Data Flow: Settings Sync

graph TD
    A["Admin writes settings"]

    subgraph table["EcomIndexSettingsTable"]
        B["DDB Stream (NEW_IMAGE)"]
        C["Settings Exporter Lambda"]
        D["Cloudflare KV (read by search_proxy worker)"]
    end

    A --> B
    B --> C
    C --> D

If KV data is stale, check:

1. DDB stream status on the table
2. Settings exporter Lambda logs
3. Cloudflare KV namespace content

Typical Investigation Paths

Search returning wrong results:

1. Check index settings in DDB: query EcomIndexSettingsTable for the account
2. Check KV cache: npx wrangler kv key get --namespace-id {id} "{account_id}"
3. Check search proxy logs: npx wrangler tail {env}-ecom-api

Indexing not working:

1. Check indexer jobs: query EcomIndexerJobsTable GSI_JobsByStatus for the shop
2. Check indexer Lambda logs: aws logs tail /aws/lambda/{env}-EcomIndexerFunction
3. Check SQS queue for the shop (dynamically created)

Metrics missing:

1. Check metrics queue depth: SQS
2. Check DLQ for failed messages
3. Check metrics worker Lambda logs

Accounts

Env	Account	Ecom API Domain
Staging	468036072962	staging-ecom.dev-marqo.org
Preprod	010928202142	ecom.preprod-marqo.org
Prod	023568249301	ecom.marqo-ep.ai