Skip to content

Controller

  • Infra code: infra/controller/
  • Component code: components/auth_lambda/ (external), components/merchandising_exporter/

The original control plane (Python/Django on Elastic Beanstalk). Manages user accounts, Cognito, billing, and the cloud controller dashboard.

Architecture

graph TD
    A["Browser"] --> B["ALB (EBS-managed)"]

    subgraph ebs["Elastic Beanstalk (Python/Django)"]
        C["DDB (UsersAccountsTable)"]
        D["Cognito (user management)"]
        E["Secrets Manager (config)"]
        F["S3 (feature flags, app versions)"]
    end

    B --> C
    B --> D
    B --> E
    B --> F

    G["API"]

    subgraph apigw["API Gateway (REST)"]
        H["Auth Lambda (authorizer)"]
        I["Marqtune Lambda"]
    end

    J["Controller EBS"]

    G --> H
    G --> I
    H --> J

    K["EventBridge (5 min)"] --> L["Merchandising Exporter Lambda"] --> M["Cloudflare KV"]

AWS Resources

Resource Name Pattern How to Inspect
Elastic Beanstalk {env}-cloud-controller-{env}-AL2023 EBS
DynamoDB UsersAccountsTable (or UsersAccountsTable-{user} in dev) DynamoDB
DynamoDB {env}-MerchandisingTable DynamoDB
DynamoDB {env}-SearchRedirectTable DynamoDB
Cognito MarqoUserPool (prod), MarqoUserPool-staging, dev-{branch}-MarqoUserPool (dev) Cognito
Lambda ControllerAuthLambda-{env} Lambda
Lambda MerchandisingExporterLambda-{env} Lambda
API Gateway ControllerApi-{env} (REST) API Gateway
WAF ControllerWebACL WAF
S3 {env}-cloud-controller S3
VPC ControllerVpc-{env} Network stack
CloudWatch Dashboard CloudControllerDashboard-{env} CloudWatch

DynamoDB Tables

UsersAccountsTable

  • pk (S), sk (S)
  • User accounts, organizations, roles. Shared with console monolith.
  • Cross-account read access granted to Marqtune account.

MerchandisingTable

  • pk (S), sk (S)
  • Merchandising rules and search configuration data.
  • Exported to Cloudflare KV every 5 minutes.

SearchRedirectTable

  • pk (S), sk (S)
  • Search redirect mappings (query -> URL).
  • Read by search_proxy Cloudflare worker via DDB (IAM user credentials).

Merchandising Exporter

Runs every 5 minutes via EventBridge. Reads from MerchandisingTable, writes to Cloudflare KV.

Key env vars:

  • CF_ACCOUNT_ID, CF_KV_NAMESPACE_ID, CF_API_TOKEN_SECRET_NAME
  • MERCHANDISING_TABLE, INDEX_UPDATE_RECENCY_MINUTES

Cloudflare KV namespace IDs per env:

Env KV Namespace ID
Dev baee10aa63cf4a60bfad1f0a921090f8
Staging 8553b710ce884390a9058351ab7c3ca5
Preprod 91d28d0451364cb9b09e3eea128e1ad9
Prod 900f2bd661f649748701fff155764a3b

Cross-Account Integrations

Target Account Purpose
Data plane cells (API Gateway) Various per cell Index/cluster operations
Marqtune 975050354766 (staging) / 905418443936 (prod) ML workflow proxy
Marqtune (cross-account Lambda invoke) Same Auth Lambda shared as authorizer

Typical Investigation Paths

Controller not responding:

  1. Check EBS health: aws elasticbeanstalk describe-environment-health
  2. Check recent events: aws elasticbeanstalk describe-events
  3. Check ALB target health
  4. Check CloudWatch logs

Merchandising not syncing to search:

  1. Check MerchandisingExporter Lambda logs
  2. Check CloudWatch alarm: MerchandisingExporterHeartbeatAlarm-{env}
  3. Check Cloudflare KV namespace content
  4. Verify CF API token secret exists: aws secretsmanager describe-secret --secret-id {env}/CloudFlareApiToken

Auth failures:

  1. Check ControllerAuthLambda-{env} logs
  2. Verify Cognito user pool config
  3. Check Secrets Manager for {env}/CloudController secret