Skip to main content

Polo v2

Infrastructure observability and cost analytics platform for Marqo Cloud.

Commands​

CLI lives in components/cli/commands.py. Run via Pants:

PANTS_CONCURRENT=True pants run components/cli:cli -- <command>
  • polo up / polo down — start/stop ClickHouse via Docker Compose
  • polo migrate — apply schema migrations to local ClickHouse
  • polo migrate-remote <instance-id> — run migrations against remote CH via SSM
  • polo seed — seed test data
  • polo sync — snapshot prod data into local ClickHouse
  • polo health — run repo health checks
  • polo test schema — schema tests (requires CH running)
  • polo test unit — unit tests only
  • polo test integration — integration tests (requires CH running)
  • polo test perf — performance benchmarks
  • polo test e2e — e2e tests (requires AWS credentials + CH)
  • polo test all — all of the above in order
  • polo build ui — build React SPA
  • polo deploy worker [--env staging] — deploy Cloudflare Worker
  • polo ralph — autonomous agent loop: pick up beads issues, implement via worker agent, review via reviewer agent, merge on approval. Flags: -n/--max-issues (default 10), --budget (USD per agent, default 1.0), --dry-run (preview only), --model (e.g. sonnet, opus), --permission-mode (default auto), -s/-t/-l/-p (filter by status/type/label/priority)
  • pants lint :: — lint all Pants-managed code
  • pants test :: -//tests/:: — run unit tests via Pants

Conventions​

  • Python 3.11+, type hints everywhere
  • All collectors return list[ResourceEvent] (Pydantic model)
  • ClickHouse inserts are always batched (DEFAULT_BATCH_SIZE = 1000)
  • Every module has a corresponding test file
  • AWS calls are wrapped with retry + exponential backoff
  • All test resources in AWS get tag polo:test=true
  • Use AWS CLI profile "polo" to access the AWS account

Architecture​

  • ClickHouse for storage (resource_events, resource_snapshots, cost tables, hierarchy)
  • Python collectors run as Lambda functions, collect from AWS APIs
  • Cloudflare Worker serves the API (queries ClickHouse directly)
  • React SPA on Cloudflare (Vite + TanStack Query + Tailwind CSS)
  • TanStack Router provides URL-based routing (code-based, not file-based)
  • See docs/ for canonical documentation
  • To inspect deployed resources, logs, and metrics: see docs/operations/inspecting-aws.md

Browsing Staging with Playwright​

The staging app at polo.dev-marqo.org is behind Cloudflare Access. To browse it with Playwright:

  1. Get a CF Access token: cloudflared access login https://polo.dev-marqo.org
  2. Retrieve the cached token: cloudflared access token -app=https://polo.dev-marqo.org
  3. Set the CF_Authorization cookie on the browser context before navigating:
const context = page.context();
await context.addCookies([{
  name: 'CF_Authorization',
  value: '<token from step 2>',
  domain: '.polo.dev-marqo.org',
  path: '/',
  secure: true,
  httpOnly: true,
  sameSite: 'None'
}]);
await page.goto('https://polo.dev-marqo.org/');

Non-Interactive Shell Commands​

ALWAYS use non-interactive flags with file operations to avoid hanging on confirmation prompts.

Shell commands like cp, mv, and rm may be aliased to include -i (interactive) mode on some systems, causing the agent to hang indefinitely waiting for y/n input.

# Force overwrite without prompting
cp -f source dest           # NOT: cp source dest
mv -f source dest           # NOT: mv source dest
rm -f file                  # NOT: rm file

# For recursive operations
rm -rf directory            # NOT: rm -r directory
cp -rf source dest          # NOT: cp -r source dest

Other commands that may prompt:

  • scp — use -o BatchMode=yes for non-interactive
  • ssh — use -o BatchMode=yes to fail instead of prompting
  • apt-get — use -y flag
  • brew — use HOMEBREW_NO_AUTO_UPDATE=1 env var

Beads Issue Tracker​

This project uses bd (beads) for issue tracking. Run bd prime for detailed workflow context and commands.

bd ready              # Find available work
bd show <id>          # View issue details
bd update <id> --claim  # Claim work atomically
bd close <id>         # Complete work
bd dolt push          # Push beads data to remote

Rules​

  • Use bd for ALL task tracking — do NOT use TodoWrite, TaskCreate, or markdown TODO lists
  • Run bd prime for detailed command reference and session close protocol
  • Use bd remember for persistent knowledge — do NOT use MEMORY.md files

Session Completion​

When ending a work session, you MUST complete ALL steps below. Work is NOT complete until git push succeeds.

MANDATORY WORKFLOW:

  1. File issues for remaining work — Create issues for anything that needs follow-up
  2. Run quality gates (if code changed) — Tests, linters, builds
  3. Update issue status — Close finished work, update in-progress items
  4. PUSH TO REMOTE — This is MANDATORY: 
    git pull --rebase
    bd dolt push
    git push
    git status  # MUST show "up to date with origin"
  5. Clean up — Clear stashes, prune remote branches
  6. Verify — All changes committed AND pushed
  7. Hand off — Provide context for next session

CRITICAL RULES:

  • Work is NOT complete until git push succeeds
  • NEVER stop before pushing — that leaves work stranded locally
  • NEVER say "ready to push when you are" — YOU must push
  • If push fails, resolve and retry until it succeeds

Code Quality​

Before pushing any code, you MUST ensure it is formatted and lint-free:

PANTS_CONCURRENT=True pants fmt :: check ::  # Auto-fix formatting

Beads Issue Tracker​

This project uses bd (beads) for issue tracking. Run bd prime to see full workflow context and commands.

Quick Reference​

bd ready              # Find available work
bd show <id>          # View issue details
bd update <id> --claim  # Claim work
bd close <id>         # Complete work

Rules​

  • Use bd for ALL task tracking — do NOT use TodoWrite, TaskCreate, or markdown TODO lists
  • Run bd prime for detailed command reference and session close protocol
  • Use bd remember for persistent knowledge — do NOT use MEMORY.md files

Session Completion​

When ending a work session, you MUST complete ALL steps below. Work is NOT complete until git push succeeds.

MANDATORY WORKFLOW:

  1. File issues for remaining work - Create issues for anything that needs follow-up
  2. Run quality gates (if code changed) - Tests, linters, builds
  3. Update issue status - Close finished work, update in-progress items
  4. PUSH TO REMOTE - This is MANDATORY: 
    git pull --rebase
    bd dolt push
    git push
    git status  # MUST show "up to date with origin"
  5. Clean up - Clear stashes, prune remote branches
  6. Verify - All changes committed AND pushed
  7. Hand off - Provide context for next session

CRITICAL RULES:

  • Work is NOT complete until git push succeeds
  • NEVER stop before pushing - that leaves work stranded locally
  • NEVER say "ready to push when you are" - YOU must push
  • If push fails, resolve and retry until it succeeds