Multi-Platform Cost Ingestion

Status: PLANNED — This feature is not yet implemented. The cost_github, cost_cloudflare, and cost_datadog collectors do not exist yet. Currently Polo only collects from AWS. This document is the design specification for future development.

Polo tracks costs across AWS, GitHub, Cloudflare, and Datadog using the same resource_events schema. The resource_arn field serves as a universal resource identifier across platforms.

Resource ID Conventions

Platform	Format	Example
AWS	Standard ARN	arn:aws:ec2:us-east-1:123:instance/i-abc
GitHub	github:{org}/{type}/{name}	github:marqo-ai/repo/marqo, github:marqo-ai/actions/workflow/ci.yml
Cloudflare	cloudflare:{product}/{id}	cloudflare:worker/polo-api, cloudflare:zone/marqo.ai, cloudflare:r2/model-storage
Datadog	datadog:{product}	datadog:infrastructure, datadog:logs, datadog:apm

Platform Collectors

Collector	Source API	Granularity	Auth
cost_github	GitHub Billing API + Workflow Runs API	Monthly (billing), per-run (actions)	PAT or GitHub App with admin:org
cost_cloudflare	Cloudflare Analytics + Billing API	Daily	API token with Billing:Read
cost_datadog	Datadog Usage API (/api/v1/usage/*)	Hourly	API key + App key

For GitHub Actions, the interesting breakdown is per-workflow and per-repo, since CI sprawl is a real cost driver that nobody monitors.

Normalisation

All costs normalised to daily USD in ResourceEvent with appropriate collector value. aws_account_id and aws_region are empty for non-AWS resources.

Hierarchy Integration

Non-AWS costs roll up through a platform hierarchy:

platform:aws         → all AWS costs
platform:github      → all GitHub costs
platform:cloudflare  → all Cloudflare costs
platform:datadog     → all Datadog costs

resource_ancestry gets rows linking non-AWS resources to their platform node.

Secrets

All platform API keys stored in AWS Secrets Manager:

• polo/github-pat — GitHub PAT
• polo/cloudflare-token — Cloudflare API token
• polo/datadog-api-key + polo/datadog-app-key — Datadog credentials