Shift-right on AWS: canaries by revision and tenant

This is the downstream part of an argument I introduced in The cost does not disappear, it moves: development in the age of agents: with agents, development cost moves upstream to specification and downstream to verification. Here I focus downstream.

The argument in one sentence: Definition of Done (DoD) criteria become the alarms governing release. A canary, a gradual release to a small share of traffic or users, catches technical problems by revision; a tenant, an isolated customer or organisation sharing an application, provides the unit for functional rollout. My proposed combination is an ECS canary for the new revision plus tenant rollout through feature flags, switches that enable production functionality without redeploying.

Native canaries on ECS Fargate

Since 2025, ECS natively manages blue/green, linear and canary deployment without CodeDeploy, with lifecycle hooks, bake time and rollback in service configuration.

1. An Application Load Balancer (ALB) with two weighted target groups, sets of ECS tasks receiving traffic, on the production listener rule that determines request routing.
2. A CANARY strategy: for example, 10% of traffic to the new revision, a bake time observation period, then 100%.
3. A lifecycle hook, usually a serverless Lambda function, runs end-to-end tests or synthetic monitoring, simulated requests behaving like users, against test traffic before moving real traffic.
4. CloudWatch alarms for the new revision’s error rate and p95 latency, the threshold below which 95% of request durations fall, linked to automatic rollback.

The specification connection: if the DoD says “p95 latency below 300 ms” or “no increase in 5xx errors”, HTTP server error responses such as 500 or 502, those criteria become canary alarms.

The limitation: an ECS canary routes a random percentage of requests. It does not choose who sees the change.

Tenant rollout with feature flags

In a multitenant system, the canary moves from deployment to release: code reaches production switched off and is enabled for groups of tenants.

• AppConfig multivariant flags evaluate application-supplied tenant_id context against rules you define.
• Gradual rollout by entity, using AppConfig Agent: a tenant receiving a version keeps it throughout deployment, regardless of which task serves the request.
• Automatic rollback if CloudWatch alarms fire during configuration deployment.

A pipeline can automate this sequence:

1. Internal or test tenant.
2. Low-risk cohort with enough traffic.
3. 25% of tenants.
4. Everyone.

Each step has bake time and alarm-triggered rollback.

Alternatives include the commercial LaunchDarkly platform, open-source and self-hostable Unleash, and open-source Flagsmith. Where EU data residency is required, self-hosted Unleash or AppConfig in an EU region are the simplest options.

Prerequisites and limitations

Automatic rollback works only if metrics can distinguish canary participants. That is the real prerequisite, rather than the flag.

Topic	Problem	Action
Per-tenant metrics	Alarms cannot distinguish the canary cohort	Label metrics and logs with tenant identity
Traffic volume	With little traffic, 10% or a small tenant gives no signal within minutes	Size bake time and cohorts for actual volumes
Database migrations	Two versions share one schema; a flag does not isolate the database	Require expand/contract migrations in the spec: add backwards-compatible schema first, remove old schema after rollout
Shared resources	Queues and batch jobs affect all tenants	Assess them separately in the release plan
Flag debt	Every flag adds a code branch	DoD rule: remove the flag within N days of reaching 100%
Functional behaviour	Shift-right catches technical issues better than functional problems with contractual or compliance consequences	Validate functionality beforehand, rather than on users

Shift-right, moving some verification into production through flags, canaries and observability, does not replace shift-left, testing as early as possible. Used as an excuse to test less beforehand, it becomes testing on users.

---

References

• re:Invent 2025 deep dive: ECS Managed Instances and native deployments.
• AWS samples: ECS deployment strategies.
• AWS samples: ECS service and deployment.
• AWS AppConfig: creating feature flags.
• AWS AppConfig: deploying feature flags.