Home / Documentation

Documentation

BYOC Setup

Deploy TraceLoom infrastructure in your own AWS account. All test execution happens on your EC2 instances, and all Playwright traces stay in your S3 bucket — zero data egress.

Overview

BYOC (Bring Your Own Cloud) means TraceLoom coordinates test execution from its control plane, but your EC2 instances do the actual work. The architecture:

  • TraceLoom control plane — manages test scheduling, sharding, and result aggregation. Hosted by TraceLoom.
  • Your AWS account — runs EC2 Spot instances, stores traces in S3, queues jobs in SQS. You own and control all of this.
  • Cross-account IAM role — TraceLoom assumes this role with a unique ExternalId to communicate with your infrastructure. Revoke at any time by deleting the CloudFormation stack.

Your Playwright traces, test artifacts, and test data never leave your infrastructure. TraceLoom reads test results and metadata — it does not retain your actual trace files.

Prerequisites

  • AWS account with admin access or CloudFormation permissions
  • TraceLoom account — sign up free

Step 1: Launch CloudFormation Stack

From the TraceLoom onboarding wizard, click Deploy to AWS. This opens the AWS CloudFormation console with the template pre-loaded and parameters pre-filled.

The CloudFormation template creates:

  • IAM cross-account role (TraceLoomExecutionRole) with ExternalId protection
  • S3 bucket for Playwright trace storage with server-side encryption
  • SQS queue for test job distribution
  • EC2 Auto Scaling Group configured for Spot instances

Review the CloudFormation parameters before deploying:

TraceLoomAccountId: 153550591373
ExternalId:        <your-project-id>    # Pre-filled from your TraceLoom project
InstanceType:      c5.xlarge            # Recommended for Playwright workloads
MaxWorkers:        10                   # Maximum concurrent Spot instances
TraceBucketName:   traceloom-<random>  # Auto-generated, globally unique

Click Create stack. Deployment takes approximately 3 minutes. When the stack status shows CREATE_COMPLETE, return to TraceLoom — it will validate the connection automatically.

Step 2: Configure IAM Permissions

The CloudFormation template creates an IAM role that TraceLoom assumes using the cross-account access pattern. No long-lived credentials are stored — TraceLoom uses temporary credentials from STS AssumeRole, scoped to your project only.

The IAM role grants TraceLoom these permissions:

# EC2 permissions (test execution)
ec2:DescribeInstances
ec2:DescribeSpotPriceHistory
autoscaling:DescribeAutoScalingGroups
autoscaling:SetDesiredCapacity

# S3 permissions (trace storage - your bucket only)
s3:GetObject
s3:PutObject
s3:ListBucket

# SQS permissions (job queue - your queue only)
sqs:SendMessage
sqs:ReceiveMessage
sqs:DeleteMessage
sqs:GetQueueAttributes

# CloudWatch permissions (metrics and logs)
cloudwatch:PutMetricData
logs:CreateLogGroup
logs:CreateLogStream
logs:PutLogEvents

The ExternalId is a unique identifier for your project. It prevents the confused deputy attack — even if another AWS account knew your role ARN, they cannot assume the role without the correct ExternalId. TraceLoom generates this automatically and it never changes for your project.

Step 3: Validate Connection

After stack creation, TraceLoom runs automated validation checks. All four checks must pass before you can run tests. 

Checking connection...
✓ STS AssumeRole     — cross-account access confirmed
✓ S3 write access    — trace storage accessible
✓ SQS send/receive   — job queue accessible
✓ EC2 describe       — instance fleet visible

Connection validated. You're ready to run tests.

Troubleshooting

If validation fails, check these common issues:

  • AssumeRole failed — verify the ExternalId matches between TraceLoom and your CloudFormation parameters. The ExternalId in the IAM trust policy must exactly match the value from your TraceLoom project settings.
  • S3 access denied — check the S3 bucket policy. The IAM role must have s3:GetObject, s3:PutObject, and s3:ListBucket permissions on the trace bucket.
  • EC2 quota exceeded — your account may need a service quota increase for Spot instances. Request an increase in the EC2 console under Service Quotas for Running On-Demand Standard instances or the relevant Spot instance category.
  • Stack creation failed — review the CloudFormation Events tab for the specific error. Common causes: IAM permission boundaries in your account, VPC limits, or S3 bucket name conflicts.

Infrastructure Components

After deployment, these AWS resources exist in your account and are fully visible in your AWS Console:

  • S3 Bucket — stores Playwright trace files (.trace.zip) organized by run ID and test name. Retention policy is configurable. Default: no expiration.
  • SQS Queue — receives test job messages from TraceLoom. Each message contains the test shard assignment, repository reference, and run configuration.
  • EC2 Auto Scaling Group — launches Spot instances on demand when jobs arrive. Instances run the bootstrap script, execute the assigned test shard, upload the trace to S3, and then terminate if idle.
  • IAM Role — cross-account role with ExternalId. Grants TraceLoom control plane access scoped to your S3 bucket, SQS queue, and EC2 fleet only.

Security Model

TraceLoom is designed for teams where data sovereignty is non-negotiable. The security guarantees:

  • No credential storage — TraceLoom never stores AWS access keys or secrets. All access uses temporary STS credentials scoped to your role.
  • ExternalId prevents confused deputy attacks — the IAM trust policy requires a unique ExternalId that only TraceLoom knows for your project. Prevents unauthorized role assumption even if the role ARN is discovered.
  • Encryption in transit and at rest — all API calls use TLS. S3 traces use server-side encryption (SSE-S3 or SSE-KMS if you configure your own KMS key).
  • Principle of least privilege — the IAM role grants only the specific permissions TraceLoom needs. Resource ARNs in the policy are scoped to your specific S3 bucket and SQS queue.
  • Full customer control — revoke all TraceLoom access instantly by deleting the CloudFormation stack. No cleanup required on the TraceLoom side.

Cost Considerations

BYOC means you pay AWS directly for compute. TraceLoom Spot instances use c5.xlarge by default — a good balance of CPU (4 vCPU) and memory (8 GB) for Playwright workloads. 

# Estimated AWS costs per month (us-east-1, approximate)

# Small team: ~200 test runs/month, 50 tests per run, 5 workers
c5.xlarge Spot:  ~$0.05/hr × 5 instances × ~40 hrs/month = ~$10/month
S3 storage:      ~$0.023/GB (traces are small, typically <5 MB each)
SQS requests:    ~$0.40/million (negligible at this scale)
Estimated total: ~$15-25/month in AWS costs

# Medium team: ~1,000 test runs/month, 200 tests per run, 10 workers
c5.xlarge Spot:  ~$0.05/hr × 10 instances × ~100 hrs/month = ~$50/month
S3 storage:      ~$5/month (1,000 runs × 200 tests × ~25 KB avg trace)
Estimated total: ~$60-80/month in AWS costs

Cost optimization strategies:

  • Spot instances save ~70% vs on-demand. The Auto Scaling Group is configured to use Spot by default with on-demand fallback.
  • Auto-scaling scales to zero — instances terminate when idle. No cost when tests are not running.
  • S3 lifecycle rules — configure trace expiration (e.g., 90 days) to control storage costs. Set via the S3 console on your trace bucket.
  • Instance type selection — adjust InstanceType in the CloudFormation parameters. c5.large (2 vCPU, 4 GB) works for smaller test suites and costs half as much.

Back to Getting Started