Running on AWS
Updated Nov 18, 2026 · today
This page says what to install for each AWS compute service, and what we measured when we ran it in a real AWS account. Three kinds of path exist:
- An agent you run (the OpenTelemetry Collector). It sends OTLP, which the hosted service accepts today.
- No agent: CloudWatch Logs and CloudWatch metrics reach Epok through Amazon Data Firehose. Status: not yet available on Epok's hosted service. Use OpenTelemetry (OTLP) for now.
- Account changes from CloudTrail, read through a read-only IAM role: AWS CloudTrail change events. Status: not yet available on Epok's hosted service. Use OpenTelemetry (OTLP) for now.
What is supported
| Service | Logs | Traces | Metrics | Change events | How it arrives |
|---|---|---|---|---|---|
| EC2 | Yes | Yes | Yes | Not listed | OpenTelemetry Collector on the instance |
| CloudWatch Logs | Yes (not yet on the hosted service) | Not listed | Not listed | Not listed | Subscription filter and Amazon Data Firehose |
| CloudWatch metrics | Not listed | Not listed | Yes (not yet on the hosted service) | Not listed | CloudWatch Metric Streams and Firehose |
| Lambda | Yes (not yet on the hosted service) | Yes | Not listed | Not listed | CloudWatch Logs and Firehose; OpenTelemetry Lambda layers |
| ECS on Fargate | Not listed | Yes | Not listed | Not listed | OpenTelemetry Collector sidecar |
| CloudTrail | Not listed | Not listed | Not listed | Yes (not yet on the hosted service) | Read-only IAM role (CloudTrail lookup) |
On this page: Choose a pathEC2CloudWatch Logs (no agent)ECSLambdaEKSKnown limits
Choose a path
| Where it runs | Logs | Metrics | Traces |
|---|---|---|---|
| EC2 | OTel Collector on the host (tested), or CloudWatch Logs + Firehose (tested, not yet hosted) | OTel Collector host metrics (tested) | OTLP to the Collector on the host (tested) |
| ECS on Fargate | awslogs driver to CloudWatch Logs, then Firehose (not yet hosted; not yet tested from an ECS task) | CloudWatch metric stream (tested 2026-11-09, not yet hosted) | OTel Collector sidecar (setup manifest) |
| Lambda | CloudWatch Logs + Firehose (tested with a real function, not yet hosted) | CloudWatch metric stream (tested 2026-11-09, not yet hosted) | OpenTelemetry Lambda layers (setup manifest) |
| EKS | Epok Helm chart (tested on kind, not on EKS) | Epok Helm chart (tested on kind, not on EKS) | OTLP to the chart's collector |
EC2: the OpenTelemetry Collector on the host
One Collector per instance reads your log files, collects host metrics (CPU, memory, disk, filesystem, network, load) and forwards OTLP from apps on the same host. The resourcedetection processor reads the instance ID, type, region, zone and account from the instance metadata service, so the instance needs no IAM role for it. Today they reach Epok on host metrics only, not on logs or spans (see Known limits).
Install on Amazon Linux 2023 (arm64 shown; use amd64 on x86). The Collector project publishes no 0.123.0 Linux packages; use 0.123.1.
V=0.123.1; ARCH=arm64 # amd64 on x86 instances
curl -fsSLo otelcol-contrib.rpm \
"https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v${V}/otelcol-contrib_${V}_linux_${ARCH}.rpm"
sudo rpm -i otelcol-contrib.rpmPut the key in the service's environment file, readable by root only. Do not put it in the configuration file.
read -rsp "Epok ingest key: " EPOK_INGEST_KEY; echo
echo "EPOK_INGEST_KEY=$EPOK_INGEST_KEY" | sudo tee -a /etc/otelcol-contrib/otelcol-contrib.conf >/dev/null
sudo chmod 600 /etc/otelcol-contrib/otelcol-contrib.conf
unset EPOK_INGEST_KEYThen write /etc/otelcol-contrib/config.yaml and run sudo systemctl restart otelcol-contrib:
# /etc/otelcol-contrib/config.yaml for a VM (EC2 or Compute Engine) sending to Epok:
# app log files + host metrics + OTLP from local apps.
# The key comes from the environment (EPOK_INGEST_KEY), never this file.
receivers:
otlp:
protocols:
grpc:
endpoint: 127.0.0.1:4317
http:
endpoint: 127.0.0.1:4318
filelog:
include: [/var/log/myapp/*.log]
start_at: end
# Plain-text lines carry no OTel severity; without this every line is
# stored as level=info. Take the first level word in the line.
operators:
- type: regex_parser
regex: '\b(?P<sev>TRACE|DEBUG|INFO|WARN|WARNING|ERROR|FATAL)\b'
on_error: send_quiet
severity:
parse_from: attributes.sev
- type: remove
field: attributes.sev
if: 'attributes.sev != nil'
hostmetrics:
collection_interval: 60s
scrapers:
cpu: {}
memory: {}
load: {}
disk: {}
filesystem: {}
network: {}
processors:
resourcedetection:
detectors: [env, ec2, system]
timeout: 5s
override: false
resource/service:
attributes:
- key: service.name
value: myapp
action: insert
batch: {}
exporters:
otlphttp/epok:
endpoint: https://ingest.epok.cloud
compression: gzip
headers:
x-api-key: "${env:EPOK_INGEST_KEY}"
service:
pipelines:
logs:
receivers: [otlp, filelog]
processors: [resourcedetection, resource/service, batch]
exporters: [otlphttp/epok]
metrics:
receivers: [otlp, hostmetrics]
processors: [resourcedetection, batch]
exporters: [otlphttp/epok]
traces:
receivers: [otlp]
processors: [resourcedetection, batch]
exporters: [otlphttp/epok]
Measured on a t4g.small, Amazon Linux 2023
- 60 of 60 lines written to the log file were stored once, with
service=myapp. - One OTLP log and one OTLP span sent by an app to the local Collector were stored.
- 22 host metric names arrived, each labeled with
host_id(the instance ID),cloud_region,cloud_availability_zoneandcloud_account_id. - Without the
regex_parserstep, every plain-text line was stored aslevel=info, including lines that say ERROR. With it, the level comes from the line.
CloudWatch Logs without an agent (Firehose)
Status: not yet available on Epok's hosted service. Use OpenTelemetry (OTLP) for now.
Anything that already writes to CloudWatch Logs (Lambda, ECS with the awslogs driver, the CloudWatch agent on EC2, API Gateway, RDS) can reach Epok through one CloudFormation stack: a subscription filter per log group, a Firehose stream, and an S3 bucket for what Firehose cannot deliver. Setup, checks, troubleshooting and removal are on AWS: logs and metrics.
Measured against a real Firehose stream
- Stack creation: 160 s. First canary event stored 17 s after it was written.
- Latency, with the buffer interval at 0: 60 of 60 events within 60 s; median 5.2 s, 95th percentile 7.5 s, slowest 8.4 s. At the default 60 s interval, add up to a minute.
- 1,000 of 1,000 events stored, once each.
- When Epok answered 429 or 503, Firehose re-sent the same request about 2 s later and every event was stored once. When Epok stored a request but the answer was lost (a 500 after the write), Firehose re-sent it and Epok stored nothing twice and billed it once.
- Billing: the bytes Epok counted equalled the decoded bytes Firehose sent and the copy in the stack's S3 bucket, to the byte.
- A wrong key is refused with a 403 whose message Firehose writes to the stack's delivery-error log group. Another account's key never sees the data.
Each event gets service from the last part of its log group name, log_group, log_stream, cloud.provider, cloud.account_id and cloud.region. The level is the first of FATAL or CRITICAL, ERROR, WARN, DEBUG and TRACE found as a word in the message, else info. So a web access-log line with status 502 is stored as info.
ECS on Fargate
Logs: keep the awslogslog driver and add the task's log group (for example /ecs/api, stored as service=api) to the Firehose stack above. We tested this stack with Lambda and EC2 log groups, not yet with an ECS task.
Traces: add an OpenTelemetry Collector sidecar to the task definition. The steps, as a machine-readable manifest an agent or a human can run: aws-ecs-otel-traces.json (skill: SKILL.md). The key is stored as an SSM SecureString and injected into the sidecar only.
Lambda
Logs: add /aws/lambda/<function> to the Firehose stack. Measured with a real Python function: every line it printed or logged was stored once with service=<function name>; a line logged with logging.error was stored as level=error.
Traces: the OpenTelemetry Lambda layers (Python instrumentation plus the Collector extension). Manifest: aws-lambda-otel-traces.json (skill: SKILL.md). The key is stored in Secrets Manager and read by the Collector extension at start.
EKS
Use the Epok Helm chart: Kubernetes. We have run it on kind, not yet on EKS. Logs that EKS writes to CloudWatch (the control plane, or Container Insights) can also go through the Firehose stack, but they arrive with service set to the last part of the log group name (for example application), not the pod's app.
Known limits (measured)
- Firehose stack:
serviceis the last part of the log group name. For the CloudWatch agent's default group names (the file path, such as/var/log/nginx/access) that isaccess; setlog_group_namein the agent to/ec2/<app>instead. - Firehose stack: to add a log group to an existing stack, run
aws cloudformation update-stackwith the template file again (--template-body file://epok-integration.yaml, fromhttps://epok.cloud/setup/aws/epok-integration.yaml), the fullLogGroupNameslist, andUsePreviousValue=truefor every other parameter. With--use-previous-templatenothing changes ("No updates are to be performed"): AWS stores the template with itsFn::ForEachalready expanded. Re-runningaws-logs.shdoes not add groups either; it leaves an existing stack as it is. - Collector: the instance, region and account fields are on host metrics only (as the labels
host_id,cloud_region,cloud_account_id). OTLP log records and spans keepservice,hostand the Kubernetes fields, but not these, until the ingest change that adds them to logs and spans is switched on. Then they are stored ascloud.region,cloud.account_idandhost_id, the samecloud.*names the Firehose rows carry.