Skip to content

JULIO-18: tag AppSec invocation spans with the AWS access key ID - #887

Draft
Julio-Guerra wants to merge 3 commits into
DataDog:mainfrom
Julio-Guerra:agent/software-engineer/3624e28377f2
Draft

Julio-Guerra wants to merge 3 commits into
DataDog:mainfrom
Julio-Guerra:agent/software-engineer/3624e28377f2

Conversation

@Julio-Guerra

Copy link
Copy Markdown
Contributor

What does this PR do?

When DD_APPSEC_AWS_ACCESS_KEY_ID_TAG_ENABLED=true (default false) and the wrapper's AppSec path runs (DD_APPSEC_ENABLED=true), asm_start_request tags the aws.lambda span of every AppSec-monitored HTTP invocation with aws.lambda.access_key_id. The value is the execution environment's AWS_ACCESS_KEY_ID.

  • datadog_lambda/config.py: new appsec_aws_access_key_id_tag_enabled, parsed with as_bool, default false. Environment variable only, never from Remote Configuration.
  • datadog_lambda/asm.py: the value is read once per process, at the first monitored invocation rather than at import, and cached even when it is missing or invalid. It is used only on a full match of (ASIA|AKIA)[A-Z0-9]{16}, so a value with a trailing newline is rejected. It is never logged. SnapStart functions (AWS_LAMBDA_INITIALIZATION_TYPE=snap-start) are not read and not tagged, because their environment predates the snapshot. The tag is set before the aws_lambda.start_request dispatch, so it is in place before the WAF runs.
  • The secret access key and the session token are never read.
  • No change to the WAF, the rules, blocking, Remote Configuration or wrapper.py.

Motivation

Cross-product correlation between AAP signals and CloudTrail activity needs a join key on the invocation span. The execution environment's access key ID stays the same for the environment's life, and CloudTrail records it as userIdentity.accessKeyId. The feature is off by default because the tag is a credential identifier and has not passed the privacy review yet.

When both AppSec paths could run: with the extension v102 wrapper or later, DD_SERVERLESS_APPSEC_ENABLED=true makes the wrapper export DD_APPSEC_ENABLED=false, so this code path does not run and the extension writes the tag. If a custom exec wrapper keeps both paths on, this wrapper still tags, and the extension does not overwrite a value it finds. That extension behavior lands in its own PR.

Testing Guidelines

  • pytest tests: 1043 passed locally (macOS arm64, Python 3.12.8, ddtrace 4.17.0). CI runs Python 3.8 to 3.14.
  • New tests in tests/test_asm.py, tests/test_wrapper.py and tests/test_config.py cover: every monitored HTTP event type (ALB single and multi-value headers, Function URL, API Gateway REST, HTTP and WebSocket); default off and the kill switch; canary secret and token absent from span meta, metrics and the AppSec event; empty, malformed and trailing-newline values (no tag, no log above debug); read once; SnapStart; other initialization types; no tag on SQS, SNS or EventBridge; the tag present at WAF dispatch; the real WAF on crs-930-120 in a fresh interpreter; AppSec off; extension-only AppSec; and both AppSec paths on.
  • Micro benchmark tests/test_benchmarks.py::test_asm_start_request_access_key_id, 20,000 rounds with GC off: median 11.71 µs with the feature off and 12.63 µs with it on, so about +0.9 µs per monitored invocation.
  • black --check datadog_lambda/ tests (25.11.0) is clean. flake8 on the changed source files is clean. On Python 3.12, flake8 5 still reports pre-existing E231 false positives in f-strings in files this PR does not touch.

Additional Notes

  • Draft. Not for merge until the owners review it.
  • The commits are unsigned: the signing key was not available to this automated run. They must be re-signed before merge.
  • The tag name aws.lambda.access_key_id still has to be confirmed by the APM span-tag owners and the privacy review. It is defined in one place (asm.AWS_ACCESS_KEY_ID_TAG).
  • Supersedes the design prototype JULIO-13: prototype for the datadog-lambda-python LLD (do not merge) #886, which remains a reference only.

Types of Changes

  • Bug fix
  • New feature
  • Breaking change
  • Misc (docs, refactoring, dependency upgrade, etc.)

Check all that apply

  • This PR's description is comprehensive
  • This PR contains breaking changes that are documented in the description
  • This PR introduces new APIs or parameters that are documented and unlikely to change in the foreseeable future
  • This PR impacts documentation, and it has been updated (or a ticket has been logged)
  • This PR's changes are covered by the automated tests
  • This PR collects user input/sensitive content into Datadog
  • This PR passes the integration tests (ask a Datadog member to run the tests)

🤖 Generated with Claude Code

Julio-Guerra and others added 3 commits November 13, 2026 00:24
When DD_APPSEC_AWS_ACCESS_KEY_ID_TAG_ENABLED is true (default false) and
the wrapper's AppSec path runs (DD_APPSEC_ENABLED=true), asm_start_request
sets aws.lambda.access_key_id on the aws.lambda span of every monitored
HTTP invocation, before the WAF runs.

The value is AWS_ACCESS_KEY_ID only, read once per execution environment
at the first monitored invocation, used only on a full match of
(ASIA|AKIA)[A-Z0-9]{16}, and never logged. SnapStart functions get no tag.
The secret access key and the session token are never read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: multica-agent <github@multica.ai>
Unit and wrapper tests for the LLD test specification (RFC "CADR for AWS
Lambda" S1 to S9c, D1 to D7): every monitored HTTP event type, default off
and kill switch, canary secret and token, validation and trailing newline,
read once, SnapStart, non-HTTP triggers, tag set before the WAF, the real
WAF on crs-930-120, extension-only AppSec, and both AppSec paths on.
Adds a micro benchmark of asm_start_request with the feature off and on.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: multica-agent <github@multica.ai>
Use patch.dict so the wrapper tests put back any variable they set, and
show the child stderr when the real-WAF test fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: multica-agent <github@multica.ai>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant