Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions apps/docs/content/docs/platform/self-hosting/networking.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -278,7 +278,7 @@ This depends on the runtime. The published images run Bun. If you build the stan
| Everything through the SSRF guard — SSO OIDC discovery, the HTTP block, tools, connectors, outbound webhooks, content fetches, MCP servers, and model endpoints typed into a block | Yes — see [Proxying guarded requests](#proxying-guarded-requests) |
| SSO sign-in — the token, JWKS, and userinfo calls made while a user signs in | Yes — they use the default `fetch` |
| SSO domain verification (TXT lookup) | No — DNS, sent to `1.1.1.1` and `8.8.8.8` unless `DOMAIN_VERIFICATION_NAMESERVERS` names your resolvers |
| Amazon Bedrock, object storage on S3, and every AWS integration (DynamoDB, SQS, Lambda, Textract, and the rest) | Yes — each AWS client gets a proxy-aware agent while a proxy is set. Bedrock then uses HTTP/1.1 instead of HTTP/2 |
| Amazon Bedrock, object storage on S3, and every AWS integration (DynamoDB, SQS, Lambda, Textract, and the rest) | Yes — each AWS service client gets a proxy-aware agent while a proxy is set, and STS calls made for its credentials inherit it. Bedrock then uses HTTP/1.1 instead of HTTP/2. IAM Identity Center (SSO) credential lookups do not use the proxy |
| Email via SMTP, and the SMTP block | No — SMTP is not HTTP. Use a mail relay the app can reach directly, or Resend or SES |
| Email via Amazon SES | Yes — the AWS SDK client above |
| OTLP export from the server SDK | Yes — the exporters get the same proxy-aware agent |
Expand All @@ -289,7 +289,7 @@ The practical consequence: a mandatory-egress-proxy environment can route LLM tr

### Proxying guarded requests

A request through the SSRF guard keeps its checks when it is proxied. Before each request, and again for each redirect hop, the app resolves the destination's hostname with its own DNS and refuses it if any policy in [Security](/platform/self-hosting/security#the-ssrf-boundary) blocks the address. It then hands the hostname, not the address, to the proxy, so a proxy that allowlists by domain sees the name it expects.
A request through the SSRF guard keeps its checks when it is proxied. Before each request, and again for each redirect hop, the app resolves the destination's hostname with its own DNS and refuses it if any policy in [Security](/platform/self-hosting/security#the-ssrf-boundary) blocks any of the addresses it resolves to, since the proxy may connect to any of them. It then hands the hostname, not the address, to the proxy, so a proxy that allowlists by domain sees the name it expects.

- **DNS must still resolve inside the container.** A destination that the app cannot resolve is refused before the proxy is asked.
- **The proxy resolves the name again.** The app cannot pin the address across a proxy, so the proxy's own egress policy is the last check.
Expand Down
4 changes: 2 additions & 2 deletions apps/docs/openapi-v2-resources.json
Original file line number Diff line number Diff line change
Expand Up @@ -23929,7 +23929,7 @@
"type": "string",
"minLength": 1,
"maxLength": 4194304,
"description": "UTF-8 CSV contents, up to 4 MiB."
"description": "UTF-8 CSV contents, up to 4 MiB. Both files travel in one request body limited to 10 MiB after JSON encoding."
}
},
"required": ["filename", "content"],
Expand All @@ -23949,7 +23949,7 @@
"type": "string",
"minLength": 1,
"maxLength": 4194304,
"description": "UTF-8 CSV contents, up to 4 MiB."
"description": "UTF-8 CSV contents, up to 4 MiB. Both files travel in one request body limited to 10 MiB after JSON encoding."
}
},
"required": ["filename", "content"],
Expand Down
113 changes: 113 additions & 0 deletions apps/sim/app/api/auth/oauth/token/route.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import {
createMockRequest,
hybridAuthMockFns,
} from '@sim/testing'
import { createRouteContext } from '@sim/testing/helpers/http'
import { NextRequest } from 'next/server'
import { beforeEach, describe, expect, it, vi } from 'vitest'

Expand Down Expand Up @@ -36,11 +37,13 @@ vi.mock('@/lib/credentials/application/resolve-managed-oauth-token', () => ({
resolveManagedOAuthCredentialToken: { execute: mockResolveManagedOAuthCredentialToken },
}))

import { AuthType, checkSessionOrInternalAuth } from '@/lib/auth/hybrid'
import { createQuickBooksAccountId } from '@/lib/oauth/quickbooks'
import { GET, POST } from '@/app/api/auth/oauth/token/route'
import { getToolMetadata } from '@/tools/metadata'

const mockGetToolMetadata = vi.mocked(getToolMetadata)
const mockCheckSessionOrInternalAuth = vi.mocked(checkSessionOrInternalAuth)

describe('OAuth Token API Routes', () => {
beforeEach(() => {
Expand Down Expand Up @@ -290,6 +293,116 @@ describe('OAuth Token API Routes', () => {
})
})

describe('saved service-account credentials', () => {
const authorized = {
ok: true,
requesterUserId: 'test-user-id',
credentialOwnerUserId: 'test-user-id',
workspaceId: 'workspace-id',
}

beforeEach(() => {
authOAuthUtilsMockFns.mockResolveOAuthAccountId.mockResolvedValueOnce({
accountId: '',
credentialId: 'saved-credential-id',
credentialType: 'service_account',
providerId: 'vanta-service-account',
workspaceId: 'workspace-id',
usedCredentialTable: true,
})
authOAuthUtilsMockFns.mockResolveServiceAccountToken.mockResolvedValue({
accessToken: 'saved-secret',
})
mockGetToolMetadata.mockReturnValue({
id: 'vanta_list_tests',
params: {},
oauth: { required: true, provider: 'vanta', credentialKind: 'service-account' },
})
})

it('refuses to release the secret to a session caller', async () => {
mockCheckSessionOrInternalAuth.mockResolvedValueOnce({
success: true,
authType: AuthType.SESSION,
userId: 'test-user-id',
})
mockAuthorizeCredentialUse.mockResolvedValueOnce(authorized)

const response = await POST(
createMockRequest('POST', {
credentialId: 'saved-credential-id',
toolId: 'vanta_list_tests',
}),
createRouteContext({})
)

expect(response.status).toBe(403)
const body = await response.json()
expect(body).toMatchObject({ code: 'SERVICE_ACCOUNT_SESSION_FORBIDDEN' })
expect(JSON.stringify(body)).not.toContain('saved-secret')
expect(authOAuthUtilsMockFns.mockResolveServiceAccountToken).not.toHaveBeenCalled()
})

it('resolves the secret for an internal-JWT execution caller', async () => {
mockCheckSessionOrInternalAuth.mockResolvedValueOnce({
success: true,
authType: AuthType.INTERNAL_JWT,
userId: 'test-user-id',
})
mockAuthorizeCredentialUse.mockResolvedValueOnce(authorized)

const response = await POST(
createMockRequest('POST', {
credentialId: 'saved-credential-id',
toolId: 'vanta_list_tests',
workflowId: 'workflow-id',
}),
createRouteContext({})
)

expect(response.status).toBe(200)
await expect(response.json()).resolves.toMatchObject({
accessToken: 'saved-secret',
credentialType: 'service_account',
})
})
})

it('still returns an OAuth token to a session caller', async () => {
mockCheckSessionOrInternalAuth.mockResolvedValueOnce({
success: true,
authType: AuthType.SESSION,
userId: 'test-user-id',
})
mockAuthorizeCredentialUse.mockResolvedValueOnce({
ok: true,
requesterUserId: 'test-user-id',
credentialOwnerUserId: 'test-user-id',
})
authOAuthUtilsMockFns.mockGetCredential.mockResolvedValueOnce({
id: 'credential-id',
accessToken: 'test-token',
refreshToken: 'refresh-token',
accessTokenExpiresAt: new Date(Date.now() + 3600 * 1000),
providerId: 'google',
})
authOAuthUtilsMockFns.mockRefreshTokenIfNeeded.mockResolvedValueOnce({
accessToken: 'session-oauth-token',
refreshed: false,
})

const response = await POST(
createMockRequest('POST', { credentialId: 'credential-id' }),
createRouteContext({})
)

expect(response.status).toBe(200)
await expect(response.json()).resolves.toMatchObject({
accessToken: 'session-oauth-token',
credentialType: 'oauth',
})
})

describe('credentialAccountUserId + providerId path', () => {
it('should reject internal JWT authentication', async () => {
hybridAuthMockFns.mockCheckSessionOrInternalAuth.mockResolvedValueOnce({
Expand Down
4 changes: 3 additions & 1 deletion apps/sim/lib/api/contracts/knowledge/gitlab-permissions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,9 @@ const gitLabCsvUploadSchema = z
.string()
.min(1)
.max(GITLAB_CSV_MAX_BYTES)
.describe('UTF-8 CSV contents, up to 4 MiB.'),
.describe(
'UTF-8 CSV contents, up to 4 MiB. Both files travel in one request body limited to 10 MiB after JSON encoding.'
),
})
.strict()
export const gitLabPermissionInputSchema = z
Expand Down
30 changes: 24 additions & 6 deletions apps/sim/lib/core/network/environment-proxy.server.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,16 +17,17 @@ import { afterAll, beforeAll, beforeEach, describe, expect, it, vi } from 'vites
vi.mock('@/lib/core/network/config.server', () => networkConfigMock)
vi.mock('@sim/security/dns', async (importOriginal) => {
const actual = await importOriginal<typeof import('@sim/security/dns')>()
const records: Record<string, string> = {
'origin.invalid': '1.1.1.1',
'internal.invalid': '10.0.0.5',
const records: Record<string, [string, ...string[]]> = {
'origin.invalid': ['1.1.1.1', '93.184.216.34'],
'internal.invalid': ['10.0.0.5'],
'split.invalid': ['93.184.216.34', '169.254.169.254'],
}
return {
...actual,
resolveHostAddresses: async (host: string) => {
const address = records[host]
if (!address) throw new Error(`No addresses for ${host}`)
return { addresses: [address], preferred: address }
const addresses = records[host]
if (!addresses) throw new Error(`No addresses for ${host}`)
return { addresses, preferred: addresses[0] }
},
}
})
Expand Down Expand Up @@ -250,6 +251,23 @@ describe('guarded requests under HTTPS_PROXY', () => {
expect(tunnels).toEqual([])
})

it('refuses a name with any blocked record, since the proxy may dial any of them', async () => {
const guarded = createSsrfGuardedFetchWithDispatcher(options)
try {
await expect(guarded.fetch('https://split.invalid/')).rejects.toThrow(/metadata/i)
await expect(secureFetchWithValidation('https://split.invalid/', options)).rejects.toThrow(
/metadata/i
)
} finally {
await guarded.dispatcher.destroy()
}
expect(tunnels).toEqual([])

const response = await secureFetchWithValidation('https://origin.invalid/', options)
expect(await response.text()).toBe('tls reached')
expect(tunnels.map((tunnel) => tunnel.authority)).toEqual(['origin.invalid:443'])
})

it('judges each redirect hop again before proxying it', async () => {
const pinned = createPinnedFetchWithDispatcher('1.1.1.1', options)
try {
Expand Down
9 changes: 5 additions & 4 deletions apps/sim/lib/core/network/transport.server.ts
Original file line number Diff line number Diff line change
Expand Up @@ -42,10 +42,10 @@ export function createOutboundTransport(options: OutboundTransportOptions): Outb
]
/**
* The proxy resolves the destination itself, so pinning cannot hold across it. Every request
* (each redirect hop included) is resolved and judged here first, so a name that points at
* a refused address is stopped before the proxy is asked to reach it. A name that changes
* its answer between this check and the proxy's own lookup is left to the proxy's egress
* policy, which is why the self-hosting docs make the proxy the final egress control.
* (each redirect hop included) is resolved and judged here first, and a name is refused when
* any of its records is refused, since the proxy may connect to any of them. A name that
* changes its answer between this check and the proxy's own lookup is left to the proxy's
* egress policy, which is why the self-hosting docs make the proxy the final egress control.
*/
const selectEnvironmentProxy = async (
target: string,
Expand All @@ -56,6 +56,7 @@ export function createOutboundTransport(options: OutboundTransportOptions): Outb
const validation = await validateEgressUrl(target, 'url', options.profile, {
logDetails: false,
signal,
requireEveryAddress: true,
})
if (!validation.isValid) throw new Error(validation.error, { cause: validation.cause })
if (closed) throw new OutboundRoutingError('GATEWAY_UNAVAILABLE')
Expand Down
6 changes: 5 additions & 1 deletion apps/sim/lib/core/security/egress/validate.ts
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,8 @@ export interface EgressValidationOptions {
logDetails?: boolean
/** Cancels DNS validation before a guarded connection can begin. */
signal?: AbortSignal
/** Refuse when any record is refused; for transports that cannot pin (a forward proxy re-resolves). */
requireEveryAddress?: boolean
}

type EgressDenial = Extract<EgressDecision, { allowed: false }>
Expand Down Expand Up @@ -133,7 +135,7 @@ export async function validateEgressUrl(
// Refused records are filtered rather than failing the whole host: pinning to a
// surviving permitted address is just as safe as refusing outright, and a
// split-horizon resolver that answers with a private record alongside the
// public one would otherwise be unusable.
// public one would otherwise be unusable. A caller that cannot pin opts out.
let refusal: EgressDenial | undefined
const usable = addresses.filter((address) => {
const decision = evaluateAddress(parsed, address, policy)
Expand All @@ -142,6 +144,8 @@ export async function validateEgressUrl(
return false
})

if (refusal && options.requireEveryAddress) return fail(refusal, paramName, profile, options)

if (usable.length === 0) {
return fail(
refusal ?? { allowed: false, reason: 'address-blocked', detail: 'no addresses resolved' },
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ vi.mock('@/lib/core/security/input-validation.server', () => inputValidationMock

import {
normalizeHttpBaseUrl,
refuseUnsafeTypedUrl,
tokenCheckValidator,
} from '@/lib/credentials/token-service-accounts/validators/http'

Expand Down Expand Up @@ -159,8 +160,11 @@ describe('tokenCheckValidator with a typed host', () => {
)
})

it('refuses a base URL with a mistyped scheme instead of reading the scheme as a host', () => {
it('refuses a mistyped scheme instead of reading the scheme as a host', () => {
expect(() => normalizeHttpBaseUrl('https:/grafana.example.com')).toThrow(/scheme/)
expect(() => refuseUnsafeTypedUrl('https:/evil.com')).toThrow(/scheme/)
expect(() => refuseUnsafeTypedUrl('HTTP:evil.com')).toThrow(/scheme/)
expect(refuseUnsafeTypedUrl('https://grafana.example.com')).toBe('https://grafana.example.com')
expect(normalizeHttpBaseUrl('grafana.example.com:3000/')).toBe(
'https://grafana.example.com:3000'
)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,7 @@ export function normalizeHttpBaseUrl(raw: string | undefined): string {
*/
export function refuseUnsafeTypedUrl(raw: string | undefined): string {
const typed = (raw ?? '').trim()
if (/^https?:(?!\/\/)/i.test(typed)) throw new Error('malformed scheme')
if (/^(?!https?:\/\/)[a-z][a-z\d+.-]*:\/\//i.test(typed)) throw new Error('http(s) only')
const authority = typed.replace(/^[a-z][a-z\d+.-]*:\/\//i, '').split(/[/?#]/)[0]
if (authority.includes('@')) throw new Error('credentials in URL')
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -420,7 +420,7 @@ export const validateDubServiceAccount = tokenCheckValidator({
*/
export const validateDynatraceServiceAccount = tokenCheckValidator({
serviceLabel: 'Dynatrace',
domain: ({ domain }) => normalizeHttpBaseUrl(domain).replace(/\/api\/v[12]$/, ''),
domain: ({ domain }) => normalizeHttpBaseUrl(domain).replace(/\/api\/v[12]$/i, ''),
egress: 'configuredEndpoint',
request: ({ apiToken, domain = '' }) => ({
url: buildDynatraceUrl(domain, '/apiTokens/lookup'),
Expand Down
16 changes: 16 additions & 0 deletions apps/sim/lib/credentials/workload-identity/issuer.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,22 @@ describe('workload identity issuer', () => {
).rejects.toThrow(/an hour/)
})

it.each([
['another workspace', 'sim:workspace:ws-2:credential:abc123def456'],
['an organization', 'sim:organization:ws-1:credential:abc123def456'],
['a lookalike workspace id', 'sim:workspace:ws-10:credential:abc123def456'],
])('refuses to mint a subject owned by %s', async (_owner, subject) => {
const issuer = await loadIssuer()

await expect(
issuer.mintWorkloadIdentityToken({
subject,
audience: 'sts.amazonaws.com',
owner: { workspaceId: 'ws-1' },
})
).rejects.toThrow(/another owner/)
})

it('rejects a token presented to a different audience', async () => {
const issuer = await loadIssuer()
const token = await issuer.mintWorkloadIdentityToken({
Expand Down
2 changes: 2 additions & 0 deletions apps/sim/lib/credentials/workload-identity/issuer.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import { generateId } from '@sim/utils/id'
import { calculateJwkThumbprint, type JWK, SignJWT } from 'jose'
import { env } from '@/lib/core/config/env'
import { getBaseUrl } from '@/lib/core/utils/urls'
import { assertWorkloadIdentitySubjectOwner } from '@/lib/credentials/workload-identity/subject'

/**
* Sim's workload-identity issuer: a minimal OpenID Connect token issuer that
Expand Down Expand Up @@ -194,6 +195,7 @@ export interface MintWorkloadIdentityTokenInput {
export async function mintWorkloadIdentityToken(
input: MintWorkloadIdentityTokenInput
): Promise<string> {
assertWorkloadIdentitySubjectOwner(input.subject, input.owner)
const ttl = input.ttlSeconds ?? WORKLOAD_IDENTITY_TOKEN_TTL_SECONDS
if (!Number.isInteger(ttl) || ttl <= 0 || ttl > MAX_WORKLOAD_IDENTITY_TOKEN_TTL_SECONDS) {
throw new Error('Workload identity token lifetime must be between 1 second and an hour')
Expand Down
18 changes: 16 additions & 2 deletions apps/sim/lib/credentials/workload-identity/subject.ts
Original file line number Diff line number Diff line change
Expand Up @@ -40,12 +40,26 @@ function ownerSegment(owner: WorkloadIdentityOwner): string {
throw new WorkloadIdentitySubjectError('A workload identity subject needs exactly one owner')
}

function subjectPrefix(owner: WorkloadIdentityOwner): string {
return `sim:${ownerSegment(owner)}:credential:`
}

/** Throws unless the subject was built for this owner. */
export function assertWorkloadIdentitySubjectOwner(
subject: string,
owner: WorkloadIdentityOwner
): void {
if (!subject.startsWith(subjectPrefix(owner))) {
throw new WorkloadIdentitySubjectError('Workload identity subject belongs to another owner')
}
}

/** `sim:workspace:<id>:credential:<subjectId>` (or `sim:organization:…`). */
function buildWorkloadIdentitySubject(owner: WorkloadIdentityOwner, subjectId: string): string {
if (!SUBJECT_ID_PATTERN.test(subjectId)) {
throw new WorkloadIdentitySubjectError('Invalid workload identity subject')
}
return `sim:${ownerSegment(owner)}:credential:${subjectId}`
return `${subjectPrefix(owner)}${subjectId}`
}

function proofKey(): Buffer {
Expand Down Expand Up @@ -114,7 +128,7 @@ export function verifyWorkloadIdentitySubject(
const expiresAt = Number(subjectProof.slice(0, separator))
const signature = subjectProof.slice(separator + 1)
if (!Number.isSafeInteger(expiresAt) || expiresAt <= now) throw invalid()
if (!subject.startsWith(`sim:${ownerSegment(owner)}:credential:`)) throw invalid()
if (!subject.startsWith(subjectPrefix(owner))) throw invalid()
if (!safeCompare(signature, sign(provider, owner, userId, subject, expiresAt))) throw invalid()
}

Expand Down
Loading
Loading