Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
20a924c
fix(docker): keep ISR and suffix-list caches writable on a read-only …
waleedlatif1 Nov 10, 2026
8d459fd
fix(cli): bind sim-setup's containers to loopback, fix reset and env …
waleedlatif1 Nov 10, 2026
61843fe
improvement(search): return a longer live read window when an MCP cli…
waleedlatif1 Nov 10, 2026
bf575fe
feat(credentials): saved credentials and credential-only block versio…
waleedlatif1 Nov 10, 2026
ddc7f04
ci: bump the actions group across 2 directories with 7 updates (#9839)
dependabot[bot] Nov 10, 2026
77ed96b
feat(library): AI Agent Builders With Built-In Web Search and Researc…
waleedlatif1 Nov 10, 2026
27997f4
feat(library): Best AI Agent Platforms That Support the A2A Protocol …
waleedlatif1 Nov 10, 2026
ce4f216
feat(library): AI Agent Development Company vs Building In House: How…
waleedlatif1 Nov 10, 2026
5bffff6
feat(search): update GitLab permission CSVs through the API (#9841)
waleedlatif1 Nov 10, 2026
ba644c0
chore(deps): upgrade pi to 1.5.0 (#9826)
waleedlatif1 Nov 10, 2026
5836fce
feat(search): manage source permissions through a shared API and CLI …
waleedlatif1 Nov 10, 2026
b68b2d9
chore(deps): take vitest 5.1, shiki 4.6, react-hook-form 7.91, MCP SD…
waleedlatif1 Nov 10, 2026
106310e
feat(credentials): saved credentials for 17 more blocks, GitLab token…
waleedlatif1 Nov 10, 2026
7299c3e
chore(sdk): remove the TypeScript and Python SDKs (#9848)
waleedlatif1 Nov 10, 2026
4c52ece
fix(firecrawl): complete v2 response and depth compatibility (#9847)
waleedlatif1 Nov 10, 2026
8577aa4
chore(sdk): use a live release family as the desktop feed's unrelated…
waleedlatif1 Nov 10, 2026
1174092
feat(api): manage workspace members through the v2 API (#9852)
waleedlatif1 Nov 10, 2026
e1a3f29
feat(credentials): saved credentials for 21 token-and-host blocks on …
waleedlatif1 Nov 10, 2026
8718b9c
feat(network): honor HTTP_PROXY, HTTPS_PROXY and NO_PROXY for guarded…
waleedlatif1 Nov 10, 2026
4711852
feat(api): workflow trigger health and enablement through the v2 API …
waleedlatif1 Nov 10, 2026
02293a9
feat(api-keys): read-only API keys and expiry at creation (#9855)
waleedlatif1 Nov 10, 2026
24aab3f
feat(api): run a pinned deployment version and expose paused run poin…
waleedlatif1 Nov 10, 2026
81c10c9
feat(mistral): extract PDF images as files and expose table format in…
waleedlatif1 Nov 10, 2026
a12832d
fix(release): cap proxied plain-http responses, count Firecrawl depth…
waleedlatif1 Nov 10, 2026
03bf275
fix(sim-setup): quote env values containing $ so dotenv, Bun, and Com…
waleedlatif1 Nov 10, 2026
e1a8ea0
improvement(api): release audit follow-ups for v2 member analytics, w…
waleedlatif1 Nov 11, 2026
b03f0c7
fix(release): run pinned async versions on their own variables, map C…
waleedlatif1 Nov 11, 2026
15cf51d
fix(release): redact deploy keys, check operation-backed tools in the…
waleedlatif1 Nov 11, 2026
411cc9c
fix(security): harden saved-credential, proxy egress, and keyless-min…
waleedlatif1 Nov 11, 2026
0b8f632
fix(release): reuse only full-access delegation keys and treat a blan…
waleedlatif1 Nov 11, 2026
4266ef6
fix(release): small release-audit cleanups (#9863)
waleedlatif1 Nov 11, 2026
9b10389
fix(release): keyless subject uniqueness, NO_PROXY IPv6/CIDR, proxied…
waleedlatif1 Nov 11, 2026
1a605bc
fix(credentials): report a refused host as the user's input, persist …
waleedlatif1 Nov 11, 2026
a17b8a8
fix(sim-setup): stop doctor --fix from turning ${VAR} references into…
waleedlatif1 Nov 11, 2026
1b1639c
fix(release): host_not_allowed for refused hosts, internal-JWT-only s…
waleedlatif1 Nov 11, 2026
3c2a91c
docs(skills): name refuseUnsafeTypedUrl in the add-integration token-…
waleedlatif1 Nov 11, 2026
c7a4966
fix(mistral): say V3 parses an uploaded or referenced PDF, and bound …
waleedlatif1 Nov 11, 2026
aabee30
docs(release): correct the Discord allowlist note, Mistral image nami…
waleedlatif1 Nov 11, 2026
9ab4e6f
feat(credentials): keyless Snowflake, Microsoft app-only and Google k…
waleedlatif1 Nov 11, 2026
cac0148
feat(credentials): set up keyless credentials from the v2 API, CLI an…
waleedlatif1 Nov 11, 2026
a4e9694
test(credentials): use a non-token-shaped GitHub fixture so secret sc…
waleedlatif1 Nov 11, 2026
219b39c
test(api-keys): use a neutral database role name in the internal-erro…
waleedlatif1 Nov 11, 2026
a4bbaf7
fix(credentials): correct keyless setup instructions and enforce the …
waleedlatif1 Nov 11, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
The diff you're trying to view is too large. We only load the first 3000 changed files.
66 changes: 64 additions & 2 deletions .agents/skills/add-block/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,8 @@ export const {ServiceName}Block: BlockConfig = {
// Multi-line input
{ id: 'field', title: 'Label', type: 'long-input', placeholder: '...', rows: 6 }

// Password input
// Password input: only for legacy blocks or a secret with no saved-credential fit
// (model-provider keys, hosted-key pairs); a service token uses a saved credential
{ id: 'apiKey', title: 'API Key', type: 'short-input', password: true }
```

Expand Down Expand Up @@ -136,6 +137,32 @@ export const {ServiceName}Block: BlockConfig = {
}
```

A service-account-only picker (a saved token or key credential) instead sets
`credentialKind: 'service-account'` and `mode: 'basic'` (no scopes), and pairs with an advanced
twin; `isSavedCredentialPicker` matches only this shape:
```typescript
{
id: 'credential',
title: '{Service} Account',
type: 'oauth-input',
serviceId: '{service}',
credentialKind: 'service-account',
canonicalParamId: 'oauthCredential',
mode: 'basic',
placeholder: 'Select {Service} credential',
required: true,
},
{
id: 'manualCredential',
title: '{Service} Account',
type: 'short-input',
canonicalParamId: 'oauthCredential',
mode: 'advanced',
placeholder: 'Enter credential ID',
required: true,
}
```

**Scopes:** Always use `getScopesForService(serviceId)` from `@/lib/oauth/utils` for `requiredScopes`. Never hardcode scope arrays — the single source of truth is `OAUTH_PROVIDERS` in `lib/oauth/oauth.ts`.

**Scope descriptions:** When adding a new OAuth provider, also add human-readable descriptions for all scopes in `SCOPE_DESCRIPTIONS` within `lib/oauth/utils.ts`.
Expand All @@ -159,6 +186,40 @@ export const {ServiceName}Block: BlockConfig = {

Optional companions: `credentialLabels` (override the picker's section/connect-row copy) and `allowServiceAccounts: true` (trigger-mode only — list service accounts, which triggers otherwise exclude; set only when the trigger's polling path can resolve a service-account token). The connect modal, provider families (Google JSON key, Atlassian token, token-paste, client-credential, Slack bot), and the preview gate are all resolved from `serviceAccountProviderId` — you don't wire them per block.

### Credential-only successor (replacing inline secrets)

A block that takes a secret inline keeps running for existing workflows; the saved-credential
version is a new block type that spreads it. Keep the successor an object literal with the picker
written inline, because the docs generator reads the source text:

```typescript
export const ServiceV2Block: BlockConfig = {
...ServiceBlock,
type: 'service_v2',
authMode: AuthMode.OAuth,
hideFromToolbar: false,
sunset: undefined,
...withSavedCredential(ServiceBlock, {
id: 'credential',
type: 'oauth-input',
serviceId: 'service',
title: 'Service Credential',
}),
}
```

- The legacy block changes only by `hideFromToolbar: true` and `sunset: { status: 'legacy',
replacedBy: 'service_v2' }` (an older `_v2` points at the new `_v3`). Register both types.
- `withSavedCredential` (`blocks/utils.ts`) swaps exactly the fields its policy in
`SAVED_CREDENTIAL_OAUTH` supplies for the picker, places it directly under the operation
dropdown, inherits the replaced fields' `condition` and `required`, and leaves trigger-mode
fields alone. Pass `required` when the replaced fields disagree (an optional host), and `params`
only when the legacy params builder validates the secret: extract that builder so the legacy
block calls it unchanged (`blocks/legacy-builders.test.ts`) and the successor skips the check.
- Add the pair to `serializer/saved-credentials.test.ts` (`optionalIds` for replaced fields the
legacy block never required), and grep for code that hard-codes the legacy type (`getBlock`,
`integration.type ===`, availability lookups), which now resolves to the successor.

### OAuth deployment availability (required for integration blocks)

A visible tools-category block with OAuth is deployment-gated. Its `oauth-input.serviceId` is
Expand Down Expand Up @@ -661,7 +722,8 @@ outputs: {
When creating V2 blocks (alongside legacy V1):

```typescript
// V1 Block - mark as legacy
// V1 Block - mark as legacy. A credential-only successor is the exception: the
// successor keeps the legacy block's name, so neither block is renamed.
export const ServiceBlock: BlockConfig = {
type: 'service',
name: 'Service (Legacy)',
Expand Down
53 changes: 52 additions & 1 deletion .agents/skills/add-integration/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,12 @@ supported paths and choose the simplest supported setup for the intended user:
| Customer-owned client credentials | Saved `service_account` credential with a client ID and secret | Internal-app eligibility, grant activation, scope syntax, token lifetime and revocation behavior |
| API token or personal access token | Saved token service account when reusable connections are useful | Token permissions, identity verification, expiry and rotation |
| Private key, certificate, or service-account JSON | Existing key-based service-account framework | Signing algorithm, audience, subject, tenant binding and key rotation |
| Cloud workload identity (AWS role, Databricks/Snowflake/Entra/GCP federation) | Keyless saved credential: Sim signs a short-lived OIDC token the customer's cloud trusts. Add the provider to every table in `lib/credentials/workload-identity/audiences.ts` (`WORKLOAD_IDENTITY_PROVIDERS`, its service-account provider ID map, `WORKLOAD_IDENTITY_KEYLESS_CONFIGS`, `WORKLOAD_IDENTITY_AUDIENCES`) and to `MIXED_MODE_PROVIDERS` in `lib/credentials/application/prepare-workload-identity.ts` | Issuer and audience the provider accepts, how the trust pins the subject, token lifetime |

API, CLI, and MCP callers set up keyless in two steps: `POST /api/v2/credentials/workload-identity/subjects`
(or `sim credentials keyless-subject <providerId> --output json`) returns the subject to trust and a
`keylessConfig`; they add its `requiredConfigFields` and create the credential with it. The provider
catalog advertises `keyless` only when the deployment can sign tokens.

Do not present customer-owned credentials as a way around a provider's approval requirements
for a shared public integration. Explain the supported account/app type in the setup docs. Keep
Expand All @@ -66,6 +72,50 @@ Use explicit permission choices when the supported operations have different acc
region and permission selections on reconnect unless the user changes them. Verify documented
identity endpoints rather than guessing which person a client-credentials token represents.

### Token credentials (API key, personal access token, token + host)

A service authenticated by a pasted key or token takes it through a saved token credential, never an
inline `password` field, so one shared, verified credential serves every block. Wire it by the
service id:

1. `apps/sim/lib/credentials/token-service-accounts/descriptors.ts`: a provider id and descriptor.
Fields are `apiToken`, plus `domain` for a host, instance URL, or region and
`providerWorkspaceId` for an account or org id; every listed field is required. Set `url: true`
on a `domain` that is a URL whose scheme or path matters, so the connect form sends it as typed.
2. `validators/token-checks.ts`: one `tokenCheckValidator` declaration (register it in
`server.ts`). Make one cheap, documented read on the tools' own host with the tools' own auth,
reusing their header and URL builders. `domain` and `workspaceId` normalize to exactly the form
the tools build URLs from and throw for anything that could steer the host or path
(`normalizeHttpBaseUrl`, `allowedDomain`, `refuseUnsafeTypedUrl`); the normalized domain is what
the credential stores. Set `egress` when the user chooses the host. Add `forbiddenMeansScoped`,
`rejectsKey`, or `provesKey` only for documented replies (probe a fake key to see how a bad one
answers), and `errorBodyEchoesToken` when an error body echoes the token.
3. `apps/sim/lib/oauth/oauth.ts` and `types.ts`: the service entry (`authType: 'service_account'`,
`serviceAccountProviderId`).
4. The tools' credential policy. A NEW integration's tools take a hidden `accessToken` and declare
`oauth: { required: true, provider: '<service>', credentialKind: 'service-account' }`, pinning
any host or account param with `authoritativeParams` (Flow's and Snowflake's tools are the
pattern). `tokenCredentialOAuth('<service>', { <toolParam>: 'accessToken' | 'domain' |
'providerWorkspaceId' })` in `apps/sim/tools/shared/saved-credentials.ts`, listed in
`SAVED_CREDENTIAL_OAUTH`, is for existing tools that already take the token inline (successors).
Either way, the executor overwrites mapped and pinned params after authorizing the credential,
and `check-block-registry` probes the real URL builders of every block with a saved-credential
picker and fails on a host-steering param left unmapped or unpinned.
5. A credential-only block (including a successor) uses `authMode: AuthMode.OAuth`, so the
generated catalog maps the service; a token-only block that keeps `AuthMode.ApiKey`, as Coda
does, registers in `tokenCredentialIntegrationTypes` instead (see below). The credential picker
sits directly under the operation dropdown (`add-block` → OAuth/Credentials). A new block
writes the picker inline, as Higgsfield does: `{ id: 'credential', type: 'oauth-input', serviceId,
credentialKind: 'service-account', canonicalParamId: 'oauthCredential', mode: 'basic',
required: true }`, plus a `manualCredential` `short-input` with the same `canonicalParamId` in
`mode: 'advanced'`. An existing block that already takes the secret inline is never edited in
place: add a credential-only successor (`add-block` → Credential-only successor).

Then run `generate-docs`, `generate:block-successors`, `generate:deployment-config`, and
`generate:tool-metadata`, and `check-block-registry.ts origin/staging`. This pattern does not fit
hosted-key tools, model-provider keys, two-secret or username/password logins, or a host chosen
per call.

### Pair saved credentials with block and resource selectors

`oauth-input` is the shared saved-credential picker, including for service accounts that never
Expand Down Expand Up @@ -425,7 +475,8 @@ If creating V2 versions (API-aligned outputs):

1. **V2 Tools** - Add `_v2` suffix, version `2.0.0`, flat outputs
2. **V2 Block** - Add `_v2` type, use `createVersionedToolSelector`
3. **V1 Block** - Add `(Legacy)` to name, set `hideFromToolbar: true`, and add
3. **V1 Block** - Add `(Legacy)` to name (a credential-only successor is the exception: it keeps
the legacy block's name), set `hideFromToolbar: true`, and add
`sunset: { status: 'legacy', replacedBy: '{service}_v2' }` — `check-block-registry`
fails a legacy block with no `replacedBy`, and the amber legacy badge plus its
click-to-upgrade action read from that field.
Expand Down
4 changes: 4 additions & 0 deletions .agents/skills/add-tools/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,10 @@ export const {serviceName}{Action}Tool: ToolConfig<
provider: '{service}', // Must match the service id in OAUTH_PROVIDERS
// credentialKind: 'service-account', // service-account-only integrations
},
// A token credential (API key, PAT, token + host) instead uses the service's shared policy from
// tools/shared/saved-credentials.ts, built with tokenCredentialOAuth('{service}', { apiKey:
// 'accessToken' }). Its keys are the tool's own token/host/account params, which the executor
// fills after authorizing the credential.

params: {
// Hidden params (system-injected, e.g. the OAuth accessToken)
Expand Down
18 changes: 18 additions & 0 deletions .agents/skills/validate-integration/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -218,6 +218,24 @@ For **each tool** in `tools.access`:
- True/false → `switch` (a Yes/No `dropdown` only when the tool needs a third "unset" state)
- Credentials → `oauth-input` with correct `serviceId`
- Every tool with a `credential`/`oauthCredential` param declares `oauth: { required: true, provider }` matching that `serviceId`, plus `credentialKind: 'service-account'` when only service accounts are supported
- [ ] In a new block or credential-only successor, a pasted key, token, or token + host comes from a
saved token credential (add-integration → Token credentials), not an inline `password` field
(legacy blocks keep theirs); a successor changes its legacy block only by `hideFromToolbar`
and `sunset`, and its picker sits under the operation
- [ ] For a token-credential integration, every tool in `tools.access` declares the service's
`tokenCredentialOAuth` policy, every
param that decides the request host is mapped, and the stored domain is in the form the tools
build URLs from (`check-block-registry.ts origin/staging` passes)
- [ ] A non-successor token block's tools declare `oauth: { required: true, provider,
credentialKind: 'service-account' }` and pin every host or account param in
`authoritativeParams` (`check-block-registry` enforces this for every block with a
saved-credential picker)
- [ ] A keyless provider appears in every table in `lib/credentials/workload-identity/audiences.ts`
and in `MIXED_MODE_PROVIDERS` (`prepare-workload-identity.ts`), and the provider catalog
shows `keyless` only when the deployment can sign tokens
- [ ] The `tokenCheckValidator` request uses the tools' host and auth, its normalizers refuse
anything that could steer the host or path, and a fake key is rejected without a valid
scoped key being refused
- [ ] Dropdown `value: () => 'default'` is set for dropdowns with a sensible default
- [ ] Every `short-input`, `long-input`, `code`, and selector subBlock has a `placeholder` — including
password fields (`Enter your API key`). Formatted values show the shape
Expand Down
4 changes: 2 additions & 2 deletions .github/CI.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,13 @@ The `changes` job runs Turborepo 2.9.14's `affectedTasks` query before installin

A PR compares the checked-out GitHub merge commit with its first parent: the actual target revision tested by that merge. The PR event's base SHA can become stale when the target advances. Scoping requires a matching GitHub merge ref and SHA, exactly two merge parents, and a usable target parent; otherwise every check runs. This works with the existing depth-two checkout without fetching older ancestry. Main/staging pushes execute all checks; the parent `ci` workflow skips checks on `dev` pushes, while PRs targeting `dev` still run checks. A manual `checks` run executes all checks unless supplied a full base commit SHA to reproduce a scoped plan. Missing history, an empty diff, unknown files outside the workspace graph, root script/CI changes, or a planner error run every check. Missing selection outputs fail the aggregate gate. The gate also rejects an omitted dependency, any failed/cancelled job, and any skip that its explicit selection flag did not authorize.

Dedicated docs verification uses the same planner, comparing pushes with their previous commit and PRs with the tested merge's first parent. Site acceptance follows `docs#build` and its transitive package dependencies. The app's docs-consumer task depends on server runtime and that docs build: backend-only changes run the platform consumer against the docs app, while site changes also run corpus and browser checks. Independent SDK and unit-only edits skip those production jobs. The required `docs HTTP` gate verifies explicit site/platform selections before accepting any skip.
Dedicated docs verification uses the same planner, comparing pushes with their previous commit and PRs with the tested merge's first parent. Site acceptance follows `docs#build` and its transitive package dependencies. The app's docs-consumer task depends on server runtime and that docs build: backend-only changes run the platform consumer against the docs app, while site changes also run corpus and browser checks. Standalone package and unit-only edits skip those production jobs. The required `docs HTTP` gate verifies explicit site/platform selections before accepting any skip.

Docs production builds run once and publish an artifact to isolated HTTP, corpus, Chromium, and WebKit jobs. Source checks keep their own checkout because generator verification mutates inputs. Content caches use separate lint/source/build namespaces; generation validates every restore, repairs missing declarations, and removes obsolete modules. The Next build cache keeps its supported incremental behavior. Content and Next caches have size limits, and Next cache files expire after seven days; successful content eviction happens after all consumers finish.

| Change | Expensive checks selected |
| --- | --- |
| Independent SDK | That workspace's tests and type-check; no app build, services, or browsers |
| Standalone package | That workspace's tests and type-check; no app build, services, or browsers |
| App unit test | App test shards and test type check |
| Integration test | Both provision paths; no HTTP/browser job merely for the test edit |
| Workspace UI | Database integration, mobile, SEO and live Electron, plus app tests/build |
Expand Down
14 changes: 5 additions & 9 deletions .github/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,23 +140,19 @@ Using clear and consistent commit messages makes it easier for everyone to under

To set up your local development environment:

### Option 1: Using NPM Package (Simplest)
### Option 1: Using the Setup Wizard (Simplest)

The easiest way to run Sim locally is using our NPM package:
The easiest way to run Sim locally is the interactive setup wizard:

```bash
npx simstudio
npx sim-setup
```

After running this command, open [http://localhost:3000/](http://localhost:3000/) in your browser.

#### Options

- `-p, --port <port>`: Specify the port to run Sim on (default: 3000)
- `--no-pull`: Skip pulling the latest Docker images
When it finishes, open [http://localhost:3000/](http://localhost:3000/) in your browser. Inside a cloned Sim repository, run `bun run sim-setup --mode dev` for the local development mode.

#### Requirements

- Node.js 20+
- Docker must be installed and running on your machine

### Option 2: Using Docker Compose
Expand Down
2 changes: 1 addition & 1 deletion .github/actions/image/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ runs:
# under 300 MB, sat at 249 GB. Sticky disks bill at ~$0.51/GB-month.
- name: Set up Blacksmith builder
if: inputs.provider == '' || inputs.provider == 'blacksmith'
uses: useblacksmith/setup-docker-builder@19215110ab936351210feebdfa5b440b4493e184 # v2
uses: useblacksmith/setup-docker-builder@06dd9c83a5d541274d2e3ac67625560979275267 # v2
with:
cache-key: ${{ steps.cache-key.outputs.value }}

Expand Down
3 changes: 3 additions & 0 deletions .github/scripts/http-e2e.sh
Original file line number Diff line number Diff line change
Expand Up @@ -259,6 +259,9 @@ case "$group" in
bun --no-env-file scripts/test-assistant-actions-e2e.ts
;;
organization-search)
GITLAB_CSV_E2E_BASE_URL="$local_app_url" \
GITLAB_CSV_E2E_REPORT_PATH="$report_dir/gitlab-csv-permissions-http-report.json" \
bun --no-env-file scripts/test-gitlab-csv-permissions-e2e.ts
ORGANIZATION_SEARCH_E2E_BASE_URL="$local_app_url" \
ORGANIZATION_SEARCH_E2E_DATABASE_URL="$DATABASE_URL" \
ORGANIZATION_SEARCH_E2E_AUTH_SECRET="$BETTER_AUTH_SECRET" \
Expand Down
Loading
Loading