Compare commits

..

106 Commits

Author SHA1 Message Date
Scott Wilson
72780c61b4 fix: check create member permission for invite ability 2025-02-11 16:37:25 -08:00
Scott Wilson
c4da0305ba improvement: supress eslint error and improve text 2025-02-11 16:19:37 -08:00
Scott Wilson
4fdfdc1a39 improvements: ui improvements & add users to org from project member invite modal 2025-02-11 16:14:13 -08:00
Daniel Hougaard
4e48ab1eeb Merge pull request #3097 from Infisical/daniel/aws-arn-validate-fix
fix: improve arn validation regex
2025-02-10 18:55:13 +01:00
Daniel Hougaard
a6671b4355 fix: improve arn validation regex 2025-02-10 21:47:07 +04:00
Daniel Hougaard
4c7ae3475a Merge pull request #3092 from Infisical/daniel/azure-app-connection
feat(secret-syncs): azure app config & key vault support
2025-02-08 02:02:12 +01:00
Scott Wilson
49797c3c13 improvements: minor textual improvements 2025-02-07 16:58:11 -08:00
Daniel Hougaard
7d9c5657aa Update AzureKeyVaultSyncFields.tsx 2025-02-08 04:50:25 +04:00
Daniel Hougaard
eda4abb610 fix: orphan labels when switching from no label -> label 2025-02-08 04:27:17 +04:00
Daniel Hougaard
e341bbae9d chore: requested changes 2025-02-08 04:27:17 +04:00
Daniel Hougaard
7286f9a9e6 fix: secrets being deleted 2025-02-08 04:27:17 +04:00
Daniel Hougaard
1c9a9283ae added import support 2025-02-08 04:27:17 +04:00
Daniel Hougaard
8d52011173 requested changes 2025-02-08 04:27:17 +04:00
Daniel Hougaard
1b5b937db5 requested changes 2025-02-08 04:27:17 +04:00
Daniel Hougaard
7b8b024654 Update SecretSyncConnectionField.tsx 2025-02-08 04:27:17 +04:00
Daniel Hougaard
a67badf660 requested changes 2025-02-08 04:27:17 +04:00
Daniel Hougaard
ba42ea736b docs: azure connection & syncs 2025-02-08 04:27:17 +04:00
Daniel Hougaard
6c7289ebe6 fix: smaller fixes 2025-02-08 04:27:17 +04:00
Daniel Hougaard
5cd6a66989 feat(secret-syncs): azure app config & key vault support 2025-02-08 04:27:17 +04:00
Scott Wilson
4e41e84491 Merge pull request #3093 from Infisical/cmek-additions
Improvement: CMEK Additions and Normalization
2025-02-07 10:04:55 -08:00
Scott Wilson
85d71b1085 Merge pull request #3095 from Infisical/secret-sync-handle-larger-messages
Improvement: Increase Max Error Message Size for Secret Syncs
2025-02-07 10:04:30 -08:00
Daniel Hougaard
9d66659f72 Merge pull request #3060 from Infisical/daniel/query-secrets-by-metadata
feat(api): list secrets filter by metadata
2025-02-07 04:53:30 +01:00
Daniel Hougaard
70c9761abe requested changes 2025-02-07 07:49:42 +04:00
Scott Wilson
6047c4489b improvement: increase max error message size for secret syncs and handle messages that exceed limit 2025-02-06 17:14:22 -08:00
Scott Wilson
c9d7559983 Merge pull request #3072 from Infisical/secret-metadata-audit-log
Improvement: Include Secret Metadata in Audit Logs
2025-02-06 15:10:49 -08:00
Scott Wilson
66251403bf Merge pull request #3086 from Infisical/aws-secrets-manager-sync
Feature: AWS Secrets Manager Sync
2025-02-06 11:26:26 -08:00
Scott Wilson
b9c4407507 fix: skip empty values for create 2025-02-06 10:12:51 -08:00
Scott Wilson
624be80768 improvement: address feedback 2025-02-06 08:25:39 -08:00
Daniel Hougaard
8d7b5968d3 requested changes 2025-02-06 07:39:47 +04:00
Scott Wilson
b7d4bb0ce2 improvement: add name constraint error feedback to update cmek 2025-02-05 17:36:52 -08:00
Scott Wilson
598dea0dd3 improvements: cmek additions, normalization and remove kms key slug col 2025-02-05 17:28:57 -08:00
Maidul Islam
7154b19703 update azure app connection docs 2025-02-05 19:26:14 -05:00
Maidul Islam
9ce465b3e2 Update azure-app-configuration.mdx 2025-02-05 19:22:05 -05:00
Maidul Islam
598e5c0be5 Update azure-app-configuration.mdx 2025-02-05 19:16:57 -05:00
Scott Wilson
72f08a6b89 Merge pull request #3090 from Infisical/fix-dashboard-search-exclude-replicas
Fix: Exclude Reserved Folders from Deep Search Folder Query
2025-02-05 13:58:05 -08:00
Scott Wilson
55d8762351 fix: exclude reserved folders from deep search 2025-02-05 13:53:14 -08:00
Akhil Mohan
3c92ec4dc3 Merge pull request #3088 from akhilmhdh/fix/increare-gcp-sa-limit
feat: increased identity gcp auth cred limit from 255 to respective limits
2025-02-06 01:53:55 +05:30
Maidul Islam
f2224262a4 Merge pull request #3089 from Infisical/misc/removed-unused-and-outdated-metadata-field
misc: removed outdated metadata field
2025-02-05 12:19:24 -05:00
Scott Wilson
23eac40740 Merge pull request #3081 from Infisical/secrets-overview-page-move-secrets
Feature: Secrets Overview Page Move Secrets
2025-02-05 08:54:06 -08:00
Sheen Capadngan
4ae88c0447 misc: removed outdated metadata field 2025-02-05 18:55:16 +08:00
=
7aecaad050 feat: increased identity gcp auth cred limit from 255 to respective limits 2025-02-05 10:38:10 +05:30
Scott Wilson
cf61390e52 improvements: address feedback 2025-02-04 20:14:47 -08:00
Scott Wilson
3f02481e78 feature: aws secrets manager sync 2025-02-04 19:58:30 -08:00
Scott Wilson
7adc103ed2 Merge pull request #3082 from Infisical/app-connections-and-secret-syncs-unique-constraint
Fix: Move App Connection and Secret Sync Unique Name Constraint to DB
2025-02-04 09:42:02 -08:00
Scott Wilson
5bdbf37171 improvement: add error codes enum for re-use 2025-02-04 08:37:06 -08:00
Maidul Islam
4f874734ab Update operator version 2025-02-04 10:10:59 -05:00
Maidul Islam
eb6fd8259b Merge pull request #3085 from Infisical/combine-helm-release
Combine image release with helm
2025-02-04 10:07:52 -05:00
Maidul Islam
1766a44dd0 Combine image release with helm
Combine image release with helm release so that one happens after the other. This will help reduce manual work.
2025-02-04 09:59:32 -05:00
Akhil Mohan
624c9ef8da Merge pull request #3083 from akhilmhdh/fix/base64-decode-issue
Resolved base64 decode saving file as ansii
2025-02-04 20:04:02 +05:30
=
dfd4b13574 fix: resolved base64 decode saving file as ansii 2025-02-04 16:14:28 +05:30
Scott Wilson
22b57b7a74 chore: add migration file 2025-02-03 19:40:00 -08:00
Scott Wilson
1ba0b9c204 improvement: move unique name constraint to db for secret syncs and app connections 2025-02-03 19:36:37 -08:00
Scott Wilson
a903537441 fix: clear selection if modal is closed through cancel button and secrets have been moved 2025-02-03 18:44:52 -08:00
Scott Wilson
92c4d83714 improvement: make results look better 2025-02-03 18:29:38 -08:00
Scott Wilson
a6414104ad feature: secrets overview page move secrets 2025-02-03 18:18:00 -08:00
Daniel Hougaard
071f37666e Update secret-v2-bridge-dal.ts 2025-02-03 23:22:27 +04:00
Daniel Hougaard
cd5078d8b7 Update secret-router.ts 2025-02-03 23:22:20 +04:00
Maidul Islam
110d0e95b0 Merge pull request #3077 from carlosvargas9103/carlosvargas9103-fix-typo-readme
fixed typo in README.md
2025-02-03 13:26:32 -05:00
BlackMagiq
a8c0bbb7ca Merge pull request #3080 from Infisical/update-security-docs
Update Security Docs
2025-02-03 10:13:26 -08:00
Tuan Dang
6af8a4fab8 Update security docs 2025-02-03 10:07:57 -08:00
Daniel Hougaard
407fd8eda7 chore: rename to metadata filter 2025-02-03 21:16:07 +04:00
Daniel Hougaard
9d976de19b Revert "fix: improved filter"
This reverts commit be99e40050.
2025-02-03 21:13:47 +04:00
Carlos Vargas
43ecd31b74 fixed typo in README.md 2025-02-03 16:18:17 +01:00
Daniel Hougaard
be99e40050 fix: improved filter 2025-02-03 12:54:54 +04:00
Scott Wilson
800d2c0454 improvement: add secret metadata type 2025-01-31 17:38:58 -08:00
Scott Wilson
6d0534b165 improvement: include secret metadata in audit logs 2025-01-31 17:31:17 -08:00
Vlad Matsiiako
ccee0f5428 Merge pull request #3071 from Infisical/fix-oidc-doc-images
Fix: Remove Relative Paths for ODIC Overview Docs
2025-01-31 15:33:40 -08:00
Scott Wilson
14586c7cd0 fix: remove relative path for oidc docs 2025-01-31 15:30:38 -08:00
Scott Wilson
7090eea716 Merge pull request #3069 from Infisical/oidc-group-membership-mapping
Feature: OIDC Group Membership Mapping
2025-01-31 11:32:38 -08:00
Scott Wilson
01d3443139 improvement: update docker dev and makefile for keycloak dev 2025-01-31 11:14:49 -08:00
Scott Wilson
c4b23a8d4f improvement: improve grammar 2025-01-31 11:05:56 -08:00
Scott Wilson
90a2a11fff improvement: update tooltips 2025-01-31 11:04:20 -08:00
Scott Wilson
95d7c2082c improvements: address feedback 2025-01-31 11:01:54 -08:00
Sheen
ab5eb4c696 Merge pull request #3070 from Infisical/misc/readded-operator-installation-flag
misc: readded operator installation flag for secret CRD
2025-01-31 16:53:57 +08:00
Akhil Mohan
65aeb81934 Merge pull request #3011 from xinbenlv/patch-1
Fix grammar on overview.mdx
2025-01-31 14:22:03 +05:30
Akhil Mohan
a406511405 Merge pull request #3048 from isaiahmartin847/refactor/copy-secret
Improve Visibility and Alignment of Tooltips and Copy Secret Key Icon
2025-01-31 14:20:02 +05:30
Sheen Capadngan
61da0db49e misc: readded operator installation flag for CRD 2025-01-31 16:03:42 +08:00
Daniel Hougaard
0968893d4b improved filtering format 2025-01-30 21:41:17 +01:00
Scott Wilson
59666740ca chore: revert license and remove unused query key/doc reference 2025-01-30 10:35:23 -08:00
Scott Wilson
9cc7edc869 feature: oidc group membership mapping 2025-01-30 10:21:30 -08:00
Daniel Hougaard
e1b016f76d Merge pull request #3068 from nicogiard/patch-1
fix: wrong client variable in c# code example
2025-01-29 22:24:03 +01:00
Nicolas Giard
1175b9b5af fix: wrong client variable
The InfisicalClient variable was wrong
2025-01-29 21:57:57 +01:00
Maidul Islam
09521144ec Merge pull request #3066 from akhilmhdh/fix/secret-list-plain
Resolved list secret plain to have key as well
2025-01-29 14:04:49 -05:00
=
8759944077 feat: resolved list secret plain to have key as well 2025-01-30 00:31:47 +05:30
Maidul Islam
aac3c355e9 Merge pull request #3061 from Infisical/secret-sync-ui-doc-improvements
improvements: Import Behavior Doc/UI Clarification and Minor Integration Layout Adjustments
2025-01-29 13:16:21 -05:00
Akhil Mohan
2a28a462a5 Merge pull request #3053 from Infisical/daniel/k8s-insight
k8s: bug fixes and better prints
2025-01-29 23:16:46 +05:30
Maidul Islam
216cae9b33 Merge pull request #3058 from Infisical/misc/improved-helper-text-for-gcp-sa-field
misc: improved helper text for GCP sa field
2025-01-29 09:54:20 -05:00
Daniel Hougaard
d24a5d96e3 requested changes 2025-01-29 14:24:23 +01:00
Akhil Mohan
89d4d4bc92 Merge pull request #3064 from akhilmhdh/fix/secret-path-validation-permission
feat: added validation for secret path in permission
2025-01-29 18:46:38 +05:30
=
cffcb28bc9 feat: removed secret path check in glob 2025-01-29 17:50:02 +05:30
=
61388753cf feat: updated to support in error in ui 2025-01-29 17:32:13 +05:30
=
a6145120e6 feat: added validation for secret path in permission 2025-01-29 17:01:45 +05:30
Sheen Capadngan
dacffbef08 doc: documentation updates for gcp app connection 2025-01-29 18:12:17 +08:00
Sheen Capadngan
4db3e5d208 Merge remote-tracking branch 'origin/main' into misc/improved-helper-text-for-gcp-sa-field 2025-01-29 17:43:48 +08:00
Maidul Islam
2a84d61862 add guide for how to wrote a design doc 2025-01-28 23:31:12 -05:00
Daniel Hougaard
55b0dc7f81 chore: cleanup 2025-01-28 23:35:07 +01:00
Daniel Hougaard
ba03fc256b Update secret-router.ts 2025-01-28 23:30:28 +01:00
Daniel Hougaard
ea28c374a7 feat(api): filter secrets by metadata 2025-01-28 23:29:02 +01:00
Sheen Capadngan
9fcb1c2161 misc: added emphasis on suffix 2025-01-29 04:38:16 +08:00
Sheen Capadngan
ee49f714b9 misc: added valid example to error thrown for sa mismatch 2025-01-29 03:41:24 +08:00
Sheen Capadngan
b5d60398d6 misc: improved helper text for GCP sa field 2025-01-29 03:10:37 +08:00
Daniel Hougaard
2c75e23acf helm 2025-01-28 04:21:29 +01:00
Daniel Hougaard
907dd4880a fix(k8): reconcile on status update 2025-01-28 04:20:51 +01:00
isaiahmartin847
92f697e195 I removed the hover opacity on the 'copy secret name' icon so the icon is always visible instead of appearing only on hover. I believe this will make it more noticeable to users.
As a user myself, I didn't realize it was possible to copy a secret name until I accidentally hovered over it.
2025-01-27 12:26:22 -07:00
isaiahmartin847
8062f0238b I added a wrapper div with a class of relative to make the icon and tooltip align vertically inline. 2025-01-27 12:25:38 -07:00
xinbenlv
645dfafba0 Fix grammar on overview.mdx 2025-01-20 09:02:18 -08:00
419 changed files with 7730 additions and 1496 deletions

View File

@@ -92,20 +92,24 @@ ENABLE_MSSQL_SECRET_ROTATION_ENCRYPT=true
# App Connections
# aws assume-role
# aws assume-role connection
INF_APP_CONNECTION_AWS_ACCESS_KEY_ID=
INF_APP_CONNECTION_AWS_SECRET_ACCESS_KEY=
# github oauth
# github oauth connection
INF_APP_CONNECTION_GITHUB_OAUTH_CLIENT_ID=
INF_APP_CONNECTION_GITHUB_OAUTH_CLIENT_SECRET=
#github app
#github app connection
INF_APP_CONNECTION_GITHUB_APP_CLIENT_ID=
INF_APP_CONNECTION_GITHUB_APP_CLIENT_SECRET=
INF_APP_CONNECTION_GITHUB_APP_PRIVATE_KEY=
INF_APP_CONNECTION_GITHUB_APP_SLUG=
INF_APP_CONNECTION_GITHUB_APP_ID=
#gcp app
#gcp app connection
INF_APP_CONNECTION_GCP_SERVICE_ACCOUNT_CREDENTIAL=
# azure app connection
INF_APP_CONNECTION_AZURE_CLIENT_ID=
INF_APP_CONNECTION_AZURE_CLIENT_SECRET=

View File

@@ -1,4 +1,4 @@
name: Release Helm Charts
name: Release Infisical Core Helm chart
on: [workflow_dispatch]
@@ -17,6 +17,6 @@ jobs:
- name: Install Cloudsmith CLI
run: pip install --upgrade cloudsmith-cli
- name: Build and push helm package to Cloudsmith
run: cd helm-charts && sh upload-to-cloudsmith.sh
run: cd helm-charts && sh upload-infisical-core-helm-cloudsmith.sh
env:
CLOUDSMITH_API_KEY: ${{ secrets.CLOUDSMITH_API_KEY }}

View File

@@ -1,4 +1,4 @@
name: Release Docker image for K8 operator
name: Release image + Helm chart K8s Operator
on:
push:
tags:
@@ -35,3 +35,18 @@ jobs:
tags: |
infisical/kubernetes-operator:latest
infisical/kubernetes-operator:${{ steps.extract_version.outputs.version }}
- name: Checkout
uses: actions/checkout@v2
- name: Install Helm
uses: azure/setup-helm@v3
with:
version: v3.10.0
- name: Install python
uses: actions/setup-python@v4
- name: Install Cloudsmith CLI
run: pip install --upgrade cloudsmith-cli
- name: Build and push helm package to Cloudsmith
run: cd helm-charts && sh upload-k8s-operator-cloudsmith.sh
env:
CLOUDSMITH_API_KEY: ${{ secrets.CLOUDSMITH_API_KEY }}

View File

@@ -30,3 +30,6 @@ reviewable-api:
npm run type:check
reviewable: reviewable-ui reviewable-api
up-dev-sso:
docker compose -f docker-compose.dev.yml --profile sso up --build

View File

@@ -125,7 +125,7 @@ Install pre commit hook to scan each commit before you push to your repository
infisical scan install --pre-commit-hook
```
Lean about Infisical's code scanning feature [here](https://infisical.com/docs/cli/scanning-overview)
Learn about Infisical's code scanning feature [here](https://infisical.com/docs/cli/scanning-overview)
## Open-source vs. paid

View File

@@ -0,0 +1,23 @@
import { Knex } from "knex";
import { TableName } from "../schemas";
export async function up(knex: Knex): Promise<void> {
const hasManageGroupMembershipsCol = await knex.schema.hasColumn(TableName.OidcConfig, "manageGroupMemberships");
await knex.schema.alterTable(TableName.OidcConfig, (tb) => {
if (!hasManageGroupMembershipsCol) {
tb.boolean("manageGroupMemberships").notNullable().defaultTo(false);
}
});
}
export async function down(knex: Knex): Promise<void> {
const hasManageGroupMembershipsCol = await knex.schema.hasColumn(TableName.OidcConfig, "manageGroupMemberships");
await knex.schema.alterTable(TableName.OidcConfig, (t) => {
if (hasManageGroupMembershipsCol) {
t.dropColumn("manageGroupMemberships");
}
});
}

View File

@@ -0,0 +1,23 @@
import { Knex } from "knex";
import { TableName } from "@app/db/schemas";
export async function up(knex: Knex): Promise<void> {
await knex.schema.alterTable(TableName.AppConnection, (t) => {
t.unique(["orgId", "name"]);
});
await knex.schema.alterTable(TableName.SecretSync, (t) => {
t.unique(["projectId", "name"]);
});
}
export async function down(knex: Knex): Promise<void> {
await knex.schema.alterTable(TableName.AppConnection, (t) => {
t.dropUnique(["orgId", "name"]);
});
await knex.schema.alterTable(TableName.SecretSync, (t) => {
t.dropUnique(["projectId", "name"]);
});
}

View File

@@ -0,0 +1,37 @@
import { Knex } from "knex";
import { TableName } from "../schemas";
export async function up(knex: Knex): Promise<void> {
const hasTable = await knex.schema.hasTable(TableName.IdentityGcpAuth);
const hasAllowedProjectsColumn = await knex.schema.hasColumn(TableName.IdentityGcpAuth, "allowedProjects");
const hasAllowedServiceAccountsColumn = await knex.schema.hasColumn(
TableName.IdentityGcpAuth,
"allowedServiceAccounts"
);
const hasAllowedZones = await knex.schema.hasColumn(TableName.IdentityGcpAuth, "allowedZones");
if (hasTable) {
await knex.schema.alterTable(TableName.IdentityGcpAuth, (t) => {
if (hasAllowedProjectsColumn) t.string("allowedProjects", 2500).alter();
if (hasAllowedServiceAccountsColumn) t.string("allowedServiceAccounts", 5000).alter();
if (hasAllowedZones) t.string("allowedZones", 2500).alter();
});
}
}
export async function down(knex: Knex): Promise<void> {
const hasTable = await knex.schema.hasTable(TableName.IdentityGcpAuth);
const hasAllowedProjectsColumn = await knex.schema.hasColumn(TableName.IdentityGcpAuth, "allowedProjects");
const hasAllowedServiceAccountsColumn = await knex.schema.hasColumn(
TableName.IdentityGcpAuth,
"allowedServiceAccounts"
);
const hasAllowedZones = await knex.schema.hasColumn(TableName.IdentityGcpAuth, "allowedZones");
if (hasTable) {
await knex.schema.alterTable(TableName.IdentityGcpAuth, (t) => {
if (hasAllowedProjectsColumn) t.string("allowedProjects").alter();
if (hasAllowedServiceAccountsColumn) t.string("allowedServiceAccounts").alter();
if (hasAllowedZones) t.string("allowedZones").alter();
});
}
}

View File

@@ -0,0 +1,27 @@
import { Knex } from "knex";
import { TableName } from "@app/db/schemas";
export async function up(knex: Knex): Promise<void> {
if (await knex.schema.hasTable(TableName.KmsKey)) {
const hasSlugCol = await knex.schema.hasColumn(TableName.KmsKey, "slug");
if (hasSlugCol) {
await knex.schema.alterTable(TableName.KmsKey, (t) => {
t.dropColumn("slug");
});
}
}
}
export async function down(knex: Knex): Promise<void> {
if (await knex.schema.hasTable(TableName.KmsKey)) {
const hasSlugCol = await knex.schema.hasColumn(TableName.KmsKey, "slug");
if (!hasSlugCol) {
await knex.schema.alterTable(TableName.KmsKey, (t) => {
t.string("slug", 32);
});
}
}
}

View File

@@ -0,0 +1,31 @@
import { Knex } from "knex";
import { TableName } from "@app/db/schemas";
export async function up(knex: Knex): Promise<void> {
if (await knex.schema.hasTable(TableName.SecretSync)) {
const hasLastSyncMessage = await knex.schema.hasColumn(TableName.SecretSync, "lastSyncMessage");
const hasLastImportMessage = await knex.schema.hasColumn(TableName.SecretSync, "lastImportMessage");
const hasLastRemoveMessage = await knex.schema.hasColumn(TableName.SecretSync, "lastRemoveMessage");
await knex.schema.alterTable(TableName.SecretSync, (t) => {
if (hasLastSyncMessage) t.string("lastSyncMessage", 1024).alter();
if (hasLastImportMessage) t.string("lastImportMessage", 1024).alter();
if (hasLastRemoveMessage) t.string("lastRemoveMessage", 1024).alter();
});
}
}
export async function down(knex: Knex): Promise<void> {
if (await knex.schema.hasTable(TableName.SecretSync)) {
const hasLastSyncMessage = await knex.schema.hasColumn(TableName.SecretSync, "lastSyncMessage");
const hasLastImportMessage = await knex.schema.hasColumn(TableName.SecretSync, "lastImportMessage");
const hasLastRemoveMessage = await knex.schema.hasColumn(TableName.SecretSync, "lastRemoveMessage");
await knex.schema.alterTable(TableName.SecretSync, (t) => {
if (hasLastSyncMessage) t.string("lastSyncMessage").alter();
if (hasLastImportMessage) t.string("lastImportMessage").alter();
if (hasLastRemoveMessage) t.string("lastRemoveMessage").alter();
});
}
}

View File

@@ -17,9 +17,9 @@ export const IdentityGcpAuthsSchema = z.object({
updatedAt: z.date(),
identityId: z.string().uuid(),
type: z.string(),
allowedServiceAccounts: z.string(),
allowedProjects: z.string(),
allowedZones: z.string()
allowedServiceAccounts: z.string().nullable().optional(),
allowedProjects: z.string().nullable().optional(),
allowedZones: z.string().nullable().optional()
});
export type TIdentityGcpAuths = z.infer<typeof IdentityGcpAuthsSchema>;

View File

@@ -16,8 +16,7 @@ export const KmsKeysSchema = z.object({
name: z.string(),
createdAt: z.date(),
updatedAt: z.date(),
projectId: z.string().nullable().optional(),
slug: z.string().nullable().optional()
projectId: z.string().nullable().optional()
});
export type TKmsKeys = z.infer<typeof KmsKeysSchema>;

View File

@@ -27,7 +27,8 @@ export const OidcConfigsSchema = z.object({
createdAt: z.date(),
updatedAt: z.date(),
orgId: z.string().uuid(),
lastUsed: z.date().nullable().optional()
lastUsed: z.date().nullable().optional(),
manageGroupMemberships: z.boolean().default(false)
});
export type TOidcConfigs = z.infer<typeof OidcConfigsSchema>;

View File

@@ -153,7 +153,8 @@ export const registerOidcRouter = async (server: FastifyZodProvider) => {
discoveryURL: true,
isActive: true,
orgId: true,
allowedEmailDomains: true
allowedEmailDomains: true,
manageGroupMemberships: true
}).extend({
clientId: z.string(),
clientSecret: z.string()
@@ -207,7 +208,8 @@ export const registerOidcRouter = async (server: FastifyZodProvider) => {
userinfoEndpoint: z.string().trim(),
clientId: z.string().trim(),
clientSecret: z.string().trim(),
isActive: z.boolean()
isActive: z.boolean(),
manageGroupMemberships: z.boolean().optional()
})
.partial()
.merge(z.object({ orgSlug: z.string() })),
@@ -223,7 +225,8 @@ export const registerOidcRouter = async (server: FastifyZodProvider) => {
userinfoEndpoint: true,
orgId: true,
allowedEmailDomains: true,
isActive: true
isActive: true,
manageGroupMemberships: true
})
}
},
@@ -272,7 +275,8 @@ export const registerOidcRouter = async (server: FastifyZodProvider) => {
clientId: z.string().trim(),
clientSecret: z.string().trim(),
isActive: z.boolean(),
orgSlug: z.string().trim()
orgSlug: z.string().trim(),
manageGroupMemberships: z.boolean().optional().default(false)
})
.superRefine((data, ctx) => {
if (data.configurationType === OIDCConfigurationType.CUSTOM) {
@@ -334,7 +338,8 @@ export const registerOidcRouter = async (server: FastifyZodProvider) => {
userinfoEndpoint: true,
orgId: true,
isActive: true,
allowedEmailDomains: true
allowedEmailDomains: true,
manageGroupMemberships: true
})
}
},
@@ -350,4 +355,25 @@ export const registerOidcRouter = async (server: FastifyZodProvider) => {
return oidc;
}
});
server.route({
method: "GET",
url: "/manage-group-memberships",
schema: {
querystring: z.object({
orgId: z.string().trim().min(1, "Org ID is required")
}),
response: {
200: z.object({
isEnabled: z.boolean()
})
}
},
onRequest: verifyAuth([AuthMode.JWT]),
handler: async (req) => {
const isEnabled = await server.services.oidc.isOidcManageGroupMembershipsEnabled(req.query.orgId, req.permission);
return { isEnabled };
}
});
};

View File

@@ -223,6 +223,7 @@ export enum EventType {
UPDATE_CMEK = "update-cmek",
DELETE_CMEK = "delete-cmek",
GET_CMEKS = "get-cmeks",
GET_CMEK = "get-cmek",
CMEK_ENCRYPT = "cmek-encrypt",
CMEK_DECRYPT = "cmek-decrypt",
UPDATE_EXTERNAL_GROUP_ORG_ROLE_MAPPINGS = "update-external-group-org-role-mapping",
@@ -249,7 +250,9 @@ export enum EventType {
DELETE_SECRET_SYNC = "delete-secret-sync",
SECRET_SYNC_SYNC_SECRETS = "secret-sync-sync-secrets",
SECRET_SYNC_IMPORT_SECRETS = "secret-sync-import-secrets",
SECRET_SYNC_REMOVE_SECRETS = "secret-sync-remove-secrets"
SECRET_SYNC_REMOVE_SECRETS = "secret-sync-remove-secrets",
OIDC_GROUP_MEMBERSHIP_MAPPING_ASSIGN_USER = "oidc-group-membership-mapping-assign-user",
OIDC_GROUP_MEMBERSHIP_MAPPING_REMOVE_USER = "oidc-group-membership-mapping-remove-user"
}
interface UserActorMetadata {
@@ -315,6 +318,8 @@ interface GetSecretsEvent {
};
}
type TSecretMetadata = { key: string; value: string }[];
interface GetSecretEvent {
type: EventType.GET_SECRET;
metadata: {
@@ -323,6 +328,7 @@ interface GetSecretEvent {
secretId: string;
secretKey: string;
secretVersion: number;
secretMetadata?: TSecretMetadata;
};
}
@@ -334,6 +340,7 @@ interface CreateSecretEvent {
secretId: string;
secretKey: string;
secretVersion: number;
secretMetadata?: TSecretMetadata;
};
}
@@ -342,7 +349,12 @@ interface CreateSecretBatchEvent {
metadata: {
environment: string;
secretPath: string;
secrets: Array<{ secretId: string; secretKey: string; secretVersion: number }>;
secrets: Array<{
secretId: string;
secretKey: string;
secretVersion: number;
secretMetadata?: TSecretMetadata;
}>;
};
}
@@ -354,6 +366,7 @@ interface UpdateSecretEvent {
secretId: string;
secretKey: string;
secretVersion: number;
secretMetadata?: TSecretMetadata;
};
}
@@ -362,7 +375,7 @@ interface UpdateSecretBatchEvent {
metadata: {
environment: string;
secretPath: string;
secrets: Array<{ secretId: string; secretKey: string; secretVersion: number }>;
secrets: Array<{ secretId: string; secretKey: string; secretVersion: number; secretMetadata?: TSecretMetadata }>;
};
}
@@ -760,9 +773,9 @@ interface AddIdentityGcpAuthEvent {
metadata: {
identityId: string;
type: string;
allowedServiceAccounts: string;
allowedProjects: string;
allowedZones: string;
allowedServiceAccounts?: string | null;
allowedProjects?: string | null;
allowedZones?: string | null;
accessTokenTTL: number;
accessTokenMaxTTL: number;
accessTokenNumUsesLimit: number;
@@ -782,9 +795,9 @@ interface UpdateIdentityGcpAuthEvent {
metadata: {
identityId: string;
type?: string;
allowedServiceAccounts?: string;
allowedProjects?: string;
allowedZones?: string;
allowedServiceAccounts?: string | null;
allowedProjects?: string | null;
allowedZones?: string | null;
accessTokenTTL?: number;
accessTokenMaxTTL?: number;
accessTokenNumUsesLimit?: number;
@@ -1835,6 +1848,13 @@ interface GetCmeksEvent {
};
}
interface GetCmekEvent {
type: EventType.GET_CMEK;
metadata: {
keyId: string;
};
}
interface CmekEncryptEvent {
type: EventType.CMEK_ENCRYPT;
metadata: {
@@ -2044,6 +2064,26 @@ interface SecretSyncRemoveSecretsEvent {
};
}
interface OidcGroupMembershipMappingAssignUserEvent {
type: EventType.OIDC_GROUP_MEMBERSHIP_MAPPING_ASSIGN_USER;
metadata: {
assignedToGroups: { id: string; name: string }[];
userId: string;
userEmail: string;
userGroupsClaim: string[];
};
}
interface OidcGroupMembershipMappingRemoveUserEvent {
type: EventType.OIDC_GROUP_MEMBERSHIP_MAPPING_REMOVE_USER;
metadata: {
removedFromGroups: { id: string; name: string }[];
userId: string;
userEmail: string;
userGroupsClaim: string[];
};
}
export type Event =
| GetSecretsEvent
| GetSecretEvent
@@ -2205,6 +2245,7 @@ export type Event =
| CreateCmekEvent
| UpdateCmekEvent
| DeleteCmekEvent
| GetCmekEvent
| GetCmeksEvent
| CmekEncryptEvent
| CmekDecryptEvent
@@ -2232,4 +2273,6 @@ export type Event =
| DeleteSecretSyncEvent
| SecretSyncSyncSecretsEvent
| SecretSyncImportSecretsEvent
| SecretSyncRemoveSecretsEvent;
| SecretSyncRemoveSecretsEvent
| OidcGroupMembershipMappingAssignUserEvent
| OidcGroupMembershipMappingRemoveUserEvent;

View File

@@ -2,6 +2,7 @@ import { ForbiddenError } from "@casl/ability";
import slugify from "@sindresorhus/slugify";
import { OrgMembershipRole, TOrgRoles } from "@app/db/schemas";
import { TOidcConfigDALFactory } from "@app/ee/services/oidc/oidc-config-dal";
import { isAtLeastAsPrivileged } from "@app/lib/casl";
import { BadRequestError, ForbiddenRequestError, NotFoundError, UnauthorizedError } from "@app/lib/errors";
import { alphaNumericNanoId } from "@app/lib/nanoid";
@@ -45,6 +46,7 @@ type TGroupServiceFactoryDep = {
projectKeyDAL: Pick<TProjectKeyDALFactory, "find" | "delete" | "findLatestProjectKey" | "insertMany">;
permissionService: Pick<TPermissionServiceFactory, "getOrgPermission" | "getOrgPermissionByRole">;
licenseService: Pick<TLicenseServiceFactory, "getPlan">;
oidcConfigDAL: Pick<TOidcConfigDALFactory, "findOne">;
};
export type TGroupServiceFactory = ReturnType<typeof groupServiceFactory>;
@@ -59,7 +61,8 @@ export const groupServiceFactory = ({
projectBotDAL,
projectKeyDAL,
permissionService,
licenseService
licenseService,
oidcConfigDAL
}: TGroupServiceFactoryDep) => {
const createGroup = async ({ name, slug, role, actor, actorId, actorAuthMethod, actorOrgId }: TCreateGroupDTO) => {
if (!actorOrgId) throw new UnauthorizedError({ message: "No organization ID provided in request" });
@@ -311,6 +314,18 @@ export const groupServiceFactory = ({
message: `Failed to find group with ID ${id}`
});
const oidcConfig = await oidcConfigDAL.findOne({
orgId: group.orgId,
isActive: true
});
if (oidcConfig?.manageGroupMemberships) {
throw new BadRequestError({
message:
"Cannot add user to group: OIDC group membership mapping is enabled - user must be assigned to this group in your OIDC provider."
});
}
const { permission: groupRolePermission } = await permissionService.getOrgPermissionByRole(group.role, actorOrgId);
// check if user has broader or equal to privileges than group
@@ -366,6 +381,18 @@ export const groupServiceFactory = ({
message: `Failed to find group with ID ${id}`
});
const oidcConfig = await oidcConfigDAL.findOne({
orgId: group.orgId,
isActive: true
});
if (oidcConfig?.manageGroupMemberships) {
throw new BadRequestError({
message:
"Cannot remove user from group: OIDC group membership mapping is enabled - user must be removed from this group in your OIDC provider."
});
}
const { permission: groupRolePermission } = await permissionService.getOrgPermissionByRole(group.role, actorOrgId);
// check if user has broader or equal to privileges than group

View File

@@ -5,6 +5,11 @@ import { Issuer, Issuer as OpenIdIssuer, Strategy as OpenIdStrategy, TokenSet }
import { OrgMembershipStatus, SecretKeyEncoding, TableName, TUsers } from "@app/db/schemas";
import { TOidcConfigsUpdate } from "@app/db/schemas/oidc-configs";
import { TAuditLogServiceFactory } from "@app/ee/services/audit-log/audit-log-service";
import { EventType } from "@app/ee/services/audit-log/audit-log-types";
import { TGroupDALFactory } from "@app/ee/services/group/group-dal";
import { addUsersToGroupByUserIds, removeUsersFromGroupByUserIds } from "@app/ee/services/group/group-fns";
import { TUserGroupMembershipDALFactory } from "@app/ee/services/group/user-group-membership-dal";
import { TLicenseServiceFactory } from "@app/ee/services/license/license-service";
import { OrgPermissionActions, OrgPermissionSubjects } from "@app/ee/services/permission/org-permission";
import { TPermissionServiceFactory } from "@app/ee/services/permission/permission-service";
@@ -18,13 +23,18 @@ import {
infisicalSymmetricEncypt
} from "@app/lib/crypto/encryption";
import { BadRequestError, ForbiddenRequestError, NotFoundError, OidcAuthError } from "@app/lib/errors";
import { AuthMethod, AuthTokenType } from "@app/services/auth/auth-type";
import { OrgServiceActor } from "@app/lib/types";
import { ActorType, AuthMethod, AuthTokenType } from "@app/services/auth/auth-type";
import { TAuthTokenServiceFactory } from "@app/services/auth-token/auth-token-service";
import { TokenType } from "@app/services/auth-token/auth-token-types";
import { TGroupProjectDALFactory } from "@app/services/group-project/group-project-dal";
import { TOrgBotDALFactory } from "@app/services/org/org-bot-dal";
import { TOrgDALFactory } from "@app/services/org/org-dal";
import { getDefaultOrgMembershipRole } from "@app/services/org/org-role-fns";
import { TOrgMembershipDALFactory } from "@app/services/org-membership/org-membership-dal";
import { TProjectDALFactory } from "@app/services/project/project-dal";
import { TProjectBotDALFactory } from "@app/services/project-bot/project-bot-dal";
import { TProjectKeyDALFactory } from "@app/services/project-key/project-key-dal";
import { SmtpTemplates, TSmtpService } from "@app/services/smtp/smtp-service";
import { getServerCfg } from "@app/services/super-admin/super-admin-service";
import { LoginMethod } from "@app/services/super-admin/super-admin-types";
@@ -45,7 +55,14 @@ import {
type TOidcConfigServiceFactoryDep = {
userDAL: Pick<
TUserDALFactory,
"create" | "findOne" | "transaction" | "updateById" | "findById" | "findUserEncKeyByUserId"
| "create"
| "findOne"
| "updateById"
| "findById"
| "findUserEncKeyByUserId"
| "findUserEncKeyByUserIdsBatch"
| "find"
| "transaction"
>;
userAliasDAL: Pick<TUserAliasDALFactory, "create" | "findOne">;
orgDAL: Pick<
@@ -57,8 +74,23 @@ type TOidcConfigServiceFactoryDep = {
licenseService: Pick<TLicenseServiceFactory, "getPlan" | "updateSubscriptionOrgMemberCount">;
tokenService: Pick<TAuthTokenServiceFactory, "createTokenForUser">;
smtpService: Pick<TSmtpService, "sendMail" | "verify">;
permissionService: Pick<TPermissionServiceFactory, "getOrgPermission">;
permissionService: Pick<TPermissionServiceFactory, "getOrgPermission" | "getUserOrgPermission">;
oidcConfigDAL: Pick<TOidcConfigDALFactory, "findOne" | "update" | "create">;
groupDAL: Pick<TGroupDALFactory, "findByOrgId">;
userGroupMembershipDAL: Pick<
TUserGroupMembershipDALFactory,
| "find"
| "transaction"
| "insertMany"
| "findGroupMembershipsByUserIdInOrg"
| "delete"
| "filterProjectsByUserMembership"
>;
groupProjectDAL: Pick<TGroupProjectDALFactory, "find">;
projectKeyDAL: Pick<TProjectKeyDALFactory, "find" | "findLatestProjectKey" | "insertMany" | "delete">;
projectDAL: Pick<TProjectDALFactory, "findProjectGhostUser">;
projectBotDAL: Pick<TProjectBotDALFactory, "findOne">;
auditLogService: Pick<TAuditLogServiceFactory, "createAuditLog">;
};
export type TOidcConfigServiceFactory = ReturnType<typeof oidcConfigServiceFactory>;
@@ -73,7 +105,14 @@ export const oidcConfigServiceFactory = ({
tokenService,
orgBotDAL,
smtpService,
oidcConfigDAL
oidcConfigDAL,
userGroupMembershipDAL,
groupDAL,
groupProjectDAL,
projectKeyDAL,
projectDAL,
projectBotDAL,
auditLogService
}: TOidcConfigServiceFactoryDep) => {
const getOidc = async (dto: TGetOidcCfgDTO) => {
const org = await orgDAL.findOne({ slug: dto.orgSlug });
@@ -156,11 +195,21 @@ export const oidcConfigServiceFactory = ({
isActive: oidcCfg.isActive,
allowedEmailDomains: oidcCfg.allowedEmailDomains,
clientId,
clientSecret
clientSecret,
manageGroupMemberships: oidcCfg.manageGroupMemberships
};
};
const oidcLogin = async ({ externalId, email, firstName, lastName, orgId, callbackPort }: TOidcLoginDTO) => {
const oidcLogin = async ({
externalId,
email,
firstName,
lastName,
orgId,
callbackPort,
groups = [],
manageGroupMemberships
}: TOidcLoginDTO) => {
const serverCfg = await getServerCfg();
if (serverCfg.enabledLoginMethods && !serverCfg.enabledLoginMethods.includes(LoginMethod.OIDC)) {
@@ -315,6 +364,83 @@ export const oidcConfigServiceFactory = ({
});
}
if (manageGroupMemberships) {
const userGroups = await userGroupMembershipDAL.findGroupMembershipsByUserIdInOrg(user.id, orgId);
const orgGroups = await groupDAL.findByOrgId(orgId);
const userGroupsNames = userGroups.map((membership) => membership.groupName);
const missingGroupsMemberships = groups.filter((groupName) => !userGroupsNames.includes(groupName));
const groupsToAddUserTo = orgGroups.filter((group) => missingGroupsMemberships.includes(group.name));
for await (const group of groupsToAddUserTo) {
await addUsersToGroupByUserIds({
userIds: [user.id],
group,
userDAL,
userGroupMembershipDAL,
orgDAL,
groupProjectDAL,
projectKeyDAL,
projectDAL,
projectBotDAL
});
}
if (groupsToAddUserTo.length) {
await auditLogService.createAuditLog({
actor: {
type: ActorType.PLATFORM,
metadata: {}
},
orgId,
event: {
type: EventType.OIDC_GROUP_MEMBERSHIP_MAPPING_ASSIGN_USER,
metadata: {
userId: user.id,
userEmail: user.email ?? user.username,
assignedToGroups: groupsToAddUserTo.map(({ id, name }) => ({ id, name })),
userGroupsClaim: groups
}
}
});
}
const membershipsToRemove = userGroups
.filter((membership) => !groups.includes(membership.groupName))
.map((membership) => membership.groupId);
const groupsToRemoveUserFrom = orgGroups.filter((group) => membershipsToRemove.includes(group.id));
for await (const group of groupsToRemoveUserFrom) {
await removeUsersFromGroupByUserIds({
userIds: [user.id],
group,
userDAL,
userGroupMembershipDAL,
groupProjectDAL,
projectKeyDAL
});
}
if (groupsToRemoveUserFrom.length) {
await auditLogService.createAuditLog({
actor: {
type: ActorType.PLATFORM,
metadata: {}
},
orgId,
event: {
type: EventType.OIDC_GROUP_MEMBERSHIP_MAPPING_REMOVE_USER,
metadata: {
userId: user.id,
userEmail: user.email ?? user.username,
removedFromGroups: groupsToRemoveUserFrom.map(({ id, name }) => ({ id, name })),
userGroupsClaim: groups
}
}
});
}
}
await licenseService.updateSubscriptionOrgMemberCount(organization.id);
const userEnc = await userDAL.findUserEncKeyByUserId(user.id);
@@ -385,7 +511,8 @@ export const oidcConfigServiceFactory = ({
tokenEndpoint,
userinfoEndpoint,
clientId,
clientSecret
clientSecret,
manageGroupMemberships
}: TUpdateOidcCfgDTO) => {
const org = await orgDAL.findOne({
slug: orgSlug
@@ -448,7 +575,8 @@ export const oidcConfigServiceFactory = ({
userinfoEndpoint,
jwksUri,
isActive,
lastUsed: null
lastUsed: null,
manageGroupMemberships
};
if (clientId !== undefined) {
@@ -491,7 +619,8 @@ export const oidcConfigServiceFactory = ({
tokenEndpoint,
userinfoEndpoint,
clientId,
clientSecret
clientSecret,
manageGroupMemberships
}: TCreateOidcCfgDTO) => {
const org = await orgDAL.findOne({
slug: orgSlug
@@ -589,7 +718,8 @@ export const oidcConfigServiceFactory = ({
clientIdTag,
encryptedClientSecret,
clientSecretIV,
clientSecretTag
clientSecretTag,
manageGroupMemberships
});
return oidcCfg;
@@ -683,7 +813,9 @@ export const oidcConfigServiceFactory = ({
firstName: claims.given_name ?? "",
lastName: claims.family_name ?? "",
orgId: org.id,
callbackPort
groups: claims.groups as string[] | undefined,
callbackPort,
manageGroupMemberships: oidcCfg.manageGroupMemberships
})
.then(({ isUserCompleted, providerAuthToken }) => {
cb(null, { isUserCompleted, providerAuthToken });
@@ -697,5 +829,16 @@ export const oidcConfigServiceFactory = ({
return strategy;
};
return { oidcLogin, getOrgAuthStrategy, getOidc, updateOidcCfg, createOidcCfg };
const isOidcManageGroupMembershipsEnabled = async (orgId: string, actor: OrgServiceActor) => {
await permissionService.getUserOrgPermission(actor.id, orgId, actor.authMethod, actor.orgId);
const oidcConfig = await oidcConfigDAL.findOne({
orgId,
isActive: true
});
return Boolean(oidcConfig?.manageGroupMemberships);
};
return { oidcLogin, getOrgAuthStrategy, getOidc, updateOidcCfg, createOidcCfg, isOidcManageGroupMembershipsEnabled };
};

View File

@@ -12,6 +12,8 @@ export type TOidcLoginDTO = {
lastName?: string;
orgId: string;
callbackPort?: string;
groups?: string[];
manageGroupMemberships?: boolean | null;
};
export type TGetOidcCfgDTO =
@@ -37,6 +39,7 @@ export type TCreateOidcCfgDTO = {
clientSecret: string;
isActive: boolean;
orgSlug: string;
manageGroupMemberships: boolean;
} & TGenericPermission;
export type TUpdateOidcCfgDTO = Partial<{
@@ -52,5 +55,6 @@ export type TUpdateOidcCfgDTO = Partial<{
clientSecret: string;
isActive: boolean;
orgSlug: string;
manageGroupMemberships: boolean;
}> &
TGenericPermission;

View File

@@ -163,6 +163,27 @@ export type ProjectPermissionSet =
| [ProjectPermissionActions.Create, ProjectPermissionSub.SecretRollback]
| [ProjectPermissionActions.Edit, ProjectPermissionSub.Kms];
const SECRET_PATH_MISSING_SLASH_ERR_MSG = "Invalid Secret Path; it must start with a '/'";
const SECRET_PATH_PERMISSION_OPERATOR_SCHEMA = z.union([
z.string().refine((val) => val.startsWith("/"), SECRET_PATH_MISSING_SLASH_ERR_MSG),
z
.object({
[PermissionConditionOperators.$EQ]: PermissionConditionSchema[PermissionConditionOperators.$EQ].refine(
(val) => val.startsWith("/"),
SECRET_PATH_MISSING_SLASH_ERR_MSG
),
[PermissionConditionOperators.$NEQ]: PermissionConditionSchema[PermissionConditionOperators.$NEQ].refine(
(val) => val.startsWith("/"),
SECRET_PATH_MISSING_SLASH_ERR_MSG
),
[PermissionConditionOperators.$IN]: PermissionConditionSchema[PermissionConditionOperators.$IN].refine(
(val) => val.every((el) => el.startsWith("/")),
SECRET_PATH_MISSING_SLASH_ERR_MSG
),
[PermissionConditionOperators.$GLOB]: PermissionConditionSchema[PermissionConditionOperators.$GLOB]
})
.partial()
]);
// akhilmhdh: don't modify this for v2
// if you want to update create a new schema
const SecretConditionV1Schema = z
@@ -177,17 +198,7 @@ const SecretConditionV1Schema = z
})
.partial()
]),
secretPath: z.union([
z.string(),
z
.object({
[PermissionConditionOperators.$EQ]: PermissionConditionSchema[PermissionConditionOperators.$EQ],
[PermissionConditionOperators.$NEQ]: PermissionConditionSchema[PermissionConditionOperators.$NEQ],
[PermissionConditionOperators.$IN]: PermissionConditionSchema[PermissionConditionOperators.$IN],
[PermissionConditionOperators.$GLOB]: PermissionConditionSchema[PermissionConditionOperators.$GLOB]
})
.partial()
])
secretPath: SECRET_PATH_PERMISSION_OPERATOR_SCHEMA
})
.partial();
@@ -204,17 +215,7 @@ const SecretConditionV2Schema = z
})
.partial()
]),
secretPath: z.union([
z.string(),
z
.object({
[PermissionConditionOperators.$EQ]: PermissionConditionSchema[PermissionConditionOperators.$EQ],
[PermissionConditionOperators.$NEQ]: PermissionConditionSchema[PermissionConditionOperators.$NEQ],
[PermissionConditionOperators.$IN]: PermissionConditionSchema[PermissionConditionOperators.$IN],
[PermissionConditionOperators.$GLOB]: PermissionConditionSchema[PermissionConditionOperators.$GLOB]
})
.partial()
]),
secretPath: SECRET_PATH_PERMISSION_OPERATOR_SCHEMA,
secretName: z.union([
z.string(),
z

View File

@@ -688,7 +688,9 @@ export const RAW_SECRETS = {
environment: "The slug of the environment to list secrets from.",
secretPath: "The secret path to list secrets from.",
includeImports: "Weather to include imported secrets or not.",
tagSlugs: "The comma separated tag slugs to filter secrets."
tagSlugs: "The comma separated tag slugs to filter secrets.",
metadataFilter:
"The secret metadata key-value pairs to filter secrets by. When querying for multiple metadata pairs, the query is treated as an AND operation. Secret metadata format is key=value1,value=value2|key=value3,value=value4."
},
CREATE: {
secretName: "The name of the secret to create.",
@@ -1591,6 +1593,13 @@ export const KMS = {
orderDirection: "The direction to order keys in.",
search: "The text string to filter key names by."
},
GET_KEY_BY_ID: {
keyId: "The ID of the KMS key to retrieve."
},
GET_KEY_BY_NAME: {
keyName: "The name of the KMS key to retrieve.",
projectId: "The ID of the project the key belongs to."
},
ENCRYPT: {
keyId: "The ID of the key to encrypt the data with.",
plaintext: "The plaintext to be encrypted (base64 encoded)."
@@ -1719,11 +1728,26 @@ export const SecretSyncs = {
REGION: "The AWS region to sync secrets to.",
PATH: "The Parameter Store path to sync secrets to."
},
AWS_SECRETS_MANAGER: {
REGION: "The AWS region to sync secrets to.",
MAPPING_BEHAVIOR:
"How secrets from Infisical should be mapped to AWS Secrets Manager; one-to-one or many-to-one.",
SECRET_NAME: "The secret name in AWS Secrets Manager to sync to when using mapping behavior many-to-one."
},
GITHUB: {
ORG: "The name of the GitHub organization.",
OWNER: "The name of the GitHub account owner of the repository.",
REPO: "The name of the GitHub repository.",
ENV: "The name of the GitHub environment."
},
AZURE_KEY_VAULT: {
VAULT_BASE_URL:
"The base URL of the Azure Key Vault to sync secrets to. Example: https://example.vault.azure.net/"
},
AZURE_APP_CONFIGURATION: {
CONFIGURATION_URL:
"The URL of the Azure App Configuration to sync secrets to. Example: https://example.azconfig.io/",
LABEL: "An optional label to assign to secrets created in Azure App Configuration."
}
}
};

View File

@@ -204,6 +204,10 @@ const envSchema = z
// gcp app
INF_APP_CONNECTION_GCP_SERVICE_ACCOUNT_CREDENTIAL: zpStr(z.string().optional()),
// azure app
INF_APP_CONNECTION_AZURE_CLIENT_ID: zpStr(z.string().optional()),
INF_APP_CONNECTION_AZURE_CLIENT_SECRET: zpStr(z.string().optional()),
/* CORS ----------------------------------------------------------------------------- */
CORS_ALLOWED_ORIGINS: zpStr(

View File

@@ -0,0 +1,4 @@
export enum DatabaseErrorCode {
ForeignKeyViolation = "23503",
UniqueViolation = "23505"
}

View File

@@ -0,0 +1 @@
export * from "./database";

View File

@@ -7,6 +7,7 @@ import { buildDynamicKnexQuery, TKnexDynamicOperator } from "./dynamic";
export * from "./connection";
export * from "./join";
export * from "./prependTableNameToFindFilter";
export * from "./select";
export const withTransaction = <K extends object>(db: Knex, dal: K) => ({

View File

@@ -0,0 +1,13 @@
import { TableName } from "@app/db/schemas";
import { buildFindFilter } from "@app/lib/knex/index";
type TFindFilterParameters = Parameters<typeof buildFindFilter<object>>[0];
export const prependTableNameToFindFilter = (tableName: TableName, filterObj: object): TFindFilterParameters =>
Object.fromEntries(
Object.entries(filterObj).map(([key, value]) =>
key.startsWith("$")
? [key, prependTableNameToFindFilter(tableName, value as object)]
: [`${tableName}.${key}`, value]
)
);

View File

@@ -467,7 +467,8 @@ export const registerRoutes = async (
projectBotDAL,
projectKeyDAL,
permissionService,
licenseService
licenseService,
oidcConfigDAL
});
const groupProjectService = groupProjectServiceFactory({
groupDAL,
@@ -848,7 +849,8 @@ export const registerRoutes = async (
secretVersionTagDAL,
secretVersionV2BridgeDAL,
secretVersionTagV2BridgeDAL,
resourceMetadataDAL
resourceMetadataDAL,
appConnectionDAL
});
const secretQueueService = secretQueueFactory({
@@ -1337,7 +1339,14 @@ export const registerRoutes = async (
smtpService,
orgBotDAL,
permissionService,
oidcConfigDAL
oidcConfigDAL,
projectBotDAL,
projectKeyDAL,
projectDAL,
userGroupMembershipDAL,
groupProjectDAL,
groupDAL,
auditLogService
});
const userEngagementService = userEngagementServiceFactory({

View File

@@ -110,7 +110,6 @@ export const secretRawSchema = z.object({
secretReminderNote: z.string().nullable().optional(),
secretReminderRepeatDays: z.number().nullable().optional(),
skipMultilineEncoding: z.boolean().default(false).nullable().optional(),
metadata: z.unknown().nullable().optional(),
createdAt: z.date(),
updatedAt: z.date()
});

View File

@@ -73,7 +73,13 @@ export const registerAppConnectionEndpoints = <T extends TAppConnection, I exten
description: `List the ${appName} Connections the current user has permission to establish connections with.`,
response: {
200: z.object({
appConnections: z.object({ app: z.literal(app), name: z.string(), id: z.string().uuid() }).array()
appConnections: z
.object({
app: z.literal(app),
name: z.string(),
id: z.string().uuid()
})
.array()
})
}
},

View File

@@ -4,6 +4,14 @@ import { EventType } from "@app/ee/services/audit-log/audit-log-types";
import { readLimit } from "@app/server/config/rateLimiter";
import { verifyAuth } from "@app/server/plugins/auth/verify-auth";
import { AwsConnectionListItemSchema, SanitizedAwsConnectionSchema } from "@app/services/app-connection/aws";
import {
AzureAppConfigurationConnectionListItemSchema,
SanitizedAzureAppConfigurationConnectionSchema
} from "@app/services/app-connection/azure-app-configuration";
import {
AzureKeyVaultConnectionListItemSchema,
SanitizedAzureKeyVaultConnectionSchema
} from "@app/services/app-connection/azure-key-vault";
import { GcpConnectionListItemSchema, SanitizedGcpConnectionSchema } from "@app/services/app-connection/gcp";
import { GitHubConnectionListItemSchema, SanitizedGitHubConnectionSchema } from "@app/services/app-connection/github";
import { AuthMode } from "@app/services/auth/auth-type";
@@ -12,13 +20,17 @@ import { AuthMode } from "@app/services/auth/auth-type";
const SanitizedAppConnectionSchema = z.union([
...SanitizedAwsConnectionSchema.options,
...SanitizedGitHubConnectionSchema.options,
...SanitizedGcpConnectionSchema.options
...SanitizedGcpConnectionSchema.options,
...SanitizedAzureKeyVaultConnectionSchema.options,
...SanitizedAzureAppConfigurationConnectionSchema.options
]);
const AppConnectionOptionsSchema = z.discriminatedUnion("app", [
AwsConnectionListItemSchema,
GitHubConnectionListItemSchema,
GcpConnectionListItemSchema
GcpConnectionListItemSchema,
AzureKeyVaultConnectionListItemSchema,
AzureAppConfigurationConnectionListItemSchema
]);
export const registerAppConnectionRouter = async (server: FastifyZodProvider) => {

View File

@@ -0,0 +1,18 @@
import { AppConnection } from "@app/services/app-connection/app-connection-enums";
import {
CreateAzureAppConfigurationConnectionSchema,
SanitizedAzureAppConfigurationConnectionSchema,
UpdateAzureAppConfigurationConnectionSchema
} from "@app/services/app-connection/azure-app-configuration";
import { registerAppConnectionEndpoints } from "./app-connection-endpoints";
export const registerAzureAppConfigurationConnectionRouter = async (server: FastifyZodProvider) => {
registerAppConnectionEndpoints({
app: AppConnection.AzureAppConfiguration,
server,
sanitizedResponseSchema: SanitizedAzureAppConfigurationConnectionSchema,
createSchema: CreateAzureAppConfigurationConnectionSchema,
updateSchema: UpdateAzureAppConfigurationConnectionSchema
});
};

View File

@@ -0,0 +1,18 @@
import { AppConnection } from "@app/services/app-connection/app-connection-enums";
import {
CreateAzureKeyVaultConnectionSchema,
SanitizedAzureKeyVaultConnectionSchema,
UpdateAzureKeyVaultConnectionSchema
} from "@app/services/app-connection/azure-key-vault";
import { registerAppConnectionEndpoints } from "./app-connection-endpoints";
export const registerAzureKeyVaultConnectionRouter = async (server: FastifyZodProvider) => {
registerAppConnectionEndpoints({
app: AppConnection.AzureKeyVault,
server,
sanitizedResponseSchema: SanitizedAzureKeyVaultConnectionSchema,
createSchema: CreateAzureKeyVaultConnectionSchema,
updateSchema: UpdateAzureKeyVaultConnectionSchema
});
};

View File

@@ -1,6 +1,8 @@
import { AppConnection } from "@app/services/app-connection/app-connection-enums";
import { registerAwsConnectionRouter } from "./aws-connection-router";
import { registerAzureAppConfigurationConnectionRouter } from "./azure-app-configuration-connection-router";
import { registerAzureKeyVaultConnectionRouter } from "./azure-key-vault-connection-router";
import { registerGcpConnectionRouter } from "./gcp-connection-router";
import { registerGitHubConnectionRouter } from "./github-connection-router";
@@ -10,5 +12,7 @@ export const APP_CONNECTION_REGISTER_ROUTER_MAP: Record<AppConnection, (server:
{
[AppConnection.AWS]: registerAwsConnectionRouter,
[AppConnection.GitHub]: registerGitHubConnectionRouter,
[AppConnection.GCP]: registerGcpConnectionRouter
[AppConnection.GCP]: registerGcpConnectionRouter,
[AppConnection.AzureKeyVault]: registerAzureKeyVaultConnectionRouter,
[AppConnection.AzureAppConfiguration]: registerAzureAppConfigurationConnectionRouter
};

View File

@@ -15,6 +15,10 @@ import { CmekOrderBy } from "@app/services/cmek/cmek-types";
const keyNameSchema = slugSchema({ min: 1, max: 32, field: "Name" });
const keyDescriptionSchema = z.string().trim().max(500).optional();
const CmekSchema = KmsKeysSchema.merge(InternalKmsSchema.pick({ version: true, encryptionAlgorithm: true })).omit({
isReserved: true
});
const base64Schema = z.string().superRefine((val, ctx) => {
if (!isBase64(val)) {
ctx.addIssue({
@@ -53,7 +57,7 @@ export const registerCmekRouter = async (server: FastifyZodProvider) => {
}),
response: {
200: z.object({
key: KmsKeysSchema
key: CmekSchema
})
}
},
@@ -106,7 +110,7 @@ export const registerCmekRouter = async (server: FastifyZodProvider) => {
}),
response: {
200: z.object({
key: KmsKeysSchema
key: CmekSchema
})
}
},
@@ -150,7 +154,7 @@ export const registerCmekRouter = async (server: FastifyZodProvider) => {
}),
response: {
200: z.object({
key: KmsKeysSchema
key: CmekSchema
})
}
},
@@ -201,7 +205,7 @@ export const registerCmekRouter = async (server: FastifyZodProvider) => {
}),
response: {
200: z.object({
keys: KmsKeysSchema.merge(InternalKmsSchema.pick({ version: true, encryptionAlgorithm: true })).array(),
keys: CmekSchema.array(),
totalCount: z.number()
})
}
@@ -230,6 +234,92 @@ export const registerCmekRouter = async (server: FastifyZodProvider) => {
}
});
server.route({
method: "GET",
url: "/keys/:keyId",
config: {
rateLimit: readLimit
},
schema: {
description: "Get KMS key by ID",
params: z.object({
keyId: z.string().uuid().describe(KMS.GET_KEY_BY_ID.keyId)
}),
response: {
200: z.object({
key: CmekSchema
})
}
},
onRequest: verifyAuth([AuthMode.JWT, AuthMode.IDENTITY_ACCESS_TOKEN]),
handler: async (req) => {
const {
params: { keyId },
permission
} = req;
const key = await server.services.cmek.findCmekById(keyId, permission);
await server.services.auditLog.createAuditLog({
...req.auditLogInfo,
projectId: key.projectId!,
event: {
type: EventType.GET_CMEK,
metadata: {
keyId: key.id
}
}
});
return { key };
}
});
server.route({
method: "GET",
url: "/keys/key-name/:keyName",
config: {
rateLimit: readLimit
},
schema: {
description: "Get KMS key by Name",
params: z.object({
keyName: slugSchema({ field: "Key name" }).describe(KMS.GET_KEY_BY_NAME.keyName)
}),
querystring: z.object({
projectId: z.string().min(1, "Project ID is required").describe(KMS.GET_KEY_BY_NAME.projectId)
}),
response: {
200: z.object({
key: CmekSchema
})
}
},
onRequest: verifyAuth([AuthMode.JWT, AuthMode.IDENTITY_ACCESS_TOKEN]),
handler: async (req) => {
const {
params: { keyName },
query: { projectId },
permission
} = req;
const key = await server.services.cmek.findCmekByName(keyName, projectId, permission);
await server.services.auditLog.createAuditLog({
...req.auditLogInfo,
projectId: key.projectId!,
event: {
type: EventType.GET_CMEK,
metadata: {
keyId: key.id
}
}
});
return { key };
}
});
// encrypt data
server.route({
method: "POST",

View File

@@ -0,0 +1,17 @@
import {
AwsSecretsManagerSyncSchema,
CreateAwsSecretsManagerSyncSchema,
UpdateAwsSecretsManagerSyncSchema
} from "@app/services/secret-sync/aws-secrets-manager";
import { SecretSync } from "@app/services/secret-sync/secret-sync-enums";
import { registerSyncSecretsEndpoints } from "./secret-sync-endpoints";
export const registerAwsSecretsManagerSyncRouter = async (server: FastifyZodProvider) =>
registerSyncSecretsEndpoints({
destination: SecretSync.AWSSecretsManager,
server,
responseSchema: AwsSecretsManagerSyncSchema,
createSchema: CreateAwsSecretsManagerSyncSchema,
updateSchema: UpdateAwsSecretsManagerSyncSchema
});

View File

@@ -0,0 +1,17 @@
import {
AzureAppConfigurationSyncSchema,
CreateAzureAppConfigurationSyncSchema,
UpdateAzureAppConfigurationSyncSchema
} from "@app/services/secret-sync/azure-app-configuration";
import { SecretSync } from "@app/services/secret-sync/secret-sync-enums";
import { registerSyncSecretsEndpoints } from "./secret-sync-endpoints";
export const registerAzureAppConfigurationSyncRouter = async (server: FastifyZodProvider) =>
registerSyncSecretsEndpoints({
destination: SecretSync.AzureAppConfiguration,
server,
responseSchema: AzureAppConfigurationSyncSchema,
createSchema: CreateAzureAppConfigurationSyncSchema,
updateSchema: UpdateAzureAppConfigurationSyncSchema
});

View File

@@ -0,0 +1,17 @@
import {
AzureKeyVaultSyncSchema,
CreateAzureKeyVaultSyncSchema,
UpdateAzureKeyVaultSyncSchema
} from "@app/services/secret-sync/azure-key-vault";
import { SecretSync } from "@app/services/secret-sync/secret-sync-enums";
import { registerSyncSecretsEndpoints } from "./secret-sync-endpoints";
export const registerAzureKeyVaultSyncRouter = async (server: FastifyZodProvider) =>
registerSyncSecretsEndpoints({
destination: SecretSync.AzureKeyVault,
server,
responseSchema: AzureKeyVaultSyncSchema,
createSchema: CreateAzureKeyVaultSyncSchema,
updateSchema: UpdateAzureKeyVaultSyncSchema
});

View File

@@ -1,6 +1,9 @@
import { SecretSync } from "@app/services/secret-sync/secret-sync-enums";
import { registerAwsParameterStoreSyncRouter } from "./aws-parameter-store-sync-router";
import { registerAwsSecretsManagerSyncRouter } from "./aws-secrets-manager-sync-router";
import { registerAzureAppConfigurationSyncRouter } from "./azure-app-configuration-sync-router";
import { registerAzureKeyVaultSyncRouter } from "./azure-key-vault-sync-router";
import { registerGcpSyncRouter } from "./gcp-sync-router";
import { registerGitHubSyncRouter } from "./github-sync-router";
@@ -8,6 +11,9 @@ export * from "./secret-sync-router";
export const SECRET_SYNC_REGISTER_ROUTER_MAP: Record<SecretSync, (server: FastifyZodProvider) => Promise<void>> = {
[SecretSync.AWSParameterStore]: registerAwsParameterStoreSyncRouter,
[SecretSync.AWSSecretsManager]: registerAwsSecretsManagerSyncRouter,
[SecretSync.GitHub]: registerGitHubSyncRouter,
[SecretSync.GCPSecretManager]: registerGcpSyncRouter
[SecretSync.GCPSecretManager]: registerGcpSyncRouter,
[SecretSync.AzureKeyVault]: registerAzureKeyVaultSyncRouter,
[SecretSync.AzureAppConfiguration]: registerAzureAppConfigurationSyncRouter
};

View File

@@ -9,19 +9,34 @@ import {
AwsParameterStoreSyncListItemSchema,
AwsParameterStoreSyncSchema
} from "@app/services/secret-sync/aws-parameter-store";
import {
AwsSecretsManagerSyncListItemSchema,
AwsSecretsManagerSyncSchema
} from "@app/services/secret-sync/aws-secrets-manager";
import {
AzureAppConfigurationSyncListItemSchema,
AzureAppConfigurationSyncSchema
} from "@app/services/secret-sync/azure-app-configuration";
import { AzureKeyVaultSyncListItemSchema, AzureKeyVaultSyncSchema } from "@app/services/secret-sync/azure-key-vault";
import { GcpSyncListItemSchema, GcpSyncSchema } from "@app/services/secret-sync/gcp";
import { GitHubSyncListItemSchema, GitHubSyncSchema } from "@app/services/secret-sync/github";
const SecretSyncSchema = z.discriminatedUnion("destination", [
AwsParameterStoreSyncSchema,
AwsSecretsManagerSyncSchema,
GitHubSyncSchema,
GcpSyncSchema
GcpSyncSchema,
AzureKeyVaultSyncSchema,
AzureAppConfigurationSyncSchema
]);
const SecretSyncOptionsSchema = z.discriminatedUnion("destination", [
AwsParameterStoreSyncListItemSchema,
AwsSecretsManagerSyncListItemSchema,
GitHubSyncListItemSchema,
GcpSyncListItemSchema
GcpSyncListItemSchema,
AzureKeyVaultSyncListItemSchema,
AzureAppConfigurationSyncListItemSchema
]);
export const registerSecretSyncRouter = async (server: FastifyZodProvider) => {

View File

@@ -181,6 +181,66 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
}
],
querystring: z.object({
metadataFilter: z
.string()
.optional()
.transform((val) => {
if (!val) return undefined;
const result: { key?: string; value?: string }[] = [];
const pairs = val.split("|");
for (const pair of pairs) {
const keyValuePair: { key?: string; value?: string } = {};
const parts = pair.split(/[,=]/);
for (let i = 0; i < parts.length; i += 2) {
const identifier = parts[i].trim().toLowerCase();
const value = parts[i + 1]?.trim();
if (identifier === "key" && value) {
keyValuePair.key = value;
} else if (identifier === "value" && value) {
keyValuePair.value = value;
}
}
if (keyValuePair.key && keyValuePair.value) {
result.push(keyValuePair);
}
}
return result.length ? result : undefined;
})
.superRefine((metadata, ctx) => {
if (metadata && !Array.isArray(metadata)) {
ctx.addIssue({
code: z.ZodIssueCode.custom,
message:
"Invalid secretMetadata format. Correct format is key=value1,value=value2|key=value3,value=value4."
});
}
if (metadata) {
if (metadata.length > 10) {
ctx.addIssue({
code: z.ZodIssueCode.custom,
message: "You can only filter by up to 10 metadata fields"
});
}
for (const item of metadata) {
if (!item.key && !item.value) {
ctx.addIssue({
code: z.ZodIssueCode.custom,
message:
"Invalid secretMetadata format, key or value must be provided. Correct format is key=value1,value=value2|key=value3,value=value4."
});
}
}
}
})
.describe(RAW_SECRETS.LIST.metadataFilter),
workspaceId: z.string().trim().optional().describe(RAW_SECRETS.LIST.workspaceId),
workspaceSlug: z.string().trim().optional().describe(RAW_SECRETS.LIST.workspaceSlug),
environment: z.string().trim().optional().describe(RAW_SECRETS.LIST.environment),
@@ -281,6 +341,7 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
actorAuthMethod: req.permission.authMethod,
projectId: workspaceId,
path: secretPath,
metadataFilter: req.query.metadataFilter,
includeImports: req.query.include_imports,
recursive: req.query.recursive,
tagSlugs: req.query.tagSlugs
@@ -411,7 +472,8 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
secretPath: req.query.secretPath,
secretId: secret.id,
secretKey: req.params.secretName,
secretVersion: secret.version
secretVersion: secret.version,
secretMetadata: secret.secretMetadata
}
}
});
@@ -519,7 +581,8 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
secretPath: req.body.secretPath,
secretId: secret.id,
secretKey: req.params.secretName,
secretVersion: secret.version
secretVersion: secret.version,
secretMetadata: req.body.secretMetadata
}
}
});
@@ -631,7 +694,8 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
secretPath: req.body.secretPath,
secretId: secret.id,
secretKey: req.params.secretName,
secretVersion: secret.version
secretVersion: secret.version,
secretMetadata: req.body.secretMetadata
}
}
});
@@ -1904,6 +1968,10 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
}
const { secrets } = secretOperation;
const secretMetadataMap = new Map(
inputSecrets.map(({ secretKey, secretMetadata }) => [secretKey, secretMetadata])
);
await server.services.auditLog.createAuditLog({
projectId: secrets[0].workspace,
...req.auditLogInfo,
@@ -1915,7 +1983,8 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
secrets: secrets.map((secret) => ({
secretId: secret.id,
secretKey: secret.secretKey,
secretVersion: secret.version
secretVersion: secret.version,
secretMetadata: secretMetadataMap.get(secret.secretKey)
}))
}
}
@@ -2010,6 +2079,10 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
}
const { secrets } = secretOperation;
const secretMetadataMap = new Map(
inputSecrets.map(({ secretKey, secretMetadata }) => [secretKey, secretMetadata])
);
await server.services.auditLog.createAuditLog({
projectId: secrets[0].workspace,
...req.auditLogInfo,
@@ -2021,7 +2094,8 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
secrets: secrets.map((secret) => ({
secretId: secret.id,
secretKey: secret.secretKey,
secretVersion: secret.version
secretVersion: secret.version,
secretMetadata: secretMetadataMap.get(secret.secretKey)
}))
}
}

View File

@@ -1,7 +1,9 @@
export enum AppConnection {
GitHub = "github",
AWS = "aws",
GCP = "gcp"
GCP = "gcp",
AzureKeyVault = "azure-key-vault",
AzureAppConfiguration = "azure-app-configuration"
}
export enum AWSRegion {

View File

@@ -20,10 +20,25 @@ import {
} from "@app/services/app-connection/github";
import { KmsDataKey } from "@app/services/kms/kms-types";
import {
AzureAppConfigurationConnectionMethod,
getAzureAppConfigurationConnectionListItem,
validateAzureAppConfigurationConnectionCredentials
} from "./azure-app-configuration";
import {
AzureKeyVaultConnectionMethod,
getAzureKeyVaultConnectionListItem,
validateAzureKeyVaultConnectionCredentials
} from "./azure-key-vault";
export const listAppConnectionOptions = () => {
return [getAwsAppConnectionListItem(), getGitHubConnectionListItem(), getGcpAppConnectionListItem()].sort((a, b) =>
a.name.localeCompare(b.name)
);
return [
getAwsAppConnectionListItem(),
getGitHubConnectionListItem(),
getGcpAppConnectionListItem(),
getAzureKeyVaultConnectionListItem(),
getAzureAppConfigurationConnectionListItem()
].sort((a, b) => a.name.localeCompare(b.name));
};
export const encryptAppConnectionCredentials = async ({
@@ -79,6 +94,10 @@ export const validateAppConnectionCredentials = async (
return validateGitHubConnectionCredentials(appConnection);
case AppConnection.GCP:
return validateGcpConnectionCredentials(appConnection);
case AppConnection.AzureKeyVault:
return validateAzureKeyVaultConnectionCredentials(appConnection);
case AppConnection.AzureAppConfiguration:
return validateAzureAppConfigurationConnectionCredentials(appConnection);
default:
// eslint-disable-next-line @typescript-eslint/restrict-template-expressions
throw new Error(`Unhandled App Connection ${app}`);
@@ -89,6 +108,8 @@ export const getAppConnectionMethodName = (method: TAppConnection["method"]) =>
switch (method) {
case GitHubConnectionMethod.App:
return "GitHub App";
case AzureKeyVaultConnectionMethod.OAuth:
case AzureAppConfigurationConnectionMethod.OAuth:
case GitHubConnectionMethod.OAuth:
return "OAuth";
case AwsConnectionMethod.AccessKey:

View File

@@ -3,5 +3,7 @@ import { AppConnection } from "./app-connection-enums";
export const APP_CONNECTION_NAME_MAP: Record<AppConnection, string> = {
[AppConnection.AWS]: "AWS",
[AppConnection.GitHub]: "GitHub",
[AppConnection.GCP]: "GCP"
[AppConnection.GCP]: "GCP",
[AppConnection.AzureKeyVault]: "Azure Key Vault",
[AppConnection.AzureAppConfiguration]: "Azure App Configuration"
};

View File

@@ -3,6 +3,7 @@ import { ForbiddenError, subject } from "@casl/ability";
import { OrgPermissionAppConnectionActions, OrgPermissionSubjects } from "@app/ee/services/permission/org-permission";
import { TPermissionServiceFactory } from "@app/ee/services/permission/permission-service";
import { generateHash } from "@app/lib/crypto/encryption";
import { DatabaseErrorCode } from "@app/lib/error-codes";
import { BadRequestError, DatabaseError, NotFoundError } from "@app/lib/errors";
import { DiscriminativePick, OrgServiceActor } from "@app/lib/types";
import { AppConnection } from "@app/services/app-connection/app-connection-enums";
@@ -27,6 +28,8 @@ import { githubConnectionService } from "@app/services/app-connection/github/git
import { TKmsServiceFactory } from "@app/services/kms/kms-service";
import { TAppConnectionDALFactory } from "./app-connection-dal";
import { ValidateAzureAppConfigurationConnectionCredentialsSchema } from "./azure-app-configuration";
import { ValidateAzureKeyVaultConnectionCredentialsSchema } from "./azure-key-vault";
import { ValidateGcpConnectionCredentialsSchema } from "./gcp";
import { gcpConnectionService } from "./gcp/gcp-connection-service";
@@ -41,7 +44,9 @@ export type TAppConnectionServiceFactory = ReturnType<typeof appConnectionServic
const VALIDATE_APP_CONNECTION_CREDENTIALS_MAP: Record<AppConnection, TValidateAppConnectionCredentials> = {
[AppConnection.AWS]: ValidateAwsConnectionCredentialsSchema,
[AppConnection.GitHub]: ValidateGitHubConnectionCredentialsSchema,
[AppConnection.GCP]: ValidateGcpConnectionCredentialsSchema
[AppConnection.GCP]: ValidateGcpConnectionCredentialsSchema,
[AppConnection.AzureKeyVault]: ValidateAzureKeyVaultConnectionCredentialsSchema,
[AppConnection.AzureAppConfiguration]: ValidateAzureAppConfigurationConnectionCredentialsSchema
};
export const appConnectionServiceFactory = ({
@@ -144,54 +149,40 @@ export const appConnectionServiceFactory = ({
OrgPermissionSubjects.AppConnections
);
const appConnection = await appConnectionDAL.transaction(async (tx) => {
const isConflictingName = Boolean(
await appConnectionDAL.findOne(
{
name: params.name,
orgId: actor.orgId
},
tx
)
);
const validatedCredentials = await validateAppConnectionCredentials({
app,
credentials,
method,
orgId: actor.orgId
} as TAppConnectionConfig);
if (isConflictingName)
throw new BadRequestError({
message: `An App Connection with the name "${params.name}" already exists`
});
const encryptedCredentials = await encryptAppConnectionCredentials({
credentials: validatedCredentials,
orgId: actor.orgId,
kmsService
});
const validatedCredentials = await validateAppConnectionCredentials({
app,
credentials,
method,
orgId: actor.orgId
} as TAppConnectionConfig);
const encryptedCredentials = await encryptAppConnectionCredentials({
credentials: validatedCredentials,
try {
const connection = await appConnectionDAL.create({
orgId: actor.orgId,
kmsService
encryptedCredentials,
method,
app,
...params
});
const connection = await appConnectionDAL.create(
{
orgId: actor.orgId,
encryptedCredentials,
method,
app,
...params
},
tx
);
return {
...connection,
credentialsHash: generateHash(connection.encryptedCredentials),
credentials: validatedCredentials
};
});
} as TAppConnection;
} catch (err) {
if (err instanceof DatabaseError && (err.error as { code: string })?.code === DatabaseErrorCode.UniqueViolation) {
throw new BadRequestError({ message: `An App Connection with the name "${params.name}" already exists` });
}
return appConnection as TAppConnection;
throw err;
}
};
const updateAppConnection = async (
@@ -215,72 +206,55 @@ export const appConnectionServiceFactory = ({
OrgPermissionSubjects.AppConnections
);
const updatedAppConnection = await appConnectionDAL.transaction(async (tx) => {
if (params.name && appConnection.name !== params.name) {
const isConflictingName = Boolean(
await appConnectionDAL.findOne(
{
name: params.name,
orgId: appConnection.orgId
},
tx
)
);
let encryptedCredentials: undefined | Buffer;
if (isConflictingName)
throw new BadRequestError({
message: `An App Connection with the name "${params.name}" already exists`
});
}
if (credentials) {
const { app, method } = appConnection as DiscriminativePick<TAppConnectionConfig, "app" | "method">;
let encryptedCredentials: undefined | Buffer;
if (credentials) {
const { app, method } = appConnection as DiscriminativePick<TAppConnectionConfig, "app" | "method">;
if (
!VALIDATE_APP_CONNECTION_CREDENTIALS_MAP[app].safeParse({
method,
credentials
}).success
)
throw new BadRequestError({
message: `Invalid credential format for ${
APP_CONNECTION_NAME_MAP[app]
} Connection with method ${getAppConnectionMethodName(method)}`
});
const validatedCredentials = await validateAppConnectionCredentials({
app,
orgId: actor.orgId,
credentials,
method
} as TAppConnectionConfig);
if (!validatedCredentials)
throw new BadRequestError({ message: "Unable to validate connection - check credentials" });
encryptedCredentials = await encryptAppConnectionCredentials({
credentials: validatedCredentials,
orgId: actor.orgId,
kmsService
if (
!VALIDATE_APP_CONNECTION_CREDENTIALS_MAP[app].safeParse({
method,
credentials
}).success
)
throw new BadRequestError({
message: `Invalid credential format for ${
APP_CONNECTION_NAME_MAP[app]
} Connection with method ${getAppConnectionMethodName(method)}`
});
const validatedCredentials = await validateAppConnectionCredentials({
app,
orgId: actor.orgId,
credentials,
method
} as TAppConnectionConfig);
if (!validatedCredentials)
throw new BadRequestError({ message: "Unable to validate connection - check credentials" });
encryptedCredentials = await encryptAppConnectionCredentials({
credentials: validatedCredentials,
orgId: actor.orgId,
kmsService
});
}
try {
const updatedConnection = await appConnectionDAL.updateById(connectionId, {
orgId: actor.orgId,
encryptedCredentials,
...params
});
return await decryptAppConnection(updatedConnection, kmsService);
} catch (err) {
if (err instanceof DatabaseError && (err.error as { code: string })?.code === DatabaseErrorCode.UniqueViolation) {
throw new BadRequestError({ message: `An App Connection with the name "${params.name}" already exists` });
}
const updatedConnection = await appConnectionDAL.updateById(
connectionId,
{
orgId: actor.orgId,
encryptedCredentials,
...params
},
tx
);
return updatedConnection;
});
return decryptAppConnection(updatedAppConnection, kmsService);
throw err;
}
};
const deleteAppConnection = async (app: AppConnection, connectionId: string, actor: OrgServiceActor) => {
@@ -311,7 +285,10 @@ export const appConnectionServiceFactory = ({
return await decryptAppConnection(deletedAppConnection, kmsService);
} catch (err) {
if (err instanceof DatabaseError && (err.error as { code: string })?.code === "23503") {
if (
err instanceof DatabaseError &&
(err.error as { code: string })?.code === DatabaseErrorCode.ForeignKeyViolation
) {
throw new BadRequestError({
message:
"Cannot delete App Connection with existing connections. Remove all existing connections and try again."

View File

@@ -11,11 +11,35 @@ import {
TValidateGitHubConnectionCredentials
} from "@app/services/app-connection/github";
import {
TAzureAppConfigurationConnection,
TAzureAppConfigurationConnectionConfig,
TAzureAppConfigurationConnectionInput,
TValidateAzureAppConfigurationConnectionCredentials
} from "./azure-app-configuration";
import {
TAzureKeyVaultConnection,
TAzureKeyVaultConnectionConfig,
TAzureKeyVaultConnectionInput,
TValidateAzureKeyVaultConnectionCredentials
} from "./azure-key-vault";
import { TGcpConnection, TGcpConnectionConfig, TGcpConnectionInput, TValidateGcpConnectionCredentials } from "./gcp";
export type TAppConnection = { id: string } & (TAwsConnection | TGitHubConnection | TGcpConnection);
export type TAppConnection = { id: string } & (
| TAwsConnection
| TGitHubConnection
| TGcpConnection
| TAzureKeyVaultConnection
| TAzureAppConfigurationConnection
);
export type TAppConnectionInput = { id: string } & (TAwsConnectionInput | TGitHubConnectionInput | TGcpConnectionInput);
export type TAppConnectionInput = { id: string } & (
| TAwsConnectionInput
| TGitHubConnectionInput
| TGcpConnectionInput
| TAzureKeyVaultConnectionInput
| TAzureAppConfigurationConnectionInput
);
export type TCreateAppConnectionDTO = Pick<
TAppConnectionInput,
@@ -26,9 +50,16 @@ export type TUpdateAppConnectionDTO = Partial<Omit<TCreateAppConnectionDTO, "met
connectionId: string;
};
export type TAppConnectionConfig = TAwsConnectionConfig | TGitHubConnectionConfig | TGcpConnectionConfig;
export type TAppConnectionConfig =
| TAwsConnectionConfig
| TGitHubConnectionConfig
| TGcpConnectionConfig
| TAzureKeyVaultConnectionConfig
| TAzureAppConfigurationConnectionConfig;
export type TValidateAppConnectionCredentials =
| TValidateAwsConnectionCredentials
| TValidateGitHubConnectionCredentials
| TValidateGcpConnectionCredentials;
| TValidateGcpConnectionCredentials
| TValidateAzureKeyVaultConnectionCredentials
| TValidateAzureAppConfigurationConnectionCredentials;

View File

@@ -0,0 +1,3 @@
export enum AzureAppConfigurationConnectionMethod {
OAuth = "oauth"
}

View File

@@ -0,0 +1,98 @@
import { AxiosError, AxiosResponse } from "axios";
import { getConfig } from "@app/lib/config/env";
import { request } from "@app/lib/config/request";
import { BadRequestError, InternalServerError } from "@app/lib/errors";
import { getAppConnectionMethodName } from "@app/services/app-connection/app-connection-fns";
import { IntegrationUrls } from "@app/services/integration-auth/integration-list";
import { AppConnection } from "../app-connection-enums";
import { AzureAppConfigurationConnectionMethod } from "./azure-app-configuration-connection-enums";
import {
ExchangeCodeAzureResponse,
TAzureAppConfigurationConnectionConfig
} from "./azure-app-configuration-connection-types";
export const getAzureAppConfigurationConnectionListItem = () => {
const { INF_APP_CONNECTION_AZURE_CLIENT_ID } = getConfig();
return {
name: "Azure App Configuration" as const,
app: AppConnection.AzureAppConfiguration as const,
methods: Object.values(AzureAppConfigurationConnectionMethod) as [AzureAppConfigurationConnectionMethod.OAuth],
oauthClientId: INF_APP_CONNECTION_AZURE_CLIENT_ID
};
};
export const validateAzureAppConfigurationConnectionCredentials = async (
config: TAzureAppConfigurationConnectionConfig
) => {
const { credentials: inputCredentials, method } = config;
const { INF_APP_CONNECTION_AZURE_CLIENT_ID, INF_APP_CONNECTION_AZURE_CLIENT_SECRET, SITE_URL } = getConfig();
if (!INF_APP_CONNECTION_AZURE_CLIENT_ID || !INF_APP_CONNECTION_AZURE_CLIENT_SECRET) {
throw new InternalServerError({
message: `Azure ${getAppConnectionMethodName(method)} environment variables have not been configured`
});
}
let tokenResp: AxiosResponse<ExchangeCodeAzureResponse> | null = null;
let tokenError: AxiosError | null = null;
try {
tokenResp = await request.post<ExchangeCodeAzureResponse>(
IntegrationUrls.AZURE_TOKEN_URL.replace("common", inputCredentials.tenantId || "common"),
new URLSearchParams({
grant_type: "authorization_code",
code: inputCredentials.code,
scope: `openid offline_access https://azconfig.io/.default`,
client_id: INF_APP_CONNECTION_AZURE_CLIENT_ID,
client_secret: INF_APP_CONNECTION_AZURE_CLIENT_SECRET,
redirect_uri: `${SITE_URL}/organization/app-connections/azure/oauth/callback`
})
);
} catch (e: unknown) {
if (e instanceof AxiosError) {
tokenError = e;
} else {
throw new BadRequestError({
message: `Unable to validate connection - verify credentials`
});
}
}
if (tokenError) {
if (tokenError instanceof AxiosError) {
throw new BadRequestError({
message: `Failed to get access token: ${
(tokenError?.response?.data as { error_description?: string })?.error_description || "Unknown error"
}`
});
} else {
throw new InternalServerError({
message: "Failed to get access token"
});
}
}
if (!tokenResp) {
throw new InternalServerError({
message: `Failed to get access token: Token was empty with no error`
});
}
switch (method) {
case AzureAppConfigurationConnectionMethod.OAuth:
return {
tenantId: inputCredentials.tenantId,
accessToken: tokenResp.data.access_token,
refreshToken: tokenResp.data.refresh_token,
expiresAt: Date.now() + tokenResp.data.expires_in * 1000
};
default:
throw new InternalServerError({
message: `Unhandled Azure connection method: ${method as AzureAppConfigurationConnectionMethod}`
});
}
};

View File

@@ -0,0 +1,76 @@
import { z } from "zod";
import { AppConnections } from "@app/lib/api-docs";
import { AppConnection } from "@app/services/app-connection/app-connection-enums";
import {
BaseAppConnectionSchema,
GenericCreateAppConnectionFieldsSchema,
GenericUpdateAppConnectionFieldsSchema
} from "@app/services/app-connection/app-connection-schemas";
import { AzureAppConfigurationConnectionMethod } from "./azure-app-configuration-connection-enums";
export const AzureAppConfigurationConnectionOAuthInputCredentialsSchema = z.object({
code: z.string().trim().min(1, "OAuth code required"),
tenantId: z.string().trim().optional()
});
export const AzureAppConfigurationConnectionOAuthOutputCredentialsSchema = z.object({
tenantId: z.string().optional(),
accessToken: z.string(),
refreshToken: z.string(),
expiresAt: z.number()
});
export const ValidateAzureAppConfigurationConnectionCredentialsSchema = z.discriminatedUnion("method", [
z.object({
method: z
.literal(AzureAppConfigurationConnectionMethod.OAuth)
.describe(AppConnections.CREATE(AppConnection.AzureAppConfiguration).method),
credentials: AzureAppConfigurationConnectionOAuthInputCredentialsSchema.describe(
AppConnections.CREATE(AppConnection.AzureAppConfiguration).credentials
)
})
]);
export const CreateAzureAppConfigurationConnectionSchema = ValidateAzureAppConfigurationConnectionCredentialsSchema.and(
GenericCreateAppConnectionFieldsSchema(AppConnection.AzureAppConfiguration)
);
export const UpdateAzureAppConfigurationConnectionSchema = z
.object({
credentials: AzureAppConfigurationConnectionOAuthInputCredentialsSchema.optional().describe(
AppConnections.UPDATE(AppConnection.AzureAppConfiguration).credentials
)
})
.and(GenericUpdateAppConnectionFieldsSchema(AppConnection.AzureAppConfiguration));
const BaseAzureAppConfigurationConnectionSchema = BaseAppConnectionSchema.extend({
app: z.literal(AppConnection.AzureAppConfiguration)
});
export const AzureAppConfigurationConnectionSchema = z.intersection(
BaseAzureAppConfigurationConnectionSchema,
z.discriminatedUnion("method", [
z.object({
method: z.literal(AzureAppConfigurationConnectionMethod.OAuth),
credentials: AzureAppConfigurationConnectionOAuthOutputCredentialsSchema
})
])
);
export const SanitizedAzureAppConfigurationConnectionSchema = z.discriminatedUnion("method", [
BaseAzureAppConfigurationConnectionSchema.extend({
method: z.literal(AzureAppConfigurationConnectionMethod.OAuth),
credentials: AzureAppConfigurationConnectionOAuthOutputCredentialsSchema.pick({
tenantId: true
})
})
]);
export const AzureAppConfigurationConnectionListItemSchema = z.object({
name: z.literal("Azure App Configuration"),
app: z.literal(AppConnection.AzureAppConfiguration),
methods: z.nativeEnum(AzureAppConfigurationConnectionMethod).array(),
oauthClientId: z.string().optional()
});

View File

@@ -0,0 +1,41 @@
import z from "zod";
import { DiscriminativePick } from "@app/lib/types";
import { AppConnection } from "../app-connection-enums";
import {
AzureAppConfigurationConnectionOAuthOutputCredentialsSchema,
AzureAppConfigurationConnectionSchema,
CreateAzureAppConfigurationConnectionSchema,
ValidateAzureAppConfigurationConnectionCredentialsSchema
} from "./azure-app-configuration-connection-schemas";
export type TAzureAppConfigurationConnection = z.infer<typeof AzureAppConfigurationConnectionSchema>;
export type TAzureAppConfigurationConnectionInput = z.infer<typeof CreateAzureAppConfigurationConnectionSchema> & {
app: AppConnection.AzureAppConfiguration;
};
export type TValidateAzureAppConfigurationConnectionCredentials =
typeof ValidateAzureAppConfigurationConnectionCredentialsSchema;
export type TAzureAppConfigurationConnectionConfig = DiscriminativePick<
TAzureAppConfigurationConnectionInput,
"method" | "app" | "credentials"
> & {
orgId: string;
};
export type ExchangeCodeAzureResponse = {
token_type: string;
scope: string;
expires_in: number;
ext_expires_in: number;
access_token: string;
refresh_token: string;
id_token: string;
};
export type TAzureAppConfigurationConnectionCredentials = z.infer<
typeof AzureAppConfigurationConnectionOAuthOutputCredentialsSchema
>;

View File

@@ -0,0 +1,4 @@
export * from "./azure-app-configuration-connection-enums";
export * from "./azure-app-configuration-connection-fns";
export * from "./azure-app-configuration-connection-schemas";
export * from "./azure-app-configuration-connection-types";

View File

@@ -0,0 +1,3 @@
export enum AzureKeyVaultConnectionMethod {
OAuth = "oauth"
}

View File

@@ -0,0 +1,170 @@
import { AxiosError, AxiosResponse } from "axios";
import { getConfig } from "@app/lib/config/env";
import { request } from "@app/lib/config/request";
import { BadRequestError, InternalServerError, NotFoundError } from "@app/lib/errors";
import {
decryptAppConnectionCredentials,
encryptAppConnectionCredentials,
getAppConnectionMethodName
} from "@app/services/app-connection/app-connection-fns";
import { IntegrationUrls } from "@app/services/integration-auth/integration-list";
import { TKmsServiceFactory } from "@app/services/kms/kms-service";
import { TAppConnectionDALFactory } from "../app-connection-dal";
import { AppConnection } from "../app-connection-enums";
import { AzureKeyVaultConnectionMethod } from "./azure-key-vault-connection-enums";
import {
ExchangeCodeAzureResponse,
TAzureKeyVaultConnectionConfig,
TAzureKeyVaultConnectionCredentials
} from "./azure-key-vault-connection-types";
export const getAzureConnectionAccessToken = async (
connectionId: string,
appConnectionDAL: Pick<TAppConnectionDALFactory, "findById" | "update">,
kmsService: Pick<TKmsServiceFactory, "createCipherPairWithDataKey">
) => {
const appCfg = getConfig();
if (!appCfg.INF_APP_CONNECTION_AZURE_CLIENT_ID || !appCfg.INF_APP_CONNECTION_AZURE_CLIENT_SECRET) {
throw new BadRequestError({
message: `Azure environment variables have not been configured`
});
}
const appConnection = await appConnectionDAL.findById(connectionId);
if (!appConnection) {
throw new NotFoundError({ message: `Connection with ID '${connectionId}' not found` });
}
if (appConnection.app !== AppConnection.AzureKeyVault && appConnection.app !== AppConnection.AzureAppConfiguration) {
throw new BadRequestError({ message: `Connection with ID '${connectionId}' is not an Azure Key Vault connection` });
}
const credentials = (await decryptAppConnectionCredentials({
orgId: appConnection.orgId,
kmsService,
encryptedCredentials: appConnection.encryptedCredentials
})) as TAzureKeyVaultConnectionCredentials;
const { data } = await request.post<ExchangeCodeAzureResponse>(
IntegrationUrls.AZURE_TOKEN_URL.replace("common", credentials.tenantId || "common"),
new URLSearchParams({
grant_type: "refresh_token",
scope: `openid offline_access`,
client_id: appCfg.INF_APP_CONNECTION_AZURE_CLIENT_ID,
client_secret: appCfg.INF_APP_CONNECTION_AZURE_CLIENT_SECRET,
refresh_token: credentials.refreshToken
})
);
const accessExpiresAt = new Date();
accessExpiresAt.setSeconds(accessExpiresAt.getSeconds() + data.expires_in);
const updatedCredentials = {
...credentials,
accessToken: data.access_token,
expiresAt: accessExpiresAt.getTime(),
refreshToken: data.refresh_token
};
const encryptedCredentials = await encryptAppConnectionCredentials({
credentials: updatedCredentials,
orgId: appConnection.orgId,
kmsService
});
await appConnectionDAL.update(
{ id: connectionId },
{
encryptedCredentials
}
);
return {
accessToken: data.access_token
};
};
export const getAzureKeyVaultConnectionListItem = () => {
const { INF_APP_CONNECTION_AZURE_CLIENT_ID } = getConfig();
return {
name: "Azure Key Vault" as const,
app: AppConnection.AzureKeyVault as const,
methods: Object.values(AzureKeyVaultConnectionMethod) as [AzureKeyVaultConnectionMethod.OAuth],
oauthClientId: INF_APP_CONNECTION_AZURE_CLIENT_ID
};
};
export const validateAzureKeyVaultConnectionCredentials = async (config: TAzureKeyVaultConnectionConfig) => {
const { credentials: inputCredentials, method } = config;
const { INF_APP_CONNECTION_AZURE_CLIENT_ID, INF_APP_CONNECTION_AZURE_CLIENT_SECRET, SITE_URL } = getConfig();
if (!INF_APP_CONNECTION_AZURE_CLIENT_ID || !INF_APP_CONNECTION_AZURE_CLIENT_SECRET) {
throw new InternalServerError({
message: `Azure ${getAppConnectionMethodName(method)} environment variables have not been configured`
});
}
let tokenResp: AxiosResponse<ExchangeCodeAzureResponse> | null = null;
let tokenError: AxiosError | null = null;
try {
tokenResp = await request.post<ExchangeCodeAzureResponse>(
IntegrationUrls.AZURE_TOKEN_URL.replace("common", inputCredentials.tenantId || "common"),
new URLSearchParams({
grant_type: "authorization_code",
code: inputCredentials.code,
scope: `openid offline_access https://vault.azure.net/.default`,
client_id: INF_APP_CONNECTION_AZURE_CLIENT_ID,
client_secret: INF_APP_CONNECTION_AZURE_CLIENT_SECRET,
redirect_uri: `${SITE_URL}/organization/app-connections/azure/oauth/callback`
})
);
} catch (e: unknown) {
if (e instanceof AxiosError) {
tokenError = e;
} else {
throw new BadRequestError({
message: `Unable to validate connection - verify credentials`
});
}
}
if (tokenError) {
if (tokenError instanceof AxiosError) {
throw new BadRequestError({
message: `Failed to get access token: ${
(tokenError?.response?.data as { error_description?: string })?.error_description || "Unknown error"
}`
});
} else {
throw new InternalServerError({
message: "Failed to get access token"
});
}
}
if (!tokenResp) {
throw new InternalServerError({
message: `Failed to get access token: Token was empty with no error`
});
}
switch (method) {
case AzureKeyVaultConnectionMethod.OAuth:
return {
tenantId: inputCredentials.tenantId,
accessToken: tokenResp.data.access_token,
refreshToken: tokenResp.data.refresh_token,
expiresAt: Date.now() + tokenResp.data.expires_in * 1000
};
default:
throw new InternalServerError({
message: `Unhandled Azure connection method: ${method as AzureKeyVaultConnectionMethod}`
});
}
};

View File

@@ -0,0 +1,76 @@
import { z } from "zod";
import { AppConnections } from "@app/lib/api-docs";
import { AppConnection } from "@app/services/app-connection/app-connection-enums";
import {
BaseAppConnectionSchema,
GenericCreateAppConnectionFieldsSchema,
GenericUpdateAppConnectionFieldsSchema
} from "@app/services/app-connection/app-connection-schemas";
import { AzureKeyVaultConnectionMethod } from "./azure-key-vault-connection-enums";
export const AzureKeyVaultConnectionOAuthInputCredentialsSchema = z.object({
code: z.string().trim().min(1, "OAuth code required"),
tenantId: z.string().trim().optional()
});
export const AzureKeyVaultConnectionOAuthOutputCredentialsSchema = z.object({
tenantId: z.string().optional(),
accessToken: z.string(),
refreshToken: z.string(),
expiresAt: z.number()
});
export const ValidateAzureKeyVaultConnectionCredentialsSchema = z.discriminatedUnion("method", [
z.object({
method: z
.literal(AzureKeyVaultConnectionMethod.OAuth)
.describe(AppConnections.CREATE(AppConnection.AzureKeyVault).method),
credentials: AzureKeyVaultConnectionOAuthInputCredentialsSchema.describe(
AppConnections.CREATE(AppConnection.AzureKeyVault).credentials
)
})
]);
export const CreateAzureKeyVaultConnectionSchema = ValidateAzureKeyVaultConnectionCredentialsSchema.and(
GenericCreateAppConnectionFieldsSchema(AppConnection.AzureKeyVault)
);
export const UpdateAzureKeyVaultConnectionSchema = z
.object({
credentials: AzureKeyVaultConnectionOAuthInputCredentialsSchema.optional().describe(
AppConnections.UPDATE(AppConnection.AzureKeyVault).credentials
)
})
.and(GenericUpdateAppConnectionFieldsSchema(AppConnection.AzureKeyVault));
const BaseAzureKeyVaultConnectionSchema = BaseAppConnectionSchema.extend({
app: z.literal(AppConnection.AzureKeyVault)
});
export const AzureKeyVaultConnectionSchema = z.intersection(
BaseAzureKeyVaultConnectionSchema,
z.discriminatedUnion("method", [
z.object({
method: z.literal(AzureKeyVaultConnectionMethod.OAuth),
credentials: AzureKeyVaultConnectionOAuthOutputCredentialsSchema
})
])
);
export const SanitizedAzureKeyVaultConnectionSchema = z.discriminatedUnion("method", [
BaseAzureKeyVaultConnectionSchema.extend({
method: z.literal(AzureKeyVaultConnectionMethod.OAuth),
credentials: AzureKeyVaultConnectionOAuthOutputCredentialsSchema.pick({
tenantId: true
})
})
]);
export const AzureKeyVaultConnectionListItemSchema = z.object({
name: z.literal("Azure Key Vault"),
app: z.literal(AppConnection.AzureKeyVault),
methods: z.nativeEnum(AzureKeyVaultConnectionMethod).array(),
oauthClientId: z.string().optional()
});

View File

@@ -0,0 +1,38 @@
import z from "zod";
import { DiscriminativePick } from "@app/lib/types";
import { AppConnection } from "../app-connection-enums";
import {
AzureKeyVaultConnectionOAuthOutputCredentialsSchema,
AzureKeyVaultConnectionSchema,
CreateAzureKeyVaultConnectionSchema,
ValidateAzureKeyVaultConnectionCredentialsSchema
} from "./azure-key-vault-connection-schemas";
export type TAzureKeyVaultConnection = z.infer<typeof AzureKeyVaultConnectionSchema>;
export type TAzureKeyVaultConnectionInput = z.infer<typeof CreateAzureKeyVaultConnectionSchema> & {
app: AppConnection.AzureKeyVault;
};
export type TValidateAzureKeyVaultConnectionCredentials = typeof ValidateAzureKeyVaultConnectionCredentialsSchema;
export type TAzureKeyVaultConnectionConfig = DiscriminativePick<
TAzureKeyVaultConnectionInput,
"method" | "app" | "credentials"
> & {
orgId: string;
};
export type ExchangeCodeAzureResponse = {
token_type: string;
scope: string;
expires_in: number;
ext_expires_in: number;
access_token: string;
refresh_token: string;
id_token: string;
};
export type TAzureKeyVaultConnectionCredentials = z.infer<typeof AzureKeyVaultConnectionOAuthOutputCredentialsSchema>;

View File

@@ -0,0 +1,4 @@
export * from "./azure-key-vault-connection-enums";
export * from "./azure-key-vault-connection-fns";
export * from "./azure-key-vault-connection-schemas";
export * from "./azure-key-vault-connection-types";

View File

@@ -153,7 +153,7 @@ export const validateGcpConnectionCredentials = async (appConnection: TGcpConnec
const serviceAccountId = appConnection.credentials.serviceAccountEmail.split("@")[0];
if (!serviceAccountId.endsWith(expectedAccountIdSuffix)) {
throw new BadRequestError({
message: `GCP service account ID (the part of the email before '@') must have a suffix of "${expectedAccountIdSuffix}"`
message: `GCP service account ID must have a suffix of "${expectedAccountIdSuffix}" e.g. service-account-${expectedAccountIdSuffix}@my-project.iam.gserviceaccount.com"`
});
}
}

View File

@@ -3,7 +3,8 @@ import { ForbiddenError } from "@casl/ability";
import { ActionProjectType, ProjectType } from "@app/db/schemas";
import { TPermissionServiceFactory } from "@app/ee/services/permission/permission-service";
import { ProjectPermissionCmekActions, ProjectPermissionSub } from "@app/ee/services/permission/project-permission";
import { BadRequestError, NotFoundError } from "@app/lib/errors";
import { DatabaseErrorCode } from "@app/lib/error-codes";
import { BadRequestError, DatabaseError, NotFoundError } from "@app/lib/errors";
import { OrgServiceActor } from "@app/lib/types";
import {
TCmekDecryptDTO,
@@ -44,17 +45,31 @@ export const cmekServiceFactory = ({ kmsService, kmsDAL, permissionService, proj
});
ForbiddenError.from(permission).throwUnlessCan(ProjectPermissionCmekActions.Create, ProjectPermissionSub.Cmek);
const cmek = await kmsService.generateKmsKey({
...dto,
projectId,
isReserved: false
});
try {
const cmek = await kmsService.generateKmsKey({
...dto,
projectId,
isReserved: false
});
return cmek;
return {
...cmek,
version: 1,
encryptionAlgorithm: dto.encryptionAlgorithm
};
} catch (err) {
if (err instanceof DatabaseError && (err.error as { code: string })?.code === DatabaseErrorCode.UniqueViolation) {
throw new BadRequestError({
message: `A KMS key with the name "${dto.name}" already exists for the project with ID "${projectId}"`
});
}
throw err;
}
};
const updateCmekById = async ({ keyId, ...data }: TUpdabteCmekByIdDTO, actor: OrgServiceActor) => {
const key = await kmsDAL.findById(keyId);
const key = await kmsDAL.findCmekById(keyId);
if (!key) throw new NotFoundError({ message: `Key with ID ${keyId} not found` });
@@ -71,13 +86,27 @@ export const cmekServiceFactory = ({ kmsService, kmsDAL, permissionService, proj
ForbiddenError.from(permission).throwUnlessCan(ProjectPermissionCmekActions.Edit, ProjectPermissionSub.Cmek);
const cmek = await kmsDAL.updateById(keyId, data);
try {
const cmek = await kmsDAL.updateById(keyId, data);
return cmek;
return {
...cmek,
version: key.version,
encryptionAlgorithm: key.encryptionAlgorithm
};
} catch (err) {
if (err instanceof DatabaseError && (err.error as { code: string })?.code === DatabaseErrorCode.UniqueViolation) {
throw new BadRequestError({
message: `A KMS key with the name "${data.name!}" already exists for the project with ID "${key.projectId}"`
});
}
throw err;
}
};
const deleteCmekById = async (keyId: string, actor: OrgServiceActor) => {
const key = await kmsDAL.findById(keyId);
const key = await kmsDAL.findCmekById(keyId);
if (!key) throw new NotFoundError({ message: `Key with ID ${keyId} not found` });
@@ -94,9 +123,9 @@ export const cmekServiceFactory = ({ kmsService, kmsDAL, permissionService, proj
ForbiddenError.from(permission).throwUnlessCan(ProjectPermissionCmekActions.Delete, ProjectPermissionSub.Cmek);
const cmek = kmsDAL.deleteById(keyId);
await kmsDAL.deleteById(keyId);
return cmek;
return key;
};
const listCmeksByProjectId = async (
@@ -120,15 +149,58 @@ export const cmekServiceFactory = ({ kmsService, kmsDAL, permissionService, proj
ForbiddenError.from(permission).throwUnlessCan(ProjectPermissionCmekActions.Read, ProjectPermissionSub.Cmek);
const { keys: cmeks, totalCount } = await kmsDAL.findKmsKeysByProjectId({ projectId, ...filters });
const { keys: cmeks, totalCount } = await kmsDAL.listCmeksByProjectId({ projectId, ...filters });
return { cmeks, totalCount };
};
const findCmekById = async (keyId: string, actor: OrgServiceActor) => {
const key = await kmsDAL.findCmekById(keyId);
if (!key) throw new NotFoundError({ message: `Key with ID "${keyId}" not found` });
if (!key.projectId || key.isReserved) throw new BadRequestError({ message: "Key is not customer managed" });
const { permission } = await permissionService.getProjectPermission({
actor: actor.type,
actorId: actor.id,
projectId: key.projectId,
actorAuthMethod: actor.authMethod,
actorOrgId: actor.orgId,
actionProjectType: ActionProjectType.KMS
});
ForbiddenError.from(permission).throwUnlessCan(ProjectPermissionCmekActions.Read, ProjectPermissionSub.Cmek);
return key;
};
const findCmekByName = async (keyName: string, projectId: string, actor: OrgServiceActor) => {
const key = await kmsDAL.findCmekByName(keyName, projectId);
if (!key)
throw new NotFoundError({ message: `Key with name "${keyName}" not found for project with ID "${projectId}"` });
if (!key.projectId || key.isReserved) throw new BadRequestError({ message: "Key is not customer managed" });
const { permission } = await permissionService.getProjectPermission({
actor: actor.type,
actorId: actor.id,
projectId: key.projectId,
actorAuthMethod: actor.authMethod,
actorOrgId: actor.orgId,
actionProjectType: ActionProjectType.KMS
});
ForbiddenError.from(permission).throwUnlessCan(ProjectPermissionCmekActions.Read, ProjectPermissionSub.Cmek);
return key;
};
const cmekEncrypt = async ({ keyId, plaintext }: TCmekEncryptDTO, actor: OrgServiceActor) => {
const key = await kmsDAL.findById(keyId);
if (!key) throw new NotFoundError({ message: `Key with ID ${keyId} not found` });
if (!key) throw new NotFoundError({ message: `Key with ID "${keyId}" not found` });
if (!key.projectId || key.isReserved) throw new BadRequestError({ message: "Key is not customer managed" });
@@ -155,7 +227,7 @@ export const cmekServiceFactory = ({ kmsService, kmsDAL, permissionService, proj
const cmekDecrypt = async ({ keyId, ciphertext }: TCmekDecryptDTO, actor: OrgServiceActor) => {
const key = await kmsDAL.findById(keyId);
if (!key) throw new NotFoundError({ message: `Key with ID ${keyId} not found` });
if (!key) throw new NotFoundError({ message: `Key with ID "${keyId}" not found` });
if (!key.projectId || key.isReserved) throw new BadRequestError({ message: "Key is not customer managed" });
@@ -185,6 +257,8 @@ export const cmekServiceFactory = ({ kmsService, kmsDAL, permissionService, proj
deleteCmekById,
listCmeksByProjectId,
cmekEncrypt,
cmekDecrypt
cmekDecrypt,
findCmekById,
findCmekByName
};
};

View File

@@ -1,7 +1,7 @@
import { z } from "zod";
const twelveDigitRegex = /^\d{12}$/;
const arnRegex = /^arn:aws:iam::\d{12}:(user\/[\w-]+|role\/[\w-]+|\*)$/;
const arnRegex = /^arn:aws:iam::\d{12}:(user\/[\w+=,.@/-]+|role\/[\w+=,.@/-]+|\*)$/;
export const validateAccountIds = z
.string()

View File

@@ -3,12 +3,32 @@ import { Knex } from "knex";
import { TDbClient } from "@app/db";
import { KmsKeysSchema, TableName, TInternalKms, TKmsKeys } from "@app/db/schemas";
import { DatabaseError } from "@app/lib/errors";
import { ormify, selectAllTableCols } from "@app/lib/knex";
import { buildFindFilter, ormify, prependTableNameToFindFilter, selectAllTableCols } from "@app/lib/knex";
import { OrderByDirection } from "@app/lib/types";
import { CmekOrderBy, TListCmeksByProjectIdDTO } from "@app/services/cmek/cmek-types";
export type TKmsKeyDALFactory = ReturnType<typeof kmskeyDALFactory>;
type TCmekFindFilter = Parameters<typeof buildFindFilter<TKmsKeys>>[0];
const baseCmekQuery = ({ filter, db, tx }: { db: TDbClient; filter?: TCmekFindFilter; tx?: Knex }) => {
const query = (tx || db.replicaNode())(TableName.KmsKey)
.where(`${TableName.KmsKey}.isReserved`, false)
.join(TableName.InternalKms, `${TableName.InternalKms}.kmsKeyId`, `${TableName.KmsKey}.id`)
.select(
selectAllTableCols(TableName.KmsKey),
db.ref("encryptionAlgorithm").withSchema(TableName.InternalKms),
db.ref("version").withSchema(TableName.InternalKms)
);
if (filter) {
/* eslint-disable @typescript-eslint/no-misused-promises */
void query.where(buildFindFilter(prependTableNameToFindFilter(TableName.KmsKey, filter)));
}
return query;
};
export const kmskeyDALFactory = (db: TDbClient) => {
const kmsOrm = ormify(db, TableName.KmsKey);
@@ -73,7 +93,7 @@ export const kmskeyDALFactory = (db: TDbClient) => {
}
};
const findKmsKeysByProjectId = async (
const listCmeksByProjectId = async (
{
projectId,
offset = 0,
@@ -92,6 +112,7 @@ export const kmskeyDALFactory = (db: TDbClient) => {
void qb.whereILike("name", `%${search}%`);
}
})
.where(`${TableName.KmsKey}.isReserved`, false)
.join(TableName.InternalKms, `${TableName.InternalKms}.kmsKeyId`, `${TableName.KmsKey}.id`)
.select<
(TKmsKeys &
@@ -118,5 +139,33 @@ export const kmskeyDALFactory = (db: TDbClient) => {
}
};
return { ...kmsOrm, findByIdWithAssociatedKms, findKmsKeysByProjectId };
const findCmekById = async (id: string, tx?: Knex) => {
try {
const key = await baseCmekQuery({
filter: { id },
db,
tx
}).first();
return key;
} catch (error) {
throw new DatabaseError({ error, name: "Find by ID - KMS Key" });
}
};
const findCmekByName = async (keyName: string, projectId: string, tx?: Knex) => {
try {
const key = await baseCmekQuery({
filter: { name: keyName, projectId },
db,
tx
}).first();
return key;
} catch (error) {
throw new DatabaseError({ error, name: "Find by Name - KMS Key" });
}
};
return { ...kmsOrm, findByIdWithAssociatedKms, listCmeksByProjectId, findCmekById, findCmekByName };
};

View File

@@ -493,6 +493,7 @@ export const secretFolderDALFactory = (db: TDbClient) => {
db.ref("parents.environment")
)
.from(TableName.SecretFolder)
.where(`${TableName.SecretFolder}.isReserved`, false)
.join("parents", `${TableName.SecretFolder}.parentId`, "parents.id");
})
)

View File

@@ -69,6 +69,8 @@ const getParametersByPath = async (ssm: AWS.SSM, path: string): Promise<TAWSPara
attempt += 1;
// eslint-disable-next-line no-await-in-loop
await sleep();
// eslint-disable-next-line no-continue
continue;
}
throw e;

View File

@@ -0,0 +1,10 @@
import { AppConnection } from "@app/services/app-connection/app-connection-enums";
import { SecretSync } from "@app/services/secret-sync/secret-sync-enums";
import { TSecretSyncListItem } from "@app/services/secret-sync/secret-sync-types";
export const AWS_SECRETS_MANAGER_SYNC_LIST_OPTION: TSecretSyncListItem = {
name: "AWS Secrets Manager",
destination: SecretSync.AWSSecretsManager,
connection: AppConnection.AWS,
canImportSecrets: true
};

View File

@@ -0,0 +1,4 @@
export enum AwsSecretsManagerSyncMappingBehavior {
OneToOne = "one-to-one",
ManyToOne = "many-to-one"
}

View File

@@ -0,0 +1,352 @@
import {
BatchGetSecretValueCommand,
CreateSecretCommand,
CreateSecretCommandInput,
DeleteSecretCommand,
DeleteSecretResponse,
ListSecretsCommand,
SecretsManagerClient,
UpdateSecretCommand,
UpdateSecretCommandInput
} from "@aws-sdk/client-secrets-manager";
import { AWSError } from "aws-sdk";
import { CreateSecretResponse, SecretListEntry, SecretValueEntry } from "aws-sdk/clients/secretsmanager";
import { getAwsConnectionConfig } from "@app/services/app-connection/aws/aws-connection-fns";
import { AwsSecretsManagerSyncMappingBehavior } from "@app/services/secret-sync/aws-secrets-manager/aws-secrets-manager-sync-enums";
import { SecretSyncError } from "@app/services/secret-sync/secret-sync-errors";
import { TSecretMap } from "@app/services/secret-sync/secret-sync-types";
import { TAwsSecretsManagerSyncWithCredentials } from "./aws-secrets-manager-sync-types";
type TAwsSecretsRecord = Record<string, SecretListEntry>;
type TAwsSecretValuesRecord = Record<string, SecretValueEntry>;
const MAX_RETRIES = 5;
const BATCH_SIZE = 20;
const getSecretsManagerClient = async (secretSync: TAwsSecretsManagerSyncWithCredentials) => {
const { destinationConfig, connection } = secretSync;
const config = await getAwsConnectionConfig(connection, destinationConfig.region);
const secretsManagerClient = new SecretsManagerClient({
region: config.region,
credentials: config.credentials!
});
return secretsManagerClient;
};
const sleep = async () =>
new Promise((resolve) => {
setTimeout(resolve, 1000);
});
const getSecretsRecord = async (client: SecretsManagerClient): Promise<TAwsSecretsRecord> => {
const awsSecretsRecord: TAwsSecretsRecord = {};
let hasNext = true;
let nextToken: string | undefined;
let attempt = 0;
while (hasNext) {
try {
// eslint-disable-next-line no-await-in-loop
const output = await client.send(new ListSecretsCommand({ NextToken: nextToken }));
attempt = 0;
if (output.SecretList) {
output.SecretList.forEach((secretEntry) => {
if (secretEntry.Name) {
awsSecretsRecord[secretEntry.Name] = secretEntry;
}
});
}
hasNext = Boolean(output.NextToken);
nextToken = output.NextToken;
} catch (e) {
if ((e as AWSError).code === "ThrottlingException" && attempt < MAX_RETRIES) {
attempt += 1;
// eslint-disable-next-line no-await-in-loop
await sleep();
// eslint-disable-next-line no-continue
continue;
}
throw e;
}
}
return awsSecretsRecord;
};
const getSecretValuesRecord = async (
client: SecretsManagerClient,
awsSecretsRecord: TAwsSecretsRecord
): Promise<TAwsSecretValuesRecord> => {
const awsSecretValuesRecord: TAwsSecretValuesRecord = {};
let attempt = 0;
const secretIdList = Object.keys(awsSecretsRecord);
for (let i = 0; i < secretIdList.length; i += BATCH_SIZE) {
const batchSecretIds = secretIdList.slice(i, i + BATCH_SIZE);
let hasNext = true;
let nextToken: string | undefined;
while (hasNext) {
try {
// eslint-disable-next-line no-await-in-loop
const output = await client.send(
new BatchGetSecretValueCommand({
SecretIdList: batchSecretIds,
NextToken: nextToken
})
);
attempt = 0;
if (output.SecretValues) {
output.SecretValues.forEach((secretValueEntry) => {
if (secretValueEntry.Name) {
awsSecretValuesRecord[secretValueEntry.Name] = secretValueEntry;
}
});
}
hasNext = Boolean(output.NextToken);
nextToken = output.NextToken;
} catch (e) {
if ((e as AWSError).code === "ThrottlingException" && attempt < MAX_RETRIES) {
attempt += 1;
// eslint-disable-next-line no-await-in-loop
await sleep();
// eslint-disable-next-line no-continue
continue;
}
throw e;
}
}
}
return awsSecretValuesRecord;
};
const createSecret = async (
client: SecretsManagerClient,
input: CreateSecretCommandInput,
attempt = 0
): Promise<CreateSecretResponse> => {
try {
return await client.send(new CreateSecretCommand(input));
} catch (error) {
if ((error as AWSError).code === "ThrottlingException" && attempt < MAX_RETRIES) {
await sleep();
// retry
return createSecret(client, input, attempt + 1);
}
throw error;
}
};
const updateSecret = async (
client: SecretsManagerClient,
input: UpdateSecretCommandInput,
attempt = 0
): Promise<CreateSecretResponse> => {
try {
return await client.send(new UpdateSecretCommand(input));
} catch (error) {
if ((error as AWSError).code === "ThrottlingException" && attempt < MAX_RETRIES) {
await sleep();
// retry
return updateSecret(client, input, attempt + 1);
}
throw error;
}
};
const deleteSecret = async (
client: SecretsManagerClient,
secretKey: string,
attempt = 0
): Promise<DeleteSecretResponse> => {
try {
return await client.send(new DeleteSecretCommand({ SecretId: secretKey, ForceDeleteWithoutRecovery: true }));
} catch (error) {
if ((error as AWSError).code === "ThrottlingException" && attempt < MAX_RETRIES) {
await sleep();
// retry
return deleteSecret(client, secretKey, attempt + 1);
}
throw error;
}
};
export const AwsSecretsManagerSyncFns = {
syncSecrets: async (secretSync: TAwsSecretsManagerSyncWithCredentials, secretMap: TSecretMap) => {
const { destinationConfig } = secretSync;
const client = await getSecretsManagerClient(secretSync);
const awsSecretsRecord = await getSecretsRecord(client);
const awsValuesRecord = await getSecretValuesRecord(client, awsSecretsRecord);
if (destinationConfig.mappingBehavior === AwsSecretsManagerSyncMappingBehavior.OneToOne) {
for await (const entry of Object.entries(secretMap)) {
const [key, { value }] = entry;
// skip secrets that don't have a value set
if (!value) {
// eslint-disable-next-line no-continue
continue;
}
if (awsSecretsRecord[key]) {
// skip secrets that haven't changed
if (awsValuesRecord[key]?.SecretString === value) {
// eslint-disable-next-line no-continue
continue;
}
try {
await updateSecret(client, {
SecretId: key,
SecretString: value
});
} catch (error) {
throw new SecretSyncError({
error,
secretKey: key
});
}
} else {
try {
await createSecret(client, {
Name: key,
SecretString: value
});
} catch (error) {
throw new SecretSyncError({
error,
secretKey: key
});
}
}
}
for await (const secretKey of Object.keys(awsSecretsRecord)) {
if (!(secretKey in secretMap) || !secretMap[secretKey].value) {
try {
await deleteSecret(client, secretKey);
} catch (error) {
throw new SecretSyncError({
error,
secretKey
});
}
}
}
} else {
// Many-To-One Mapping
const secretValue = JSON.stringify(
Object.fromEntries(Object.entries(secretMap).map(([key, secretData]) => [key, secretData.value]))
);
if (awsValuesRecord[destinationConfig.secretName]) {
await updateSecret(client, {
SecretId: destinationConfig.secretName,
SecretString: secretValue
});
} else {
await createSecret(client, {
Name: destinationConfig.secretName,
SecretString: secretValue
});
}
for await (const secretKey of Object.keys(awsSecretsRecord)) {
if (secretKey === destinationConfig.secretName) {
// eslint-disable-next-line no-continue
continue;
}
try {
await deleteSecret(client, secretKey);
} catch (error) {
throw new SecretSyncError({
error,
secretKey
});
}
}
}
},
getSecrets: async (secretSync: TAwsSecretsManagerSyncWithCredentials): Promise<TSecretMap> => {
const client = await getSecretsManagerClient(secretSync);
const awsSecretsRecord = await getSecretsRecord(client);
const awsValuesRecord = await getSecretValuesRecord(client, awsSecretsRecord);
const { destinationConfig } = secretSync;
if (destinationConfig.mappingBehavior === AwsSecretsManagerSyncMappingBehavior.OneToOne) {
return Object.fromEntries(
Object.keys(awsSecretsRecord).map((key) => [key, { value: awsValuesRecord[key].SecretString ?? "" }])
);
}
// Many-To-One Mapping
const secretValueEntry = awsValuesRecord[destinationConfig.secretName];
if (!secretValueEntry) return {};
try {
const parsedValue = (secretValueEntry.SecretString ? JSON.parse(secretValueEntry.SecretString) : {}) as Record<
string,
string
>;
return Object.fromEntries(Object.entries(parsedValue).map(([key, value]) => [key, { value }]));
} catch {
throw new SecretSyncError({
message:
"Failed to import secrets. Invalid format for Many-To-One mapping behavior: requires key/value configuration.",
shouldRetry: false
});
}
},
removeSecrets: async (secretSync: TAwsSecretsManagerSyncWithCredentials, secretMap: TSecretMap) => {
const { destinationConfig } = secretSync;
const client = await getSecretsManagerClient(secretSync);
const awsSecretsRecord = await getSecretsRecord(client);
if (destinationConfig.mappingBehavior === AwsSecretsManagerSyncMappingBehavior.OneToOne) {
for await (const secretKey of Object.keys(awsSecretsRecord)) {
if (secretKey in secretMap) {
try {
await deleteSecret(client, secretKey);
} catch (error) {
throw new SecretSyncError({
error,
secretKey
});
}
}
}
} else {
await deleteSecret(client, destinationConfig.secretName);
}
}
};

View File

@@ -0,0 +1,63 @@
import { z } from "zod";
import { SecretSyncs } from "@app/lib/api-docs";
import { AppConnection, AWSRegion } from "@app/services/app-connection/app-connection-enums";
import { AwsSecretsManagerSyncMappingBehavior } from "@app/services/secret-sync/aws-secrets-manager/aws-secrets-manager-sync-enums";
import { SecretSync } from "@app/services/secret-sync/secret-sync-enums";
import {
BaseSecretSyncSchema,
GenericCreateSecretSyncFieldsSchema,
GenericUpdateSecretSyncFieldsSchema
} from "@app/services/secret-sync/secret-sync-schemas";
const AwsSecretsManagerSyncDestinationConfigSchema = z
.discriminatedUnion("mappingBehavior", [
z.object({
mappingBehavior: z
.literal(AwsSecretsManagerSyncMappingBehavior.OneToOne)
.describe(SecretSyncs.DESTINATION_CONFIG.AWS_SECRETS_MANAGER.MAPPING_BEHAVIOR)
}),
z.object({
mappingBehavior: z
.literal(AwsSecretsManagerSyncMappingBehavior.ManyToOne)
.describe(SecretSyncs.DESTINATION_CONFIG.AWS_SECRETS_MANAGER.MAPPING_BEHAVIOR),
secretName: z
.string()
.regex(
/^[a-zA-Z0-9/_+=.@-]+$/,
"Secret name must contain only alphanumeric characters and the characters /_+=.@-"
)
.min(1, "Secret name is required")
.max(256, "Secret name cannot exceed 256 characters")
.describe(SecretSyncs.DESTINATION_CONFIG.AWS_SECRETS_MANAGER.SECRET_NAME)
})
])
.and(
z.object({
region: z.nativeEnum(AWSRegion).describe(SecretSyncs.DESTINATION_CONFIG.AWS_SECRETS_MANAGER.REGION)
})
);
export const AwsSecretsManagerSyncSchema = BaseSecretSyncSchema(SecretSync.AWSSecretsManager).extend({
destination: z.literal(SecretSync.AWSSecretsManager),
destinationConfig: AwsSecretsManagerSyncDestinationConfigSchema
});
export const CreateAwsSecretsManagerSyncSchema = GenericCreateSecretSyncFieldsSchema(
SecretSync.AWSSecretsManager
).extend({
destinationConfig: AwsSecretsManagerSyncDestinationConfigSchema
});
export const UpdateAwsSecretsManagerSyncSchema = GenericUpdateSecretSyncFieldsSchema(
SecretSync.AWSSecretsManager
).extend({
destinationConfig: AwsSecretsManagerSyncDestinationConfigSchema.optional()
});
export const AwsSecretsManagerSyncListItemSchema = z.object({
name: z.literal("AWS Secrets Manager"),
connection: z.literal(AppConnection.AWS),
destination: z.literal(SecretSync.AWSSecretsManager),
canImportSecrets: z.literal(true)
});

View File

@@ -0,0 +1,19 @@
import { z } from "zod";
import { TAwsConnection } from "@app/services/app-connection/aws";
import {
AwsSecretsManagerSyncListItemSchema,
AwsSecretsManagerSyncSchema,
CreateAwsSecretsManagerSyncSchema
} from "./aws-secrets-manager-sync-schemas";
export type TAwsSecretsManagerSync = z.infer<typeof AwsSecretsManagerSyncSchema>;
export type TAwsSecretsManagerSyncInput = z.infer<typeof CreateAwsSecretsManagerSyncSchema>;
export type TAwsSecretsManagerSyncListItem = z.infer<typeof AwsSecretsManagerSyncListItemSchema>;
export type TAwsSecretsManagerSyncWithCredentials = TAwsSecretsManagerSync & {
connection: TAwsConnection;
};

View File

@@ -0,0 +1,4 @@
export * from "./aws-secrets-manager-sync-constants";
export * from "./aws-secrets-manager-sync-fns";
export * from "./aws-secrets-manager-sync-schemas";
export * from "./aws-secrets-manager-sync-types";

View File

@@ -0,0 +1,10 @@
import { AppConnection } from "@app/services/app-connection/app-connection-enums";
import { SecretSync } from "@app/services/secret-sync/secret-sync-enums";
import { TSecretSyncListItem } from "@app/services/secret-sync/secret-sync-types";
export const AZURE_APP_CONFIGURATION_SYNC_LIST_OPTION: TSecretSyncListItem = {
name: "Azure App Configuration",
destination: SecretSync.AzureAppConfiguration,
connection: AppConnection.AzureAppConfiguration,
canImportSecrets: true
};

View File

@@ -0,0 +1,214 @@
/* eslint-disable no-await-in-loop */
import https from "https";
import { request } from "@app/lib/config/request";
import { BadRequestError } from "@app/lib/errors";
import { TAppConnectionDALFactory } from "@app/services/app-connection/app-connection-dal";
import { getAzureConnectionAccessToken } from "@app/services/app-connection/azure-key-vault";
import { isAzureKeyVaultReference } from "@app/services/integration-auth/integration-sync-secret-fns";
import { TKmsServiceFactory } from "@app/services/kms/kms-service";
import { TSecretMap } from "@app/services/secret-sync/secret-sync-types";
import { TAzureAppConfigurationSyncWithCredentials } from "./azure-app-configuration-sync-types";
type TAzureAppConfigurationSecretSyncFactoryDeps = {
appConnectionDAL: Pick<TAppConnectionDALFactory, "findById" | "update">;
kmsService: Pick<TKmsServiceFactory, "createCipherPairWithDataKey">;
};
interface AzureAppConfigKeyValue {
key: string;
value: string;
label?: string;
}
export const azureAppConfigurationSecretSyncFactory = ({
kmsService,
appConnectionDAL
}: TAzureAppConfigurationSecretSyncFactoryDeps) => {
const $getCompleteAzureAppConfigValues = async (accessToken: string, baseURL: string, url: string) => {
let result: AzureAppConfigKeyValue[] = [];
let currentUrl = url;
while (currentUrl) {
const res = await request.get<{ items: AzureAppConfigKeyValue[]; ["@nextLink"]: string }>(currentUrl, {
baseURL,
headers: {
Authorization: `Bearer ${accessToken}`
},
// we force IPV4 because docker setup fails with ipv6
httpsAgent: new https.Agent({
family: 4
})
});
result = result.concat(res.data.items);
currentUrl = res.data?.["@nextLink"];
}
return result;
};
const $deleteAzureSecret = async (accessToken: string, configurationUrl: string, key: string, label?: string) => {
await request.delete(`${configurationUrl}/kv/${key}?api-version=2023-11-01`, {
headers: {
Authorization: `Bearer ${accessToken}`
},
...(label &&
label.length > 0 && {
params: {
label
}
}),
httpsAgent: new https.Agent({
family: 4
})
});
};
const syncSecrets = async (secretSync: TAzureAppConfigurationSyncWithCredentials, secretMap: TSecretMap) => {
if (!secretSync.destinationConfig.configurationUrl.endsWith(".azconfig.io")) {
throw new BadRequestError({
message: "Invalid Azure App Configuration URL provided."
});
}
const { accessToken } = await getAzureConnectionAccessToken(secretSync.connectionId, appConnectionDAL, kmsService);
const azureAppConfigValuesUrl = `/kv?api-version=2023-11-01${
secretSync.destinationConfig.label ? `&label=${secretSync.destinationConfig.label}` : "&label=%00"
}`;
const azureAppConfigValuesUrlAllSecrets = `/kv?api-version=2023-11-01`;
const azureAppConfigSecretsLabeled = Object.fromEntries(
(
await $getCompleteAzureAppConfigValues(
accessToken,
secretSync.destinationConfig.configurationUrl,
azureAppConfigValuesUrl
)
).map((entry) => [entry.key, entry.value])
);
const azureAppConfigSecrets = Object.fromEntries(
(
await $getCompleteAzureAppConfigValues(
accessToken,
secretSync.destinationConfig.configurationUrl,
azureAppConfigValuesUrlAllSecrets
)
).map((entry) => [
entry.key,
{
value: entry.value,
label: entry.label
}
])
);
// add the secrets to azure app config, that are in infisical
for await (const key of Object.keys(secretMap)) {
if (!(key in azureAppConfigSecretsLabeled) || secretMap[key]?.value !== azureAppConfigSecretsLabeled[key]) {
await request.put(
`${secretSync.destinationConfig.configurationUrl}/kv/${key}?api-version=2023-11-01`,
{
value: secretMap[key]?.value,
...(isAzureKeyVaultReference(secretMap[key]?.value || "") && {
content_type: "application/vnd.microsoft.appconfig.keyvaultref+json;charset=utf-8"
})
},
{
...(secretSync.destinationConfig.label && {
params: {
label: secretSync.destinationConfig.label
}
}),
headers: {
Authorization: `Bearer ${accessToken}`
},
httpsAgent: new https.Agent({
family: 4
})
}
);
}
}
for await (const key of Object.keys(azureAppConfigSecrets)) {
const azureSecret = azureAppConfigSecrets[key];
if (
!(key in secretMap) ||
secretMap[key] === null ||
(azureSecret.label && azureSecret.label !== secretSync.destinationConfig.label) ||
(!azureSecret.label && secretSync.destinationConfig.label)
) {
await $deleteAzureSecret(accessToken, secretSync.destinationConfig.configurationUrl, key, azureSecret.label);
}
}
};
const removeSecrets = async (secretSync: TAzureAppConfigurationSyncWithCredentials, secretMap: TSecretMap) => {
const { accessToken } = await getAzureConnectionAccessToken(secretSync.connectionId, appConnectionDAL, kmsService);
const azureAppConfigValuesUrl = `/kv?api-version=2023-11-01${
secretSync.destinationConfig.label ? `&label=${secretSync.destinationConfig.label}` : "&label=%00"
}`;
const azureAppConfigSecrets = Object.fromEntries(
(
await $getCompleteAzureAppConfigValues(
accessToken,
secretSync.destinationConfig.configurationUrl,
azureAppConfigValuesUrl
)
).map((entry) => [entry.key, entry.value])
);
for await (const infisicalKey of Object.keys(secretMap)) {
if (infisicalKey in azureAppConfigSecrets) {
await $deleteAzureSecret(
accessToken,
secretSync.destinationConfig.configurationUrl,
infisicalKey,
secretSync.destinationConfig.label
);
}
}
};
const getSecrets = async (secretSync: TAzureAppConfigurationSyncWithCredentials) => {
const { accessToken } = await getAzureConnectionAccessToken(secretSync.connectionId, appConnectionDAL, kmsService);
const secretMap: TSecretMap = {};
const azureAppConfigValuesUrl = `/kv?api-version=2023-11-01${
secretSync.destinationConfig.label ? `&label=${secretSync.destinationConfig.label}` : "&label=%00"
}`;
const azureAppConfigSecrets = Object.fromEntries(
(
await $getCompleteAzureAppConfigValues(
accessToken,
secretSync.destinationConfig.configurationUrl,
azureAppConfigValuesUrl
)
).map((entry) => [entry.key, entry.value])
);
Object.keys(azureAppConfigSecrets).forEach((key) => {
secretMap[key] = {
value: azureAppConfigSecrets[key]
};
});
return secretMap;
};
return {
syncSecrets,
removeSecrets,
getSecrets
};
};

View File

@@ -0,0 +1,50 @@
import { z } from "zod";
import { SecretSyncs } from "@app/lib/api-docs";
import { AppConnection } from "@app/services/app-connection/app-connection-enums";
import { SecretSync } from "@app/services/secret-sync/secret-sync-enums";
import {
BaseSecretSyncSchema,
GenericCreateSecretSyncFieldsSchema,
GenericUpdateSecretSyncFieldsSchema
} from "@app/services/secret-sync/secret-sync-schemas";
import { TSyncOptionsConfig } from "@app/services/secret-sync/secret-sync-types";
const AzureAppConfigurationSyncDestinationConfigSchema = z.object({
configurationUrl: z
.string()
.min(1, "App Configuration URL required")
.describe(SecretSyncs.DESTINATION_CONFIG.AZURE_APP_CONFIGURATION.CONFIGURATION_URL),
label: z.string().optional().describe(SecretSyncs.DESTINATION_CONFIG.AZURE_APP_CONFIGURATION.LABEL)
});
const AzureAppConfigurationSyncOptionsConfig: TSyncOptionsConfig = { canImportSecrets: true };
export const AzureAppConfigurationSyncSchema = BaseSecretSyncSchema(
SecretSync.AzureAppConfiguration,
AzureAppConfigurationSyncOptionsConfig
).extend({
destination: z.literal(SecretSync.AzureAppConfiguration),
destinationConfig: AzureAppConfigurationSyncDestinationConfigSchema
});
export const CreateAzureAppConfigurationSyncSchema = GenericCreateSecretSyncFieldsSchema(
SecretSync.AzureAppConfiguration,
AzureAppConfigurationSyncOptionsConfig
).extend({
destinationConfig: AzureAppConfigurationSyncDestinationConfigSchema
});
export const UpdateAzureAppConfigurationSyncSchema = GenericUpdateSecretSyncFieldsSchema(
SecretSync.AzureAppConfiguration,
AzureAppConfigurationSyncOptionsConfig
).extend({
destinationConfig: AzureAppConfigurationSyncDestinationConfigSchema.optional()
});
export const AzureAppConfigurationSyncListItemSchema = z.object({
name: z.literal("Azure App Configuration"),
connection: z.literal(AppConnection.AzureAppConfiguration),
destination: z.literal(SecretSync.AzureAppConfiguration),
canImportSecrets: z.literal(true)
});

View File

@@ -0,0 +1,19 @@
import { z } from "zod";
import { TAzureAppConfigurationConnection } from "@app/services/app-connection/azure-app-configuration";
import {
AzureAppConfigurationSyncListItemSchema,
AzureAppConfigurationSyncSchema,
CreateAzureAppConfigurationSyncSchema
} from "./azure-app-configuration-sync-schemas";
export type TAzureAppConfigurationSync = z.infer<typeof AzureAppConfigurationSyncSchema>;
export type TAzureAppConfigurationSyncInput = z.infer<typeof CreateAzureAppConfigurationSyncSchema>;
export type TAzureAppConfigurationSyncListItem = z.infer<typeof AzureAppConfigurationSyncListItemSchema>;
export type TAzureAppConfigurationSyncWithCredentials = TAzureAppConfigurationSync & {
connection: TAzureAppConfigurationConnection;
};

View File

@@ -0,0 +1,4 @@
export * from "./azure-app-configuration-sync-constants";
export * from "./azure-app-configuration-sync-fns";
export * from "./azure-app-configuration-sync-schemas";
export * from "./azure-app-configuration-sync-types";

View File

@@ -0,0 +1,10 @@
import { AppConnection } from "@app/services/app-connection/app-connection-enums";
import { SecretSync } from "@app/services/secret-sync/secret-sync-enums";
import { TSecretSyncListItem } from "@app/services/secret-sync/secret-sync-types";
export const AZURE_KEY_VAULT_SYNC_LIST_OPTION: TSecretSyncListItem = {
name: "Azure Key Vault",
destination: SecretSync.AzureKeyVault,
connection: AppConnection.AzureKeyVault,
canImportSecrets: true
};

View File

@@ -0,0 +1,256 @@
/* eslint-disable no-await-in-loop */
import { AxiosError } from "axios";
import { request } from "@app/lib/config/request";
import { TAppConnectionDALFactory } from "@app/services/app-connection/app-connection-dal";
import { getAzureConnectionAccessToken } from "@app/services/app-connection/azure-key-vault";
import { TKmsServiceFactory } from "@app/services/kms/kms-service";
import { TSecretMap } from "@app/services/secret-sync/secret-sync-types";
import { SecretSyncError } from "../secret-sync-errors";
import { GetAzureKeyVaultSecret, TAzureKeyVaultSyncWithCredentials } from "./azure-key-vault-sync-types";
type TAzureKeyVaultSecretSyncFactoryDeps = {
appConnectionDAL: Pick<TAppConnectionDALFactory, "findById" | "update">;
kmsService: Pick<TKmsServiceFactory, "createCipherPairWithDataKey">;
};
export const azureKeyVaultSecretSyncFactory = ({
kmsService,
appConnectionDAL
}: TAzureKeyVaultSecretSyncFactoryDeps) => {
const $getAzureKeyVaultSecrets = async (accessToken: string, vaultBaseUrl: string) => {
const paginateAzureKeyVaultSecrets = async () => {
let result: GetAzureKeyVaultSecret[] = [];
let currentUrl = `${vaultBaseUrl}/secrets?api-version=7.3`;
while (currentUrl) {
const res = await request.get<{ value: GetAzureKeyVaultSecret; nextLink: string }>(currentUrl, {
headers: {
Authorization: `Bearer ${accessToken}`
}
});
result = result.concat(res.data.value);
currentUrl = res.data.nextLink;
}
return result;
};
const getAzureKeyVaultSecrets = await paginateAzureKeyVaultSecrets();
const enabledAzureKeyVaultSecrets = getAzureKeyVaultSecrets.filter((secret) => secret.attributes.enabled);
// disabled keys to skip sending updates to
const disabledAzureKeyVaultSecretKeys = getAzureKeyVaultSecrets
.filter(({ attributes }) => !attributes.enabled)
.map((getAzureKeyVaultSecret) => {
return getAzureKeyVaultSecret.id.substring(getAzureKeyVaultSecret.id.lastIndexOf("/") + 1);
});
let lastSlashIndex: number;
const res = (
await Promise.all(
enabledAzureKeyVaultSecrets.map(async (getAzureKeyVaultSecret) => {
if (!lastSlashIndex) {
lastSlashIndex = getAzureKeyVaultSecret.id.lastIndexOf("/");
}
const azureKeyVaultSecret = await request.get<GetAzureKeyVaultSecret>(
`${getAzureKeyVaultSecret.id}?api-version=7.3`,
{
headers: {
Authorization: `Bearer ${accessToken}`
}
}
);
return {
...azureKeyVaultSecret.data,
key: getAzureKeyVaultSecret.id.substring(lastSlashIndex + 1)
};
})
)
).reduce(
(obj, secret) => ({
...obj,
[secret.key]: secret
}),
{} as Record<string, GetAzureKeyVaultSecret>
);
return {
vaultSecrets: res,
disabledAzureKeyVaultSecretKeys
};
};
const syncSecrets = async (secretSync: TAzureKeyVaultSyncWithCredentials, secretMap: TSecretMap) => {
const { accessToken } = await getAzureConnectionAccessToken(secretSync.connection.id, appConnectionDAL, kmsService);
const { vaultSecrets, disabledAzureKeyVaultSecretKeys } = await $getAzureKeyVaultSecrets(
accessToken,
secretSync.destinationConfig.vaultBaseUrl
);
const setSecrets: {
key: string;
value: string;
}[] = [];
const deleteSecrets: string[] = [];
Object.keys(secretMap).forEach((infisicalKey) => {
const hyphenatedKey = infisicalKey.replace(/_/g, "-");
if (!(hyphenatedKey in vaultSecrets)) {
// case: secret has been created
setSecrets.push({
key: hyphenatedKey,
value: secretMap[infisicalKey].value
});
} else if (secretMap[infisicalKey].value !== vaultSecrets[hyphenatedKey].value) {
// case: secret has been updated
setSecrets.push({
key: hyphenatedKey,
value: secretMap[infisicalKey].value
});
}
});
Object.keys(vaultSecrets).forEach((key) => {
const underscoredKey = key.replace(/-/g, "_");
if (!(underscoredKey in secretMap)) {
deleteSecrets.push(key);
}
});
const setSecretAzureKeyVault = async ({ key, value }: { key: string; value: string }) => {
let isSecretSet = false;
let syncError: Error | null = null;
let maxTries = 6;
if (disabledAzureKeyVaultSecretKeys.includes(key)) return;
while (!isSecretSet && maxTries > 0) {
// try to set secret
try {
await request.put(
`${secretSync.destinationConfig.vaultBaseUrl}/secrets/${key}?api-version=7.3`,
{
value
},
{
headers: {
Authorization: `Bearer ${accessToken}`
}
}
);
isSecretSet = true;
} catch (err) {
syncError = err as Error;
if (err instanceof AxiosError) {
// eslint-disable-next-line
if (err.response?.data?.error?.innererror?.code === "ObjectIsDeletedButRecoverable") {
await request.post(
`${secretSync.destinationConfig.vaultBaseUrl}/deletedsecrets/${key}/recover?api-version=7.3`,
{},
{
headers: {
Authorization: `Bearer ${accessToken}`
}
}
);
await new Promise((resolve) => {
setTimeout(resolve, 10_000);
});
} else {
await new Promise((resolve) => {
setTimeout(resolve, 10_000);
});
maxTries -= 1;
}
}
}
}
if (!isSecretSet) {
throw new SecretSyncError({
error: syncError,
secretKey: key
});
}
};
for await (const setSecret of setSecrets) {
const { key, value } = setSecret;
await setSecretAzureKeyVault({
key,
value
});
}
for await (const deleteSecretKey of deleteSecrets.filter(
(secret) => !setSecrets.find((setSecret) => setSecret.key === secret)
)) {
await request.delete(`${secretSync.destinationConfig.vaultBaseUrl}/secrets/${deleteSecretKey}?api-version=7.3`, {
headers: {
Authorization: `Bearer ${accessToken}`
}
});
}
};
const removeSecrets = async (secretSync: TAzureKeyVaultSyncWithCredentials, secretMap: TSecretMap) => {
const { accessToken } = await getAzureConnectionAccessToken(secretSync.connection.id, appConnectionDAL, kmsService);
const { vaultSecrets, disabledAzureKeyVaultSecretKeys } = await $getAzureKeyVaultSecrets(
accessToken,
secretSync.destinationConfig.vaultBaseUrl
);
for await (const [key] of Object.entries(vaultSecrets)) {
const underscoredKey = key.replace(/-/g, "_");
if (underscoredKey in secretMap) {
if (!disabledAzureKeyVaultSecretKeys.includes(underscoredKey)) {
await request.delete(`${secretSync.destinationConfig.vaultBaseUrl}/secrets/${key}?api-version=7.3`, {
headers: {
Authorization: `Bearer ${accessToken}`
}
});
}
}
}
};
const getSecrets = async (secretSync: TAzureKeyVaultSyncWithCredentials) => {
const { accessToken } = await getAzureConnectionAccessToken(secretSync.connection.id, appConnectionDAL, kmsService);
const { vaultSecrets, disabledAzureKeyVaultSecretKeys } = await $getAzureKeyVaultSecrets(
accessToken,
secretSync.destinationConfig.vaultBaseUrl
);
const secretMap: TSecretMap = {};
Object.keys(vaultSecrets).forEach((key) => {
if (!disabledAzureKeyVaultSecretKeys.includes(key)) {
const underscoredKey = key.replace(/-/g, "_");
secretMap[underscoredKey] = {
value: vaultSecrets[key].value
};
}
});
return secretMap;
};
return {
syncSecrets,
removeSecrets,
getSecrets
};
};

View File

@@ -0,0 +1,50 @@
import { z } from "zod";
import { SecretSyncs } from "@app/lib/api-docs";
import { AppConnection } from "@app/services/app-connection/app-connection-enums";
import { SecretSync } from "@app/services/secret-sync/secret-sync-enums";
import {
BaseSecretSyncSchema,
GenericCreateSecretSyncFieldsSchema,
GenericUpdateSecretSyncFieldsSchema
} from "@app/services/secret-sync/secret-sync-schemas";
import { TSyncOptionsConfig } from "@app/services/secret-sync/secret-sync-types";
const AzureKeyVaultSyncDestinationConfigSchema = z.object({
vaultBaseUrl: z
.string()
.url("Invalid vault base URL format")
.min(1, "Vault base URL required")
.describe(SecretSyncs.DESTINATION_CONFIG.AZURE_KEY_VAULT.VAULT_BASE_URL)
});
const AzureKeyVaultSyncOptionsConfig: TSyncOptionsConfig = { canImportSecrets: true };
export const AzureKeyVaultSyncSchema = BaseSecretSyncSchema(
SecretSync.AzureKeyVault,
AzureKeyVaultSyncOptionsConfig
).extend({
destination: z.literal(SecretSync.AzureKeyVault),
destinationConfig: AzureKeyVaultSyncDestinationConfigSchema
});
export const CreateAzureKeyVaultSyncSchema = GenericCreateSecretSyncFieldsSchema(
SecretSync.AzureKeyVault,
AzureKeyVaultSyncOptionsConfig
).extend({
destinationConfig: AzureKeyVaultSyncDestinationConfigSchema
});
export const UpdateAzureKeyVaultSyncSchema = GenericUpdateSecretSyncFieldsSchema(
SecretSync.AzureKeyVault,
AzureKeyVaultSyncOptionsConfig
).extend({
destinationConfig: AzureKeyVaultSyncDestinationConfigSchema.optional()
});
export const AzureKeyVaultSyncListItemSchema = z.object({
name: z.literal("Azure Key Vault"),
connection: z.literal(AppConnection.AzureKeyVault),
destination: z.literal(SecretSync.AzureKeyVault),
canImportSecrets: z.literal(true)
});

View File

@@ -0,0 +1,35 @@
import { z } from "zod";
import { TAzureKeyVaultConnection } from "@app/services/app-connection/azure-key-vault";
import {
AzureKeyVaultSyncListItemSchema,
AzureKeyVaultSyncSchema,
CreateAzureKeyVaultSyncSchema
} from "./azure-key-vault-sync-schemas";
export type TAzureKeyVaultSync = z.infer<typeof AzureKeyVaultSyncSchema>;
export type TAzureKeyVaultSyncInput = z.infer<typeof CreateAzureKeyVaultSyncSchema>;
export type TAzureKeyVaultSyncListItem = z.infer<typeof AzureKeyVaultSyncListItemSchema>;
export type TAzureKeyVaultSyncWithCredentials = TAzureKeyVaultSync & {
connection: TAzureKeyVaultConnection;
};
export interface GetAzureKeyVaultSecret {
id: string; // secret URI
value: string;
attributes: {
enabled: boolean;
created: number;
updated: number;
recoveryLevel: string;
recoverableDays: number;
};
}
export interface AzureKeyVaultSecret extends GetAzureKeyVaultSecret {
key: string;
}

View File

@@ -0,0 +1,4 @@
export * from "./azure-key-vault-sync-constants";
export * from "./azure-key-vault-sync-fns";
export * from "./azure-key-vault-sync-schemas";
export * from "./azure-key-vault-sync-types";

View File

@@ -4,7 +4,7 @@ import { TDbClient } from "@app/db";
import { TableName } from "@app/db/schemas";
import { TSecretSyncs } from "@app/db/schemas/secret-syncs";
import { DatabaseError } from "@app/lib/errors";
import { buildFindFilter, ormify, selectAllTableCols } from "@app/lib/knex";
import { buildFindFilter, ormify, prependTableNameToFindFilter, selectAllTableCols } from "@app/lib/knex";
import { TSecretFolderDALFactory } from "@app/services/secret-folder/secret-folder-dal";
export type TSecretSyncDALFactory = ReturnType<typeof secretSyncDALFactory>;
@@ -34,17 +34,9 @@ const baseSecretSyncQuery = ({ filter, db, tx }: { db: TDbClient; filter?: Secre
db.ref("updatedAt").withSchema(TableName.AppConnection).as("connectionUpdatedAt")
);
// prepends table name to filter keys to avoid ambiguous col references, skipping utility filters like $in, etc.
const prependTableName = (filterObj: object): SecretSyncFindFilter =>
Object.fromEntries(
Object.entries(filterObj).map(([key, value]) =>
key.startsWith("$") ? [key, prependTableName(value as object)] : [`${TableName.SecretSync}.${key}`, value]
)
);
if (filter) {
/* eslint-disable @typescript-eslint/no-misused-promises */
void query.where(buildFindFilter(prependTableName(filter)));
void query.where(buildFindFilter(prependTableNameToFindFilter(TableName.SecretSync, filter)));
}
return query;
@@ -123,47 +115,39 @@ export const secretSyncDALFactory = (
};
const create = async (data: Parameters<(typeof secretSyncOrm)["create"]>[0]) => {
try {
const secretSync = (await secretSyncOrm.transaction(async (tx) => {
const sync = await secretSyncOrm.create(data, tx);
const secretSync = (await secretSyncOrm.transaction(async (tx) => {
const sync = await secretSyncOrm.create(data, tx);
return baseSecretSyncQuery({
filter: { id: sync.id },
db,
tx
}).first();
}))!;
return baseSecretSyncQuery({
filter: { id: sync.id },
db,
tx
}).first();
}))!;
// TODO (scott): replace with cached folder path once implemented
const [folderWithPath] = secretSync.folderId
? await folderDAL.findSecretPathByFolderIds(secretSync.projectId, [secretSync.folderId])
: [];
return expandSecretSync(secretSync, folderWithPath);
} catch (error) {
throw new DatabaseError({ error, name: "Create - Secret Sync" });
}
// TODO (scott): replace with cached folder path once implemented
const [folderWithPath] = secretSync.folderId
? await folderDAL.findSecretPathByFolderIds(secretSync.projectId, [secretSync.folderId])
: [];
return expandSecretSync(secretSync, folderWithPath);
};
const updateById = async (syncId: string, data: Parameters<(typeof secretSyncOrm)["updateById"]>[1]) => {
try {
const secretSync = (await secretSyncOrm.transaction(async (tx) => {
const sync = await secretSyncOrm.updateById(syncId, data, tx);
const secretSync = (await secretSyncOrm.transaction(async (tx) => {
const sync = await secretSyncOrm.updateById(syncId, data, tx);
return baseSecretSyncQuery({
filter: { id: sync.id },
db,
tx
}).first();
}))!;
return baseSecretSyncQuery({
filter: { id: sync.id },
db,
tx
}).first();
}))!;
// TODO (scott): replace with cached folder path once implemented
const [folderWithPath] = secretSync.folderId
? await folderDAL.findSecretPathByFolderIds(secretSync.projectId, [secretSync.folderId])
: [];
return expandSecretSync(secretSync, folderWithPath);
} catch (error) {
throw new DatabaseError({ error, name: "Update by ID - Secret Sync" });
}
// TODO (scott): replace with cached folder path once implemented
const [folderWithPath] = secretSync.folderId
? await folderDAL.findSecretPathByFolderIds(secretSync.projectId, [secretSync.folderId])
: [];
return expandSecretSync(secretSync, folderWithPath);
};
const findOne = async (filter: Parameters<(typeof secretSyncOrm)["findOne"]>[0], tx?: Knex) => {

View File

@@ -1,7 +1,10 @@
export enum SecretSync {
AWSParameterStore = "aws-parameter-store",
AWSSecretsManager = "aws-secrets-manager",
GitHub = "github",
GCPSecretManager = "gcp-secret-manager"
GCPSecretManager = "gcp-secret-manager",
AzureKeyVault = "azure-key-vault",
AzureAppConfiguration = "azure-app-configuration"
}
export enum SecretSyncInitialSyncBehavior {

View File

@@ -4,6 +4,10 @@ import {
AWS_PARAMETER_STORE_SYNC_LIST_OPTION,
AwsParameterStoreSyncFns
} from "@app/services/secret-sync/aws-parameter-store";
import {
AWS_SECRETS_MANAGER_SYNC_LIST_OPTION,
AwsSecretsManagerSyncFns
} from "@app/services/secret-sync/aws-secrets-manager";
import { GITHUB_SYNC_LIST_OPTION, GithubSyncFns } from "@app/services/secret-sync/github";
import { SecretSync } from "@app/services/secret-sync/secret-sync-enums";
import { SecretSyncError } from "@app/services/secret-sync/secret-sync-errors";
@@ -13,19 +17,34 @@ import {
TSecretSyncWithCredentials
} from "@app/services/secret-sync/secret-sync-types";
import { TAppConnectionDALFactory } from "../app-connection/app-connection-dal";
import { TKmsServiceFactory } from "../kms/kms-service";
import {
AZURE_APP_CONFIGURATION_SYNC_LIST_OPTION,
azureAppConfigurationSecretSyncFactory
} from "./azure-app-configuration";
import { AZURE_KEY_VAULT_SYNC_LIST_OPTION, azureKeyVaultSecretSyncFactory } from "./azure-key-vault";
import { GCP_SYNC_LIST_OPTION } from "./gcp";
import { GcpSyncFns } from "./gcp/gcp-sync-fns";
const SECRET_SYNC_LIST_OPTIONS: Record<SecretSync, TSecretSyncListItem> = {
[SecretSync.AWSParameterStore]: AWS_PARAMETER_STORE_SYNC_LIST_OPTION,
[SecretSync.AWSSecretsManager]: AWS_SECRETS_MANAGER_SYNC_LIST_OPTION,
[SecretSync.GitHub]: GITHUB_SYNC_LIST_OPTION,
[SecretSync.GCPSecretManager]: GCP_SYNC_LIST_OPTION
[SecretSync.GCPSecretManager]: GCP_SYNC_LIST_OPTION,
[SecretSync.AzureKeyVault]: AZURE_KEY_VAULT_SYNC_LIST_OPTION,
[SecretSync.AzureAppConfiguration]: AZURE_APP_CONFIGURATION_SYNC_LIST_OPTION
};
export const listSecretSyncOptions = () => {
return Object.values(SECRET_SYNC_LIST_OPTIONS).sort((a, b) => a.name.localeCompare(b.name));
};
type TSyncSecretDeps = {
appConnectionDAL: Pick<TAppConnectionDALFactory, "findById" | "update">;
kmsService: Pick<TKmsServiceFactory, "createCipherPairWithDataKey">;
};
// const addAffixes = (secretSync: TSecretSyncWithCredentials, unprocessedSecretMap: TSecretMap) => {
// let secretMap = { ...unprocessedSecretMap };
//
@@ -67,34 +86,68 @@ export const listSecretSyncOptions = () => {
// };
export const SecretSyncFns = {
syncSecrets: (secretSync: TSecretSyncWithCredentials, secretMap: TSecretMap): Promise<void> => {
syncSecrets: (
secretSync: TSecretSyncWithCredentials,
secretMap: TSecretMap,
{ kmsService, appConnectionDAL }: TSyncSecretDeps
): Promise<void> => {
// const affixedSecretMap = addAffixes(secretSync, secretMap);
switch (secretSync.destination) {
case SecretSync.AWSParameterStore:
return AwsParameterStoreSyncFns.syncSecrets(secretSync, secretMap);
case SecretSync.AWSSecretsManager:
return AwsSecretsManagerSyncFns.syncSecrets(secretSync, secretMap);
case SecretSync.GitHub:
return GithubSyncFns.syncSecrets(secretSync, secretMap);
case SecretSync.GCPSecretManager:
return GcpSyncFns.syncSecrets(secretSync, secretMap);
case SecretSync.AzureKeyVault:
return azureKeyVaultSecretSyncFactory({
appConnectionDAL,
kmsService
}).syncSecrets(secretSync, secretMap);
case SecretSync.AzureAppConfiguration:
return azureAppConfigurationSecretSyncFactory({
appConnectionDAL,
kmsService
}).syncSecrets(secretSync, secretMap);
default:
throw new Error(
`Unhandled sync destination for sync secrets fns: ${(secretSync as TSecretSyncWithCredentials).destination}`
);
}
},
getSecrets: async (secretSync: TSecretSyncWithCredentials): Promise<TSecretMap> => {
getSecrets: async (
secretSync: TSecretSyncWithCredentials,
{ kmsService, appConnectionDAL }: TSyncSecretDeps
): Promise<TSecretMap> => {
let secretMap: TSecretMap;
switch (secretSync.destination) {
case SecretSync.AWSParameterStore:
secretMap = await AwsParameterStoreSyncFns.getSecrets(secretSync);
break;
case SecretSync.AWSSecretsManager:
secretMap = await AwsSecretsManagerSyncFns.getSecrets(secretSync);
break;
case SecretSync.GitHub:
secretMap = await GithubSyncFns.getSecrets(secretSync);
break;
case SecretSync.GCPSecretManager:
secretMap = await GcpSyncFns.getSecrets(secretSync);
break;
case SecretSync.AzureKeyVault:
secretMap = await azureKeyVaultSecretSyncFactory({
appConnectionDAL,
kmsService
}).getSecrets(secretSync);
break;
case SecretSync.AzureAppConfiguration:
secretMap = await azureAppConfigurationSecretSyncFactory({
appConnectionDAL,
kmsService
}).getSecrets(secretSync);
break;
default:
throw new Error(
`Unhandled sync destination for get secrets fns: ${(secretSync as TSecretSyncWithCredentials).destination}`
@@ -104,16 +157,32 @@ export const SecretSyncFns = {
return secretMap;
// return stripAffixes(secretSync, secretMap);
},
removeSecrets: (secretSync: TSecretSyncWithCredentials, secretMap: TSecretMap): Promise<void> => {
removeSecrets: (
secretSync: TSecretSyncWithCredentials,
secretMap: TSecretMap,
{ kmsService, appConnectionDAL }: TSyncSecretDeps
): Promise<void> => {
// const affixedSecretMap = addAffixes(secretSync, secretMap);
switch (secretSync.destination) {
case SecretSync.AWSParameterStore:
return AwsParameterStoreSyncFns.removeSecrets(secretSync, secretMap);
case SecretSync.AWSSecretsManager:
return AwsSecretsManagerSyncFns.removeSecrets(secretSync, secretMap);
case SecretSync.GitHub:
return GithubSyncFns.removeSecrets(secretSync, secretMap);
case SecretSync.GCPSecretManager:
return GcpSyncFns.removeSecrets(secretSync, secretMap);
case SecretSync.AzureKeyVault:
return azureKeyVaultSecretSyncFactory({
appConnectionDAL,
kmsService
}).removeSecrets(secretSync, secretMap);
case SecretSync.AzureAppConfiguration:
return azureAppConfigurationSecretSyncFactory({
appConnectionDAL,
kmsService
}).removeSecrets(secretSync, secretMap);
default:
throw new Error(
`Unhandled sync destination for remove secrets fns: ${(secretSync as TSecretSyncWithCredentials).destination}`
@@ -122,17 +191,25 @@ export const SecretSyncFns = {
}
};
const MAX_MESSAGE_LENGTH = 1024;
export const parseSyncErrorMessage = (err: unknown): string => {
let errorMessage: string;
if (err instanceof SecretSyncError) {
return JSON.stringify({
errorMessage = JSON.stringify({
secretKey: err.secretKey,
error: err.message || parseSyncErrorMessage(err.error)
});
} else if (err instanceof AxiosError) {
errorMessage = err?.response?.data
? JSON.stringify(err?.response?.data)
: err?.message ?? "An unknown error occurred.";
} else {
errorMessage = (err as Error)?.message || "An unknown error occurred.";
}
if (err instanceof AxiosError) {
return err?.response?.data ? JSON.stringify(err?.response?.data) : err?.message ?? "An unknown error occurred.";
}
return (err as Error)?.message || "An unknown error occurred.";
return errorMessage.length <= MAX_MESSAGE_LENGTH
? errorMessage
: `${errorMessage.substring(0, MAX_MESSAGE_LENGTH - 3)}...`;
};

View File

@@ -3,12 +3,18 @@ import { SecretSync } from "@app/services/secret-sync/secret-sync-enums";
export const SECRET_SYNC_NAME_MAP: Record<SecretSync, string> = {
[SecretSync.AWSParameterStore]: "AWS Parameter Store",
[SecretSync.AWSSecretsManager]: "AWS Secrets Manager",
[SecretSync.GitHub]: "GitHub",
[SecretSync.GCPSecretManager]: "GCP Secret Manager"
[SecretSync.GCPSecretManager]: "GCP Secret Manager",
[SecretSync.AzureKeyVault]: "Azure Key Vault",
[SecretSync.AzureAppConfiguration]: "Azure App Configuration"
};
export const SECRET_SYNC_CONNECTION_MAP: Record<SecretSync, AppConnection> = {
[SecretSync.AWSParameterStore]: AppConnection.AWS,
[SecretSync.AWSSecretsManager]: AppConnection.AWS,
[SecretSync.GitHub]: AppConnection.GitHub,
[SecretSync.GCPSecretManager]: AppConnection.GCP
[SecretSync.GCPSecretManager]: AppConnection.GCP,
[SecretSync.AzureKeyVault]: AppConnection.AzureKeyVault,
[SecretSync.AzureAppConfiguration]: AppConnection.AzureAppConfiguration
};

View File

@@ -57,11 +57,14 @@ import { TSecretVersionV2DALFactory } from "@app/services/secret-v2-bridge/secre
import { TSecretVersionV2TagDALFactory } from "@app/services/secret-v2-bridge/secret-version-tag-dal";
import { SmtpTemplates, TSmtpService } from "@app/services/smtp/smtp-service";
import { TAppConnectionDALFactory } from "../app-connection/app-connection-dal";
export type TSecretSyncQueueFactory = ReturnType<typeof secretSyncQueueFactory>;
type TSecretSyncQueueFactoryDep = {
queueService: Pick<TQueueServiceFactory, "queue" | "start">;
kmsService: Pick<TKmsServiceFactory, "createCipherPairWithDataKey">;
appConnectionDAL: Pick<TAppConnectionDALFactory, "findById" | "update">;
keyStore: Pick<TKeyStoreFactory, "acquireLock" | "setItemWithExpiry" | "getItem">;
folderDAL: TSecretFolderDALFactory;
secretV2BridgeDAL: Pick<
@@ -111,6 +114,7 @@ const getRequeueDelay = (failureCount?: number) => {
export const secretSyncQueueFactory = ({
queueService,
kmsService,
appConnectionDAL,
keyStore,
folderDAL,
secretV2BridgeDAL,
@@ -322,7 +326,10 @@ export const secretSyncQueueFactory = ({
"Invalid Secret Sync source configuration: folder no longer exists. Please update source environment and secret path."
);
const importedSecrets = await SecretSyncFns.getSecrets(secretSync);
const importedSecrets = await SecretSyncFns.getSecrets(secretSync, {
appConnectionDAL,
kmsService
});
if (!Object.keys(importedSecrets).length) return {};
@@ -434,7 +441,10 @@ export const secretSyncQueueFactory = ({
});
}
await SecretSyncFns.syncSecrets(secretSyncWithCredentials, secretMap);
await SecretSyncFns.syncSecrets(secretSyncWithCredentials, secretMap, {
appConnectionDAL,
kmsService
});
isSynced = true;
} catch (err) {
@@ -672,7 +682,11 @@ export const secretSyncQueueFactory = ({
credentials
}
} as TSecretSyncWithCredentials,
secretMap
secretMap,
{
appConnectionDAL,
kmsService
}
);
isSuccess = true;

View File

@@ -8,7 +8,8 @@ import {
ProjectPermissionSub
} from "@app/ee/services/permission/project-permission";
import { KeyStorePrefixes, TKeyStoreFactory } from "@app/keystore/keystore";
import { BadRequestError, NotFoundError } from "@app/lib/errors";
import { DatabaseErrorCode } from "@app/lib/error-codes";
import { BadRequestError, DatabaseError, NotFoundError } from "@app/lib/errors";
import { OrgServiceActor } from "@app/lib/types";
import { TAppConnectionServiceFactory } from "@app/services/app-connection/app-connection-service";
import { TProjectBotServiceFactory } from "@app/services/project-bot/project-bot-service";
@@ -197,37 +198,26 @@ export const secretSyncServiceFactory = ({
// validates permission to connect and app is valid for sync destination
await appConnectionService.connectAppConnectionById(destinationApp, params.connectionId, actor);
const secretSync = await secretSyncDAL.transaction(async (tx) => {
const isConflictingName = Boolean(
(
await secretSyncDAL.find(
{
name: params.name,
projectId
},
tx
)
).length
);
if (isConflictingName)
throw new BadRequestError({
message: `A Secret Sync with the name "${params.name}" already exists for the project with ID "${folder.projectId}"`
});
const sync = await secretSyncDAL.create({
try {
const secretSync = await secretSyncDAL.create({
folderId: folder.id,
...params,
...(params.isAutoSyncEnabled && { syncStatus: SecretSyncStatus.Pending }),
projectId
});
return sync;
});
if (secretSync.isAutoSyncEnabled) await secretSyncQueue.queueSecretSyncSyncSecretsById({ syncId: secretSync.id });
if (secretSync.isAutoSyncEnabled) await secretSyncQueue.queueSecretSyncSyncSecretsById({ syncId: secretSync.id });
return secretSync as TSecretSync;
} catch (err) {
if (err instanceof DatabaseError && (err.error as { code: string })?.code === DatabaseErrorCode.UniqueViolation) {
throw new BadRequestError({
message: `A Secret Sync with the name "${params.name}" already exists for the project with ID "${folder.projectId}"`
});
}
return secretSync as TSecretSync;
throw err;
}
};
const updateSecretSync = async (
@@ -260,78 +250,65 @@ export const secretSyncServiceFactory = ({
message: `Secret sync with ID "${secretSync.id}" is not configured for ${SECRET_SYNC_NAME_MAP[destination]}`
});
const updatedSecretSync = await secretSyncDAL.transaction(async (tx) => {
let { folderId } = secretSync;
let { folderId } = secretSync;
if (params.connectionId) {
const destinationApp = SECRET_SYNC_CONNECTION_MAP[secretSync.destination as SecretSync];
if (params.connectionId) {
const destinationApp = SECRET_SYNC_CONNECTION_MAP[secretSync.destination as SecretSync];
// validates permission to connect and app is valid for sync destination
await appConnectionService.connectAppConnectionById(destinationApp, params.connectionId, actor);
}
// validates permission to connect and app is valid for sync destination
await appConnectionService.connectAppConnectionById(destinationApp, params.connectionId, actor);
}
if (
(secretPath && secretPath !== secretSync.folder?.path) ||
(environment && environment !== secretSync.environment?.slug)
) {
const updatedEnvironment = environment ?? secretSync.environment?.slug;
const updatedSecretPath = secretPath ?? secretSync.folder?.path;
if (
(secretPath && secretPath !== secretSync.folder?.path) ||
(environment && environment !== secretSync.environment?.slug)
) {
const updatedEnvironment = environment ?? secretSync.environment?.slug;
const updatedSecretPath = secretPath ?? secretSync.folder?.path;
if (!updatedEnvironment || !updatedSecretPath)
throw new BadRequestError({ message: "Must specify both source environment and secret path" });
if (!updatedEnvironment || !updatedSecretPath)
throw new BadRequestError({ message: "Must specify both source environment and secret path" });
ForbiddenError.from(permission).throwUnlessCan(
ProjectPermissionActions.Read,
subject(ProjectPermissionSub.Secrets, {
environment: updatedEnvironment,
secretPath: updatedSecretPath
})
);
ForbiddenError.from(permission).throwUnlessCan(
ProjectPermissionActions.Read,
subject(ProjectPermissionSub.Secrets, {
environment: updatedEnvironment,
secretPath: updatedSecretPath
})
);
const newFolder = await folderDAL.findBySecretPath(secretSync.projectId, updatedEnvironment, updatedSecretPath);
const newFolder = await folderDAL.findBySecretPath(secretSync.projectId, updatedEnvironment, updatedSecretPath);
if (!newFolder)
throw new BadRequestError({
message: `Could not find folder with path "${secretPath}" in environment "${environment}" for project with ID "${secretSync.projectId}"`
});
if (!newFolder)
throw new BadRequestError({
message: `Could not find folder with path "${secretPath}" in environment "${environment}" for project with ID "${secretSync.projectId}"`
});
folderId = newFolder.id;
}
folderId = newFolder.id;
}
if (params.name && secretSync.name !== params.name) {
const isConflictingName = Boolean(
(
await secretSyncDAL.find(
{
name: params.name,
projectId: secretSync.projectId
},
tx
)
).length
);
const isAutoSyncEnabled = params.isAutoSyncEnabled ?? secretSync.isAutoSyncEnabled;
if (isConflictingName)
throw new BadRequestError({
message: `A Secret Sync with the name "${params.name}" already exists for project with ID "${secretSync.projectId}"`
});
}
const isAutoSyncEnabled = params.isAutoSyncEnabled ?? secretSync.isAutoSyncEnabled;
const updatedSync = await secretSyncDAL.updateById(syncId, {
try {
const updatedSecretSync = await secretSyncDAL.updateById(syncId, {
...params,
...(isAutoSyncEnabled && folderId && { syncStatus: SecretSyncStatus.Pending }),
folderId
});
return updatedSync;
});
if (updatedSecretSync.isAutoSyncEnabled)
await secretSyncQueue.queueSecretSyncSyncSecretsById({ syncId: secretSync.id });
if (updatedSecretSync.isAutoSyncEnabled)
await secretSyncQueue.queueSecretSyncSyncSecretsById({ syncId: secretSync.id });
return updatedSecretSync as TSecretSync;
} catch (err) {
if (err instanceof DatabaseError && (err.error as { code: string })?.code === DatabaseErrorCode.UniqueViolation) {
throw new BadRequestError({
message: `A Secret Sync with the name "${params.name}" already exists for the project with ID "${secretSync.projectId}"`
});
}
return updatedSecretSync as TSecretSync;
throw err;
}
};
const deleteSecretSync = async (

View File

@@ -2,6 +2,12 @@ import { Job } from "bullmq";
import { TCreateAuditLogDTO } from "@app/ee/services/audit-log/audit-log-types";
import { QueueJobs } from "@app/queue";
import {
TAwsSecretsManagerSync,
TAwsSecretsManagerSyncInput,
TAwsSecretsManagerSyncListItem,
TAwsSecretsManagerSyncWithCredentials
} from "@app/services/secret-sync/aws-secrets-manager";
import {
TGitHubSync,
TGitHubSyncInput,
@@ -17,18 +23,51 @@ import {
TAwsParameterStoreSyncListItem,
TAwsParameterStoreSyncWithCredentials
} from "./aws-parameter-store";
import {
TAzureAppConfigurationSync,
TAzureAppConfigurationSyncInput,
TAzureAppConfigurationSyncListItem,
TAzureAppConfigurationSyncWithCredentials
} from "./azure-app-configuration";
import {
TAzureKeyVaultSync,
TAzureKeyVaultSyncInput,
TAzureKeyVaultSyncListItem,
TAzureKeyVaultSyncWithCredentials
} from "./azure-key-vault";
import { TGcpSync, TGcpSyncInput, TGcpSyncListItem, TGcpSyncWithCredentials } from "./gcp";
export type TSecretSync = TAwsParameterStoreSync | TGitHubSync | TGcpSync;
export type TSecretSync =
| TAwsParameterStoreSync
| TAwsSecretsManagerSync
| TGitHubSync
| TGcpSync
| TAzureKeyVaultSync
| TAzureAppConfigurationSync;
export type TSecretSyncWithCredentials =
| TAwsParameterStoreSyncWithCredentials
| TAwsSecretsManagerSyncWithCredentials
| TGitHubSyncWithCredentials
| TGcpSyncWithCredentials;
| TGcpSyncWithCredentials
| TAzureKeyVaultSyncWithCredentials
| TAzureAppConfigurationSyncWithCredentials;
export type TSecretSyncInput = TAwsParameterStoreSyncInput | TGitHubSyncInput | TGcpSyncInput;
export type TSecretSyncInput =
| TAwsParameterStoreSyncInput
| TAwsSecretsManagerSyncInput
| TGitHubSyncInput
| TGcpSyncInput
| TAzureKeyVaultSyncInput
| TAzureAppConfigurationSyncInput;
export type TSecretSyncListItem = TAwsParameterStoreSyncListItem | TGitHubSyncListItem | TGcpSyncListItem;
export type TSecretSyncListItem =
| TAwsParameterStoreSyncListItem
| TAwsSecretsManagerSyncListItem
| TGitHubSyncListItem
| TGcpSyncListItem
| TAzureKeyVaultSyncListItem
| TAzureAppConfigurationSyncListItem;
export type TSyncOptionsConfig = {
canImportSecrets: boolean;

View File

@@ -414,6 +414,20 @@ export const secretV2BridgeDALFactory = (db: TDbClient) => {
`${TableName.SecretTag}.id`
)
.leftJoin(TableName.ResourceMetadata, `${TableName.SecretV2}.id`, `${TableName.ResourceMetadata}.secretId`)
.where((qb) => {
if (filters?.metadataFilter && filters.metadataFilter.length > 0) {
filters.metadataFilter.forEach((meta) => {
void qb.whereExists((subQuery) => {
void subQuery
.select("secretId")
.from(TableName.ResourceMetadata)
.whereRaw(`"${TableName.ResourceMetadata}"."secretId" = "${TableName.SecretV2}"."id"`)
.where(`${TableName.ResourceMetadata}.key`, meta.key)
.where(`${TableName.ResourceMetadata}.value`, meta.value);
});
});
}
})
.select(
selectAllTableCols(TableName.SecretV2),
db.raw(
@@ -481,6 +495,7 @@ export const secretV2BridgeDALFactory = (db: TDbClient) => {
}
]
});
return data;
} catch (error) {
throw new DatabaseError({ error, name: "get all secret" });

View File

@@ -1291,8 +1291,13 @@ export const secretV2BridgeServiceFactory = ({
]
}
});
if (secretsToUpdate.length !== inputSecrets.length)
throw new NotFoundError({ message: `Secret does not exist: ${secretsToUpdate.map((el) => el.key).join(",")}` });
if (secretsToUpdate.length !== inputSecrets.length) {
const secretsToUpdateNames = secretsToUpdate.map((secret) => secret.key);
const invalidSecrets = inputSecrets.filter((secret) => !secretsToUpdateNames.includes(secret.secretKey));
throw new NotFoundError({
message: `Secret does not exist: ${invalidSecrets.map((el) => el.secretKey).join(",")}`
});
}
const secretsToUpdateInDBGroupedByKey = groupBy(secretsToUpdate, (i) => i.key);
secretsToUpdate.forEach((el) => {

View File

@@ -30,6 +30,10 @@ export type TGetSecretsDTO = {
includeImports?: boolean;
recursive?: boolean;
tagSlugs?: string[];
metadataFilter?: {
key?: string;
value?: string;
}[];
orderBy?: SecretsOrderBy;
orderDirection?: OrderByDirection;
offset?: number;
@@ -310,6 +314,7 @@ export type TFindSecretsByFolderIdsFilter = {
orderDirection?: OrderByDirection;
search?: string;
tagSlugs?: string[];
metadataFilter?: { key?: string; value?: string }[];
includeTagsInSearch?: boolean;
keys?: string[];
};

View File

@@ -1263,6 +1263,13 @@ export const secretServiceFactory = ({
name: "bot_not_found_error"
});
if (paramsV2.metadataFilter) {
throw new BadRequestError({
message: "Please upgrade your project to filter secrets by metadata",
name: "SecretMetadataNotSupported"
});
}
const { secrets, imports } = await getSecrets({
actorId,
projectId,
@@ -1444,7 +1451,7 @@ export const secretServiceFactory = ({
decryptedSecret.secretValue = expandedSecretValue || "";
}
return decryptedSecret;
return { secretMetadata: undefined, ...decryptedSecret };
};
const createSecretRaw = async ({

View File

@@ -182,6 +182,10 @@ export type TGetSecretsRawDTO = {
includeImports?: boolean;
recursive?: boolean;
tagSlugs?: string[];
metadataFilter?: {
key?: string;
value?: string;
}[];
orderBy?: SecretsOrderBy;
orderDirection?: OrderByDirection;
offset?: number;

View File

@@ -110,7 +110,7 @@ var secretsCmd = &cobra.Command{
if plainOutput {
for _, secret := range secrets {
fmt.Println(secret.Value)
fmt.Println(fmt.Sprintf("%s=%s", secret.Key, secret.Value))
}
} else {
visualize.PrintAllSecretDetails(secrets)

View File

@@ -0,0 +1,67 @@
---
title: "How to write a design document"
sidebarTitle: "Writing Design Docs"
description: "Learn how to write a design document at Infisical"
---
## **Why write a design document?**
Writing a design document helps you efficiently solve broad, complex engineering problems at Infisical. While planning is important, we are a startup, so speed and urgency should be your top of mind. Keep the process lightweight and time boxed so that we can get the most out of it.
**Writing a design will help you:**
- **Understand the problem space:** Deeply understand the problem youre solving to make sure it is well scoped.
- **Stay on the right path:** Without proper planning, you risk cycling between partial implementation and replanning, encountering roadblocks that force you back to square one. A solid plan minimizes wasted engineering hours.
- **An opportunity to collaborate:** Bring relevant engineers into the discussion to develop well-thought-out solutions and catch potential issues you might have overlooked.
- **Faster implementation:** A well-thought-out plan will help you catch roadblocks early and ship quickly because you know exactly what needs to get implemented.
**When to write a design document:**
- **Write a design doc**: If the feature is not well defined, high-security, or will take more than **1 full engineering week** to build.
- **Skip the design doc**: For small, straightforward features that can be built quickly with informal discussions.
If you are unsure when to create a design doc, chat with @maidul.
## **What to Include in your Design Document**
Every feature/problem is unique, but your design docs should generally include the following sections. If you need to include additional sections, feel free to do so.
1. **Title**
- A descriptive title.
- Name of document owner and name of reviewer(s).
2. **Overview**
- A high-level summary of the problem and proposed solution. Keep it brief (max 3 paragraphs).
3. **Context**
- Explain the problems background, why its important to solve now, and any constraints (e.g., technical, sales, or timeline-related). What do we get out of solving this problem? (needed to close a deal, scale, performance, etc.).
4. **Solution**
- Provide a big-picture explanation of the solution, followed by detailed technical architecture.
- Use diagrams/charts where needed.
- Write clearly so that another engineer could implement the solution in your absence.
5. **Milestones**
- Break the project into phases with clear start and end dates estimates. Use a table or bullet points.
6. **FAQ**
- Common questions or concerns someone might have while reading your document that can be quickly addressed.
## **How to Write a Design Doc**
- **Keep it Simple**: Use clear, simple language. Opt for short sentences, bullet points, and concrete examples over fluff writing.
- **Use Visuals**: Add diagrams and charts for clarity to convey your ideas.
- **Make it Self-Explanatory**: Ensure that anyone reading the document can understand and implement the plan without needing additional context.
Before sharing your design docs with others, review your design doc as if you were a teammate seeing it for the first time. Anticipate questions and address them.
## **Process from start to finish**
1. **Research/Discuss**
- Before you start writing, take some time to research and get a solid understanding of the problem space. Look into how other well-established companies are tackling similar challenges, if they are.
Talk through the problem and your initial solution with other engineers on the team—bounce ideas around and get their feedback. If you have ideas on how the system could if implemented in Infisical, would it effect any downstream features/systems, etc?
Once youve got a general direction, you might need to test a some theories. This is where quick proof of concepts (POCs) come in handy, but dont get too caught up in the details. The goal of a POC is simply to validate a core idea or concept so you can get to the rest of your planning.
2. **Write the Doc**
- Based on your research/discussions, write the design doc and include all relevant sections. Your goal is to come up with a convincing plan on why this is the correct why to solve the problem at hand.
3. **Assign Reviewers**
- Ask a relevant engineer(s) to review your document. Their role is to identify blind spots, challenge assumptions, and ensure everything is clear. Once you and the reviewer are on the same page on the approach, update the document with any missing details they brought up.
4. **Team Review and Feedback**
- Invite the relevant engineers to a design doc review meeting and give them 10-15 minutes to read through the document. After everyone has had a chance to review it, open the floor up for discussion. Address any feedback or concerns raised during this meeting. If significant points were overlooked during your initial planning, you may need to revisit the drawing board. Your goal is to think about the feature holistically and minimize the need for drastic changes to your design doc later on.

View File

@@ -66,7 +66,8 @@
{
"group": "Engineering",
"pages": [
"documentation/engineering/oncall"
"documentation/engineering/oncall",
"documentation/engineering/how-to-write-design-doc"
]
}
]

View File

@@ -192,6 +192,17 @@ services:
depends_on:
- openldap
profiles: [ldap]
keycloak:
image: quay.io/keycloak/keycloak:26.1.0
restart: always
environment:
- KC_BOOTSTRAP_ADMIN_PASSWORD=admin
- KC_BOOTSTRAP_ADMIN_USERNAME=admin
command: start-dev
ports:
- 8088:8080
profiles: [ sso ]
volumes:
postgres-data:

View File

@@ -0,0 +1,4 @@
---
title: "Available"
openapi: "GET /api/v1/app-connections/azure-app-configuration/available"
---

View File

@@ -0,0 +1,10 @@
---
title: "Create"
openapi: "POST /api/v1/app-connections/azure-app-configuration"
---
<Note>
Azure App Configuration Connections must be created through the Infisical UI.
Check out the configuration docs for [Azure App Configuration Connections](/integrations/app-connections/azure-app-configuration) for a step-by-step
guide.
</Note>

View File

@@ -0,0 +1,4 @@
---
title: "Delete"
openapi: "DELETE /api/v1/app-connections/azure-app-configuration/{connectionId}"
---

View File

@@ -0,0 +1,4 @@
---
title: "Get by ID"
openapi: "GET /api/v1/app-connections/azure-app-configuration/{connectionId}"
---

View File

@@ -0,0 +1,4 @@
---
title: "Get by Name"
openapi: "GET /api/v1/app-connections/azure-app-configuration/connection-name/{connectionName}"
---

View File

@@ -0,0 +1,4 @@
---
title: "List"
openapi: "GET /api/v1/app-connections/azure-app-configuration"
---

View File

@@ -0,0 +1,10 @@
---
title: "Update"
openapi: "PATCH /api/v1/app-connections/azure-app-configuration/{connectionId}"
---
<Note>
Azure App Configuration Connections must be updated through the Infisical UI.
Check out the configuration docs for [Azure App Configuration Connections](/integrations/app-connections/azure-app-configuration) for a step-by-step
guide.
</Note>

View File

@@ -0,0 +1,4 @@
---
title: "Available"
openapi: "GET /api/v1/app-connections/azure-key-vault/available"
---

Some files were not shown because too many files have changed in this diff Show More