Compare commits

...

110 Commits

Author SHA1 Message Date
b5801af9a8 improvements: address feedback 2025-06-30 18:32:36 -07:00
20366a8c07 improvement: address feedback 2025-06-30 18:09:50 -07:00
447e28511c improvement: update stale/conflict text 2025-06-30 16:44:29 -07:00
650ed656e3 improvement: color/layout styling adjustments to change request page 2025-06-30 16:30:37 -07:00
3871fa552c Merge pull request #3888 from Infisical/revert-3885-misc/add-indices-for-referencing-columns-in-identity-access-token
Revert "misc: add indices for referencing columns in identity access token"
2025-06-30 17:27:31 -04:00
9c72ee7f10 Revert "misc: add indices for referencing columns in identity access token" 2025-07-01 05:23:51 +08:00
22e8617661 Merge pull request #3885 from Infisical/misc/add-indices-for-referencing-columns-in-identity-access-token
misc: add indices for referencing columns in identity access token
2025-06-30 17:01:20 -04:00
2f29a513cc misc: make index creation concurrently 2025-07-01 03:36:55 +08:00
d3833c33b3 Merge pull request #3878 from Infisical/fix-approval-policy-bypassing
Fix bypassing approval policies
2025-06-30 13:37:28 -04:00
978a3e5828 misc: add indices for referencing columns in identity access token 2025-07-01 01:25:11 +08:00
27bf91e58f Merge pull request #3873 from Infisical/org-access-control-improvements
improvement(org-access-control): Standardize and improve org access control UI
2025-06-30 09:54:42 -07:00
f2c3c76c60 improvement: address feedback on remove rule policy edit 2025-06-30 09:21:00 -07:00
85023916e4 improvement: address feedback 2025-06-30 09:12:47 -07:00
02afd6a8e7 Merge pull request #3882 from Infisical/feat/fix-access-token-ips
feat: resolved inefficient join for ip restriction in access token
2025-06-30 21:22:28 +05:30
=
929eac4350 feat: resolved inefficient join for ip restriction in access token 2025-06-30 20:13:26 +05:30
c6074dd69a Merge pull request #3881 from Infisical/docs-update
update spend policy
2025-06-29 18:10:54 -07:00
a9b26755ba update spend policy 2025-06-29 17:43:05 -07:00
033e5d3f81 Merge pull request #3880 from Infisical/docs-update
update logos in docs
2025-06-28 16:38:05 -07:00
90634e1913 update logos in docs 2025-06-28 16:26:58 -07:00
58b61a861a Fix bypassing approval policies 2025-06-28 04:17:09 -04:00
3c8ec7d7fb Merge pull request #3869 from Infisical/sequence-approval-policy-ui-additions
improvement(access-policies): Revamp approval sequence table display and access request modal
2025-06-28 04:07:41 -04:00
26a59286c5 Merge pull request #3877 from Infisical/remove-datadog-logs
Remove debug logs for DataDog stream
2025-06-28 03:45:14 -04:00
392792bb1e Remove debug logs for DataDog stream 2025-06-28 03:37:32 -04:00
48f40ff938 improvement: address feedback 2025-06-27 21:00:48 -07:00
969896e431 Merge pull request #3874 from Infisical/remove-certauth-join
Remove cert auth left join
2025-06-27 20:41:58 -04:00
fd85da5739 set trusted ip to empty 2025-06-27 20:36:32 -04:00
2caf6ff94b remove cert auth left join 2025-06-27 20:21:28 -04:00
ed7d709a70 improvement: standardize and improve org access control 2025-06-27 15:15:12 -07:00
aff97374a9 Merge pull request #3868 from Infisical/misc/add-mention-of-service-usage-api-for-gcp
misc: add mention of service usage API for GCP
2025-06-28 04:26:21 +08:00
e8e90585ca Merge pull request #3871 from Infisical/project-role-type-col
improvement(project-roles): Add type col to project roles table and default sort
2025-06-27 11:42:47 -07:00
abd9dbf714 improvement: add type col to project roles table and default sort 2025-06-27 11:34:54 -07:00
89aed3640b Merge pull request #3852 from akhilmhdh/feat/tls-identity-auth
feat: TLS cert identity auth
2025-06-28 02:29:25 +08:00
5513ff7631 Merge pull request #3866 from Infisical/feat/posthogEventBatch
feat(telemetry): Add aggregated events and groups to posthog
2025-06-27 14:42:55 -03:00
9fb7676739 misc: reordered doc for mi auth 2025-06-28 01:35:46 +08:00
6ac734d6c4 removed unnecessary changes 2025-06-28 01:32:53 +08:00
8044999785 feat(telemetry): increase even redis key exp to 15 mins 2025-06-27 14:31:54 -03:00
be51e4372d feat(telemetry): addressed PR suggestions 2025-06-27 14:30:31 -03:00
460b545925 Merge branch 'feat/tls-identity-auth' of https://github.com/akhilmhdh/infisical into HEAD 2025-06-28 01:29:49 +08:00
2f26c1930b misc: doc updates 2025-06-28 01:26:24 +08:00
953cc3a850 improvements: revise approval sequence table display and access request modal 2025-06-27 09:30:11 -07:00
fc9ae05f89 misc: updated TLS acronym 2025-06-28 00:21:08 +08:00
de22a3c56b misc: updated casing of acronym 2025-06-28 00:17:42 +08:00
7c4baa6fd4 misc: added image for service usage API 2025-06-27 13:19:14 +00:00
f285648c95 misc: add mention of service usage API for GCP 2025-06-27 21:10:02 +08:00
0f04890d8f feat(telemetry): addressed PR suggestions 2025-06-26 21:18:07 -03:00
61274243e2 feat(telemetry): add batch events and groups logic 2025-06-26 20:58:01 -03:00
9366428091 Merge pull request #3865 from Infisical/remove-manual-styled-css-on-checkboxes
fix(checkbox): Remove manual css overrides of checkbox checked state
2025-06-26 15:38:05 -07:00
62482852aa fix: remove manual css overrides of checkbox checked state 2025-06-26 15:33:27 -07:00
cc02c00b61 Merge pull request #3864 from Infisical/update-aws-param-store-docs
Clarify relationship between path and key schema for AWS parameter store
2025-06-26 18:19:06 -04:00
2e256e4282 Tooltip 2025-06-26 18:14:48 -04:00
1b4bae6a84 Merge pull request #3863 from Infisical/remove-secret-scanning-v1-backend
chore(secret-scanning-v1): remove secret scanning v1 queue and webhook endpoint
2025-06-26 14:51:23 -07:00
1f0bcae0fc Merge pull request #3860 from Infisical/secret-sync-selection-improvements
improvement(secret-sync/app-connection): Add search/pagination to secret sync and app connection selection modals
2025-06-26 14:50:44 -07:00
dcd21883d1 Clarify relationship between path and key schema for AWS parameter store
docs
2025-06-26 17:02:21 -04:00
d7913a75c2 chore: remove secret scanning v1 queue and webhook endpoint 2025-06-26 11:32:45 -07:00
205442bff5 Merge pull request #3859 from Infisical/overview-ui-improvements
improvement(secret-overview): Add collapsed environment view to secret overview page
2025-06-26 09:24:33 -07:00
8ab51aba12 improvement: add search/pagination app connection select 2025-06-26 09:21:35 -07:00
3d1f054b87 improvement: add pagination/search to secret sync selection 2025-06-26 08:13:57 -07:00
29fedfdde5 Merge pull request #3850 from Infisical/policy-edit-revisions
improvement(project-policies): Revamp edit role page and access tree
2025-06-26 07:46:35 -07:00
b5317d1d75 fix: add ability to remove non-conditional rules 2025-06-26 07:37:30 -07:00
6446311b6d Merge pull request #3835 from Infisical/feat/gitlabSecretSync
feat(secret-sync): Add gitlab secret sync
2025-06-25 17:53:12 -03:00
3e80f1907c Merge pull request #3857 from Infisical/daniel/fix-dotnet-docs
docs: fix redirect for .NET SDK
2025-06-25 23:18:14 +04:00
79e62eec25 docs: fix redirect for .NET SDK 2025-06-25 23:11:11 +04:00
c41730c5fb Merge pull request #3856 from Infisical/daniel/fix-docs
fix(docs): sdk and changelog tab not loading
2025-06-25 22:34:09 +04:00
aac63d3097 fix(docs): sdk and changelog tab not working 2025-06-25 22:32:08 +04:00
f0b9d3c816 feat(secret-sync): improve hide secrets tooltip message 2025-06-25 14:10:28 -03:00
ea393d144a feat(secret-sync): minor change on docs 2025-06-25 13:57:07 -03:00
c4c0f86598 feat(secret-sync): improve update logic and add warning on docs for gitlab limitation on hidden variables 2025-06-25 13:51:38 -03:00
1f7617d132 Merge pull request #3851 from Infisical/ENG-3013
Allow undefined value for tags to prevent unwanted overrides
2025-06-25 12:45:43 -04:00
c95680b95d feat(secret-sync): type fix 2025-06-25 13:33:43 -03:00
18f1f93b5f Review fixes 2025-06-25 12:29:23 -04:00
70ea761375 feat(secret-sync): fix update masked_and_hidden field to not be sent unless it's true 2025-06-25 13:17:41 -03:00
5b4790ee78 improvements: truncate environment selection and only show visualize access when expanded 2025-06-25 09:09:08 -07:00
5ab2a6bb5d Feedback 2025-06-25 11:56:11 -04:00
dcac85fe6c Merge pull request #3847 from Infisical/share-your-own-secret-link-fix
fix(secret-sharing): Support self-hosted for "share your own secret" link
2025-06-25 08:31:13 -07:00
2f07471404 Merge pull request #3853 from akhilmhdh/feat/copy-token
feat: added copy token button
2025-06-25 10:55:07 -04:00
137fd5ef07 added minor text updates 2025-06-25 10:50:16 -04:00
=
883c7835a1 feat: added copy token button 2025-06-25 15:28:58 +05:30
=
e33f34ceb4 fix: corrected the doc key 2025-06-25 14:46:13 +05:30
=
af5805a5ca feat: resolved incorrect invalidation 2025-06-25 14:46:13 +05:30
bcf1c49a1b Update docs/documentation/platform/identities/tls-cert-auth.mdx
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-06-25 14:45:14 +05:30
84fedf8eda Update docs/documentation/platform/identities/tls-cert-auth.mdx
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-06-25 14:44:45 +05:30
97755981eb Update docs/documentation/platform/identities/tls-cert-auth.mdx
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-06-25 14:43:01 +05:30
8291663802 Update frontend/src/pages/organization/AccessManagementPage/components/OrgIdentityTab/components/IdentitySection/IdentityTlsCertAuthForm.tsx
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-06-25 14:42:24 +05:30
d9aed45504 Update frontend/src/pages/organization/AccessManagementPage/components/OrgIdentityTab/components/IdentitySection/IdentityTlsCertAuthForm.tsx
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-06-25 14:42:11 +05:30
=
8ada11edf3 feat: docs for tls cert auth 2025-06-25 14:27:04 +05:30
=
4bd62aa462 feat: updated frontend to have the tls cert auth login 2025-06-25 14:26:55 +05:30
0366e58a5b Type fix 2025-06-25 00:24:24 -03:00
9f6dca23db Greptile reviews 2025-06-24 23:19:42 -04:00
18e733c71f feat(secret-sync): minor fixes 2025-06-25 00:16:44 -03:00
f0a95808e7 Allow undefined value for tags to prevent unwanted overrides 2025-06-24 23:13:53 -04:00
90a0d0f744 Merge pull request #3848 from Infisical/improve-audit-log-streams
improve audit log streams: add backend logs + DD source
2025-06-24 22:18:04 -04:00
7f9c9be2c8 review fix 2025-06-24 22:00:45 -04:00
070982081c Merge remote-tracking branch 'origin/main' into feat/gitlabSecretSync 2025-06-24 22:42:28 -03:00
f462c3f85d feat(secret-sync): minor fixes 2025-06-24 21:38:33 -03:00
8683693103 improvement: address greptile feedback 2025-06-24 15:35:42 -07:00
737fffcceb improvement: address greptile feedback 2025-06-24 15:35:08 -07:00
ffac24ce75 improvement: revise edit role page and access tree 2025-06-24 15:23:27 -07:00
c505c5877f feat(secret-sync): updated docs 2025-06-24 18:11:18 -03:00
d4bf8a33dc feat(secret-sync): rework GitLab secret-sync to add group variables 2025-06-24 18:01:32 -03:00
6566393e21 Review fixes 2025-06-24 14:39:46 -04:00
af245b1f16 Add "service: audit-logs" entry for DataDog 2025-06-24 14:22:26 -04:00
c17df7e951 Improve URL detection 2025-06-24 12:44:16 -04:00
4d4953e95a improve audit log streams: add backend logs + DD source 2025-06-24 12:35:49 -04:00
43e0d400f9 feat(secret-sync): add Gitlab PR comments suggestions 2025-06-24 10:05:46 -03:00
=
b80b77ec36 feat: completed backend changes for tls auth 2025-06-24 16:46:46 +05:30
198e74cd88 fix: include nooppener in window.open 2025-06-23 18:05:48 -07:00
8ed0a1de84 fix: correct window open for share your own secret link to handle self-hosted 2025-06-23 18:01:38 -07:00
c305ddd463 feat(secret-sync): Gitlab PR suggestions 2025-06-23 10:52:59 -03:00
27cb686216 feat(secret-sync): Fix frontend file names 2025-06-20 21:26:12 -03:00
e201d77a8f feat(secret-sync): Add gitlab secret sync 2025-06-20 21:13:14 -03:00
255 changed files with 10259 additions and 3409 deletions

View File

@ -107,6 +107,10 @@ INF_APP_CONNECTION_GITHUB_APP_PRIVATE_KEY=
INF_APP_CONNECTION_GITHUB_APP_SLUG=
INF_APP_CONNECTION_GITHUB_APP_ID=
#gitlab app connection
INF_APP_CONNECTION_GITLAB_OAUTH_CLIENT_ID=
INF_APP_CONNECTION_GITLAB_OAUTH_CLIENT_SECRET=
#github radar app connection
INF_APP_CONNECTION_GITHUB_RADAR_APP_CLIENT_ID=
INF_APP_CONNECTION_GITHUB_RADAR_APP_CLIENT_SECRET=

View File

@ -8,6 +8,9 @@ import { Lock } from "@app/lib/red-lock";
export const mockKeyStore = (): TKeyStoreFactory => {
const store: Record<string, string | number | Buffer> = {};
const getRegex = (pattern: string) =>
new RE2(`^${pattern.replace(/[-[\]/{}()+?.\\^$|]/g, "\\$&").replace(/\*/g, ".*")}$`);
return {
setItem: async (key, value) => {
store[key] = value;
@ -23,7 +26,7 @@ export const mockKeyStore = (): TKeyStoreFactory => {
return 1;
},
deleteItems: async ({ pattern, batchSize = 500, delay = 1500, jitter = 200 }) => {
const regex = new RE2(`^${pattern.replace(/[-[\]/{}()+?.\\^$|]/g, "\\$&").replace(/\*/g, ".*")}$`);
const regex = getRegex(pattern);
let totalDeleted = 0;
const keys = Object.keys(store);
@ -53,6 +56,27 @@ export const mockKeyStore = (): TKeyStoreFactory => {
incrementBy: async () => {
return 1;
},
getItems: async (keys) => {
const values = keys.map((key) => {
const value = store[key];
if (typeof value === "string") {
return value;
}
return null;
});
return values;
},
getKeysByPattern: async (pattern) => {
const regex = getRegex(pattern);
const keys = Object.keys(store);
return keys.filter((key) => regex.test(key));
},
deleteItemsByKeyIn: async (keys) => {
for (const key of keys) {
delete store[key];
}
return keys.length;
},
acquireLock: () => {
return Promise.resolve({
release: () => {}

View File

@ -30,6 +30,7 @@
"@fastify/static": "^7.0.4",
"@fastify/swagger": "^8.14.0",
"@fastify/swagger-ui": "^2.1.0",
"@gitbeaker/rest": "^42.5.0",
"@google-cloud/kms": "^4.5.0",
"@infisical/quic": "^1.0.8",
"@node-saml/passport-saml": "^5.0.1",
@ -7807,6 +7808,48 @@
"p-limit": "^3.1.0"
}
},
"node_modules/@gitbeaker/core": {
"version": "42.5.0",
"resolved": "https://registry.npmjs.org/@gitbeaker/core/-/core-42.5.0.tgz",
"integrity": "sha512-rMWpOPaZi1iLiifnOIoVO57p2EmQQdfIwP4txqNyMvG4WjYP5Ez0U7jRD9Nra41x6K5kTPBZkuQcAdxVWRJcEQ==",
"license": "MIT",
"dependencies": {
"@gitbeaker/requester-utils": "^42.5.0",
"qs": "^6.12.2",
"xcase": "^2.0.1"
},
"engines": {
"node": ">=18.20.0"
}
},
"node_modules/@gitbeaker/requester-utils": {
"version": "42.5.0",
"resolved": "https://registry.npmjs.org/@gitbeaker/requester-utils/-/requester-utils-42.5.0.tgz",
"integrity": "sha512-HLdLS9LPBMVQumvroQg/4qkphLDtwDB+ygEsrD2u4oYCMUtXV4V1xaVqU4yTXjbTJ5sItOtdB43vYRkBcgueBw==",
"license": "MIT",
"dependencies": {
"picomatch-browser": "^2.2.6",
"qs": "^6.12.2",
"rate-limiter-flexible": "^4.0.1",
"xcase": "^2.0.1"
},
"engines": {
"node": ">=18.20.0"
}
},
"node_modules/@gitbeaker/rest": {
"version": "42.5.0",
"resolved": "https://registry.npmjs.org/@gitbeaker/rest/-/rest-42.5.0.tgz",
"integrity": "sha512-oC5cM6jS7aFOp0luTw5mWSRuMgdxwHRLZQ/aWkI+ETMfsprR/HyxsXfljlMY/XJ/fRxTbRJiodR5Axf66WjO3w==",
"license": "MIT",
"dependencies": {
"@gitbeaker/core": "^42.5.0",
"@gitbeaker/requester-utils": "^42.5.0"
},
"engines": {
"node": ">=18.20.0"
}
},
"node_modules/@google-cloud/kms": {
"version": "4.5.0",
"resolved": "https://registry.npmjs.org/@google-cloud/kms/-/kms-4.5.0.tgz",
@ -24628,6 +24671,18 @@
"url": "https://github.com/sponsors/jonschlinkert"
}
},
"node_modules/picomatch-browser": {
"version": "2.2.6",
"resolved": "https://registry.npmjs.org/picomatch-browser/-/picomatch-browser-2.2.6.tgz",
"integrity": "sha512-0ypsOQt9D4e3hziV8O4elD9uN0z/jtUEfxVRtNaAAtXIyUx9m/SzlO020i8YNL2aL/E6blOvvHQcin6HZlFy/w==",
"license": "MIT",
"engines": {
"node": ">=8.6"
},
"funding": {
"url": "https://github.com/sponsors/jonschlinkert"
}
},
"node_modules/pify": {
"version": "4.0.1",
"resolved": "https://registry.npmjs.org/pify/-/pify-4.0.1.tgz",
@ -25562,6 +25617,12 @@
"node": ">= 0.6"
}
},
"node_modules/rate-limiter-flexible": {
"version": "4.0.1",
"resolved": "https://registry.npmjs.org/rate-limiter-flexible/-/rate-limiter-flexible-4.0.1.tgz",
"integrity": "sha512-2/dGHpDFpeA0+755oUkW+EKyklqLS9lu0go9pDsbhqQjZcxfRyJ6LA4JI0+HAdZ2bemD/oOjUeZQB2lCZqXQfQ==",
"license": "ISC"
},
"node_modules/raw-body": {
"version": "2.5.2",
"resolved": "https://registry.npmjs.org/raw-body/-/raw-body-2.5.2.tgz",
@ -31039,6 +31100,12 @@
}
}
},
"node_modules/xcase": {
"version": "2.0.1",
"resolved": "https://registry.npmjs.org/xcase/-/xcase-2.0.1.tgz",
"integrity": "sha512-UmFXIPU+9Eg3E9m/728Bii0lAIuoc+6nbrNUKaRPJOFp91ih44qqGlWtxMB6kXFrRD6po+86ksHM5XHCfk6iPw==",
"license": "MIT"
},
"node_modules/xml-crypto": {
"version": "6.0.1",
"resolved": "https://registry.npmjs.org/xml-crypto/-/xml-crypto-6.0.1.tgz",

View File

@ -149,6 +149,7 @@
"@fastify/static": "^7.0.4",
"@fastify/swagger": "^8.14.0",
"@fastify/swagger-ui": "^2.1.0",
"@gitbeaker/rest": "^42.5.0",
"@google-cloud/kms": "^4.5.0",
"@infisical/quic": "^1.0.8",
"@node-saml/passport-saml": "^5.0.1",

View File

@ -74,6 +74,7 @@ import { TAllowedFields } from "@app/services/identity-ldap-auth/identity-ldap-a
import { TIdentityOciAuthServiceFactory } from "@app/services/identity-oci-auth/identity-oci-auth-service";
import { TIdentityOidcAuthServiceFactory } from "@app/services/identity-oidc-auth/identity-oidc-auth-service";
import { TIdentityProjectServiceFactory } from "@app/services/identity-project/identity-project-service";
import { TIdentityTlsCertAuthServiceFactory } from "@app/services/identity-tls-cert-auth/identity-tls-cert-auth-types";
import { TIdentityTokenAuthServiceFactory } from "@app/services/identity-token-auth/identity-token-auth-service";
import { TIdentityUaServiceFactory } from "@app/services/identity-ua/identity-ua-service";
import { TIntegrationServiceFactory } from "@app/services/integration/integration-service";
@ -218,6 +219,7 @@ declare module "fastify" {
identityKubernetesAuth: TIdentityKubernetesAuthServiceFactory;
identityGcpAuth: TIdentityGcpAuthServiceFactory;
identityAliCloudAuth: TIdentityAliCloudAuthServiceFactory;
identityTlsCertAuth: TIdentityTlsCertAuthServiceFactory;
identityAwsAuth: TIdentityAwsAuthServiceFactory;
identityAzureAuth: TIdentityAzureAuthServiceFactory;
identityOciAuth: TIdentityOciAuthServiceFactory;

View File

@ -164,6 +164,9 @@ import {
TIdentityProjectMemberships,
TIdentityProjectMembershipsInsert,
TIdentityProjectMembershipsUpdate,
TIdentityTlsCertAuths,
TIdentityTlsCertAuthsInsert,
TIdentityTlsCertAuthsUpdate,
TIdentityTokenAuths,
TIdentityTokenAuthsInsert,
TIdentityTokenAuthsUpdate,
@ -794,6 +797,11 @@ declare module "knex/types/tables" {
TIdentityAlicloudAuthsInsert,
TIdentityAlicloudAuthsUpdate
>;
[TableName.IdentityTlsCertAuth]: KnexOriginal.CompositeTableType<
TIdentityTlsCertAuths,
TIdentityTlsCertAuthsInsert,
TIdentityTlsCertAuthsUpdate
>;
[TableName.IdentityAwsAuth]: KnexOriginal.CompositeTableType<
TIdentityAwsAuths,
TIdentityAwsAuthsInsert,

View File

@ -0,0 +1,28 @@
import { Knex } from "knex";
import { TableName } from "../schemas";
import { createOnUpdateTrigger, dropOnUpdateTrigger } from "../utils";
export async function up(knex: Knex): Promise<void> {
if (!(await knex.schema.hasTable(TableName.IdentityTlsCertAuth))) {
await knex.schema.createTable(TableName.IdentityTlsCertAuth, (t) => {
t.uuid("id", { primaryKey: true }).defaultTo(knex.fn.uuid());
t.bigInteger("accessTokenTTL").defaultTo(7200).notNullable();
t.bigInteger("accessTokenMaxTTL").defaultTo(7200).notNullable();
t.bigInteger("accessTokenNumUsesLimit").defaultTo(0).notNullable();
t.jsonb("accessTokenTrustedIps").notNullable();
t.timestamps(true, true, true);
t.uuid("identityId").notNullable().unique();
t.foreign("identityId").references("id").inTable(TableName.Identity).onDelete("CASCADE");
t.string("allowedCommonNames").nullable();
t.binary("encryptedCaCertificate").notNullable();
});
}
await createOnUpdateTrigger(knex, TableName.IdentityTlsCertAuth);
}
export async function down(knex: Knex): Promise<void> {
await knex.schema.dropTableIfExists(TableName.IdentityTlsCertAuth);
await dropOnUpdateTrigger(knex, TableName.IdentityTlsCertAuth);
}

View File

@ -0,0 +1,27 @@
// Code generated by automation script, DO NOT EDIT.
// Automated by pulling database and generating zod schema
// To update. Just run npm run generate:schema
// Written by akhilmhdh.
import { z } from "zod";
import { zodBuffer } from "@app/lib/zod";
import { TImmutableDBKeys } from "./models";
export const IdentityTlsCertAuthsSchema = z.object({
id: z.string().uuid(),
accessTokenTTL: z.coerce.number().default(7200),
accessTokenMaxTTL: z.coerce.number().default(7200),
accessTokenNumUsesLimit: z.coerce.number().default(0),
accessTokenTrustedIps: z.unknown(),
createdAt: z.date(),
updatedAt: z.date(),
identityId: z.string().uuid(),
allowedCommonNames: z.string().nullable().optional(),
encryptedCaCertificate: zodBuffer
});
export type TIdentityTlsCertAuths = z.infer<typeof IdentityTlsCertAuthsSchema>;
export type TIdentityTlsCertAuthsInsert = Omit<z.input<typeof IdentityTlsCertAuthsSchema>, TImmutableDBKeys>;
export type TIdentityTlsCertAuthsUpdate = Partial<Omit<z.input<typeof IdentityTlsCertAuthsSchema>, TImmutableDBKeys>>;

View File

@ -52,6 +52,7 @@ export * from "./identity-org-memberships";
export * from "./identity-project-additional-privilege";
export * from "./identity-project-membership-role";
export * from "./identity-project-memberships";
export * from "./identity-tls-cert-auths";
export * from "./identity-token-auths";
export * from "./identity-ua-client-secrets";
export * from "./identity-universal-auths";

View File

@ -86,6 +86,7 @@ export enum TableName {
IdentityOidcAuth = "identity_oidc_auths",
IdentityJwtAuth = "identity_jwt_auths",
IdentityLdapAuth = "identity_ldap_auths",
IdentityTlsCertAuth = "identity_tls_cert_auths",
IdentityOrgMembership = "identity_org_memberships",
IdentityProjectMembership = "identity_project_memberships",
IdentityProjectMembershipRole = "identity_project_membership_role",
@ -251,6 +252,7 @@ export enum IdentityAuthMethod {
ALICLOUD_AUTH = "alicloud-auth",
AWS_AUTH = "aws-auth",
AZURE_AUTH = "azure-auth",
TLS_CERT_AUTH = "tls-cert-auth",
OCI_AUTH = "oci-auth",
OIDC_AUTH = "oidc-auth",
JWT_AUTH = "jwt-auth",

View File

@ -80,6 +80,7 @@ export const registerSshCertRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SignSshKey,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
certificateTemplateId: req.body.certificateTemplateId,
principals: req.body.principals,
@ -171,6 +172,7 @@ export const registerSshCertRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.IssueSshCreds,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
certificateTemplateId: req.body.certificateTemplateId,
principals: req.body.principals,

View File

@ -358,6 +358,7 @@ export const registerSshHostRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.IssueSshHostUserCert,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
sshHostId: req.params.sshHostId,
hostname: host.hostname,
@ -427,6 +428,7 @@ export const registerSshHostRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.IssueSshHostHostCert,
organizationId: req.permission.orgId,
distinctId: getTelemetryDistinctId(req),
properties: {
sshHostId: req.params.sshHostId,

View File

@ -350,6 +350,12 @@ export const accessApprovalRequestServiceFactory = ({
const canBypass = !policy.bypassers.length || policy.bypassers.some((bypasser) => bypasser.userId === actorId);
const cannotBypassUnderSoftEnforcement = !(isSoftEnforcement && canBypass);
// Calculate break glass attempt before sequence checks
const isBreakGlassApprovalAttempt =
policy.enforcementLevel === EnforcementLevel.Soft &&
actorId === accessApprovalRequest.requestedByUserId &&
status === ApprovalStatus.APPROVED;
const isApprover = policy.approvers.find((approver) => approver.userId === actorId);
// If user is (not an approver OR cant self approve) AND can't bypass policy
if ((!isApprover || (!policy.allowedSelfApprovals && isSelfApproval)) && cannotBypassUnderSoftEnforcement) {
@ -409,15 +415,14 @@ export const accessApprovalRequestServiceFactory = ({
const isApproverOfTheSequence = policy.approvers.find(
(el) => el.sequence === presentSequence.step && el.userId === actorId
);
if (!isApproverOfTheSequence) throw new BadRequestError({ message: "You are not reviewer in this step" });
// Only throw if actor is not the approver and not bypassing
if (!isApproverOfTheSequence && !isBreakGlassApprovalAttempt) {
throw new BadRequestError({ message: "You are not a reviewer in this step" });
}
}
const reviewStatus = await accessApprovalRequestReviewerDAL.transaction(async (tx) => {
const isBreakGlassApprovalAttempt =
policy.enforcementLevel === EnforcementLevel.Soft &&
actorId === accessApprovalRequest.requestedByUserId &&
status === ApprovalStatus.APPROVED;
let reviewForThisActorProcessing: {
id: string;
requestId: string;

View File

@ -0,0 +1,21 @@
export function providerSpecificPayload(url: string) {
const { hostname } = new URL(url);
const payload: Record<string, string> = {};
switch (hostname) {
case "http-intake.logs.datadoghq.com":
case "http-intake.logs.us3.datadoghq.com":
case "http-intake.logs.us5.datadoghq.com":
case "http-intake.logs.datadoghq.eu":
case "http-intake.logs.ap1.datadoghq.com":
case "http-intake.logs.ddog-gov.com":
payload.ddsource = "infisical";
payload.service = "audit-logs";
break;
default:
break;
}
return payload;
}

View File

@ -13,6 +13,7 @@ import { TLicenseServiceFactory } from "../license/license-service";
import { OrgPermissionActions, OrgPermissionSubjects } from "../permission/org-permission";
import { TPermissionServiceFactory } from "../permission/permission-service-types";
import { TAuditLogStreamDALFactory } from "./audit-log-stream-dal";
import { providerSpecificPayload } from "./audit-log-stream-fns";
import { LogStreamHeaders, TAuditLogStreamServiceFactory } from "./audit-log-stream-types";
type TAuditLogStreamServiceFactoryDep = {
@ -69,10 +70,11 @@ export const auditLogStreamServiceFactory = ({
headers.forEach(({ key, value }) => {
streamHeaders[key] = value;
});
await request
.post(
url,
{ ping: "ok" },
{ ...providerSpecificPayload(url), ping: "ok" },
{
headers: streamHeaders,
// request timeout
@ -137,7 +139,7 @@ export const auditLogStreamServiceFactory = ({
await request
.post(
url || logStream.url,
{ ping: "ok" },
{ ...providerSpecificPayload(url || logStream.url), ping: "ok" },
{
headers: streamHeaders,
// request timeout

View File

@ -1,13 +1,15 @@
import { RawAxiosRequestHeaders } from "axios";
import { AxiosError, RawAxiosRequestHeaders } from "axios";
import { SecretKeyEncoding } from "@app/db/schemas";
import { getConfig } from "@app/lib/config/env";
import { request } from "@app/lib/config/request";
import { infisicalSymmetricDecrypt } from "@app/lib/crypto/encryption";
import { logger } from "@app/lib/logger";
import { QueueJobs, QueueName, TQueueServiceFactory } from "@app/queue";
import { TProjectDALFactory } from "@app/services/project/project-dal";
import { TAuditLogStreamDALFactory } from "../audit-log-stream/audit-log-stream-dal";
import { providerSpecificPayload } from "../audit-log-stream/audit-log-stream-fns";
import { LogStreamHeaders } from "../audit-log-stream/audit-log-stream-types";
import { TLicenseServiceFactory } from "../license/license-service";
import { TAuditLogDALFactory } from "./audit-log-dal";
@ -128,13 +130,25 @@ export const auditLogQueueServiceFactory = async ({
headers[key] = value;
});
return request.post(url, auditLog, {
try {
const response = await request.post(
url,
{ ...providerSpecificPayload(url), ...auditLog },
{
headers,
// request timeout
timeout: AUDIT_LOG_STREAM_TIMEOUT,
// connection timeout
signal: AbortSignal.timeout(AUDIT_LOG_STREAM_TIMEOUT)
});
}
);
return response;
} catch (error) {
logger.error(
`Failed to stream audit log [url=${url}] for org [orgId=${orgId}] [error=${(error as AxiosError).message}]`
);
return error;
}
}
)
);
@ -218,13 +232,25 @@ export const auditLogQueueServiceFactory = async ({
headers[key] = value;
});
return request.post(url, auditLog, {
try {
const response = await request.post(
url,
{ ...providerSpecificPayload(url), ...auditLog },
{
headers,
// request timeout
timeout: AUDIT_LOG_STREAM_TIMEOUT,
// connection timeout
signal: AbortSignal.timeout(AUDIT_LOG_STREAM_TIMEOUT)
});
}
);
return response;
} catch (error) {
logger.error(
`Failed to stream audit log [url=${url}] for org [orgId=${orgId}] [error=${(error as AxiosError).message}]`
);
return error;
}
}
)
);

View File

@ -202,6 +202,12 @@ export enum EventType {
REVOKE_IDENTITY_ALICLOUD_AUTH = "revoke-identity-alicloud-auth",
GET_IDENTITY_ALICLOUD_AUTH = "get-identity-alicloud-auth",
LOGIN_IDENTITY_TLS_CERT_AUTH = "login-identity-tls-cert-auth",
ADD_IDENTITY_TLS_CERT_AUTH = "add-identity-tls-cert-auth",
UPDATE_IDENTITY_TLS_CERT_AUTH = "update-identity-tls-cert-auth",
REVOKE_IDENTITY_TLS_CERT_AUTH = "revoke-identity-tls-cert-auth",
GET_IDENTITY_TLS_CERT_AUTH = "get-identity-tls-cert-auth",
LOGIN_IDENTITY_AWS_AUTH = "login-identity-aws-auth",
ADD_IDENTITY_AWS_AUTH = "add-identity-aws-auth",
UPDATE_IDENTITY_AWS_AUTH = "update-identity-aws-auth",
@ -1141,6 +1147,53 @@ interface GetIdentityAliCloudAuthEvent {
};
}
interface LoginIdentityTlsCertAuthEvent {
type: EventType.LOGIN_IDENTITY_TLS_CERT_AUTH;
metadata: {
identityId: string;
identityTlsCertAuthId: string;
identityAccessTokenId: string;
};
}
interface AddIdentityTlsCertAuthEvent {
type: EventType.ADD_IDENTITY_TLS_CERT_AUTH;
metadata: {
identityId: string;
allowedCommonNames: string | null | undefined;
accessTokenTTL: number;
accessTokenMaxTTL: number;
accessTokenNumUsesLimit: number;
accessTokenTrustedIps: Array<TIdentityTrustedIp>;
};
}
interface DeleteIdentityTlsCertAuthEvent {
type: EventType.REVOKE_IDENTITY_TLS_CERT_AUTH;
metadata: {
identityId: string;
};
}
interface UpdateIdentityTlsCertAuthEvent {
type: EventType.UPDATE_IDENTITY_TLS_CERT_AUTH;
metadata: {
identityId: string;
allowedCommonNames: string | null | undefined;
accessTokenTTL?: number;
accessTokenMaxTTL?: number;
accessTokenNumUsesLimit?: number;
accessTokenTrustedIps?: Array<TIdentityTrustedIp>;
};
}
interface GetIdentityTlsCertAuthEvent {
type: EventType.GET_IDENTITY_TLS_CERT_AUTH;
metadata: {
identityId: string;
};
}
interface LoginIdentityOciAuthEvent {
type: EventType.LOGIN_IDENTITY_OCI_AUTH;
metadata: {
@ -3358,6 +3411,11 @@ export type Event =
| UpdateIdentityAliCloudAuthEvent
| GetIdentityAliCloudAuthEvent
| DeleteIdentityAliCloudAuthEvent
| LoginIdentityTlsCertAuthEvent
| AddIdentityTlsCertAuthEvent
| UpdateIdentityTlsCertAuthEvent
| GetIdentityTlsCertAuthEvent
| DeleteIdentityTlsCertAuthEvent
| LoginIdentityOciAuthEvent
| AddIdentityOciAuthEvent
| UpdateIdentityOciAuthEvent

View File

@ -1,17 +1,11 @@
import { ProbotOctokit } from "probot";
import { OrgMembershipRole, TableName } from "@app/db/schemas";
import { getConfig } from "@app/lib/config/env";
import { logger } from "@app/lib/logger";
import { QueueJobs, QueueName, TQueueServiceFactory } from "@app/queue";
import { InternalServerError } from "@app/lib/errors";
import { TQueueServiceFactory } from "@app/queue";
import { TOrgDALFactory } from "@app/services/org/org-dal";
import { SmtpTemplates, TSmtpService } from "@app/services/smtp/smtp-service";
import { TSmtpService } from "@app/services/smtp/smtp-service";
import { TTelemetryServiceFactory } from "@app/services/telemetry/telemetry-service";
import { PostHogEventTypes } from "@app/services/telemetry/telemetry-types";
import { TSecretScanningDALFactory } from "../secret-scanning-dal";
import { scanContentAndGetFindings, scanFullRepoContentAndGetFindings } from "./secret-scanning-fns";
import { SecretMatch, TScanFullRepoEventPayload, TScanPushEventPayload } from "./secret-scanning-queue-types";
import { TScanFullRepoEventPayload, TScanPushEventPayload } from "./secret-scanning-queue-types";
type TSecretScanningQueueFactoryDep = {
queueService: TQueueServiceFactory;
@ -23,227 +17,21 @@ type TSecretScanningQueueFactoryDep = {
export type TSecretScanningQueueFactory = ReturnType<typeof secretScanningQueueFactory>;
export const secretScanningQueueFactory = ({
queueService,
secretScanningDAL,
smtpService,
telemetryService,
orgMembershipDAL: orgMemberDAL
}: TSecretScanningQueueFactoryDep) => {
const startFullRepoScan = async (payload: TScanFullRepoEventPayload) => {
await queueService.queue(QueueName.SecretFullRepoScan, QueueJobs.SecretScan, payload, {
attempts: 3,
backoff: {
type: "exponential",
delay: 5000
},
removeOnComplete: true,
removeOnFail: {
count: 20 // keep the most recent 20 jobs
}
// eslint-disable-next-line @typescript-eslint/no-unused-vars
export const secretScanningQueueFactory = (_props: TSecretScanningQueueFactoryDep) => {
// eslint-disable-next-line @typescript-eslint/no-unused-vars
const startFullRepoScan = async (_payload: TScanFullRepoEventPayload) => {
throw new InternalServerError({
message: "Secret Scanning V1 has been deprecated. Please migrate to Secret Scanning V2"
});
};
const startPushEventScan = async (payload: TScanPushEventPayload) => {
await queueService.queue(QueueName.SecretPushEventScan, QueueJobs.SecretScan, payload, {
attempts: 3,
backoff: {
type: "exponential",
delay: 5000
},
removeOnComplete: true,
removeOnFail: {
count: 20 // keep the most recent 20 jobs
}
// eslint-disable-next-line @typescript-eslint/no-unused-vars
const startPushEventScan = async (_payload: TScanPushEventPayload) => {
throw new InternalServerError({
message: "Secret Scanning V1 has been deprecated. Please migrate to Secret Scanning V2"
});
};
const getOrgAdminEmails = async (organizationId: string) => {
// get emails of admins
const adminsOfWork = await orgMemberDAL.findMembership({
[`${TableName.Organization}.id` as string]: organizationId,
role: OrgMembershipRole.Admin
});
return adminsOfWork.filter((userObject) => userObject.email).map((userObject) => userObject.email as string);
};
queueService.start(QueueName.SecretPushEventScan, async (job) => {
const appCfg = getConfig();
const { organizationId, commits, pusher, repository, installationId } = job.data;
const [owner, repo] = repository.fullName.split("/");
const octokit = new ProbotOctokit({
auth: {
appId: appCfg.SECRET_SCANNING_GIT_APP_ID,
privateKey: appCfg.SECRET_SCANNING_PRIVATE_KEY,
installationId
}
});
const allFindingsByFingerprint: { [key: string]: SecretMatch } = {};
for (const commit of commits) {
for (const filepath of [...commit.added, ...commit.modified]) {
// eslint-disable-next-line
const fileContentsResponse = await octokit.repos.getContent({
owner,
repo,
path: filepath
});
const { data } = fileContentsResponse;
const fileContent = Buffer.from((data as { content: string }).content, "base64").toString();
// eslint-disable-next-line
const findings = await scanContentAndGetFindings(`\n${fileContent}`); // extra line to count lines correctly
for (const finding of findings) {
const fingerPrintWithCommitId = `${commit.id}:${filepath}:${finding.RuleID}:${finding.StartLine}`;
const fingerPrintWithoutCommitId = `${filepath}:${finding.RuleID}:${finding.StartLine}`;
finding.Fingerprint = fingerPrintWithCommitId;
finding.FingerPrintWithoutCommitId = fingerPrintWithoutCommitId;
finding.Commit = commit.id;
finding.File = filepath;
finding.Author = commit.author.name;
finding.Email = commit?.author?.email ? commit?.author?.email : "";
allFindingsByFingerprint[fingerPrintWithCommitId] = finding;
}
}
}
await secretScanningDAL.transaction(async (tx) => {
if (!Object.keys(allFindingsByFingerprint).length) return;
await secretScanningDAL.upsert(
Object.keys(allFindingsByFingerprint).map((key) => ({
installationId,
email: allFindingsByFingerprint[key].Email,
author: allFindingsByFingerprint[key].Author,
date: allFindingsByFingerprint[key].Date,
file: allFindingsByFingerprint[key].File,
tags: allFindingsByFingerprint[key].Tags,
commit: allFindingsByFingerprint[key].Commit,
ruleID: allFindingsByFingerprint[key].RuleID,
endLine: String(allFindingsByFingerprint[key].EndLine),
entropy: String(allFindingsByFingerprint[key].Entropy),
message: allFindingsByFingerprint[key].Message,
endColumn: String(allFindingsByFingerprint[key].EndColumn),
startLine: String(allFindingsByFingerprint[key].StartLine),
startColumn: String(allFindingsByFingerprint[key].StartColumn),
fingerPrintWithoutCommitId: allFindingsByFingerprint[key].FingerPrintWithoutCommitId,
description: allFindingsByFingerprint[key].Description,
symlinkFile: allFindingsByFingerprint[key].SymlinkFile,
orgId: organizationId,
pusherEmail: pusher.email,
pusherName: pusher.name,
repositoryFullName: repository.fullName,
repositoryId: String(repository.id),
fingerprint: allFindingsByFingerprint[key].Fingerprint
})),
tx
);
});
const adminEmails = await getOrgAdminEmails(organizationId);
if (pusher?.email) {
adminEmails.push(pusher.email);
}
if (Object.keys(allFindingsByFingerprint).length) {
await smtpService.sendMail({
template: SmtpTemplates.SecretLeakIncident,
subjectLine: `Incident alert: leaked secrets found in Github repository ${repository.fullName}`,
recipients: adminEmails.filter((email) => email).map((email) => email),
substitutions: {
numberOfSecrets: Object.keys(allFindingsByFingerprint).length,
pusher_email: pusher.email,
pusher_name: pusher.name
}
});
}
await telemetryService.sendPostHogEvents({
event: PostHogEventTypes.SecretScannerPush,
distinctId: repository.fullName,
properties: {
numberOfRisks: Object.keys(allFindingsByFingerprint).length
}
});
});
queueService.start(QueueName.SecretFullRepoScan, async (job) => {
const appCfg = getConfig();
const { organizationId, repository, installationId } = job.data;
const octokit = new ProbotOctokit({
auth: {
appId: appCfg.SECRET_SCANNING_GIT_APP_ID,
privateKey: appCfg.SECRET_SCANNING_PRIVATE_KEY,
installationId
}
});
const findings = await scanFullRepoContentAndGetFindings(
// this is because of collision of octokit in probot and github
// eslint-disable-next-line
octokit as any,
installationId,
repository.fullName
);
await secretScanningDAL.transaction(async (tx) => {
if (!findings.length) return;
// eslint-disable-next-line
await secretScanningDAL.upsert(
findings.map((finding) => ({
installationId,
email: finding.Email,
author: finding.Author,
date: finding.Date,
file: finding.File,
tags: finding.Tags,
commit: finding.Commit,
ruleID: finding.RuleID,
endLine: String(finding.EndLine),
entropy: String(finding.Entropy),
message: finding.Message,
endColumn: String(finding.EndColumn),
startLine: String(finding.StartLine),
startColumn: String(finding.StartColumn),
fingerPrintWithoutCommitId: finding.FingerPrintWithoutCommitId,
description: finding.Description,
symlinkFile: finding.SymlinkFile,
orgId: organizationId,
repositoryFullName: repository.fullName,
repositoryId: String(repository.id),
fingerprint: finding.Fingerprint
})),
tx
);
});
const adminEmails = await getOrgAdminEmails(organizationId);
if (findings.length) {
await smtpService.sendMail({
template: SmtpTemplates.SecretLeakIncident,
subjectLine: `Incident alert: leaked secrets found in Github repository ${repository.fullName}`,
recipients: adminEmails.filter((email) => email).map((email) => email),
substitutions: {
numberOfSecrets: findings.length
}
});
}
await telemetryService.sendPostHogEvents({
event: PostHogEventTypes.SecretScannerFull,
distinctId: repository.fullName,
properties: {
numberOfRisks: findings.length
}
});
});
queueService.listen(QueueName.SecretPushEventScan, "failed", (job, err) => {
logger.error(err, "Failed to secret scan on push", job?.data);
});
queueService.listen(QueueName.SecretFullRepoScan, "failed", (job, err) => {
logger.error(err, "Failed to do full repo secret scan", job?.data);
});
return { startFullRepoScan, startPushEventScan };
};

View File

@ -98,6 +98,7 @@ export const secretScanningServiceFactory = ({
if (canUseSecretScanning(actorOrgId)) {
await Promise.all(
repositories.map(({ id, full_name }) =>
// eslint-disable-next-line @typescript-eslint/no-unsafe-call,@typescript-eslint/no-unsafe-return,@typescript-eslint/no-unsafe-member-access
secretScanningQueue.startFullRepoScan({
organizationId: session.orgId,
installationId,
@ -180,6 +181,7 @@ export const secretScanningServiceFactory = ({
if (!installationLink) return;
if (canUseSecretScanning(installationLink.orgId)) {
// eslint-disable-next-line @typescript-eslint/no-unsafe-call,@typescript-eslint/no-unsafe-return,@typescript-eslint/no-unsafe-member-access
await secretScanningQueue.startPushEventScan({
commits,
pusher: { name: pusher.name, email: pusher.email },

View File

@ -73,6 +73,7 @@ type TWaitTillReady = {
export type TKeyStoreFactory = {
setItem: (key: string, value: string | number | Buffer, prefix?: string) => Promise<"OK">;
getItem: (key: string, prefix?: string) => Promise<string | null>;
getItems: (keys: string[], prefix?: string) => Promise<(string | null)[]>;
setExpiry: (key: string, expiryInSeconds: number) => Promise<number>;
setItemWithExpiry: (
key: string,
@ -81,6 +82,7 @@ export type TKeyStoreFactory = {
prefix?: string
) => Promise<"OK">;
deleteItem: (key: string) => Promise<number>;
deleteItemsByKeyIn: (keys: string[]) => Promise<number>;
deleteItems: (arg: TDeleteItems) => Promise<number>;
incrementBy: (key: string, value: number) => Promise<number>;
acquireLock(
@ -89,6 +91,7 @@ export type TKeyStoreFactory = {
settings?: Partial<Settings>
): Promise<{ release: () => Promise<ExecutionResult> }>;
waitTillReady: ({ key, waitingCb, keyCheckCb, waitIteration, delay, jitter }: TWaitTillReady) => Promise<void>;
getKeysByPattern: (pattern: string, limit?: number) => Promise<string[]>;
};
export const keyStoreFactory = (redisConfigKeys: TRedisConfigKeys): TKeyStoreFactory => {
@ -100,6 +103,9 @@ export const keyStoreFactory = (redisConfigKeys: TRedisConfigKeys): TKeyStoreFac
const getItem = async (key: string, prefix?: string) => redis.get(prefix ? `${prefix}:${key}` : key);
const getItems = async (keys: string[], prefix?: string) =>
redis.mget(keys.map((key) => (prefix ? `${prefix}:${key}` : key)));
const setItemWithExpiry = async (
key: string,
expiryInSeconds: number | string,
@ -109,6 +115,11 @@ export const keyStoreFactory = (redisConfigKeys: TRedisConfigKeys): TKeyStoreFac
const deleteItem = async (key: string) => redis.del(key);
const deleteItemsByKeyIn = async (keys: string[]) => {
if (keys.length === 0) return 0;
return redis.del(keys);
};
const deleteItems = async ({ pattern, batchSize = 500, delay = 1500, jitter = 200 }: TDeleteItems) => {
let cursor = "0";
let totalDeleted = 0;
@ -164,6 +175,24 @@ export const keyStoreFactory = (redisConfigKeys: TRedisConfigKeys): TKeyStoreFac
}
};
const getKeysByPattern = async (pattern: string, limit?: number) => {
let cursor = "0";
const allKeys: string[] = [];
do {
// eslint-disable-next-line no-await-in-loop
const [nextCursor, keys] = await redis.scan(cursor, "MATCH", pattern, "COUNT", 1000);
cursor = nextCursor;
allKeys.push(...keys);
if (limit && allKeys.length >= limit) {
return allKeys.slice(0, limit);
}
} while (cursor !== "0");
return allKeys;
};
return {
setItem,
getItem,
@ -175,6 +204,9 @@ export const keyStoreFactory = (redisConfigKeys: TRedisConfigKeys): TKeyStoreFac
acquireLock(resources: string[], duration: number, settings?: Partial<Settings>) {
return redisLock.acquire(resources, duration, settings);
},
waitTillReady
waitTillReady,
getKeysByPattern,
deleteItemsByKeyIn,
getItems
};
};

View File

@ -8,6 +8,8 @@ import { TKeyStoreFactory } from "./keystore";
export const inMemoryKeyStore = (): TKeyStoreFactory => {
const store: Record<string, string | number | Buffer> = {};
const getRegex = (pattern: string) =>
new RE2(`^${pattern.replace(/[-[\]/{}()+?.\\^$|]/g, "\\$&").replace(/\*/g, ".*")}$`);
return {
setItem: async (key, value) => {
@ -24,7 +26,7 @@ export const inMemoryKeyStore = (): TKeyStoreFactory => {
return 1;
},
deleteItems: async ({ pattern, batchSize = 500, delay = 1500, jitter = 200 }) => {
const regex = new RE2(`^${pattern.replace(/[-[\]/{}()+?.\\^$|]/g, "\\$&").replace(/\*/g, ".*")}$`);
const regex = getRegex(pattern);
let totalDeleted = 0;
const keys = Object.keys(store);
@ -59,6 +61,27 @@ export const inMemoryKeyStore = (): TKeyStoreFactory => {
release: () => {}
}) as Promise<Lock>;
},
waitTillReady: async () => {}
waitTillReady: async () => {},
getKeysByPattern: async (pattern) => {
const regex = getRegex(pattern);
const keys = Object.keys(store);
return keys.filter((key) => regex.test(key));
},
deleteItemsByKeyIn: async (keys) => {
for (const key of keys) {
delete store[key];
}
return keys.length;
},
getItems: async (keys) => {
const values = keys.map((key) => {
const value = store[key];
if (typeof value === "string") {
return value;
}
return null;
});
return values;
}
};
};

View File

@ -22,6 +22,7 @@ export enum ApiDocsTags {
UniversalAuth = "Universal Auth",
GcpAuth = "GCP Auth",
AliCloudAuth = "Alibaba Cloud Auth",
TlsCertAuth = "TLS Certificate Auth",
AwsAuth = "AWS Auth",
OciAuth = "OCI Auth",
AzureAuth = "Azure Auth",
@ -283,6 +284,38 @@ export const ALICLOUD_AUTH = {
}
} as const;
export const TLS_CERT_AUTH = {
LOGIN: {
identityId: "The ID of the identity to login."
},
ATTACH: {
identityId: "The ID of the identity to attach the configuration onto.",
allowedCommonNames:
"The comma-separated list of trusted common names that are allowed to authenticate with Infisical.",
caCertificate: "The PEM-encoded CA certificate to validate client certificates.",
accessTokenTTL: "The lifetime for an access token in seconds.",
accessTokenMaxTTL: "The maximum lifetime for an access token in seconds.",
accessTokenNumUsesLimit: "The maximum number of times that an access token can be used.",
accessTokenTrustedIps: "The IPs or CIDR ranges that access tokens can be used from."
},
UPDATE: {
identityId: "The ID of the identity to update the auth method for.",
allowedCommonNames:
"The comma-separated list of trusted common names that are allowed to authenticate with Infisical.",
caCertificate: "The PEM-encoded CA certificate to validate client certificates.",
accessTokenTTL: "The new lifetime for an access token in seconds.",
accessTokenMaxTTL: "The new maximum lifetime for an access token in seconds.",
accessTokenNumUsesLimit: "The new maximum number of times that an access token can be used.",
accessTokenTrustedIps: "The new IPs or CIDR ranges that access tokens can be used from."
},
RETRIEVE: {
identityId: "The ID of the identity to retrieve the auth method for."
},
REVOKE: {
identityId: "The ID of the identity to revoke the auth method for."
}
} as const;
export const AWS_AUTH = {
LOGIN: {
identityId: "The ID of the identity to login.",
@ -2228,6 +2261,12 @@ export const AppConnections = {
},
FLYIO: {
accessToken: "The Access Token used to access fly.io."
},
GITLAB: {
instanceUrl: "The GitLab instance URL to connect with.",
accessToken: "The Access Token used to access GitLab.",
code: "The OAuth code to use to connect with GitLab.",
accessTokenType: "The type of token used to connect with GitLab."
}
}
};
@ -2402,6 +2441,17 @@ export const SecretSyncs = {
FLYIO: {
appId: "The ID of the Fly.io app to sync secrets to."
},
GITLAB: {
projectId: "The GitLab Project ID to sync secrets to.",
projectName: "The GitLab Project Name to sync secrets to.",
groupId: "The GitLab Group ID to sync secrets to.",
groupName: "The GitLab Group Name to sync secrets to.",
scope: "The GitLab scope that secrets should be synced to. (default: project)",
targetEnvironment: "The GitLab environment scope that secrets should be synced to. (default: *)",
shouldProtectSecrets: "Whether variables should be protected",
shouldMaskSecrets: "Whether variables should be masked in logs",
shouldHideSecrets: "Whether variables should be hidden"
},
CLOUDFLARE_PAGES: {
projectName: "The name of the Cloudflare Pages project to sync secrets to.",
environment: "The environment of the Cloudflare Pages project to sync secrets to."

View File

@ -193,6 +193,9 @@ const envSchema = z
PYLON_API_KEY: zpStr(z.string().optional()),
DISABLE_AUDIT_LOG_GENERATION: zodStrBool.default("false"),
SSL_CLIENT_CERTIFICATE_HEADER_KEY: zpStr(z.string().optional()).default("x-ssl-client-cert"),
IDENTITY_TLS_CERT_AUTH_CLIENT_CERTIFICATE_HEADER_KEY: zpStr(z.string().optional()).default(
"x-identity-tls-cert-auth-client-cert"
),
WORKFLOW_SLACK_CLIENT_ID: zpStr(z.string().optional()),
WORKFLOW_SLACK_CLIENT_SECRET: zpStr(z.string().optional()),
ENABLE_MSSQL_SECRET_ROTATION_ENCRYPT: zodStrBool.default("true"),
@ -247,6 +250,10 @@ const envSchema = z
INF_APP_CONNECTION_GITHUB_RADAR_APP_ID: zpStr(z.string().optional()),
INF_APP_CONNECTION_GITHUB_RADAR_APP_WEBHOOK_SECRET: zpStr(z.string().optional()),
// gitlab oauth
INF_APP_CONNECTION_GITLAB_OAUTH_CLIENT_ID: zpStr(z.string().optional()),
INF_APP_CONNECTION_GITLAB_OAUTH_CLIENT_SECRET: zpStr(z.string().optional()),
// gcp app
INF_APP_CONNECTION_GCP_SERVICE_ACCOUNT_CREDENTIAL: zpStr(z.string().optional()),

View File

@ -62,7 +62,8 @@ export enum QueueName {
SecretRotationV2 = "secret-rotation-v2",
FolderTreeCheckpoint = "folder-tree-checkpoint",
InvalidateCache = "invalidate-cache",
SecretScanningV2 = "secret-scanning-v2"
SecretScanningV2 = "secret-scanning-v2",
TelemetryAggregatedEvents = "telemetry-aggregated-events"
}
export enum QueueJobs {
@ -101,7 +102,8 @@ export enum QueueJobs {
SecretScanningV2DiffScan = "secret-scanning-v2-diff-scan",
SecretScanningV2SendNotification = "secret-scanning-v2-notification",
CaOrderCertificateForSubscriber = "ca-order-certificate-for-subscriber",
PkiSubscriberDailyAutoRenewal = "pki-subscriber-daily-auto-renewal"
PkiSubscriberDailyAutoRenewal = "pki-subscriber-daily-auto-renewal",
TelemetryAggregatedEvents = "telemetry-aggregated-events"
}
export type TQueueJobTypes = {
@ -292,6 +294,10 @@ export type TQueueJobTypes = {
name: QueueJobs.PkiSubscriberDailyAutoRenewal;
payload: undefined;
};
[QueueName.TelemetryAggregatedEvents]: {
name: QueueJobs.TelemetryAggregatedEvents;
payload: undefined;
};
};
const SECRET_SCANNING_JOBS = [

View File

@ -193,6 +193,8 @@ import { identityOidcAuthServiceFactory } from "@app/services/identity-oidc-auth
import { identityProjectDALFactory } from "@app/services/identity-project/identity-project-dal";
import { identityProjectMembershipRoleDALFactory } from "@app/services/identity-project/identity-project-membership-role-dal";
import { identityProjectServiceFactory } from "@app/services/identity-project/identity-project-service";
import { identityTlsCertAuthDALFactory } from "@app/services/identity-tls-cert-auth/identity-tls-cert-auth-dal";
import { identityTlsCertAuthServiceFactory } from "@app/services/identity-tls-cert-auth/identity-tls-cert-auth-service";
import { identityTokenAuthDALFactory } from "@app/services/identity-token-auth/identity-token-auth-dal";
import { identityTokenAuthServiceFactory } from "@app/services/identity-token-auth/identity-token-auth-service";
import { identityUaClientSecretDALFactory } from "@app/services/identity-ua/identity-ua-client-secret-dal";
@ -297,7 +299,6 @@ import { injectAssumePrivilege } from "../plugins/auth/inject-assume-privilege";
import { injectIdentity } from "../plugins/auth/inject-identity";
import { injectPermission } from "../plugins/auth/inject-permission";
import { injectRateLimits } from "../plugins/inject-rate-limits";
import { registerSecretScannerGhApp } from "../plugins/secret-scanner";
import { registerV1Routes } from "./v1";
import { registerV2Routes } from "./v2";
import { registerV3Routes } from "./v3";
@ -326,7 +327,6 @@ export const registerRoutes = async (
}
) => {
const appCfg = getConfig();
await server.register(registerSecretScannerGhApp, { prefix: "/ss-webhook" });
await server.register(registerSecretScanningV2Webhooks, {
prefix: "/secret-scanning/webhooks"
});
@ -386,6 +386,7 @@ export const registerRoutes = async (
const identityKubernetesAuthDAL = identityKubernetesAuthDALFactory(db);
const identityUaClientSecretDAL = identityUaClientSecretDALFactory(db);
const identityAliCloudAuthDAL = identityAliCloudAuthDALFactory(db);
const identityTlsCertAuthDAL = identityTlsCertAuthDALFactory(db);
const identityAwsAuthDAL = identityAwsAuthDALFactory(db);
const identityGcpAuthDAL = identityGcpAuthDALFactory(db);
const identityOciAuthDAL = identityOciAuthDALFactory(db);
@ -686,7 +687,8 @@ export const registerRoutes = async (
const telemetryQueue = telemetryQueueServiceFactory({
keyStore,
telemetryDAL,
queueService
queueService,
telemetryService
});
const invalidateCacheQueue = invalidateCacheQueueFactory({
@ -1417,7 +1419,8 @@ export const registerRoutes = async (
const identityAccessTokenService = identityAccessTokenServiceFactory({
identityAccessTokenDAL,
identityOrgMembershipDAL,
accessTokenQueue
accessTokenQueue,
identityDAL
});
const identityProjectService = identityProjectServiceFactory({
@ -1493,6 +1496,15 @@ export const registerRoutes = async (
permissionService
});
const identityTlsCertAuthService = identityTlsCertAuthServiceFactory({
identityAccessTokenDAL,
identityTlsCertAuthDAL,
identityOrgMembershipDAL,
licenseService,
permissionService,
kmsService
});
const identityAwsAuthService = identityAwsAuthServiceFactory({
identityAccessTokenDAL,
identityAwsAuthDAL,
@ -1947,6 +1959,7 @@ export const registerRoutes = async (
identityAwsAuth: identityAwsAuthService,
identityAzureAuth: identityAzureAuthService,
identityOciAuth: identityOciAuthService,
identityTlsCertAuth: identityTlsCertAuthService,
identityOidcAuth: identityOidcAuthService,
identityJwtAuth: identityJwtAuthService,
identityLdapAuth: identityLdapAuthService,

View File

@ -722,6 +722,7 @@ export const registerAdminRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.InvalidateCache,
organizationId: req.permission.orgId,
distinctId: getTelemetryDistinctId(req),
properties: {
...req.auditLogInfo

View File

@ -35,6 +35,10 @@ import {
CamundaConnectionListItemSchema,
SanitizedCamundaConnectionSchema
} from "@app/services/app-connection/camunda";
import {
CloudflareConnectionListItemSchema,
SanitizedCloudflareConnectionSchema
} from "@app/services/app-connection/cloudflare/cloudflare-connection-schema";
import {
DatabricksConnectionListItemSchema,
SanitizedDatabricksConnectionSchema
@ -46,6 +50,7 @@ import {
GitHubRadarConnectionListItemSchema,
SanitizedGitHubRadarConnectionSchema
} from "@app/services/app-connection/github-radar";
import { GitLabConnectionListItemSchema, SanitizedGitLabConnectionSchema } from "@app/services/app-connection/gitlab";
import {
HCVaultConnectionListItemSchema,
SanitizedHCVaultConnectionSchema
@ -80,10 +85,6 @@ import {
WindmillConnectionListItemSchema
} from "@app/services/app-connection/windmill";
import { AuthMode } from "@app/services/auth/auth-type";
import {
CloudflareConnectionListItemSchema,
SanitizedCloudflareConnectionSchema
} from "@app/services/app-connection/cloudflare/cloudflare-connection-schema";
// can't use discriminated due to multiple schemas for certain apps
const SanitizedAppConnectionSchema = z.union([
@ -114,6 +115,7 @@ const SanitizedAppConnectionSchema = z.union([
...SanitizedHerokuConnectionSchema.options,
...SanitizedRenderConnectionSchema.options,
...SanitizedFlyioConnectionSchema.options,
...SanitizedGitLabConnectionSchema.options,
...SanitizedCloudflareConnectionSchema.options
]);
@ -145,6 +147,7 @@ const AppConnectionOptionsSchema = z.discriminatedUnion("app", [
HerokuConnectionListItemSchema,
RenderConnectionListItemSchema,
FlyioConnectionListItemSchema,
GitLabConnectionListItemSchema,
CloudflareConnectionListItemSchema
]);

View File

@ -0,0 +1,90 @@
import z from "zod";
import { readLimit } from "@app/server/config/rateLimiter";
import { verifyAuth } from "@app/server/plugins/auth/verify-auth";
import { AppConnection } from "@app/services/app-connection/app-connection-enums";
import {
CreateGitLabConnectionSchema,
SanitizedGitLabConnectionSchema,
TGitLabGroup,
TGitLabProject,
UpdateGitLabConnectionSchema
} from "@app/services/app-connection/gitlab";
import { AuthMode } from "@app/services/auth/auth-type";
import { registerAppConnectionEndpoints } from "./app-connection-endpoints";
export const registerGitLabConnectionRouter = async (server: FastifyZodProvider) => {
registerAppConnectionEndpoints({
app: AppConnection.GitLab,
server,
sanitizedResponseSchema: SanitizedGitLabConnectionSchema,
createSchema: CreateGitLabConnectionSchema,
updateSchema: UpdateGitLabConnectionSchema
});
// The below endpoints are not exposed and for Infisical App use
server.route({
method: "GET",
url: `/:connectionId/projects`,
config: {
rateLimit: readLimit
},
schema: {
params: z.object({
connectionId: z.string().uuid()
}),
response: {
200: z
.object({
id: z.string(),
name: z.string()
})
.array()
}
},
onRequest: verifyAuth([AuthMode.JWT]),
handler: async (req) => {
const { connectionId } = req.params;
const projects: TGitLabProject[] = await server.services.appConnection.gitlab.listProjects(
connectionId,
req.permission
);
return projects;
}
});
server.route({
method: "GET",
url: `/:connectionId/groups`,
config: {
rateLimit: readLimit
},
schema: {
params: z.object({
connectionId: z.string().uuid()
}),
response: {
200: z
.object({
id: z.string(),
name: z.string()
})
.array()
}
},
onRequest: verifyAuth([AuthMode.JWT]),
handler: async (req) => {
const { connectionId } = req.params;
const groups: TGitLabGroup[] = await server.services.appConnection.gitlab.listGroups(
connectionId,
req.permission
);
return groups;
}
});
};

View File

@ -10,11 +10,13 @@ import { registerAzureClientSecretsConnectionRouter } from "./azure-client-secre
import { registerAzureDevOpsConnectionRouter } from "./azure-devops-connection-router";
import { registerAzureKeyVaultConnectionRouter } from "./azure-key-vault-connection-router";
import { registerCamundaConnectionRouter } from "./camunda-connection-router";
import { registerCloudflareConnectionRouter } from "./cloudflare-connection-router";
import { registerDatabricksConnectionRouter } from "./databricks-connection-router";
import { registerFlyioConnectionRouter } from "./flyio-connection-router";
import { registerGcpConnectionRouter } from "./gcp-connection-router";
import { registerGitHubConnectionRouter } from "./github-connection-router";
import { registerGitHubRadarConnectionRouter } from "./github-radar-connection-router";
import { registerGitLabConnectionRouter } from "./gitlab-connection-router";
import { registerHCVaultConnectionRouter } from "./hc-vault-connection-router";
import { registerHerokuConnectionRouter } from "./heroku-connection-router";
import { registerHumanitecConnectionRouter } from "./humanitec-connection-router";
@ -27,7 +29,6 @@ import { registerTeamCityConnectionRouter } from "./teamcity-connection-router";
import { registerTerraformCloudConnectionRouter } from "./terraform-cloud-router";
import { registerVercelConnectionRouter } from "./vercel-connection-router";
import { registerWindmillConnectionRouter } from "./windmill-connection-router";
import { registerCloudflareConnectionRouter } from "./cloudflare-connection-router";
export * from "./app-connection-router";
@ -60,5 +61,6 @@ export const APP_CONNECTION_REGISTER_ROUTER_MAP: Record<AppConnection, (server:
[AppConnection.Heroku]: registerHerokuConnectionRouter,
[AppConnection.Render]: registerRenderConnectionRouter,
[AppConnection.Flyio]: registerFlyioConnectionRouter,
[AppConnection.GitLab]: registerGitLabConnectionRouter,
[AppConnection.Cloudflare]: registerCloudflareConnectionRouter
};

View File

@ -692,6 +692,7 @@ export const registerCaRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.IssueCert,
organizationId: req.permission.orgId,
distinctId: getTelemetryDistinctId(req),
properties: {
caId: ca.id,
@ -786,6 +787,7 @@ export const registerCaRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SignCert,
organizationId: req.permission.orgId,
distinctId: getTelemetryDistinctId(req),
properties: {
caId: ca.id,

View File

@ -266,6 +266,7 @@ export const registerCertRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.IssueCert,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
caId: req.body.caId,
certificateTemplateId: req.body.certificateTemplateId,
@ -442,6 +443,7 @@ export const registerCertRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SignCert,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
caId: req.body.caId,
certificateTemplateId: req.body.certificateTemplateId,

View File

@ -475,6 +475,7 @@ export const registerDashboardRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretPulled,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
numberOfSecrets: secretCountFromEnv,
workspaceId: projectId,
@ -979,6 +980,7 @@ export const registerDashboardRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretPulled,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
numberOfSecrets: secretCount,
workspaceId: projectId,
@ -1144,6 +1146,7 @@ export const registerDashboardRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretPulled,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
numberOfSecrets: secretCountForEnv,
workspaceId: projectId,
@ -1336,6 +1339,7 @@ export const registerDashboardRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretPulled,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
numberOfSecrets: secrets.length,
workspaceId: projectId,

View File

@ -85,6 +85,7 @@ export const registerIdentityRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.MachineIdentityCreated,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
orgId: req.body.organizationId,
name: identity.name,

View File

@ -0,0 +1,396 @@
import crypto from "node:crypto";
import { z } from "zod";
import { IdentityTlsCertAuthsSchema } from "@app/db/schemas";
import { EventType } from "@app/ee/services/audit-log/audit-log-types";
import { ApiDocsTags, TLS_CERT_AUTH } from "@app/lib/api-docs";
import { getConfig } from "@app/lib/config/env";
import { BadRequestError } from "@app/lib/errors";
import { readLimit, writeLimit } from "@app/server/config/rateLimiter";
import { verifyAuth } from "@app/server/plugins/auth/verify-auth";
import { AuthMode } from "@app/services/auth/auth-type";
import { TIdentityTrustedIp } from "@app/services/identity/identity-types";
import { isSuperAdmin } from "@app/services/super-admin/super-admin-fns";
const validateCommonNames = z
.string()
.min(1)
.trim()
.transform((el) =>
el
.split(",")
.map((i) => i.trim())
.join(",")
);
const validateCaCertificate = (caCert: string) => {
if (!caCert) return true;
try {
// eslint-disable-next-line no-new
new crypto.X509Certificate(caCert);
return true;
} catch (err) {
return false;
}
};
export const registerIdentityTlsCertAuthRouter = async (server: FastifyZodProvider) => {
server.route({
method: "POST",
url: "/login",
config: {
rateLimit: writeLimit
},
schema: {
hide: false,
tags: [ApiDocsTags.TlsCertAuth],
description: "Login with TLS Certificate Auth",
body: z.object({
identityId: z.string().trim().describe(TLS_CERT_AUTH.LOGIN.identityId)
}),
response: {
200: z.object({
accessToken: z.string(),
expiresIn: z.coerce.number(),
accessTokenMaxTTL: z.coerce.number(),
tokenType: z.literal("Bearer")
})
}
},
handler: async (req) => {
const appCfg = getConfig();
const clientCertificate = req.headers[appCfg.IDENTITY_TLS_CERT_AUTH_CLIENT_CERTIFICATE_HEADER_KEY];
if (!clientCertificate) {
throw new BadRequestError({ message: "Missing TLS certificate in header" });
}
const { identityTlsCertAuth, accessToken, identityAccessToken, identityMembershipOrg } =
await server.services.identityTlsCertAuth.login({
identityId: req.body.identityId,
clientCertificate: clientCertificate as string
});
await server.services.auditLog.createAuditLog({
...req.auditLogInfo,
orgId: identityMembershipOrg?.orgId,
event: {
type: EventType.LOGIN_IDENTITY_TLS_CERT_AUTH,
metadata: {
identityId: identityTlsCertAuth.identityId,
identityAccessTokenId: identityAccessToken.id,
identityTlsCertAuthId: identityTlsCertAuth.id
}
}
});
return {
accessToken,
tokenType: "Bearer" as const,
expiresIn: identityTlsCertAuth.accessTokenTTL,
accessTokenMaxTTL: identityTlsCertAuth.accessTokenMaxTTL
};
}
});
server.route({
method: "POST",
url: "/identities/:identityId",
config: {
rateLimit: writeLimit
},
onRequest: verifyAuth([AuthMode.JWT, AuthMode.IDENTITY_ACCESS_TOKEN]),
schema: {
hide: false,
tags: [ApiDocsTags.TlsCertAuth],
description: "Attach TLS Certificate Auth configuration onto identity",
security: [
{
bearerAuth: []
}
],
params: z.object({
identityId: z.string().trim().describe(TLS_CERT_AUTH.ATTACH.identityId)
}),
body: z
.object({
allowedCommonNames: validateCommonNames
.optional()
.nullable()
.describe(TLS_CERT_AUTH.ATTACH.allowedCommonNames),
caCertificate: z
.string()
.min(1)
.max(10240)
.refine(validateCaCertificate, "Invalid CA Certificate.")
.describe(TLS_CERT_AUTH.ATTACH.caCertificate),
accessTokenTrustedIps: z
.object({
ipAddress: z.string().trim()
})
.array()
.min(1)
.default([{ ipAddress: "0.0.0.0/0" }, { ipAddress: "::/0" }])
.describe(TLS_CERT_AUTH.ATTACH.accessTokenTrustedIps),
accessTokenTTL: z
.number()
.int()
.min(0)
.max(315360000)
.default(2592000)
.describe(TLS_CERT_AUTH.ATTACH.accessTokenTTL),
accessTokenMaxTTL: z
.number()
.int()
.min(1)
.max(315360000)
.default(2592000)
.describe(TLS_CERT_AUTH.ATTACH.accessTokenMaxTTL),
accessTokenNumUsesLimit: z
.number()
.int()
.min(0)
.default(0)
.describe(TLS_CERT_AUTH.ATTACH.accessTokenNumUsesLimit)
})
.refine(
(val) => val.accessTokenTTL <= val.accessTokenMaxTTL,
"Access Token TTL cannot be greater than Access Token Max TTL."
),
response: {
200: z.object({
identityTlsCertAuth: IdentityTlsCertAuthsSchema
})
}
},
handler: async (req) => {
const identityTlsCertAuth = await server.services.identityTlsCertAuth.attachTlsCertAuth({
actor: req.permission.type,
actorId: req.permission.id,
actorAuthMethod: req.permission.authMethod,
actorOrgId: req.permission.orgId,
...req.body,
identityId: req.params.identityId,
isActorSuperAdmin: isSuperAdmin(req.auth)
});
await server.services.auditLog.createAuditLog({
...req.auditLogInfo,
orgId: req.permission.orgId,
event: {
type: EventType.ADD_IDENTITY_TLS_CERT_AUTH,
metadata: {
identityId: identityTlsCertAuth.identityId,
allowedCommonNames: identityTlsCertAuth.allowedCommonNames,
accessTokenTTL: identityTlsCertAuth.accessTokenTTL,
accessTokenMaxTTL: identityTlsCertAuth.accessTokenMaxTTL,
accessTokenTrustedIps: identityTlsCertAuth.accessTokenTrustedIps as TIdentityTrustedIp[],
accessTokenNumUsesLimit: identityTlsCertAuth.accessTokenNumUsesLimit
}
}
});
return { identityTlsCertAuth };
}
});
server.route({
method: "PATCH",
url: "/identities/:identityId",
config: {
rateLimit: writeLimit
},
onRequest: verifyAuth([AuthMode.JWT, AuthMode.IDENTITY_ACCESS_TOKEN]),
schema: {
hide: false,
tags: [ApiDocsTags.TlsCertAuth],
description: "Update TLS Certificate Auth configuration on identity",
security: [
{
bearerAuth: []
}
],
params: z.object({
identityId: z.string().describe(TLS_CERT_AUTH.UPDATE.identityId)
}),
body: z
.object({
caCertificate: z
.string()
.min(1)
.max(10240)
.refine(validateCaCertificate, "Invalid CA Certificate.")
.optional()
.describe(TLS_CERT_AUTH.UPDATE.caCertificate),
allowedCommonNames: validateCommonNames
.optional()
.nullable()
.describe(TLS_CERT_AUTH.UPDATE.allowedCommonNames),
accessTokenTrustedIps: z
.object({
ipAddress: z.string().trim()
})
.array()
.min(1)
.optional()
.describe(TLS_CERT_AUTH.UPDATE.accessTokenTrustedIps),
accessTokenTTL: z
.number()
.int()
.min(0)
.max(315360000)
.optional()
.describe(TLS_CERT_AUTH.UPDATE.accessTokenTTL),
accessTokenNumUsesLimit: z
.number()
.int()
.min(0)
.optional()
.describe(TLS_CERT_AUTH.UPDATE.accessTokenNumUsesLimit),
accessTokenMaxTTL: z
.number()
.int()
.max(315360000)
.min(0)
.optional()
.describe(TLS_CERT_AUTH.UPDATE.accessTokenMaxTTL)
})
.refine(
(val) => (val.accessTokenMaxTTL && val.accessTokenTTL ? val.accessTokenTTL <= val.accessTokenMaxTTL : true),
"Access Token TTL cannot be greater than Access Token Max TTL."
),
response: {
200: z.object({
identityTlsCertAuth: IdentityTlsCertAuthsSchema
})
}
},
handler: async (req) => {
const identityTlsCertAuth = await server.services.identityTlsCertAuth.updateTlsCertAuth({
actor: req.permission.type,
actorId: req.permission.id,
actorAuthMethod: req.permission.authMethod,
actorOrgId: req.permission.orgId,
...req.body,
identityId: req.params.identityId
});
await server.services.auditLog.createAuditLog({
...req.auditLogInfo,
orgId: req.permission.orgId,
event: {
type: EventType.UPDATE_IDENTITY_TLS_CERT_AUTH,
metadata: {
identityId: identityTlsCertAuth.identityId,
allowedCommonNames: identityTlsCertAuth.allowedCommonNames,
accessTokenTTL: identityTlsCertAuth.accessTokenTTL,
accessTokenMaxTTL: identityTlsCertAuth.accessTokenMaxTTL,
accessTokenTrustedIps: identityTlsCertAuth.accessTokenTrustedIps as TIdentityTrustedIp[],
accessTokenNumUsesLimit: identityTlsCertAuth.accessTokenNumUsesLimit
}
}
});
return { identityTlsCertAuth };
}
});
server.route({
method: "GET",
url: "/identities/:identityId",
config: {
rateLimit: readLimit
},
onRequest: verifyAuth([AuthMode.JWT, AuthMode.IDENTITY_ACCESS_TOKEN]),
schema: {
hide: false,
tags: [ApiDocsTags.TlsCertAuth],
description: "Retrieve TLS Certificate Auth configuration on identity",
security: [
{
bearerAuth: []
}
],
params: z.object({
identityId: z.string().describe(TLS_CERT_AUTH.RETRIEVE.identityId)
}),
response: {
200: z.object({
identityTlsCertAuth: IdentityTlsCertAuthsSchema.extend({
caCertificate: z.string()
})
})
}
},
handler: async (req) => {
const identityTlsCertAuth = await server.services.identityTlsCertAuth.getTlsCertAuth({
identityId: req.params.identityId,
actor: req.permission.type,
actorId: req.permission.id,
actorOrgId: req.permission.orgId,
actorAuthMethod: req.permission.authMethod
});
await server.services.auditLog.createAuditLog({
...req.auditLogInfo,
orgId: req.permission.orgId,
event: {
type: EventType.GET_IDENTITY_TLS_CERT_AUTH,
metadata: {
identityId: identityTlsCertAuth.identityId
}
}
});
return { identityTlsCertAuth };
}
});
server.route({
method: "DELETE",
url: "/identities/:identityId",
config: {
rateLimit: writeLimit
},
onRequest: verifyAuth([AuthMode.JWT, AuthMode.IDENTITY_ACCESS_TOKEN]),
schema: {
hide: false,
tags: [ApiDocsTags.TlsCertAuth],
description: "Delete TLS Certificate Auth configuration on identity",
security: [
{
bearerAuth: []
}
],
params: z.object({
identityId: z.string().describe(TLS_CERT_AUTH.REVOKE.identityId)
}),
response: {
200: z.object({
identityTlsCertAuth: IdentityTlsCertAuthsSchema
})
}
},
handler: async (req) => {
const identityTlsCertAuth = await server.services.identityTlsCertAuth.revokeTlsCertAuth({
actor: req.permission.type,
actorId: req.permission.id,
actorAuthMethod: req.permission.authMethod,
actorOrgId: req.permission.orgId,
identityId: req.params.identityId
});
await server.services.auditLog.createAuditLog({
...req.auditLogInfo,
orgId: req.permission.orgId,
event: {
type: EventType.REVOKE_IDENTITY_TLS_CERT_AUTH,
metadata: {
identityId: identityTlsCertAuth.identityId
}
}
});
return { identityTlsCertAuth };
}
});
};

View File

@ -25,6 +25,7 @@ import { registerIdentityLdapAuthRouter } from "./identity-ldap-auth-router";
import { registerIdentityOciAuthRouter } from "./identity-oci-auth-router";
import { registerIdentityOidcAuthRouter } from "./identity-oidc-auth-router";
import { registerIdentityRouter } from "./identity-router";
import { registerIdentityTlsCertAuthRouter } from "./identity-tls-cert-auth-router";
import { registerIdentityTokenAuthRouter } from "./identity-token-auth-router";
import { registerIdentityUaRouter } from "./identity-universal-auth-router";
import { registerIntegrationAuthRouter } from "./integration-auth-router";
@ -66,6 +67,7 @@ export const registerV1Routes = async (server: FastifyZodProvider) => {
await authRouter.register(registerIdentityAccessTokenRouter);
await authRouter.register(registerIdentityAliCloudAuthRouter);
await authRouter.register(registerIdentityAwsAuthRouter);
await authRouter.register(registerIdentityTlsCertAuthRouter, { prefix: "/tls-cert-auth" });
await authRouter.register(registerIdentityAzureAuthRouter);
await authRouter.register(registerIdentityOciAuthRouter);
await authRouter.register(registerIdentityOidcAuthRouter);

View File

@ -103,6 +103,7 @@ export const registerIntegrationRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.IntegrationCreated,
organizationId: req.permission.orgId,
distinctId: getTelemetryDistinctId(req),
properties: {
...createIntegrationEventProperty,

View File

@ -64,6 +64,7 @@ export const registerInviteOrgRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.UserOrgInvitation,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
inviteeEmails: req.body.inviteeEmails,
organizationRoleSlug: req.body.organizationRoleSlug,

View File

@ -331,6 +331,7 @@ export const registerPkiSubscriberRouter = async (server: FastifyZodProvider) =>
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.IssueCert,
organizationId: req.permission.orgId,
distinctId: getTelemetryDistinctId(req),
properties: {
subscriberId: subscriber.id,
@ -399,6 +400,7 @@ export const registerPkiSubscriberRouter = async (server: FastifyZodProvider) =>
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.IssueCert,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
subscriberId: subscriber.id,
commonName: subscriber.commonName,
@ -471,6 +473,7 @@ export const registerPkiSubscriberRouter = async (server: FastifyZodProvider) =>
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SignCert,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
subscriberId: subscriber.id,
commonName: subscriber.commonName,

View File

@ -165,6 +165,7 @@ export const registerSecretRequestsRouter = async (server: FastifyZodProvider) =
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretRequestDeleted,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
secretRequestId: req.params.id,
organizationId: req.permission.orgId,
@ -256,6 +257,7 @@ export const registerSecretRequestsRouter = async (server: FastifyZodProvider) =
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretRequestCreated,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
secretRequestId: shareRequest.id,
organizationId: req.permission.orgId,

View File

@ -1,10 +1,11 @@
import { SecretSync } from "@app/services/secret-sync/secret-sync-enums";
import { registerSyncSecretsEndpoints } from "./secret-sync-endpoints";
import {
CloudflarePagesSyncSchema,
CreateCloudflarePagesSyncSchema,
UpdateCloudflarePagesSyncSchema
} from "@app/services/secret-sync/cloudflare-pages/cloudflare-pages-schema";
import { SecretSync } from "@app/services/secret-sync/secret-sync-enums";
import { registerSyncSecretsEndpoints } from "./secret-sync-endpoints";
export const registerCloudflarePagesSyncRouter = async (server: FastifyZodProvider) =>
registerSyncSecretsEndpoints({

View File

@ -0,0 +1,13 @@
import { CreateGitLabSyncSchema, GitLabSyncSchema, UpdateGitLabSyncSchema } from "@app/services/secret-sync/gitlab";
import { SecretSync } from "@app/services/secret-sync/secret-sync-enums";
import { registerSyncSecretsEndpoints } from "./secret-sync-endpoints";
export const registerGitLabSyncRouter = async (server: FastifyZodProvider) =>
registerSyncSecretsEndpoints({
destination: SecretSync.GitLab,
server,
responseSchema: GitLabSyncSchema,
createSchema: CreateGitLabSyncSchema,
updateSchema: UpdateGitLabSyncSchema
});

View File

@ -13,6 +13,7 @@ import { registerDatabricksSyncRouter } from "./databricks-sync-router";
import { registerFlyioSyncRouter } from "./flyio-sync-router";
import { registerGcpSyncRouter } from "./gcp-sync-router";
import { registerGitHubSyncRouter } from "./github-sync-router";
import { registerGitLabSyncRouter } from "./gitlab-sync-router";
import { registerHCVaultSyncRouter } from "./hc-vault-sync-router";
import { registerHerokuSyncRouter } from "./heroku-sync-router";
import { registerHumanitecSyncRouter } from "./humanitec-sync-router";
@ -45,5 +46,6 @@ export const SECRET_SYNC_REGISTER_ROUTER_MAP: Record<SecretSync, (server: Fastif
[SecretSync.Heroku]: registerHerokuSyncRouter,
[SecretSync.Render]: registerRenderSyncRouter,
[SecretSync.Flyio]: registerFlyioSyncRouter,
[SecretSync.GitLab]: registerGitLabSyncRouter,
[SecretSync.CloudflarePages]: registerCloudflarePagesSyncRouter
};

View File

@ -22,10 +22,15 @@ import {
import { AzureDevOpsSyncListItemSchema, AzureDevOpsSyncSchema } from "@app/services/secret-sync/azure-devops";
import { AzureKeyVaultSyncListItemSchema, AzureKeyVaultSyncSchema } from "@app/services/secret-sync/azure-key-vault";
import { CamundaSyncListItemSchema, CamundaSyncSchema } from "@app/services/secret-sync/camunda";
import {
CloudflarePagesSyncListItemSchema,
CloudflarePagesSyncSchema
} from "@app/services/secret-sync/cloudflare-pages/cloudflare-pages-schema";
import { DatabricksSyncListItemSchema, DatabricksSyncSchema } from "@app/services/secret-sync/databricks";
import { FlyioSyncListItemSchema, FlyioSyncSchema } from "@app/services/secret-sync/flyio";
import { GcpSyncListItemSchema, GcpSyncSchema } from "@app/services/secret-sync/gcp";
import { GitHubSyncListItemSchema, GitHubSyncSchema } from "@app/services/secret-sync/github";
import { GitLabSyncListItemSchema, GitLabSyncSchema } from "@app/services/secret-sync/gitlab";
import { HCVaultSyncListItemSchema, HCVaultSyncSchema } from "@app/services/secret-sync/hc-vault";
import { HerokuSyncListItemSchema, HerokuSyncSchema } from "@app/services/secret-sync/heroku";
import { HumanitecSyncListItemSchema, HumanitecSyncSchema } from "@app/services/secret-sync/humanitec";
@ -34,10 +39,6 @@ import { TeamCitySyncListItemSchema, TeamCitySyncSchema } from "@app/services/se
import { TerraformCloudSyncListItemSchema, TerraformCloudSyncSchema } from "@app/services/secret-sync/terraform-cloud";
import { VercelSyncListItemSchema, VercelSyncSchema } from "@app/services/secret-sync/vercel";
import { WindmillSyncListItemSchema, WindmillSyncSchema } from "@app/services/secret-sync/windmill";
import {
CloudflarePagesSyncListItemSchema,
CloudflarePagesSyncSchema
} from "@app/services/secret-sync/cloudflare-pages/cloudflare-pages-schema";
const SecretSyncSchema = z.discriminatedUnion("destination", [
AwsParameterStoreSyncSchema,
@ -60,6 +61,7 @@ const SecretSyncSchema = z.discriminatedUnion("destination", [
HerokuSyncSchema,
RenderSyncSchema,
FlyioSyncSchema,
GitLabSyncSchema,
CloudflarePagesSyncSchema
]);
@ -84,6 +86,7 @@ const SecretSyncOptionsSchema = z.discriminatedUnion("destination", [
HerokuSyncListItemSchema,
RenderSyncListItemSchema,
FlyioSyncListItemSchema,
GitLabSyncListItemSchema,
CloudflarePagesSyncListItemSchema
]);

View File

@ -198,6 +198,7 @@ export const registerProjectRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.ProjectCreated,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
orgId: project.orgId,
name: project.name,

View File

@ -333,6 +333,7 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretPulled,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
numberOfSecrets: secrets.length,
workspaceId,
@ -489,6 +490,7 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
if (getUserAgentType(req.headers["user-agent"]) !== UserAgentType.K8_OPERATOR) {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretPulled,
organizationId: req.permission.orgId,
distinctId: getTelemetryDistinctId(req),
properties: {
numberOfSecrets: 1,
@ -615,6 +617,7 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretCreated,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
numberOfSecrets: 1,
workspaceId: projectId,
@ -750,6 +753,7 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretUpdated,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
numberOfSecrets: 1,
workspaceId: projectId,
@ -850,6 +854,7 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretDeleted,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
numberOfSecrets: 1,
workspaceId: projectId,
@ -957,6 +962,7 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretPulled,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
numberOfSecrets: secrets.length,
workspaceId: req.query.workspaceId,
@ -1036,6 +1042,7 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretPulled,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
numberOfSecrets: 1,
workspaceId: req.query.workspaceId,
@ -1207,6 +1214,7 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretCreated,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
numberOfSecrets: 1,
workspaceId: req.body.workspaceId,
@ -1396,6 +1404,7 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretUpdated,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
numberOfSecrets: 1,
workspaceId: req.body.workspaceId,
@ -1519,6 +1528,7 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretDeleted,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
numberOfSecrets: 1,
workspaceId: req.body.workspaceId,
@ -1702,6 +1712,7 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretCreated,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
numberOfSecrets: secrets.length,
workspaceId: req.body.workspaceId,
@ -1828,6 +1839,7 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretUpdated,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
numberOfSecrets: secrets.length,
workspaceId: req.body.workspaceId,
@ -1946,6 +1958,7 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretDeleted,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
numberOfSecrets: secrets.length,
workspaceId: req.body.workspaceId,
@ -2054,6 +2067,7 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretCreated,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
numberOfSecrets: secrets.length,
workspaceId: secrets[0].workspace,
@ -2209,6 +2223,7 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretUpdated,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
numberOfSecrets: secrets.length,
workspaceId: secrets[0].workspace,
@ -2307,6 +2322,7 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretDeleted,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
numberOfSecrets: secrets.length,
workspaceId: secrets[0].workspace,

View File

@ -26,6 +26,7 @@ export enum AppConnection {
Heroku = "heroku",
Render = "render",
Flyio = "flyio",
GitLab = "gitlab",
Cloudflare = "cloudflare"
}

View File

@ -51,6 +51,11 @@ import {
validateAzureKeyVaultConnectionCredentials
} from "./azure-key-vault";
import { CamundaConnectionMethod, getCamundaConnectionListItem, validateCamundaConnectionCredentials } from "./camunda";
import { CloudflareConnectionMethod } from "./cloudflare/cloudflare-connection-enum";
import {
getCloudflareConnectionListItem,
validateCloudflareConnectionCredentials
} from "./cloudflare/cloudflare-connection-fns";
import {
DatabricksConnectionMethod,
getDatabricksConnectionListItem,
@ -64,6 +69,7 @@ import {
GitHubRadarConnectionMethod,
validateGitHubRadarConnectionCredentials
} from "./github-radar";
import { getGitLabConnectionListItem, GitLabConnectionMethod, validateGitLabConnectionCredentials } from "./gitlab";
import {
getHCVaultConnectionListItem,
HCVaultConnectionMethod,
@ -99,11 +105,6 @@ import {
validateWindmillConnectionCredentials,
WindmillConnectionMethod
} from "./windmill";
import {
getCloudflareConnectionListItem,
validateCloudflareConnectionCredentials
} from "./cloudflare/cloudflare-connection-fns";
import { CloudflareConnectionMethod } from "./cloudflare/cloudflare-connection-enum";
export const listAppConnectionOptions = () => {
return [
@ -134,6 +135,7 @@ export const listAppConnectionOptions = () => {
getHerokuConnectionListItem(),
getRenderConnectionListItem(),
getFlyioConnectionListItem(),
getGitLabConnectionListItem(),
getCloudflareConnectionListItem()
].sort((a, b) => a.name.localeCompare(b.name));
};
@ -213,6 +215,7 @@ export const validateAppConnectionCredentials = async (
[AppConnection.Heroku]: validateHerokuConnectionCredentials as TAppConnectionCredentialsValidator,
[AppConnection.Render]: validateRenderConnectionCredentials as TAppConnectionCredentialsValidator,
[AppConnection.Flyio]: validateFlyioConnectionCredentials as TAppConnectionCredentialsValidator,
[AppConnection.GitLab]: validateGitLabConnectionCredentials as TAppConnectionCredentialsValidator,
[AppConnection.Cloudflare]: validateCloudflareConnectionCredentials as TAppConnectionCredentialsValidator
};
@ -230,6 +233,7 @@ export const getAppConnectionMethodName = (method: TAppConnection["method"]) =>
case GitHubConnectionMethod.OAuth:
case AzureDevOpsConnectionMethod.OAuth:
case HerokuConnectionMethod.OAuth:
case GitLabConnectionMethod.OAuth:
return "OAuth";
case HerokuConnectionMethod.AuthToken:
return "Auth Token";
@ -327,6 +331,7 @@ export const TRANSITION_CONNECTION_CREDENTIALS_TO_PLATFORM: Record<
[AppConnection.Heroku]: platformManagedCredentialsNotSupported,
[AppConnection.Render]: platformManagedCredentialsNotSupported,
[AppConnection.Flyio]: platformManagedCredentialsNotSupported,
[AppConnection.GitLab]: platformManagedCredentialsNotSupported,
[AppConnection.Cloudflare]: platformManagedCredentialsNotSupported
};

View File

@ -28,6 +28,7 @@ export const APP_CONNECTION_NAME_MAP: Record<AppConnection, string> = {
[AppConnection.Heroku]: "Heroku",
[AppConnection.Render]: "Render",
[AppConnection.Flyio]: "Fly.io",
[AppConnection.GitLab]: "GitLab",
[AppConnection.Cloudflare]: "Cloudflare"
};
@ -59,5 +60,6 @@ export const APP_CONNECTION_PLAN_MAP: Record<AppConnection, AppConnectionPlanTyp
[AppConnection.Heroku]: AppConnectionPlanType.Regular,
[AppConnection.Render]: AppConnectionPlanType.Regular,
[AppConnection.Flyio]: AppConnectionPlanType.Regular,
[AppConnection.GitLab]: AppConnectionPlanType.Regular,
[AppConnection.Cloudflare]: AppConnectionPlanType.Regular
};

View File

@ -58,6 +58,8 @@ import { gcpConnectionService } from "./gcp/gcp-connection-service";
import { ValidateGitHubConnectionCredentialsSchema } from "./github";
import { githubConnectionService } from "./github/github-connection-service";
import { ValidateGitHubRadarConnectionCredentialsSchema } from "./github-radar";
import { ValidateGitLabConnectionCredentialsSchema } from "./gitlab";
import { gitlabConnectionService } from "./gitlab/gitlab-connection-service";
import { ValidateHCVaultConnectionCredentialsSchema } from "./hc-vault";
import { hcVaultConnectionService } from "./hc-vault/hc-vault-connection-service";
import { ValidateHerokuConnectionCredentialsSchema } from "./heroku";
@ -116,6 +118,7 @@ const VALIDATE_APP_CONNECTION_CREDENTIALS_MAP: Record<AppConnection, TValidateAp
[AppConnection.Heroku]: ValidateHerokuConnectionCredentialsSchema,
[AppConnection.Render]: ValidateRenderConnectionCredentialsSchema,
[AppConnection.Flyio]: ValidateFlyioConnectionCredentialsSchema,
[AppConnection.GitLab]: ValidateGitLabConnectionCredentialsSchema,
[AppConnection.Cloudflare]: ValidateCloudflareConnectionCredentialsSchema
};
@ -524,7 +527,8 @@ export const appConnectionServiceFactory = ({
onepass: onePassConnectionService(connectAppConnectionById),
heroku: herokuConnectionService(connectAppConnectionById, appConnectionDAL, kmsService),
render: renderConnectionService(connectAppConnectionById),
cloudflare: cloudflareConnectionService(connectAppConnectionById),
flyio: flyioConnectionService(connectAppConnectionById)
flyio: flyioConnectionService(connectAppConnectionById),
gitlab: gitlabConnectionService(connectAppConnectionById, appConnectionDAL, kmsService),
cloudflare: cloudflareConnectionService(connectAppConnectionById)
};
};

View File

@ -62,6 +62,12 @@ import {
TCamundaConnectionInput,
TValidateCamundaConnectionCredentialsSchema
} from "./camunda";
import {
TCloudflareConnection,
TCloudflareConnectionConfig,
TCloudflareConnectionInput,
TValidateCloudflareConnectionCredentialsSchema
} from "./cloudflare/cloudflare-connection-types";
import {
TDatabricksConnection,
TDatabricksConnectionConfig,
@ -92,6 +98,12 @@ import {
TGitHubRadarConnectionInput,
TValidateGitHubRadarConnectionCredentialsSchema
} from "./github-radar";
import {
TGitLabConnection,
TGitLabConnectionConfig,
TGitLabConnectionInput,
TValidateGitLabConnectionCredentialsSchema
} from "./gitlab";
import {
THCVaultConnection,
THCVaultConnectionConfig,
@ -153,12 +165,6 @@ import {
TWindmillConnectionConfig,
TWindmillConnectionInput
} from "./windmill";
import {
TCloudflareConnection,
TCloudflareConnectionConfig,
TCloudflareConnectionInput,
TValidateCloudflareConnectionCredentialsSchema
} from "./cloudflare/cloudflare-connection-types";
export type TAppConnection = { id: string } & (
| TAwsConnection
@ -188,6 +194,7 @@ export type TAppConnection = { id: string } & (
| THerokuConnection
| TRenderConnection
| TFlyioConnection
| TGitLabConnection
| TCloudflareConnection
);
@ -223,6 +230,7 @@ export type TAppConnectionInput = { id: string } & (
| THerokuConnectionInput
| TRenderConnectionInput
| TFlyioConnectionInput
| TGitLabConnectionInput
| TCloudflareConnectionInput
);
@ -266,6 +274,7 @@ export type TAppConnectionConfig =
| THerokuConnectionConfig
| TRenderConnectionConfig
| TFlyioConnectionConfig
| TGitLabConnectionConfig
| TCloudflareConnectionConfig;
export type TValidateAppConnectionCredentialsSchema =
@ -296,6 +305,7 @@ export type TValidateAppConnectionCredentialsSchema =
| TValidateHerokuConnectionCredentialsSchema
| TValidateRenderConnectionCredentialsSchema
| TValidateFlyioConnectionCredentialsSchema
| TValidateGitLabConnectionCredentialsSchema
| TValidateCloudflareConnectionCredentialsSchema;
export type TListAwsConnectionKmsKeys = {

View File

@ -1,6 +1,7 @@
import z from "zod";
import { AppConnections } from "@app/lib/api-docs";
import { CharacterType, characterValidator } from "@app/lib/validator/validate-string";
import { AppConnection } from "@app/services/app-connection/app-connection-enums";
import {
BaseAppConnectionSchema,
@ -9,7 +10,6 @@ import {
} from "@app/services/app-connection/app-connection-schemas";
import { CloudflareConnectionMethod } from "./cloudflare-connection-enum";
import { CharacterType, characterValidator } from "@app/lib/validator/validate-string";
const accountIdCharacterValidator = characterValidator([
CharacterType.AlphaNumeric,

View File

@ -1,3 +1,4 @@
import { logger } from "@app/lib/logger";
import { OrgServiceActor } from "@app/lib/types";
import { AppConnection } from "../app-connection-enums";
@ -19,6 +20,7 @@ export const gcpConnectionService = (getAppConnection: TGetAppConnectionFunc) =>
return projects;
} catch (error) {
logger.error(error, "Error listing GCP secret manager projects");
return [];
}
};

View File

@ -0,0 +1,9 @@
export enum GitLabConnectionMethod {
OAuth = "oauth",
AccessToken = "access-token"
}
export enum GitLabAccessTokenType {
Project = "project",
Personal = "personal"
}

View File

@ -0,0 +1,351 @@
/* eslint-disable no-await-in-loop */
import { GitbeakerRequestError, Gitlab } from "@gitbeaker/rest";
import { AxiosError } from "axios";
import { getConfig } from "@app/lib/config/env";
import { request } from "@app/lib/config/request";
import { BadRequestError, InternalServerError } from "@app/lib/errors";
import { removeTrailingSlash } from "@app/lib/fn";
import { logger } from "@app/lib/logger";
import { blockLocalAndPrivateIpAddresses } from "@app/lib/validator";
import { AppConnection } from "@app/services/app-connection/app-connection-enums";
import { encryptAppConnectionCredentials } from "@app/services/app-connection/app-connection-fns";
import { IntegrationUrls } from "@app/services/integration-auth/integration-list";
import { TKmsServiceFactory } from "@app/services/kms/kms-service";
import { TAppConnectionDALFactory } from "../app-connection-dal";
import { GitLabAccessTokenType, GitLabConnectionMethod } from "./gitlab-connection-enums";
import { TGitLabConnection, TGitLabConnectionConfig, TGitLabGroup, TGitLabProject } from "./gitlab-connection-types";
interface GitLabOAuthTokenResponse {
access_token: string;
token_type: string;
expires_in: number;
refresh_token: string;
created_at: number;
scope?: string;
}
export const getGitLabConnectionListItem = () => {
const { INF_APP_CONNECTION_GITLAB_OAUTH_CLIENT_ID } = getConfig();
return {
name: "GitLab" as const,
app: AppConnection.GitLab as const,
methods: Object.values(GitLabConnectionMethod) as [
GitLabConnectionMethod.AccessToken,
GitLabConnectionMethod.OAuth
],
oauthClientId: INF_APP_CONNECTION_GITLAB_OAUTH_CLIENT_ID
};
};
export const getGitLabInstanceUrl = async (instanceUrl?: string) => {
const gitLabInstanceUrl = instanceUrl ? removeTrailingSlash(instanceUrl) : IntegrationUrls.GITLAB_URL;
await blockLocalAndPrivateIpAddresses(gitLabInstanceUrl);
return gitLabInstanceUrl;
};
export const getGitLabClient = async (accessToken: string, instanceUrl?: string, isOAuth = false) => {
const host = await getGitLabInstanceUrl(instanceUrl);
const client = new Gitlab<true>({
host,
...(isOAuth ? { oauthToken: accessToken } : { token: accessToken }),
camelize: true
});
return client;
};
export const refreshGitLabToken = async (
refreshToken: string,
appId: string,
orgId: string,
appConnectionDAL: Pick<TAppConnectionDALFactory, "updateById">,
kmsService: Pick<TKmsServiceFactory, "createCipherPairWithDataKey">,
instanceUrl?: string
): Promise<string> => {
const { INF_APP_CONNECTION_GITLAB_OAUTH_CLIENT_ID, INF_APP_CONNECTION_GITLAB_OAUTH_CLIENT_SECRET, SITE_URL } =
getConfig();
if (!INF_APP_CONNECTION_GITLAB_OAUTH_CLIENT_SECRET || !INF_APP_CONNECTION_GITLAB_OAUTH_CLIENT_ID || !SITE_URL) {
throw new InternalServerError({
message: `GitLab environment variables have not been configured`
});
}
const payload = new URLSearchParams({
grant_type: "refresh_token",
refresh_token: refreshToken,
client_id: INF_APP_CONNECTION_GITLAB_OAUTH_CLIENT_ID,
client_secret: INF_APP_CONNECTION_GITLAB_OAUTH_CLIENT_SECRET,
redirect_uri: `${SITE_URL}/organization/app-connections/gitlab/oauth/callback`
});
try {
const url = await getGitLabInstanceUrl(instanceUrl);
const { data } = await request.post<GitLabOAuthTokenResponse>(`${url}/oauth/token`, payload.toString(), {
headers: {
"Content-Type": "application/x-www-form-urlencoded",
Accept: "application/json"
}
});
const expiresAt = new Date(Date.now() + data.expires_in * 1000 - 600000);
const encryptedCredentials = await encryptAppConnectionCredentials({
credentials: {
instanceUrl,
tokenType: data.token_type,
createdAt: new Date(data.created_at * 1000).toISOString(),
refreshToken: data.refresh_token,
accessToken: data.access_token,
expiresAt
},
orgId,
kmsService
});
await appConnectionDAL.updateById(appId, { encryptedCredentials });
return data.access_token;
} catch (error: unknown) {
if (error instanceof AxiosError) {
throw new BadRequestError({
message: `Failed to refresh GitLab token: ${error.message}`
});
}
throw new BadRequestError({
message: "Unable to refresh GitLab token"
});
}
};
export const exchangeGitLabOAuthCode = async (
code: string,
instanceUrl?: string
): Promise<GitLabOAuthTokenResponse> => {
const { INF_APP_CONNECTION_GITLAB_OAUTH_CLIENT_ID, INF_APP_CONNECTION_GITLAB_OAUTH_CLIENT_SECRET, SITE_URL } =
getConfig();
if (!INF_APP_CONNECTION_GITLAB_OAUTH_CLIENT_SECRET || !INF_APP_CONNECTION_GITLAB_OAUTH_CLIENT_ID || !SITE_URL) {
throw new InternalServerError({
message: `GitLab environment variables have not been configured`
});
}
try {
const payload = new URLSearchParams({
grant_type: "authorization_code",
code,
client_id: INF_APP_CONNECTION_GITLAB_OAUTH_CLIENT_ID,
client_secret: INF_APP_CONNECTION_GITLAB_OAUTH_CLIENT_SECRET,
redirect_uri: `${SITE_URL}/organization/app-connections/gitlab/oauth/callback`
});
const url = await getGitLabInstanceUrl(instanceUrl);
const response = await request.post<GitLabOAuthTokenResponse>(`${url}/oauth/token`, payload.toString(), {
headers: {
"Content-Type": "application/x-www-form-urlencoded",
Accept: "application/json"
}
});
if (!response.data) {
throw new InternalServerError({
message: "Failed to exchange OAuth code: Empty response"
});
}
return response.data;
} catch (error: unknown) {
if (error instanceof AxiosError) {
throw new BadRequestError({
message: `Failed to exchange OAuth code: ${error.message}`
});
}
throw new BadRequestError({
message: "Unable to exchange OAuth code"
});
}
};
export const validateGitLabConnectionCredentials = async (config: TGitLabConnectionConfig) => {
const { credentials: inputCredentials, method } = config;
let accessToken: string;
let oauthData: GitLabOAuthTokenResponse | null = null;
if (method === GitLabConnectionMethod.OAuth && "code" in inputCredentials) {
oauthData = await exchangeGitLabOAuthCode(inputCredentials.code, inputCredentials.instanceUrl);
accessToken = oauthData.access_token;
} else if (method === GitLabConnectionMethod.AccessToken && "accessToken" in inputCredentials) {
accessToken = inputCredentials.accessToken;
} else {
throw new BadRequestError({
message: "Invalid credentials for the selected connection method"
});
}
try {
const client = await getGitLabClient(
accessToken,
inputCredentials.instanceUrl,
method === GitLabConnectionMethod.OAuth
);
await client.Users.showCurrentUser();
} catch (error: unknown) {
logger.error(error, "Error validating GitLab connection credentials");
if (error instanceof GitbeakerRequestError) {
throw new BadRequestError({
message: `Failed to validate credentials: ${error.message ?? "Unknown error"}${error.cause?.description && error.message !== "Unauthorized" ? `. Cause: ${error.cause.description}` : ""}`
});
}
throw new BadRequestError({
message: `Failed to validate credentials: ${(error as Error)?.message || "verify credentials"}`
});
}
if (method === GitLabConnectionMethod.OAuth && oauthData) {
return {
accessToken,
instanceUrl: inputCredentials.instanceUrl,
refreshToken: oauthData.refresh_token,
expiresAt: new Date(Date.now() + oauthData.expires_in * 1000 - 60000),
tokenType: oauthData.token_type,
createdAt: new Date(oauthData.created_at * 1000)
};
}
return inputCredentials;
};
export const listGitLabProjects = async ({
appConnection,
appConnectionDAL,
kmsService
}: {
appConnection: TGitLabConnection;
appConnectionDAL: Pick<TAppConnectionDALFactory, "updateById">;
kmsService: Pick<TKmsServiceFactory, "createCipherPairWithDataKey">;
}): Promise<TGitLabProject[]> => {
let { accessToken } = appConnection.credentials;
if (
appConnection.method === GitLabConnectionMethod.OAuth &&
appConnection.credentials.refreshToken &&
new Date(appConnection.credentials.expiresAt) < new Date()
) {
accessToken = await refreshGitLabToken(
appConnection.credentials.refreshToken,
appConnection.id,
appConnection.orgId,
appConnectionDAL,
kmsService,
appConnection.credentials.instanceUrl
);
}
try {
const client = await getGitLabClient(
accessToken,
appConnection.credentials.instanceUrl,
appConnection.method === GitLabConnectionMethod.OAuth
);
const projects = await client.Projects.all({
archived: false,
includePendingDelete: false,
membership: true,
includeHidden: false,
imported: false
});
return projects.map((project) => ({
name: project.pathWithNamespace,
id: project.id.toString()
}));
} catch (error: unknown) {
if (error instanceof GitbeakerRequestError) {
throw new BadRequestError({
message: `Failed to fetch GitLab projects: ${error.message ?? "Unknown error"}${error.cause?.description && error.message !== "Unauthorized" ? `. Cause: ${error.cause.description}` : ""}`
});
}
if (error instanceof InternalServerError) {
throw error;
}
throw new InternalServerError({
message: "Unable to fetch GitLab projects"
});
}
};
export const listGitLabGroups = async ({
appConnection,
appConnectionDAL,
kmsService
}: {
appConnection: TGitLabConnection;
appConnectionDAL: Pick<TAppConnectionDALFactory, "updateById">;
kmsService: Pick<TKmsServiceFactory, "createCipherPairWithDataKey">;
}): Promise<TGitLabGroup[]> => {
let { accessToken } = appConnection.credentials;
if (
appConnection.method === GitLabConnectionMethod.AccessToken &&
appConnection.credentials.accessTokenType === GitLabAccessTokenType.Project
) {
return [];
}
if (
appConnection.method === GitLabConnectionMethod.OAuth &&
appConnection.credentials.refreshToken &&
new Date(appConnection.credentials.expiresAt) < new Date()
) {
accessToken = await refreshGitLabToken(
appConnection.credentials.refreshToken,
appConnection.id,
appConnection.orgId,
appConnectionDAL,
kmsService,
appConnection.credentials.instanceUrl
);
}
try {
const client = await getGitLabClient(
accessToken,
appConnection.credentials.instanceUrl,
appConnection.method === GitLabConnectionMethod.OAuth
);
const groups = await client.Groups.all({
orderBy: "name",
sort: "asc",
minAccessLevel: 50
});
return groups.map((group) => ({
id: group.id.toString(),
name: group.name
}));
} catch (error: unknown) {
if (error instanceof GitbeakerRequestError) {
throw new BadRequestError({
message: `Failed to fetch GitLab groups: ${error.message ?? "Unknown error"}${error.cause?.description && error.message !== "Unauthorized" ? `. Cause: ${error.cause.description}` : ""}`
});
}
if (error instanceof InternalServerError) {
throw error;
}
throw new InternalServerError({
message: "Unable to fetch GitLab groups"
});
}
};

View File

@ -0,0 +1,138 @@
import z from "zod";
import { AppConnections } from "@app/lib/api-docs";
import { AppConnection } from "@app/services/app-connection/app-connection-enums";
import {
BaseAppConnectionSchema,
GenericCreateAppConnectionFieldsSchema,
GenericUpdateAppConnectionFieldsSchema
} from "@app/services/app-connection/app-connection-schemas";
import { GitLabAccessTokenType, GitLabConnectionMethod } from "./gitlab-connection-enums";
export const GitLabConnectionAccessTokenCredentialsSchema = z.object({
accessToken: z
.string()
.trim()
.min(1, "Access Token required")
.describe(AppConnections.CREDENTIALS.GITLAB.accessToken),
instanceUrl: z
.string()
.trim()
.url("Invalid Instance URL")
.optional()
.describe(AppConnections.CREDENTIALS.GITLAB.instanceUrl),
accessTokenType: z.nativeEnum(GitLabAccessTokenType).describe(AppConnections.CREDENTIALS.GITLAB.accessTokenType)
});
export const GitLabConnectionOAuthCredentialsSchema = z.object({
code: z.string().trim().min(1, "OAuth code required").describe(AppConnections.CREDENTIALS.GITLAB.code),
instanceUrl: z
.string()
.trim()
.url("Invalid Instance URL")
.optional()
.describe(AppConnections.CREDENTIALS.GITLAB.instanceUrl)
});
export const GitLabConnectionOAuthOutputCredentialsSchema = z.object({
accessToken: z.string().trim(),
refreshToken: z.string().trim(),
expiresAt: z.date(),
tokenType: z.string().optional().default("bearer"),
createdAt: z.string().optional(),
instanceUrl: z
.string()
.trim()
.url("Invalid Instance URL")
.optional()
.describe(AppConnections.CREDENTIALS.GITLAB.instanceUrl)
});
export const GitLabConnectionRefreshTokenCredentialsSchema = z.object({
refreshToken: z.string().trim().min(1, "Refresh token required"),
instanceUrl: z
.string()
.trim()
.url("Invalid Instance URL")
.optional()
.describe(AppConnections.CREDENTIALS.GITLAB.instanceUrl)
});
const BaseGitLabConnectionSchema = BaseAppConnectionSchema.extend({
app: z.literal(AppConnection.GitLab)
});
export const GitLabConnectionSchema = z.intersection(
BaseGitLabConnectionSchema,
z.discriminatedUnion("method", [
z.object({
method: z.literal(GitLabConnectionMethod.AccessToken),
credentials: GitLabConnectionAccessTokenCredentialsSchema
}),
z.object({
method: z.literal(GitLabConnectionMethod.OAuth),
credentials: GitLabConnectionOAuthOutputCredentialsSchema
})
])
);
export const SanitizedGitLabConnectionSchema = z.discriminatedUnion("method", [
BaseGitLabConnectionSchema.extend({
method: z.literal(GitLabConnectionMethod.AccessToken),
credentials: GitLabConnectionAccessTokenCredentialsSchema.pick({
instanceUrl: true,
accessTokenType: true
})
}),
BaseGitLabConnectionSchema.extend({
method: z.literal(GitLabConnectionMethod.OAuth),
credentials: GitLabConnectionOAuthOutputCredentialsSchema.pick({
instanceUrl: true
})
})
]);
export const ValidateGitLabConnectionCredentialsSchema = z.discriminatedUnion("method", [
z.object({
method: z.literal(GitLabConnectionMethod.AccessToken).describe(AppConnections.CREATE(AppConnection.GitLab).method),
credentials: GitLabConnectionAccessTokenCredentialsSchema.describe(
AppConnections.CREATE(AppConnection.GitLab).credentials
)
}),
z.object({
method: z.literal(GitLabConnectionMethod.OAuth).describe(AppConnections.CREATE(AppConnection.GitLab).method),
credentials: z
.union([
GitLabConnectionOAuthCredentialsSchema,
GitLabConnectionRefreshTokenCredentialsSchema,
GitLabConnectionOAuthOutputCredentialsSchema
])
.describe(AppConnections.CREATE(AppConnection.GitLab).credentials)
})
]);
export const CreateGitLabConnectionSchema = ValidateGitLabConnectionCredentialsSchema.and(
GenericCreateAppConnectionFieldsSchema(AppConnection.GitLab)
);
export const UpdateGitLabConnectionSchema = z
.object({
credentials: z
.union([
GitLabConnectionAccessTokenCredentialsSchema,
GitLabConnectionOAuthOutputCredentialsSchema,
GitLabConnectionRefreshTokenCredentialsSchema,
GitLabConnectionOAuthCredentialsSchema
])
.optional()
.describe(AppConnections.UPDATE(AppConnection.GitLab).credentials)
})
.and(GenericUpdateAppConnectionFieldsSchema(AppConnection.GitLab));
export const GitLabConnectionListItemSchema = z.object({
name: z.literal("GitLab"),
app: z.literal(AppConnection.GitLab),
methods: z.nativeEnum(GitLabConnectionMethod).array(),
oauthClientId: z.string().optional()
});

View File

@ -0,0 +1,47 @@
import { logger } from "@app/lib/logger";
import { OrgServiceActor } from "@app/lib/types";
import { TKmsServiceFactory } from "@app/services/kms/kms-service";
import { TAppConnectionDALFactory } from "../app-connection-dal";
import { AppConnection } from "../app-connection-enums";
import { listGitLabGroups, listGitLabProjects } from "./gitlab-connection-fns";
import { TGitLabConnection } from "./gitlab-connection-types";
type TGetAppConnectionFunc = (
app: AppConnection,
connectionId: string,
actor: OrgServiceActor
) => Promise<TGitLabConnection>;
export const gitlabConnectionService = (
getAppConnection: TGetAppConnectionFunc,
appConnectionDAL: Pick<TAppConnectionDALFactory, "updateById">,
kmsService: Pick<TKmsServiceFactory, "createCipherPairWithDataKey">
) => {
const listProjects = async (connectionId: string, actor: OrgServiceActor) => {
try {
const appConnection = await getAppConnection(AppConnection.GitLab, connectionId, actor);
const projects = await listGitLabProjects({ appConnection, appConnectionDAL, kmsService });
return projects;
} catch (error) {
logger.error(error, `Failed to establish connection with GitLab for app ${connectionId}`);
return [];
}
};
const listGroups = async (connectionId: string, actor: OrgServiceActor) => {
try {
const appConnection = await getAppConnection(AppConnection.GitLab, connectionId, actor);
const groups = await listGitLabGroups({ appConnection, appConnectionDAL, kmsService });
return groups;
} catch (error) {
logger.error(error, `Failed to establish connection with GitLab for app ${connectionId}`);
return [];
}
};
return {
listProjects,
listGroups
};
};

View File

@ -0,0 +1,56 @@
import z from "zod";
import { DiscriminativePick } from "@app/lib/types";
import { AppConnection } from "../app-connection-enums";
import {
CreateGitLabConnectionSchema,
GitLabConnectionSchema,
ValidateGitLabConnectionCredentialsSchema
} from "./gitlab-connection-schemas";
export type TGitLabConnection = z.infer<typeof GitLabConnectionSchema>;
export type TGitLabConnectionInput = z.infer<typeof CreateGitLabConnectionSchema> & {
app: AppConnection.GitLab;
};
export type TValidateGitLabConnectionCredentialsSchema = typeof ValidateGitLabConnectionCredentialsSchema;
export type TGitLabConnectionConfig = DiscriminativePick<TGitLabConnectionInput, "method" | "app" | "credentials"> & {
orgId: string;
};
export type TGitLabProject = {
name: string;
id: string;
};
export type TGitLabAccessTokenCredentials = {
accessToken: string;
instanceUrl: string;
};
export type TGitLabOAuthCredentials = {
accessToken: string;
refreshToken: string;
expiresAt: Date;
tokenType?: string;
createdAt?: Date;
instanceUrl: string;
};
export type TGitLabOAuthCodeCredentials = {
code: string;
instanceUrl: string;
};
export type TGitLabRefreshTokenCredentials = {
refreshToken: string;
instanceUrl: string;
};
export interface TGitLabGroup {
id: string;
name: string;
}

View File

@ -0,0 +1,4 @@
export * from "./gitlab-connection-enums";
export * from "./gitlab-connection-fns";
export * from "./gitlab-connection-schemas";
export * from "./gitlab-connection-types";

View File

@ -17,70 +17,11 @@ export const identityAccessTokenDALFactory = (db: TDbClient) => {
const doc = await (tx || db.replicaNode())(TableName.IdentityAccessToken)
.where(filter)
.join(TableName.Identity, `${TableName.Identity}.id`, `${TableName.IdentityAccessToken}.identityId`)
.leftJoin(
TableName.IdentityUaClientSecret,
`${TableName.IdentityAccessToken}.identityUAClientSecretId`,
`${TableName.IdentityUaClientSecret}.id`
)
.leftJoin(
TableName.IdentityUniversalAuth,
`${TableName.IdentityUaClientSecret}.identityUAId`,
`${TableName.IdentityUniversalAuth}.id`
)
.leftJoin(TableName.IdentityGcpAuth, `${TableName.Identity}.id`, `${TableName.IdentityGcpAuth}.identityId`)
.leftJoin(
TableName.IdentityAliCloudAuth,
`${TableName.Identity}.id`,
`${TableName.IdentityAliCloudAuth}.identityId`
)
.leftJoin(TableName.IdentityAwsAuth, `${TableName.Identity}.id`, `${TableName.IdentityAwsAuth}.identityId`)
.leftJoin(TableName.IdentityAzureAuth, `${TableName.Identity}.id`, `${TableName.IdentityAzureAuth}.identityId`)
.leftJoin(TableName.IdentityLdapAuth, `${TableName.Identity}.id`, `${TableName.IdentityLdapAuth}.identityId`)
.leftJoin(
TableName.IdentityKubernetesAuth,
`${TableName.Identity}.id`,
`${TableName.IdentityKubernetesAuth}.identityId`
)
.leftJoin(TableName.IdentityOciAuth, `${TableName.Identity}.id`, `${TableName.IdentityOciAuth}.identityId`)
.leftJoin(TableName.IdentityOidcAuth, `${TableName.Identity}.id`, `${TableName.IdentityOidcAuth}.identityId`)
.leftJoin(TableName.IdentityTokenAuth, `${TableName.Identity}.id`, `${TableName.IdentityTokenAuth}.identityId`)
.leftJoin(TableName.IdentityJwtAuth, `${TableName.Identity}.id`, `${TableName.IdentityJwtAuth}.identityId`)
.select(selectAllTableCols(TableName.IdentityAccessToken))
.select(
db.ref("accessTokenTrustedIps").withSchema(TableName.IdentityUniversalAuth).as("accessTokenTrustedIpsUa"),
db.ref("accessTokenTrustedIps").withSchema(TableName.IdentityGcpAuth).as("accessTokenTrustedIpsGcp"),
db
.ref("accessTokenTrustedIps")
.withSchema(TableName.IdentityAliCloudAuth)
.as("accessTokenTrustedIpsAliCloud"),
db.ref("accessTokenTrustedIps").withSchema(TableName.IdentityAwsAuth).as("accessTokenTrustedIpsAws"),
db.ref("accessTokenTrustedIps").withSchema(TableName.IdentityAzureAuth).as("accessTokenTrustedIpsAzure"),
db.ref("accessTokenTrustedIps").withSchema(TableName.IdentityKubernetesAuth).as("accessTokenTrustedIpsK8s"),
db.ref("accessTokenTrustedIps").withSchema(TableName.IdentityOciAuth).as("accessTokenTrustedIpsOci"),
db.ref("accessTokenTrustedIps").withSchema(TableName.IdentityOidcAuth).as("accessTokenTrustedIpsOidc"),
db.ref("accessTokenTrustedIps").withSchema(TableName.IdentityTokenAuth).as("accessTokenTrustedIpsToken"),
db.ref("accessTokenTrustedIps").withSchema(TableName.IdentityJwtAuth).as("accessTokenTrustedIpsJwt"),
db.ref("accessTokenTrustedIps").withSchema(TableName.IdentityLdapAuth).as("accessTokenTrustedIpsLdap"),
db.ref("name").withSchema(TableName.Identity)
)
.select(db.ref("name").withSchema(TableName.Identity))
.first();
if (!doc) return;
return {
...doc,
trustedIpsUniversalAuth: doc.accessTokenTrustedIpsUa,
trustedIpsGcpAuth: doc.accessTokenTrustedIpsGcp,
trustedIpsAliCloudAuth: doc.accessTokenTrustedIpsAliCloud,
trustedIpsAwsAuth: doc.accessTokenTrustedIpsAws,
trustedIpsAzureAuth: doc.accessTokenTrustedIpsAzure,
trustedIpsKubernetesAuth: doc.accessTokenTrustedIpsK8s,
trustedIpsOciAuth: doc.accessTokenTrustedIpsOci,
trustedIpsOidcAuth: doc.accessTokenTrustedIpsOidc,
trustedIpsAccessTokenAuth: doc.accessTokenTrustedIpsToken,
trustedIpsAccessJwtAuth: doc.accessTokenTrustedIpsJwt,
trustedIpsAccessLdapAuth: doc.accessTokenTrustedIpsLdap
};
return doc;
} catch (error) {
throw new DatabaseError({ error, name: "IdAccessTokenFindOne" });
}

View File

@ -7,12 +7,14 @@ import { checkIPAgainstBlocklist, TIp } from "@app/lib/ip";
import { TAccessTokenQueueServiceFactory } from "../access-token-queue/access-token-queue";
import { AuthTokenType } from "../auth/auth-type";
import { TIdentityDALFactory } from "../identity/identity-dal";
import { TIdentityOrgDALFactory } from "../identity/identity-org-dal";
import { TIdentityAccessTokenDALFactory } from "./identity-access-token-dal";
import { TIdentityAccessTokenJwtPayload, TRenewAccessTokenDTO } from "./identity-access-token-types";
type TIdentityAccessTokenServiceFactoryDep = {
identityAccessTokenDAL: TIdentityAccessTokenDALFactory;
identityDAL: Pick<TIdentityDALFactory, "getTrustedIpsByAuthMethod">;
identityOrgMembershipDAL: TIdentityOrgDALFactory;
accessTokenQueue: Pick<
TAccessTokenQueueServiceFactory,
@ -25,7 +27,8 @@ export type TIdentityAccessTokenServiceFactory = ReturnType<typeof identityAcces
export const identityAccessTokenServiceFactory = ({
identityAccessTokenDAL,
identityOrgMembershipDAL,
accessTokenQueue
accessTokenQueue,
identityDAL
}: TIdentityAccessTokenServiceFactoryDep) => {
const validateAccessTokenExp = async (identityAccessToken: TIdentityAccessTokens) => {
const {
@ -190,23 +193,11 @@ export const identityAccessTokenServiceFactory = ({
message: "Failed to authorize revoked access token, access token is revoked"
});
const trustedIpsMap: Record<IdentityAuthMethod, unknown> = {
[IdentityAuthMethod.UNIVERSAL_AUTH]: identityAccessToken.trustedIpsUniversalAuth,
[IdentityAuthMethod.GCP_AUTH]: identityAccessToken.trustedIpsGcpAuth,
[IdentityAuthMethod.ALICLOUD_AUTH]: identityAccessToken.trustedIpsAliCloudAuth,
[IdentityAuthMethod.AWS_AUTH]: identityAccessToken.trustedIpsAwsAuth,
[IdentityAuthMethod.OCI_AUTH]: identityAccessToken.trustedIpsOciAuth,
[IdentityAuthMethod.AZURE_AUTH]: identityAccessToken.trustedIpsAzureAuth,
[IdentityAuthMethod.KUBERNETES_AUTH]: identityAccessToken.trustedIpsKubernetesAuth,
[IdentityAuthMethod.OIDC_AUTH]: identityAccessToken.trustedIpsOidcAuth,
[IdentityAuthMethod.TOKEN_AUTH]: identityAccessToken.trustedIpsAccessTokenAuth,
[IdentityAuthMethod.JWT_AUTH]: identityAccessToken.trustedIpsAccessJwtAuth,
[IdentityAuthMethod.LDAP_AUTH]: identityAccessToken.trustedIpsAccessLdapAuth
};
const trustedIps = trustedIpsMap[identityAccessToken.authMethod as IdentityAuthMethod];
if (ipAddress) {
const trustedIps = await identityDAL.getTrustedIpsByAuthMethod(
identityAccessToken.identityId,
identityAccessToken.authMethod as IdentityAuthMethod
);
if (ipAddress && trustedIps) {
checkIPAgainstBlocklist({
ipAddress,
trustedIps: trustedIps as TIp[]

View File

@ -0,0 +1,10 @@
import { TDbClient } from "@app/db";
import { TableName } from "@app/db/schemas";
import { ormify, TOrmify } from "@app/lib/knex";
export type TIdentityTlsCertAuthDALFactory = TOrmify<TableName.IdentityTlsCertAuth>;
export const identityTlsCertAuthDALFactory = (db: TDbClient) => {
const orm = ormify(db, TableName.IdentityTlsCertAuth);
return orm;
};

View File

@ -0,0 +1,423 @@
import crypto from "node:crypto";
import { ForbiddenError } from "@casl/ability";
import jwt from "jsonwebtoken";
import { IdentityAuthMethod } from "@app/db/schemas";
import { TLicenseServiceFactory } from "@app/ee/services/license/license-service";
import { OrgPermissionIdentityActions, OrgPermissionSubjects } from "@app/ee/services/permission/org-permission";
import {
constructPermissionErrorMessage,
validatePrivilegeChangeOperation
} from "@app/ee/services/permission/permission-fns";
import { TPermissionServiceFactory } from "@app/ee/services/permission/permission-service-types";
import { extractX509CertFromChain } from "@app/lib/certificates/extract-certificate";
import { getConfig } from "@app/lib/config/env";
import { BadRequestError, NotFoundError, PermissionBoundaryError, UnauthorizedError } from "@app/lib/errors";
import { extractIPDetails, isValidIpOrCidr } from "@app/lib/ip";
import { ActorType, AuthTokenType } from "../auth/auth-type";
import { TIdentityOrgDALFactory } from "../identity/identity-org-dal";
import { TIdentityAccessTokenDALFactory } from "../identity-access-token/identity-access-token-dal";
import { TIdentityAccessTokenJwtPayload } from "../identity-access-token/identity-access-token-types";
import { TKmsServiceFactory } from "../kms/kms-service";
import { KmsDataKey } from "../kms/kms-types";
import { validateIdentityUpdateForSuperAdminPrivileges } from "../super-admin/super-admin-fns";
import { TIdentityTlsCertAuthDALFactory } from "./identity-tls-cert-auth-dal";
import { TIdentityTlsCertAuthServiceFactory } from "./identity-tls-cert-auth-types";
type TIdentityTlsCertAuthServiceFactoryDep = {
identityAccessTokenDAL: Pick<TIdentityAccessTokenDALFactory, "create" | "delete">;
identityTlsCertAuthDAL: Pick<
TIdentityTlsCertAuthDALFactory,
"findOne" | "transaction" | "create" | "updateById" | "delete"
>;
identityOrgMembershipDAL: Pick<TIdentityOrgDALFactory, "findOne">;
licenseService: Pick<TLicenseServiceFactory, "getPlan">;
permissionService: Pick<TPermissionServiceFactory, "getOrgPermission">;
kmsService: Pick<TKmsServiceFactory, "createCipherPairWithDataKey">;
};
const parseSubjectDetails = (data: string) => {
const values: Record<string, string> = {};
data.split("\n").forEach((el) => {
const [key, value] = el.split("=");
values[key.trim()] = value.trim();
});
return values;
};
export const identityTlsCertAuthServiceFactory = ({
identityAccessTokenDAL,
identityTlsCertAuthDAL,
identityOrgMembershipDAL,
licenseService,
permissionService,
kmsService
}: TIdentityTlsCertAuthServiceFactoryDep): TIdentityTlsCertAuthServiceFactory => {
const login: TIdentityTlsCertAuthServiceFactory["login"] = async ({ identityId, clientCertificate }) => {
const identityTlsCertAuth = await identityTlsCertAuthDAL.findOne({ identityId });
if (!identityTlsCertAuth) {
throw new NotFoundError({
message: "TLS Certificate auth method not found for identity, did you configure TLS Certificate auth?"
});
}
const identityMembershipOrg = await identityOrgMembershipDAL.findOne({
identityId: identityTlsCertAuth.identityId
});
if (!identityMembershipOrg) {
throw new NotFoundError({
message: `Identity organization membership for identity with ID '${identityTlsCertAuth.identityId}' not found`
});
}
const { decryptor } = await kmsService.createCipherPairWithDataKey({
type: KmsDataKey.Organization,
orgId: identityMembershipOrg.orgId
});
const caCertificate = decryptor({
cipherTextBlob: identityTlsCertAuth.encryptedCaCertificate
}).toString();
const leafCertificate = extractX509CertFromChain(decodeURIComponent(clientCertificate))?.[0];
if (!leafCertificate) {
throw new BadRequestError({ message: "Missing client certificate" });
}
const clientCertificateX509 = new crypto.X509Certificate(leafCertificate);
const caCertificateX509 = new crypto.X509Certificate(caCertificate);
const isValidCertificate = clientCertificateX509.verify(caCertificateX509.publicKey);
if (!isValidCertificate)
throw new UnauthorizedError({
message: "Access denied: Certificate not issued by the provided CA."
});
if (new Date(clientCertificateX509.validTo) < new Date()) {
throw new UnauthorizedError({
message: "Access denied: Certificate has expired."
});
}
if (new Date(clientCertificateX509.validFrom) > new Date()) {
throw new UnauthorizedError({
message: "Access denied: Certificate not yet valid."
});
}
const subjectDetails = parseSubjectDetails(clientCertificateX509.subject);
if (identityTlsCertAuth.allowedCommonNames) {
const isValidCommonName = identityTlsCertAuth.allowedCommonNames.split(",").includes(subjectDetails.CN);
if (!isValidCommonName) {
throw new UnauthorizedError({
message: "Access denied: TLS Certificate Auth common name not allowed."
});
}
}
// Generate the token
const identityAccessToken = await identityTlsCertAuthDAL.transaction(async (tx) => {
const newToken = await identityAccessTokenDAL.create(
{
identityId: identityTlsCertAuth.identityId,
isAccessTokenRevoked: false,
accessTokenTTL: identityTlsCertAuth.accessTokenTTL,
accessTokenMaxTTL: identityTlsCertAuth.accessTokenMaxTTL,
accessTokenNumUses: 0,
accessTokenNumUsesLimit: identityTlsCertAuth.accessTokenNumUsesLimit,
authMethod: IdentityAuthMethod.TLS_CERT_AUTH
},
tx
);
return newToken;
});
const appCfg = getConfig();
const accessToken = jwt.sign(
{
identityId: identityTlsCertAuth.identityId,
identityAccessTokenId: identityAccessToken.id,
authTokenType: AuthTokenType.IDENTITY_ACCESS_TOKEN
} as TIdentityAccessTokenJwtPayload,
appCfg.AUTH_SECRET,
Number(identityAccessToken.accessTokenTTL) === 0
? undefined
: {
expiresIn: Number(identityAccessToken.accessTokenTTL)
}
);
return {
identityTlsCertAuth,
accessToken,
identityAccessToken,
identityMembershipOrg
};
};
const attachTlsCertAuth: TIdentityTlsCertAuthServiceFactory["attachTlsCertAuth"] = async ({
identityId,
accessTokenTTL,
accessTokenMaxTTL,
accessTokenNumUsesLimit,
accessTokenTrustedIps,
actorId,
actorAuthMethod,
actor,
actorOrgId,
isActorSuperAdmin,
caCertificate,
allowedCommonNames
}) => {
await validateIdentityUpdateForSuperAdminPrivileges(identityId, isActorSuperAdmin);
const identityMembershipOrg = await identityOrgMembershipDAL.findOne({ identityId });
if (!identityMembershipOrg) throw new NotFoundError({ message: `Failed to find identity with ID ${identityId}` });
if (identityMembershipOrg.identity.authMethods.includes(IdentityAuthMethod.TLS_CERT_AUTH)) {
throw new BadRequestError({
message: "Failed to add TLS Certificate Auth to already configured identity"
});
}
if (accessTokenMaxTTL > 0 && accessTokenTTL > accessTokenMaxTTL) {
throw new BadRequestError({ message: "Access token TTL cannot be greater than max TTL" });
}
const { permission } = await permissionService.getOrgPermission(
actor,
actorId,
identityMembershipOrg.orgId,
actorAuthMethod,
actorOrgId
);
ForbiddenError.from(permission).throwUnlessCan(OrgPermissionIdentityActions.Create, OrgPermissionSubjects.Identity);
const plan = await licenseService.getPlan(identityMembershipOrg.orgId);
const reformattedAccessTokenTrustedIps = accessTokenTrustedIps.map((accessTokenTrustedIp) => {
if (
!plan.ipAllowlisting &&
accessTokenTrustedIp.ipAddress !== "0.0.0.0/0" &&
accessTokenTrustedIp.ipAddress !== "::/0"
)
throw new BadRequestError({
message:
"Failed to add IP access range to access token due to plan restriction. Upgrade plan to add IP access range."
});
if (!isValidIpOrCidr(accessTokenTrustedIp.ipAddress))
throw new BadRequestError({
message: "The IP is not a valid IPv4, IPv6, or CIDR block"
});
return extractIPDetails(accessTokenTrustedIp.ipAddress);
});
const { encryptor } = await kmsService.createCipherPairWithDataKey({
type: KmsDataKey.Organization,
orgId: identityMembershipOrg.orgId
});
const identityTlsCertAuth = await identityTlsCertAuthDAL.transaction(async (tx) => {
const doc = await identityTlsCertAuthDAL.create(
{
identityId: identityMembershipOrg.identityId,
accessTokenMaxTTL,
allowedCommonNames,
accessTokenTTL,
encryptedCaCertificate: encryptor({ plainText: Buffer.from(caCertificate) }).cipherTextBlob,
accessTokenNumUsesLimit,
accessTokenTrustedIps: JSON.stringify(reformattedAccessTokenTrustedIps)
},
tx
);
return doc;
});
return { ...identityTlsCertAuth, orgId: identityMembershipOrg.orgId };
};
const updateTlsCertAuth: TIdentityTlsCertAuthServiceFactory["updateTlsCertAuth"] = async ({
identityId,
caCertificate,
allowedCommonNames,
accessTokenTTL,
accessTokenMaxTTL,
accessTokenNumUsesLimit,
accessTokenTrustedIps,
actorId,
actorAuthMethod,
actor,
actorOrgId
}) => {
const identityMembershipOrg = await identityOrgMembershipDAL.findOne({ identityId });
if (!identityMembershipOrg) throw new NotFoundError({ message: `Failed to find identity with ID ${identityId}` });
if (!identityMembershipOrg.identity.authMethods.includes(IdentityAuthMethod.TLS_CERT_AUTH)) {
throw new NotFoundError({
message: "The identity does not have TLS Certificate Auth attached"
});
}
const identityTlsCertAuth = await identityTlsCertAuthDAL.findOne({ identityId });
if (
(accessTokenMaxTTL || identityTlsCertAuth.accessTokenMaxTTL) > 0 &&
(accessTokenTTL || identityTlsCertAuth.accessTokenTTL) >
(accessTokenMaxTTL || identityTlsCertAuth.accessTokenMaxTTL)
) {
throw new BadRequestError({ message: "Access token TTL cannot be greater than max TTL" });
}
const { permission } = await permissionService.getOrgPermission(
actor,
actorId,
identityMembershipOrg.orgId,
actorAuthMethod,
actorOrgId
);
ForbiddenError.from(permission).throwUnlessCan(OrgPermissionIdentityActions.Edit, OrgPermissionSubjects.Identity);
const plan = await licenseService.getPlan(identityMembershipOrg.orgId);
const reformattedAccessTokenTrustedIps = accessTokenTrustedIps?.map((accessTokenTrustedIp) => {
if (
!plan.ipAllowlisting &&
accessTokenTrustedIp.ipAddress !== "0.0.0.0/0" &&
accessTokenTrustedIp.ipAddress !== "::/0"
)
throw new BadRequestError({
message:
"Failed to add IP access range to access token due to plan restriction. Upgrade plan to add IP access range."
});
if (!isValidIpOrCidr(accessTokenTrustedIp.ipAddress))
throw new BadRequestError({
message: "The IP is not a valid IPv4, IPv6, or CIDR block"
});
return extractIPDetails(accessTokenTrustedIp.ipAddress);
});
const { encryptor } = await kmsService.createCipherPairWithDataKey({
type: KmsDataKey.Organization,
orgId: identityMembershipOrg.orgId
});
const updatedTlsCertAuth = await identityTlsCertAuthDAL.updateById(identityTlsCertAuth.id, {
allowedCommonNames,
encryptedCaCertificate: caCertificate
? encryptor({ plainText: Buffer.from(caCertificate) }).cipherTextBlob
: undefined,
accessTokenMaxTTL,
accessTokenTTL,
accessTokenNumUsesLimit,
accessTokenTrustedIps: reformattedAccessTokenTrustedIps
? JSON.stringify(reformattedAccessTokenTrustedIps)
: undefined
});
return { ...updatedTlsCertAuth, orgId: identityMembershipOrg.orgId };
};
const getTlsCertAuth: TIdentityTlsCertAuthServiceFactory["getTlsCertAuth"] = async ({
identityId,
actorId,
actor,
actorAuthMethod,
actorOrgId
}) => {
const identityMembershipOrg = await identityOrgMembershipDAL.findOne({ identityId });
if (!identityMembershipOrg) throw new NotFoundError({ message: `Failed to find identity with ID ${identityId}` });
if (!identityMembershipOrg.identity.authMethods.includes(IdentityAuthMethod.TLS_CERT_AUTH)) {
throw new BadRequestError({
message: "The identity does not have TLS Certificate Auth attached"
});
}
const identityAuth = await identityTlsCertAuthDAL.findOne({ identityId });
const { permission } = await permissionService.getOrgPermission(
actor,
actorId,
identityMembershipOrg.orgId,
actorAuthMethod,
actorOrgId
);
ForbiddenError.from(permission).throwUnlessCan(OrgPermissionIdentityActions.Read, OrgPermissionSubjects.Identity);
const { decryptor } = await kmsService.createCipherPairWithDataKey({
type: KmsDataKey.Organization,
orgId: identityMembershipOrg.orgId
});
let caCertificate = "";
if (identityAuth.encryptedCaCertificate) {
caCertificate = decryptor({ cipherTextBlob: identityAuth.encryptedCaCertificate }).toString();
}
return { ...identityAuth, caCertificate, orgId: identityMembershipOrg.orgId };
};
const revokeTlsCertAuth: TIdentityTlsCertAuthServiceFactory["revokeTlsCertAuth"] = async ({
identityId,
actorId,
actor,
actorAuthMethod,
actorOrgId
}) => {
const identityMembershipOrg = await identityOrgMembershipDAL.findOne({ identityId });
if (!identityMembershipOrg) throw new NotFoundError({ message: `Failed to find identity with ID ${identityId}` });
if (!identityMembershipOrg.identity.authMethods.includes(IdentityAuthMethod.TLS_CERT_AUTH)) {
throw new BadRequestError({
message: "The identity does not have TLS Certificate auth"
});
}
const { permission, membership } = await permissionService.getOrgPermission(
actor,
actorId,
identityMembershipOrg.orgId,
actorAuthMethod,
actorOrgId
);
ForbiddenError.from(permission).throwUnlessCan(OrgPermissionIdentityActions.Edit, OrgPermissionSubjects.Identity);
const { permission: rolePermission } = await permissionService.getOrgPermission(
ActorType.IDENTITY,
identityMembershipOrg.identityId,
identityMembershipOrg.orgId,
actorAuthMethod,
actorOrgId
);
const permissionBoundary = validatePrivilegeChangeOperation(
membership.shouldUseNewPrivilegeSystem,
OrgPermissionIdentityActions.RevokeAuth,
OrgPermissionSubjects.Identity,
permission,
rolePermission
);
if (!permissionBoundary.isValid)
throw new PermissionBoundaryError({
message: constructPermissionErrorMessage(
"Failed to revoke TLS Certificate auth of identity with more privileged role",
membership.shouldUseNewPrivilegeSystem,
OrgPermissionIdentityActions.RevokeAuth,
OrgPermissionSubjects.Identity
),
details: { missingPermissions: permissionBoundary.missingPermissions }
});
const revokedIdentityTlsCertAuth = await identityTlsCertAuthDAL.transaction(async (tx) => {
const deletedTlsCertAuth = await identityTlsCertAuthDAL.delete({ identityId }, tx);
await identityAccessTokenDAL.delete({ identityId, authMethod: IdentityAuthMethod.TLS_CERT_AUTH }, tx);
return { ...deletedTlsCertAuth?.[0], orgId: identityMembershipOrg.orgId };
});
return revokedIdentityTlsCertAuth;
};
return {
login,
attachTlsCertAuth,
updateTlsCertAuth,
getTlsCertAuth,
revokeTlsCertAuth
};
};

View File

@ -0,0 +1,49 @@
import { TIdentityAccessTokens, TIdentityOrgMemberships, TIdentityTlsCertAuths } from "@app/db/schemas";
import { TProjectPermission } from "@app/lib/types";
export type TLoginTlsCertAuthDTO = {
identityId: string;
clientCertificate: string;
};
export type TAttachTlsCertAuthDTO = {
identityId: string;
caCertificate: string;
allowedCommonNames?: string | null;
accessTokenTTL: number;
accessTokenMaxTTL: number;
accessTokenNumUsesLimit: number;
accessTokenTrustedIps: { ipAddress: string }[];
isActorSuperAdmin?: boolean;
} & Omit<TProjectPermission, "projectId">;
export type TUpdateTlsCertAuthDTO = {
identityId: string;
caCertificate?: string;
allowedCommonNames?: string | null;
accessTokenTTL?: number;
accessTokenMaxTTL?: number;
accessTokenNumUsesLimit?: number;
accessTokenTrustedIps?: { ipAddress: string }[];
} & Omit<TProjectPermission, "projectId">;
export type TGetTlsCertAuthDTO = {
identityId: string;
} & Omit<TProjectPermission, "projectId">;
export type TRevokeTlsCertAuthDTO = {
identityId: string;
} & Omit<TProjectPermission, "projectId">;
export type TIdentityTlsCertAuthServiceFactory = {
login: (dto: TLoginTlsCertAuthDTO) => Promise<{
identityTlsCertAuth: TIdentityTlsCertAuths;
accessToken: string;
identityAccessToken: TIdentityAccessTokens;
identityMembershipOrg: TIdentityOrgMemberships;
}>;
attachTlsCertAuth: (dto: TAttachTlsCertAuthDTO) => Promise<TIdentityTlsCertAuths>;
updateTlsCertAuth: (dto: TUpdateTlsCertAuthDTO) => Promise<TIdentityTlsCertAuths>;
revokeTlsCertAuth: (dto: TRevokeTlsCertAuthDTO) => Promise<TIdentityTlsCertAuths>;
getTlsCertAuth: (dto: TGetTlsCertAuthDTO) => Promise<TIdentityTlsCertAuths & { caCertificate: string }>;
};

View File

@ -1,5 +1,5 @@
import { TDbClient } from "@app/db";
import { TableName, TIdentities } from "@app/db/schemas";
import { IdentityAuthMethod, TableName, TIdentities } from "@app/db/schemas";
import { DatabaseError } from "@app/lib/errors";
import { ormify, selectAllTableCols } from "@app/lib/knex";
@ -8,6 +8,28 @@ export type TIdentityDALFactory = ReturnType<typeof identityDALFactory>;
export const identityDALFactory = (db: TDbClient) => {
const identityOrm = ormify(db, TableName.Identity);
const getTrustedIpsByAuthMethod = async (identityId: string, authMethod: IdentityAuthMethod) => {
const authMethodToTableName = {
[IdentityAuthMethod.TOKEN_AUTH]: TableName.IdentityTokenAuth,
[IdentityAuthMethod.UNIVERSAL_AUTH]: TableName.IdentityUniversalAuth,
[IdentityAuthMethod.KUBERNETES_AUTH]: TableName.IdentityKubernetesAuth,
[IdentityAuthMethod.GCP_AUTH]: TableName.IdentityGcpAuth,
[IdentityAuthMethod.ALICLOUD_AUTH]: TableName.IdentityAliCloudAuth,
[IdentityAuthMethod.AWS_AUTH]: TableName.IdentityAwsAuth,
[IdentityAuthMethod.AZURE_AUTH]: TableName.IdentityAzureAuth,
[IdentityAuthMethod.TLS_CERT_AUTH]: TableName.IdentityTlsCertAuth,
[IdentityAuthMethod.OCI_AUTH]: TableName.IdentityOciAuth,
[IdentityAuthMethod.OIDC_AUTH]: TableName.IdentityOidcAuth,
[IdentityAuthMethod.JWT_AUTH]: TableName.IdentityJwtAuth,
[IdentityAuthMethod.LDAP_AUTH]: TableName.IdentityLdapAuth
} as const;
const tableName = authMethodToTableName[authMethod];
if (!tableName) return;
const data = await db(tableName).where({ identityId }).first();
if (!data) return;
return data.accessTokenTrustedIps;
};
const getIdentitiesByFilter = async ({
limit,
offset,
@ -38,5 +60,5 @@ export const identityDALFactory = (db: TDbClient) => {
}
};
return { ...identityOrm, getIdentitiesByFilter };
return { ...identityOrm, getTrustedIpsByAuthMethod, getIdentitiesByFilter };
};

View File

@ -11,7 +11,8 @@ export const buildAuthMethods = ({
azureId,
tokenId,
jwtId,
ldapId
ldapId,
tlsCertId
}: {
uaId?: string;
gcpId?: string;
@ -24,6 +25,7 @@ export const buildAuthMethods = ({
tokenId?: string;
jwtId?: string;
ldapId?: string;
tlsCertId?: string;
}) => {
return [
...[uaId ? IdentityAuthMethod.UNIVERSAL_AUTH : null],
@ -36,6 +38,7 @@ export const buildAuthMethods = ({
...[azureId ? IdentityAuthMethod.AZURE_AUTH : null],
...[tokenId ? IdentityAuthMethod.TOKEN_AUTH : null],
...[jwtId ? IdentityAuthMethod.JWT_AUTH : null],
...[ldapId ? IdentityAuthMethod.LDAP_AUTH : null]
...[ldapId ? IdentityAuthMethod.LDAP_AUTH : null],
...[tlsCertId ? IdentityAuthMethod.TLS_CERT_AUTH : null]
].filter((authMethod) => authMethod) as IdentityAuthMethod[];
};

View File

@ -12,6 +12,7 @@ import {
TIdentityOciAuths,
TIdentityOidcAuths,
TIdentityOrgMemberships,
TIdentityTlsCertAuths,
TIdentityTokenAuths,
TIdentityUniversalAuths,
TOrgRoles
@ -99,7 +100,11 @@ export const identityOrgDALFactory = (db: TDbClient) => {
`${TableName.IdentityOrgMembership}.identityId`,
`${TableName.IdentityLdapAuth}.identityId`
)
.leftJoin<TIdentityTlsCertAuths>(
TableName.IdentityTlsCertAuth,
`${TableName.IdentityOrgMembership}.identityId`,
`${TableName.IdentityTlsCertAuth}.identityId`
)
.select(
selectAllTableCols(TableName.IdentityOrgMembership),
@ -114,6 +119,7 @@ export const identityOrgDALFactory = (db: TDbClient) => {
db.ref("id").as("tokenId").withSchema(TableName.IdentityTokenAuth),
db.ref("id").as("jwtId").withSchema(TableName.IdentityJwtAuth),
db.ref("id").as("ldapId").withSchema(TableName.IdentityLdapAuth),
db.ref("id").as("tlsCertId").withSchema(TableName.IdentityTlsCertAuth),
db.ref("name").withSchema(TableName.Identity),
db.ref("hasDeleteProtection").withSchema(TableName.Identity)
);
@ -238,7 +244,11 @@ export const identityOrgDALFactory = (db: TDbClient) => {
"paginatedIdentity.identityId",
`${TableName.IdentityLdapAuth}.identityId`
)
.leftJoin<TIdentityTlsCertAuths>(
TableName.IdentityTlsCertAuth,
"paginatedIdentity.identityId",
`${TableName.IdentityTlsCertAuth}.identityId`
)
.select(
db.ref("id").withSchema("paginatedIdentity"),
db.ref("role").withSchema("paginatedIdentity"),
@ -260,7 +270,8 @@ export const identityOrgDALFactory = (db: TDbClient) => {
db.ref("id").as("azureId").withSchema(TableName.IdentityAzureAuth),
db.ref("id").as("tokenId").withSchema(TableName.IdentityTokenAuth),
db.ref("id").as("jwtId").withSchema(TableName.IdentityJwtAuth),
db.ref("id").as("ldapId").withSchema(TableName.IdentityLdapAuth)
db.ref("id").as("ldapId").withSchema(TableName.IdentityLdapAuth),
db.ref("id").as("tlsCertId").withSchema(TableName.IdentityTlsCertAuth)
)
// cr stands for custom role
.select(db.ref("id").as("crId").withSchema(TableName.OrgRoles))
@ -306,6 +317,7 @@ export const identityOrgDALFactory = (db: TDbClient) => {
azureId,
tokenId,
ldapId,
tlsCertId,
createdAt,
updatedAt
}) => ({
@ -313,7 +325,6 @@ export const identityOrgDALFactory = (db: TDbClient) => {
roleId,
identityId,
id,
orgId,
createdAt,
updatedAt,
@ -341,7 +352,8 @@ export const identityOrgDALFactory = (db: TDbClient) => {
azureId,
tokenId,
jwtId,
ldapId
ldapId,
tlsCertId
})
}
}),
@ -380,7 +392,12 @@ export const identityOrgDALFactory = (db: TDbClient) => {
.join(TableName.Identity, `${TableName.Identity}.id`, `${TableName.IdentityOrgMembership}.identityId`)
.where(`${TableName.IdentityOrgMembership}.orgId`, orgId)
.leftJoin(TableName.OrgRoles, `${TableName.IdentityOrgMembership}.roleId`, `${TableName.OrgRoles}.id`)
.orderBy(`${TableName.Identity}.${orderBy}`, orderDirection)
.orderBy(
orderBy === OrgIdentityOrderBy.Role
? `${TableName.IdentityOrgMembership}.${orderBy}`
: `${TableName.Identity}.${orderBy}`,
orderDirection
)
.select(`${TableName.IdentityOrgMembership}.id`)
.select<{ id: string; total_count: string }>(
db.raw(
@ -511,6 +528,23 @@ export const identityOrgDALFactory = (db: TDbClient) => {
if (orderBy === OrgIdentityOrderBy.Name) {
void query.orderBy("identityName", orderDirection);
} else if (orderBy === OrgIdentityOrderBy.Role) {
void query.orderByRaw(
`
CASE
WHEN ??.role = ?
THEN ??.slug
ELSE ??.role
END ?
`,
[
TableName.IdentityOrgMembership,
"custom",
TableName.OrgRoles,
TableName.IdentityOrgMembership,
db.raw(orderDirection)
]
);
}
const docs = await query;

View File

@ -46,8 +46,8 @@ export type TListOrgIdentitiesByOrgIdDTO = {
} & TOrgPermission;
export enum OrgIdentityOrderBy {
Name = "name"
// Role = "role"
Name = "name",
Role = "role"
}
export type TSearchOrgIdentitiesByOrgIdDAL = {

View File

@ -307,7 +307,6 @@ export const AwsParameterStoreSyncFns = {
awsParameterStoreSecretsRecord,
Boolean(syncOptions.tags?.length || syncOptions.syncSecretMetadataAsTags)
);
const syncTagsRecord = Object.fromEntries(syncOptions.tags?.map((tag) => [tag.key, tag.value]) ?? []);
for await (const entry of Object.entries(secretMap)) {
const [key, { value, secretMetadata }] = entry;
@ -342,13 +341,13 @@ export const AwsParameterStoreSyncFns = {
}
}
if (shouldManageTags) {
if ((syncOptions.tags !== undefined || syncOptions.syncSecretMetadataAsTags) && shouldManageTags) {
const { tagsToAdd, tagKeysToRemove } = processParameterTags({
syncTagsRecord: {
// configured sync tags take preference over secret metadata
...(syncOptions.syncSecretMetadataAsTags &&
Object.fromEntries(secretMetadata?.map((tag) => [tag.key, tag.value]) ?? [])),
...syncTagsRecord
...(syncOptions.tags && Object.fromEntries(syncOptions.tags?.map((tag) => [tag.key, tag.value]) ?? []))
},
awsTagsRecord: awsParameterStoreTagsRecord[key] ?? {}
});

View File

@ -366,12 +366,13 @@ export const AwsSecretsManagerSyncFns = {
}
}
if (syncOptions.tags !== undefined || syncOptions.syncSecretMetadataAsTags) {
const { tagsToAdd, tagKeysToRemove } = processTags({
syncTagsRecord: {
// configured sync tags take preference over secret metadata
...(syncOptions.syncSecretMetadataAsTags &&
Object.fromEntries(secretMetadata?.map((tag) => [tag.key, tag.value]) ?? [])),
...syncTagsRecord
...(syncOptions.tags !== undefined && syncTagsRecord)
},
awsTagsRecord: Object.fromEntries(
awsDescriptionsRecord[key]?.Tags?.map((tag) => [tag.Key!, tag.Value!]) ?? []
@ -400,6 +401,7 @@ export const AwsSecretsManagerSyncFns = {
}
}
}
}
if (syncOptions.disableSecretDeletion) return;
@ -439,6 +441,7 @@ export const AwsSecretsManagerSyncFns = {
});
}
if (syncOptions.tags !== undefined) {
const { tagsToAdd, tagKeysToRemove } = processTags({
syncTagsRecord,
awsTagsRecord: Object.fromEntries(
@ -468,6 +471,7 @@ export const AwsSecretsManagerSyncFns = {
}
}
}
}
},
getSecrets: async (secretSync: TAwsSecretsManagerSyncWithCredentials): Promise<TSecretMap> => {
const client = await getSecretsManagerClient(secretSync);

View File

@ -0,0 +1,10 @@
import { AppConnection } from "@app/services/app-connection/app-connection-enums";
import { SecretSync } from "@app/services/secret-sync/secret-sync-enums";
import { TSecretSyncListItem } from "@app/services/secret-sync/secret-sync-types";
export const GITLAB_SYNC_LIST_OPTION: TSecretSyncListItem = {
name: "GitLab",
destination: SecretSync.GitLab,
connection: AppConnection.GitLab,
canImportSecrets: false
};

View File

@ -0,0 +1,4 @@
export enum GitLabSyncScope {
Project = "project",
Group = "group"
}

View File

@ -0,0 +1,452 @@
/* eslint-disable no-await-in-loop */
import { GitbeakerRequestError } from "@gitbeaker/rest";
import { TAppConnectionDALFactory } from "@app/services/app-connection/app-connection-dal";
import {
getGitLabClient,
GitLabConnectionMethod,
refreshGitLabToken,
TGitLabConnection
} from "@app/services/app-connection/gitlab";
import { TKmsServiceFactory } from "@app/services/kms/kms-service";
import { TGitLabSyncWithCredentials, TGitLabVariable } from "@app/services/secret-sync/gitlab/gitlab-sync-types";
import { SecretSyncError } from "@app/services/secret-sync/secret-sync-errors";
import { matchesSchema } from "@app/services/secret-sync/secret-sync-fns";
import { TSecretMap } from "@app/services/secret-sync/secret-sync-types";
import { SECRET_SYNC_NAME_MAP } from "../secret-sync-maps";
import { GitLabSyncScope } from "./gitlab-sync-enums";
interface TGitLabVariablePayload {
key?: string;
value: string;
variable_type?: "env_var" | "file";
environment_scope?: string;
protected?: boolean;
masked?: boolean;
masked_and_hidden?: boolean;
description?: string;
}
interface TGitLabVariableCreate extends TGitLabVariablePayload {
key: string;
}
interface TGitLabVariableUpdate extends Omit<TGitLabVariablePayload, "key"> {}
type TGitLabSyncFactoryDeps = {
appConnectionDAL: Pick<TAppConnectionDALFactory, "updateById">;
kmsService: Pick<TKmsServiceFactory, "createCipherPairWithDataKey">;
};
const getValidAccessToken = async (
connection: TGitLabConnection,
appConnectionDAL: Pick<TAppConnectionDALFactory, "updateById">,
kmsService: Pick<TKmsServiceFactory, "createCipherPairWithDataKey">
): Promise<string> => {
if (
connection.method === GitLabConnectionMethod.OAuth &&
connection.credentials.refreshToken &&
new Date(connection.credentials.expiresAt) < new Date()
) {
const accessToken = await refreshGitLabToken(
connection.credentials.refreshToken,
connection.id,
connection.orgId,
appConnectionDAL,
kmsService,
connection.credentials.instanceUrl
);
return accessToken;
}
return connection.credentials.accessToken;
};
const getGitLabVariables = async ({
accessToken,
connection,
scope,
resourceId,
targetEnvironment
}: {
accessToken: string;
connection: TGitLabConnection;
scope: GitLabSyncScope;
resourceId: string;
targetEnvironment?: string;
}): Promise<TGitLabVariable[]> => {
try {
const client = await getGitLabClient(
accessToken,
connection.credentials.instanceUrl,
connection.method === GitLabConnectionMethod.OAuth
);
let variables: TGitLabVariable[] = [];
if (scope === GitLabSyncScope.Project) {
variables = await client.ProjectVariables.all(resourceId);
} else {
variables = await client.GroupVariables.all(resourceId);
}
if (targetEnvironment) {
variables = variables.filter((v) => v.environmentScope === targetEnvironment);
}
return variables;
} catch (error) {
if (error instanceof GitbeakerRequestError) {
throw new SecretSyncError({
error: new Error(
`Failed to fetch variables: ${error.message ?? "Unknown error"}${error.cause?.description && error.message !== "Unauthorized" ? `. Cause: ${error.cause.description}` : ""}`
)
});
}
throw new SecretSyncError({
error
});
}
};
const createGitLabVariable = async ({
accessToken,
connection,
scope,
resourceId,
variable
}: {
accessToken: string;
connection: TGitLabConnection;
scope: GitLabSyncScope;
resourceId: string;
variable: TGitLabVariableCreate;
}): Promise<void> => {
try {
const client = await getGitLabClient(
accessToken,
connection.credentials.instanceUrl,
connection.method === GitLabConnectionMethod.OAuth
);
const payload = {
key: variable.key,
value: variable.value,
variableType: "env_var",
environmentScope: variable.environment_scope || "*",
protected: variable.protected || false,
masked: variable.masked || false,
masked_and_hidden: variable.masked_and_hidden || false,
raw: false
};
if (scope === GitLabSyncScope.Project) {
await client.ProjectVariables.create(resourceId, payload.key, payload.value, {
variableType: "env_var",
environmentScope: payload.environmentScope,
protected: payload.protected,
masked: payload.masked,
masked_and_hidden: payload.masked_and_hidden,
raw: false
});
} else {
await client.GroupVariables.create(resourceId, payload.key, payload.value, {
variableType: "env_var",
environmentScope: payload.environmentScope,
protected: payload.protected,
masked: payload.masked,
...(payload.masked_and_hidden && { masked_and_hidden: payload.masked_and_hidden }),
raw: false
});
}
} catch (error) {
if (error instanceof GitbeakerRequestError) {
throw new SecretSyncError({
error: new Error(
`Failed to create variable: ${error.message ?? "Unknown error"}${error.cause?.description && error.message !== "Unauthorized" ? `. Cause: ${error.cause.description}` : ""}`
),
secretKey: variable.key
});
}
throw new SecretSyncError({
error,
secretKey: variable.key
});
}
};
const updateGitLabVariable = async ({
accessToken,
connection,
scope,
resourceId,
key,
variable,
targetEnvironment
}: {
accessToken: string;
connection: TGitLabConnection;
scope: GitLabSyncScope;
resourceId: string;
key: string;
variable: TGitLabVariableUpdate;
targetEnvironment?: string;
}): Promise<void> => {
try {
const client = await getGitLabClient(
accessToken,
connection.credentials.instanceUrl,
connection.method === GitLabConnectionMethod.OAuth
);
const options = {
...(variable.environment_scope && { environmentScope: variable.environment_scope }),
...(variable.protected !== undefined && { protected: variable.protected }),
...(variable.masked !== undefined && { masked: variable.masked })
};
if (targetEnvironment) {
options.environmentScope = targetEnvironment;
}
if (scope === GitLabSyncScope.Project) {
await client.ProjectVariables.edit(resourceId, key, variable.value, {
...options,
filter: { environment_scope: targetEnvironment || "*" }
});
} else {
await client.GroupVariables.edit(resourceId, key, variable.value, {
...options,
filter: { environment_scope: targetEnvironment || "*" }
});
}
} catch (error) {
if (error instanceof GitbeakerRequestError) {
throw new SecretSyncError({
error: new Error(
`Failed to update variable: ${error.message ?? "Unknown error"}${error.cause?.description && error.message !== "Unauthorized" ? `. Cause: ${error.cause.description}` : ""}`
),
secretKey: key
});
}
throw new SecretSyncError({
error,
secretKey: key
});
}
};
const deleteGitLabVariable = async ({
accessToken,
connection,
scope,
resourceId,
key,
targetEnvironment,
allVariables
}: {
accessToken: string;
connection: TGitLabConnection;
scope: GitLabSyncScope;
resourceId: string;
key: string;
targetEnvironment?: string;
allVariables?: TGitLabVariable[];
}): Promise<void> => {
if (allVariables && !allVariables.find((v) => v.key === key)) {
return;
}
try {
const client = await getGitLabClient(
accessToken,
connection.credentials.instanceUrl,
connection.method === GitLabConnectionMethod.OAuth
);
const options: { filter?: { environment_scope: string } } = {};
if (targetEnvironment) {
options.filter = { environment_scope: targetEnvironment || "*" };
}
if (scope === GitLabSyncScope.Project) {
await client.ProjectVariables.remove(resourceId, key, options);
} else {
await client.GroupVariables.remove(resourceId, key);
}
} catch (error: unknown) {
if (error instanceof GitbeakerRequestError) {
throw new SecretSyncError({
error: new Error(
`Failed to delete variable: ${error.message ?? "Unknown error"}${error.cause?.description && error.message !== "Unauthorized" ? `. Cause: ${error.cause.description}` : ""}`
),
secretKey: key
});
}
throw new SecretSyncError({
error,
secretKey: key
});
}
};
export const GitLabSyncFns = {
syncSecrets: async (
secretSync: TGitLabSyncWithCredentials,
secretMap: TSecretMap,
{ appConnectionDAL, kmsService }: TGitLabSyncFactoryDeps
): Promise<void> => {
const { connection, environment, destinationConfig } = secretSync;
const { scope, targetEnvironment } = destinationConfig;
const resourceId = scope === GitLabSyncScope.Project ? destinationConfig.projectId : destinationConfig.groupId;
const accessToken = await getValidAccessToken(connection, appConnectionDAL, kmsService);
try {
const currentVariables = await getGitLabVariables({
accessToken,
connection,
scope,
resourceId,
targetEnvironment
});
const currentVariableMap = new Map(currentVariables.map((v) => [v.key, v]));
for (const [key, { value }] of Object.entries(secretMap)) {
if (value?.length < 8 && destinationConfig.shouldMaskSecrets) {
throw new SecretSyncError({
message: `Secret ${key} is too short to be masked. GitLab requires a minimum of 8 characters for masked secrets.`,
secretKey: key
});
}
try {
const existingVariable = currentVariableMap.get(key);
if (existingVariable) {
if (
existingVariable.value !== value ||
existingVariable.environmentScope !== targetEnvironment ||
existingVariable.protected !== destinationConfig.shouldProtectSecrets ||
existingVariable.masked !== destinationConfig.shouldMaskSecrets
) {
await updateGitLabVariable({
accessToken,
connection,
scope,
resourceId,
key,
variable: {
value,
environment_scope: targetEnvironment,
protected: destinationConfig.shouldProtectSecrets,
masked: destinationConfig.shouldMaskSecrets || existingVariable.hidden
},
targetEnvironment
});
}
} else {
await createGitLabVariable({
accessToken,
connection,
scope,
resourceId,
variable: {
key,
value,
variable_type: "env_var",
environment_scope: targetEnvironment || "*",
protected: destinationConfig.shouldProtectSecrets || false,
masked: destinationConfig.shouldMaskSecrets || false,
masked_and_hidden: destinationConfig.shouldHideSecrets || false
}
});
}
} catch (error) {
throw new SecretSyncError({
error,
secretKey: key
});
}
}
if (!secretSync.syncOptions.disableSecretDeletion) {
for (const variable of currentVariables) {
try {
const shouldDelete =
matchesSchema(variable.key, environment?.slug || "", secretSync.syncOptions.keySchema) &&
!(variable.key in secretMap);
if (shouldDelete) {
await deleteGitLabVariable({
accessToken,
connection,
scope,
resourceId,
key: variable.key,
targetEnvironment
});
}
} catch (error) {
throw new SecretSyncError({
error,
secretKey: variable.key
});
}
}
}
} catch (error) {
if (error instanceof SecretSyncError) {
throw error;
}
throw new SecretSyncError({
message: "Failed to sync secrets",
error
});
}
},
removeSecrets: async (
secretSync: TGitLabSyncWithCredentials,
secretMap: TSecretMap,
{ appConnectionDAL, kmsService }: TGitLabSyncFactoryDeps
): Promise<void> => {
const { connection, destinationConfig } = secretSync;
const { scope, targetEnvironment } = destinationConfig;
const resourceId = scope === GitLabSyncScope.Project ? destinationConfig.projectId : destinationConfig.groupId;
const accessToken = await getValidAccessToken(connection, appConnectionDAL, kmsService);
const allVariables = await getGitLabVariables({
accessToken,
connection,
scope,
resourceId,
targetEnvironment
});
for (const key of Object.keys(secretMap)) {
try {
await deleteGitLabVariable({
accessToken,
connection,
scope,
resourceId,
key,
targetEnvironment,
allVariables
});
} catch (error) {
throw new SecretSyncError({
error,
secretKey: key
});
}
}
},
getSecrets: async (secretSync: TGitLabSyncWithCredentials): Promise<TSecretMap> => {
throw new Error(`${SECRET_SYNC_NAME_MAP[secretSync.destination]} does not support importing secrets.`);
}
};

View File

@ -0,0 +1,97 @@
import { z } from "zod";
import { SecretSyncs } from "@app/lib/api-docs";
import { AppConnection } from "@app/services/app-connection/app-connection-enums";
import { SecretSync } from "@app/services/secret-sync/secret-sync-enums";
import {
BaseSecretSyncSchema,
GenericCreateSecretSyncFieldsSchema,
GenericUpdateSecretSyncFieldsSchema
} from "@app/services/secret-sync/secret-sync-schemas";
import { TSyncOptionsConfig } from "@app/services/secret-sync/secret-sync-types";
import { GitLabSyncScope } from "./gitlab-sync-enums";
const GitLabSyncDestinationConfigSchema = z.discriminatedUnion("scope", [
z.object({
scope: z.literal(GitLabSyncScope.Project).describe(SecretSyncs.DESTINATION_CONFIG.GITLAB.scope),
projectId: z.string().min(1, "Project ID is required").describe(SecretSyncs.DESTINATION_CONFIG.GITLAB.projectId),
projectName: z
.string()
.min(1, "Project name is required")
.describe(SecretSyncs.DESTINATION_CONFIG.GITLAB.projectName),
targetEnvironment: z
.string()
.optional()
.default("*")
.describe(SecretSyncs.DESTINATION_CONFIG.GITLAB.targetEnvironment),
shouldProtectSecrets: z
.boolean()
.optional()
.default(false)
.describe(SecretSyncs.DESTINATION_CONFIG.GITLAB.shouldProtectSecrets),
shouldMaskSecrets: z
.boolean()
.optional()
.default(false)
.describe(SecretSyncs.DESTINATION_CONFIG.GITLAB.shouldMaskSecrets),
shouldHideSecrets: z
.boolean()
.optional()
.default(false)
.describe(SecretSyncs.DESTINATION_CONFIG.GITLAB.shouldHideSecrets)
}),
z.object({
scope: z.literal(GitLabSyncScope.Group).describe(SecretSyncs.DESTINATION_CONFIG.GITLAB.scope),
groupId: z.string().min(1, "Group ID is required").describe(SecretSyncs.DESTINATION_CONFIG.GITLAB.groupId),
groupName: z.string().min(1, "Group name is required").describe(SecretSyncs.DESTINATION_CONFIG.GITLAB.groupName),
targetEnvironment: z
.string()
.optional()
.default("*")
.describe(SecretSyncs.DESTINATION_CONFIG.GITLAB.targetEnvironment),
shouldProtectSecrets: z
.boolean()
.optional()
.default(false)
.describe(SecretSyncs.DESTINATION_CONFIG.GITLAB.shouldProtectSecrets),
shouldMaskSecrets: z
.boolean()
.optional()
.default(false)
.describe(SecretSyncs.DESTINATION_CONFIG.GITLAB.shouldMaskSecrets),
shouldHideSecrets: z
.boolean()
.optional()
.default(false)
.describe(SecretSyncs.DESTINATION_CONFIG.GITLAB.shouldHideSecrets)
})
]);
const GitLabSyncOptionsConfig: TSyncOptionsConfig = { canImportSecrets: false };
export const GitLabSyncSchema = BaseSecretSyncSchema(SecretSync.GitLab, GitLabSyncOptionsConfig).extend({
destination: z.literal(SecretSync.GitLab),
destinationConfig: GitLabSyncDestinationConfigSchema
});
export const CreateGitLabSyncSchema = GenericCreateSecretSyncFieldsSchema(
SecretSync.GitLab,
GitLabSyncOptionsConfig
).extend({
destinationConfig: GitLabSyncDestinationConfigSchema
});
export const UpdateGitLabSyncSchema = GenericUpdateSecretSyncFieldsSchema(
SecretSync.GitLab,
GitLabSyncOptionsConfig
).extend({
destinationConfig: GitLabSyncDestinationConfigSchema.optional()
});
export const GitLabSyncListItemSchema = z.object({
name: z.literal("GitLab"),
connection: z.literal(AppConnection.GitLab),
destination: z.literal(SecretSync.GitLab),
canImportSecrets: z.literal(false)
});

View File

@ -0,0 +1,58 @@
import { z } from "zod";
import { TGitLabConnection } from "@app/services/app-connection/gitlab";
import { CreateGitLabSyncSchema, GitLabSyncListItemSchema, GitLabSyncSchema } from "./gitlab-sync-schemas";
export type TGitLabSync = z.infer<typeof GitLabSyncSchema>;
export type TGitLabSyncInput = z.infer<typeof CreateGitLabSyncSchema>;
export type TGitLabSyncListItem = z.infer<typeof GitLabSyncListItemSchema>;
export type TGitLabSyncWithCredentials = TGitLabSync & {
connection: TGitLabConnection;
};
export type TGitLabVariable = {
key: string;
value: string;
protected: boolean;
masked: boolean;
environmentScope?: string;
hidden?: boolean;
};
export type TGitLabVariableCreate = {
key: string;
value: string;
variable_type?: "env_var" | "file";
protected?: boolean;
masked?: boolean;
raw?: boolean;
environment_scope?: string;
description?: string;
};
export type TGitLabVariableUpdate = {
value: string;
variable_type?: "env_var" | "file";
protected?: boolean;
masked?: boolean;
raw?: boolean;
environment_scope?: string;
description?: string | null;
};
export type TGitLabListVariables = {
accessToken: string;
projectId: string;
environmentScope?: string;
};
export type TGitLabCreateVariable = TGitLabListVariables & {
variable: TGitLabVariableCreate;
};
export type TGitLabUpdateVariable = TGitLabListVariables & {
key: string;
variable: TGitLabVariableUpdate;
};

View File

@ -0,0 +1,4 @@
export * from "./gitlab-sync-constants";
export * from "./gitlab-sync-fns";
export * from "./gitlab-sync-schemas";
export * from "./gitlab-sync-types";

View File

@ -19,6 +19,7 @@ export enum SecretSync {
Heroku = "heroku",
Render = "render",
Flyio = "flyio",
GitLab = "gitlab",
CloudflarePages = "cloudflare-pages"
}

View File

@ -34,6 +34,7 @@ import { CloudflarePagesSyncFns } from "./cloudflare-pages/cloudflare-pages-fns"
import { FLYIO_SYNC_LIST_OPTION, FlyioSyncFns } from "./flyio";
import { GCP_SYNC_LIST_OPTION } from "./gcp";
import { GcpSyncFns } from "./gcp/gcp-sync-fns";
import { GITLAB_SYNC_LIST_OPTION, GitLabSyncFns } from "./gitlab";
import { HC_VAULT_SYNC_LIST_OPTION, HCVaultSyncFns } from "./hc-vault";
import { HEROKU_SYNC_LIST_OPTION, HerokuSyncFns } from "./heroku";
import { HUMANITEC_SYNC_LIST_OPTION } from "./humanitec";
@ -66,6 +67,7 @@ const SECRET_SYNC_LIST_OPTIONS: Record<SecretSync, TSecretSyncListItem> = {
[SecretSync.Heroku]: HEROKU_SYNC_LIST_OPTION,
[SecretSync.Render]: RENDER_SYNC_LIST_OPTION,
[SecretSync.Flyio]: FLYIO_SYNC_LIST_OPTION,
[SecretSync.GitLab]: GITLAB_SYNC_LIST_OPTION,
[SecretSync.CloudflarePages]: CLOUDFLARE_PAGES_SYNC_LIST_OPTION
};
@ -230,6 +232,8 @@ export const SecretSyncFns = {
return RenderSyncFns.syncSecrets(secretSync, schemaSecretMap);
case SecretSync.Flyio:
return FlyioSyncFns.syncSecrets(secretSync, schemaSecretMap);
case SecretSync.GitLab:
return GitLabSyncFns.syncSecrets(secretSync, schemaSecretMap, { appConnectionDAL, kmsService });
case SecretSync.CloudflarePages:
return CloudflarePagesSyncFns.syncSecrets(secretSync, schemaSecretMap);
default:
@ -318,6 +322,9 @@ export const SecretSyncFns = {
case SecretSync.Flyio:
secretMap = await FlyioSyncFns.getSecrets(secretSync);
break;
case SecretSync.GitLab:
secretMap = await GitLabSyncFns.getSecrets(secretSync);
break;
case SecretSync.CloudflarePages:
secretMap = await CloudflarePagesSyncFns.getSecrets(secretSync);
break;
@ -394,6 +401,8 @@ export const SecretSyncFns = {
return RenderSyncFns.removeSecrets(secretSync, schemaSecretMap);
case SecretSync.Flyio:
return FlyioSyncFns.removeSecrets(secretSync, schemaSecretMap);
case SecretSync.GitLab:
return GitLabSyncFns.removeSecrets(secretSync, schemaSecretMap, { appConnectionDAL, kmsService });
case SecretSync.CloudflarePages:
return CloudflarePagesSyncFns.removeSecrets(secretSync, schemaSecretMap);
default:

View File

@ -22,6 +22,7 @@ export const SECRET_SYNC_NAME_MAP: Record<SecretSync, string> = {
[SecretSync.Heroku]: "Heroku",
[SecretSync.Render]: "Render",
[SecretSync.Flyio]: "Fly.io",
[SecretSync.GitLab]: "GitLab",
[SecretSync.CloudflarePages]: "Cloudflare Pages"
};
@ -46,6 +47,7 @@ export const SECRET_SYNC_CONNECTION_MAP: Record<SecretSync, AppConnection> = {
[SecretSync.Heroku]: AppConnection.Heroku,
[SecretSync.Render]: AppConnection.Render,
[SecretSync.Flyio]: AppConnection.Flyio,
[SecretSync.GitLab]: AppConnection.GitLab,
[SecretSync.CloudflarePages]: AppConnection.Cloudflare
};
@ -70,5 +72,6 @@ export const SECRET_SYNC_PLAN_MAP: Record<SecretSync, SecretSyncPlanType> = {
[SecretSync.Heroku]: SecretSyncPlanType.Regular,
[SecretSync.Render]: SecretSyncPlanType.Regular,
[SecretSync.Flyio]: SecretSyncPlanType.Regular,
[SecretSync.GitLab]: SecretSyncPlanType.Regular,
[SecretSync.CloudflarePages]: SecretSyncPlanType.Regular
};

View File

@ -72,8 +72,15 @@ import {
TAzureKeyVaultSyncListItem,
TAzureKeyVaultSyncWithCredentials
} from "./azure-key-vault";
import {
TCloudflarePagesSync,
TCloudflarePagesSyncInput,
TCloudflarePagesSyncListItem,
TCloudflarePagesSyncWithCredentials
} from "./cloudflare-pages/cloudflare-pages-types";
import { TFlyioSync, TFlyioSyncInput, TFlyioSyncListItem, TFlyioSyncWithCredentials } from "./flyio/flyio-sync-types";
import { TGcpSync, TGcpSyncInput, TGcpSyncListItem, TGcpSyncWithCredentials } from "./gcp";
import { TGitLabSync, TGitLabSyncInput, TGitLabSyncListItem, TGitLabSyncWithCredentials } from "./gitlab";
import {
THCVaultSync,
THCVaultSyncInput,
@ -106,12 +113,6 @@ import {
TTerraformCloudSyncWithCredentials
} from "./terraform-cloud";
import { TVercelSync, TVercelSyncInput, TVercelSyncListItem, TVercelSyncWithCredentials } from "./vercel";
import {
TCloudflarePagesSync,
TCloudflarePagesSyncInput,
TCloudflarePagesSyncListItem,
TCloudflarePagesSyncWithCredentials
} from "./cloudflare-pages/cloudflare-pages-types";
export type TSecretSync =
| TAwsParameterStoreSync
@ -134,6 +135,7 @@ export type TSecretSync =
| THerokuSync
| TRenderSync
| TFlyioSync
| TGitLabSync
| TCloudflarePagesSync;
export type TSecretSyncWithCredentials =
@ -157,6 +159,7 @@ export type TSecretSyncWithCredentials =
| THerokuSyncWithCredentials
| TRenderSyncWithCredentials
| TFlyioSyncWithCredentials
| TGitLabSyncWithCredentials
| TCloudflarePagesSyncWithCredentials;
export type TSecretSyncInput =
@ -180,6 +183,7 @@ export type TSecretSyncInput =
| THerokuSyncInput
| TRenderSyncInput
| TFlyioSyncInput
| TGitLabSyncInput
| TCloudflarePagesSyncInput;
export type TSecretSyncListItem =
@ -203,6 +207,7 @@ export type TSecretSyncListItem =
| THerokuSyncListItem
| TRenderSyncListItem
| TFlyioSyncListItem
| TGitLabSyncListItem
| TCloudflarePagesSyncListItem;
export type TSyncOptionsConfig = {

View File

@ -7,13 +7,18 @@ import { QueueJobs, QueueName, TQueueServiceFactory } from "@app/queue";
import { getServerCfg } from "../super-admin/super-admin-service";
import { TTelemetryDALFactory } from "./telemetry-dal";
import { TELEMETRY_SECRET_OPERATIONS_KEY, TELEMETRY_SECRET_PROCESSED_KEY } from "./telemetry-service";
import {
TELEMETRY_SECRET_OPERATIONS_KEY,
TELEMETRY_SECRET_PROCESSED_KEY,
TTelemetryServiceFactory
} from "./telemetry-service";
import { PostHogEventTypes } from "./telemetry-types";
type TTelemetryQueueServiceFactoryDep = {
queueService: TQueueServiceFactory;
keyStore: Pick<TKeyStoreFactory, "getItem" | "deleteItem">;
telemetryDAL: TTelemetryDALFactory;
telemetryService: TTelemetryServiceFactory;
};
export type TTelemetryQueueServiceFactory = ReturnType<typeof telemetryQueueServiceFactory>;
@ -21,7 +26,8 @@ export type TTelemetryQueueServiceFactory = ReturnType<typeof telemetryQueueServ
export const telemetryQueueServiceFactory = ({
queueService,
keyStore,
telemetryDAL
telemetryDAL,
telemetryService
}: TTelemetryQueueServiceFactoryDep) => {
const appCfg = getConfig();
const postHog =
@ -48,6 +54,10 @@ export const telemetryQueueServiceFactory = ({
await keyStore.deleteItem(TELEMETRY_SECRET_OPERATIONS_KEY);
});
queueService.start(QueueName.TelemetryAggregatedEvents, async () => {
await telemetryService.processAggregatedEvents();
});
// every day at midnight a telemetry job executes on self-hosted instances
// this sends some telemetry information like instance id secrets operated etc
const startTelemetryCheck = async () => {
@ -60,11 +70,26 @@ export const telemetryQueueServiceFactory = ({
{ pattern: "0 0 * * *", utc: true },
QueueName.TelemetryInstanceStats // just a job id
);
// clear previous aggregated events job
await queueService.stopRepeatableJob(
QueueName.TelemetryAggregatedEvents,
QueueJobs.TelemetryAggregatedEvents,
{ pattern: "*/5 * * * *", utc: true },
QueueName.TelemetryAggregatedEvents // just a job id
);
if (postHog) {
await queueService.queue(QueueName.TelemetryInstanceStats, QueueJobs.TelemetryInstanceStats, undefined, {
jobId: QueueName.TelemetryInstanceStats,
repeat: { pattern: "0 0 * * *", utc: true }
});
// Start aggregated events job (runs every five minutes)
await queueService.queue(QueueName.TelemetryAggregatedEvents, QueueJobs.TelemetryAggregatedEvents, undefined, {
jobId: QueueName.TelemetryAggregatedEvents,
repeat: { pattern: "*/5 * * * *", utc: true }
});
}
};
@ -72,6 +97,10 @@ export const telemetryQueueServiceFactory = ({
logger.error(err?.failedReason, `${QueueName.TelemetryInstanceStats}: failed`);
});
queueService.listen(QueueName.TelemetryAggregatedEvents, "failed", (err) => {
logger.error(err?.failedReason, `${QueueName.TelemetryAggregatedEvents}: failed`);
});
return {
startTelemetryCheck
};

View File

@ -1,3 +1,4 @@
import { createHash, randomUUID } from "crypto";
import { PostHog } from "posthog-node";
import { TLicenseServiceFactory } from "@app/ee/services/license/license-service";
@ -12,12 +13,49 @@ import { PostHogEventTypes, TPostHogEvent, TSecretModifiedEvent } from "./teleme
export const TELEMETRY_SECRET_PROCESSED_KEY = "telemetry-secret-processed";
export const TELEMETRY_SECRET_OPERATIONS_KEY = "telemetry-secret-operations";
export const POSTHOG_AGGREGATED_EVENTS = [PostHogEventTypes.SecretPulled];
const TELEMETRY_AGGREGATED_KEY_EXP = 900; // 15mins
// Bucket configuration
const TELEMETRY_BUCKET_COUNT = 30;
const TELEMETRY_BUCKET_NAMES = Array.from(
{ length: TELEMETRY_BUCKET_COUNT },
(_, i) => `bucket-${i.toString().padStart(2, "0")}`
);
type AggregatedEventData = Record<string, unknown>;
type SingleEventData = {
distinctId: string;
event: string;
properties: unknown;
organizationId: string;
};
export type TTelemetryServiceFactory = ReturnType<typeof telemetryServiceFactory>;
export type TTelemetryServiceFactoryDep = {
keyStore: Pick<TKeyStoreFactory, "getItem" | "incrementBy">;
keyStore: Pick<
TKeyStoreFactory,
"incrementBy" | "deleteItemsByKeyIn" | "setItemWithExpiry" | "getKeysByPattern" | "getItems"
>;
licenseService: Pick<TLicenseServiceFactory, "getInstanceType">;
};
const getBucketForDistinctId = (distinctId: string): string => {
// Use SHA-256 hash for consistent distribution
const hash = createHash("sha256").update(distinctId).digest("hex");
// Take first 8 characters and convert to number for better distribution
const hashNumber = parseInt(hash.substring(0, 8), 16);
const bucketIndex = hashNumber % TELEMETRY_BUCKET_COUNT;
return TELEMETRY_BUCKET_NAMES[bucketIndex];
};
export const createTelemetryEventKey = (event: string, distinctId: string): string => {
const bucketId = getBucketForDistinctId(distinctId);
return `telemetry-event-${event}-${bucketId}-${distinctId}-${randomUUID()}`;
};
export const telemetryServiceFactory = ({ keyStore, licenseService }: TTelemetryServiceFactoryDep) => {
const appCfg = getConfig();
@ -64,11 +102,33 @@ To opt into telemetry, you can set "TELEMETRY_ENABLED=true" within the environme
const instanceType = licenseService.getInstanceType();
// capture posthog only when its cloud or signup event happens in self-hosted
if (instanceType === InstanceType.Cloud || event.event === PostHogEventTypes.UserSignedUp) {
if (event.organizationId) {
try {
postHog.groupIdentify({ groupType: "organization", groupKey: event.organizationId });
} catch (error) {
logger.error(error, "Failed to identify PostHog organization");
}
}
if (POSTHOG_AGGREGATED_EVENTS.includes(event.event)) {
const eventKey = createTelemetryEventKey(event.event, event.distinctId);
await keyStore.setItemWithExpiry(
eventKey,
TELEMETRY_AGGREGATED_KEY_EXP,
JSON.stringify({
distinctId: event.distinctId,
event: event.event,
properties: event.properties,
organizationId: event.organizationId
})
);
} else {
postHog.capture({
event: event.event,
distinctId: event.distinctId,
properties: event.properties
properties: event.properties,
...(event.organizationId ? { groups: { organization: event.organizationId } } : {})
});
}
return;
}
@ -89,6 +149,160 @@ To opt into telemetry, you can set "TELEMETRY_ENABLED=true" within the environme
}
};
const aggregateGroupProperties = (events: SingleEventData[]): AggregatedEventData => {
const aggregatedData: AggregatedEventData = {};
// Set the total count
aggregatedData.count = events.length;
events.forEach((event) => {
if (!event.properties) return;
Object.entries(event.properties as Record<string, unknown>).forEach(([key, value]: [string, unknown]) => {
if (Array.isArray(value)) {
// For arrays, count occurrences of each item
const existingCounts =
aggregatedData[key] &&
typeof aggregatedData[key] === "object" &&
aggregatedData[key]?.constructor === Object
? (aggregatedData[key] as Record<string, number>)
: {};
value.forEach((item) => {
const itemKey = typeof item === "object" ? JSON.stringify(item) : String(item);
existingCounts[itemKey] = (existingCounts[itemKey] || 0) + 1;
});
aggregatedData[key] = existingCounts;
} else if (typeof value === "object" && value?.constructor === Object) {
// For objects, count occurrences of each field value
const existingCounts =
aggregatedData[key] &&
typeof aggregatedData[key] === "object" &&
aggregatedData[key]?.constructor === Object
? (aggregatedData[key] as Record<string, number>)
: {};
if (value) {
Object.values(value).forEach((fieldValue) => {
const valueKey = typeof fieldValue === "object" ? JSON.stringify(fieldValue) : String(fieldValue);
existingCounts[valueKey] = (existingCounts[valueKey] || 0) + 1;
});
}
aggregatedData[key] = existingCounts;
} else if (typeof value === "number") {
// For numbers, add to existing sum
aggregatedData[key] = ((aggregatedData[key] as number) || 0) + value;
} else if (value !== undefined && value !== null) {
// For other types (strings, booleans, etc.), count occurrences
const stringValue = String(value);
const existingValue = aggregatedData[key];
if (!existingValue) {
aggregatedData[key] = { [stringValue]: 1 };
} else if (existingValue && typeof existingValue === "object" && existingValue.constructor === Object) {
const countObject = existingValue as Record<string, number>;
countObject[stringValue] = (countObject[stringValue] || 0) + 1;
} else {
const oldValue = String(existingValue);
aggregatedData[key] = {
[oldValue]: 1,
[stringValue]: 1
};
}
}
});
});
return aggregatedData;
};
const processBucketEvents = async (eventType: string, bucketId: string) => {
if (!postHog) return 0;
try {
const bucketPattern = `telemetry-event-${eventType}-${bucketId}-*`;
const bucketKeys = await keyStore.getKeysByPattern(bucketPattern);
if (bucketKeys.length === 0) return 0;
const bucketEvents = await keyStore.getItems(bucketKeys);
let bucketEventsParsed: SingleEventData[] = [];
try {
bucketEventsParsed = bucketEvents
.filter((event) => event !== null)
.map((event) => JSON.parse(event as string) as SingleEventData);
} catch (error) {
logger.error(error, `Failed to parse bucket events for ${eventType} in ${bucketId}`);
return 0;
}
const eventsGrouped = new Map<string, SingleEventData[]>();
bucketEventsParsed.forEach((event) => {
const key = JSON.stringify({ id: event.distinctId, org: event.organizationId });
if (!eventsGrouped.has(key)) {
eventsGrouped.set(key, []);
}
eventsGrouped.get(key)!.push(event);
});
if (eventsGrouped.size === 0) return 0;
for (const [eventsKey, events] of eventsGrouped) {
const key = JSON.parse(eventsKey) as { id: string; org?: string };
if (key.org) {
try {
postHog.groupIdentify({ groupType: "organization", groupKey: key.org });
} catch (error) {
logger.error(error, "Failed to identify PostHog organization");
}
}
const properties = aggregateGroupProperties(events);
postHog.capture({
event: `${eventType} aggregated`,
distinctId: key.id,
properties,
...(key.org ? { groups: { organization: key.org } } : {})
});
}
// Clean up processed data for this bucket
await keyStore.deleteItemsByKeyIn(bucketKeys);
logger.info(`Processed ${bucketEventsParsed.length} events from bucket ${bucketId} for ${eventType}`);
return bucketEventsParsed.length;
} catch (error) {
logger.error(error, `Failed to process bucket ${bucketId} for ${eventType}`);
return 0;
}
};
const processAggregatedEvents = async () => {
if (!postHog) return;
for (const eventType of POSTHOG_AGGREGATED_EVENTS) {
let totalProcessed = 0;
logger.info(`Starting bucket processing for ${eventType}`);
// Process each bucket sequentially to control memory usage
for (const bucketId of TELEMETRY_BUCKET_NAMES) {
try {
// eslint-disable-next-line no-await-in-loop
const processed = await processBucketEvents(eventType, bucketId);
totalProcessed += processed;
} catch (error) {
logger.error(error, `Failed to process bucket ${bucketId} for ${eventType}`);
}
}
logger.info(`Completed processing ${totalProcessed} total events for ${eventType}`);
}
};
const flushAll = async () => {
if (postHog) {
await postHog.shutdownAsync();
@ -98,6 +312,8 @@ To opt into telemetry, you can set "TELEMETRY_ENABLED=true" within the environme
return {
sendLoopsEvent,
sendPostHogEvents,
flushAll
processAggregatedEvents,
flushAll,
getBucketForDistinctId
};
};

View File

@ -1,3 +1,13 @@
import {
IdentityActor,
KmipClientActor,
PlatformActor,
ScimClientActor,
ServiceActor,
UnknownUserActor,
UserActor
} from "@app/ee/services/audit-log/audit-log-types";
export enum PostHogEventTypes {
SecretPush = "secrets pushed",
SecretPulled = "secrets pulled",
@ -40,6 +50,14 @@ export type TSecretModifiedEvent = {
secretPath: string;
channel?: string;
userAgent?: string;
actor?:
| UserActor
| IdentityActor
| ServiceActor
| ScimClientActor
| PlatformActor
| UnknownUserActor
| KmipClientActor;
};
};
@ -214,7 +232,7 @@ export type TInvalidateCacheEvent = {
};
};
export type TPostHogEvent = { distinctId: string } & (
export type TPostHogEvent = { distinctId: string; organizationId?: string } & (
| TSecretModifiedEvent
| TAdminInitEvent
| TUserSignedUpEvent

View File

@ -11,9 +11,9 @@ Fairly frequently, you might run into situations when you need to spend company
As a perk of working at Infisical, we cover some of your meal expenses.
HQ team members: meals and unlimited snacks are provided on-site at no cost.
**HQ team members**: meals and unlimited snacks are provided **on-site** at no cost.
Remote team members: a food stipend is allocated based on location.
**Remote team members**: a food stipend is allocated based on location.
# Trivial expenses
@ -27,21 +27,28 @@ This means expenses that are:
Please spend money in a way that you think is in the best interest of the company.
</Note>
## Saving receipts
Make sure you keep copies for all receipts. If you expense something on a company card and cannot provide a receipt, this may be deducted from your pay.
# Travel
You should default to using your company card in all cases - it has no transaction fees. If using your personal card is unavoidable, please reach out to Maidul to get it reimbursed manually.
If you need to travel on Infisicals behalf for in-person onboarding, meeting customers, and offsites, again please spend money in the best interests of the company.
## Training
We do not pre-approve your travel expenses, and trust team members to make the right decisions here. Some guidance:
- Please find a flight ticket that is reasonably priced. We all travel economy by default we cannot afford for folks to fly premium or business class. Feel free to upgrade using your personal money/airmiles if youd like to.
- Feel free to pay for the Uber/subway/bus to and from the airport with your Brex card.
- For business travel, Infisical will cover reasonable expenses for breakfast, lunch, and dinner.
- When traveling internationally, Infisical does not cover roaming charges for your phone. You can expense a reasonable eSIM, which usually is no more than $20.
<Note>
Note that this only applies to business travel. It is not applicable for personal travel or day-to-day commuting.
</Note>
For engineers, youre welcome to take an approved Udemy course. Please reach out to Maidul. For the GTM team, you may buy a book a month if its relevant to your work.
# Equipment
Infisical is a remote first company so we understand the importance of having a comfortable work setup. To support this, we provide allowances for essential office equipment.
### Desk & Chair
### 1. Desk & Chair
Most people already have a comfortable desk and chair, but if you need an upgrade, we offer the following allowances.
While we're not yet able to provide the latest and greatest, we strive to be reasonable given the stage of our company.
@ -50,10 +57,10 @@ While we're not yet able to provide the latest and greatest, we strive to be rea
**Chair**: $150 USD
### Laptop
### 2. Laptop
Each team member will receive a company-issued Macbook Pro before they start their first day.
### Notes
### 3. Notes
1. All equipment purchased using company allowances remains the property of Infisical.
2. Keep all receipts for equipment purchases and submit them for reimbursement.
@ -65,6 +72,28 @@ This is because we don't yet have a formal HR department to handle such logistic
For any equipment related questions, please reach out to Maidul.
## Brex
# Brex
We use Brex as our primary credit card provider. Don't have a company card yet? Reach out to Maidul.
### Budgets
You will generally have multiple budgets assigned to you. "General Company Expenses" primarily covers quick SaaS purchases (not food). Remote team members should have a "Lunch Stipend" budget that applies to food.
If your position involves a lot of travel, you may also have a "Travel" budget that applies to expenses related to business travel (e.g., you can not use it for transportation or food during personal travel).
### Saving receipts
Make sure you keep copies for all receipts. If you expense something on a company card and cannot provide a receipt, this may be deducted from your pay.
You should default to using your company card in all cases - it has no transaction fees. If using your personal card is unavoidable, please reach out to Maidul to get it reimbursed manually.
### Need a one-off budget increase?
You can do this directly within Brex - just request the amount and duration for the relevant budget in the app, and your hiring manager will automatically be notified for approval.
# Training
For engineers, youre welcome to take an approved Udemy course. Please reach out to Maidul. For the GTM team, you may buy a book a month if its relevant to your work.

View File

@ -4,7 +4,7 @@ services:
nginx:
container_name: infisical-dev-nginx
image: nginx
restart: always
restart: "always"
ports:
- 8080:80
- 8443:443

View File

@ -0,0 +1,4 @@
---
title: "Available"
openapi: "GET /api/v1/app-connections/gitlab/available"
---

View File

@ -0,0 +1,10 @@
---
title: "Create"
openapi: "POST /api/v1/app-connections/gitlab"
---
<Note>
Gitlab OAuth Connections must be created through the Infisical UI.
Check out the configuration docs for [Gitlab OAuth Connections](/integrations/app-connections/gitlab) for a step-by-step
guide.
</Note>

View File

@ -0,0 +1,4 @@
---
title: "Delete"
openapi: "DELETE /api/v1/app-connections/gitlab/{connectionId}"
---

View File

@ -0,0 +1,4 @@
---
title: "Get by ID"
openapi: "GET /api/v1/app-connections/gitlab/{connectionId}"
---

View File

@ -0,0 +1,4 @@
---
title: "Get by Name"
openapi: "GET /api/v1/app-connections/gitlab/connection-name/{connectionName}"
---

View File

@ -0,0 +1,4 @@
---
title: "List"
openapi: "GET /api/v1/app-connections/gitlab"
---

View File

@ -0,0 +1,10 @@
---
title: "Update"
openapi: "PATCH /api/v1/app-connections/gitlab/{connectionId}"
---
<Note>
Gitlab OAuth Connections must be updated through the Infisical UI.
Check out the configuration docs for [Gitlab OAuth Connections](/integrations/app-connections/gitlab) for a step-by-step
guide.
</Note>

View File

@ -0,0 +1,4 @@
---
title: "Create"
openapi: "POST /api/v1/secret-syncs/gitlab"
---

View File

@ -0,0 +1,4 @@
---
title: "Delete"
openapi: "DELETE /api/v1/secret-syncs/gitlab/{syncId}"
---

View File

@ -0,0 +1,4 @@
---
title: "Get by ID"
openapi: "GET /api/v1/secret-syncs/gitlab/{syncId}"
---

View File

@ -0,0 +1,4 @@
---
title: "Get by Name"
openapi: "GET /api/v1/secret-syncs/gitlab/sync-name/{syncName}"
---

View File

@ -0,0 +1,4 @@
---
title: "List"
openapi: "GET /api/v1/secret-syncs/gitlab"
---

View File

@ -0,0 +1,4 @@
---
title: "Remove Secrets"
openapi: "POST /api/v1/secret-syncs/gitlab/{syncId}/remove-secrets"
---

View File

@ -0,0 +1,4 @@
---
title: "Sync Secrets"
openapi: "POST /api/v1/secret-syncs/gitlab/{syncId}/sync-secrets"
---

View File

@ -0,0 +1,4 @@
---
title: "Update"
openapi: "PATCH /api/v1/secret-syncs/gitlab/{syncId}"
---

View File

@ -0,0 +1,4 @@
---
title: "Attach"
openapi: "POST /api/v1/auth/tls-cert-auth/identities/{identityId}"
---

Some files were not shown because too many files have changed in this diff Show More