Compare commits

...

83 Commits

Author SHA1 Message Date
fd85da5739 set trusted ip to empty 2025-06-27 20:36:32 -04:00
2caf6ff94b remove cert auth left join 2025-06-27 20:21:28 -04:00
aff97374a9 Merge pull request #3868 from Infisical/misc/add-mention-of-service-usage-api-for-gcp
misc: add mention of service usage API for GCP
2025-06-28 04:26:21 +08:00
e8e90585ca Merge pull request #3871 from Infisical/project-role-type-col
improvement(project-roles): Add type col to project roles table and default sort
2025-06-27 11:42:47 -07:00
abd9dbf714 improvement: add type col to project roles table and default sort 2025-06-27 11:34:54 -07:00
89aed3640b Merge pull request #3852 from akhilmhdh/feat/tls-identity-auth
feat: TLS cert identity auth
2025-06-28 02:29:25 +08:00
5513ff7631 Merge pull request #3866 from Infisical/feat/posthogEventBatch
feat(telemetry): Add aggregated events and groups to posthog
2025-06-27 14:42:55 -03:00
9fb7676739 misc: reordered doc for mi auth 2025-06-28 01:35:46 +08:00
6ac734d6c4 removed unnecessary changes 2025-06-28 01:32:53 +08:00
8044999785 feat(telemetry): increase even redis key exp to 15 mins 2025-06-27 14:31:54 -03:00
be51e4372d feat(telemetry): addressed PR suggestions 2025-06-27 14:30:31 -03:00
460b545925 Merge branch 'feat/tls-identity-auth' of https://github.com/akhilmhdh/infisical into HEAD 2025-06-28 01:29:49 +08:00
2f26c1930b misc: doc updates 2025-06-28 01:26:24 +08:00
fc9ae05f89 misc: updated TLS acronym 2025-06-28 00:21:08 +08:00
de22a3c56b misc: updated casing of acronym 2025-06-28 00:17:42 +08:00
7c4baa6fd4 misc: added image for service usage API 2025-06-27 13:19:14 +00:00
f285648c95 misc: add mention of service usage API for GCP 2025-06-27 21:10:02 +08:00
0f04890d8f feat(telemetry): addressed PR suggestions 2025-06-26 21:18:07 -03:00
61274243e2 feat(telemetry): add batch events and groups logic 2025-06-26 20:58:01 -03:00
9366428091 Merge pull request #3865 from Infisical/remove-manual-styled-css-on-checkboxes
fix(checkbox): Remove manual css overrides of checkbox checked state
2025-06-26 15:38:05 -07:00
62482852aa fix: remove manual css overrides of checkbox checked state 2025-06-26 15:33:27 -07:00
cc02c00b61 Merge pull request #3864 from Infisical/update-aws-param-store-docs
Clarify relationship between path and key schema for AWS parameter store
2025-06-26 18:19:06 -04:00
2e256e4282 Tooltip 2025-06-26 18:14:48 -04:00
1b4bae6a84 Merge pull request #3863 from Infisical/remove-secret-scanning-v1-backend
chore(secret-scanning-v1): remove secret scanning v1 queue and webhook endpoint
2025-06-26 14:51:23 -07:00
1f0bcae0fc Merge pull request #3860 from Infisical/secret-sync-selection-improvements
improvement(secret-sync/app-connection): Add search/pagination to secret sync and app connection selection modals
2025-06-26 14:50:44 -07:00
dcd21883d1 Clarify relationship between path and key schema for AWS parameter store
docs
2025-06-26 17:02:21 -04:00
d7913a75c2 chore: remove secret scanning v1 queue and webhook endpoint 2025-06-26 11:32:45 -07:00
205442bff5 Merge pull request #3859 from Infisical/overview-ui-improvements
improvement(secret-overview): Add collapsed environment view to secret overview page
2025-06-26 09:24:33 -07:00
8ab51aba12 improvement: add search/pagination app connection select 2025-06-26 09:21:35 -07:00
3d1f054b87 improvement: add pagination/search to secret sync selection 2025-06-26 08:13:57 -07:00
29fedfdde5 Merge pull request #3850 from Infisical/policy-edit-revisions
improvement(project-policies): Revamp edit role page and access tree
2025-06-26 07:46:35 -07:00
b5317d1d75 fix: add ability to remove non-conditional rules 2025-06-26 07:37:30 -07:00
6446311b6d Merge pull request #3835 from Infisical/feat/gitlabSecretSync
feat(secret-sync): Add gitlab secret sync
2025-06-25 17:53:12 -03:00
3e80f1907c Merge pull request #3857 from Infisical/daniel/fix-dotnet-docs
docs: fix redirect for .NET SDK
2025-06-25 23:18:14 +04:00
79e62eec25 docs: fix redirect for .NET SDK 2025-06-25 23:11:11 +04:00
c41730c5fb Merge pull request #3856 from Infisical/daniel/fix-docs
fix(docs): sdk and changelog tab not loading
2025-06-25 22:34:09 +04:00
aac63d3097 fix(docs): sdk and changelog tab not working 2025-06-25 22:32:08 +04:00
f0b9d3c816 feat(secret-sync): improve hide secrets tooltip message 2025-06-25 14:10:28 -03:00
ea393d144a feat(secret-sync): minor change on docs 2025-06-25 13:57:07 -03:00
c4c0f86598 feat(secret-sync): improve update logic and add warning on docs for gitlab limitation on hidden variables 2025-06-25 13:51:38 -03:00
1f7617d132 Merge pull request #3851 from Infisical/ENG-3013
Allow undefined value for tags to prevent unwanted overrides
2025-06-25 12:45:43 -04:00
c95680b95d feat(secret-sync): type fix 2025-06-25 13:33:43 -03:00
18f1f93b5f Review fixes 2025-06-25 12:29:23 -04:00
70ea761375 feat(secret-sync): fix update masked_and_hidden field to not be sent unless it's true 2025-06-25 13:17:41 -03:00
5b4790ee78 improvements: truncate environment selection and only show visualize access when expanded 2025-06-25 09:09:08 -07:00
5ab2a6bb5d Feedback 2025-06-25 11:56:11 -04:00
dcac85fe6c Merge pull request #3847 from Infisical/share-your-own-secret-link-fix
fix(secret-sharing): Support self-hosted for "share your own secret" link
2025-06-25 08:31:13 -07:00
2f07471404 Merge pull request #3853 from akhilmhdh/feat/copy-token
feat: added copy token button
2025-06-25 10:55:07 -04:00
137fd5ef07 added minor text updates 2025-06-25 10:50:16 -04:00
=
883c7835a1 feat: added copy token button 2025-06-25 15:28:58 +05:30
=
e33f34ceb4 fix: corrected the doc key 2025-06-25 14:46:13 +05:30
=
af5805a5ca feat: resolved incorrect invalidation 2025-06-25 14:46:13 +05:30
bcf1c49a1b Update docs/documentation/platform/identities/tls-cert-auth.mdx
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-06-25 14:45:14 +05:30
84fedf8eda Update docs/documentation/platform/identities/tls-cert-auth.mdx
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-06-25 14:44:45 +05:30
97755981eb Update docs/documentation/platform/identities/tls-cert-auth.mdx
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-06-25 14:43:01 +05:30
8291663802 Update frontend/src/pages/organization/AccessManagementPage/components/OrgIdentityTab/components/IdentitySection/IdentityTlsCertAuthForm.tsx
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-06-25 14:42:24 +05:30
d9aed45504 Update frontend/src/pages/organization/AccessManagementPage/components/OrgIdentityTab/components/IdentitySection/IdentityTlsCertAuthForm.tsx
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-06-25 14:42:11 +05:30
=
8ada11edf3 feat: docs for tls cert auth 2025-06-25 14:27:04 +05:30
=
4bd62aa462 feat: updated frontend to have the tls cert auth login 2025-06-25 14:26:55 +05:30
0366e58a5b Type fix 2025-06-25 00:24:24 -03:00
9f6dca23db Greptile reviews 2025-06-24 23:19:42 -04:00
18e733c71f feat(secret-sync): minor fixes 2025-06-25 00:16:44 -03:00
f0a95808e7 Allow undefined value for tags to prevent unwanted overrides 2025-06-24 23:13:53 -04:00
90a0d0f744 Merge pull request #3848 from Infisical/improve-audit-log-streams
improve audit log streams: add backend logs + DD source
2025-06-24 22:18:04 -04:00
7f9c9be2c8 review fix 2025-06-24 22:00:45 -04:00
070982081c Merge remote-tracking branch 'origin/main' into feat/gitlabSecretSync 2025-06-24 22:42:28 -03:00
f462c3f85d feat(secret-sync): minor fixes 2025-06-24 21:38:33 -03:00
8683693103 improvement: address greptile feedback 2025-06-24 15:35:42 -07:00
737fffcceb improvement: address greptile feedback 2025-06-24 15:35:08 -07:00
ffac24ce75 improvement: revise edit role page and access tree 2025-06-24 15:23:27 -07:00
c505c5877f feat(secret-sync): updated docs 2025-06-24 18:11:18 -03:00
d4bf8a33dc feat(secret-sync): rework GitLab secret-sync to add group variables 2025-06-24 18:01:32 -03:00
6566393e21 Review fixes 2025-06-24 14:39:46 -04:00
af245b1f16 Add "service: audit-logs" entry for DataDog 2025-06-24 14:22:26 -04:00
c17df7e951 Improve URL detection 2025-06-24 12:44:16 -04:00
4d4953e95a improve audit log streams: add backend logs + DD source 2025-06-24 12:35:49 -04:00
43e0d400f9 feat(secret-sync): add Gitlab PR comments suggestions 2025-06-24 10:05:46 -03:00
=
b80b77ec36 feat: completed backend changes for tls auth 2025-06-24 16:46:46 +05:30
198e74cd88 fix: include nooppener in window.open 2025-06-23 18:05:48 -07:00
8ed0a1de84 fix: correct window open for share your own secret link to handle self-hosted 2025-06-23 18:01:38 -07:00
c305ddd463 feat(secret-sync): Gitlab PR suggestions 2025-06-23 10:52:59 -03:00
27cb686216 feat(secret-sync): Fix frontend file names 2025-06-20 21:26:12 -03:00
e201d77a8f feat(secret-sync): Add gitlab secret sync 2025-06-20 21:13:14 -03:00
230 changed files with 9193 additions and 2631 deletions

View File

@ -107,6 +107,10 @@ INF_APP_CONNECTION_GITHUB_APP_PRIVATE_KEY=
INF_APP_CONNECTION_GITHUB_APP_SLUG=
INF_APP_CONNECTION_GITHUB_APP_ID=
#gitlab app connection
INF_APP_CONNECTION_GITLAB_OAUTH_CLIENT_ID=
INF_APP_CONNECTION_GITLAB_OAUTH_CLIENT_SECRET=
#github radar app connection
INF_APP_CONNECTION_GITHUB_RADAR_APP_CLIENT_ID=
INF_APP_CONNECTION_GITHUB_RADAR_APP_CLIENT_SECRET=

View File

@ -8,6 +8,9 @@ import { Lock } from "@app/lib/red-lock";
export const mockKeyStore = (): TKeyStoreFactory => {
const store: Record<string, string | number | Buffer> = {};
const getRegex = (pattern: string) =>
new RE2(`^${pattern.replace(/[-[\]/{}()+?.\\^$|]/g, "\\$&").replace(/\*/g, ".*")}$`);
return {
setItem: async (key, value) => {
store[key] = value;
@ -23,7 +26,7 @@ export const mockKeyStore = (): TKeyStoreFactory => {
return 1;
},
deleteItems: async ({ pattern, batchSize = 500, delay = 1500, jitter = 200 }) => {
const regex = new RE2(`^${pattern.replace(/[-[\]/{}()+?.\\^$|]/g, "\\$&").replace(/\*/g, ".*")}$`);
const regex = getRegex(pattern);
let totalDeleted = 0;
const keys = Object.keys(store);
@ -53,6 +56,27 @@ export const mockKeyStore = (): TKeyStoreFactory => {
incrementBy: async () => {
return 1;
},
getItems: async (keys) => {
const values = keys.map((key) => {
const value = store[key];
if (typeof value === "string") {
return value;
}
return null;
});
return values;
},
getKeysByPattern: async (pattern) => {
const regex = getRegex(pattern);
const keys = Object.keys(store);
return keys.filter((key) => regex.test(key));
},
deleteItemsByKeyIn: async (keys) => {
for (const key of keys) {
delete store[key];
}
return keys.length;
},
acquireLock: () => {
return Promise.resolve({
release: () => {}

View File

@ -30,6 +30,7 @@
"@fastify/static": "^7.0.4",
"@fastify/swagger": "^8.14.0",
"@fastify/swagger-ui": "^2.1.0",
"@gitbeaker/rest": "^42.5.0",
"@google-cloud/kms": "^4.5.0",
"@infisical/quic": "^1.0.8",
"@node-saml/passport-saml": "^5.0.1",
@ -7807,6 +7808,48 @@
"p-limit": "^3.1.0"
}
},
"node_modules/@gitbeaker/core": {
"version": "42.5.0",
"resolved": "https://registry.npmjs.org/@gitbeaker/core/-/core-42.5.0.tgz",
"integrity": "sha512-rMWpOPaZi1iLiifnOIoVO57p2EmQQdfIwP4txqNyMvG4WjYP5Ez0U7jRD9Nra41x6K5kTPBZkuQcAdxVWRJcEQ==",
"license": "MIT",
"dependencies": {
"@gitbeaker/requester-utils": "^42.5.0",
"qs": "^6.12.2",
"xcase": "^2.0.1"
},
"engines": {
"node": ">=18.20.0"
}
},
"node_modules/@gitbeaker/requester-utils": {
"version": "42.5.0",
"resolved": "https://registry.npmjs.org/@gitbeaker/requester-utils/-/requester-utils-42.5.0.tgz",
"integrity": "sha512-HLdLS9LPBMVQumvroQg/4qkphLDtwDB+ygEsrD2u4oYCMUtXV4V1xaVqU4yTXjbTJ5sItOtdB43vYRkBcgueBw==",
"license": "MIT",
"dependencies": {
"picomatch-browser": "^2.2.6",
"qs": "^6.12.2",
"rate-limiter-flexible": "^4.0.1",
"xcase": "^2.0.1"
},
"engines": {
"node": ">=18.20.0"
}
},
"node_modules/@gitbeaker/rest": {
"version": "42.5.0",
"resolved": "https://registry.npmjs.org/@gitbeaker/rest/-/rest-42.5.0.tgz",
"integrity": "sha512-oC5cM6jS7aFOp0luTw5mWSRuMgdxwHRLZQ/aWkI+ETMfsprR/HyxsXfljlMY/XJ/fRxTbRJiodR5Axf66WjO3w==",
"license": "MIT",
"dependencies": {
"@gitbeaker/core": "^42.5.0",
"@gitbeaker/requester-utils": "^42.5.0"
},
"engines": {
"node": ">=18.20.0"
}
},
"node_modules/@google-cloud/kms": {
"version": "4.5.0",
"resolved": "https://registry.npmjs.org/@google-cloud/kms/-/kms-4.5.0.tgz",
@ -24628,6 +24671,18 @@
"url": "https://github.com/sponsors/jonschlinkert"
}
},
"node_modules/picomatch-browser": {
"version": "2.2.6",
"resolved": "https://registry.npmjs.org/picomatch-browser/-/picomatch-browser-2.2.6.tgz",
"integrity": "sha512-0ypsOQt9D4e3hziV8O4elD9uN0z/jtUEfxVRtNaAAtXIyUx9m/SzlO020i8YNL2aL/E6blOvvHQcin6HZlFy/w==",
"license": "MIT",
"engines": {
"node": ">=8.6"
},
"funding": {
"url": "https://github.com/sponsors/jonschlinkert"
}
},
"node_modules/pify": {
"version": "4.0.1",
"resolved": "https://registry.npmjs.org/pify/-/pify-4.0.1.tgz",
@ -25562,6 +25617,12 @@
"node": ">= 0.6"
}
},
"node_modules/rate-limiter-flexible": {
"version": "4.0.1",
"resolved": "https://registry.npmjs.org/rate-limiter-flexible/-/rate-limiter-flexible-4.0.1.tgz",
"integrity": "sha512-2/dGHpDFpeA0+755oUkW+EKyklqLS9lu0go9pDsbhqQjZcxfRyJ6LA4JI0+HAdZ2bemD/oOjUeZQB2lCZqXQfQ==",
"license": "ISC"
},
"node_modules/raw-body": {
"version": "2.5.2",
"resolved": "https://registry.npmjs.org/raw-body/-/raw-body-2.5.2.tgz",
@ -31039,6 +31100,12 @@
}
}
},
"node_modules/xcase": {
"version": "2.0.1",
"resolved": "https://registry.npmjs.org/xcase/-/xcase-2.0.1.tgz",
"integrity": "sha512-UmFXIPU+9Eg3E9m/728Bii0lAIuoc+6nbrNUKaRPJOFp91ih44qqGlWtxMB6kXFrRD6po+86ksHM5XHCfk6iPw==",
"license": "MIT"
},
"node_modules/xml-crypto": {
"version": "6.0.1",
"resolved": "https://registry.npmjs.org/xml-crypto/-/xml-crypto-6.0.1.tgz",

View File

@ -149,6 +149,7 @@
"@fastify/static": "^7.0.4",
"@fastify/swagger": "^8.14.0",
"@fastify/swagger-ui": "^2.1.0",
"@gitbeaker/rest": "^42.5.0",
"@google-cloud/kms": "^4.5.0",
"@infisical/quic": "^1.0.8",
"@node-saml/passport-saml": "^5.0.1",

View File

@ -74,6 +74,7 @@ import { TAllowedFields } from "@app/services/identity-ldap-auth/identity-ldap-a
import { TIdentityOciAuthServiceFactory } from "@app/services/identity-oci-auth/identity-oci-auth-service";
import { TIdentityOidcAuthServiceFactory } from "@app/services/identity-oidc-auth/identity-oidc-auth-service";
import { TIdentityProjectServiceFactory } from "@app/services/identity-project/identity-project-service";
import { TIdentityTlsCertAuthServiceFactory } from "@app/services/identity-tls-cert-auth/identity-tls-cert-auth-types";
import { TIdentityTokenAuthServiceFactory } from "@app/services/identity-token-auth/identity-token-auth-service";
import { TIdentityUaServiceFactory } from "@app/services/identity-ua/identity-ua-service";
import { TIntegrationServiceFactory } from "@app/services/integration/integration-service";
@ -218,6 +219,7 @@ declare module "fastify" {
identityKubernetesAuth: TIdentityKubernetesAuthServiceFactory;
identityGcpAuth: TIdentityGcpAuthServiceFactory;
identityAliCloudAuth: TIdentityAliCloudAuthServiceFactory;
identityTlsCertAuth: TIdentityTlsCertAuthServiceFactory;
identityAwsAuth: TIdentityAwsAuthServiceFactory;
identityAzureAuth: TIdentityAzureAuthServiceFactory;
identityOciAuth: TIdentityOciAuthServiceFactory;

View File

@ -164,6 +164,9 @@ import {
TIdentityProjectMemberships,
TIdentityProjectMembershipsInsert,
TIdentityProjectMembershipsUpdate,
TIdentityTlsCertAuths,
TIdentityTlsCertAuthsInsert,
TIdentityTlsCertAuthsUpdate,
TIdentityTokenAuths,
TIdentityTokenAuthsInsert,
TIdentityTokenAuthsUpdate,
@ -794,6 +797,11 @@ declare module "knex/types/tables" {
TIdentityAlicloudAuthsInsert,
TIdentityAlicloudAuthsUpdate
>;
[TableName.IdentityTlsCertAuth]: KnexOriginal.CompositeTableType<
TIdentityTlsCertAuths,
TIdentityTlsCertAuthsInsert,
TIdentityTlsCertAuthsUpdate
>;
[TableName.IdentityAwsAuth]: KnexOriginal.CompositeTableType<
TIdentityAwsAuths,
TIdentityAwsAuthsInsert,

View File

@ -0,0 +1,28 @@
import { Knex } from "knex";
import { TableName } from "../schemas";
import { createOnUpdateTrigger, dropOnUpdateTrigger } from "../utils";
export async function up(knex: Knex): Promise<void> {
if (!(await knex.schema.hasTable(TableName.IdentityTlsCertAuth))) {
await knex.schema.createTable(TableName.IdentityTlsCertAuth, (t) => {
t.uuid("id", { primaryKey: true }).defaultTo(knex.fn.uuid());
t.bigInteger("accessTokenTTL").defaultTo(7200).notNullable();
t.bigInteger("accessTokenMaxTTL").defaultTo(7200).notNullable();
t.bigInteger("accessTokenNumUsesLimit").defaultTo(0).notNullable();
t.jsonb("accessTokenTrustedIps").notNullable();
t.timestamps(true, true, true);
t.uuid("identityId").notNullable().unique();
t.foreign("identityId").references("id").inTable(TableName.Identity).onDelete("CASCADE");
t.string("allowedCommonNames").nullable();
t.binary("encryptedCaCertificate").notNullable();
});
}
await createOnUpdateTrigger(knex, TableName.IdentityTlsCertAuth);
}
export async function down(knex: Knex): Promise<void> {
await knex.schema.dropTableIfExists(TableName.IdentityTlsCertAuth);
await dropOnUpdateTrigger(knex, TableName.IdentityTlsCertAuth);
}

View File

@ -0,0 +1,27 @@
// Code generated by automation script, DO NOT EDIT.
// Automated by pulling database and generating zod schema
// To update. Just run npm run generate:schema
// Written by akhilmhdh.
import { z } from "zod";
import { zodBuffer } from "@app/lib/zod";
import { TImmutableDBKeys } from "./models";
export const IdentityTlsCertAuthsSchema = z.object({
id: z.string().uuid(),
accessTokenTTL: z.coerce.number().default(7200),
accessTokenMaxTTL: z.coerce.number().default(7200),
accessTokenNumUsesLimit: z.coerce.number().default(0),
accessTokenTrustedIps: z.unknown(),
createdAt: z.date(),
updatedAt: z.date(),
identityId: z.string().uuid(),
allowedCommonNames: z.string().nullable().optional(),
encryptedCaCertificate: zodBuffer
});
export type TIdentityTlsCertAuths = z.infer<typeof IdentityTlsCertAuthsSchema>;
export type TIdentityTlsCertAuthsInsert = Omit<z.input<typeof IdentityTlsCertAuthsSchema>, TImmutableDBKeys>;
export type TIdentityTlsCertAuthsUpdate = Partial<Omit<z.input<typeof IdentityTlsCertAuthsSchema>, TImmutableDBKeys>>;

View File

@ -52,6 +52,7 @@ export * from "./identity-org-memberships";
export * from "./identity-project-additional-privilege";
export * from "./identity-project-membership-role";
export * from "./identity-project-memberships";
export * from "./identity-tls-cert-auths";
export * from "./identity-token-auths";
export * from "./identity-ua-client-secrets";
export * from "./identity-universal-auths";

View File

@ -86,6 +86,7 @@ export enum TableName {
IdentityOidcAuth = "identity_oidc_auths",
IdentityJwtAuth = "identity_jwt_auths",
IdentityLdapAuth = "identity_ldap_auths",
IdentityTlsCertAuth = "identity_tls_cert_auths",
IdentityOrgMembership = "identity_org_memberships",
IdentityProjectMembership = "identity_project_memberships",
IdentityProjectMembershipRole = "identity_project_membership_role",
@ -251,6 +252,7 @@ export enum IdentityAuthMethod {
ALICLOUD_AUTH = "alicloud-auth",
AWS_AUTH = "aws-auth",
AZURE_AUTH = "azure-auth",
TLS_CERT_AUTH = "tls-cert-auth",
OCI_AUTH = "oci-auth",
OIDC_AUTH = "oidc-auth",
JWT_AUTH = "jwt-auth",

View File

@ -80,6 +80,7 @@ export const registerSshCertRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SignSshKey,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
certificateTemplateId: req.body.certificateTemplateId,
principals: req.body.principals,
@ -171,6 +172,7 @@ export const registerSshCertRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.IssueSshCreds,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
certificateTemplateId: req.body.certificateTemplateId,
principals: req.body.principals,

View File

@ -358,6 +358,7 @@ export const registerSshHostRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.IssueSshHostUserCert,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
sshHostId: req.params.sshHostId,
hostname: host.hostname,
@ -427,6 +428,7 @@ export const registerSshHostRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.IssueSshHostHostCert,
organizationId: req.permission.orgId,
distinctId: getTelemetryDistinctId(req),
properties: {
sshHostId: req.params.sshHostId,

View File

@ -0,0 +1,21 @@
export function providerSpecificPayload(url: string) {
const { hostname } = new URL(url);
const payload: Record<string, string> = {};
switch (hostname) {
case "http-intake.logs.datadoghq.com":
case "http-intake.logs.us3.datadoghq.com":
case "http-intake.logs.us5.datadoghq.com":
case "http-intake.logs.datadoghq.eu":
case "http-intake.logs.ap1.datadoghq.com":
case "http-intake.logs.ddog-gov.com":
payload.ddsource = "infisical";
payload.service = "audit-logs";
break;
default:
break;
}
return payload;
}

View File

@ -13,6 +13,7 @@ import { TLicenseServiceFactory } from "../license/license-service";
import { OrgPermissionActions, OrgPermissionSubjects } from "../permission/org-permission";
import { TPermissionServiceFactory } from "../permission/permission-service-types";
import { TAuditLogStreamDALFactory } from "./audit-log-stream-dal";
import { providerSpecificPayload } from "./audit-log-stream-fns";
import { LogStreamHeaders, TAuditLogStreamServiceFactory } from "./audit-log-stream-types";
type TAuditLogStreamServiceFactoryDep = {
@ -69,10 +70,11 @@ export const auditLogStreamServiceFactory = ({
headers.forEach(({ key, value }) => {
streamHeaders[key] = value;
});
await request
.post(
url,
{ ping: "ok" },
{ ...providerSpecificPayload(url), ping: "ok" },
{
headers: streamHeaders,
// request timeout
@ -137,7 +139,7 @@ export const auditLogStreamServiceFactory = ({
await request
.post(
url || logStream.url,
{ ping: "ok" },
{ ...providerSpecificPayload(url || logStream.url), ping: "ok" },
{
headers: streamHeaders,
// request timeout

View File

@ -1,13 +1,15 @@
import { RawAxiosRequestHeaders } from "axios";
import { AxiosError, RawAxiosRequestHeaders } from "axios";
import { SecretKeyEncoding } from "@app/db/schemas";
import { getConfig } from "@app/lib/config/env";
import { request } from "@app/lib/config/request";
import { infisicalSymmetricDecrypt } from "@app/lib/crypto/encryption";
import { logger } from "@app/lib/logger";
import { QueueJobs, QueueName, TQueueServiceFactory } from "@app/queue";
import { TProjectDALFactory } from "@app/services/project/project-dal";
import { TAuditLogStreamDALFactory } from "../audit-log-stream/audit-log-stream-dal";
import { providerSpecificPayload } from "../audit-log-stream/audit-log-stream-fns";
import { LogStreamHeaders } from "../audit-log-stream/audit-log-stream-types";
import { TLicenseServiceFactory } from "../license/license-service";
import { TAuditLogDALFactory } from "./audit-log-dal";
@ -128,13 +130,29 @@ export const auditLogQueueServiceFactory = async ({
headers[key] = value;
});
return request.post(url, auditLog, {
headers,
// request timeout
timeout: AUDIT_LOG_STREAM_TIMEOUT,
// connection timeout
signal: AbortSignal.timeout(AUDIT_LOG_STREAM_TIMEOUT)
});
try {
logger.info(`Streaming audit log [url=${url}] for org [orgId=${orgId}]`);
const response = await request.post(
url,
{ ...providerSpecificPayload(url), ...auditLog },
{
headers,
// request timeout
timeout: AUDIT_LOG_STREAM_TIMEOUT,
// connection timeout
signal: AbortSignal.timeout(AUDIT_LOG_STREAM_TIMEOUT)
}
);
logger.info(
`Successfully streamed audit log [url=${url}] for org [orgId=${orgId}] [response=${JSON.stringify(response.data)}]`
);
return response;
} catch (error) {
logger.error(
`Failed to stream audit log [url=${url}] for org [orgId=${orgId}] [error=${(error as AxiosError).message}]`
);
return error;
}
}
)
);
@ -218,13 +236,29 @@ export const auditLogQueueServiceFactory = async ({
headers[key] = value;
});
return request.post(url, auditLog, {
headers,
// request timeout
timeout: AUDIT_LOG_STREAM_TIMEOUT,
// connection timeout
signal: AbortSignal.timeout(AUDIT_LOG_STREAM_TIMEOUT)
});
try {
logger.info(`Streaming audit log [url=${url}] for org [orgId=${orgId}]`);
const response = await request.post(
url,
{ ...providerSpecificPayload(url), ...auditLog },
{
headers,
// request timeout
timeout: AUDIT_LOG_STREAM_TIMEOUT,
// connection timeout
signal: AbortSignal.timeout(AUDIT_LOG_STREAM_TIMEOUT)
}
);
logger.info(
`Successfully streamed audit log [url=${url}] for org [orgId=${orgId}] [response=${JSON.stringify(response.data)}]`
);
return response;
} catch (error) {
logger.error(
`Failed to stream audit log [url=${url}] for org [orgId=${orgId}] [error=${(error as AxiosError).message}]`
);
return error;
}
}
)
);

View File

@ -202,6 +202,12 @@ export enum EventType {
REVOKE_IDENTITY_ALICLOUD_AUTH = "revoke-identity-alicloud-auth",
GET_IDENTITY_ALICLOUD_AUTH = "get-identity-alicloud-auth",
LOGIN_IDENTITY_TLS_CERT_AUTH = "login-identity-tls-cert-auth",
ADD_IDENTITY_TLS_CERT_AUTH = "add-identity-tls-cert-auth",
UPDATE_IDENTITY_TLS_CERT_AUTH = "update-identity-tls-cert-auth",
REVOKE_IDENTITY_TLS_CERT_AUTH = "revoke-identity-tls-cert-auth",
GET_IDENTITY_TLS_CERT_AUTH = "get-identity-tls-cert-auth",
LOGIN_IDENTITY_AWS_AUTH = "login-identity-aws-auth",
ADD_IDENTITY_AWS_AUTH = "add-identity-aws-auth",
UPDATE_IDENTITY_AWS_AUTH = "update-identity-aws-auth",
@ -1141,6 +1147,53 @@ interface GetIdentityAliCloudAuthEvent {
};
}
interface LoginIdentityTlsCertAuthEvent {
type: EventType.LOGIN_IDENTITY_TLS_CERT_AUTH;
metadata: {
identityId: string;
identityTlsCertAuthId: string;
identityAccessTokenId: string;
};
}
interface AddIdentityTlsCertAuthEvent {
type: EventType.ADD_IDENTITY_TLS_CERT_AUTH;
metadata: {
identityId: string;
allowedCommonNames: string | null | undefined;
accessTokenTTL: number;
accessTokenMaxTTL: number;
accessTokenNumUsesLimit: number;
accessTokenTrustedIps: Array<TIdentityTrustedIp>;
};
}
interface DeleteIdentityTlsCertAuthEvent {
type: EventType.REVOKE_IDENTITY_TLS_CERT_AUTH;
metadata: {
identityId: string;
};
}
interface UpdateIdentityTlsCertAuthEvent {
type: EventType.UPDATE_IDENTITY_TLS_CERT_AUTH;
metadata: {
identityId: string;
allowedCommonNames: string | null | undefined;
accessTokenTTL?: number;
accessTokenMaxTTL?: number;
accessTokenNumUsesLimit?: number;
accessTokenTrustedIps?: Array<TIdentityTrustedIp>;
};
}
interface GetIdentityTlsCertAuthEvent {
type: EventType.GET_IDENTITY_TLS_CERT_AUTH;
metadata: {
identityId: string;
};
}
interface LoginIdentityOciAuthEvent {
type: EventType.LOGIN_IDENTITY_OCI_AUTH;
metadata: {
@ -3358,6 +3411,11 @@ export type Event =
| UpdateIdentityAliCloudAuthEvent
| GetIdentityAliCloudAuthEvent
| DeleteIdentityAliCloudAuthEvent
| LoginIdentityTlsCertAuthEvent
| AddIdentityTlsCertAuthEvent
| UpdateIdentityTlsCertAuthEvent
| GetIdentityTlsCertAuthEvent
| DeleteIdentityTlsCertAuthEvent
| LoginIdentityOciAuthEvent
| AddIdentityOciAuthEvent
| UpdateIdentityOciAuthEvent

View File

@ -1,17 +1,11 @@
import { ProbotOctokit } from "probot";
import { OrgMembershipRole, TableName } from "@app/db/schemas";
import { getConfig } from "@app/lib/config/env";
import { logger } from "@app/lib/logger";
import { QueueJobs, QueueName, TQueueServiceFactory } from "@app/queue";
import { InternalServerError } from "@app/lib/errors";
import { TQueueServiceFactory } from "@app/queue";
import { TOrgDALFactory } from "@app/services/org/org-dal";
import { SmtpTemplates, TSmtpService } from "@app/services/smtp/smtp-service";
import { TSmtpService } from "@app/services/smtp/smtp-service";
import { TTelemetryServiceFactory } from "@app/services/telemetry/telemetry-service";
import { PostHogEventTypes } from "@app/services/telemetry/telemetry-types";
import { TSecretScanningDALFactory } from "../secret-scanning-dal";
import { scanContentAndGetFindings, scanFullRepoContentAndGetFindings } from "./secret-scanning-fns";
import { SecretMatch, TScanFullRepoEventPayload, TScanPushEventPayload } from "./secret-scanning-queue-types";
import { TScanFullRepoEventPayload, TScanPushEventPayload } from "./secret-scanning-queue-types";
type TSecretScanningQueueFactoryDep = {
queueService: TQueueServiceFactory;
@ -23,227 +17,21 @@ type TSecretScanningQueueFactoryDep = {
export type TSecretScanningQueueFactory = ReturnType<typeof secretScanningQueueFactory>;
export const secretScanningQueueFactory = ({
queueService,
secretScanningDAL,
smtpService,
telemetryService,
orgMembershipDAL: orgMemberDAL
}: TSecretScanningQueueFactoryDep) => {
const startFullRepoScan = async (payload: TScanFullRepoEventPayload) => {
await queueService.queue(QueueName.SecretFullRepoScan, QueueJobs.SecretScan, payload, {
attempts: 3,
backoff: {
type: "exponential",
delay: 5000
},
removeOnComplete: true,
removeOnFail: {
count: 20 // keep the most recent 20 jobs
}
// eslint-disable-next-line @typescript-eslint/no-unused-vars
export const secretScanningQueueFactory = (_props: TSecretScanningQueueFactoryDep) => {
// eslint-disable-next-line @typescript-eslint/no-unused-vars
const startFullRepoScan = async (_payload: TScanFullRepoEventPayload) => {
throw new InternalServerError({
message: "Secret Scanning V1 has been deprecated. Please migrate to Secret Scanning V2"
});
};
const startPushEventScan = async (payload: TScanPushEventPayload) => {
await queueService.queue(QueueName.SecretPushEventScan, QueueJobs.SecretScan, payload, {
attempts: 3,
backoff: {
type: "exponential",
delay: 5000
},
removeOnComplete: true,
removeOnFail: {
count: 20 // keep the most recent 20 jobs
}
// eslint-disable-next-line @typescript-eslint/no-unused-vars
const startPushEventScan = async (_payload: TScanPushEventPayload) => {
throw new InternalServerError({
message: "Secret Scanning V1 has been deprecated. Please migrate to Secret Scanning V2"
});
};
const getOrgAdminEmails = async (organizationId: string) => {
// get emails of admins
const adminsOfWork = await orgMemberDAL.findMembership({
[`${TableName.Organization}.id` as string]: organizationId,
role: OrgMembershipRole.Admin
});
return adminsOfWork.filter((userObject) => userObject.email).map((userObject) => userObject.email as string);
};
queueService.start(QueueName.SecretPushEventScan, async (job) => {
const appCfg = getConfig();
const { organizationId, commits, pusher, repository, installationId } = job.data;
const [owner, repo] = repository.fullName.split("/");
const octokit = new ProbotOctokit({
auth: {
appId: appCfg.SECRET_SCANNING_GIT_APP_ID,
privateKey: appCfg.SECRET_SCANNING_PRIVATE_KEY,
installationId
}
});
const allFindingsByFingerprint: { [key: string]: SecretMatch } = {};
for (const commit of commits) {
for (const filepath of [...commit.added, ...commit.modified]) {
// eslint-disable-next-line
const fileContentsResponse = await octokit.repos.getContent({
owner,
repo,
path: filepath
});
const { data } = fileContentsResponse;
const fileContent = Buffer.from((data as { content: string }).content, "base64").toString();
// eslint-disable-next-line
const findings = await scanContentAndGetFindings(`\n${fileContent}`); // extra line to count lines correctly
for (const finding of findings) {
const fingerPrintWithCommitId = `${commit.id}:${filepath}:${finding.RuleID}:${finding.StartLine}`;
const fingerPrintWithoutCommitId = `${filepath}:${finding.RuleID}:${finding.StartLine}`;
finding.Fingerprint = fingerPrintWithCommitId;
finding.FingerPrintWithoutCommitId = fingerPrintWithoutCommitId;
finding.Commit = commit.id;
finding.File = filepath;
finding.Author = commit.author.name;
finding.Email = commit?.author?.email ? commit?.author?.email : "";
allFindingsByFingerprint[fingerPrintWithCommitId] = finding;
}
}
}
await secretScanningDAL.transaction(async (tx) => {
if (!Object.keys(allFindingsByFingerprint).length) return;
await secretScanningDAL.upsert(
Object.keys(allFindingsByFingerprint).map((key) => ({
installationId,
email: allFindingsByFingerprint[key].Email,
author: allFindingsByFingerprint[key].Author,
date: allFindingsByFingerprint[key].Date,
file: allFindingsByFingerprint[key].File,
tags: allFindingsByFingerprint[key].Tags,
commit: allFindingsByFingerprint[key].Commit,
ruleID: allFindingsByFingerprint[key].RuleID,
endLine: String(allFindingsByFingerprint[key].EndLine),
entropy: String(allFindingsByFingerprint[key].Entropy),
message: allFindingsByFingerprint[key].Message,
endColumn: String(allFindingsByFingerprint[key].EndColumn),
startLine: String(allFindingsByFingerprint[key].StartLine),
startColumn: String(allFindingsByFingerprint[key].StartColumn),
fingerPrintWithoutCommitId: allFindingsByFingerprint[key].FingerPrintWithoutCommitId,
description: allFindingsByFingerprint[key].Description,
symlinkFile: allFindingsByFingerprint[key].SymlinkFile,
orgId: organizationId,
pusherEmail: pusher.email,
pusherName: pusher.name,
repositoryFullName: repository.fullName,
repositoryId: String(repository.id),
fingerprint: allFindingsByFingerprint[key].Fingerprint
})),
tx
);
});
const adminEmails = await getOrgAdminEmails(organizationId);
if (pusher?.email) {
adminEmails.push(pusher.email);
}
if (Object.keys(allFindingsByFingerprint).length) {
await smtpService.sendMail({
template: SmtpTemplates.SecretLeakIncident,
subjectLine: `Incident alert: leaked secrets found in Github repository ${repository.fullName}`,
recipients: adminEmails.filter((email) => email).map((email) => email),
substitutions: {
numberOfSecrets: Object.keys(allFindingsByFingerprint).length,
pusher_email: pusher.email,
pusher_name: pusher.name
}
});
}
await telemetryService.sendPostHogEvents({
event: PostHogEventTypes.SecretScannerPush,
distinctId: repository.fullName,
properties: {
numberOfRisks: Object.keys(allFindingsByFingerprint).length
}
});
});
queueService.start(QueueName.SecretFullRepoScan, async (job) => {
const appCfg = getConfig();
const { organizationId, repository, installationId } = job.data;
const octokit = new ProbotOctokit({
auth: {
appId: appCfg.SECRET_SCANNING_GIT_APP_ID,
privateKey: appCfg.SECRET_SCANNING_PRIVATE_KEY,
installationId
}
});
const findings = await scanFullRepoContentAndGetFindings(
// this is because of collision of octokit in probot and github
// eslint-disable-next-line
octokit as any,
installationId,
repository.fullName
);
await secretScanningDAL.transaction(async (tx) => {
if (!findings.length) return;
// eslint-disable-next-line
await secretScanningDAL.upsert(
findings.map((finding) => ({
installationId,
email: finding.Email,
author: finding.Author,
date: finding.Date,
file: finding.File,
tags: finding.Tags,
commit: finding.Commit,
ruleID: finding.RuleID,
endLine: String(finding.EndLine),
entropy: String(finding.Entropy),
message: finding.Message,
endColumn: String(finding.EndColumn),
startLine: String(finding.StartLine),
startColumn: String(finding.StartColumn),
fingerPrintWithoutCommitId: finding.FingerPrintWithoutCommitId,
description: finding.Description,
symlinkFile: finding.SymlinkFile,
orgId: organizationId,
repositoryFullName: repository.fullName,
repositoryId: String(repository.id),
fingerprint: finding.Fingerprint
})),
tx
);
});
const adminEmails = await getOrgAdminEmails(organizationId);
if (findings.length) {
await smtpService.sendMail({
template: SmtpTemplates.SecretLeakIncident,
subjectLine: `Incident alert: leaked secrets found in Github repository ${repository.fullName}`,
recipients: adminEmails.filter((email) => email).map((email) => email),
substitutions: {
numberOfSecrets: findings.length
}
});
}
await telemetryService.sendPostHogEvents({
event: PostHogEventTypes.SecretScannerFull,
distinctId: repository.fullName,
properties: {
numberOfRisks: findings.length
}
});
});
queueService.listen(QueueName.SecretPushEventScan, "failed", (job, err) => {
logger.error(err, "Failed to secret scan on push", job?.data);
});
queueService.listen(QueueName.SecretFullRepoScan, "failed", (job, err) => {
logger.error(err, "Failed to do full repo secret scan", job?.data);
});
return { startFullRepoScan, startPushEventScan };
};

View File

@ -98,6 +98,7 @@ export const secretScanningServiceFactory = ({
if (canUseSecretScanning(actorOrgId)) {
await Promise.all(
repositories.map(({ id, full_name }) =>
// eslint-disable-next-line @typescript-eslint/no-unsafe-call,@typescript-eslint/no-unsafe-return,@typescript-eslint/no-unsafe-member-access
secretScanningQueue.startFullRepoScan({
organizationId: session.orgId,
installationId,
@ -180,6 +181,7 @@ export const secretScanningServiceFactory = ({
if (!installationLink) return;
if (canUseSecretScanning(installationLink.orgId)) {
// eslint-disable-next-line @typescript-eslint/no-unsafe-call,@typescript-eslint/no-unsafe-return,@typescript-eslint/no-unsafe-member-access
await secretScanningQueue.startPushEventScan({
commits,
pusher: { name: pusher.name, email: pusher.email },

View File

@ -73,6 +73,7 @@ type TWaitTillReady = {
export type TKeyStoreFactory = {
setItem: (key: string, value: string | number | Buffer, prefix?: string) => Promise<"OK">;
getItem: (key: string, prefix?: string) => Promise<string | null>;
getItems: (keys: string[], prefix?: string) => Promise<(string | null)[]>;
setExpiry: (key: string, expiryInSeconds: number) => Promise<number>;
setItemWithExpiry: (
key: string,
@ -81,6 +82,7 @@ export type TKeyStoreFactory = {
prefix?: string
) => Promise<"OK">;
deleteItem: (key: string) => Promise<number>;
deleteItemsByKeyIn: (keys: string[]) => Promise<number>;
deleteItems: (arg: TDeleteItems) => Promise<number>;
incrementBy: (key: string, value: number) => Promise<number>;
acquireLock(
@ -89,6 +91,7 @@ export type TKeyStoreFactory = {
settings?: Partial<Settings>
): Promise<{ release: () => Promise<ExecutionResult> }>;
waitTillReady: ({ key, waitingCb, keyCheckCb, waitIteration, delay, jitter }: TWaitTillReady) => Promise<void>;
getKeysByPattern: (pattern: string, limit?: number) => Promise<string[]>;
};
export const keyStoreFactory = (redisConfigKeys: TRedisConfigKeys): TKeyStoreFactory => {
@ -100,6 +103,9 @@ export const keyStoreFactory = (redisConfigKeys: TRedisConfigKeys): TKeyStoreFac
const getItem = async (key: string, prefix?: string) => redis.get(prefix ? `${prefix}:${key}` : key);
const getItems = async (keys: string[], prefix?: string) =>
redis.mget(keys.map((key) => (prefix ? `${prefix}:${key}` : key)));
const setItemWithExpiry = async (
key: string,
expiryInSeconds: number | string,
@ -109,6 +115,11 @@ export const keyStoreFactory = (redisConfigKeys: TRedisConfigKeys): TKeyStoreFac
const deleteItem = async (key: string) => redis.del(key);
const deleteItemsByKeyIn = async (keys: string[]) => {
if (keys.length === 0) return 0;
return redis.del(keys);
};
const deleteItems = async ({ pattern, batchSize = 500, delay = 1500, jitter = 200 }: TDeleteItems) => {
let cursor = "0";
let totalDeleted = 0;
@ -164,6 +175,24 @@ export const keyStoreFactory = (redisConfigKeys: TRedisConfigKeys): TKeyStoreFac
}
};
const getKeysByPattern = async (pattern: string, limit?: number) => {
let cursor = "0";
const allKeys: string[] = [];
do {
// eslint-disable-next-line no-await-in-loop
const [nextCursor, keys] = await redis.scan(cursor, "MATCH", pattern, "COUNT", 1000);
cursor = nextCursor;
allKeys.push(...keys);
if (limit && allKeys.length >= limit) {
return allKeys.slice(0, limit);
}
} while (cursor !== "0");
return allKeys;
};
return {
setItem,
getItem,
@ -175,6 +204,9 @@ export const keyStoreFactory = (redisConfigKeys: TRedisConfigKeys): TKeyStoreFac
acquireLock(resources: string[], duration: number, settings?: Partial<Settings>) {
return redisLock.acquire(resources, duration, settings);
},
waitTillReady
waitTillReady,
getKeysByPattern,
deleteItemsByKeyIn,
getItems
};
};

View File

@ -8,6 +8,8 @@ import { TKeyStoreFactory } from "./keystore";
export const inMemoryKeyStore = (): TKeyStoreFactory => {
const store: Record<string, string | number | Buffer> = {};
const getRegex = (pattern: string) =>
new RE2(`^${pattern.replace(/[-[\]/{}()+?.\\^$|]/g, "\\$&").replace(/\*/g, ".*")}$`);
return {
setItem: async (key, value) => {
@ -24,7 +26,7 @@ export const inMemoryKeyStore = (): TKeyStoreFactory => {
return 1;
},
deleteItems: async ({ pattern, batchSize = 500, delay = 1500, jitter = 200 }) => {
const regex = new RE2(`^${pattern.replace(/[-[\]/{}()+?.\\^$|]/g, "\\$&").replace(/\*/g, ".*")}$`);
const regex = getRegex(pattern);
let totalDeleted = 0;
const keys = Object.keys(store);
@ -59,6 +61,27 @@ export const inMemoryKeyStore = (): TKeyStoreFactory => {
release: () => {}
}) as Promise<Lock>;
},
waitTillReady: async () => {}
waitTillReady: async () => {},
getKeysByPattern: async (pattern) => {
const regex = getRegex(pattern);
const keys = Object.keys(store);
return keys.filter((key) => regex.test(key));
},
deleteItemsByKeyIn: async (keys) => {
for (const key of keys) {
delete store[key];
}
return keys.length;
},
getItems: async (keys) => {
const values = keys.map((key) => {
const value = store[key];
if (typeof value === "string") {
return value;
}
return null;
});
return values;
}
};
};

View File

@ -22,6 +22,7 @@ export enum ApiDocsTags {
UniversalAuth = "Universal Auth",
GcpAuth = "GCP Auth",
AliCloudAuth = "Alibaba Cloud Auth",
TlsCertAuth = "TLS Certificate Auth",
AwsAuth = "AWS Auth",
OciAuth = "OCI Auth",
AzureAuth = "Azure Auth",
@ -283,6 +284,38 @@ export const ALICLOUD_AUTH = {
}
} as const;
export const TLS_CERT_AUTH = {
LOGIN: {
identityId: "The ID of the identity to login."
},
ATTACH: {
identityId: "The ID of the identity to attach the configuration onto.",
allowedCommonNames:
"The comma-separated list of trusted common names that are allowed to authenticate with Infisical.",
caCertificate: "The PEM-encoded CA certificate to validate client certificates.",
accessTokenTTL: "The lifetime for an access token in seconds.",
accessTokenMaxTTL: "The maximum lifetime for an access token in seconds.",
accessTokenNumUsesLimit: "The maximum number of times that an access token can be used.",
accessTokenTrustedIps: "The IPs or CIDR ranges that access tokens can be used from."
},
UPDATE: {
identityId: "The ID of the identity to update the auth method for.",
allowedCommonNames:
"The comma-separated list of trusted common names that are allowed to authenticate with Infisical.",
caCertificate: "The PEM-encoded CA certificate to validate client certificates.",
accessTokenTTL: "The new lifetime for an access token in seconds.",
accessTokenMaxTTL: "The new maximum lifetime for an access token in seconds.",
accessTokenNumUsesLimit: "The new maximum number of times that an access token can be used.",
accessTokenTrustedIps: "The new IPs or CIDR ranges that access tokens can be used from."
},
RETRIEVE: {
identityId: "The ID of the identity to retrieve the auth method for."
},
REVOKE: {
identityId: "The ID of the identity to revoke the auth method for."
}
} as const;
export const AWS_AUTH = {
LOGIN: {
identityId: "The ID of the identity to login.",
@ -2228,6 +2261,12 @@ export const AppConnections = {
},
FLYIO: {
accessToken: "The Access Token used to access fly.io."
},
GITLAB: {
instanceUrl: "The GitLab instance URL to connect with.",
accessToken: "The Access Token used to access GitLab.",
code: "The OAuth code to use to connect with GitLab.",
accessTokenType: "The type of token used to connect with GitLab."
}
}
};
@ -2402,6 +2441,17 @@ export const SecretSyncs = {
FLYIO: {
appId: "The ID of the Fly.io app to sync secrets to."
},
GITLAB: {
projectId: "The GitLab Project ID to sync secrets to.",
projectName: "The GitLab Project Name to sync secrets to.",
groupId: "The GitLab Group ID to sync secrets to.",
groupName: "The GitLab Group Name to sync secrets to.",
scope: "The GitLab scope that secrets should be synced to. (default: project)",
targetEnvironment: "The GitLab environment scope that secrets should be synced to. (default: *)",
shouldProtectSecrets: "Whether variables should be protected",
shouldMaskSecrets: "Whether variables should be masked in logs",
shouldHideSecrets: "Whether variables should be hidden"
},
CLOUDFLARE_PAGES: {
projectName: "The name of the Cloudflare Pages project to sync secrets to.",
environment: "The environment of the Cloudflare Pages project to sync secrets to."

View File

@ -193,6 +193,9 @@ const envSchema = z
PYLON_API_KEY: zpStr(z.string().optional()),
DISABLE_AUDIT_LOG_GENERATION: zodStrBool.default("false"),
SSL_CLIENT_CERTIFICATE_HEADER_KEY: zpStr(z.string().optional()).default("x-ssl-client-cert"),
IDENTITY_TLS_CERT_AUTH_CLIENT_CERTIFICATE_HEADER_KEY: zpStr(z.string().optional()).default(
"x-identity-tls-cert-auth-client-cert"
),
WORKFLOW_SLACK_CLIENT_ID: zpStr(z.string().optional()),
WORKFLOW_SLACK_CLIENT_SECRET: zpStr(z.string().optional()),
ENABLE_MSSQL_SECRET_ROTATION_ENCRYPT: zodStrBool.default("true"),
@ -247,6 +250,10 @@ const envSchema = z
INF_APP_CONNECTION_GITHUB_RADAR_APP_ID: zpStr(z.string().optional()),
INF_APP_CONNECTION_GITHUB_RADAR_APP_WEBHOOK_SECRET: zpStr(z.string().optional()),
// gitlab oauth
INF_APP_CONNECTION_GITLAB_OAUTH_CLIENT_ID: zpStr(z.string().optional()),
INF_APP_CONNECTION_GITLAB_OAUTH_CLIENT_SECRET: zpStr(z.string().optional()),
// gcp app
INF_APP_CONNECTION_GCP_SERVICE_ACCOUNT_CREDENTIAL: zpStr(z.string().optional()),

View File

@ -62,7 +62,8 @@ export enum QueueName {
SecretRotationV2 = "secret-rotation-v2",
FolderTreeCheckpoint = "folder-tree-checkpoint",
InvalidateCache = "invalidate-cache",
SecretScanningV2 = "secret-scanning-v2"
SecretScanningV2 = "secret-scanning-v2",
TelemetryAggregatedEvents = "telemetry-aggregated-events"
}
export enum QueueJobs {
@ -101,7 +102,8 @@ export enum QueueJobs {
SecretScanningV2DiffScan = "secret-scanning-v2-diff-scan",
SecretScanningV2SendNotification = "secret-scanning-v2-notification",
CaOrderCertificateForSubscriber = "ca-order-certificate-for-subscriber",
PkiSubscriberDailyAutoRenewal = "pki-subscriber-daily-auto-renewal"
PkiSubscriberDailyAutoRenewal = "pki-subscriber-daily-auto-renewal",
TelemetryAggregatedEvents = "telemetry-aggregated-events"
}
export type TQueueJobTypes = {
@ -292,6 +294,10 @@ export type TQueueJobTypes = {
name: QueueJobs.PkiSubscriberDailyAutoRenewal;
payload: undefined;
};
[QueueName.TelemetryAggregatedEvents]: {
name: QueueJobs.TelemetryAggregatedEvents;
payload: undefined;
};
};
const SECRET_SCANNING_JOBS = [

View File

@ -193,6 +193,8 @@ import { identityOidcAuthServiceFactory } from "@app/services/identity-oidc-auth
import { identityProjectDALFactory } from "@app/services/identity-project/identity-project-dal";
import { identityProjectMembershipRoleDALFactory } from "@app/services/identity-project/identity-project-membership-role-dal";
import { identityProjectServiceFactory } from "@app/services/identity-project/identity-project-service";
import { identityTlsCertAuthDALFactory } from "@app/services/identity-tls-cert-auth/identity-tls-cert-auth-dal";
import { identityTlsCertAuthServiceFactory } from "@app/services/identity-tls-cert-auth/identity-tls-cert-auth-service";
import { identityTokenAuthDALFactory } from "@app/services/identity-token-auth/identity-token-auth-dal";
import { identityTokenAuthServiceFactory } from "@app/services/identity-token-auth/identity-token-auth-service";
import { identityUaClientSecretDALFactory } from "@app/services/identity-ua/identity-ua-client-secret-dal";
@ -297,7 +299,6 @@ import { injectAssumePrivilege } from "../plugins/auth/inject-assume-privilege";
import { injectIdentity } from "../plugins/auth/inject-identity";
import { injectPermission } from "../plugins/auth/inject-permission";
import { injectRateLimits } from "../plugins/inject-rate-limits";
import { registerSecretScannerGhApp } from "../plugins/secret-scanner";
import { registerV1Routes } from "./v1";
import { registerV2Routes } from "./v2";
import { registerV3Routes } from "./v3";
@ -326,7 +327,6 @@ export const registerRoutes = async (
}
) => {
const appCfg = getConfig();
await server.register(registerSecretScannerGhApp, { prefix: "/ss-webhook" });
await server.register(registerSecretScanningV2Webhooks, {
prefix: "/secret-scanning/webhooks"
});
@ -386,6 +386,7 @@ export const registerRoutes = async (
const identityKubernetesAuthDAL = identityKubernetesAuthDALFactory(db);
const identityUaClientSecretDAL = identityUaClientSecretDALFactory(db);
const identityAliCloudAuthDAL = identityAliCloudAuthDALFactory(db);
const identityTlsCertAuthDAL = identityTlsCertAuthDALFactory(db);
const identityAwsAuthDAL = identityAwsAuthDALFactory(db);
const identityGcpAuthDAL = identityGcpAuthDALFactory(db);
const identityOciAuthDAL = identityOciAuthDALFactory(db);
@ -686,7 +687,8 @@ export const registerRoutes = async (
const telemetryQueue = telemetryQueueServiceFactory({
keyStore,
telemetryDAL,
queueService
queueService,
telemetryService
});
const invalidateCacheQueue = invalidateCacheQueueFactory({
@ -1493,6 +1495,15 @@ export const registerRoutes = async (
permissionService
});
const identityTlsCertAuthService = identityTlsCertAuthServiceFactory({
identityAccessTokenDAL,
identityTlsCertAuthDAL,
identityOrgMembershipDAL,
licenseService,
permissionService,
kmsService
});
const identityAwsAuthService = identityAwsAuthServiceFactory({
identityAccessTokenDAL,
identityAwsAuthDAL,
@ -1947,6 +1958,7 @@ export const registerRoutes = async (
identityAwsAuth: identityAwsAuthService,
identityAzureAuth: identityAzureAuthService,
identityOciAuth: identityOciAuthService,
identityTlsCertAuth: identityTlsCertAuthService,
identityOidcAuth: identityOidcAuthService,
identityJwtAuth: identityJwtAuthService,
identityLdapAuth: identityLdapAuthService,

View File

@ -722,6 +722,7 @@ export const registerAdminRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.InvalidateCache,
organizationId: req.permission.orgId,
distinctId: getTelemetryDistinctId(req),
properties: {
...req.auditLogInfo

View File

@ -35,6 +35,10 @@ import {
CamundaConnectionListItemSchema,
SanitizedCamundaConnectionSchema
} from "@app/services/app-connection/camunda";
import {
CloudflareConnectionListItemSchema,
SanitizedCloudflareConnectionSchema
} from "@app/services/app-connection/cloudflare/cloudflare-connection-schema";
import {
DatabricksConnectionListItemSchema,
SanitizedDatabricksConnectionSchema
@ -46,6 +50,7 @@ import {
GitHubRadarConnectionListItemSchema,
SanitizedGitHubRadarConnectionSchema
} from "@app/services/app-connection/github-radar";
import { GitLabConnectionListItemSchema, SanitizedGitLabConnectionSchema } from "@app/services/app-connection/gitlab";
import {
HCVaultConnectionListItemSchema,
SanitizedHCVaultConnectionSchema
@ -80,10 +85,6 @@ import {
WindmillConnectionListItemSchema
} from "@app/services/app-connection/windmill";
import { AuthMode } from "@app/services/auth/auth-type";
import {
CloudflareConnectionListItemSchema,
SanitizedCloudflareConnectionSchema
} from "@app/services/app-connection/cloudflare/cloudflare-connection-schema";
// can't use discriminated due to multiple schemas for certain apps
const SanitizedAppConnectionSchema = z.union([
@ -114,6 +115,7 @@ const SanitizedAppConnectionSchema = z.union([
...SanitizedHerokuConnectionSchema.options,
...SanitizedRenderConnectionSchema.options,
...SanitizedFlyioConnectionSchema.options,
...SanitizedGitLabConnectionSchema.options,
...SanitizedCloudflareConnectionSchema.options
]);
@ -145,6 +147,7 @@ const AppConnectionOptionsSchema = z.discriminatedUnion("app", [
HerokuConnectionListItemSchema,
RenderConnectionListItemSchema,
FlyioConnectionListItemSchema,
GitLabConnectionListItemSchema,
CloudflareConnectionListItemSchema
]);

View File

@ -0,0 +1,90 @@
import z from "zod";
import { readLimit } from "@app/server/config/rateLimiter";
import { verifyAuth } from "@app/server/plugins/auth/verify-auth";
import { AppConnection } from "@app/services/app-connection/app-connection-enums";
import {
CreateGitLabConnectionSchema,
SanitizedGitLabConnectionSchema,
TGitLabGroup,
TGitLabProject,
UpdateGitLabConnectionSchema
} from "@app/services/app-connection/gitlab";
import { AuthMode } from "@app/services/auth/auth-type";
import { registerAppConnectionEndpoints } from "./app-connection-endpoints";
export const registerGitLabConnectionRouter = async (server: FastifyZodProvider) => {
registerAppConnectionEndpoints({
app: AppConnection.GitLab,
server,
sanitizedResponseSchema: SanitizedGitLabConnectionSchema,
createSchema: CreateGitLabConnectionSchema,
updateSchema: UpdateGitLabConnectionSchema
});
// The below endpoints are not exposed and for Infisical App use
server.route({
method: "GET",
url: `/:connectionId/projects`,
config: {
rateLimit: readLimit
},
schema: {
params: z.object({
connectionId: z.string().uuid()
}),
response: {
200: z
.object({
id: z.string(),
name: z.string()
})
.array()
}
},
onRequest: verifyAuth([AuthMode.JWT]),
handler: async (req) => {
const { connectionId } = req.params;
const projects: TGitLabProject[] = await server.services.appConnection.gitlab.listProjects(
connectionId,
req.permission
);
return projects;
}
});
server.route({
method: "GET",
url: `/:connectionId/groups`,
config: {
rateLimit: readLimit
},
schema: {
params: z.object({
connectionId: z.string().uuid()
}),
response: {
200: z
.object({
id: z.string(),
name: z.string()
})
.array()
}
},
onRequest: verifyAuth([AuthMode.JWT]),
handler: async (req) => {
const { connectionId } = req.params;
const groups: TGitLabGroup[] = await server.services.appConnection.gitlab.listGroups(
connectionId,
req.permission
);
return groups;
}
});
};

View File

@ -10,11 +10,13 @@ import { registerAzureClientSecretsConnectionRouter } from "./azure-client-secre
import { registerAzureDevOpsConnectionRouter } from "./azure-devops-connection-router";
import { registerAzureKeyVaultConnectionRouter } from "./azure-key-vault-connection-router";
import { registerCamundaConnectionRouter } from "./camunda-connection-router";
import { registerCloudflareConnectionRouter } from "./cloudflare-connection-router";
import { registerDatabricksConnectionRouter } from "./databricks-connection-router";
import { registerFlyioConnectionRouter } from "./flyio-connection-router";
import { registerGcpConnectionRouter } from "./gcp-connection-router";
import { registerGitHubConnectionRouter } from "./github-connection-router";
import { registerGitHubRadarConnectionRouter } from "./github-radar-connection-router";
import { registerGitLabConnectionRouter } from "./gitlab-connection-router";
import { registerHCVaultConnectionRouter } from "./hc-vault-connection-router";
import { registerHerokuConnectionRouter } from "./heroku-connection-router";
import { registerHumanitecConnectionRouter } from "./humanitec-connection-router";
@ -27,7 +29,6 @@ import { registerTeamCityConnectionRouter } from "./teamcity-connection-router";
import { registerTerraformCloudConnectionRouter } from "./terraform-cloud-router";
import { registerVercelConnectionRouter } from "./vercel-connection-router";
import { registerWindmillConnectionRouter } from "./windmill-connection-router";
import { registerCloudflareConnectionRouter } from "./cloudflare-connection-router";
export * from "./app-connection-router";
@ -60,5 +61,6 @@ export const APP_CONNECTION_REGISTER_ROUTER_MAP: Record<AppConnection, (server:
[AppConnection.Heroku]: registerHerokuConnectionRouter,
[AppConnection.Render]: registerRenderConnectionRouter,
[AppConnection.Flyio]: registerFlyioConnectionRouter,
[AppConnection.GitLab]: registerGitLabConnectionRouter,
[AppConnection.Cloudflare]: registerCloudflareConnectionRouter
};

View File

@ -692,6 +692,7 @@ export const registerCaRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.IssueCert,
organizationId: req.permission.orgId,
distinctId: getTelemetryDistinctId(req),
properties: {
caId: ca.id,
@ -786,6 +787,7 @@ export const registerCaRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SignCert,
organizationId: req.permission.orgId,
distinctId: getTelemetryDistinctId(req),
properties: {
caId: ca.id,

View File

@ -266,6 +266,7 @@ export const registerCertRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.IssueCert,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
caId: req.body.caId,
certificateTemplateId: req.body.certificateTemplateId,
@ -442,6 +443,7 @@ export const registerCertRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SignCert,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
caId: req.body.caId,
certificateTemplateId: req.body.certificateTemplateId,

View File

@ -475,6 +475,7 @@ export const registerDashboardRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretPulled,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
numberOfSecrets: secretCountFromEnv,
workspaceId: projectId,
@ -979,6 +980,7 @@ export const registerDashboardRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretPulled,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
numberOfSecrets: secretCount,
workspaceId: projectId,
@ -1144,6 +1146,7 @@ export const registerDashboardRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretPulled,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
numberOfSecrets: secretCountForEnv,
workspaceId: projectId,
@ -1336,6 +1339,7 @@ export const registerDashboardRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretPulled,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
numberOfSecrets: secrets.length,
workspaceId: projectId,

View File

@ -85,6 +85,7 @@ export const registerIdentityRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.MachineIdentityCreated,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
orgId: req.body.organizationId,
name: identity.name,

View File

@ -0,0 +1,396 @@
import crypto from "node:crypto";
import { z } from "zod";
import { IdentityTlsCertAuthsSchema } from "@app/db/schemas";
import { EventType } from "@app/ee/services/audit-log/audit-log-types";
import { ApiDocsTags, TLS_CERT_AUTH } from "@app/lib/api-docs";
import { getConfig } from "@app/lib/config/env";
import { BadRequestError } from "@app/lib/errors";
import { readLimit, writeLimit } from "@app/server/config/rateLimiter";
import { verifyAuth } from "@app/server/plugins/auth/verify-auth";
import { AuthMode } from "@app/services/auth/auth-type";
import { TIdentityTrustedIp } from "@app/services/identity/identity-types";
import { isSuperAdmin } from "@app/services/super-admin/super-admin-fns";
const validateCommonNames = z
.string()
.min(1)
.trim()
.transform((el) =>
el
.split(",")
.map((i) => i.trim())
.join(",")
);
const validateCaCertificate = (caCert: string) => {
if (!caCert) return true;
try {
// eslint-disable-next-line no-new
new crypto.X509Certificate(caCert);
return true;
} catch (err) {
return false;
}
};
export const registerIdentityTlsCertAuthRouter = async (server: FastifyZodProvider) => {
server.route({
method: "POST",
url: "/login",
config: {
rateLimit: writeLimit
},
schema: {
hide: false,
tags: [ApiDocsTags.TlsCertAuth],
description: "Login with TLS Certificate Auth",
body: z.object({
identityId: z.string().trim().describe(TLS_CERT_AUTH.LOGIN.identityId)
}),
response: {
200: z.object({
accessToken: z.string(),
expiresIn: z.coerce.number(),
accessTokenMaxTTL: z.coerce.number(),
tokenType: z.literal("Bearer")
})
}
},
handler: async (req) => {
const appCfg = getConfig();
const clientCertificate = req.headers[appCfg.IDENTITY_TLS_CERT_AUTH_CLIENT_CERTIFICATE_HEADER_KEY];
if (!clientCertificate) {
throw new BadRequestError({ message: "Missing TLS certificate in header" });
}
const { identityTlsCertAuth, accessToken, identityAccessToken, identityMembershipOrg } =
await server.services.identityTlsCertAuth.login({
identityId: req.body.identityId,
clientCertificate: clientCertificate as string
});
await server.services.auditLog.createAuditLog({
...req.auditLogInfo,
orgId: identityMembershipOrg?.orgId,
event: {
type: EventType.LOGIN_IDENTITY_TLS_CERT_AUTH,
metadata: {
identityId: identityTlsCertAuth.identityId,
identityAccessTokenId: identityAccessToken.id,
identityTlsCertAuthId: identityTlsCertAuth.id
}
}
});
return {
accessToken,
tokenType: "Bearer" as const,
expiresIn: identityTlsCertAuth.accessTokenTTL,
accessTokenMaxTTL: identityTlsCertAuth.accessTokenMaxTTL
};
}
});
server.route({
method: "POST",
url: "/identities/:identityId",
config: {
rateLimit: writeLimit
},
onRequest: verifyAuth([AuthMode.JWT, AuthMode.IDENTITY_ACCESS_TOKEN]),
schema: {
hide: false,
tags: [ApiDocsTags.TlsCertAuth],
description: "Attach TLS Certificate Auth configuration onto identity",
security: [
{
bearerAuth: []
}
],
params: z.object({
identityId: z.string().trim().describe(TLS_CERT_AUTH.ATTACH.identityId)
}),
body: z
.object({
allowedCommonNames: validateCommonNames
.optional()
.nullable()
.describe(TLS_CERT_AUTH.ATTACH.allowedCommonNames),
caCertificate: z
.string()
.min(1)
.max(10240)
.refine(validateCaCertificate, "Invalid CA Certificate.")
.describe(TLS_CERT_AUTH.ATTACH.caCertificate),
accessTokenTrustedIps: z
.object({
ipAddress: z.string().trim()
})
.array()
.min(1)
.default([{ ipAddress: "0.0.0.0/0" }, { ipAddress: "::/0" }])
.describe(TLS_CERT_AUTH.ATTACH.accessTokenTrustedIps),
accessTokenTTL: z
.number()
.int()
.min(0)
.max(315360000)
.default(2592000)
.describe(TLS_CERT_AUTH.ATTACH.accessTokenTTL),
accessTokenMaxTTL: z
.number()
.int()
.min(1)
.max(315360000)
.default(2592000)
.describe(TLS_CERT_AUTH.ATTACH.accessTokenMaxTTL),
accessTokenNumUsesLimit: z
.number()
.int()
.min(0)
.default(0)
.describe(TLS_CERT_AUTH.ATTACH.accessTokenNumUsesLimit)
})
.refine(
(val) => val.accessTokenTTL <= val.accessTokenMaxTTL,
"Access Token TTL cannot be greater than Access Token Max TTL."
),
response: {
200: z.object({
identityTlsCertAuth: IdentityTlsCertAuthsSchema
})
}
},
handler: async (req) => {
const identityTlsCertAuth = await server.services.identityTlsCertAuth.attachTlsCertAuth({
actor: req.permission.type,
actorId: req.permission.id,
actorAuthMethod: req.permission.authMethod,
actorOrgId: req.permission.orgId,
...req.body,
identityId: req.params.identityId,
isActorSuperAdmin: isSuperAdmin(req.auth)
});
await server.services.auditLog.createAuditLog({
...req.auditLogInfo,
orgId: req.permission.orgId,
event: {
type: EventType.ADD_IDENTITY_TLS_CERT_AUTH,
metadata: {
identityId: identityTlsCertAuth.identityId,
allowedCommonNames: identityTlsCertAuth.allowedCommonNames,
accessTokenTTL: identityTlsCertAuth.accessTokenTTL,
accessTokenMaxTTL: identityTlsCertAuth.accessTokenMaxTTL,
accessTokenTrustedIps: identityTlsCertAuth.accessTokenTrustedIps as TIdentityTrustedIp[],
accessTokenNumUsesLimit: identityTlsCertAuth.accessTokenNumUsesLimit
}
}
});
return { identityTlsCertAuth };
}
});
server.route({
method: "PATCH",
url: "/identities/:identityId",
config: {
rateLimit: writeLimit
},
onRequest: verifyAuth([AuthMode.JWT, AuthMode.IDENTITY_ACCESS_TOKEN]),
schema: {
hide: false,
tags: [ApiDocsTags.TlsCertAuth],
description: "Update TLS Certificate Auth configuration on identity",
security: [
{
bearerAuth: []
}
],
params: z.object({
identityId: z.string().describe(TLS_CERT_AUTH.UPDATE.identityId)
}),
body: z
.object({
caCertificate: z
.string()
.min(1)
.max(10240)
.refine(validateCaCertificate, "Invalid CA Certificate.")
.optional()
.describe(TLS_CERT_AUTH.UPDATE.caCertificate),
allowedCommonNames: validateCommonNames
.optional()
.nullable()
.describe(TLS_CERT_AUTH.UPDATE.allowedCommonNames),
accessTokenTrustedIps: z
.object({
ipAddress: z.string().trim()
})
.array()
.min(1)
.optional()
.describe(TLS_CERT_AUTH.UPDATE.accessTokenTrustedIps),
accessTokenTTL: z
.number()
.int()
.min(0)
.max(315360000)
.optional()
.describe(TLS_CERT_AUTH.UPDATE.accessTokenTTL),
accessTokenNumUsesLimit: z
.number()
.int()
.min(0)
.optional()
.describe(TLS_CERT_AUTH.UPDATE.accessTokenNumUsesLimit),
accessTokenMaxTTL: z
.number()
.int()
.max(315360000)
.min(0)
.optional()
.describe(TLS_CERT_AUTH.UPDATE.accessTokenMaxTTL)
})
.refine(
(val) => (val.accessTokenMaxTTL && val.accessTokenTTL ? val.accessTokenTTL <= val.accessTokenMaxTTL : true),
"Access Token TTL cannot be greater than Access Token Max TTL."
),
response: {
200: z.object({
identityTlsCertAuth: IdentityTlsCertAuthsSchema
})
}
},
handler: async (req) => {
const identityTlsCertAuth = await server.services.identityTlsCertAuth.updateTlsCertAuth({
actor: req.permission.type,
actorId: req.permission.id,
actorAuthMethod: req.permission.authMethod,
actorOrgId: req.permission.orgId,
...req.body,
identityId: req.params.identityId
});
await server.services.auditLog.createAuditLog({
...req.auditLogInfo,
orgId: req.permission.orgId,
event: {
type: EventType.UPDATE_IDENTITY_TLS_CERT_AUTH,
metadata: {
identityId: identityTlsCertAuth.identityId,
allowedCommonNames: identityTlsCertAuth.allowedCommonNames,
accessTokenTTL: identityTlsCertAuth.accessTokenTTL,
accessTokenMaxTTL: identityTlsCertAuth.accessTokenMaxTTL,
accessTokenTrustedIps: identityTlsCertAuth.accessTokenTrustedIps as TIdentityTrustedIp[],
accessTokenNumUsesLimit: identityTlsCertAuth.accessTokenNumUsesLimit
}
}
});
return { identityTlsCertAuth };
}
});
server.route({
method: "GET",
url: "/identities/:identityId",
config: {
rateLimit: readLimit
},
onRequest: verifyAuth([AuthMode.JWT, AuthMode.IDENTITY_ACCESS_TOKEN]),
schema: {
hide: false,
tags: [ApiDocsTags.TlsCertAuth],
description: "Retrieve TLS Certificate Auth configuration on identity",
security: [
{
bearerAuth: []
}
],
params: z.object({
identityId: z.string().describe(TLS_CERT_AUTH.RETRIEVE.identityId)
}),
response: {
200: z.object({
identityTlsCertAuth: IdentityTlsCertAuthsSchema.extend({
caCertificate: z.string()
})
})
}
},
handler: async (req) => {
const identityTlsCertAuth = await server.services.identityTlsCertAuth.getTlsCertAuth({
identityId: req.params.identityId,
actor: req.permission.type,
actorId: req.permission.id,
actorOrgId: req.permission.orgId,
actorAuthMethod: req.permission.authMethod
});
await server.services.auditLog.createAuditLog({
...req.auditLogInfo,
orgId: req.permission.orgId,
event: {
type: EventType.GET_IDENTITY_TLS_CERT_AUTH,
metadata: {
identityId: identityTlsCertAuth.identityId
}
}
});
return { identityTlsCertAuth };
}
});
server.route({
method: "DELETE",
url: "/identities/:identityId",
config: {
rateLimit: writeLimit
},
onRequest: verifyAuth([AuthMode.JWT, AuthMode.IDENTITY_ACCESS_TOKEN]),
schema: {
hide: false,
tags: [ApiDocsTags.TlsCertAuth],
description: "Delete TLS Certificate Auth configuration on identity",
security: [
{
bearerAuth: []
}
],
params: z.object({
identityId: z.string().describe(TLS_CERT_AUTH.REVOKE.identityId)
}),
response: {
200: z.object({
identityTlsCertAuth: IdentityTlsCertAuthsSchema
})
}
},
handler: async (req) => {
const identityTlsCertAuth = await server.services.identityTlsCertAuth.revokeTlsCertAuth({
actor: req.permission.type,
actorId: req.permission.id,
actorAuthMethod: req.permission.authMethod,
actorOrgId: req.permission.orgId,
identityId: req.params.identityId
});
await server.services.auditLog.createAuditLog({
...req.auditLogInfo,
orgId: req.permission.orgId,
event: {
type: EventType.REVOKE_IDENTITY_TLS_CERT_AUTH,
metadata: {
identityId: identityTlsCertAuth.identityId
}
}
});
return { identityTlsCertAuth };
}
});
};

View File

@ -25,6 +25,7 @@ import { registerIdentityLdapAuthRouter } from "./identity-ldap-auth-router";
import { registerIdentityOciAuthRouter } from "./identity-oci-auth-router";
import { registerIdentityOidcAuthRouter } from "./identity-oidc-auth-router";
import { registerIdentityRouter } from "./identity-router";
import { registerIdentityTlsCertAuthRouter } from "./identity-tls-cert-auth-router";
import { registerIdentityTokenAuthRouter } from "./identity-token-auth-router";
import { registerIdentityUaRouter } from "./identity-universal-auth-router";
import { registerIntegrationAuthRouter } from "./integration-auth-router";
@ -66,6 +67,7 @@ export const registerV1Routes = async (server: FastifyZodProvider) => {
await authRouter.register(registerIdentityAccessTokenRouter);
await authRouter.register(registerIdentityAliCloudAuthRouter);
await authRouter.register(registerIdentityAwsAuthRouter);
await authRouter.register(registerIdentityTlsCertAuthRouter, { prefix: "/tls-cert-auth" });
await authRouter.register(registerIdentityAzureAuthRouter);
await authRouter.register(registerIdentityOciAuthRouter);
await authRouter.register(registerIdentityOidcAuthRouter);

View File

@ -103,6 +103,7 @@ export const registerIntegrationRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.IntegrationCreated,
organizationId: req.permission.orgId,
distinctId: getTelemetryDistinctId(req),
properties: {
...createIntegrationEventProperty,

View File

@ -64,6 +64,7 @@ export const registerInviteOrgRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.UserOrgInvitation,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
inviteeEmails: req.body.inviteeEmails,
organizationRoleSlug: req.body.organizationRoleSlug,

View File

@ -331,6 +331,7 @@ export const registerPkiSubscriberRouter = async (server: FastifyZodProvider) =>
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.IssueCert,
organizationId: req.permission.orgId,
distinctId: getTelemetryDistinctId(req),
properties: {
subscriberId: subscriber.id,
@ -399,6 +400,7 @@ export const registerPkiSubscriberRouter = async (server: FastifyZodProvider) =>
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.IssueCert,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
subscriberId: subscriber.id,
commonName: subscriber.commonName,
@ -471,6 +473,7 @@ export const registerPkiSubscriberRouter = async (server: FastifyZodProvider) =>
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SignCert,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
subscriberId: subscriber.id,
commonName: subscriber.commonName,

View File

@ -165,6 +165,7 @@ export const registerSecretRequestsRouter = async (server: FastifyZodProvider) =
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretRequestDeleted,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
secretRequestId: req.params.id,
organizationId: req.permission.orgId,
@ -256,6 +257,7 @@ export const registerSecretRequestsRouter = async (server: FastifyZodProvider) =
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretRequestCreated,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
secretRequestId: shareRequest.id,
organizationId: req.permission.orgId,

View File

@ -1,10 +1,11 @@
import { SecretSync } from "@app/services/secret-sync/secret-sync-enums";
import { registerSyncSecretsEndpoints } from "./secret-sync-endpoints";
import {
CloudflarePagesSyncSchema,
CreateCloudflarePagesSyncSchema,
UpdateCloudflarePagesSyncSchema
} from "@app/services/secret-sync/cloudflare-pages/cloudflare-pages-schema";
import { SecretSync } from "@app/services/secret-sync/secret-sync-enums";
import { registerSyncSecretsEndpoints } from "./secret-sync-endpoints";
export const registerCloudflarePagesSyncRouter = async (server: FastifyZodProvider) =>
registerSyncSecretsEndpoints({

View File

@ -0,0 +1,13 @@
import { CreateGitLabSyncSchema, GitLabSyncSchema, UpdateGitLabSyncSchema } from "@app/services/secret-sync/gitlab";
import { SecretSync } from "@app/services/secret-sync/secret-sync-enums";
import { registerSyncSecretsEndpoints } from "./secret-sync-endpoints";
export const registerGitLabSyncRouter = async (server: FastifyZodProvider) =>
registerSyncSecretsEndpoints({
destination: SecretSync.GitLab,
server,
responseSchema: GitLabSyncSchema,
createSchema: CreateGitLabSyncSchema,
updateSchema: UpdateGitLabSyncSchema
});

View File

@ -13,6 +13,7 @@ import { registerDatabricksSyncRouter } from "./databricks-sync-router";
import { registerFlyioSyncRouter } from "./flyio-sync-router";
import { registerGcpSyncRouter } from "./gcp-sync-router";
import { registerGitHubSyncRouter } from "./github-sync-router";
import { registerGitLabSyncRouter } from "./gitlab-sync-router";
import { registerHCVaultSyncRouter } from "./hc-vault-sync-router";
import { registerHerokuSyncRouter } from "./heroku-sync-router";
import { registerHumanitecSyncRouter } from "./humanitec-sync-router";
@ -45,5 +46,6 @@ export const SECRET_SYNC_REGISTER_ROUTER_MAP: Record<SecretSync, (server: Fastif
[SecretSync.Heroku]: registerHerokuSyncRouter,
[SecretSync.Render]: registerRenderSyncRouter,
[SecretSync.Flyio]: registerFlyioSyncRouter,
[SecretSync.GitLab]: registerGitLabSyncRouter,
[SecretSync.CloudflarePages]: registerCloudflarePagesSyncRouter
};

View File

@ -22,10 +22,15 @@ import {
import { AzureDevOpsSyncListItemSchema, AzureDevOpsSyncSchema } from "@app/services/secret-sync/azure-devops";
import { AzureKeyVaultSyncListItemSchema, AzureKeyVaultSyncSchema } from "@app/services/secret-sync/azure-key-vault";
import { CamundaSyncListItemSchema, CamundaSyncSchema } from "@app/services/secret-sync/camunda";
import {
CloudflarePagesSyncListItemSchema,
CloudflarePagesSyncSchema
} from "@app/services/secret-sync/cloudflare-pages/cloudflare-pages-schema";
import { DatabricksSyncListItemSchema, DatabricksSyncSchema } from "@app/services/secret-sync/databricks";
import { FlyioSyncListItemSchema, FlyioSyncSchema } from "@app/services/secret-sync/flyio";
import { GcpSyncListItemSchema, GcpSyncSchema } from "@app/services/secret-sync/gcp";
import { GitHubSyncListItemSchema, GitHubSyncSchema } from "@app/services/secret-sync/github";
import { GitLabSyncListItemSchema, GitLabSyncSchema } from "@app/services/secret-sync/gitlab";
import { HCVaultSyncListItemSchema, HCVaultSyncSchema } from "@app/services/secret-sync/hc-vault";
import { HerokuSyncListItemSchema, HerokuSyncSchema } from "@app/services/secret-sync/heroku";
import { HumanitecSyncListItemSchema, HumanitecSyncSchema } from "@app/services/secret-sync/humanitec";
@ -34,10 +39,6 @@ import { TeamCitySyncListItemSchema, TeamCitySyncSchema } from "@app/services/se
import { TerraformCloudSyncListItemSchema, TerraformCloudSyncSchema } from "@app/services/secret-sync/terraform-cloud";
import { VercelSyncListItemSchema, VercelSyncSchema } from "@app/services/secret-sync/vercel";
import { WindmillSyncListItemSchema, WindmillSyncSchema } from "@app/services/secret-sync/windmill";
import {
CloudflarePagesSyncListItemSchema,
CloudflarePagesSyncSchema
} from "@app/services/secret-sync/cloudflare-pages/cloudflare-pages-schema";
const SecretSyncSchema = z.discriminatedUnion("destination", [
AwsParameterStoreSyncSchema,
@ -60,6 +61,7 @@ const SecretSyncSchema = z.discriminatedUnion("destination", [
HerokuSyncSchema,
RenderSyncSchema,
FlyioSyncSchema,
GitLabSyncSchema,
CloudflarePagesSyncSchema
]);
@ -84,6 +86,7 @@ const SecretSyncOptionsSchema = z.discriminatedUnion("destination", [
HerokuSyncListItemSchema,
RenderSyncListItemSchema,
FlyioSyncListItemSchema,
GitLabSyncListItemSchema,
CloudflarePagesSyncListItemSchema
]);

View File

@ -198,6 +198,7 @@ export const registerProjectRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.ProjectCreated,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
orgId: project.orgId,
name: project.name,

View File

@ -333,6 +333,7 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretPulled,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
numberOfSecrets: secrets.length,
workspaceId,
@ -489,6 +490,7 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
if (getUserAgentType(req.headers["user-agent"]) !== UserAgentType.K8_OPERATOR) {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretPulled,
organizationId: req.permission.orgId,
distinctId: getTelemetryDistinctId(req),
properties: {
numberOfSecrets: 1,
@ -615,6 +617,7 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretCreated,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
numberOfSecrets: 1,
workspaceId: projectId,
@ -750,6 +753,7 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretUpdated,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
numberOfSecrets: 1,
workspaceId: projectId,
@ -850,6 +854,7 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretDeleted,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
numberOfSecrets: 1,
workspaceId: projectId,
@ -957,6 +962,7 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretPulled,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
numberOfSecrets: secrets.length,
workspaceId: req.query.workspaceId,
@ -1036,6 +1042,7 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretPulled,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
numberOfSecrets: 1,
workspaceId: req.query.workspaceId,
@ -1207,6 +1214,7 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretCreated,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
numberOfSecrets: 1,
workspaceId: req.body.workspaceId,
@ -1396,6 +1404,7 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretUpdated,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
numberOfSecrets: 1,
workspaceId: req.body.workspaceId,
@ -1519,6 +1528,7 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretDeleted,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
numberOfSecrets: 1,
workspaceId: req.body.workspaceId,
@ -1702,6 +1712,7 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretCreated,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
numberOfSecrets: secrets.length,
workspaceId: req.body.workspaceId,
@ -1828,6 +1839,7 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretUpdated,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
numberOfSecrets: secrets.length,
workspaceId: req.body.workspaceId,
@ -1946,6 +1958,7 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretDeleted,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
numberOfSecrets: secrets.length,
workspaceId: req.body.workspaceId,
@ -2054,6 +2067,7 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretCreated,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
numberOfSecrets: secrets.length,
workspaceId: secrets[0].workspace,
@ -2209,6 +2223,7 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretUpdated,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
numberOfSecrets: secrets.length,
workspaceId: secrets[0].workspace,
@ -2307,6 +2322,7 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
await server.services.telemetry.sendPostHogEvents({
event: PostHogEventTypes.SecretDeleted,
distinctId: getTelemetryDistinctId(req),
organizationId: req.permission.orgId,
properties: {
numberOfSecrets: secrets.length,
workspaceId: secrets[0].workspace,

View File

@ -26,6 +26,7 @@ export enum AppConnection {
Heroku = "heroku",
Render = "render",
Flyio = "flyio",
GitLab = "gitlab",
Cloudflare = "cloudflare"
}

View File

@ -51,6 +51,11 @@ import {
validateAzureKeyVaultConnectionCredentials
} from "./azure-key-vault";
import { CamundaConnectionMethod, getCamundaConnectionListItem, validateCamundaConnectionCredentials } from "./camunda";
import { CloudflareConnectionMethod } from "./cloudflare/cloudflare-connection-enum";
import {
getCloudflareConnectionListItem,
validateCloudflareConnectionCredentials
} from "./cloudflare/cloudflare-connection-fns";
import {
DatabricksConnectionMethod,
getDatabricksConnectionListItem,
@ -64,6 +69,7 @@ import {
GitHubRadarConnectionMethod,
validateGitHubRadarConnectionCredentials
} from "./github-radar";
import { getGitLabConnectionListItem, GitLabConnectionMethod, validateGitLabConnectionCredentials } from "./gitlab";
import {
getHCVaultConnectionListItem,
HCVaultConnectionMethod,
@ -99,11 +105,6 @@ import {
validateWindmillConnectionCredentials,
WindmillConnectionMethod
} from "./windmill";
import {
getCloudflareConnectionListItem,
validateCloudflareConnectionCredentials
} from "./cloudflare/cloudflare-connection-fns";
import { CloudflareConnectionMethod } from "./cloudflare/cloudflare-connection-enum";
export const listAppConnectionOptions = () => {
return [
@ -134,6 +135,7 @@ export const listAppConnectionOptions = () => {
getHerokuConnectionListItem(),
getRenderConnectionListItem(),
getFlyioConnectionListItem(),
getGitLabConnectionListItem(),
getCloudflareConnectionListItem()
].sort((a, b) => a.name.localeCompare(b.name));
};
@ -213,6 +215,7 @@ export const validateAppConnectionCredentials = async (
[AppConnection.Heroku]: validateHerokuConnectionCredentials as TAppConnectionCredentialsValidator,
[AppConnection.Render]: validateRenderConnectionCredentials as TAppConnectionCredentialsValidator,
[AppConnection.Flyio]: validateFlyioConnectionCredentials as TAppConnectionCredentialsValidator,
[AppConnection.GitLab]: validateGitLabConnectionCredentials as TAppConnectionCredentialsValidator,
[AppConnection.Cloudflare]: validateCloudflareConnectionCredentials as TAppConnectionCredentialsValidator
};
@ -230,6 +233,7 @@ export const getAppConnectionMethodName = (method: TAppConnection["method"]) =>
case GitHubConnectionMethod.OAuth:
case AzureDevOpsConnectionMethod.OAuth:
case HerokuConnectionMethod.OAuth:
case GitLabConnectionMethod.OAuth:
return "OAuth";
case HerokuConnectionMethod.AuthToken:
return "Auth Token";
@ -327,6 +331,7 @@ export const TRANSITION_CONNECTION_CREDENTIALS_TO_PLATFORM: Record<
[AppConnection.Heroku]: platformManagedCredentialsNotSupported,
[AppConnection.Render]: platformManagedCredentialsNotSupported,
[AppConnection.Flyio]: platformManagedCredentialsNotSupported,
[AppConnection.GitLab]: platformManagedCredentialsNotSupported,
[AppConnection.Cloudflare]: platformManagedCredentialsNotSupported
};

View File

@ -28,6 +28,7 @@ export const APP_CONNECTION_NAME_MAP: Record<AppConnection, string> = {
[AppConnection.Heroku]: "Heroku",
[AppConnection.Render]: "Render",
[AppConnection.Flyio]: "Fly.io",
[AppConnection.GitLab]: "GitLab",
[AppConnection.Cloudflare]: "Cloudflare"
};
@ -59,5 +60,6 @@ export const APP_CONNECTION_PLAN_MAP: Record<AppConnection, AppConnectionPlanTyp
[AppConnection.Heroku]: AppConnectionPlanType.Regular,
[AppConnection.Render]: AppConnectionPlanType.Regular,
[AppConnection.Flyio]: AppConnectionPlanType.Regular,
[AppConnection.GitLab]: AppConnectionPlanType.Regular,
[AppConnection.Cloudflare]: AppConnectionPlanType.Regular
};

View File

@ -58,6 +58,8 @@ import { gcpConnectionService } from "./gcp/gcp-connection-service";
import { ValidateGitHubConnectionCredentialsSchema } from "./github";
import { githubConnectionService } from "./github/github-connection-service";
import { ValidateGitHubRadarConnectionCredentialsSchema } from "./github-radar";
import { ValidateGitLabConnectionCredentialsSchema } from "./gitlab";
import { gitlabConnectionService } from "./gitlab/gitlab-connection-service";
import { ValidateHCVaultConnectionCredentialsSchema } from "./hc-vault";
import { hcVaultConnectionService } from "./hc-vault/hc-vault-connection-service";
import { ValidateHerokuConnectionCredentialsSchema } from "./heroku";
@ -116,6 +118,7 @@ const VALIDATE_APP_CONNECTION_CREDENTIALS_MAP: Record<AppConnection, TValidateAp
[AppConnection.Heroku]: ValidateHerokuConnectionCredentialsSchema,
[AppConnection.Render]: ValidateRenderConnectionCredentialsSchema,
[AppConnection.Flyio]: ValidateFlyioConnectionCredentialsSchema,
[AppConnection.GitLab]: ValidateGitLabConnectionCredentialsSchema,
[AppConnection.Cloudflare]: ValidateCloudflareConnectionCredentialsSchema
};
@ -524,7 +527,8 @@ export const appConnectionServiceFactory = ({
onepass: onePassConnectionService(connectAppConnectionById),
heroku: herokuConnectionService(connectAppConnectionById, appConnectionDAL, kmsService),
render: renderConnectionService(connectAppConnectionById),
cloudflare: cloudflareConnectionService(connectAppConnectionById),
flyio: flyioConnectionService(connectAppConnectionById)
flyio: flyioConnectionService(connectAppConnectionById),
gitlab: gitlabConnectionService(connectAppConnectionById, appConnectionDAL, kmsService),
cloudflare: cloudflareConnectionService(connectAppConnectionById)
};
};

View File

@ -62,6 +62,12 @@ import {
TCamundaConnectionInput,
TValidateCamundaConnectionCredentialsSchema
} from "./camunda";
import {
TCloudflareConnection,
TCloudflareConnectionConfig,
TCloudflareConnectionInput,
TValidateCloudflareConnectionCredentialsSchema
} from "./cloudflare/cloudflare-connection-types";
import {
TDatabricksConnection,
TDatabricksConnectionConfig,
@ -92,6 +98,12 @@ import {
TGitHubRadarConnectionInput,
TValidateGitHubRadarConnectionCredentialsSchema
} from "./github-radar";
import {
TGitLabConnection,
TGitLabConnectionConfig,
TGitLabConnectionInput,
TValidateGitLabConnectionCredentialsSchema
} from "./gitlab";
import {
THCVaultConnection,
THCVaultConnectionConfig,
@ -153,12 +165,6 @@ import {
TWindmillConnectionConfig,
TWindmillConnectionInput
} from "./windmill";
import {
TCloudflareConnection,
TCloudflareConnectionConfig,
TCloudflareConnectionInput,
TValidateCloudflareConnectionCredentialsSchema
} from "./cloudflare/cloudflare-connection-types";
export type TAppConnection = { id: string } & (
| TAwsConnection
@ -188,6 +194,7 @@ export type TAppConnection = { id: string } & (
| THerokuConnection
| TRenderConnection
| TFlyioConnection
| TGitLabConnection
| TCloudflareConnection
);
@ -223,6 +230,7 @@ export type TAppConnectionInput = { id: string } & (
| THerokuConnectionInput
| TRenderConnectionInput
| TFlyioConnectionInput
| TGitLabConnectionInput
| TCloudflareConnectionInput
);
@ -266,6 +274,7 @@ export type TAppConnectionConfig =
| THerokuConnectionConfig
| TRenderConnectionConfig
| TFlyioConnectionConfig
| TGitLabConnectionConfig
| TCloudflareConnectionConfig;
export type TValidateAppConnectionCredentialsSchema =
@ -296,6 +305,7 @@ export type TValidateAppConnectionCredentialsSchema =
| TValidateHerokuConnectionCredentialsSchema
| TValidateRenderConnectionCredentialsSchema
| TValidateFlyioConnectionCredentialsSchema
| TValidateGitLabConnectionCredentialsSchema
| TValidateCloudflareConnectionCredentialsSchema;
export type TListAwsConnectionKmsKeys = {

View File

@ -1,6 +1,7 @@
import z from "zod";
import { AppConnections } from "@app/lib/api-docs";
import { CharacterType, characterValidator } from "@app/lib/validator/validate-string";
import { AppConnection } from "@app/services/app-connection/app-connection-enums";
import {
BaseAppConnectionSchema,
@ -9,7 +10,6 @@ import {
} from "@app/services/app-connection/app-connection-schemas";
import { CloudflareConnectionMethod } from "./cloudflare-connection-enum";
import { CharacterType, characterValidator } from "@app/lib/validator/validate-string";
const accountIdCharacterValidator = characterValidator([
CharacterType.AlphaNumeric,

View File

@ -1,3 +1,4 @@
import { logger } from "@app/lib/logger";
import { OrgServiceActor } from "@app/lib/types";
import { AppConnection } from "../app-connection-enums";
@ -19,6 +20,7 @@ export const gcpConnectionService = (getAppConnection: TGetAppConnectionFunc) =>
return projects;
} catch (error) {
logger.error(error, "Error listing GCP secret manager projects");
return [];
}
};

View File

@ -0,0 +1,9 @@
export enum GitLabConnectionMethod {
OAuth = "oauth",
AccessToken = "access-token"
}
export enum GitLabAccessTokenType {
Project = "project",
Personal = "personal"
}

View File

@ -0,0 +1,351 @@
/* eslint-disable no-await-in-loop */
import { GitbeakerRequestError, Gitlab } from "@gitbeaker/rest";
import { AxiosError } from "axios";
import { getConfig } from "@app/lib/config/env";
import { request } from "@app/lib/config/request";
import { BadRequestError, InternalServerError } from "@app/lib/errors";
import { removeTrailingSlash } from "@app/lib/fn";
import { logger } from "@app/lib/logger";
import { blockLocalAndPrivateIpAddresses } from "@app/lib/validator";
import { AppConnection } from "@app/services/app-connection/app-connection-enums";
import { encryptAppConnectionCredentials } from "@app/services/app-connection/app-connection-fns";
import { IntegrationUrls } from "@app/services/integration-auth/integration-list";
import { TKmsServiceFactory } from "@app/services/kms/kms-service";
import { TAppConnectionDALFactory } from "../app-connection-dal";
import { GitLabAccessTokenType, GitLabConnectionMethod } from "./gitlab-connection-enums";
import { TGitLabConnection, TGitLabConnectionConfig, TGitLabGroup, TGitLabProject } from "./gitlab-connection-types";
interface GitLabOAuthTokenResponse {
access_token: string;
token_type: string;
expires_in: number;
refresh_token: string;
created_at: number;
scope?: string;
}
export const getGitLabConnectionListItem = () => {
const { INF_APP_CONNECTION_GITLAB_OAUTH_CLIENT_ID } = getConfig();
return {
name: "GitLab" as const,
app: AppConnection.GitLab as const,
methods: Object.values(GitLabConnectionMethod) as [
GitLabConnectionMethod.AccessToken,
GitLabConnectionMethod.OAuth
],
oauthClientId: INF_APP_CONNECTION_GITLAB_OAUTH_CLIENT_ID
};
};
export const getGitLabInstanceUrl = async (instanceUrl?: string) => {
const gitLabInstanceUrl = instanceUrl ? removeTrailingSlash(instanceUrl) : IntegrationUrls.GITLAB_URL;
await blockLocalAndPrivateIpAddresses(gitLabInstanceUrl);
return gitLabInstanceUrl;
};
export const getGitLabClient = async (accessToken: string, instanceUrl?: string, isOAuth = false) => {
const host = await getGitLabInstanceUrl(instanceUrl);
const client = new Gitlab<true>({
host,
...(isOAuth ? { oauthToken: accessToken } : { token: accessToken }),
camelize: true
});
return client;
};
export const refreshGitLabToken = async (
refreshToken: string,
appId: string,
orgId: string,
appConnectionDAL: Pick<TAppConnectionDALFactory, "updateById">,
kmsService: Pick<TKmsServiceFactory, "createCipherPairWithDataKey">,
instanceUrl?: string
): Promise<string> => {
const { INF_APP_CONNECTION_GITLAB_OAUTH_CLIENT_ID, INF_APP_CONNECTION_GITLAB_OAUTH_CLIENT_SECRET, SITE_URL } =
getConfig();
if (!INF_APP_CONNECTION_GITLAB_OAUTH_CLIENT_SECRET || !INF_APP_CONNECTION_GITLAB_OAUTH_CLIENT_ID || !SITE_URL) {
throw new InternalServerError({
message: `GitLab environment variables have not been configured`
});
}
const payload = new URLSearchParams({
grant_type: "refresh_token",
refresh_token: refreshToken,
client_id: INF_APP_CONNECTION_GITLAB_OAUTH_CLIENT_ID,
client_secret: INF_APP_CONNECTION_GITLAB_OAUTH_CLIENT_SECRET,
redirect_uri: `${SITE_URL}/organization/app-connections/gitlab/oauth/callback`
});
try {
const url = await getGitLabInstanceUrl(instanceUrl);
const { data } = await request.post<GitLabOAuthTokenResponse>(`${url}/oauth/token`, payload.toString(), {
headers: {
"Content-Type": "application/x-www-form-urlencoded",
Accept: "application/json"
}
});
const expiresAt = new Date(Date.now() + data.expires_in * 1000 - 600000);
const encryptedCredentials = await encryptAppConnectionCredentials({
credentials: {
instanceUrl,
tokenType: data.token_type,
createdAt: new Date(data.created_at * 1000).toISOString(),
refreshToken: data.refresh_token,
accessToken: data.access_token,
expiresAt
},
orgId,
kmsService
});
await appConnectionDAL.updateById(appId, { encryptedCredentials });
return data.access_token;
} catch (error: unknown) {
if (error instanceof AxiosError) {
throw new BadRequestError({
message: `Failed to refresh GitLab token: ${error.message}`
});
}
throw new BadRequestError({
message: "Unable to refresh GitLab token"
});
}
};
export const exchangeGitLabOAuthCode = async (
code: string,
instanceUrl?: string
): Promise<GitLabOAuthTokenResponse> => {
const { INF_APP_CONNECTION_GITLAB_OAUTH_CLIENT_ID, INF_APP_CONNECTION_GITLAB_OAUTH_CLIENT_SECRET, SITE_URL } =
getConfig();
if (!INF_APP_CONNECTION_GITLAB_OAUTH_CLIENT_SECRET || !INF_APP_CONNECTION_GITLAB_OAUTH_CLIENT_ID || !SITE_URL) {
throw new InternalServerError({
message: `GitLab environment variables have not been configured`
});
}
try {
const payload = new URLSearchParams({
grant_type: "authorization_code",
code,
client_id: INF_APP_CONNECTION_GITLAB_OAUTH_CLIENT_ID,
client_secret: INF_APP_CONNECTION_GITLAB_OAUTH_CLIENT_SECRET,
redirect_uri: `${SITE_URL}/organization/app-connections/gitlab/oauth/callback`
});
const url = await getGitLabInstanceUrl(instanceUrl);
const response = await request.post<GitLabOAuthTokenResponse>(`${url}/oauth/token`, payload.toString(), {
headers: {
"Content-Type": "application/x-www-form-urlencoded",
Accept: "application/json"
}
});
if (!response.data) {
throw new InternalServerError({
message: "Failed to exchange OAuth code: Empty response"
});
}
return response.data;
} catch (error: unknown) {
if (error instanceof AxiosError) {
throw new BadRequestError({
message: `Failed to exchange OAuth code: ${error.message}`
});
}
throw new BadRequestError({
message: "Unable to exchange OAuth code"
});
}
};
export const validateGitLabConnectionCredentials = async (config: TGitLabConnectionConfig) => {
const { credentials: inputCredentials, method } = config;
let accessToken: string;
let oauthData: GitLabOAuthTokenResponse | null = null;
if (method === GitLabConnectionMethod.OAuth && "code" in inputCredentials) {
oauthData = await exchangeGitLabOAuthCode(inputCredentials.code, inputCredentials.instanceUrl);
accessToken = oauthData.access_token;
} else if (method === GitLabConnectionMethod.AccessToken && "accessToken" in inputCredentials) {
accessToken = inputCredentials.accessToken;
} else {
throw new BadRequestError({
message: "Invalid credentials for the selected connection method"
});
}
try {
const client = await getGitLabClient(
accessToken,
inputCredentials.instanceUrl,
method === GitLabConnectionMethod.OAuth
);
await client.Users.showCurrentUser();
} catch (error: unknown) {
logger.error(error, "Error validating GitLab connection credentials");
if (error instanceof GitbeakerRequestError) {
throw new BadRequestError({
message: `Failed to validate credentials: ${error.message ?? "Unknown error"}${error.cause?.description && error.message !== "Unauthorized" ? `. Cause: ${error.cause.description}` : ""}`
});
}
throw new BadRequestError({
message: `Failed to validate credentials: ${(error as Error)?.message || "verify credentials"}`
});
}
if (method === GitLabConnectionMethod.OAuth && oauthData) {
return {
accessToken,
instanceUrl: inputCredentials.instanceUrl,
refreshToken: oauthData.refresh_token,
expiresAt: new Date(Date.now() + oauthData.expires_in * 1000 - 60000),
tokenType: oauthData.token_type,
createdAt: new Date(oauthData.created_at * 1000)
};
}
return inputCredentials;
};
export const listGitLabProjects = async ({
appConnection,
appConnectionDAL,
kmsService
}: {
appConnection: TGitLabConnection;
appConnectionDAL: Pick<TAppConnectionDALFactory, "updateById">;
kmsService: Pick<TKmsServiceFactory, "createCipherPairWithDataKey">;
}): Promise<TGitLabProject[]> => {
let { accessToken } = appConnection.credentials;
if (
appConnection.method === GitLabConnectionMethod.OAuth &&
appConnection.credentials.refreshToken &&
new Date(appConnection.credentials.expiresAt) < new Date()
) {
accessToken = await refreshGitLabToken(
appConnection.credentials.refreshToken,
appConnection.id,
appConnection.orgId,
appConnectionDAL,
kmsService,
appConnection.credentials.instanceUrl
);
}
try {
const client = await getGitLabClient(
accessToken,
appConnection.credentials.instanceUrl,
appConnection.method === GitLabConnectionMethod.OAuth
);
const projects = await client.Projects.all({
archived: false,
includePendingDelete: false,
membership: true,
includeHidden: false,
imported: false
});
return projects.map((project) => ({
name: project.pathWithNamespace,
id: project.id.toString()
}));
} catch (error: unknown) {
if (error instanceof GitbeakerRequestError) {
throw new BadRequestError({
message: `Failed to fetch GitLab projects: ${error.message ?? "Unknown error"}${error.cause?.description && error.message !== "Unauthorized" ? `. Cause: ${error.cause.description}` : ""}`
});
}
if (error instanceof InternalServerError) {
throw error;
}
throw new InternalServerError({
message: "Unable to fetch GitLab projects"
});
}
};
export const listGitLabGroups = async ({
appConnection,
appConnectionDAL,
kmsService
}: {
appConnection: TGitLabConnection;
appConnectionDAL: Pick<TAppConnectionDALFactory, "updateById">;
kmsService: Pick<TKmsServiceFactory, "createCipherPairWithDataKey">;
}): Promise<TGitLabGroup[]> => {
let { accessToken } = appConnection.credentials;
if (
appConnection.method === GitLabConnectionMethod.AccessToken &&
appConnection.credentials.accessTokenType === GitLabAccessTokenType.Project
) {
return [];
}
if (
appConnection.method === GitLabConnectionMethod.OAuth &&
appConnection.credentials.refreshToken &&
new Date(appConnection.credentials.expiresAt) < new Date()
) {
accessToken = await refreshGitLabToken(
appConnection.credentials.refreshToken,
appConnection.id,
appConnection.orgId,
appConnectionDAL,
kmsService,
appConnection.credentials.instanceUrl
);
}
try {
const client = await getGitLabClient(
accessToken,
appConnection.credentials.instanceUrl,
appConnection.method === GitLabConnectionMethod.OAuth
);
const groups = await client.Groups.all({
orderBy: "name",
sort: "asc",
minAccessLevel: 50
});
return groups.map((group) => ({
id: group.id.toString(),
name: group.name
}));
} catch (error: unknown) {
if (error instanceof GitbeakerRequestError) {
throw new BadRequestError({
message: `Failed to fetch GitLab groups: ${error.message ?? "Unknown error"}${error.cause?.description && error.message !== "Unauthorized" ? `. Cause: ${error.cause.description}` : ""}`
});
}
if (error instanceof InternalServerError) {
throw error;
}
throw new InternalServerError({
message: "Unable to fetch GitLab groups"
});
}
};

View File

@ -0,0 +1,138 @@
import z from "zod";
import { AppConnections } from "@app/lib/api-docs";
import { AppConnection } from "@app/services/app-connection/app-connection-enums";
import {
BaseAppConnectionSchema,
GenericCreateAppConnectionFieldsSchema,
GenericUpdateAppConnectionFieldsSchema
} from "@app/services/app-connection/app-connection-schemas";
import { GitLabAccessTokenType, GitLabConnectionMethod } from "./gitlab-connection-enums";
export const GitLabConnectionAccessTokenCredentialsSchema = z.object({
accessToken: z
.string()
.trim()
.min(1, "Access Token required")
.describe(AppConnections.CREDENTIALS.GITLAB.accessToken),
instanceUrl: z
.string()
.trim()
.url("Invalid Instance URL")
.optional()
.describe(AppConnections.CREDENTIALS.GITLAB.instanceUrl),
accessTokenType: z.nativeEnum(GitLabAccessTokenType).describe(AppConnections.CREDENTIALS.GITLAB.accessTokenType)
});
export const GitLabConnectionOAuthCredentialsSchema = z.object({
code: z.string().trim().min(1, "OAuth code required").describe(AppConnections.CREDENTIALS.GITLAB.code),
instanceUrl: z
.string()
.trim()
.url("Invalid Instance URL")
.optional()
.describe(AppConnections.CREDENTIALS.GITLAB.instanceUrl)
});
export const GitLabConnectionOAuthOutputCredentialsSchema = z.object({
accessToken: z.string().trim(),
refreshToken: z.string().trim(),
expiresAt: z.date(),
tokenType: z.string().optional().default("bearer"),
createdAt: z.string().optional(),
instanceUrl: z
.string()
.trim()
.url("Invalid Instance URL")
.optional()
.describe(AppConnections.CREDENTIALS.GITLAB.instanceUrl)
});
export const GitLabConnectionRefreshTokenCredentialsSchema = z.object({
refreshToken: z.string().trim().min(1, "Refresh token required"),
instanceUrl: z
.string()
.trim()
.url("Invalid Instance URL")
.optional()
.describe(AppConnections.CREDENTIALS.GITLAB.instanceUrl)
});
const BaseGitLabConnectionSchema = BaseAppConnectionSchema.extend({
app: z.literal(AppConnection.GitLab)
});
export const GitLabConnectionSchema = z.intersection(
BaseGitLabConnectionSchema,
z.discriminatedUnion("method", [
z.object({
method: z.literal(GitLabConnectionMethod.AccessToken),
credentials: GitLabConnectionAccessTokenCredentialsSchema
}),
z.object({
method: z.literal(GitLabConnectionMethod.OAuth),
credentials: GitLabConnectionOAuthOutputCredentialsSchema
})
])
);
export const SanitizedGitLabConnectionSchema = z.discriminatedUnion("method", [
BaseGitLabConnectionSchema.extend({
method: z.literal(GitLabConnectionMethod.AccessToken),
credentials: GitLabConnectionAccessTokenCredentialsSchema.pick({
instanceUrl: true,
accessTokenType: true
})
}),
BaseGitLabConnectionSchema.extend({
method: z.literal(GitLabConnectionMethod.OAuth),
credentials: GitLabConnectionOAuthOutputCredentialsSchema.pick({
instanceUrl: true
})
})
]);
export const ValidateGitLabConnectionCredentialsSchema = z.discriminatedUnion("method", [
z.object({
method: z.literal(GitLabConnectionMethod.AccessToken).describe(AppConnections.CREATE(AppConnection.GitLab).method),
credentials: GitLabConnectionAccessTokenCredentialsSchema.describe(
AppConnections.CREATE(AppConnection.GitLab).credentials
)
}),
z.object({
method: z.literal(GitLabConnectionMethod.OAuth).describe(AppConnections.CREATE(AppConnection.GitLab).method),
credentials: z
.union([
GitLabConnectionOAuthCredentialsSchema,
GitLabConnectionRefreshTokenCredentialsSchema,
GitLabConnectionOAuthOutputCredentialsSchema
])
.describe(AppConnections.CREATE(AppConnection.GitLab).credentials)
})
]);
export const CreateGitLabConnectionSchema = ValidateGitLabConnectionCredentialsSchema.and(
GenericCreateAppConnectionFieldsSchema(AppConnection.GitLab)
);
export const UpdateGitLabConnectionSchema = z
.object({
credentials: z
.union([
GitLabConnectionAccessTokenCredentialsSchema,
GitLabConnectionOAuthOutputCredentialsSchema,
GitLabConnectionRefreshTokenCredentialsSchema,
GitLabConnectionOAuthCredentialsSchema
])
.optional()
.describe(AppConnections.UPDATE(AppConnection.GitLab).credentials)
})
.and(GenericUpdateAppConnectionFieldsSchema(AppConnection.GitLab));
export const GitLabConnectionListItemSchema = z.object({
name: z.literal("GitLab"),
app: z.literal(AppConnection.GitLab),
methods: z.nativeEnum(GitLabConnectionMethod).array(),
oauthClientId: z.string().optional()
});

View File

@ -0,0 +1,47 @@
import { logger } from "@app/lib/logger";
import { OrgServiceActor } from "@app/lib/types";
import { TKmsServiceFactory } from "@app/services/kms/kms-service";
import { TAppConnectionDALFactory } from "../app-connection-dal";
import { AppConnection } from "../app-connection-enums";
import { listGitLabGroups, listGitLabProjects } from "./gitlab-connection-fns";
import { TGitLabConnection } from "./gitlab-connection-types";
type TGetAppConnectionFunc = (
app: AppConnection,
connectionId: string,
actor: OrgServiceActor
) => Promise<TGitLabConnection>;
export const gitlabConnectionService = (
getAppConnection: TGetAppConnectionFunc,
appConnectionDAL: Pick<TAppConnectionDALFactory, "updateById">,
kmsService: Pick<TKmsServiceFactory, "createCipherPairWithDataKey">
) => {
const listProjects = async (connectionId: string, actor: OrgServiceActor) => {
try {
const appConnection = await getAppConnection(AppConnection.GitLab, connectionId, actor);
const projects = await listGitLabProjects({ appConnection, appConnectionDAL, kmsService });
return projects;
} catch (error) {
logger.error(error, `Failed to establish connection with GitLab for app ${connectionId}`);
return [];
}
};
const listGroups = async (connectionId: string, actor: OrgServiceActor) => {
try {
const appConnection = await getAppConnection(AppConnection.GitLab, connectionId, actor);
const groups = await listGitLabGroups({ appConnection, appConnectionDAL, kmsService });
return groups;
} catch (error) {
logger.error(error, `Failed to establish connection with GitLab for app ${connectionId}`);
return [];
}
};
return {
listProjects,
listGroups
};
};

View File

@ -0,0 +1,56 @@
import z from "zod";
import { DiscriminativePick } from "@app/lib/types";
import { AppConnection } from "../app-connection-enums";
import {
CreateGitLabConnectionSchema,
GitLabConnectionSchema,
ValidateGitLabConnectionCredentialsSchema
} from "./gitlab-connection-schemas";
export type TGitLabConnection = z.infer<typeof GitLabConnectionSchema>;
export type TGitLabConnectionInput = z.infer<typeof CreateGitLabConnectionSchema> & {
app: AppConnection.GitLab;
};
export type TValidateGitLabConnectionCredentialsSchema = typeof ValidateGitLabConnectionCredentialsSchema;
export type TGitLabConnectionConfig = DiscriminativePick<TGitLabConnectionInput, "method" | "app" | "credentials"> & {
orgId: string;
};
export type TGitLabProject = {
name: string;
id: string;
};
export type TGitLabAccessTokenCredentials = {
accessToken: string;
instanceUrl: string;
};
export type TGitLabOAuthCredentials = {
accessToken: string;
refreshToken: string;
expiresAt: Date;
tokenType?: string;
createdAt?: Date;
instanceUrl: string;
};
export type TGitLabOAuthCodeCredentials = {
code: string;
instanceUrl: string;
};
export type TGitLabRefreshTokenCredentials = {
refreshToken: string;
instanceUrl: string;
};
export interface TGitLabGroup {
id: string;
name: string;
}

View File

@ -0,0 +1,4 @@
export * from "./gitlab-connection-enums";
export * from "./gitlab-connection-fns";
export * from "./gitlab-connection-schemas";
export * from "./gitlab-connection-types";

View File

@ -79,7 +79,8 @@ export const identityAccessTokenDALFactory = (db: TDbClient) => {
trustedIpsOidcAuth: doc.accessTokenTrustedIpsOidc,
trustedIpsAccessTokenAuth: doc.accessTokenTrustedIpsToken,
trustedIpsAccessJwtAuth: doc.accessTokenTrustedIpsJwt,
trustedIpsAccessLdapAuth: doc.accessTokenTrustedIpsLdap
trustedIpsAccessLdapAuth: doc.accessTokenTrustedIpsLdap,
trustedIpsAccessTlsCertAuth: []
};
} catch (error) {
throw new DatabaseError({ error, name: "IdAccessTokenFindOne" });

View File

@ -201,7 +201,8 @@ export const identityAccessTokenServiceFactory = ({
[IdentityAuthMethod.OIDC_AUTH]: identityAccessToken.trustedIpsOidcAuth,
[IdentityAuthMethod.TOKEN_AUTH]: identityAccessToken.trustedIpsAccessTokenAuth,
[IdentityAuthMethod.JWT_AUTH]: identityAccessToken.trustedIpsAccessJwtAuth,
[IdentityAuthMethod.LDAP_AUTH]: identityAccessToken.trustedIpsAccessLdapAuth
[IdentityAuthMethod.LDAP_AUTH]: identityAccessToken.trustedIpsAccessLdapAuth,
[IdentityAuthMethod.TLS_CERT_AUTH]: identityAccessToken.trustedIpsAccessTlsCertAuth
};
const trustedIps = trustedIpsMap[identityAccessToken.authMethod as IdentityAuthMethod];

View File

@ -0,0 +1,10 @@
import { TDbClient } from "@app/db";
import { TableName } from "@app/db/schemas";
import { ormify, TOrmify } from "@app/lib/knex";
export type TIdentityTlsCertAuthDALFactory = TOrmify<TableName.IdentityTlsCertAuth>;
export const identityTlsCertAuthDALFactory = (db: TDbClient) => {
const orm = ormify(db, TableName.IdentityTlsCertAuth);
return orm;
};

View File

@ -0,0 +1,423 @@
import crypto from "node:crypto";
import { ForbiddenError } from "@casl/ability";
import jwt from "jsonwebtoken";
import { IdentityAuthMethod } from "@app/db/schemas";
import { TLicenseServiceFactory } from "@app/ee/services/license/license-service";
import { OrgPermissionIdentityActions, OrgPermissionSubjects } from "@app/ee/services/permission/org-permission";
import {
constructPermissionErrorMessage,
validatePrivilegeChangeOperation
} from "@app/ee/services/permission/permission-fns";
import { TPermissionServiceFactory } from "@app/ee/services/permission/permission-service-types";
import { extractX509CertFromChain } from "@app/lib/certificates/extract-certificate";
import { getConfig } from "@app/lib/config/env";
import { BadRequestError, NotFoundError, PermissionBoundaryError, UnauthorizedError } from "@app/lib/errors";
import { extractIPDetails, isValidIpOrCidr } from "@app/lib/ip";
import { ActorType, AuthTokenType } from "../auth/auth-type";
import { TIdentityOrgDALFactory } from "../identity/identity-org-dal";
import { TIdentityAccessTokenDALFactory } from "../identity-access-token/identity-access-token-dal";
import { TIdentityAccessTokenJwtPayload } from "../identity-access-token/identity-access-token-types";
import { TKmsServiceFactory } from "../kms/kms-service";
import { KmsDataKey } from "../kms/kms-types";
import { validateIdentityUpdateForSuperAdminPrivileges } from "../super-admin/super-admin-fns";
import { TIdentityTlsCertAuthDALFactory } from "./identity-tls-cert-auth-dal";
import { TIdentityTlsCertAuthServiceFactory } from "./identity-tls-cert-auth-types";
type TIdentityTlsCertAuthServiceFactoryDep = {
identityAccessTokenDAL: Pick<TIdentityAccessTokenDALFactory, "create" | "delete">;
identityTlsCertAuthDAL: Pick<
TIdentityTlsCertAuthDALFactory,
"findOne" | "transaction" | "create" | "updateById" | "delete"
>;
identityOrgMembershipDAL: Pick<TIdentityOrgDALFactory, "findOne">;
licenseService: Pick<TLicenseServiceFactory, "getPlan">;
permissionService: Pick<TPermissionServiceFactory, "getOrgPermission">;
kmsService: Pick<TKmsServiceFactory, "createCipherPairWithDataKey">;
};
const parseSubjectDetails = (data: string) => {
const values: Record<string, string> = {};
data.split("\n").forEach((el) => {
const [key, value] = el.split("=");
values[key.trim()] = value.trim();
});
return values;
};
export const identityTlsCertAuthServiceFactory = ({
identityAccessTokenDAL,
identityTlsCertAuthDAL,
identityOrgMembershipDAL,
licenseService,
permissionService,
kmsService
}: TIdentityTlsCertAuthServiceFactoryDep): TIdentityTlsCertAuthServiceFactory => {
const login: TIdentityTlsCertAuthServiceFactory["login"] = async ({ identityId, clientCertificate }) => {
const identityTlsCertAuth = await identityTlsCertAuthDAL.findOne({ identityId });
if (!identityTlsCertAuth) {
throw new NotFoundError({
message: "TLS Certificate auth method not found for identity, did you configure TLS Certificate auth?"
});
}
const identityMembershipOrg = await identityOrgMembershipDAL.findOne({
identityId: identityTlsCertAuth.identityId
});
if (!identityMembershipOrg) {
throw new NotFoundError({
message: `Identity organization membership for identity with ID '${identityTlsCertAuth.identityId}' not found`
});
}
const { decryptor } = await kmsService.createCipherPairWithDataKey({
type: KmsDataKey.Organization,
orgId: identityMembershipOrg.orgId
});
const caCertificate = decryptor({
cipherTextBlob: identityTlsCertAuth.encryptedCaCertificate
}).toString();
const leafCertificate = extractX509CertFromChain(decodeURIComponent(clientCertificate))?.[0];
if (!leafCertificate) {
throw new BadRequestError({ message: "Missing client certificate" });
}
const clientCertificateX509 = new crypto.X509Certificate(leafCertificate);
const caCertificateX509 = new crypto.X509Certificate(caCertificate);
const isValidCertificate = clientCertificateX509.verify(caCertificateX509.publicKey);
if (!isValidCertificate)
throw new UnauthorizedError({
message: "Access denied: Certificate not issued by the provided CA."
});
if (new Date(clientCertificateX509.validTo) < new Date()) {
throw new UnauthorizedError({
message: "Access denied: Certificate has expired."
});
}
if (new Date(clientCertificateX509.validFrom) > new Date()) {
throw new UnauthorizedError({
message: "Access denied: Certificate not yet valid."
});
}
const subjectDetails = parseSubjectDetails(clientCertificateX509.subject);
if (identityTlsCertAuth.allowedCommonNames) {
const isValidCommonName = identityTlsCertAuth.allowedCommonNames.split(",").includes(subjectDetails.CN);
if (!isValidCommonName) {
throw new UnauthorizedError({
message: "Access denied: TLS Certificate Auth common name not allowed."
});
}
}
// Generate the token
const identityAccessToken = await identityTlsCertAuthDAL.transaction(async (tx) => {
const newToken = await identityAccessTokenDAL.create(
{
identityId: identityTlsCertAuth.identityId,
isAccessTokenRevoked: false,
accessTokenTTL: identityTlsCertAuth.accessTokenTTL,
accessTokenMaxTTL: identityTlsCertAuth.accessTokenMaxTTL,
accessTokenNumUses: 0,
accessTokenNumUsesLimit: identityTlsCertAuth.accessTokenNumUsesLimit,
authMethod: IdentityAuthMethod.TLS_CERT_AUTH
},
tx
);
return newToken;
});
const appCfg = getConfig();
const accessToken = jwt.sign(
{
identityId: identityTlsCertAuth.identityId,
identityAccessTokenId: identityAccessToken.id,
authTokenType: AuthTokenType.IDENTITY_ACCESS_TOKEN
} as TIdentityAccessTokenJwtPayload,
appCfg.AUTH_SECRET,
Number(identityAccessToken.accessTokenTTL) === 0
? undefined
: {
expiresIn: Number(identityAccessToken.accessTokenTTL)
}
);
return {
identityTlsCertAuth,
accessToken,
identityAccessToken,
identityMembershipOrg
};
};
const attachTlsCertAuth: TIdentityTlsCertAuthServiceFactory["attachTlsCertAuth"] = async ({
identityId,
accessTokenTTL,
accessTokenMaxTTL,
accessTokenNumUsesLimit,
accessTokenTrustedIps,
actorId,
actorAuthMethod,
actor,
actorOrgId,
isActorSuperAdmin,
caCertificate,
allowedCommonNames
}) => {
await validateIdentityUpdateForSuperAdminPrivileges(identityId, isActorSuperAdmin);
const identityMembershipOrg = await identityOrgMembershipDAL.findOne({ identityId });
if (!identityMembershipOrg) throw new NotFoundError({ message: `Failed to find identity with ID ${identityId}` });
if (identityMembershipOrg.identity.authMethods.includes(IdentityAuthMethod.TLS_CERT_AUTH)) {
throw new BadRequestError({
message: "Failed to add TLS Certificate Auth to already configured identity"
});
}
if (accessTokenMaxTTL > 0 && accessTokenTTL > accessTokenMaxTTL) {
throw new BadRequestError({ message: "Access token TTL cannot be greater than max TTL" });
}
const { permission } = await permissionService.getOrgPermission(
actor,
actorId,
identityMembershipOrg.orgId,
actorAuthMethod,
actorOrgId
);
ForbiddenError.from(permission).throwUnlessCan(OrgPermissionIdentityActions.Create, OrgPermissionSubjects.Identity);
const plan = await licenseService.getPlan(identityMembershipOrg.orgId);
const reformattedAccessTokenTrustedIps = accessTokenTrustedIps.map((accessTokenTrustedIp) => {
if (
!plan.ipAllowlisting &&
accessTokenTrustedIp.ipAddress !== "0.0.0.0/0" &&
accessTokenTrustedIp.ipAddress !== "::/0"
)
throw new BadRequestError({
message:
"Failed to add IP access range to access token due to plan restriction. Upgrade plan to add IP access range."
});
if (!isValidIpOrCidr(accessTokenTrustedIp.ipAddress))
throw new BadRequestError({
message: "The IP is not a valid IPv4, IPv6, or CIDR block"
});
return extractIPDetails(accessTokenTrustedIp.ipAddress);
});
const { encryptor } = await kmsService.createCipherPairWithDataKey({
type: KmsDataKey.Organization,
orgId: identityMembershipOrg.orgId
});
const identityTlsCertAuth = await identityTlsCertAuthDAL.transaction(async (tx) => {
const doc = await identityTlsCertAuthDAL.create(
{
identityId: identityMembershipOrg.identityId,
accessTokenMaxTTL,
allowedCommonNames,
accessTokenTTL,
encryptedCaCertificate: encryptor({ plainText: Buffer.from(caCertificate) }).cipherTextBlob,
accessTokenNumUsesLimit,
accessTokenTrustedIps: JSON.stringify(reformattedAccessTokenTrustedIps)
},
tx
);
return doc;
});
return { ...identityTlsCertAuth, orgId: identityMembershipOrg.orgId };
};
const updateTlsCertAuth: TIdentityTlsCertAuthServiceFactory["updateTlsCertAuth"] = async ({
identityId,
caCertificate,
allowedCommonNames,
accessTokenTTL,
accessTokenMaxTTL,
accessTokenNumUsesLimit,
accessTokenTrustedIps,
actorId,
actorAuthMethod,
actor,
actorOrgId
}) => {
const identityMembershipOrg = await identityOrgMembershipDAL.findOne({ identityId });
if (!identityMembershipOrg) throw new NotFoundError({ message: `Failed to find identity with ID ${identityId}` });
if (!identityMembershipOrg.identity.authMethods.includes(IdentityAuthMethod.TLS_CERT_AUTH)) {
throw new NotFoundError({
message: "The identity does not have TLS Certificate Auth attached"
});
}
const identityTlsCertAuth = await identityTlsCertAuthDAL.findOne({ identityId });
if (
(accessTokenMaxTTL || identityTlsCertAuth.accessTokenMaxTTL) > 0 &&
(accessTokenTTL || identityTlsCertAuth.accessTokenTTL) >
(accessTokenMaxTTL || identityTlsCertAuth.accessTokenMaxTTL)
) {
throw new BadRequestError({ message: "Access token TTL cannot be greater than max TTL" });
}
const { permission } = await permissionService.getOrgPermission(
actor,
actorId,
identityMembershipOrg.orgId,
actorAuthMethod,
actorOrgId
);
ForbiddenError.from(permission).throwUnlessCan(OrgPermissionIdentityActions.Edit, OrgPermissionSubjects.Identity);
const plan = await licenseService.getPlan(identityMembershipOrg.orgId);
const reformattedAccessTokenTrustedIps = accessTokenTrustedIps?.map((accessTokenTrustedIp) => {
if (
!plan.ipAllowlisting &&
accessTokenTrustedIp.ipAddress !== "0.0.0.0/0" &&
accessTokenTrustedIp.ipAddress !== "::/0"
)
throw new BadRequestError({
message:
"Failed to add IP access range to access token due to plan restriction. Upgrade plan to add IP access range."
});
if (!isValidIpOrCidr(accessTokenTrustedIp.ipAddress))
throw new BadRequestError({
message: "The IP is not a valid IPv4, IPv6, or CIDR block"
});
return extractIPDetails(accessTokenTrustedIp.ipAddress);
});
const { encryptor } = await kmsService.createCipherPairWithDataKey({
type: KmsDataKey.Organization,
orgId: identityMembershipOrg.orgId
});
const updatedTlsCertAuth = await identityTlsCertAuthDAL.updateById(identityTlsCertAuth.id, {
allowedCommonNames,
encryptedCaCertificate: caCertificate
? encryptor({ plainText: Buffer.from(caCertificate) }).cipherTextBlob
: undefined,
accessTokenMaxTTL,
accessTokenTTL,
accessTokenNumUsesLimit,
accessTokenTrustedIps: reformattedAccessTokenTrustedIps
? JSON.stringify(reformattedAccessTokenTrustedIps)
: undefined
});
return { ...updatedTlsCertAuth, orgId: identityMembershipOrg.orgId };
};
const getTlsCertAuth: TIdentityTlsCertAuthServiceFactory["getTlsCertAuth"] = async ({
identityId,
actorId,
actor,
actorAuthMethod,
actorOrgId
}) => {
const identityMembershipOrg = await identityOrgMembershipDAL.findOne({ identityId });
if (!identityMembershipOrg) throw new NotFoundError({ message: `Failed to find identity with ID ${identityId}` });
if (!identityMembershipOrg.identity.authMethods.includes(IdentityAuthMethod.TLS_CERT_AUTH)) {
throw new BadRequestError({
message: "The identity does not have TLS Certificate Auth attached"
});
}
const identityAuth = await identityTlsCertAuthDAL.findOne({ identityId });
const { permission } = await permissionService.getOrgPermission(
actor,
actorId,
identityMembershipOrg.orgId,
actorAuthMethod,
actorOrgId
);
ForbiddenError.from(permission).throwUnlessCan(OrgPermissionIdentityActions.Read, OrgPermissionSubjects.Identity);
const { decryptor } = await kmsService.createCipherPairWithDataKey({
type: KmsDataKey.Organization,
orgId: identityMembershipOrg.orgId
});
let caCertificate = "";
if (identityAuth.encryptedCaCertificate) {
caCertificate = decryptor({ cipherTextBlob: identityAuth.encryptedCaCertificate }).toString();
}
return { ...identityAuth, caCertificate, orgId: identityMembershipOrg.orgId };
};
const revokeTlsCertAuth: TIdentityTlsCertAuthServiceFactory["revokeTlsCertAuth"] = async ({
identityId,
actorId,
actor,
actorAuthMethod,
actorOrgId
}) => {
const identityMembershipOrg = await identityOrgMembershipDAL.findOne({ identityId });
if (!identityMembershipOrg) throw new NotFoundError({ message: `Failed to find identity with ID ${identityId}` });
if (!identityMembershipOrg.identity.authMethods.includes(IdentityAuthMethod.TLS_CERT_AUTH)) {
throw new BadRequestError({
message: "The identity does not have TLS Certificate auth"
});
}
const { permission, membership } = await permissionService.getOrgPermission(
actor,
actorId,
identityMembershipOrg.orgId,
actorAuthMethod,
actorOrgId
);
ForbiddenError.from(permission).throwUnlessCan(OrgPermissionIdentityActions.Edit, OrgPermissionSubjects.Identity);
const { permission: rolePermission } = await permissionService.getOrgPermission(
ActorType.IDENTITY,
identityMembershipOrg.identityId,
identityMembershipOrg.orgId,
actorAuthMethod,
actorOrgId
);
const permissionBoundary = validatePrivilegeChangeOperation(
membership.shouldUseNewPrivilegeSystem,
OrgPermissionIdentityActions.RevokeAuth,
OrgPermissionSubjects.Identity,
permission,
rolePermission
);
if (!permissionBoundary.isValid)
throw new PermissionBoundaryError({
message: constructPermissionErrorMessage(
"Failed to revoke TLS Certificate auth of identity with more privileged role",
membership.shouldUseNewPrivilegeSystem,
OrgPermissionIdentityActions.RevokeAuth,
OrgPermissionSubjects.Identity
),
details: { missingPermissions: permissionBoundary.missingPermissions }
});
const revokedIdentityTlsCertAuth = await identityTlsCertAuthDAL.transaction(async (tx) => {
const deletedTlsCertAuth = await identityTlsCertAuthDAL.delete({ identityId }, tx);
await identityAccessTokenDAL.delete({ identityId, authMethod: IdentityAuthMethod.TLS_CERT_AUTH }, tx);
return { ...deletedTlsCertAuth?.[0], orgId: identityMembershipOrg.orgId };
});
return revokedIdentityTlsCertAuth;
};
return {
login,
attachTlsCertAuth,
updateTlsCertAuth,
getTlsCertAuth,
revokeTlsCertAuth
};
};

View File

@ -0,0 +1,49 @@
import { TIdentityAccessTokens, TIdentityOrgMemberships, TIdentityTlsCertAuths } from "@app/db/schemas";
import { TProjectPermission } from "@app/lib/types";
export type TLoginTlsCertAuthDTO = {
identityId: string;
clientCertificate: string;
};
export type TAttachTlsCertAuthDTO = {
identityId: string;
caCertificate: string;
allowedCommonNames?: string | null;
accessTokenTTL: number;
accessTokenMaxTTL: number;
accessTokenNumUsesLimit: number;
accessTokenTrustedIps: { ipAddress: string }[];
isActorSuperAdmin?: boolean;
} & Omit<TProjectPermission, "projectId">;
export type TUpdateTlsCertAuthDTO = {
identityId: string;
caCertificate?: string;
allowedCommonNames?: string | null;
accessTokenTTL?: number;
accessTokenMaxTTL?: number;
accessTokenNumUsesLimit?: number;
accessTokenTrustedIps?: { ipAddress: string }[];
} & Omit<TProjectPermission, "projectId">;
export type TGetTlsCertAuthDTO = {
identityId: string;
} & Omit<TProjectPermission, "projectId">;
export type TRevokeTlsCertAuthDTO = {
identityId: string;
} & Omit<TProjectPermission, "projectId">;
export type TIdentityTlsCertAuthServiceFactory = {
login: (dto: TLoginTlsCertAuthDTO) => Promise<{
identityTlsCertAuth: TIdentityTlsCertAuths;
accessToken: string;
identityAccessToken: TIdentityAccessTokens;
identityMembershipOrg: TIdentityOrgMemberships;
}>;
attachTlsCertAuth: (dto: TAttachTlsCertAuthDTO) => Promise<TIdentityTlsCertAuths>;
updateTlsCertAuth: (dto: TUpdateTlsCertAuthDTO) => Promise<TIdentityTlsCertAuths>;
revokeTlsCertAuth: (dto: TRevokeTlsCertAuthDTO) => Promise<TIdentityTlsCertAuths>;
getTlsCertAuth: (dto: TGetTlsCertAuthDTO) => Promise<TIdentityTlsCertAuths & { caCertificate: string }>;
};

View File

@ -11,7 +11,8 @@ export const buildAuthMethods = ({
azureId,
tokenId,
jwtId,
ldapId
ldapId,
tlsCertId
}: {
uaId?: string;
gcpId?: string;
@ -24,6 +25,7 @@ export const buildAuthMethods = ({
tokenId?: string;
jwtId?: string;
ldapId?: string;
tlsCertId?: string;
}) => {
return [
...[uaId ? IdentityAuthMethod.UNIVERSAL_AUTH : null],
@ -36,6 +38,7 @@ export const buildAuthMethods = ({
...[azureId ? IdentityAuthMethod.AZURE_AUTH : null],
...[tokenId ? IdentityAuthMethod.TOKEN_AUTH : null],
...[jwtId ? IdentityAuthMethod.JWT_AUTH : null],
...[ldapId ? IdentityAuthMethod.LDAP_AUTH : null]
...[ldapId ? IdentityAuthMethod.LDAP_AUTH : null],
...[tlsCertId ? IdentityAuthMethod.TLS_CERT_AUTH : null]
].filter((authMethod) => authMethod) as IdentityAuthMethod[];
};

View File

@ -12,6 +12,7 @@ import {
TIdentityOciAuths,
TIdentityOidcAuths,
TIdentityOrgMemberships,
TIdentityTlsCertAuths,
TIdentityTokenAuths,
TIdentityUniversalAuths,
TOrgRoles
@ -99,7 +100,11 @@ export const identityOrgDALFactory = (db: TDbClient) => {
`${TableName.IdentityOrgMembership}.identityId`,
`${TableName.IdentityLdapAuth}.identityId`
)
.leftJoin<TIdentityTlsCertAuths>(
TableName.IdentityTlsCertAuth,
`${TableName.IdentityOrgMembership}.identityId`,
`${TableName.IdentityTlsCertAuth}.identityId`
)
.select(
selectAllTableCols(TableName.IdentityOrgMembership),
@ -114,6 +119,7 @@ export const identityOrgDALFactory = (db: TDbClient) => {
db.ref("id").as("tokenId").withSchema(TableName.IdentityTokenAuth),
db.ref("id").as("jwtId").withSchema(TableName.IdentityJwtAuth),
db.ref("id").as("ldapId").withSchema(TableName.IdentityLdapAuth),
db.ref("id").as("tlsCertId").withSchema(TableName.IdentityTlsCertAuth),
db.ref("name").withSchema(TableName.Identity),
db.ref("hasDeleteProtection").withSchema(TableName.Identity)
);
@ -238,7 +244,11 @@ export const identityOrgDALFactory = (db: TDbClient) => {
"paginatedIdentity.identityId",
`${TableName.IdentityLdapAuth}.identityId`
)
.leftJoin<TIdentityTlsCertAuths>(
TableName.IdentityTlsCertAuth,
"paginatedIdentity.identityId",
`${TableName.IdentityTlsCertAuth}.identityId`
)
.select(
db.ref("id").withSchema("paginatedIdentity"),
db.ref("role").withSchema("paginatedIdentity"),
@ -260,7 +270,8 @@ export const identityOrgDALFactory = (db: TDbClient) => {
db.ref("id").as("azureId").withSchema(TableName.IdentityAzureAuth),
db.ref("id").as("tokenId").withSchema(TableName.IdentityTokenAuth),
db.ref("id").as("jwtId").withSchema(TableName.IdentityJwtAuth),
db.ref("id").as("ldapId").withSchema(TableName.IdentityLdapAuth)
db.ref("id").as("ldapId").withSchema(TableName.IdentityLdapAuth),
db.ref("id").as("tlsCertId").withSchema(TableName.IdentityTlsCertAuth)
)
// cr stands for custom role
.select(db.ref("id").as("crId").withSchema(TableName.OrgRoles))
@ -306,6 +317,7 @@ export const identityOrgDALFactory = (db: TDbClient) => {
azureId,
tokenId,
ldapId,
tlsCertId,
createdAt,
updatedAt
}) => ({
@ -313,7 +325,6 @@ export const identityOrgDALFactory = (db: TDbClient) => {
roleId,
identityId,
id,
orgId,
createdAt,
updatedAt,
@ -341,7 +352,8 @@ export const identityOrgDALFactory = (db: TDbClient) => {
azureId,
tokenId,
jwtId,
ldapId
ldapId,
tlsCertId
})
}
}),

View File

@ -307,7 +307,6 @@ export const AwsParameterStoreSyncFns = {
awsParameterStoreSecretsRecord,
Boolean(syncOptions.tags?.length || syncOptions.syncSecretMetadataAsTags)
);
const syncTagsRecord = Object.fromEntries(syncOptions.tags?.map((tag) => [tag.key, tag.value]) ?? []);
for await (const entry of Object.entries(secretMap)) {
const [key, { value, secretMetadata }] = entry;
@ -342,13 +341,13 @@ export const AwsParameterStoreSyncFns = {
}
}
if (shouldManageTags) {
if ((syncOptions.tags !== undefined || syncOptions.syncSecretMetadataAsTags) && shouldManageTags) {
const { tagsToAdd, tagKeysToRemove } = processParameterTags({
syncTagsRecord: {
// configured sync tags take preference over secret metadata
...(syncOptions.syncSecretMetadataAsTags &&
Object.fromEntries(secretMetadata?.map((tag) => [tag.key, tag.value]) ?? [])),
...syncTagsRecord
...(syncOptions.tags && Object.fromEntries(syncOptions.tags?.map((tag) => [tag.key, tag.value]) ?? []))
},
awsTagsRecord: awsParameterStoreTagsRecord[key] ?? {}
});

View File

@ -366,37 +366,39 @@ export const AwsSecretsManagerSyncFns = {
}
}
const { tagsToAdd, tagKeysToRemove } = processTags({
syncTagsRecord: {
// configured sync tags take preference over secret metadata
...(syncOptions.syncSecretMetadataAsTags &&
Object.fromEntries(secretMetadata?.map((tag) => [tag.key, tag.value]) ?? [])),
...syncTagsRecord
},
awsTagsRecord: Object.fromEntries(
awsDescriptionsRecord[key]?.Tags?.map((tag) => [tag.Key!, tag.Value!]) ?? []
)
});
if (syncOptions.tags !== undefined || syncOptions.syncSecretMetadataAsTags) {
const { tagsToAdd, tagKeysToRemove } = processTags({
syncTagsRecord: {
// configured sync tags take preference over secret metadata
...(syncOptions.syncSecretMetadataAsTags &&
Object.fromEntries(secretMetadata?.map((tag) => [tag.key, tag.value]) ?? [])),
...(syncOptions.tags !== undefined && syncTagsRecord)
},
awsTagsRecord: Object.fromEntries(
awsDescriptionsRecord[key]?.Tags?.map((tag) => [tag.Key!, tag.Value!]) ?? []
)
});
if (tagsToAdd.length) {
try {
await addTags(client, key, tagsToAdd);
} catch (error) {
throw new SecretSyncError({
error,
secretKey: key
});
if (tagsToAdd.length) {
try {
await addTags(client, key, tagsToAdd);
} catch (error) {
throw new SecretSyncError({
error,
secretKey: key
});
}
}
}
if (tagKeysToRemove.length) {
try {
await removeTags(client, key, tagKeysToRemove);
} catch (error) {
throw new SecretSyncError({
error,
secretKey: key
});
if (tagKeysToRemove.length) {
try {
await removeTags(client, key, tagKeysToRemove);
} catch (error) {
throw new SecretSyncError({
error,
secretKey: key
});
}
}
}
}
@ -439,32 +441,34 @@ export const AwsSecretsManagerSyncFns = {
});
}
const { tagsToAdd, tagKeysToRemove } = processTags({
syncTagsRecord,
awsTagsRecord: Object.fromEntries(
awsDescriptionsRecord[destinationConfig.secretName]?.Tags?.map((tag) => [tag.Key!, tag.Value!]) ?? []
)
});
if (syncOptions.tags !== undefined) {
const { tagsToAdd, tagKeysToRemove } = processTags({
syncTagsRecord,
awsTagsRecord: Object.fromEntries(
awsDescriptionsRecord[destinationConfig.secretName]?.Tags?.map((tag) => [tag.Key!, tag.Value!]) ?? []
)
});
if (tagsToAdd.length) {
try {
await addTags(client, destinationConfig.secretName, tagsToAdd);
} catch (error) {
throw new SecretSyncError({
error,
secretKey: destinationConfig.secretName
});
if (tagsToAdd.length) {
try {
await addTags(client, destinationConfig.secretName, tagsToAdd);
} catch (error) {
throw new SecretSyncError({
error,
secretKey: destinationConfig.secretName
});
}
}
}
if (tagKeysToRemove.length) {
try {
await removeTags(client, destinationConfig.secretName, tagKeysToRemove);
} catch (error) {
throw new SecretSyncError({
error,
secretKey: destinationConfig.secretName
});
if (tagKeysToRemove.length) {
try {
await removeTags(client, destinationConfig.secretName, tagKeysToRemove);
} catch (error) {
throw new SecretSyncError({
error,
secretKey: destinationConfig.secretName
});
}
}
}
}

View File

@ -0,0 +1,10 @@
import { AppConnection } from "@app/services/app-connection/app-connection-enums";
import { SecretSync } from "@app/services/secret-sync/secret-sync-enums";
import { TSecretSyncListItem } from "@app/services/secret-sync/secret-sync-types";
export const GITLAB_SYNC_LIST_OPTION: TSecretSyncListItem = {
name: "GitLab",
destination: SecretSync.GitLab,
connection: AppConnection.GitLab,
canImportSecrets: false
};

View File

@ -0,0 +1,4 @@
export enum GitLabSyncScope {
Project = "project",
Group = "group"
}

View File

@ -0,0 +1,452 @@
/* eslint-disable no-await-in-loop */
import { GitbeakerRequestError } from "@gitbeaker/rest";
import { TAppConnectionDALFactory } from "@app/services/app-connection/app-connection-dal";
import {
getGitLabClient,
GitLabConnectionMethod,
refreshGitLabToken,
TGitLabConnection
} from "@app/services/app-connection/gitlab";
import { TKmsServiceFactory } from "@app/services/kms/kms-service";
import { TGitLabSyncWithCredentials, TGitLabVariable } from "@app/services/secret-sync/gitlab/gitlab-sync-types";
import { SecretSyncError } from "@app/services/secret-sync/secret-sync-errors";
import { matchesSchema } from "@app/services/secret-sync/secret-sync-fns";
import { TSecretMap } from "@app/services/secret-sync/secret-sync-types";
import { SECRET_SYNC_NAME_MAP } from "../secret-sync-maps";
import { GitLabSyncScope } from "./gitlab-sync-enums";
interface TGitLabVariablePayload {
key?: string;
value: string;
variable_type?: "env_var" | "file";
environment_scope?: string;
protected?: boolean;
masked?: boolean;
masked_and_hidden?: boolean;
description?: string;
}
interface TGitLabVariableCreate extends TGitLabVariablePayload {
key: string;
}
interface TGitLabVariableUpdate extends Omit<TGitLabVariablePayload, "key"> {}
type TGitLabSyncFactoryDeps = {
appConnectionDAL: Pick<TAppConnectionDALFactory, "updateById">;
kmsService: Pick<TKmsServiceFactory, "createCipherPairWithDataKey">;
};
const getValidAccessToken = async (
connection: TGitLabConnection,
appConnectionDAL: Pick<TAppConnectionDALFactory, "updateById">,
kmsService: Pick<TKmsServiceFactory, "createCipherPairWithDataKey">
): Promise<string> => {
if (
connection.method === GitLabConnectionMethod.OAuth &&
connection.credentials.refreshToken &&
new Date(connection.credentials.expiresAt) < new Date()
) {
const accessToken = await refreshGitLabToken(
connection.credentials.refreshToken,
connection.id,
connection.orgId,
appConnectionDAL,
kmsService,
connection.credentials.instanceUrl
);
return accessToken;
}
return connection.credentials.accessToken;
};
const getGitLabVariables = async ({
accessToken,
connection,
scope,
resourceId,
targetEnvironment
}: {
accessToken: string;
connection: TGitLabConnection;
scope: GitLabSyncScope;
resourceId: string;
targetEnvironment?: string;
}): Promise<TGitLabVariable[]> => {
try {
const client = await getGitLabClient(
accessToken,
connection.credentials.instanceUrl,
connection.method === GitLabConnectionMethod.OAuth
);
let variables: TGitLabVariable[] = [];
if (scope === GitLabSyncScope.Project) {
variables = await client.ProjectVariables.all(resourceId);
} else {
variables = await client.GroupVariables.all(resourceId);
}
if (targetEnvironment) {
variables = variables.filter((v) => v.environmentScope === targetEnvironment);
}
return variables;
} catch (error) {
if (error instanceof GitbeakerRequestError) {
throw new SecretSyncError({
error: new Error(
`Failed to fetch variables: ${error.message ?? "Unknown error"}${error.cause?.description && error.message !== "Unauthorized" ? `. Cause: ${error.cause.description}` : ""}`
)
});
}
throw new SecretSyncError({
error
});
}
};
const createGitLabVariable = async ({
accessToken,
connection,
scope,
resourceId,
variable
}: {
accessToken: string;
connection: TGitLabConnection;
scope: GitLabSyncScope;
resourceId: string;
variable: TGitLabVariableCreate;
}): Promise<void> => {
try {
const client = await getGitLabClient(
accessToken,
connection.credentials.instanceUrl,
connection.method === GitLabConnectionMethod.OAuth
);
const payload = {
key: variable.key,
value: variable.value,
variableType: "env_var",
environmentScope: variable.environment_scope || "*",
protected: variable.protected || false,
masked: variable.masked || false,
masked_and_hidden: variable.masked_and_hidden || false,
raw: false
};
if (scope === GitLabSyncScope.Project) {
await client.ProjectVariables.create(resourceId, payload.key, payload.value, {
variableType: "env_var",
environmentScope: payload.environmentScope,
protected: payload.protected,
masked: payload.masked,
masked_and_hidden: payload.masked_and_hidden,
raw: false
});
} else {
await client.GroupVariables.create(resourceId, payload.key, payload.value, {
variableType: "env_var",
environmentScope: payload.environmentScope,
protected: payload.protected,
masked: payload.masked,
...(payload.masked_and_hidden && { masked_and_hidden: payload.masked_and_hidden }),
raw: false
});
}
} catch (error) {
if (error instanceof GitbeakerRequestError) {
throw new SecretSyncError({
error: new Error(
`Failed to create variable: ${error.message ?? "Unknown error"}${error.cause?.description && error.message !== "Unauthorized" ? `. Cause: ${error.cause.description}` : ""}`
),
secretKey: variable.key
});
}
throw new SecretSyncError({
error,
secretKey: variable.key
});
}
};
const updateGitLabVariable = async ({
accessToken,
connection,
scope,
resourceId,
key,
variable,
targetEnvironment
}: {
accessToken: string;
connection: TGitLabConnection;
scope: GitLabSyncScope;
resourceId: string;
key: string;
variable: TGitLabVariableUpdate;
targetEnvironment?: string;
}): Promise<void> => {
try {
const client = await getGitLabClient(
accessToken,
connection.credentials.instanceUrl,
connection.method === GitLabConnectionMethod.OAuth
);
const options = {
...(variable.environment_scope && { environmentScope: variable.environment_scope }),
...(variable.protected !== undefined && { protected: variable.protected }),
...(variable.masked !== undefined && { masked: variable.masked })
};
if (targetEnvironment) {
options.environmentScope = targetEnvironment;
}
if (scope === GitLabSyncScope.Project) {
await client.ProjectVariables.edit(resourceId, key, variable.value, {
...options,
filter: { environment_scope: targetEnvironment || "*" }
});
} else {
await client.GroupVariables.edit(resourceId, key, variable.value, {
...options,
filter: { environment_scope: targetEnvironment || "*" }
});
}
} catch (error) {
if (error instanceof GitbeakerRequestError) {
throw new SecretSyncError({
error: new Error(
`Failed to update variable: ${error.message ?? "Unknown error"}${error.cause?.description && error.message !== "Unauthorized" ? `. Cause: ${error.cause.description}` : ""}`
),
secretKey: key
});
}
throw new SecretSyncError({
error,
secretKey: key
});
}
};
const deleteGitLabVariable = async ({
accessToken,
connection,
scope,
resourceId,
key,
targetEnvironment,
allVariables
}: {
accessToken: string;
connection: TGitLabConnection;
scope: GitLabSyncScope;
resourceId: string;
key: string;
targetEnvironment?: string;
allVariables?: TGitLabVariable[];
}): Promise<void> => {
if (allVariables && !allVariables.find((v) => v.key === key)) {
return;
}
try {
const client = await getGitLabClient(
accessToken,
connection.credentials.instanceUrl,
connection.method === GitLabConnectionMethod.OAuth
);
const options: { filter?: { environment_scope: string } } = {};
if (targetEnvironment) {
options.filter = { environment_scope: targetEnvironment || "*" };
}
if (scope === GitLabSyncScope.Project) {
await client.ProjectVariables.remove(resourceId, key, options);
} else {
await client.GroupVariables.remove(resourceId, key);
}
} catch (error: unknown) {
if (error instanceof GitbeakerRequestError) {
throw new SecretSyncError({
error: new Error(
`Failed to delete variable: ${error.message ?? "Unknown error"}${error.cause?.description && error.message !== "Unauthorized" ? `. Cause: ${error.cause.description}` : ""}`
),
secretKey: key
});
}
throw new SecretSyncError({
error,
secretKey: key
});
}
};
export const GitLabSyncFns = {
syncSecrets: async (
secretSync: TGitLabSyncWithCredentials,
secretMap: TSecretMap,
{ appConnectionDAL, kmsService }: TGitLabSyncFactoryDeps
): Promise<void> => {
const { connection, environment, destinationConfig } = secretSync;
const { scope, targetEnvironment } = destinationConfig;
const resourceId = scope === GitLabSyncScope.Project ? destinationConfig.projectId : destinationConfig.groupId;
const accessToken = await getValidAccessToken(connection, appConnectionDAL, kmsService);
try {
const currentVariables = await getGitLabVariables({
accessToken,
connection,
scope,
resourceId,
targetEnvironment
});
const currentVariableMap = new Map(currentVariables.map((v) => [v.key, v]));
for (const [key, { value }] of Object.entries(secretMap)) {
if (value?.length < 8 && destinationConfig.shouldMaskSecrets) {
throw new SecretSyncError({
message: `Secret ${key} is too short to be masked. GitLab requires a minimum of 8 characters for masked secrets.`,
secretKey: key
});
}
try {
const existingVariable = currentVariableMap.get(key);
if (existingVariable) {
if (
existingVariable.value !== value ||
existingVariable.environmentScope !== targetEnvironment ||
existingVariable.protected !== destinationConfig.shouldProtectSecrets ||
existingVariable.masked !== destinationConfig.shouldMaskSecrets
) {
await updateGitLabVariable({
accessToken,
connection,
scope,
resourceId,
key,
variable: {
value,
environment_scope: targetEnvironment,
protected: destinationConfig.shouldProtectSecrets,
masked: destinationConfig.shouldMaskSecrets || existingVariable.hidden
},
targetEnvironment
});
}
} else {
await createGitLabVariable({
accessToken,
connection,
scope,
resourceId,
variable: {
key,
value,
variable_type: "env_var",
environment_scope: targetEnvironment || "*",
protected: destinationConfig.shouldProtectSecrets || false,
masked: destinationConfig.shouldMaskSecrets || false,
masked_and_hidden: destinationConfig.shouldHideSecrets || false
}
});
}
} catch (error) {
throw new SecretSyncError({
error,
secretKey: key
});
}
}
if (!secretSync.syncOptions.disableSecretDeletion) {
for (const variable of currentVariables) {
try {
const shouldDelete =
matchesSchema(variable.key, environment?.slug || "", secretSync.syncOptions.keySchema) &&
!(variable.key in secretMap);
if (shouldDelete) {
await deleteGitLabVariable({
accessToken,
connection,
scope,
resourceId,
key: variable.key,
targetEnvironment
});
}
} catch (error) {
throw new SecretSyncError({
error,
secretKey: variable.key
});
}
}
}
} catch (error) {
if (error instanceof SecretSyncError) {
throw error;
}
throw new SecretSyncError({
message: "Failed to sync secrets",
error
});
}
},
removeSecrets: async (
secretSync: TGitLabSyncWithCredentials,
secretMap: TSecretMap,
{ appConnectionDAL, kmsService }: TGitLabSyncFactoryDeps
): Promise<void> => {
const { connection, destinationConfig } = secretSync;
const { scope, targetEnvironment } = destinationConfig;
const resourceId = scope === GitLabSyncScope.Project ? destinationConfig.projectId : destinationConfig.groupId;
const accessToken = await getValidAccessToken(connection, appConnectionDAL, kmsService);
const allVariables = await getGitLabVariables({
accessToken,
connection,
scope,
resourceId,
targetEnvironment
});
for (const key of Object.keys(secretMap)) {
try {
await deleteGitLabVariable({
accessToken,
connection,
scope,
resourceId,
key,
targetEnvironment,
allVariables
});
} catch (error) {
throw new SecretSyncError({
error,
secretKey: key
});
}
}
},
getSecrets: async (secretSync: TGitLabSyncWithCredentials): Promise<TSecretMap> => {
throw new Error(`${SECRET_SYNC_NAME_MAP[secretSync.destination]} does not support importing secrets.`);
}
};

View File

@ -0,0 +1,97 @@
import { z } from "zod";
import { SecretSyncs } from "@app/lib/api-docs";
import { AppConnection } from "@app/services/app-connection/app-connection-enums";
import { SecretSync } from "@app/services/secret-sync/secret-sync-enums";
import {
BaseSecretSyncSchema,
GenericCreateSecretSyncFieldsSchema,
GenericUpdateSecretSyncFieldsSchema
} from "@app/services/secret-sync/secret-sync-schemas";
import { TSyncOptionsConfig } from "@app/services/secret-sync/secret-sync-types";
import { GitLabSyncScope } from "./gitlab-sync-enums";
const GitLabSyncDestinationConfigSchema = z.discriminatedUnion("scope", [
z.object({
scope: z.literal(GitLabSyncScope.Project).describe(SecretSyncs.DESTINATION_CONFIG.GITLAB.scope),
projectId: z.string().min(1, "Project ID is required").describe(SecretSyncs.DESTINATION_CONFIG.GITLAB.projectId),
projectName: z
.string()
.min(1, "Project name is required")
.describe(SecretSyncs.DESTINATION_CONFIG.GITLAB.projectName),
targetEnvironment: z
.string()
.optional()
.default("*")
.describe(SecretSyncs.DESTINATION_CONFIG.GITLAB.targetEnvironment),
shouldProtectSecrets: z
.boolean()
.optional()
.default(false)
.describe(SecretSyncs.DESTINATION_CONFIG.GITLAB.shouldProtectSecrets),
shouldMaskSecrets: z
.boolean()
.optional()
.default(false)
.describe(SecretSyncs.DESTINATION_CONFIG.GITLAB.shouldMaskSecrets),
shouldHideSecrets: z
.boolean()
.optional()
.default(false)
.describe(SecretSyncs.DESTINATION_CONFIG.GITLAB.shouldHideSecrets)
}),
z.object({
scope: z.literal(GitLabSyncScope.Group).describe(SecretSyncs.DESTINATION_CONFIG.GITLAB.scope),
groupId: z.string().min(1, "Group ID is required").describe(SecretSyncs.DESTINATION_CONFIG.GITLAB.groupId),
groupName: z.string().min(1, "Group name is required").describe(SecretSyncs.DESTINATION_CONFIG.GITLAB.groupName),
targetEnvironment: z
.string()
.optional()
.default("*")
.describe(SecretSyncs.DESTINATION_CONFIG.GITLAB.targetEnvironment),
shouldProtectSecrets: z
.boolean()
.optional()
.default(false)
.describe(SecretSyncs.DESTINATION_CONFIG.GITLAB.shouldProtectSecrets),
shouldMaskSecrets: z
.boolean()
.optional()
.default(false)
.describe(SecretSyncs.DESTINATION_CONFIG.GITLAB.shouldMaskSecrets),
shouldHideSecrets: z
.boolean()
.optional()
.default(false)
.describe(SecretSyncs.DESTINATION_CONFIG.GITLAB.shouldHideSecrets)
})
]);
const GitLabSyncOptionsConfig: TSyncOptionsConfig = { canImportSecrets: false };
export const GitLabSyncSchema = BaseSecretSyncSchema(SecretSync.GitLab, GitLabSyncOptionsConfig).extend({
destination: z.literal(SecretSync.GitLab),
destinationConfig: GitLabSyncDestinationConfigSchema
});
export const CreateGitLabSyncSchema = GenericCreateSecretSyncFieldsSchema(
SecretSync.GitLab,
GitLabSyncOptionsConfig
).extend({
destinationConfig: GitLabSyncDestinationConfigSchema
});
export const UpdateGitLabSyncSchema = GenericUpdateSecretSyncFieldsSchema(
SecretSync.GitLab,
GitLabSyncOptionsConfig
).extend({
destinationConfig: GitLabSyncDestinationConfigSchema.optional()
});
export const GitLabSyncListItemSchema = z.object({
name: z.literal("GitLab"),
connection: z.literal(AppConnection.GitLab),
destination: z.literal(SecretSync.GitLab),
canImportSecrets: z.literal(false)
});

View File

@ -0,0 +1,58 @@
import { z } from "zod";
import { TGitLabConnection } from "@app/services/app-connection/gitlab";
import { CreateGitLabSyncSchema, GitLabSyncListItemSchema, GitLabSyncSchema } from "./gitlab-sync-schemas";
export type TGitLabSync = z.infer<typeof GitLabSyncSchema>;
export type TGitLabSyncInput = z.infer<typeof CreateGitLabSyncSchema>;
export type TGitLabSyncListItem = z.infer<typeof GitLabSyncListItemSchema>;
export type TGitLabSyncWithCredentials = TGitLabSync & {
connection: TGitLabConnection;
};
export type TGitLabVariable = {
key: string;
value: string;
protected: boolean;
masked: boolean;
environmentScope?: string;
hidden?: boolean;
};
export type TGitLabVariableCreate = {
key: string;
value: string;
variable_type?: "env_var" | "file";
protected?: boolean;
masked?: boolean;
raw?: boolean;
environment_scope?: string;
description?: string;
};
export type TGitLabVariableUpdate = {
value: string;
variable_type?: "env_var" | "file";
protected?: boolean;
masked?: boolean;
raw?: boolean;
environment_scope?: string;
description?: string | null;
};
export type TGitLabListVariables = {
accessToken: string;
projectId: string;
environmentScope?: string;
};
export type TGitLabCreateVariable = TGitLabListVariables & {
variable: TGitLabVariableCreate;
};
export type TGitLabUpdateVariable = TGitLabListVariables & {
key: string;
variable: TGitLabVariableUpdate;
};

View File

@ -0,0 +1,4 @@
export * from "./gitlab-sync-constants";
export * from "./gitlab-sync-fns";
export * from "./gitlab-sync-schemas";
export * from "./gitlab-sync-types";

View File

@ -19,6 +19,7 @@ export enum SecretSync {
Heroku = "heroku",
Render = "render",
Flyio = "flyio",
GitLab = "gitlab",
CloudflarePages = "cloudflare-pages"
}

View File

@ -34,6 +34,7 @@ import { CloudflarePagesSyncFns } from "./cloudflare-pages/cloudflare-pages-fns"
import { FLYIO_SYNC_LIST_OPTION, FlyioSyncFns } from "./flyio";
import { GCP_SYNC_LIST_OPTION } from "./gcp";
import { GcpSyncFns } from "./gcp/gcp-sync-fns";
import { GITLAB_SYNC_LIST_OPTION, GitLabSyncFns } from "./gitlab";
import { HC_VAULT_SYNC_LIST_OPTION, HCVaultSyncFns } from "./hc-vault";
import { HEROKU_SYNC_LIST_OPTION, HerokuSyncFns } from "./heroku";
import { HUMANITEC_SYNC_LIST_OPTION } from "./humanitec";
@ -66,6 +67,7 @@ const SECRET_SYNC_LIST_OPTIONS: Record<SecretSync, TSecretSyncListItem> = {
[SecretSync.Heroku]: HEROKU_SYNC_LIST_OPTION,
[SecretSync.Render]: RENDER_SYNC_LIST_OPTION,
[SecretSync.Flyio]: FLYIO_SYNC_LIST_OPTION,
[SecretSync.GitLab]: GITLAB_SYNC_LIST_OPTION,
[SecretSync.CloudflarePages]: CLOUDFLARE_PAGES_SYNC_LIST_OPTION
};
@ -230,6 +232,8 @@ export const SecretSyncFns = {
return RenderSyncFns.syncSecrets(secretSync, schemaSecretMap);
case SecretSync.Flyio:
return FlyioSyncFns.syncSecrets(secretSync, schemaSecretMap);
case SecretSync.GitLab:
return GitLabSyncFns.syncSecrets(secretSync, schemaSecretMap, { appConnectionDAL, kmsService });
case SecretSync.CloudflarePages:
return CloudflarePagesSyncFns.syncSecrets(secretSync, schemaSecretMap);
default:
@ -318,6 +322,9 @@ export const SecretSyncFns = {
case SecretSync.Flyio:
secretMap = await FlyioSyncFns.getSecrets(secretSync);
break;
case SecretSync.GitLab:
secretMap = await GitLabSyncFns.getSecrets(secretSync);
break;
case SecretSync.CloudflarePages:
secretMap = await CloudflarePagesSyncFns.getSecrets(secretSync);
break;
@ -394,6 +401,8 @@ export const SecretSyncFns = {
return RenderSyncFns.removeSecrets(secretSync, schemaSecretMap);
case SecretSync.Flyio:
return FlyioSyncFns.removeSecrets(secretSync, schemaSecretMap);
case SecretSync.GitLab:
return GitLabSyncFns.removeSecrets(secretSync, schemaSecretMap, { appConnectionDAL, kmsService });
case SecretSync.CloudflarePages:
return CloudflarePagesSyncFns.removeSecrets(secretSync, schemaSecretMap);
default:

View File

@ -22,6 +22,7 @@ export const SECRET_SYNC_NAME_MAP: Record<SecretSync, string> = {
[SecretSync.Heroku]: "Heroku",
[SecretSync.Render]: "Render",
[SecretSync.Flyio]: "Fly.io",
[SecretSync.GitLab]: "GitLab",
[SecretSync.CloudflarePages]: "Cloudflare Pages"
};
@ -46,6 +47,7 @@ export const SECRET_SYNC_CONNECTION_MAP: Record<SecretSync, AppConnection> = {
[SecretSync.Heroku]: AppConnection.Heroku,
[SecretSync.Render]: AppConnection.Render,
[SecretSync.Flyio]: AppConnection.Flyio,
[SecretSync.GitLab]: AppConnection.GitLab,
[SecretSync.CloudflarePages]: AppConnection.Cloudflare
};
@ -70,5 +72,6 @@ export const SECRET_SYNC_PLAN_MAP: Record<SecretSync, SecretSyncPlanType> = {
[SecretSync.Heroku]: SecretSyncPlanType.Regular,
[SecretSync.Render]: SecretSyncPlanType.Regular,
[SecretSync.Flyio]: SecretSyncPlanType.Regular,
[SecretSync.GitLab]: SecretSyncPlanType.Regular,
[SecretSync.CloudflarePages]: SecretSyncPlanType.Regular
};

View File

@ -72,8 +72,15 @@ import {
TAzureKeyVaultSyncListItem,
TAzureKeyVaultSyncWithCredentials
} from "./azure-key-vault";
import {
TCloudflarePagesSync,
TCloudflarePagesSyncInput,
TCloudflarePagesSyncListItem,
TCloudflarePagesSyncWithCredentials
} from "./cloudflare-pages/cloudflare-pages-types";
import { TFlyioSync, TFlyioSyncInput, TFlyioSyncListItem, TFlyioSyncWithCredentials } from "./flyio/flyio-sync-types";
import { TGcpSync, TGcpSyncInput, TGcpSyncListItem, TGcpSyncWithCredentials } from "./gcp";
import { TGitLabSync, TGitLabSyncInput, TGitLabSyncListItem, TGitLabSyncWithCredentials } from "./gitlab";
import {
THCVaultSync,
THCVaultSyncInput,
@ -106,12 +113,6 @@ import {
TTerraformCloudSyncWithCredentials
} from "./terraform-cloud";
import { TVercelSync, TVercelSyncInput, TVercelSyncListItem, TVercelSyncWithCredentials } from "./vercel";
import {
TCloudflarePagesSync,
TCloudflarePagesSyncInput,
TCloudflarePagesSyncListItem,
TCloudflarePagesSyncWithCredentials
} from "./cloudflare-pages/cloudflare-pages-types";
export type TSecretSync =
| TAwsParameterStoreSync
@ -134,6 +135,7 @@ export type TSecretSync =
| THerokuSync
| TRenderSync
| TFlyioSync
| TGitLabSync
| TCloudflarePagesSync;
export type TSecretSyncWithCredentials =
@ -157,6 +159,7 @@ export type TSecretSyncWithCredentials =
| THerokuSyncWithCredentials
| TRenderSyncWithCredentials
| TFlyioSyncWithCredentials
| TGitLabSyncWithCredentials
| TCloudflarePagesSyncWithCredentials;
export type TSecretSyncInput =
@ -180,6 +183,7 @@ export type TSecretSyncInput =
| THerokuSyncInput
| TRenderSyncInput
| TFlyioSyncInput
| TGitLabSyncInput
| TCloudflarePagesSyncInput;
export type TSecretSyncListItem =
@ -203,6 +207,7 @@ export type TSecretSyncListItem =
| THerokuSyncListItem
| TRenderSyncListItem
| TFlyioSyncListItem
| TGitLabSyncListItem
| TCloudflarePagesSyncListItem;
export type TSyncOptionsConfig = {

View File

@ -7,13 +7,18 @@ import { QueueJobs, QueueName, TQueueServiceFactory } from "@app/queue";
import { getServerCfg } from "../super-admin/super-admin-service";
import { TTelemetryDALFactory } from "./telemetry-dal";
import { TELEMETRY_SECRET_OPERATIONS_KEY, TELEMETRY_SECRET_PROCESSED_KEY } from "./telemetry-service";
import {
TELEMETRY_SECRET_OPERATIONS_KEY,
TELEMETRY_SECRET_PROCESSED_KEY,
TTelemetryServiceFactory
} from "./telemetry-service";
import { PostHogEventTypes } from "./telemetry-types";
type TTelemetryQueueServiceFactoryDep = {
queueService: TQueueServiceFactory;
keyStore: Pick<TKeyStoreFactory, "getItem" | "deleteItem">;
telemetryDAL: TTelemetryDALFactory;
telemetryService: TTelemetryServiceFactory;
};
export type TTelemetryQueueServiceFactory = ReturnType<typeof telemetryQueueServiceFactory>;
@ -21,7 +26,8 @@ export type TTelemetryQueueServiceFactory = ReturnType<typeof telemetryQueueServ
export const telemetryQueueServiceFactory = ({
queueService,
keyStore,
telemetryDAL
telemetryDAL,
telemetryService
}: TTelemetryQueueServiceFactoryDep) => {
const appCfg = getConfig();
const postHog =
@ -48,6 +54,10 @@ export const telemetryQueueServiceFactory = ({
await keyStore.deleteItem(TELEMETRY_SECRET_OPERATIONS_KEY);
});
queueService.start(QueueName.TelemetryAggregatedEvents, async () => {
await telemetryService.processAggregatedEvents();
});
// every day at midnight a telemetry job executes on self-hosted instances
// this sends some telemetry information like instance id secrets operated etc
const startTelemetryCheck = async () => {
@ -60,11 +70,26 @@ export const telemetryQueueServiceFactory = ({
{ pattern: "0 0 * * *", utc: true },
QueueName.TelemetryInstanceStats // just a job id
);
// clear previous aggregated events job
await queueService.stopRepeatableJob(
QueueName.TelemetryAggregatedEvents,
QueueJobs.TelemetryAggregatedEvents,
{ pattern: "*/5 * * * *", utc: true },
QueueName.TelemetryAggregatedEvents // just a job id
);
if (postHog) {
await queueService.queue(QueueName.TelemetryInstanceStats, QueueJobs.TelemetryInstanceStats, undefined, {
jobId: QueueName.TelemetryInstanceStats,
repeat: { pattern: "0 0 * * *", utc: true }
});
// Start aggregated events job (runs every five minutes)
await queueService.queue(QueueName.TelemetryAggregatedEvents, QueueJobs.TelemetryAggregatedEvents, undefined, {
jobId: QueueName.TelemetryAggregatedEvents,
repeat: { pattern: "*/5 * * * *", utc: true }
});
}
};
@ -72,6 +97,10 @@ export const telemetryQueueServiceFactory = ({
logger.error(err?.failedReason, `${QueueName.TelemetryInstanceStats}: failed`);
});
queueService.listen(QueueName.TelemetryAggregatedEvents, "failed", (err) => {
logger.error(err?.failedReason, `${QueueName.TelemetryAggregatedEvents}: failed`);
});
return {
startTelemetryCheck
};

View File

@ -1,3 +1,4 @@
import { createHash, randomUUID } from "crypto";
import { PostHog } from "posthog-node";
import { TLicenseServiceFactory } from "@app/ee/services/license/license-service";
@ -12,12 +13,49 @@ import { PostHogEventTypes, TPostHogEvent, TSecretModifiedEvent } from "./teleme
export const TELEMETRY_SECRET_PROCESSED_KEY = "telemetry-secret-processed";
export const TELEMETRY_SECRET_OPERATIONS_KEY = "telemetry-secret-operations";
export const POSTHOG_AGGREGATED_EVENTS = [PostHogEventTypes.SecretPulled];
const TELEMETRY_AGGREGATED_KEY_EXP = 900; // 15mins
// Bucket configuration
const TELEMETRY_BUCKET_COUNT = 30;
const TELEMETRY_BUCKET_NAMES = Array.from(
{ length: TELEMETRY_BUCKET_COUNT },
(_, i) => `bucket-${i.toString().padStart(2, "0")}`
);
type AggregatedEventData = Record<string, unknown>;
type SingleEventData = {
distinctId: string;
event: string;
properties: unknown;
organizationId: string;
};
export type TTelemetryServiceFactory = ReturnType<typeof telemetryServiceFactory>;
export type TTelemetryServiceFactoryDep = {
keyStore: Pick<TKeyStoreFactory, "getItem" | "incrementBy">;
keyStore: Pick<
TKeyStoreFactory,
"incrementBy" | "deleteItemsByKeyIn" | "setItemWithExpiry" | "getKeysByPattern" | "getItems"
>;
licenseService: Pick<TLicenseServiceFactory, "getInstanceType">;
};
const getBucketForDistinctId = (distinctId: string): string => {
// Use SHA-256 hash for consistent distribution
const hash = createHash("sha256").update(distinctId).digest("hex");
// Take first 8 characters and convert to number for better distribution
const hashNumber = parseInt(hash.substring(0, 8), 16);
const bucketIndex = hashNumber % TELEMETRY_BUCKET_COUNT;
return TELEMETRY_BUCKET_NAMES[bucketIndex];
};
export const createTelemetryEventKey = (event: string, distinctId: string): string => {
const bucketId = getBucketForDistinctId(distinctId);
return `telemetry-event-${event}-${bucketId}-${distinctId}-${randomUUID()}`;
};
export const telemetryServiceFactory = ({ keyStore, licenseService }: TTelemetryServiceFactoryDep) => {
const appCfg = getConfig();
@ -64,11 +102,33 @@ To opt into telemetry, you can set "TELEMETRY_ENABLED=true" within the environme
const instanceType = licenseService.getInstanceType();
// capture posthog only when its cloud or signup event happens in self-hosted
if (instanceType === InstanceType.Cloud || event.event === PostHogEventTypes.UserSignedUp) {
postHog.capture({
event: event.event,
distinctId: event.distinctId,
properties: event.properties
});
if (event.organizationId) {
try {
postHog.groupIdentify({ groupType: "organization", groupKey: event.organizationId });
} catch (error) {
logger.error(error, "Failed to identify PostHog organization");
}
}
if (POSTHOG_AGGREGATED_EVENTS.includes(event.event)) {
const eventKey = createTelemetryEventKey(event.event, event.distinctId);
await keyStore.setItemWithExpiry(
eventKey,
TELEMETRY_AGGREGATED_KEY_EXP,
JSON.stringify({
distinctId: event.distinctId,
event: event.event,
properties: event.properties,
organizationId: event.organizationId
})
);
} else {
postHog.capture({
event: event.event,
distinctId: event.distinctId,
properties: event.properties,
...(event.organizationId ? { groups: { organization: event.organizationId } } : {})
});
}
return;
}
@ -89,6 +149,160 @@ To opt into telemetry, you can set "TELEMETRY_ENABLED=true" within the environme
}
};
const aggregateGroupProperties = (events: SingleEventData[]): AggregatedEventData => {
const aggregatedData: AggregatedEventData = {};
// Set the total count
aggregatedData.count = events.length;
events.forEach((event) => {
if (!event.properties) return;
Object.entries(event.properties as Record<string, unknown>).forEach(([key, value]: [string, unknown]) => {
if (Array.isArray(value)) {
// For arrays, count occurrences of each item
const existingCounts =
aggregatedData[key] &&
typeof aggregatedData[key] === "object" &&
aggregatedData[key]?.constructor === Object
? (aggregatedData[key] as Record<string, number>)
: {};
value.forEach((item) => {
const itemKey = typeof item === "object" ? JSON.stringify(item) : String(item);
existingCounts[itemKey] = (existingCounts[itemKey] || 0) + 1;
});
aggregatedData[key] = existingCounts;
} else if (typeof value === "object" && value?.constructor === Object) {
// For objects, count occurrences of each field value
const existingCounts =
aggregatedData[key] &&
typeof aggregatedData[key] === "object" &&
aggregatedData[key]?.constructor === Object
? (aggregatedData[key] as Record<string, number>)
: {};
if (value) {
Object.values(value).forEach((fieldValue) => {
const valueKey = typeof fieldValue === "object" ? JSON.stringify(fieldValue) : String(fieldValue);
existingCounts[valueKey] = (existingCounts[valueKey] || 0) + 1;
});
}
aggregatedData[key] = existingCounts;
} else if (typeof value === "number") {
// For numbers, add to existing sum
aggregatedData[key] = ((aggregatedData[key] as number) || 0) + value;
} else if (value !== undefined && value !== null) {
// For other types (strings, booleans, etc.), count occurrences
const stringValue = String(value);
const existingValue = aggregatedData[key];
if (!existingValue) {
aggregatedData[key] = { [stringValue]: 1 };
} else if (existingValue && typeof existingValue === "object" && existingValue.constructor === Object) {
const countObject = existingValue as Record<string, number>;
countObject[stringValue] = (countObject[stringValue] || 0) + 1;
} else {
const oldValue = String(existingValue);
aggregatedData[key] = {
[oldValue]: 1,
[stringValue]: 1
};
}
}
});
});
return aggregatedData;
};
const processBucketEvents = async (eventType: string, bucketId: string) => {
if (!postHog) return 0;
try {
const bucketPattern = `telemetry-event-${eventType}-${bucketId}-*`;
const bucketKeys = await keyStore.getKeysByPattern(bucketPattern);
if (bucketKeys.length === 0) return 0;
const bucketEvents = await keyStore.getItems(bucketKeys);
let bucketEventsParsed: SingleEventData[] = [];
try {
bucketEventsParsed = bucketEvents
.filter((event) => event !== null)
.map((event) => JSON.parse(event as string) as SingleEventData);
} catch (error) {
logger.error(error, `Failed to parse bucket events for ${eventType} in ${bucketId}`);
return 0;
}
const eventsGrouped = new Map<string, SingleEventData[]>();
bucketEventsParsed.forEach((event) => {
const key = JSON.stringify({ id: event.distinctId, org: event.organizationId });
if (!eventsGrouped.has(key)) {
eventsGrouped.set(key, []);
}
eventsGrouped.get(key)!.push(event);
});
if (eventsGrouped.size === 0) return 0;
for (const [eventsKey, events] of eventsGrouped) {
const key = JSON.parse(eventsKey) as { id: string; org?: string };
if (key.org) {
try {
postHog.groupIdentify({ groupType: "organization", groupKey: key.org });
} catch (error) {
logger.error(error, "Failed to identify PostHog organization");
}
}
const properties = aggregateGroupProperties(events);
postHog.capture({
event: `${eventType} aggregated`,
distinctId: key.id,
properties,
...(key.org ? { groups: { organization: key.org } } : {})
});
}
// Clean up processed data for this bucket
await keyStore.deleteItemsByKeyIn(bucketKeys);
logger.info(`Processed ${bucketEventsParsed.length} events from bucket ${bucketId} for ${eventType}`);
return bucketEventsParsed.length;
} catch (error) {
logger.error(error, `Failed to process bucket ${bucketId} for ${eventType}`);
return 0;
}
};
const processAggregatedEvents = async () => {
if (!postHog) return;
for (const eventType of POSTHOG_AGGREGATED_EVENTS) {
let totalProcessed = 0;
logger.info(`Starting bucket processing for ${eventType}`);
// Process each bucket sequentially to control memory usage
for (const bucketId of TELEMETRY_BUCKET_NAMES) {
try {
// eslint-disable-next-line no-await-in-loop
const processed = await processBucketEvents(eventType, bucketId);
totalProcessed += processed;
} catch (error) {
logger.error(error, `Failed to process bucket ${bucketId} for ${eventType}`);
}
}
logger.info(`Completed processing ${totalProcessed} total events for ${eventType}`);
}
};
const flushAll = async () => {
if (postHog) {
await postHog.shutdownAsync();
@ -98,6 +312,8 @@ To opt into telemetry, you can set "TELEMETRY_ENABLED=true" within the environme
return {
sendLoopsEvent,
sendPostHogEvents,
flushAll
processAggregatedEvents,
flushAll,
getBucketForDistinctId
};
};

View File

@ -1,3 +1,13 @@
import {
IdentityActor,
KmipClientActor,
PlatformActor,
ScimClientActor,
ServiceActor,
UnknownUserActor,
UserActor
} from "@app/ee/services/audit-log/audit-log-types";
export enum PostHogEventTypes {
SecretPush = "secrets pushed",
SecretPulled = "secrets pulled",
@ -40,6 +50,14 @@ export type TSecretModifiedEvent = {
secretPath: string;
channel?: string;
userAgent?: string;
actor?:
| UserActor
| IdentityActor
| ServiceActor
| ScimClientActor
| PlatformActor
| UnknownUserActor
| KmipClientActor;
};
};
@ -214,7 +232,7 @@ export type TInvalidateCacheEvent = {
};
};
export type TPostHogEvent = { distinctId: string } & (
export type TPostHogEvent = { distinctId: string; organizationId?: string } & (
| TSecretModifiedEvent
| TAdminInitEvent
| TUserSignedUpEvent

View File

@ -4,7 +4,7 @@ services:
nginx:
container_name: infisical-dev-nginx
image: nginx
restart: always
restart: "always"
ports:
- 8080:80
- 8443:443

View File

@ -0,0 +1,4 @@
---
title: "Available"
openapi: "GET /api/v1/app-connections/gitlab/available"
---

View File

@ -0,0 +1,10 @@
---
title: "Create"
openapi: "POST /api/v1/app-connections/gitlab"
---
<Note>
Gitlab OAuth Connections must be created through the Infisical UI.
Check out the configuration docs for [Gitlab OAuth Connections](/integrations/app-connections/gitlab) for a step-by-step
guide.
</Note>

View File

@ -0,0 +1,4 @@
---
title: "Delete"
openapi: "DELETE /api/v1/app-connections/gitlab/{connectionId}"
---

View File

@ -0,0 +1,4 @@
---
title: "Get by ID"
openapi: "GET /api/v1/app-connections/gitlab/{connectionId}"
---

View File

@ -0,0 +1,4 @@
---
title: "Get by Name"
openapi: "GET /api/v1/app-connections/gitlab/connection-name/{connectionName}"
---

View File

@ -0,0 +1,4 @@
---
title: "List"
openapi: "GET /api/v1/app-connections/gitlab"
---

View File

@ -0,0 +1,10 @@
---
title: "Update"
openapi: "PATCH /api/v1/app-connections/gitlab/{connectionId}"
---
<Note>
Gitlab OAuth Connections must be updated through the Infisical UI.
Check out the configuration docs for [Gitlab OAuth Connections](/integrations/app-connections/gitlab) for a step-by-step
guide.
</Note>

View File

@ -0,0 +1,4 @@
---
title: "Create"
openapi: "POST /api/v1/secret-syncs/gitlab"
---

View File

@ -0,0 +1,4 @@
---
title: "Delete"
openapi: "DELETE /api/v1/secret-syncs/gitlab/{syncId}"
---

View File

@ -0,0 +1,4 @@
---
title: "Get by ID"
openapi: "GET /api/v1/secret-syncs/gitlab/{syncId}"
---

View File

@ -0,0 +1,4 @@
---
title: "Get by Name"
openapi: "GET /api/v1/secret-syncs/gitlab/sync-name/{syncName}"
---

View File

@ -0,0 +1,4 @@
---
title: "List"
openapi: "GET /api/v1/secret-syncs/gitlab"
---

View File

@ -0,0 +1,4 @@
---
title: "Remove Secrets"
openapi: "POST /api/v1/secret-syncs/gitlab/{syncId}/remove-secrets"
---

View File

@ -0,0 +1,4 @@
---
title: "Sync Secrets"
openapi: "POST /api/v1/secret-syncs/gitlab/{syncId}/sync-secrets"
---

View File

@ -0,0 +1,4 @@
---
title: "Update"
openapi: "PATCH /api/v1/secret-syncs/gitlab/{syncId}"
---

View File

@ -0,0 +1,4 @@
---
title: "Attach"
openapi: "POST /api/v1/auth/tls-cert-auth/identities/{identityId}"
---

View File

@ -0,0 +1,4 @@
---
title: "Login"
openapi: "POST /api/v1/auth/tls-cert-auth/login"
---

View File

@ -0,0 +1,4 @@
---
title: "Retrieve"
openapi: "GET /api/v1/auth/tls-cert-auth/identities/{identityId}"
---

View File

@ -0,0 +1,4 @@
---
title: "Revoke"
openapi: "DELETE /api/v1/auth/tls-cert-auth/identities/{identityId}"
---

View File

@ -0,0 +1,4 @@
---
title: "Update"
openapi: "PATCH /api/v1/auth/tls-cert-auth/identities/{identityId}"
---

Some files were not shown because too many files have changed in this diff Show More