feat: Add high availability for multiple replicas (#4555)

* feat: HA tailnet coordinator * fixup! feat: HA tailnet coordinator * fixup! feat: HA tailnet coordinator * remove printlns * close all connections on coordinator * impelement high availability feature * fixup! impelement high availability feature * fixup! impelement high availability feature * fixup! impelement high availability feature * fixup! impelement high availability feature * Add replicas * Add DERP meshing to arbitrary addresses * Move packages to highavailability folder * Move coordinator to high availability package * Add flags for HA * Rename to replicasync * Denest packages for replicas * Add test for multiple replicas * Fix coordination test * Add HA to the helm chart * Rename function pointer * Add warnings for HA * Add the ability to block endpoints * Add flag to disable P2P connections * Wow, I made the tests pass * Add replicas endpoint * Ensure close kills replica * Update sql * Add database latency to high availability * Pipe TLS to DERP mesh * Fix DERP mesh with TLS * Add tests for TLS * Fix replica sync TLS * Fix RootCA for replica meshing * Remove ID from replicasync * Fix getting certificates for meshing * Remove excessive locking * Fix linting * Store mesh key in the database * Fix replica key for tests * Fix types gen * Fix unlocking unlocked * Fix race in tests * Update enterprise/derpmesh/derpmesh.go Co-authored-by: Colin Adler <colin1adler@gmail.com> * Rename to syncReplicas * Reuse http client * Delete old replicas on a CRON * Fix race condition in connection tests * Fix linting * Fix nil type * Move pubsub to in-memory for twenty test * Add comment for configuration tweaking * Fix leak with transport * Fix close leak in derpmesh * Fix race when creating server * Remove handler update * Skip test on Windows * Fix DERP mesh test * Wrap HTTP handler replacement in mutex * Fix error message for relay * Fix API handler for normal tests * Fix speedtest * Fix replica resend * Fix derpmesh send * Ping async * Increase wait time of template version jobd * Fix race when closing replica sync * Add name to client * Log the derpmap being used * Don't connect if DERP is empty * Improve agent coordinator logging * Fix lock in coordinator * Fix relay addr * Fix race when updating durations * Fix client publish race * Run pubsub loop in a queue * Store agent nodes in order * Fix coordinator locking * Check for closed pipe Co-authored-by: Colin Adler <colin1adler@gmail.com>
2025-07-15 22:20:27 +00:00 · 2022-10-17 08:43:30 -05:00
parent dc3519e973
commit 2ba4a62a0d
76 changed files with 3437 additions and 404 deletions
--- a/coderd/database/queries/replicas.sql
+++ b/coderd/database/queries/replicas.sql
@ -0,0 +1,31 @@
+-- name: GetReplicasUpdatedAfter :many
+SELECT * FROM replicas WHERE updated_at > $1 AND stopped_at IS NULL;
+
+-- name: InsertReplica :one
+INSERT INTO replicas (
+    id,
+    created_at,
+    started_at,
+    updated_at,
+    hostname,
+    region_id,
+    relay_address,
+    version,
+    database_latency
+) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9) RETURNING *;
+
+-- name: UpdateReplica :one
+UPDATE replicas SET
+    updated_at = $2,
+    started_at = $3,
+    stopped_at = $4,
+    relay_address = $5,
+    region_id = $6,
+    hostname = $7,
+    version = $8,
+    error = $9,
+    database_latency = $10
+WHERE id = $1 RETURNING *;
+
+-- name: DeleteReplicasUpdatedBefore :exec
+DELETE FROM replicas WHERE updated_at < $1;
--- a/coderd/database/queries/siteconfig.sql
+++ b/coderd/database/queries/siteconfig.sql
@ -3,3 +3,9 @@ INSERT INTO site_configs (key, value) VALUES ('deployment_id', $1);

 -- name: GetDeploymentID :one
 SELECT value FROM site_configs WHERE key = 'deployment_id';
+
+-- name: InsertDERPMeshKey :exec
+INSERT INTO site_configs (key, value) VALUES ('derp_mesh_key', $1);
+
+-- name: GetDERPMeshKey :one
+SELECT value FROM site_configs WHERE key = 'derp_mesh_key';