60 Commits

Author SHA1 Message Date
761d53a418 feat: update dependencies
Rekres, update dependencies.

Mostly to bring in
https://github.com/siderolabs/discovery-client/releases/tag/v0.1.11 and
get it tested, as this repo has client tests.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2025-02-26 15:34:11 +04:00
7c1129e3e7 chore: bump deps
Bump direct deps:
- run rekres
- github.com/fsnotify/fsnotify to v1.8.0
- github.com/grpc-ecosystem/go-grpc-middleware/v2 to v2.2.0
- github.com/siderolabs/gen to v0.8.0
- github.com/stretchr/testify to v1.10.0
- golang.org/x/net to v0.32.0
- golang.org/x/sync to v0.10.0
- golang.org/x/time to v0.8.0
- google.golang.org/grpc to v1.69.0
- google.golang.org/protobuf to v1.36.0

Bump indirect deps:
- github.com/klauspost/compress v1.17.11
- github.com/prometheus/common v0.61.0
- golang.org/x/sys v0.28.0
- golang.org/x/text v0.21.0
- google.golang.org/genproto/googleapis/rpc v0.0.0-20241216192217-9240e9c98484

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
v1.0.9
2024-12-17 18:38:38 +03:00
2bb245aa38 fix: do not register storage metric collectors if it is not enabled
If discovery service storage is not enabled, we were still wrongly registering it as prometheus metrics collector, because the nil check was done against the interface. It had to be done on the concrete type instead.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
v1.0.8
2024-11-28 13:34:40 +01:00
b8da986b5a fix: reduce memory allocations (logger)
Rework the gRPC logger by using hand-rolled simple version, rework
version parsing to remove regexp matching.

The baseline (via benchmark):

```
BenchmarkViaClientSimulator-32    	    2934	    387398 ns/op	  101921 B/op	     832 allocs/op
```

The baseline + removed logging middleware:

```
BenchmarkViaClientSimulator-32    	    3543	    331166 ns/op	   73581 B/op	     543 allocs/op
```

Reworked logging middleware:

```
BenchmarkViaClientSimulator-32    	    3394	    334066 ns/op	   77985 B/op	     568 allocs/op
```

Plus reworked version parsing:

```
BenchmarkViaClientSimulator-32    	    3510	    325714 ns/op	   66215 B/op	     561 allocs/op
```

So overall, baseline to this PR:

* allocs 101921 -> 66215 B/op
* alloc ops 832 -> 561 allocs/op

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
v1.0.7
2024-10-22 13:30:21 +04:00
3367c7b349 chore: add proto-codec/codec
Unify usage of proto codec v2 across out projects.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
v1.0.6
2024-10-02 20:34:45 +03:00
efbb10bdfd fix: properly parse peer address
After switch to Go's http/server, the peer address comes wrapped, so use
a different method to unwrap it.

The tests haven't caught that, as they were using gRPC's server, so
switch tests to use same approach as production, ans enable HTTP/2 over
TLS, as otherwise h2c is a mess, and it doesn't abort connections
properly for test purposes.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
v1.0.5
2024-09-26 21:02:42 +04:00
cf39974104 feat: support direct TLS serving
Support certificate reload on the fly.

Slice version to just `vX.Y` in the metrics.

Bump IP-based limits.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
v1.0.4
2024-09-26 18:05:36 +04:00
270f2575e7 chore: bump deps
Run rekres and bump deps.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
v1.0.3
2024-09-02 16:34:00 +03:00
74bca2da5c feat: export service entrypoint
Move the entrypoint logic into its own package, so it can be imported from other projects.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2024-05-30 11:25:11 +02:00
86e131779a feat: log the state file size on load and save
Log the raw file size in addition to the number of clusters, affiliates etc.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2024-05-29 10:15:59 +02:00
417251c0ba fix: fix the panic in loading state from storage
When importing clusters from a file, grow the used buffer to the correct size of the next cluster, so that it does not panic when unmarshaling it.

Handle panics on the storage's save & load, so that it will never crash the discovery service when it fails.

Additionally:
- fix the slice growing logic when exporting clusters so that we avoid over-growing the slices for the affiliates and the endpoints.
- modify the storage tests to use the real state instead of a mock, replace the assertions to ignore the order accordingly.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
v1.0.2
2024-05-28 18:45:28 +02:00
10c83d2eab release(v1.0.1): prepare release
This is the official v1.0.1 release.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
v1.0.1
2024-05-28 17:33:42 +04:00
196c609d1e fix: use shared gRPC buffers, lower buffer size
As Discovery Service handles lots of connections with relatively low
traffic on each connection, lower the buffer size and re-use the
read/write buffers.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-05-28 14:27:14 +04:00
a2217bd298 chore: migrate from wrapped sync.Pool to HashTrieMap
This should lower memory consumption because HashTrieMap doesn't use any and doesn't have double maps.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-05-28 11:54:47 +03:00
8a7a0d4a43 chore: bump deps
Run rekres and bump gen to 0.5.0. Preparation for the next PR.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-05-28 03:03:31 +03:00
ea8b8f1493 feat: implement state storage
On a best-effort basis, store the state on the disk periodically and on shutdown & restore it from the disk on startup.

Additionally, bump Go version, deps & rekres.

Closes siderolabs/discovery-service#54.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
v1.0.0
2024-05-22 12:00:13 +02:00
c55142668f feat: migrate grpc-middleware to v2, update deps
Update removing multiple old middlewares, rework
the way data is passed through the context, logging fields, etc.

Fix minimum keepalive interval enforcement.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-03-12 18:14:14 +04:00
6af92b54d6 feat: add support for arm64 builds
Resolves https://github.com/siderolabs/discovery-service/issues/52.
`make rekres` also updated some of the image tags.

Signed-off-by: Dennis Marttinen <twelho@welho.tech>
Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-01-22 23:24:36 +05:30
952dc5438f refactor: update dependencies, small cleanups
No real functional changes, some small cleanups, using generic
functions, refactoring some code.

Use the actual address instead of string for the IP rate limiter
(smaller map key).

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
v0.1.3
2024-01-12 18:10:28 +04:00
4cf4b6edc8 fix: update Go to 1.21.3
Rekres, bump dependencies.

Security CVE-2023-44487

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
v0.1.2
2023-10-11 14:54:20 +04:00
c939fef8e5 chore: fix typo on landing page
Fix typo in index.html.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-05-18 19:34:51 +04:00
12d9689089 chore: update dependencies, rekres
No major changes, new vtprotobuf generator with equality methods.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-01-30 20:44:11 +04:00
912943a343 test: add test on client redirect
This matches PR https://github.com/siderolabs/discovery-client/pull/4.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-09-15 17:37:55 +04:00
8db8ef361e feat: add an option to redirect all clients to a fixed endpoint
This allows to launch discovery service with a flag like
`--redirect-endpoint=example.com:443`.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-09-08 21:27:47 +04:00
b34803b6e0 test: add a client test with affiliate deletion
Test affiliate deletion from PR
https://github.com/siderolabs/discovery-client/pull/3.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-08-25 21:35:16 +04:00
69ac844cf5 refactor: replace netaddr with netip/netipx
This removes deprecated inet.af/netaddr.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-08-25 21:11:45 +04:00
d5d0ff5a2f chore: rekres
Bump Go, rename package name to siderolabs, etc.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-08-25 20:29:58 +04:00
9baca2eb53 chore: bump prometheus/client_golang
Bump prometheus/client_golang and other deps

Fixes: [CVE-2022-21698](https://github.com/prometheus/client_golang/security/advisories/GHSA-cg3q-j54f-5p7p)

Signed-off-by: Noel Georgi <git@frezbo.dev>
2022-02-15 20:12:19 +05:30
a20b27d1b9 feat: rate limiting
enforce rate limits and a maximum burst size per IP address

Signed-off-by: Philipp Sauter <sauterp@protonmail.com>
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-02-10 00:09:57 +03:00
aab9f1f442 docs: update kubespan link
fixes #34
Signed-off-by: Rohit Dandamudi <rohit.dandamudi@siderolabs.com>
2022-01-11 14:25:15 +05:30
52b44de1f7 feat: implement inspect page
Added a new handler and respective tests to get affiliate data in a cluster.
Signed-off-by: Rohit Dandamudi <rohit.dandamudi@siderolabs.com>
2021-11-23 19:00:46 +05:30
4906c98af3 docs: update initial README docs
Updated it with more context and steps to get started.
Signed-off-by: Rohit Dandamudi <rohit.dandamudi@siderolabs.com>
2021-11-22 19:58:11 +05:30
d85ea91d43 feat: extract peer address from nginx headers
- Added additional interceptors function to log the address
- Tests to cover the client IP address case
Signed-off-by: Rohit Dandamudi <rohit.dandamudi@siderolabs.com>
2021-11-17 18:31:58 +05:30
a0e6313af7 chore: relicense under BSL-1.1
Change date is 4 years from now, change license is MPL 2.0.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-10-28 23:13:26 +03:00
5960cd240d feat: improve version parsing
Do not store versions like `v0.14.0-alpha.0-7-gf7d9f211-dirty` to avoid
a combinatorial explosion in Prometheus.

Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@talos-systems.com>
2021-10-28 17:38:33 +00:00
d4a4ec2fac chore: move api and client out of the repository
The protobuf definitions were moved out to
https://github.com/talos-systems/discovery-api.

The client was moved out to
https://github.com/talos-systems/discovery-client.

No functional changes.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-10-13 21:41:25 +03:00
95593b8685 feat: implement landing page for the discovery service
Landing page is served on a different port for easier ingress
configuration.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
v0.1.1
2021-10-11 23:16:08 +03:00
b579076e4e fix: update affiliate state correctly when they get deleted
This is client-side only fix, server side had no issues.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-10-11 22:34:28 +03:00
49e53b1473 fix: cluster with some subscriptions isn't empty
This addresses a pretty race case when cluster GC runs while the cluster
just got created without any affiliates, but with a subscription.

Client first watches cluster state, then adds an affiliate, so there
might be a case when GC runs and sees a fresh cluster without any
affiliates and GCes it.

This also fixes test instability.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-10-11 22:28:28 +03:00
9b5eeaed72 chore: add go-debug
Enable pprof, traces, expvar is compiled with `sidero.debug`.

Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@talos-systems.com>
2021-10-06 11:08:17 +00:00
1655040813 chore: improve state logging
Log cluster creation and removal.

Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@talos-systems.com>
2021-10-01 15:41:19 +00:00
ee4b2a4098 fix: retry on Hello failures
Client shouldn't go into Watch mode if Hello failed.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
v0.1.0
2021-09-30 15:59:47 +03:00
ab9c7c98dc chore: add Prometheus metrics
Closes #11.

Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@talos-systems.com>
2021-09-30 15:00:55 +03:00
b2e2079088 fix: properly encrypt IPv6 endpoints
I somehow assumes AES block size is equal to key size (32 bytes), but
that is not the fact. AES block size is always 16 bytes, so for IPv6
endpoints (and longer endpoints in general) we have to encrypt every
block.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-09-29 22:21:03 +03:00
e9d5dfa15e fix: enable connections to endpoints with public certs
gRPC enforces having some transport credentials.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-09-28 20:07:42 +03:00
509e9b2ced feat: implement client wrapper around discovery service API
Fixes #6

This wrapper handles protobuf marshaling, encryption, etc. on the client
data so that service doesn't have a way to see the data.

Client handles data refresh on TTL, updates, discovery of other
affiliates, etc.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-09-27 23:37:24 +03:00
619546696a feat: enable vtprotobuf, watch batching, more limits
Fixes #5

Batch watch responses in a single batch so that client can quickly know
that initial snapshot got delivered.

Bump go.mod deps.

Implement more limits.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-09-23 17:58:45 +03:00
7174ec1042 feat: implement new discovery service
This includes new in-memory core, new gRPC API, tests, etc.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-09-23 15:52:42 +03:00
1a43970826 feat: add node and cluster validation
Co-authored-by: Artem Chernyshev <artem.chernyshev@talos-systems.com>
Signed-off-by: C McCord <ulexus@gmail.com>
Signed-off-by: Artem Chernyshev <artem.chernyshev@talos-systems.com>
2021-08-13 19:37:32 +03:00
6454cfcb72 refactor: kresify, fix linter and rename to Kubespan manager
Tried to limit the scope of changes.

Signed-off-by: Artem Chernyshev <artem.chernyshev@talos-systems.com>
2021-08-13 16:28:29 +03:00