918 Commits

Author SHA1 Message Date
5d540e119f Move all intrinsic tag lookup to the query-frontend and prioritize them in the results (#4784)
* Move all intrinsic tag lookup to the query-frontend and prioritize them in the results

* lint, error handling, fix tests

* Fix some tests

* Revert unintended change to search/tags v1 behavior, update tests

* Reduce diff

* Revert unintended change to 'none' scope

* reduce diff

* changelog

* Update test to test intrinsic handling at the limit

* todos
2025-03-04 15:12:03 -05:00
19556c7e72 fix: flaky TestWalBlockFindTraceByID test (#4787)
Some checks failed
docker / get-tag (push) Has been cancelled
docker / docker (tempo, map[arch:amd64 runner:ubuntu-24.04]) (push) Has been cancelled
docker / docker (tempo, map[arch:arm64 runner:github-hosted-ubuntu-arm64]) (push) Has been cancelled
docker / docker (tempo-cli, map[arch:amd64 runner:ubuntu-24.04]) (push) Has been cancelled
docker / docker (tempo-cli, map[arch:arm64 runner:github-hosted-ubuntu-arm64]) (push) Has been cancelled
docker / docker (tempo-query, map[arch:amd64 runner:ubuntu-24.04]) (push) Has been cancelled
docker / docker (tempo-query, map[arch:arm64 runner:github-hosted-ubuntu-arm64]) (push) Has been cancelled
docker / docker (tempo-vulture, map[arch:amd64 runner:ubuntu-24.04]) (push) Has been cancelled
docker / docker (tempo-vulture, map[arch:arm64 runner:github-hosted-ubuntu-arm64]) (push) Has been cancelled
docker / manifest (tempo) (push) Has been cancelled
docker / manifest (tempo-cli) (push) Has been cancelled
docker / manifest (tempo-query) (push) Has been cancelled
docker / manifest (tempo-vulture) (push) Has been cancelled
docker / cd-to-dev-env (push) Has been cancelled
Snyk Monitor Scanning / snyk-scan-ci (push) Has been cancelled
Github repo and issue stats collection / main (push) Has been cancelled
* fix: flaky TestWalBlockFindTraceByID test

* changelog

* convert to uint64
2025-03-04 11:40:54 +01:00
074ab75e8b Fix query range frontend caching (#4771)
Some checks are pending
docker / get-tag (push) Waiting to run
docker / docker (tempo, map[arch:amd64 runner:ubuntu-24.04]) (push) Blocked by required conditions
docker / docker (tempo, map[arch:arm64 runner:github-hosted-ubuntu-arm64]) (push) Blocked by required conditions
docker / docker (tempo-cli, map[arch:amd64 runner:ubuntu-24.04]) (push) Blocked by required conditions
docker / docker (tempo-cli, map[arch:arm64 runner:github-hosted-ubuntu-arm64]) (push) Blocked by required conditions
docker / docker (tempo-query, map[arch:amd64 runner:ubuntu-24.04]) (push) Blocked by required conditions
docker / docker (tempo-query, map[arch:arm64 runner:github-hosted-ubuntu-arm64]) (push) Blocked by required conditions
docker / docker (tempo-vulture, map[arch:amd64 runner:ubuntu-24.04]) (push) Blocked by required conditions
docker / docker (tempo-vulture, map[arch:arm64 runner:github-hosted-ubuntu-arm64]) (push) Blocked by required conditions
docker / manifest (tempo) (push) Blocked by required conditions
docker / manifest (tempo-cli) (push) Blocked by required conditions
docker / manifest (tempo-query) (push) Blocked by required conditions
docker / manifest (tempo-vulture) (push) Blocked by required conditions
docker / cd-to-dev-env (push) Blocked by required conditions
Snyk Monitor Scanning / snyk-scan-ci (push) Waiting to run
* fix cache

Signed-off-by: Joe Elliott <number101010@gmail.com>

* changelog

Signed-off-by: Joe Elliott <number101010@gmail.com>

* remove the thing that lint doesn't like

Signed-off-by: Joe Elliott <number101010@gmail.com>

* fix changelog

Signed-off-by: Joe Elliott <number101010@gmail.com>

---------

Signed-off-by: Joe Elliott <number101010@gmail.com>
2025-02-28 14:37:17 -05:00
5d98dcd245 Convert throughput SLO metrics from histogram to counter (#4748)
For performance
2025-02-28 14:11:10 +01:00
c1f6280dd1 Do less Nexting (#4753)
* this is garbage

Signed-off-by: Joe Elliott <number101010@gmail.com>

* filtery stuff

Signed-off-by: Joe Elliott <number101010@gmail.com>

* fix

Signed-off-by: Joe Elliott <number101010@gmail.com>

* max def everywhere

Signed-off-by: Joe Elliott <number101010@gmail.com>

* clean up benches

Signed-off-by: Joe Elliott <number101010@gmail.com>

* clean up

Signed-off-by: Joe Elliott <number101010@gmail.com>

* remove vendor chagnes

Signed-off-by: Joe Elliott <number101010@gmail.com>

* changelog

Signed-off-by: Joe Elliott <number101010@gmail.com>

* add details about bench env vars

Signed-off-by: Joe Elliott <number101010@gmail.com>

---------

Signed-off-by: Joe Elliott <number101010@gmail.com>
2025-02-28 08:08:18 -05:00
9064e84aa5 [Metrics Generator] Allow running on a different source of data (#4686)
* [Metrics Generator] Allow running on a different source of data

* fix lint errors

* Address review feedback

* regenerate manifest

* do not start ring lifecycler

* make partition ring watcher a separate module

* do not join ring when gRPC is disabled

* changelog
2025-02-27 07:40:14 -05:00
2c3c357ffd Improve Tempo build options (#4755)
* Set CPU architecture API version for amd64 and arm64

This enables the compiler to use instructions from more recent API versions

* Strip debug symbols from production binary

Reduces Tempo binary size by ~20%

* Remove GO111MODULE=on environment variable

GO111MODULE defaults to 'on' since Go 1.16

* CHANGELOG.md
2025-02-27 19:57:01 +10:00
1b7a088e08 Reorder span iterators (#4754)
* Reorder iterators

* CHANGELOG.md
2025-02-27 08:50:42 +10:00
04aedaad5d Log inspected bytes for trace by id results (#4758)
We use this in dashboards

And update docs
2025-02-26 07:55:02 -05:00
8263d353e7 [TraceQL] Add ability to search for parent span id (#4692)
* [TraceQL] Add ability to search for parent span id

* docs and changelog

* update test

* add test and fix test
2025-02-25 15:28:56 -06:00
0a3fd0495a rhythm: fix block time range adjustment (#4746)
* rhythm: fix block time range adjustment

* fix pr number

* fix test
2025-02-25 13:13:36 +01:00
7cb63d54b6 rhythm: fix partition sorting (#4747) 2025-02-25 10:52:25 +01:00
5e7fe45e4e Issue 4742: fix panic on startup (#4744)
* Issue 4742: fix panic on startup

* Issue 4742: cover non-empty WAL in unit-tests
2025-02-24 18:01:01 +00:00
c2300f0f45 Issue 3948: user cfg override fix (#4741) 2025-02-24 17:39:27 +01:00
d8bf8fed20 [Rhythm] Improve metrics generator + Kafka performance and stability (#4721)
* Add new concurrency config options and validate

* Reduce allocations of IDmap when replaying wal blocks since we know the number of traces

* Generator read from kafka concurrency, add shared ingest lag metric, option to uniqify strings

* Memoize spanmetrics sanitizelabelname, move to better location

* Moved to shared queue for localblocks wal completion, allow concurrency within a tenant

* Honor max live traces in non-flushing local blocks processor

* Add metric for enqueue time

* Add missing mutex lock for enqueuing on replay

* Fix mutex lock while reloading blocks

* Increase default concurrency

* Simplify local blocks complete queue

* Remove uniqify, fix test

* Switch completequeue to reference counting and shut it down when last local blocks processor is stopped

* cache rename/cleanup

* Cleanup/denoising PR

* Lint/cleanup

* lint

* Update config manifest

* fix race condition

* cleanup

* review feedback

* Update config manifest

* changelog
2025-02-21 11:18:42 -05:00
50a9daf668 rhythm: fair partition consumption (#4655)
* rhythm: weighted partition consumption

* remove printf

* make it simpler

* better logging

* fix typo

* update the loop to priorize the laggiest partition based in time not in offset

* remove log line

* address comments

* fix bug and remove test statements

* changelog

* remove unused variable
2025-02-21 09:27:02 +01:00
40c4351a65 Rewrite traces using rebatching (#4690)
* Implement span basic rebatching and 4to4 converter

Signed-off-by: Joe Elliott <number101010@gmail.com>

* Generated trace data with unique scope and resource spans

The functions test.MakeBatch and test.MakeTrace* now create
more unique ResourceSpans and ScopeSpans. This avoids unwanted
rebatching in tests

* Fix tests in spanmetrics processor

Test data now has additional attributes on resource level
that caused these tests to fail

* Move rebatch code out of nested set function

* Make parquetTraceToTempopbTrace() private

* Add benchmark for testing traceToParquet()

* make check-fmt

* rebatchTrace() without allocating new slices for ResourceSpans and ScopeSpans

* Fix bug with duplicated rs ans ss in rebatchTrace()

* CHANGELOG.md

* Add finalizeTrace() function

* Call finalizeTrace() in combiner

---------

Signed-off-by: Joe Elliott <number101010@gmail.com>
Co-authored-by: Joe Elliott <number101010@gmail.com>
2025-02-21 09:31:03 +10:00
115795f2ad Update go to v1.24.0 (#4704)
* Update go: v1.23.3 -> v1.24.0

* Update changelog

* Minor fixes for go update

* Update linter image: go1.24 support
2025-02-17 15:18:28 +01:00
91b4a22e36 Add ability to add artificial delay to push requests (#4716) 2025-02-17 14:57:33 +01:00
068b85ca76 Improve EqualRowNumber (#4705)
* faster equal

Signed-off-by: Joe Elliott <number101010@gmail.com>

* changelog

Signed-off-by: Joe Elliott <number101010@gmail.com>

---------

Signed-off-by: Joe Elliott <number101010@gmail.com>
2025-02-15 14:47:06 +05:30
96f4128489 add sse config to tempo (#3914)
* add sse config and build functions

Signed-off-by: AvivGuiser <aviv.guiser@placer.ai>

* add sse to the config and s3 client

Signed-off-by: AvivGuiser <aviv.guiser@placer.ai>

* changelog

Signed-off-by: AvivGuiser <aviv.guiser@placer.ai>

* fix lint

Signed-off-by: AvivGuiser <aviv.guiser@placer.ai>

* remove unsued vars

Signed-off-by: AvivGuiser <aviv.guiser@placer.ai>

* fmt

Signed-off-by: AvivGuiser <aviv.guiser@placer.ai>

* added check for missing kms key id

Signed-off-by: AvivGuiser <aviv.guiser@placer.ai>

* create sse config when creating a minio client

Signed-off-by: AvivGuiser <aviv.guiser@placer.ai>

* add changelog and docs

Signed-off-by: AvivGuiser <aviv.guiser@placer.ai>

* ran go run pkg/docsgen/generate_manifest.go

Signed-off-by: AvivGuiser <aviv.guiser@placer.ai>

---------

Signed-off-by: AvivGuiser <aviv.guiser@placer.ai>
2025-02-14 13:54:06 +01:00
c8d9180f25 Set gRPC compression default to snappy in all cases (#4696)
* Re add snappy compression in gRPC

Signed-off-by: Joe Elliott <number101010@gmail.com>

* manifest

Signed-off-by: Joe Elliott <number101010@gmail.com>

* changelog

Signed-off-by: Joe Elliott <number101010@gmail.com>

* manifest

Signed-off-by: Joe Elliott <number101010@gmail.com>

* changelog

Signed-off-by: Joe Elliott <number101010@gmail.com>

---------

Signed-off-by: Joe Elliott <number101010@gmail.com>
2025-02-14 07:47:39 -05:00
135498657b Issue 4346: Fix docker compose example (#4695)
* Issue 4346: Fix memcache settings for example

Docker DNS does not create SRV record, use A record instead

* Issue 4346: Fix documentations

"memcached" is a wrong value for addresses field

* Issue 4346: Update changelog
2025-02-14 09:35:40 +01:00
6c07024064 Add TraceQL query hint to retrieve most recent results ordered by trace start time (#4238)
* Added ordered results

Signed-off-by: Joe Elliott <number101010@gmail.com>

* add most_recent query hint

Signed-off-by: Joe Elliott <number101010@gmail.com>

* changelog, docs and lint

Signed-off-by: Joe Elliott <number101010@gmail.com>

* e2e tests - fixed tag search

Signed-off-by: Joe Elliott <number101010@gmail.com>

* lint

Signed-off-by: Joe Elliott <number101010@gmail.com>

* remove clone changes

Signed-off-by: Joe Elliott <number101010@gmail.com>

* review

Signed-off-by: Joe Elliott <number101010@gmail.com>

* make shards configurable

Signed-off-by: Joe Elliott <number101010@gmail.com>

* dont mess with me lint. i will uninstall you

Signed-off-by: Joe Elliott <number101010@gmail.com>

* Make all endpoints search backwards in time

Signed-off-by: Joe Elliott <number101010@gmail.com>

* nice work on this one carles

Signed-off-by: Joe Elliott <number101010@gmail.com>

* consolidate block meta functions

Signed-off-by: Joe Elliott <number101010@gmail.com>

* fix merge :P

Signed-off-by: Joe Elliott <number101010@gmail.com>

* remove tests

Signed-off-by: Joe Elliott <number101010@gmail.com>

* don't bother tracking normal searches

Signed-off-by: Joe Elliott <number101010@gmail.com>

---------

Signed-off-by: Joe Elliott <number101010@gmail.com>
2025-02-12 16:33:10 -05:00
4c999b7178 enhancement: update dskit to latest version (#4681)
* tempo: update dskit

* manifest
2025-02-11 13:13:31 +01:00
011b8088e2 Return the operand as the only value if the tag is already filtered in the query (#4673)
* Return the operand as the only value if the tag is already filtered in the query

* chlog

* Check operands not empty
2025-02-07 16:59:20 +00:00
d7b16550b4 rythm: add block builder to resources dashboard (#4669)
* rythm: add block builder to resources dashboard

* changelog

* fmt jsonnet
2025-02-07 15:03:12 +01:00
914b76bf51 [Rhythm] Remove redudant live traces sort (#4671)
* Remove accidental trace sort left in newLiveTracesIter

* Add benchmark for live traces

* changelog
2025-02-06 13:46:41 -05:00
8e740797b5 [Query-frontend] Fix metrics streaming (#4624)
* remove existing diff functionality

Signed-off-by: Joe Elliott <number101010@gmail.com>

* maybe this works

Signed-off-by: Joe Elliott <number101010@gmail.com>

* Add benches, pre-allocs and improved tests

Signed-off-by: Joe Elliott <number101010@gmail.com>

* changelog

Signed-off-by: Joe Elliott <number101010@gmail.com>

* lint

Signed-off-by: Joe Elliott <number101010@gmail.com>

* actually fix lint

Signed-off-by: Joe Elliott <number101010@gmail.com>

* better benches

Signed-off-by: Joe Elliott <number101010@gmail.com>

* Add exemplars

Signed-off-by: Joe Elliott <number101010@gmail.com>

---------

Signed-off-by: Joe Elliott <number101010@gmail.com>
2025-01-29 14:24:01 -05:00
d544ec1822 switch max attribute bytes to per tenant and also check event links a… (#4633)
* switch max attribute bytes to per tenant and also check event links and instrumentation scopes

* changelog and lint

* remove test log lines

* add breaking change label

* lint

* add test, make func part of struct
2025-01-29 11:06:25 -06:00
40ce0afc52 [rhythm] fix: starting consuming cycle log (#4629)
* fix: starting consuming cycle log

* changelog
2025-01-28 15:38:33 +01:00
fc89a14ea6 [TraceQL] Fix to put all conditions following a select clause into the second pass (#4600)
* Fix to put all conditions following a select clause into the second pass

* more tests

* changelog
2025-01-24 09:02:09 -05:00
51aca06f9f Remove tempo serverless (#4599)
* remove tempo-serverless from cmd

* cleanup Makefile and .gitignore

* remove serverless code from pkg/api/

* remove serverless code from the querier

* clean up serverless from tempo-mixin and playbook

* Remove serverless from tempo docs

* Update changelog

* go mod vendor

* Remove tempo_feature_enabled metric

* inline internalSearchBlock

* docs alias
2025-01-24 05:02:29 +00:00
b980aa7b70 [Rhythm] Block builder performance improvement (#4596)
* block builder pool prealloc byte slices

* Convert livetraces to store bytes to reduce GC scanning, Parallelize trace unmarshal, repool prealloc slices

* changelog
2025-01-23 11:43:22 -05:00
e680d6ee8d [Rhythm] Move group partition lag metric to ingest package, export from generators too (#4571)
* Move group partition lag metric to ingest package, export from generators too

* Cleanup

* changelog

* Remove unnecessary go

Co-authored-by: Mario <mariorvinas@gmail.com>

---------

Co-authored-by: Mario <mariorvinas@gmail.com>
2025-01-22 09:13:49 +01:00
14efba0afa Use distroless base image for tempo (#4556)
This image contains busybox, making debugging easier by running /busybox/sh

In addition, the latest ca-certificates from alpin are copied, as the ones in Debian are
severely out of date.
2025-01-21 20:11:07 +05:30
9acc16dd97 [Frontend] Two fixes for gRPC query range streaming (#4576)
* Revert "Bugfix: Default step for gRPC streaming query range queries (#4546)"

This reverts commit 424274aab59a05c2865420c005b1173e1779e990.

* default step if not set

Signed-off-by: Joe Elliott <number101010@gmail.com>

* fix dumping of output to json

Signed-off-by: Joe Elliott <number101010@gmail.com>

* attachExemplars in gRPC diff

Signed-off-by: Joe Elliott <number101010@gmail.com>

* added test and bench

Signed-off-by: Joe Elliott <number101010@gmail.com>

* remove step

Signed-off-by: Joe Elliott <number101010@gmail.com>

* changelog

Signed-off-by: Joe Elliott <number101010@gmail.com>

* fix default step

Signed-off-by: Joe Elliott <number101010@gmail.com>

* lint

Signed-off-by: Joe Elliott <number101010@gmail.com>

* corrected proto import

Signed-off-by: Joe Elliott <number101010@gmail.com>

---------

Signed-off-by: Joe Elliott <number101010@gmail.com>
2025-01-17 17:54:28 -05:00
1f8d337a85 [Ingester] Create one goroutine per tenant to flush traces to disk (#4483)
* fix jsonnet example

Signed-off-by: Joe Elliott <number101010@gmail.com>

* make one goroutine per instance for wal flush

Signed-off-by: Joe Elliott <number101010@gmail.com>

* restore subservice watcher in case it did something

Signed-off-by: Joe Elliott <number101010@gmail.com>

* harden shutdown

Signed-off-by: Joe Elliott <number101010@gmail.com>

* remove flush event

Signed-off-by: Joe Elliott <number101010@gmail.com>

* undo accidental change

Signed-off-by: Joe Elliott <number101010@gmail.com>

* changelog

Signed-off-by: Joe Elliott <number101010@gmail.com>

* remove debug log

Signed-off-by: Joe Elliott <number101010@gmail.com>

* Remove broken tests

Signed-off-by: Joe Elliott <number101010@gmail.com>

* Revert "fix jsonnet example"

This reverts commit 5e586751c8b894c45f0e12613e80487f0754adeb.

* update func name for clarity

Signed-off-by: Joe Elliott <number101010@gmail.com>

---------

Signed-off-by: Joe Elliott <number101010@gmail.com>
2025-01-17 09:13:52 -05:00
bf361e1e5e Fix TraceQL results caching bug for floats ending in .0 (#4539) 2025-01-16 11:26:43 -05:00
9f224e534e feat: update minio to version 7.0.83 (#4568)
* feat: update minio to version 7.0.83

* mod tidy for serverless

* changelog
2025-01-16 16:45:25 +01:00
0eae105771 [Rhythm] Add concurrency to block-builder wal conversion and flushing (#4565)
* Add concurrency to block-builder wal conversion and flushing

* changelog

* lint
2025-01-16 14:59:05 +00:00
094a9fddb8 Update tempo operational dashboard for block builder and v2 traces api (#4559)
* Update tempo operational dashboard for block builder and v2 traces api

* changelog
2025-01-15 17:49:05 +00:00
8d9ab439e4 Tempo: remove internal error reason for discarded spans (#4554)
* remove internal error reason

Signed-off-by: Joe Elliott <number101010@gmail.com>

* changelog

Signed-off-by: Joe Elliott <number101010@gmail.com>

---------

Signed-off-by: Joe Elliott <number101010@gmail.com>
2025-01-15 08:03:17 -05:00
fad4ff786d Update blockbuilder to periodically flush wals and sort traces (#4550)
* Update blockbuilder to periodically flush wals and sort traces, to address performance bottleneck in block completion time. Also add benchmark, and disk cleanup fixes

* Add more metrics

* lint

* cleanup

* changelog

* Honor tenant max_trace_bytes setting
2025-01-15 07:14:20 -05:00
424274aab5 Bugfix: Default step for gRPC streaming query range queries (#4546)
* foo

Signed-off-by: Joe Elliott <number101010@gmail.com>

* fix test

Signed-off-by: Joe Elliott <number101010@gmail.com>

* fix an issue where exemplars were not being dumped correctly

Signed-off-by: Joe Elliott <number101010@gmail.com>

* changelog

Signed-off-by: Joe Elliott <number101010@gmail.com>

* lint

Signed-off-by: Joe Elliott <number101010@gmail.com>

---------

Signed-off-by: Joe Elliott <number101010@gmail.com>
2025-01-13 17:15:02 -05:00
00129a29df changelog 2 (#4545)
Signed-off-by: Joe Elliott <number101010@gmail.com>
2025-01-13 12:00:43 -05:00
f8728d14fd Changelog cleanup 2.7.0 (#4542)
* changelog

Signed-off-by: Joe Elliott <number101010@gmail.com>

* promote traceql metrics functions to features

Signed-off-by: Joe Elliott <number101010@gmail.com>

---------

Signed-off-by: Joe Elliott <number101010@gmail.com>
2025-01-13 10:18:24 -05:00
e709f8ac70 [rhythm] Introduce block-builder and kafka ingest path (#4533)
* Block-builder PoC

* Add unit test for block-builder (#4289)

* Add unit test for block-builder

* fmt

* Update tests

* cmon

* Deterministically build blocks for partition sections (#4327)

* Pull main (#4342)

* chore: remove gofakeit dependency (#4274)

* Further reduce Labes() calls in the metrics registry (#4283)

* Respect passed headers in read path requests (#4287)

* Ingester: Validate completed blocks (#4256)

* Add validate method to block

Signed-off-by: Joe Elliott <number101010@gmail.com>

* Add Validate usage in the ingester

Signed-off-by: Joe Elliott <number101010@gmail.com>

* changelog

Signed-off-by: Joe Elliott <number101010@gmail.com>

* add test and fix replay

Signed-off-by: Joe Elliott <number101010@gmail.com>

* increment metric

Signed-off-by: Joe Elliott <number101010@gmail.com>

---------

Signed-off-by: Joe Elliott <number101010@gmail.com>

* Add `invalid_utf8` to reasons spans could be rejected (#4293)

* Add `invalid_utf8` to reasons spans could be rejected

* Update changelog

* Update docs

* Ensure test covers invalid UTF-8 and not slack time

* add signals for duplicate rf1 data (#4296)

Signed-off-by: Joe Elliott <number101010@gmail.com>

* Bump anchore/sbom-action from 0.17.5 to 0.17.7 (#4307)

Bumps [anchore/sbom-action](https://github.com/anchore/sbom-action) from 0.17.5 to 0.17.7.
- [Release notes](https://github.com/anchore/sbom-action/releases)
- [Changelog](https://github.com/anchore/sbom-action/blob/main/RELEASE.md)
- [Commits](https://github.com/anchore/sbom-action/compare/v0.17.5...v0.17.7)

---
updated-dependencies:
- dependency-name: anchore/sbom-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* docs: Update readme with explore traces info (#4263)

* docs: Update readme with explore traces info

Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com>

* chore: remove spanlogger (#4312)

* chore: remove spanlogger

* Query-Frontend: Add middleware to drop headers (#4298)

* header strip ware

Signed-off-by: Joe Elliott <number101010@gmail.com>

* comment

Signed-off-by: Joe Elliott <number101010@gmail.com>

* changelog

Signed-off-by: Joe Elliott <number101010@gmail.com>

* remove header strip wear from metrics summary

Signed-off-by: Joe Elliott <number101010@gmail.com>

---------

Signed-off-by: Joe Elliott <number101010@gmail.com>

* Increase length of time compactions have to fail (#4315)

* increase length of time compactions have to fail

Signed-off-by: Joe Elliott <number101010@gmail.com>

* gen

Signed-off-by: Joe Elliott <number101010@gmail.com>

---------

Signed-off-by: Joe Elliott <number101010@gmail.com>

* docs: mark serverless as deprecated (#4017)

* docs: mark serverless as deprecated

* Changelog + readme

* docs: Remove duplicated examples (#4295)

This removes duplicates examples from the Configure TraceQL
metrics page.

Signed-off-by: Alex Bikfalvi <alex.bikfalvi@grafana.com>

* tempo-cli: support dropping multiple traces in a single operation (#4266)

* tempo-cli: support dropping multiple traces in a single operation

* update final log message

---------

Co-authored-by: Suraj Nath <9503187+electron0zero@users.noreply.github.com>

* [DOC] Add clarification for metrics summary and traceQL metrics (#4316)

* Add clarification for metrics summary and traceQL metrics

* Apply suggestions from code review

Co-authored-by: Jennifer Villa <jvilla2013@gmail.com>

* Update docs/sources/tempo/api_docs/metrics-summary.md

---------

Co-authored-by: Jennifer Villa <jvilla2013@gmail.com>

* TraceQL metrics time range fixes (#4325)

* Disconnect job time range filtering from step, so that results in split backend/recent range is accurate

* changelog

* Fix to assert metrics query range before alignment because alignment may increase it, which is not the responsibility of the caller to account for (#4331)

* Add doc about configuring TLS with Helm (#4328)

* Add doc about configuring TLS with Helm

* Add memberlist and readinessProbe to example

* Include server config for listening on TLS

* Add note about scraping

* Update docs/sources/tempo/configuration/network/tls.md

Co-authored-by: Markus Toivonen <markus.toivonen@hoxhunt.com>

* Update docs/sources/tempo/configuration/network/tls.md

Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com>

* Update docs/sources/tempo/configuration/network/tls.md

Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com>

* Add memcached config for TLS

---------

Co-authored-by: Markus Toivonen <markus.toivonen@hoxhunt.com>
Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com>

* [DOC] Add TLS info to Helm chart doc (#4334)

---------

Signed-off-by: Joe Elliott <number101010@gmail.com>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Alex Bikfalvi <alex.bikfalvi@grafana.com>
Co-authored-by: Javier Molina Reyes <javiermolinar@live.com>
Co-authored-by: Zach Leslie <zach.leslie@grafana.com>
Co-authored-by: Joe Elliott <number101010@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Ryan Perry <Rperry2174@gmail.com>
Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com>
Co-authored-by: Suraj Nath <9503187+electron0zero@users.noreply.github.com>
Co-authored-by: Alex Bikfalvi <alex@bikfalvi.com>
Co-authored-by: Andrey Karpov <ndk@users.noreply.github.com>
Co-authored-by: Jennifer Villa <jvilla2013@gmail.com>
Co-authored-by: Martin Disibio <martin.disibio@grafana.com>
Co-authored-by: Markus Toivonen <markus.toivonen@hoxhunt.com>

* WIP: Rhythm ingest path (#4314)

* Validate distributor config. Finish encoder/decoder tests

* Repair tests

* Make SingleBinary work out of the box by defaulting to partition 0

* Fix first time startup where blockbuilder fails before ingester can create topic

* Fix initial startup cycle time and delay

* Add more failure modes to the block-builder (#4345)

* Add more tests to the block-builder

* stuff

* Add comments

* [Rhythm] Metrics generator read from kafka first pass (#4359)

* Metrics generator read from kafka first pass

* review feedback

* Multiple fixes in block-builder (#4364)

* [rhythm] git merge origin/main (#4376)

* chore: remove gofakeit dependency (#4274)

* Further reduce Labes() calls in the metrics registry (#4283)

* Respect passed headers in read path requests (#4287)

* Ingester: Validate completed blocks (#4256)

* Add validate method to block

Signed-off-by: Joe Elliott <number101010@gmail.com>

* Add Validate usage in the ingester

Signed-off-by: Joe Elliott <number101010@gmail.com>

* changelog

Signed-off-by: Joe Elliott <number101010@gmail.com>

* add test and fix replay

Signed-off-by: Joe Elliott <number101010@gmail.com>

* increment metric

Signed-off-by: Joe Elliott <number101010@gmail.com>

---------

Signed-off-by: Joe Elliott <number101010@gmail.com>

* Add `invalid_utf8` to reasons spans could be rejected (#4293)

* Add `invalid_utf8` to reasons spans could be rejected

* Update changelog

* Update docs

* Ensure test covers invalid UTF-8 and not slack time

* add signals for duplicate rf1 data (#4296)

Signed-off-by: Joe Elliott <number101010@gmail.com>

* Bump anchore/sbom-action from 0.17.5 to 0.17.7 (#4307)

Bumps [anchore/sbom-action](https://github.com/anchore/sbom-action) from 0.17.5 to 0.17.7.
- [Release notes](https://github.com/anchore/sbom-action/releases)
- [Changelog](https://github.com/anchore/sbom-action/blob/main/RELEASE.md)
- [Commits](https://github.com/anchore/sbom-action/compare/v0.17.5...v0.17.7)

---
updated-dependencies:
- dependency-name: anchore/sbom-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* docs: Update readme with explore traces info (#4263)

* docs: Update readme with explore traces info

Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com>

* chore: remove spanlogger (#4312)

* chore: remove spanlogger

* Query-Frontend: Add middleware to drop headers (#4298)

* header strip ware

Signed-off-by: Joe Elliott <number101010@gmail.com>

* comment

Signed-off-by: Joe Elliott <number101010@gmail.com>

* changelog

Signed-off-by: Joe Elliott <number101010@gmail.com>

* remove header strip wear from metrics summary

Signed-off-by: Joe Elliott <number101010@gmail.com>

---------

Signed-off-by: Joe Elliott <number101010@gmail.com>

* Increase length of time compactions have to fail (#4315)

* increase length of time compactions have to fail

Signed-off-by: Joe Elliott <number101010@gmail.com>

* gen

Signed-off-by: Joe Elliott <number101010@gmail.com>

---------

Signed-off-by: Joe Elliott <number101010@gmail.com>

* docs: mark serverless as deprecated (#4017)

* docs: mark serverless as deprecated

* Changelog + readme

* docs: Remove duplicated examples (#4295)

This removes duplicates examples from the Configure TraceQL
metrics page.

Signed-off-by: Alex Bikfalvi <alex.bikfalvi@grafana.com>

* tempo-cli: support dropping multiple traces in a single operation (#4266)

* tempo-cli: support dropping multiple traces in a single operation

* update final log message

---------

Co-authored-by: Suraj Nath <9503187+electron0zero@users.noreply.github.com>

* [DOC] Add clarification for metrics summary and traceQL metrics (#4316)

* Add clarification for metrics summary and traceQL metrics

* Apply suggestions from code review

Co-authored-by: Jennifer Villa <jvilla2013@gmail.com>

* Update docs/sources/tempo/api_docs/metrics-summary.md

---------

Co-authored-by: Jennifer Villa <jvilla2013@gmail.com>

* TraceQL metrics time range fixes (#4325)

* Disconnect job time range filtering from step, so that results in split backend/recent range is accurate

* changelog

* Fix to assert metrics query range before alignment because alignment may increase it, which is not the responsibility of the caller to account for (#4331)

* Add doc about configuring TLS with Helm (#4328)

* Add doc about configuring TLS with Helm

* Add memberlist and readinessProbe to example

* Include server config for listening on TLS

* Add note about scraping

* Update docs/sources/tempo/configuration/network/tls.md

Co-authored-by: Markus Toivonen <markus.toivonen@hoxhunt.com>

* Update docs/sources/tempo/configuration/network/tls.md

Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com>

* Update docs/sources/tempo/configuration/network/tls.md

Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com>

* Add memcached config for TLS

---------

Co-authored-by: Markus Toivonen <markus.toivonen@hoxhunt.com>
Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com>

* [DOC] Add TLS info to Helm chart doc (#4334)

* fix deprecation warning by switching to DoBatchWithOptions (#4343)

Signed-off-by: Daniel Strobusch <1847260+dastrobu@users.noreply.github.com>

* bump dskit to v0.0.0-20241115082728-f2a7eb3aa0e9 to leverage benefits for context causes for DoBatch calls. (#4341)

See https://github.com/grafana/dskit/issues/576

Signed-off-by: Daniel Strobusch <1847260+dastrobu@users.noreply.github.com>

* Bump github.com/minio/minio-go/v7 from 7.0.70 to 7.0.80 (#4282)

* Bump github.com/minio/minio-go/v7 from 7.0.70 to 7.0.80

Bumps [github.com/minio/minio-go/v7](https://github.com/minio/minio-go) from 7.0.70 to 7.0.80.
- [Release notes](https://github.com/minio/minio-go/releases)
- [Commits](https://github.com/minio/minio-go/compare/v7.0.70...v7.0.80)

---
updated-dependencies:
- dependency-name: github.com/minio/minio-go/v7
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update serverless vendor

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Zach Leslie <zach.leslie@grafana.com>

* update default config values to better align with production workloads (#4340)

* update default config values to better align with production workloads

* Update CHANGELOG.md and config docs

* Ingester memory improvements by adjusting prealloc (#4344)

* remove trace ids

Signed-off-by: Joe Elliott <number101010@gmail.com>

* linear buckets

Signed-off-by: Joe Elliott <number101010@gmail.com>

* changelog

Signed-off-by: Joe Elliott <number101010@gmail.com>

* tuney tune

Signed-off-by: Joe Elliott <number101010@gmail.com>

* metric misses and increase pool size

Signed-off-by: Joe Elliott <number101010@gmail.com>

* lint

Signed-off-by: Joe Elliott <number101010@gmail.com>

---------

Signed-off-by: Joe Elliott <number101010@gmail.com>

* Bump github.com/Azure/azure-sdk-for-go/sdk/azcore from 1.13.0 to 1.16.0 (#4302)

* Bump github.com/Azure/azure-sdk-for-go/sdk/azcore from 1.13.0 to 1.16.0

Bumps [github.com/Azure/azure-sdk-for-go/sdk/azcore](https://github.com/Azure/azure-sdk-for-go) from 1.13.0 to 1.16.0.
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases)
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md)
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/azcore/v1.13.0...sdk/azcore/v1.16.0)

---
updated-dependencies:
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/azcore
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update serverless vendor

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Zach Leslie <zach.leslie@grafana.com>

* Use Prometheus fast regexp (#4329)

* basic integration

Signed-off-by: Joe Elliott <number101010@gmail.com>

* patch tests for new meaning

Signed-off-by: Joe Elliott <number101010@gmail.com>

* patch up more tests

Signed-off-by: Joe Elliott <number101010@gmail.com>

* add basic tests

Signed-off-by: Joe Elliott <number101010@gmail.com>

* changelog + docs

Signed-off-by: Joe Elliott <number101010@gmail.com>

* remove benches

Signed-off-by: Joe Elliott <number101010@gmail.com>

* Cleaned up + tests

Signed-off-by: Joe Elliott <number101010@gmail.com>

* comment

Signed-off-by: Joe Elliott <number101010@gmail.com>

* lint

Signed-off-by: Joe Elliott <number101010@gmail.com>

* Update docs/sources/tempo/traceql/_index.md

Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com>

* comment

Signed-off-by: Joe Elliott <number101010@gmail.com>

---------

Signed-off-by: Joe Elliott <number101010@gmail.com>
Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com>

* Fix broken link in service-graphs docs (#4351)

* Fix minor typo in TraceQL docs (#4356)

* Bump default memcached version (#4363)

* Exemplar fixes (#4366)

* Fix exemplars based on duration to convert to seconds, fix various other issues

* changelog

* fix: initialize histogram buckets to 0 to avoid them being downsampled (#4368)

* initialized histogram buckets to 0 to avoid them being downsampled

* Ingester/Generator Live trace cleanup (#4365)

* moved trace sizes somewhere shareable

Signed-off-by: Joe Elliott <number101010@gmail.com>

* use tracesizes in ingester

Signed-off-by: Joe Elliott <number101010@gmail.com>

* make tests work

Signed-off-by: Joe Elliott <number101010@gmail.com>

* trace bytes in generator

Signed-off-by: Joe Elliott <number101010@gmail.com>

* remove traceCount

Signed-off-by: Joe Elliott <number101010@gmail.com>

* live trace shenanigans

Signed-off-by: Joe Elliott <number101010@gmail.com>

* changelog

Signed-off-by: Joe Elliott <number101010@gmail.com>

* Update modules/generator/processor/localblocks/livetraces.go

Co-authored-by: Mario <mariorvinas@gmail.com>

* Update modules/ingester/instance.go

Co-authored-by: Mario <mariorvinas@gmail.com>

* Test cleanup. Add sz test, restore commented out and fix e2e

Signed-off-by: Joe Elliott <number101010@gmail.com>

* remove todo comment

Signed-off-by: Joe Elliott <number101010@gmail.com>

---------

Signed-off-by: Joe Elliott <number101010@gmail.com>
Co-authored-by: Mario <mariorvinas@gmail.com>

* Bump anchore/sbom-action from 0.17.7 to 0.17.8 (#4371)

Bumps [anchore/sbom-action](https://github.com/anchore/sbom-action) from 0.17.7 to 0.17.8.
- [Release notes](https://github.com/anchore/sbom-action/releases)
- [Changelog](https://github.com/anchore/sbom-action/blob/main/RELEASE.md)
- [Commits](https://github.com/anchore/sbom-action/compare/v0.17.7...v0.17.8)

---
updated-dependencies:
- dependency-name: anchore/sbom-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Update for IDs change

* Only run blockbuilder if ingest enabled

---------

Signed-off-by: Joe Elliott <number101010@gmail.com>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Alex Bikfalvi <alex.bikfalvi@grafana.com>
Signed-off-by: Daniel Strobusch <1847260+dastrobu@users.noreply.github.com>
Co-authored-by: Javier Molina Reyes <javiermolinar@live.com>
Co-authored-by: Zach Leslie <zach.leslie@grafana.com>
Co-authored-by: Joe Elliott <number101010@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Ryan Perry <Rperry2174@gmail.com>
Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com>
Co-authored-by: Suraj Nath <9503187+electron0zero@users.noreply.github.com>
Co-authored-by: Alex Bikfalvi <alex@bikfalvi.com>
Co-authored-by: Andrey Karpov <ndk@users.noreply.github.com>
Co-authored-by: Jennifer Villa <jvilla2013@gmail.com>
Co-authored-by: Martin Disibio <martin.disibio@grafana.com>
Co-authored-by: Markus Toivonen <markus.toivonen@hoxhunt.com>
Co-authored-by: Daniel Strobusch <1847260+dastrobu@users.noreply.github.com>
Co-authored-by: Carles Garcia <carles.garciacabot@grafana.com>

* [rhythm] Changes to simplify operations (#4389)

* Use mapping for assigning partitions

* Use mapping for assigning partitions in the generator too

* Add support for SASL auth to kafka clients

* Add metrics to ingest (#4395)

* [rhythm] Extract block-builder into its own module (#4396)

* Extract block-builder into its own module

* Update /operations and examples

* No ephemeral storage

* No rolling strategy either

* fmt and compile

* Address review comment

* [rhythm] Correctly pass start/end time when appending a trace (#4410)

* Correctly pass start/end times

* Different code, same result

* [rhythm] Multiple fixes to block-builder consumption (#4413)

* Multiple fixes to cycle consumption

* fmt

* happy now?

* ups

* Rhythm: Separate non-flushing local blocks processor to store new queue data for reads (#4411)

* wip: separate non-flushing local blocks processor to store new queue data for reads

* Make real config for non-flushing local blocks processor, optional, validate wal config and use defaults if needed

* Fix defaulting of second WAL config

* [rhythm] Make ID generator more robust (#4416)

* Make ID generator more robust

* Simplify

* Update to e50f5d96b

* Fix registering of kafka read client metrics (#4502)

* [rhythm] Make ID generator more robust (#4416) (#4507)

* Make ID generator more robust

* Simplify

* Removed references to Loki and Mimir (#4509)

Signed-off-by: Joe Elliott <number101010@gmail.com>

* [Rhythm] Block builder test updates (#4510)

* Make blockbuilder tests closer to real kafka and less implementation specific by always enabling support for consumer groups, call commit control func in order

* Verify last committed offset in each test

* hide test function

* lint

* lint

* [Rhythm] Block-builder consumption loop (#4480)

* Alternate block-builder consume

* Set timeout on PollFetches, reduce initial poll delay, update 1 test to work using real consumergroup functionality

* restore metrics

* Re-add original partition lag metric, polled in separate goroutine. Fix consume loop to only consume full-duration cycles for more determinism

* merge conflict

* Review feedback

* Review feedback

* Comment

* code cleanup, lint

* logs

* code cleanup

* lint

* Review feedback

* Remove missed lookback_on_no_commit config in e2e tests and regen manifest

* Review feedback

* Fix rewind to latest commit to init correctly, it didn't work in some clusters (#4532)

* [rhythm] merge main at 71e8531 (#4531)

* Fixes

* More fixes

---------

Signed-off-by: Joe Elliott <number101010@gmail.com>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Alex Bikfalvi <alex.bikfalvi@grafana.com>
Signed-off-by: Daniel Strobusch <1847260+dastrobu@users.noreply.github.com>
Co-authored-by: Javier Molina Reyes <javiermolinar@live.com>
Co-authored-by: Zach Leslie <zach.leslie@grafana.com>
Co-authored-by: Joe Elliott <number101010@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Ryan Perry <Rperry2174@gmail.com>
Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com>
Co-authored-by: Suraj Nath <9503187+electron0zero@users.noreply.github.com>
Co-authored-by: Alex Bikfalvi <alex@bikfalvi.com>
Co-authored-by: Andrey Karpov <ndk@users.noreply.github.com>
Co-authored-by: Jennifer Villa <jvilla2013@gmail.com>
Co-authored-by: Martin Disibio <martin.disibio@grafana.com>
Co-authored-by: Markus Toivonen <markus.toivonen@hoxhunt.com>
Co-authored-by: Daniel Strobusch <1847260+dastrobu@users.noreply.github.com>
Co-authored-by: Carles Garcia <carles.garciacabot@grafana.com>
2025-01-10 16:05:42 +01:00
9e8f582cfe v2.7.0-rc.0 (#4508)
Signed-off-by: Joe Elliott <number101010@gmail.com>
2025-01-03 10:55:31 -05:00
c34cf3ae3b Bump alpine base image to 3.21 (#4504)
* Bump alpine base image to 3.21

* changelog
2025-01-02 17:32:06 -05:00