678 Commits

Author SHA1 Message Date
3f420b6a4a vParquet4: update opentelemetry-proto and add scope attrs (#3649)
* Update opentelemetry-proto submodule to v1.3.0

* Generate proto

* Populate scope attributes

* Mention breaking change in CHANGELOG.md

* Run make check-fmt
2024-05-10 09:34:13 +10:00
29f4167bba [TraceQL Metrics] histogram_over_time (#3644)
* Add lang/parser support for quantile_over_time, fix missing stringify of metrics and hints

* First working draft of quantile_over_time implementation

* Validate the query in the frontend

* Histogram accumulate jobs by bucket as they come in instead of at the end. Refactor label handling to be shared. Validate quantiles, support quantile_over_time by any integer attribute

* Fix language definition to allow both floats or ints for quantiles

* Remove layer of proto->seriesset conversion. Fix roundtrip of __bucket label

* Rename to SimpleAdditionCombiner, slight interval calc cleanup

* Fix p0 returning 1 instead of minimum value, comments cleanup

* Rename frontend param and fix handling of ints

* Fix pre-existing bug in metrics optimization when asserting multiple conditions on the same attribute

* Fix to support 3 flavors of the metrics pipeline: query-frontend, across blocks, and across generators.  Error handling, move code around

* Update query_range frontend test for new behavior

* Consolidate histogram code between traceql and traceqlmetrics. quantile_over_time test, code cleanup

* lint

* changelog

* histogram_over_time

* Redo histograms to set __bucket label to the actual value instead of log2, for several reasons:  support non-log2 or native histogram buckets in the future, and where queriers on different versions may have different buckets during rollout

* Revert all changes to traceqlmetrics package, was getting too noisy

* Fix after merge

* lint

* changelog

* Add test for histogram_over_time

* Change quantile_over_time test to use more exported methods
2024-05-09 12:08:09 -04:00
e06c33905e 👋 So long vParquet1, and thanks for all the fish! (#3663)
* good bye vParquet!

* changelog
2024-05-09 11:04:36 -04:00
b76ab98594 Golangci deprecations and update using tools pattern (#3610)
* Breadcrumbs for golangci

* Include golangci in tools

* Use tools image for lint make target

* Update changelog
2024-05-08 20:56:43 +00:00
9032132d0c Bump x/net package to 0.24/latest (#3613)
* Bump x/net package to 0.24/latest

Signed-off-by: Pavol Loffay <p.loffay@gmail.com>

* Fix

Signed-off-by: Pavol Loffay <p.loffay@gmail.com>

* Fix

Signed-off-by: Pavol Loffay <p.loffay@gmail.com>

---------

Signed-off-by: Pavol Loffay <p.loffay@gmail.com>
2024-05-08 17:21:01 +00:00
5b250a7e85 fix(traceql): fix extract matcher regexes to work with regexp-type matchers (#3641)
* fix(traceql): fix extract matcher regexes to work with regexp-type matchers

Partially fixes #3635.

I'm going to add a further test case which this doesn't fix.

* Add more test cases for extract matcher

* Add CHANGELOG entry
2024-05-08 12:38:22 -04:00
947de3531d Update deb and rpm packages to create and assign permissions for /var/tempo (#3657)
* test deb stuff

* more stuff

* create /var/tempo and assign perms

* Create and assign permissions of /var/tempo after creating user

* More precise dir perms

* whitespace

* changelog
2024-05-08 12:28:58 -04:00
8df96701a3 [TraceQL] quantile_over_time 2 of 2 - engine (#3633)
* Add lang/parser support for quantile_over_time, fix missing stringify of metrics and hints

* First working draft of quantile_over_time implementation

* Validate the query in the frontend

* Histogram accumulate jobs by bucket as they come in instead of at the end. Refactor label handling to be shared. Validate quantiles, support quantile_over_time by any integer attribute

* Fix language definition to allow both floats or ints for quantiles

* Remove layer of proto->seriesset conversion. Fix roundtrip of __bucket label

* Rename to SimpleAdditionCombiner, slight interval calc cleanup

* Fix p0 returning 1 instead of minimum value, comments cleanup

* Rename frontend param and fix handling of ints

* Fix pre-existing bug in metrics optimization when asserting multiple conditions on the same attribute

* Fix to support 3 flavors of the metrics pipeline: query-frontend, across blocks, and across generators.  Error handling, move code around

* Update query_range frontend test for new behavior

* Consolidate histogram code between traceql and traceqlmetrics. quantile_over_time test, code cleanup

* lint

* changelog

* Redo histograms to set __bucket label to the actual value instead of log2, for several reasons:  support non-log2 or native histogram buckets in the future, and where queriers on different versions may have different buckets during rollout

* Revert all changes to traceqlmetrics package, was getting too noisy
2024-05-07 14:23:16 -04:00
884a344b60 SearchTagValuesV2 perf improvements (#3650)
* remove keep

Signed-off-by: Joe Elliott <number101010@gmail.com>

* null test

Signed-off-by: Joe Elliott <number101010@gmail.com>

* remove attr

Signed-off-by: Joe Elliott <number101010@gmail.com>

* consolidate

Signed-off-by: Joe Elliott <number101010@gmail.com>

* dont sent 2x

Signed-off-by: Joe Elliott <number101010@gmail.com>

* port to vp4

Signed-off-by: Joe Elliott <number101010@gmail.com>

* changelog

Signed-off-by: Joe Elliott <number101010@gmail.com>

* fix wal

Signed-off-by: Joe Elliott <number101010@gmail.com>

---------

Signed-off-by: Joe Elliott <number101010@gmail.com>
2024-05-07 13:15:53 -04:00
fc33ae5fba Add vparquet4 encoding (#3368)
* Copy vparquet3 to vparquet4 folder

* Create initial vParquet4 encoding

* vParquet4: handle unsupported / dropped attributes (#3123)

* Count dropped attributes and store them in separate column

* Remove current support for arrays and kv-lists

* Store per-trace service statistics in vparquet4 blocks (#2941)

* Add support for attributes with array values (#3221)

* Implement single values as arrays and introduce type column

* Add support for array attributes

* Tests more array values and fix attribute conversion

* Use a combination of snappy and delta for ValueType

* Vendor module google/go-cmp/cmp/cmpopts

* Use all types of array attributes in TestTraceToParquet

* Improve TestFieldsAreCleared

* links and events schema changes (#3163)

* Add attributes to instrumentation scope (#3322)

* Fix typo in vParquet4 event name enconding (#3336)

* Precalculate and reuse the vParquet4 schema before opening blocks

* Convert block in vparquet4/test-data directory

* CHANGELOG.md

* Update test cases in BenchmarkBackendBlockTraceQL

* Unsupported attribute values are no longer dropped

* Replace ValueType column with IsArray column

* Skip service stats map allocation when none are present

---------

Signed-off-by: Andreas Gerstmayr <agerstmayr@redhat.com>
Co-authored-by: Andreas Gerstmayr <agerstmayr@redhat.com>
Co-authored-by: Jennie Pham <94262131+ie-pham@users.noreply.github.com>
Co-authored-by: Andreas Gerstmayr <andreas@gerstmayr.me>
2024-05-07 10:01:03 +10:00
37f3308aeb Fix span-metrics' subprocessors bug (#3612)
* Fix span-metrics' subprocessors bug that applied wrong configs when running multiple tenants

* Add test

* Fix race condition

* Update CHANGELOG.md

Co-authored-by: Joe Elliott <joe.elliott@grafana.com>

* Copy when copying the config

* Fix bug

---------

Co-authored-by: Joe Elliott <joe.elliott@grafana.com>
2024-05-06 10:39:43 +02:00
3f831f9e31 Implement tenant deletion for empty tenants (#3611)
* Extend backend.Reader with Find() function

* Extend backend.Writer with Delete() function

* Add EmptyTenantDeletion age config to tempodb

* Add delete handling to poller

* Add integration tests for poller change

* Update backend implementations

* Fix lint

* Setup test defaults

* Integrate prefix test permutations

* Add doc for the new configuration option

* Update changelog

* Fix local implementation to ignore directories

* Tidy up todo after test coverage

* Update docs/sources/tempo/configuration/_index.md

Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com>

* Address some PR feedback

* Rename backend.FindOpts -> backend.FindMatch
* Adjust default delete time from 20m -> 12h

* Add log message for object deletion

* Add additional safety check to avoid deletion

* Add test default value

* Godoc update after rename

* Drop logging from the backend used for testing

* Require empty tenant deletion to be enabled in the config

* Add docs for _enabled config

---------

Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com>
2024-05-03 20:48:42 +00:00
35aa72e692 Add new histogram to generator - messaging system latency (#3453)
* introduce new service-graph metric for messaging-system latency

* added tests for new histogram values

* fix linting

* make new metric optional via config

* fix typo

* fix failing tests

* add feature to changelog

* negative times diff consistency - return 0 instead of negative

* update docs

* Update docs/sources/tempo/metrics-generator/service_graphs/estimate-cardinality.md

use present when possible

Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com>

* change 1e9 to time const

* added a reference to the "wait" config of the processor

* fixed indentations and formatting stuff from rebasing

* removed mistaken println found by linter

---------

Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com>
2024-05-03 12:49:23 -04:00
854cacaa42 TraceQL - Add support for scoped intrinsics using : (#3629)
* add scoped intrinsics support :

* lint

* test & changelog

* more test & changelog & doc

* rename

* add comment
2024-05-02 19:25:22 -05:00
251bf5a9a7 Surface new labels for uninstrumented services and systems (#3543)
* Surface new labels for uninstrumented services and systems

* Update CHANGELOG.md

* remove unnecessary Println in test

* Reuse dimensions and prefixes for this use-case

* Add docs

* keep only virtual_node behind the new feature

* add overrides

* Update docs/sources/tempo/metrics-generator/service_graphs/_index.md

Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com>

* forgot this test (fixed)

* add benchmarks

* add edge pooling/reuse

* update flag

* move label out of dimensions to avoid prefix logic

* lint

* Update modules/generator/processor/servicegraphs/config.go

Co-authored-by: Mario <mariorvinas@gmail.com>

* minor amends to names and docs

* leave the new virtual_node label as an extra dimension

* keep edge sync.Pool ops inside store

* Update modules/generator/processor/servicegraphs/store/store.go

The edge is not expired here, so it shouldn't be returned to the pool.

Co-authored-by: Mario <mariorvinas@gmail.com>

* leave the new label un-prefixed

---------

Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com>
Co-authored-by: Mario <mariorvinas@gmail.com>
2024-04-30 16:53:05 -04:00
9caf167fb7 Update docker compose examples (#3603)
* Update local example to mention the path ownership required

* Drop grafana7.4 docker-compose example

* Drop docker-compose example for gcs

* Drop docker-compose example for azure

* Drop docker-compose example for s3

* Update docker compose examples

* Drop deprecated `version` field
* Add init service to chown the volume on startup and set dependency for ordered startup
* Use yaml reference for docker image specification

* Update docker compose example commands for deprecation

* Update top docker-compose readme to point to our docs

* Update integration readme for docker compose command deprecation

* Update tk readme for docker compose command deprecation

* Remove s3 docker compose example and point to the top of the example/docker-compose

* Update wording slightly

* Update changelog

* Be less specific about the possible backend configurations.
2024-04-24 16:16:07 +00:00
7f6d590b89 clear metrics generator wal on start up (#3548)
* clear metrics generator wal on start up

* add log line

* changelogs and move clean wal to even earlier

* clear tenant specific folders only
2024-04-19 12:03:11 -05:00
c59bb01bcb Query Frontend Refactor: Metrics Query Range (#3584)
* things technically compile!

Signed-off-by: Joe Elliott <number101010@gmail.com>

* Succeed with no data

Signed-off-by: Joe Elliott <number101010@gmail.com>

* remove prom compat

Signed-off-by: Joe Elliott <number101010@gmail.com>

* Add tests for the metrics pipeline and fix

Signed-off-by: Joe Elliott <number101010@gmail.com>

* cleanup

Signed-off-by: Joe Elliott <number101010@gmail.com>

* restore sorting

Signed-off-by: Joe Elliott <number101010@gmail.com>

* removed unreferenced files

Signed-off-by: Joe Elliott <number101010@gmail.com>

* migrate to response interface

Signed-off-by: Joe Elliott <number101010@gmail.com>

* Sampling rate!

Signed-off-by: Joe Elliott <number101010@gmail.com>

* remove no-op

Signed-off-by: Joe Elliott <number101010@gmail.com>

* wip: add cli

Signed-off-by: Joe Elliott <number101010@gmail.com>

* QueryRange -> MetricsQueryRange

Signed-off-by: Joe Elliott <number101010@gmail.com>

* changelog and other cleanup

Signed-off-by: Joe Elliott <number101010@gmail.com>

* lint

Signed-off-by: Joe Elliott <number101010@gmail.com>

* lint + guardcode

Signed-off-by: Joe Elliott <number101010@gmail.com>

* change the way we're ignoring tests

Signed-off-by: Joe Elliott <number101010@gmail.com>

* clean up url parsing

Signed-off-by: Joe Elliott <number101010@gmail.com>

* fixed url parsing and add test

Signed-off-by: Joe Elliott <number101010@gmail.com>

* adjust sanity check

Signed-off-by: Joe Elliott <number101010@gmail.com>

---------

Signed-off-by: Joe Elliott <number101010@gmail.com>
2024-04-19 09:07:30 -04:00
2935e9cf38 Add TLS support for Memcached client (#3585)
* Adding TLS support for Memcached Client

* Update CHANGELOG.md

* Resolve linting and build failures

* newClient not netClient

* Include username in CHANGELOG entry
2024-04-19 07:33:44 -04:00
5e26604dcf Add support for ingester query shards (#3574)
* Add support for ingester query shards

* Update changelog

* Drop unused test method

* Avoid boring lint error

* Fix /0 panic

* Fix first query param in test

* Drop TODO after fixed in config

* Tidy

* Test for case where we skip doing too much work

* Reduce shards until we are above the hardcode 1m interval threshold

* Test greater than query count

* Tidy

* Avoid /0

* Require 5 occurrences for goconst

* Set goconst to ignore tests, drop previous config

* Avoid shard count checking and set interval floor

* Clean and lint

* Handle case where more shards than the window will tollerate

* Ensure tests handle delta in both postivie and negative directions

* Grow last shard to fill the gap and add test

* Tidy
2024-04-18 17:40:20 +00:00
646d3dcbd2 Compactor tweaks: Fix pool leak when iter exhausted and async pages for better throughput (#3579)
* Fix pool leak when iter exhausted. Async pages for better throughput

* changelog
2024-04-17 11:27:34 -04:00
fa7aeb11f0 changelog patch up (#3573)
Signed-off-by: Joe Elliott <number101010@gmail.com>
2024-04-15 17:27:35 -04:00
f9b37a8589 Use os path separator to split blocks path (#3568)
* Use os path separator to split blocks path

* Add changelog

* Fix order
2024-04-12 10:20:13 -04:00
475312b319 Tempo: Refactor Trace By ID and Metrics Summary (#3522)
* extend buildUpstreamRequest

Signed-off-by: Joe Elliott <number101010@gmail.com>

* metrics summary

Signed-off-by: Joe Elliott <number101010@gmail.com>

* trace by id first pass

Signed-off-by: Joe Elliott <number101010@gmail.com>

* fixed inf loop

Signed-off-by: Joe Elliott <number101010@gmail.com>

* works

Signed-off-by: Joe Elliott <number101010@gmail.com>

* removed trace id hedging in the frontend

Signed-off-by: Joe Elliott <number101010@gmail.com>

* todos, tests and cleanup

Signed-off-by: Joe Elliott <number101010@gmail.com>

* add 429 test

Signed-off-by: Joe Elliott <number101010@gmail.com>

* changelog

Signed-off-by: Joe Elliott <number101010@gmail.com>

* lint

Signed-off-by: Joe Elliott <number101010@gmail.com>

* restore api prefix

Signed-off-by: Joe Elliott <number101010@gmail.com>

* pass the right param

Signed-off-by: Joe Elliott <number101010@gmail.com>

* fix race in test

Signed-off-by: Joe Elliott <number101010@gmail.com>

* move deduper into combiner

Signed-off-by: Joe Elliott <number101010@gmail.com>

* Move content type marshalling into combiner

Signed-off-by: Joe Elliott <number101010@gmail.com>

* lint: check err in test

Signed-off-by: Joe Elliott <number101010@gmail.com>

* Update integration/e2e/limits_test.go

Co-authored-by: Koenraad Verheyden <koenraad.verheyden@posteo.net>

* Apply suggestions from code review

Co-authored-by: Koenraad Verheyden <koenraad.verheyden@posteo.net>

* rename func for clarity

Signed-off-by: Joe Elliott <number101010@gmail.com>

* review

Signed-off-by: Joe Elliott <number101010@gmail.com>

---------

Signed-off-by: Joe Elliott <number101010@gmail.com>
Co-authored-by: Koenraad Verheyden <koenraad.verheyden@posteo.net>
2024-04-11 15:07:36 -04:00
b6312f4a8e Add spss and limit to search request hash (#3557)
* Add spss and limit to search request hash

Signed-off-by: Joe Elliott <number101010@gmail.com>

* changelog

Signed-off-by: Joe Elliott <number101010@gmail.com>

* lint

Signed-off-by: Joe Elliott <number101010@gmail.com>

---------

Signed-off-by: Joe Elliott <number101010@gmail.com>
2024-04-10 12:54:25 -04:00
0d3bde58b2 Update docker image to run as non-root (#2265)
* Update Tempo image to run as non-root

Signed-off-by: Zach Leslie <zach.leslie@grafana.com>

* Change ownership of /var/tempo

Signed-off-by: Zach Leslie <zach.leslie@grafana.com>

* Breadcrumbs for ingester filesystem permissions

Signed-off-by: Zach Leslie <zach.leslie@grafana.com>

* Create the directory we attempt to chown

Signed-off-by: Zach Leslie <zach.leslie@grafana.com>

* Include initcontainer and adjust statefulset security for new UID

* Compile jsonnet

* Drop securityContext since the chown handles the permissions

* Adjust test path for ownership

* Update changelog to note breaking change

* Drop extra object and include additional hardcode

* Improve language for CHANGELOG

---------

Signed-off-by: Zach Leslie <zach.leslie@grafana.com>
2024-04-04 16:43:53 +00:00
d806d4bbaf Frontend: Add tenant to cache key (#3527)
* add tenant to cache key

Signed-off-by: Joe Elliott <number101010@gmail.com>

* changelog

Signed-off-by: Joe Elliott <number101010@gmail.com>

---------

Signed-off-by: Joe Elliott <number101010@gmail.com>
2024-03-28 15:29:01 -04:00
6cafca9c29 Add querier metrics for requests executed (#3524)
* Add metrics for concurrency and querier requests executed

* update changelog

* turn metric into log line

* update
2024-03-27 20:44:59 +05:30
3d2850d457 TraceQL: Nested set intrinsics (#3497)
* add nested set

Signed-off-by: Joe Elliott <number101010@gmail.com>

* only return nested set params if explicitly requested

Signed-off-by: Joe Elliott <number101010@gmail.com>

* tests, tests, tests !

Signed-off-by: Joe Elliott <number101010@gmail.com>

* lint

Signed-off-by: Joe Elliott <number101010@gmail.com>

* changelog

Signed-off-by: Joe Elliott <number101010@gmail.com>

* added/fixed tests

Signed-off-by: Joe Elliott <number101010@gmail.com>

* fix predicate adding

Signed-off-by: Joe Elliott <number101010@gmail.com>

---------

Signed-off-by: Joe Elliott <number101010@gmail.com>
2024-03-21 15:32:43 -04:00
c61bd1b9d6 Frontend Refactor: Streaming tags (#3460)
* log request

Signed-off-by: Joe Elliott <number101010@gmail.com>

* move stuff a bit

Signed-off-by: Joe Elliott <number101010@gmail.com>

* oh my. e2e tests pass

Signed-off-by: Joe Elliott <number101010@gmail.com>

* add handlers

Signed-off-by: Joe Elliott <number101010@gmail.com>

* streaming tags

Signed-off-by: Joe Elliott <number101010@gmail.com>

* add cli support

Signed-off-by: Joe Elliott <number101010@gmail.com>

* improve logging

Signed-off-by: Joe Elliott <number101010@gmail.com>

* fix

Signed-off-by: Joe Elliott <number101010@gmail.com>

* docs

Signed-off-by: Joe Elliott <number101010@gmail.com>

* pipe overrides

Signed-off-by: Joe Elliott <number101010@gmail.com>

* cleanup

Signed-off-by: Joe Elliott <number101010@gmail.com>

* cleanup

Signed-off-by: Joe Elliott <number101010@gmail.com>

* support limits

Signed-off-by: Joe Elliott <number101010@gmail.com>

* docs

Signed-off-by: Joe Elliott <number101010@gmail.com>

* e2e tests and caching

Signed-off-by: Joe Elliott <number101010@gmail.com>

* key prefixes

Signed-off-by: Joe Elliott <number101010@gmail.com>

* cache keys

Signed-off-by: Joe Elliott <number101010@gmail.com>

* Fixed distinct collection in combiners

Signed-off-by: Joe Elliott <number101010@gmail.com>

* fixed combiner bugs and revived tests

Signed-off-by: Joe Elliott <number101010@gmail.com>

* restored all tests

Signed-off-by: Joe Elliott <number101010@gmail.com>

* lint

Signed-off-by: Joe Elliott <number101010@gmail.com>

* made search handler utilities generic

Signed-off-by: Joe Elliott <number101010@gmail.com>

* Added handler tests for tags

Signed-off-by: Joe Elliott <number101010@gmail.com>

* add diff support

Signed-off-by: Joe Elliott <number101010@gmail.com>

* lint

Signed-off-by: Joe Elliott <number101010@gmail.com>

* add distinct value collector tests

Signed-off-by: Joe Elliott <number101010@gmail.com>

* fix integration tests

Signed-off-by: Joe Elliott <number101010@gmail.com>

* diff tests

Signed-off-by: Joe Elliott <number101010@gmail.com>

* swapped query for the more robust ExtractMatchers(query)

Signed-off-by: Joe Elliott <number101010@gmail.com>

* tests

Signed-off-by: Joe Elliott <number101010@gmail.com>

* moved e2e tests to a more sensible place

Signed-off-by: Joe Elliott <number101010@gmail.com>

* fix non-deterministic  test

Signed-off-by: Joe Elliott <number101010@gmail.com>

* changelog

Signed-off-by: Joe Elliott <number101010@gmail.com>

* fix tests for 429 handling

Signed-off-by: Joe Elliott <number101010@gmail.com>

* Update docs/sources/tempo/operations/tempo_cli.md

Co-authored-by: Mario <mariorvinas@gmail.com>

* review

Signed-off-by: Joe Elliott <number101010@gmail.com>

* Update docs/sources/tempo/api_docs/_index.md

Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com>

* Correctly cancel GRPC context beneath the HTTP server (#3443)

* cancel context

Signed-off-by: Joe Elliott <number101010@gmail.com>

* update dskit

Signed-off-by: Joe Elliott <number101010@gmail.com>

* focused timeouts

Signed-off-by: Joe Elliott <number101010@gmail.com>

* docs

Signed-off-by: Joe Elliott <number101010@gmail.com>

* lint N docs

Signed-off-by: Joe Elliott <number101010@gmail.com>

* more lint

Signed-off-by: Joe Elliott <number101010@gmail.com>

* make update-mod

Signed-off-by: Joe Elliott <number101010@gmail.com>

---------

Signed-off-by: Joe Elliott <number101010@gmail.com>

* Bump anchore/sbom-action from 0.15.8 to 0.15.9 (#3476)

Bumps [anchore/sbom-action](https://github.com/anchore/sbom-action) from 0.15.8 to 0.15.9.
- [Release notes](https://github.com/anchore/sbom-action/releases)
- [Commits](https://github.com/anchore/sbom-action/compare/v0.15.8...v0.15.9)

---
updated-dependencies:
- dependency-name: anchore/sbom-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Doc update (#3482)

* doc: remove reference to previously purged script

* doc: correct label for docs updates

* [TraceQL Metrics] Use new per-tenant max_metrics_duration and fix duration check (#3484)

* Use new per-tenant max_metrics_duration, and fix duration timestamp handling

* Update docs and defaults

* Handle prefixes when listing blocks from S3 and GCS (#3466)

* Handle prefixes when listing blocks from S3

fixes #3465

* Handle prefixes when listing blocks from GCS

* Add test for prefixes when listing blocks from Azure

* Update unit tests to check for actual block IDs instead of just length of the slices

Cleanup unit tests

* Further refine S3/GCS backend for ListBlocks

Brings logic more in line with Azure object parsing.
Also has the benefit of handling prefixes without a trailing slash.

* Update poller integration test to exercise prefixes

* Update e2e test to exercise prefixes

* Fix format check error

* Fix failing e2e tests

* Remove unnecessary prefix permutations from e2e test

* Remove unnecessary test config file copy

* Ignore lint

---------

Co-authored-by: Zach Leslie <zach.leslie@grafana.com>

* Update doc-validator.yml (#3483)

Updates the doc-validator to the latest version. Note that this changes the reference format to use the full URL (https://....) instead of /docs/blah

* [DOC] Document Tempo Operator Monolithic mode (#3474)

* [DOC] Document Tempo Operator Monolithic mode

Signed-off-by: Andreas Gerstmayr <agerstmayr@redhat.com>

* clarify supported storages

Signed-off-by: Andreas Gerstmayr <agerstmayr@redhat.com>

* fix case of title

Signed-off-by: Andreas Gerstmayr <agerstmayr@redhat.com>

* Apply suggestions from code review

* Apply suggestions from code review

---------

Signed-off-by: Andreas Gerstmayr <agerstmayr@redhat.com>
Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com>

* [DOC] document Grafana data source setup using Grafana and Tempo operators (#3473)

* [docs] document Grafana data source setup using Grafana and Tempo operators

* move the Grafana data source setup page to the operator folder (this
  page is only relevant for the operator)
* document Grafana data source setup using Grafana and Tempo operators

Signed-off-by: Andreas Gerstmayr <agerstmayr@redhat.com>

* Apply suggestions from code review

---------

Signed-off-by: Andreas Gerstmayr <agerstmayr@redhat.com>
Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com>

* [DOC] fix typo in setup/operator/monolithic.md (#3496)

Signed-off-by: Andreas Gerstmayr <agerstmayr@redhat.com>

* Bump github.com/prometheus/client_golang from 1.18.0 to 1.19.0 (#3455)

* Bump github.com/prometheus/client_golang from 1.18.0 to 1.19.0

Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.18.0 to 1.19.0.
- [Release notes](https://github.com/prometheus/client_golang/releases)
- [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prometheus/client_golang/compare/v1.18.0...v1.19.0)

---
updated-dependencies:
- dependency-name: github.com/prometheus/client_golang
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update serverless gomod

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: grafanabot <bot@grafana.com>

* Add support for dashes, quotes and spaces in attribute names (#3458)

* Add support for dashes, quotes and spaces in attribute names

* chlog

* [TraceQL Metrics] Step align query_range time range (#3490)

* Step align query_range time range

* Time range error: improve message and fix format for prom format.

* oops remove printlns

* lint

* changelog

* 2.4.1 changelog (#3503)

Signed-off-by: Joe Elliott <number101010@gmail.com>

* [DOC] Add 2.4.1 release notes (#3504)

* fix tests due to interface changing

Signed-off-by: Joe Elliott <number101010@gmail.com>

* Pass context

Signed-off-by: Joe Elliott <number101010@gmail.com>

---------

Signed-off-by: Joe Elliott <number101010@gmail.com>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Andreas Gerstmayr <agerstmayr@redhat.com>
Co-authored-by: Mario <mariorvinas@gmail.com>
Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Matt Simonsen <matt.simonsen@gmail.com>
Co-authored-by: Martin Disibio <martin.disibio@grafana.com>
Co-authored-by: Ben Foster <bpfoster@gmail.com>
Co-authored-by: Zach Leslie <zach.leslie@grafana.com>
Co-authored-by: Andreas Gerstmayr <agerstmayr@redhat.com>
Co-authored-by: grafanabot <bot@grafana.com>
2024-03-20 15:53:00 -04:00
c41b078b13 2.4.1 changelog (#3503)
Signed-off-by: Joe Elliott <number101010@gmail.com>
2024-03-19 13:50:10 -04:00
b8cdf38b4f [TraceQL Metrics] Step align query_range time range (#3490)
* Step align query_range time range

* Time range error: improve message and fix format for prom format.

* oops remove printlns

* lint

* changelog
2024-03-19 12:40:18 -04:00
a751904ccc Add support for dashes, quotes and spaces in attribute names (#3458)
* Add support for dashes, quotes and spaces in attribute names

* chlog
2024-03-19 12:14:13 +00:00
8e6e7fe86f Handle prefixes when listing blocks from S3 and GCS (#3466)
* Handle prefixes when listing blocks from S3

fixes #3465

* Handle prefixes when listing blocks from GCS

* Add test for prefixes when listing blocks from Azure

* Update unit tests to check for actual block IDs instead of just length of the slices

Cleanup unit tests

* Further refine S3/GCS backend for ListBlocks

Brings logic more in line with Azure object parsing.
Also has the benefit of handling prefixes without a trailing slash.

* Update poller integration test to exercise prefixes

* Update e2e test to exercise prefixes

* Fix format check error

* Fix failing e2e tests

* Remove unnecessary prefix permutations from e2e test

* Remove unnecessary test config file copy

* Ignore lint

---------

Co-authored-by: Zach Leslie <zach.leslie@grafana.com>
2024-03-14 18:39:37 +00:00
f34a137131 [TraceQL Metrics] Use new per-tenant max_metrics_duration and fix duration check (#3484)
* Use new per-tenant max_metrics_duration, and fix duration timestamp handling

* Update docs and defaults
2024-03-14 08:55:19 -04:00
eb81b92881 Correctly cancel GRPC context beneath the HTTP server (#3443)
* cancel context

Signed-off-by: Joe Elliott <number101010@gmail.com>

* update dskit

Signed-off-by: Joe Elliott <number101010@gmail.com>

* focused timeouts

Signed-off-by: Joe Elliott <number101010@gmail.com>

* docs

Signed-off-by: Joe Elliott <number101010@gmail.com>

* lint N docs

Signed-off-by: Joe Elliott <number101010@gmail.com>

* more lint

Signed-off-by: Joe Elliott <number101010@gmail.com>

* make update-mod

Signed-off-by: Joe Elliott <number101010@gmail.com>

---------

Signed-off-by: Joe Elliott <number101010@gmail.com>
2024-03-12 12:18:42 -04:00
ae083c34b8 Fix 429s tripping SLOs (#3469)
* fix 429s tripping SLOs

Signed-off-by: Joe Elliott <number101010@gmail.com>

* changelog

Signed-off-by: Joe Elliott <number101010@gmail.com>

* test fix

Signed-off-by: Joe Elliott <number101010@gmail.com>

---------

Signed-off-by: Joe Elliott <number101010@gmail.com>
2024-03-06 17:00:07 -05:00
8079874157 [Traceql metrics] New (unsafe) query hints (#3396)
* Update traceql metrics to use the trace-level timestamp columns conditionally

* comments

* Update benchmark, comment

* lint

* Change overlap cutoff to 20%

* add more instrumentation and a little cleanup

* Add many new hints, with the unsafe hints enabled by per-tenant flag

* Collect all hints in one place, decomplicate code

* Make func signature golang convention

* make querier block_concurrency configurable

* changelog

* Make time_overlap_cutoff configurable. Rename block_concurrency to concurrent_blocks to match naming convention of other concurrent_* fields.

* fix test
2024-03-05 07:26:58 -05:00
781bcaf827 [TraceQL Metrics] Fix handling of empties and nils + other fixes (#3440)
* Preserve empty string timeseries labels all the way to the api, fix various small issues with step, json, etc

* Support other data types for prom

* changelog

* gen-proto
2024-02-29 14:01:31 -05:00
bb1627c0d4 Fix metrics results when filtering and rating on the same attribute (#3436)
* Fix metrics by() conditions to not interfere with existing condtions

* changelog
2024-02-28 14:48:22 -05:00
014cc3a04a Return unfiltered results when a bad TraceQL query is provided (#3426)
* Return unfiltered results when a bad TraceQL query is provided in autocomplete

* chlog num
2024-02-28 16:27:59 +01:00
1b1bab0f65 Add string interning to SyncIterator (#3411)
* Intern pq.Value

* Some improvements

* No noticeable improvement with pool

* Add tests

* Don't intern UUID columns

* fmt

* Add interning to older formats

* Add simple benchmark

* Please linter

* Some improvements

* Add comments

* chlog

* Add high cardinality attr to benchmark
2024-02-27 16:54:52 +01:00
8026c874db changelog 2.4 (#3427)
Signed-off-by: Joe Elliott <number101010@gmail.com>
2024-02-26 13:57:19 -05:00
3bb0c2c4d0 chore: add query-frontend option to select request headers in query logs (#3383)
* chore: add query-frontend option to select request headers in query logs

* chore: added changelog entry

* Update modules/frontend/handler.go

Co-authored-by: Mario <mariorvinas@gmail.com>

* Update modules/frontend/handler.go

Co-authored-by: Mario <mariorvinas@gmail.com>

* chore: fix typo in a name

---------

Co-authored-by: Mario <mariorvinas@gmail.com>
2024-02-26 12:14:14 +01:00
1c125aaff8 Fix spanmetrics latency underflow, prevent partial histogram scrape (#3412)
* Fix spanmetrics latency underflow, prevent partial histogram scrape

* changelog
2024-02-22 19:41:40 +01:00
63562bcd7e changelog (#3415)
Signed-off-by: Joe Elliott <number101010@gmail.com>
2024-02-20 21:03:41 +00:00
c041e08768 Restore tempo_request_duration_seconds metrics for querier_api_* requests (#3403)
* Restore tempo_request_duration_seconds metrics for querier_api_* requests

* Update CHANGELOG.md
2024-02-16 19:20:53 +00:00
4f53eb9eda Add shutdown delay and e2e test (#3395)
* Add shutdown delay and e2e test

Signed-off-by: Joe Elliott <number101010@gmail.com>

* changelog

Signed-off-by: Joe Elliott <number101010@gmail.com>

* determinism + comment

Signed-off-by: Joe Elliott <number101010@gmail.com>

* .Stop in a different goroutine then .Start creates a race

Signed-off-by: Joe Elliott <number101010@gmail.com>

---------

Signed-off-by: Joe Elliott <number101010@gmail.com>
2024-02-15 11:29:19 -05:00
e446bda47c Fix trace context propagation in the read path (#3387)
* Correctly propagate trace context between query-frontend and querier

* Add tracing to distributed docker-compose example

* Changelog
2024-02-14 11:59:55 +01:00
9d82759ddd deprecating vparquet v1 (#3377)
* deprecating vparquet v1

* update docs and tests

* meow

* remove line
2024-02-09 11:56:53 -06:00