* Allow compaction disable per tenant
* Update mock
* Rename legacy yaml key
* Rename methods and fields for clarity about disablement
* Rename methods and fields for clarity about disablement
* Update changelog
* Add enum to registry for allowed histogram modes
* Update overrides and generator for new enum values
* Update servicegraph tests for signature
* Update generator tests for interface change
* Update generator storage tests for interface change
* Update generator storage tests for interface change
* Drop signature update
* Update validation for enum lookup
* Update test for signature change
* Fix servicegraphs signature
* Fix default validation when empty
* Remove unused function
* Set default value for generator config to avoid empty checking enum
* Initial setup native histograms
* Bump prometheus dependencies, map exemplars over
* Add GenerateNativeHistograms from legacy
* Add test coverage for legacy overrides
* Plumb overrides into remote write config generation and test
* Lint for unused vars and duplicate imports
* Map 'classic' histograms out of prom.Histogram
* More tweaking to get classic histograms working, not there yet though :(
* Lint increment
* Lint increment
* Refactor native histogram tests
* Track and reset the buckets for which exemplars have been recorded
* Split multiplier test into integer and floating point
* Fix expectedSeriesCount and examplars in test
* Drop expectedSeriesCount and just len of expectedSamples instead
* Reduce log spam
* Apply existing interface constraint to Gauge and Counter
* Update test instances for metric interface
* Use int64 in place of atomic since always under lock
* Lint for interface updates
* Drop series[0] check
* Convert override from bool to string and test values
* Push mode func into native histograms implementation for update handling
* Drop unused variable
* Set --enable-feature=native-histograms on all prometheus docker-compose setups
* Update generate_native_histograms setting
---------
Co-authored-by: Koenraad Verheyden <koenraad.verheyden@grafana.com>
Co-authored-by: yuna <yuna.verheyden@posteo.net>
* introduce new service-graph metric for messaging-system latency
* added tests for new histogram values
* fix linting
* make new metric optional via config
* fix typo
* fix failing tests
* add feature to changelog
* negative times diff consistency - return 0 instead of negative
* update docs
* Update docs/sources/tempo/metrics-generator/service_graphs/estimate-cardinality.md
use present when possible
Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com>
* change 1e9 to time const
* added a reference to the "wait" config of the processor
* fixed indentations and formatting stuff from rebasing
* removed mistaken println found by linter
---------
Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com>
* Surface new labels for uninstrumented services and systems
* Update CHANGELOG.md
* remove unnecessary Println in test
* Reuse dimensions and prefixes for this use-case
* Add docs
* keep only virtual_node behind the new feature
* add overrides
* Update docs/sources/tempo/metrics-generator/service_graphs/_index.md
Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com>
* forgot this test (fixed)
* add benchmarks
* add edge pooling/reuse
* update flag
* move label out of dimensions to avoid prefix logic
* lint
* Update modules/generator/processor/servicegraphs/config.go
Co-authored-by: Mario <mariorvinas@gmail.com>
* minor amends to names and docs
* leave the new virtual_node label as an extra dimension
* keep edge sync.Pool ops inside store
* Update modules/generator/processor/servicegraphs/store/store.go
The edge is not expired here, so it shouldn't be returned to the pool.
Co-authored-by: Mario <mariorvinas@gmail.com>
* leave the new label un-prefixed
---------
Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com>
Co-authored-by: Mario <mariorvinas@gmail.com>
* print limit according to ingestion rate strategy
* clean
* overrides always return default overrides for ingestion rate strategy
* remove test logs
* babo
* add both local and global limits
* Update traceql metrics to use the trace-level timestamp columns conditionally
* comments
* Update benchmark, comment
* lint
* Change overlap cutoff to 20%
* add more instrumentation and a little cleanup
* Add many new hints, with the unsafe hints enabled by per-tenant flag
* Collect all hints in one place, decomplicate code
* Make func signature golang convention
* make querier block_concurrency configurable
* changelog
* Make time_overlap_cutoff configurable. Rename block_concurrency to concurrent_blocks to match naming convention of other concurrent_* fields.
* fix test
* Add support for per-tenant compaction window
* Update legacy round trip
* Metric the override config value
* Drop comment
* Use default config window in case no override for tenant
* Add docs for compaction_window
* Add test for round-trip
* Drop metric for default overrides compaction window
* Update changelog
* Update grafana/dskit and google/grpc
* Updates for dskit usage
* Update serverless go.mod
* Avoid localhost name lookup in memberlist test
* Update serverless go.mod for grpc
* Standardise error formatting using standard library errors package
* Standardise improve non-idiomatic error messages
* Make github.com/pkg/errors an indirect dependency
Co-authored-by: A. Stoewer <adrian@stoewer.me>
* testing
* undo logs
* make mg ingestion slack per tenant
* changelog
* fix label and updated docs
* move mg slack to update processor
* store duration correctly
* oops
* rebase
* make it spicy level: atomic
* use uber/atomic
* up up and away
* undo limits.go
* testing
* undo logs
* add config to drop certain labels from target info
* comment and refactor
* documentation
* hopefully improve perf
* rebase
* lint
* oops
* lint
* [vParquet3] create new block encoding by copying vParquet2
* vParquet3: add dedicated columns to parquet schema and block meta (#2517)
* Re-order schema to keep columns affected by column index changes low
* Add spare columns for dedicated attributes to schema struct
* Add dedicated column config to block meta
* Read and write attributes in dedicated columns
* Make order of dedicated attributes predictable when reading
* Fix existing tests and benchmark
* Run exiting benchmarks and tests with dedicated columns
Co-authored-by: Mario <mariorvinas@gmail.com>
* Add dedicated columns to overrides module (#2551)
* [vParquet3] Write path (#2555)
* Add dedicated columns to overrides and blocks
* Improvements
* Change test
* Fix tests
* Extend ingester_test:
* Add dedicated columns config to storage block
* Review comments
* Add comment
* [vParquet3] dedicated columns read path (#2592)
* Refactor and rename function blockMetaToDedicatedColumnMapping
* Query dedicated attribute columns with TraceQL
* Search tag values in dedicated attribute columns
* Search tags in dedicated attribute columns
* Search for values in dedicated attribute columns in tests
* More consistent naming
* Update block and meta.json in vparquet2/test-data
* Test dedicated column in traceToParquet test
* Format Go code
* Introduce types for dedicated column type and scope
Replace StaticTypeFromString() with DedicatedColumnType.ToStaticType()
* The function dedicatedColumnsToColumnMapping() can receive multiple scopes
* [vParquet3] Add support for dedicated columns in compactor (#2561)
* Re-order schema to keep columns affected by column index changes low
* Add spare columns for dedicated attributes to schema struct
* Add dedicated column config to block meta
* Read and write attributes in dedicated columns
* Make order of dedicated attributes predictable when reading
* Fix existing tests and benchmark
* Run exiting benchmarks and tests with dedicated columns
* Add dedicated columns to overrides and blocks
* Support dedicated columns in compactor block selection
* Changes to hash
* More tests
---------
Co-authored-by: A. Stoewer <adrian@stoewer.me>
* [vParquet3] pass dedicated columns to querier (#2603)
* Add dedicated columns to SearchBlockRequest message
* Assign SearchBlockRequest dedicated cols from BlockMeta and vice versa
* Encode SearchBlockRequest to http request and vice versa
* Don't add empty dedicated columns when building a search request
* Unit tests with dedicated columns
* Implement dedicated column scope and type as protobuf enums
* [vParquet3] validate dedicated columns configuration (#2616)
* Add validate function
* Refactor: use DedicatedColumns type instead of []DedicatedColumn
* Initialize logger before verifying the config
This fixes the config verification output
* Check for invalid dedicated columns with '-config.verify true'
* Use ToTempopb() to validate dedicated column scope and type
* [vParquet3] mention feature in CHANGELOG.md
* [vParquet3] Address review comments
* Remove TODO comment about caching the dedicated column hash
* Shorten url param for dedicated columns to 'dc'
* Add function to get latest encoding and use it in tests
* Fix name DedicateColumnsFromTempopb
* [vParquet3] Address more review comments
* Remove 'Test' columns from vParquet3 schema
* Rename async iterator environment variable
* Do not export methods of dedicatedColumnMapping
* Skip dedicated attribute lookup depending on scope in searchTagValues
* Validate maximum number of configured dedicated columns
* Test data for vparquet3 uses dedicated columns
* Reduce size of block meta JSON
* Use 'parquet_' prefix for dedicated column configuration
* [vParquet3] Integration tests with dedicated attribute columns
* Add e2e tests for encodings and dedicated attribute columns
* Use dedicated attribute columns in TestSearchCompleteBlock
* Add support for v2 in encodings test
---------
Co-authored-by: Mario <mariorvinas@gmail.com>
* Include gofumpt and goimports in tools
* Replace gofmt with gofumpt in Makefile
* Run `make fmt`
* Adhere to goconst lint rule
* Include gofumpt note in the CONTRIBUTING.md
* Update CHANGELOG
* Add user-configurable overrides module
* Add /api/overrides and fix crash on boot
* Add overridesHandler and WriteStatusRuntimeConfig
* return json and only return overrides for the tenant
* Implement delete
* Fix test I think?
* Fix tests
* clean up handler and TODOs
* Add tests for overridesHandler
* Add e2e test
* Refactor:
- clean up integration tests for overrides
- rename ReloadInterval -> PollInterval
- linting
* Linting
* address more Linting
* fix lint and add test for PATCH
* fix lint error unparam
* remove todo
* use tenantLimits as return type
* Add prometheus.Collector to overrides.Interface
* Test tempo_overrides_user_configurable_overrides_fetch_total metric in e2e tests
* Sprinkle in some tracing
* Update CHANGELOG.md
* Rename loop to running for consistency
* Have mux handle method routing; split up GET, POST and DELETE handlers
* Use built in contains
* Split up user-configurable overrides manager, api and backend client
* Move overrides API to httpclient
* Clean up, linting, fmt
* Remove version field from json
* Address review comments
* If overrides.json does not exist, properly delete it from cache
* Add config warning for conflicting storage
* Check in my tests as well
* Simplify API handler, return 404 on overrides not found
* Typo, linting, fix test
* Use backend constants
---------
Co-authored-by: Suraj Nath <9503187+electron0zero@users.noreply.github.com>
