* remove tempo-serverless from cmd
* cleanup Makefile and .gitignore
* remove serverless code from pkg/api/
* remove serverless code from the querier
* clean up serverless from tempo-mixin and playbook
* Remove serverless from tempo docs
* Update changelog
* go mod vendor
* Remove tempo_feature_enabled metric
* inline internalSearchBlock
* docs alias
* First working draft of cost attribution usage tracker
* Add missing tracker name label, more efficient batch proportioning, cleanup
* Reduce series hashing
* Fix user-configurable overrides tests for new json element
* lint
* Add per-tenant override for max cardinality
* lint, review feedback
* Default to not enabled, cleanup test config
* Explicitly check for usage_metrics handler
* review feedback
* Update tracker to support many-to-one mapping with relabel
* lint
* New behavior for missing and overflow
* Fix issue where subsequent spans would incorrectly reuse the series of the previous span if they were missing values
* Revert maps back to slices now that we can depend on a dimension always having a value
* Please ignore benchmark profiles
* Tweak config to have specific cost attribution tracker section. Update manifest and config index
* lint
* changelog
* Update api docs for new endpoint
* Review feedback
* review feedback
* Swap loop order for a tad more performance
propagate the bytes read from the storage layer to the frontend, and use bytes read to compute throughout, and use that in SLO computation for metadata endpoints.
metadata SLO thresholds can be configured via `metadata_slo` config.
```
search:
metadata_slo:
duration_slo: 5s
throughput_bytes_slo: 1.073741824e+09
```
we will also return the `metrics` in the response of all the metadata endpoints:
- `/search/tags`
- `/v2/search/tags`
- `/search/tag/<tagName>/values`
- `/v2/search/tag/<tagName>/values`
here is what the it looks like in the response:
```json
{
"<existing keys>": "<existing response>",
"metrics": {
"inspectedBytes": "630188"
}
}
```
we return `"metrics": {}` when response is empty or only contains `intrinsics`
it will also expose these new label to existing metrics with `op="metadata"` label
- total metadata queries counter
- metadata queries within SLO counter
- metadata queries throughput histogram
* add local pprof files in .gitignore
* fast distinct_string_collector
* faster scoped_distinct_string collector
* Update the usage of the combiners
* add a benchmark for duplicates
* Add benchmarks
* Update CHANGELOG.md
* return error on calls to Diff with diff disabled
* return exceeded bool on Collect in ScopedDistinctString
* test collect return and Exceeded
* Add a note about DistinctString collector's Collect return bool
* only new in withDiff versions
* remove extra note
* move the lock down in the Diff method
* fix flaky test
* ingore /tmp dir in the root
* instrument localblock seacher methods
* insturment walBlock
* disk cache works locally
* fix fmt
* fix lint
* refactor the disk caching logic
* add tests and cleanup
* update CHANGELOG.md
* address review comments
* cacheKey without start and end
* rename batches property of Trace to ResourceSpans to be OTEL compatible
* added test to assert that the response hasn't change
* update test data
* fix unmarshall test
* changelog
* fix e2e
* try to fix flaky test
* create helper functions to marshal and unmarshal traces to OTEL compatible schema
* added better documentation
* Add combiners to frontend
---------
Co-authored-by: Mario <mariorvinas@gmail.com>
* Add MultiTenantQueriesEnabled config in frontend
* Add MultiTenantMiddleware and tests
---------
Co-authored-by: Mario <mariorvinas@gmail.com>
* Expose dedicated handler in queryfrontend for each http endpoint
* Add multi-tenant middleware in multi-tenant routes
* ignore e2e test folder
* fix up
* search methods on client
* Add e2e test TestMultiTenantSearch
* Add NewSearchTagsV2 combiner
* test are passing...
* test cleanup
* docs & todos
* make lint happy
* use distinct value collector
* Add docs
* remove callback from combiner AddRequest
* leave a todo to merge CombineSearchResults
* return multi-tenant query unsupported for metrics endpoint
* add a note on testing search streaming locally
* update docs for lowercase headers
* streaming search endpoints unsupported error
* multi-tenant works in streaming search? who knows?
* remove todo
* update gitignore & local docker-compose file
* test unsupported endpoints in e2e search
* Update CHANGELOG.md
* Add tests for unsupprted endpoints
* more cleanup
* fix lint
* fix golangci
* assert response for tags endpoints
* docs feedback
Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com>
* reword and fix docs
---------
Co-authored-by: Mario <mariorvinas@gmail.com>
Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com>
* Add traceid index to vparquet2 and use it when finding trace by id
* Update to detect when trace ID is beyond the last row group and bail
* simplify
* Add index to vparquet3
* go away
* lint
* changelog
* Fix index to not flush duplicate entry, which could happen if the block ended on exactly the configured rowgroup size while flushing
* the opposite of move fast and break things
* more optionalality
* Add new env var VPARQUET_INDEX=0 to disable the index lookup
* Remove forced volume mounts for generator deployment which prevents downstream jsonnet from changing them via $.tempo_metrics_generator_container, make volume mounts idential between deployment and statefulset
* Sort gitignore and add more paths
* Fix command and regen output
* changelog
* parse ast
Signed-off-by: Joe Elliott <number101010@gmail.com>
* stringer
Signed-off-by: Joe Elliott <number101010@gmail.com>
* Union and sibling
Signed-off-by: Joe Elliott <number101010@gmail.com>
* disallow static/static comparisons
Signed-off-by: Joe Elliott <number101010@gmail.com>
* added status type
Signed-off-by: Joe Elliott <number101010@gmail.com>
* attributes
Signed-off-by: Joe Elliott <number101010@gmail.com>
* test cleanup
Signed-off-by: Joe Elliott <number101010@gmail.com>
* made names slightly less bad
Signed-off-by: Joe Elliott <number101010@gmail.com>
* stringer tests
Signed-off-by: Joe Elliott <number101010@gmail.com>
* removed empty {}
Signed-off-by: Joe Elliott <number101010@gmail.com>
* started validate. stringer
Signed-off-by: Joe Elliott <number101010@gmail.com>
* Unwound recursive relationships in grammar
Signed-off-by: Joe Elliott <number101010@gmail.com>
* Added TestAll
Signed-off-by: Joe Elliott <number101010@gmail.com>
* Validate
Signed-off-by: Joe Elliott <number101010@gmail.com>
* Fix attribute parsing
Signed-off-by: Joe Elliott <number101010@gmail.com>
* cleaned up image definitions
Signed-off-by: Joe Elliott <number101010@gmail.com>
* vendor check
Signed-off-by: Joe Elliott <number101010@gmail.com>
* go.mod
Signed-off-by: Joe Elliott <number101010@gmail.com>
* enums
Signed-off-by: Joe Elliott <number101010@gmail.com>
* removed stringer maps
Signed-off-by: Joe Elliott <number101010@gmail.com>
* nolint
Signed-off-by: Joe Elliott <number101010@gmail.com>
* Improved tests and cleanup
Signed-off-by: Joe Elliott <number101010@gmail.com>
* Added static expressions in scalar filters
Signed-off-by: Joe Elliott <number101010@gmail.com>
* adjusted examples
Signed-off-by: Joe Elliott <number101010@gmail.com>
* regenerated expr.y
Signed-off-by: Joe Elliott <number101010@gmail.com>
* require aggregates and grouping expressions to reference the span
Signed-off-by: Joe Elliott <number101010@gmail.com>
* added todo
Signed-off-by: Joe Elliott <number101010@gmail.com>
* Added basic structure for compression
Signed-off-by: Joe Elliott <number101010@gmail.com>
* AppenderBuffered impl
Signed-off-by: Joe Elliott <number101010@gmail.com>
* mild commenting
Signed-off-by: Joe Elliott <number101010@gmail.com>
* moved encoding to meta json
Signed-off-by: Joe Elliott <number101010@gmail.com>
* Added pagedReader and backend.ReaderAt
Signed-off-by: Joe Elliott <number101010@gmail.com>
* dedupingfinder => pagedfinder
Signed-off-by: Joe Elliott <number101010@gmail.com>
* Abstracted away index reading. Used pagedFinder in backend block
Signed-off-by: Joe Elliott <number101010@gmail.com>
* Abstracted PagedIterator to use indexReader
Signed-off-by: Joe Elliott <number101010@gmail.com>
* improved comments
Signed-off-by: Joe Elliott <number101010@gmail.com>
* created versioned encoding
Signed-off-by: Joe Elliott <number101010@gmail.com>
* removed findmetrics
Signed-off-by: Joe Elliott <number101010@gmail.com>
* Moved backend block to encoding
Signed-off-by: Joe Elliott <number101010@gmail.com>
* Moved indexreader records to common
Signed-off-by: Joe Elliott <number101010@gmail.com>
* Added index reader
Signed-off-by: Joe Elliott <number101010@gmail.com>
* jpe cleanup
Signed-off-by: Joe Elliott <number101010@gmail.com>
* v1 LIVES
Signed-off-by: Joe Elliott <number101010@gmail.com>
* Tests pass on v1
Signed-off-by: Joe Elliott <number101010@gmail.com>
* encoding tests pass
Signed-off-by: Joe Elliott <number101010@gmail.com>
* wooo snappy tests
Signed-off-by: Joe Elliott <number101010@gmail.com>
* Moved block config to its own struct
Signed-off-by: Joe Elliott <number101010@gmail.com>
* Add custom marshaller
Signed-off-by: Joe Elliott <number101010@gmail.com>
* updated configs added default
Signed-off-by: Joe Elliott <number101010@gmail.com>
* Test all encodings
Signed-off-by: Joe Elliott <number101010@gmail.com>
* Added benchmark. Default to snappy
Signed-off-by: Joe Elliott <number101010@gmail.com>
* fixed panic in reader
Signed-off-by: Joe Elliott <number101010@gmail.com>
* Fleshed out benchmarks
Signed-off-by: Joe Elliott <number101010@gmail.com>
* lint
Signed-off-by: Joe Elliott <number101010@gmail.com>
* backend_block => benchmark_block
Signed-off-by: Joe Elliott <number101010@gmail.com>
* Added 256 and 1M LZ4 tests
Signed-off-by: Joe Elliott <number101010@gmail.com>
* Removed findmetrics from operational
Signed-off-by: Joe Elliott <number101010@gmail.com>
* indexreaderbytes => indexreader
Signed-off-by: Joe Elliott <number101010@gmail.com>
* jpe cleanup
Signed-off-by: Joe Elliott <number101010@gmail.com>
* Fixed bytes flushed metric
Signed-off-by: Joe Elliott <number101010@gmail.com>
* Changed default encoding
Signed-off-by: Joe Elliott <number101010@gmail.com>
* Pass tracker correctly. Add guard code
Signed-off-by: Joe Elliott <number101010@gmail.com>
* Added marshal tests
Signed-off-by: Joe Elliott <number101010@gmail.com>
* lint
Signed-off-by: Joe Elliott <number101010@gmail.com>
* page reader tests
Signed-off-by: Joe Elliott <number101010@gmail.com>
* actually add tests
Signed-off-by: Joe Elliott <number101010@gmail.com>
* Added reader tess
Signed-off-by: Joe Elliott <number101010@gmail.com>
* changelog
Signed-off-by: Joe Elliott <number101010@gmail.com>
* zstd
Signed-off-by: Joe Elliott <number101010@gmail.com>
* vendor
Signed-off-by: Joe Elliott <number101010@gmail.com>
* Convert tempo-cli to kong cli parser, add some basic command structure
* Rename commands to 'list block' 'list bucket'
* Update list bucket command to put index analysis behind --load-index flag, print results more dynamically
* Rename list bucket to list blocks, improve parameter descriptions and locations
* vendor-check fixes
* ignore tempo-cli binary
* Fix lint re: unused values
* Add tempo-cli section to docs website. Update readme.md to point to docs for tempo-cli
* Apply suggestions from code review
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
* Apply suggestions from code review
* Update changelog
* Wording tweak based on feedback
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
* Update blocklist in memory for blocks add/removed/compacted by compactor, to reduce compaction noise/errors
* Remove unused parameter to updateBlocklist
* Test to ensure traces are still Find'able after compaction
* Change updateBlocklist to compare blockmeta by id instead of pointer. Add tests
* Ignore .vscode
* Update changelog
* Update docker-compose local example for clarity and compatibility (#287)
* Revamp docker-compose example folder: make local storage example the default. Move static config files to ./etc/ folder. Move tempo and other storage to ./example-data/ folder
* Update gitignore path for new example path
* Added goreleaser steps. Cleaned up github workflows
Signed-off-by: Joe Elliott <number101010@gmail.com>
* Added releaser
Signed-off-by: Joe Elliott <number101010@gmail.com>
* Add release-process for testing
Signed-off-by: Joe Elliott <number101010@gmail.com>
* Removed vendor-deps. They should already be current and present
Signed-off-by: Joe Elliott <number101010@gmail.com>
* changed name for clarity
Signed-off-by: Joe Elliott <number101010@gmail.com>
* Restrict goreleaser to v tags
Signed-off-by: Joe Elliott <number101010@gmail.com>
* Added build folder
Signed-off-by: Joe Elliott <number101010@gmail.com>
* removed windows
Signed-off-by: Joe Elliott <number101010@gmail.com>
* Added dist to gitignore
Signed-off-by: Joe Elliott <number101010@gmail.com>
* Removed test release-process branch
Signed-off-by: Joe Elliott <number101010@gmail.com>
