41 Commits

Author SHA1 Message Date
2e7cd0fe22 chore(coderd/database/dbpurge): remove dbmem from tests (#18151)
Related to https://github.com/coder/coder/issues/15109
2025-06-02 16:48:38 +02:00
6c0bed0f53 chore: update to coder/quartz v0.2.0 (#18007)
Upgrade to coder/quartz v0.2.0 including fixing up a minor API breaking change.
2025-05-27 16:05:03 +04:00
04c33968cf refactor: replace golang.org/x/exp/slices with slices (#16772)
The experimental functions in `golang.org/x/exp/slices` are now
available in the standard library since Go 1.21.

Reference: https://go.dev/doc/go1.21#slices

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2025-03-04 00:46:49 +11:00
44499315ed chore: reduce log volume on server startup (#16608)
Addresses https://github.com/coder/coder/issues/16231.

This PR reduces the volume of logs we print after server startup in
order to surface the web UI URL better.

Here are the logs after the changes a couple of seconds after starting
the server:

<img width="868" alt="Screenshot 2025-02-18 at 16 31 32"
src="https://github.com/user-attachments/assets/786dc4b8-7383-48c8-a5c3-a997c01ca915"
/>

The warning is due to running a development site-less build. It wouldn't
show in a release build.
2025-02-20 16:33:14 +01:00
6ea5c6f0ef fix: show user-auth provisioners for all organizations (#16350) 2025-01-30 14:08:27 -07:00
c069563af1 test: fix use of t.Logf where t.Log would suffice (#16328) 2025-01-29 14:35:04 +00:00
7b88776403 chore(testutil): add testutil.GoleakOptions (#16070)
- Adds `testutil.GoleakOptions` and consolidates existing options to
this location
- Pre-emptively adds required ignore for this Dependabot PR to pass CI
https://github.com/coder/coder/pull/16066
2025-01-08 15:38:37 +00:00
83c493e832 chore: fix more flaky tests on Windows with Postgres (#15629)
Addresses the following flakes:

- https://github.com/coder/internal/issues/222
- https://github.com/coder/internal/issues/223
- https://github.com/coder/internal/issues/224
- https://github.com/coder/internal/issues/225
- https://github.com/coder/internal/issues/226
- https://github.com/coder/internal/issues/227
- https://github.com/coder/internal/issues/228
- https://github.com/coder/internal/issues/229
- https://github.com/coder/internal/issues/230
2024-11-26 11:56:07 +01:00
5861e516b9 chore: add standard test logger ignoring db canceled (#15556)
Refactors our use of `slogtest` to instantiate a "standard logger" across most of our tests.  This standard logger incorporates https://github.com/coder/slog/pull/217 to also ignore database query canceled errors by default, which are a source of low-severity flakes.

Any test that has set non-default `slogtest.Options` is left alone. In particular, `coderdtest` defaults to ignoring all errors. We might consider revisiting that decision now that we have better tools to target the really common flaky Error logs on shutdown.
2024-11-18 14:09:22 +04:00
ccfffc6911 chore: add tx metrics and logs for serialization errors (#15215)
Before db_metrics were all or nothing. Now `InTx` metrics are always recorded, and query metrics are opt in.


Adds instrumentation & logging around serialization failures in the database.
2024-10-25 12:14:15 -04:00
343f8ec9ab chore: join owner, template, and org in new workspace view (#15116)
Joins in fields like `username`, `avatar_url`, `organization_name`,
`template_name` to `workspaces` via a **view**. 
The view must be maintained moving forward, but this prevents needing to
add RBAC permissions to fetch related workspace fields.
2024-10-22 09:20:54 -05:00
335eb05223 feat: add keys to organization provision daemons (#14627) 2024-09-16 20:02:08 +00:00
0f8251be41 feat(coderd/database/dbpurge): retain most recent agent build logs (#14460)
Updates the `DeleteOldWorkspaceAgentLogs` to:
- Retain logs for the most recent build regardless of age,
- Delete logs for agents that never connected and were created before
   the cutoff for deleting logs while still retaining the logs most recent build.
2024-08-30 17:39:09 +01:00
a74273f1fd chore(coderd/database/dbpurge): replace usage of time.* with quartz (#14480)
Related to #10576

This PR introduces quartz to coderd/database/dbpurge and updates the following unit tests to make use of Quartz's functionality:

- TestPurge
- TestDeleteOldWorkspaceAgentLogs

Additionally, updates DeleteOldWorkspaceAgentLogs to replace the hard-coded interval with a parameter passed into the query. This aids in testing and brings us a step towards allowing operators to configure the cutoff interval for workspace agent logs.
2024-08-30 11:55:47 +01:00
628750232f fix: delete workspace agent stats after 180 days (#14489)
Fixes #13430.

The test for purging old workspace agent stats from the DB was consistently failing when ran with Postgres towards the end of the month, but not with the in-memory DB. 

This was because month intervals are calculated differently for `time.Time` and the `interval` type in Postgres:

```
ethan=# SELECT
    '2024-08-30'::DATE AS original_date,
    ('2024-08-30'::DATE - INTERVAL '6 months') AS sub_date;
 original_date |      sub_date
---------------+---------------------
 2024-08-30    | 2024-02-29 00:00:00
(1 row)
```

Using `func (t Time) AddDate(years int, months int, days int) Time`, where `months` is `-6`:
```
Original: 2024-08-30 00:00:00 +0000 UTC
6 Months Earlier: 2024-03-01 00:00:00 +0000 UTC
```

Since 6 months was chosen arbitrarily, we should be able to change it to 180 days, to remove any ambiguity between the in-memory DB, and the Postgres DB. The alternative solution would involve implementing Postgres' month interval algorithm in Go.

The UI only shows stats as old as 168 days (24 weeks), so a frontend change isn't required for the extra days of stats we lose in some cases.
2024-08-30 18:30:04 +10:00
70046ea08d fix: missing nolint comment (#13862) 2024-07-10 09:55:30 +00:00
54898033b1 fix: dbpurge: disable parallel tests (#13848) 2024-07-09 15:10:57 +02:00
10aa32ca08 chore: refactor AgentHasNotConnectedSinceWeek_LogsExpired (#13802) 2024-07-08 13:52:56 +02:00
0a221e8d5b feat: create database tables and queries for notifications (#13536) 2024-06-28 09:21:25 +00:00
619ec927e9 test(coderd/database): fix DST issue in dbpurge test (#13170)
Fixes #13165
2024-05-06 14:14:38 +03:00
886a97b425 chore: fix build ci (#13164) 2024-05-06 05:01:47 +00:00
3adcccb618 fix(coderd/database): reduce db load via dbpurge advisory locking (#13021) 2024-04-22 11:10:32 +00:00
e17e8aa3c9 feat(coderd/database): keep only 1 day of workspace_agent_stats after rollup (#12674) 2024-04-22 13:11:50 +03:00
12e6fbf11e feat(coderd/database): add dbrollup service to rollup insights (#12665)
Add `dbrollup` service that runs the `UpsertTemplateUsageStats` query
every 5 minutes, on the minute. This allows us to have fairly real-time
insights data when viewing "today".
2024-03-22 18:42:43 +02:00
bae0a747ed test(coderd): skip flaky dau test (#12517)
* test(coderd): skip flaky dau test

* chore(coderd/database/dbpurge): fix failing test (#12530)

---------

Co-authored-by: Cian Johnston <cian@coder.com>
2024-03-11 12:54:38 +00:00
b5f866c1cb chore: add organization_id column to provisioner daemons (#12356)
* chore: add organization_id column to provisioner daemons
* Update upsert to include organization id on set
2024-03-06 12:04:50 -06:00
475c3650ca feat: add support for optional external auth providers (#12021) 2024-02-21 11:18:38 -07:00
643c3ee54b refactor(provisionerd): move provisionersdk.VersionCurrent -> provisionerdproto.VersionCurrent (#12225) 2024-02-20 12:44:19 +00:00
04fd96a014 feat(coderd): add provisioner_daemons to /debug/health endpoint (#11393)
Adds a healthcheck for provisioner daemons to /debug/health endpoint.
2024-01-08 09:29:04 +00:00
1ef96022b0 feat(coderd): add provisioner build version and api_version on serve (#11369)
* assert provisioner daemon version and api_version in unit tests
* add build info in HTTP header, extract codersdk.BuildVersionHeader
* add api_version to codersdk.ProvisionerDaemon
* testutil.MustString -> testutil.MustRandString
2024-01-03 09:01:57 +00:00
5b0e6bfa2a feat(coderd/database): add api_version to provisioner_daemons table (#11204)
Adds column api_version to the provisioner_daemons table.
This is distinct from the coderd version, and is used to handle breaking changes in the provisioner daemon API.
2023-12-14 12:52:41 +00:00
4f7ae6461b feat(coderd/database): add UpsertProvisionerDaemons query (#11178)
Co-authored-by: Marcin Tojek <marcin@coder.com>
2023-12-13 12:31:40 +00:00
b02796655e fix(coderd/database): remove column updated_at from provisioner_daemons table (#11108) 2023-12-12 11:19:28 +00:00
941e3873a8 fix: implement fake DeleteOldWorkspaceAgentStats (#11076) 2023-12-07 14:08:16 +01:00
f6891bc465 fix: implement fake DeleteOldWorkspaceAgentLogs (#11042) 2023-12-06 14:31:43 +01:00
13b89f79df feat: purge old provisioner daemons (#10949) 2023-12-01 12:43:05 +00:00
5abfe5afd0 chore: rename dbfake to dbmem (#10432) 2023-10-30 17:42:20 +00:00
22e781eced chore: add /v2 to import module path (#9072)
* chore: add /v2 to import module path

go mod requires semantic versioning with versions greater than 1.x

This was a mechanical update by running:
```
go install github.com/marwan-at-work/mod/cmd/mod@latest
mod upgrade
```

Migrate generated files to import /v2

* Fix gen
2023-08-18 18:55:43 +00:00
bd944e0d21 chore: rename startup logs to agent logs (#8649)
* chore: rename startup logs to agent logs

This also adds a `source` property to every agent log. It
should allow us to group logs and display them nicer in
the UI as they stream in.

* Fix migration order

* Fix naming

* Rename the frontend

* Fix tests

* Fix down migration

* Match enums for workspace agent logs

* Fix inserting log source

* Fix migration order

* Fix logs tests

* Fix psql insert
2023-07-28 15:57:23 +00:00
407f80a1ef fix(dbpurge): use dbauthz.AsSystemRestricted (#7017) 2023-04-05 17:48:39 +00:00
cb7375450b feat: add startup script logs to the ui (#6558)
* Add startup script logs to the database

* Add coderd endpoints for startup script logs

* Push startup script logs from agent

* Pull startup script logs on frontend

* Rename queries

* Add constraint

* Start creating log sending loop

* Add log sending to the agent

* Add tests for streaming logs

* Shorten notify channel name

* Add FE

* Improve bulk log performance

* Finish UI display

* Fix startup log visibility

* Add warning for overflow

* Fix agent queue logs overflow

* Display staartup logs in a virtual DOM for performance

* Fix agent queue with loads of logs

* Fix authorize test

* Remove faulty test

* Fix startup and shutdown reporting error

* Fix gen

* Fix comments

* Periodically purge old database entries

* Add test fixture for migration

* Add Storybook

* Check if there are logs when displaying features

* Fix startup component overflow gap

* Fix startup log wrapping

---------

Co-authored-by: Asher <ash@coder.com>
2023-03-23 14:09:13 -05:00