Commit Graph

979 Commits

Author SHA1 Message Date
e1df5368a5 add tests for prebuilds
finalise database integration tests for prebuilds

reintegrate with danny's latest changes

add back assertions for deletion integration tests of prebuilds

tidy up prebuilds tests
2025-03-05 08:30:27 +00:00
f3a5b8e4d8 Merge branch 'main' of github.com:/coder/coder into dk/prebuilds
Signed-off-by: Danny Kopping <dannykopping@gmail.com>
2025-03-04 10:08:54 +00:00
24f3445e00 chore: track workspace resource monitors in telemetry (#16776)
Addresses https://github.com/coder/nexus/issues/195. Specifically, just
the "tracking templates" requirement:

> ## Tracking in templates
> To enable resource alerts, a user must add the resource_monitoring
block to a template's coder_agent resource. We'd like to track if
customers have any resource monitoring enabled on a per-deployment
basis. Even better, we could identify which templates are using resource
monitoring.
2025-03-03 18:41:01 +01:00
b85ba586ee fix(coderd/database): consider tag sets when calculating queue position (#16685)
Relates to https://github.com/coder/coder/issues/15843

## PR Contents

- Reimplementation of the `GetProvisionerJobsByIDsWithQueuePosition` SQL
query to **take into account** provisioner job tags and provisioner
daemon tags.
- Unit tests covering different **tag sets**, **job statuses**, and
**job ordering** scenarios.

## Notes

- The original row order is preserved by introducing the `ordinality`
field.
- Unnecessary rows are filtered as early as possible to ensure that
expensive joins operate on a smaller dataset.
- A "fake" join with `provisioner_jobs` is added at the end to ensure
`sqlc.embed` compiles successfully.
- **Backward compatibility is preserved**—only the SQL query has been
updated, while the Go code remains unchanged.
2025-03-03 10:02:18 -05:00
df743e6961 Add prebuild claim test
Some checks failed
Deploy PR / check_pr (push) Has been cancelled
Deploy PR / get_info (push) Has been cancelled
Deploy PR / comment-pr (push) Has been cancelled
Deploy PR / build (push) Has been cancelled
Deploy PR / deploy (push) Has been cancelled
Signed-off-by: Danny Kopping <dannykopping@gmail.com>
2025-03-03 14:29:56 +00:00
04c33968cf refactor: replace golang.org/x/exp/slices with slices (#16772)
The experimental functions in `golang.org/x/exp/slices` are now
available in the standard library since Go 1.21.

Reference: https://go.dev/doc/go1.21#slices

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2025-03-04 00:46:49 +11:00
88f0131abb fix: use dbtime in dbmem query to fix flake (#16773)
Closes https://github.com/coder/internal/issues/447.

The test was failing 30% of the time on Windows without the rounding
applied by `dbtime`. `dbtime` was used on the timestamps inserted into
the DB, but not within the query. Once using `dbtime` within the query
there were no failures in 200 runs.
2025-03-03 13:42:13 +00:00
59d406f3b2 Merge branch 'dk/prebuilds' of github.com:/coder/coder into dk/prebuilds-tests 2025-03-03 12:46:44 +00:00
e16d763241 add tests to ensure that preubilds are correctly provisioned for active template versions 2025-03-03 10:20:47 +00:00
c074f77a4f feat: add notifications inbox db (#16599)
This PR is linked [to the following
issue](https://github.com/coder/internal/issues/334).

The objective is to create the DB layer and migration for the new `Coder
Inbox`.
2025-03-03 10:12:48 +01:00
c9736f26af For the love of god, wrap errors!
Signed-off-by: Danny Kopping <dannykopping@gmail.com>
2025-02-28 15:36:23 +00:00
91a4a98c27 chore: add an unassign action for roles (#16728) 2025-02-27 10:39:06 -07:00
6dd51f92fb chore: test metricscache on postgres (#16711)
metricscache_test has been running tests against dbmem only, instead of
against postgres. Unfortunately the implementations of
GetTemplateAverageBuildTime have diverged between dbmem and postgres.
This change gets the tests working on Postgres and test for the
behaviour postgres provides.
2025-02-27 09:43:51 +00:00
7cd6e9cdd6 fix: return provisioners in desc order and add limit to cli (#16720) 2025-02-26 21:06:51 +02:00
5e32ed22ac Merge branch 'main' of github.com:coder/coder into dk/prebuilds
Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-26 14:31:56 +02:00
98dfc70f31 fix(coderd/database): remove linux build tags from db package (#16633)
Remove linux build tags from database package to make sure we can run
tests on Mac OS.
2025-02-25 11:39:37 -05:00
d3a56ae3ef feat: enable GitHub OAuth2 login by default on new deployments (#16662)
Third and final PR to address
https://github.com/coder/coder/issues/16230.

This PR enables GitHub OAuth2 login by default on new deployments.
Combined with https://github.com/coder/coder/pull/16629, this will allow
the first admin user to sign up with GitHub rather than email and
password.

We take care not to enable the default on deployments that would upgrade
to a Coder version with this change.

To disable the default provider an admin can set the
`CODER_OAUTH2_GITHUB_DEFAULT_PROVIDER` env variable to false.
2025-02-25 16:31:33 +01:00
4e1e745912 add prebuild metrics and observability
Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-25 11:17:10 +00:00
e005e4e51d chore: merge provisioner key and provisioner permissions (#16628)
Provisioner key permissions were never any different than provisioners.
Merging them for a cleaner permission story until they are required (if
ever) to be seperate.

This removed `ResourceProvisionerKey` from RBAC and just uses the
existing `ResourceProvisioner`.
2025-02-24 13:31:11 -06:00
546a549dcf feat: enable soft delete for organizations (#16584)
- Add deleted column to organizations table
- Add trigger to check for existing workspaces, templates, groups and
members in a org before allowing the soft delete

---------

Co-authored-by: Steven Masley <stevenmasley@gmail.com>
Co-authored-by: Steven Masley <Emyrk@users.noreply.github.com>
2025-02-24 12:59:41 -05:00
fad3f23791 Correct queries; running prebuilds only needs to return current preset ID
Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-24 16:19:08 +00:00
c88c04c022 Determine latest chosen preset using latest non-null preset selection
See https://github.com/coder/internal/issues/398

Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-21 14:36:35 +00:00
44d12aa722 Refactoring reconciliation loop into control & logic, adding initial (incomplete) tests
Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-21 14:34:58 +00:00
473d53a870 Merge branch 'main' of github.com:/coder/coder into dk/prebuilds 2025-02-20 21:01:33 +00:00
44499315ed chore: reduce log volume on server startup (#16608)
Addresses https://github.com/coder/coder/issues/16231.

This PR reduces the volume of logs we print after server startup in
order to surface the web UI URL better.

Here are the logs after the changes a couple of seconds after starting
the server:

<img width="868" alt="Screenshot 2025-02-18 at 16 31 32"
src="https://github.com/user-attachments/assets/786dc4b8-7383-48c8-a5c3-a997c01ca915"
/>

The warning is due to running a development site-less build. It wouldn't
show in a release build.
2025-02-20 16:33:14 +01:00
d1b159ad12 Merge branch 'main' of github.com:/coder/coder into dk/prebuilds 2025-02-20 14:57:50 +00:00
4d97580666 Decompose GetTemplatePrebuildState into separate queries, reimplement logic in Go
This is in service of testability

Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-20 14:57:17 +00:00
b07b33ec9d feat: add agentapi endpoint to report connections for audit (#16507)
This change adds a new `ReportConnection` endpoint to the `agentapi`.

The protocol version was bumped previously, so it has been omitted here.

This allows the agent to report connection events, for example when the
user connects to the workspace via SSH or VS Code.

Updates #15139
2025-02-20 14:52:01 +02:00
9469b78290 fix!: enforce regex for agent names (#16641)
Underscores and double hyphens are now blocked. The regex is almost the
exact same as the `coder_app` `slug` regex, but uppercase characters are
still permitted.
2025-02-20 05:09:26 +00:00
64d476545b Merge branch 'main' of github.com:/coder/coder into dk/prebuilds
Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-19 15:42:15 +00:00
d2419c89ac feat: add tool to send a test notification (#16611)
Relates to https://github.com/coder/coder/issues/16463

Adds a CLI command, and API endpoint, to trigger a test notification for
administrators of a deployment.
2025-02-19 13:08:38 +00:00
9dd9fedc12 Implement strict prebuilds eligibility
See https://github.com/coder/internal/issues/372

Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-18 09:39:04 +00:00
e9fdd86c45 Merge branch 'jjs/presets' of github.com:/coder/coder into dk/prebuilds 2025-02-18 08:32:25 +00:00
d6b9806098 chore: implement oom/ood processing component (#16436)
Implements the processing logic as set out in the OOM/OOD RFC.
2025-02-17 16:56:52 +00:00
b5329ae1cd feat: add workspace agent connect and app open audit types (#16493)
This commit adds new audit resource types for workspace agents and
workspace apps, as well as connect/disconnect and open/close actions.

The idea is that we will log new audit events for connecting to the
agent via SSH/editor.

Likewise, we will log openings of `coder_app`s.

This change also introduces support for filtering by `request_id`.

Updates #15139
2025-02-17 13:02:30 +00:00
5ba7ba6bfc fix(coderd): add strict org ID joins for provisioner job metadata (#16588)
References #16558
2025-02-17 14:16:45 +02:00
77306f3de1 feat(coderd): add filters and fix template for provisioner daemons (#16558)
This change adds provisioner daemon ID filter to the provisioner daemons
endpoint, and also implements the limiting to 50 results.

Test coverage is greatly improved and template information for jobs
associated to the daemon was also fixed.

Updates #15084
Updates #15192
Related #16532
2025-02-14 17:26:46 +02:00
c3f67f8169 Claim prebuild by preset ID
Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-14 13:38:17 +00:00
2b296244a5 Taking presets into account when provisioning prebuilds
Fixing migrations

Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-14 12:17:54 +00:00
b60f2f66c6 Persisting presets defined with prebuilds to DB
Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-14 12:17:11 +00:00
7e1abd141c Claim prebuild and rename
Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-14 12:17:08 +00:00
eebbeb54bc Discrimination between "outdated" and "extraneous" prebuilds, hardening reconciliation
Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-14 12:17:08 +00:00
fdabb8cf07 Very basic prebuild reassignment
Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-14 12:17:08 +00:00
9d5c6633de Generating short ID for prebuilds
Also dropped unnecessary CASTs

Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-14 12:17:07 +00:00
c227bb842f Control loop now handles reconciliation of multiple prebuilds-configured template versions
Correctly calculates extraneous prebuilds and returns offending prebuild IDs

Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-14 12:17:07 +00:00
b7c43f663e Fixed bug in state query relating to multiple template versions & workspaces in partially-deleted statuses
Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-14 12:17:06 +00:00
41a9778af0 Improving control loop resilience, fixing calculations
Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-14 12:17:06 +00:00
cffd63450d Add prebuilds user to default org
Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-14 12:17:05 +00:00
815ecbc74b Specify progress type
Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-14 12:17:05 +00:00
981f61e272 Incorporate in-progress jobs into state calculation
Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-14 12:17:04 +00:00