Commit Graph

555 Commits

Author SHA1 Message Date
e1df5368a5 add tests for prebuilds
finalise database integration tests for prebuilds

reintegrate with danny's latest changes

add back assertions for deletion integration tests of prebuilds

tidy up prebuilds tests
2025-03-05 08:30:27 +00:00
f3a5b8e4d8 Merge branch 'main' of github.com:/coder/coder into dk/prebuilds
Signed-off-by: Danny Kopping <dannykopping@gmail.com>
2025-03-04 10:08:54 +00:00
24f3445e00 chore: track workspace resource monitors in telemetry (#16776)
Addresses https://github.com/coder/nexus/issues/195. Specifically, just
the "tracking templates" requirement:

> ## Tracking in templates
> To enable resource alerts, a user must add the resource_monitoring
block to a template's coder_agent resource. We'd like to track if
customers have any resource monitoring enabled on a per-deployment
basis. Even better, we could identify which templates are using resource
monitoring.
2025-03-03 18:41:01 +01:00
b85ba586ee fix(coderd/database): consider tag sets when calculating queue position (#16685)
Relates to https://github.com/coder/coder/issues/15843

## PR Contents

- Reimplementation of the `GetProvisionerJobsByIDsWithQueuePosition` SQL
query to **take into account** provisioner job tags and provisioner
daemon tags.
- Unit tests covering different **tag sets**, **job statuses**, and
**job ordering** scenarios.

## Notes

- The original row order is preserved by introducing the `ordinality`
field.
- Unnecessary rows are filtered as early as possible to ensure that
expensive joins operate on a smaller dataset.
- A "fake" join with `provisioner_jobs` is added at the end to ensure
`sqlc.embed` compiles successfully.
- **Backward compatibility is preserved**—only the SQL query has been
updated, while the Go code remains unchanged.
2025-03-03 10:02:18 -05:00
e16d763241 add tests to ensure that preubilds are correctly provisioned for active template versions 2025-03-03 10:20:47 +00:00
c074f77a4f feat: add notifications inbox db (#16599)
This PR is linked [to the following
issue](https://github.com/coder/internal/issues/334).

The objective is to create the DB layer and migration for the new `Coder
Inbox`.
2025-03-03 10:12:48 +01:00
91a4a98c27 chore: add an unassign action for roles (#16728) 2025-02-27 10:39:06 -07:00
6dd51f92fb chore: test metricscache on postgres (#16711)
metricscache_test has been running tests against dbmem only, instead of
against postgres. Unfortunately the implementations of
GetTemplateAverageBuildTime have diverged between dbmem and postgres.
This change gets the tests working on Postgres and test for the
behaviour postgres provides.
2025-02-27 09:43:51 +00:00
7cd6e9cdd6 fix: return provisioners in desc order and add limit to cli (#16720) 2025-02-26 21:06:51 +02:00
5e32ed22ac Merge branch 'main' of github.com:coder/coder into dk/prebuilds
Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-26 14:31:56 +02:00
d3a56ae3ef feat: enable GitHub OAuth2 login by default on new deployments (#16662)
Third and final PR to address
https://github.com/coder/coder/issues/16230.

This PR enables GitHub OAuth2 login by default on new deployments.
Combined with https://github.com/coder/coder/pull/16629, this will allow
the first admin user to sign up with GitHub rather than email and
password.

We take care not to enable the default on deployments that would upgrade
to a Coder version with this change.

To disable the default provider an admin can set the
`CODER_OAUTH2_GITHUB_DEFAULT_PROVIDER` env variable to false.
2025-02-25 16:31:33 +01:00
4e1e745912 add prebuild metrics and observability
Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-25 11:17:10 +00:00
546a549dcf feat: enable soft delete for organizations (#16584)
- Add deleted column to organizations table
- Add trigger to check for existing workspaces, templates, groups and
members in a org before allowing the soft delete

---------

Co-authored-by: Steven Masley <stevenmasley@gmail.com>
Co-authored-by: Steven Masley <Emyrk@users.noreply.github.com>
2025-02-24 12:59:41 -05:00
fad3f23791 Correct queries; running prebuilds only needs to return current preset ID
Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-24 16:19:08 +00:00
c88c04c022 Determine latest chosen preset using latest non-null preset selection
See https://github.com/coder/internal/issues/398

Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-21 14:36:35 +00:00
4d97580666 Decompose GetTemplatePrebuildState into separate queries, reimplement logic in Go
This is in service of testability

Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-20 14:57:17 +00:00
64d476545b Merge branch 'main' of github.com:/coder/coder into dk/prebuilds
Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-19 15:42:15 +00:00
9dd9fedc12 Implement strict prebuilds eligibility
See https://github.com/coder/internal/issues/372

Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-18 09:39:04 +00:00
e9fdd86c45 Merge branch 'jjs/presets' of github.com:/coder/coder into dk/prebuilds 2025-02-18 08:32:25 +00:00
d6b9806098 chore: implement oom/ood processing component (#16436)
Implements the processing logic as set out in the OOM/OOD RFC.
2025-02-17 16:56:52 +00:00
b5329ae1cd feat: add workspace agent connect and app open audit types (#16493)
This commit adds new audit resource types for workspace agents and
workspace apps, as well as connect/disconnect and open/close actions.

The idea is that we will log new audit events for connecting to the
agent via SSH/editor.

Likewise, we will log openings of `coder_app`s.

This change also introduces support for filtering by `request_id`.

Updates #15139
2025-02-17 13:02:30 +00:00
5ba7ba6bfc fix(coderd): add strict org ID joins for provisioner job metadata (#16588)
References #16558
2025-02-17 14:16:45 +02:00
77306f3de1 feat(coderd): add filters and fix template for provisioner daemons (#16558)
This change adds provisioner daemon ID filter to the provisioner daemons
endpoint, and also implements the limiting to 50 results.

Test coverage is greatly improved and template information for jobs
associated to the daemon was also fixed.

Updates #15084
Updates #15192
Related #16532
2025-02-14 17:26:46 +02:00
c3f67f8169 Claim prebuild by preset ID
Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-14 13:38:17 +00:00
2b296244a5 Taking presets into account when provisioning prebuilds
Fixing migrations

Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-14 12:17:54 +00:00
b60f2f66c6 Persisting presets defined with prebuilds to DB
Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-14 12:17:11 +00:00
7e1abd141c Claim prebuild and rename
Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-14 12:17:08 +00:00
eebbeb54bc Discrimination between "outdated" and "extraneous" prebuilds, hardening reconciliation
Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-14 12:17:08 +00:00
fdabb8cf07 Very basic prebuild reassignment
Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-14 12:17:08 +00:00
9d5c6633de Generating short ID for prebuilds
Also dropped unnecessary CASTs

Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-14 12:17:07 +00:00
c227bb842f Control loop now handles reconciliation of multiple prebuilds-configured template versions
Correctly calculates extraneous prebuilds and returns offending prebuild IDs

Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-14 12:17:07 +00:00
b7c43f663e Fixed bug in state query relating to multiple template versions & workspaces in partially-deleted statuses
Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-14 12:17:06 +00:00
41a9778af0 Improving control loop resilience, fixing calculations
Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-14 12:17:06 +00:00
815ecbc74b Specify progress type
Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-14 12:17:05 +00:00
981f61e272 Incorporate in-progress jobs into state calculation
Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-14 12:17:04 +00:00
4b5555e918 Preset tables, latest workspace view, state logic improved
Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-14 12:17:04 +00:00
1150e206e0 Migrations
Exclude system users from users.sql queries

Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-14 12:17:01 +00:00
e38bd27183 feat(coderd): add support for provisioner job id and tag filter (#16556)
This change adds to new filters to the provisionerjobs endpoint, id
(array) and tags (map).

Updates #15084
Updates #15192
Related #16532
2025-02-13 18:24:27 +02:00
71cbf735e5 feat(coderd): add support for presets to the coder API (#16526)
This pull request builds on the existing migrations and queries to add
support for presets to the coder API.
2025-02-12 14:41:14 +02:00
e85da8b02b chore: return template data for provisioner daemons (#16514)
Return template data in provisioner jobs to be displayed in the
provisioners page.
2025-02-11 14:22:34 -03:00
34b46f9205 feat(coderd/database): add support for presets (#16509)
This pull requests adds the necessary migrations and queries to support
presets within the coderd database. Future PRs will build functionality
to the provisioners and the frontend.
2025-02-11 13:55:09 +02:00
e9b3561677 refactor: return template_icon and make metadata required (#16496) 2025-02-10 10:00:34 -03:00
b04d883348 feat: add provisioner job metadata (#16454)
This change adds metadata to provisioner jobs to help with rendering
related tempaltes and workspaces in the UI.

Updates #15084
2025-02-06 16:19:20 +02:00
7cbd77fd94 feat: improve resources_monitoring for OOM & OOD monitoring (#16241)
As requested for [this
issue](https://github.com/coder/internal/issues/245) we need to have a
new resource `resources_monitoring` in the agent.

It needs to be parsed from the provisioner and inserted into a new db
table.
2025-02-04 18:45:33 +01:00
2ace044e0b chore: track the first time html is served in telemetry (#16334)
Addresses https://github.com/coder/nexus/issues/175.

## Changes

- Adds the `telemetry_items` database table. It's a key value store for
telemetry events that don't fit any other database tables.
- Adds a telemetry report when HTML is served for the first time in
`site.go`.
2025-01-31 13:55:46 +01:00
1336925c9f feat(flake.nix): switch dogfood dev image to buildNixShellImage from dockerTools (#16223)
Replace Depot build action with Nix for Nix dogfood image builds

The dogfood Nix image is now built using Nix's native container tooling instead of Depot. This change:

- Adds Nix setup steps to the GitHub Actions workflow
- Removes the Dockerfile.nix in favor of a Nix-native container build
- Updates the flake.nix to support building Docker images
- Introduces a hash file to track Nix-related changes
- Updates the vendorHash for Go dependencies

Change-Id: I4e011fe3a19d9a1375fbfd5223c910e59d66a5d9
Signed-off-by: Thomas Kosiewski <tk@coder.com>
2025-01-28 16:38:37 +01:00
75c899ff71 feat(cli): add provisioner job cancel command (#16252)
Fixes #16117
Updates #15084
2025-01-27 16:26:56 +00:00
5762d8add4 fix: return only the first workspace agent script timing per script (#16203)
Fixes https://github.com/coder/coder/issues/16124

If a workspace agent crashes, it is possible for any startup scripts to
be ran again. This PR makes it so that the
`GetWorkspaceAgentScriptTimingsByBuildID` query only returns the first
timing recorded per-script.
2025-01-21 11:54:43 +00:00
3864c7e3b0 feat(coderd): add endpoint to list provisioner jobs (#16029)
Closes #15190
Updates #15084
2025-01-20 11:18:53 +02:00
50bf5ca8fe fix(coderd/database): aggregate user engagement statistics by interval (#16150)
This PR addresses the TODO comment here:

https://github.com/coder/coder/pull/16134/files#diff-1844f895bb005f036da11d800fe2a76b54bfddd481c5d8cb15f210c64679caa5R47

The backend now backfills entries for dates with no status changes.
2025-01-16 17:34:53 +02:00