Commit Graph

517 Commits

Author SHA1 Message Date
44d12aa722 Refactoring reconciliation loop into control & logic, adding initial (incomplete) tests
Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-21 14:34:58 +00:00
d1b159ad12 Merge branch 'main' of github.com:/coder/coder into dk/prebuilds 2025-02-20 14:57:50 +00:00
4d97580666 Decompose GetTemplatePrebuildState into separate queries, reimplement logic in Go
This is in service of testability

Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-20 14:57:17 +00:00
9469b78290 fix!: enforce regex for agent names (#16641)
Underscores and double hyphens are now blocked. The regex is almost the
exact same as the `coder_app` `slug` regex, but uppercase characters are
still permitted.
2025-02-20 05:09:26 +00:00
64d476545b Merge branch 'main' of github.com:/coder/coder into dk/prebuilds
Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-19 15:42:15 +00:00
7d949e5ee0 Log lock acquisition time for https://github.com/coder/internal/issues/371
Setting default claimer to avoid panics

Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-19 11:00:22 +00:00
9dd9fedc12 Implement strict prebuilds eligibility
See https://github.com/coder/internal/issues/372

Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-18 09:39:04 +00:00
e9fdd86c45 Merge branch 'jjs/presets' of github.com:/coder/coder into dk/prebuilds 2025-02-18 08:32:25 +00:00
0a94405c3f Move prebuilds code to enterprise top-level package, refactor into agpl pointers
Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-18 08:20:17 +00:00
7498980c5f Hide prebuilds behind premium license & experiment
Signed-off-by: Danny Kopping <danny@coder.com>
2025-02-17 13:03:56 +00:00
7f061b9faf fix(coderd): add stricter authorization for provisioners endpoint (#16587)
References #16558
2025-02-17 14:34:47 +02:00
77306f3de1 feat(coderd): add filters and fix template for provisioner daemons (#16558)
This change adds provisioner daemon ID filter to the provisioner daemons
endpoint, and also implements the limiting to 50 results.

Test coverage is greatly improved and template information for jobs
associated to the daemon was also fixed.

Updates #15084
Updates #15192
Related #16532
2025-02-14 17:26:46 +02:00
d0a534e30d chore: prevent authentication of non-unique oidc subjects (#16498)
Any IdP returning an empty field here breaks the assumption of a
unique subject id. This is defined in the OIDC spec.
2025-02-10 09:31:08 -06:00
0e2ae10b47 feat: add additional patch routes for group and role idp sync (#16351) 2025-01-31 12:14:24 -07:00
6ea5c6f0ef fix: show user-auth provisioners for all organizations (#16350) 2025-01-30 14:08:27 -07:00
b256b204d0 feat: add endpoint for partial updates to org sync field and assign_default (#16337) 2025-01-30 13:55:17 -07:00
2371153a37 feat: add endpoint for partial updates to org sync mapping (#16316) 2025-01-30 10:52:50 -07:00
92d22e296b chore: track usage of organizations in telemetry (#16323)
Addresses https://github.com/coder/internal/issues/317.

## Changes

Requirements are quoted below:

> how many orgs does deployment have

Adds the Organization entity to telemetry.

> ensuring resources are associated with orgs

All resources that reference an org already report the org id to
telemetry. Adds a test to check that.

> whether org sync is configured

Adds the `IDPOrgSync` boolean field to the Deployment entity.

## Implementation of the org sync check

While there's an `OrganizationSyncEnabled` method on the IDPSync
interface, I decided not to use it directly and implemented a
counterpart just for telemetry purposes. It's a compromise I'm not happy
about, but I found that it's a simpler approach than the alternative.
There are multiple reasons:

1. The telemetry package cannot statically access the IDPSync interface
due to a circular import.
2. We can't dynamically pass a reference to the
`OrganizationSyncEnabled` function at the time of instantiating the
telemetry object, because our server initialization logic depends on the
telemetry object being created before the IDPSync object.
3. If we circumvent that problem by passing the reference as an
initially empty pointer, initializing telemetry, then IDPSync, then
updating the pointer to point to `OrganizationSyncEnabled`, we have to
refactor the initialization logic of the telemetry object itself to
avoid a race condition where the first telemetry report is performed
without a valid reference.

I actually implemented that approach in
https://github.com/coder/coder/pull/16307, but realized I'm unable to
fully test it. It changed the initialization order in the server
command, and I wanted to test our CLI with Org Sync configured with a
premium license. As far as I'm aware, we don't have the tooling to do
that. I couldn't figure out a way to start the CLI with a mock license,
and I didn't want to go down further into the refactoring rabbit hole.

So I decided that reimplementing the org sync checking logic is simpler.
2025-01-29 15:54:31 +01:00
c069563af1 test: fix use of t.Logf where t.Log would suffice (#16328) 2025-01-29 14:35:04 +00:00
76adde91dc fix(provisioner/terraform/tfparse): allow empty values in coder_workspace_tag defaults (#16303)
* chore(docs): update docs re workspace tag default values
* chore(coderdenttest): use random name instead of t.Name() in newExternalProvisionerDaemon
* fix(provisioner/terraform/tfparse): allow empty values in coder_workspace_tag defaults
2025-01-28 09:11:39 +00:00
5841c0aacb fix: fetch custom roles from workspace agent context (#16237) 2025-01-23 12:57:09 -06:00
f34e6fd92c chore: implement 'use' verb to template object, read has less scope now (#16075)
Template `use` is now a verb.
- Template admins can `use` all templates (org template admins same in
org)
- Members get the `use` perm from the `everyone` group in the
`group_acl`.
2025-01-17 11:55:41 -06:00
f32f7c6862 test(enterprise/coderd): fix ctx init in multiple workspace tests (#16176) 2025-01-17 14:33:58 +00:00
3a81aac318 chore(enterprise/coderd): use filesystem mirror for providers in TestWorkspaceTagsTerraform (#16155)
Fixes https://github.com/coder/internal/issues/266 (hopefully)

Each instance of this test has to download the coder/coder Terraform
provider.
To mitigate this, only download the providers once using a
`filesystem_mirror` (ref:
https://developer.hashicorp.com/terraform/cli/config/config-file#provider-installation)
2025-01-15 20:27:18 +00:00
12991ff43e fix(coderd): change the order of precedence between coder_workspace_tags and request tags (#16119)
This PR switches the order of precedence of workspace tags when posting
a template version.
Previously, user-specified tags in the request could not override those
detected from our parsing of the template file. Now, they can do.

This addresses a customer issue where were attempting to set a workspace
tag via variable.

Note: there is a possible follow-up item here where we could pass in the
workspace tag values from the request into `tfparse` and let it take
those user-specified values into account. This is covered in a separate
test.
2025-01-14 16:43:42 +00:00
071bb26018 feat(coderd): add endpoint to list provisioner daemons (#16028)
Updates #15190
Updates #15084
Supersedes #15940
2025-01-14 16:40:26 +00:00
5e88289493 chore(enterprise/coderd): conditionally disable parallelism on windows (#16072)
Fixes https://github.com/coder/internal/issues/266
2025-01-09 09:14:20 +00:00
6ca1e5973e chore: add api endpoints to get idp field values (#16063)
Supports coder/internal#210
2025-01-08 16:07:02 -05:00
7b88776403 chore(testutil): add testutil.GoleakOptions (#16070)
- Adds `testutil.GoleakOptions` and consolidates existing options to
this location
- Pre-emptively adds required ignore for this Dependabot PR to pass CI
https://github.com/coder/coder/pull/16066
2025-01-08 15:38:37 +00:00
1ab10cf80c fix(provisioner/terraform/tfparse): skip evaluation of unrelated parameters (#16023)
* Improves tfparse test coverage to include more parameter types and values
* Adds tests with unrelated parameters that should be ignored by tfparse
* Modifies tfparse to only attempt evaluation of parameters referenced by coder_workspace_tags
2025-01-03 19:32:17 +00:00
94f5d52fdc chore: adopt markdownlint and markdown-table-formatter for *.md (#15831)
Co-authored-by: Edward Angert <EdwardAngert@users.noreply.github.com>
2025-01-03 13:12:59 +00:00
73ec6b2635 chore: audit log entries for all idp sync changes (#15919) 2025-01-02 15:02:04 -06:00
2c7f8ac65f chore: migrate to coder/websocket 1.8.12 (#15898)
Migrates us to `coder/websocket` v1.8.12 rather than `nhooyr/websocket` on an older version.

Works around https://github.com/coder/websocket/issues/504 by adding an explicit test for `xerrors.Is(err, io.EOF)` where we were previously getting `io.EOF` from the netConn.
2024-12-19 00:51:30 +04:00
91875c2b47 fix: allow users to extend their running workspace's deadline (#15895)
Fixes https://github.com/coder/coder/issues/15515

This change effectively reverts the changes introduced by
https://github.com/coder/coder/pull/13182 (for
https://github.com/coder/coder/issues/13078).

We also rename the `override-stop` command name to `extend` to match the
API endpoint's name (keeping an alias to allow `override-stop` to be used).
2024-12-18 10:58:33 +00:00
dcf51535bb fix(coderd/wsbuilder): correctly evaluate dynamic workspace tag values (#15897)
Relates to https://github.com/coder/coder/issues/15894:

- Adds `coderdenttest.NewExternalProvisionerDaemonTerraform`
- Adds integration-style test coverage for creating a workspace with
`coder_workspace_tags` specified in `main.tf`
- Modifies `coderd/wsbuilder` to fetch template version variables and
  includes them in eval context for evaluating `coder_workspace_tags`
2024-12-17 21:57:03 +00:00
21cc6f77fe docs: move premium and enterprise badges from heading to info in admonition (#15774)
closes https://github.com/coder/coder.com/issues/666


[preview](https://coder.com/docs/@666-unheading-premium/user-guides/workspace-scheduling#autostop-requirement)

---------

Co-authored-by: EdwardAngert <2408959-EdwardAngert@users.noreply.gitlab.com>
Co-authored-by: EdwardAngert <17991901+EdwardAngert@users.noreply.github.com>
2024-12-17 18:20:01 +00:00
d31c2f1fe7 chore: implement SCIM PUT endpoint, protect against missing active (#15829)
Closes https://github.com/coder/coder/issues/15828
2024-12-12 08:11:13 -06:00
b39becba66 feat(site): add a provisioner warning to workspace builds (#15686)
This PR adds warnings about provisioner health to workspace build pages.
It closes https://github.com/coder/coder/issues/15048


![image](https://github.com/user-attachments/assets/fa54d0e8-c51f-427a-8f66-7e5dbbc9baca)

![image](https://github.com/user-attachments/assets/b5169669-ab05-43d5-8553-315a3099b4fd)
2024-12-11 13:38:13 +02:00
40624bf78b fix: update workspace TTL on template TTL change (#15761)
Relates to https://github.com/coder/coder/issues/15390

Currently when a user creates a workspace, their workspace's TTL is
determined by the template's default TTL. If the Coder instance is AGPL,
or if the template has disallowed the user from configuring autostop,
then it is not possible to change the workspace's TTL after creation.
Any changes to the template's default TTL only takes effect on _new_
workspaces.

This PR modifies the behaviour slightly so that on AGPL Coder, or on
enterprise when a template does not allow user's to configure their
workspace's TTL, updating the template's default TTL will also update
any workspace's TTL to match this value.
2024-12-06 11:01:39 +00:00
67553a7bbe chore: fix TestWorkspaceAutobuild/NextStartAtIsValid flake (#15772)
Fixes https://github.com/coder/internal/issues/238

It appears we haven't got quartz piped through to enough of Coder for
the test to work as I had expected. Using the current time should be
sufficient for this test.
2024-12-06 10:41:31 +00:00
e21a301682 fix: make GetWorkspacesEligibleForTransition return even less false positives (#15594)
Relates to https://github.com/coder/coder/issues/15082

Further to https://github.com/coder/coder/pull/15429, this reduces the
amount of false-positives returned by the 'is eligible for autostart'
part of the query. We achieve this by calculating the 'next start at'
time of the workspace, storing it in the database, and using it in our
`GetWorkspacesEligibleForTransition` query.

The prior implementation of the 'is eligible for autostart' query would
return _all_ workspaces that at some point in the future _might_ be
eligible for autostart. This now ensures we only return workspaces that
_should_ be eligible for autostart.

We also now pass `currentTick` instead of `t` to the
`GetWorkspacesEligibleForTransition` query as otherwise we'll have one
round of workspaces that are skipped by `isEligibleForTransition` due to
`currentTick` being a truncated version of `t`.
2024-12-02 21:02:36 +00:00
2b57dcc68c feat(coderd): add matched provisioner daemons information to more places (#15688)
- Refactors `checkProvisioners` into `db2sdk.MatchedProvisioners`
- Adds a separate RBAC subject just for reading provisioner daemons
- Adds matched provisioners information to additional endpoints relating to
  workspace builds and templates
-Updates existing unit tests for above endpoints
-Adds API endpoint for matched provisioners of template dry-run job
-Updates CLI to show warning when creating/starting/stopping/deleting
 workspaces for which no provisoners are available

---------

Co-authored-by: Danny Kopping <danny@coder.com>
2024-12-02 20:54:32 +00:00
8afb10e090 chore: improve validation of Security tag in swaggerparser (#15660)
Aims to resolve #15605 

There's currently one option valid for the `@Security` tag in
swaggerparser - which fails in the CI if we try to put any other value.

At least one of our endpoints does not accept `CoderSessionToken` as an
option for the authentication and so we need to add new possibilities in
order to keep the documentation up-to-date.

In this PR , I added `ProvisionerKey` which is the way our provisioner
daemon can authenticate to the backend - also modified a bit the code to
simplify other options later.
2024-11-26 07:19:43 +01:00
60ddcf5de2 chore: improve testing coverage on ExtractProvisionerDaemonAuthenticated middleware (#15622)
This one aims to resolve #15604 

Created some table tests for the main cases - 
also preferred to create two isolated cases for the most complicated
cases in order to keep table tests simple enough.

Give us full coverage on the middleware logic, for both optional and non
optional cases - PSK and ProvisionerKey.
2024-11-26 04:02:20 +01:00
0896f339c4 refactor(coderd/provisionerdserver): use quartz.Clock instead of TimeNowFn (#15642)
Replace `TimeNowFn` in `provisionerdserver` with `quartz.Clock` as
well as pass `coderd`'s `Clock` to `provisionerdserver`.
2024-11-25 16:25:36 +00:00
b5fbfd727c fix: fix hang in teardown of TestConn_CoordinatorRollingRestart (#15624)
fixes a flake seen on main: https://github.com/coder/coder/actions/runs/11967210463/job/33364072261

the TCP echo server had a waitgroup to ensure that all accepted connections get torn down, but no explicit teardown of the connection. We depended on the tailnet agent closing its side of the connection, which depends on closing the tunneled connection. The tunneled `FIN` could race with tearing down the tunnel itself.

So, this PR adds explicit `t.Cleanup` to close the echo connection. It also removes the waitgroup. The purpose of the waitgroup was to ensure that all goroutines created by the echo listener get shut down, but we have `goleak` for that, which fails much faster than the 20 minutes this test run took.
2024-11-22 11:44:56 +04:00
a518017a88 feat(coderd): add endpoint to fetch provisioner key details (#15505)
This PR is the first step aiming to resolve #15126 - 

Creating a new endpoint to return the details associated to a
provisioner key.

This is an authenticated endpoints aiming to be used by the provisioner
daemons - using the provisioner key as authentication method.

This endpoint is not ment to be used with PSK or User Sessions.
2024-11-20 18:04:47 +01:00
6ed76921dd chore: fix windows postgres tests (#15593)
Patches tests that caused Windows Postgres CI in
https://github.com/coder/coder/pull/15520 to consistently fail.

I tested this by temporarily adding Postgres Windows CI to this PR.
However, I reverted those changes to merge them with
https://github.com/coder/coder/pull/15520. For reference, here's [a
passing CI
run](https://github.com/coder/coder/actions/runs/11918816662/job/33219786238)
from an earlier commit.

**Note:** Although Windows tests now pass, they remain quite flaky. I
recommend running Postgres Windows CI to gather data on these flakes,
but I don’t think it should be a required job just yet.
2024-11-20 13:30:31 +01:00
fbe2fa66f5 chore: add test for coord rolling restart (#14680)
Closes https://github.com/coder/team-coconut/issues/50

---------

Co-authored-by: Ethan Dickson <ethan@coder.com>
2024-11-20 18:04:33 +11:00
c3c23ed3d9 chore: add query to fetch top level idp claim fields (#15525)
Adds an api endpoint to grab all available sync field options for IDP
sync. This is for autocomplete on idp sync forms. This is required for
organization admins to have some insight into the claim fields available
when configuring group/role sync.
2024-11-18 14:31:39 -06:00