Commit Graph

2680 Commits

Author SHA1 Message Date
f569d9c33d feat: add separate max token lifetime for administrators (#18267)
# Add separate token lifetime limits for administrators

This PR introduces a new configuration option `--max-admin-token-lifetime` that allows administrators to create API tokens with longer lifetimes than regular users. By default, administrators can create tokens with a lifetime of up to 7 days (168 hours), while the existing `--max-token-lifetime` setting continues to apply to regular users.

The implementation:
- Adds a new `MaximumAdminTokenDuration` field to the session configuration
- Modifies the token validation logic to check the user's role and apply the appropriate lifetime limit
- Updates the token configuration endpoint to return the correct maximum lifetime based on the user's role
- Adds tests to verify that administrators can create tokens with longer and shorter lifetimes
- Updates documentation and help text to reflect the new option

This change allows organizations to grant administrators extended token lifetimes while maintaining tighter security controls for regular users.

Fixes #17395
2025-06-06 17:36:30 +02:00
a12429e9f8 feat(agent/agentcontainers): refactor Lister to ContainerCLI and implement new methods (#18243) 2025-06-06 10:33:09 +00:00
533c6dcbbe fix: remove error log when notification manager is already closed (#18264)
Closes https://github.com/coder/internal/issues/677

Resolves flakes such as:

```
$ go test -race -run "TestRunStopRace" github.com/coder/coder/v2/coderd/notifications -count=10000 -parallel $(nproc)
--- FAIL: TestRunStopRace (0.00s)
    t.go:106: 2025-06-06 02:44:39.348 [debu]  notifications-manager: notification manager started
    t.go:106: 2025-06-06 02:44:39.348 [debu]  notifications-manager: graceful stop requested
    t.go:106: 2025-06-06 02:44:39.348 [debu]  notifications-manager: notification manager stopped
    t.go:115: 2025-06-06 02:44:39.348 [erro]  notifications-manager: notification manager stopped with error ...
        error= manager already closed:
                   github.com/coder/coder/v2/coderd/notifications.(*Manager).loop
                       /home/coder/coder/coderd/notifications/manager.go:166
         *** slogtest: log detected at level ERROR; TEST FAILURE ***
--- FAIL: TestRunStopRace (0.00s)
    t.go:106: 2025-06-06 02:44:41.632 [debu]  notifications-manager: notification manager started
    t.go:106: 2025-06-06 02:44:41.632 [debu]  notifications-manager: graceful stop requested
    t.go:106: 2025-06-06 02:44:41.632 [debu]  notifications-manager: notification manager stopped
    t.go:115: 2025-06-06 02:44:41.633 [erro]  notifications-manager: notification manager stopped with error ...
        error= manager already closed:
                   github.com/coder/coder/v2/coderd/notifications.(*Manager).loop
                       /home/coder/coder/coderd/notifications/manager.go:166
         *** slogtest: log detected at level ERROR; TEST FAILURE ***
FAIL
FAIL    github.com/coder/coder/v2/coderd/notifications  6.847s
FAIL
```

These error logs are caused as a result of the `Manager` `Run` start operation being asynchronous. In the flaking test case we immediately call `Stop` after `Run`. It's possible for `Stop` to be scheduled to completion before the goroutine spawned by `Run` calls `loop` and checks `closed`. If this happens, `loop` returns an error and produces the error log.

We'll address this by replacing this specific error log with a warning log.

```
$ go test -run "TestRunStopRace" github.com/coder/coder/v2/coderd/notifications -count=10000 -parallel $(nproc)
ok      github.com/coder/coder/v2/coderd/notifications  1.294s

$ go test -race github.com/coder/coder/v2/coderd/notifications -count=100 -parallel $(nproc)
ok      github.com/coder/coder/v2/coderd/notifications  26.525s
```
2025-06-06 19:28:10 +10:00
0076e8479f chore(vpn): send ping results over tunnel (#18200)
Closes #17982.

The purpose of this PR is to expose network latency via the API used by Coder Desktop.

This PR has the tunnel ping all known agents every 5 seconds, in order to produce an instance of:
```proto
message LastPing {
	// latency is the RTT of the ping to the agent.
	google.protobuf.Duration latency = 1;
	// did_p2p indicates whether the ping was sent P2P, or over DERP.
	bool did_p2p = 2;
	// preferred_derp is the human readable name of the preferred DERP region,
	// or the region used for the last ping, if it was sent over DERP.
	string preferred_derp = 3;
	// preferred_derp_latency is the last known latency to the preferred DERP
	// region. Unset if the region does not appear in the DERP map.
	optional google.protobuf.Duration preferred_derp_latency = 4;
}
```
The contents of this message are stored and included on all subsequent upsertions of the agent. 
Note that we upsert existing agents every 5 seconds to update the `last_handshake` value.

On the desktop apps, this message will be used to produce a tooltip similar to that of the VS Code extension:
<img width="495" alt="image" src="https://github.com/user-attachments/assets/d8b65f3d-f536-4c64-9af9-35c1a42c92d2" />
(wording not final)

Unlike the VS Code extension, we omit:
- The Latency of *all* available DERP regions. It seems not ideal to send a copy of this entire map for every online agent, and it certainly doesn't make sense for it to be on the `Agent` or `LastPing` message. 
If we do want to expose this info on Coder Desktop, we should consider how best to do so; maybe we want to include it on a more generic `Netcheck` message.
- The current throughput (Bytes up/down). This is out of scope of the linked issue, and is non-trivial to implement. I'm also not sure of the value given the frequency we're doing these status updates (every 5 seconds).
If we want to expose it, it'll be in a separate PR.

<img width="343" alt="image" src="https://github.com/user-attachments/assets/8447d03b-9721-4111-8ac1-332d70a1e8f1" />
2025-06-06 14:18:57 +10:00
0428c5ec1c chore: include 'everyone' group in template importing (#18257) 2025-06-05 19:25:36 +00:00
623dcd97dc fix(cli): fix flakes related to context cancellation when establishing pg connections (#18246)
Since https://github.com/coder/coder/pull/18195 was merged, we started
running CLI tests with postgres instead of just dbmem. This surfaced
errors related to context cancellation while establishing postgres
connections.

This PR should fix https://github.com/coder/internal/issues/672. Related
to https://github.com/coder/coder/issues/15109.
2025-06-05 15:54:13 +02:00
b5fd3dd855 feat(coderd/agentapi): allow inserting apps for sub agents (#18129)
Allow creating workspace apps for a sub agent when the agent is being
created with `CreateSubAgent` in the agent api.
2025-06-05 11:57:02 +01:00
277c2c7ea7 chore(coderd/prometheusmetrics): remove dbmem from tests (#18238) 2025-06-05 09:30:27 +02:00
5f7e5d7097 feat: support prebuilt workspaces in non-default organizations (#18010)
closes https://github.com/coder/internal/issues/527
2025-06-04 14:20:29 +02:00
4d0fe20ca6 chore(coderd/database/dbauthz): update RBAC for InsertWorkspaceApp (#18223)
Instead of using `ResourceSystem` as the resource for
`InsertWorkspaceApp`, we instead use the associated workspace (if it
exists), with the action `ActionUpdate`.
2025-06-04 11:22:01 +00:00
246a829ea9 feat: evaluate dynamic parameters http endpoint (#18182)
Used when a websocket is too heavy. This implements a single request to
the preview engine.
2025-06-02 13:50:07 -05:00
2e7cd0fe22 chore(coderd/database/dbpurge): remove dbmem from tests (#18151)
Related to https://github.com/coder/coder/issues/15109
2025-06-02 16:48:38 +02:00
1d48131e98 chore(coderd/externalauth): remove dbmem from tests (#18147)
Related to https://github.com/coder/coder/issues/15109
2025-06-02 14:42:18 +02:00
d1fc0dd2c5 chore(coderd/database/dbmetrics): remove dbmem from tests (#18150)
Related to https://github.com/coder/coder/issues/15109
2025-06-02 14:26:56 +02:00
8ca5519f57 chore(coderd/idpsync): run all tests with postgres (#18149)
Related to https://github.com/coder/coder/issues/15109.

Running postgres tests used to create a new postgres docker container
every time. I believe the slow down might've been caused by that and was
misattributed to postgres performance.

```
coder@main ~/coder ((0e90ac29))> DB=ci gotestsum --packages="./coderd/idpsync" -- -count=1
✓  coderd/idpsync (1.471s)

DONE 91 tests in 4.766s
```
2025-06-02 14:07:31 +02:00
bdf227c1f9 chore(coderd/database/dbgen): remove dbmem from tests (#18152)
Related to https://github.com/coder/coder/issues/15109
2025-06-02 14:05:40 +02:00
d3ed6fe652 chore(coderd/autobuild): use dbtestutil.WillUsePostgres instead of os.Getenv in test (#18145)
Standardizing on `WillUsePostgres` will make it easier to remove the
check entirely once dbmem is removed.

Related to https://github.com/coder/coder/issues/15109.
2025-06-02 13:58:07 +02:00
782d01bae2 chore(coderd/httpmw): remove dbmem usage from tests (#18146)
Related to https://github.com/coder/coder/issues/15109
2025-06-02 13:57:56 +02:00
9a28cb0545 chore(coderd/telemetry): remove dbmem from tests (#18148)
Related to https://github.com/coder/coder/issues/15109
2025-06-02 13:57:27 +02:00
9812162190 chore(coderd/database/dbrollup): remove dbmem from tests (#18153)
Related to https://github.com/coder/coder/issues/15109
2025-06-02 13:56:50 +02:00
9db114d17c feat: add filecache prometheus metrics (#18089)
Dynamic parameters has an in memory file cache. This adds prometheus
metrics to monitor said cache.
2025-05-30 11:54:54 -05:00
562c4696de fix(coderd/database/dbmem): fill DisplayGroup field for InsertWorkspaceApp (#18136)
It appears `dbmem` was missed in the new app groups feature
https://github.com/coder/coder/pull/17977.
2025-05-30 17:52:31 +01:00
9afdd33e64 fix(coderd/database/dbmem): apply rlock/runlock on GetTelemetryItems (#18133)
Fixes https://github.com/coder/coder/issues/18132
2025-05-30 16:39:32 +01:00
216fe441cf chore: align CSRF settings with deployment config (#18116) 2025-05-30 09:30:49 -05:00
f974add373 chore: rollback PR #18025 (#18118)
Rollback https://github.com/coder/coder/pull/18025
2025-05-30 11:00:26 -03:00
d779126ee3 chore: rollback PR #18081 (#18104)
Rollback https://github.com/coder/coder/pull/18081
2025-05-29 13:12:13 -03:00
e4648b6fc1 feat: allow iframing urls on the same domain as the deployment (#18102)
Used for AI tasks. We should eventually add regions to this csp header.
2025-05-29 10:07:57 -05:00
8387dd27ab chore: add form_type parameter argument to db (#17920)
`form_type` is a new parameter field in the terraform provider. Bring
that field into coder/coder.

Validation for `multi-select` has also been added.
2025-05-29 08:55:19 -05:00
776c144128 fix(coderd): ensure agent timings are non-zero on insert (#18065)
Relates to https://github.com/coder/coder/issues/15432

Ensures that no workspace build timings with zero values for started_at or ended_at are inserted into the DB or returned from the API.
2025-05-29 13:36:06 +01:00
b712d0b23f feat(coderd/agentapi): implement sub agent api (#17823)
Closes https://github.com/coder/internal/issues/619

Implement the `coderd` side of the AgentAPI for the upcoming
dev-container agents work.

`agent/agenttest/client.go` is left unimplemented for a future PR
working to implement the agent side of this feature.
2025-05-29 12:15:47 +01:00
bc83de2a72 feat: add prebuilt workspaces telemetry (#18084)
Adds telemetry for a _global_ account of prebuilt workspaces created,
failed to build, and claimed.

Partitioning this data by template/preset tuple is not currently in
scope.

---------

Signed-off-by: Danny Kopping <dannykopping@gmail.com>
2025-05-29 13:13:44 +02:00
2ec7404197 chore: make owner_name and owner_username consistent (#18081)
We've been using owner_name inconsistently as username. So this PR fixes
it by making the attribute naming more consistent.
2025-05-28 17:25:32 -03:00
b330c0803c fix: reimplement reporting of preset-hard-limited metric (#18055)
Addresses concerns raised in https://github.com/coder/coder/pull/18045
2025-05-28 14:18:32 -04:00
ca8660cea6 chore: keep previous workspace build parameters for dynamic params (#18059)
The existing code persists all static parameters and their values. Using
the previous build as the source if no new inputs are found.

Dynamic params do not have a state of the parameters saved to disk. So
instead, all previous values are persisted always, and new inputs
override.
2025-05-28 10:00:39 -05:00
6e255c72c6 chore(coderd/database): enforce agent name unique within workspace build (#18052)
Adds a database trigger that runs on insert and update of the
`workspace_agents` table. The trigger ensures that the agent name is
unique within the context of the workspace build it is being inserted
into.
2025-05-28 14:21:17 +01:00
110102a60a fix: optimize queue position sql query (#17974)
Use only `online provisioner daemons` for
`GetProvisionerJobsByIDsWithQueuePosition` query. It should improve
performance of the query.
2025-05-28 08:21:16 -04:00
b4531c4218 feat: make dynamic parameters respect owner in form (#18013)
Closes https://github.com/coder/coder/issues/18012

---------

Co-authored-by: Jaayden Halko <jaayden.halko@gmail.com>
2025-05-27 15:43:00 -05:00
9fc3329575 feat: persist app groups in the database (#17977) 2025-05-27 13:13:08 -06:00
a18eb9d08f feat(site): allow recreating devcontainers and showing dirty status (#18049)
This change allows showing the devcontainer dirty status in the UI as
well as a recreate button to update the devcontainer.

Closes #16424
2025-05-27 19:42:24 +03:00
d63417b542 fix: update WorkspaceOwnerName to use user.name instead of user.username (#18025)
We have been using the user.username instead of user.name in wrong
places, making it very confusing for the UI.
2025-05-27 11:42:07 -03:00
9827c97f32 feat: add AI Tasks page (#18047)
**Preview:**

<img width="1624" alt="Screenshot 2025-05-26 at 21 25 04"
src="https://github.com/user-attachments/assets/2a51915d-2527-4467-bf99-1f2d876b953b"
/>
2025-05-27 11:34:07 -03:00
6c0bed0f53 chore: update to coder/quartz v0.2.0 (#18007)
Upgrade to coder/quartz v0.2.0 including fixing up a minor API breaking change.
2025-05-27 16:05:03 +04:00
6f6e73af03 feat: implement expiration policy logic for prebuilds (#17996)
## Summary 

This PR introduces support for expiration policies in prebuilds. The TTL
(time-to-live) is retrieved from the Terraform configuration
([terraform-provider-coder
PR](https://github.com/coder/terraform-provider-coder/pull/404)):
```
prebuilds = {
	  instances = 2
	  expiration_policy {
		  ttl = 86400
	  }
  }
```
**Note**: Since there is no need for precise TTL enforcement down to the
second, in this implementation expired prebuilds are handled in a single
reconciliation cycle: they are deleted, and new instances are created
only if needed to match the desired count.

## Changes

* The outcome of a reconciliation cycle is now expressed as a slice of
reconciliation actions, instead of a single aggregated action.
* Adjusted reconciliation logic to delete expired prebuilds and
guarantee that the number of desired instances is correct.
* Updated relevant data structures and methods to support expiration
policies parameters.
* Added documentation to `Prebuilt workspaces` page
* Update `terraform-provider-coder` to version 2.5.0:
https://github.com/coder/terraform-provider-coder/releases/tag/v2.5.0

Depends on: https://github.com/coder/terraform-provider-coder/pull/404
Fixes: https://github.com/coder/coder/issues/17916
2025-05-26 20:31:24 +01:00
0731304905 feat(agent/agentcontainers): recreate devcontainers concurrently (#18042)
This change introduces a refactor of the devcontainers recreation logic
which is now handled asynchronously rather than being request scoped.
The response was consequently changed from "No Content" to "Accepted" to
reflect this.

A new `Status` field was introduced to the devcontainer struct which
replaces `Running` (bool). This reflects that the devcontainer can now
be in various states (starting, running, stopped or errored).

The status field also protects against multiple concurrent recrations,
as long as they are initiated via the API.

Updates #16424
2025-05-26 18:30:52 +03:00
d6c14f3d8a feat(agent/agentcontainers): update containers periodically (#17972)
This change introduces a significant refactor to the agentcontainers API
and enables periodic updates of Docker containers rather than on-demand.
Consequently this change also allows us to move away from using a
locking channel and replace it with a mutex, which simplifies usage.

Additionally a previous oversight was fixed, and testing added, to clear
devcontainer running/dirty status when the container has been removed.

Updates coder/coder#16424
Updates coder/internal#621
2025-05-22 19:44:33 +03:00
f825477a5c fix: add timeouts to test telemetry snapshot (#17879)
This PR ensures that waits on channels will time out according to the
test context, rather than waiting indefinitely. This should alleviate
the panic seen in https://github.com/coder/internal/issues/645 and, if
the deadlock recurs, allow the test to be retried automatically in CI.
2025-05-22 13:51:24 +02:00
34494fb330 chore: avoid depending on rbac in slim builds (#17959)
I noticed the `coder-vpn.dylib` (of course alongside the Agent/CLI binaries) had grown substantially (from 29MB to 37MB for the dylib), and discovered that importing RBAC in slim builds was the issue

This PR removes the dependency on RBAC in slim builds, and adds a compile-time check to ensure it can't be imported in the future:

```
$ make build
# github.com/coder/coder/v2/coderd/rbac
coderd/rbac/no_slim.go:8:2: initialization cycle: _DO_NOT_IMPORT_THIS_PACKAGE_IN_SLIM_BUILDS refers to itself
make: *** [Makefile:224: build/coder-slim_2.22.1-devel+7e46d24b4_linux_amd64] Error 1
```

Before and after for `coder-slim_darwin_arm64`:
```
$ gsa before after
┌───────────────────────────────────────────────────────────────────────────────────┐
│ Diff between before and after                                                     │
├─────────┬─────────────────────────────────────────┬──────────┬──────────┬─────────┤
│ PERCENT │ NAME                                    │ OLD SIZE │ NEW SIZE │ DIFF    │
├─────────┼─────────────────────────────────────────┼──────────┼──────────┼─────────┤
│ -100%   │ github.com/gorilla/mux                  │          │          │ +0 B    │
│ -100%   │ github.com/ammario/tlru                 │          │          │ +0 B    │
│ -100%   │ github.com/armon/go-radix               │          │          │ +0 B    │
│ -0.00%  │ gvisor.dev/gvisor                       │ 2.4 MB   │ 2.4 MB   │ -4 B    │
│ -0.21%  │ os                                      │ 155 kB   │ 155 kB   │ -328 B  │
│ -0.23%  │ regexp                                  │ 152 kB   │ 152 kB   │ -346 B  │
│ -0.04%  │ runtime                                 │ 876 kB   │ 876 kB   │ -372 B  │
│ -100%   │ github.com/rcrowley/go-metrics          │ 675 B    │          │ -675 B  │
│ -23.79% │ github.com/cespare/xxhash/v2            │ 3.0 kB   │ 2.3 kB   │ -715 B  │
│ -100%   │ github.com/agnivade/levenshtein         │ 1.4 kB   │          │ -1.4 kB │
│ -100%   │ github.com/go-ini/ini                   │ 1.5 kB   │          │ -1.5 kB │
│ -100%   │ github.com/xeipuuv/gojsonreference      │ 2.4 kB   │          │ -2.4 kB │
│ -100%   │ github.com/xeipuuv/gojsonpointer        │ 5.2 kB   │          │ -5.2 kB │
│ -2.43%  │ go.opentelemetry.io/otel                │ 316 kB   │ 309 kB   │ -7.7 kB │
│ -2.40%  │ slices                                  │ 381 kB   │ 372 kB   │ -9.2 kB │
│ -0.68%  │ crypto                                  │ 1.4 MB   │ 1.4 MB   │ -9.5 kB │
│ -100%   │ github.com/tchap/go-patricia/v2         │ 23 kB    │          │ -23 kB  │
│ -100%   │ github.com/yashtewari/glob-intersection │ 28 kB    │          │ -28 kB  │
│ -4.35%  │ <autogenerated>                         │ 754 kB   │ 721 kB   │ -33 kB  │
│ -100%   │ github.com/sirupsen/logrus              │ 72 kB    │          │ -72 kB  │
│ -2.56%  │ github.com/coder/coder/v2               │ 3.3 MB   │ 3.2 MB   │ -84 kB  │
│ -100%   │ github.com/gobwas/glob                  │ 107 kB   │          │ -107 kB │
│ -100%   │ sigs.k8s.io/yaml                        │ 244 kB   │          │ -244 kB │
│ -100%   │ github.com/open-policy-agent/opa        │ 2.2 MB   │          │ -2.2 MB │
├─────────┼─────────────────────────────────────────┼──────────┼──────────┼─────────┤
│ -7.79%  │ __go_buildinfo __DATA                   │ 18 kB    │ 17 kB    │ -1.4 kB │
│ -6.81%  │ __itablink __DATA_CONST                 │ 23 kB    │ 22 kB    │ -1.6 kB │
│ -6.61%  │ __typelink __DATA_CONST                 │ 71 kB    │ 66 kB    │ -4.7 kB │
│ -2.86%  │ __noptrdata __DATA                      │ 1.0 MB   │ 993 kB   │ -29 kB  │
│ -21.49% │ __data __DATA                           │ 320 kB   │ 251 kB   │ -69 kB  │
│ -6.19%  │ __rodata __DATA_CONST                   │ 6.0 MB   │ 5.6 MB   │ -372 kB │
│ -47.19% │ __rodata __TEXT                         │ 7.6 MB   │ 4.0 MB   │ -3.6 MB │
├─────────┼─────────────────────────────────────────┼──────────┼──────────┼─────────┤
│ -14.02% │ before                                  │ 50 MB    │ 43 MB    │ -7.0 MB │
│         │ after                                   │          │          │         │
└─────────┴─────────────────────────────────────────┴──────────┴──────────┴─────────┘
```
2025-05-22 19:48:23 +10:00
53e8e9c7cd fix: reduce cost of prebuild failure (#17697)
Relates to https://github.com/coder/coder/issues/17432

### Part 1:

Notes:
- `GetPresetsAtFailureLimit` SQL query is added, which is similar to
`GetPresetsBackoff`, they use same CTEs: `filtered_builds`,
`time_sorted_builds`, but they are still different.

- Query is executed on every loop iteration. We can consider marking
specific preset as permanently failed as an optimization to avoid
executing query on every loop iteration. But I decided don't do it for
now.

- By default `FailureHardLimit` is set to 3.

- `FailureHardLimit` is configurable. Setting it to zero - means that
hard limit is disabled.

### Part 2

Notes:
- `PrebuildFailureLimitReached` notification is added.
- Notification is sent to template admins.
- Notification is sent only the first time, when hard limit is reached.
But it will `log.Warn` on every loop iteration.
- I introduced this enum:
```sql
CREATE TYPE prebuild_status AS ENUM (
  'normal',           -- Prebuilds are working as expected; this is the default, healthy state.
  'hard_limited',     -- Prebuilds have failed repeatedly and hit the configured hard failure limit; won't be retried anymore.
  'validation_failed' -- Prebuilds failed due to a non-retryable validation error (e.g. template misconfiguration); won't be retried.
);
```
`validation_failed` not used in this PR, but I think it will be used in
next one, so I wanted to save us an extra migration.

- Notification looks like this:
<img width="472" alt="image"
src="https://github.com/user-attachments/assets/e10efea0-1790-4e7f-a65c-f94c40fced27"
/>

### Latest notification views:
<img width="463" alt="image"
src="https://github.com/user-attachments/assets/11310c58-68d1-4075-a497-f76d854633fe"
/>
<img width="725" alt="image"
src="https://github.com/user-attachments/assets/6bbfe21a-91ac-47c3-a9d1-21807bb0c53a"
/>
2025-05-21 15:16:38 -04:00
b7462fb256 feat: improve transaction safety in CompleteJob function (#17970)
This PR refactors the CompleteJob function to use database transactions
more consistently for better atomicity guarantees. The large function
was broken down into three specialized handlers:

- completeTemplateImportJob
- completeWorkspaceBuildJob
- completeTemplateDryRunJob

Each handler now uses the Database.InTx wrapper to ensure all database
operations for a job completion are performed within a single
transaction, preventing partial updates in case of failures.

Added comprehensive tests for transaction behavior for each job type.

Fixes #17694

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: Claude <noreply@anthropic.com>
2025-05-21 16:48:51 +02:00
3e7ff9d9e1 chore(coderd/rbac): add Action{Create,Delete}Agent to ResourceWorkspace (#17932) 2025-05-20 21:20:56 +01:00