571 Commits

Author SHA1 Message Date
8ea956fc11 feat: add app status tracking to the backend (#17163)
This does ~95% of the backend work required to integrate the AI work.

Most left to integrate from the tasks branch is just frontend, which
will be a lot smaller I believe.

The real difference between this branch and that one is the abstraction
-- this now attaches statuses to apps, and returns the latest status
reported as part of a workspace.

This change enables us to have a similar UX to in the tasks branch, but
for agents other than Claude Code as well. Any app can report status
now.
2025-03-31 10:55:44 -04:00
06e5d9ef21 feat(coderd): add webpush package (#17091)
* Adds `codersdk.ExperimentWebPush` (`web-push`)
* Adds a `coderd/webpush` package that allows sending native push
notifications via `github.com/SherClockHolmes/webpush-go`
* Adds database tables to store push notification subscriptions.
* Adds an API endpoint that allows users to subscribe/unsubscribe, and
send a test notification (404 without experiment, excluded from API docs)
* Adds server CLI command to regenerate VAPID keys (note: regenerating
the VAPID keypair requires deleting all existing subscriptions)

---------

Co-authored-by: Kyle Carberry <kyle@carberry.com>
2025-03-27 10:03:53 +00:00
cf10d98aab fix: improve error message when deleting organization with resources (#17049)
Closes
[coder/internal#477](https://github.com/coder/internal/issues/477)

![Screenshot 2025-03-21 at 11 25
57 AM](https://github.com/user-attachments/assets/50cc03e9-395d-4fc7-8882-18cb66b1fac9)

I'm solving this issue in two parts:

1. Updated the postgres function so that it doesn't omit 0 values in the
error
2. Created a new query to fetch the number of resources associated with
an organization and using that information to provider a cleaner error
message to the frontend

> **_NOTE:_** SQL is not my strong suit, and the code was created with
the help of AI. So I'd take extra time looking over what I wrote there
2025-03-25 15:31:24 -04:00
5c8cac9fb7 feat: add name to workspace agent devcontainers (#17089)
In the presence of multiple devcontainers, it would be nice to
differentiate them by name. This change inherits the resource name from
terraform.

Refs #17076
2025-03-25 12:59:20 +00:00
4c33846f6d chore: add prebuilds system user (#16916)
Pre-requisite for https://github.com/coder/coder/pull/16891

Closes https://github.com/coder/internal/issues/515

This PR introduces a new concept of a "system" user.

Our data model requires that all workspaces have an owner (a `users`
relation), and prebuilds is a feature that will spin up workspaces to be
claimed later by actual users - and thus needs to own the workspaces in
the interim.

Naturally, introducing a change like this touches a few aspects around
the codebase and we've taken the approach _default hidden_ here; in
other words, queries for users will by default _exclude_ all system
users, but there is a flag to ensure they can be displayed. This keeps
the changeset relatively small.

This user has minimal permissions (it's equivalent to a `member` since
it has no roles). It will be associated with the default org in the
initial migration, and thereafter we'll need to somehow ensure its
membership aligns with templates (which are org-scoped) for which it'll
need to provision prebuilds; that's a solution we'll have in a
subsequent PR.

---------

Signed-off-by: Danny Kopping <dannykopping@gmail.com>
Co-authored-by: Sas Swart <sas.swart.cdk@gmail.com>
2025-03-25 12:18:06 +00:00
5b3eda6719 chore: persist template import terraform plan in postgres (#17012) 2025-03-24 10:01:50 -06:00
a71aa202dc feat: filter users by github user id in the users list CLI command (#17029)
Add the `--github-user-id` option to `coder users list`, which makes the
command only return users with a matching GitHub user id. This will
enable https://github.com/coder/start-workspace-action to find a Coder
user that corresponds to a GitHub user requesting to start a workspace.
2025-03-21 13:30:47 +01:00
69ba27e347 feat: allow specifying devcontainer on agent in terraform (#16997)
This change allows specifying devcontainers in terraform and plumbs it
through to the agent via agent manifest.

This will be used for autostarting devcontainers in a workspace.

Depends on coder/terraform-provider-coder#368
Updates #16423
2025-03-20 19:09:39 +02:00
72d9876c76 fix(coderd/workspaceapps): prevent race in workspace app audit session updates (#17020)
Fixes coder/internal#520
2025-03-20 14:10:45 +00:00
4960a1e85a feat(coderd): add mark-all-as-read endpoint for inbox notifications (#16976)
[Resolve this issue](https://github.com/coder/internal/issues/506)

Add a mark-all-as-read endpoint which is marking as read all
notifications that are not read for the authenticated user.
Also adds the DB logic.
2025-03-20 13:41:54 +01:00
ef62e626c8 fix: ensure targets are propagated to inbox (#16985)
Currently the `targets` column in `inbox_notifications` doesn't get
filled. This PR fixes that. Rather than give targets special treatment,
we should put it in the payload like everything else. This correctly
propagates notification targets to the inbox table without much code
change.
2025-03-19 09:51:49 +00:00
de41bd6b95 feat: add support for workspace app audit (#16801)
This change adds support for workspace app auditing.

To avoid audit log spam, we introduce the concept of app audit sessions.
An audit session is unique per workspace app, user, ip, user agent and
http status code. The sessions are stored in a separate table from audit
logs to allow use-case specific optimizations. Sessions are ephemeral
and the table does not function as a log.

The logic for auditing is placed in the DBTokenProvider for workspace
apps so that wsproxies are included.

This is the final change affecting the API fo #15139.

Updates #15139
2025-03-18 13:50:52 +02:00
3ae55bbbf4 feat(coderd): add inbox notifications endpoints (#16889)
This PR is part of the inbox notifications topic, and rely on previous
PRs merged - it adds :

- Endpoints to : 
  - WS : watch new inbox notifications
  - REST : list inbox notifications
  - REST : update the read status of a notification

Also, this PR acts as a follow-up PR from previous work and : 

- fix DB query issues
- fix DBMem logic to match DB
2025-03-18 00:02:47 +01:00
8c0350e20c feat: add a paginated organization members endpoint (#16835)
Closes
[coder/internal#460](https://github.com/coder/internal/issues/460)
2025-03-10 14:42:07 -04:00
eddccbca5c fix: hide deleted users from org members query (#16830) 2025-03-06 11:50:08 -07:00
522181fead feat(coderd): add new dispatch logic for coder inbox (#16764)
This PR is [resolving the dispatch part of Coder
Inbocx](https://github.com/coder/internal/issues/403).

Since the DB layer has been merged - we now want to insert notifications
into Coder Inbox in parallel of the other delivery target.

To do so, we push two messages instead of one using the `Enqueue`
method.
2025-03-05 22:43:18 +01:00
9041646b81 chore: add "user_configs" db table (#16564) 2025-03-05 10:46:03 -07:00
24f3445e00 chore: track workspace resource monitors in telemetry (#16776)
Addresses https://github.com/coder/nexus/issues/195. Specifically, just
the "tracking templates" requirement:

> ## Tracking in templates
> To enable resource alerts, a user must add the resource_monitoring
block to a template's coder_agent resource. We'd like to track if
customers have any resource monitoring enabled on a per-deployment
basis. Even better, we could identify which templates are using resource
monitoring.
2025-03-03 18:41:01 +01:00
b85ba586ee fix(coderd/database): consider tag sets when calculating queue position (#16685)
Relates to https://github.com/coder/coder/issues/15843

## PR Contents

- Reimplementation of the `GetProvisionerJobsByIDsWithQueuePosition` SQL
query to **take into account** provisioner job tags and provisioner
daemon tags.
- Unit tests covering different **tag sets**, **job statuses**, and
**job ordering** scenarios.

## Notes

- The original row order is preserved by introducing the `ordinality`
field.
- Unnecessary rows are filtered as early as possible to ensure that
expensive joins operate on a smaller dataset.
- A "fake" join with `provisioner_jobs` is added at the end to ensure
`sqlc.embed` compiles successfully.
- **Backward compatibility is preserved**—only the SQL query has been
updated, while the Go code remains unchanged.
2025-03-03 10:02:18 -05:00
c074f77a4f feat: add notifications inbox db (#16599)
This PR is linked [to the following
issue](https://github.com/coder/internal/issues/334).

The objective is to create the DB layer and migration for the new `Coder
Inbox`.
2025-03-03 10:12:48 +01:00
91a4a98c27 chore: add an unassign action for roles (#16728) 2025-02-27 10:39:06 -07:00
6dd51f92fb chore: test metricscache on postgres (#16711)
metricscache_test has been running tests against dbmem only, instead of
against postgres. Unfortunately the implementations of
GetTemplateAverageBuildTime have diverged between dbmem and postgres.
This change gets the tests working on Postgres and test for the
behaviour postgres provides.
2025-02-27 09:43:51 +00:00
7cd6e9cdd6 fix: return provisioners in desc order and add limit to cli (#16720) 2025-02-26 21:06:51 +02:00
d3a56ae3ef feat: enable GitHub OAuth2 login by default on new deployments (#16662)
Third and final PR to address
https://github.com/coder/coder/issues/16230.

This PR enables GitHub OAuth2 login by default on new deployments.
Combined with https://github.com/coder/coder/pull/16629, this will allow
the first admin user to sign up with GitHub rather than email and
password.

We take care not to enable the default on deployments that would upgrade
to a Coder version with this change.

To disable the default provider an admin can set the
`CODER_OAUTH2_GITHUB_DEFAULT_PROVIDER` env variable to false.
2025-02-25 16:31:33 +01:00
546a549dcf feat: enable soft delete for organizations (#16584)
- Add deleted column to organizations table
- Add trigger to check for existing workspaces, templates, groups and
members in a org before allowing the soft delete

---------

Co-authored-by: Steven Masley <stevenmasley@gmail.com>
Co-authored-by: Steven Masley <Emyrk@users.noreply.github.com>
2025-02-24 12:59:41 -05:00
d6b9806098 chore: implement oom/ood processing component (#16436)
Implements the processing logic as set out in the OOM/OOD RFC.
2025-02-17 16:56:52 +00:00
b5329ae1cd feat: add workspace agent connect and app open audit types (#16493)
This commit adds new audit resource types for workspace agents and
workspace apps, as well as connect/disconnect and open/close actions.

The idea is that we will log new audit events for connecting to the
agent via SSH/editor.

Likewise, we will log openings of `coder_app`s.

This change also introduces support for filtering by `request_id`.

Updates #15139
2025-02-17 13:02:30 +00:00
5ba7ba6bfc fix(coderd): add strict org ID joins for provisioner job metadata (#16588)
References #16558
2025-02-17 14:16:45 +02:00
77306f3de1 feat(coderd): add filters and fix template for provisioner daemons (#16558)
This change adds provisioner daemon ID filter to the provisioner daemons
endpoint, and also implements the limiting to 50 results.

Test coverage is greatly improved and template information for jobs
associated to the daemon was also fixed.

Updates #15084
Updates #15192
Related #16532
2025-02-14 17:26:46 +02:00
e38bd27183 feat(coderd): add support for provisioner job id and tag filter (#16556)
This change adds to new filters to the provisionerjobs endpoint, id
(array) and tags (map).

Updates #15084
Updates #15192
Related #16532
2025-02-13 18:24:27 +02:00
71cbf735e5 feat(coderd): add support for presets to the coder API (#16526)
This pull request builds on the existing migrations and queries to add
support for presets to the coder API.
2025-02-12 14:41:14 +02:00
e85da8b02b chore: return template data for provisioner daemons (#16514)
Return template data in provisioner jobs to be displayed in the
provisioners page.
2025-02-11 14:22:34 -03:00
34b46f9205 feat(coderd/database): add support for presets (#16509)
This pull requests adds the necessary migrations and queries to support
presets within the coderd database. Future PRs will build functionality
to the provisioners and the frontend.
2025-02-11 13:55:09 +02:00
e9b3561677 refactor: return template_icon and make metadata required (#16496) 2025-02-10 10:00:34 -03:00
b04d883348 feat: add provisioner job metadata (#16454)
This change adds metadata to provisioner jobs to help with rendering
related tempaltes and workspaces in the UI.

Updates #15084
2025-02-06 16:19:20 +02:00
7cbd77fd94 feat: improve resources_monitoring for OOM & OOD monitoring (#16241)
As requested for [this
issue](https://github.com/coder/internal/issues/245) we need to have a
new resource `resources_monitoring` in the agent.

It needs to be parsed from the provisioner and inserted into a new db
table.
2025-02-04 18:45:33 +01:00
2ace044e0b chore: track the first time html is served in telemetry (#16334)
Addresses https://github.com/coder/nexus/issues/175.

## Changes

- Adds the `telemetry_items` database table. It's a key value store for
telemetry events that don't fit any other database tables.
- Adds a telemetry report when HTML is served for the first time in
`site.go`.
2025-01-31 13:55:46 +01:00
75c899ff71 feat(cli): add provisioner job cancel command (#16252)
Fixes #16117
Updates #15084
2025-01-27 16:26:56 +00:00
5762d8add4 fix: return only the first workspace agent script timing per script (#16203)
Fixes https://github.com/coder/coder/issues/16124

If a workspace agent crashes, it is possible for any startup scripts to
be ran again. This PR makes it so that the
`GetWorkspaceAgentScriptTimingsByBuildID` query only returns the first
timing recorded per-script.
2025-01-21 11:54:43 +00:00
3864c7e3b0 feat(coderd): add endpoint to list provisioner jobs (#16029)
Closes #15190
Updates #15084
2025-01-20 11:18:53 +02:00
50bf5ca8fe fix(coderd/database): aggregate user engagement statistics by interval (#16150)
This PR addresses the TODO comment here:

https://github.com/coder/coder/pull/16134/files#diff-1844f895bb005f036da11d800fe2a76b54bfddd481c5d8cb15f210c64679caa5R47

The backend now backfills entries for dates with no status changes.
2025-01-16 17:34:53 +02:00
ab880904e2 feat(coderd/database): order user engagement statistics by date (#16142)
This PR orders user engagement statistics by date as mentioned
[here](https://github.com/coder/coder/pull/16134/files#diff-1844f895bb005f036da11d800fe2a76b54bfddd481c5d8cb15f210c64679caa5R47)
2025-01-15 13:19:25 +02:00
071bb26018 feat(coderd): add endpoint to list provisioner daemons (#16028)
Updates #15190
Updates #15084
Supersedes #15940
2025-01-14 16:40:26 +00:00
4543b21b7c feat(coderd/database): track user status changes over time (#16019)
RE: https://github.com/coder/coder/issues/15740,
https://github.com/coder/coder/issues/15297

In order to add a graph to the coder frontend to show user status over
time as an indicator of license usage, this PR adds the following:

* a new `api.insightsUserStatusCountsOverTime` endpoint to the API
* which calls a new `GetUserStatusCountsOverTime` query from postgres
* which relies on two new tables `user_status_changes` and
`user_deleted`
* which are populated by a new trigger and function that tracks updates
to the users table

The chart itself will be added in a subsequent PR

---------

Co-authored-by: Mathias Fredriksson <mafredri@gmail.com>
2025-01-13 13:08:16 +02:00
106b1cd3bc chore: convert dbauthz tests to also run with Postgres (#15862)
Another PR to address https://github.com/coder/coder/issues/15109.

- adds the DisableForeignKeysAndTriggers utility, which simplifies
converting tests from in-mem to postgres
- converts the dbauthz test suite to pass on both the in-mem db and
Postgres
2025-01-08 16:22:51 +01:00
08463c27d8 feat: add OpenIn option to coder_app (#15743)
This PR is the coder/coder part of [the open_in parameter
issue](https://github.com/coder/terraform-provider-coder/issues/297)
aiming to add a new optional parameter to choose how to open modules.

This PR is heavily linked [to this
PR](https://github.com/coder/terraform-provider-coder/pull/321).

ℹ️ For now, some integrations tests can not be pushed as it requires a
release on the terraform-provider repo.
2025-01-03 11:27:02 +01:00
e191d9650c feat: support created_at filter for the GET /users endpoint (#15633)
Closes https://github.com/coder/coder/issues/12747

We support these filters currently:
https://coder.com/docs/v2/latest/admin/users#user-filtering, adding
`created_at` filter as well.
2024-12-17 15:24:54 +11:00
36c2cf8a40 fix(coderd/database): exclude canceled jobs in queue position (#15835)
When calculating the queue position in
`GetProvisionerJobsByIDsWithQueuePosition` we only counted jobs with
`started_at = NULL`. This is misleading, as it allows canceling or
canceled jobs to take up rows in the computed queue position, giving an
impression that the queue is larger than it really is.

This modifies the query to also exclude jobs with a null `canceled_at`,
`completed_at`, or `error` field for the purposes of calculating the
queue position, and also adds a test to validate this behaviour.

(Note: due to the behaviour of `dbgen.ProvisionerJob` with `dbmem` I had
to use other proxy methods to validate the corresponding dbmem
implementation.)

---------

Co-authored-by: Mathias Fredriksson <mafredri@gmail.com>
2024-12-12 12:37:45 +00:00
b39becba66 feat(site): add a provisioner warning to workspace builds (#15686)
This PR adds warnings about provisioner health to workspace build pages.
It closes https://github.com/coder/coder/issues/15048


![image](https://github.com/user-attachments/assets/fa54d0e8-c51f-427a-8f66-7e5dbbc9baca)

![image](https://github.com/user-attachments/assets/b5169669-ab05-43d5-8553-315a3099b4fd)
2024-12-11 13:38:13 +02:00
40624bf78b fix: update workspace TTL on template TTL change (#15761)
Relates to https://github.com/coder/coder/issues/15390

Currently when a user creates a workspace, their workspace's TTL is
determined by the template's default TTL. If the Coder instance is AGPL,
or if the template has disallowed the user from configuring autostop,
then it is not possible to change the workspace's TTL after creation.
Any changes to the template's default TTL only takes effect on _new_
workspaces.

This PR modifies the behaviour slightly so that on AGPL Coder, or on
enterprise when a template does not allow user's to configure their
workspace's TTL, updating the template's default TTL will also update
any workspace's TTL to match this value.
2024-12-06 11:01:39 +00:00