Commit Graph

171 Commits

Author SHA1 Message Date
7ad87505c8 chore: move agent functions from codersdk into agentsdk (#5903)
* chore: rename `AgentConn` to `WorkspaceAgentConn`

The codersdk was becoming bloated with consts for the workspace
agent that made no sense to a reader. `Tailnet*` is an example
of these consts.

* chore: remove `Get` prefix from *Client functions

* chore: remove `BypassRatelimits` option in `codersdk.Client`

It feels wrong to have this as a direct option because it's so infrequently
needed by API callers. It's better to directly modify headers in the two
places that we actually use it.

* Merge `appearance.go` and `buildinfo.go` into `deployment.go`

* Merge `experiments.go` and `features.go` into `deployment.go`

* Fix `make gen` referencing old type names

* Merge `error.go` into `client.go`

`codersdk.Response` lived in `error.go`, which is wrong.

* chore: refactor workspace agent functions into agentsdk

It was odd conflating the codersdk that clients should use
with functions that only the agent should use. This separates
them into two SDKs that are closely coupled, but separate.

* Merge `insights.go` into `deployment.go`

* Merge `organizationmember.go` into `organizations.go`

* Merge `quota.go` into `workspaces.go`

* Rename `sse.go` to `serversentevents.go`

* Rename `codersdk.WorkspaceAppHostResponse` to `codersdk.AppHostResponse`

* Format `.vscode/settings.json`

* Fix outdated naming in `api.ts`

* Fix app host response

* Fix unsupported type

* Fix imported type
2023-01-29 15:47:24 -06:00
8735f51047 feat(coder): add authz_querier experiment (#5858)
* feat(coderd): add authz_querier experiment

* coderdtest: wire up authz_querier

* wire up AuthzQuerier in coderd

* remove things that do not yet exist in this timeline

* add newline

* comment unreachable code
2023-01-25 21:35:53 +00:00
56b996532f feat: add --experiments flag to replace --experimental (#5767)
- Deprecates the --experimental flag
- Adds a new flag --experiments which supports passing multiple comma-separated values or a wildcard value.
- Exposes a new endpoint /api/v2/experiments that returns the list of enabled experiments.
- Deprecates the field Features.Experimental in favour of this new API.
- Updates apidocgen to support type aliases (shoutout to @mtojek).
- Modifies apitypings to support generating slice types.
- Updates develop.sh to pass additional args after -- to $CODERD_SHIM.
2023-01-18 19:12:53 +00:00
bbc1a9a1d8 fix: use UserInfo endpoint with OIDC (#5735)
This resolves a user issue surfaced in Discord:
https://discord.com/channels/747933592273027093/1064566338875576361/1064566338875576361

Both methods of obtaining claims need to be used according
to the OIDC specification.
2023-01-16 16:06:39 -06:00
5a968e2f93 feat: add flag to disaable all rate limits (#5570) 2023-01-05 18:05:20 +00:00
04d45f3c1c fix!: remove AUTO_IMPORT_TEMPLATE for Kubernetes installs (#5401)
* fix!: remove AUTO_IMPORT_TEMPLATE

* chore: remove template auto importing

Co-authored-by: Dean Sheather <dean@deansheather.com>
2023-01-05 04:04:32 +00:00
de0601d611 feat: allow configurable username claim field in OIDC (#5507)
Co-authored-by: Colin Adler <colin1adler@gmail.com>
2023-01-04 15:16:31 -06:00
dc6d271293 feat: Build framework for generating API docs (#5383)
* WIP

* Gen

* WIP

* chi swagger

* WIP

* WIP

* WIP

* GetWorkspaces

* GetWorkspaces

* Markdown

* Use widdershins

* WIP

* WIP

* WIP

* Markdown template

* Fix: makefile

* fmt

* Fix: comment

* Enable swagger conditionally

* fix: site

* Default false

* Flag tests

* fix

* fix

* template fixes

* Fix

* Fix

* Fix

* WIP

* Formatted

* Cleanup

* Templates

* BEGIN END SECTION

* subshell exit code

* Fix

* Fix merge

* WIP

* Fix

* Fix fmt

* Fix

* Generic api.md page

* Fix merge

* Link pages

* Fix

* Fix

* Fix: links

* Add icon

* Write manifest file

* Fix fmt

* Fix: enterprise

* Fix: Swagger.Enable

* Fix: rename apidocs to apidoc

* Fix: find -not -prune

* Fix: json not available

* Fix: rename Coderd API to Coder API

* Fix: npm exec

* Fix: api dir

* Fix: by ID

* Fix: string uuid

* Fix: include deleted

* Fix: indirect go.mod

* Fix: source lib.sh

* Fix: shellcheck

* Fix: pushd popd

* Fix: fmt

* Fix: improve workspaces

* Fix: swagger-enable

* Fix

* Fix: mention only HTTP 200

* Fix: IDs

* Fix: https

* Fix: icon

* More APis

* Fix: format swagger.json

* Fix: SwaggerEndpoint

* Fix: SCRIPT_DIR

* Fix: PROJECT_ROOT

* Fix: use code tags in schemas.md

* Fix: examples

* Fix: examples

* Fix: improve format

* Fix: date-time,enums

* Fix: include_deleted

* Fix: array of

* Fix: parameter, response

* Fix: string time or null

* Workspaces: more docs

* Workspaces: more docs

* Fix: renderDisplayName

* Fix: ActiveUserCount

* Fix

* Fix: typo

* Templates: docs

* Notice: incomplete
2022-12-19 18:43:46 +01:00
787b8b2a51 fix: fix app hostname returning port number (#5441) 2022-12-16 04:43:00 +10:00
4fc4c01cea fix: Enable reconnectingpty loadtest and fix/improve logging (#5403)
* fix: Enable reconnectingpty loadtest and fix/improve logging

This commit re-enabled reconnectingpty loadtests after a logging
refactor of `(*agent).handleReconnectingPTY`. The reasons the tests were
flaking was that `logger.Error` was being called and `slogtest` failing
the test.

We could have set the option for `slogtest` to disable failing, but that
could hide real issues. The current approach improves reconnectingpty
logging overall and provides more insight into what's happening. It's
expected that reconnectingpty sessions fail after the agent is closed,
so calling `logger.Error` at that point is not wanted.

Ref: #5322
2022-12-13 21:28:07 +02:00
ab3b3d5fca feat: add debouncing to provisionerd rpc calls (#5198) 2022-12-01 16:54:53 -06:00
d9f2aaf3b4 feat: Add support for update checks and notifications (#4810)
Co-authored-by: Kira Pilot <kira@coder.com>
2022-12-01 19:43:28 +02:00
1f20cab110 fix: don't use yamux for in-memory provisioner{,d} streams (#5136) 2022-11-22 12:19:32 -06:00
fb9ca7b830 feat: Add the option to generate a trial license during setup (#5110)
This allows users to generate a 30 day free license during setup to
test out Enterprise features.
2022-11-16 17:09:49 -06:00
b6703b11c6 feat: Add external provisioner daemons (#4935)
* Start to port over provisioner daemons PR

* Move to Enterprise

* Begin adding tests for external registration

* Move provisioner daemons query to enterprise

* Move around provisioner daemons schema

* Add tags to provisioner daemons

* make gen

* Add user local provisioner daemons

* Add provisioner daemons

* Add feature for external daemons

* Add command to start a provisioner daemon

* Add provisioner tags to template push and create

* Rename migration files

* Fix tests

* Fix entitlements test

* PR comments

* Update migration

* Fix FE types
2022-11-16 16:34:06 -06:00
97dbd4dc5d Implement Quotas v3 (#5012)
* provisioner/terraform: add cost to resource_metadata

* provisionerd/runner: use Options struct

* Complete provisionerd implementation

* Add quota_allowance to groups

* Combine Quota and RBAC licenses

* Add Opts to InTx
2022-11-14 17:57:33 +00:00
49c7648af5 chore: Close idle connections on test cleanup (#4993)
It's possible that this was the source of a leak on Windows...
2022-11-13 14:06:03 -06:00
927c241995 fix: Debounce AcquireJob when no jobs are available (#5017)
This prevents constant database spam at scale to a maximum
of 60 queries/s per coderd instance.
2022-11-10 22:37:33 +00:00
26ab0d37c1 fix: Protect codersdk.Client SessionToken so it can be updated (#4965)
This feature is used by the coder agent to exchange a new token. By
protecting the SessionToken via mutex we ensure there are no data races
when accessing it.
2022-11-09 15:31:24 +02:00
2789fb7cac fix: move experimental flag to server (#4959) 2022-11-08 16:59:39 +00:00
eec406b739 feat: Add Git auth for GitHub, GitLab, Azure DevOps, and BitBucket (#4670)
* Add scaffolding

* Move migration

* Add endpoints for gitauth

* Add configuration files and tests!

* Update typesgen

* Convert configuration format for git auth

* Fix unclosed database conn

* Add overriding VS Code configuration

* Fix Git screen

* Write VS Code special configuration if providers exist

* Enable automatic cloning from VS Code

* Add tests for gitaskpass

* Fix feature visibiliy

* Add banner for too many configurations

* Fix update loop for oauth token

* Jon comments

* Add deployment config page
2022-10-24 19:46:24 -05:00
f75a54cd1e feat: Support x-forwarded-for headers for IPs (#4684)
* feat: Support x-forwarded-for headers for IPs

Fixes #4430.

* Fix realip accepting headers

* Fix unused headers
2022-10-23 13:21:49 -05:00
7bc5b89f7a feat: Support config files with viper (#4696) 2022-10-21 17:08:23 -05:00
e8537067ef Revert "Revert "Revert "feat: Support config files with viper"" (#4693)" (#4695)
This reverts commit 372fb1f345.
2022-10-21 16:07:38 -05:00
372fb1f345 Revert "Revert "feat: Support config files with viper"" (#4693) 2022-10-21 20:55:20 +00:00
a2fb444911 Revert "feat: Support config files with viper (#4558)" (#4692)
This reverts commit c8e299c8f1.
2022-10-21 20:04:27 +00:00
c8e299c8f1 feat: Support config files with viper (#4558) 2022-10-21 19:26:39 +00:00
61683f1961 fix: allow for alternate usernames on conflict (#4614) 2022-10-17 22:07:11 -05:00
18c4368571 chore: Move deployment UI and HA into experimental (#4595)
These are new, so they deserve a bit of gel-time!
2022-10-17 18:29:40 +00:00
2ba4a62a0d feat: Add high availability for multiple replicas (#4555)
* feat: HA tailnet coordinator

* fixup! feat: HA tailnet coordinator

* fixup! feat: HA tailnet coordinator

* remove printlns

* close all connections on coordinator

* impelement high availability feature

* fixup! impelement high availability feature

* fixup! impelement high availability feature

* fixup! impelement high availability feature

* fixup! impelement high availability feature

* Add replicas

* Add DERP meshing to arbitrary addresses

* Move packages to highavailability folder

* Move coordinator to high availability package

* Add flags for HA

* Rename to replicasync

* Denest packages for replicas

* Add test for multiple replicas

* Fix coordination test

* Add HA to the helm chart

* Rename function pointer

* Add warnings for HA

* Add the ability to block endpoints

* Add flag to disable P2P connections

* Wow, I made the tests pass

* Add replicas endpoint

* Ensure close kills replica

* Update sql

* Add database latency to high availability

* Pipe TLS to DERP mesh

* Fix DERP mesh with TLS

* Add tests for TLS

* Fix replica sync TLS

* Fix RootCA for replica meshing

* Remove ID from replicasync

* Fix getting certificates for meshing

* Remove excessive locking

* Fix linting

* Store mesh key in the database

* Fix replica key for tests

* Fix types gen

* Fix unlocking unlocked

* Fix race in tests

* Update enterprise/derpmesh/derpmesh.go

Co-authored-by: Colin Adler <colin1adler@gmail.com>

* Rename to syncReplicas

* Reuse http client

* Delete old replicas on a CRON

* Fix race condition in connection tests

* Fix linting

* Fix nil type

* Move pubsub to in-memory for twenty test

* Add comment for configuration tweaking

* Fix leak with transport

* Fix close leak in derpmesh

* Fix race when creating server

* Remove handler update

* Skip test on Windows

* Fix DERP mesh test

* Wrap HTTP handler replacement in mutex

* Fix error message for relay

* Fix API handler for normal tests

* Fix speedtest

* Fix replica resend

* Fix derpmesh send

* Ping async

* Increase wait time of template version jobd

* Fix race when closing replica sync

* Add name to client

* Log the derpmap being used

* Don't connect if DERP is empty

* Improve agent coordinator logging

* Fix lock in coordinator

* Fix relay addr

* Fix race when updating durations

* Fix client publish race

* Run pubsub loop in a queue

* Store agent nodes in order

* Fix coordinator locking

* Check for closed pipe

Co-authored-by: Colin Adler <colin1adler@gmail.com>
2022-10-17 13:43:30 +00:00
a029817d3d feat: allow suffix after wildcard in wildcard access URL (#4524) 2022-10-14 18:25:11 +00:00
4e57b9fbdc fix: allow regular users to push files (#4500)
- As part of merging support for Template RBAC
  and user groups a permission check on reading files
  was relaxed.

  With the addition of admin roles on individual templates, regular
  users are now able to push template versions if they have
  inherited the 'admin' role for a template. In order to do so
  they need to be able to create and read their own files. Since
  collisions on hash in the past were ignored, this means that a regular user
  who pushes a template version with a file hash that collides with
  an existing hash will not be able to read the file (since it belongs to
  another user).

  This commit fixes the underlying problem which was that
  the files table had a primary key on the 'hash' column.
  This was not a problem at the time because only template
  admins and other users with similar elevated roles were
  able to read all files regardless of ownership. To fix this
  a new column and primary key 'id' has been introduced to the files
  table. The unique constraint has been updated to be hash+created_by.
  Tables (provisioner_jobs) that referenced files.hash have been updated
  to reference files.id. Relevant API endpoints have also been updated.
2022-10-13 18:02:52 -05:00
3120c94c22 feat: add template RBAC/groups (#4235) 2022-10-10 15:37:06 -05:00
b1faaef482 feat: deployment flags (#4426) 2022-10-10 15:04:15 -04:00
738a38d71f chore: remove resources calls (#4344) 2022-10-03 21:01:13 +00:00
39cf329404 fix: Replace access URL for built-in DERP servers (#4197)
Fixes #4195.
2022-09-26 12:56:04 -05:00
3c215a83b6 feat: Allow admins to create workspaces (#4183)
Fixes #3263.

This is now possible via the API, but still isn't possible via the UI.
2022-09-23 20:17:10 -05:00
6deef06ad2 feat: secure and cross-domain subdomain-based proxying (#4136)
Co-authored-by: Kyle Carberry <kyle@carberry.com>
2022-09-22 22:30:32 +00:00
db0ba8588e chore: Refactor Enterprise code to layer on top of AGPL (#4034)
* chore: Refactor Enterprise code to layer on top of AGPL

This is an experiment to invert the import order of the Enterprise
code to layer on top of AGPL.

* Fix Garrett's comments

* Add pointer.Handle to atomically obtain references

This uses a context to ensure the same value persists through
multiple executions to `Load()`.

* Remove entitlements API from AGPL coderd

* Remove AGPL Coder entitlements endpoint test

* Fix warnings output

* Add command-line flag to toggle audit logging

* Fix hasLicense being set

* Remove features interface

* Fix audit logging default

* Add bash as a dependency

* Add comment

* Add tests for resync and pubsub, and add back previous exp backoff retry

* Separate authz code again

* Add pointer loading example from comment

* Fix duplicate test, remove pointer.Handle

* Fix expired license

* Add entitlements struct

* Fix context passing
2022-09-19 23:11:01 -05:00
714c366d16 chore: Remove WebRTC networking (#3881)
* chore: Remove WebRTC networking

* Fix race condition

* Fix WebSocket not closing
2022-09-19 19:46:29 -05:00
e847e7386a fix: Resolve flake in TestPortForward (#4069) 2022-09-14 20:21:53 -05:00
9ab437d6e2 feat: Add serving applications on subdomains and port-based proxying (#3753)
Co-authored-by: Dean Sheather <dean@deansheather.com>
2022-09-14 03:31:33 +10:00
4e26e325a6 feat: add auditing to user routes (#3961) 2022-09-08 21:16:16 -05:00
4f0105ef7e feat: add orphan support (#3849)
* feat: add resource orphanage

* feat: deny custom state in build for regular users

* Minor protoc improvements
2022-09-06 17:07:00 +00:00
67c4605370 chore: Reduce test times (#3856)
* chore: Reduce test times

* Rename IncludeProvisionerD to IncludeProvisionerDaemon

* Make  TestTemplateDAUs use Tailnet
2022-09-04 11:28:09 -05:00
30f8fd9b95 Daily Active User Metrics (#3735)
* agent: add StatsReporter

* Stabilize protoc
2022-09-01 14:58:23 -05:00
9bd83e5ec7 feat: Add Tailscale networking (#3505)
* fix: Add coder user to docker group on installation

This makes for a simpler setup, and reduces the likelihood
a user runs into a strange issue.

* Add wgnet

* Add ping

* Add listening

* Finish refactor to make this work

* Add interface for swapping

* Fix conncache with interface

* chore: update gvisor

* fix tailscale types

* linting

* more linting

* Add coordinator

* Add coordinator tests

* Fix coordination

* It compiles!

* Move all connection negotiation in-memory

* Migrate coordinator to use net.conn

* Add closed func

* Fix close listener func

* Make reconnecting PTY work

* Fix reconnecting PTY

* Update CI to Go 1.19

* Add CLI flags for DERP mapping

* Fix Tailnet test

* Rename ConnCoordinator to TailnetCoordinator

* Remove print statement from workspace agent test

* Refactor wsconncache to use tailnet

* Remove STUN from unit tests

* Add migrate back to dump

* chore: Upgrade to Go 1.19

This is required as part of #3505.

* Fix reconnecting PTY tests

* fix: update wireguard-go to fix devtunnel

* fix migration numbers

* linting

* Return early for status if endpoints are empty

* Update cli/server.go

Co-authored-by: Colin Adler <colin1adler@gmail.com>

* Update cli/server.go

Co-authored-by: Colin Adler <colin1adler@gmail.com>

* Fix frontend entites

* Fix agent bicopy

* Fix race condition for the last node

* Fix down migration

* Fix connection RBAC

* Fix migration numbers

* Fix forwarding TCP to a local port

* Implement ping for tailnet

* Rename to ForceHTTP

* Add external derpmapping

* Expose DERP region names to the API

* Add global option to enable Tailscale networking for web

* Mark DERP flags hidden while testing

* Update DERP map on reconnect

* Add close func to workspace agents

* Fix race condition in upstream dependency

* Fix feature columns race condition

Co-authored-by: Colin Adler <colin1adler@gmail.com>
2022-08-31 20:09:44 -05:00
14a9576b77 Auto import kubernetes template in Helm charts (#3550) 2022-08-26 05:32:35 +10:00
9e35793b43 Enterprise rbac testing (#3653)
* WIP refactor Auth tests to allow enterprise

Signed-off-by: Spike Curtis <spike@coder.com>

* enterprise RBAC testing

Signed-off-by: Spike Curtis <spike@coder.com>

* Fix import ordering

Signed-off-by: Spike Curtis <spike@coder.com>

Signed-off-by: Spike Curtis <spike@coder.com>
2022-08-24 12:05:46 -07:00
b101a6f3f4 POST license API endpoint (#3570)
* POST license API

Signed-off-by: Spike Curtis <spike@coder.com>

* Support interface{} types in generated Typescript

Signed-off-by: Spike Curtis <spike@coder.com>

* Disable linting on empty interface any

Signed-off-by: Spike Curtis <spike@coder.com>

* Code review updates

Signed-off-by: Spike Curtis <spike@coder.com>

* Enforce unique licenses

Signed-off-by: Spike Curtis <spike@coder.com>

* Renames from code review

Signed-off-by: Spike Curtis <spike@coder.com>

* Code review renames and comments

Signed-off-by: Spike Curtis <spike@coder.com>

Signed-off-by: Spike Curtis <spike@coder.com>
2022-08-22 15:02:50 -07:00