Commit Graph

213 Commits

Author SHA1 Message Date
b675aec4dd feat: Add endpoint to get all workspaces a user can access (#1354)
This iterates through user organizations to get permitted
workspaces. This will allow admins to manage user workspaces!
2022-05-10 02:38:20 +00:00
57bb108465 feat: Add update user password endpoint (#1310) 2022-05-06 09:20:08 -05:00
d0293e4d33 feat: Implement list roles & enforce authorize examples (#1273) 2022-05-03 16:10:19 -05:00
e530ab2838 chore: add api specific 404 (#1272)
Prevents weird errors when routes are moved, like
https://github.com/coder/coder/issues/1205
2022-05-03 09:22:08 -05:00
55ad97bbd7 feat: add pprof and prometheus metrics to coder server (#1266) 2022-05-03 12:48:02 +00:00
fd49a18b47 feat: Add "state" command to pull and push workspace state (#1264)
It's possible for a workspace to become in an invalid state.
This is something we'll detect for jobs, and allow monitoring of.

These commands will allow admins to manually reconcile state.
2022-05-02 17:51:58 -05:00
81577f120a feat: Add web terminal with reconnecting TTYs (#1186)
* feat: Add web terminal with reconnecting TTYs

This adds a web terminal that can reconnect to resume sessions!
No more disconnects, and no more bad bufferring!

* Add xstate service

* Add the webpage for accessing a web terminal

* Add terminal page tests

* Use Ticker instead of Timer

* Active Windows mode on Windows
2022-04-29 17:30:10 -05:00
35211e2190 feat: Add user roles, but do not yet enforce them (#1200)
* chore: Rework roles to be expandable by name alone
2022-04-29 09:04:19 -05:00
441ffd6a0b feat: add PUT /api/v2/users/:user-id/suspend endpoint (#1154) 2022-04-26 09:00:07 -05:00
744a00a55d feat: Add GIT_COMMITTER information to agent env vars (#1171)
This makes setting up git a bit simpler, and users
can always override these values!

We'll probably add a way to disable our Git integration
anyways, so these could be part of that.
2022-04-25 20:03:54 -05:00
88669fd578 feat: Move workspaces under organizations (#1109)
This removes split ownership for workspaces. They are now
a resource of organizations and have a designated owner,
which is a user.

This enables simple administration for commands like:
- `coder stop ben/dev`
- `coder build logs colin/arch`

or if we decide to allow administrators to access workspaces,
they could even SSH using this syntax: `coder ssh colin/dev`.
2022-04-25 16:11:03 -05:00
2a57ea757a feat: add audit package (#1046) 2022-04-25 18:57:59 +00:00
a2dd618849 feat: Use environment variables and startup script in agent (#1147)
These values were ignored. Environment variables are applied to
new sessions, and are refreshed on reconnect. This is cool because
a workspace could be updated with new environment variables without
requiring a complete start/stop.

The startup script is only ran once regardless of changes, which
feels like the expected behavior.
2022-04-25 18:30:39 +00:00
7496c3da81 feat: Add GitHub OAuth (#1050)
* Initial oauth

* Add Github authentication

* Add AuthMethods endpoint

* Add frontend

* Rename basic authentication to password

* Add flags for configuring GitHub auth

* Remove name from API keys

* Fix authmethods in test

* Add stories and display auth methods error
2022-04-23 22:58:57 +00:00
548de7d6f3 feat: User pagination using offsets (#1062)
Offset pagination and cursor pagination supported
2022-04-22 15:27:55 -05:00
c8246e3e8a feat: Add Azure instance identitity authentication (#1064)
This enables zero-trust authentication for Azure instances. Now
we support the three major clouds: AWS, Azure, and GCP 😎.
2022-04-19 13:48:13 +00:00
d202f20fdb feat: Add TURN proxying to enable offline deployments (#1000)
* Add turnconn

* Add option for passing ICE servers

* Log TURN remote address

* Add TURN server to coder start
2022-04-18 22:40:25 +00:00
1df750bf1a feat: add GET /api/v2/users (#1028) 2022-04-18 17:19:47 +00:00
d9d4599ba9 chore: idea: unify http responses further (#941) 2022-04-12 10:17:33 -05:00
63d1465019 feat: Add update profile endpoint (#916) 2022-04-12 14:05:21 +00:00
19b4323512 feat: Allow workspace resources to attach multiple agents (#942)
This enables a "kubernetes_pod" to attach multiple agents that
could be for multiple services. Each agent is required to have
a unique name, so SSH syntax is:

`coder ssh <workspace>.<agent>`

A resource can have zero agents too, they aren't required.
2022-04-11 16:06:15 -05:00
94ab6f3d8e feat: add debug-level request logging (#923)
This commit adds a small middleware to coderd that logs all requests at DEBUG level.
2022-04-08 14:35:29 +00:00
18595791c0 feat: add version to footer (#882)
* Add endpoint for getting build info

* Add build info XService

* Add version with link to page footer

Partially addresses #376.

* Lift buildinfo package
2022-04-07 12:18:58 -05:00
23f989127d coderd: autostart: codersdk, http api, database plumbing (#879)
* feat: add columns autostart_schedule, autostop_schedule to database schema
* feat: database: add UpdateWorkspaceAutostart and UpdateWorkspaceAutostop methods
* feat: add AutostartSchedule/AutostopSchedule to api workspace struct
* feat: codersdk: implement update workspace autostart and autostop methods
* chore: add unit tests for workspace autostarat and autostop methods
2022-04-07 10:03:35 +01:00
02ad3f14f5 chore: Rename Projects to Templates (#880)
Customer feedback indicated projects was a confusing name.
After querying the team internally, it seemed unanimous
that it is indeed a confusing name.

Here's for a lil less confusion @ashmeer7 🥂
2022-04-06 12:42:40 -05:00
9da17be61e feat: Add user scoped git ssh keys (#834) 2022-04-06 00:18:26 +00:00
31536186f7 feat: Add rate-limits to the API (#848)
Closes #285.
2022-04-04 17:32:05 -05:00
0d53795c0d feat: Add strict transport security and secure cookie options (#741) 2022-03-31 12:31:06 -05:00
82dfd6c72f feat: Add UI for awaiting agent connections (#578)
* feat: Add stage to build logs

This adds a stage property to logs, and refactors the job logs
cliui.

It also adds tests to the cliui for build logs!

* feat: Add stage to build logs

This adds a stage property to logs, and refactors the job logs
cliui.

It also adds tests to the cliui for build logs!

* feat: Add config-ssh and tests for resiliency

* Rename "Echo" test to "ImmediateExit"

* Fix Terraform resource agent association

* Fix logs post-cancel

* Fix select on Windows

* Remove terraform init logs

* Move timer into it's own loop

* Fix race condition in provisioner jobs

* Fix requested changes
2022-03-28 19:19:28 -05:00
bd20d9ee7f feat: Add datadog tracing to http middleware (#530)
* add datadog tracing to http handlers
2022-03-28 22:11:52 +00:00
a502a5fa14 feat: Add AWS instance identity authentication (#570)
* feat: Add AWS instance identity authentication

This allows zero-trust authentication for all AWS instances.

Prior to this, AWS instances could be used by passing `CODER_TOKEN`
as an environment variable to the startup script. AWS explicitly
states that secrets should not be passed in startup scripts because
it's user-readable.

* Fix sha256 verbosity

* Fix HTTP client being exposed on auth
2022-03-28 19:31:03 +00:00
3a48e4000e fix: Race when shutting down and opening WebSockets (#576)
Adding to a WaitGroup while calling wait is a race condition. Surrounding
this in a mutex should solve the problem. Since context is used for
cancellation on all sockets, cleanup should occur properly.

See: https://github.com/coder/coder/runs/5701221057?check_suite_focus=true#step:10:98
2022-03-26 13:53:50 -05:00
591523a078 chore: Move httpapi, httpmw, & database into coderd (#568)
* chore: Move httpmw to /coderd directory
httpmw is specific to coderd and should be scoped under coderd

* chore: Move httpapi to /coderd directory
httpapi is specific to coderd and should be scoped under coderd

* chore: Move database  to /coderd directory
database is specific to coderd and should be scoped under coderd

* chore: Update codecov & gitattributes for generated files
* chore: Update Makefile
2022-03-25 16:07:45 -05:00
c451f4e685 feat: Add templates to create working release (#422)
* Add templates

* Move API structs to codersdk

* Back to green tests!

* It all works, but now with tea! 🧋

* It works!

* Add cancellation to provisionerd

* Tests pass!

* Add deletion of workspaces and projects

* Fix agent lock

* Add clog

* Fix linting errors

* Remove unused CLI tests

* Rename daemon to start

* Fix leaking command

* Fix promptui test

* Update agent connection frequency

* Skip login tests on Windows

* Increase tunnel connect timeout

* Fix templater

* Lower test requirements

* Fix embed

* Disable promptui tests for Windows

* Fix write newline

* Fix PTY write newline

* Fix CloseReader

* Fix compilation on Windows

* Fix linting error

* Remove bubbletea

* Cleanup readwriter

* Use embedded templates instead of serving over API

* Move templates to examples

* Improve workspace create flow

* Fix Windows build

* Fix tests

* Fix linting errors

* Fix untar with extracting max size

* Fix newline char
2022-03-22 13:17:50 -06:00
706e60bb3b chore: pluralize table names (#463) 2022-03-17 18:10:46 -05:00
ec077c6191 refactor: Migrate from Next.js to pure webpack config (#360)
Fix for #348 - migrate our NextJS project to a pure webpack project w/ a single bundle

- [x] Switch from `next/link` to `react-router-dom`'s link 

> This part was easy - just change the import to `import { Link } from "react-router-dom"` and `<Link href={...} />` to `<Link to={...} />`

- [x] Switch from `next/router` to `react-router-dom`'s paradigms (`useNavigation`, `useLocation`, and `useParams`)

> `router.push` can be converted to `navigate(...)` (provided by the `useNavigate` hook)
> `router.replace` can be converted `navigate(..., {replace: true})` 
>  Query parameters (`const { query } = useRouter`) can be converted to `const query = useParams()`)

- [x] Implement client-side routing with `react-router-dom`

> Parameterized routes in NextJS like `projects/[organization]/[project]` would look like:
> ```
>               <Route path="projects">
>                    <Route path=":organization/:project">
>                    <Route index element={<ProjectPage />} />
>                  </Route>
>               </Route>
> ```

I've hooked up a `build:analyze` command that spins up a server to show the bundle size:
<img width="1303" alt="image" src="https://user-images.githubusercontent.com/88213859/157496889-87c5fdcd-fad1-4f2e-b7b6-437aebf99641.png">

The bundle looks OK, but there are some opportunities for improvement - the heavy-weight dependencies, like React, ReactDOM, Material-UI, and lodash could be brought in via a CDN: https://stackoverflow.com/questions/50645796/how-to-import-reactjs-material-ui-using-a-cdn-through-webpacks-externals
2022-03-12 12:51:05 -08:00
bf0ae8f573 feat: Refactor API routes to use UUIDs instead of friendly names (#401)
* Add client for agent

* Cleanup code

* Fix linting error

* Rename routes to be simpler

* Rename workspace history to workspace build

* Refactor HTTP middlewares to use UUIDs

* Cleanup routes

* Compiles!

* Fix files and organizations

* Fix querying

* Fix agent lock

* Cleanup database abstraction

* Add parameters

* Fix linting errors

* Fix log race

* Lock on close wait

* Fix log cleanup

* Fix e2e tests

* Fix upstream version of opencensus-go

* Update coderdtest.go

* Fix coverpkg

* Fix codecov ignore
2022-03-07 11:40:54 -06:00
e5c95552cd feat: Remove magical parameters from being injected (#371)
* ci: Update DataDog GitHub branch to fallback to GITHUB_REF

This was detecting branches, but not our "main" branch before.
Hopefully this fixes it!

* Add basic Terraform Provider

* Rename post files to upload

* Add tests for resources

* Skip instance identity test

* Add tests for ensuring agent get's passed through properly

* Fix linting errors

* Add echo path

* Fix agent authentication

* fix: Convert all jobs to use a common resource and agent type

This enables a consistent API for project import and provisioned resources.

* Add "coder_workspace" data source

* feat: Remove magical parameters from being injected

This is a much cleaner abstraction. Explicitly declaring the user
parameters for each provisioner makes for significantly simpler
testing.
2022-02-28 18:26:01 +00:00
35ae532f7c feat: Add Terraform Provider for agent resources (#368)
* ci: Update DataDog GitHub branch to fallback to GITHUB_REF

This was detecting branches, but not our "main" branch before.
Hopefully this fixes it!

* Add basic Terraform Provider

* Rename post files to upload

* Add tests for resources

* Skip instance identity test

* Add tests for ensuring agent get's passed through properly

* Fix linting errors

* Add echo path

* Fix agent authentication

* Update codersdk/files.go

Co-authored-by: Bryan <bryan@coder.com>

Co-authored-by: Bryan <bryan@coder.com>
2022-02-28 17:16:44 +00:00
8958b641e9 feat: Add agent authentication based on instance ID (#336)
* feat: Add agent authentication based on instance ID

Each cloud has it's own unique instance identity signatures, which
can be used for zero-token authentication. This change adds support
for tracking by "instance_id", and automatically authenticating
with Google Cloud.

* Add test for CLI

* Fix workspace agent request name

* Fix race with adding to wait group

* Fix name of instance identity token
2022-02-21 20:36:29 +00:00
d04570ad29 fix: Use sync.WaitGroup to await hijacked HTTP connections (#337)
WebSockets hijack the HTTP connection from the server, causing
server.Close() to not wait for these connections to fully cleanup.

This adds a global wait-group to the coderd API, which ensures all
WebSocket HTTP handlers have properly exited before returning.
2022-02-20 16:29:16 -06:00
3f7781403d feat: Login via CLI (#298)
Fixes #210 - this isPR implements `coder login` in the case where the default user is already created.

This change adds:
- A prompt in the case where there is not an initial user that opens the server URL + requests a session token
  - This ports over some code from v1 for the `openURL` and `isWSL` functions to support opening the browser
- A `/api/v2/api-keys` endpoint that can be `POST`'d to in order to request a new api key for a user
  - This route was inspired by the v1 functionality
- A `cli-auth` route + page that shows the generated api key
- Tests for the new code + storybook for the new UI

The `/cli-auth` route, like in v1, is very minimal:

<img width="624" alt="Screen Shot 2022-02-16 at 5 05 07 PM" src="https://user-images.githubusercontent.com/88213859/154384627-78ab9841-27bf-490f-9bbe-23f8173c9e97.png">

And the terminal UX looks like this:

![2022-02-16 17 13 29](https://user-images.githubusercontent.com/88213859/154385225-509c78d7-840c-4cab-8f1e-074fede8f97e.gif)
2022-02-17 20:09:33 -08:00
154b9bce57 feat: Add "coder projects create" command (#246)
* Refactor parameter parsing to return nil values if none computed

* Refactor parameter to allow for hiding redisplay

* Refactor parameters to enable schema matching

* Refactor provisionerd to dynamically update parameter schemas

* Refactor job update for provisionerd

* Handle multiple states correctly when provisioning a project

* Add project import job resource table

* Basic creation flow works!

* Create project fully works!!!

* Only show job status if completed

* Add create workspace support

* Replace Netflix/go-expect with ActiveState

* Fix linting errors

* Use forked chzyer/readline

* Add create workspace CLI

* Add CLI test

* Move jobs to their own APIs

* Remove go-expect

* Fix requested changes

* Skip workspacecreate test on windows
2022-02-12 13:34:04 -06:00
07fe5ced68 feat: Add "coder" CLI (#221)
* feat: Add "coder" CLI

* Add CLI test for login

* Add "bin/coder" target to Makefile

* Update promptui to fix race

* Fix error scope

* Don't run CLI tests on Windows

* Fix requested changes
2022-02-10 08:33:27 -06:00
78bf4c6d21 feat: nextrouter pkg to handle nextjs routing rules (#167)
An issue came up last week... our `embed.go` strategy doesn't handle dynamic NextJS-style routes! This is a blocker, because I'm aiming to set up CD on Monday, and the v2 UI makes heavy use of dynamic routing.

As a potential solution, this implements a go pkg `nextrouter` that serves `html` files, but respecting the dynamic routing behavior of NextJS:
- Files that have square brackets - ie `[providers]` provide a single-level dynamic route
- Files that have `[[...` prefix - ie `[[...any]]` - are catch-all routes.
- Files should be preferred over folders (ie, `providers.html` is preferred over `/providers`)
- Fixes the trailing-slash bug we hit in the previous `embed` strategy

This also integrates with `slog.Logger` for tracing, and handles injecting template parameters - a feature we need in v1 and v2 to be able to inject stuff like CSRF tokens.

This implements testing by using an in-memory file-system, so that we can exercise all of these cases.

In addition, this adjust V2's `embed.go` strategy to use `nextrouter`, which simplifies that file considerably. I'm tempted to factor out the `secureheaders` logic into a separate package, too.

If this works OK, it could be used for V1 too (although that scenario is more complex due to our hybrid-routing strategy). Based on our FE variety meeting, there's always a chance we could move away from NextJS in v1 - if that's the case, this router will still work and be more tested than our previous strategy (it just won't make use of dynamic routing). So I figured this was worth doing to make sure we can make forward progress in V2.
2022-02-08 17:01:19 -08:00
7364933e65 refactor: Allow provisioner jobs to be disconnected from projects (#194)
* Nest jobs under an organization

* Rename project parameter to parameter schema

* Update references when computing project parameters

* Add files endpoint

* Allow one-off project import jobs

* Allow variables to be injected that are not defined by the schema

* Update API to use jobs first

* Fix CLI tests

* Fix linting

* Fix hex length for files table

* Reduce memory allocation for windows
2022-02-08 12:00:44 -06:00
ed705f6af2 refactor: Generalize log ownership to allow for scratch jobs (#182)
* refactor: Generalize log ownership to allow for scratch jobs

Importing may fail when creating a project. We don't want to lose this output,
but we don't want to allow users to create a failing project.

This generalizes logs to soon enable one-off situations where a user can upload
their archive, create a project, and watch the output parse to completion.

* Improve file table schema by using hash

* Fix racey test by allowing logs before

* Add debug logging for PostgreSQL insert
2022-02-07 15:32:37 -06:00
1796dc6c2f chore: Add test helpers to improve coverage (#166)
* chore: Rename ProjectHistory to ProjectVersion

Version more accurately represents version storage. This
forks from the WorkspaceHistory name, but I think it's
easier to understand Workspace history.

* Rename files

* Standardize tests a bit more

* Remove Server struct from coderdtest

* Improve test coverage for workspace history

* Fix linting errors

* Fix coderd test leak

* Fix coderd test leak

* Improve workspace history logs

* Standardize test structure for codersdk

* Fix linting errors

* Fix WebSocket compression

* Update coderd/workspaces.go

Co-authored-by: Bryan <bryan@coder.com>

* Add test for listing project parameters

* Cache npm dependencies with setup node

* Remove windows npm cache key

Co-authored-by: Bryan <bryan@coder.com>
2022-02-05 18:24:51 -06:00
f19770b2c6 chore: Rename ProjectHistory to ProjectVersion (#165)
* chore: Rename ProjectHistory to ProjectVersion

Version more accurately represents version storage. This
forks from the WorkspaceHistory name, but I think it's
easier to understand Workspace history.

* Rename files
2022-02-04 20:47:58 -06:00
65de6eef9c feat: Add streaming endpoint for workspace history (#157)
* feat: Add parameter querying to the API

* feat: Add streaming endpoint for workspace history

Enables a buildlog-like flow for reading job output.

* Fix empty parameter source and destination

* Add comment for usage of workspace history logs endpoint
2022-02-04 19:36:43 +00:00