Commit Graph

143 Commits

Author SHA1 Message Date
7e8692b0fd fix: Update routing for workspace schedule (#2113)
* fix: Update routing for workspace schedule

This was broken as part of #2101. It was a silly mistake,
but unfortunate our tests didn't catch it.

This is a rare change so unlikely to occur again, so I won't
make an issue adding tests.

* Update site/src/pages/WorkspaceSchedulePage/WorkspaceSchedulePage.tsx

Co-authored-by: Presley Pizzo <1290996+presleyp@users.noreply.github.com>

Co-authored-by: Presley Pizzo <1290996+presleyp@users.noreply.github.com>
2022-06-07 14:28:47 +00:00
3e419ddb3d feat: enforce template-level constraints for TTL and autostart (#2018)
This PR adds fields to templates that constrain values for workspaces derived from that template.

- Autostop: Adds a field max_ttl on the template which limits the maximum value of ttl on all workspaces derived from that template. Defaulting to 168 hours, enforced on edits to workspace metadata. New workspaces will default to the templates's `max_ttl` if not specified.
- Autostart: Adds a field min_autostart_duration which limits the minimum duration between successive autostarts of a template, measured from a single reference time. Defaulting to 1 hour, enforced on edits to workspace metadata.
2022-06-07 13:37:45 +01:00
37b0aaa018 fix: add workspace option 'deleted' to options type (#2095)
* fix: add workspace option 'deleted' to options type

* dead code
2022-06-06 17:23:02 +00:00
c9a4642a12 chore: Update BE http errors to be ui friendly (#1994)
* chore: More UI friendly errors

Mainly capitlization + messages prefix error
2022-06-03 21:48:09 +00:00
43f622a52d fix: Remove unused workspace routes in favor of list with filter (#2038)
* fix: Remove unused workspace routes in favor of list with filter

This consolidates the workspace routes into a single place.
It allows users to fetch a workspace by their username and
workspace name, which will be used by the frontend for routing.

* Fix RBAC

* Fix CLI usages
2022-06-03 14:36:08 -05:00
8b03e2b0e1 feat: Workspaces filtering (#1972)
Co-authored-by: G r e y <grey@coder.com>
Co-authored-by: Kira Pilot <kira@coder.com>
2022-06-03 17:20:28 +00:00
dcf03d8ba3 chore: refactor time.Duration -> int64 milliseconds for FE consumption (#1944)
* Changes all public-facing codersdk types to use a plain int64 (milliseconds) instead of time.Duration.
* Makes autostart_schedule a *string as it may not be present.
* Adds a utils/ptr package with some useful methods.
2022-06-02 11:23:34 +01:00
9e3a625898 Show workspace name in WorkspaceBuildStats component (#1933)
* Show workspace name in WorkspaceBuildStats component

Signed-off-by: Spike Curtis <spike@coder.com>

* Fix WorkspaceBuildPage tests

Signed-off-by: Spike Curtis <spike@coder.com>
2022-06-01 16:49:43 -07:00
a8ae9b39b3 feat: enforce upper bounds on workspace TTL and Deadline (#1902)
* Enforces upper bound for workspace TTL
* Enforces upper bound for workspace deadline
2022-05-30 20:19:17 +01:00
ff542afe87 feat: allow bumping workspace deadline (#1828)
* Adds a `bump` command to extend workspace build deadline
 * Reduces WARN-level logging spam from autobuild executor
 * Modifies `cli/ssh` notifications to read from workspace build deadline and to notify relative time instead (sidestepping the problem of figuring out a user's timezone across multiple OSes)
 * Shows workspace extension time in `coder list` output e.g.
    ```
    WORKSPACE        TEMPLATE  STATUS   LAST BUILT  OUTDATED  AUTOSTART        TTL        
    developer/test1  docker    Running  4m          false     0 9 * * MON-FRI  15m (+5m)  
    ```
2022-05-27 20:04:33 +01:00
ec1fe46138 feat: Move create organizations route (#1831)
* feat: last rbac routes
- move create organization to /organizations.
2022-05-27 11:19:13 -05:00
8f0a5a81f1 feat: add API/SDK support for autostop extension (#1778)
* Adds deadline column to workspace_builds, associated DB/API plumbing
* database: Upon inserting a row into workspace_builds, deadline will 
  initially be zero.
* autobuild: Executor now checks the Deadline field of the workspace_build
  for the purpose of autostop logic.
* coderd: Adds a new route /api/v2/workspaces/:workspace/extend which allows
  updating the deadline of the currently active workspace build. The new
  deadline must be after the existing deadline, and not the zero time.
* provisionerd: updates workspace_build.deadline upon successful workspace 
  build completion (equal to now plus workspace TTL, if it exists).
2022-05-26 18:08:11 +01:00
31b819e83f chore: Remove interface from coderd and lift API surface (#1772)
Abstracting coderd into an interface added misdirection because
the interface was never intended to be fulfilled outside of a single
implementation.

This lifts the abstraction, and attaches all handlers to a root struct
named `*coderd.API`.
2022-05-26 03:14:08 +00:00
c7ca86d374 feat: Implement RBAC checks on /templates endpoints (#1678)
* feat: Generic Filter method for rbac objects
2022-05-24 08:43:34 -05:00
b2020761d9 feat: add default autostart and ttl for new workspaces (#1632)
* database: add autostart_schedule and ttl to InsertWorkspace; make gen
* coderd: workspaces: consume additional fields of CreateWorkspaceRequest
* cli: update: add support for TTL and autostart_schedule
* cli: create: add unit tests
* coder: import  `time/tzdata` for embedded timezone database
* autobuild: fix unit test that only runs with a real db
2022-05-23 23:31:41 +01:00
d72c45e483 refactor: workspace autostop_schedule -> ttl (#1578)
Co-authored-by: G r e y <grey@coder.com>
2022-05-19 15:09:27 -04:00
6c1117094d chore: Force codersdk to not import anything from database (#1576)
* chore: Force codersdk to not import anything from database (linter rule)
* chore: Move all database types in codersdk out
2022-05-19 13:04:44 -05:00
67333b6186 feat: getWorkspaces filter site api (#1564) 2022-05-19 12:08:55 -04:00
4eb0bb6afd feat: don't return 200 for deleted workspaces (#1556) 2022-05-20 00:29:10 +10:00
c034e8389e feat: Add RBAC to /workspace endpoints (#1566)
* feat: Add RBAC to /workspace endpoints
2022-05-18 18:15:19 -05:00
0706c60445 chore: Add watch workspace endpoint (#1493) 2022-05-18 16:16:26 -05:00
9f402fa27f Spike/222 workspace build order (#1534)
* chore: refactor before_id/after_id to build_number

Signed-off-by: Spike Curtis <spike@coder.com>

* pagination of workspace_builds

Signed-off-by: Spike Curtis <spike@coder.com>

* Disable parallel on postgres tests

Signed-off-by: Spike Curtis <spike@coder.com>

* Fix lint

Signed-off-by: Spike Curtis <spike@coder.com>

* Fix workspace build postgres query

Signed-off-by: Spike Curtis <spike@coder.com>

* Fix JS tests

Signed-off-by: Spike Curtis <spike@coder.com>

* Fix workspace builds postgres query

Signed-off-by: Spike Curtis <spike@coder.com>
2022-05-18 16:33:33 +00:00
a3556b12da feat: Single query for all workspaces with optional filter (#1537)
* feat: Add single query for all workspaces using a filter
2022-05-18 10:09:07 -05:00
4ad5ac2d4a feat: Rbac more coderd endpoints, unit test to confirm (#1437)
* feat: Enforce authorize call on all endpoints
- Make 'request()' exported for running custom requests
* Rbac users endpoints
* 401 -> 403
2022-05-17 13:43:19 -05:00
680de709a5 chore: organize http handlers (#1486)
They're currently randomly in a bunch of different files. This cleans up
the handler functions to be in the file of the type they return.
2022-05-16 14:36:27 -05:00
dbd5b4a47b feat: Add workspace owner name to response (#1448)
This will be rendered in the workspace page!
2022-05-13 20:41:21 -05:00
f4da5d4f3a feat: add lifecycle.Executor to manage autostart and autostop (#1183)
This PR adds a package lifecycle and an Executor implementation that attempts to schedule a build of workspaces with autostart configured.

- lifecycle.Executor takes a chan time.Time in its constructor (e.g. time.Tick(time.Minute))
- Whenever a value is received from this channel, it executes one iteration of looping through the workspaces and triggering lifecycle operations.
- When the context passed to the executor is Done, it exits.
- Only workspaces that meet the following criteria will have a lifecycle operation applied to them:
  - Workspace has a valid and non-empty autostart or autostop schedule (either)
  - Workspace's last build was successful
- The following transitions will be applied depending on the current workspace state:
  - If the workspace is currently running, it will be stopped.
  - If the workspace is currently stopped, it will be started.
  - Otherwise, nothing will be done.
- Workspace builds will be created with the same parameters and template version as the last successful build (for example, template version)
2022-05-11 23:03:02 +01:00
97a95f1377 chore: upgrade golangci-lint to v1.46.0 (#1373) 2022-05-10 16:04:23 -05:00
fd49a18b47 feat: Add "state" command to pull and push workspace state (#1264)
It's possible for a workspace to become in an invalid state.
This is something we'll detect for jobs, and allow monitoring of.

These commands will allow admins to manually reconcile state.
2022-05-02 17:51:58 -05:00
88669fd578 feat: Move workspaces under organizations (#1109)
This removes split ownership for workspaces. They are now
a resource of organizations and have a designated owner,
which is a user.

This enables simple administration for commands like:
- `coder stop ben/dev`
- `coder build logs colin/arch`

or if we decide to allow administrators to access workspaces,
they could even SSH using this syntax: `coder ssh colin/dev`.
2022-04-25 16:11:03 -05:00
23295f7f07 fix: Check for job status on another incoming (#1117)
If a job silently failed, it wasn't possible for another one
to execute. This fixes it by using the API status to return
active state.
2022-04-25 02:22:36 +00:00
3311c2f65d refactor: replace Code by Detail in the http API error (#1011) 2022-04-18 11:02:54 -05:00
d9d4599ba9 chore: idea: unify http responses further (#941) 2022-04-12 10:17:33 -05:00
23f989127d coderd: autostart: codersdk, http api, database plumbing (#879)
* feat: add columns autostart_schedule, autostop_schedule to database schema
* feat: database: add UpdateWorkspaceAutostart and UpdateWorkspaceAutostop methods
* feat: add AutostartSchedule/AutostopSchedule to api workspace struct
* feat: codersdk: implement update workspace autostart and autostop methods
* chore: add unit tests for workspace autostarat and autostop methods
2022-04-07 10:03:35 +01:00
02ad3f14f5 chore: Rename Projects to Templates (#880)
Customer feedback indicated projects was a confusing name.
After querying the team internally, it seemed unanimous
that it is indeed a confusing name.

Here's for a lil less confusion @ashmeer7 🥂
2022-04-06 12:42:40 -05:00
2b1a0ee126 chore: update v1 schema (#643) 2022-04-01 14:42:36 -05:00
591523a078 chore: Move httpapi, httpmw, & database into coderd (#568)
* chore: Move httpmw to /coderd directory
httpmw is specific to coderd and should be scoped under coderd

* chore: Move httpapi to /coderd directory
httpapi is specific to coderd and should be scoped under coderd

* chore: Move database  to /coderd directory
database is specific to coderd and should be scoped under coderd

* chore: Update codecov & gitattributes for generated files
* chore: Update Makefile
2022-03-25 16:07:45 -05:00
c451f4e685 feat: Add templates to create working release (#422)
* Add templates

* Move API structs to codersdk

* Back to green tests!

* It all works, but now with tea! 🧋

* It works!

* Add cancellation to provisionerd

* Tests pass!

* Add deletion of workspaces and projects

* Fix agent lock

* Add clog

* Fix linting errors

* Remove unused CLI tests

* Rename daemon to start

* Fix leaking command

* Fix promptui test

* Update agent connection frequency

* Skip login tests on Windows

* Increase tunnel connect timeout

* Fix templater

* Lower test requirements

* Fix embed

* Disable promptui tests for Windows

* Fix write newline

* Fix PTY write newline

* Fix CloseReader

* Fix compilation on Windows

* Fix linting error

* Remove bubbletea

* Cleanup readwriter

* Use embedded templates instead of serving over API

* Move templates to examples

* Improve workspace create flow

* Fix Windows build

* Fix tests

* Fix linting errors

* Fix untar with extracting max size

* Fix newline char
2022-03-22 13:17:50 -06:00
bf0ae8f573 feat: Refactor API routes to use UUIDs instead of friendly names (#401)
* Add client for agent

* Cleanup code

* Fix linting error

* Rename routes to be simpler

* Rename workspace history to workspace build

* Refactor HTTP middlewares to use UUIDs

* Cleanup routes

* Compiles!

* Fix files and organizations

* Fix querying

* Fix agent lock

* Cleanup database abstraction

* Add parameters

* Fix linting errors

* Fix log race

* Lock on close wait

* Fix log cleanup

* Fix e2e tests

* Fix upstream version of opencensus-go

* Update coderdtest.go

* Fix coverpkg

* Fix codecov ignore
2022-03-07 11:40:54 -06:00
1796dc6c2f chore: Add test helpers to improve coverage (#166)
* chore: Rename ProjectHistory to ProjectVersion

Version more accurately represents version storage. This
forks from the WorkspaceHistory name, but I think it's
easier to understand Workspace history.

* Rename files

* Standardize tests a bit more

* Remove Server struct from coderdtest

* Improve test coverage for workspace history

* Fix linting errors

* Fix coderd test leak

* Fix coderd test leak

* Improve workspace history logs

* Standardize test structure for codersdk

* Fix linting errors

* Fix WebSocket compression

* Update coderd/workspaces.go

Co-authored-by: Bryan <bryan@coder.com>

* Add test for listing project parameters

* Cache npm dependencies with setup node

* Remove windows npm cache key

Co-authored-by: Bryan <bryan@coder.com>
2022-02-05 18:24:51 -06:00
e75bde4e31 feat: Add provisionerdaemon to coderd (#141)
* feat: Add history middleware parameters

These will be used for streaming logs, checking status,
and other operations related to workspace and project
history.

* refactor: Move all HTTP routes to top-level struct

Nesting all structs behind their respective structures
is leaky, and promotes naming conflicts between handlers.

Our HTTP routes cannot have conflicts, so neither should
function naming.

* Add provisioner daemon routes

* Add periodic updates

* Skip pubsub if short

* Return jobs with WorkspaceHistory

* Add endpoints for extracting singular history

* The full end-to-end operation works

* fix: Disable compression for websocket dRPC transport (#145)

There is a race condition in the interop between the websocket and `dRPC`: https://github.com/coder/coder/runs/5038545709?check_suite_focus=true#step:7:117 - it seems both the websocket and dRPC feel like they own the `byte[]` being sent between them. This can lead to data races, in which both `dRPC` and the websocket are writing.

This is just tracking some experimentation to fix that race condition

## Run results: ##
- Run 1: peer test failure
- Run 2: peer test failure
- Run 3: `TestWorkspaceHistory/CreateHistory`  - https://github.com/coder/coder/runs/5040858460?check_suite_focus=true#step:8:45
```
status code 412: The provided project history is running. Wait for it to complete importing!`
```
- Run 4: `TestWorkspaceHistory/CreateHistory` - https://github.com/coder/coder/runs/5040957999?check_suite_focus=true#step:7:176
```
    workspacehistory_test.go:122: 
        	Error Trace:	workspacehistory_test.go:122
        	Error:      	Condition never satisfied
        	Test:       	TestWorkspaceHistory/CreateHistory
```
- Run 5: peer failure
- Run 6: Pass  
- Run 7: Peer failure

## Open Questions: ##

### Is `dRPC` or `websocket` at fault for the data race?

It looks like this condition is specifically happening when `dRPC` decides to [`SendError`]). This constructs a new byte payload from [`MarshalError`](f6e369438f/drpcwire/error.go (L15)) - so `dRPC` has created this buffer and owns it.

From `dRPC`'s perspective, the callstack looks like this:
- [`sendPacket`](f6e369438f/drpcstream/stream.go (L253))
  - [`writeFrame`](f6e369438f/drpcwire/writer.go (L65))
    - [`AppendFrame`](f6e369438f/drpcwire/packet.go (L128))
      - with finally the data race happening here:
```go
// AppendFrame appends a marshaled form of the frame to the provided buffer.
func AppendFrame(buf []byte, fr Frame) []byte {
...
	out := buf
	out = append(out, control).   // <---------
```

This should be fine, since `dPRC` create this buffer, and is taking the byte buffer constructed from `MarshalError` and tacking a bunch of headers on it to create a proper frame.

Once `dRPC` is done writing, it _hangs onto the buffer and resets it here__: f6e369438f/drpcwire/writer.go (L73)

However... the websocket implementation, once it gets the buffer, it runs a `statelessDeflate` [here](8dee580a7f/write.go (L180)), which compresses the buffer on the fly. This functionality actually [mutates the buffer in place](a1a9cfc821/flate/stateless.go (L94)), which is where get our race.

In the case where the `byte[]` aren't being manipulated anywhere else, this compress-in-place operation would be safe, and that's probably the case for most over-the-wire usages. In this case, though, where we're plumbing `dRPC` -> websocket, they both are manipulating it (`dRPC` is reusing the buffer for the next `write`, and `websocket` is compressing on the fly).

### Why does cloning on `Read` fail?

Get a bunch of errors like:
```
2022/02/02 19:26:10 [WARN] yamux: frame for missing stream: Vsn:0 Type:0 Flags:0 StreamID:0 Length:0
2022/02/02 19:26:25 [ERR] yamux: Failed to read header: unexpected EOF
2022/02/02 19:26:25 [ERR] yamux: Failed to read header: unexpected EOF
2022/02/02 19:26:25 [WARN] yamux: frame for missing stream: Vsn:0 Type:0 Flags:0 StreamID:0 Length:0
```

# UPDATE:

We decided we could disable websocket compression, which would avoid the race because the in-place `deflate` operaton would no longer be run. Trying that out now:

- Run 1:  
- Run 2: https://github.com/coder/coder/runs/5042645522?check_suite_focus=true#step:8:338
- Run 3:  
- Run 4: https://github.com/coder/coder/runs/5042988758?check_suite_focus=true#step:7:168
- Run 5: 

* fix: Remove race condition with acquiredJobDone channel (#148)

Found another data race while running the tests: https://github.com/coder/coder/runs/5044320845?check_suite_focus=true#step:7:83

__Issue:__ There is a race in the p.acquiredJobDone chan - in particular, there can be a case where we're waiting on the channel to finish (in close) with <-p.acquiredJobDone, but in parallel, an acquireJob could've been started, which would create a new channel for p.acquiredJobDone. There is a similar race in `close(..)`ing the channel, which also came up in test runs.

__Fix:__ Instead of recreating the channel everytime, we can use `sync.WaitGroup` to accomplish the same functionality - a semaphore to make close wait for the current job to wrap up.

* fix: Bump up workspace history timeout (#149)

This is an attempted fix for failures like: https://github.com/coder/coder/runs/5043435263?check_suite_focus=true#step:7:32

Looking at the timing of the test:
```
    t.go:56: 2022-02-02 21:33:21.964 [DEBUG]	(terraform-provisioner)	<provision.go:139>	ran apply
    t.go:56: 2022-02-02 21:33:21.991 [DEBUG]	(provisionerd)	<provisionerd.go:162>	skipping acquire; job is already running
    t.go:56: 2022-02-02 21:33:22.050 [DEBUG]	(provisionerd)	<provisionerd.go:162>	skipping acquire; job is already running
    t.go:56: 2022-02-02 21:33:22.090 [DEBUG]	(provisionerd)	<provisionerd.go:162>	skipping acquire; job is already running
    t.go:56: 2022-02-02 21:33:22.140 [DEBUG]	(provisionerd)	<provisionerd.go:162>	skipping acquire; job is already running
    t.go:56: 2022-02-02 21:33:22.195 [DEBUG]	(provisionerd)	<provisionerd.go:162>	skipping acquire; job is already running
    t.go:56: 2022-02-02 21:33:22.240 [DEBUG]	(provisionerd)	<provisionerd.go:162>	skipping acquire; job is already running
    workspacehistory_test.go:122: 
        	Error Trace:	workspacehistory_test.go:122
        	Error:      	Condition never satisfied
        	Test:       	TestWorkspaceHistory/CreateHistory
```

It  appears that the `terraform apply` job had just finished - with less than a second to spare until our `require.Eventually` completes - but there's still work to be done (ie, collecting the state files). So my suspicion is that terraform might, in some cases, exceed our 5s timeout.

Note that in the setup for this test - there is a similar project history wait that waits for 15s, so I borrowed that here.

In the future - we can look at potentially using a simple echo provider to exercise this in the unit test, in a way that is more reliable in terms of timing. I'll log an issue to track that.

Co-authored-by: Bryan <bryan@coder.com>
2022-02-03 20:34:50 +00:00
177eba87b6 refactor: Move all HTTP routes to top-level struct (#130)
* feat: Add history middleware parameters

These will be used for streaming logs, checking status,
and other operations related to workspace and project
history.

* refactor: Move all HTTP routes to top-level struct

Nesting all structs behind their respective structures
is leaky, and promotes naming conflicts between handlers.

Our HTTP routes cannot have conflicts, so neither should
function naming.
2022-02-01 22:15:26 +00:00
5b01f615eb feat: Add APIs for querying workspaces (#61)
* Add SQL migration

* Add query functions for workspaces

* Add create routes

* Add tests for codersdk

* Add workspace parameter route

* Add workspace query

* Move workspace function

* Add querying for workspace history

* Fix query

* Fix syntax error

* Move workspace routes

* Fix version

* Add CLI tests

* Fix syntax error

* Remove error

* Fix history error

* Add new user test

* Fix test

* Lower target to 70%

* Improve comments

* Add comment
2022-01-25 19:52:58 +00:00