With just a few workspaces, the autobuild executor can slow down API
requests every time it runs. This is because we started a long running
transaction and checked all eligible (for autostart) workspaces inside
that transaction. PostgreSQL doesn't know if we're modifying rows and as
such is locking the tables for read operations.
This commit changes the behavior so each workspace is checked in its own
transaction reducing the time the table/rows needs to stay locked.
For now concurrency has been arbitrarily limited to 10 workspaces at a
time, this could be made configurable or adjusted as the need arises.
* fix: Improve `coder server` shutdown procedure
This commit improves the `coder server` shutdown procedure so that all
triggers for shutdown do so in a graceful way without skipping any
steps.
We also improve cancellation and shutdown of services by ensuring
resources are cleaned up at the end.
Notable changes:
- We wrap `cmd.Context()` to allow us to control cancellation better
- We attempt graceful shutdown of the http server (`server.Shutdown`)
because it's less abrupt (compared to `shutdownConns`)
- All exit paths share the same shutdown procedure (except for early
exit)
- `provisionerd`s are now shutdown concurrently instead of one at a
time, the also now get a new context for shutdown because
`cmd.Context()` may be cancelled
- Resources created by `newProvisionerDaemon` are cleaned up
- Lifecycle `Executor` exits its goroutine on context cancellation
Fixes#3245
* add reason field for workspace build
* add the reason field to FE via API
* update BuildReasonMember to BuildReasonInitiator
* add unit tests
* add more unit tests
* add error for unknown transition
* fix lint
* add documentation
* fix unit tests
* fix generated types
* remove nested transaction
* rename migration file
This commit makes the following changes:
- Partially reverts the changes of feat: update workspace deadline when workspace ttl updated #2165, making the deadline of a running workspace build independant of TTL, once started.
- CLI: updating a workspace TTL no longer updates the deadline of the workspace.
- UI: updating a workspace TTL no longer updates the deadline of the workspace.
- Drive-by: API: When creating a workspace, default TTL to min(12 hours, template max_ttl) if not instructed otherwise.
- Drive-by: CLI: list: measure workspace extension correctly (+X in last column) from the time the provisioner job was completed
- Drive-by: WorkspaceSchedule: show timezone of schedule if it is set, defaulting to dayjs guess otherwise.
- Drive-by: WorkspaceScheduleForm: fixed an issue where deleting the "TTL" value in the form would show the text "Your workspace will shut down a few seconds after start".
This commit adds the following changes to workspace scheduling behaviour:
* CLI: updating a workspace TTL updates the deadline of the workspace.
* If the TTL is being un-set, the workspace deadline is set to zero.
* If the TTL is being set, the workspace deadline is updated to be the last updated time of the workspace build plus the requested TTL. Additionally, the user is prompted to confirm interactively (can be bypassed with -y).
* UI: updating the workspace schedule behaves similarly to the CLI, showing a message to the user if the updated TTL/time to shutdown would effect changes to the lifetime of the running workspace.
This PR adds fields to templates that constrain values for workspaces derived from that template.
- Autostop: Adds a field max_ttl on the template which limits the maximum value of ttl on all workspaces derived from that template. Defaulting to 168 hours, enforced on edits to workspace metadata. New workspaces will default to the templates's `max_ttl` if not specified.
- Autostart: Adds a field min_autostart_duration which limits the minimum duration between successive autostarts of a template, measured from a single reference time. Defaulting to 1 hour, enforced on edits to workspace metadata.
* Changes all public-facing codersdk types to use a plain int64 (milliseconds) instead of time.Duration.
* Makes autostart_schedule a *string as it may not be present.
* Adds a utils/ptr package with some useful methods.
Removes 5-second wait in autobuild.executor unit tests:
- Adds a write-only channel to Executor and plumbs through to unit tests
- Modifies runOnce to return an executor.RunStats struct and write to statsCh if not nil
* Adds a `bump` command to extend workspace build deadline
* Reduces WARN-level logging spam from autobuild executor
* Modifies `cli/ssh` notifications to read from workspace build deadline and to notify relative time instead (sidestepping the problem of figuring out a user's timezone across multiple OSes)
* Shows workspace extension time in `coder list` output e.g.
```
WORKSPACE TEMPLATE STATUS LAST BUILT OUTDATED AUTOSTART TTL
developer/test1 docker Running 4m false 0 9 * * MON-FRI 15m (+5m)
```
* Adds deadline column to workspace_builds, associated DB/API plumbing
* database: Upon inserting a row into workspace_builds, deadline will
initially be zero.
* autobuild: Executor now checks the Deadline field of the workspace_build
for the purpose of autostop logic.
* coderd: Adds a new route /api/v2/workspaces/:workspace/extend which allows
updating the deadline of the currently active workspace build. The new
deadline must be after the existing deadline, and not the zero time.
* provisionerd: updates workspace_build.deadline upon successful workspace
build completion (equal to now plus workspace TTL, if it exists).
* database: add autostart_schedule and ttl to InsertWorkspace; make gen
* coderd: workspaces: consume additional fields of CreateWorkspaceRequest
* cli: update: add support for TTL and autostart_schedule
* cli: create: add unit tests
* coder: import `time/tzdata` for embedded timezone database
* autobuild: fix unit test that only runs with a real db
This PR adds a package lifecycle and an Executor implementation that attempts to schedule a build of workspaces with autostart configured.
- lifecycle.Executor takes a chan time.Time in its constructor (e.g. time.Tick(time.Minute))
- Whenever a value is received from this channel, it executes one iteration of looping through the workspaces and triggering lifecycle operations.
- When the context passed to the executor is Done, it exits.
- Only workspaces that meet the following criteria will have a lifecycle operation applied to them:
- Workspace has a valid and non-empty autostart or autostop schedule (either)
- Workspace's last build was successful
- The following transitions will be applied depending on the current workspace state:
- If the workspace is currently running, it will be stopped.
- If the workspace is currently stopped, it will be started.
- Otherwise, nothing will be done.
- Workspace builds will be created with the same parameters and template version as the last successful build (for example, template version)
