Commit Graph

49 Commits

Author SHA1 Message Date
6639c69fad feat: add "display_order" column to coder_parameter to keep parameters sorted in UI (#8227) 2023-06-30 12:41:55 +02:00
98a5ae7f48 feat: add provisioner job hang detector (#7927) 2023-06-25 13:17:00 +00:00
41e7216df7 chore: upgrade terraform to 1.5.1 (#8160) 2023-06-22 19:08:40 +00:00
52ead3d933 feat(provisioner): add support for .tf.json templates (#7835)
Co-authored-by: Colin Adler <colin1adler@gmail.com>
2023-06-08 08:06:50 +10:00
a7366a8b76 feat!: drop support for legacy parameters (#7663) 2023-06-02 11:16:46 +02:00
c7fb5f960c feat: preserve original order of rich parameters (#6747)
* WIP

* orderedParameters

* fix

* WIP

* TestS
2023-03-24 09:37:27 +01:00
df31636e72 feat: pass access_token to coder_git_auth resource (#6713)
This allows template authors to leverage git auth to perform
custom actions, like clone repositories.
2023-03-22 19:37:08 +00:00
f05609b4da chore: format Go more aggressively 2023-02-18 18:32:09 -06:00
bbb208e29c feat: Add CLI support for workspace build parameters (#5768)
* WIP

* WIP

* CLI: handle workspace build parameters

* fix: golintci

* Fix: dry run

* fix

* CLI: is mutable

* coderd: mutable

* fix: golanci

* fix: richParameterFile

* CLI: create unit tests

* CLI: update test

* Fix

* fix: order

* fix
2023-01-23 15:01:22 +01:00
6651c1632d fix: avoid terraform state concurrent access, remove global mutex (#5273) 2022-12-06 17:05:14 +00:00
1f20cab110 fix: don't use yamux for in-memory provisioner{,d} streams (#5136) 2022-11-22 12:19:32 -06:00
95fb59696e Refactor Provisioner to distinguish Plan and Apply (#5036) 2022-11-11 16:45:58 -06:00
795ed3dc97 provisioner: fix multi-dir installs (#4690)
In the previous implementation, tests would occasionally fail since the original install directory was deleted.
2022-10-22 20:44:05 +00:00
d49d57ec4b provisioner: don't pass CODER_ variables (#4638) 2022-10-19 09:42:29 -04:00
8a94b72c7d feat: Allow hide resources (#3977) 2022-09-09 16:38:00 -03:00
b0fe9bcdd1 chore: Upgrade to Go 1.19 (#3617)
This is required as part of #3505.
2022-08-21 22:32:53 +00:00
3fdae47b87 fix: Shadow err in TestProvision_Cancel to fix test race (#3579)
Fixes #3574
2022-08-19 11:56:28 +03:00
f1423450bd fix: Allow terraform provisions to be gracefully cancelled (#3526)
* fix: Allow terraform provisions to be gracefully cancelled

This change allows terraform commands to be gracefully cancelled on
Unix-like platforms by signaling interrupt on provision cancellation.

One implementation detail to note is that we do not necessarily kill a
running terraform command immediately even if the stream is closed. The
reason for this is to allow for graceful cancellation even in such an
event. Currently the timeout is set to 5 minutes by default.

Related: #2683

The above issue may be partially or fully fixed by this change.

* fix: Remove incorrect minimumTerraformVersion variable

* Allow init to return provision complete response
2022-08-18 17:03:55 +03:00
fb9fca8bc9 fix: Ensure terraform tests have a cache path and logger (#3161)
* fix: Ensure terraform tests have a cache path and logger

* fix: Protect against concurrent `terraform init`
2022-08-04 20:37:07 +03:00
54a585dbf6 Log provisioner outputs from TestProvision_ExtraEnv (#2427)
Signed-off-by: Spike Curtis <spike@coder.com>
2022-06-17 14:47:05 -07:00
552dad6919 Remove tfexec, allow TF_ environment vars and log them (#2264)
* Remove tfexec, allow TF_ environment vars and log them

Signed-off-by: Spike Curtis <spike@coder.com>

* fixup: commented code, long lines

Signed-off-by: Spike Curtis <spike@coder.com>

* rename executor methods to remove get

Signed-off-by: Spike Curtis <spike@coder.com>

* don't log terraform environment variables we don't know are safe

Signed-off-by: Spike Curtis <spike@coder.com>

* Disable linting of fake secret

Signed-off-by: Spike Curtis <spike@coder.com>

* drop parse support and move logger into terraform package

Signed-off-by: Spike Curtis <spike@coder.com>

* disable testpackage linter on internal package test

Signed-off-by: Spike Curtis <spike@coder.com>
2022-06-16 17:50:39 +00:00
45eb1b4980 feat: improve terraform template parsing errors (#2331) 2022-06-16 04:12:17 +10:00
14701498c9 fix: Improve Terraform agent<->resource association testing (#2187) 2022-06-08 17:40:34 -05:00
013f028e55 feat: Add workspace application support (#1773)
* feat: Add app support

This adds apps as a property to a workspace agent.

The resource is added to the Terraform provider here:
https://github.com/coder/terraform-provider-coder/pull/17

Apps will be opened in the dashboard or via the CLI
with `coder open <name>`. If `command` is specified, a
terminal will appear locally and in the web. If `target`
is specified, the browser will open to an exposed instance
of that target.

* Compare fields in apps test

* Update Terraform provider to use relative path

* Add some basic structure for routing

* chore: Remove interface from coderd and lift API surface

Abstracting coderd into an interface added misdirection because
the interface was never intended to be fulfilled outside of a single
implementation.

This lifts the abstraction, and attaches all handlers to a root struct
named `*coderd.API`.

* Add basic proxy logic

* Add proxying based on path

* Add app proxying for wildcards

* Add wsconncache

* fix: Race when writing to a closed pipe

This is such an intermittent race it's difficult to track,
but regardless this is an improvement to the code.

* fix: Race when writing to a closed pipe

This is such an intermittent race it's difficult to track,
but regardless this is an improvement to the code.

* fix: Race when writing to a closed pipe

This is such an intermittent race it's difficult to track,
but regardless this is an improvement to the code.

* fix: Race when writing to a closed pipe

This is such an intermittent race it's difficult to track,
but regardless this is an improvement to the code.

* Add workspace route proxying endpoint

- Makes the workspace conn cache concurrency-safe
- Reduces unnecessary open checks in `peer.Channel`
- Fixes the use of a temporary context when dialing a workspace agent

* Add embed errors

* chore: Refactor site to improve testing

It was difficult to develop this package due to the
embed build tag being mandatory on the tests. The logic
to test doesn't require any embedded files.

* Add test for error handler

* Remove unused access url

* Add RBAC tests

* Fix dial agent syntax

* Fix linting errors

* Fix gen

* Fix icon required

* Adjust migration number

* Fix proxy error status code

* Fix empty db lookup
2022-06-04 15:13:37 -05:00
74329f479f fix: Use Terraform address to index resource + agent association (#1727)
Closes #1705.

There was an issue in the implementation brought by #1577 by not trimming
the array value when resources use counts. This should fix it, and adds
a test to be sure!
2022-05-24 20:00:34 -05:00
c2f74f3cc2 chore: avoid concurrent usage of t.FailNow (#1683)
* chore: golangci: add linter rule to report usage of t.FailNow inside goroutines
* chore: avoid t.FailNow in goroutines to appease the race detector
2022-05-24 08:58:39 +01:00
adb7d20c16 feat: skip terraform destroy if there is no state when deleting (#1594) 2022-05-20 14:07:23 +10:00
43c6bff5ae fix: Use "terraform state pull" instead of "terraform show" (#1262)
Although the terraform-exec docs don't indicate this, the result of
"terraform show" isn't actually the state... it's a trimmed version
of the state that excludes resource identifiers, essentially removing
all state that did exist.

Tests will be written to ensure Terraform state reconciliation can occur.
This will happen in another PR, as dogfood is currently broken because of this.
2022-05-02 20:02:38 +00:00
d139a16446 fix: Allow remote state to be used with Terraform (#1242)
The Terraform Provisioner depended on the statefile content
being at a specific path, which disallowed the use of external
state providers. This fixes it!
2022-05-02 15:41:27 +00:00
8701e0084c feat: Update Terraform provider to support "dir" in "coder_agent" (#1219)
This allows users to specify a starting directory for shell sessions.
2022-05-02 10:27:34 -05:00
b948f2dab5 fix: Use environment variables for agent authentication (#1238)
* fix: Update GIT_COMMITTER_NAME to use username

This was a mistake when adding the committer fields 🤦.

* fix: Use environment variables for agent authentication

Using files led to situations where running "coder server --dev" would
break `gitssh`. This is applicable in a production environment too. Users
should be able to log into another Coder deployment from their workspace.

Users can still set "CODER_URL" if they'd like with agent env vars!
2022-04-30 16:40:30 +00:00
65d77383d0 fix: Allow nested Terraform resources (#1093)
This fixes the dependency tree by adding recursion. It
now finds indirect connections and associates it with
an agent.

An example is attached which surfaced this issue.
2022-04-20 12:28:48 -05:00
19b4323512 feat: Allow workspace resources to attach multiple agents (#942)
This enables a "kubernetes_pod" to attach multiple agents that
could be for multiple services. Each agent is required to have
a unique name, so SSH syntax is:

`coder ssh <workspace>.<agent>`

A resource can have zero agents too, they aren't required.
2022-04-11 16:06:15 -05:00
21fdb80825 fix: Parsing dynamic values for agent results in error (#564)
The logic required a constant value before, which disallowed dynamic
value injection into the agent. This isn't an accurate limitation,
so inverting the logic resolves it.
2022-03-25 17:21:39 +00:00
a06821c103 feat: Update Coder Terraform Provider to v0.2.1 (#563)
This update exposes the workspace name and owner, and changes
authentication methods to be explicit. Implicit authentication
added unnecessary complexity and introduced inconsistency.
2022-03-25 16:34:45 +00:00
c451f4e685 feat: Add templates to create working release (#422)
* Add templates

* Move API structs to codersdk

* Back to green tests!

* It all works, but now with tea! 🧋

* It works!

* Add cancellation to provisionerd

* Tests pass!

* Add deletion of workspaces and projects

* Fix agent lock

* Add clog

* Fix linting errors

* Remove unused CLI tests

* Rename daemon to start

* Fix leaking command

* Fix promptui test

* Update agent connection frequency

* Skip login tests on Windows

* Increase tunnel connect timeout

* Fix templater

* Lower test requirements

* Fix embed

* Disable promptui tests for Windows

* Fix write newline

* Fix PTY write newline

* Fix CloseReader

* Fix compilation on Windows

* Fix linting error

* Remove bubbletea

* Cleanup readwriter

* Use embedded templates instead of serving over API

* Move templates to examples

* Improve workspace create flow

* Fix Windows build

* Fix tests

* Fix linting errors

* Fix untar with extracting max size

* Fix newline char
2022-03-22 13:17:50 -06:00
18c929c8ab feat: Use open-source Terraform Provider (#403)
This removes our internal Terraform Provider, and opens
it to the world!
2022-03-07 17:39:00 -06:00
8acd58a095 fix: Use plan to detect resource agent association (#381)
* ci: Update DataDog GitHub branch to fallback to GITHUB_REF

This was detecting branches, but not our "main" branch before.
Hopefully this fixes it!

* Add basic Terraform Provider

* Rename post files to upload

* Add tests for resources

* Skip instance identity test

* Add tests for ensuring agent get's passed through properly

* Fix linting errors

* Add echo path

* Fix agent authentication

* fix: Convert all jobs to use a common resource and agent type

This enables a consistent API for project import and provisioned resources.

* Add "coder_workspace" data source

* feat: Remove magical parameters from being injected

This is a much cleaner abstraction. Explicitly declaring the user
parameters for each provisioner makes for significantly simpler
testing.

* feat: Add graceful exits to provisionerd

Terraform (or other provisioners) may need to cleanup state, or
cancel actions before exit. This adds the ability to gracefully
exit provisionerd.

* Fix cancel error check

* feat: Add destroy to workspace provision job

This enables the full flow of create/update/delete.

* fix: Use plan to detect resource agent association

Before this used the configuration object which detected all resources
regardless of count.
2022-02-28 19:32:13 +00:00
e5c95552cd feat: Remove magical parameters from being injected (#371)
* ci: Update DataDog GitHub branch to fallback to GITHUB_REF

This was detecting branches, but not our "main" branch before.
Hopefully this fixes it!

* Add basic Terraform Provider

* Rename post files to upload

* Add tests for resources

* Skip instance identity test

* Add tests for ensuring agent get's passed through properly

* Fix linting errors

* Add echo path

* Fix agent authentication

* fix: Convert all jobs to use a common resource and agent type

This enables a consistent API for project import and provisioned resources.

* Add "coder_workspace" data source

* feat: Remove magical parameters from being injected

This is a much cleaner abstraction. Explicitly declaring the user
parameters for each provisioner makes for significantly simpler
testing.
2022-02-28 18:26:01 +00:00
35ae532f7c feat: Add Terraform Provider for agent resources (#368)
* ci: Update DataDog GitHub branch to fallback to GITHUB_REF

This was detecting branches, but not our "main" branch before.
Hopefully this fixes it!

* Add basic Terraform Provider

* Rename post files to upload

* Add tests for resources

* Skip instance identity test

* Add tests for ensuring agent get's passed through properly

* Fix linting errors

* Add echo path

* Fix agent authentication

* Update codersdk/files.go

Co-authored-by: Bryan <bryan@coder.com>

Co-authored-by: Bryan <bryan@coder.com>
2022-02-28 17:16:44 +00:00
795bba2af4 feat: Add dry run for provisioners (#178)
* refactor: Rename ProjectParameter to ProjectVersionParameter

This was confusing with ParameterValue before. It still is a bit,
but this should help distinguish scope.

* Add project version resources table

* Allow project parameters to optionally have user and workspace

* Add dry run for provisioners

* Add resource detection on project import
2022-02-07 19:35:18 -06:00
75468fad48 chore: Add testpackage linter (#156)
Enforces a consistent test package layout.
This makes it difficult to test internal
functionality, which I believe promotes
healthy decomposition, and minimal package
exports.
2022-02-04 12:06:06 -06:00
3ba8242764 feat: Add provisionerd service (#127)
This brings an async service that parses and
provisions to life! It's separated from coderd
intentionally to allow for simpler testing.

Integration with coderd will come in another PR!
2022-02-01 12:15:54 -06:00
ac617e1fa8 feat: Update generated schema for provisionerd (#121) 2022-01-31 23:36:15 -06:00
9db5fb0952 refactor: Improve handshake resiliency of peer (#95)
* fix: Synchronize peer logging with a channel

We were depending on the close mutex to properly
report connection state. This ensures the RTC
connection is properly closed before returning.

* Disable pion logging

* Remove buffer

* Try ICE servers

* Remove flushed

* Add diagram explaining handshake

* Fix candidate accept ordering

* Add debug logging to peerbroker

* Fix send ordering

* Lock adding ICE candidate

* Add test for negotiating out of order

* Reduce connection to a single negotiation channel

* Improve test times by pre-installing Terraform

* Lock remote session description being applied

* Organize conn

* Revert to multi-channel setup

* Properly close ICE gatherer

* Improve comments

* Try removing buffered candidates

* Buffer local and remote messages

* Log dTLS transport state

* Add pion logging
2022-01-30 20:11:18 -06:00
b3c5bb3576 feat: Compute project build parameters (#82)
* feat: Add parameter and jobs database schema

This modifies a prior migration which is typically forbidden,
but because we're pre-production deployment I felt grouping
would be helpful to future contributors.

This adds database functions that are required for the provisioner
daemon and job queue logic.

* feat: Compute project build parameters

Adds a projectparameter package to compute build-time project
values for a provided scope.

This package will be used to return which variables are being
used for a build, and can visually indicate the hierarchy to
a user.

* Fix terraform provisioner

* Improve naming, abstract inject to consume scope

* Run CI on all branches
2022-01-29 17:45:42 -06:00
2654a93132 chore: Fix golangci-lint configuration and patch errors (#34)
* chore: Fix golangci-lint configuration and patch errors

Due to misconfiguration of a linting rules directory, our linter has not been
working properly. This change fixes the configuration issue, and all remaining
linting errors.

* Fix race in peer logging

* Fix race and return

* Lock on bufferred amount low

* Fix mutex lock
2022-01-20 10:00:13 -06:00
6a919aea79 feat: Add authentication and personal user endpoint (#29)
* feat: Add authentication and personal user endpoint

This contribution adds a lot of scaffolding for the database fake
and testability of coderd.

A new endpoint "/user" is added to return the currently authenticated
user to the requester.

* Use TestMain to catch leak instead

* Add userpassword package

* Add WIP

* Add user auth

* Fix test

* Add comments

* Fix login response

* Fix order

* Fix generated code

* Update httpapi/httpapi.go

Co-authored-by: Bryan <bryan@coder.com>

Co-authored-by: Bryan <bryan@coder.com>
2022-01-20 13:46:51 +00:00
7c260f88d1 feat: Create provisioner abstraction (#12)
* feat: Create provisioner abstraction

Creates a provisioner abstraction that takes prior art from the Terraform plugin system. It's safe to assume this code will change a lot when it becomes integrated with provisionerd.

Closes #10.

* Ignore generated files in diff view

* Check for unstaged file changes

* Install protoc-gen-go

* Use proper drpc plugin version

* Fix serve closed pipe

* Install sqlc with curl for speed

* Fix install command

* Format CI action

* Add linguist-generated and closed pipe test

* Cleanup code from comments

* Add dRPC comment

* Add Terraform installer for cross-platform

* Build provisioner tests on Linux only
2022-01-08 11:24:02 -06:00