concourse-ci Cookbook
=====================

This cookbook sets up a Concourse CI service by Docker Compose.

## Contents

- [Requirements](#requirements)
    - [platforms](#platforms)
    - [packages](#packages)
    - [cookbooks](#cookbooks)
- [Attributes](#attributes)
- [Usage](#usage)
    - [Recipes](#recipes)
        - [concourse-ci::default](#concourse-cidefault)
        - [concourse-ci::fly](#concourse-cifly)
        - [concourse-ci::docker-compose](#concourse-cidocker-compose)
    - [Role Examples](#role-examples)
    - [SSL server keys and certificates management by ssl_cert cookbook](#ssl-server-keys-and-certificates-management-by-ssl_cert-cookbook)
    - [Encryption key management by Chef Vault](#encryption-key-management-by-chef-vault)
    - [OAuth client ID and secret management by Chef Vault](#oauth-client-id-and-secret-management-by-chef-vault)
- [License and Authors](#license-and-authors)

## Requirements

### platforms
- none.

### packages
- none.

### cookbooks
- `docker-grid`
- `ssl_cert`

## Attributes

|Key|Type|Description, example|Default|
|:--|:--|:--|:--|
|`['concourse-ci']['fly']['version']`|String||`'3.3.4'`|
|`['concourse-ci']['fly']['release_url']`|String||`"https://github.com/concourse/concourse/releases/download/v#{node['concourse-ci']['fly']['version']}/fly_linux_amd64"`|
|`['concourse-ci']['fly']['release_checksum']`|String||`nil`|
|`['concourse-ci']['fly']['auto_upgrade']`|Boolean||`false`|
|`['concourse-ci']['fly']['install_path']`|String||`'/usr/local/bin/fly'`|
|`['concourse-ci']['with_ssl_cert_cookbook']`|Boolean|See `attributes/default.rb`|`false`|
|`['concourse-ci']['ssl_cert']['ca_names']`|Array|Internal CA names that are imported by the ssl_cert cookbook.|`[]`|
|`['concourse-ci']['ssl_cert']['common_name']`|String|Server common name for TLS|`node['fqdn']`|
|`['concourse-ci']['docker-image']['entrypoint']`|String|Concourse Docker image's entrypoint setting to import an internal CA certificate.|`'/usr/local/bin/dumb-init /usr/local/bin/concourse'`|
|`['concourse-ci']['docker-compose']['import_ca']`|Boolean|whether import internal CA certificates or not.|`false`|
|`['concourse-ci']['docker-compose']['app_dir']`|String||`"#{node['docker-grid']['compose']['app_dir']}/concourse"`|
|`['concourse-ci']['docker-compose']['pgdata_dir']`|String|Path string or nil (unset).|`"#{node['concourse-ci']['docker-compose']['app_dir']}/database"`|
|`['concourse-ci']['docker-compose']['web_keys_dir']`|String|Path string.|`"#{node['concourse-ci']['docker-compose']['app_dir']}/keys/web"`|
|`['concourse-ci']['docker-compose']['worker_keys_dir']`|String|Path string.|`"#{node['concourse-ci']['docker-compose']['app_dir']}/keys/worker"`|
|`['concourse-ci']['docker-compose']['pgdata_dir']`|String|Path string or nil (unset, non-persistent).|`"#{node['concourse-ci']['docker-compose']['app_dir']}/database"`|
|`['concourse-ci']['docker-compose']['db_password_reset']`|String|Only available if the password is automatically generated by Chef.|`false`|
|`['concourse-ci']['docker-compose']['db_password_vault_item']`|Hash|See `attributes/default.rb`|`{}`|
|`['concourse-ci']['docker-compose']['web_encryption_key_vault_item']`|Hash|See `attributes/default.rb`|`{}`|
|`['concourse-ci']['docker-compose']['web_password_reset']`|String|Only available if the password is automatically generated by Chef.|`false`|
|`['concourse-ci']['docker-compose']['web_password_vault_item']`|Hash|See `attributes/default.rb`|`{}`|
|`['concourse-ci']['docker-compose']['web_oauth_client_id_vault_item']`|Hash|See `attributes/default.rb`|`{}`|
|`['concourse-ci']['docker-compose']['web_oauth_client_secret_vault_item']`|Hash|See `attributes/default.rb`|`{}`|
|`['concourse-ci']['docker-compose']['ssh_keys_reset']`|String|Resets all SSH keys forcely.|`false`|
|`['concourse-ci']['docker-compose']['config_format_version']`|String|Read only. `docker-compose.yml` format version. Only version 1 is supported now.|`'1'`|
|`['concourse-ci']['docker-compose']['config']`|Hash|`docker-compose.yml` configurations.|See `attributes/default.rb`|

## Usage

### Recipes

#### concourse-ci::default

This recipe does nothing.

#### concourse-ci::fly

This recipe installs the `fly` CLI and the `fly_prune_workers` command.

#### concourse-ci::docker-compose

This recipe generates SSH keys of each node and a `docker-compose.yml` file for the Concourse CI service.

### Role Examples

- `roles/concourse.rb`

```ruby
name 'concourse'
description 'Concourse'

run_list(
  'role[docker]',
  'recipe[concourse-ci::docker-compose]',
)

image = 'concourse/concourse:latest'
port = '18080'

override_attributes(
  'concourse-ci' => {
    'docker-compose' => {
      'config' => {
        # Version 1 docker-compose format
        'concourse-web' => {
          'image' => image,
          'ports' => [
            "#{port}:8080",
          ],
          #'volumes' => [
          #  # These volumes will be set by the concourse-ci::docker-compose recipe automatically.
          #  #"#{node['concourse-ci']['docker-compose']['web_keys_dir']}:/concourse-keys",
          #],
          'environment' => {
            'CONCOURSE_EXTERNAL_URL' => "http://192.168.1.3:#{port}",
            #'CONCOURSE_RESOURCE_CHECKING_INTERVAL' => '1m',  # default
          },
        },
        'concourse-worker' => {
          'image' => image,
          #'volumes' => [
          #  # These volumes will be set by the concourse-ci::docker-compose recipe automatically.
          #  #"#{node['concourse-ci']['docker-compose']['worker_keys_dir']}:/concourse-keys",
          #],
        },
      },
    },
  },
)
```

- `roles/concourse-with-ssl.rb`

```ruby
name 'concourse-with-ssl'
description 'Concourse with SSL'

run_list(
  #'recipe[ssl_cert::server_key_pairs]',  # concourse-ci <= 0.2.1
  'role[docker]',
  'recipe[concourse-ci::docker-compose]',
)

image = 'concourse/concourse:2.6.0'
port = '18443'
cn = 'concourse.io.example.com'

override_attributes(
  'ssl_cert' => {
    'common_names' => [
      cn,
    ],
  },
  'concourse-ci' => {
    'with_ssl_cert_cookbook' => true,
    'ssl_cert' => {
      'common_name' => cn,
    },
    'docker-compose' => {
      'config' => {
        # Version 1 docker-compose format
        'concourse-web' => {
          'image' => image,
          'ports' => [
            "#{port}:8443",
          ],
          'environment' => {
            'CONCOURSE_EXTERNAL_URL' => "https://192.168.1.3:#{port}",
            'CONCOURSE_TLS_BIND_PORT' => '8443',  # activate HTTPS
            # These environments will be set by the concourse-ci::docker-compose recipe automatically.
            #'CONCOURSE_TLS_CERT' => '/root/server.crt',
            #'CONCOURSE_TLS_KEY' => '/root/server.key',
          },
          #'volumes' => [
          #  # These volumes will be set by the concourse-ci::docker-compose recipe automatically.
          #  #"#{node['concourse-ci']['docker-compose']['web_keys_dir']}:/concourse-keys",
          #  #"#{server_cert_path(node['concourse-ci']['ssl_cert']['common_name'])}:/root/server.crt:ro",
          #  #"#{server_key_path(node['concourse-ci']['ssl_cert']['common_name'])}:/root/server.key:ro",
          #],
        },
        'concourse-worker' => {
          'image' => image,
          #'volumes' => [
          #  # These volumes will be set by the concourse-ci::docker-compose recipe automatically.
          #  #"#{node['concourse-ci']['docker-compose']['worker_keys_dir']}:/concourse-keys",
          #],
        },
      },
    },
  },
)
```

- `roles/concourse-with-oauth.rb`

```ruby
name 'concourse-with-oauth'
description 'Concourse with OAuth'

run_list(
  #'recipe[ssl_cert::ca_certs]',  # concourse-ci <= 0.2.1
  'recipe[ssl_cert::server_key_pairs]',
  'role[docker]',
  'recipe[concourse-ci::docker-compose]',
)

image = 'concourse/concourse:latest'
port = '18443'
ca_name = 'grid_ca'
cn = 'concourse.io.example.com'
gitlab_cn = 'gitlab.io.example.com'

override_attributes(
  'ssl_cert' => {
    'ca_names' => [
      ca_name,
    ],
    'common_names' => [
      cn,
    ],
  },
  'concourse-ci' => {
    'with_ssl_cert_cookbook' => true,
    'ssl_cert' => {
      'ca_names' => [
        ca_name,
      ],
      'common_name' => cn,
    },
    'docker-compose' => {
      'import_ca' => true,
      'web_oauth_client_id_vault_item' => {
        'vault' => 'concourse',
        'name' => 'web_oauth_client_id',
        'env_context' => false,
        'key' => 'cid',
      },
      'web_oauth_client_secret_vault_item' => {
        'vault' => 'concourse',
        'name' => 'web_oauth_client_secret',
        'env_context' => false,
        'key' => 'secret',
      },
      'config' => {
        # Version 1 docker-compose format
        'concourse-web' => {
          'ports' => [
            #'4080:8080',
            "#{port}:8443",
          ],
          'environment' => {
            'CONCOURSE_TLS_BIND_PORT' => '8443',
            'CONCOURSE_EXTERNAL_URL' => "https://#{cn}:#{port}",
            # OAuth for the default `main`` team
            'CONCOURSE_GENERIC_OAUTH_DISPLAY_NAME' => 'GitLab',
            # The following 2 variables are set automatically,
            # if the ['concourse-ci']['docker-compose']['web_oauth_client_(id|secret)_vault_item'] attributes are specified.
            #'CONCOURSE_GENERIC_OAUTH_CLIENT_ID' => '${CONCOURSE_GENERIC_OAUTH_CLIENT_ID}',
            #'CONCOURSE_GENERIC_OAUTH_CLIENT_SECRET' => '${CONCOURSE_GENERIC_OAUTH_CLIENT_SECRET}',
            'CONCOURSE_GENERIC_OAUTH_AUTH_URL' => "https://#{gitlab_cn}/oauth/authorize",
            'CONCOURSE_GENERIC_OAUTH_TOKEN_URL' => "https://#{gitlab_cn}/oauth/token",
          },
        },
      },
    },
  },
)
```

### SSL server keys and certificates management by ssl_cert cookbook

- create vault items.

```text
$ ruby -rjson -e 'puts JSON.generate({"private" => File.read("concourse.io.example.com.prod.key")})' \
> > ~/tmp/concourse.io.example.com.prod.key.json

$ ruby -rjson -e 'puts JSON.generate({"public" => File.read("concourse.io.example.com.prod.crt")})' \
> > ~/tmp/concourse.io.example.com.prod.crt.json

$ cd $CHEF_REPO_PATH

$ knife vault create ssl_server_keys concourse.io.example.com.prod \
> --json ~/tmp/concourse.io.example.com.prod.key.json

$ knife vault create ssl_server_certs concourse.io.example.com.prod \
> --json ~/tmp/concourse.io.example.com.prod.crt.json
```

- grant reference permission to the Concourse host

```text
$ knife vault update ssl_server_keys  concourse.io.example.com.prod -S 'name:concourse-host.example.com'
$ knife vault update ssl_server_certs concourse.io.example.com.prod -S 'name:concourse-host.example.com'
```

- modify run_list and attributes

```ruby
run_list(
  #'recipe[ssl_cert::server_key_pairs]',  # concourse-ci <= 0.2.1
  'recipe[concourse-ci::docker-compose]',
)

override_attributes(
  'ssl_cert' => {
    'common_names' => [
      'concourse.io.example.com',
    ],
  },
  'concourse-ci' => {
    'with_ssl_cert_cookbook' => true,
    'ssl_cert' => {
      'common_name' => 'concourse.io.example.com',
    },
    # ...
  },
)
```

### Encryption key management by Chef Vault

- create vault items.

```text
# a 16 or 32-byte random character sequence.
$ cat ~/tmp/concourse_ekey.json
{"ekey":"********************************"}

$ knife vault create concourse web_encryption_key --json ~/tmp/concourse_ekey.json
```

- grant reference permission to the Concourse host

```text
$ knife vault update concourse web_encryption_key -S 'name:concourse-host.example.com'
```

- modify attributes

```ruby
override_attributes(
  'concourse-ci' => {
    # ...
    'docker-compose' => {
      'web_encryption_key_vault_item' => {
        'vault' => 'concourse',
        'name' => 'web_encryption_key',
        'env_context' => false,
        'key' => 'ekey',
      },
      # ...
    },
  },
)
```

### OAuth client ID and secret management by Chef Vault

- create vault items.

```text
$ cat ~/tmp/concourse_oauth_client_id.json
{"cid":"***************************************************************"}
$ cat ~/tmp/concourse_oauth_client_secret.json
{"secret":"***************************************************************"}

$ knife vault create concourse web_oauth_client_id --json ~/tmp/concourse_oauth_client_id.json
$ knife vault create concourse web_oauth_client_secret --json ~/tmp/concourse_oauth_client_secret.json
```

- grant reference permission to the Concourse host

```text
$ knife vault update concourse web_oauth_client_id -S 'name:concourse-host.example.com'
$ knife vault update concourse web_oauth_client_secret -S 'name:concourse-host.example.com'
```

- modify attributes

```ruby
override_attributes(
  'concourse-ci' => {
    # ...
    'docker-compose' => {
      'web_oauth_client_id_vault_item' => {
        'vault' => 'concourse',
        'name' => 'web_oauth_client_id',
        'env_context' => false,
        'key' => 'cid',
      },
      'web_oauth_client_secret_vault_item' => {
        'vault' => 'concourse',
        'name' => 'web_oauth_client_secret',
        'env_context' => false,
        'key' => 'secret',
      },
      # ...
    },
  },
)
```

## License and Authors

- Author:: whitestar at osdn.jp

```text
Copyright 2017, whitestar

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
```
