Skip to main content

Teaching vacancies - # Application secrets

Application secrets

The application requires sensitive configuration which may be specific to each environment. It is stored securely in AWS SSM parameter store.

To read the whole application configuration of an environment (e.g. qa), run:

% aws-vault exec SecretEditor -- make qa print-env

To edit the application secrets of an environment (e.g. qa), run:

% aws-vault exec SecretEditor -- make qa edit-app-secrets

You will have the opportunity to verify and confirm before pushing the change. All the secrets are versioned, so in case of issue, a previous version can be restored.

cloudwatch_slack_hook_url

Terraform variable containing encrypted URL to the Slack webhook. cloudwatch_ops_genie_api_key is not actually used but is commonly set to the same value as cloudwatch_slack_hook_url. They need to be in an encrypted and base64 encoded format. There currently isn't a way to do this with Terraform, so to work around this:

  1. Set cloudwatch_slack_hook_url and cloudwatch_ops_genie_api_key as their clear text values and apply, making sure to strip the protocol off cloudwatch_slack_hook_url.
  2. Go to the Lambda function settings within the AWS Console.
  3. Under 'Environment Variables', expand 'Encryption configuration' and select 'Enable helpers for encryption in transit'.
  4. Select the 'tvs2-<env>-cloudwatch-lambda' key.
  5. Select 'Encrypt' on both the opsGenieApiKey and slackHookUrl (or if editing, the one that's changed).
  6. Click 'Save'.
  7. Reload the page, and copy both of the (now encrypted) values, and replace cloudwatch_slack_hook_url and cloudwatch_ops_genie_api_key in your terraform variables file.
  8. Run a terraform plan to ensure everything has been done correctly (You should not have any changes required for the lambda resource).

SECRET_KEY_BASE

The secret_key_base is used as the input secret to the application's key generator, which in turn is used to create all MessageVerifiers/MessageEncryptors, including the ones that sign and encrypt cookies. If you change this key, all old signed cookies will become invalid! In order to generate a new secret key:

  1. Run the rails secret task from the repo, it will generate a new secret key
  2. You need to generate a different key per environment
  3. Update SECRET_KEY_BASE in AWS Systems Manager Parameter Store /teaching-vacancies/<env>/app/secrets files

DFE_SIGN_IN_PASSWORD and DFE_SIGN_IN_SECRET

  • DFE_SIGN_IN_PASSWORD is used to encrypt JWT tokens to authorise a user with DFE sign-in.
  • DFE_SIGN_IN_SECRET is OAuth2 client secret. It is only know to the application and the authorising server.

In order to update the password you need to have access to the DSI Manage console, it is specific to the environment (test, preprod, prod). This is the URL for test: https://test-manage.signin.education.gov.uk

Once you are inside:

  1. Follow the Service configuration link
  2. Regenerate the Client secret (DFE_SIGN_IN_SECRET env variable) and API secret (DFE_SIGN_IN_PASSWORD env variable)
  3. Update DFE_SIGN_IN_SECRET and DFE_SIGN_IN_PASSWORD in AWS Systems Manager Parameter Store /teaching-vacancies/<env>/app/secrets files

Google API Keys

There are several different API keys in use in different environments. There are keys for Google Maps, as well as service accounts for Google Analytics, BigQuery and Google Drive.

  • GOOGLE_LOCATION_SEARCH_API_KEY is used for both the Google Places API and the Google Geocoding API
  • GOOGLE_MAPS_API_KEY is used for Google Maps
  • GOOGLE_API_JSON_KEY is used for analytics, indexing and drive
  • BIG_QUERY_API_JSON_KEY is used for writing tables into BigQuery

NOTE: Keys with JSON in the name are JSON objects, not simple strings. They will need to be normalized in to JSON strings to be used in ENV variables.

  1. Go to the Google Cloud Console API credentials section
  2. Always use your own login if it has sufficient permissions.

For a string-based API key follow this workflow:

  1. Click 'CREATE CREDENTIALS' in the toolbar at the top of the page
  2. Choose 'API key'
  3. Click 'RESTRICT KEY' in the modal that appears
  4. In 'Name' give the key a clear, descriptive name including environment details where applicable
  5. Under 'Application Restrictions' select 'HTTP referrers (web sites)'
  6. Click 'ADD AN ITEM' under 'Website restrictions' and put in desired URL pattern.
  7. Under 'API restrictions' choose 'Restrict key'
  8. Select the API and access level for the key
  9. Create one key per API and environment and use the minimum necessary permission(s) for that key
  10. Click 'Save'
  11. Copy your new key from the table and update it in AWS Systems Manager Parameter Store /teaching-vacancies/<env>/app/secrets files
  12. Do a rolling restart on the updated environment for the application
  13. Check that everything works as expected
  14. Delete the old API key from the Credentials table in the Google Cloud console
  15. Notify anyone who needs to know that the key has been changed

For a JSON API keys follow this workflow:

To change a key on an existing service account (most common scenario):
  1. You are now back on the dashboard. Find your new service account and click on it.
  2. Click 'ADD KEY'
  3. Pick 'Create new key'
  4. Choose 'JSON' and click CREATE
  5. The new key will be automatically downloaded. Once you have it, you can click 'CLOSE' on the modal
  6. If you are on a mac, the easiest way to get the key into an ENV friendly format is using the jq tool. In a terminal: 
    brew install jq
cd <key download directory>
jq -c . teacher-vacancy-service-<some UID>.json | pbcopy
  7. Copy the full body of the new key as a json string (not necessary if you used ...| pbcopy in the jq example, above)
  8. Paste the full string of the new key in AWS Systems Manager Parameter Store /teaching-vacancies/<env>/app/BIG_QUERY_API_JSON_KEY or /teaching-vacancies/<env>/app/GOOGLE_API_JSON_KEY files
  9. Do a rolling restart on the updated environment for the application
  10. Check that everything works as expected
  11. Delete the old key from the 'Keys' section in the Service Account window
  12. Notify anyone who needs to know that the key has been changed
To create a new service account (usually not necessary):
  1. Click 'CREATE CREDENTIALS' in the toolbar at the top of the page
  2. Choose 'Service Account'
  3. In 'Service account name' give the account a clear, descriptive name including environment details where applicable
  4. Add a concise description of what the service account is for and what environments it is to be used in 'Service account description'.
  5. Click 'Create'
  6. Add at least one role to restrict the service account to a service. Use the minimum necessary permission(s) for the role.
  7. Click 'Continue'
  8. So far, our service account have not required user or group access, so you can skip the next step (click 'DONE'). This may change in the future.

ORDNANCE_SURVEY_API_KEY

Used for geocoding. The key cannot be revoked through the portal. Contact Ordnance survey support.

To create a new key:

  1. Connect to https://developer.ordnancesurvey.co.uk/
  2. Navigate to My Keys > Add a new key
  3. Enter key name, select OS Names API and click Save Key
  4. Update ORDNANCE_SURVEY_API_KEY in AWS Systems Manager Parameter Store /teaching-vacancies/<env>/app/secrets files

SENTRY_DSN and SENTRY_FRONTEND_DSN

Two separate DSNs for Sentry:

  • SENTRY_DSN is used by the sentry-ruby gem
  • SENTRY_FRONTEND_DSN is used to track Javascript errors

They don't necessarily need to be separate, but as SENTRY_FRONTEND_DSN must by definition be public, this gives us some kind of separation. These are identical for all environments for the sake of simplicity, and Rails.application.configuration.app_role is used to set Sentry's concept of environment.

Locally, you shouldn't need those API keys as you wouldn't want to do error tracking, but you can set them for testing purposes in .env.local if needed.

To rotate keys:

  1. Go to https://sentry.io/settings/teaching-vacancies/projects/teaching-vacancies/keys/
  2. Revoke/recreate the key you want to change
  3. Update the values in Parameter Store in all of the environments' secrets files

SKYLIGHT_AUTHENTICATION

Used by the app to report performance data to Skylight. Managed by digital-tools.

NOTIFY_KEY

Used to integrate with the GovUK Notify API.

  1. Access https://www.notifications.service.gov.uk/
  2. Navigate to API integration > API keys
  3. Click Revoke on the old key
  4. Click Create an API key
  5. Add a specific name, select the type of key and click Continue
  6. Update NOTIFY_KEY in AWS Systems Manager Parameter Store /teaching-vacancies/<env>/app/secrets files