Skip to content

About

The Early Years Covid Recovery project application

Topics

Resources

Code of conduct

Security policy

Stars

5 stars

Watchers

3 watching

Forks

Latest commit

 

History

1,529 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Early years child development training

ci brakeman pa11y

This is an application, written in Ruby on Rails (Version 7), based on the DFE-Digital template. It uses a Contentful workspace for the content, managed by the content editors in the Early years child development training service.

Optionally create .env to override or set default variables.

Dependencies

Ruby version 3.4.x Node version 22.12.x locally (24.x in the production image) PostgreSQL version 15.17 Yarn version 4.0.x

Suggest using asdf for local development. It is recommended to install Ruby, Postgres and Nodejs using asdf (see instructions in 'Working locally' section). You may also be required to install dependencies to use these versions

Getting Started

Run locally

  1. Clone the repository
  2. Install git-secrets
  3. Obtain the master keys
  4. Run bundle install to install the gem dependencies
  5. Run yarn to install node dependencies
  6. Copy the .env.example settings into the .env file
  7. Run bin/dev to launch the app on http://localhost:3000

Run through containerised environment

  1. Clone the repository
  2. Install git-secrets
  3. Obtain the master keys
  4. Run bundle install to install the gem dependencies
  5. Run yarn to install node dependencies
  6. Copy the .env.example settings into the .env file
  7. Run bin/docker-dev or bin/podman-dev to launch the app on http://localhost:3000

Useful Links

Rails Credentials

We use rails credentials to manage secrets; obtain the encryption keys from the dev team.

To edit, use either:

$ EDITOR=vi rails credentials:edit --environment <env>
$ ./bin/docker-rails credentials:edit --environment <env>

Full instructions can be found by running rails credentials:help

Git Secrets

This will help to prevent unintentional commits of access keys.

$ brew install git-secrets
$ cd ./path/to/repo
$ git secrets --install
$ git secrets --register-aws

Find advanced settings and other installation options at the git-secrets project.


Working locally

$ asdf plugin add ruby
$ asdf install ruby
$ asdf plugin add postgres
$ asdf install postgres
$ asdf plugin add nodejs
$ asdf install nodejs

Development

Gemfile group :development

Use bin/dev to start the process workers (watching for changes to asset files).

Testing

Gemfile group :test

Use bin/rspec to run the test suite under /spec. Rails system specs use RackTest only for efficiency.

Production

Running locally in the production rails environment requires generating a self-signed certificate. Use bin/docker-certs

Using Docker

Production image

The app target uses the Docker Official Image ruby:3.4.10-alpine3.23. Compilation and asset-building tools stay in build stages; the runtime includes only application dependencies and the system packages required by the app.

Build and smoke-test the image with:

docker build --pull --target app -t recovery:app .
bash .github/scripts/smoke_image.sh recovery:app

The PR workflow scans the image and review deployments run the sign-in smoke check.

Containerised services are available for use by developers and testers. There are a number of convenience scripts to make working with Docker easier. All containers for the project are named with the prefix recovery_. The project uses chained Docker Compose files to prepare different environments.

There are also several scripts which can be run if Podman is installed. All of these Podman-specific scripts start with bin/podman-.... All Docker-related scripts start with bin/docker-...

These commands help maintain your containerised workspace:

  • bin/docker-build creates tagged images for all the services
  • bin/docker-certs generates a self-signed certificate for running the app in production
  • bin/docker-files changes the ownership of files to your current user, files generated inside containers are created by root
  • bin/docker-down stop any active services
  • bin/docker-prune purge project containers, volumes and images

The commands run common tasks inside containers:

  • bin/docker-adr rebuilds the architecture decision records table of contents
  • bin/docker-dev starts Procfile.dev, containerised equivalent of bin/dev, using the docker-compose.dev.yml override Additionally, it will install bundle and yarn dependencies before starting the web server and asset watchers, including when node_modules already exists.
  • bin/podman-dev starts the same development stack via Podman Compose
  • bin/docker-rails erd generate an Entity Relationship Diagram
  • bin/docker-rails db:seed populates the containerised postgres database
  • bin/docker-rails console drops into a running development environment or starts one, containerised equivalent of bin/rails console
  • bin/docker-rspec -f doc runs the test suite with optional arguments, containerised equivalent of bin/rspec
  • bin/podman-rspec -f doc starts the same test stack via Podman Compose
  • bin/docker-doc runs a YARD documentation server
  • bin/docker-uml exports UML diagrams as default PNGs
  • bin/docker-pa11y runs WCAG checks against a generated sitemap.xml
  • bin/docker-stylelint "name_of_file_goes_here" runs stylelint (css linter)

Using Rake

Custom tasks are namespaced under eyfs, list them using rake --tasks eyfs.

# Generate secure bot user
$ rake eyfs:bot
# Enable the post login 'What's new' page
$ rake eyfs:whats_new

Deployment Pipelines

Visit the Github Container Registry.

Development is deployed automatically with the latest commit from main.

Adding the review label to a pull request in Github will trigger a deployment for review. Once a feature branch is deployed, the URL to access it is added as a comment in the PR conversation in the format: https://eyrecovery-review-pr-##.azurewebsites.net

We intend to use semantic versioning.

Staging is deployed from this workflow. Production is deployed from this workflow.

Smoke Tests

A smoke test script is provided to verify the sign-in journey is working after a deployment. It checks that the health endpoint is reachable and that the sign-in page renders a valid GOV.UK One Login authorisation link.

Run it manually against any deployed environment:

$ .github/scripts/smoke_sign_in.sh <base_url>

For example:

$ .github/scripts/smoke_sign_in.sh https://eyrecovery-dev.azurewebsites.net

The script will:

  1. Poll GET /health until the app responds (up to 30 attempts, 10 seconds apart)
  2. Poll GET /users/sign-in until the page contains a valid GOV.UK One Login authorize link with the expected OIDC query parameters (client_id, redirect_uri, response_type=code)

It exits with a non-zero status code if either check fails, making it suitable for use in CI/CD pipelines.

The smoke tests run automatically as part of the staging and production deployment workflows.

End-to-end login test

The separate One Login end-to-end workflow runs on pull requests and merge groups, and can also be started manually. It uses Puppeteer and a pinned version of the official GOV.UK One Login simulator, not the live identity service.

The test follows the actual authorisation redirect, callback, token exchange, ID-token signature validation and user-info request. It verifies that a new user can access authenticated registration pages and save their terms acceptance and name, then signs in again in a fresh browser session to verify that the same account is reused. No authentication responses are mocked.

To run locally, install the project's Ruby and JavaScript dependencies (including Puppeteer's browser), provide the test Rails credentials key as for the existing test suite, and start Docker. Then run:

$ bash bin/login-e2e

The runner starts Rails in test mode on port 3001, the simulator on port 4001, and a disposable PostgreSQL database on port 5434. These ports must be free. It prepares the database and compiles assets, and stops the server and containers on exit. The database uses temporary storage and is discarded afterwards. Failure screenshots and server output are saved under tmp/login-e2e/.

This configuration uses only synthetic user data and a temporary RSA keypair generated for each run. The private key is stored outside the failure-artifact directory and deleted on exit; no committed One Login key or additional CI secret is needed. It does not change any deployed environment, use production credentials, or add a production authentication bypass. The existing deployment smoke checks remain unchanged.


Monitoring

Sentry is used to monitor production environments

$ brew install getsentry/tools/sentry-cli

$ sentry-cli projects list --org early-years-foundation-reform

+---------+--------------+-------------------------------+--------------+
| ID      | Slug         | Team                          | Name         |
+---------+--------------+-------------------------------+--------------+
| 6274627 | eyf-reform   | early-years-foundation-reform | Rails        |
| 6274651 | eyf-recovery | early-years-foundation-reform | eyf-recovery |
+---------+--------------+-------------------------------+--------------+

Accessibility Standards

An automated accessibility audit can be run against a development server running in Docker using ./bin/docker-pa11y. The test uses pa11y-ci and a dynamic sitemap.xml file to ensure the project meets WCAG2AA standards. A secure HTTP header BOT is used to provide access to pages that require authentication. The secret $BOT_TOKEN environment variable defines the account to seed.

curl -i -L -H "BOT: ${BOT_TOKEN}" http://localhost:3000/my-account

docker-pa11y accepts an optional argument to test external sites.


Emails

Emails are sent using the GOV.UK Notify.

Getting a GovUK Notify account

You need an account before you can use GOV.UK Notify to send emails. To obtain one ask a current member of the team to add you to the "Early Years Foundation Recovery" service, by navigating to the Team members page and clicking the Invite a team member button and entering a government email address. This will send an email inviting you to use the service.

The credentials file for each environment holds an API key for Notify:

  • railsdevelopment-... Team and guest list (limits who you can send to)
  • railstest-... Test (pretends to send messages)
  • railsproduction-... Live (sends to anyone)

It is possible to temporarily override the key by defining GOVUK_NOTIFY_API_KEY in .env.

Accessing information in the Notify service

Once you have an account you can view the Dashboard with details of how many emails have been sent out and any that have failed to send.

You can update the content of the emails in the Templates section.

For more information

Documentation for GovUK Notify can be found here: https://docs.notifications.service.gov.uk/ruby.html

The status of GovUK notify can be checked here: https://status.notifications.service.gov.uk/

For more information the Notify team can be contacted here: https://www.notifications.service.gov.uk/support, or in the UK Government digital slack workspace in the #govuk-notify channel.


GOV.UK One Login

GOV.UK One Login admin tool exists for managing the integration environment client config. It can be accessed at https://admin.sign-in.service.gov.uk (currently only 1 email can access the client config).

Local development with the simulator

For local development CDT runs against the official GOV.UK One Login simulator (docs)

The simulator is started automatically by bin/podman-dev as the gov-one-simulator service in docker-compose.dev.yml.

Sanity-check with:

curl http://localhost:4000/.well-known/openid-configuration

The app reads its GOV.UK One Login configuration with env-var-first precedence (see config/application.rb); the dev defaults are in .env.example:

Variable Dev default
GOV_ONE_BASE_URI http://localhost:4000
GOV_ONE_CLIENT_ID cdt-local-dev
GOV_ONE_PRIVATE_KEY_PATH /srv/config/gov_one/dev_private_key.pem

The committed config/gov_one/dev_private_key.pem is the publicly-documented default keypair from the GOV.UK simulator docs — development only, never used in any deployed environment. The matching public key is pinned on the simulator service in docker-compose.dev.yml so signature validation works regardless of any future change to the simulator image's bundled defaults.

Account Registration

Register an account on the integration OIDC used in development https://signin.integration.account.gov.uk/sign-in-or-create. Using this authentication method also requires basic HTTP auth credentials.

Status Updates

For status updates see https://status.account.gov.uk/

Support

Questions can be directed to the #govuk-one-login slack channel https://ukgovernmentdigital.slack.com/archives/C02AQUJ6WTC

Credentials

  • Integration GOV.UK One Login environment credentials are stored in Rails development credentials (config/credentials/development.yml.enc)
  • Both production and integration GOV.UK One Login environment credentials are stored in Rails production credentials (config/credentials/production.yml.enc).

Service Dashboard

Key performance metrics are surfaced in a Looker Studio dashboard and refreshed daily. User service accounts can authenticate using the Google Cloud SDK.

Storage and Reporting

Downloading exported data

  • gcloud auth login
  • gcloud config set project eyfsdashboard
  • gsutil ls (list buckets)
  • gsutil -m cp -r "gs://eyfs-recovery-production/events" "gs://eyfs-recovery-production/training" . (export folders recursively)

Azure

Production console access

Application Insights

Application Insights via an open telemetry collector is configured to pass through to Azure Application Insights. This is being used to retrieve metrics, and more diagnostic logging, for the service.

Programmatic testing

bravo = Training::Module.by_name('bravo')
data = ContentTestSchema.new(mod: bravo).call(pass: false).compact
file = Rails.root.join('spec/support/ast/bravo-fail.yml')
File.open(file, 'w') { |file| file.write(data.to_yaml) }

Creating data exports (WIP)

Scripts

The data export scripts are located within app/models/data_analysis

The script will then need to be added to DATA_SOURCES within app/services/dashboard.rb e.g adding in { model: 'DataAnalysis::DeviceEngagement', folder: 'training', file: 'device_engagement' }, this will then generate the file tmp/<current date>/training/device_engagement.csv

Seeding

An example of creating seed data can be found here lib/seed_device_events.rb

seeds scripts can be executed by running bin/docker-rails runner <seed script>

e.g bin/docker-rails runner lib/seed_device_events.rb

Running

bin/docker-rails console to get into the console then run DashboardJob.run(upload: false)

files can be found within tmp/

About

The Early Years Covid Recovery project application

Topics

Resources

Code of conduct

Security policy

Stars

5 stars

Watchers

3 watching

Forks

Releases

Packages

Used by

Contributors

Languages