Skip to content

Commit 66c44af

Browse files
lotaslotas
committed
feat(dev): Introduce compose profiles
Compose profiles allow to run background jobs that are not started by default. Refactored docker-compose.yml to load environment variables for services from dedicated env files.
1 parent bcc6e69 commit 66c44af

File tree

19 files changed

+1740
-465
lines changed

19 files changed

+1740
-465
lines changed

‎changelog/issue-5602.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
audience: developers
2+
level: patch
3+
reference: issue 5602
4+
---
5+
6+
Introduced docker compose profiles to allow running background tasks and cron jobs.

‎dev-docs/development-process.md‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -275,6 +275,47 @@ yarn dev:start queue-web auth-web
275275
yarn stop
276276
```
277277

278+
### Using compose profiles
279+
280+
By default `yarn start` (`docker compose up`) will only start web services and workers. Background tasks and cron jobs do not start with the rest of the services.
281+
282+
If you need to run those services you can use [profiles](https://docs.docker.com/compose/profiles/) functionality of `docker compose`.
283+
284+
Each background and cron job have special profiles defined for them (see `docker-compose.yml`):
285+
286+
* service name (`queue`, `github`, ...)
287+
* job type (`background`, `cron`)
288+
* service + job type (`queue-cron`, `worker-manager-background`, ...)
289+
290+
That can help to run only what is needed:
291+
292+
```sh
293+
# start all services + all queue background and cron jobs
294+
export COMPOSE_PROFILES=queue
295+
yarn start
296+
# stop all
297+
yarn stop
298+
299+
# start all services + all cron jobs in development mode
300+
export COMPOSE_PROFILES=cron
301+
yarn dev:start
302+
# stop all
303+
yarn dev:stop
304+
305+
# start all services + background tasks of worker-manager
306+
export COMPOSE_PROFILES=worker-manager-background
307+
yarn start
308+
# stop all
309+
yarn stop
310+
311+
# start only one specific background job by name
312+
yarn start notify-background-handler
313+
```
314+
315+
> Note: although `docker compose` command allows passing profile names through arguments like `docker compose --profile cron up -d`, those services will not be stopped if you run `docker compose down` afterwards. To stop them you would need to use same arguments `docker compose --profile cron down`.
316+
>
317+
> This way using exported environment variable makes compose behaviour more intuitive.
318+
278319
## Running tasks with local generic-worker
279320

280321
`generic-worker` is configured to automatically start with `docker compose` and connect to taskcluster using `docker-compose/generic-worker` task queue id.

0 commit comments

Comments
 (0)