feat(doc): Updated dev deployments documentation

lotas · lotas · commit 5d5ee8e7abe1 · 2022-03-01T21:00:44.000+01:00
Describing how to run own rabbitmq/postgres for testing purposes
diff --git a/dev-docs/dev-deployment.md b/dev-docs/dev-deployment.md
@@ -13,10 +13,10 @@ You will need to have the following
 * A running kubernetes cluster with at least 2000 mCPU and 4GB RAM available.
   * Helm 3 installed
   * The latest version of kubectl installed, and credentials configured to talk to the cluster
-* A RabbitMQ cluster running the latest available version.
+* A RabbitMQ cluster running the latest available version. (see also [install RabbitMQ](#own-rabbitmq-in-cluster))
   The deployment process requires administrative access (the RabbitMQ management API) and creates multiple users.
   The free levels of CloudAMQP's service do not support this.
-* A Postgres server running Postgres 11.x (see below for Google Cloud SQL, or use another provider).
+* A Postgres server running Postgres 11.x (see below for Google Cloud SQL, or use another provider). (see also [install Postgres](#own-postgres-in-cluster))
   The Postgres server must be initialized with the `en_US.utf8` locale; see [the deployment docs](../ui/docs/manual/deploying/database.mdx).
 * An AWS account and an IAM user in that account
   Set up your `aws` command-line to use the IAM user (`aws configure`).
@@ -222,3 +222,134 @@ If you set up a taskcluster-github app, you probably want to test a variety of i
 2. Put together a body of your pulse message. Make sure you use the schemas. It should be in JSON format.
 3. Look up the routing key and exchange you need (most likely you are testing a handler - so look up the bindings for that handler in the code).
 3. Navigate to the management UI on the RabbitMQ server (the url from `pulseHostname`), login using the above credentials and go to the exchange of interest. You will see *Publish Message* section in the UI. Fill out the *Routing Key* and *Payload* fields (the result of the step 2 goes into the latter). Press *Publish Message* and you're done.
+
+## Own RabbitMQ in cluster
+
+It is possible to run RabbitMQ directly in the same cluster for dev purposes.
+
+Warning: by using this approach, you are responsible for maintenance and backups.
+
+You can use one of the existing helm charts, steps listed below:
+
+1. Add helm repository:
+
+   `helm repo add bitnami https://charts.bitnami.com/bitnami`
+2. To enable TLS you need to create certificates according to [RabbitMQ Documentation](https://www.rabbitmq.com/ssl.html#automated-certificate-generation)
+   <details>
+   <summary>
+   Generate and upload rabbitmq-certificates
+   </summary>
+
+   ```sh
+   git clone https://github.com/michaelklishin/tls-gen tls-gen
+
+   cd tls-gen/basic
+   make && make verify && make info
+
+   cd result
+   cp ca_certificate.pem ca.crt && cp server_certificate.pem tls.crt && cp server_key.pem tls.key
+
+   kubectl -n $NAMESPACE create secret generic rabbitmq-certificates \
+      --from-file=./ca.crt \
+      --from-file=./tls.crt \
+      --from-file=./tls.key
+   ```
+   </details>
+3. Create config with chart values:
+   <details>
+   <summary><code>rabbitmq/values.yaml</code></summary>
+
+   ```yaml
+   auth:
+      username: admin
+      password: adminpassword      # set some strong management password
+      erlangCookie: secretcookie   # set some cookie secret
+
+   tls:
+      enabled: true
+      existingSecret: rabbitmq-certificates  # same name of the secret from the previous step
+   ```
+
+   More details at <https://github.com/bitnami/charts/tree/master/bitnami/rabbitmq>
+   </details>
+4. Install chart:
+
+   `helm install tc-rabbitmq --namespace $NAMESPACE -f ./rabbitmq/values.yaml bitnami/rabbitmq`
+
+   This will create several resources. By default it will create Persisted Volume Claim for `8Gb` of disk that will be persisted by kubernetes cluster.
+5. Create proxy (port forwarding) to be able to access management panel:
+
+   `kubectl port-forward --namespace $NAMESPACE svc/tc-rabbitmq 15672:15672`
+
+   You are now able to access it via <http://localhost:15672> using username and password configured in `rabbitmq/values.yaml`.
+
+6. Ensure that vhost and users are created.
+
+   In your `dev-config.yml` (that is created with `yarn dev:init`) you should have `meta.rabbitAdminManagementOrigin = http://127.0.0.1:15672` to be able to properly access it
+
+   Run `yarn dev:ensure:rabbit` to create missing users and vhost. It will prompt for management password (see `rabbitmq/values.yaml`), and assumes that `dev-config.yml` was already generated.
+
+7. Update `dev-config.yml` to have:
+
+   `pulseHostname = tc-rabbitmq-headless` if running in the same namespace, or `pulseHostname: tc-rabbitmq-headless.$NAMESPACE.svc.cluster.local` otherwise
+
+   In case of connection issues, set non tls mode with `pulseAmqps = false`
+
+8. Run `yarn dev:apply` if needed, to recreate configs and secrets with new pulse values.
+
+## Own Postgres in cluster
+
+It is possible to run Postgres directly in the same cluster for dev purposes.
+
+Warning: by using this approach, you are responsible for maintenance and backups.
+
+1. Add helm repository:
+
+   `helm repo add bitnami https://charts.bitnami.com/bitnami`
+
+2. Create config with chart values:
+   <details>
+   <summary><code>postgres/values.yaml</code></summary>
+
+   ```yaml
+   auth:
+      postgresPassword: rootpassword  # set something strong
+      username: taskcluster
+      password: taskcluster  # set something strong
+      database: taskcluster
+
+   tls:
+      enabled: true
+      autoGenerated: true
+
+   volumePermissions:
+      enabled: true
+
+   image:
+      tag: 11  # TaskCluster currently supports version 11
+   ```
+
+   More details at <https://github.com/bitnami/charts/tree/master/bitnami/postgresql>
+   </details>
+3. Install chart:
+
+   `helm install tc-postgresql --namespace $NAMESPACE -f ./postgresql/values.yaml bitnami/postgresql`
+
+   This will create several resources. By default it will create Persisted Volume Claim for `8Gb` of disk that will be persisted by kubernetes cluster.
+4. Create proxy (port forwarding) to be able to access Postgres locally:
+
+   `kubectl port-forward --namespace $NAMESPACE svc/tc-postgresql 5432:5432`
+
+   You are now able to connect to it via <http://localhost:5432> using username and password configured in `postgres/values.yaml`.
+
+5. Ensure that users are created.
+
+   In your `dev-config.yml` (that is created with `yarn dev:init`) you should have `meta.dbPrivateIp = tc-postgresql.$NAMESPACE.svc.cluster.local` to be able to properly access it
+
+   `yarn dev:ensure:db` will create all necessary users for all services
+
+6. Migrate database
+
+   Once users are set, you can run `yarn dev:db:upgrade` (with running port-forward) to create db schema and stored functions
+
+Now it should be possible to use postgres for dev purposes inside the cluster. Make sure to make backups.
diff --git a/infrastructure/tooling/src/dev/index.js b/infrastructure/tooling/src/dev/index.js
@@ -5,7 +5,7 @@ const _ = require('lodash');
 const { readRepoYAML, writeRepoYAML } = require('../utils');
 const inquirer = require('inquirer');
 const commonPrompts = require('./common');
-const { rabbitPrompts, rabbitResources } = require('./rabbit');
+const { rabbitPrompts, rabbitResources, rabbitAdminPasswordPrompt, rabbitEnsureResources } = require('./rabbit');
 const { azureResources } = require('./azure');
 const { postgresPrompts, postgresResources, postgresEnsureDb } = require('./postgres');
 const { k8sResources } = require('./k8s');
@@ -109,8 +109,18 @@ const ensureDb = async (options) => {
   await postgresEnsureDb({ userConfig });
 };
 
+const ensureRabbit = async (options) => {
+  const userConfig = await readUserConfig();
+  const prompts = [];
+
+  await rabbitAdminPasswordPrompt({ userConfig, prompts });
+  const answer = await inquirer.prompt(prompts);
+
+  await rabbitEnsureResources({ userConfig, answer });
+};
+
 const delete_ = async (options) => {
   await helm('delete');
 };
 
-module.exports = { init, apply, verify, ensureDb, delete_, dbUpgrade, dbDowngrade };
+module.exports = { init, apply, verify, ensureDb, ensureRabbit, delete_, dbUpgrade, dbDowngrade };
diff --git a/infrastructure/tooling/src/dev/rabbit.js b/infrastructure/tooling/src/dev/rabbit.js
@@ -111,7 +111,44 @@ const rabbitResources = async ({ userConfig, answer, configTmpl }) => {
   return userConfig;
 };
 
+const rabbitAdminPasswordPrompt = ({ userConfig, prompts }) => {
+  prompts.push({
+    type: 'password',
+    name: 'rabbitAdminPassword',
+    message: 'RabbitMq admin password.',
+  });
+};
+
+const rabbitEnsureResources = async ({ userConfig, answer }) => {
+  const apiUrl = `${userConfig.meta?.rabbitAdminManagementOrigin}/api`;
+  const agent = request.agent().auth(userConfig.meta?.rabbitAdminUser, answer.rabbitAdminPassword).type('json');
+  const vhost = userConfig.pulseVhost;
+
+  console.log(`(Re-)creating RabbitMQ vhost ${vhost}`);
+  await agent.put(`${apiUrl}/vhosts/${encodeURIComponent(vhost)}`);
+
+  for (const [name, cfg] of Object.entries(userConfig)) {
+    if (!cfg.pulse_username || !cfg.pulse_password) {
+      continue;
+    }
+
+    console.log(`Creating RabbitMQ user ${cfg.pulse_username}`);
+    await agent.put(`${apiUrl}/users/${encodeURIComponent(cfg.pulse_username)}`).send({
+      password: cfg.pulse_password,
+      tags: '',
+    });
+    const regexName = `taskcluster\\-${name.replace(/_/g, '\\-')}`;
+    await agent.put(`${apiUrl}/permissions/${encodeURIComponent(vhost)}/${encodeURIComponent(cfg.pulse_username)}`).send({
+      configure: `^(queue/${regexName}/.*|exchange/${regexName}/.*)`,
+      write: `^(queue/${regexName}/.*|exchange/${regexName}/.*)`,
+      read: `^(queue/${regexName}/.*|exchange/.*)`,
+    });
+  }
+};
+
 module.exports = {
   rabbitPrompts,
   rabbitResources,
+  rabbitAdminPasswordPrompt,
+  rabbitEnsureResources,
 };
diff --git a/infrastructure/tooling/src/main.js b/infrastructure/tooling/src/main.js
@@ -180,6 +180,13 @@ program.command('dev:ensure:db')
     run(ensureDb, options);
   }));
 
+program.command('dev:ensure:rabbit')
+  .description('Verify that rabbitmq users and vhost are properly set')
+  .action(actFn(({ options }) => {
+    const { ensureRabbit } = require('./dev');
+    run(ensureRabbit, options);
+  }));
+
 program.command('test:meta')
   .action(actFn(({ options }) => {
     const { main } = require('./meta');
diff --git a/package.json b/package.json
@@ -192,6 +192,7 @@
     "dev:delete": "node infrastructure/tooling/src/main.js dev:delete",
     "dev:verify": "node infrastructure/tooling/src/main.js dev:verify",
     "dev:ensure:db": "node infrastructure/tooling/src/main.js dev:ensure:db",
+    "dev:ensure:rabbit": "node infrastructure/tooling/src/main.js dev:ensure:rabbit",
     "smoketest": "node infrastructure/tooling/src/main.js smoketest",
     "test:meta": "node infrastructure/tooling/src/main.js test:meta",
     "db:upgrade": "node db/src/main.js upgrade",
