Example NixOps State Backends by grahamc · Pull Request #1264 · NixOS/nixops

grahamc · 2020-03-24T23:01:03Z

Store in S3:

{
  network = {
    storage.s3 = {
      profile = "rootlol";
      region = "us-east-1";
      bucket = "grahamc-nixops-state";
      key = "personal.nixops";
      kms_keyid = "166c5cbe-b827-4105-bdf4-a2db9b52efb4";
    };
  };

  flexo = {
    deployment = {
      targetHost = "147.75.105.137";
    };
    ...
  };
}

Store in Memory:

{
  network = {
    storage.memory = {};
  };

  flexo = {
    deployment = {
      targetHost = "147.75.105.137";
    };
    ...
  };
}

Legacy, exactly as master works now:

{
  network = {
    storage.legacy = {};
  };

  flexo = {
    deployment = {
      targetHost = "147.75.105.137";
    };
    ...
  };
}

grahamc · 2020-03-26T16:01:57Z

nixops/storage/legacy.py

+    # Making it part of the type definition allows adding new
+    # arguments later.
+    def fetchToFile(self, path: str, **kwargs) -> None:
+        os.symlink(os.path.abspath(self.state_location()), path)


This at one point did not abspath, which was causing problems with legacy paths to state files.

It seems to me like it would be a bit safer to make an actual copy rather than a symlink.

That way we know that if something goes wrong in the middle of writing the state file to disk, the "proper" statefile won't be corrupted.

If we were to copy, we would need to implement our own implementation of locking on top. The legacy backend already supports its own locking. My thinking is it would be better to find a way to "escape" the legacy backend, or implement a "backup" feature in the Legacy backend, than implement copying. What do you think?

domenkozar · 2020-03-26T17:36:35Z

nixops/storage/s3.py

+            ConditionExpression="attribute_not_exists(LockID)",
+        )
+
+    def unlock(self, path: str) -> None:


Would be cool to use contextmanager for locking as well to ensure a bit of safety

This way it can be moved into Storage abstraction and then backend can implement it or not

I'm not sure we should completely conflate looking & storage.
S3, Backblaze B2 & friends don't have any locking mechanisms themselves, so you may want to utilise a dynamodb table for locking.

It's fine if a storage backend implements a default locking mechanism though, as long as it can be overriden by the user.

nixos-discourse · 2020-03-28T17:04:18Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/using-nixops-in-a-team/6459/4

grahamc · 2020-04-01T00:49:50Z

Right now if you don't define network, you get a bad error:

RuntimeError: error: attribute 'network' in selection path 'network' not found

grahamc · 2020-04-01T00:50:32Z

If you define network.storage = {} you get a bad error:

    key = list(storage.keys()).pop()
IndexError: pop from empty list

nixos-discourse · 2020-04-01T16:38:41Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/tweag-nix-dev-update/6525/1

ip1981 · 2020-04-20T19:50:35Z

A PR can be (and converted to) to a draft, so you wouldn't need a "do not merge" warning.

grahamc · 2020-04-23T14:09:24Z

`nixops.nix` is a much more obvious name and really indicates what tooling is supposed to be used to build/deploy by just eyeballing a directory layout. We also don't want to occupy generic words possibly already used in some deployments.

…ers such as is_flake

This will soon contain a reference to readthedocs.

adisbladis · 2021-07-07T10:51:46Z

We're never going to move forward with this without getting this change out to more people for review.

Therefore I'm merging this, with the caveat that it might be somewhat changed before a final release.

rehno-lindeque · 2021-07-10T17:12:47Z

nixops/evaluation.py

+1. Open %s
+2. Add:
+    network.storage.legacy = {
+      databasefile = "~/.nixops/deployments.nixops"


There's a reference to databasefile here that doesn't seem to exist in the code

rehno-lindeque · 2021-07-11T12:36:37Z

remove the S3 one and relocate it to nixops-aws

Looks like the relocation part of this one still needs doing if I'm not mistaken? I couldn't find any reference to StorageBackend in nixops-aws.

roberth · 2021-07-30T13:13:28Z

Whew, I finally got around to playing with this. Good call merging this though!

My observations so far:

The interface is quite coarse grained, dealing with the whole db only and having no knowledge of which deployment is used. This coarseness means that state file history becomes noisy and locking more problematic as the number of deployments increases. Users can choose distinct backend configurations to compensate for this, but that's effectively duplicating the -d feature that already exists. I'd be ok removing -d altogether, but that's perhaps too radical. Making the interface fine grained seems like the way to go.

State is always written, even for read-only operations like nixops info. I could compensate for this in a backend, but that's not the right solution. The same problem applies to locking, where it can't be worked around in the backend. So if NixOps needs to know what it's about to do anyway, it may as well use that knowledge in order not to write state redundantly.

Sqlite isn't really an interchange format. If something goes wrong, it's hard to read. I don't know about its format/version strategy (maybe it's good though?) and it's not easy to diff. Perhaps the backends could be based on nixops import/export instead?

Which brings us to: nixops import --include-keys-like functionality doesn't seem to be included. I haven't tested it and I may have read the code wrong, but it also seems like a good idea to have a single code path for this stuff anyway.

Mic92 · 2021-09-20T20:16:26Z

Is the memory backend functional at all? We tried to use both legacy backend and memory backend in nix-community/infra#123 but they both seem not to work. nixops stable is also broken, which means there is no functional nixops left.

talyz · 2021-10-21T14:53:33Z

Yeah, the legacy backend was broken by this, since it now ignores the nixAttrs state attribute. I've solved this by using the require attribute to import the deployment files from the network file. Specifying the network file is however cumbersome, since the --network flag currently expects a directory, not a file. To make it easier, I've opened #1480.

With that in place, I think we could essentially drop deployments altogether without losing granularity: instead of running nixops deploy -d my_deployment you'd get the same result by running nixops deploy -n my_deployment.nix, as long as my_deployment.nix requires the appropriate files and specifies the backend to use. Note however that this strategy doesn't work with the current legacy backend, since it doesn't allow choosing where the state is stored. A simple json file backend based on import/export, like @roberth is suggesting, would solve this.

grahamc force-pushed the context-manager-states-rebase branch 3 times, most recently from cb46a0a to d105b44 Compare March 26, 2020 16:01

grahamc commented Mar 26, 2020

View reviewed changes

grahamc force-pushed the context-manager-states-rebase branch from d105b44 to 63b64d0 Compare March 26, 2020 17:21

This was referenced Mar 26, 2020

Key/value backend #624

Closed

Key/Value backend #865

Closed

domenkozar reviewed Mar 26, 2020

View reviewed changes

grahamc force-pushed the context-manager-states-rebase branch from 63b64d0 to 2834cbd Compare March 26, 2020 21:10

peterhoeg mentioned this pull request Mar 29, 2020

Feature Request: Support remote state storage and locking #600

Closed

adisbladis mentioned this pull request Apr 15, 2020

WIP: Add deploy without creating a state file #1248

Closed

grahamc mentioned this pull request Apr 20, 2020

Release 2.0 old tracking issue #1242

Open

16 tasks

grahamc added this to the 2.0 milestone Apr 20, 2020

grahamc marked this pull request as draft April 23, 2020 14:04

grahamc force-pushed the context-manager-states-rebase branch 2 times, most recently from 6d13093 to 9f3d08e Compare April 29, 2020 19:22

adisbladis force-pushed the context-manager-states-rebase branch from 9f3d08e to 8becbe2 Compare June 4, 2020 14:10

adisbladis force-pushed the context-manager-states-rebase branch from 8becbe2 to 35a0a89 Compare July 10, 2020 14:51

grahamc force-pushed the context-manager-states-rebase branch 2 times, most recently from 8fcb3da to e039f1e Compare July 10, 2020 17:14

adisbladis added 18 commits July 7, 2021 05:41

nixops/backends/none.py: Add more static typing

b879324

nixops/deployment.py: Add more static typing

8256550

nixops/nix_expr.py: Add more static typing

9687d74

nixops/plugins/__init__.py: Add more static typing

a92ebfe

nixops/plugins/manager.py: Add more static typing

b3bad1c

nixops/resources/__init__.py: Add more static typing

7ce24cf

nixops/resources/ssh_keypair.py: Add more static typing

e48422a

nixops/script_defs.py: Add more static typing

9feab77

nixops/plugins/__init__.py: Add more static typing

6bcdee4

nixops/plugins/manager.py: Add more static typing

0c83259

nixops/util.py: Add more static typing

5f542a0

Reduce nix-instantiate calls from 3 to 2

7d81a21

Remove all traces of s3 storage/locking implementations

4ea1728

Rename network.nix to nixops.nix

bc31429

`nixops.nix` is a much more obvious name and really indicates what tooling is supposed to be used to build/deploy by just eyeballing a directory layout. We also don't want to occupy generic words possibly already used in some deployments.

Add migration step docs

d87cf56

Add state migration docs

b2b79b4

Fix output of migration error message to not include internal paramet…

e14c835

…ers such as is_flake

Remove TODO from migration error message output

3f7d77a

This will soon contain a reference to readthedocs.

adisbladis force-pushed the context-manager-states-rebase branch from e9bb679 to 3f7d77a Compare July 7, 2021 10:42

adisbladis merged commit 9b3ac91 into NixOS:master Jul 7, 2021

rehno-lindeque reviewed Jul 10, 2021

View reviewed changes

talyz mentioned this pull request Jul 27, 2021

Legacy deployments are broken #1460

Open

roberth mentioned this pull request Aug 6, 2021

Always provide our known_hosts in addition to user known_hosts #1464

Merged

brprice mentioned this pull request Jan 30, 2022

Regression: self.rev is absent #1503

Open

steve-chavez mentioned this pull request Jul 19, 2022

avoid copy when encoding request body PostgREST/postgrest#2349

Merged

Uh oh!

Conversation

grahamc commented Mar 24, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

grahamc Mar 26, 2020

Choose a reason for hiding this comment

Uh oh!

kyrias Apr 20, 2020

Choose a reason for hiding this comment

Uh oh!

grahamc Apr 23, 2020

Choose a reason for hiding this comment

Uh oh!

domenkozar Mar 26, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

domenkozar Mar 26, 2020

Choose a reason for hiding this comment

Uh oh!

adisbladis Apr 2, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nixos-discourse commented Mar 28, 2020

Uh oh!

grahamc commented Apr 1, 2020

Uh oh!

grahamc commented Apr 1, 2020

Uh oh!

nixos-discourse commented Apr 1, 2020

Uh oh!

ip1981 commented Apr 20, 2020

Uh oh!

grahamc commented Apr 23, 2020 • edited by adisbladis Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

adisbladis commented Jul 7, 2021

Uh oh!

rehno-lindeque Jul 10, 2021

Choose a reason for hiding this comment

Uh oh!

rehno-lindeque commented Jul 11, 2021

Uh oh!

roberth commented Jul 30, 2021

Uh oh!

Mic92 commented Sep 20, 2021

Uh oh!

talyz commented Oct 21, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

12 participants

grahamc commented Mar 24, 2020 •

edited

Loading

domenkozar Mar 26, 2020 •

edited

Loading

adisbladis Apr 2, 2020 •

edited

Loading

grahamc commented Apr 23, 2020 •

edited by adisbladis

Loading