Stand-alone container for a BigchainDB node by muawiakh · Pull Request #2424 · bigchaindb/bigchaindb

muawiakh · 2018-07-31T12:14:57Z

Background

Some customers have asked for a single stand-alone BigchainDB image.
Easier for early users of BigchainDB

Solution

Make BigchainDB, MongoDB and Tendermint be part of the same container, even though it is an anti-pattern but this has been requested at several instances by customers/partners.
Easier for early adopters of BigchainDB i.e. 2 step approach:
- docker pull
- docker run

TODOs

Add documentation of this workflow.
Tag image once this PR is merged.

Issues this Resolves

Resolves #2238 (or at least I think it does. -Troy)

codecov-io · 2018-07-31T12:23:23Z

Codecov Report

Merging #2424 into master will increase coverage by 1.2%.
The diff coverage is n/a.

@@            Coverage Diff            @@
##           master    #2424     +/-   ##
=========================================
+ Coverage   88.43%   89.64%   +1.2%     
=========================================
  Files          39       40      +1     
  Lines        2275     2589    +314     
=========================================
+ Hits         2012     2321    +309     
- Misses        263      268      +5

ldmberman · 2018-08-01T09:06:39Z

docs/server/source/appendices/all-in-one-bigchaindb.md

@@ -0,0 +1,84 @@
+# Run BigchainDB with all-in-one Docker
+
+**NOT for Production Use**


What are the production use criteria? We do not have such warnings in the docs of other setups.

For starters, a single container for multiple stateful services is not a good sign. This is a very opinionated image. For production use cases someone might need more than a single Docker image.

I think we all acknowledge that having all services running in a single container is an anti-pattern. This all-in-one container is just a convenience and we should have that warning here if not anywhere else.

anti-pattern is not really an explanation.

Moreover, any setup we have might be labeled "not for production use" at least for a reason that we do not have a stable release yet.

Therefore, to avoid disorienting the users I would refrain from such labels. Instead, we can add a recommendation to use a multi-image setup with a brief explanation of how it is better.

anti-pattern is not really an explanation.

Why?

anti-pattern is just a label.. Does such a setup perform worse? If yes, why? Is it more difficult to maintain such a setup?

I have no experience regarding this. There are some sources which suggest this is an anti-pattern but the docker documentation has no mention of it.

I am not really sure if we can say anything about the production readiness of either of these approaches until we try them ourselves?

If someone wants to use just a single node in production then maybe yes, because in terms of performance or maintenance it is not a big overhead. If you want to have a network it will be more complex.

Maybe if we have dynamically adding validators implemented, i.e. you deploy one node with this and dynamically add using upsert, there are some questions yet to be answered. That is why I suggested not to be used in production.

Nodes are supposed to be managed by different operators. Operators may choose different ways of deploying them. What are the particular implications of one operator deciding to deploy a single image? I do not see any so far.

This image does a tendermint init, which implies all the nodes spawn different chains. So operators will have to change that behavior or come up with a new workflow.

Ah, makes sense!

In other words, a single-node cluster configuration is hardcoded in the image. It is not currently well suited for deploying a network. Could you, perhaps, add such a clarification?

ldmberman · 2018-08-01T09:09:35Z

docs/server/source/appendices/all-in-one-bigchaindb.md

+- MongoDB
+- Tendermint
+
+*We understand that it is an anti-pattern to run multiple services inside a docker.*


It makes sense to recommend a multi-image alternative, add a link to its description, and briefly mention its benefits. it is an anti-pattern is not really an explanation so I would omit it.

ldmberman · 2018-08-01T09:10:36Z

Dockerfile-all-in-one

+
+# When developing with Python in a docker container, we are using PYTHONBUFFERED
+# to force stdin, stdout and stderr to be totally unbuffered and to capture logs/outputs
+ENV PYTHONUNBUFFERED 0


Where is this env variable used?

Actually this should be removed, since I will be removing:

pip install --no-cache-dir --process-dependency-links -e .[dev]

But this parameter is mostly required for dev use cases and since monit is maintaining the services, it does not make sense.

kansi · 2018-08-01T09:16:00Z

executed docker build - < Dockerfile-all-in-on, got the following error

  Running setup.py install for pycparser: started
    Running setup.py install for pycparser: finished with status 'done'
  Running setup.py install for cffi: started
    Running setup.py install for cffi: finished with status 'done'
Successfully installed cffi-1.11.5 pip-18.0 pycparser-2.18
Directory '.' is not installable. File 'setup.py' not found.
The command '/bin/sh -c apk --update add sudo bash     && apk --update add python3 openssl ca-certificates git     && apk --update add --virtual build-dependencies python3-dev         libffi-dev openssl-dev build-base jq     && apk add --no-cache libstdc++ dpkg gnupg     && pip3 install --upgrade pip cffi     && pip install --no-cache-dir --process-dependency-links -e .[dev]     && apk del build-dependencies     && rm -f /var/cache/apk/*' returned a non-zero code: 1

kansi · 2018-08-01T09:17:09Z

Dockerfile-all-in-one

@@ -0,0 +1,58 @@
+ FROM alpine:latest


Nitpick: Indentation is not right here.

kansi · 2018-08-01T09:20:42Z

Dockerfile-all-in-one

+        libffi-dev openssl-dev build-base jq \
+    && apk add --no-cache libstdc++ dpkg gnupg \
+    && pip3 install --upgrade pip cffi \
+    && pip install --no-cache-dir --process-dependency-links -e .[dev] \


Is there a need to have dev level dependencies too?

Probably not, I wrote it with another use case in mind but you are right, it is not needed anymore, atleast for this file.

kansi · 2018-08-01T09:24:27Z

Dockerfile-all-in-one

+ENV TMHOME=/tendermint
+
+RUN addgroup tmuser \
+    && adduser -S -G tmuser tmuser -h "$TMHOME"


Is there a reason to have a separate user/group for Tendermint?

This is migrated from the official Tendermint docker image. Let me see if we can work without it.

kansi · 2018-08-01T09:31:53Z

pkg/scripts/bigchaindb-monit-config


    pushd \$4
-      nohup bigchaindb start >> \$3/bigchaindb.out.log 2>> \$3/bigchaindb.err.log &
+      nohup bigchaindb -l DEBUG start >> \$3/bigchaindb.out.log 2>> \$3/bigchaindb.err.log &


Should the log level be a variable? if bigchaindb-monit-config is intended to run and monitor production system then user should be able to select the necessary/required log level.

This is just a step towards that, not the production template. The user can still change it using the BigchainDB configuration file. I can remove it but without Debug we have very little logs.

- Fix nitpicks - Remove -e flag while installing because it is not needed - Remove unwanted env variables - Update some docs

- Clarify that this image is not well suited for a network deployment

ldmberman · 2018-08-02T10:33:17Z

Created #2434, restarted the build.

Stand-alone container for a BigchainDB node

973dec4

muawiakh requested a review from ldmberman July 31, 2018 12:14

muawiakh requested a review from shahbazn July 31, 2018 12:41

Add basic docs for all-in-one BigchainDB image

7c7b824

ldmberman reviewed Aug 1, 2018

View reviewed changes

kansi reviewed Aug 1, 2018

View reviewed changes

muawiakh added 2 commits August 1, 2018 12:12

Address comments

d3216bc

- Fix nitpicks - Remove -e flag while installing because it is not needed - Remove unwanted env variables - Update some docs

Add -e while installing otherwise doesn't work! Strange.

ee10173

kansi approved these changes Aug 1, 2018

View reviewed changes

Address comments II

b977ed6

- Clarify that this image is not well suited for a network deployment

ldmberman approved these changes Aug 2, 2018

View reviewed changes

ldmberman merged commit 66b243a into bigchaindb:master Aug 2, 2018

		@@ -0,0 +1,84 @@
		# Run BigchainDB with all-in-one Docker

		NOT for Production Use

Conversation

muawiakh commented Jul 31, 2018 • edited by ttmc Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Background

Solution

TODOs

Issues this Resolves

Uh oh!

codecov-io commented Jul 31, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ldmberman Aug 1, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kansi commented Aug 1, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ldmberman commented Aug 2, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

muawiakh commented Jul 31, 2018 •

edited by ttmc

Loading

codecov-io commented Jul 31, 2018 •

edited

Loading

ldmberman Aug 1, 2018 •

edited

Loading

kansi commented Aug 1, 2018 •

edited

Loading