You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
How to add new add-on software to elasticluster? #687
I have used ElastiCluster to create an SGE cluster successfully. I still need to install some dependencies to run some Snakemake jobs on the cluster. It is OK for me but a little challenge for other users. So, I would like to install those dependencies using ansible. I think the ElastiCluster add-on software like "Rstudio sever" is a good example. I may create new role like "my-appinstaller" to handle the tasks.
Here, I would like to know whether there is any instruction to create new roles for ElastiCluster and configure the setting to deploy some configurations and packages to front nodes and worker nodes.
Hello @weizhu365 and thanks for your interest in ElastiCluster!
I think there are two options to add new roles to be used by ElastiCluster:
Create your own .yml file with the playbook, then use it with elasticluster setup my-cluster -- myplaybook.yml
Note that, to save time, this requires that you create a cluster in two stages:
first create the cluster but do not configure it:
elasticluster start --no-setup my-cluster
then set it up with the additional playbook(s):
elasticluster setup my-cluster -- myplaybook.yml
Note that:
You can have more than one additional playbook file; they will be run in the order given.
Nothing bad will happen if you do a normal elasticluster start my-cluster (i.e., without the --no-setup option) and then a elasticluster setup -- myplaybook.yml; it's just that the standard ElastiCluster playbooks will run twice.
Install ElastiCluster from sources, navigate to the source directory, then cd elasticluster/share/playbooks and add your own roles in the roles directory. You can then call them from the gridengine.yml file (shortcut), or:
add a new .yml file for your new role (like the rstudio.yml file); the purpose of this file is to just call the additional role or roles for hosts in a specific group (let's say, group mysoftware);
add an include: statement for this new .yml file in main.yml
in the [setup/...] section of the config file, add the group mysoftware to all hosts where you want the software installed.
Whichever way you choose, if the software can be freely installed, I would be happy to work on a PR for making it available to other users of ElastiCluster!
Riccardo;
Thank you for the reply, which is very helpful. Actually, I had figured out to use the solution two, following the Rstudio example. Nevertheless, your reply makes me more confident. :-) The solution 1 is new to me.
The software to be installed are all open-source, and I believe that a general solution like this is useful to all ElastiCluster users.
Best,
Wei
On Nov 18, 2020, at 3:25 PM, Riccardo Murri ***@***.***> wrote:
Hello @weizhu365 and thanks for your interest in ElastiCluster!
I think there are two options to add new roles to be used by ElastiCluster:
• Create your own .yml file with the playbook, then use it with elasticluster setup my-cluster -- myplaybook.yml
Note that, to save time, this requires that you create a cluster in two stages:
• first create the cluster but do not configure it:
elasticluster start --no-setup my-cluster
• then set it up with the additional playbook(s):
elasticluster setup my-cluster -- myplaybook.yml
Note that:
• You can have more than one additional playbook file; they will be run in the order given.
• Nothing bad will happen if you do a normal elasticluster start my-cluster (i.e., without the --no-setup option) and then a elasticluster setup -- myplaybook.yml; it's just that the standard ElastiCluster playbooks will run twice.
• Install ElastiCluster from sources, navigate to the source directory, then cd elasticluster/share/playbooks and add your own roles in the roles directory. You can then call them from the gridengine.yml file (shortcut), or:
• add a new .yml file for your new role (like the rstudio.yml file); the purpose of this file is to just call the additional role or roles for hosts in a specific group (let's say, group mysoftware);
• add an include: statement for this new .yml file in main.yml
• in the [setup/...] section of the config file, add the group mysoftware to all hosts where you want the software installed.
Whichever way you choose, if the software can be freely installed, I would be happy to work on a PR for making it available to other users of ElastiCluster!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
Activity
riccardomurri commentedon Nov 18, 2020
Hello @weizhu365 and thanks for your interest in ElastiCluster!
I think there are two options to add new roles to be used by ElastiCluster:
Create your own
.ymlfile with the playbook, then use it withelasticluster setup my-cluster -- myplaybook.ymlNote that, to save time, this requires that you create a cluster in two stages:
first create the cluster but do not configure it:
elasticluster start --no-setup my-cluster
then set it up with the additional playbook(s):
elasticluster setup my-cluster -- myplaybook.yml
Note that:
elasticluster start my-cluster(i.e., without the--no-setupoption) and then aelasticluster setup -- myplaybook.yml; it's just that the standard ElastiCluster playbooks will run twice.Install ElastiCluster from sources, navigate to the source directory, then
cd elasticluster/share/playbooksand add your own roles in the roles directory. You can then call them from thegridengine.ymlfile (shortcut), or:.ymlfile for your new role (like therstudio.ymlfile); the purpose of this file is to just call the additional role or roles for hosts in a specific group (let's say, groupmysoftware);include:statement for this new.ymlfile inmain.yml[setup/...]section of the config file, add the groupmysoftwareto all hosts where you want the software installed.Whichever way you choose, if the software can be freely installed, I would be happy to work on a PR for making it available to other users of ElastiCluster!
weizhu365 commentedon Nov 18, 2020