Configuring an MNP Environment

This guide contains instructions for installing HampsonRussell Geoview (HRS) on an

MNP network for processing large, computation-intensive jobs by specific HRS
modules.

An MNP environment enables you to process jobs quickly by using the aggregate
power and memory of many computers, that is, a cluster. Computation-intensive
jobs can be run on a heterogeneous network of 64-bit computers as if they were
running on a single large, parallel computer. In this way, large computational
problems can be solved more cost-effectively.

This information is primarily intended for use by personnel who install applications,
set up licensing, and configure MNP environments. This document contains
information about system requirements and other factors to consider as you plan
the installation. Read it carefully before you install the software.

The HRS implementation of MNP is based on Message Passing Interface (MPI).

Currently, only the native scheduler is supported.

Please note that new capabilities are being added for MNP every week. This guide
will help get you started, but it is an evolving document. If you want to start testing
and using MNP, we want to know. Contact your local sales or support person for
additional support resources.
 Configuring an MNP Environment

Contents
DOCUMENT CONVENTIONS..........................................................................................................................................3

NOTATION .................................................................................................................................................................... 3
TERMINOLOGY............................................................................................................................................................... 3
LICENSING......................................................................................................................................................................5

THIRD-PARTY SOFTWARE .............................................................................................................................................6

SUPPORTED OPERATING SYSTEMS...............................................................................................................................7
LINUX PLATFORM: LOCAL NETWORK ...........................................................................................................................8
BEFORE YOU BEGIN......................................................................................................................................................... 8
VERIFY SYSTEMS ............................................................................................................................................................. 9
SET UP PASSWORDLESS SSH ........................................................................................................................................... 10
SET UP THE NODES........................................................................................................................................................ 11
SET UP COMMUNICATION .............................................................................................................................................. 12
Tips for [Link] .................................................................................................................................... 12
TROUBLESHOOTING LINUX ............................................................................................................................................. 13
LINUX PLATFORM: AZURE CLOUD ..............................................................................................................................16

REQUIREMENTS ........................................................................................................................................................... 16
GET STARTED ON AZURE ................................................................................................................................................ 17
SET UP AZURE STORAGE ACCOUNT ................................................................................................................................... 17
EDIT THE SCRIPT ........................................................................................................................................................... 19
POPULATE THE AZURE STORAGE ACCOUNT ........................................................................................................................ 20
RUN THE SCRIPT AND THE HRS PROCESS ........................................................................................................................... 22
WINDOWS PLATFORM: LOCAL NETWORK .................................................................................................................23

BEFORE YOU BEGIN....................................................................................................................................................... 23

SET UP THE SYSTEM ...................................................................................................................................................... 24
SET UP THE NODES........................................................................................................................................................ 24
SET UP COMMUNICATION .............................................................................................................................................. 25
KNOWN LIMITATIONS.................................................................................................................................................26
TECHNICAL SUPPORT & ASSISTANCE .........................................................................................................................27

Updated for HRS 10.6 2 Last Updated: September 2020

 Configuring an MNP Environment

Document conventions
Notation
The following conventions are used throughout this guide:

Notation Description

terminal This font is used for paths, user input, program output, or error
messages.
Example: tar xv

<user> This identifies a generic element that must be replaced by the

appropriate value for your installation.

<instdir> This refers to the path and directory used for installing the
GeoSoftware application that is being discussed.
For example, on Linux, the installation path is:

/net/apps/cgg/geosoftware/hrs

text The indicates text in the HRS user interface (such as buttons and
menus)

Terminology
This terminology is consistent with that commonly used for network and cloud
computing.

System: the combination of all hardware, software, and data used for MNP

User computer: the work station where the user will click Run Multi-node

Node: one or more CPUs and threads on a computer Cluster: the master and
the worker nodes

Hostname: the unique identifier of a computer, the computer’s unique given

name

Updated for HRS 10.6 3 Last Updated: September 2020

 Configuring an MNP Environment

Master: the node that divides the job among the workers and collates the
completed output into a single file

Worker: one of several nodes that process portions of the input data, one
portion per node

Updated for HRS 10.6 4 Last Updated: September 2020

 Configuring an MNP Environment

Licensing
MNP requires a specific license to run, one HRS-mnp license per node in the cluster
(master and workers).

In addition to consuming a license for the type of process being run, each worker in
your cluster will consume one MNP license. If the number of workers exceeds the
number of available licenses, the job will only run on the number of nodes
associated with the number of licenses available. If no licenses are available, the
job will not run. The job may then be run at a later time when more licenses are
available.

The license file should have proper file permissions for each user and/or machine
that will be operating the software.

The license must be a server license (not a local license).

Follow these steps to set up the license file:

1. Place the license file on a server (and directory) that the user computer can
access. The master computer must also be able to access the licenses.

2. Make the license manager aware of your new license.

3. Configure HampsonRussell to use the new license.

Full instructions are in the HRS Installation Guide.

Updated for HRS 10.6 5 Last Updated: September 2020

 Configuring an MNP Environment

Third-party software
When you install HampsonRussell Geoview, the following types of third-party
software are also installed if they are not already present on the system. These lists
are not complete; they identify key third-party software.

The following third-party software will be installed:

• Objectivity [Link]
• Open MPI for MNP support

Updated for HRS 10.6 6 Last Updated: September 2020

 Configuring an MNP Environment

Supported operating systems

MNP can be used on all operating systems supported by HampsonRussell.

Note: Different operating systems cannot be used together either in the cluster or to
connect to the cluster

Updated for HRS 10.6 7 Last Updated: September 2020

 Configuring an MNP Environment

Linux Platform: Local Network

These steps configure MNP for Linux computers used as the user computer and the
cluster nodes. The cluster computers are part of a traditional private network.

Before you begin

Before you configure the MNP environment, review the following requirements and
recommendations:

• You need specific licenses for MNP (see Licensing)

• You need the HampsonRussell installer file

• The systems must have access to the same shared network for the HRS
installation directory and the project data directory.

• The user computer and the master computer must have acess to the same
license server that has the MNP licenses and the licenses for the processes
being run.

• Workers access input data over the network using NFS, so it is best for the
workers to have a high-speed, low-latency network (GigE or better).

• The systems should have the same amount of memory and use the same
operating system, and the processors should be capable of roughly equal
performance.

• The firewall between all cluster nodes must be open. This is required by Open
MPI. This requires Administrator access.

Updated for HRS 10.6 8 Last Updated: September 2020

 Configuring an MNP Environment

Verify systems
Verify that all computers that you want to use in the MNP system meet the
following requirements:

1. The user must have a shared home directory.

Note: If you are not using a shared home directory, the network computers must have
have an identical home directory accessible by the same directory path on the
same physical disk. The network computers must also have an identical login shell,
user ID, and group ID.

2. The HampsonRussell software (HRS) is installed on the user computer and

the cluster computers in the same directory path and on the same physical
disk (or the installation directory can be on a shared network accesssible by
the user computer and all cluster computers).

3. The HRS project data is saved in a directory with the same directory path
and on the same physical disk (or the directory with the project data can be
on a shared network that is accessible by the user computer and the cluster
computers).
Updated for HRS 10.6 9 Last Updated: September 2020
 Configuring an MNP Environment

4. Name resolution must be set up for the network. The nslookup command
must return the correct IP address for all master and worker hosts.

The best way to satisfy all of these requirements is by using NIS (yellow pages) in
combination with the automounter. For more information, see the applicable
sections of your operating system documentation.

Note: Do NOT use /net/<hostname>/ to access remote directories.

Using this will result in performance and operational problems.

Set up passwordless SSH

MNP uses the Secure Shell (SSH) protocol to create the cluster. To create the
cluster in an automated manner, it is necessary for the user computer that runs
Geoview to establish an SSH session with each computer in the cluster without
being prompted for a password.

This section describes how to get SSH to use “password-less” logins for MNP
processing. This method is built into the Linux infrastructure and is not a feature of
our software. For this reason, we do not provide customer support for SSH-related
issues.

1. Verify that SSH is installed on all of the systems that you use for MNP. Use
SSH to log into each computer that will be used in the cluster (using the
user’s name and password). Accept the prompt to confirm the authorized
connection.

2. Log on to the user computer.

3. Create a key pair for the master. Enter this command and press Enter:

ssh-keygen -t dsa

4. When you are asked to supply filenames for the key pairs, accept the default
filenames.

5. When you are asked for a passphrase, leave the line empty and press Enter.

6. Do the same when you are asked for the passphrase again.
Updated for HRS 10.6 10 Last Updated: September 2020
 Configuring an MNP Environment

Two new files are created: ~/.ssh/id_dsa and ~/.ssh/id_dsa.pub. The first
file is the private key which should not be given to anyone. (Set the
permissions for it to 700.) The second file is the public key.

7. Log in to any worker.

8. Copy the public key to the worker. Enter:

ssh-copy-id -i ~/.ssh/id_dsa.pub [user]@[hostname]

9. Use SSH to log into all computers in the cluster and confirm their identities.

This time you will not be prompted for a password.

Note: If you’ve configured MNP previously, remove the source <instdir>/environ_mnp

line from the .cshrc file. This modification is no longer necessary as of HRS
[Link] that SSH is installed on all of the systems that you use for MNP.

Set up the nodes

Specify the nodes to be used in the MNP cluster, at least 2 nodes.

1. Create a folder named mnp in the .hrs directory

2. Create a text file, [Link], in the .hrs\mnp folder

3. Enter the hostnames of the cluster computers in the file. The first entry is the
master, and the other entries are the workers. Hostnames can be repeated
to create multiple nodes on the same host. We recommend repeating the
master’s hostname since it is idle for most of the process runtime.

Note: You can enter the IP addresses instead of the hostnames. However, we
recommend using the hostnames because they are less likely to change, unlike IP
addresses.

Updated for HRS 10.6 11 Last Updated: September 2020

 Configuring an MNP Environment

Set up communication
HRS MNP uses Open MPI to provide clustering communication between nodes. The
mpiexec command will execute geoviewBatch to create the MNP cluster. mpiexec
requires parameters that are specific for your environment. The [Link]
file is where you specify these parameters.

1. Create a text file, [Link], in C:\\users\[username]\.hrs\mnp

2. Enter one of the below example commands, edited for your environment.

Example of a typical [Link] file content:

mpiexec -bind-to none --mca -btl_tcp_if_include eth0

Typical parameters for mpiexec include:

• -bind-to none
o This allows a process to use multiple cores.
o We recommend always including this parameter.
• --mca -btl_tcp_if_include [network interface1],[network_interface2]...
o mpiexec uses all the network interfaces on the computer to
communicate with other cluster nodes.
o Use this command to include only certain interfaces.
• --mca -btl_tcp_if_exclude [network interface1],[network_interface2]...
o mpiexec uses all the network interfaces on the computer to
communicate with other cluster nodes.
o Use this command to exclude certain interfaces.
• --mca -btl tcp,self
o Directs mpiexec to use TCP-based communications.

Refer to [Link] for common questions about using Open

MPI.

Tips for [Link]

• Do not specify the mpiexec “-host” argument. Geoview will use the contents
of the [Link] file to generate this argument.
• Do not specify the executable that mpiexec will run. That will be automatically
added at runtime.

Updated for HRS 10.6 12 Last Updated: September 2020

 Configuring an MNP Environment

• To test your mpiexec command, use the Linux hostname command with your
mpiexec command in a terminal. You will need to specify hosts in this
command. mpiexec should respond with the hosts you specified.

For example:
>source [path_to_hrs]/environ

>[path_to_hrs]/3rdparty/openmpi-3.1.0-linux64/bin/mpiexec --mca -
btl_tcp_if_include eth0 -bind-to none -host host1,host2 hostname

host1

host2

Troubleshooting Linux
Problems using MNP are usually due to improper SSH configuration, improper MPI
configuration (nodes and communication) or improper network configuration. The
best way to diagnose the problem is to view the runtime message files. Check the
messages for failed or incomplete actions.

Is the password-less SSH is working properly?

At a terminal, SSH to each machine in your ~/.hrs/mnp/[Link] file. Verify that

you successfully log into each machine without being prompted for a password.

Common problems with password-less SSH are permission issues with your ~/.ssh
directory. Verify that your permissions are set as per the instructions in Set up
passwordless SSH, p.10.

Another common issue is SELinux preventing password-less SSH. Try running chcon
-R -v system_u:object_r:usr_t:s0 ~/.ssh/ to see if that fixes the problem.

Are the firewall ports are open?

Open MPI requires this. The simplest way to ensure this is to disable the firewall on
the hosts in your MNP network.

Is the mpiexec command configured correctly?

The two most common configuration issues are ensuring the correct network
interface is specified and that the TCP protocol is used. Configure the network
interfaces with the --mca -btl_tcp_if_include and --mca -btl_tcp_if_exclude options.
Specify the TCP protocol using the --mca btl tcp,self option. The mpiexec command
is configured in the ~/.hrs/mnp/[Link] file. The runtime message output
Updated for HRS 10.6 13 Last Updated: September 2020
 Configuring an MNP Environment

may also help determine problems using mpiexec. Examine

the<[Link]>/[Link]/ directory. You will find a number of <n>.job
directories. Open the directory with the date that corresponds to when the MNP was
run. In that directory you will find a messages directory. The contents of those files
may help determine why the mpiexec command failed.

Multiple Geoview Batch Controller windows open?

On slower computers, multiple Geoview Batch Controller windows may open when
starting a batch or MNP job. To fix this problem, you can set an environment
variable in your Linux login script. In your login script, set the
HRS_BATCH_STARTUP_DELAY variable to a value greater than 10. Twenty is a
good value to try. For example, if you are using csh, add this line to your .cshrc
file:

setenv HRS_BATCH_STARTUP_DELAY 20

On slower computers, Geoview may crash while drawing the results of an MNP job.
To fix this problem, you can set an environment variable in your Linux login script.
In your login script, set the HRS_BATCH_UPDATE_DELAY variable to a value
greater than 5. Ten is a good value to try. For example, if you are using csh, add
this line to your .cshrc file:

setenv HRS_BATCH_UPDATE_DELAY 20

OpenSSH authentication

If you add a computer to your MNP cluster, a dialog may open asking you to verify
the authenticity of the host when the job starts. As long as you are certain the host
is correct, type “yes” into the dialog to continue.

Updated for HRS 10.6 14 Last Updated: September 2020

 Configuring an MNP Environment

Updated for HRS 10.6 15 Last Updated: September 2020

 Configuring an MNP Environment

Linux Platform: Azure Cloud

These steps configure MNP for Linux computers used as the user computer and the
cluster nodes. The cluster computers are part of the Azure cloud platform.

The MNP environment is set up on Azure using a script. The Azure Deployment
Script is a text file of python code named “python [Link]”. It performs these
actions:

• Creates the virtual machines and the resource group on Azure

• Allocates two computers on Azure, one to be the master node and one a
client

• The master has the HRS software and the project data. It mounts the project
onto the workers and starts them

• Installs NFS on all virtual machines in the resource group

• States the key (password) for the Azure subscription

• Installs HRS Geoview on the master node

• Mounts Geoview on the master and worker nodes

• Uploads the project data onto the master, and sets the data drives to the
same filepath as Geoview

Note: The script sets up the master to use the Standard D16s v3 (16 vcpus, 64 GB
memory) Azure series with an attached premium ssd data disk for the project
data. All VMs have NFS installed. Geoview is installed in an NFS sharable directory.
The workers are set up to use Standard F8s (8 vcpus, 16 GB memory). They share
the Geoview installation and the project data on the master node.

Requirements
Before you configure the MNP environment, verify that you have the following
items:

• Specific licenses (see Licensing)

• Azure deployment script (request this from your regional Sales contact)

• HRS installation file

Updated for HRS 10.6 16 Last Updated: September 2020

 Configuring an MNP Environment

• Azure subscription (your customer account with Microsoft)

Microsoft has a large volume of documents and videos describing how Azure and
cloud computing works as well as the terminology used in cloud computing. Visit
their site for more information about signing up for a subscription.

Get started on Azure

First, open your Azure subscription site.

Record your subscription id, client id, and key (secret password). You’ll need these
to configure MNP on Azure.

Next, create a single virtual machine (VM) for the HRS Geoview application and
licenses. This will be a preliminary VM solely for setting up Azure for MNP. We
recommend using the Azure Standard D v3 series for this VM. Your company may
already have a VM with their applications that are being used in the cloud.

The script is written in Python and, therefore, you will need to install several Python
items. The items are commented in the script along with an example. Open the
script in a text editor to see the Python items. Install them on the preliminary VM.

Install NFS on the preliminary VM. Use the Python command yum install nfs-
utils

Save the HRS project (folders and files) to the preliminary VM’s SSD volume.

Note: Project files can be permanently stored in a Azure storage account, but it is better
to download it to the user’s VM using “azcopy”. This overcomes I/O bottleneck.
After a project is finished, you can upload the project files back to the Azure
unmanaged storage or download to a local drive.

On the preliminary VM, create a folder named HRS10_4.

Install HRS to the VM HRS10_4 folder using the same steps as those for installing it
on a local computer or your network including your licenses. Tar the HRS10_4
folder, then gzip it. The final folder name will be HRS10_4.[Link].

Set up Azure storage account

You will need to create a storage account on Azure for the scipt and files
necessary for MNP to run. Record the account name and key.

Updated for HRS 10.6 17 Last Updated: September 2020

 Configuring an MNP Environment

In your storage account, create two folders for Blobs-type data: deployment and
installers. The “gsdeployment” folder in the below image will be created by the
script.

Azure deployment scripts

Geoview installation
package

Register the active directory. In Azure Active Directory>App registrations, create a

new application registration. Enter app ID and the keys.

Updated for HRS 10.6 18 Last Updated: September 2020

 Configuring an MNP Environment

Record the Application ID.

Edit the script

We have sent you the script and corresponding folders. After the script is edited,
these will be saved to the storage account along with the compressed HRS
application and the project data.

Open the script in a text editor. Edit the below bolded items:

SUBSCRIPTION_ID = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"

CLIENT_ID = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"

CLIENT_SECRET = “xxxxxxxxxxxxxxx"

Updated for HRS 10.6 19 Last Updated: September 2020

 Configuring an MNP Environment

TEMPLATE_PARAMS = ['nWorkers', 'nClients', 'clientVmSize', 'serverDiskSize']

REGION = "canadacentral"

DEPLOYMENT_NAME = "geoview_test_environment"

"serverVmSize": {
"defaultValue": "Standard_D4s_v3",
"type": "string",
"metadata": {
"description": "Data server VM size"
}
},
"numberOfWorkers": {
"minValue": 1,
"defaultValue": 2,
"type": "int",
"metadata": {
"description": "Number of MNP worker VMs"
}
},
"variables": {
"vmName": "geoviews-vm",
"vmUserName": "gs-user",
"vmPassword": “xxxxxxxxxx",
"nsgName": "geoview-nsg",
"nicName": "geoview-nic",
"storageAccountName": "gsresources",
"storageAccountKey":"/X2cccccccchQqRFFPl27zwdJEL0oKQ==",
"deploymentURI":
"[Link]
"sshKeyData": "ssh-rsa AAAAxxxxxxxxxx=="
},

Populate the Azure storage account

Upload the script to your Azure storage account’s “deployment” folder.

Upload HRS10_4.[Link] to your Azure storage account’s “installers” folder.

Updated for HRS 10.6 20 Last Updated: September 2020

 Configuring an MNP Environment

Updated for HRS 10.6 21 Last Updated: September 2020

 Configuring an MNP Environment

Run the script and the HRS process

On the preliminary VM, run “python [Link]”. As an argument, enter the number
of nodes to be used to run the HRS process.

Remotely connect to the new master (created by the script).

Start HRS.

Setup the process parameters and run the process using Run Multi-node.

After the job finishes and if no more jobs are to be run, delete the resource group
on Azure. As long as it exists, you may be charged for using it.

Updated for HRS 10.6 22 Last Updated: September 2020

 Configuring an MNP Environment

Windows Platform: Local Network

These steps configure MNP for Window computers used as the user computer and
the cluster nodes. The cluster computers are part of a traditional private network.

Before you begin

You need:

• Specific licenses (see Licensing)

• HampsonRussell installer file

• Web access to download the MS MPI installer

• Physical or remote access to the user computer and every cluster node

Also:

• The user needs access to all network drives (passwords are usually not
required once the user is signed in to their computer)

• The user computer and the cluster computers must have access to the
project data directory on a shared network.

• The user computer and the master computer must have acess to the same
license server that has the MNP licenses and the licenses for the processes
being run.

• Workers access input data over the network using SMB, so it is best for the
workers to have a high-speed, low-latency network (GigE or better).

• The systems should have the same amount of memory and use the same
operating system, and the processors should be capable of roughly equal
performance.

• The firewall between all cluster nodes must be open. This is required by Open
MPI. This requires Administrator access.

Updated for HRS 10.6 23 Last Updated: September 2020

 Configuring an MNP Environment

Set up the system

On every cluster node and the user computer:

1. Install HampsonRussell (HRS) to the same directory path

2. Install MS MPI

3. Run the Windows Services application, change the MS-MPI Launch Service
start type to Automatic, and start this service.

On the network:

1. Set up name resolution.

2. Perform a DNS lookup to verify that the IP addresses and corresponding host
names are correct for all cluster computers.

Set up the nodes

On the user computer:

Updated for HRS 10.6 24 Last Updated: September 2020

 Configuring an MNP Environment

1. Create a folder named mnp in the C:\\users\[username]\.hrs directory

2. Create a text file, [Link], in the mnp folder

3. Enter the hostnames of the cluster computers in the file. The first entry is the
master, and the others are the workers. Hostnames can be repeated to
create multiple nodes on the same host. We recommend repeating the
master’s hostname since it is idle for most of the process runtime.

Note: You can enter the IP addresses instead of the hostnames. However, we
recommend using the hostnames because they are less likely to change, unlike IP
addresses.

Set up communication
HRS MNP uses Open MPI to provide clustering communication between nodes. The
mpiexec command will execute geoviewBatch to create the MNP cluster. mpiexec
requires parameters that are specific for your environment. For Windows, it is
highly unlikely that you will need to edit the [Link] file.

1. Create a text file, [Link], in C:\\users\[username]\hrs\mnp

2. Enter “mpiexec” in the first line, and save and close the file

If communication is not set up correctly, Windows will open an error message.

Updated for HRS 10.6 25 Last Updated: September 2020

 Configuring an MNP Environment

Known Limitations
MNP currently has a number of known limitations. Please be aware of these:

1. Pause and stop functionality for MNP jobs is not currently available in
Geoview Batch Controller. To stop a job, you must kill the mpiexec process
associated with the job. The mpiexec process runs on the computer that
started the job.

2. Running multiple MNP jobs for the same user on the same hosts is not
supported and may cause a failure with the associated jobs. Users should
queue all MNP jobs to avoid this problem.

Updated for HRS 10.6 26 Last Updated: September 2020

 Configuring an MNP Environment

TECHNICAL SUPPORT & ASSISTANCE

If you are experiencing issues with installation or require assistance with any items outlined in
this manual, please contact our support staff specific to your region. To access email addresses
for each office listed below, please visit: [Link]
Do/GeoSoftware/GeoSoftware-Support-Contact-Information

Updated for HRS 10.6 27 Last Updated: September 2020