Configuration¶

Many aspects of NOMAD and its operation can be modified through configuration. Most configuration items have reasonable defaults and typically only a small subset has to be overwritten. Configuration items are structured hierarchically. For example, the configuration item services.api_host denotes the attribute api_host in the configuration section services.

Configuration sources¶

Configuration items get their value based on a hierarchy of sources. The sources are applied in the following order of precedence, where later sources override earlier ones (see merging rules below):

Environment Variables: A variable like NOMAD_SERVICES_API_HOST. These have the highest priority and will override all other settings. NOMAD services will inspect the environment for any variables starting with NOMAD_. The rest of the name is interpreted as a configuration item, where sections and attributes are concatenated with a _. For example, the environment variable NOMAD_SERVICES_API_HOST will set the value for the api_host attribute in the services section.
Command-Line Configuration Files: Files passed via the -f or --config-file flag to the NOMAD CLI. If multiple files are given, they are merged in order, with later files overriding earlier ones. For example, to load an additional configuration file on top of a default configuration, you could do the following:
```
nomad admin run appworker -f nomad.yaml -f nomad-dev.yaml
```
Default nomad.yaml: A file named nomad.yaml in the current working directory, or a file pointed to by the NOMAD_CONFIG environment variable. This serves as the base configuration. This file is only automatically read if no file(s) have been specified using the command flag above.
Built-in Defaults: The default values hard-coded in the NOMAD source code. These have the lowest priority. These default values can be found from the nomad/config/defaults.yaml file in the source code.

Merging Rules¶

When configuration is loaded from multiple sources (e.g., a default file and an override file), the values are merged according to the following rules:

Objects (Dictionaries): When overwriting an object, the new value is recursively merged with the existing value. The final merged object will have all attributes from the new object, plus any attributes from the old object that were not overwritten. This allows you to change an individual setting deep in the configuration hierarchy without having to restate the entire structure.
Other Types (Lists, Strings, Numbers): When overwriting any other data type, such as a list, string, or number, the new value completely replaces the old one.

It is crucial to remember that lists are not appended or merged item-by-item. When you provide a new list in an override file, it will completely replace the original list.

For example, consider a default configuration that enables several plugins:

# In the base nomad.yaml
plugins:
  entry_points:
    include:
      - "systemnormalizer:system_normalizer_entry_point"
      - "atomisticparsers:amber_parser_entry_point"

If you provide an override file to run only the systemnormalizer:system_normalizer_entry_point for your nomad oasis:

# In override.yaml
plugins:
  entry_points:
    include:
      - "atomisticparsers:amber_parser_entry_point"

The final list of normalizers for that run will be ["atomisticparsers:amber_parser_entry_point"]. The systemnormalizer will be removed for that run because the entire include list was replaced.

If you intend to add an item to a list, you must repeat all the original items in your override file and add the new one.

Configuration examples¶

Many of the configuration options use a data model that contains the following three fields: include, exclude and options. This structure allows you to easily disable, enable, reorder and modify the configuration values with minimal config rewrite. Here are examples of common customization tasks:

Disable plugin entry point

plugins:
  entry_points:
    exclude:
      - <plugin-entry-point-id>

Explicitly select the list of plugins to use:

plugins:
  entry_points:
    include:
      - <plugin-entry-point-id-1>
      - <plugin-entry-point-id-2>

Modify plugin configuration

plugins:
  entry_points:
    options:
      <plugin-entry-point-id>:
        name: "Custom name"

Add a new item that does not yet exist in options. Note that by default all options are shown in the order they have been declared unless the order is explicitly given in include.

plugins:
  entry_points:
    options:
      <plugin-entry-point-id>:
        menus:
          options:
            my_menu: # This option does not exist yet, create it here
              title: "My Menu"
              ...

Configuration Reference¶

The following is a reference of all configuration sections and attributes.

Services¶

services¶

Contains basic configuration of the NOMAD services (app, worker, north).

name	type
api_host	`str`	The external hostname that clients can use to reach this NOMAD installation. default: `localhost`
api_port	`str \| int`	The port used to expose the NOMAD app and api to clients. default: `8000`
api_base_path	`str`	The base path prefix for the NOMAD app and api. default: `/nomad-oasis`
api_secret	`str`	A secret that is used to issue download and other tokens. default: `default-api-secret-that-is-long-enough`
mode	`str`	The mode to run NOMAD in, defaults to production mode. Affects the security features and you should only run NOMAD in development mode if the instance is not publicly available. default: `ModeEnum.PRODUCTION` options: - `production` - `development`
api_timeout	`int`	If the NOMAD app is run with gunicorn as process manager, this timeout (in s) is passed and worker processes will be restarted, if they do not respond in time. default: `600`
https	`bool`	Set to `True`, if external clients are using SSL to connect to this installation. Requires to setup a reverse-proxy (e.g. the one used in the docker-compose based installation) that handles the SSL encryption. default: `False`
https_upload	`bool`	Set to `True`, if upload curl commands should suggest the use of SSL for file uploads. This can be configured independently of `https` to suggest large file via regular HTTP. default: `False`
admin_user_id	`str`	The admin user `user_id`. All users are treated the same; there are no particular authorization information attached to user accounts. However, the API will grant the user with the given `user_id` more rights, e.g. using the `admin` owner setting in accessing data. default: `00000000-0000-0000-0000-000000000000`
encyclopedia_base	`str`	This enables links to the given encyclopedia installation in the UI. default: `https://nomad-lab.eu/prod/rae/encyclopedia/#`
optimade_enabled	`bool`	If true, the app will serve the optimade API. default: `True`
dcat_enabled	`bool`	If true the app will serve the DCAT API. default: `True`
h5grove_enabled	`bool`	If true the app will serve the h5grove API. default: `True`
console_log_level	`int \| str`	The log level that controls console logging for all NOMAD services (app, worker, north). The level is given in Python `logging` log level numbers. default: `30`
upload_limit	`int`	The maximum allowed unpublished uploads per user. If a user exceeds this amount, the user cannot add more uploads. default: `10`
force_raw_file_decoding	`bool`	By default, text raw-files are interpreted with utf-8 encoding. If this fails, the actual encoding is guessed. With this setting, we force to assume iso-8859-1 encoding, if a file is not decodable with utf-8. default: `False`
max_entry_download	`int`	There is an inherent limit in page-based pagination with Elasticsearch. If you increased this limit with your Elasticsearch, you can also adopt this setting accordingly, changing the maximum amount of entries that can be paginated with page-base pagination. Page-after-value-based pagination is independent and can be used without limitations. default: `50000`
max_entry_metadata_download	`int`	The maximum amount of entries metadata that can be downloaded. default: `100000`
unavailable_value	`str`	Value that is used in `results` section Enum fields (e.g. system type, spacegroup, etc.) to indicate that the value could not be determined. default: `unavailable`
app_token_max_expires_in	`int`	Maximum expiration time for an app token in seconds. Requests with a higher value will be declined. default: `2592000`
html_resource_http_max_age	`int`	Used for the max_age cache-control directive on statically served html, js, css resources. default: `60`
image_resource_http_max_age	`int`	Used for the max_age cache-control directive on statically served image resources. default: `2592000`
upload_members_group_search_enabled	`bool`	If true, the GUI will show a search for groups as upload members. default: `True`
log_api_queries	`bool`	If true, all queries to the /entries/query API endpoint will be logged. default: `True`

meta¶

Metadata about the deployment and how it is presented to clients.

name	type
version	`str`	The NOMAD version string. default: `1.4.1rc1`
commit	`str`	The source-code commit that this installation's NOMAD version is build from. default: ""
deployment	`str`	Human-friendly name of this nomad deployment. default: `devel`
service	`str`	Name for the service that is added to all logs. Depending on how NOMAD is installed, services get a name (app, worker, north) automatically. default: `unknown nomad service`
name	`str`	Web-site title for the NOMAD UI. default: `NOMAD` deprecated
description	`str`	Description of the NOMAD deployment. Shown at the GUI homepage. default: `This is the central NOMAD deployment hosted by
FAIRmat. This deployment hosts a wide range of
research data with primary focus on condensed-matter physics and the chemical
physics of solids. You can access all published data without an account. If you
want to provide your own data, please log in or register for an account.`
homepage	`str`	Provider homepage. default: `https://nomad-lab.eu` deprecated
source_url	`str`	URL of the NOMAD source-code repository. default: `https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR` deprecated
maintainer_email	`str`	Email of the NOMAD deployment maintainer. default: `[email protected]`
beta	`dict`	Additional data that describes how the deployment is labeled as a beta-version in the UI. default: Complex object, default value not displayed.
footer_links	`list[FooterLink]`	A list of links to be displayed in the footer. default: `[]`
deployment_url	`str \| None`	The NOMAD deployment's url. If not explicitly set, will default to the (api url) read from the configuration.
label	`str \| None`	An additional log-stash data key-value pair added to all logs. Can be used to differentiate deployments when analyzing logs.

FooterLink¶

A model for links to be displayed in the footer.

name	type
title	`str`	The title of the link. default: `PydanticUndefined`
url	`str`	The URL of the link. default: `PydanticUndefined`

oasis¶

Settings related to the configuration of a NOMAD Oasis deployment.

name	type
is_oasis	`bool`	Set to `True` to indicate that this deployment is a NOMAD Oasis. default: `False`
allowed_users	`list[str]`	A list of usernames or user account emails. These represent a white-list of allowed users. With this, users will need to login right-away and only the listed users might use this deployment. All API requests must have authentication information as well.
uses_central_user_management	`bool`	Set to True to use the central user-management. Typically the NOMAD backend is using the configured `keycloak` to access user data. With this, the backend will use the API of the central NOMAD (`central_nomad_deployment_url`) instead. default: `False`
central_nomad_deployment_url	`str`	The URL of the API of the NOMAD deployment that is considered the central NOMAD. default: `https://nomad-lab.eu/prod/v1/api`
terms_of_service_url	`str`	The URL of the terms of service. default: ""
require_authentication	`bool`	If True, authentication is required to access sensitive API endpoints. default: `False`

north¶

Settings related to the operation of the NOMAD remote tools hub service north.

name	type
hub_ip	`str`	This setting is forwarded to jupyterhub; refer to the jupyterhub documentation. default: `0.0.0.0`
hub_port	`int \| str`	The internal port that NOMAD services use to connect to the jupyterhub API. default: `9000`
windows	`bool`	Enable windows OS hacks. default: `True`
nomad_access_token_expiry_time	`int`	All tools are run with an access token for the NOMAD api in the NOMAD_CLIENT_ACCESS_TOKEN environment variable. This token will be automatically used by the nomad-lab Python package, e.g. if you use the ArchiveQuery to access data. This option sets the amount of seconds that this token is valid for. default: `86400`
tools	`NORTHTools`	The available north tools. Either the tools definitions as dict or a path to a .json file. default: Complex object, default value not displayed.
hub_service_api_token	`str`	A secret token shared between NOMAD and the NORTH jupyterhub. This needs to be the token of an admin service. default: `secret-token`
enabled	`bool \| None`	Enables or disables the NORTH API and UI views. This is independent of whether you run a jupyter hub or not. default: `True`
hub_connect_ip	`str \| None`	Overwrites the default hostname that can be used from within a north container to reach the host system. Typically has to be set for non Linux hosts. Set this to `host.docker.internal` on windows/macos.
hub_connect_url	`str \| None`	This setting is forwarded to jupyterhub; refer to the jupyterhub documentation.
docker_network	`str \| None`	This setting is forwarded to jupyterhub; refer to the jupyterhub documentation.
hub_host	`str \| None`	The internal host name that NOMAD services use to connect to the jupyterhub API. default: `localhost`
jupyterhub_crypt_key	`str \| None`	This setting is forwarded to jupyterhub; refer to the jupyterhub documentation.
nomad_host	`str \| None`	The NOMAD app host name that spawned containers use.

NORTHTools¶

name	type
options	`dict[str, NORTHTool]`	The available plugin. default: Complex object, default value not displayed.
include	`list[str] \| None`	List of included options. If not explicitly defined, all of the options will be included by default.
exclude	`list[str] \| None`	List of excluded options. Has higher precedence than include.

NORTHTool¶

name	type
image_pull_policy	`str`	The image pull policy used in k8s deployments. default: `Always`
privileged	`bool`	Whether the tool needs to run in privileged mode. default: `False`
seccomp_unconfined	`bool`	Whether the tool runs with seccomp=unconfined. default: `False`
use_gpu	`bool`	Whether the tool is provided access to available GPU resources. default: `False`
with_path	`bool`	Whether the tool supports a path to a file or directory. This also enables tools to be launched from files in the NOMAD UI. default: `False`
file_extensions	`list[str]`	The file extensions of files that this tool should be launchable for. default: `[]`
maintainer	`list[NORTHToolMaintainer]`	The maintainers of the tool. default: `[]`
external_mounts	`list[NORTHExternalMount]`	Additional mounts to be added to tool containers. default: `[]`
display_name	`str \| None`	A display name for the tool, e.g. shown in the NOMAD GUI.
short_description	`str \| None`	A short description of the tool, e.g. shown in the NOMAD GUI.
description	`str \| None`	A description of the tool, e.g. shown in the NOMAD GUI.
image	`str \| None`	The docker image (incl. tags) to use for the tool.
cmd	`str \| None`	The container cmd that is passed to the spawner.
default_url	`str \| None`	An optional path prefix that is added to the container URL to reach the tool, e.g. "/lab" for jupyterlab.
path_prefix	`str \| None`	An optional path prefix that is added to the container URL to reach the files, e.g. "lab/tree" for jupyterlab.
mount_path	`str \| None`	The path in the container where uploads and work directories will be mounted, e.g. /home/jovyan for Jupyter containers.
icon	`str \| None`	A URL to an icon that is used to represent the tool in the NOMAD UI.

NORTHExternalMount¶

name	type
host_path	`str`	default: `PydanticUndefined`
bind	`str`	default: `PydanticUndefined`
mode	`str`	default: `ReadMode.ro` options: - `ro` - `rw`

NORTHToolMaintainer¶

name	type
name	`str`	default: `PydanticUndefined`
email	`str`	default: `PydanticUndefined`

Files, databases, external services¶

fs¶

name	type
tmp	`str`	Internal temporary filesystem path used during processing. default: `.volumes/fs/tmp`
staging	`str`	Internal path used for staging uploads before they are processed. default: `.volumes/fs/staging`
public	`str`	Internal path where public files (e.g. published uploads) are stored. default: `.volumes/fs/public`
north_home	`str`	Internal base path for NORTH user home directories. default: `.volumes/fs/north/users`
local_tmp	`str`	Local temporary directory on the host system (outside of NOMAD volumes). default: `/tmp`
prefix_size	`int`	Number of characters from upload/entry IDs used as directory prefixes in storage. default: `2`
archive_version_suffix	`str \| list[str]`	This allows to add an additional segment to the names of archive files and thereby allows different NOMAD installations to work with the same storage directories and raw files, but with separate archives. If this is a list, the first string is used. If the file with the first string does not exist on read, the system will look for the file with the next string, etc. default: `['v1.2', 'v1']`
working_directory	`str`	Base working directory used to resolve relative filesystem paths. default: `PydanticUndefined`
staging_external	`str \| None`	External/absolute path to staging storage. If None, derived from working_directory.
public_external	`str \| None`	External/absolute path to the public storage. If None, derived from working_directory.
north_home_external	`str \| None`	External/absolute path for NORTH user home directories. If None, derived from working_directory.
north_home_user_folder_map	`dict[str, str] \| None`	This can be used to mount external folders with already existing user data for every user's work folder in the North tools. For example, if you already store user's data files under their own folders on the server, you can mount them with this into the user's launched North tool e.g. Jupyter notebook. The username is on the left hand side and the external folder path on disk on the right hand side. For example: north_home_user_folder_map: 'nomad username': '/path/on/disk/to/work/folder/specific/for/user' default: Complex object, default value not displayed.
external_working_directory	`str \| None`	Optional external working directory overriding working_directory for derived paths.

elastic¶

name	type
username	`str`	Username for authenticating with the Elasticsearch server. default: ""
password	`str`	Password for authenticating with the Elasticsearch server. default: ""
host	`str`	Hostname or IP address of the Elasticsearch server. default: `localhost`
port	`int`	Port on which the Elasticsearch server is listening. default: `9200`
timeout	`int`	Default request timeout (in seconds) for Elasticsearch operations. default: `60`
bulk_timeout	`int`	Timeout (in seconds) for bulk Elasticsearch operations. default: `600`
bulk_size	`int`	Number of documents per bulk indexing/request batch. default: `1000`
max_payload_size	`int`	Maximum payload size sent to the Elasticsearch server in bytes. Note that Elasticsearch has an internal limit of 100MB that you can configure as well. default: `94371840`
entries_per_material_cap	`int`	Maximum number of entries per material used when aggregating entry data to materials. default: `1000`
entries_index	`str`	Name of the Elasticsearch index storing entries. default: `nomad_entries_v1`
materials_index	`str`	Name of the Elasticsearch index storing materials. default: `nomad_materials_v1`

keycloak¶

name	type
server_url	`str`	Internal base URL of the Keycloak server used by NOMAD. default: `https://nomad-lab.eu/fairdi/keycloak/auth/`
realm_name	`str`	Keycloak realm name used for this NOMAD deployment. default: `fairdi_nomad_prod`
username	`str`	Administrative Keycloak username used for service-level operations. default: `admin`
password	`str`	Administrative Keycloak password used for service-level operations. default: `password`
client_id	`str`	Keycloak client ID used by the NOMAD backend/UI. default: `nomad_public`
public_server_url	`str \| None`	Publicly reachable Keycloak server URL. Defaults to server_url if not explicitly set.
client_secret	`str \| None`	Optional Keycloak client secret used for confidential clients.

mongo¶

Connection and usage settings for MongoDB.

name	type
host	`str`	The name of the host that runs mongodb. default: `localhost`
port	`int`	The port to connect with mongodb. default: `27017`
db_name	`str`	The used mongodb database name. default: `nomad_v1`
username	`str \| None`	Optional username for MongoDB authentication.
password	`str \| None`	Optional password for MongoDB authentication.

logstash¶

name	type
enabled	`bool`	If True, logs are forwarded to a Logstash instance. default: `False`
host	`str`	Hostname or IP address of the Logstash server. default: `localhost`
tcp_port	`str`	TCP port on which the Logstash server receives logs. default: `5000`
level	`int \| str`	Minimum log level for logs sent to Logstash. default: `10`

mail¶

name	type
enabled	`bool`	If True, NOMAD will attempt to send emails using the configured SMTP server. default: `False`
with_login	`bool`	If True, NOMAD will authenticate to the SMTP server using the given user and password. default: `False`
host	`str`	Hostname or IP address of the SMTP server. default: ""
port	`int`	Port of the SMTP server. default: `8995`
user	`str`	SMTP username used for authentication (if with_login is True). default: ""
password	`str`	SMTP password used for authentication (if with_login is True). default: ""
from_address	`str`	Email address used in the From header of outgoing emails. default: `[email protected]`
cc_address	`str \| None`	Optional email address to CC on outgoing emails.

datacite¶

name	type
mds_host	`str`	Base URL of the DataCite MDS (Metadata Store) service. default: `https://mds.datacite.org`
enabled	`bool`	If True, NOMAD will register DOIs via DataCite for published uploads/datasets. default: `False`
prefix	`str`	DataCite DOI prefix assigned to this NOMAD deployment. default: `10.17172`
user	`str`	DataCite MDS username. default: `*`
password	`str`	DataCite MDS password. default: `*`

rfc3161_timestamp¶

name	type
server	`str`	The rfc3161ng timestamping host. default: `http://zeitstempel.dfn.de`
cert	`str`	Path to the optional rfc3161ng timestamping server certificate.
hash_algorithm	`str`	Hash algorithm used by the rfc3161ng timestamping server. default: `sha256`
username	`str \| None`	Optional username for authenticating to the RFC3161 timestamping server.
password	`str \| None`	Optional password for authenticating to the RFC3161 timestamping server.

Processing¶

normalize¶

name	type
normalizers	`Options`	default: Complex object, default value not displayed.
system_classification_with_clusters_threshold	`int`	The system size limit for running the dimensionality analysis. For very large systems the dimensionality analysis will get too expensive. default: `64`
clustering_size_limit	`int`	The system size limit for running the system clustering. For very large systems the clustering will get too expensive. default: `1000`
symmetry_tolerance	`float`	Symmetry tolerance controls the precision used by spglib in order to find symmetries. The atoms are allowed to move this much from their symmetry positions in order for spglib to still detect symmetries. The unit is angstroms. The value of 0.1 is used e.g. by Materials Project according to https://pymatgen.org/pymatgen.symmetry.html#pymatgen.symmetry.analyzer.SpacegroupAnalyzer default: `0.1`
prototype_symmetry_tolerance	`float`	The symmetry tolerance used in aflow prototype matching. Should only be changed before re-running the prototype detection. default: `0.1`
max_2d_single_cell_size	`int`	Maximum number of atoms in the single cell of a 2D material for it to be considered valid. default: `7`
cluster_threshold	`float`	The distance tolerance between atoms for grouping them into the same cluster. Used in detecting system type. default: `2.5`
angle_rounding	`float`	Defines the "bin size" for rounding cell angles for the material hash in degree. default: `10.0`
flat_dim_threshold	`float`	The threshold for a system to be considered "flat". Used e.g. when determining if a 2D structure is purely 2-dimensional to allow extra rigid transformations that are improper in 3D but proper in 2D. default: `0.1`
k_space_precision	`float`	The threshold for point equality in k-space. Unit: 1/m. default: `150000000.0`
band_structure_energy_tolerance	`float`	The energy threshold for how much a band can be on top or below the fermi level in order to still detect a gap. Unit: Joule. default: `8.01088e-21`
springer_db_path	`str \| None`	default: `/opt/venv/lib/python3.12/site-packages/nomad/config/models/normalizing/data/springer.msg`

Options¶

Common configuration class used for enabling/disabling certain elements and defining the configuration of each element.

name	type
include	`list[str] \| None`	List of included options. If not explicitly defined, all of the options will be included by default.
exclude	`list[str] \| None`	List of excluded options. Has higher precedence than include.
options	`dict[str, Any] \| None`	Contains the available options. default: Complex object, default value not displayed.

process¶

name	type
index_materials	`bool`	If True, material-level indices are created/updated during processing. default: `False`
reuse_parser	`bool`	If True, parser instances may be reused between entries to improve performance. default: `True`
metadata_file_name	`str`	Base name (without extension) for per-upload metadata files. default: `nomad`
metadata_file_extensions	`tuple`	Allowed file extensions for per-upload metadata files. default: `('json', 'yaml', 'yml')`
auxfile_cutoff	`int`	Maximum number of auxiliary files considered when matching parsers. default: `100`
parser_matching_size	`int`	Heuristic size parameter used when selecting parsers based on file size. default: `12000`
max_upload_size	`int`	Maximum allowed upload size in bytes. default: `34359738368`
use_empty_parsers	`bool`	If True, allow parsers that produce empty archives to be used. default: `False`
redirect_stdouts	`bool`	True will redirect lines to stdout (e.g. print output) that occur during processing (e.g. created by parsers or normalizers) as log entries. default: `False`
rfc3161_skip_published	`bool`	If True, RFC3161 timestamping is skipped for already published uploads. default: `False`
exclude_potcar	`bool`	True will exclude original VASP POTCAR file from upload. default: `True`

reprocess¶

name	type
rematch_published	`bool`	If True, published uploads are rematched with parsers during reprocessing. default: `True`
reprocess_existing_entries	`bool`	If True, existing entries in the database are reprocessed. default: `True`
use_original_parser	`bool`	If True, the originally used parser is forced during reprocessing (if available). default: `False`
add_matched_entries_to_published	`bool`	If True, newly matched entries are added to the published set. default: `True`
delete_unmatched_published_entries	`bool`	If True, previously published entries that no longer match are deleted. default: `False`
index_individual_entries	`bool`	If True, individual entries are re-indexed during reprocessing. default: `False`

bundle_export¶

Controls behaviour related to exporting bundles.

name	type
default_cli_bundle_export_path	`str`	Default path used when exporting bundles using the CLI command. default: `./bundles`
default_settings	`BundleExportSettings`	General default settings. default: Complex object, default value not displayed.
default_settings_cli	`BundleExportSettings`	Additional default settings, applied when exporting using the CLI. This allows to override some of the settings specified in the general default settings above.

BundleExportSettings¶

name	type
include_raw_files	`bool`	If the raw files should be included in the export default: `True`
include_archive_files	`bool`	If the parsed archive files should be included in the export default: `True`
include_datasets	`bool`	If the datasets should be included in the export default: `True`

bundle_import¶

Controls behaviour related to importing bundles.

name	type
required_nomad_version	`str`	Minimum NOMAD version of bundles required for import. default: `1.1.2`
default_cli_bundle_import_path	`str`	Default path used when importing bundles using the CLI command. default: `./bundles`
allow_bundles_from_oasis	`bool`	If oasis admins can "push" bundles to this NOMAD deployment. default: `False`
allow_unpublished_bundles_from_oasis	`bool`	If oasis admins can "push" bundles of unpublished uploads. default: `False`
default_settings	`BundleImportSettings`	General default settings. default: Complex object, default value not displayed.
default_settings_cli	`BundleImportSettings`	Additional default settings, applied when importing using the CLI. This allows to override some of the settings specified in the general default settings above. default: Complex object, default value not displayed.

BundleImportSettings¶

name	type
include_raw_files	`bool`	If the raw files should be included in the import default: `True`
include_archive_files	`bool`	If the parsed archive files should be included in the import default: `True`
include_datasets	`bool`	If the datasets should be included in the import default: `True`
include_bundle_info	`bool`	If the bundle_info.json file should be kept (not necessary but may be nice to have. default: `True`
keep_original_timestamps	`bool`	If all timestamps (create time, publish time etc) should be imported from the bundle. default: `False`
set_from_oasis	`bool`	If the from_oasis flag and oasis_deployment_url should be set. default: `True`
delete_upload_on_fail	`bool`	If False, it is just removed from the ES index on failure. default: `False`
delete_bundle_on_fail	`bool`	Deletes the source bundle if the import fails. default: `True`
delete_bundle_on_success	`bool`	Deletes the source bundle if the import succeeds. default: `True`
delete_bundle_include_parent_folder	`bool`	When deleting the bundle, also include parent folder, if empty. default: `True`
trigger_processing	`bool`	If the upload should be processed when the import is done (not recommended). default: `False`
process_settings	`Reprocess`	When trigger_processing is set to True, these settings control the reprocessing behaviour (see the config for `reprocess` for more info). NOTE: reprocessing is no longer the recommended method to import bundles. default: Complex object, default value not displayed.

Reprocess¶

name	type
rematch_published	`bool`	If True, published uploads are rematched with parsers during reprocessing. default: `True`
reprocess_existing_entries	`bool`	If True, existing entries in the database are reprocessed. default: `True`
use_original_parser	`bool`	If True, the originally used parser is forced during reprocessing (if available). default: `False`
add_matched_entries_to_published	`bool`	If True, newly matched entries are added to the published set. default: `True`
delete_unmatched_published_entries	`bool`	If True, previously published entries that no longer match are deleted. default: `False`
index_individual_entries	`bool`	If True, individual entries are re-indexed during reprocessing. default: `False`

archive¶

name	type
block_size	`int`	In case of using blocked TOC, this is the size of each block. default: `1048576`
read_buffer_size	`int`	GPFS needs at least 256K to achieve decent performance. default: `1048576`
copy_chunk_size	`int`	The chunk size of every read of binary data. It is used to copy data from one file to another. A small value will result in more syscalls, a large value will result in higher peak memory usage. default: `16777216`
toc_depth	`int`	Depths of table of contents in the archive. default: `10`
small_obj_optimization_threshold	`int`	For any child of lists/dicts whose encoded size is smaller than this value, no TOC will be generated. default: `1048576`
fast_loading	`bool`	When enabled, this flag determines whether to read the whole dict/list at once when a certain mount of children has been visited. This reduces the number of syscalls although data may be repeatedly read. Otherwise, always read children one by one. This may slow down the loading as more syscalls are needed. default: `True`
fast_loading_threshold	`float`	If the fraction of children that have been visited is less than this threshold, fast loading will be used. default: `0.6`
trivial_size	`int`	To identify numerical lists. default: `20`

User Interface¶

These settings affect the behaviour of the user interface. Note that the preferred way for creating custom apps is by using app plugin entry points.

ui¶

Used to customize the user interface.

name	type
north	`NORTHUI`	NORTH (NOMAD Remote Tools Hub) UI configuration. default: Complex object, default value not displayed.
example_uploads	`ExampleUploads`	Controls the available example uploads. default: Complex object, default value not displayed.
app_base	`str \| None`	This is automatically set.
north_base	`str \| None`	This is automatically set.
theme	`Theme \| None`	Controls the site theme and identity.
unit_systems	`UnitSystems \| None`	Controls the available unit systems.
entry	`Entry \| None`	Controls the entry visualization.
apps	`Apps \| None`	deprecated

Theme¶

Theme and identity settings.

name	type
title	`str`	Site name in the browser tab. default: `PydanticUndefined`

Apps¶

Contains App definitions and controls their availability.

name	type
include	`list[str] \| None`	List of included options. If not explicitly defined, all of the options will be included by default.
exclude	`list[str] \| None`	List of excluded options. Has higher precedence than include.
options	`dict[str, App] \| None`	Contains the available app options.

ExampleUploads¶

Controls the availability of example uploads.

name	type
include	`list[str] \| None`	List of included options. If not explicitly defined, all of the options will be included by default.
exclude	`list[str] \| None`	List of excluded options. Has higher precedence than include.

Entry¶

Controls the entry visualization.

name	type
cards	`Cards`	Controls the cards that are displayed on the entry overview page. default: `PydanticUndefined`

Cards¶

Contains the overview page card definitions and controls their visibility.

name	type
include	`list[str] \| None`	List of included options. If not explicitly defined, all of the options will be included by default.
exclude	`list[str] \| None`	List of excluded options. Has higher precedence than include.
options	`dict[str, Card] \| None`	Contains the available card options.

Card¶

Definition for a card shown in the entry overview page.

name	type
error	`str`	The error message to show if an error is encountered within the card. default: `PydanticUndefined`

NORTHUI¶

NORTH (NOMAD Remote Tools Hub) UI configuration.

name	type
enabled	`bool`	Whether the NORTH tools are available in the UI. The default value is read from the root-level NORTH configuration. default: `True`

UnitSystems¶

Controls the available unit systems.

name	type
selected	`str`	Selected option. default: `PydanticUndefined`
include	`list[str] \| None`	List of included options. If not explicitly defined, all of the options will be included by default.
exclude	`list[str] \| None`	List of excluded options. Has higher precedence than include.
options	`dict[str, UnitSystem] \| None`	Contains the available unit systems.

UnitSystem¶

name	type
label	`str`	Short, descriptive label used for this unit system. default: `PydanticUndefined`
units	`dict[str, UnitSystemUnit] \| None`	Contains a mapping from each dimension to a unit. If a unit is not specified for a dimension, the SI equivalent will be used by default. The following dimensions are available: - dimensionless - length - mass - time - current - temperature - luminosity - luminous_flux - substance - angle - information - force - energy - power - pressure - charge - solid_angle - resistance - conductance - inductance - magnetic_flux - magnetic_field - frequency - luminance - illuminance - electric_potential - capacitance - activity

UnitSystemUnit¶

name	type
definition	`str`	The unit definition. Can be a mathematical expression that combines several units, e.g. `(kg * m) / s^2`. You should only use units that are registered in the NOMAD unit registry (`nomad.units.ureg`). default: `PydanticUndefined`
locked	`bool \| None`	Whether the unit is locked in the unit system it is defined in. default: `False`

Others¶

temporal¶

name	type
host	`str`	Hostname or IP address of the Temporal server. default: `localhost`
port	`int`	Port on which the Temporal server is listening. default: `7233`
namespace	`str`	Temporal namespace used for NOMAD workflows. default: `default`
enabled	`bool`	If False, Temporal-based processing is disabled and alternative mechanisms may be used. default: `True`
graceful_shutdown_timeout	`int`	Graceful shutdown timeout (in seconds) for Temporal workers. default: `1`
processing_timeouts	`ProcessingTimeouts`	Timeout configuration for individual processing workflows and activities. default: `PydanticUndefined`
prometheus_bind_address	`str \| None`	The bind address for the Prometheus metrics server. If not set, the runtime will not be configured with Prometheus metrics.

ProcessingTimeouts¶

name	type
internal_processing_heartbeat_timeout	`int`	The maximum interval in seconds that an internal processing activity (e.g. entry processing) can run without reporting progress (sending a heartbeat) before Temporal considers it failed. default: `600`
delete_upload_timeout	`int`	The timeout for activities in the delete upload workflow in seconds. default: `3600`
process_entry_timeout	`int`	The timeout for the process entry activity in seconds. default: `7200`
process_upload_timeout	`int`	The timeout for activities in the process upload workflow in seconds. default: `3600`
edit_upload_metadata_timeout	`int`	The timeout for activities in the edit upload metadata workflow in seconds. default: `3600`
import_bundle_timeout	`int`	The timeout for activities in the import bundle workflow in seconds. default: `3600`
publish_upload_timeout	`int`	The timeout for activities in the publish upload workflow in seconds. default: `3600`
publish_externally_timeout	`int`	The timeout for activities in the publish externally workflow in seconds. default: `3600`
process_example_upload_timeout	`int`	The timeout for activities in the process example upload workflow in seconds. default: `3600`
setup_upload_timeout	`int`	The timeout for the setup upload activity in seconds. default: `300`
update_files_timeout	`int`	The timeout for the update files activity in seconds. default: `3600`
match_all_timeout	`int`	The timeout for the match all activity in seconds. default: `3600`
next_level_entries_timeout	`int`	The timeout for the next level entries activity in seconds. default: `3600`
cleanup_timeout	`int`	The timeout for the cleanup activity in seconds. default: `3600`
process_upload_success_timeout	`int`	The timeout for the process upload success activity in seconds. default: `300`
remove_workflow_id_timeout	`int`	The timeout for the remove workflow id activity in seconds. default: `300`
cleanup_workflow_tmp_dir_timeout	`int`	The timeout for the cleanup workflow tmp dir activity in seconds. default: `300`

logtransfer¶

Configuration of logtransfer and statistics service.

When enabled (enabled) an additional logger will write logs to a log file (log_file). At regular intervals (transfer_interval) a celery task is scheduled. It will log a set of statistics. It will copy the log file (transfer_log_files). Transfer the contents of the copy to the central NOMAD (oasis.central_nomad_deployment_url) and delete the copy. The transfer is only done if the the log file has a certain size (transfer_threshold). Only a maximum amount of logs are transferred (transfer_capacity). Only logs with a certain level (level) are considered. The files will be stored in fs.tmp.

name	type
enabled	`bool`	If enabled this starts process that frequently generates logs with statistics. default: `False`
transfer_threshold	`int`	The minimum size in bytes of stored logs before logs are transferred. 0 means transfer at every transfer interval. default: `0`
transfer_capacity	`int`	The maximum number of bytes of stored logs that are transferred. Excess is dropped. default: `1000000`
transfer_interval	`int`	Time interval in seconds after which stored logs are potentially transferred. default: `600`
level	`int \| str`	The min log level for logs to be transferred. default: `20`
log_file	`str`	The log file that is used to store logs for transfer. default: `nomad.log`
transfer_log_file	`str`	The log file that is used to copy logs for transfer. default: `.transfer.log`
file_rollover_wait_time	`float`	Time in seconds to wait after log file was "rolled over" for transfer. default: `1`

tests¶

name	type
default_timeout	`int`	Default timeout (in seconds) used in test utilities and helpers. default: `60`
assume_auth_for_username	`str \| None`	Will assume that all API calls with no authentication have authentication for the user with the given username.

client¶

name	type
url	`str`	Base URL of the NOMAD API used by the Python client. default: `https://nomad-lab.eu/prod/v1/api`
user	`str \| None`	Optional username used by the Python NOMAD client for authentication.
password	`str \| None`	Optional password used by the Python NOMAD client for authentication.
access_token	`str \| None`	Optional bearer access token used by the Python NOMAD client.

gitlab¶

name	type
private_token	`str`	Private token used for accessing the GitLab API (e.g. for CI integrations). default: `not set`

plugins¶

Represent a PEP 604 union type

E.g. for int | str

name	type
#### plugins
name	type
----	----	-
entry_points	`EntryPoints`	Used to control plugin entry points. default: `PydanticUndefined`
plugin_packages	`dict[str, PluginPackage]`	Contains the installed installed plugin packages with the package name used as a key. This is autogenerated and should not be modified. default: `PydanticUndefined`

EntryPoints¶

name	type
options	`dict[str, SchemaPackageEntryPoint \| ParserEntryPoint \| NormalizerEntryPoint \| AppEntryPoint \| ExampleUploadEntryPoint \| APIEntryPoint \| ActionEntryPoint \| NorthToolEntryPoint]`	The available plugin entry points. default: Complex object, default value not displayed.
include	`list[str] \| None`	List of included options. If not explicitly defined, all of the options will be included by default.
exclude	`list[str] \| None`	List of excluded options. Has higher precedence than include.

SchemaPackageEntryPoint¶

Base model for schema package plugin entry points.

name	type
id_url_safe	`str \| None`	URL-safe identifier for the plugin entry point. If no custom name is provided, one is automatically generated from the entry point id. Any custom identifier is checked for URL-safety and for collisions with other entry points with the same type within the same NOMAD deployment.
name	`str \| None`	Name of the plugin entry point.
description	`str \| None`	A human readable description of the plugin entry point.

ParserEntryPoint¶

Base model for parser plugin entry points.

name	type
level	`int`	Integer that determines the execution order of this parser within an upload. Parser with lowest level will be executed first. Note that this only controls the order in which matched parsers are executed, but does not affect the order in which parsers are matched to files. default: `0`
aliases	`list[str]`	List of alternative parser names. default: `[]`
mainfile_name_re	`str`	A regular expression that is applied the name of a potential mainfile. If this expression is given, the parser is only considered for a file, if the expression matches. default: `.*`
mainfile_mime_re	`str`	A regular expression that is applied the mime type of a potential mainfile. If this expression is given, the parser is only considered for a file, if the expression matches. default: `.*`
mainfile_alternative	`bool`	If True, the parser only matches a file, if no other file in the same directory matches a parser. default: `False`
supported_compressions	`list[str]`	Files compressed with the given formats (e.g. xz, gz) are uncompressed and matched like normal files. default: `[]`
id_url_safe	`str \| None`	URL-safe identifier for the plugin entry point. If no custom name is provided, one is automatically generated from the entry point id. Any custom identifier is checked for URL-safety and for collisions with other entry points with the same type within the same NOMAD deployment.
name	`str \| None`	Name of the plugin entry point.
description	`str \| None`	A human readable description of the plugin entry point.
mainfile_contents_re	`str \| None`	A regular expression that is applied the content of a potential mainfile. If this expression is given, the parser is only considered for a file, if the expression matches.
mainfile_binary_header	`bytes \| None`	Matches a binary file if the given bytes are included in the file.
mainfile_binary_header_re	`bytes \| None`	Matches a binary file if the given binary regular expression bytes matches the file contents.
mainfile_contents_dict	`dict \| None`	Used to check the contents of structured data files like JSON, HDF5 or csv/excel files. Parser will match the file if the dictionary provided matches the contents of the file. One can also use the following reserved keys for special queries: - `__has_key: str` Plain string or regular expression of a key in the data to match - `__has_all_keys: list[str]` List of keys that must be present in the data to match - `__has_only_keys: list[str]` List of keys that must exclusively be present in the data - `__has_comment: str` (only for csv/xlsx files) Comments that are ignored, must be at the top level of the dictionary In case of a csv/excel file for example, one can set this attribute to `{'__has_all_keys': [<column names>]}` in order to check if certain columns exist in a given sheet. Also in order to check if a certain sheet name with specific column names exist, one may set this attribute to: `{'<sheet name>': {'__has_all_keys': [<column names>]}}`.

NormalizerEntryPoint¶

Base model for normalizer plugin entry points.

name	type
level	`int`	Integer that determines the execution order of this normalizer within the processing of an individual entry. Normalizers with the lowest level is run first. default: `0`
id_url_safe	`str \| None`	URL-safe identifier for the plugin entry point. If no custom name is provided, one is automatically generated from the entry point id. Any custom identifier is checked for URL-safety and for collisions with other entry points with the same type within the same NOMAD deployment.
name	`str \| None`	Name of the plugin entry point.
description	`str \| None`	A human readable description of the plugin entry point.

ExampleUploadEntryPoint¶

Base model for example upload plugin entry points.

name	type
id_url_safe	`str \| None`	URL-safe identifier for the plugin entry point. If no custom name is provided, one is automatically generated from the entry point id. Any custom identifier is checked for URL-safety and for collisions with other entry points with the same type within the same NOMAD deployment.
name	`str \| None`	Name of the plugin entry point.
description	`str \| None`	Longer description of the example upload. default: `PydanticUndefined`
category	`str \| None`	Category for the example upload. default: `PydanticUndefined`
title	`str \| None`	Title of the example upload. default: `PydanticUndefined`
resources	`None \| list[UploadResource \| str] \| UploadResource \| str`	List of data resources for this example upload.
path	`str \| None`	deprecated
url	`str \| None`	deprecated

UploadResource¶

Represents a request to include a certain resource into an example upload. Can point to a local folder/file, or alternatively to an online resource that can be downloaded.

name	type
path	`str`	Path to a file/folder within the python package (filepaths should start from the package root directory) or to an online URL. default: `PydanticUndefined`
target	`str`	File path within the upload where the file should be stored. default: ""

ActionEntryPoint¶

Base model for action plugin entry points.

name	type
task_queue	`str`	Determines the task queue for this action default: `TaskQueue.CPU`
id_url_safe	`str \| None`	URL-safe identifier for the plugin entry point. If no custom name is provided, one is automatically generated from the entry point id. Any custom identifier is checked for URL-safety and for collisions with other entry points with the same type within the same NOMAD deployment.
name	`str \| None`	Name of the plugin entry point.
description	`str \| None`	A human readable description of the plugin entry point.

AppEntryPoint¶

Base model for app plugin entry points.

name	type
app	`App`	The app configuration. default: `PydanticUndefined`
id_url_safe	`str \| None`	URL-safe identifier for the plugin entry point. If no custom name is provided, one is automatically generated from the entry point id. Any custom identifier is checked for URL-safety and for collisions with other entry points with the same type within the same NOMAD deployment.
name	`str \| None`	Name of the plugin entry point.
description	`str \| None`	A human readable description of the plugin entry point.

APIEntryPoint¶

Base model for API plugin entry points.

name	type
id_url_safe	`str \| None`	URL-safe identifier for the plugin entry point. If no custom name is provided, one is automatically generated from the entry point id. Any custom identifier is checked for URL-safety and for collisions with other entry points with the same type within the same NOMAD deployment.
name	`str \| None`	Name of the plugin entry point.
description	`str \| None`	A human readable description of the plugin entry point.
prefix	`str \| None`	The prefix for the API. The URL for the API will be the base URL of the NOMAD installation followed by this prefix. The prefix must not collide with any other API prefixes. There is no default, this field must be set.

NorthToolEntryPoint¶

Base model for NORTH tool plugin entry points.

name	type
north_tool	`NORTHTool`	default: `PydanticUndefined`
id_url_safe	`str \| None`	URL-safe identifier for the plugin entry point. If no custom name is provided, one is automatically generated from the entry point id. Any custom identifier is checked for URL-safety and for collisions with other entry points with the same type within the same NOMAD deployment.
name	`str \| None`	Name of the plugin entry point.
description	`str \| None`	A human readable description of the plugin entry point.

PluginPackage¶

name	type
name	`str`	Name of the plugin Python package, read from pyproject.toml. default: `PydanticUndefined`
entry_points	`list[str]`	List of entry point ids contained in this package, read form pyproject.toml default: `PydanticUndefined`
description	`str \| None`	Package description, read from pyproject.toml.
version	`str \| None`	Plugin package version, read from pyproject.toml.
homepage	`str \| None`	Link to the plugin package homepage, read from pyproject.toml.
documentation	`str \| None`	Link to the plugin package documentation page, read from pyproject.toml.
repository	`str \| None`	Link to the plugin package source code repository, read from pyproject.toml.

uploads¶

name	type
pagination	`Pagination`	default: Complex object, default value not displayed.
entries	`Entries`	default: Complex object, default value not displayed.

Entries¶

name	type
pagination	`Pagination`	default: Complex object, default value not displayed.