Python Tutorial
Python Tutorial
Release 2.0
Anaconda Inc.
1 Anaconda Enterprise 1
2 Anaconda Distribution 3
3 Anaconda Cloud 5
3.1 Anaconda Enterprise 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3.2 Anaconda Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 935
3.3 Anaconda Cloud . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1406
3.4 Archive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1447
Index 1691
i
ii
CHAPTER 1
Anaconda Enterprise
Anaconda Enterprise is an enterprise-ready, secure and scalable data science platform that empowers teams to govern
data science assets, collaborate and deploy data science projects.
Enterprise 5 includes these capabilities:
• Easily deploy your projects into interactive data applications, live notebooks and machine learning models with
APIs.
• Share those applications with colleagues and collaborators.
• Manage your data science assets: notebooks, packages, environments and projects in an integrated data science
experience.
View Anaconda Enterprise 5 documentation.
1
Anaconda Documentation, Release 2.0
Anaconda Distribution
Anaconda Distribution is a free, easy-to-install package manager, environment manager and Python distribution with
a collection of 1,000+ open source packages with free community support. Anaconda is platform-agnostic, so you can
use it whether you are on Windows, macOS or Linux.
View Anaconda Distribution 5 documentation.
3
Anaconda Documentation, Release 2.0
Anaconda Cloud
Anaconda Cloud is a package management service that makes it easy to find, access, store and share public notebooks,
and environments, as well as conda and PyPI packages.
View Anaconda Cloud documentation.
Anaconda Enterprise 4 includes Anaconda Distribution, Anaconda Enterprise 4 Repository and Anaconda Enterprise
4 Notebooks. This is our previous-generation product, and documentation is provided for our current customers.
Anaconda Enterprise 4 Repository is an enterprise server on your network or your private cloud where open source
and proprietary packages may be stored, retrieved and shared.
Anaconda Enterprise 4 Notebooks is a browser-based Python data analysis environment and visualization tool in a
secure, governed environment.
Please refer to Anaconda Enterprise 5 documentation for our current-generation product.
5
Anaconda Documentation, Release 2.0
User guide
Getting started
You do not need to have an Anaconda Repository account or be logged in to search for, download, or install packages,
notebooks, environments or installers.
You do need an account to access private packages without an access token and to upload and share your own packages,
notebooks, environments and installers with others.
1. In the top Search box, type part or all of the name of a file you are searching for, and then press Enter.
2. Packages that match your search string are displayed. To see more information, click the package name.
• Access: All, Public and/or Private—available only if you are logged in.
• Platform: All, Source, Linux-32, Linux-64, Noarch, OSX-64, Win-32 and Win-64.
NOTE: Source packages are source code only, not yet built for any specific platform. Noarch packages are built to
work on all platforms.
You can download and install packages using Anaconda Navigator, the graphical user interface for Anaconda®.
Advanced users may prefer a Terminal window or an Anaconda Prompt.
Using Navigator
NOTE: If your organization does not use HTTPS, use http in the domain box and clear the Enable SSL
verification checkbox.
NOTE: If your organization uses subdomains, enter the address as https://api.
<your-anaconda-repo>:8080. Replace <your-anaconda-repo> with the name of your local
Repository. If that does not work, contact your system administrator.
3. Click the Apply button.
3. Sign Navigator into your local Anaconda Cloud—Repository— so you can search for packages marked as
private:
1. Click the top right Sign in to Anaconda Cloud button.
2. Type your Repository username and password:
Building and uploading new packages is optional, and best suited for advanced users who are comfortable using a
Terminal application. It requires the anaconda-client, which is easy to get if you have installed Anaconda.
Use Terminal window or Anaconda Prompt to run the following command line commands.
1. To build and upload packages, first install the Anaconda Client CLI:
conda install anaconda-client
4. To build your test package, first install conda-build and turn off automatic Client uploading, then run the conda
build command:
conda install conda-build
conda config --set anaconda_upload no
conda build .
5. Find the path where the newly-built package was placed, so that you can use it in the next step:
conda build . --output
NOTE: Replace /your/path/ with the path you found in the previous step.
For more information, see Working with conda packages.
Sharing notebooks
To upload a notebook to Anaconda Repository with anaconda-client, open Anaconda Prompt or Terminal and then
enter:
Viewing notebooks
You can view an HTML version of your notebook in Anaconda Repository. Log into your account, then from the
drop-down menu of the view button, select Notebooks. Click the name of the notebook you want to view.
You can also view an HTML version of your notebook directly from:
http://<your-anaconda-repo>/USERNAME/my-notebook
NOTE: Replace <your-anaconda-repo> with your Repository name, USERNAME with your username and
my-notebook with the name of your notebook.
Anyone who has anaconda-client and access to Repository can download your notebook. To download the notebook,
open Anaconda Prompt or Terminal and enter:
Sharing environments
A saved conda environment can be uploaded to Anaconda Repository with the web interface or the anaconda
upload command.
To save the environment, run this command in an Anaconda Prompt or Terminal window:
https://<your-anaconda-repo>/<USERNAME>/environments
http://envs.<your-anaconda-repo>/USERNAME
NOTE: Replace <your-anaconda-repo> with the name of your local Repository and USERNAME with your
username.
Anyone who has access can download and install your environment. Open Anaconda Prompt or Terminal and then
enter:
NOTE: Replace user with your username and my-environment with the name of your environment.
How to. . .
• Use packages
• Use the Anaconda Client CLI
• Build packages
Use packages
Find a package
From Anaconda Repository, you can search for packages by package name. From the top navigation bar of any page,
in the search box enter the package name. You can filter your searches using type of the packages, access or labels and
you can sort results by number of favorites or number of downloads by clicking the search results column heading.
NOTE: Replace USERNAME with your username and PACKAGE with the name of the desired package.
Install Client
Your credentials for Client are those that you used to create an account on Repository.
To get help:
1. In a browser, navigate to your Repository.
2. Select the Sign In tab.
3. Click either the I forgot my password link or the I forgot my username link.
After you have downloaded and configured Client, in a Terminal window or an Anaconda Prompt, run:
anaconda login
anaconda --help
anaconda COMMANDNAME -h
NOTE: Replace COMMANDNAME with the name of the command about which you want more information.
If you have a question that you cannot answer using the help command or documentation, contact your system admin-
istrator who has access to Anaconda Enterprise Support.
Build packages
anaconda upload -h
Tutorials
Anaconda Repository labels can be used to facilitate a development cycle and organize the code that is in development,
in testing and in production, without affecting non-development users.
In this tutorial, we show how to use a “test” label, so that you can upload files without affecting your production-quality
packages. Without a --label argument the default label is “main.”
1. You need to begin with a conda package. If you do not have one, use our example conda package. Before you
build the package, edit the version in the meta.yaml file in anaconda-client/example-packages/
conda/ to be 2.0:
2. Upload your test package to Repository using the Client upload command. Adding the --label option tells
Repository to make the upload visible only to users who specify that label:
NOTE: Replace /path/to/ with the path to where you stored the package.
3. You now can see that even when you search conda “main,” you do not see the 2.0 version of the test package.
This is because you need to tell conda to look for your new “test” label.
4. The --override argument tells conda not to use any channels in your ~/.condarc file.
The following command produces no 2.0 results:
Replace OLD with the old label, NEW with the new label, and SPEC with the package to move. SPEC can be either
“user/package/version/file”, or “user/package/version” in which case it moves all files in that version.
In addition to uploading or downloading packages, you can also upload or download other file types to/from Anaconda
Repository.
You can upload any type of file with Anaconda Client command line interface (CLI) by using the steps below.
PyPI package files, conda package files and notebook files are automatically detected. There is no auto-detect for other
types of files, so you must explicitly specify the package, package-type and version fields.
In the following example, we upload a spreadsheet named baby-names in comma separated value (CSV) format.
1. Create a new package, which creates a namespace that can hold multiple files:
anaconda login
anaconda package --create jsmith/baby-names
https://<your-anaconda-repo>/USERNAME/PACKAGE
NOTE: Replace <your-anaconda-repo> with the name of your local Repository, USERNAME with the desired
username and PACKAGE with the desired package name.
If the repository has multiple files with the same name and different extensions, anaconda download will down-
load all of them by default. If you use anaconda-client 1.7 or higher, you can use anaconda download with
the option --package-type or -t to specify only one of these files. This option can work with the values pypi,
conda, ipynb, and env.
Tasks
This guide covers all the everyday tasks for a user of Anaconda Repository.
Creating an account
The information below applies to personal Anaconda Repository accounts. For information on organization accounts,
see Working with organizations.
You do not need an Anaconda Repository account to find, download and use packages.
However, you do need a Repository account to:
• Author packages.
• Upload packages, notebooks and environments.
• Access private packages that are shared with you.
• Create organizations.
If your administrator sent you an email with a login address, username and password, use them.
Otherwise, to sign up for an Anaconda Repository account:
1. In a browser, go to the address your administrator gave you.
In the top navigation bar, the currently active user or organization is shown at the far right.
TIP: If the email address on your account is associated with a Gravatar account, Repository displays your profile photo.
To associate your email address with Gravatar or to change your Gravatar profile photo, see gravatar.com.
Packages, notebooks, environments, projects and installers that you have created with this account appear on your
Landscape.
Click the view button to see the following options:
• Landscape: Your home page.
• Favorites: Other users’ packages that you have starred.
• Packages: Only packages you have created.
• Notebooks: Only notebooks you have created.
You can use the Anaconda Client command line interface (CLI) in an Anaconda Prompt or Terminal window to:
• Connect to and manage your Anaconda Repository account.
• Upload packages you have created.
• Generate access tokens to allow access to private packages.
NOTE: Anaconda Client is not necessary to search for and download packages.
Anaconda Client requires conda to be installed. If you have Anaconda, conda is already installed.
To install with conda, open Anaconda Prompt or Terminal window and enter:
After installing, view the complete list of Client tasks with this command from Anaconda Prompt or Terminal window:
anaconda -h
• Creating an organization
• Uploading packages to an organization
• Transferring packages to an organization
• Customizing users and groups
• Creating groups for differing access levels
• Deleting an organization
Creating an organization
NOTE: Replace ORGANIZATION with the name of the organization, and package.tar.bz2 with the name of the
package.
Only the co-owners of an organization may customize users and groups of the organization.
To add, remove, or edit group and user access for an organization you administer:
From the top right drop-down menu, select Groups, then click the name of the group you want to edit.
In the Members box, type the username of the user you want to add as a member, thenc lick the Add button.
To remove a member, in the list on the Members page, click the delete icon (trash can).
Users receive a dashboard notification when you add them to an organization.
Within an organization, you can create a group to customize access for a group of users:
1. From the top right drop-down menu, select Groups, then click the + New Group button.
2. Give the group a name and click the Create group button.
3. In the Members box, add the desired members by username.
4. Add installers, packages, notebooks, projects or environments that this group can access.
5. Click the Save Group button.
Deleting an organization
To delete an organization you administer and erase all data associated with it:
1. At the top right of the Repository interface, in the Profile list, select Switch To.
2. Select the organization you want to delete.
3. In the Profile list, select Settings.
4. Select the Account option. You may be asked to verify your password.
5. In the Delete Account section, click the Delete button.
A confirmation page requests that you provide the full name of the organization.
All files uploaded to Anaconda Repository are stored in packages. Each Repository package is visible at its own
unique URL based on the name of the user who owns the package and the name of the package. You can create a
Repository package and then upload files into it.
Each user and organization has their own location called a namespace where they may host packages.
A label is part of the URLs for Repository where conda looks for packages. Each file within a package may be tagged
with one or more labels, or not tagged at all to accept the default label of main. Labels are searched only if you
specify a label.
Repository supports two package managers, conda and PyPI. To work with conda or PyPI packages, you must use
their corresponding subdomains.
EXAMPLE: To install conda packages from the user “travis,” use the Repository URL:
https://conda.<your-anaconda-repo>/travis
EXAMPLE: To install PyPI packages from the user “travis,” use the Repository URL:
https://pypi.<your-anaconda-repo>/travis
2. Choose the repository for which you would like to build the package. In this example, we use a simple, public
conda test package:
In this directory, there are two required files, build.sh, and meta.yaml.
NOTE: Linux and macOS systems are Unix systems. Packages built for Unix systems require a build.sh file,
packages built for Windows require a bld.bat file, and packages built for both Unix and Windows systems
require both a build.sh file and a bld.bat file. All packages require a meta.yaml file.
3. To build the package, turn off automatic Client uploading and then run the conda build command:
All packages built in this way are placed in a subdirectory of Anaconda’s conda-bld directory.
4. You can check where the resulting file was placed with the --output option:
For more information on conda’s overall build framework, you may also want to read the articles Building conda
packages and Tutorials on conda build.
Upload the test package to Repository with the anaconda upload command:
anaconda login
anaconda upload /path/to/conda-package.tar.bz2
NOTE: Replace /path/to/ with the path to where you stored the package.
You can install conda packages from Repository by adding channels to your conda configuration.
1. Because conda knows how to interact with Repository, specifying the channel “sean” translates to https://
<your-anaconda-repo>/sean:
2. You can now install public conda packages from Sean’s Repository account. Try installing the testci package at
https://<your-anaconda-repo>/sean/testci:
You can also install a package from a channel with a token and a label:
NOTE: Replace <token> with the provided token,‘‘<channel>‘‘ with the user channel, <labelname> with the
label name and <package> with the package name you want to install.
You can test PyPI package uploading with a small, public example package saved in the anaconda-client repository:
1. Begin by cloning the repository from the command line:
2. You can now create your PyPI package with the setup.py script:
3. Your package now is built as a source “tarball” and is ready to be uploaded with:
http://<your-anaconda-repo>/USERNAME/PACKAGE
NOTE: Replace <your-anaconda-repo> with the name of your local Repository, USERNAME with your user-
name and PACKAGE with the package name.
The best way to install a PyPI package is using pip. For the following command, you can use the package you authored
in the above steps:
NOTE: Replace <your-anaconda-repo> with the name of your local Repository, USERNAME with your user-
name and PACKAGE with the test-package name.
As of Anaconda Repository version 2.6.0, your Repository supports conda “noarch” packages that contain no operating
system-specific files.
The conda build system allows you to specify “no architecture” when building a package, so it is compatible with all
platforms and architectures. Noarch packages from your Repository instance can be downloaded and installed on any
platform.
NOTE: Noarch packages are not compatible with Anaconda constructor. If you intend to use the packages with
Anaconda constructor, build the packages for specific operating systems.
To specify a noarch build, use the noarch key in the build section of your conda recipe’s meta.yaml file:
build:
noarch: generic
You can upload noarch packages to Repository in the same manner as any other package:
Uploading a package
To upload a package to Repository, using the Client CLI, run the upload command:
anaconda login
anaconda upload PACKAGE
https://<your-anaconda-repo>/USERNAME/PACKAGE
NOTE: <your-anaconda-repo> is the name of your local Repository, USERNAME is your username and
PACKAGE is the package name.
Anyone can download your package by using Client:
By default, all packages, notebooks and environments uploaded to Repository are public, meaning they are accessible
to anyone who has access to Repository.
When you make a package private, only you and the users you authorize can access it.
You can authorize users to access your private package in two ways:
• Use a group inside an organization account—only group members who are logged in can access the package.
This is the best way to control access to your private packages because it allows you to set separate permissions
for each package, notebook or environment.
• Use a token control system—only users who have the appropriate token can access the private package or chan-
nel.
After you grant other users access, they can download and install your package using the Web UI or Client.
https://<your-anaconda-repo>/USERNAME/PACKAGE/settings/admin
Replace <your-anaconda-repo> with the name of your local Repository, USERNAME with your username
and PACKAGE with the name of the package.
5. Click Set access, then select Private.
NOTE: You can use the same procedure and URL to make Jupyter Notebooks and conda environments private.
1. Create an organization.
2. Upload or transfer the package to the organization.
3. Within the organization, create a group with the appropriate users, permissions, and packages.
You can control access to private packages and channels with the token system. All Repository URLs can be prefixed
with /t/<token> to allow access.
The degree of access a token grants is completely configurable when you generate it. You can generate multiple tokens
to control which groups of users have access to certain features if they have the appropriate token.
Tokens provide access to all packages in a specified channel. Separate permissions per package, notebook or environ-
ment may be better handled with organizations and groups.
You can generate tokens using the Web UI or Anaconda Client.
NOTE: By default, tokens expire after one year.
1. Navigate to:
https://<your-anaconda-repo>/<channel>/settings/access
NOTE: Replace <your-anaconda-repo> with the name of your local Repository, and <channel> with
the name of the desired channel.
2. In the Token Name box, type a name for the token:
3. Select the appropriate checkboxes for the type of access you want to allow for users of this token.
EXAMPLE: To allow users to download private packages or packages from private channels, select Allow
private downloads from Conda repositories.
4. Click the Create button.
NOTE: Replace <token> with your token string,‘‘<channel>‘‘ with the desired channel name, and
<labelname> with the label name.
NOTE: If you lose the token’s random alphanumeric string, you must revoke the token and create a new one.
Using a token
NOTE: Replace <your-anaconda-repo> with the name of your local Repository, <token> with the
provided token and <channel> with a user channel.
• Install a private package without first adding a channel:
NOTE: Replace <your-anaconda-repo> with the name of your local Repository, <token> with the
provided token, <channel> with a user channel, <labelname> with the label name and <package> with
the name of the package to install.
• Install a private PyPI package:
NOTE: Replace <your-anaconda-repo> with the name of your local Repository, <token> with the
provided token, <channel> with a user channel and PACKAGE with the name of the desired package.
NOTE: Private PyPI packages can also be installed using:
https://pypi.<your-anaconda-repo>/t/<token>/<channel>
Revoking a token
NOTE: Replace YOUR-TOKEN-NAME with the name of the token you want to revoke.
To download a package using the Web UI, in a web browser, navigate to the organization’s or user’s channel.
To download a package using Client:
• Run:
NOTE: Replace OrgName with the organization or username and PACKAGE with the package name.
• Or run:
NOTE: Replace <your-anaconda-repo> with the name of your local Repository, OrgName with the
organization name or username and PACKAGE with the package name.
When you create or add a package, by default it is attached to your individual profile. You can transfer ownership to
another owner account you control, such as an organization profile you manage.
To transfer a package to a new owner:
1. On your dashboard—or the dashboard of an organization you administer—select the package for which you
want to transfer ownership.
The system displays options for that package.
2. To display the package settings, select the Settings option.
3. Select the Admin option.
4. Under Transfer this package to a new owner, click the Transfer button.
5. Select the organization name for the new owner.
6. Click the Transfer Ownership button.
You can add other users that are not part of an organization to collaborate on your packages. You need the usernames
of the other users. You can also remove collaborators at any time.
All collaborators have full read/write permissions to the package, even if the package is private.
1. On your dashboard, click the package name.
2. Select the Settings option.
3. In the package settings, select the Collaborators option.
4. To add a collaborator, in the current collaborators, type the username of the person you want to add, then click
the Add button.
5. To remove a collaborator, click the red X button next to the collaborator name.
NOTE: Replace jsmith with your username, testpack with the package name and 0.2 with the desired version.
You can now see the change on your profile page:
https://<your-anaconda-repo>/USERNAME/PACKAGE
NOTE: Replace <your-anaconda-repo> with the name of your local Repository, USERNAME with your user-
name and PACKAGE with the package name.
Copying a package
To copy a package from the channel conda-forge to a personal channel such as jsmith:
Deprecated options
Previously labels were called “channels”, and the anaconda copy command has deprecated options
from-channel and to-channel that expect to operate on labels.
These deprecated options should not be used.
If you attempt to use them in a command such as anaconda copy --from-channel conda-forge
--to-channel jsmith glueviz, you will get an error that Label conda-forge does not exist.
You can delete individual files from a package in Repository, without deleting the entire package.
CAUTION: There is no undo for deleting a file.
To delete individual files from a package in Repository:
1. Access Repository using the Web UI.
2. In the Tools menu, select Packages.
3. Click the Files tab.
4. OPTIONAL: If the files you want to delete are not visible, under Filters, in the Type list, select All.
5. To select individual files, expand the package in which the files are located.
6. Select the checkboxes next to the files you want to delete.
7. Click the Delete button.
8. Enter your account name in the confirmation window.
Deleting a package
You can delete an entire package from Repository, including all of its versions.
CAUTION: There is no undo for deleting a package.
To delete a package from Repository:
1. Access Repository using the Web UI.
2. In the Tools menu, select Packages.
3. OPTIONAL: If the packages that you want to delete are not visible, under Filters, in the Type list, select All.
NOTE: Replace jsmith with your user name, and testpak with the package name.
You can now see the change on your profile page:
https://<your-anaconda-repo>/USERNAME
NOTE: Replace <your-anaconda-repo> with the name of your local Repository and USERNAME with your
username.
Also see the tutorial Using labels in the development cycle.
As of Anaconda Repository version 2.3.0, you can upload and download Jupyter notebooks like other files.
You can also view a static copy of notebooks in Repository. The ability to run notebooks inside Repository will
become available in a future release.
To run notebooks, use Anaconda Navigator or AE Notebooks server.
Uploading a notebook
The default maximum allowed size for notebooks is 25 MB. This limit can be changed by setting the
MAX_IPYNB_SIZE variable in the config.yaml file.
To upload a notebook to your user account, at the Anaconda Prompt or in a Terminal window, run:
NOTE: Replace my-notebook with the name of the notebook you want to upload.
To upload a new version of your notebook, while retaining the original version, upload it with the version switch from
a Terminal window or an Anaconda Prompt:
NOTE: Replace my-notebook with the name of the notebook you want to upload.
Finding a notebook
http://notebooks.<your-anaconda-repo>/USERNAME/my-notebook
NOTE: Replace <your-anaconda-repo> with the name of your local Repository, USERNAME with your user-
name and my-notebook with the name of your notebook.
To see another user’s notebook, browse to the associated user account on your Repository installation.
Downloading a notebook
Anyone with access to Repository can download your notebook using the Anaconda Prompt or Terminal window:
NOTE: Replace USERNAME with your username, and my-notebook with the name of your notebook.
A saved conda environment can be uploaded to Anaconda Repository with the web interface or the anaconda
upload command.
# To save the environment, run this command in an Anaconda Prompt or Terminal window:
https://<your-anaconda-repo>/<USERNAME>/environments
http://envs.anaconda.org/USERNAME
You can add Anaconda Projects to Anaconda Repository. Projects can be any directory of code and assets. For
example, projects often contain notebooks or Bokeh apps.
Adding a project
4. Under Settings for an individual project, you can change options, set groups and collaborations, and manage
administration of the project, including making it public, private or authenticated, transferring membership, or
deleting it.
Anaconda Repository provides a way to integrate with Cloudera Manager to distribute your Anaconda data science
artifacts to your Hadoop cluster.
You can create custom Cloudera Manager parcels with the packages you want, including your own packages.
NOTE: Creating custom parcels requires a local mirror of the Anaconda packages.
When creating a parcel, Repository generates a 64-bit Linux installer including the specified packages and a file named
construct.yaml, which can be used with conda constructor.
To create a custom parcel, management pack or installer:
1. From the view button, select Installers.
2. Click the Create new installer button.
NOTE: Use only letters, numbers, dashes and underscores in the installer name.
When creating a parcel, Anaconda Repository generates a 64-bit Linux installer with the specified packages,
and a file named construct.yaml which can be used with conda constructor.
To create just the installer script, click Create installer; to create a parcel, click Create parcel.
4. Click the Create management pack button, Create parcel button or Create installer button.
NOTE: By default, conda is not included in a custom parcel. To add additional packages to your environment, you can
add them using the Repository Web UI.
A parcel is generated with the prefix of /opt/cloudera/parcels/PARCEL_NAME. This is the default location
where activated parcels are loaded. If you are deploying parcels in a different directory, you can change this prefix
with the PARCELS_ROOT configuration setting.
/opt/cloudera/parcels/PARCEL_NAME/meta/parcel.json
After you have created a custom parcel, you can distribute it to your cluster by adding http://<repository
ip>:<port>/USERNAME/installers/parcels/ as a Remote Parcel Repository URL.
NOTE: Replace <repository ip> with the Repository IP address, <port> with the port address and USERNAME
with your user name.
Cloudera Manager detects the parcels hosted on Repository and provides the option to download and distribute the
parcels.
By default, Repository generates a parcel file for every compatible distribution.
You can customize which parcel distributions are created by configuring the PARCEL_DISTRO_SUFFIXES configu-
ration setting.
NOTE: If you have configured conda via ~/.condarc on your server for use of a proxy—for example, to mirror
behind a proxy—you must disable proxying for Repository. For more information, see the conda documentation.
EXAMPLE:
proxy_servers:
https: http://proxy.corp.example.com
http: http://proxy.corp.example.com
'http://<repository ip>': false
Once you have an installer created, you can return to this page and create a management pack or a parcel from a
specific version. Use the Create from a previous version option to choose which version you want to use. The
Create Installer button will be disabled since you have already created an installer. The other buttons will be disabled
if you have already created management packs or parcels for those versions.
The versions listed on the drop-down list are the versions that successfully created an installer. An installer is needed
to create a management pack or a parcel, so versions which failed won’t be listed.
The Sign In tab provides two links to help regain access to your account:
• I forgot my username. Click this link to have the username emailed to the email address of record.
• I forgot my password. Click this link to have a reset password link sent to the email address of record.
NOTE: The reset password link expires within 24 hours. If you no longer have access to the email account, you can
create a new account or email your administrator for assistance.
Also see Tutorials.
Anaconda Repository is package management server software that makes it easy to find, access, store and share public
and private notebooks, projects, installers, environments, and conda and PyPI packages. Repository also makes it easy
to stay current with updates made to the packages and environments you are using.
Anaconda also makes an instance of Anaconda Repository for private enterprises at Anaconda Cloud.
To begin using Repository, read Getting started, then the remaining sections of the user guide.
Administration guide
This Anaconda Repository Administration guide is intended for installers and administrators of Anaconda Repository
version 2.33.
Installation
This guide provides instructions for installing and configuring Anaconda Repository.
System requirements
Your server must meet the requirements for hardware, software, security and network. Please review and verify that
you have met all system requirements before beginning your installation.
• Hardware requirements
• Software requirements
• Security requirements
• Network requirements
• Hardware verification
• Software verification
• Security verification
Hardware requirements
Software requirements
• Linux environment: Installations have been tested on Red Hat Enterprise Linux/CentOS 6.7, 7.3, 7.4, and 7.5,
and Ubuntu 12.04+. Verify Linux version.
• Client environment may be Windows, macOS or Linux.
• Ubuntu users may need to install cURL. Verify cURL access.
• MongoDB version 2.6+ installed as root and running. Versions through 3.6 are supported. Verify MongoDB
installation.
• bzip2. Verify bzip2 installation.
Security requirements
• Root access or sudo capabilities. Verify root access and sudo privileges.
• OPTIONAL: Ability to make IPTables modifications.
• SELinux policy edit privileges.
NOTE: SELinux does not have to be disabled for Repository operation.
Network requirements
Hardware verification
Machine architecture
arch
This command displays what your system is: 32-bit “i686” or 64-bit “x86_64.”
Memory requirements
You need a minimum RAM size of 32 GB, or 16 GB RAM with 1600 MHz DDR3.
In a terminal window, run:
free -m
Storage requirements
To check your available disk space—hard drive or virtual environment size—use the built-in Linux df utility with the
-h parameter for human readable format:
df -h
Software verification
Please contact us by filing a GitHub issue if you have problems with a version other than Redhat, CentOS or Ubuntu.
Prompts may vary slightly depending on your version.
RedHat and CentOS Linux distributions have cURL pre-installed, but Ubuntu does not.
To verify cURL access, in a terminal window, run:
curl --version
If cURL is not found, Ubuntu users can use the Advanced Packaging Tool (APT) to get and install cURL:
TIP: If you already have Miniconda or Anaconda installed, in all versions of Linux you can use the conda command:
MongoDB version 2.4 or higher must installed as root and running. Versions through 3.4 are supported. To check for
the existence of MongoDB and its version number, in a terminal window, run:
mongod --version
If you get a “not found” message or if the MongoDB version is 2.3 or earlier, then install MongoDB 2.4 or higher
using the official installation instructions. Remember to install as root with the sudo command.
MongoDB must always be running before Repository can be started.
To start MongoDB:
bzip2 is installed
To check for the existence of bzip2 and its version number, in a terminal window, run:
bzip2 --version
Security verification
sudo -v
These instructions are for normal Linux installations on machines that have access to the internet.
NOTE: If the destination server is an air gapped system or otherwise does not have access to the internet, see Installing
on an air gapped system.
Your server must meet the requirements for hardware, software, security and network. Please review and verify that
you have met all system requirements before beginning your installation.
Your support representative provides you with a download URL for the Anaconda Repository installer. Make sure you
have the download URL.
In a terminal window, create the yum repo file as the root user:
RPM_CDN="https://820451f3d8380952ce65-4cc6343b423784e82fd202bb87cf87cf.ssl.cf1.
˓→rackcdn.com"
curl -O $RPM_CDN/mongodb-org-tools-2.6.8-1.x86_64.rpm
curl -O $RPM_CDN/mongodb-org-shell-2.6.8-1.x86_64.rpm
curl -O $RPM_CDN/mongodb-org-server-2.6.8-1.x86_64.rpm
curl -O $RPM_CDN/mongodb-org-mongos-2.6.8-1.x86_64.rpm
curl -O $RPM_CDN/mongodb-org-2.6.8-1.x86_64.rpm
1. Install MongoDB:
2. Start MongoDB:
1. Install MongoDB:
2. Start MongoDB:
1. Install MongoDB:
NOTE: If you do not specify a version, such as 2.6.9, apt-get installs the latest stable version, which is 3.x.
2. Start MongoDB:
1. In a Terminal window, create a new user account for Anaconda Repository named “anaconda-server,” and switch
to this new account:
sudo useradd -m anaconda-server
NOTE: The anaconda-server user is the default for installing Repository. Any username can be used, but using
the root user is discouraged.
2. Create a Repository package storage directory:
sudo mkdir -m 0770 -p /opt/anaconda-server/package-storage
3. Install Repository
Download the Repository installer from the download URL provided by your support representative:
curl "$INSTALLER_URL" > anaconda_repository.sh
2. Once you have reviewed the license terms, approve them by typing yes:
Do you approve the license terms? [yes|no] yes
4. At the end of the installation routine, update the anaconda-server user’s path—prepending /home/
anaconda-server/repo—by answering “yes” at the prompt to add the install location to your path:
source ~/.bashrc
4. Configure Repository
1. Initialize the web server and indicate the filepath for the package storage location:
anaconda-server-config --init
anaconda-server-config --set fs_storage_root /opt/anaconda-server/package-storage
NOTE: The location for file storage can be any location owned by the anaconda-server user that you created in
section 2 above.
NOTE: As of Repository 2.33.8, the fs_storage_root configuration setting is mandatory for local filesys-
tem storage and the Repository server will not run without it.
2. Configure the connection to your MongoDB database:
NOTE: Replace superuser with a username of your choice and [email protected] with an email address
where you wish to receive system email notifications.
NOTE: To ensure the bash shell does not process any of the characters in this password, limit the password to
letters and numbers, with no punctuation. After setup, you can change the password in the web UI.
4. Initialize the Repository database:
anaconda-server-db-setup --execute
NOTE: The above command is also run when upgrading Repository. Upgrade and then run:
anaconda-server-db-setup --execute
NOTE: More configuration options can be controlled with one or more .yaml configuration files. Repository reads
configuration files in this order:
1. From /etc/anaconda-server/*.yaml.
2. From $PREFIX/etc/anaconda-server/*.yaml.
3. From the path specified in the environment variable ANACONDA_SERVER_CONFIG, if it is set and the com-
mand line argument --config-file was not used.
4. From the path specified in the command line argument --config-file, if it was used.
All configuration is merged, and options from files read earlier are overwritten by files read later. If there are multiple
files in the same directory, they are read in alphabetical order.
anaconda-server-install-supervisord-config.sh
@reboot /home/anaconda-server/repo/bin/supervisord
supervisorctl status
$PREFIX/var/log/anaconda-server/application.log
2. If you are using LDAP or Kerberos authentication, modify your user account to be a superuser.
EXAMPLE: If your user account is “jsmith”:
anaconda-server-admin set-superuser "jsmith"
NOTE: See Troubleshooting if you have issues starting the repo server.
7. Client configuration
Follow the Configuring Anaconda Client instructions so you can use one or more clients to communicate with the
server.
Miniconda and Anaconda installers can be served by Repository via the static directory located at /home/
anaconda-server/repo/opt/anaconda-server/installers. To serve up the latest installers for each
platform, download them to this directory.
Define the URL for miniconda installers:
URL="https://repo.anaconda.com/miniconda/"
The Miniconda*latest*.sh always point to the latest Miniconda installers. Either these can be mirrored or the
ones with the latest version number can be mirrored. Define the list of installers to mirror:
versions="Miniconda2-4.5.4-Linux-ppc64le.sh
Miniconda2-4.5.4-Linux-x86.sh
Miniconda2-4.5.4-Linux-x86_64.sh
Miniconda2-4.5.4-MacOSX-x86_64.pkg
Miniconda2-4.5.4-MacOSX-x86_64.sh
Miniconda2-4.5.4-Windows-x86.exe
Miniconda2-4.5.4-Windows-x86_64.exe
Miniconda3-4.5.4-Linux-ppc64le.sh
Miniconda3-4.5.4-Linux-x86.sh
(continues on next page)
# miniconda installers
pushd /home/anaconda-server/repo/opt/anaconda-server/installers
URL="https://repo.anaconda.com/archive/"
versions="Anaconda3-5.2.0-Linux-ppc64le.sh
Anaconda3-5.2.0-Linux-x86.sh
Anaconda3-5.2.0-Linux-x86_64.sh
Anaconda3-5.2.0-MacOSX-x86_64.pkg
Anaconda3-5.2.0-MacOSX-x86_64.sh
Anaconda3-5.2.0-Windows-x86.exe
Anaconda3-5.2.0-Windows-x86_64.exe
Anaconda2-5.2.0-Linux-ppc64le.sh
Anaconda2-5.2.0-Linux-x86.sh
Anaconda2-5.2.0-Linux-x86_64.sh
Anaconda2-5.2.0-MacOSX-x86_64.pkg
Anaconda2-5.2.0-MacOSX-x86_64.sh
Anaconda2-5.2.0-Windows-x86.exe
Anaconda2-5.2.0-Windows-x86_64.exe"
# miniconda installers
pushd /home/anaconda-server/repo/opt/anaconda-server/installers
popd
Users can download the installers using curl from the following URL:
# Fill in server name, port, and specific installer for your platform
curl -s -O http://your.anaconda.repository:8080/downloads/Miniconda-latest-Linux-x86_
˓→64.sh
NOTE: Replace your.anaconda.repository with the IP address or domain name of your repository.
The final step is to mirror the packages from a subset of channels on Anaconda Cloud to the local Repository. The
channels to mirror are as follows:
Channel Description
anaconda Default anaconda channel containing all packages built
and supported by Anaconda, Inc. Also contains custom
packages.
r If you would like conda packages for r, mirror this chan-
nel. It is typically done under an r account.
If the local Repository will be used by Anaconda Enter-
prise Notebooks the recommended method is to mirror
wakari
these channels under the wakari account.
anaconda-nb-extensions
The packages will be mirrored to the package store defined by the fs_storage_root key as described in section 4 above.
Mirror Anaconda
anaconda-server-sync-conda
NOTE: Due to the size of the Cloud repository and depending on the available internet bandwidth, the mirroring
process can take hours.
Mirroring an Anaconda repository contains documentation and advanced yaml configuration for mirroring other
channels.
These instructions are for installation on air gapped systems or other machines that do not have access to the internet.
The air gap archives contain installers, dependencies and packages to mirror.
Your server must meet the requirements for hardware, software, security and network. Please review and verify that
you have met all system requirements before beginning your installation.
Download the installers archive and the appropriate mirrors archive for your needs. The Air gap archives page lists
the archives and their contents.
NOTE: These installation instructions assume the air gap media is available on the target server at $IN-
STALLER_PATH.
EXAMPLE:
Also download and expand the archive of conda packages you plan to mirror. These instructions assume packages are
expanded to $INSTALLER_PATH:
This section provides information about where to get the air gap archives and their contents.
The air gap archives are generated monthly, generally on the 1st of each month. Monthly archives are hosted at
http://airgap.demo.continuum.io/ organized in folders by date.
Installers Archive
All the installers and the latest Miniconda and Anaconda installers for all platforms are in the archive titled:
anaconda-enterprise-`date +%Y-%m-%d`.tar
The archive size is about 14 GB. It contains everything to install Anaconda Repository, Anaconda Enterprise Note-
books, Anaconda Adam and Anaconda Scale.
The archive contains:
Contents Description
aen-*.sh anaconda-enterprise-notebooks server, gateway, compute installers
anaconda_repository*.sh anaconda-repository installer
adam-installer*.sh adam installer
conda/ latest version of Miniconda and Anaconda for all platforms
rpms6x/ dependencies for installing on RHEL-6x/CentOS-6x
rpms7x/ dependencies for installing on RHEL-7x/CentOS-7x
Mirror archives
In addition, the anaconda-server-sync-conda subdirectory contains mirror archives. These are platform-
specific conda packages that must be mirrored after AE-Repo is installed. If you only need packages for a subset of
platforms, download the platform-based installers as they will be much smaller in size.
Each component has an md5 file and a list file which are both small and included for convenience.
NOTE: The archives contain packages for channels: Anaconda, R, Adam, Wakari. The anaconda-nb-extensions
packages are in the anaconda-nb-extensions channel.
cd $INSTALLER_PATH/rpms*x/
1. Install MongoDB:
2. Start MongoDB:
1. Install MongoDB:
2. Start MongoDB:
1. Install MongoDB:
NOTE: If you do not specify a version, such as 2.6.9, apt-get installs the latest stable version, which is 3.x.
2. Start MongoDB:
1. In a Terminal window, create a new user account for Anaconda Repository named “anaconda-server,” and switch
to this new account:
NOTE: The anaconda-server user is the default for installing Repository. Any username can be used, but using
the root user is discouraged.
2. Create a Repository package storage directory:
sudo su - anaconda-server
3. Install Repository
bash $INSTALLER_PATH/anaconda_repository-*-linux-64.sh
NOTE: Path should have only one installer that is for the latest stable version of Repository.
1. Review and accept the license terms:
2. Once you have reviewed the license terms, approve them by typing yes:
4. At the end of the installation routine, update the anaconda-server user’s path—prepending /home/
anaconda-server/repo—by answering “yes” at the prompt to add the install location to your path:
6. For the new path changes to take effect, source your .bashrc:
source ~/.bashrc
4. Configure Repository
1. Initialize the web server and indicate the filepath for the package storage location:
anaconda-server-config --init
anaconda-server-config --set fs_storage_root /opt/anaconda-server/package-storage
NOTE: The location for file storage can be any location owned by the anaconda-server user that you created in
section 2 above.
NOTE: As of Repository 2.33.8, the fs_storage_root configuration setting is mandatory for local filesys-
tem storage and the Repository server will not run without it.
2. Configure the connection to your MongoDB database:
NOTE: Replace superuser with a username of your choice and [email protected] with an email address
where you wish to receive system email notifications.
NOTE: To ensure the bash shell does not process any of the characters in this password, limit the password to
letters and numbers, with no punctuation. After setup, you can change the password in the web UI.
4. Initialize the Repository database:
anaconda-server-db-setup --execute
NOTE: The above command is also run when upgrading Repository. Upgrade and then run:
anaconda-server-db-setup --execute
anaconda-server-install-supervisord-config.sh
@reboot /home/anaconda-server/repo/bin/supervisord
supervisorctl status
$PREFIX/var/log/anaconda-server/application.log
NOTE: See Troubleshooting if you have issues starting the repo server.
7. Client configuration
Follow the Configuring Anaconda Client instructions so you can use one or more clients to communicate with the
server.
Miniconda and Anaconda installers can be served by Repository via the static directory located at /home/
anaconda-server/repo/opt/anaconda-server/installers. To serve up the latest installers for each
platform, copy them from your air gap archive to this directory.:
cp Miniconda-latest-Linux-x86_64.sh /home/anaconda-server/repo/opt/anaconda-server/
˓→installers
# Fill in server name, port, and specific installer for your platform
curl -s -O http://your.anaconda.repository:8080/downloads/Miniconda-latest-Linux-x86_
˓→64.sh
NOTE: Replace your.anaconda.repository with the IP address or domain name of your repository.
The final step is to mirror the packages from a subset of channels on Anaconda Cloud to the local Repository. The
channels to mirror are as follows:
Channel Description
anaconda Default anaconda channel containing all packages built
and supported by Anaconda, Inc. Also contains custom
packages.
r If you would like conda packages for r, mirror this chan-
nel. It is typically done under an r account.
If the local Repository will be used by Anaconda Enter-
prise Notebooks the recommended method is to mirror
wakari
these channels under the wakari account.
anaconda-nb-extensions
The packages will be mirrored to the package store defined by the fs_storage_root key as described in section 4 above.
Mirror Anaconda
Since we are mirroring from a local file system, some additional configuration is necessary. The steps are the same for
each channel:
1. Create a mirror configuration yaml typically stored in $PREFIX/etc/anaconda-server/mirror/.
2. Customize your mirror. An example is if you only need to mirror packages for a subset of platforms. By default,
it mirrors all packages found in the channels linux-64, osx-64, win-64, win-32 and linux-32.
3. Invoke the mirror command by pointing it to the config file:
echo "channels:" > ~/repo/etc/anaconda-server/mirror/conda.yaml
echo " - file://$MIRRORS_ARCHIVE/anaconda-suite/pkgs" >> \
~/repo/etc/anaconda-server/mirror/conda.yaml
Mirroring an Anaconda repository contains documentation and advanced yaml config for mirroring other chan-
nels.
Configuration
Enabling HTTPS
Before you begin, purchase an SSL certificate and download the SSL *.cert file and SSL *.key file.
NOTE: If security is not an issue, for testing, you may set up a self-signed SSL certificate. For more information, see
http://www.selfsignedcertificate.com/.
1. Save the SSL *.cert file and an SSL *.key file in your home directory.
2. Configure the server to use those keys and the correct ports:
Configure conda
If your conda client is configured to point to this local repo, update the configuration file .condarc to contain the
ssl_verify flag. If you’re using a self-signed certificate, configure the ssl_verify flag in .condarc to point
to the root CA used to sign the Anaconda Enterprise Repository server certificate.
Configure anaconda-client
If you’re using anaconda-client to connect to Anaconda Enterprise Repository with the command line, set the
ssl_verify flag.
Use anaconda config --files to find the anaconda-client configuration files.
SEE the command reference for updating the client sites configuration for anaconda-client.
To send emails such as password reset emails, Repository must have the email settings configured.
The easiest way to enable clients to access a Repository server on standard ports is to configure the server to redirect
traffic received on standard HTTP port 80 to the standard Repository HTTP port 8080:
HTTPS
To use HTTPS, redirect traffic from standard HTTPS port 443 to standard Repository HTTPS port 8443:
sudo iptables -t nat -A OUTPUT -d localhost -p tcp --dport 443 -j REDIRECT --to-ports
˓→8443
sudo iptables -t nat -I PREROUTING -p tcp --dport 443 -j REDIRECT --to-port 8443
Alternate port
Enable clients to access a Repository on standard ports by configuring the server to redirect traffic received on standard
HTTP port 80 to the standard Repository HTTP port 8080.
NOTE: These commands assume the default state of IPTables, which is on and allowing inbound SSH access on port
22. This is the factory default state for CentOS 6.7. If this default has been changed, you can reset it:
sudo iptables -L
CAUTION: Mistakes with IPTables rules can render a remote machine inaccessible.
1. Allow inbound access to tcp port 80:
sudo iptables -I INPUT -i eth0 -p tcp --dport 80 -m comment --comment "# Anaconda
˓→Repo #" -j ACCEPT
sudo iptables -I INPUT -i eth0 -p tcp --dport 8080 -m comment --comment "#
˓→Anaconda Repo #" -j ACCEPT
iptables -L -n
Chain INPUT (policy ACCEPT)
target prot opt source destination
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:8080 /* #
˓→Anaconda Repo # */
NOTE: The PREROUTING (nat) IPTables chain is not displayed by default. To display the chain:
iptables -L -n -t nat
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
REDIRECT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:80 /* #
˓→Anaconda Repo # */ redir ports 8080
If you already have a MongoDB server running, you can connect to it by setting the MONGO_URL configuration
variable:
For more information, see the MongoDB Connection String URI Format manual.
See also Configuring MongoDB authentication.
By default, MongoDB does not require a username or password to access or modify the database. We recommend
enabling and configuring mandatory authentication.
1. Open a MongoDB shell:
mongo
2. Repository requires read/write access to the database binstar. Enter the following commands into the Mon-
goDB shell to create an administrative user and a service user:
use admin
auth=true
• If you are using the current MongoDB configuration format, add the security.authorization key to /etc/
mongod.conf:
security:
authorization: enabled
8. Edit the Repository configuration file and set the MONGO_URL parameter to mongodb://
<username>:<password>@<hostname>.
After editing the configuration file, restart Repository for the changes to take effect.
9. Edit the Repository configuration file and set the MONGO_URL parameter to mongodb://
<username>:<password>@<hostname>.
After editing the configuration file, restart Repository for the changes to take effect.
NOTE: For more information about MongoDB authentication and authorization, see https://docs.mongodb.com/v2.6/
core/authentication/ and https://docs.mongodb.com/v2.6/core/authorization/.
Sometimes you do not want to replicate all the packages from Repository into your mirror. The
anaconda-server-sync-conda tool includes whitelist/blacklist functionality to manipulate your list of mir-
rored packages in a variety of ways.
A mirror config file can be specified when you run anaconda-server-sync-conda with the flag
--mirror-config=FILEPATH and replace FILEPATH with the path to your config file.
NOTE: Configuration files are yaml files.
To customize your distribution, you have the following options:
• remote_url: Repository mirrors packages from this source URL.
• mirror_dir: Repository stores packages in this directory on the machine where the script is executed.
mirror_dir: /opt/anaconda-server/package-storage
platforms:
- linux-32
- linux-64
license_blacklist: GPL
whitelist:
- distribute
- conda
blacklist:
- flask
- readline
The step-by-step algorithm that is used by cas-mirror to create the ultimate list of packages to mirror follows this
order:
1. Get a full list of packages from default_url.
2. If the platforms option is present, only those packages available to the platforms listed here are left on the list.
3. If license_blacklist is present, then all the packages subject to any of the licenses mentioned here are removed
from the list. See the list of license families that can be blacklisted.
4. If blacklist is present, then all member packages explicitly mentioned here are removed from the list.
5. If whitelist is present, then those assigned member packages are added to the list. The whitelist option overrides
license_blacklist and blacklist, so that a package listed here is mirrored even when under a GPL license or if it
appears in the blacklist option.
After performing all of the above actions sequentially, the script produces the ultimate list of packages that are mir-
rored.
To prevent cross-site scripting attacks (XSS), user content—such as Jupyter Notebooks—can be served from a separate
domain.
To enable this:
1. Configure the project to use a separate content domain:
1. If your user content domain is a subdomain of your Repository domain, you must also configure the session
cookie to only send to the root domain:
account_names_filter: false
USER_REGEX: ^[a-z0-9_][a-z0-9_-.]+$
LDAP:
# Replace with company LDAP server
URI: 'ldap://<ldap.company.com>'
2. When switching authentication to LDAP, the admin account is lost, so you need to add your admin account
again:
NOTE: Replace jsmith and abc123DEF with your LDAP username and password.
4. To apply the changes, restart the Repository server:
5. Open a new browser window and navigate to your local Repository installation:
http://your.anaconda.repository
NOTE: Replace your.anaconda.repository with your Repository server IP address or domain name.
6. Log in using your LDAP credentials.
7. Optional. You may set an LDAP network timeout in seconds with the options OPT_NETWORK_TIMEOUT
and OPT_TIMEOUT. The default value is 0, meaning no timeout.
For example, to set the timeout to 60 seconds, add this block to the LDAP settings in your configuration file:
OPTIONS:
OPT_NETWORK_TIMEOUT: 60
OPT_TIMEOUT: 60
Microsoft Active Directory is a server program that provides directory services and uses the open industry standard
Lightweight Directory Access Protocol (LDAP).
To enable Active Directory support:
1. Open the Repository configuration file $PREFIX/etc/anaconda-server/config.yaml and add the
following configuration:
account_names_filter: false
USER_REGEX: ^[a-z0-9_][a-z0-9_-.]+$
LDAP:
# Replace with company LDAP server
'URI': 'ldap://<ldap.server.url>'
# The values '' perform an anonymous bind so we may use search/bind method
BIND_DN: ''
BIND_AUTH: ''
NOTE: Replace jsmith and abc123DEF with your Active Directory username and password.
You see a response similar to the following:
http://your.anaconda.repository
Repository can be configured to allow synchronizing the membership of organization groups with groups in an LDAP
directory. Owners of an organization can select a specific LDAP group as the source of group members.
Once this is enabled, users who sign in to Repository who are members of the LDAP group automatically are granted
the permissions of the organization group.
To enable LDAP groups, configure the following:
• Authenticated bind to LDAP. Repository needs to perform searches against the directory to determine the avail-
able groups and the membership of those groups.
• A query for Repository to identify the groups in your LDAP directory. For more information, see
GROUP_SEARCH.
If LDAP synchronization is disabled or the LDAP server is unreachable, the member list at the time is used for the
group.
To administer and debug LDAP synchronization, a superuser can visit:
http://your.anaconda.repository/admin/ldap
To enable a secure Transport Layer Security (TLS) connection on LDAP/Active Directory, add the following to the
LDAP configuration section of the file $PREFIX/etc/anaconda-server/config.yaml, replacing /path/
to/certfile with the actual path to the certfile.:
LDAP:
... # Rest of the LDAP config
START_TLS: true,
OPTIONS:
OPT_PROTOCOL_VERSION: 3
OPT_X_TLS_DEMAND: true
OPT_X_TLS_REQUIRE_CERT: 'OPT_X_TLS_NEVER'
OPT_X_TLS_CACERTFILE: '/path/to/certfile'
NOTE: START_TLS is not compatible with LDAPS. When using START_TLS, the URI value in the LDAP configu-
ration section must start with ldap://. When using START_TLS, the connection starts as a regular connection, and
is upgraded to use TLS after connection has been established.
If you’re using self-signed certificates, you’ll need to add OPT_X_TLS_NEWCTX as the last entry of the OPTIONS
field of the LDAP options:
LDAP:
... # Rest of the LDAP config
START_TLS: true,
OPTIONS:
OPT_PROTOCOL_VERSION: 3
OPT_X_TLS_DEMAND: true
OPT_X_TLS_REQUIRE_CERT: 'OPT_X_TLS_NEVER'
OPT_X_TLS_CACERTFILE: '/path/to/certfile'
OPT_X_TLS_NEWCTX: 0
URI
Start by setting URI to point to your server. The value of this setting can be anything that your LDAP library supports.
For instance, openldap may allow you to give a comma- or space-separated list of URI values to try in sequence.
BIND_DN
The distinguished name to use when binding to the LDAP server with BIND_AUTH. Use the empty string—the
default—for an anonymous bind.
BIND_AUTH
USER_SEARCH
A dictionary that locates a user in the directory. The dict object must contain the required entries base and filter
and may contain the optional entry scope.
• base: The base DN to search.
• filter: Should contain the placeholder %(username)s for the username.
• scope: One of LDAP_SCOPE_BASE, LDAP_SCOPE_ONELEVEL or LDAP_SCOPE_SUBTREE.
EXAMPLE:
SUPERUSER_SEARCH
A dict that will determine whether a valid user is a superuser. The dict object must contain the required entries base
and filter and may contain the optional entry scope. If the search is successful, then something is returned by the
LDAP server, and the user is given superuser permissions.
• base: The base DN to search.
• filter: Should contain the placeholder %(username)s for the username.
• scope: One of LDAP_SCOPE_BASE, LDAP_SCOPE_ONELEVEL, or LDAP_SCOPE_SUBTREE.
For example:
Notice that this check is done during the login procedure, so even though privileges might have been removed from
(or added to) the LDAP server, the user will have to authenticate again to see the changes.
ENABLE_GROUPS
This attribute enables LDAP group synchronization, allowing users to synchronize group membership with an LDAP
directory. Defaults to false.
EXAMPLE:
ENABLE_GROUPS: true
GROUP_SEARCH
A dictionary that locates a group in the directory. An LDAP search is performed using the base distinguished name
and filter.
NOTE: Unlike USER_SEARCH, you must put parenthesis around the GROUP_SEARCH filter. It may appear to
work without parenthesis, when it’s actually failing or behaving unpredictably.
EXAMPLE:
GROUP_SEARCH:
base: dc=example,dc=com
filter: (objectClass=group)
NOTE: Anaconda Repository assumes that the groups’ objectClass is groupOfNames (or a compatible
schema). The following LDIF snippet shows an example group instance:
GROUP_MEMBERS_ATTR
The LDAP attribute on a group object that indicates the users that are members of the group. Defaults to member.
EXAMPLE:
GROUP_MEMBERS_ATTR: 'member'
NOTE: Anaconda Repository assumes that the groups’ objectClass is groupOfNames (or a compatible schema).
REFRESH_INTERVAL
The number of seconds that group membership information from LDAP is used before being fetched from the directory
server again. Defaults to 3600, which is 1 hour.
EXAMPLE:
REFRESH_INTERVAL: 600
KEY_MAP
This is a dict mapping application context to LDAP. An application may expect user data to be consistent, and not all
LDAP setups use the same configuration:
'application_key': 'ldap_key'
EXAMPLE:
START_TLS
If true, each connection to the LDAP server calls start_tls_s() to enable TLS encryption over the standard
LDAP port. There are a number of configuration options that can be given to OPTIONS that affect the TLS connection.
For example, OPT_X_TLS_REQUIRE_CERT can be set to OPT_X_TLS_NEVER to disable certificate verification,
perhaps to allow self-signed certificates.
OPTIONS
LDAP:
OPTIONS:
OPT_PROTOCOL_VERSION: 3
OPT_X_TLS_REQUIRE_CERT: 'OPT_X_TLS_NEVER'
TLS—secure LDAP
To enable a secure TLS connection you must set START_TLS to true. There are a number of configuration options
for OPTIONS that affect the TLS connection.
EXAMPLE: OPT_X_TLS_REQUIRE_CERT set to OPT_X_TLS_NEVER disables certificate verification, perhaps to
allow self-signed certificates:
LDAP:
START_TLS: true
OPTIONS:
OPT_PROTOCOL_VERSION: 3
OPT_X_TLS_DEMAND: true
OPT_X_TLS_REQUIRE_CERT: 'OPT_X_TLS_NEVER'
OPT_X_TLS_CACERTFILE: '/path/to/certfile'
Kerberos is an authentication protocol designed to allow nodes communicating over an insecure network to verify
identity. Repository can use Kerberos to authenticate users.
The Kerberos protocol uses timestamps to prevent replay attacks on expired credentials, so the Network Time Protocol
(NTP) service must be set up and working correctly.
Several aspects of Kerberos rely on name service. Your domain name system (DNS) entries and your hosts must
have the correct information. The hostname command and the configuration file /etc/hostname must reflect
the fully-qualified domain name (FQDN) of the machine. The configuration file /etc/hosts must include an entry
with the FQDN, to allow reverse-DNS lookups to be performed.
To allow clients to authenticate against Anaconda Repository, create a principal for the service with a private key that
identifies the service. Create a service principal HTTP/your.anaconda.repository, and create the keytab
containing this principal to $PREFIX/etc/anaconda-server/http.keytab:
SERVER_NAME=your.anaconda.repository
7. In the Properties dialog, select the Account tab, and ensure the Do not require Kerberos preauthentication option
is selected.
8. Open an Administrative prompt and run:
AUTH_TYPE: KERBEROS
KRB5_KTNAME: /home/anaconda-server/repo/etc/anaconda-server/http.keytab
AUTH_TYPE string Configures the authentication scheme used for Repository. Set to
KERBEROS to enable Kerberos authentication. Default: NATIVE.
KRB5_KTNAME string The file path of the keytab containing the service principal for
Repository. Default: /etc/krb5.keytab.
KRB5_SERVICE_NAME
string The service type used to identify the service principal for Reposi-
tory. HTTP in HTTP/your.anaconda.repository@YOUR.
REALM. Default: HTTP.
KRB5_HOSTNAME string The hostname used to identify the service principal for Repos-
itory. your.anaconda.repository in HTTP/your.
[email protected]. Default: the host-
name of the machine on which Repository is running.
Kerberos authentication adds a layer of security to Anaconda Repository. The following example show how to set up a
minimal working installation with three machines: One running anaconda server, one running the MIT Kerberos Key
Distribution Center (KDC), and a client from where we are going to connect to both services.
For this example we assume that both the KDC and Anaconda Repository are already configured and the 3 systems
have the Network Time Protocol (NTP) service working.
Initial Setup
All 3 machines are running CentOS 7 but the configurations mentioned here apply for many other Linux distributions.
We are going to use the following domain names:
• Anaconda Repository: anaconda.kerberos.local
• Kerberos KDC: kdc.kerberos.local
• Client: client.kerberos.local
Make sure that the information is correct in the configuration files /etc/hostname and /etc/hosts to allow
reverse-DNS lookups.
The name of the Kerberos realm is KERBEROS.LOCAL. The 3 machines have the same configuration file /etc/
krb5.conf:
[logging]
kdc = FILE:/var/log/krb5kdc.log
admin_server = FILE:/var/log/kadmind.log
default = SYSLOG:NOTICE:DAEMON
[libdefaults]
dns_lookup_realm = true
dns_lookup_kdc = true
ticket_lifetime = 24h
renew_lifetime = 7d
forwardable = true
rdns = false
default_realm = KERBEROS.LOCAL
default_ccache_name = KEYRING:persistent:%{uid}
[realms]
KERBEROS.LOCAL = {
kdc = kdc.kerberos.local
admin_server = kdc.kerberos.local
}
[domain_realm]
.kerberos.local = KERBEROS.LOCAL
kerberos.local = KERBEROS.LOCAL
At this point Anaconda Repository is up and running, it’s installed on /home/anaconda-server/repo, the
administrator account in this example is superuser. To allow authentication we first create a service principal and
the keytab containing this principal. This is accomplished running the following commands as root from a terminal on
anaconda.kerberos.local.
chown anaconda-server:anaconda-server \
/home/anaconda-server/repo/etc/anaconda-server/http.keytab
chmod 600 /home/anaconda-server/repo/etc/anaconda-server/http.keytab
AUTH_TYPE: KERBEROS
KRB5_KTNAME: /home/anaconda-server/repo/etc/anaconda-server/http.keytab
Finally, add the principal for the admin account on the kerberos realm:
Client Configuration
To log in to Anaconda Repository with Kerberos Authentication, a browser that supports said authentication protocol
is necessary. In this example we are using Firefox. Some extra tweaking is required.
• Open Firefox and type about:config in the navigation bar, click the confirmation button if necessary to proceed
to the configuration page.
• Type negotiate in the Search field to filter out the options, double click network.negotiate-auth.trusted-uris
and enter .kerberos.local in the text box.
• Do the same for network.negotiate-auth.delegation-uris.
Finally a ticket for the superuser should be stored on the local machine. The following command will request it:
kinit [email protected]
Now it is possible to open anaconda server on firefox, in this case the URL is anaconda.kerberos.
local:8080, after clicking Sign In, the user should be able to log in immediately without having to enter any
credentials.
AUTH_TYPE: PAM
When switching authentication to PAM the admin account is lost, so you need to add your admin account again:
Open a new browser window and navigate to your local Anaconda Repository installation:
http://your.anaconda.server
NOTE: Replace “your.anaconda.server” with your actual Anaconda Repository server IP address or domain name.
You can now log in using your PAM credentials.
NOTE: To use the “shadow” PAM backend, add the user under which Anaconda Repository is running (usually
“anaconda-server”) to the “shadow” group:
The site can be put into read only mode to disable any action that modifies the database.
This may be useful when mirroring from the site.
NOTE: Logging in modifies the database, so in read only mode users and admins may not log in.
To enable read only mode use the setting:
READ_ONLY: true
• Client configuration
• Conda configuration
• Pip configuration
• Kerberos configuration
Anaconda Client gives you the ability to upload packages to your on-site Anaconda Repository and provides highly
granular access control capabilities. The instructions below describe how to configure Client to use your local Repos-
itory instead of Anaconda Cloud.
Client configuration
On each machine that accesses your on-site Repository, run this command as the machine’s local user:
Or, to set the default repo on a system-wide basis, run this command:
NOTE: Replace your.server.name with the name of your local Repository and <port> with the name of the
port used by Repository.
The system level config file is used only if no user-level config file is present.
To show the system and user config file locations and configuration settings:
Conda configuration
When the above anaconda config steps are completed, you can access all packages and channels from the local
on-site Repository instead of the public Anaconda.org.
Users can then add individual accounts to their .condarc file by running the following command:
If you still want to access certain channels from the public Anaconda.org, run:
To set a preferred priority for the channels conda searches for package installs, edit your ~/.condarc file and
change the order. Channels at the top are searched first.
For example:
channels:
- channel
- https://conda.anaconda.org/t/<token>/<channel2>
- http://conda.anaconda.org/<channel1>
- defaults
Pip configuration
To install PyPI packages from your Repository, add your channel to your ~/.pip/pip.conf configuration file.
Edit the file and add an extra-index-url entry to the global config section:
[global]
extra-index-url = http://your.server.name:<port>/pypi/USERNAME/simple
NOTE: Replace your.server.name with the name of your local Repository, <port> with the name of the port
used by Repository and USERNAME with your username.
Kerberos configuration
If you have enabled Kerberos authentication as described in Configuring Repository to use Kerberos, your browser
and Client should be able to authenticate to Repository using Kerberos.
In macOS/Unix, configure the file /etc/krb5.conf:
[libdefaults]
default_realm = YOUR.DOMAIN
[realms]
YOUR.DOMAIN = {
(continues on next page)
[domain_realm]
your.anaconda.repository = YOUR.DOMAIN
NOTE: Replace YOUR.DOMAIN with your domain, your.kdc.server with your Kerberos key distribution center
(KDC) and your.anaconda.repository with your local Repository server.
If your configuration is correct, you should be able to authenticate using the command line tool kinit:
kinit jsmith
anaconda login
Browser Setup
Many browsers do not present your Kerberos credentials by default, to prevent leaking credentials to untrusted parties.
In order to use Kerberos authentication, you must whitelist Repository as a trusted party to receive credentials.
You must restart your browser after configuring the whitelist in order for changes to be reflected.
Safari
Chrome
The AuthServerWhitelist policy must be set to your.anaconda.repository to allow Chrome to present cre-
dentials to Repository with the hostname your.anaconda.repository. Depending on your DNS configuration,
DisableAuthNegotiateCnameLookup may also be required to prevent Chrome from canonicalizing the hostname be-
fore generating a service name.
NOTE: Replace your.anaconda.repository with your local Repository server.
To configure on macOS:
On Linux:
mkdir -p /etc/opt/chrome/policies/managed
mkdir -p /etc/opt/chrome/policies/recommended
chmod -w /etc/opt/chrome/policies/managed
echo '{"AuthServerWhitelist": "your.anaconda.repository"}' > /etc/opt/chrome/policies/
˓→managed/anaconda_repo_policy.json
On Windows, use Group Policy objects to set the Authentication server whitelist setting to your.anaconda.
repository.
For more information, see Chrome’s SPNEGO authentication and administration documentation.
Firefox
Internet Explorer
You can add a local copy—mirror—of Anaconda or PyPI repositories to your Anaconda Repository installation.
This section explains how to use Repository’s convenient syncing tools to create and configure local mirrors:
NOTE: This documentation is for versions of Anaconda Repository 2.26 and newer. For versions from 2.23 through
2.25, see 2.23 documentation. For versions before 2.23, see 2.20 documentation.
You need to have already installed and configured your Repository instance. Due to the size of Repository, it is
important that you have configured a file storage location with sufficient disk space. If necessary, see the requirements
for the file storage location.
A full Anaconda mirror requires approximately 90 GB.
You will also need to install cas-mirror as it is the recommended mirroring tool.
NOTE: The anaconda-mirror tool has been deprecated and will not be updated any further.
You can mirror some or all of the contents of the Anaconda repository using the cas-sync-api-v4 command:
$ cas-sync-api-v4 --help
usage: cas-sync-api-v4 [-h] [-f FILENAME] [--config] [--version]
[-l LOG_LEVEL] [-v]
optional arguments:
-h, --help show this help message and exit
-f FILENAME, --file FILENAME
Configuration file location (Defaults to:
/home/abarto/.cas-mirror or /etc/cas-mirror)
--config, --show-config
Show running configuration and exit
--version Print version and exit
-l LOG_LEVEL, --log-level LOG_LEVEL
Set the log level (CRITICAL, ERROR, WARNING, INFO,
DEBUG)Default: INFO.
-v, --verbose Shorthand for --log-level DEBUG
This tools mirrors all of the packages from the Anaconda repository default channels into the anaconda user account.
It leverages the functionality exposed by the anaconda-client package to import the remote packages into an
existing site.
1. Associate the URL of your Anaconda repository with a site using the anaconda command:
anaconda config --set sites.myrepo.url "http://your-anaconda-repo/"
fetch_installers: True
repodata_source: False
dir_names: ['archive']
server_log_dir: None
server_port: None
dest_site: myrepo
dest_channel: anaconda
verify_checksum: False
delta: False
delta_dir: None
log_dir: None
log_level: 20
python_versions: []
(continues on next page)
With this configuration, cas-sync-api-v4 mirrors the contents of all the default channels into the
anaconda account of the myrepo site.
4. Run cas-sync-api-v4:
cas-sync-api-v4 -f sync.yaml
Alternately, you may not want to mirror all packages. To mirror a subset of the total repository, specify which platforms
you want to include, or use the whitelist, blacklist or license_blacklist functionality to control which packages are
mirrored, by copying the default configuration file $PREFIX/etc/anaconda-server/mirror/anaconda.
yaml to $PREFIX/etc/anaconda-server/mirror/anaconda-custom.yaml.
For Repository 2.27 or newer, there are sample yaml config files located at: $PREFIX/etc/anaconda-server/
mirror. PREFIX is the install location of Repository, which by default is ~anaconda-server/repo/etc/
anaconda-server/mirror.
This command mirrors the repository according to the settings in the configuration file anaconda-custom.yaml:
cas-sync-api-v4 -f anaconda-custom.yaml
Offline mirroring
Offline mirroring is done by using both cas-sync and cas-sync-api-v4. First download all the packages onto
a host with Internet access.
EXAMPLE:
To download the packages, create a configuration file named export.yaml:
mirror_dir: /opt/mirror/export/
platforms:
- linux-64
- win-64
python_versions:
- 2.7
- 3.6
(continues on next page)
After it finishes, a conda repository for each platform is created in the directory /opt/mirror/export/pkgs.
We can now take the contents of the directory /opt/mirror/export/ to the air-gapped environment.
To import the packages, create a config file named import.yaml:
dest_site: mysite
dest_channel: anaconda
channels:
- file:///opt/mirror/export/pkgs/
platforms:
- linux-64
- win-64
python_versions:
- 2.7
- 3.6
It is used with the command anaconda-mirror sync, or with a configuration file such as anaconda-custom.
yaml with the command anaconda-mirror --config-file anaconda-custom sync.
Resetting packages
This resets the “last sync” time for the repository, so anaconda-mirror requests all packages, not just those
changed or added since the last sync. As the requests are processed, anaconda-mirror still automatically down-
loads only those files that differ from the files currently in the repository.
Exporting a mirror
This command dumps the packages, according to the configured settings, into the file mirror.tar.
This mirror can be used in an air gapped environment.
Importing a mirror
To mirror the Anaconda repository in an air gapped environment, point anaconda-mirror to the exported mirror
archive.
Mount the USB drive and then run:
This command mirrors the contents of the local Anaconda repository to your Repository installation under the user-
name “anaconda.”
Filtering
If you want to update the filters on your mirror—for example, to exclude additional licenses—running
anaconda-mirror sync again retrieves new packages that match this filter, but it does not remove existing
packages that no longer match the filter.
To see which packages no longer match your filter:
anaconda-mirror clean
If mirroring from an air gap archive, the channel in the following configuration points to a local directory to which the
archive is expanded.
In addition, if a platform-specific archive is downloaded, then the config file needs the platforms section. The examples
in the following sections assume x64-repo-mirrors-\*.tar <airgap-archive-mirrors> is expanded
to $MIRRORS_ARCHIVE.
Similarly, for an online system, the channel points to Anaconda Cloud. The platforms are optional and limit the
mirrored conda packages to the specified platforms.
Mirroring R channel
cat $PREFIX/etc/anaconda-server/mirror/r.yaml
channels:
- file://$MIRRORS_ARCHIVE/r/pkgs
cat $PREFIX/etc/anaconda-server/mirror/r.yaml
channels:
- https://conda.anaconda.org/r
anaconda-server-sync-conda --mirror-config \
$PREFIX/etc/anaconda-server/mirror/r.yaml --account=r-channel
cat $PREFIX/etc/anaconda-server/mirror/wakari.yaml
channels:
- file://$MIRRORS_ARCHIVE/wakari/pkgs
cat $PREFIX/etc/anaconda-server/mirror/wakari.yaml
channels:
- https://conda.anaconda.org/t/<TOKEN>/anaconda-nb-extensions
- https://conda.anaconda.org/wakari
NOTE: Replace <TOKEN> with the token for the anaconda-nb-extensions channel that you should have received
along with your Repository license.
2. Mirror the packages to the Wakari channel:
anaconda-server-sync-conda --mirror-config \
$PREFIX/etc/anaconda-server/mirror/wakari.yaml --account=wakari
cat $PREFIX/etc/anaconda-server/mirror/anaconda-adam.yaml
channels:
- file://$MIRRORS_ARCHIVE/anaconda-adam/pkgs
cat $PREFIX/etc/anaconda-server/mirror/anaconda-adam.yaml
channels:
- https://conda.anaconda.org/anaconda-adam
anaconda-server-sync-conda --mirror-config \
$PREFIX/etc/anaconda-server/mirror/anaconda-adam.yaml --account=anaconda-adam
Configuring conda
Having created the mirror, you still need to configure conda to search for packages here rather than on the default
Anaconda repository. You can do that by editing your ~/.condarc file to add the appropriate channel:
channels:
- http://<anaconda.repo.ipaddress>:<port>/conda/anaconda/
You need to have already installed and configured your Repository instance. Due to the size of Repository, it is
important that you have configured a file storage location with sufficient disk space. If necessary, see the requirements
for the file storage location.
The full PyPI mirror requires approximately 120 GB.
anaconda-server-sync-pypi
This command loads all of the packages on pypi.python.org into the ~pypi binstar user account.
Verify that the command ran successfully by opening your browser to http://your-anaconda-repo/pypi/
~pypi.
NOTE: Replace your-anaconda-repo with the URL to your installation of Repository.
It is possible to customize the mirror behavior by creating a configuration file such as $PREFIX/etc/
anaconda-server/mirror/pypi.yaml and using the --mirror-config option:
Name Description
user The local user under which the PyPI packages are imported. Default: pypi.
pkg_list A list of packages to mirror. Only packages listed are mirrored. If this is set, blacklist and
whitelist settings are ignored. Default: [].
whitelistA list of packages to mirror. Only packages listed are mirrored. If the list is empty, all packages are
checked. Default: [].
blacklistA list of packages to skip. The packages listed are ignored. Default: [].
Only download the latest versions of the packages. Default: false.
latest_only
The URL of the PyPI mirror. /pypi is appended to build the XML RPC API URL, /simple for the
remote_url
simple index and /pypi/{package}/{version}/json for the JSON API. Default: https:/
/pypi.python.org/.
A custom value for XML RPC URL. If this value is present, it takes precedence over the URL built
xml_rpc_api_url
using remote_url. Default: null.
A custom value for the simple index URL. If this value is present, it takes precedence over the URL
simple_index_url
built using remote_url. Default: null.
Whether to use the XML RPC API as specified by PEP381. If this is set to true, the XML RPC API
use_xml_rpc
is used to determine which packages to check. Otherwise the scripts falls back to the simple index. If
the XML RPC fails, the simple index is used. Default: true.
Whether to use the serial number provided by the XML RPC API. Only packages updated since the
use_serial
last serial saved are checked. If this is set to false, all PyPI packages are checked for updates. Default:
true.
Create the mirror user as an organization instead of a regular user account. All superusers are added to
create_org
the “Owners” group of the organization. Default: false.
private Save the mirrored packages as private. Default: false.
EXAMPLE:
whitelist:
- requests
- six
- numpy
- simplejson
latest_only: true
remote_url: http://pypimirror.local/
use_xml_rpc: true
Configuring pip
[global]
index-url=https://pypi.anaconda.org/pypi/simple
Customizing mirrors
This section explains how you can customize your PyPI or Anaconda repository mirror.
NOTE: This documentation applies to versions of Repository 2.26 and newer. For versions between 2.23 and 2.26,
see 2.23 documentation. For older versions, see 2.20 documentation.
Your Anaconda Repository should already be installed and configured, as well as the mirroring tool cas-mirror.
See mirroring an Anaconda repository.
remote_url
Conda packages, Anaconda installers and Miniconda installers are fetched from this remote URL.
DEFAULT: https://repo.anaconda.com/
channels
mirror_dir
platforms
Conda packages and installers for this list of platforms are mirrored.
DEFAULT: A list of all platforms. This is:
['osx-32', 'osx-64', 'win-32', 'win-64', 'linux-32', 'linux-64',
'linux-armv6l', 'linux-armv7l', 'linux-ppc64le']
fetch_installers
python_versions
pkg_list
license_blacklist
blacklist
whitelist
dest_channel
dest_site
delta
delta_dir
max_retries
safe
If safe is true, synchronizing repositories or merging delta directories never delete anything.
DEFAULT: false
repodata_source
By default, cas-sync and cas-sync-api-v4 mirror all platforms. If you do not need all platforms, you can save
time and disk space by editing the yaml file to specify which platform(s) you want to mirror.
EXAMPLE:
platforms:
- linux-64
- win-32
You may want to mirror only a small subset of Repository. Rather than blacklisting a long list of packages you do not
want to be mirrored, you can instead simply enumerate the list of packages you do want to mirror.
EXAMPLE: This example mirrors only the three packages Accelerate, PyQt and Zope. All other packages are ignored:
package_list:
- accelerate
- pyqt
- zope
python_versions:
- 3.3
As of Repository 2.26.0, the Anaconda mirroring script supports license blacklisting for the following license families:
• AGPL
• APACHE
• BSD
• GPL2
• GPL3
• LGPL
• MIT
• PROPRIETARY
• PUBLICDOMAIN
• PSF
• OTHER
• NONE
EXAMPLE: This example mirrors all the packages in the repository except those that are GPL2-, GPL3- or BSD-
licensed:
license_blacklist:
- GPL2
- GPL3
- BSD
NOTE: Older versions of Anaconda mirror support only license blacklisting for GPL. If you are using an older version
of Repository, see the documentation for customizing your PyPI or Anaconda Repository mirror.
The blacklist allows access to all packages except those explicitly listed.
EXAMPLE: This example mirrors the entire Repository except the bzip2, tk and openssl packages:
blacklist:
- bzip2
- tk
- openssl
The whitelist functions in combination with either the license_blacklist or blacklist arguments, and re-adds packages
that were excluded by a previous argument.
EXAMPLE: This example mirrors the entire Repository except any GPL2- or GPL3-licensed packages, but including
readline, despite the fact that it is GPL3-licensed:
license_blacklist:
- GPL2
- GPL3
whitelist:
- readline
You may find that combining two or more of the arguments above is the simplest way to get the exact combination of
packages that you want.
The platforms argument is evaluated before any other argument.
EXAMPLE: This example mirrors only linux-64 distributions of the dnspython, shapely and gdal packages:
platforms:
- linux-64
package_list:
- dnspython
- shapely
- gdal
If the license_blacklist and blacklist arguments are combined, the license_blacklist is evaluated first, and the blacklist
is a supplemental modifier.
EXAMPLE: In this example, the mirror configuration does not mirror GPL2-licensed packages. It does not mirror the
GPL3-licensed package PyQt because it has been blacklisted. It does mirror all other packages in Repository:
license_blacklist:
- GPL2
blacklist:
- pyqt
If the blacklist and whitelist arguments are both employed, the blacklist is evaluated first, with the whitelist functioning
as a modifier.
EXAMPLE: This example mirrors all packages in the repository except astropy and pygments. Despite being listed
on the blacklist, Accelerate is mirrored because it is listed on the whitelist:
blacklist:
- accelerate
- astropy
- pygments
whitelist:
- accelerate
To help ensure that a file was correctly uploaded or synced you can use the checksum tool. This routine fetches a file
from a database and verifies that the stored hash checksum and the calculated hash checksum of the file on disk are
the same.
On a package’s page, view the file list and click the Info button next to a file to see the file’s keyname.
To check the file’s hash checksum run:
anaconda-server-checksum keyname
Customizing installers
Anaconda Repository can distribute copies of the Anaconda distribution and the Miniconda installer that are pre-
configured to use your installation of Repository.
This applies to Anaconda Distribution version 4.1 and higher, and Miniconda version 4.1.11 and higher.
By default the installers will be stored in $PREFIX/opt/anaconda-server/installers. If you prefer to
store the installers in a different location, configure a new path:
NOTE: Replace “/preferred/directory” with the path to the directory where you prefer to store the installers.
If necessary, edit the script below and replace “5.2.0” with the current version number.
To download the installers:
mkdir -p /tmp/extras
pushd /tmp/extras
URL="https://repo.anaconda.com"
version="5.2.0"
miniconda="Miniconda3-latest-Linux-x86_64.sh \
Miniconda3-latest-MacOSX-x86_64.sh \
Miniconda3-latest-Windows-x86.exe \
Miniconda3-latest-Windows-x86_64.exe \
Miniconda-latest-Linux-x86_64.sh \
(continues on next page)
anaconda-server-admin update-installers
This command requires that the SERVER_NAME setting be set in order to generate the correct URLs.
By default, the included conda installation will point to the default anaconda and r-channel accounts on your
Anaconda Repository server, if those accounts exist.
You can change these default channels by setting the DEFAULT_CHANNELS setting, and then running the
anaconda-server-admin update-installers command.
Configuration reference
• Files
• Logging
• Usernames
– USER_REGEX
• Database
– MONGO_URL
– MONGO_DBNAME
– MQ_DBNAME
– MONGO_REPLICA_SET
• File storage
– Storage_type
– keyname_full_path
– Fs_storage_root
– PACKAGE_BUCKET_ID
– S3_REGION_NAME
– S3_SERVER_SIDE_ENCRYPTION
• Notebooks
– MAX_IPYNB_SIZE
• Web server
– SERVER_NAME
– port
– subdomains
– SESSION_COOKIE_DOMAIN
– USER_CONTENT_DOMAIN
– ssl_options
* certfile
* keyfile
* ssl_version
– PREFERRED_URL_SCHEME
– gunicorn
* timeout
* workers
• Authentication
– AUTH_TYPE
– KRB5_HOSTNAME
– KRB5_SERVICE_NAME
– KRB5_KTNAME
– LDAP
– LOCK_DOWN
• Email
– SMTP_HOST
– SMTP_PORT
– SMTP_TLS
– SMTP_USERNAME
– SMTP_PASSWORD
– USE_SES
– RETURN_ADDRESS
– ALLOW_DUPLICATED_EMAILS
– require_email_validation
• Advanced
– AVATAR_METHOD
– AVATAR_GRAVATAR_URL
– AVATAR_STATIC_URL
– CONSTRUCTOR_TIMEOUT
– CONSTRUCTOR_TOKEN_TIMEOUT
– CONSTRUCTOR_ALLOWED_OPTIONS
– PARCELS_ROOT
– PARCEL_DISTRO_SUFFIXES
– DEFAULT_CHANNELS
– CONSTRUCTOR_TMPDIR
– STANDARD_LABELS
– CONDA_CACHE_SIZE
– CACHE_METHOD
• REMEMBER_COOKIE_ENABLED
• PERMANENT_SESSION_LIFETIME
• REMEMBER_COOKIE_DURATION
• SUPERUSER_ORG_ADMIN
• NEXT_URL_WHITELIST
• NEXT_URL_WHITELIST_REGEXP
Files
Anaconda Repository loads configuration files with the extension .yaml from the following locations:
• /etc/binstar/
• /etc/anaconda-server/
• $PREFIX/etc/anaconda-server
VALUE_ONE: 50
Or, you can set a value named VALUE_ONE to 50 with this command:
Logging
The location of the server’s log file is defined in the supervisord configuration file $PREFIX/etc/supervisord.
conf by the stdout_logfile config entry located in the [program:anaconda-server] section.
Advanced configuration of logging requires setting a LOGGING key on the server’s config.yaml. It uses Python’s
logging module config structure.
Usernames
USER_REGEX
USER_REGEX: '^[a-z.+-]+$'
NOTE: The default value for USER_REGEX is ^[a-z0-9_][a-z0-9_-]+$ which translates to: at least one
alphanumeric character or underscore, followed by zero or more alphanumeric, dash or underscore characters.
NOTE: Escape any extra instances of the single quote character ' as \'. Do not use the slash and ampersand characters
/ and &, which have special meanings in URLs.
NOTE: If USER_REGEX is changed and the server is restarted, existing usernames that do not match the new
USER_REGEX do not cause errors.
Database
MONGO_URL
A MongoDB connection URI is used to connect to the MongoDB database server. It can be used to configure the
hostname and port, as well as database authentication.
For example:
MONGO_URL: mongodb://anaconda-server:[email protected]/
MONGO_DBNAME
MQ_DBNAME
The MongoDB database where Repository stores data used for asynchronous processing.
MONGO_REPLICA_SET
The name of a MongoDB replica set Repository connects to after establishing a connection to the database server.
File storage
Repository can serve package contents from a local file-system, or from Amazon Web Services Simple Storage Ser-
vice: AWS S3.
Storage_type
The storage mechanism to use. Valid choices are fs, for file-system storage, or s3, for AWS S3 storage.
keyname_full_path
When this option is set, Repository stores the files by full paths and not just by hashes. This way a tensor-
flow file uploaded by the user Bob will be stored on <fs_storage_root>/Bob/tensorflow/osx-64/
tensorflow-1.1.0-np112py36_0.tar.bz2-594ac56e7e042600648defdb.
NOTE: The storage path does not always contain the current file owner and their user name. This is because the file
location on the storage does not change when you rename a user or transfer a file to a different user.
Fs_storage_root
If configured to use file-system storage, the absolute path to a directory where Repository stores all uploaded packages.
PACKAGE_BUCKET_ID
If configured to use AWS S3 storage, the name of an AWS S3 bucket where Repository stores uploaded packages.
You can identify the name of your bucket by using <bucket> in your http://<bucket>.s3.amazonaws.
com URL.
S3_REGION_NAME
The S3 region that the bucket is located in. The available regions can be found in the Amazon AWS documentation.
S3_SERVER_SIDE_ENCRYPTION
This variable can be set to AES256 to enable server-side encryption for packages stored in the S3 bucket.
Notebooks
MAX_IPYNB_SIZE
Specifies the maximum allowed size when uploading notebooks to the server. The default is 25 MB. This variable can
be set in config.yaml.
Web server
SERVER_NAME
The name and port number of the server. This option is required for subdomain support.
For example:
SERVER_NAME: anaconda.srv:8080
port
subdomains
If set to true, Repository serves conda package from a separate subdomain. Defaults to false.
For example:
SERVER_NAME: anaconda.srv:8080
subdomains: true
SESSION_COOKIE_DOMAIN
The domain that Repository sets on the session cookie. If this is not set, the cookie is valid for all subdomains of
SERVER_NAME.
See Securing user-created content.
USER_CONTENT_DOMAIN
As a cross-site scripting (XSS) protection, notebook content can be served from a separate domain name. If this option
is configured, Repository only serves rendered notebooks from this domain.
See Securing user-created content.
ssl_options
Repository can serve content over HTTPS, using user-provided SSL certificates.
For example:
ssl_options:
certfile: /etc/anaconda-server/server.crt
keyfile: /etc/anaconda-server/server.key
PREFERRED_URL_SCHEME: https
certfile
keyfile
The absolute path to a PEM-formatted private key for the associated certificate.
ssl_version
An integer that specifies the SSL protocol version as defined by Python’s ssl module:
PROTOCOL_SSLv2 = 0
PROTOCOL_SSLv23 = 2
PROTOCOL_SSLv3 = 1
PROTOCOL_TLS = 2
PROTOCOL_TLSv1 = 3
PROTOCOL_TLSv1_1 = 4
PROTOCOL_TLSv1_2 = 5
PREFERRED_URL_SCHEME
The preferred scheme that is used to generate URLs. Set this to https if HTTPS is configured.
gunicorn
Repository uses Gunicorn. The most commonly used options are timeout and workers. A complete list of settings
can be found in Gunicorn’s documentation.
For example:
gunicorn:
timeout: 60
workers: 5
timeout
The number of seconds for which a worker is allowed to process a request, before being forcefully terminated.
Default: 120
workers
The number of workers that Gunicorn spawns to serve Repository. Defaults to 2 × the number of CPUs + 1.
Authentication
AUTH_TYPE
The method Repository uses to authenticate users. Valid choices are NATIVE, for built-in authentication, KERBEROS,
for Kerberos, and LDAP.
KRB5_HOSTNAME
KRB5_SERVICE_NAME
KRB5_KTNAME
LDAP
LDAP:
# Replace with company LDAP server
URI: 'ldap://<ldap.company.com>'
# Replace <uid=%(username)s,ou=People,dc=company,dc=com> with your company specific
˓→LDAP Bind/Base DN
USER_SEARCH:
base: cn=Users,dc=example,dc=com
filter: sAMAccountName=%(username)s
OPTIONS:
OPT_NETWORK_TIMEOUT: 60
OPT_TIMEOUT: 60
NOTE: To use LDAP with SSL, set the USER_REGEX and account_names_filter options:
account_names_filter: false
USER_REGEX: ^[a-z0-9_][a-z0-9_-.]+$
LDAP:
[configuration continues as above with URI, BIND_DN, and so on]
LOCK_DOWN
Makes all views with the exception of the login form and welcome page, unaccessible to anonymous users.
Repository can be configured to send email for various reasons, including to reset forgotten usernames and passwords.
Email can be sent using SMTP protocol, or through Amazon Web Services Simple Email Service (AWS SES).
SMTP_HOST
SMTP_PORT
SMTP_TLS
SMTP_USERNAME
The username to authenticate against the SMTP server before attempting to send email.
SMTP_PASSWORD
The password to authenticate against the SMTP server before attempting to send email.
USE_SES
If set to true, Repository sends email with AWS SES. To authenticate to AWS, the server should be configured with
an appropriate IAM role, or have credentials specified in a Boto configuration file.
RETURN_ADDRESS
ALLOW_DUPLICATED_EMAILS
If set to true, Repository allows different users to share the same email or secondary email. Defaults to false.
require_email_validation
If set to true, Repository emails new users a unique token to validate their email address before permitting them to
log in.
Advanced
AVATAR_METHOD
The method to use to generate the user avatar URL. Valid choices are:
• ‘gravatar’ to use the gravatar.com service
• ‘default’ to show a predefined static icon
• ‘static’ to use a custom static URL
AVATAR_GRAVATAR_URL
A URL for a Gravatar compatible service. Default: https://www.gravatar.com/. This URL is used as the
prefix to build a valid gravatar URL.
AVATAR_STATIC_URL
A static URL to use when AVATAR_METHOD is set to static. Defaults to an empty string.
CONSTRUCTOR_TIMEOUT
The timeout in seconds for the call to constructor while building installers, parcels and management packs.
Defaults to 60 seconds.
CONSTRUCTOR_TOKEN_TIMEOUT
To provide access to private packages while building an installer, a temporary token is created. It must be valid during
the call to constructor and it should expire soon after the call completes. CONSTRUCTOR_TOKEN_TIMEOUT
sets the token’s valid lifetime in seconds. Defaults to 60 seconds. This value should be greater than or equal to
CONSTRUCTOR_TIMEOUT.
CONSTRUCTOR_ALLOWED_OPTIONS
A list of constructor option names that are allowed to be included in the installer construction form. The default
is [] (no options are allowed).
PARCELS_ROOT
The prefix with which Cloudera parcels are generated. Defaults to /opt/cloudera/parcels.
PARCEL_DISTRO_SUFFIXES
The distributions for which Cloudera parcels are generated. Defaults to ['el5', 'el6', 'el7', 'lucid',
'precise', 'trusty', 'wheezy', 'jessie', 'squeeze', 'sles11', 'sles12'].
For example, if you want to support only Ubuntu:
PARCEL_DISTRO_SUFFIXES:
- lucid
- precise
- trusty
DEFAULT_CHANNELS
The Repository accounts that environments installed with the bundled Anaconda distributions pull packages from.
Defaults to ['anaconda', 'r-channel'].
For example, to add an additional custom account:
DEFAULT_CHANNELS:
- anaconda
- r-channel
- custom
CONSTRUCTOR_TMPDIR
When constructor builds an installer it stores the configuration in this temporary directory. The default is None,
which tells constructor to create a temporary directory using Python’s tempfile.mkdtemp.
STANDARD_LABELS
A list of standarized labels. If a user defines a label that is not listed as standard, a warning notice will be shown in the
package’s page. Defaults to ['main', 'dev', 'alpha', 'beta', 'broken'].
CONDA_CACHE_SIZE
The maximum size (in bytes) of the repodata.json requests cache. Set to 0 to disable repodata.json caching.
Default: 1 Gb. When the maximum size is reached, the 10 least recently used entries of the cache are evicted.
CACHE_METHOD
The method used for caching repodata info. It can either be tempfile (the prior method of caching) or diskcache,
which uses SQLite as a back-end. Default: diskcache.
REMEMBER_COOKIE_ENABLED
Sets whether to use the remember me cookie to keep the session alive. If it’s set to
true the REMEMBER_COOKIE_DURATION setting is relevant, and if it’s set to false, the
PERMANENT_SESSION_LIFETIME is relevant. Defaults to true.
PERMANENT_SESSION_LIFETIME
An integer that sets how many minutes the session will live. Only used when REMEMBER_COOKIE_ENABLED is
false. Default is 44640 (31 days).
REMEMBER_COOKIE_DURATION
An integer that sets how many minutes the session will live when using the remember me cookie. Only used when
REMEMBER_COOKIE_ENABLED is true. Default is 525600 (365 days).
SUPERUSER_ORG_ADMIN
Whether superusers should automatically be granted admin rights on organizations. Default is false.
NEXT_URL_WHITELIST
List of hostnames that are marked as safe when redirecting requests due to the presence of a “next” request parameter.
It is mainly used under an Anaconda Enterprise Notebooks Single Sign-on Set-up. The default is [] (no external
redirects are safe).
NEXT_URL_WHITELIST_REGEXP
A regular expression to match hostnames that are marked as safe when redirecting requests due to the presence of
a “next” request parameter. It is mainly used under an Anaconda Enterprise Notebooks Single Sign-on Set-up. The
default is ‘(?!)’ which matches nothing, so only local redirects are allowed.
Repository has two installation options:
• Online: If you have internet access on the destination server, follow the online instructions.
• Air gap: If you have an air gapped system or the destination server does not have internet access, follow the air
gap instructions.
Repository provides advanced configuration options that can be used to meet site-specific needs.
Repository includes a number of optional components that can be installed and used individually.
You may also want to see Updating Repository and Uninstalling Repository.
User management
• Adding a user
• Searching for users
• Promoting an existing user
• Resetting user passwords
• Resending welcome emails to new users
• Changing a user’s storage size or changing their plan to free unlimited
• Removing a user
Adding a user
New users can navigate in a browser to your local Repository web page and sign themselves up for an account, or you
can add them using the command line:
1. Set the USER_PASSWORD environment variable (e.g., export USER_PASSWORD=abc123DEF).
2. Run the anaconda-server-create-user command, with the following syntax:
EXAMPLE:
anaconda-server-create-user --username jsmith --email [email protected]
--superuser
As of Anaconda Repository version 2.33.5, you can search for users by username or email address. Open this URL in
a browser:
http://your.anaconda.server:port/admin/users
NOTE: Replace “your.anaconda.server:port” with the name or IP address and port of your Anaconda server.
As of Anaconda Repository version 2.28, administrators can use the Administration Accounts page to promote users
to staff or superuser status.
The staff user and superuser can view the following sections of the Administration page:
• Reports.
• User administration.
• License downloads.
• Trial licenses.
• Current LDAP configuration.
• Current server configuration.
• Downloads summary.
• Downloads from a specific address.
• Security feed and security feed details.
• Storage administration.
In addition to all of the above, a superuser can also view the following on the Administration page:
• Plans.
• User details.
To promote an existing user to a staff user or superuser:
3. Click the Set Staff button to give the user staff privileges or click the Set Superuser button to give the user
superuser privileges.
4. In the dialog box that appears, retype the user’s name.
5. Click the Set button.
If a user forgets their password, you can request a reset link to provide to the user.
To send emails, Repository must have the email settings configured.
1. Log into your Repository administrative account.
2. From the top Tools menu, select Admin.
3. From the left navigation pane, select Password Reset.
4. Enter the user’s email address.
The Web UI generates a password reset link.
5. Email the link to the user.
You can also reset passwords without sending emails:
NOTE: Replace jsmith with the username whose password you want to reset.
Removing a user
System management
Recommended workflow
One of the most useful features of Anaconda Repository is its ability to help manage package development and
deployment in a seamless fashion. This page describes the development process and channel usage employed by
one of our internal teams, to serve as an example of how you can leverage channels for workflow separation.
Multiple channels allow our team to maintain separate package states and easily earmark and control the versions and
states of packages that users can install.
Our team created the following channels:
• Master.
• Staging.
• Release.
We have used this workflow through 4 release cycles and it has worked out well for us.
Master
A master is created any time something is merged into our master branch. It is considered the development build of
all of the components that make up the software. Code that makes it to this channel should be stable and should have
been confirmed independently, but a full QA test has not been run on it yet.
Staging
Once we are ready to start working on a release, we create a staging:X.Y.Z branch. This contains all code that is going
to go into a release. No new features should be introduced at this point, just any last minute bug fixes to existing code.
Release
The staging channel gets culled so that only the latest package is maintained in it. Any alpha, beta, or dev packages
are removed. After all testing is complete, all issues are resolved, and the channel contains only one version of each
package, we copy that package into a release:X.Y.Z channel, then lock that channel.
This guide is for backing up and restoring an Anaconda Repository instance that uses local file system storage. If your
instance uses Amazon S3 or any other storage provider, please consult their specific documentation on backup and
restore procedures.
If any of these items are different for your instance, modify these instructions accordingly.
• Unless noted, run all shell commands while logged in as the anaconda-server user. Using sudo privileges,
log in as the anaconda-server user with this command:
sudo su - anaconda-server
$ pwd
/home/anaconda-server
Backup
Before starting the backup process, shut the service down using supervisorctl:
Make a $VERSION environment variable and set it to the version of the currently installed Anaconda Repository:
$ echo $VERSION
2.33.27
$ TIMESTAMP=`date +%Y-%m-%d`
$ echo $TIMESTAMP
2018-07-30
Code/Binaries
Generate a tarfile archive with the installed code, binaries and any dependencies:
Notice that this also generated a SHA1 checksum. This checksum will be used to verify when you restore the archive.
Configuration
This step is necessary only if you stored Anaconda Repository’s configuration in a custom location outside of the
instance installation folder (usually /home/anaconda-server/etc/).
These commands show how to generate the tarfile if the configuration is stored in /etc/anaconda-server.
Generate the tarfile with its SHA1 checksum:
Storage
As before, create a tarfile archive and its checksum with the contents of the package storage location:
Database
Generate a dump of Anaconda Repository’s MongoDB database. We recommend you follow MongoDB’s guidelines
for backup and restore. This guide uses MongoDB tools:
.bashrc
If you chose to let the Anaconda Repository installer update the .bashrc file of the user anaconda-server, back
it up:
$ cp /home/anaconda-server/.bashrc anaconda-server-bashrc-$VERSION-$TIMESTAMP.sh
$ sha1sum anaconda-server-bashrc-$VERSION-$TIMESTAMP.sh > anaconda-server-bashrc-
˓→$VERSION-$TIMESTAMP.sh.sha1
Restore
• Verify that the restore environment meets the requirements listed in the Installation Guide for Anaconda Repos-
itory. You will need:
– MongoDB (any supported version) installed
– A user account (usually anaconda-server)
– A storage directory (usually /opt/anaconda-server/package-storage) owned by the Ana-
conda Repository user account. This is only needed if you’re using a local filesystem as a storage backend.
• Run all shell commands while logged in as the anaconda-server user, as you did when backing up Ana-
conda Repository. Using sudo privileges, log in as the anaconda-server user with this command:
sudo su - anaconda-server
Verify checksums
.bashrc
If you backed up the .bashrc file of the user anaconda-server, restore it:
cp anaconda-server-bashrc-$VERSION-$TIMESTAMP.sh /home/anaconda-server/.bashrc
After restoring this file, log out and log in as anaconda-server again for the changes to take effect.
Database
If you followed the Anaconda Repository Installation Guide, MongoDB is up and running and you can use
mongorestore to restore the database archive:
Storage
Code/Binaries
repo/bin/anaconda-server-install-supervisord-config.sh
The server should now be up and running. Check the status with supervisorctl:
$ repo/bin/supervisorctl status
anaconda-server RUNNING pid 8446, uptime 0:03:18
You can use the “orphan-check” tool to resynchronize the filesystem and the database if the filesystem and the database
get out of sync.
The system can get out of sync when files in the filesystem are not referenced from the database, or when packages in
the database do not have a corresponding file in the filesystem.
The orphan-check tool prints on stdout a list of files on the filesystem that are not referenced from the database:
anaconda-server-orphan-check --dryrun
You can use the –json option if you want a JSON representation of the output:
anaconda-server-orphan-check --json
anaconda-server-orphan-check --clean
You can also check for packages that have missing files:
anaconda-server-orphan-check --reverse
Then you can delete those file objects from the database:
Anaconda Repository includes a number of components that can be installed and used individually.
This section describes how to install and use two such tools:
• cas-mirror.
• cas-installer.
As a convention, all packages and commands that are part of the Repository product share the common cas prefix,
which is short for Continuum Anaconda Server. (Anaconda, Inc. was formerly known as Continuum Analytics, Inc.)
All packages are installed using the conda command, which is part of the Miniconda installer. For Repository
installation and configuration instructions, see Installation.
Using cas-mirror
Installing cas-mirror
After cas-mirror has been installed, the following commands are available:
cas-sync --help
cas-merge --help
cas-sync-api-v4 --help
cas-server --help
The cas-sync command brings the local mirror of Repository up-to-date with our remote servers.
To configure the location of the mirror on your file system, check the output of:
cas-sync --config
If necessary, create a configuration file, either ~/.cas-mirror or system-wise /etc/cas-mirror, which con-
tains the desired location of the local mirror on the file system, the platforms that should be mirrored and an optional
blacklist of packages that which should not be mirrored.
EXAMPLE:
mirror_dir: /home/data/mirror
remote_url: "" # where to get miniconda and anaconda installers -- blank to skip
# possible platforms are: linux-64, linux-32, osx-64, win-32, win-64 platforms:
- linux-64
- win-32
blacklist:
- dnspython
- shapely
- gdal
Once you are satisfied with the mirror directory—which may be the default—run:
cas-sync
Running this command for the first time takes many hours, because the entire Repository is being downloaded. Sub-
sequent runs take significantly less time.
You need to run cas-server as root when you intend to serve on port 80.
To serve repository over HTTP, run:
cas-server
If needed, use the --port option to change the port on which the repository is being served.
If you’ve already downloaded most of the anaconda repository, and you’re only interested in the changes since
cas-sync was last run, you can use the delta configuration option:
mirror_dir: /home/data/mirror
remote_url: "" # where to get miniconda and anaconda installers -- blank to skip
# possible platforms are: linux-64, linux-32, osx-64, win-32, win-64
platforms:
- linux-64
- win-32
blacklist:
- dnspython
- shapely
- gdal
delta: true
delta_dir: delta_pkgs
Instead of mirroring to the existing local repository, it will record the necessary changes to bring the mirror up to date
in a separate directory (delta_pkgs in this case). You can then use this generated directory to update air-gapped
mirrors using the cas-merge command.
The cas-merge command takes a delta directory and combines its contents with an existing mirror directory. New
packages are added, missing packages are deleted, and the repodata is updated.
If instead of mirroring to a local directory, you want to make the changes directly into an existing Anaconda Repository
instance, the cas-sync-api-v4 can be used. You’ll need to use the dest_site config option:
mirror_dir: /home/data/mirror
remote_url: "" # where to get miniconda and anaconda installers -- blank to skip
# possible platforms are: linux-64, linux-32, osx-64, win-32, win-64
platforms:
- linux-64
- win-32
blacklist:
- dnspython
- shapely
- gdal
dest_site: some_site
Make sure that the site is defined in the anaconda config and you’re properly logged into it before invoking
cas-sync-api-v4.
More extensive information about the cas-mirror tool’s functionality and configurable options is available at Customiz-
ing mirrors.
Using cas-installer
The cas-installer tool makes an environment installer, which is a bash script or Windows executable file that can be
run on any machine to install an exact copy of a conda environment and its packages on that machine.
Installing cas-installer
A token from Anaconda is required to install cas-installer, and you should have received it when your organization
purchased Repository, Workgroup or Enterprise. If you no longer have access to your token, submit a support ticket
or contact us at Enterprise Support. You can also email support at the email address given to you by your sales
representative.
When you have the token, run:
export TOKEN=<your_anaconda_cloud-token>
conda config --add channels https://conda.anaconda.org/t/$TOKEN/anaconda-server
Because this tool allows you to create an installer for a conda environment, it is important that the cas-installer package
is installed into the root conda environment, not root user. The following command ensures that this happens:
cas-installer -h
The command takes an installer specification file as its argument, which specifies the name of the installer, the conda
channel to pull packages from, the conda packages included in the installer, and so on.
EXAMPLE:
# specifications
specs:
- python
- grin
# The conda default channels which are used when running a conda which
# was installed be the cas-installer created: requires conda---3.6.2 or
# greater---in the specifications. The *channels is a YAML reference to
# &channels above. It inserts all the channels from the channels key, so
# that they do not have to be typed twice.
conda_default_channels: *channels
# installer filename
# installer_filename: grin.sh
For Windows, the tool creates nsis-based .exe installers, which can only be created on a Windows platform, although
the architecture may be different. For Unix, the tool creates bash-based .sh installer, which can only be created on
Unix—Linux or macOS—systems.
Updating Repository
CAUTION: You must have a tested backup of your installation before starting the update process. If updating more
than one version, all updates must be performed in sequential order.
Your support representative can provide you with a download URL for an updated Repository installer.
To update to the latest Repository release:
NOTE: To use a Repository version from 2.33.3 through 2.33.10 and Anaconda Enterprise Notebooks with single sign-
on (SSO), you must set USE_SERVER_BASED_SESSIONS: false in the Repository configuration. This setting
affects the network security properties of AEN and Repository. Specifically, if USE_SERVER_BASED_SESSIONS
is set to false, and if a new cross-site scripting (XSS) vulnerability is discovered, it could expose an additional server
fixation vulnerability. Please discuss this with your Anaconda representative and be sure the feature is compatible with
your network requirements before setting USE_SERVER_BASED_SESSIONS: false.
NOTE: As of Repository 2.33.8, the fs_storage_root configuration setting is mandatory for local filesystem
storage and the Repository server will not run without it. You can set it with this command:
Updating to 2.33:
NOTE: To use a Repository version from 2.33.3 through 2.33.10 and Anaconda Enterprise Notebooks with single sign-
on (SSO), you must set USE_SERVER_BASED_SESSIONS: false in the Repository configuration. This setting
affects the network security properties of AEN and Repository. Specifically, if USE_SERVER_BASED_SESSIONS
is set to false, and if a new cross-site scripting (XSS) vulnerability is discovered, it could expose an additional server
fixation vulnerability. Please discuss this with your Anaconda representative and be sure the feature is compatible with
your network requirements before setting USE_SERVER_BASED_SESSIONS: false.
NOTE: As of Repository 2.33.8, the fs_storage_root configuration setting is mandatory for local filesystem
storage and the Repository server will not run without it. You can set it with this command:
anaconda-server-config --set fs_storage_root /opt/anaconda-server/package-storage
Updating to 2.31:
curl '$INSTALLER_URL' > anaconda_repository.sh
bash anaconda_repository.sh -u
anaconda-server-db-setup --execute
supervisorctl stop all
supervisorctl reload
supervisorctl start all
Updating to 2.30:
curl '$INSTALLER_URL' > anaconda_repository.sh
bash anaconda_repository.sh -u
anaconda-server-db-setup --execute
supervisorctl stop all
supervisorctl reload
supervisorctl start all
Updating to 2.29:
curl '$INSTALLER_URL' > anaconda_repository.sh
bash anaconda_repository.sh -u
anaconda-server-db-setup --execute
supervisorctl stop all
supervisorctl reload
supervisorctl start all
Updating to 2.28:
curl '$INSTALLER_URL' > anaconda_repository.sh
bash anaconda_repository.sh -u
anaconda-server-db-setup --execute
supervisorctl stop all
supervisorctl reload
supervisorctl start all
Updating to 2.27:
curl '$INSTALLER_URL' > anaconda_repository.sh
bash anaconda_repository.sh -u
anaconda-server-db-setup --execute
(continues on next page)
Updating to 2.26.6:
The logging configuration can be removed. Logs have been moved to $PREFIX/var/log/anaconda-server/:
Updating to 2.26:
Updating to 2.25:
Updating to 2.24:
Updating to 2.23:
Updating to 2.22:
Updating to 2.21:
conda update binstar-server binstar-static anaconda-client
anaconda-server-db-setup --execute
supervisorctl stop all
supervisorctl reload
supervisorctl start all
Updating to 2.20:
conda update binstar-server binstar-static anaconda-client
anaconda-server-db-setup --execute
supervisorctl stop all
supervisorctl reload
supervisorctl start all
Updating to 2.19:
conda update binstar-server anaconda-client anaconda-build
anaconda-server-db-setup --execute
supervisorctl stop all
supervisorctl reload
supervisorctl start all
Updating to 2.18:
conda update binstar-server anaconda-client anaconda-build
anaconda-server-db-setup --execute
supervisorctl stop
supervisorctl reload
supervisorctl start all
Updating to 2.17:
conda update binstar-server anaconda-client anaconda-build
anaconda-server-db-setup --execute
supervisorctl stop
supervisorctl reload
supervisorctl start all
Updating to 2.16:
conda update binstar-server anaconda-client anaconda-build
anaconda-server-db-setup --execute
supervisorctl restart all
Updating to 2.15:
conda update binstar-server anaconda-client anaconda-build
anaconda-server-db-setup --execute
supervisorctl restart all
Updating to 2.14:
conda update binstar-server anaconda-client anaconda-build
anaconda-server-db-setup --execute
supervisorctl restart all
Updating to 2.13:
Updating to 2.12:
Updating to 2.9:
Updating to 2.8:
Updating to 2.6.0:
Updating to 2.5.1:
Updating to 2.3:
Updating to 2.2:
Uninstalling Repository
Before deleting Repository, you may want to make a backup for security reasons. For suggestions on mongo backups,
see https://docs.mongodb.org/manual/reference/program/mongodump/ .
To delete Repository:
1. Check the file storage path:
rm -rf /home/anaconda-server/repo
rm -rf /etc/binstar
Troubleshooting
This page provides instructions for troubleshooting issues that may occur with your Anaconda Repository installation.
This could be because you are behind a firewall. Check if your IPTables rules are blocking your ports:
iptables -L -n
If a rule blocks a port you want to use, then you must allow the port:
If Anaconda Client is not yet installed and you try to search for a package on Anaconda.org using the anaconda
command, you may receive the following error message:
This error occurs because the Windows version of Anaconda contains an anaconda.bat file that is used for setting
environment paths and switching environments. If Client is not installed, this batch file is called instead. Once you
install Client, the Anaconda search command will work:
When configuring Client to connect to a Repository behind a reverse proxy, the anaconda upload command may
appear to try connecting to the internal hostname rather than the external configured one.
This can be corrected in the settings of the reverse proxy, such as NGINX or Apache.
In NGINX, add the setting proxy_set_header Host $host; to access the internal host with the external
hostname.
In Apache, turn on the option ProxyPreserveHost.
Other reverse proxies each have their own settings to handle hostnames correctly.
EXAMPLE: Some other reverse proxies use a settings syntax such as
http_proxy=id:passwd@proxyhost:port.
Repository should normally be started as a daemon. For troubleshooting, it can instead be started as a foreground
process on a specified port:
Administrative commands
Many of these actions can be done in the web interface. This command reference is for those administrators who
prefer to use command line shortcuts.
In all examples below, replace “jsmith” with the name of the user whose settings you wish to change.
Reset a user’s password interactively:
The above command will prompt you to enter the new password twice. You may also reset the password directly:
Set all users with a given email domain to a free and unlimited plan:
You can do a “dry run” of the command to display what the command will do without changing anything:
NOTE: Replace “yourdomain.com” with the domain whose users you wish to upgrade.
Give the user the privileges of a superuser or remove them:
NOTE: Replace “old_name” with the current username, and “new_name” with the new username.
Ensure the files recorded in the database exist.
anaconda-server-admin verify-storage
Ensure that files recorded in the database exist and have the correct checksum:
anaconda-server-admin clean-storage
anaconda-server-admin update-installers
Delete a user:
Convert a regular user account to an organization and add another user to the owners group:
FAQs
• What is Anaconda?
• How do I get started with Repository?
• What is an organization account, and how is it different from an individual account?
• Who can upload packages to an organization?
Anaconda Repository is package management server software that makes it easy to find, access, store and share public
and private notebooks, projects, installers, environments, and conda and PyPI packages. Repository also makes it easy
to stay current with updates made to the packages and environments you are using.
Many enterprises have customized local instances of Repository. Anaconda also makes an instance of Repository
available for public use at Anaconda Cloud.
Repository supports any type of package. It is primarily used for conda, PyPI and R packages, as well as notebooks
and environments.
What is Anaconda?
Anaconda is a software development and consulting company of passionate, open source advocates based in Austin,
Texas, USA. We are committed to the open source community. We created the Anaconda Python distribution and
contribute to many other open source-based data analytics tools. You can find out more about us by reading our story.
If you have access to Repository, you can search, download and install hundreds of public packages without having an
account.
If you want to upload packages to Repository, you need to sign up for a Repository account, get Anaconda and the
Anaconda Client. For more information, see Creating an account or ask your system administrator.
An organization account allows multiple individual users to administer packages and have more control over package
access by other users. An individual account is for use by one person.
Only users who are co-owners of an organization may upload packages to that organization. Administrators who are
not co-owners cannot upload packages to the organization. Users who are members of groups with read/write access
but who are not co-owners, cannot upload packages to the organization.
Your organization receives Professional Support with your purchase of Anaconda Repository. Please contact your
system administrator for help.
You are also welcome to join our community support mailing lists for both Anaconda and conda. On these lists you
can ask questions, answer questions and discuss ways to use Anaconda. You can also submit requests for new features
and make any other comments you may have.
Note that the community support forums cannot provide Anaconda Repository support.
Reporting a bug
Issues with Repository are tracked on GitHub. If you think you have found a bug, search to see if it has been reported,
and report it if no one else has.
Release notes
The Anaconda Repository 2.33 release is available to all Anaconda Repository customers as of September 19, 2017.
NOTE: If you have a subscription but do not have a license, contact support to receive that license. Otherwise contact
sales to acquire it.
Administrators can update to the new Anaconda Repository release as described in Updating Repository.
Please contact your enterprise support representative if you have any questions or problems regarding the release.
Changelog
2.33.27 - 2018-07-30
2.33.26 - 2018-07-19
2.33.25 - 2018-07-19
2.33.24 - 2018-07-03
2.33.23 - 2018-05-23
2.33.22 - 2018-05-21
2.33.21 - 2018-05-03
2.33.20 - 2018-04-24
2.33.19 - 2018-04-19
2.33.18 - 2018-04-03
2.33.17 - 2018-03-09
2.33.16 - 2018-03-08
2.33.15 - 2018-02-27
2.33.14 - 2018-02-20
2.33.13 - 2018-02-19
2.33.12 - 2018-02-07
2.33.11 - 2018-02-06
• A warning message is now shown when all packages are added to a given group
Admin facing changes
• Disable password reset admin option while using LDAP
Non visible changes
• Added proper HEAD support on download endpoints
• Disabled USE_SERVER_BASED_SESSIONS by default
• Disabled database based settings
• Strengthened validation of labels
• Fixed popups for operations when no files or packages are selected
• Fixed deletion of files by collaborators
• Fixed access to static content while on LOCK_DOWN
• Fixed transfer of ownership of items between orgs
2.33.10 - 2018-01-19
2.33.9 - 2018-01-16
2.33.8 - 2018-01-15
2.33.7 - 2017-12-11
2.33.6 - 2017-11-27
Added
• Filter for authenticated packages in search/favorites view
• Updated message for input field when copying label
• Updated support links
• Added email notification when group member is added
• Installation info for R and pypi packages using labels
• Support for defining standard labels
• Added support to remove user using anaconda-server-admin
• Show licence url for packages, environments and notebooks
• Validation for empty fields in credit card info for plan upgrade
Fixed
• Updated instructions to create initial user
• Prevent organizations to have admin access for ownables of its original user
2.33.5 - 2017-11-07
Added
• Added a CLI tool to manage group membership
• Missing tooltips on header and admin section
• Added some missing feed items
• Account search now uses both names and emails
• Filter for authenticated packages
Fixed
• File info modal now works with list attributes that contain dicts
• Validate uploaded environment name
• Improved UX of installer creation form
• Updated feed icon for group collaboration removal
• Show all feed items related to a particular ownable in the History tab for that ownable
• Added main channel to default mirror config
• Generic exceptions during LDAP auth are now caught and logged
• Fixed supervisord script crontab option
• Updated read only rules on projects and installers
• Disable autocomplete suggestions for confirmation input fields
• Added authenticated packages to the billing package limit notice
• Add quotes around conda install help message if label has spaces
• Specify correct package type on tooltip text for label removal
• Updated flask-login-ldap dependency
• Validate name of copied label
• Removed validation of label name on deletion
• Removed duplicate HTTP headers on cached responses
• Do not allow pypi packages in installers
• Updated mirroring docs
• Make installers/projects summary optional
• Replaced some occurrences of word kapsel
• Align upvote icon
• Set package access from packages list
• Only owners can upload installers/environments to its own channel
2.33.4 - 2017-10-24
Added
• Use environment variable to set initial user’s password
• Usernames blacklist
• Show projects and installers summaries on header
• Added tooltips to package page buttons
• Instructions to generate tokens for organizations
• Feed items for projects and installers
• Settings for session timeout
• Supervisor script creates folder for extra config
• Updated EULA
• Set private packages and storage to unlimited individually
• Added progress indicator on installer upload
• Command to mirror only latest versions of conda packages
Fixed
2.33.3 - 2017-10-20
Added
• Added support for server based sessions
2.33.2 - 2017-10-10
Fixed
• Exception in admin after updating private packages for a user
2.33.1 - 2017-10-03
Added
• New feed items for group membership and groups collaborations
• Download stats for files API endpoint
• Option to set amount of private packages for a user from admin
• Improved license creation page
• Added distribution_types to downloads feed
• Set packages access as authenticated from packages list
• Added option –authenticated to anaconda-server-sync-conda
• Added conda-build as dependency
• Relaxed expired tokens restrictions for public endpoints
• Add organizations as collaborators for packages, environments and notebooks
• Send email when adding collaborator to a project/installer
Fixed
• Link to docs in packages view
• Catch all exceptions raised when loading environment file
• Return json responses on api calls when an error is encountered
• Error message when uploading an invalid installer file
• Group permissions moved to the settings
• Fix wrong autocomplete using firefox
• Fix typeahead initial suggestions in installers form
• Updated callout in contact us form
• Hiding package access settings for collaborators
• Fixed refresh when closing user menu on the navbar
• Show info about installers downloads on admin interface
• Fixed downloads stats on admin
• Prevent adding package owner as package collaborator
• Storing package_type when API package upload
• Collaborators can now access a package’s history page
• API docs are back up again
• Refactored mirroring tools
2.33.0 - 2017-09-19
Added
• SUPERUSER_SEARCH to set superuser status in LDAP
• File format validation on installer upload
• Show which users are admin in users list
• Use simillar settings for typeahead package suggestions
• Require user to be logged in to see user typeahead suggestions
• Cleaned output from test suite
• READ_ONLY mode setting and admin option
• Added tooltips to social media icons on footer
• Hide license download buttons from add ons page
• PAM authentication support
• Added reCAPTCHA to contact us form
Fixed
• Exception in group collaborations list for a package after group delete
• Maintain consistency in redirections after item deletion
• Fix group link in project collaborator view
• Flash message after issues with email validation
• Exception on admin downloads list for a user when package/file was had no owner
• Change dashboard user on item transfer
• Prevent adding current user as a collaborator
• Remove current owner from items ownership transfer options
• Fixed UI issues
• Validate name and version of installers only when full form is submitted
• Fixed some redirections to documentation in Anaconda Cloud
• Hiding delete package for collaborators
• Updated links to slideshare and youtube accounts
2.32.9 - 2017-09-15
Fixed
• Fixed forgot password link
2.32.8 - 2017-09-11
Fixed
• Temporarily disabled contact page for anaconda cloud
2.32.7 - 2017-09-07
Fixed
• Exception in token expiration warning code
• Fixed error when displaying a group that no longer exists
2.32.6 - 2017-09-06
Added
• Separated package groups collaborations in three tabs (packages, notebooks, environments)
• Added icons to all feed items
• Provided more info on feed for uploaded packages/environments/notebooks
• Unicode validation on signup form
• Package api returns builds and adds filter for search platform
• Validation for profile name
• Added option to upload all packages to a group at once
• Added page to see feed for a user
• Added some reserved names for packages
• Semantic versions validation for installers version field
• Warning header when token is about to expire
• Make favorites page public
Fixed
• Allowing anaconda login under lockdown
• Fixed potential exceptions on old cache code
• Changed typeahead environment query to use dashboard user
• Fixed issues with the upload of previously deleted installers
• Fixed Cache-Control headers on old repodata caching code
• Display all packages by default, not just only conda packages
• Show correct label for Groups & Collaborators depending on type of user
• Changed owner of uploaded installer to current dashboard user
• Remove groups permissions from all items after group delete
• Color schemes of some flashing messages
• Prevent project/installer transfer if recipient already has one with the same name
• Empty environment field from installer created from environment if the environment was deleted
• Fixed email validation when other user is logged in
• Maintain consistency on headers from dashboard
• Fixed token generalizations
• Exception on admin downloads list for a user when package/file was missing
• Fixed “View Docs” URL
• Use dashboard user in redirects after file delete
2.32.5 - 2017-08-29
Fixed
• Removed Continuum references
2.32.4 - 2017-08-28
Fixed
• Fixed old continuum links
2.32.3 - 2017-08-24
Fixed
• Navbar logo responsiveness issues
2.32.2 - 2017-08-24
Fixed
• Changed navbar buttons order to the new design schema
2.32.1 - 2017-08-24
Fixed
• Bigger logo on the navbar
• Fixed home page screenshot of the site
2.32.0 - 2017-08-22
Added
• New top bar button structure
• Added option ‘any’ to display all kind of packages in repo view
• Support for custom user avatar methods
2.31.6 - 2017-08-08
Added
• Email notification when added as a collaborator
• Labels regex now distinguishes uppercase characters
• Added site export tool
• Upload option for installers
• Added button to review the build log next to each installer file
• Version specific landing page for packages
• Added group info to site export tool
• Upload option for environments
Fixed
• Layout issues on dashboard and repo pages
• Remove user from groups when the account is removed
• Remove user as a collaborator for installers/packages/projects/notebooks/environments when is deleted
• Show only projects with files in projects list
• CSV export of users
• Typeahead for multi-type packages
• Added user validation to remove collaborator form
• Metadata display on package API
• Cache key generation and diskcache size limit parameter
• Empty packages will no longer appear in search results
2.31.4 - 2017-08-03
Fixed
• Fixed repodata caching
2.31.3 - 2017-07-27
Fixed
2.31.2 - 2017-07-24
Added
Fixed
2.31.1 - 2017-07-13
Added
Fixed
2.31.0 - 2017-06-28
Added
Fixed
2.30.3 - 2017-06-06
Added
Fixed
## 2.30.2 - 2017-05-24
## 2.30.1 - 2017-05-24
Added
• Warning before deleting a package and all of its messages
• Tool to check orphan files and packages
• Added option to lock down all public pages
• Added link to contact us for custom plans.
Fixed
• Display error message when config file is not found
• Fixes resend confirmation email
• Retrying on 502 error while mirroring conda
• Replace non-ascii characters from filename when downloading a file
• Added password validation to password reset form
• Handling 404 on s3 key_exists
• Removed files and packages will appear on package history
• Using user’s name and email on contact emails
• Changed Resend Email label to Password Reset in admin page
• Added placeholder to collaborators form
• Added singular form messages in flash notifications
• Fixed popup label for pkg/nbk/env settings
• Fixed scrollbar blocking content in installers documentation
• Added searchbox on navigation header for non-authenticated users
• Added more database indexes for better query performance
• Removed admin monitor page
• Increased request timeout default to 120 seconds
• Updated the mirror configuration examples
• Fixed URL for pricing info
• Fixed token creation API
• Removed outdated mirror documentation
• Added migration to normalize files data
• Added brand as key in api endpoint
• Declined credit card info is no longer stored
• Improved UI for group members page
## 2.30.0 - 2017-05-08
Added
• Added EULA to the installer
• Contact Us form now sends emails to [email protected]
• Create organization instead of user on mirroring tools
Fixed
• Fixed anaconda-server-sync-conda settings message
• Secured web helpers views
• Removed unused test endpoints
• Fixed insecure groups endpoint
• Fixed filename too long exception on type filter for installers
• Server side encryption on S3 storage
• Fixed documentation link in /settings/access API token page
• Year in footer matches current year
• Fixed password restrictions checks
• Replaced binstar-* message for anaconda-server-* on mirroring script
• Added missing instruction to Project upload instructions
• Fixed the way we load the license data from the database
• Added validation for reserved usernames
• Fixed package set-access on firefox
• Fixed encoding error on package information page
• Changed error message on installer creation form
## 2.29.1 - 2017-04-19
Fixed
• Fixed access to LDAP views
• Removed unused remove_user view
• Add support for expired marketo access token
• Fixed exception on installer creation
## 2.29.0 - 2017-04-19
Added
• Visibility on Projects feature
• Sortable account list by package count
• Change plan button for organizations
## 2.28.1 - 2017-04-03
Added
• Installers can now be created from uploaded environments
Fixed
• Fixed outdated version badges
• Add “jessie” and “sles12” parcel suffixes
• Only allows alphanumeric characters on installers name
• Fix group installers page
• Fix edition of existing installers
2.28 - 2017-03-22
Added
• Add UI to delete and set access of packages on the repo page
• Add UI to delete files in the files section of the repo page
• Add UI to set and unset superuser and staff status on admin page
• Staff users can now access licensing
• Pagination on history for the account admin page
Fixed
2.27.5 - 2017-03-14
Fixed
• Added boto dependency back
2.27.4 - 2017-03-03
Added
• Support for S3 regions that only use V4 signatures
• Support for S3 server-side encryption
• Support for custom PyPI repo sync
• New releases overrides package’s description, summary, license and icon
• Update mongodb to 3.4
Fixed
• API endpoint DELETE /dist/{owner_login}/{package_name}/{version}/-/{_id} should delete the file with the
associated ID
• Email validation on profile page
• Downloading files with spaces or special characters should result in the correct filename
• Some documentation URLs showed up without styling
• Fixed text overlap in admin deployment page
• /downloads installers should generate configuration files correctly
• “Not Found” errors are more consistent and clear
• Package search timeout
• Displaying validation in the popup on account’s admin page
2.27.3 - 2017-03-02
Fixed
• Add support for string license attribute
2.27.2 - 2017-02-27
Fixed
• Remove pyc from ambari mpack templates folder
2.27.1 - 2017-02-23
Added
• Limit to the cache
Fixed
• Added missing ambari mpack templates folder
2.27.0 - 2017-02-15
Added
• Support for generating custom Hortonworks/Ambari management packs
• License and license url to packages api
• Update email confirmation code to more secure and flexible hash
• Support for multiple users with same email (if option enabled)
• Admin support to remove an account
• Admin support to change storage size or change plan to free unlimited
• Package versions on installers and parcels are now optional
• Specify a configuration file with the environment variable ANACONDA_SERVER_CONFIG
Fixed
• Noarch repodata should not include files that are missing platform and arch
• Fix attribute errors kapsel unit tests
• Mirror configuration python_versions should not require quotes
• Add link to package on Favorites page breadcrumbs.
• Improve support for POWER and ARM architectures.
• Provide a useful error page when MongoDB is unreachable.
• Fix notebook and env with same name
• Suppress form errors when adding or removing package/channels
• anaconda-server-config will work on the config file you actually have
• Remove temporary redirects on user settings and org groups settings
2.26.5 - 2017-01-30
2.26.4 - 2017-01-30
Fixed
• Some files would not be cleaned up correctly when the corresponding user was deleted in the interface. Added
a migration to clean up any existing unused files.
• Add anaconda-server-admin clean-storage command to clean up unused files.
• Fix csv column order on package info at the admin page.
• Render URLs in the package summary as links.
• Display correct breadcrumb for a selected environment.
• Sort labels on package page alphabetically.
• Pluralize storage information correctly.
2.26.3 - 2017-01-10
2.26.2 - 2017-01-06
Added
• Conda repodata is now cached more frequently.
• Constructor installer creation will now be terminated if it takes longer than 60 seconds.
Fixed
• An error that occurred when PyPI packages that were deleted were re-uploaded.
• Anaconda Repository has a new mirroring tool with reproducible results, and improved support for delta mir-
roring.
• Documentation updates
• Fix task queue (removed mtq library)
• Improve performance of PyPI simple index page
• Move licensing code to anaconda_platform.component.licensing
• Add hotjar (cloud)
• Move username regex to check into settings
• Parcels: Add anaconda and python 2.7 as default packages
• Added AIC (Anaconda Installer Configuration) installers
• Remove Kapsel Execution
Added
• Gevent in as the server worker_class
• Remove check of key existence in s3 for anaconda.org
Fixed
• API: added correct handling when user is deleted
• DOWNLOADS: allow unlimited storage of download stats
• UI: terms and conditions link was incorrectly escaped
• NOTEBOOKS: fix sorting of notebook versions
• REPO: copied package files would sometimes return 404
• UI: added default sorting to more tables
• REPO: usability and functionality fixes for CDH parcel generation
Added
• REPO: users can create custom CDH parcels through Anaconda Repository
• UI: standardize sorting on tables
• UI: the software version is included in the footer of Anaconda Repository
Fixed
• NOTEBOOKS: added iframe sandboxing to notebooks
• NOTEBOOKS: fixed rendering of thumbnails uploaded by nb_anacondacloud
• REPO: copied package files were sometimes incorrectly garbage collected
Changed
• UI: The pages on the conda.anaconda.org and pypi.anaconda.org domains redirect to anaconda.org.
Removed
• BUILD: The deprecated build feature has been removed from Anaconda Repository.
Fixed
• NOTEBOOKS: links in the notebook will open in the browser window directly, instead of inside of a frame.
Fixed
• Allow numeric usernames
Added
• REPO: package types will correctly update from added files (#2492)
• UI: cluster pages now list apps associated with that cluster
• Notebooks larger than 25mb will not be rendered (#2336)
• API: the endpoint /user/{account}/downloads/{start}--{end} now provides download activity
aggregated by package for an account.
Fixed
• Improve the performance of the security feed (#2335)
Changed
• UI: the reminder to use beta will be hidden for 24 hours when a user clicks the “close” button.
• BUILD: remove welcome to build message for build deprecation notice.
• UI: rename project to kapsel everywhere (except imports) (#2563)
Fixed
• Fixed generation of URLs to user notebook content server over https
Fixed
• DB: improved group migration to handle more corner cases
Fixed
• REPO: package types will correctly update from added files (#2492)
Added
• Basic Cluster Pages
• Conda Caching - Conda endpoints now use Last-Modified/if-modified-since headers
Added
• API: add an endpoint /user/{account}/downloads/{start}--{end} that provides an aggregated
summary of download activity for an account.
• BUILD: automatically scroll to the bottom of log when new lines are appended
• REPO: improve support for R packages
• WEB: license warning message includes a link to the license configuration page
Fixed
• Users do not need to be logged into GitHub to trigger builds
• BUILD: remote address for workers will be detected correctly when running behind a proxy (#2036)
• API: LDAP users logging in for the first time via anaconda login are created correctly.
• PIP v8.1.2 fixed package name lookup
Added
• Queue administration page that displays build worker details and history (#1847)
• An additional configuration file can be specified with the environment variable ANACONDA_SERVER_CONFIG
or the command line argument --config-file
• Configuration files in the directory $PREFIX/etc/anaconda-server/ will now be automatically loaded
• Better logging for login logic
• Failed logins are now recorded in the security log
• docs.anaconda.org content is now bundled with Anaconda Repository
• New privacy policy
• Project’s API
• Show notebooks with nbpresent metadata as presentations (#1583)
• Can now view different versions of notebooks (#1764)
• Complete list of current settings on /admin/deployment (#1928)
• Decorator to validate params in a requests. (#1970)
• api.anaconda.org returns conda_url, pypi_url and main_url (#1984)
• keyname is displayed for superusers on the file details modal, allowing an administrator to locate a file on disk
(#1985)
Fixed
• Editing package description should not add extra whitespace (#1710)
• Starred packages owned by other users will appear on the dashboard (#1706)
• Notebook output that is too wide will display a scroll-bar (#1581)
• Cleaned up styling on CI settings page (#1713)
• Security log details modal should appear for non-administrator users
• More graceful handling of notebook rendering failure (#1548)
• GitHub OAuth flow in the user settings page (#1931)
• Changed conda install instructions to use short channel name
• Group API exceptions when viewing group members (#1959)
• Fixed error in sample enterprise config file (#1968)
Changed
• Renamed “upvotes” to “favorites” (#1707)
• adjusted helptext for conda install from specific user channel (#1914)
• Clean up build workers that have been idle too long (#1749)
• Add SMTP support for sending email (#1747)
• Implemented UI Improvements to align with Anaconda branding, making A-Cloud easier to use
• Added confirmation after sending a message to support from the “contact us” page
• Removed left nav on dashboard
• Moved channel manager to the apps dropdown
• Made it easier for Academic users to access features by adding extended subdomain access for institutions
• Created a landing page for bug reporting to help A-Cloud users better self-select which repo for issue logging
• Implemented UI Improvements
• Fixed minor issues
• Improved user profile
• Improved password validation
• Updated plans and pricing pages
• Implemented UI Improvements
• Increased specificity when mirroring the Anaconda repository including more robust license-blacklisting capac-
ity and new python version-filtering capacity
• Implemented the ability to upload iPython notebooks to your Anaconda Server user account
• Improved the user interface for channel-based interactions, which allowed users to manage multiple package
and channel interactions from a single dashboard
• Performed additional unit testing
• Due to a lack of backwards compatibility, this release locks the following two versions of the dependency packages:
– flask-wtf=0.8.4
– werkzeug=0.9.6
Command reference
Anaconda Client is the command line interface (CLI) to Anaconda Repository. You can use it to log in, log out,
manage your account, upload files, generate access tokens, view tokens and other tasks.
The full Client command reference is shown below. You can also view this command reference in a terminal window
with the command anaconda --help or anaconda -h.
See also: Anaconda Cloud API Reference.
• anaconda
• Authentication
– auth
– login
– logout
– whoami
• Informational
– show
– search
– config
• Package management
– package
– upload
– download
– remove
– groups
– label
– copy
– move
anaconda
optional arguments:
-h, --help show this help message and exit
-V, --version show program's version number and exit
output:
--disable-ssl-warnings
Disable SSL warnings (default: False)
--show-traceback Show the full traceback for chalmers user errors
(default: False)
-v, --verbose print debug information ot the console
-q, --quiet Only show warnings or errors the console
anaconda-client options:
-t TOKEN, --token TOKEN
Authentication token to use. May be a token or a path
to a file containing a token
-s SITE, --site SITE select the anaconda-client site to use
Commands:
Authentication
auth
optional arguments:
-h, --help show this help message and exit
-n NAME, --name NAME A unique name so you can identify this token later.
View your tokens at anaconda.org/settings/access
-o ORGANIZATION, --org ORGANIZATION, --organization ORGANIZATION
Set the token owner (must be an organization)
--strength {strong,weak}
--strong Create a longer token (default)
-w, --weak Create a shorter token
--url URL The url of the application that will use this token
--max-age MAX_AGE The maximum age in seconds that this token will be
valid for
-s SCOPES, --scopes SCOPES
Scopes for token. For example if you want to limit
this token to conda downloads only you would use
--scopes "repo conda:download"
--out OUT
actions:
-x, --list-scopes list all authentication scopes
-l, --list list all user authentication tokens
-r NAME [NAME ...], --remove NAME [NAME ...]
remove authentication tokens
-c, --create Create an authentication token
(continues on next page)
login
Authenticate a user
optional arguments:
-h, --help show this help message and exit
--hostname HOSTNAME Specify the host name of this login, this should be
unique (default: hq-phone-114.corp.continuum.io)
--username LOGIN_USERNAME
Specify your username. If this is not given, you will
be prompted
--password LOGIN_PASSWORD
Specify your password. If this is not given, you will
be prompted
logout
optional arguments:
-h, --help show this help message and exit
whoami
optional arguments:
-h, --help show this help message and exit
Informational
show
positional arguments:
spec Package written as USER[/PACKAGE[/VERSION[/FILE]]]
optional arguments:
-h, --help show this help message and exit
EXAMPLE:
anaconda show anaconda
anaconda show anaconda/python
anaconda show anaconda/python/2.7.5
anaconda show anaconda/python/2.7.5/linux-64/python-2.7.5-0.tar.bz2
search
name
positional arguments:
name Search string
optional arguments:
-h, --help show this help message and exit
-t {conda,pypi}, --package-type {conda,pypi}
only search for packages of this type
-p {osx-32,osx-64,win-32,win-64,linux-32,linux-64,linux-armv6l,linux-armv7l,linux-
˓→ppc64le,noarch}, --platform {osx-32,osx-64,win-32,win-64,linux-32,linux-64,linux-
˓→armv6l,linux-armv7l,linux-ppc64le,noarch}
config
usage: anaconda config [-h] [--type TYPE] [--set name value] [--get name]
[--remove REMOVE] [--show] [-f] [--show-sources] [-u]
[-s]
optional arguments:
(continues on next page)
actions:
--set name value sets a new variable: name value
--get name get value: name
--remove REMOVE removes a variable
--show show all variables
-f, --files show the config file names
--show-sources Display all identified config sources
location:
-u, --user set a variable for this user
-s, --system, --site set a variable for all users on this machine
anaconda-client configuration
* Invoke the anaconda command with the `-s/--site` option like this:
All options can be set as global options that affect all sites
or site options that affect only one site.
Package management
package
positional arguments:
USER/PACKAGE Package to operate on
optional arguments:
-h, --help show this help message and exit
actions:
--add-collaborator user
username of the collaborator you want to add
--list-collaborators list all of the collaborators in a package
--create Create a package
metadata arguments:
--summary SUMMARY Set the package short summary
--license LICENSE Set the package license
--license-url LICENSE_URL
Set the package license url
privacy:
--personal Set the package access to personal This package will
be available only on your personal registries
--private Set the package access to private This package will
require authorized and authenticated access to install
upload
positional arguments:
files Distributions to upload
optional arguments:
-h, --help show this help message and exit
-c CHANNELS, --channel CHANNELS
[DEPRECATED] Add this file to a specific channel.
Warning: if the file channels do not include "main",
the file will not show up in your user channel
-l LABELS, --label LABELS
Add this file to a specific label. Warning: if the
file labels do not include "main", the file will not
show up in your user label
--no-progress Don't show upload progress
-u USER, --user USER User account or Organization, defaults to the current
user
--all Use conda convert to generate packages for all
platforms and upload them
--no-register Don't create a new package namespace if it does not
exist
--register Create a new package namespace if it does not exist
--build-id BUILD_ID Anaconda Repository Build ID (internal only)
-i, --interactive Run an interactive prompt if any packages are missing
-f, --fail Fail if a package or release does not exist (default)
--force Force a package upload regardless of errors
metadata options:
-p PACKAGE, --package PACKAGE
Defaults to the package name in the uploaded file
-v VERSION, --version VERSION
Defaults to the package version in the uploaded file
-s SUMMARY, --summary SUMMARY
Set the summary of the package
-t PACKAGE_TYPE, --package-type PACKAGE_TYPE
Set the package type [ipynb, env]. Defaults to
autodetect
-d DESCRIPTION, --description DESCRIPTION
description of the file(s)
--thumbnail THUMBNAIL
Notebook's thumbnail image
--private Create the package with private access
See also:
• Uploading a conda package.
• Uploading PyPI packages.
download
positional arguments:
handle user/notebook
optional arguments:
-h, --help show this help message and exit
-f, --force Overwrite
-o OUTPUT, --output OUTPUT
Download as
Usage:
anaconda download notebook
anaconda download user/notebook
remove
example::
positional arguments:
specs Package written as <user>[/<package>[/<version>[/<filename>]]]
optional arguments:
-h, --help show this help message and exit
-f, --force Do not prompt removal
groups
spec
positional arguments:
{add,show,members,add_member,remove_member,packages,add_package,remove_package}
The group management command to execute
(continues on next page)
optional arguments:
-h, --help show this help message and exit
--perms {read,write,admin}
The permission the group should provide
label
optional arguments:
-h, --help show this help message and exit
-o ORGANIZATION, --organization ORGANIZATION
Manage an organizations labels
--copy LABEL LABEL
--list list all labels for a user
--show LABEL Show all of the files in a label
--lock LABEL Lock a label
--unlock LABEL Unlock a label
--remove LABEL Remove a label
copy
positional arguments:
spec Package - written as user/package/version[/filename]
If filename is not given, copy all files in the
version
optional arguments:
-h, --help show this help message and exit
--to-owner TO_OWNER User account to copy package to (default: your
account)
--from-label FROM_LABEL
Label to copy packages from
--to-label TO_LABEL Label to put all packages into
move
positional arguments:
spec Package - written as user/package/version[/filename]
If filename is not given, move all files in the
version
optional arguments:
-h, --help show this help message and exit
--from-label FROM_LABEL
Label to move packages from
--to-label TO_LABEL Label to move packages to
Glossary
• Anaconda
• Anaconda Client CLI
• Anaconda Repository
• conda
• conda build
• conda package
• label
• Miniconda
• namespace
• noarch package
• on-site repository
• organization account
• package
• package manager
• project
• repository
• source package
• token
Anaconda
An easy-to-install, free collection of open source packages, including Python and the conda package manager. Over
150 packages are installed with Anaconda. After installing Anaconda, you can install or update over 250 additional
open source packages contained in the Anaconda Repository using the conda install PACKAGE command.
The Anaconda Client command line interface (CLI) allows you to log into Anaconda Repository directly from your
Terminal window or Anaconda Prompt and manage your account. It is not necessary for downloading or installing
packages from Repository.
Anaconda Repository
Repository hosts hundreds of useful Python packages, notebooks and environments for a wide variety of applications.
You do not need to be logged in, or even need a Repository account, to search for packages, download and install
them.
conda
The conda package manager and environment manager program that installs and updates packages and their depen-
dencies, and lets you easily switch between environments on your local computer.
conda build
The command line interface that lets you build packages for your local operating system.
conda package
A compressed file containing system-level libraries, Python modules, executable programs or other components. The
file uses the tarball format.
label
Part of the URLs for Repository where conda looks for packages. Labels are searched only if you specify a label.
The default label is “main,” so packages that are uploaded without specifying a label are automatically labeled “main.”
The version labeled main is also downloaded by default, unless a user specifies a different label. So, if a file is labeled
main, then the label name may be omitted from the URL.
EXAMPLE: The following repositories are equivalent:
https://<your-anaconda-repo>/sean/label/main
https://<your-anaconda-repo>/sean
Commands such as conda install can be used with a channel, or used with a channel and a label:
conda install --channel sean selenium
conda install --channel sean/label/dev selenium
conda install --channel sean/label/stable selenium
Using Anaconda Client, package developers can create labels such as development labels/dev, test labels/
test or other labels that are searched only if the user specifies the label.
EXAMPLE: The following search examples use a namespace of “travis”:
Miniconda
A minimal installer for conda. Like Anaconda, Miniconda is a software package that includes the conda package
manager and Python and its dependencies, but does not include any other packages. Once conda is installed by
installing either Anaconda or Miniconda, you can install other software packages directly from the command line
using conda install.
namespace
Each user and organization has their own location called a “namespace” where they may host packages. You can view
the public packages in a user or organization’s namespace by navigating to their user page.
EXAMPLE: The “travis” user namespace located at https://<your-anaconda-repo>/travis contains
packages that were uploaded and shared by the user whose account is named “travis.”
noarch package
A conda package that contains nothing specific to any system architecture, so it may be installed on any system. When
conda searches for packages on any system in a channel, conda always checks both the system-specific subdirec-
tory—such as linux-64—and the noarch directory.
on-site repository
Repository is powered by Anaconda Server. You can run your own server behind firewalls or in air gapped environ-
ments. For more information, contact [email protected].
organization account
An organization account is a type of account on Repository that allows multiple individual users to administer packages
and control package access to different user groups. It also includes a large amount of storage space.
Use organization accounts to:
• Share packages, environments or notebooks under an organization’s account rather than your personal account.
• Assign multiple account administrators.
• Assign different access permissions to groups of users and customize per-package access by group.
package
All files uploaded to Repository are stored in packages. Each Repository package is visible at its own unique URL
based on the name of the user who owns the package and the name of the package.
EXAMPLE: If a user “travis” uploads a test package named “testpkg,” it is visible at:
https://<your-anaconda-repo>/travis/testpkg
Repository packages may contain multiple files, and these files may be data files such as comma separated value (CSV),
tab separated value (TSV), or text (TXT), or package files such as conda packages, PyPI packages or R packages.
package manager
A tool that facilitates the process of installing, upgrading, configuring and removing packages on Repository. Reposi-
tory supports two package managers, conda and PyPI.
For more information, see Using package managers.
project
Anaconda Project is an open source tool created by Anaconda that delivers light-weight, efficient encapsulation and
portability of data science projects.
repository
A storage location from which software packages may be retrieved and installed on a computer.
source package
“Source” packages are source code only, not yet built for any specific platform, and might be compatible with all,
some or only one of the platforms.
token
An access control token is a random alphanumeric string that is inserted into a URL that you give to another Repository
user. The token allows them to download a package or add a channel that you have marked private. Only those users
with the correct access token can access the private file. You can use Client to generate tokens to give other users
specifically scoped access to packages and collections.
Previous versions
This documentation is provided for the use of our customers who have not yet upgraded to the current version. Your
version number is located in the footer.*
Adding a PyPI or Anaconda mirror to your Anaconda repository installation (AER 2.32)
Adding a PyPI or Anaconda mirror to your Anaconda repository installation (AER 2.31)
Adding a PyPI or Anaconda mirror to your Anaconda repository installation (AER 2.30)
Adding a PyPI or Anaconda mirror to your Anaconda repository installation (AER 2.29)
Adding a PyPI or Anaconda mirror to your Anaconda repository installation (AER 2.28)
Adding a PyPI or Anaconda mirror to your Anaconda repository installation (AER 2.27)
Adding a PyPI or Anaconda mirror to your Anaconda repository installation (AER 2.26)
Adding a PyPI or Anaconda mirror to your Anaconda repository installation (AER 2.25)
Customizing your PyPI or Anaconda repository mirror - v 2.2.0 or earlier (AER 2.25)
Adding a PyPI or Anaconda mirror to your Anaconda repository installation (AER 2.24)
Customizing your PyPI or Anaconda repository mirror - v 2.2.0 or earlier (AER 2.24)
User guide
Quickstart
Reference
Command Reference
User administration
Client configuration
Recommended Workflow
Configuration reference
* Anaconda Repository Versions 2.21.0 and earlier do not contain the version number in the footer. Please contact
your Enterprise Support representative to get your version number.
User guide
AEN’s browser-based management of private packages, notebooks, and environments allows data science team mem-
bers to:
• Create, share and manage private notebooks.
• Control notebook revisions.
• Compare and identify differences across notebook versions.
• Search notebooks for keywords and packages.
• Use enhanced collaborative notebook features including revision control and locking.
• Access on-premises and/or cloud-based collaborative notebook servers.
• Utilize multiple language kernels like Python and R language in the same notebook.
• Create new notebook environments on the fly without leaving the notebook or entering commands in a prompt.
• Publish results to business stakeholders as interactive visualizations and presentations.
To quickly get up and running with AEN, see Getting started.
Download the Cheat sheet for easy reference.
Concepts
• Projects
• Team collaboration
• Access control
• Sharing projects
• Project tags
Projects
Team collaboration
Teams collaborate in AEN using projects. Projects allow a team to easily come together by sharing the resources,
applications, and environments that are necessary to collaborate effectively.
The AEN project owner and any team members connected to their project will have access to the same:
• Shared files and home directories.
• Shared Python and R environments.
• Shared nodes and hardware.
• Common applications.
• Web user interface.
For more information, see Working with projects.
Access control
Sharing projects
In addition to general project sharing capabilities, you can also publish Jupyter Notebooks to Anaconda Repository.
This automatically versions the notebook and allows you to define who can view the notebook.
Project tags
Getting started
This section contains information and tasks for first-time AEN users.
In this getting started guide, you will:
Before you start, download and print the AEN cheat sheet for easy reference.
After your administrator has set up your server and new Anaconda account, you will receive a welcome email.
1. Click the link in the email to open the AEN login page.
NOTE: Use the domain name and not the IP address when you connect to AEN. Using the IP address can cause
TLS and security certificate errors.
2. Enter your AEN account username and password.
NOTE: Some administrators allow you to create your own account. If your administrator has allowed this, in
the create a new account section, create your own username and password.
3. Click the Login button.
Your user home page, where all good things happen, is displayed:
• On the right side of the AEN task bar, click on the New Project icon:
2. On the Project page that is displayed, type a name for your project, such as “Testing.”
7. To change the project settings, click the Project Settings icon on at the top right.
4. Add collaborators
You can add team members to your project as collaborators. Adding team members to your projects makes collabora-
tion easy because they have full access to the project’s applications, files and services.
When you add team members, their home directory is mounted in the project. There is no need to download and email
data or scripts—team members can work on the same files in the same environment in which you are working.
To add collaborators to your project:
1. From your project home page, in the Team box, begin typing a teammate’s username.
2. In the list that is displayed, select the teammate’s username.
3. Click the Add button.
1. Repeat these steps for each team member you want to add as a collaborator.
TIP: You can add or remove team members any time from the Team menu in Project Settings. You can also modify a
team member’s read, write or execute permissions at any time from the Using Workbench.
1. From your project home page, click the Jupyter Notebooks icon.
2. On the File View page, click the Examples folder.
If you are already familiar with creating notebooks, you can easily set up a new environment with the programs you
need—like SciPy and NumPy—then open a new notebook and make your edits.
To create a new environment:
1. From your project home page, click the Jupyter Notebooks icon.
1. In the available packages section, search for the package name numpy—all lower case.
2. In the results section, next to numpy, select the checkbox.
NOTE: If you do not see your new environment listed under Notebooks, next to the New
button, click the Refresh button.
A new locked notebook is displayed. Paste or write some code to execute when you are ready.
Whether you are exploring an existing notebook, or creating a new one, you can easily create checkpoints, return to
an earlier version, compare two different versions and save them for reference.
To create a checkpoint, in the File menu, select Save and Checkpoint:
To revert your notebook to a previous checkpoint, in the File menu, select Revert to Checkpoint.
NOTE: For more information about revision control features, including creating commits and comparing differences,
see Using the Revision Control Mechanism extension.
Now that you have completed the Getting Started guide, you are ready to move on to basic tasks and advanced tasks.
Basic tasks
This section contains information and tasks that use the web browser to manage projects and is best-suited for any
beginning AEN user:
Almost everything in AEN starts by opening an existing project or creating a new one.
After that, you can set up a special environment with the packages you want, set their access permissions and modify
your project settings.
To search for projects and files, use the Search box in the AEN navigation bar. The search provides different results
depending on which page you search from:
• On a project home page, search results include any files that match your search criteria within the current project.
• On any other AEN page, search results include any files that match your search criteria within all projects.
TIP: Your search results include only files and projects that you can view: public projects, and private projects to
which you have a minimum of view access.
Search indexing
Files that are modified while a project is running are automatically re-indexed shortly after the files are modified. If
you create or update a large number of files—such as cloning a git repository or copying a directory—search results
may take several minutes to update.
Files that are modified while the project is not running are re-indexed only after the project is started.
Regular expression patterns can be embedded in the query string by wrapping them in forward-slashes (/):
name:/joh?n(ath[oa]n)/
Searching a project
TIP: Search by glob patterns, which are similar to file matching in the command line.
EXAMPLE: To find projects in the test family that are numbered from 00 to 99, search for Test-??. To find
all projects whose name ends with “Stats,” search for *Stats.
2. Press Enter.
3. In the search results, click the plus + icon above a project name to show a list of matching files in the selected
project:
TIP: Click the project name to open the project’s home page.
4. To view a file, click its file name in the matching files list:
Saving a search
The “save this search” text changes to “stored” and your search is saved. Your saved searches are listed on your home
page.
On your home page, in the Saved searches section, click X next the saved search that you want to remove:
1. On the project home page, click the Project Settings icon to open the Project Settings page.
1. In the username box, type in the first few letters of the username for the team member you want to add to the
project.
2. In the list of usernames that displays, click the user to add.
3. Click the Add button.
Click the red Remove link next to the name of the user you want to remove from the project.
By default, all of the team members on a project have read and write access permissions for all project assets.
The available permissions are read, write and execute. If you remove all individual or group permissions for a project
asset, team members will not be able to access that asset.
To change a project’s permissions:
1. Open the project’s home page.
2. Click the Workbench icon.
3. In the Workbench app, right-click the file or folder you want to limit access to.
NOTE: When you change a folder’s permissions, the permissions of files and folders inside it do not change.
You may change the permissions of those files and folders manually.
4. In the menu that displays, select Permissions:
A list of owners and team members who have access to your project is displayed.
5. Find the team member you want to change access for:
6. Next to the team member’s name, select or deselect the permissions for that user.
NOTE: You can add a team member and set their access at the same time by typing their name in a username
box, setting their permissions, and then clicking the Add button.
7. Click the Submit button.
The selected permissions are added, and the deselected permissions are removed.
NOTE: If a team member is in the Workbench application when you give them access, they must refresh their browser
window to see their current permissions.
You can choose to grant file or folder access to someone who is not part of the project team, as long as that person has
an AEN account.
Sharing with individuals outside the team is a four step process:
1. Copy or move the file or folder to your home directory.
2. Give the user read and execute access to your home directory.
3. Add the user to the file’s permissions.
4. Have the user add your directory to their workbench.
Your home directory is displayed at the bottom of the File Manager pane in the Workbench.
To protect the other files and folders in your home directory—those you are not providing permissions to a user to
access—we recommended that you:
1. Create a sub-folder.
2. Rename the folder with the name of the user you are granting access to.
3. Copy or move the file you want to grant permissions for to the renamed folder.
The file is copied or moved to the new location and is ready for you to update the file permissions.
You must select read and execute access for a user to be able to view, but not edit, the files or folders.
1. Right-click the name of the file or folder you are granting access to.
2. In the menu that is displayed, select Permissions.
3. Click the Add button.
4. Type the username of the user to whom you are granting file access and press Enter.
TIP: If you grant access to a folder instead of a specific file, you only have to set permissions the first time you share
the folder with each user, unless you need to update the permissions.
Once a user is included in your Permissions list, you must add the correct permissions for the user, in the same way
as you would for a team member.
Once complete, depending on the access granted, the user will be able to view, read, change, and execute the file.
NOTE: If you change permissions for a folder instead of a file, the user will be able to see and access any files within
that folder.
The user can now add your home directory to their Workbench File Manager.
To add your home directory to another user’s workbench, have the other user follow these steps:
1. Click the Show Directory button at the top of the Workbench File Manager:
5. Close the Show Directories dialog box by clicking the X in the upper-right corner or by clicking anywhere
outside the box.
TIP: Stopping a project stops all the applications launched for that project that use resources when running, such as
memory and compute cycles. It is best to stop projects when they are not in use.
1. On the project home page, click the Project Settings icon to open the Project Settings page.
3. In the Status section, click the Start or Stop button to toggle between manually starting and stopping your project.
1. On the project home page, click the Project Settings icon to open the Project Settings page.
Tagging a project
Existing tags assigned to a project are listed in the Tags section on the project’s home page.
Adding a tag
1. In the Tags box, type the name of the tag you want to add:
If the tag was not already in the Top Tags list on your user home page, it is added. If the tag was already listed
because another project used it, the number next to the tag is incremented:
Removing a tag
1. On your user home page, in the Top Tags list, click the tag name.
Starring a project makes it appear on your user home page in the Top Rated list.
Adding or removing stars for a project does not affect the stars added by other users.
1. Open the project that you want to star.
2. On the project home page, click the Star icon at the upper right:
When you claim ownership of a project, ownership of all files and folders created by the team members on the project
is transferred to you. Project files and folders are copied and renamed.
1. Stop the project to prevent team members from making changes while you are changing ownership.
2. On the project home page, click the Project Settings icon to open the Project Settings page.
1. On the project home page, click the Project Settings icon to open the Project Settings page.
3. Update your project’s summary using plain text or its description using Markdown syntax.
4. Click the Preview tab to see a preview of the Markdown description.
5. Click the Submit button.
1. On the project home page, click the Project Settings icon to open the Project Settings page.
These are projects that contain fields that are most similar to the current project.
TIP: You will only see projects to which you have been granted access: public projects, and private projects on which
you are a team member.
2. The engine performs a search, with extra weight given to the “uses” and “imports” keywords.
3. The engine finds the files and projects that are most similar to the current project and scores the results.
4. The top-scoring matches are displayed in Related Projects. Only public projects and private projects to which
you have access are included.
The number next to a project represents the number of stars that have been given to that project.
Click a project name to view the project’s home page.
The top tags used on your projects are listed on your home page:
To list all projects that share a specific tag, click the tag name:
TIP: The list includes only projects that you have access to: public projects, and private projects on which you are a
team member.
Click a project name to open the project’s home page.
These are the team members who have the most projects in common with you.
To view a collaborator’s home page—where you can see all public projects and the private projects they have shared
with you—click the collaborator’s name.
For information on sharing projects via the project settings and access control, see Sharing projects.
To upload a Jupyter Notebook to Anaconda Repository:
1. Log in to Repository by running the anaconda login command or by using the login user interface provided
by the nbextension.
CAUTION: If you are not using a secure connection, we strongly recommended that you use the command line
to log in.
2. To share your notebook environment, select the Attach conda environment checkbox. This ensures that your
team members will have the right environment for your notebook.
3. Click the Upload button to upload your notebook to your local Repository or to Anaconda.org, depending on
how your administrator has set up AEN:
NOTE: If you have not yet logged into Repository or Anaconda Cloud, or have not created an account, you will
be asked to do so.
– Download as HTML.
– Download as Markdown.
– Download as ReStructured Text.
– Download as PDF.
• Share and control team members’ direct access to read, write and/or execute your notebook file or folder. For
more information, see Controlling access to your project.
• Share and control non-team members’ file or folder access. For more information, see Controlling access to
your project.
• Create a presentation with NBPresent 4.1.
Deleting a project
CAUTION: Deleting a project deletes all project files and information! There is no undo option.
1. Download a copy of any project files that you need to save.
2. On the project home page, click the Project Settings icon to open the Project Settings page.
The applications in your project make it easy for you to interact with your files and data, manage your project’s
resources and to customize your AEN experience.
To use applications, log into AEN, then select the project you want to work on or create a new project and open it.
On the project home page, the following application icons are displayed:
TIP: Each application opens in a new browser tab. You can run multiple applications at the same time in your project.
For more information on each AEN application, see:
• Using Workbench—File viewer and manager, including permissions settings.
• Using Viewer—View-only versions of notebooks and other text files.
• Using JupyterLab—Alpha preview of the next generation notebook.
• Using Terminal—Basic bash shell Terminal.
• Using Jupyter Notebook—Jupyter Notebooks with extensions.
• Using Compute Resource Configuration—Project information, view and manage applications.
Using Workbench
• Opening Workbench
• Using File Manager
• Opening the Workbench terminal
Workbench is a file viewer and manager that includes a file editor and file permissions manager.
You can use Workbench to:
• Upload and download files using the File Manager.
• Create new files and folders using the File Manager.
• Copy and move files to new locations using the File Manager.
• Rename files and/or folders using the File Manager.
• Manage the access permissions of team members.
• Grant or revoke access to non-team members.
Workbench also includes a simple Terminal application, which is convenient because the File Manager is always
visible, making navigation simple.
When you first open Workbench, the File Manager is displayed in the left pane, and the Create a New File and Upload
a New File buttons are in the right pane:
When you open a file or Workbench Terminal, it is displayed in the right pane. To make the Create or Upload a file
options re-appear, refresh your browser window.
Two small icons are displayed in the black navigation bar at the top of the Workbench page. Hovering over them
displays tool tips that describe their use:
• The Toggle icon displays or hides the File Manager.
• The Terminal icon opens a simple terminal window.
Opening Workbench
To open Workbench:
1. Log in to AEN.
2. Select the project you want to work on, or create a new project and open it.
3. On the project home page, click the Workbench icon:
The File Manager is an intuitive way to interact with your files and folders.
2. When you finish editing the file, click the Save button.
NOTE: To close the file without saving, click the X at the top of the page under the file name.
Using Viewer
The Viewer application displays a static, view-only version of your notebooks and other text files by rendering the text
files directly and using the NBConvert tool to convert notebooks to static HTML.
1. Log in to AEN.
2. Select the project you want to work on, or create a new project and open it.
3. On the project home page, click the Viewer icon.
Viewer opens in a new browser window:
4. Click any folder to view its contents, or click any filename to view the file.
5. To search for a file or folder name, type text in the Find File box, then press the Enter key. This is not a full-text
search, but wildcards are permitted.
Using JupyterLab
JupyterLab is an early alpha-preview of the next generation of the Jupyter Notebook. It is included so that you can
take a tour and play with its capabilities.
CAUTION: JupyterLab is experimental. It is not yet intended for production work.
JupyterLab does not include any of the notebook extensions that are available in the Jupyter Notebook app.
Experiment with the application on your own, using the Notebook, Editor, Terminal and Console menus.
To review a guided tour of all of the features JupyterLab will contain when it is ready for production, click the Take a
tour link in the right pane.
Using Terminal
The Terminal application is a simple bash shell terminal that runs in your browser:
The Jupyter Notebook application allows you to create and edit documents that display the input and output of a
Python or R language script. Once saved, you can share these files with others.
NOTE: Python and R language are included by default, but with customization, Notebook can run several other kernel
environments.
This page provides a brief introduction to Jupyter Notebooks for AEN users.
For the official Jupyter Notebook user instructions, see Jupyter documentation.
For information on the notebook extensions available in AEN, see Using Jupyter Notebook extensions.
1. Log in to AEN.
2. Select the project you want to work on, or create a new project and open it.
3. On the project home page, click the Jupyter Notebook icon:
TIP: You can see the same File Manager in the Terminal, Workbench, and Viewer applications.
The Examples folder in Jupyter Notebook contains several types of Notebook examples created in Python—and one
with R language—kernel environments.
Open any example notebook to experiment and see how it works.
1. An the top right of the Files tab, click the New button.
• You can switch between Python, R language and any other custom kernels in the notebook as you work in
your notebook. For more information, see Using the Synchronize Environments extension.
The new notebook is saved in the related project directory and displayed.
The following extensions are available for use with AEN’s Jupyter Notebook application:
• Synchronize Environments with Jupyter from the Kernel menu.
• Locking adds multi-user capability from the Lock button.
• Revision Control Mechanism (RCM) adds Status, Checkout and Commit buttons.
• Conda environment and package management tab.
• Conda notebook adds conda management inside Notebook from the Kernel > Conda Packages menu option.
• Anaconda Cloud integration from the Publish to cloud button.
• Notebook Present turns your notebook into a PowerPoint-style presentation.
The Synchronize Environments extension allows you to apply a Python, R language or any other custom environment
inside your current notebook session, without needing to start up several Notebook instances using each of the selected
environments.
To change environments:
1. Open the Kernel menu.
Multi-user capabilities are engaged in AEN when multiple users work in the same notebook file.
The Locking extension allows you to lock a notebook to prevent multiple team members from making changes at the
same time. Notebooks are automatically locked when you open them.
If team members open a notebook and make changes while it is locked, their save capability is disabled, and they
cannot overwrite the notebook.
To override the lock, they must actively take control of the locked file by clicking the Lock icon in the Notebook menu
bar:
NOTE: This is a soft locking model. Team members can choose to override your lock to save their work. If you give
team members write access to your files, confirm that they understand that they should never unlock your file unless
they are making meaningful, non-destructive team contributions.
The Revision Control Mechanism (RCM) Jupyter Notebook extension provides simple version control for notebook
files. It uses the internal Jupyter functionality to perform tasks.
On the surface, RCM uses a simple linear model, but beneath that is a more complex git-based branching model. To
prevent merge conflicts, this model uses a “latest wins” policy as its main merging strategy.
• Status.
• Checkout.
• Commit.
• Configure git.
TIP: If you do not see the RCM buttons, see Setting up RCM for the first time.
The Status button allows you to see what revision you are on.
Clicking the Status button displays:
The Checkout button allows you to view a list of the previous revision points, check out a previous revision or compare
differences between revisions.
Clicking the Checkout button displays:
Comparing revisions
The Commit button allows you to save or persist the current changes, keeping a permanent record of any changes that
are introduced, so that you do not have to worry about losing important data.
Clicking the Commit button displays:
1. Enter a description of the changes in the commit as a reminder in case you need to revert back to it later.
2. Click the OK button.
Your changes are committed and a revision point is created.
If Git user name and user email are not set, the following window appears:
The Configure git button allows you to configure Git user name and email values.
After clicking the Configure Git button, the following window appears:
Enter user name and e-mail address. Click the OK button when finished.
NOTE: Change [email protected] to your email address, and Your Name to your actual name.
4. Open Jupyter Notebook and refresh the page.
The NBConda extension adds a Conda tab to your notebook for easy environment and package management from
within the notebook.
• Conda available packages list—for the selected environment in currently configured channels, search for pack-
ages and click a package name to install it.
• Installed packages list—in the selected environment, check for updates, update or delete selected packages.
TIP: While you are in any notebook, you can jump to the NBConda extension for that environment by clicking the
Kernel menu and selecting Conda Packages:
The Conda Notebook extension adds the Conda Packages option to the Kernel menu.
Select the Conda Packages option to display a list of all of the Conda packages that are currently used in the environ-
ment associated with the running kernel, as well as any available packages.
From the Conda Packages option, you can perform all of the tasks available in the Conda tab, but they will only apply
to the current environment.
The Anaconda Cloud extension adds the Cloud button to your notebook, allowing you to easily upload your notebook
to Cloud:
The AEN Notebook Present extension turns your notebook into a Microsoft PowerPoint-style presentation.
The Present extension adds 2 buttons to Notebook’s menu bar—Edit Presentation and Show Presentation:
The Notebook Present sidebar is displayed on the right side of your browser:
Clicking each icon changes the menu and layout of your notebook.
Clicking the Help icon displays 3 tours—demonstrations—of the main features of Present:
• Intro tour.
• Slides tour.
• Editor tour.
Select one of the tours to view a short presentation regarding the specifics of that feature.
Intro tour
The Intro tour is a 2-minute presentation that explains how to use the main features of Present, including a description
of each button’s purpose.
NOTE: At any time, you can pause, go back to the previous or move forward to the next slide.
The following information is covered in the Intro tour:
• App Bar—When Authoring, this allows you control the content and style of your presentation. It also can be
used to activate several keyboard shortcuts for editing:
• Stop Authoring—Clicking the Edit Presentation button again stops Authoring, and removes all keyboard short-
cuts.
• Show Presentation—If you just want to run your presentation without using any Authoring tools, just click the
Show Presentation button.
• Presenting/Authoring—Once you’ve made some slides, start Presenting, where you can use most Notebook
functions with the Theme we have defined, as well as customize slides on the fly.
• Slides button—Slides, made of Regions linked to Cell Parts are the bread and butter of any presentation, and
can be imported, created, linked, reordered, and edited here.
• Theming—Theming lets you select from existing colors, typography, and backgrounds to make distinctive pre-
sentations. The first theme you select will become the default, while you can choose custom themes for a
particular slide, like a title.
• Saving—Whenever you save your Notebook, all your presentation data will be stored right in the Notebook
.ipynb file.
• Downloading—After you’ve made a presentation, you can download it as an HTML page by choosing Download
→ Download As: Presentation (.html) in the menu.
• Help—Activate Help at any time to try other tours, connect with the Present developers and community, and
other information.
Slides tour
Slides make up a presentation. Clicking Slides toggles the sorter view and the Slide Toolbar on and off:
The Slides tour explains how to create and manage slides, including the following information:
• Slide Toolbar—Create a new slide. Clicking + Slide will offer some choices for creating your new slide.
• Import—The quickest way to create a presentation is to import each cell as a slide. If you’ve already created
slides with the official slideshow cell toolbar or RISE, you can import most of that content.
• Template Library—You can create a presentation from an existing template.
– Reuse Slide as Template—You can create a presentation based on an existing slide.
– Simple Template—A common template is the Quad Chart, with four pieces of content arranged in a grid.
• Region—The Quad Chart has four Regions. To select a region, click it.
– Link a Region to a Cell Part—Each Region can be linked to a single Cell Part using the Link Overlay,
which shows all of the parts available.
* Cell Part: Outputs (red)–Outputs, such as rich figures and script results.
* Cell Part: Widgets (purple)–Jupyter widgets, interactive widgets that provide both visualization and
user input.
* Cell Part: Whole (orange)–Finally, a Whole Cell, including its Source, Widgets and Outputs can be
linked to a single region.
– Unlink a region from a Cell Part—Unlinking removes the connection between a region and a cell part,
without deleting either one.
– Region: Trashing—Trashing a Region permanently deletes it, without affecting any linked Cell Part.
– Part Thumbnail—We’ll try to draw a part thumbnail. It can only be reliably updated when a linked Cell
Part is on-screen when you mouse over it, but you should usually be able to get an idea of what you’re
seeing. The colors of the regions correspond to the cell types.
• Presenting—Clicking the Present button while editing brings up the Presenter with editing mode still enabled:
– Linked inputs and widgets are still interactive.
– Go forward—Click to go to the next slide
– Go back—Click to go back to the previous slide
– Go back to the beginning—Click to go back to the first slide
– My work is done here—Click to go back to the Notebook.
Editor tour
Once you’ve made a few slides, you’ll likely want to customize them. The Editor tour explains how to edit your
notebook, including the following information:
• Editing Slides—Activate the Slide Editor by double-clicking it, or by clicking Edit Slide.
• Region Editor—Click to drag Regions around and resize them.
• Region Tree—Reorder Regions and see the details of how Regions will show their linked Parts.
• Add Region—Add new regions.
• Attribute Editor—Edit the properties of a region.
• Data Layouts—In addition to manually moving regions, you can apply these layouts to automatically fill your
slides.
• More Regions—Add more regions—with a weight of 1.
• Tree Weight—Make a Region bigger or smaller, based on its relative weight.
• 12 Grid—A compromise between the Free and Treemap layouts, the 12 Grid option rounds all of the values in
a layout to a factor of 12.
The Compute Resource Configuration (CRC) application displays information about the current project and allows you
to set a custom project environment and view and manage your other AEN applications, including stopping, starting,
restarting and viewing the logs of each.
The CRC application screen contains 3 sections:
• Info.
• Conda environment.
• Running apps.
Info
Conda environment
Running apps
The Running Apps section displays a list of users and the applications that are in use, as well as when the app was last
modified.
To terminate any individual application, click the Terminate button.
To stop and re-launch any individual application, click the Relaunch button.
To review the run logs of any active application, which may be useful for troubleshooting, click the Logs button.
To access your account information, click the User Settings icon in the AEN navigation bar:
Your public profile is made up of a name, a personal URL, your company and location.
1. In the left navigation pane, click the Public Profile tab.
2. To update your profile picture, create a Gravatar that is associated with the email address you used to create your
AEN account. The gravatar will automatically appear.
1. In the left navigation pane, click the Security Log tab to view a list of operations performed on your account.
2. For more information about an operation, click the Eye icon to the left of the the operation name.
Registering an application
If you want to create an application for AEN or have already done so, you must register your application.
1. In the left navigation pane, click the Applications tab.
2. Click the Register New Application button to open a form for registering your application.
Advanced tasks
Advanced tasks are best-suited for users who are comfortable working in a Terminal.
AEN runs on conda, a package management system and environment management system for installing multiple
versions of software packages and their dependencies and switching easily between them.
A conda environment usually includes 1 version of Python or R language and some packages.
The ability to have a custom project environment is one of the most powerful features of AEN. Your project environ-
ment is integrated so that all of your project applications recognize it and all of your team members have access to
it.
This section contains information about:
• Creating a default conda environment using the Jupyter Notebook application
• Creating a default conda environment using the Jupyter Notebook application
• Using your conda environment in a notebook
• Customizing your conda environment
• Installing a conda package using Terminal
• Installing a conda package using Notebook
• Uninstalling a conda package
NOTE: This conda environments guide is specific to AEN. For full conda documentation—including cheat sheets, a
conda test drive, and command reference—see the conda documentation.
You can create, activate, and install packages and deactivate environments from within the Notebook menu bar.
To install from the Notebook menu bar:
1. Click the Conda tab and select the plus sign icon.
2. Search for numpy in the package search box.
3. Select numpy from the search results.
In AEN, all new environments created with conda automatically include Python, Jupyter Notebooks and pip. You can
specify any other packages you want included in your new environment.
TIP: By default, conda creates a new environment in your project’s env directory—so that all team members have
access to the environment. For information about limiting your team member’s read, write or execute permissions, see
Workbench.
To create a new environment within your AEN account, run the command conda in a Terminal application.
EXAMPLE: To create a new environment named WeatherModel that contains Python, NumPy, pip and Jupyter
Notebooks in your project’s env directory:
1. Log in to AEN.
2. Open a project.
3. On the project home page, click the Terminal application icon to open a Terminal.
4. Create the environment:
TIP: Python, pip and Jupyter Notebooks are automatically installed in each new environment. You only need to
specify NumPy in this command.
5. Make the new environment your default:
6. To use your new environment with Jupyter Notebooks, open the Notebook application.
7. Click the New button to open a new notebook. In the drop-down menu under Notebooks, the environment you
just created is displayed.
8. To activate that environment, select it.
The environment is added to the project’s env directory.
NOTE: You can deactivate the new environment when you are finished with your notebook by opening the Terminal
application and running the command source deactivate.
Whether you have created an environment using conda in a terminal, or from the Conda tab in a notebook, you can
use the conda environment in the same way.
When working in a notebook, to select the environment you have created and want to use with that notebook, in the
Kernel menu, select Change Kernel.
EXAMPLE: If you have an environment named my_env in a project named test1 that includes NumPy and SciPy
and you want to use that environment in your notebook, in the Kernel menu, select Python [conda env:test1-my_env].
The notebook code will run in that environment and can import NumPy and SciPy functions.
If you need a Python package that AEN doesn’t include by default, you can install additional packages into your AEN
environment.
TIP: You cannot install packages into the default Anaconda environment. You must create your own environment
before installing a new package into that environment.
AEN is built on Anaconda, so you can install additional Python packages using conda or pip—both of which are
included with Anaconda.
A conda environment named mypy3, running on Python 3 and containing NumPy and SciPy is created. All
subsequent packages added to this environment will be the Python 3 compatible versions.
You can also install the package within your notebook without using the terminal app:
1. From the Notebook application, click the Conda tab.
2. Select the environment you wish to use.
3. Search for the package you want to add.
4. Click the Install button.
To uninstall a package using this method, run the command conda remove <packagename>.
NOTE: Replace <packagename> with the name of the package you are uninstalling.
Matplotlib
Matplotlib is a Python 2D and 3D plotting and visualization library that produces publication-quality figures in a
variety of hardcopy formats and interactive environments across platforms.
To display Matplotlib figures in the output cells of a notebook running the default environment, run:
Any Matplotlib figures in the notebook are displayed in it’s output cells.
EXAMPLE: The following screenshot is of a cumulative density function (CDF) plot using values taken from a normal
distribution:
For more information, including a gallery, examples, documentation and a list of plotting commands, see the Mat-
plotlib website.
Bokeh
Bokeh is an interactive visualization library that targets modern web browsers to provide elegant, concise construction
of novel graphics.
To display Bokeh figures in the output cells of a notebook running the default environment, run:
Any Bokeh figures in the notebook are displayed in its output cells.
The following screenshot is of a scatter plot of miles-per-gallon vs. horsepower for 392 automobiles using the
autompg sample dataset:
ggplot2
Ggplot2 is a plotting system for R language which is based on the grammar of graphics. Ggplot2 tries to take only the
good parts of base and lattice graphics and none of the bad parts.
To use ggplot2 with AEN:
1. Open a new Notebook using the R kernel.
2. Load the ggplot2 library with the following code:
library(ggplot2)
Cheat sheet
See the Anaconda Enterprise Notebooks cheat sheet PDF (232 KB) for a single-page summary of the
most important information about using AEN.
Troubleshooting
This troubleshooting guide provides you with ways to deal with issues that may occur with your AEN installation.
Cause
Solution
Admin guide
This administrator guide provides information about the administration of an AEN installation.
Most AEN system management is done from the administrative user interface (admin UI). Some advanced tasks are
done using the command line.
Any AEN user account can be upgraded to an administrator account to have both user and administrator privileges.
Administrators see two additional links in the AEN Navigation bar—Admin and Users:
All of the other navigation bar items are the same as for a user account.
Concepts
• System overview
• Server node
• Gateway node
• Compute node(s)
• Supervisor and supervisord
• Service Account
• Anaconda environments
• Projects and permissions
System overview
The Anaconda Enterprise Notebooks platform consists of 3 main service groups: AEN server, AEN gateway and AEN
compute, which are called “nodes”:
• Server node—The administrative front-end to the system where users login, user accounts are stored, and ad-
ministrators manage the system.
• Gateway node(s)—A reverse proxy that authenticates users and directs them to the proper compute node for
their project. Users will not notice this node after installation as it automatically routes them.
• Compute nodes—Where projects are stored and run.
These services can be run on a single machine or distributed across multiple servers.
Organizationally, each AEN installation has exactly 1 server instance and 1 or more gateway instances. Each compute
node can only be connected to a single gateway. The collection of compute nodes served by a single gateway is called
a data center. You can add data centers to the AEN installation at any time.
EXAMPLE: An AEN deployment with 2 data centers, where 1 gateway has a cluster of 20 physical computers, and
the second gateway has 30 virtual machines, must have the following services installed and running:
• 1 AEN server instance
• 2 AEN gateway instances
• 50 AEN compute instances (20 + 30)
Nodes must be configured and maintained separately.
Server node
The server node controls login, accounts, admin, project creation and management as well as interfacing with the
database. It is the main entry point to AEN for all users. The server node handles project setup and ensures that users
are sent to the correct project data center.
Since AEN is web-based, it uses the standard HTTP port 80 or HTTPS port 443 on the server.
AEN uses MongoDB for its internal data persistency. It is typically run on the same host as the server but can also be
installed on a separate host.
Server nodes use NGINX to handle the user-facing AEN web interface. NGINX acts as a request proxy for the actual
server web-process which runs on a high numbered port that only listens on localhost. NGINX is also responsible for
static content.
Server is installed in the /opt/wakari/wakari-server directory.
Server processes
When you view the status of server processes, you may see the processes explained below.
supervisord details
description Manage wakari-worker, multiple processes of wk-server.
user wakari
configuration /opt/wakari/wakari-server/etc/supervisord.conf
log /opt/wakari/wakari-server/var/log/supervisord.log
control service wakari-server
ports none
wk-server details
description Handles user interaction and passing jobs on to the wakari gateway. Access to it is managed by
NGINX.
user wakari
command /opt/wakari/wakari-server/bin/wk-server
configura- /opt/wakari/wakari-server/etc/wakari/
tion
control service wakari-server
logs /opt/wakari/wakari-server/var/log/wakari/server.log
ports Not used in versions after 4.1.2 *
* AEN 4.1.2 and earlier use port 5000. This port is used only on localhost. Later versions of AEN use Unix sockets in-
stead. The Unix socket path is: unix:/opt/wakari/wakari-server/var/run/wakari-server.sock
wakari-worker details
description Asynchronously executes tasks from wk-server.
user wakari
logs /opt/wakari/wakari-server/var/log/wakari/worker.log
control service wakari-server
nginx details
description Serves static files and acts as proxy for all other requests passed to wk-server process. *
user nginx
configura- /etc/nginx/nginx.conf /opt/wakari/wakari-server/etc/conf.d/www.
tion enterprise.conf
logs /var/log/nginx/woc.log /var/log/nginx/woc-error.log
control service nginx status
port 80
* In AEN 4.1.2 and earlier the wk-server process runs on port 5000 on localhost only. In later versions of
AEN the wk-server process uses the Unix socket path unix:/opt/wakari/wakari-server/var/run/
wakari-server.sock.
NGINX runs at least two processes:
• Master process running as root user.
• Worker processes running as nginx user.
Gateway node
The gateway node serves as an access point for a given group of compute nodes. It acts as a proxy service and
manages the authorization and mapping of URLs and ports to services that are running on those nodes. The gateway
nodes provide a consistent uniform interface for the user.
NOTE: The gateway may also be referred to as a data center because it serves as the proxy for a collection of compute
nodes.
You can put a gateway in each data center in a tiered scale-out fashion.
AEN gateway is installed in the /opt/wakari/wakari-gateway directory.
Gateway processes
When you view the status of server processes, you may see the processes explained below.
supervisord details
description Manages the wk-gateway process.
user wakari
configuration /opt/wakari/wakari-gateway/etc/supervisord.conf
log /opt/wakari/wakari-gateway/var/log/supervisord.log
control service wakari-gateway
ports none
wakari- details
gateway
descrip- Passes requests from the AEN Server to the Compute nodes.
tion
user wakari
configu- /opt/wakari/wakari-gateway/etc/wakari/wk-gateway-config.json
ration
logs /opt/wakari/wakari-gateway/var/log/wakari/gateway.application.log
/opt/wakari/wakari-gateway/var/log/wakari/gateway.log
working / (root)
dir
port 8089 (webcache)
Compute node(s)
Compute nodes are where applications such as Jupyter Notebook and Workbench actually run. They are also the
hosts that a user sees when using the Terminal app or when using SSH to access a node. Compute nodes contain all
user-visible programs.
Compute nodes only need to communicate with a gateway, so they can be completely isolated by a firewall.
Each project is associated with one or more compute nodes that are part of a single data center.
AEN compute nodes are installed in the /opt/wakari/wakari-compute directory.
Each compute node in the AEN system requires a compute launcher service to mediate access to the server and
gateway.
Compute processes
When you view the status of server processes, you may see the processes explained below.
supervisord details
description Manages the wk-compute process.
user wakari
configuration /opt/wakari/wakari-compute/etc/supervisord.conf
log /opt/wakari/wakari-compute/var/log/supervisord.log
control service wakari-compute
working dir /opt/wakari/wakari-compute/etc
ports none
wk- details
compute
de- Launches compute processes.
scrip-
tion
user wakari
con- /opt/wakari/wakari-compute/etc/wakari/wk-compute-launcher-config.json
figura- /opt/wakari/wakari-compute/etc/wakari/scripts/config.json
tion
logs /opt/wakari/wakari-compute/var/log/wakari/compute-launcher.
application.log /opt/wakari/wakari-compute/var/log/wakari/
compute-launcher.log
work- / (root)
ing
dir
con- service wakari-compute
trol
port 5002 (rfe)
AEN uses a process control system called “Supervisor” to run its services. Supervisor is run by the AEN Service
Account user, usually wakari or aen_admin.
The Supervisor daemon process is called “supervisord”. It runs in the background and should rarely need to be
restarted.
Service Account
AEN must be installed and executed by a Linux account called the AEN Service Account. The username of the
AEN Service Account is called the AEN Functional ID (NFI). The AEN Service Account is created during AEN
installation—if it does not exist—and is used to run all AEN services.
The default NFI username is wakari. Another popular choice is aen_admin.
WARNING: The Service Account should only be used for administrative tasks, and should not be used for operating
AEN the way an ordinary user would. If the Service Account creates or starts projects, the permissions on the AEN
package cache will be reset to match the Service Account, which will interfere with the normal operation of AEN for
all other users.
Anaconda environments
Each project has an associated conda environment containing the packages needed for that project. When a project is
first started, AEN clones a default environment with the name “default” into the project directory.
Each release of AEN 4 includes specific tested versions of conda and the conda packages included with AEN. These
tested conda packages include Python, R, and other packages, and these tested conda packages include all of the
packages in Anaconda.
If you upgrade or install different versions of conda or different versions of any of these conda packages, the new
packages will not have been tested as part of the AEN 4 release.
These different packages will usually work, especially if they are newer versions, but they are not tested or guaranteed
to work, and in some cases they may break product functionality.
You can use a new conda environment to test a new version of a package before installing it in your existing environ-
ments.
If using conda to change the version of a package breaks product functionality, you can use conda to change the version
of the package back to the version known to work.
For more information about environments, see Working with environments.
Files and subdirectories within the project directory have the same permissions as the project directory, except:
• The public folder and everything in it are open to anyone.
• Any files hardlinked into the root anaconda environment—/opt/wakari/anaconda—are owned by the
root or wakari users.
Project file and directory permissions are maintained by the start-project script. All files and directories
in the project will have their permissions set when the project is started, except for files owned by root or the
AEN_SRVC_ACCT user—by default, wakari or aen_admin.
The permissions set for files owned by root or the AEN_SRVC_ACCT user are not changed to avoid changing the
permissions settings of any linked files in the /opt/wakari/anaconda directory.
CAUTION: Do not start a project as the AEN_SRVC_ACCT user. The permissions system does not correctly manage
project files owned by this user.
Installation
Installation requirements
• Hardware requirements
• Software requirements
• Security requirements
• Network requirements
• Other requirements
• What’s next
Hardware requirements
Software requirements
• RHEL/CentOS on all nodes. Versions from 6.5 through 7.4 are supported. Other operating systems are sup-
ported. However, this document assumes RHEL or CentOS.
• Linux home directories—Jupyter looks in $HOME for profiles and extensions.
• Ability to install in AEN directory /opt/wakari with at least 10 GB of storage.
• Ability to install in Projects directory /projects with at least 20 GB of storage. Size depends on number and
size of projects.
NOTE: To install AEN in a different location see Installing AEN in a custom location.
Some Linux system accounts (UIDs) are added to the system during installation.
If your organization requires special actions, the following list is available:
• mongod (RHEL) or mongodb (Ubuntu/Debian)—created by the RPM or deb package.
• elasticsearch—created by RPM or deb package.
• nginx—created by RPM or deb package.
• AEN_SRVC_ACCT—created during installation of AEN, and defaults to wakari.
• ANON_USER—An account such as “public” or “anonymous” on the compute node.
NOTE: If ANON_USER is not found, AEN_SRVC_ACCT will attempt to create it. If it fails, the project(s) will
fail to start.
• ACL directories need the filesystem mounted with Posix ACL support (Posix.1e).
NOTE: You can verify ACL from the command line by running mount and tune2fs -l /path/to/
filesystem | grep options.
Software prerequisites
• AEN server:
– Mongo—Equal to or higher than version 2.6.8 and lower than version 3.0.
– NGINX—Equal to or higher than version 1.6.2.
– Elasticsearch—Equal to or higher than version 1.7.2.
– Oracle JRE version 7 or 8.
– bzip2.
• AEN Gateway:
– bzip2.
• AEN compute:
– git
– bzip2
– bash or zsh
– X Window System
NOTE: If you don’t want to install the whole X Window System, you must install the following packages
to have R plotting support:
Security requirements
/etc/sysconfig/selinux
SELINUX=enforcing
SELINUXTYPE=targeted
Network requirements
TCP Ports:
Other requirements
As long as the above requirements are met, there are no additional dependencies for AEN.
See also system requirements for Anaconda Repository and Anaconda Scale.
What’s next
RPM_CDN="https://820451f3d8380952ce65-4cc6343b423784e82fd202bb87cf87cf.ssl.cf1.
˓→rackcdn.com"
curl -O $RPM_CDN/aen-server-4.3.2-Linux-x86_64.sh
curl -O $RPM_CDN/aen-gateway-4.3.2-Linux-x86_64.sh
curl -O $RPM_CDN/aen-compute-4.3.2-Linux-x86_64.sh
NOTE: The current $RPM_CDN server will be confirmed in an email provided by your sales rep.
NOTE: These instructions use curl or wget to download packages, but you may use other means to move the necessary
files into the installation directory.
AEN is very sensitive to the IP address or domain name used to connect to the server and gateway nodes. If users
will be using the domain name, you should install the nodes using the domain name instead of the IP addresses. The
authentication system requires the proper hostnames when authenticating users between the services.
Print this page and fill in the domain names or IP addresses of the nodes below and record the user name and auto-
generated password for the administrative user account in the box below after installing the AEN server node:
AEN gateway |
AEN compute |
NOTE: The values of these IP entries or DNS entries are referred to as <AEN_SERVER_IP> or
<AEN_SERVER_FQDN>, particularly in examples of shell commands. Consider actually assigning those values to
environment variables with similar names.
Set up variables
Certain variables need to have values assigned to them before you start the installation.
NOTE: The address—FQDN or IP—specified for the AEN server must be resolvable by your intended AEN users’
web clients.
To verify your hostname, run echo $AEN_SERVER.
AEN functional ID
AEN must be installed and executed by a Linux account called the AEN Service Account. The username of the
AEN Service Account is called the AEN Functional ID (NFI). The AEN Service Account is created during AEN
installation—if it does not exist—and is used to run all AEN services.
The default NFI username is wakari. Another popular choice is aen_admin.
To set the environment variable AEN_SRVC_ACCT to wakari or your chosen name before installation, run export
AEN_SRVC_ACCT="aen_admin".
This name is now the username of the AEN Service Account and of the AEN administrator account.
When upgrading AEN, set the NFI to the NFI of the current installation.
WARNING: The Service Account should only be used for administrative tasks, and should not be used for operating
AEN the way an ordinary user would. If the Service Account creates or starts projects, the permissions on the AEN
package cache will be reset to match the Service Account, which will interfere with the normal operation of AEN for
all other users.
The AEN Functional Group (NFG) may be given any name. Most often, it is set to aen_admin or wakari. This
Linux group includes the AEN service account, so all files and directories that have the owner NFI also have the group
NFG.
When upgrading AEN, set the NFG to the NFG of the current installation.
To set the NFG before installation, run:
export AEN_SRVC_GRP="<NFG>"
During AEN installation the installers perform various operations that require root level privileges. By default, the
installers use the sudo command to perform these operations.
Before installation, set the AEN_SUDO_CMD_INSTALL environment variable to perform root level operations. You
can also set it to no command at all if the user running the installer(s) has root privileges and the sudo command is not
needed or is not available.
EXAMPLES:
export AEN_SUDO_CMD_INSTALL=""
export AEN_SUDO_CMD_INSTALL="sudo2"
By default the AEN services uses sudo -u to perform operations on behalf of other users—including mkdir,
chmod, cp and mv.
To override the default sudo command when sudo is not available on the system, before installing, set the
AEN_SUDO_CMD environment variable.
AEN must have the ability to perform operations on behalf of other users. Therefore, this environment variable cannot
be set to an empty string or to null.
CAUTION: Any command that replaces AEN_SUDO_CMD must support the -u command line parameter—similarly
to the sudo command.
EXAMPLE:
export AEN_SUDO_CMD="sudo2"
The optional environmental variable AEN_SUDO_SH is another way to customize AEN sudo operations. When AEN
executes any sudo command, it will include the value of AEN_SUDO_SH, if it is set.
EXAMPLE: If your username is “jsmith” and the values are set as:
AEN_SUDO_CMD=sudo
OWNER=jsmith
AEN_SUDO_SH=sudologger
PROJECT_HOME=/projects/jsmith/myproj
As:
In this case the sudologger utility could be a pass-through utility that logs all sudo usage and then executes the remain-
ing parameters.
While root/sudo privileges are required during installation, root/sudo privileges are not required during normal oper-
ations after install, if user accounts are managed outside the software. However root/sudo privileges are required to
start the services, thus in the service config files there may still need to be an AEN_SUDO_CMD entry.
For more information, see Configuring sudo customizations.
By default AEN server uses a local database. To override the default database location, see Install AEN connected to
a remote Mongo DB instance.
What’s next
The AEN server is the administrative front end to the system. This is where users log in to the system, where user
accounts are stored, and where admins can manage the system.
Server is installed in the /opt/wakari/wakari-server directory.
Be sure you have the bzip2 package installed. If this package is not installed on your system, install it:
RPM_CDN="https://820451f3d8380952ce65-4cc6343b423784e82fd202bb87cf87cf.ssl.cf1.
˓→rackcdn.com"
curl -O $RPM_CDN/nginx-1.6.2-1.el6.ngx.x86_64.rpm
curl -O $RPM_CDN/mongodb-org-tools-2.6.8-1.x86_64.rpm
curl -O $RPM_CDN/mongodb-org-shell-2.6.8-1.x86_64.rpm
curl -O $RPM_CDN/mongodb-org-server-2.6.8-1.x86_64.rpm
curl -O $RPM_CDN/mongodb-org-mongos-2.6.8-1.x86_64.rpm
curl -O $RPM_CDN/mongodb-org-2.6.8-1.x86_64.rpm
curl -O $RPM_CDN/elasticsearch-1.7.2.noarch.rpm
curl -O $RPM_CDN/jre-8u65-linux-x64.rpm
RPM_CDN="https://820451f3d8380952ce65-4cc6343b423784e82fd202bb87cf87cf.ssl.cf1.
˓→rackcdn.com"
curl -O $RPM_CDN/nginx-1.10.2-1.el7.ngx.x86_64.rpm
curl -O $RPM_CDN/mongodb-org-tools-2.6.12-1.x86_64.rpm
curl -O $RPM_CDN/mongodb-org-shell-2.6.12-1.x86_64.rpm
curl -O $RPM_CDN/mongodb-org-server-2.6.12-1.x86_64.rpm
curl -O $RPM_CDN/mongodb-org-mongos-2.6.12-1.x86_64.rpm
curl -O $RPM_CDN/mongodb-org-2.6.12-1.x86_64.rpm
curl -O $RPM_CDN/jre-8u112-linux-x64.rpm
curl -O $RPM_CDN/elasticsearch-1.7.6.noarch.rpm
Run:
Run:
NOTE: Change <FQDN HOSTNAME OR IP ADDRESS> to the actual fully qualified domain hostname or IP address.
Run:
PREFIX=/opt/wakari/wakari-server
Logging to /tmp/wakari_server.log
Checking server name
Ready for pre-install steps
Installing miniconda
...
...
Checking server name
Loading config from /opt/wakari/wakari-server/etc/wakari/config.json
Loading config from /opt/wakari/wakari-server/etc/wakari/wk-server-config.json
===================================
===================================
After successfully completing the installation script, the installer creates the administrator ac-
count—AEN_SRVC_ACCT user—and assigns it a password.
EXAMPLE:
Created password '<RANDOM_PASSWORD>' for user 'aen_admin'
TIP: Record this password. It will be needed in the following steps. It is also available in the installation log file
/tmp/wakari_server.log.
When SELinux is enabled, it blocks NGINX from connecting to the socket created by Gunicorn. If you have SELinux
enabled, run these commands to correct these permissions and allow connections between NGINX and Gunicorn:
sudo semanage fcontext -a -t httpd_var_run_t "/opt/wakari/wakari-server/var/run/
˓→wakari-server.sock"
TIP: If the AEN web page shows an NGINX 404 error, restart NGINX:
sudo nginx -s stop
sudo nginx
Visit http://\protect\T1\textdollarAEN_SERVER.
The License expired page is displayed.
From the License expired page, follow the onscreen instructions to upload your license file.
After your license is submitted, you will see this page:
What’s next
The gateway is a reverse proxy that authenticates users and automatically directs them to the proper AEN compute
node for their project. Users will not notice this node as it automatically routes them.
Gateway is installed in the /opt/wakari/wakari-gateway directory.
Run:
NOTE: Change <FQDN HOSTNAME OR IP ADDRESS> to the actual fully qualified domain hostname or IP address.
NOTE: You must perform the entire procedure before closing the terminal to ensure the variable export persists. If the
terminal is closed before successful installation, export the variables to continue with the installation.
Run:
PREFIX=/opt/wakari/wakari-gateway
Logging to /tmp/wakari_gateway.log
...
...
Checking server name
Please restart the Gateway after running the following command
to connect this Gateway to the AEN Server
...
sudo /opt/wakari/wakari-gateway/bin/wk-gateway-configure \
--server http://$AEN_SERVER --host $AEN_GATEWAY \
--port $AEN_GATEWAY_PORT --name Gateway --protocol http \
--summary Gateway --username $AEN_SRVC_ACCT \
--password '<NFI USER PASSWORD>'
NOTE: replace <NFI USER PASSWORD> with the password of the NFI user that was generated during server in-
stallation.
Setting permissions
Run:
Run:
1. Log into the AEN server using the Chrome or Firefox browser and the AEN_SRVC_ACCT user.
2. In the AEN navigation bar, click Admin to open the Admin Settings page.
3. In the Site Admin menu, select Data Centers:
5. Verify that your data center is registered and the status is {"status": "ok", "messages": []}:
What’s next
Run:
NOTE: Change <FQDN HOSTNAME OR IP ADDRESS> to the actual fully qualified domain hostname or IP address.
NOTE: You must perform the entire procedure before closing the terminal to ensure the variable export persists.
Run:
Once installed, you must configure the compute launcher on your server:
1. In your browser, go to your AEN server.
2. Log in as the AEN_SRVC_ACCT user.
3. In the AEN navigation bar, click Admin to open the Admin Settings page.
4. In the Providers menu, select Enterprise Resources:
5. Click the Add Resource button to open the new resource form.
6. Select the data center to associate this compute node with.
What’s next
You can configure AEN to use a local on-site Anaconda Repository server instead of Anaconda.org.
To configure AEN to use a local on-site Repository, you must:
1. Edit condarc on the compute node.
2. Configure the Anaconda client.
NOTE: If there are channels that you haven’t mirrored, you must remove them from the configuration.
Edit the file .condarc to match the following:
#/opt/wakari/anaconda/.condarc
channels:
- defaults
create_default_packages:
- anaconda-client
- ipykernel
# Default channels is needed for when users override the system .condarc
# with ~/.condarc. This ensures that "defaults" maps to your Anaconda Repository and
˓→not
# repo.anaconda.com
default_channels:
- http://<your Anaconda Repository name>:8080/conda/anaconda
- http://<your Anaconda Repository name>:8080/conda/wakari
- http://<your Anaconda Repository name>:8080/conda/r-channel
NOTE: Replace <your Anaconda Repository name> with the actual name or IP address of your local Ana-
conda Repository installation.
Anaconda client lets users work with Repository from the command-line—including searching for packages, logging
in, uploading packages, and more.
To set the default configuration of anaconda-client for all users on your compute node:
NOTE: Sudo is required because the configuration file is written to the root file system: /etc/xdg/binstar/
config.yaml.
NOTE: Replace <your Anaconda Repository> with the actual name or IP address of your local Anaconda
Repository installation.
What’s next
Review the optional configuration tasks to see if any apply to your system.
Optional configuration
NOTE: If you set uniqueEmail to false, you must drop the existing index in the database. EXAMPLE: If the
index name is email_1, run db.users.dropIndex("email_1").
uniqueUserName true A boolean setting that indicates whether unique usernames are
required.
To fix any errors, edit the configuration file and verify that it contains the correct JSON syntax.
"httpTimeout": 600
2. Update the gateway node by modifying the httpTimeout key in the /opt/wakari/wakari-gateway/
etc/wakari/wk-gateway-config.json file to match the above settings.
3. Restart the AEN compute service:
NOTE: We recommend putting /opt/wakari and /projects on the same filesystem. If the project and conda
env directories are on separate filesystems then more disk space will be required on compute nodes and performance
will be worse.
To make aen-compute service use a different directory than /projects to store your AEN projects:
1. Modify the /opt/wakari/wakari-compute/etc/wakari/wk-compute-launcher-config.
json file:
"projectRoot" : "/nfs/storage/services/wakari/projects",
To install AEN with multiple compute nodes and a /projects folder on an NFSv3 volume, manually pre-create
both the anonymous user and the $AEN_SRVC_ACCOUNT user on all nodes. Each of these users must have the same
user identity number (UID) and group identity number (GID) on all nodes.
By default AEN creates local users with a different GID on each node. To make the AEN compute service create
groups with the same GID:
1. In the /opt/wakari/wakari-compute/etc/wakari/wk-compute-launcher-config.json
file, change the identicalGID key value to true:
, "identicalGID": true
1. In the /opt/wakari/wakari-compute/etc/wakari/wk-compute-launcher-config.json
file, change the numericUsernames key value to true.
, "numericUsernames": true
The projDirsAsHome option changes the AEN home directories from the standard /home/<username> loca-
tion to the project directories and the location /projects/<username>/<project_name>/<username>/.
This ensures that AEN and AEN apps will not be affected by configuration files in a user’s home directory, such as
.bashrc or configuration files in subdirectories such as .ipython and .jupyter.
AEN version 4.1.3 stores the cache of packages in /home/<username>, while AEN versions 4.2.0 and higher
store the cache of packages in /projects/<username>/<project_name>/<username>/. By moving the
package cache to the same filesystem as the project, AEN versions 4.2.0 and higher can use hardlinks and save disk
space and time when creating or cloning environments.
These package cache locations are not affected by the projDirsAsHome option.
After upgrading from AEN 4.1.3 to AEN 4.2.0 or higher, existing projects will still use the package cache in /home/
<username>. Do not remove this cache, or the existing projects will break.
When users create new projects or install packages, the newly installed packages will use the new cache location.
If you wish to remove the older package cache in /home/<username>:
• Upgrade AEN to 4.2.0 or higher.
• Use conda remove to remove every non-default package in every project.
• Use conda install to replace them. The replaced packages will link to the new package cache in /
projects/<username>/<project_name>/<username>/.
• You can now safely remove the older package cache.
Enabling projDirsAsHome
NOTE: The projDirsAsHome option should be enabled immediately after performing the installation process and
before any users have logged in to AEN. This ensures that users will not have home directories in different places due
to some creating their home directories when the option was disabled and others creating their home directories when
the option was enabled.
1. In the /opt/wakari/wakari-compute/etc/wakari/wk-compute-launcher-config.json
file, add the projDirsAsHome key value and set it to true.
, "projDirsAsHome": true
NOTE: You must add the comma at the beginning of the line. If you add this line as the last key, you must
remove any comma at the end of the line.
2. Restart the AEN compute service:
AEN includes a full installation of the Anaconda Python distribution—along with several additional pack-
ages—located within the root conda environment in /opt/wakari/anaconda.
The first time any new AEN project is started, this default project environment is cloned into the new project’s
workspace.
To configure a different set of packages than the default:
1. Create a new conda environment in the /opt/wakari/anaconda/envs/default directory.
EXAMPLE: Using a Python 3.4 base environment, run:
For more information and examples about creating a default project environment with Microsoft R Open (MRO), see
Using MRO in AEN.
In AEN 4.2.2 and higher, you can choose to create environments with the Microsoft R Open (MRO) interpreter by
installing the mro-base package, or create environments with the R interpreter by installing the r-base package.
Unless you request a change, conda will continue to use the existing interpreter in each environment. In AEN r-base
is the default.
EXAMPLE: To create a custom environment called mro_env with MRO and R Essentials:
.. code-block:: bash
NOTE: Conda 4.4 and higher include the main channel by default. Earlier versions of conda do not.
You can also create an environment with MRO and make this the default AEN project environment.
The first time a new project is started, the default project environment is cloned into the new project’s workspace.
1. Create a new conda environment in the /opt/wakari/anaconda/envs/default directory.
The command is similar to the one used in the previous example to create a custom environment.
$ export MONGO_URL="mongodb://<username>:<password>@<host>:<port>/"
$ export MONGO_DB="<database_name>"
To configure your remote database to work with an already installed AEN server:
1. Stop the server, gateway and compute nodes:
{
"MONGO_URL": "mongodb://MONGO-USER:MONGO-PASSWORD@MONGO-URL:MONGO-PORT",
"MONGO_DB": "MONGO-DB-NAME",
"WAKARI_SERVER": "http://YOUR-IP",
"USE_SES": false,
"CDN": "http://YOUR-IP/static/",
"ANON_USER": "anonymous"
}
For more information about configuration keys, see Using configuration files.
3. Migrate the data from the former database into the new one. For more information, see the MongoDB docu-
mentation website.
4. After migration, restart the nodes:
To run SELinux in Enforcing mode, a few ports must be set up using the semanage port command.
The semange command relies on policycoreutils-python. To install policycoreutils-python, if
needed, run:
It is possible to change the domain names (hostnames) of the various AEN nodes by updating the configuration files.
NOTE: After the configuration files are updated, the associated nodes need to be restarted.
To edit the information for all of the data centers that you are changing the base domain name for:
1. Go to the Site Admin section of the Admin Settings page.
2. In the Data Centers section, click the Edit button.
3. Make any necessary updates.
NOTE: This must include the service port if it is different from the default—80 for HTTP and 443 for HTTPS.
4. In the Enterprise Resources sub-section of the Providers section, edit each compute node that has a changed
domain name.
NOTE: These URLs should include the protocol, hostname and port.
Anaconda Enterprise Notebooks performs local authentication against accounts in the AEN database by default.
To configure AEN to authenticate against accounts in an LDAP (Lightweight Directory Access Protocol) server, follow
the instructions below.
The system needs OpenLDAP libraries to be installed and accessible by AEN. AEN uses the OpenLDAP libraries to
establish an LDAP connection to your LDAP servers.
To install OpenLDAP on Ubuntu or Debian, follow the official OpenLDAP installation instructions.
Configuring OpenLDAP
{
"accounts":"wk_server.plugins.accounts.ldap2",
"LDAP" : {
"URI": "ldap://openldap.EXAMPLE.COM",
"BIND_DN": "cn=Bob Jones,ou=Users,DC=EXAMPLE,DC=COM",
"BIND_AUTH": "secretpass",
"USER_SEARCH": {"base": "DC=EXAMPLE,DC=COM",
"filter": "(| (& (ou=Payroll)
(uid=%(username)s))
(& (ou=Facilities)
(uid=%(username)s)))"
},
"KEY_MAP": {"email": "mail",
"name": "cn"
}
}
}
• URI—The IP address or hostname of your OpenLDAP server. For SSL/TLS, use the ldaps:// prefix
and specify a TLS_CACERT as described in the SSL/TLS configuration section below.
• BIND_DN—The full directory path of the user you want AEN server to bind as.
• BIND_AUTH—The password of the BIND_DN user.
• USER_SEARCH:
– base—The level at which you want to start the search.
– filter—The default is to search for the sAMAccountName attribute, and use its value for the AEN
server username field.
• KEY_MAP—Maps user attributes in AEN server to LDAP user attributes.
EXAMPLE: The mail attribute in LDAP maps to the email attribute in AEN server.
3. Restart AEN server to load new settings.
4. Log in with the admin account. This creates the admin user in the local database.
5. As soon as LDAP is installed, LDAP authentication takes over, so you need to add your admin account again:
Microsoft Active Directory is a server program that provides directory services and uses the open industry standard
Lightweight Directory Access Protocol (LDAP).
To enable Active Directory support:
1. Open the /opt/wakari/wakari-server/etc/wakari/wk-server-config.json file.
2. Add the following LDAP settings:
{
"accounts":"wk_server.plugins.accounts.ldap2",
"LDAP" : {
"URI": "ldap://<ad.EXAMPLE.COM>",
"BIND_DN": "CN=Bind User,CN=Users,DC=EXAMPLE,DC=COM",
"BIND_AUTH": "secretpass",
"USER_SEARCH": {"base": "CN=Users,DC=EXAMPLE,DC=COM",
"filter": "sAMAccountName=%(username)s"
},
"KEY_MAP": {"email": "mail",
"name": "cn"
}
}
}
• URI—The IP address or hostname of your Active Directory server. Replace <ad.EXAMPLE.COM> with
the actual URI. For SSL/TLS, use the ldaps:// prefix and specify a TLS_CACERT as described in the
SSL/TLS configuration section below.
• BIND_DN—The full directory path of the user you want AEN server to bind as.
• BIND_AUTH—The password of the BIND_DN user.
• USER_SEARCH:
– base—the level at which you want to start the search.
– filter—default is to search for the sAMAccountName attribute, and use its value for the AEN server
username field.
• KEY_MAP—Maps user attributes in AEN server to LDAP user attributes.
EXAMPLE: The mail attribute in LDAP maps to the email attribute in AEN server.
3. Restart AEN server to load new settings.
4. Log in with the admin account. This creates the admin user in the local database.
5. As soon as LDAP is installed, LDAP authentication takes over, so you need to add your admin account again:
Configuring SSL/TLS
TLS_CACERT /path/to/CA.cert
NOTE: CA.cert is the Certificate Authority used to sign the LDAP server’s SSL certificate. In the case of a self-
signed SSL certificate, this is the path to the SSL certificate itself.
/opt/wakari/wakari-server/bin/flask-ldap-login-check \
wk_server.wsgi:app \
-u [username] \
-p [password]
NOTE: username is the username of a valid user and password is that user’s BIND_AUTH password.
If your organization’s IT security policy does not allow root access or has restrictions on the use of sudo, after AEN
installation, you may customize AEN to meet their requirements.
Your organization may choose to implement any or all of the following:
• Remove root access for AEN service account (Note: this restricts AEN from managing user accounts).
• Configurable sudo command.
• Restrict sudo access to all processes.
These customizations must be done in a terminal window after copying the files to the server node.
Because root access is required for useradd, the following process restricts AEN from managing user accounts.
1. Modify the /etc/sudoers.d/wakari_sudo file to read:
NOTE: If you used a service account name other than wakari, enter that name instead of wakari.
2. Modify the /opt/wakari/wakari-compute/etc/wakari/config.json file to read:
"MANAGE_ACCOUNTS": false,
Using this option means that your IT department must create and manage all user accounts at the OS level.
After an OS-level account exists, you may create on the main AEN web page an AEN account using the same name.
The password you choose is not linked in any way to the OS-level password for the account.
Alternatively, you can configure the system to use LDAP for authenticating users.
A public account is visible to anyone who can access the AEN server. The name of this account can be configured to
any name you wish. For example, public or anonymous. To disable this feature use the special value disabled.
1. In the /opt/wakari/wakari-compute/etc/wakari/wk-compute-launcher-config.json
file, modify the ANON_USER line to read:
"ANON_USER": "public"
"ANON_USER": "public"
For more information about configuration keys, see Using configuration files.
You can use a sudo alternative as long as it supports the same execution semantics as the original sudo. The alternative
must be configured to give the service account permission to run commands on behalf of AEN users.
1. In your terminal window, open the /opt/wakari/wakari-compute/etc/wakari/config.json
file.
2. Modify the AEN_SUDO_CMD line to read:
"AEN_SUDO_CMD": "/path/to/alternative/sudo",
NOTE: If the alternate sudo command is available on PATH, then the full path is not required.
By default, sudoers is configured to allow AEN to run any command as a particular user which allows the platform to
initiate processes as the logged-in end user. If more restrictive control is required, it should be implemented using a
suitable sudoers policy. If that is not possible or practical, it is also possible to route all AEN ID-changing operations
through a single gatekeeper.
This gatekeeper wraps the desired executable and provides an alternate way to log, monitor, or control which processes
can be initiated by AEN on behalf of a user.
CAUTION: Gatekeeper is a special case configuration and should only be used if required.
To configure an AEN gatekeeper:
1. Modify the /etc/sudoers.d/wakari_sudo file to contain:
EXAMPLE: The gatekeeper can be as simple as a script with contents such as:
#!/bin/bash
first_cmd=$1
if [ 'bash' == $1 ]; then
shift
export HOME=~
export SHELL=/bin/bash
export PATH=$PATH:/opt/wakari/anaconda/bin
bash "$@"
else
exec $@
fi
Configuring SSL
The server node uses NGINX to proxy all incoming http(s) requests to the server running on a local port, and uses
NGINX for SSL termination. The default setup uses http—non-SSL—since cert files are required to configure SSL
and each enterprise will have their own cert files.
The www.enterprise.conf file is the default nginx.conf file used for AEN. It is copied to the /etc/
nginx/conf.d directory during server installation.
NOTE: This section describes setting up SSL after your gateway node has been installed and registered with the server
node.
• The Gateway CA bundle must contain the full chain: root CA, any intermediate authority and the certificate.
• The Server CA bundle must be separated into individual files for the root CA, any intermediate and the certifi-
cate.
The www.enterprise.https.conf is an NGINX configuration file for SSL. It is set up to use the server.crt
and server.key cert files.
CAUTION: You must change these values to point to the signed cert files for your domain.
NOTE: Self-signed certs or those signed by a private root CA require additional configuration.
Perform the following steps as root:
1. Stop NGINX:
6. Update the WAKARI_SERVER and CDN settings to use https instead of http in the following configuration
files:
/opt/wakari/wakari-server/etc/wakari/config.json
/opt/wakari/wakari-gateway/etc/wakari/wk-gateway-config.json
/opt/wakari/wakari-compute/etc/wakari/config.json
"verify_gateway_certificate": "/opt/wakari/wakari-server/etc/gateway.crt"
NOTE: This step may return an error since the gateway has not yet been configured for SSL.
10. In AEN, verify that the browser uses https. On the Admin Settings page, under Data Centers, click Gateway,
then select https:
{
EXISTING_CONFIGURATION,
"https": {
"key": "/opt/wakari/wakari-gateway/etc/gateway.key",
"cert": "/opt/wakari/wakari-gateway/etc/gateway.crt"
}
}
2. For a server certificate signed by a private root CA or signed by an intermediate authority, add:
{
EXISTING_CONFIGURATION,
"https": {
"key": "/opt/wakari/wakari-gateway/etc/gateway.key",
"cert": "/opt/wakari/wakari-gateway/etc/gateway.crt",
"ca": ["/opt/wakari/wakari-gateway/etc/server.crt"]
}
}
NOTE: When the certificate chain has more than one intermediate cert signed by a higher root CA authority,
you must manually break up the certs in the chain into individual files, and enumerate them in the ca key:
{
EXISTING_CONFIGURATION,
"https": {
(continues on next page)
{
EXISTING_CONFIGURATION,
"https": {
"key": "/opt/wakari/wakari-gateway/etc/gateway.key",
"cert": "/opt/wakari/wakari-gateway/etc/gateway.crt",
"passphrase": "mysecretpassphrase"
}
}
NOTE: Alternatively, the passphrase can be passed using an environment variable or entered when the wakari-
gateway service is manually started.
EXAMPLES:
Anaconda Enterprise does not support direct SSL on Compute Nodes. If you need SSL on Compute Nodes, you must
install each Compute Node on the same server as a Gateway using http://localhost:5002 for the URL value
while adding it as a resource, and you must use a Gateway for each and every Compute Node.
Security reminder
The permissions on the cert files must be set correctly to prevent them from being read by others. Since NGINX is run
by the root user, only the root user needs read access to the cert files.
EXAMPLE: If the cert files are called server.crt and server.key, then use the root account to set permissions:
AEN’s single sign-on (SSO) capability creates a new authentication provider that defers to your Anaconda Repository
for login and authentication cookies.
To enable SSO:
1. Deploy AEN and Repository on the same machine.
2. In the /opt/wakari/wakari-server/etc/wakari/config.json file, add:
{
EXISTING_CONFIGURATION,
"SECRET_KEY": "<repo signing secret>",
"REPO_LOGIN_URL":
"http://example_repo.com:8080/account/login?next=http://example_repo.com/"
}
{
EXISTING_CONFIGURATION,
"accounts": "wk_server.plugins.accounts.repo",
}
5. If you are using Repository version 2.33.3 through 2.33.10, set USE_SERVER_BASED_SESSIONS: false
in the Repository configuration.
This setting affects the network security properties of AEN and Repository. Specifically, if
USE_SERVER_BASED_SESSIONS is set to false, and if a new cross-site scripting (XSS) vulnerability
is discovered, it could expose an additional server fixation vulnerability. Please discuss this with your Ana-
conda representative and be sure the feature is compatible with your network requirements before setting
USE_SERVER_BASED_SESSIONS: false.
SSO is enabled.
Anaconda officially supports and tests functionality of the default environment(s) only for those extensions that ship
with AEN.
It is possible to add third-party and custom extensions from conda-forge or pip, but doing so may cause instability in
your default project environments or kernels.
CAUTION: Anaconda does not officially support third-party extensions. This section is informational only.
TIP: Always back up and verify your complete system before installing extensions.
The jupyter-contrib-nbextensions extensions are installed on a compute node.
The default conda executable directory for AEN is /opt/wakari/anaconda/bin/conda. If you are installing
a Jupyter extension, it must be installed in the wakari-compute directory.
EXAMPLE: Run:
For search indexing to work correctly, verify that the AEN Compute node can communicate with the AEN Server.
There must be at least one inotify watch available for the number of subdirectories within the project root filesys-
tem. Some Linux distributions default to a low number of watches, which can prevent the search indexer from moni-
toring project directories for changes.
cat /proc/sys/fs/inotify/max_user_watches
If necessary, increase the number of max user watches with the following command:
There must be at least one inotify user instance available per project.
cat /proc/sys/fs/inotify/max_user_instances
These instructions add a custom Jupyter Notebook option to allow users to select PySpark as the kernel.
Install Spark
You may create the kernel as an administrator or as a regular user. Read the instructions below to help you choose
which method to use.
1. As an administrator
Create a new kernel and point it to the root env in each project. To do so create a directory ‘pyspark’ in
/opt/wakari/wakari-compute/share/jupyter/kernels/.
Create the following kernel.json file:
{"argv": ["/opt/wakari/anaconda/bin/python",
"-m", "ipykernel", "-f", "connection_file}", "--profile", "pyspark"],
"display_name":"PySpark", "language":"python" }
To have an admin level PySpark kernel without the user .ipython space:
{"argv":
["/opt/wakari/wakari-compute/etc/ipython/pyspark.sh", "-f", "{connection_file}"],
"display_name":"PySpark", "language":"python" }
NOTE: The pyspark.sh script is defined in Without IPython profile section below.
3. As a regular user
Create a new directory in the user’s home directory: .local/share/jupyter/kernels/pyspark/. This way the user will be
using the default environment and able to upgrade or install new packages.
{"argv": ["/projects/<username>/<project_name>/envs/default/bin/python",
"-m", "ipykernel", "-f", "connection_file}", "--profile", "pyspark"],
"display_name":"PySpark", "language":"python" }
NOTE: Replace “<username>” with the correct user name and “<project_name>” with the correct project name.
You may choose any name for the ‘display_name’.
The above profile call from the kernel requires that we define a particular PySpark profile. This profile should be
created for each user that logs in to AEN to use the PySpark kernel.
In the user’s home, create the directory and file ~/.ipython/profile_pyspark/startup/
00-pyspark-setup.py with the file contents:
import os
import sys
# The place where CDH installed spark, if the user installed Spark locally it can be
˓→changed here.
os.environ["SPARK_HOME"] = "/usr/lib/spark"
os.environ["PYSPARK_PYTHON"] = "/opt/wakari/anaconda/bin/python"
Now log in using the user account that has the PySpark profile.
If it is necessary to avoid creating a local profile for the users, a script can be made to be called from the kernel. Create
a bash script that will load the environment variables:
#!/usr/bin/env bash
# setup environment variable, etc.
export PYSPARK_PYTHON="/opt/wakari/anaconda/bin/python"
export SPARK_HOME="/usr/lib/spark"
Using PySpark
When creating a new notebook in a project, now there will be the option to select PySpark as the kernel. When creating
such a notebook you’ll be able to import pyspark and start using it:
NOTE: You can always add those lines and any other command you may use frequently in the PySpark setup file
00-pyspark-setup.py as shown above.
By default, AEN uses client-side session management which is vulnerable to session replay attacks if an attacker
manages to steal a valid session ID of a user.
To enable server-side session management:
1. Modify the /opt/wakari/wakari-server/etc/wakari/wk-server-config.json file:
"USE_SERVER_BASED_SESSIONS": true,
By default, when a user logs out, their open terminal sessions will remain active.
To disable this behavior:
1. Modify the /opt/wakari/wakari-server/etc/wakari/wk-server-config.json file:
"TERMINATE_TERMINALS_ON_LOGOUT": true,
"TERMINATE_TERMINALS_ON_LOGOUT": true,
Upgrading AEN
CAUTION: These instructions are for upgrading AEN to the current version 4.3.2 from 4.3.1 ONLY. Each version
must be upgraded iteratively from the previous version. Do not skip versions.
Upgrade instructions for previous versions:
• AEN 4.3.1 upgrade instructions
• AEN 4.3.0 upgrade instructions
• AEN 4.2.2 upgrade instructions
• AEN 4.2.1 upgrade instructions
• AEN 4.2.0 upgrade instructions
• AEN 4.1.3 upgrade instructions
• AEN 4.1.2 upgrade instructions
• AEN 4.1.1 upgrade instructions.
• AEN 4.1.0 upgrade instructions.
• AEN 4.0.0 upgrade instructions.
For upgrades from versions before those listed above, please contact your enterprise support representative.
NOTE: Named Service Account functionality is available with AEN 4.0.0+ for new installations only. It is not available
for upgraded installations. Contact your enterprise support representative for more information.
An AEN platform update requires that each instance of the 3 node types be upgraded individually:
• AEN Server
• AEN Gateway
• AEN Compute
The upgrade process requires that all AEN service instances be stopped, upgraded, and then restarted.
NOTE: Any commands that call for the root user can also be done using sudo.
If you encounter any difficulty during the upgrade process, see Troubleshooting which provides guidance on:
• processes
• configuration files
• log files
• ports
If you are unable to resolve an installation or upgrade problem, please contact your enterprise support representative.
CAUTION: Make a tested backup of your installation before starting the upgrade. Upgrading to a higher version of
AEN is not reversible. Any errors during the upgrade procedure may result in partial or complete data loss and require
restoring data from backups.
CAUTION: Terminate all AEN applications and stop all projects before starting the upgrade process.
Before upgrading each service on each host:
1. Suspend the services on each of the nodes:
2. Set the AEN Functional ID (“NFI”) and AEN Functional Group (“NFG”) to the NFI and NFG of the current
installation:
export AEN_SRVC_ACCT="wakari"
export AEN_SRVC_GRP="wakari"
NOTE: The default NFI is wakari, but aen_admin or any other name may be used instead.
For more information on NFI and NFG, see the installation instructions.
3. Install wget:
NOTE: You can choose to keep the old index, but if you detect any issues with the search capabilities after the
upgrade, you will need to run the following to start with a clean index:
pushd /tmp
wget http://j.mp/aen-server-update-4.3.2
4. Start Elasticsearch:
Or, if you do not want to use the search features, edit your server’s /opt/wakari/wakari-server/etc/
wakari/config.json file by adding the line "SEARCH_ENABLED": false.
5. Restart the NGINX server:
AEN server version >= 4.1.3 uses Unix sockets for communication with NGINX. Restart NGINX to load this
new configuration:
pushd /tmp
wget http://j.mp/aen-gateway-update-4.3.2
4. Connect to the gateway using your web browser with the correct http/https, hostname and port number.
Complete the following steps on each host where an AEN compute service is running:
1. Check for any wakari-indexer processes running:
NOTE: If you stopped all the projects, you will not see any wakari-indexer processes running.
Terminate any remaining wakari-indexer processes:
NOTE: The processes killed with killall are run by the $AEN_SRVC_ACCT user, so they can be killed
as root with sudo killall or killed as the $AEN_SRVC_ACCT user with sudo -u $AEN_SRVC_ACCT
killall. Example commands show the sudo killall option.
2. Check for any AEN applications processes running—Workbench, Viewer, Terminal or Notebook:
NOTE: If you stopped all the projects, you will not see any AEN app processes running.
Terminate any remaining AEN application processes by running one or more of the following:
channels:
- https://conda.anaconda.org/t/<TOKEN>/anaconda-nb-extensions
- r
- https://conda.anaconda.org/wakari
- defaults
create_default_packages:
- anaconda-client
- ipykernel
NOTE: Contact your enterprise support representative to get your token for the Anaconda channel referenced
above. Replace <TOKEN> with the actual token from your enterprise support representative.
4. Upgrade Anaconda in the root environment:
pushd /tmp
wget http://j.mp/aen-anaconda-update-4_3_2
pushd /tmp
wget http://j.mp/aen-compute-update-4.3.2
NOTE: When upgrading the wakari-compute environment, you may see ImportError warnings with some nbex-
tensions. As long as the Validating message is OK, the ImportError warnings are harmless—a consequence of
the post-link presence on those packages.
6. Initialize the root environment to prime the package cache:
NOTE: If you don’t want to install the whole X Window System, you must install the following packages to
have R plotting support:
13. Repeat this upgrade procedure for all compute nodes in your Data Center.
After upgrading
pushd /tmp
wget http://j.mp/aen-anaconda-update-4.3.2
To upgrade the customized default environments for every user and every project at /projects/<USER>/
<PROJECT>/envs/default, run these commands for every user as that user:
pushd /tmp
wget http://j.mp/aen-anaconda-update-4.3.2
NOTE: Replace <USER> with the user’s name. Replace <PROJECT> with the project name.
NOTE: Upgrading the default environment at /opt/wakari/anaconda/envs/default does NOT au-
tomatically upgrade the default environment in the users pre-existing projects. For pre-existing projects, the
upgrade, if requested, should be done on a per-user basis.
NOTE: These commands update packages listed in aen-anaconda-update-4.3.2 and do not update any
other package.
3. If you did not stop all your projects before upgrading, then the first time you start an application you will see an
error page requesting that you restart the application.
4. Restart the application to complete the upgrade.
5. If you still see old applications or icons after restart, reload the page to reset the browser cache.
Uninstalling AEN
Begin by setting the AEN Functional ID (NFI). The NFI is the username of the AEN Service Account which is used to
run all AEN services and is also the username of the AEN Admin account. The NFI may be any name. The default NFI
is wakari. The NFI is also often set to aen_admin. The NFI (and AEN Functional Group or NFG) are described
in the installation instructions.
Set the NFI with this command:
export AEN_SRVC_ACCT="aen_admin"
Replace the name aen_admin with the NFI that was set in your installation of Anaconda Enterprise Notebooks.
To remove a server node, run the following commands as root or sudo on the server node’s host system:
1. Stop the server processes:
2. Stop MongoDB:
3. Remove AEN server software, AEN database files and NGINX configuration:
rm -Rf /opt/wakari/wakari-server
rm -Rf /opt/wakari/miniconda
rm -Rf /var/lib/mongo/wakari*
rm -Rf /etc/nginx/conf.d/www.enterprise.conf
userdel $AEN_SRVC_ACCT
7. Check for and remove any references to “aen” or “wakari” from the root user’s .condarc file:
To uninstall a gateway node, run the following commands as root or sudo on the gateway host system:
1. Stop the gateway processes:
rm -Rf /opt/wakari/wakari-gateway
userdel $AEN_SRVC_ACCT
5. Check for and remove any references to “aen” or “wakari” from the root user’s .condarc file:
To remove a compute node, run the following commands as root or sudo on each compute node host system:
1. Stop the compute processes:
service wakari-compute stop
5. Check for and remove any references to “aen” or “wakari” from the root user’s .condarc file:
grep -i aen ~/.condarc
grep -i wakari ~/.condarc
CAUTION: This is an extreme measure and is not necessary in most instances. We recommend you create and verify
a backup before doing this or any other file removal.
To remove all AEN projects from all of your compute nodes:
rm -Rf /projects
This is a step-by-step guide to installing an Anaconda Enterprise Notebooks system comprised of a front-end server, a
gateway and compute machines.
If you have any questions about these instructions or you encounter any issues while installing AEN, please contact
your sales representative or Priority Support team.
When you have completed the installation process, review the optional configuration tasks to see if any are appropriate
for your system.
Distributed install
Single-box install
In a single-box install, both the server and the gateway need separate external ports since they are independent services
that are running on the same host in the single-box installation.
Both port 80 and port 8089 must be open on the firewall for a single-box install.
The compute node only receives connections from the gateway and server nodes and typically runs on port 80 or port
443.
User management
An administrator can make any other user an administrator—or remove their administrator permissions—by using
administrator commands in the Terminal application.
A user can also be designated as a superuser or as staff, giving them greater administrative privileges within the system.
Removing an administrator/superuser
When Open Registration is enabled, anyone who has access to the URL of your AEN server can create their own
account.
1. In the AEN navigation bar, click Admin to open the Admin Settings page.
2. In the Site Admin menu, select Accounts.
3. To open user registration, select the Open Registration checkbox. To close registration, clear the checkbox.
4. Click the Update button.
1. In the AEN navigation bar, click Admin to open the Admin Settings page.
2. In the Staff menu, select Password Reset:
NOTE: Replace SOME_USER with the username and SOME_PASSWORD with the password.
3. Log into AEN as the user.
Managing permissions
This page explains the admin commands used to manage user permissions.
To verify that all files in the /opt/wakari/anaconda directory are owned by the wakari user or group:
To fix the ownership settings of any files that are listed in the output:
The acl option must be enabled on the file system that contains the project root directory.
NOTE: By default, the project root directory is /projects.
To determine the project root directory where a custom projectRoot is configured:
The mount options or default options listed by tune2fs should indicate that the acl option is enabled.
EXAMPLE:
1. In the AEN navigation bar, click Admin to open the Admin Settings page.
2. In the Site Admin menu, select Users:
The Users section lists the all users who are signed up, the number of projects they have created and the last time they
logged on to AEN.
A user’s profile page includes a summary of the projects created by that user and a list of projects on which the user is
a team member.
1. In the AEN navigation bar, click Users to see a list of users who are currently logged into the system.
2. On the Users page, click the username of the user whose profile page you want to view.
1. In the AEN navigation bar, click Admin to open the Admin Settings page.
2. In the Staff menu, select Notification:
The Notification Settings section allows you to create a system message that can be relayed to users.
By default, notifications are off.
3. To turn on email notifications, select the radio button for the type of email service to use:
• SES to use Amazon Simple Email Service (SES).
• SMTP Email Server.
4. If you select SMTP Email Server, complete the SMTP Settings.
NOTE: If you get an error message after changing the SMTP settings, you may need to restart the server.
If you have multiple compute nodes available and want to move a project from one to another, the project must exist
on both nodes.
1. Verify that the project has been created on both compute nodes. You can use rsync for this job unless you
have a shared file system like nfs.
2. On the project home page, click the Project Settings icon to open the Project Settings page.
Deleting a user
Deleting a project
NOTE: Replace <username> with the actual username and <projectname> with the actual project name you are
removing.
System management
If you have administrator privileges, you see two additional links in the AEN navigation bar—Admin and Users:
To open the Admin dashboard, click the Admin link.
• Document purpose
• Important notes
• Server component steps
– Backup
* Mongo database
* AEN Server config files (including License file)
* Nginx config (if needed)
* SSL certificates (if needed)
– Restore
* Reinstall AEN-Server
* Restore Mongo database
* AEN Server config files (including License file)
* Nginx config (if needed)
* SSL certificates (if needed)
* Restart server
• Gateway component steps
– Backup
* Config files
* Custom .condarc file (if needed)
* SSL certificates (if needed)
– Restore
* Reinstall AEN-Gateway
* Config files
* Custom .condarc file (if needed)
* SSL certificates (if needed)
* Restart gateway
• Compute component steps
– Backup
* Config files
* Custom Changes (rare)
* Create user list
* Project files
* Full Anaconda (option 1)
* Partial Anaconda (option 2)
– Restore
* Reinstall AEN-Compute
* Config files
* Custom changes (rare)
* Create users
* Project files
* Full Anaconda (option 1)
* Partial Anaconda (option 2)
* Custom environments (if needed)
* Restart compute node
Document purpose
This document lays out the steps to backup and restore Anaconda Enterprise Notebooks (AEN) for Disaster Recovery.
It is not intended to provide High Availability. Each of the components (Server, Gateway and Compute) has its own
instructions and each may be done individually as needed. The steps primarily involve creating tar files of important
configuration files and data.
This document is written for a system administrator who is comfortable with basic Linux command line navigation
and usage.
To migrate to a new cluster, use these backup and restore instructions to back up the system from the old cluster and
restore it to the new cluster.
Important notes
Review the Concepts page to become familiar with the different components and how they work together.
Root or sudo access is required for some commands.
CAUTION: All commands MUST be run by $AEN_SRVC_ACCT (the account used to run AEN) except for those
commands explicitly indicated to run as root or sudo. If the commands are not run by the correct user, the installation
will not work, and a full uninstallation and reinstallation will be required!
These instructions assume that the fully qualified domain name (FQDN) has not changed for any of the component
nodes. If any of the FQDNs are not the same, additional steps will be needed.
Backup
Mongo database
This will create a single tar file called aen_mongo_backup.tar that includes only the database named “wakari”
that is used by AEN. It also generates a log of the database backup.
NOTE: These commands must be run by $AEN_SRVC_ACCT.
Create a tar file of all of the configuration files, including any license files.
NOTE: This command must be run by $AEN_SRVC_ACCT.
Make a copy of the nginx configuration file if it has been customized. The default configuration for the AEN server is
a symlink.
NOTE: This command must be run by $AEN_SRVC_ACCT.
Make a copy of the SSL certificates files (certfiles) for the server, including the key file, and a copy of the certfile for
the gateway, which is needed for verification if using self-signed or private CA signed certs.
Restore
Reinstall AEN-Server
This assumes that mongo was reinstalled as part of the reinstallation of the server component. Untar the mongo
database and restore it.
NOTE: These commands must be run by $AEN_SRVC_ACCT.
NOTE: The --drop option resets the $AEN_SRVC_ACCT user password and restores the database to the exact state
it was in at the time of backup. Please see the MongoDB documentation for more information about mongorestore
options for Mongo 2.6.
NOTE: AEN uses Mongo 2.6 by default. If you are using a different version, consult the documentation for your
version.
Untar the tar file of all of the configuration files, including any license files.
NOTE: This command must be run by $AEN_SRVC_ACCT.
Make sure the files are in /opt/wakari/wakari-server/etc/wakari/ and are owned by the
$AEN_SRVC_ACCT.
Make sure any modifications to the nginx configuration are either in /etc/nginx/conf.d or in /opt/wakari/
wakari-server/etc/nginx/conf.d/ with a proper symlink.
NOTE: This command must be run by $AEN_SRVC_ACCT.
Move any SSL certificate files to the locations indicated in the config files.
Restart server
Backup
Config files
Make a copy of SSL certificate files for the gateway (including the key file) and the certfile for the server (needed for
verification if using self-signed or private CA signed certs).
Restore
Reinstall AEN-Gateway
NOTE: Change <FQDN HOSTNAME OR IP ADDRESS> to the actual fully qualified domain hostname or IP address.
NOTE: You must perform the entire procedure before closing the terminal to ensure the variable export persists. If the
terminal is closed before successful installation, export the variables to continue with the installation.
Run:
PREFIX=/opt/wakari/wakari-gateway
Logging to /tmp/wakari_gateway.log
...
...
Checking server name
Please restart the Gateway after running the following command
to connect this Gateway to the AEN Server
...
Config files
Verify that the files are in /opt/wakari/wakari-gateway/etc/wakari/ and are owned by the
$AEN_SRVC_ACCT.
Move any SSL certificate files to the locations indicated in the config files.
Restart gateway
Backup
Config files
Manually backup any custom changes that were applied to the code. One change might be additional files in the
skeleton folder:
/opt/wakari/wakari-compute/lib/node_modules/wakari-compute-launcher/skeleton
AEN uses POSIX access control lists (ACLs) for project sharing, so the backup must preserve the ACL information.
This is done with a script that creates a file named users.lst containing a list of all users that have access to
projects on a given compute node. Download and run the script.
NOTE: These commands must be run by $AEN_SRVC_ACCT.
wget https://s3.amazonaws.com/continuum-airgap/misc/wk-compute-get-acl-users.py
chmod 755 wk-compute-get-acl-users.py
./wk-compute-get-acl-users.py
Project files
Create a tar of the projects directory with ACLs enabled. The default projects base location is /projects.
NOTE: This command must be run as root or with sudo.
If any changes have been made to the default Anaconda installation (additional packages installed or packages re-
moved), it is necessary to backup the entire Anaconda installation.
NOTE: This command must be run by $AEN_SRVC_ACCT.
If no changes have been made to the default installation of Anaconda, you may just backup the .condarc file and
any custom environments.
NOTE: If no custom shared environments have been created, the envs folder will not be present.
Restore
Reinstall AEN-Compute
NOTE: Change <FQDN HOSTNAME OR IP ADDRESS> to the actual fully qualified domain hostname or IP address.
NOTE: You must perform the entire procedure before closing the terminal to ensure the variable export persists.
Run:
Config files
NOTE: Verify that they are located in /opt/wakari/wakari-compute/etc/wakari and are owned by the
$AEN_SRVC_ACCT.
Manually restore any custom changes you saved in the backup section. If there are changes in the skeleton directory,
these files must be world readable or projects will refuse to start.
Create users
NOTE: Only create users with these instructions if your Linux machine is not bound to LDAP.
In order for the ACLs to be set properly on restore, all users that have permissions to the files must be available on the
machine. Ask your system administrator for the proper way to do this for your system, such as using the “useradd”
tool. A list of users that are needed was created in the backup process as a file named users.lst.
A process similar to the following useradd example will be suitable for most Linux systems.
NOTE: This command must be run by $AEN_SRVC_ACCT.
Project files
If you did a full backup of the full Anaconda installation, untar this file to /opt/wakari/anaconda.
NOTE: This command must be run by $AEN_SRVC_ACCT.
A user who is promoted to administrator can access administrator commands to perform advanced administrator tasks.
NOTE: Utility files are owned by, and should only be executed by, the AEN user who owns the files.
To display a list of all administrator commands:
ls -al /opt/wakari/wakari-server/bin/wk-*
To view help information for command, run the command followed by -h or --help.
EXAMPLE: To view help for the remove-user command:
/opt/wakari/wakari-server/bin/wk-server-admin remove-user -h
/opt/wakari/wakari-server/bin/wk-server-admin remove-project -h
1. In the AEN navigation bar, click Admin to open the Admin Settings page.
2. In the Staff menu, select Daily Report:
When an error occurs, a red dot is displayed in the AEN navigation bar next to the Admin link. The red dot is removed
when all exceptions are marked as “read.”
1. In the AEN navigation bar, click Admin to open the Admin Settings page.
2. In the Staff menu, select Exceptions:
The Exceptions section lists all errors that have occurred while AEN is running.
3. To see the details of an error, click the radio button next to the error. This also marks the error as “read.”
4. To mark all errors as read without reviewing each one, click the Mark all as read button.
1. In the AEN navigation bar, click Admin to open the Admin Settings page.
2. In the Site Admin menu, select Security Log:
The Security Log section lists all errors that have occurred that could potentially affect AEN security.
3. To view a user’s profile page, click their username in the Actor column.
4. To see the details of an error, click the Eye icon next to the error.
The error details are displayed:
1. In the AEN navigation bar, click Admin to open the Admin Settings page.
2. In the Site Admin menu, select Data Centers:
1. Click the Add DataCenter button to display the the Register a datacenter form.
2. In the Name box, type a Name for the new data center:
1. In the AEN navigation bar, click Admin to open the Admin Settings page.
2. In the Providers menu, select Enterprise Resources:
The Resources section lists your existing cloud and local resources.
Adding a resource
1. Click the Add Resource button to open the new resource form.
2. Complete the form:
Removing a resource
Click the Remove button next to the resource you want to remove.
NOTE: When you remove a resource assigned to a project, the project becomes orphaned. To fix an orphaned project,
move the project to a valid Compute Resource.
Managing services
The tasks on this page assume that the 3 AEN nodes are installed in the following locations:
• Server–/opt/wakari/wakari-server/.
• Gateway–/opt/wakari/wakari-gateway/.
• Compute-Launcher–/opt/wakari/wakari-compute/.
1. Run:
OR
2. Run:
Run:
OR
Run:
OR
Services should start automatically both when they are first installed and at any point when the system is restarted.
If you need to manually start an AEN service, you must start each node independently, because they may be running
on separate machines.
NOTE: The process is basically the same for each node, but the path to the correct commands vary.
To manually start a service:
Verifying that AEN services are set to start with the system
CAUTION: Do not stop or kill supervisord without first stopping wk-compute and any other processes that use it.
You must stop services on each node independently, because they may be running on separate machines.
To stop an AEN service:
• On the server node, run:
Compute nodes may have running processes that are not automatically stopped. To stop them, run:
To get a complete list of the processes running under the wakari user account, run ps -Hu wakari.
EXAMPLE:
root@server # ps -f -C nginx
UID PID PPID C STIME TTY TIME CMD
root 26303 1 0 12:18 ? 00:00:00 nginx: master process /usr/sbin/nginx -c /
˓→etc/nginx/nginx.conf
If extra wk-server, wk-gateway, wk-compute, or supervisord processes are present, use the kill command to remove
them to prevent issues with AEN.
You can safely restart any process that you remove in error.
In order for AEN to run, the dependencies mongodb and nginx must be up and running. If either of these fail to start,
AEN will not be served on port 80.
Check if nginx and mongod are both running (RHEL 6x):
$ sudo service nginx status
nginx (pid 25956) is running...
If either of these failed to start, tail the log files. The default location of log files is:
$ tail -n 50 /var/log/mongodb/mongod.log
1. In the AEN navigation bar, click Admin to open the Admin Settings page.
2. In the Site Admin menu, select Monitor:
1. In the AEN navigation bar, click Admin to open the Admin Settings page.
2. In the Site Admin menu, select Task Queue:
The Workers section lists the workers in the task queue and whether each worker is set at high, default or low
priority.
The Queues section provides information on the default and high priority queues.
3. To view all the tasks in a particular queue, in the Queues section, click the queue name.
When the AEN nodes cannot communicate with each other as intended, it can cause issues with you AEN platform
installation.
1. On the server, in the AEN navigation bar, click Admin to open the Admin Settings page.
2. In the Site Admin menu, select Data Centers:
3. For each data center in the list, check connectivity from the server to that gateway.
EXAMPLE: The gateway in this example is http://gateway.example.com:8089:
1. On the server, in the AEN navigation bar, click Admin to open the Admin Settings page.
2. In the Providers menu, select Enterprise Resources:
"WAKARI_SERVER": "http://wakari.example.com",
4. If a connection fails:
1. Ensure that gateways (data centers) and compute nodes (Enterprise Resources) are correctly configured on
the server.
2. Verify that processes are listening on the configured ports:
$ sudo netstat -nplt
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program
tcp 0 0 *:80 *:* LISTEN 26409/nginx
tcp 0 0 *:22 *:* LISTEN 986/sshd
tcp 0 0 127.0.0.1:25 *:* LISTEN 1063/master
tcp 0 0 *:5000 *:* LISTEN 26192/python
tcp 0 0 127.0.0.1:27017 *:* LISTEN 29261/mongod
tcp 0 0 *:22 *:* LISTEN 986/sshd
tcp 0 0 127.0.0.1:25 *:* LISTEN 1063/master
3. Check the firewall setting and logs on both hosts to ensure that packets are not being blocked or discarded.
For search indexing to work correctly, a compute node must be able to communicate with the server. To verify this:
1. Run:
curl -m 5 $AEN_SERVER > /dev/null
2. Verify that there are sufficient inotify watches available for the number of subdirectories within the project root
file system:
cat /proc/sys/fs/inotify/max_user_watches
NOTE: Some Linux distributions default to a low number of watches, which may prevent the search indexer
from monitoring project directories for changes.
3. If necessary, increase the number of watches:
4. Verify that there are sufficient inotify user instances available—at least one per project:
cat /proc/sys/fs/inotify/max_user_instances
1. In the AEN navigation bar, click Admin to open the Admin Settings page.
2. In the Site Admin menu, select General:
3. In the Wakari Server box, type the main URL where the site can be viewed.
4. Click the Update button.
1. In the AEN navigation bar, click Admin to open the Admin Settings page.
2. In the Site Admin menu, select General:
3. In the Static URL box, type the static URL where JavaScript files can be accessed.
4. Click the Update button.
1. In the AEN navigation bar, click Admin to open the Admin Settings page.
2. In the Site Admin menu, select General:
1. In the AEN navigation bar, click Admin to open the Admin Settings page.
2. In the Site Admin menu, select General:
3. Under Default Project Access, select the default access type for new projects: Public or Private.
4. Click the Update button.
1. In the AEN navigation bar, click Admin to open the Admin Settings page.
2. In the Site Admin menu, select General.
3. In the Config Files section, change the configuration settings for your AEN installation. For more information
on configuration files, see Using configuration files.
4. Click the Update button.
1. In the AEN navigation bar, click Admin to open the Admin Settings page.
2. In the Site Admin menu, select License:
The Current License section displays information regarding your AEN license, including the name of the prod-
uct, vendor, license holder’s name, end and issued dates, company name, license type, and contact email.
Cheat sheet
The Admin dashboard includes three menus in the left column: Staff, Site Admin and Providers.
Staff menu
• Exceptions—If errors are raised while AEN is running, a red dot appears in the AEN navigation bar. Review
errors and mark them as read.
Providers menu
Enterprise Resources—View, add or remove local or cloud services and designate public or private to control access
to a compute node.
Troubleshooting
This troubleshooting guide provides you with ways to deal with issues that may occur with your AEN installation.
1. Clear browser cookies. When you change the AEN configuration or upgrade AEN, cookies remaining in the
browser can cause issues. Clearing cookies and logging in again can help to resolve problems.
2. Make sure NGINX and MongoDB are running.
3. Make sure that AEN services are set to start at boot, on all nodes.
4. Make sure that services are running as expected. If any services are not running or are missing, restart them.
5. Check for and remove extraneous processes.
6. Check the connectivity between nodes.
7. Check the configuration file syntax.
8. Check file ownership.
9. Verify that POSIX ACLs are enabled.
Cause
Solution
1. Log out.
2. Clear the browser’s cookies.
3. Clear the browser cache.
4. Log in.
Browser shows “Too many redirects” when the user tries to start an application.
Cause
Solution
This exception appears on the Admin > Exceptions page when a project does not have a Compute Resource assigned.
Cause
Solution
Cause
Solution
Ensure that supervisord is included in the crontab. Then restart supervisord manually.
Cause
Solution
As root, run:
NOTE: Replace SOME_USER with the administrator username and SOME_PASSWORD with the password.
3. Log into AEN as the administrator user with the new password.
Alternatively you may add an administrator user:
1. Use ssh to log into the server as root.
2. Run:
NOTE: Replace SOME_USER with the username, replace SOME_PASSWORD with the password, and replace
YOUR_EMAIL with your email address.
3. Log into AEN as the administrator user with the new password.
Cause
AEN installers log into /tmp/wakari\_{server,gateway,compute}.log. If the log files grow too large,
they might be deleted.
Solution
To set the logs to be more or less verbose, Jupyter Notebooks uses Application.log_level.
To make the logs less verbose than the default, but still informative, set Application.log_level to ERROR.
You receive the “This socket is closed” error message when you try to start an application.
Cause
When the supervisord process is killed, information sent to the standard output stdout and the standard error
stderr is held in a pipe that will eventually fill up.
Once full, attempting to start any application will cause the “This socket is closed” error.
Solution
Gateway node displays “Service Error 502: Can not connect to the application manager.”
Cause
A compute node is not responding because the wk-compute process has stopped.
Solution
You receive the “502 Communication Error: This gateway could not communicate with the Wakari server” error
message.
Cause
An AEN gateway cannot communicate with the Wakari server on AWS. There may be an issue with the IP address of
the Wakari server.
Solution
Configure your AEN gateway to use the DNS hostname of the server. On AWS this is the DNS hostname of the
Amazon Elastic Compute Cloud (EC2) instance.
Invalid username
Cause
• Other characters can be a letter, digit, period (.), underscore (_) or hyphen (-).
• The POSIX standard specifies that these characters are the portable filename character set, and that portable
usernames have the same character set.
Solution
Cause
CentOS/6 Solution
1. Install TeXLive from the TUG site. Follow the described steps. The installation may take some time.
2. Add the installation to the PATH in the file /etc/profile.d/latex.sh. Add the following, replacing the
year and architecture as needed:
PATH=/usr/local/texlive/2017/bin/x86_64-linux:$PATH
CentOS/7 Solution
Cause
Two things can cause the wk-server thread to freeze without error messages:
• LDAP freezing
• MongoDB freezing
If LDAP or MongoDB are configured with a long timeout, Gunicorn can time out first and kill the LDAP or MongoDB
process. Then the LDAP or MongoDB process dies without logging a timeout error.
Solution
Cause
If TLS is configured with a passphrase protected private key, wk-gateway will freeze without any error messages.
Solution
Update the TLS configuration so that it does not use a passphrase protected private key.
Project’s status page shows “There was an error starting this project”.
Cause
Solution
df -h /projects
Cause