Customizing a cluster profile — Anaconda Enterprise 4.4 documentation

Anaconda Enterprise: Documentation

# Customizing a cluster profile

Although the default cluster profile described in the installation guide is appropriate for most environments and use cases, you can customize profile settings as needed.

Cluster profiles are used to define information about a cluster before it is created, including the cluster name, nodes in the cluster, and other platform component options. Cluster files are located in the `~/.continuum/adam/profile.d/` directory.

Cluster definitions are used to identify information about a running cluster, including the cluster name, nodes in the cluster, and other platform component options. Cluster definitions are located in the `~/.continuum/adam/cluster.d/` directory.

- Sample profile showing all settings
- Cluster name
- SSH authentication
- Node specification
- Security settings
- Network settings–proxy configuration
- Conda settings
- Salt settings
- System settings

## Sample profile showing all settings

A cluster profile located at `~/.continuum/adam/profile.d/cluster.yaml` with all configurable settings is shown below:

    name: cluster
    provider: bare

    bare:
      username: centos
      # password: anaconda  # Optional
      port: 22
      keypair: ~/.ssh/my-private-key
      nodes:
      - host: node1.company.com
      - host: node2.company.com
      - host: node3.company.com
      - host: node4.company.com

    security:
      flush_iptables: false
      selinux_context: false
      selinux_permissive: false

    network:
      http_proxy: http://server:port/
      # https_proxy: http://server:port/

    system:
      tmp_dir: /tmp

    plugins:
      conda:
        anaconda_hash: md5=d72add23bc937ccdfc7de4f47deff843
        anaconda_url: http://repo.continuum.io/archive/Anaconda2-4.4.0-Linux-x86_64.sh
        channel_alias: https://conda.anaconda.org/
        channels:
        - defaults
        - anaconda-adam
        conda_canary: false
        enabled: true
        miniconda_hash: md5=7097150146dd3b83c805223663ebffcc
        miniconda_url: http://repo.continuum.io/miniconda/Miniconda2-4.3.21-Linux-x86_64.sh
        rootdir: /opt/continuum
        ssl_verify: False

      dask:
        bokeh_port: 8787
        bokeh_whitelist: '''*'''
        enabled: false
        host: 0.0.0.0
        http_port: 9786
        nprocs: 1
        port: 8786

      enterprise-notebooks:
        admin_email: admin@yourdomain.com
        admin_password: anaconda
        admin_user: wakari
        directory: /opt/wakari
        elasticsearch_fn: elasticsearch-1.7.2.noarch.rpm
        enabled: false
        enterprise_notebooks_version: 4.0.0
        gateway_port: 8089
        java_fn: jre-8u65-linux-x64.rpm
        mongodb_fn: mongodb-org-2.6.8-1.x86_64.rpm
        mongodb_mongos_fn: mongodb-org-mongos-2.6.8-1.x86_64.rpm
        mongodb_server_fn: mongodb-org-server-2.6.8-1.x86_64.rpm
        mongodb_shell_fn: mongodb-org-shell-2.6.8-1.x86_64.rpm
        mongodb_tools_fn: mongodb-org-tools-2.6.8-1.x86_64.rpm
        mongodb_version: 2.6.8
        nginx_fn: nginx-1.6.2-1.el6.ngx.x86_64.rpm
        root_download_url: https://820451f3d8380952ce65-4cc6343b423784e82fd202bb87cf87cf.ssl.cf1.rackcdn.com/
        user: wakari

      jupyter:
        dashboards_server:
          enabled: false
          ip: 0.0.0.0
          port: 3000
          user: anaconda
        enabled: false
        kernel_gateway:
          enabled: false
          ip: 0.0.0.0
          port: 7000
          user: anaconda
        notebook:
          directory: ~/notebooks
          enabled: false
          ip: 0.0.0.0
          password: 'anaconda'
          port: 8888
          user: anaconda

      repository:
        binstar_server_version: 2.21.0
        channel: main
        email: youremail@anaconda.com
        enabled: false
        mongodb_fn: mongodb-org-2.6.8-1.x86_64.rpm
        mongodb_mongos_fn: mongodb-org-mongos-2.6.8-1.x86_64.rpm
        mongodb_server_fn: mongodb-org-server-2.6.8-1.x86_64.rpm
        mongodb_shell_fn: mongodb-org-shell-2.6.8-1.x86_64.rpm
        mongodb_tools_fn: mongodb-org-tools-2.6.8-1.x86_64.rpm
        mongodb_version: 2.6.9
        password: anaconda
        port: 8080
        root_download_url: https://820451f3d8380952ce65-4cc6343b423784e82fd202bb87cf87cf.ssl.cf1.rackcdn.com/
        server_user: anaconda-server
        superuser: superuser
        token: qu-a49e2e69-1047-4eab-a879-a2ee9c198381

      salt_settings:
        acl:
        - anaconda
        job_pub_port: 14505
        minion_pub_port: 14510
        minion_pull_port: 14511
        minion_ret_port: 14506
        rest_port: 18000
        salt_groupname: anaconda
        salt_password: anaconda
        salt_username: anaconda
        service_scripts: false

## Cluster name

The `name` setting specifies the name of the cluster and is used by the `-n` option in Anaconda Adam commands.

EXAMPLE: To create a new cluster named “cluster” from a profile named “profile”:

    $ adam up -n cluster profile

## SSH authentication

To connect to remote machines via SSH, Adam requires a valid username, port–default is 22–and an authentication method. Adam supports the following SSH authentication methods:

- Password.
- Key pair–full path to your private key.
- Encrypted key pair–full path to your public key and ssh-agent. The private key must first be added to a running ssh-agent. Use the `agent_pubkey` setting to specify the full path to the associated public key.

For more information about system and account requirements, see Account access and security.

EXAMPLE: To use SSH with a username and password:

    bare:
      username: centos
      password: anaconda

EXAMPLE: To use SSH with a key pair:

    bare:
      username: centos
      keypair: /full/path/to/key.rsa

EXAMPLE: To use SSH with an encrypted key pair:

    bare:
      username: centos
      agent_pubkey: /full/path/to/key.pub

## Node specification

The `nodes` setting defines the nodes that exist within the cluster. By default, the first node in the list of nodes is defined as the head node and the remaining nodes are defined as compute nodes.

Each host must be able to connect to the other hosts via the specified FQDN or IP address.

TIP: We recommended using the node’s FQDN that matches the hostname on each machine:

EXAMPLE:

    bare:
      nodes:
      - host: node1.company.com
      - host: node2.company.com
      - host: node3.company.com
      - host: node4.company.com

## Security settings

The `security` settings let you configure IPTables and SELinux.

To flush iptables, reset `flush_iptables` to `true`. Default value: `false`.

    security:
      flush_iptables: false

To set SELinux to permissive, reset `selinux_permissive` to `true`. Default value: `false`.

    security:
      selinux_permissive: false

To set SELinux contexts, reset `selinux_context` to `true`. Default value: `false`.

    security:
      selinux_context: false

## Network settings–proxy configuration

The `network` settings let you install Adam to nodes that access the internet via an HTTP or HTTPS proxy.

To specify an HTTP proxy:

    network:
      http_proxy: http://server:port/

NOTE: Replace `server:port` with the actual server and port.

To specify an HTTPS proxy:

    network:
      https_proxy: http://server:port/

NOTE: Replace `server:port` with the actual server and port.

## Conda settings

These are the default settings for configuring conda:

    conda:
        anaconda_hash: md5=d72add23bc937ccdfc7de4f47deff843
        anaconda_url: http://repo.continuum.io/archive/Anaconda2-4.4.0-Linux-x86_64.sh
        channel_alias: https://conda.anaconda.org/
        channels:
        - defaults
        - anaconda-adam
        enabled: true
        miniconda_hash: md5=7097150146dd3b83c805223663ebffcc
        miniconda_url: http://repo.continuum.io/miniconda/Miniconda2-4.3.21-Linux-x86_64.sh
        rootdir: /opt/continuum
        ssl_verify: False

You can set ssl_verify to:

- `False`–no SSL verification–default.
- `True`–SSL verification is used and conda verifies certificates for SSL connections.
- `[cert path]`–the string path to a certificate to be used to verify SSL connections.

## Salt settings

Salt is the configuration management system used by Adam. The `salt_settings` let you configure the network options, access control list and API user credentials used by Salt.

Use the settings below to specify the ports used by the Salt master, minions and REST API:

    plugins:
      salt_settings:
        job_pub_port: 14505
        minion_pub_port: 14510
        minion_pull_port: 14511
        minion_ret_port: 14506
        rest_port: 18000

Use the `acl` setting to specify an access control list that defines non-root system users who can execute Salt commands:

    plugins:
      salt_settings:
        acl:
        - anaconda

Use the settings below to specify the username, password and group that gets created across the cluster and owns the directory and files located in the root installation directory–default: `/opt/continuum`:

    plugins:
      salt_settings:
        salt_username: anaconda
        salt_password: anaconda
        salt_groupname: anaconda

If you do not want to store a clear text password in the cluster profile or cluster definition file, set `salt_password` to an empty string:

    plugins:
      salt_settings:
        salt_username: anaconda
        salt_password: ''
        salt_groupname: anaconda

In this case, Adam prompts for a password in the CLI and only stores the password in memory for the duration of the command or job.

## System settings

The `tmp_dir` setting lets you specify the download directory where temporary installers are downloaded–default: `/tmp`:

    system:
      tmp_dir: /tmp