Building a custom AWS ParallelCluster AMI

Warning

Building a custom AMI is not the recommended approach for customizing AWS ParallelCluster.

Once you build your own AMI, you will no longer receive updates or bug fixes with future releases of AWS ParallelCluster. You will need to repeat the steps used to create your custom AMI with each new AWS ParallelCluster release.

Before reading any further, take a look at the Custom Bootstrap Actions section of the documentation to determine if the modifications you wish to make can be scripted and supported with future AWS ParallelCluster releases

While not ideal, there are a number of scenarios where building a custom AMI for AWS ParallelCluster is necessary. This tutorial will guide you through the process.

How to customize the AWS ParallelCluster AMI

There are three ways to use a custom AWS ParallelCluster AMI, two of them require to build a new AMI that will be available under your AWS account and one does not require to build anything in advance. Feel free to select the appropriate method based on your needs.

Modify an AWS ParallelCluster AMI

This is the safest method as the base AWS ParallelCluster AMI is often updated with new releases. This AMI has all of the components required for AWS ParallelCluster to function installed and configured and you can start with this as the base.

  1. Find the AMI which corresponds with the region you will be utilizing in the list here: https://github.com/aws/aws-parallelcluster/blob/master/amis.txt.

  2. Within the EC2 Console, choose “Launch Instance”.

  3. Navigate to “Community AMIs”, and enter the AMI id for your region into the search box.

  4. Select the AMI, choose your instance type and properties, and launch your instance.

  5. Log into your instance using the OS user and your SSH key.

  6. Customize your instance as required

  7. Run the following command to prepare your instance for AMI creation:

    sudo /usr/local/sbin/ami_cleanup.sh
    
  8. Stop the instance

  9. Create a new AMI from the instance

  10. Enter the AMI id in the custom_ami field within your cluster configuration.

Build a Custom AWS ParallelCluster AMI

If you have an AMI with a lot of customization and software already in place, you can apply the changes needed by AWS ParallelCluster on top of it.

For this method you have to install the following tools in your local system, together with the AWS ParallelCluster CLI:

  1. Packer: grab the latest version for your OS from Packer website and install it
  2. ChefDK: grab the latest version for your OS from ChefDK website and install it

Verify that the commands ‘packer’ and ‘berks’ are available in your PATH after the above tools installation.

You need to configure your AWS account credentials so that Packer can make calls to AWS API operations on your behalf. The minimal set of required permissions necessary for Packer to work are documented in the Packer doc.

Now you can use the command ‘createami’ of the AWS ParallelCluster CLI in order to build an AWS ParallelCluster AMI starting from the one you provide as base:

pcluster createami --ami-id <BASE AMI> --os <BASE OS AMI>

For other parameters, please consult the command help:

pcluster createami -h

The command executes Packer, which does the following steps:

  1. Launch an instance using the base AMI provided.
  2. Apply the AWS ParallelCluster cookbook to the instance, in order to install software and perform other necessary configuration tasks.
  3. Stop the instance.
  4. Creates an new AMI from the instance.
  5. Terminates the instance after the AMI is created.
  6. Outputs the new AMI ID string to use to create your cluster.

To create your cluster enter the AMI id in the custom_ami field within your cluster configuration.

Note

The instance type to build a custom AWS ParallelCluster AMI is a t2.xlarge and does not qualify for the AWS free tier. You are charged for any instances created when building this AMI.

Use a Custom AMI at runtime

If you don’t want to create anything in advance you can just use your AMI and create a AWS ParallelCluster from that.

Please notice that in this case the AWS ParallelCluster creation time will take longer, as it will install every software needed by AWS ParallelCluster at cluster creation time. Also scaling up for every new node will need more time.

  1. Enter the AMI id in the custom_ami field within your cluster configuration.