Nutanix remote deployments – Phoenix ISO + serial number-based JSON

Installing Nutanix in remote locations can be challenging. Fortunately, Nutanix engineers provided a solution that allows you to create generic ISO images which enables provisioning Nutanix nodes where the only requirement on site is to have the server properly connected to a network with DHCP enabled.

This approach uses Phoenix ISO that needs to be generated using Foundation VM and will point servers being built to HTTP endpoint with a serial number-based JSON file pre-created that provides all information needed for Nutanix deployment.

Setting up Foundation

If not running Foundation already go to Nutanix Portal and download the Foundation image to be deployed in your environment. If you have Foundation already deployed now is a good time to have it upgraded to the newest version. This can be done via GUI or the command line. Steps to perform the upgrade can be found in Foundation Release Notes (the latest one at the time of publication is available here).

Once you run the latest version of Foundation it is recommended to check the status of the Foundation Platforms submodule. The download is available on Nutanix Portal and upgrades instructions can be found here.

Prerequisites

For this process to work you need to have JSON with <nodeserialnumber>.json name available on the HTTP endpoint. This JSON file can be forged by automation that will fill the file with appropriate values. Example of the file below:

{
   "host_ip":"##ServerIP##",
   "svm_ip":"##CVMIP##",
   "host_subnet_mask":"##ServerNetmask##",
   "svm_subnet_mask":"##ServerNetmask##",
   "default_gw":"##ServerGateWay##",
   "node_position":"A",
   "hyp_install_type":"clean",
   "svm_install_type":"clean",
   "hyp_type":"esx",
   "hyp_version":"7.0.3",
   "svm_version":"5.20.4.6",
   "hypervisor_iso_url":{
      "url":"http://##sourceurl##/VMware-ESXi-7.0.3-19482537-HPE-703.0.0.10.9.0.11-Apr2022.iso",
      "md5sum":"68c736355a88f3fb0c13668227752421"
   },
   "hypervisor_mac":"##MACADDR##",
   "hypervisor_hostname": "##ServerFQDN##",
   "svm_num_vcpus": 12,
   "svm_gb_ram": 40,
   "svm_installer_url":{
      "url":"http://##sourceurl##/nutanix_installer_package-release-euphrates-5.20.4.6-stable-3c161846a9bca6f3adf506fa81ab5afbdd1113fa-x86_64.tar.gz",
      "md5sum":"cc154f695a232f0d4637396b6162addd"
   }
}

Replace all ##value## with appropriate values for your environment and node being deployed. Make sure to place the file on HTTP endpoint that will be accessible from all locations in your environment and which URL you will encode into custom ISO. Name of the file should be <nodeserialnumber>.json (for example JDIIE292102JHD.json). Generation of the file can be easily automated.

Generating custom Phoenix ISO

Now it is time to configure and generate custom Phoenix ISO. Logon to Foundation VM via SSH. Go to /home/nutanix/foundation/lib/phoenix/ directory. Make copy of x86_64 folder (cp -r ./x86_64 ./x86_64_orig). Go to ./x86_64 directory and edit isolinux config file (vim ./boot/isolinux/isolinux.cfg). Isolinux is used for Legacy BIOS boot. Set the content to something like this (replace XXX.XXX.XXX.XXX with real life IPs, I use IP for URL as this makes the process not rely on DNS):

# ISOLINUX configuration file for Phoenix bootloader.
ui menu.c32
default Installer
prompt 0

menu title Nutanix Installer
menu clear

# timeoutm0 should prevent the user from choosing any other option so it's
# not possible for the user to drop into a root shell. This is needed for
# STIG compliance.
timeout 10

label Installer
menu label ^Nutanix_Installer
menu default
kernel /boot/kernel
append initrd=/boot/initrd init=/installer intel_iommu=on kvm-intel.nested=1 kvm.ignore_msrs=1 kvm-intel.ept=1 quiet consoleblank=0 net.ifnames=0 mpt3sas.prot_mask=1 FOUND_IP=XXX.XXX.XXX.XXX AZ_CONF_URL=http://XXX.XXX.XXX.XXX/SerialNumber/%%NODE_SERIAL%%.json IMG=

Next step is to edit grub config which is used by EFI boot. To do that edit grub.cfg (vim ./EFI/BOOT/grub.cfg) and set the content to something like this (again put proper IPs for your environment):

set default='Installer'

insmod part_gpt
insmod part_msdos

set timeout=10

search --no-floppy --set=root -l 'PHOENIX'

menuentry 'Installer mode: Launches Phoenix GUI after Phoenix boots' {
linuxefi /boot/kernel init=/installer intel_iommu=on iommu=pt kvm-intel.nested=1 kvm.ignore_msrs=1 kvm-intel.ept=1 quiet consoleblank=0 net.ifnames=0 FOUND_IP=XXX.XXX.XXX.XXX AZ_CONF_URL=http://XXX.XXX.XXX.XXX/SerialNumber/%%NODE_SERIAL%%.json IMG=
initrdefi /boot/initrd
}

In both cases keep %%NODE_SERIAL%% as is in the config – this variable will be used by Phoenix during boot making this ISO universal.

Once boot environment config files are prepared it is time to generate the ISO. Go to /home/nutanix/foundation/bin/ and execute ./generate_iso phoenix –no-package-driver –timeout 10

Generated ISO will be in /home/nutanix/foundation/tmp/ folder called phoenix-Y.Y.Y-x86_64.iso where Y.Y.Y is version of the software.

Building the node

To build the node start it from generated ISO. Phoenix will grab IP from DHCP, take json file from HTTP endpoint based on node S/N and progress with installation. In our environment we use central S/N.json location (to have one ISO image as IP of HTTP endpoint is hardcoded there) and use variable locations for downloads so the node pulls AOS and ESXi/AHV images from nearest source to limit WAN impact. Once build is done you can perform post configuration (if required) and either form new cluster from the nodes that were provisioned or simply add node to existing cluster.