Learning How to Deploy a AlmaLinux VM With Packer (Part 1)

Learning How to Deploy a AlmaLinux VM With Packer

I looked into the process for using Packer, and while I understand its purpose, I wanted to gain a clearer insight into how to effectively implement it for my needs. Packer is a tool that automates the creation of machine images across various platforms. It allows you to define a configuration file that specifies how to build your images, ensuring consistency and repeatability in deployments. However, as I explored several templates for cloud-based systems, I found myself somewhat perplexed by the complexity of it all. I learn best by doing, but I didn’t want to dive into writing a template from scratch. So, I asked myself, what would Captain Picard do to get this thing up and running? He’d ask the ship’s computer. And in this case, I believed he might also consult Commander Data for a bit of extra insight. Consequently, I sought help from our own artificial intelligences to streamline the process, drawing on the information I had gathered from my research into this topic so far.

Iterative Approach

I didn't expect to get a fully-fledged RADV router out of the gate on the first attempt. My approach will be iterative, starting with a minimal configuration. This means I'll take small steps, testing each phase before progressing, allowing for adjustments based on what I learn along the way. Each iteration will help refine the setup, making it more robust and tailored to my needs.

I asked ChatGPT to generate the files I needed using this prompt.

Generate files: to set up a bare-bones AlmaLinux 9.2 virtual machine with the following requirements:
    - I am on a windows platform
    - Using Virtual Box

1. Packer Configuration (packer.hcl):
   - Use the VirtualBox ISO builder to install AlmaLinux 9.2 from the official boot ISO.
   - The VM should be configured with:
     - 1 vCPU, 512MB RAM, and 10GB dynamic disk.
     - Network Adapter 1: Connect to an existing NAT network named LABEXNATNetwork with a static IP in the range of 192.168.56.0/24 and virtio as the adapter type.
   - Use Packer's HTTP server to serve a Kickstart file during installation.
   - Configure SSH access with root as the username and a password for connection.

2. Kickstart File (kickstart.cfg):
   - Automate the AlmaLinux installation using a minimal install configuration.
   - Set the root password as your_password_here.
   - Create a single partition with XFS as the file system type and use the entire disk.
   - Configure networking:
     - NIC 1: Static IP configuration for the NAT network (e.g., 192.168.56.100).
   - Do not install any additional packages.

I set up a dedicated project folder within my Git repository for this machine, saved all the files, and then staged and committed them to Git. This serves as a solid baseline to build on or revert to whenever needed.

Observations and Considerations

I was quite impressed it was able to generate Packer configuration and Kickstart files. That said, it’s important never to take anything an LLM tells you at face value. You’ve got to understand the code it spits out. I see AI as a colleague with a bit of a split personality—sometimes it’s super knowledgeable and knows its stuff, but other times it can lead you down a blind alley. So, it’s all about keeping a critical eye while you work with it!

As I reviewed the generated Packer configuration and Kickstart files, a few important considerations stood out to me:

1. Readability: The structure of the files made them somewhat difficult to read. Clear formatting and comments could greatly enhance comprehension and make it easier to follow the setup process.

2. Security Concerns: A major issue with the generated files was the use of plain text for the root password. This poses a significant security risk, as anyone with access to the configuration can easily view sensitive information. In a lab designed to emulate a production environment, it’s crucial to implement secure practices. Using hashed passwords or environment variables protects credentials and aligns with industry best practices, ensuring the setup is professional rather than a quick and dirty build.

3. Wrong Localisation settings: The files did include localisation information, however the keyboard and Language is set to US and the Timezone UTC

Addressing these points before proceeding will help refine the configuration and enhance both its security and usability. before proceeding i asked ChatGPT to add comments to the file and address the points i had identified

looking at the files:
- Add comments on every line of both files
- Improve security by using pregenerarared password hash
- Add Localisagtion for EUROPE/LONDON timezone and UK keyboard

Here are the Results: ChatGPT Session

For this second version, I replaced the contents of the files, staged the updates, and committed the changes to refine the initial setup and implement improvements. This approach allows for iterative adjustments while maintaining a clear version history to track progress and roll back if necessary.

Further Observations and Considerations

Having addressed the issues identified previously the 2nd Answer looked promising, i just had a few tweaks to make before i was ready to attempt to build the machine.

1. Generate the hashed password

   • I have used this guide before to generate a hashed password so i used that and ran these commands

     #install open SSL
     choco install openssl -y
     #generate salt
     $salt=openssl rand -base64 32
     #generate hashed password
     openssl passwd -6 -salt $salt

2. Download the SHA256 Checksum for the ISO

   • I already have a script for retriving checksums from a CHECKSUM file on a Linux Repository so i ran this in Poweshell to retrieve that

     $checksums=((Invoke-RestMethod -Uri "https://repo.almalinux.org/almalinux/9/isos/x86_64/CHECKSUM" -Headers @{"Accept" = "text/plain; charset=utf-8"}).Split("`n") | Where-Object { $_.StartsWith("SHA256") }) -replace (' = ','=') | foreach-object {convertfrom-stringdata $_}
     $checksums | format-table -AutoSize

3. To ensure maximum speed, I chose to use the UK Mirror Service operated by the University of Kent.

   • I replaced the download URL with this link while ensuring that I used the checksum from the official site.

     https://www.mirrorservice.org/sites/repo.almalinux.org/almalinux/9.4/isos/x86_64/AlmaLinux-9.4-x86_64-boot.iso

4. To keep all the files in the same folder

   • I changed the http_directory clause

     http_directory     = "./"

5. The plugin URL is in the wrong format

   • I changed the URL to include the domain name

     source  = "github.com/hashicorp/virtualbox"  # Source of the VirtualBox plugin.

For the third version, I made adjustments to the files, staged the modifications, and committed the changes. This step was taken to fine-tune the setup further and make necessary improvements. This method ensures a step-by-step approach to development while keeping a comprehensive history of changes, allowing easy tracking and the ability to revert if needed.

HCL File

# Packer configuration file for building an AlmaLinux VM using VirtualBox.

# Define the required Packer plugins.
packer {
  required_plugins {
    virtualbox = {
      version = ">= 1.0.0"  # Minimum version required for the VirtualBox plugin.
      source  = "hashicorp/virtualbox"  # Source of the VirtualBox plugin.
    }
  }
}

# Define variables for SSH username and password.
variable "ssh_username" {
  type    = string
  default = "root"  # Default SSH username to access the VM.
}

# Use a pre-generated password hash for better security.
variable "ssh_password_hash" {
  type    = string
  default = "$6$RV6Nnu0diWE5fjic$GqjitWL86VdywofQgUYlnTG5cOZ76WH3pch82tCNnszf3F4Tke5zNFDRdRA9wBsy8C1/b1Bc1On/QzrfSF.x7/"  # Replace with your actual hashed password.
}

# Define the source for the VirtualBox image.
source "virtualbox-iso" "almalinux" {
  # URL to the AlmaLinux 9.4 ISO file.
  iso_url            = "https://www.mirrorservice.org/sites/repo.almalinux.org/almalinux/9.4/isos/x86_64/AlmaLinux-9.4-x86_64-boot.iso"
  # Checksum for the ISO to ensure integrity (replace with actual checksum).
  iso_checksum       = "1e5d7da3d84d5d9a5a1177858a5df21b868390bfccf7f0f419b1e59acc293160"  # Replace with the actual checksum for AlmaLinux 9.4 ISO.
  # Directory for serving files over HTTP.
  http_directory     = "./"  # Serve files from the current directory.
  # Ports for the HTTP server.
  http_port_min      = 8000
  http_port_max      = 8001
  # Boot command to initiate installation with Kickstart.
  boot_command       = [
    "<esc><wait>",  # Escape to the boot menu.
    "linux ks=http://{{ .HTTPIP }}:{{ .HTTPPort }}/kickstart.cfg<enter>"  # Specify the Kickstart file.
  ]

  guest_os_type      = "RedHat_64"  # Specify the guest OS type for VirtualBox.
  # Configure VM settings using VBoxManage commands.
  vboxmanage         = [
    ["modifyvm", "{{.Name}}", "--memory", "512"],  # Set memory to 512MB.
    ["modifyvm", "{{.Name}}", "--cpus", "1"],  # Set number of CPUs to 1.
    ["modifyvm", "{{.Name}}", "--nic1", "natnetwork"],  # Connect to a NAT network.
    ["modifyvm", "{{.Name}}", "--nat-network1", "LABEXNATNetwork"],  # Use the specified NAT network.
    ["modifyvm", "{{.Name}}", "--nictype1", "virtio"],  # Use virtio as the network adapter type.
    ["modifyvm", "{{.Name}}", "--natpf1", "ssh,tcp,,2222,,22"]  # Set up port forwarding for SSH.
  ]

  disk_size          = 10240  # Set disk size to 10GB.
  ssh_username       = var.ssh_username  # Use the defined SSH username.
  ssh_password       = var.ssh_password_hash  # Use the defined hashed password for SSH.
  ssh_wait_timeout   = "20m"  # Timeout for SSH waiting.

  vm_name            = "AlmaLinux_9_4_BareBones"  # Name of the VM.
  shutdown_command   = "shutdown -h now"  # Command to shutdown the VM.
  output_directory   = "output-almalinux-vm"  # Directory to store the output files.
}

# Define the build block that references the source.
build {
  sources = ["source.virtualbox-iso.almalinux"]  # Use the defined source.
}

Kickstart File

# Kickstart file for automating the installation of AlmaLinux 9.4.

text  # Run installer in text mode.
install  # Begin installation from the CD-ROM.
cdrom  # Use the installation media from the CD-ROM.
lang en_GB.UTF-8  # Set language to British English.
keyboard uk  # Set keyboard layout to UK.
timezone Europe/London --isUtc  # Set timezone to Europe/London and configure as UTC.

# Set the root password using a hashed password for security.
rootpw --iscrypted $6$RV6Nnu0diWE5fjic$GqjitWL86VdywofQgUYlnTG5cOZ76WH3pch82tCNnszf3F4Tke5zNFDRdRA9wBsy8C1/b1Bc1On/QzrfSF.x7/  # Replace with your actual hashed password.

# Configure network settings with static IP.
network --bootproto=static --device=eth0 --ip=192.168.56.100 --netmask=255.255.255.0 --gateway=192.168.56.1 --nameserver=8.8.8.8 --hostname=almalinux-vm
firewall --disabled  # Disable the firewall for simplicity.
selinux --disabled  # Disable SELinux for this installation.
skipx  # Skip the X Window System installation.
firstboot --disable  # Disable first boot configuration.

ignoredisk --only-use=sda  # Use only the first disk for installation.

# Partitioning setup.
clearpart --all --initlabel  # Clear all partitions and create a new partition table.
autopart --type=xfs  # Automatically partition the disk using XFS file system.

# Package selection for the installation.
%packages
@^minimal-environment  # Install the minimal environment.
%end

# Disable kdump for this installation.
%addon com_redhat_kdump --disable
%end

# Post-installation script.
%post
echo "Kickstart setup completed."  # Message indicating completion of the kickstart setup.
%end

Building the VM

With the HCL and Kickstart files generated by ChatGPT and refined by me, I truly felt like Captain Picard consulting the ship’s computer. I can only imagine how long it would have taken to write those files from scratch or to piece together templates from various sources, which would have been the standard practice in the past. At this stage, I was ready to build the VM. I opened my terminal and navigated to the project folder containing the Packer configuration and Kickstart files. I executed the following command:

packer build packer.pkr.hcl

This command instructed Packer to read the configuration file and start the VM creation process. As the build began, I monitored the terminal output for real-time feedback on various stages like downloading the ISO, setting up the virtual machine, and executing the Kickstart file.

Monitoring the Build Process

During the build, I kept an eye on the output to track progress and catch any issues that arose. Initially, everything appeared to proceed smoothly, but soon I encountered the first error that required immediate attention.

Troubleshooting

1. Disk Size Issue: The initial build failed due to an insufficient disk size of 20GB, which was inadequate for the AlmaLinux installation and all required packages. To resolve this, I increased the disk size to 256GB, providing ample space for the OS and future applications. This adjustment ensured that I wouldn't encounter space limitations during the installation process:

   disk_size          = 262144  # Set disk size to 256GB.

2. Initramfs Error: I encountered an "initramfs" error that halted the installation. I suspected this was due to insufficient RAM, as the VM had only been allocated 512MB initially. Increasing the memory to 8GB allowed the system to properly load and the process to continue

   ["modifyvm", "{{.Name}}", "--memory", "8192"],  # Set memory to 8GB.

3. Kickstart File Deprecation:

   I noticed a message indicating that Kickstart (KS) files are becoming deprecated. I noted that I’d need to investigate alternative options for automating installations moving forward.

4. Kickstart File Retrieval Error:

   An error indicated that the VM couldn't get the Kickstart file from the host, Google search revealed this was likely due to the NAT network configuration. further research revealed that some people suggested using the following option.  I also checked to ensure the initial KS file path was correct:

   # Ensure host is reachable from NAT network
   [["modifyvm", "{{.Name}}", "--nat-localhostreachable1", "on"]]

   ks=http://{{ .HTTPIP }}:{{ .HTTPPort }}/kickstart.cfg<enter>

Revising the Plan

After reading up on the issues surrounding NAT networks and serving Kickstart files from a Windows host, I made a few adjustments to my configuration. I realised that using a bridged network would provide a more stable and reliable connection for serving the Kickstart file. Additionally, I discovered that the virtio network driver was not functioning properly in my setup, prompting me to use the default driver, 82540EM, typically used in GUI-created VMs.

["modifyvm", "{{.Name}}", "--nic1", "bridged"],  # Attach NIC as bridged network
["modifyvm", "{{.Name}}", "--bridgeadapter1", "Intel(R) I211 Gigabit Network Connection"],  # Use the appropriate host adapter
["modifyvm", "{{.Name}}", "--nictype1", "82540EM"]  # Set NIC type to virtio

Since I only need one machine to connect to the network over IPv4, this setup works for now. I might consider switching back to NAT after the build, but I didn’t want to get sidetracked by that at this point. I could also look into hosting the Kickstart file on a separate server later. For now, I’m happy that the HCL file is functioning, the VM is booting, and the installer is processing the Kickstart file as expected.

Wrapping Up (For Now)

The installer flagged an error in the Kickstart file right away, which wasn’t entirely unexpected and signalled that the process would need a few more steps to refine. As this post is already quite lengthy, I’ve decided to split it into two parts. Part two, published at the same time and alongside this one, covers the troubleshooting I did to address these issues and continue with the build.

Lessons Learned

This process has already taught me a lot about Packer. While AI tools have been incredibly helpful, troubleshooting along the way has been essential. This is exactly why building a lab from scratch offers more value than deploying a pre-built solution.

Reflections on the Process

These notes have underscored the importance of iterative changes and testing. Experimentation is at the heart of IT and science, and each challenge has deepened my understanding of the process.

Moving Forward

I enjoyed the challenge and look forward to resolving the remaining issues and continuing to experiment as I transform this bare-bones VM into the baseline template i need to develop into the RADV router needed to further build out this lab. Engaging with these complex problems contributes to my skill set, which I will need for the rest of the lab and may be useful in my career.