Requirements

You must meet the following requirements before installing Longhorn on a Talos Linux cluster.

System Extensions

Some Longhorn-dependent binary executables are not present in the default Talos root filesystem. To have access to these binaries, Talos offers system extension mechanism to extend the installation.

• siderolabs/iscsi-tools: this extension enables iscsid daemon and iscsiadm to be available to all nodes for the Kubernetes persistent volumes operations.
• siderolabs/util-linux-tools: this extension enables linux tool to be available to all nodes. For example, the fstrim binary is used for Longhorn volume trimming.

The most straightforward method is patching the extensions onto existing Talos Linux nodes.

customization:
  systemExtensions:
    officialExtensions:
      - siderolabs/iscsi-tools
      - siderolabs/util-linux-tools

For detailed instructions, see the Talos documentation on System Extensions and Boot Assets.

Pod Security

Longhorn requires pod security enforce: "privileged".

By default, Talos Linux applies a baseline pod security profile across namespaces, except for the kube-system namespace. This default setting restricts Longhorn’s ability to manage and access system resources. For more information, see Root and Privileged Permission.

For detailed instructions, see Pod Security Policies Disabled & Pod Security Admission Introduction and the Talos documentation on Pod Security.

Talos Linux Version Prior to v1.10.x

Data Path Mounts

You need provide additional data path mounts to be accessible to the Kubernetes Kubelet container.

These mounts are necessary to provide access to the host directories, and attach volumes required by Longhorn components.

machine:
  kubelet:
    extraMounts:
      - destination: /var/lib/longhorn
        type: bind
        source: /var/lib/longhorn
        options:
          - bind
          - rshared
          - rw

For detailed instructions, see the Talos documentation on Editing Machine Configuration.

V2 Data Engine

To use V2 volumes, all nodes must meet the V2 Data Engine prerequisites.

machine:
  sysctls:
    vm.nr_hugepages: "1024"
  kernel:
    modules:
      - name: nvme_tcp
      - name: vfio_pci
#     - name: uio_pci_generic

Note: Talos Linux v1.7.x and earlier versions do not include the uio_pci_generic kernel module. If your system device supports vfio_pci, which is the preferred kernel module for SPDK application deployment, you are not required to install and enable the uio_pci_generic kernel driver. For more information, see System Configuration User Guide in the SPDK documentation.

You can use uio_pci_generic if vfio_pci is incompatible with your system or specific hardware. Future versions of Talos Linux are expected to include native support for uio_pci_generic. For more information, see Issue #9236. Since 1.8.0 uio_pci_generic is now supported.

Talos Linux Upgrades

Prior to v1.8.x

When upgrading a Talos Linux node, always include the --preserve option in the command. This option explicitly tells Talos to keep ephemeral data intact.

Example:

talosctl upgrade --nodes 10.20.30.40 --image ghcr.io/siderolabs/installer:v1.7.6 --preserve

Caution: If you do not include the --preserve option, Talos wipes /var/lib/longhorn, destroying all replicas stored on that node.

Recovering from an Upgraded Node without Preserving Data

If you were unable to include the --preserve option in the upgrade command, perform the following steps:

1. On the Longhorn UI, go to the Nodes page.

2. Select the upgraded node, and then select Edit node and disks in the Operation menu.

3. On the Edit Node and Disks page, set Scheduling to Disable, delete the disk, and then click Save.

4. Select the upgraded node again, and then select Edit node and disks in the Operation menu.

5. On the Edit Node and Disks page, add a disk and configure the following settings:

   • Path: Specify /var/lib/longhorn/.
   • Storage Reserved: Specify a value that matches your requirements. By default, it is set to 30% of the disk capacity.
   • Scheduling: Select Enable.

6. Click Save.

Longhorn synchronizes the replicas based on the configured settings.

After v1.8.x

The --preserve is no longer required. The flag is automatically set for talosctl upgrade command here.

Talos Linux Version v1.10.x and Later

Data Path Mounts

Because Talos Linux deprecated .machine.disks we recommend using UserVolumeConfig to mount a disk for Longhorn. See the What’s new in Talos v1.10 for more details.

You can optionally create also a VolumeConfig to specify the size of Talos System volumes, which is recommended, like this we avoid the set defaultSettings.storageReservedPercentageForDefaultDisk.

More options of disk configuration can be found in the Talos documentation.

You need provide additional data path mounts to be accessible to the Kubernetes Kubelet container.

These mounts are necessary to provide access to the host directories, and attach volumes required by Longhorn components.

The default data path for Longhorn is /var/lib/longhorn. In order to use the below configuration in Talos, we must first set our default data path to /var/mnt/longhorn. The method to do this will depend on your deployment method.

machine:
  kubelet:
    extraMounts:
      - destination: /var/mnt/longhorn
        type: bind
        source: /var/mnt/longhorn
        options:
          - bind
          - rshared
          - rw

You need to create a UserVolumeConfig to mount the disk for Longhorn, which will be automatically mounted to /var/mnt/longhorn on the configured node.

apiVersion: v1alpha1
kind: UserVolumeConfig
name: longhorn # name is used to identify the volume /var/mnt/<name>
provisioning:
  diskSelector:
    match: disk.transport == "nvme"
  grow: false
  maxSize: 1700GB

For detailed instructions on UserVolumeConfig and VolumeConfig, see the Talos documentation on Block configuration

References

• [FEATURE] Talos support