LXC Container Configuration

Install LXC

Install the package lxc-utils.

apt-get install lxc-utils

Change Container Filesystem Location

Copy the existing /etc/lxc/default.conf file to /etc/lxc/lxc.conf and add key lxc.lxcpath to specify a different one than the default /var/lib/lxc.

lxc.lxcpath = /mnt/<container-ssd>/lxc

When having existing containers which are moved.
Modify the config file and adapt the rootfs key.

Network Bridge Interface IP-range

By default, the ip-range is 10.0.3.0/24 for any system having LXC installed.
To change the network ip-range for the containers, modify the file /etc/default/lxc-net by adding the following lines to get the ip range 10.0.9.0/24.

LXC_BRIDGE="lxcbr0"
LXC_ADDR="10.0.9.1"
LXC_NETMASK="255.255.255.0"
LXC_NETWORK="10.0.9.0/24"
LXC_DHCP_RANGE="10.0.9.2,10.0.9.254"
LXC_DHCP_MAX="253"

Restart the lxc-bet service depending on the changed file.

systemctl restart lxc-net

show which ip-addresses are connected

ip addr show lxcbr0

Add the new lxc ip-address.

ip addr add 10.0.9.1/24 dev lxcbr0

Remove the old lxc default ip-address.

ip addr del 10.0.3.1/24 dev lxcbr0

A reboot is needed when ip-traffic is still not working after this.

Create a Container

Create a container from templates from the internet.

lxc-create -n <container-name> -t download

It asks for the distribution, version and the architecture.
In most cases this is for example:

Field	Value
Distribution	`ubuntu`
Version	`noble`
Architecture	`amd64`

Static IP

For some reason the ip-address of the container is not fixed anymore.
To prevent this add the following lines to the /var/lib/lxc/<container-name>/config file.

lxc.net.0.ipv4.address = 10.0.9.2/24
lxc.net.0.ipv4.gateway = 10.0.9.1

The key lxc.net.0.ipv4.gateway is probably superfluous.

Enabling FUSE (in container)

It seems a container could be lacking the fuse device /dev/fuse.
Perform the next tasks to detect a possible fix it.

1) Check existence of device

ls -l /dev/fuse

2) Create the device

Create the missing device entry in /dev execute the next command.

mknod -m 666 /dev/fuse c 10 229

Forwarding a Host Port

When a container runs xrdp for example on container having ip 10.0.9.2 and port 3389 add the following rule.

iptables \
  --table nat \
  --append PREROUTING \
  --protocol tcp \
  --dport 3389 \
  --jump DNAT \
  --to-destination 10.0.9.2:3389

To check if the rules exist.

iptables --table nat --list PREROUTING --verbose --numeric --line-numbers

To remove the rule by line-number, execute the following.

iptables --table nat --delete PREROUTING <line_number>

Delete a rule using the creation properties.

iptables \
    --table nat \
    --delete PREROUTING \
    --protocol tcp \
    --dport 3389 \
    --jump DNAT \
    --to-destination 10.0.9.2:3389

To make the rule persistent the simple way, add a @reboot in the crontab using crontab -e.

LXC Hooks

Option/Key	Description
`lxc.hook.pre-start`	A hook to be run in the host namespace before the container ttys, consoles, or mounts are loaded
`lxc.hook.pre-mount`	A hook to be run in the container's filesystem namespace, but before the rootfs has been set up
`lxc.hook.mount`	A hook to be run in the container after mounting has been done, but before the pivot_root
`lxc.hook.autodev`	A hook to be run in the container after mounting has been done and after any mount hooks have run, but before the pivot_root
`lxc.hook.start`	A hook to be run in the container right before executing the container's init
`lxc.hook.stop`	A hook to be run in the host's namespace after the container has been shut down
`lxc.hook.post-stop`	A hook to be run in the host's namespace after the container has been shut down...

Common environment variables set during hook being called are:

LXC_CONFIG_FILE = /var/lib/lxc/<container-name>/config
LXC_LOG_LEVEL = ERROR
LXC_NAME = <container-name>
LXC_ROOTFS_MOUNT = /usr/lib/x86_64-linux-gnu/lxc/rootfs
LXC_ROOTFS_PATH = /var/lib/lxc/<container-name>/rootfs
LXC_HOOK_VERSION = 0
LXC_CGNS_AWARE = 1
# In case of lxc.hook.stop
LXC_TARGET = stop

Adding/Removing iptables rule

Create a bash shell script iptable-rules.sh in directory /var/lib/lxc/<container-name>/.

#!/bin/bash

# Sanity check on arguments.
if [[ -z "${2}" ]]; then
    echo "Argument missing."
    exit 1
fi
# Rule description part. 
rule=(PREROUTING --table nat --protocol tcp --dport 3389 --jump DNAT --to-destination "${2}:3389")
# Create rule.
if [[ "$1" == "pre" ]]; then
    # When the rule does not exist create it.
    if ! iptables --check "${rule[@]}"; then
        iptables --append "${rule[@]}"
    fi
# Remove rule.
elif [[ "$1" == "post" ]]; then
    # When the rule exists delete it.
    if iptables --check "${rule[@]}"; then
        iptables --delete "${rule[@]}"
    fi
fi

Add the following content to the /var/lib/lxc/<container-name>/config/file.

Absolute script path is required.
Instead of passing arguments environment variables LXC_TARGET and LXC_CONFIG_FILE be used in the script.

# Add iptables rule when starting and stopping.
lxc.hook.pre-start = /mnt/vms-users/lxc/<container-name>/iptable-hook.sh pre 10.0.9.2
lxc.hook.stop = /mnt/vms-users/lxc/<container-name>/iptable-hook.sh post 10.0.9.2

Auto Starting Container

To also start a container when the host starts add this the container's config file normally in /var/lib/<container-name>/config.

# Enable auto start of this container.
lxc.start.auto = 1
# Delay start this container 45 seconds from when the host starts.
lxc.start.delay = 15

Migrating Container to Another Host

To this stop the container on the old-host and use the next rsync command and options to copy the rootfs directory to the new host. The option --numeric-ids prevents rsync from mapping uid and gid from the files which otherwise results in non-operational container. The same command can be used to keep old and new hosts in sync due to the --delete option which does nothing when ` rootfs.

rsync -av --numeric-ids --delete root@<old-host>:/var/lib/lxc/<container-name>/rootfs /var/lib/lxc/<container-name>/

The configuration file /var/lib/lxc/<container-name>/config must be adapted after manually copying it to the new host since the ip-range could have changed and maybe /var/lib/lxc is not the storage location for lxc containers.

Enable IPv6 (NAT) within the Container

The IPv6-range fd00::/8 is a ULA (Unique local address) is what is used internally for the containers.

Add the following lines to file /etc/default/lxc-net.

# IPv6 settings
LXC_IPV6_ADDR = "fd00::1"            # IPv6 address of the bridge
LXC_IPV6_MASK = "64"                 # Prefix length for IPv6
LXC_IPV6_NETWORK = "fd00::/64"       # IPv6 network range for containers
LXC_IPV6_NAT = "true"                # Enable NAT for IPv6 (optional)

To make the changes have effect restart the service lxc-net.

systemctl restart lxc-net

The variable LXC_IPV6_NAT="true"" can conflict with Docker and must then be commented out.
The ip6tables must be added manually.

Adding IPv6 NAT rules manually (not-persistent).

# Remove all 'nat' POSTROUTING rules.
ip6tables --table nat --flush POSTROUTING
# Add a posting rule 
ip6tables --table nat --append POSTROUTING --out-interface ens6 --jump MASQUERADE

Enable IPv6 forwarding on the host when not enabled yet.

Check if it is enabled by using the next command to query the value.

sysctl net.ipv6.conf.default.forwarding

To enable it non-persistent use the next command.

sysctl --write net.ipv6.conf.default.forwarding 1

To make this reboot/persistent add the file /etc/sysctl.d/20-ipv6-forwarding.conf with the following content.

net.ipv6.conf.all.forwarding = 1

When Docker is also installed it could be that forwarded packets are dropped.
To check this use the next command.

ip6tables --list --verbose --values
# Short options version of the same command.
ip6tables -nvL

To fix/overrule this use this command.

ip6tables --policy FORWARD ACCEPT

Adding an ip-address to the container manually avoiding changing lxc-net.

ip -6 addr add fd00::100/64 dev eth0

Add the default route (gateway) for host address fd00::1.

ip -6 route add default via fd00::1 dev eth0

Keep MAC-Adress Reboot Proof

The newer lxc-utils package does not keep a containers MAC-address static.
After each reboot the MAC-address changes and so its ip-address.

The file /etc/lxc/default.conf is missing the next line which is a MAC-address of some sort.

lxc.net.0.hwaddr = 00:16:3e:xx:xx:xx

When a container is created using lxc-create it uses that template to create the entry lxc.net.0.hwaddr in the containers config file at /var/lib/lxc/<container-name>/

Add the next line to an existing containers config file and modify the last ab:cd:ef part for each different container.

lxc.net.0.hwaddr = 00:16:3e:ab:cd:ef

Manually Set IP-address

Using the `config` file

To manually set or add an ip-address for a container, modify the config file at /var/lib/lxc/<container-name>/ and add these two lines.

Change the ipv4-range in /etc/default/lxc-net to prevent ip conflicts.
Also the lxc.net.0.ipv4.gateway setting seems not to working and is maybe only working in combi with dnsmasq.

lxc.net.0.ipv4.address=10.0.3.2/24
lxc.net.0.ipv4.gateway=10.0.3.1

Using the `dnsmasq.conf` file

Modify file /etc/default/lxc-net and LXC_DHCP_CONFILE=/etc/lxc/dnsmasq.conf.
Edit or create /etc/lxc/dnsmasq.conf with lines like dhcp-host=container-name,10.0.3.2 like below.

dhcp-host=container1,10.0.3.2
dhcp-host=container2,10.0.3.3

Ubuntu Container Auto ip-address & gateway

In the ubuntu container a netplan configuration plan is added in the form of a file
/etc/netplan/10-lxc.yaml. Disabling this file by renaming it to /etc/netplan/10-lxc.yaml-disabled makes the container not assign an ip-address related to the MAC-address and also not a default gateway.

To get some netplan information execute the following.

netplan status