System42.net Blag.

Ubuntu 10.04 LXC container script

2010-05-18T00:00:00Z

This script will create a LXC container running Ubuntu Server 10.04, you should change the variables at the top for your environment. You may want to modify some of the embedded scripts to change the use of puppet.

The script is semi-interactive. You will have to select the base console language as well as add a user and password with sudoer access throughout the script’s run.

/usr/local/bin/lxc-urizen.sh:

#!/usr/bin/env bash

# This script takes 2 or 3 arguments
# ./lxc-urizen new <hostname> <IP Address>
# or
# ./lxc-urizen delete <hostname>
# 
# This script was written by Matt Bailey for the University of Denver
# A few of the hacks to make this work properly are taken from the blog 
# at http://blog.bodhizazen.net/
#
# This is built almost exclusively for Ubuntu Server Lucid (10.04). Its 
# usefulness for other distros or ubuntu releases is limited without
# significant modification.


# ARGV
TASK="$1"
HOSTNAME="$2"
IP="$3"

NETMASK="<NETMASK>"
GATEWAY="<IP ADDRESS>"
CIDR="<CIDR SUBNET>"
BRIDGE="br0"
DIST="lucid"
ROOTFS_DIR="<PLACE TO BUILD FILESYSTEM"
PUPPETIP="<IP OF PUPPET SERVER>"
PATH="${PATH}:/opt/bin"


WORK=`mktemp -d`
echo $WORK


function new {

    echo "*** RUNNING DEBOOTSTRAP"
    debootstrap --variant=minbase ${DIST} ${ROOTFS_DIR}/${HOSTNAME}/ > /dev/null

    echo "*** BUILDING LXC CONFIG"

    cat > ${WORK}/${HOSTNAME}.conf << EOF
lxc.utsname = ${HOSTNAME}
lxc.tty = 4
lxc.network.type = veth
lxc.network.flags = up
lxc.network.link = ${BRIDGE}
lxc.network.name = eth0
lxc.network.mtu = 1500
lxc.network.ipv4 = ${IP}
lxc.rootfs = ${ROOTFS_DIR}/${HOSTNAME}
lxc.cgroup.devices.deny = a
# /dev/null and zero
lxc.cgroup.devices.allow = c 1:3 rwm
lxc.cgroup.devices.allow = c 1:5 rwm
# consoles
lxc.cgroup.devices.allow = c 5:1 rwm
lxc.cgroup.devices.allow = c 5:0 rwm
lxc.cgroup.devices.allow = c 4:0 rwm
lxc.cgroup.devices.allow = c 4:1 rwm
# /dev/{,u}random
lxc.cgroup.devices.allow = c 1:9 rwm
lxc.cgroup.devices.allow = c 1:8 rwm
# /dev/pts/* - pts namespaces are "coming soon"
lxc.cgroup.devices.allow = c 136:* rwm
lxc.cgroup.devices.allow = c 5:2 rwm
# rtc
lxc.cgroup.devices.allow = c 254:0 rwm
EOF

    echo "*** CREATING LXC CONTAINER"
    lxc-create -n ${HOSTNAME} -f ${WORK}/${HOSTNAME}.conf


    echo "*** BUILDING CLIENT SCRIPT"
    cat > ${WORK}/lxc_init.sh << EOF
#!/bin/bash

# Re-do device nodes
rm -rf /dev
mkdir /dev
mknod -m 666 /dev/null c 1 3
mknod -m 666 /dev/zero c 1 5
mknod -m 666 /dev/random c 1 8
mknod -m 666 /dev/urandom c 1 9
mkdir -m 755 /dev/pts
mkdir -m 1777 /dev/shm
mknod -m 666 /dev/tty c 5 0
mknod -m 666 /dev/tty0 c 4 0
mknod -m 666 /dev/tty1 c 4 1
mknod -m 666 /dev/tty2 c 4 2
mknod -m 666 /dev/tty3 c 4 3
mknod -m 666 /dev/tty4 c 4 4
mknod -m 600 /dev/console c 5 1
mknod -m 666 /dev/full c 1 7
mknod -m 600 /dev/initctl p
mknod -m 666 /dev/ptmx c 5 2

# Needed for GPG signed repos
apt-get install --force-yes -y gpgv
apt-get update

# Set locale and language
apt-get install -y language-pack-en
update-locale LANG="en_US.UTF-8" LANGUAGE="en_US.UTF-8" LC_ALL="en_US.UTF-8"

# Base pack for ubuntu server
apt-get install -y adduser apt-utils iproute netbase nano openssh-blacklist openssh-blacklist-extra openssh-server sudo ping
DEBCONF_FRONTEND='noninteractive' apt-get install -y console-setup

# Setup admin user
echo "*** CREATE NEW USER THAT WILL HAVE FULL SUDO ACCESS"
groupadd admin
echo -n "USERNAME: "
read USERNAME
useradd -m -G admin \${USERNAME} -s /bin/bash
echo "%admin ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers
passwd \${USERNAME}

# Write interfaces file
cat > /etc/network/interfaces << EOFNET
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
address ${IP}
netmask ${NETMASK}
gateway ${GATEWAY}
EOFNET

rm -f /etc/init/tty{4,5,6}.conf
rm -f /etc/init/plymouth.conf
rm -f /etc/init/plymouth-log.conf
rm -f /etc/init/plymouth-splash.conf
mkdir -p /var/run/network
touch /var/run/network/ifstate
mkdir -p /var/run/sshd

sed -i 's/start on filesystem and net-device-up IFACE=lo/start on filesystem # and net-device-up IFACE=lo/' /etc/init/rc-sysinit.conf

cat > /etc/init/lxc.conf << EOFLXC
# LXC – Fix init sequence to have LXC containers boot with upstart

# description "Fix LXC container - Lucid"

start on startup

task
pre-start script
mount -t proc proc /proc
mount -t devpts devpts /dev/pts
mount -t sysfs sys /sys
mount -t tmpfs varrun /var/run
mount -t tmpfs varlock /var/lock
mkdir -p /var/run/network
touch /var/run/utmp
chmod 664 /var/run/utmp
chown root.utmp /var/run/utmp
find /etc/network -name upstart -type f | xargs chmod -x 
end script

script
start networking
initctl emit filesystem --no-wait
initctl emit local-filesystems --no-wait
initctl emit virtual-filesystems --no-wait
init 2
end script
EOFLXC

echo "127.0.0.1 localhost" >> /etc/hosts
echo "${IP} ${HOSTNAME}" >> /etc/hosts
echo "${PUPPETIP} puppet" >> /etc/hosts
echo "${HOSTNAME}" > /etc/hostname


apt-get install -y puppet

echo "START=yes" > /etc/default/puppet
echo 'DAEMON_OPTS=""' >> /etc/default/puppet


EOF

    echo "*** RUNNING CLIENT SCRIPT"
    mv ${WORK}/lxc_init.sh ${ROOTFS_DIR}/${HOSTNAME}/tmp/lxc_init.sh
    chmod +x ${ROOTFS_DIR}/${HOSTNAME}/tmp/lxc_init.sh
    chroot ${ROOTFS_DIR}/${HOSTNAME} /tmp/lxc_init.sh
    rm ${ROOTFS_DIR}/${HOSTNAME}/tmp/lxc_init.sh

    echo "*** SETUP COMPLETE, RUN WITH lxc-start -n ${HOSTNAME}"

}

function delete {
    lxc-destroy -n ${HOSTNAME}
    rm -rf ${ROOTFS_DIR}/${HOSTNAME}
}

${TASK} || echo "USAGE: argv[1] needs to be either new or delete"

# Clean-up
rm -rf ${WORK}

Bonding and bridging for LXC

2010-04-19T00:00:00Z

I’m using LXC for a project at work. Setting up the networking became a bit complex, so I did a bit of scripting to go around ubuntu server’s standard interfaces file. My requirements here are that I have two (or really n) physical interfaces that need to be bonded with LACP, and a bridge interface on top of the bond0 interface.

This config is probably useful for other virtualization set ups, but YMMV; this is really just for LXC on Ubuntu Server (Lucid). Click read on to see the scripts.

My /etc/network/interfaces file looks like this:

auto lo
iface lo inet loopback

auto bond0
iface bond0 inet static
address <ADDRESS>
gateway <VALID GATEWAY>
netmask <NETMASK>
slaves eth0 eth1
bond-mode 4  # This is for 802.3ad; must match the pre-up line
bond-miimon 100  # This also needs to match the pre-up modprobe
dns-nameservers <SOME DNS SERVER>
pre-up /sbin/modprobe bonding mode=4 miimon=100  # check kernel docs for the right mode
post-up /usr/local/bin/urizen-net.sh up  # my little script for bridging
pre-down /usr/local/bin/urizen-net.sh down

/usr/local/bin/urizen-net.sh:

#!/bin/bash

IP="<PRIMARY HOST IP>"
MASK="<HOST IP NETMASK>"
GATEWAY="<HOST GATEWAY>"

if [ "$1" = "up" ]
then
    # Finish the bond
    /sbin/ifenslave bond0 eth0 eth1

    # Bring up the bridge
    /usr/sbin/brctl addbr br0
    /usr/sbin/brctl setfd br0 0
    /usr/sbin/brctl addif br0 bond0
    /sbin/ifconfig br0 ${IP} netmask ${MASK} promisc up

    # Reconfigure the bond to be 0.0.0.0 and re-set up the gw
    /sbin/ifconfig bond0 0.0.0.0 up
    /sbin/route add default gw ${GATEWAY}

    # turn scatter-gather and tcp-segmentation-offload off on the bridge
    /usr/sbin/ethtool -K br0 sg off
    /usr/sbin/ethtool -K br0 tso off
fi

if [ "$1" = "down" ]
then
    /sbin/ifconfig br0 down
    /usr/sbin/brctl delbr br0
    /sbin/ifenslave -d bond0 eth0 eth1
fi

New little blog engine

2010-02-07T00:00:00Z

Using toto now (http://github.com/cloudhead/toto). It’s pretty cool, and is almost designed to run on Heroku’s free service. I have comments disabled right now (thru disqus), but it does support them; for now, I’m keeping it super simple.

Apache per-vhost logging

2010-02-07T00:00:00Z

I wrote this little article before; after solving a problem to get zero configuration apache vhosts with separate logs in apache. It’s a minimal configuration that is extremely fast due to apache’s bulit-in perl acceleration.

Fit this in with your apache config, then you can simply create directories under /web/hosts and they will automatically be your vhosts. No restarting apache, no extra config.

<VirtualHost IP ADDRESS> 
  # get the server name from the Host: header 
  UseCanonicalName Off

  # Farm logfiles out to small script to dump into host/logs dir 
  LogFormat "%V %h %l %u %t %r %s %b" vcommon 
  CustomLog "| /usr/local/bin/logpush.pl /web/hosts" vcommon 

  # include the server name in the filenames used to satisfy requests 
  VirtualDocumentRoot /web/hosts/%0/htdocs 
  VirtualScriptAlias /web/hosts/%0/cgi-bin 
</VirtualHost>

And here is the logpush.pl file:

#!/usr/bin/perl 
use strict; 
while( my $line=<STDIN> ) { 
  my @logline = split(/ /, $line); 
  mkdir "/$ARGV[0]/$logline[0]/logs"; 
  open(LOG, ">>/$ARGV[0]/$logline[0]/logs/access_log");
  print LOG "$line"; close(LOG); 
} 
exit(0);