Latest revision as of 18:38, 29 December 2020

We run a Buildbot instance for OLA.

• Web UI: http://buildbot.openlighting.org
• IRC: OLA-buildbot-ch in #openlighting-build on irc.freenode.org

We're currently only building ola, but this could be extended to do other projects such as the QT GUI etc.

Master Configuration

The buildbot master configuration is stored in git at https://github.com/OpenLightingProject/buildbot . It needs checking out, and symlinking to replace master.cfg in the buildbot master's config directory.

N.B. If the main open-lighting git repository (as opposed to buildbot config) is ever moved or changed, ensure the master's gitpoller-workdir folder is emptied out, or the poller won't work properly.

Adding a Slave

Buildbot documentation is at http://docs.buildbot.net/current/manual/installation.html#creating-a-buildslave . The steps below should cover everything you need though.

Prerequisites & Warning

Slaves execute code directly from the git repo. Even though submits to the git repo are locked down, this is still a possible attack vector for your machine. For this reason it's best to run build slaves within a virtual machine. TODO: link to some VM solutions.

At the very least you should run the buildslave as a separate user (not root!). Slave passwords aren't stored in the git repo for security, you'll need to get Simon or Peter to add new ones.

The buildbot performs full build & test runs with all the options enabled. Please make sure you have all the necessary libraries installed on your system. You need to be able to complete a

autoconf -i
./configure --enable-e133 --enable-rdm-tests --enable-python-libs --enable-ja-rule
make
make check

cycle before proceeding. If you have trouble ask on the mailing list.

If you're running the lint check, you need cpplint.py in your path somewhere, see README.developer for info on how to obtain it.

Buildbot slaves need to connect to buildmaster.openlighting.org:9989 . Make sure your firewall allows this. No port forwarding for inbound connections is required.

Debian / Ubuntu Instructions

This requires wheezy or later. For squeeze you can use the easy_install method below.

Build Slave Installation

 sudo apt-get install buildbot-slave

Slave Configuration

Create the slave (the package creates the buildbot user automatically for you):

 sudo -u buildbot buildslave create-slave --usepty=0 /var/lib/buildbot/slaves/ola buildmaster.openlighting.org:9989 <slave user> <slave password>

Update the slave info, edit the files in /var/lib/buildbot/slaves/ola/info to be relevant to you.

Add config for the slave into /etc/default/buildslave (you'll need to increase the array id if you've got more than one slave on the same host), e.g.:

 SLAVE_ENABLED[1]=1                    # 1-enabled, 0-disabled
 SLAVE_NAME[1]="ola"         # short name printed on start/stop
 SLAVE_USER[1]="buildbot"              # user to run slave as
 SLAVE_BASEDIR[1]="/var/lib/buildbot/slaves/ola"                   # basedir to slave (absolute path)
 SLAVE_OPTIONS[1]=""                   # buildbot options
 SLAVE_PREFIXCMD[1]=""                 # prefix command, i.e. nice, linux32, dchroot

Start the slave

sudo /etc/init.d/buildslave start

Check the log if there are any issues, or confirm the slave is registered at http://buildbot.openlighting.org/buildslaves:

tail -f /var/lib/buildbot/slaves/ola/twistd.log

FreeBSD Instructions

This was tested on FreeBSD 10.

Build Slave Installation

As root:

 pkg install buildbot-slave

Slave Configuration

Setup a new user:

su -
adduser

Username: ola-build-slave Use password-based authentication? No

Setup the slave:

Switch to the slave user, I did:

su - #To root
su - ola-build-slave #To slave user
cd ~

Export the variables needed for configure on FreeBSD:

export CPPFLAGS="-I/usr/local/include/"
export LDFLAGS="-L/usr/local/lib/"

Setup the slave itself:

/usr/local/bin/buildslave create-slave ola-slave buildmaster.openlighting.org:9989 <slave user> <slave password>

Edit ola-slave/info/admin and ola-slave/info/host so your slave shows up correctly.

Then start the slave:

/usr/local/bin/buildslave start ola-slave

You can look at the logs by running

 tail -f ola-slave/twistd.log

At this point you can go to http://buildbot.openlighting.org/buildslaves and you should see your slave connected. It's probably worth asking someone to kick off a build at this point so we can check your slave is working.

Finally, if everything looks good, configure your slave to launch on startup by editing the crontab for the ola-build-slave user (crontab -e). Add the following line:

@reboot /usr/local/bin/buildslave start /home/ola-build-slave/ola-slave

NetBSD Instructions

This was tested on NetBSD 6.

Build Slave Installation

As root:

 pkgin install py27-buildbot-slave

Slave Configuration

Setup a new user:

su -
useradd -m ola-build-slave

Use password-based authentication? No

Setup the slave:

Switch to the slave user:

su - ola-build-slave
cd ~

Export the variables needed for git and configure on NetBSD (add these to the .shrc or equivalent for next time):

export CPPFLAGS="-I/usr/pkg/include/"
export LDFLAGS="-L/usr/pkg/lib/"
export GIT_SSL_NO_VERIFY=true

Setup the slave itself:

/usr/pkg/bin/buildslave create-slave ola-slave buildmaster.openlighting.org:9989 <slave user> <slave password>

Edit ola-slave/info/admin and ola-slave/info/host so your slave shows up correctly.

Then start the slave:

/usr/pkg/bin/buildslave start ola-slave

You can look at the logs by running

 tail -f ola-slave/twistd.log

At this point you can go to http://buildbot.openlighting.org/buildslaves and you should see your slave connected. It's probably worth asking someone to kick off a build at this point so we can check your slave is working.

Finally, if everything looks good, configure your slave to launch on startup by editing the crontab for the ola-build-slave user (crontab -e). Add the following line:

@reboot /usr/pkg/bin/buildslave start /home/ola-build-slave/ola-slave

OpenBSD Instructions

This was tested on OpenBSD 5.4.

Build Slave Installation

As root:

pkg_add py-buildslave

Slave Configuration

Setup a new user:

su -
useradd -m ola-build-slave

Setup the slave:

Switch to the slave user:

su - ola-build-slave
cd ~

Export the variables needed for autoconf and configure on OpenBSD (add these to the .profile or equivalent for next time), the eg++/egcc lines are needed if the installed gcc is too old to pass configure, install the eg++ one and then use that line:

export AUTOCONF_VERSION=2.69
export AUTOMAKE_VERSION=1.13
export CPPFLAGS="-I/usr/local/include/"
export LDFLAGS="-L/usr/local/lib/"
export CXX="eg++" CC="egcc"

Setup the slave itself:

/usr/local/bin/buildslave create-slave ola-slave buildmaster.openlighting.org:9989 <slave user> <slave password>

Edit ola-slave/info/admin and ola-slave/info/host so your slave shows up correctly.

Then start the slave:

/usr/local/bin/buildslave start ola-slave

You can look at the logs by running

 tail -f ola-slave/twistd.log

At this point you can go to http://buildbot.openlighting.org/buildslaves and you should see your slave connected. It's probably worth asking someone to kick off a build at this point so we can check your slave is working.

Finally, if everything looks good, configure your slave to launch on startup by editing the crontab for the ola-build-slave user (crontab -e). Add the following line:

@reboot /usr/local/bin/buildslave start /home/ola-build-slave/ola-slave

Other Platforms

Build Slave Installation

The easiest way to get started is by using easy_install. You need to have the Python headers available, so on Debian / Ubuntu run:

sudo apt-get install python-dev

Then install buildbot-slave:

easy_install buildbot-slave

Slave Configuration

Setup a new user:

sudo -s 
adduser ola-build-slave  # use a temp password for now
vi /etc/shadow  # delete the password entry

Setup the slave:

su ola-build-slave
cd ~
buildslave create-slave ola-slave buildmaster.openlighting.org:9989 <slave user> <slave password>

Edit ola-slave/info/admin and ola-slave/info/host so your slave shows up correctly.

Then start the slave:

buildslave start ola-slave

You can look at the logs by running

 tail -f ola-slave/twistd.log

At this point you can go to http://buildbot.openlighting.org/buildslaves and you should see your slave connected. It's probably worth asking someone to kick off a build at this point so we can check your slave is working.

Finally, if everything looks good, configure your slave to launch on startup by editing the crontab for the ola-build-slave user (crontab -e). Add the following line:

@reboot buildbot start /home/ola-build-slave/ola-slave

Enabling the C++ Lint Checker

The lint checker enforces C++ style. We only run this once per change but it's good to have multiple lint-enabled hosts in case one is down.

Download the cpplint checker and put it in /usr/local/bin

wget http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py
sudo cp cpplint.py /usr/local/bin
sudo chmod a+x /usr/local/bin/cpplint.py

Finally let Simon or Peter know, or raise an issue, asking them to enable C++ lint for your slave.

Enabling the Javascript Lint Checker

The lint checker enforces Javascript style. We only run this once per change but it's good to have multiple lint-enabled hosts in case one is down.

Either:

• Install the closure-linter deb package if it's present on your OS
• Install the gjslint checker by following the relevant instructions here (you man need to install python-setuptools to get easy_install on Debian/Ubuntu) https://developers.google.com/closure/utilities/docs/linter_howto

Finally let Simon or Peter know, or raise an issue, asking them to enable Javascript lint for your slave.

Enabling the heap checker

First we need libunwind:

sudo apt-get install libunwind8-dev 

Then download the latest gperftools from https://code.google.com/p/gperftools/downloads/list .

wget ....
tar -zxf gperftools-2.1.tar.gz
cd gperftools-2.1
./configure
make
sudo make install
sudo ldconfig

Finally let Simon or Peter know, or raise an issue, asking them to enable the heap checker for your slave.