SPY HILL Research
Spy-Hill.net
Poughkeepsie, New York [DIR] [UP]

Building LIGO's Global Diagnostic System (GDS)

Instructions are given for building LIGO's Global Diagnostic System (GDS) on Unix (primarily Linux), with notes in particular on dependencies on built tools, platform-specific issues, and other tricky parts.
Last updated: 6 September 2009
TOPICS
Overview
Prerequisites
Source Code
Configuration
Build and Install
General Notes
Platform-specific notes:
Links and References

LIGO's Global Diagnostic System (GDS) is a collection of monitoring and analysis tools commonly used in the LIGO control rooms. GDS is essentially a combination of DTT (Diagnostic Test Tools) and DMT (the Data Monitor Tool). In turn, DMT is both a collection of on-line "monitor" daemons and a framework for building such monitors. DMT includes C++ container classes for dealing with common LIGO data objects. DMT libraries are also the basis for "dmtroot", a collection of ROOT macros which can make use of those container classes and methods, which is distributed via the LIGOtools packaging system. GDS also includes the C and C++ libraries (Fr and FrameCPP) for accessing and manipulating data in IGWD frame file format.

GDS is primarily built and maintained for the Sun workstations in the LIGO control rooms, though it can also be built on Linux. Some extra considerations may be required to build GDS on a particular Linux distrubution (see the platform-specific notes below).

This page provides instructions for buildng GDS from source code, which can be somewhat involved, but has gotten easier over the years. It is now also possible to install GDS binaries directly from rpm files for CentOS-5 or .deb files for Debian 5 (Lenny). If you can go that way, your life will be much easier. If not, the source code for GDS and the dependencies (framecpp-1.14.0, metaio-8.2, etc) are also still available. To download either the binary packages or the source code, visit

https://www.lsc-group.phys.uwm.edu/daswg/download/repositories.html
(Or download source tarballs directly from here.)

GDS version 2.11.1 was for a long time (through S5) the best version to use, at least for my purposes (running DMT ROOT scripts for I2U2). However, there was a significant update for LIGO run S6, and GDS 2.13 is now the current version. Source code is available for 2.13.1, while the version currently running the online DMT Monitors is called 2.13.2. The proceedure for building GDS had changed in significant ways during this period; it may or may not change further during S6.

In the summer of 2009, in preparation for S6, LIGO switched to using "version 8" frame files. They previously used "version 6" files. To be able to read and process version 8 frame files you will need to get a newer version of the Frame Library and FrameCPP library, and you will probably have to modify the GDS build script to use this newer library (see below). As of 9 June 2009 the newest version of FrameCPP is 1.14.0, and this is the version used by GDS 2.13.

The instructions below, which are still being updated (as of September 2009), are for building GDS 2.13.1 with FrameCPP 1.14.0 (not 1.10.1) and metaio 8.2 on Ubuntu 8.04 (for the I2U2 cluster) and RHEL 5 (for the Teraport cluster). I have previously built GDS 2.11.1 (with FrameCPP 1.8.235) on Fedora Core 4 and 5 machines, and on Debian 3.1 (Sarge). Before that I succeeded in building GDS 2.10.8 on Fedora Core 3 (with gcc 3.4.4) and Fedora Core 4 (with gcc 4.0.2).

Technical notes or other "side" comments appear below in boxes like this one.

Prerequisites

Before you can build GDS you need to have the right tools, which are listed below. The version numbers were for building GDS-2.11.1 unless otherwise noted.

I have found that Fedora 4 and 5 have new enough versions of these tools that GDS will build on those distributions without having to update any of these tools. In contrast, I had to install newer versions of most if not all of these tools on Debian 3.1 and on Solaris 5.9.

pkg-config (version 0.18 or better) [http://pkg-config.freedesktop.org/wiki/]
This seems to be the most sensitive dependency. If you run the "configure" script and get an error about PKG_CHECK_MODULES then your version of pkg-config is too old.

libtool (version 1.5.16 or better) [http://ftp.gnu.org/gnu/libtool/]
It has been reported that building gcc-4.2.3 requires libtool-2.2, but that `make uninstall` for that release of libtool is broken and it had to be uninstalled manually. I'm not sure yet that this means that you should avoid libtool 2.2 or what.

I have also seen suggestions that newer versions of libtool (specificially 2.2.6) do not work, because it is too new. It may well be that any libtool 2.x version won't work. I'll report further findings if there are any.

autoconf (Version 2.13 or better) [http://ftp.gnu.org/gnu/autoconf/]
Regardless of what version of autoconf you have as part of your Unix distribution, if running the bootstrap script gives complaints about AC_CONFIG_AUX_DIR or any AC_.... symbols, then you probably need to build and install a newer version of autoconf. The problem may not be just in the version, but in the configuration.

automake (version 1.10 required for GDS 2.12.4) [http://ftp.gnu.org/gnu/automake/]
Regardless of what version of automake you have as part of your Unix distribution, if running the bootstrap script gives complaints about AM_PROG_LIBTOOL, AM_INIT_AUTOMAKE, or any AM_.... symbol, then you probably need to build and install a newer version of automake. The problem may not be just in the version of the code, but in how it's installed and configured on your system.
(automake version 1.72 or better was required for GDS 2.11.1)

ROOT [http://root.cern.ch]
We use version 4.04/02g for the LIGO Analysis Tool, with GDS 2.11.1. We would like to use ROOT 5, but we've not yet had experience building GDS with it. You must both build it and install it on your system, in order for GDS to properly link against it when building the DMT shared object libraries. Set the environmental ROOTSYS to point to the top-level directory of your ROOT installation.

ROOT 4.04 won't build with gcc 4.x, as it doesn't include some necessary headers files specific to that version of the compiler.

If you intend to distribute ROOT to other machines (eg. via LIGOtools) you should build it static (with the "--disable-shared" option), and you should be sure to not specify any directory paths, so that all directory names are relative to ROOTSYS.

curl [http://curl.haxx.se]
The version of the GDS source tree which you obtain from CVS does not include the source for the FrameCPP library, so when you first say `make` it will try to get the FrameCPP source tarball using curl.

Debian does not have curl, but it does have wget. To use wget simply edit External/Makefile and change HTCOPY from "curl" to "wget". Or download and build your own version of curl.

libbz2 (version 1.0.2 or newer) [http://www.bzip.org]
The bzip2 library is used in building the FrameCPP library. The libbz2-1.0 in Debian 3.1 (Sarge) is too old, or there seems to be some other problem.(Investigating further. See notes below.)

m4 (GNU version 1.4 or better) [http://ftp.gnu.org/gnu/m4]
The version of m4 which comse with ccs on Solaris 5.9 (/usr/ccs/bin/m4) is too old. You'll need to build the GNU m4. All flavours of Linux come with GNU m4, so you won't need to build m4 on Linux.

In the examples which follow below "%" is used as the Unix prompt, so you don't type it; only type the command which follows it. Similarly, "#" is used as the "root" prompt, to show commands which need to be given as root. In the examples I install things under $HOME/opt, but you could pick somewhere else, such as /opt or /usr/local/lscsoft

Obtain Source Code

The source code for GDS is available for download from the LIGO DASWG repository at the University of Wisconsin at Milwaulkee:
https://www.lsc-group.phys.uwm.edu/daswg/download/software/source/

The source code for GDS can also be obtained via CVS thus,

% cvs -d :pserver:readonly@ldas-sw.ligo.caltech.edu:/ldcg_server/common/repository_gds checkout gds
with password "m0n1t0r". However, the LDAS CVS server at Caltech is configured to only make the repository available to LIGO collaboration institutions, based on IP addresses. To check the code out from CVS you will need to be on a machine at LHO or another LIGO site.

If you need to get newer or GDS code and can't get it via CVS then you can also try to contact either Eric Myers to get him to make you a tarball (if this is for I2U2), or contact John Zweizig at Caltech (for LIGO but not I2U2).

"External" code

GDS relies on several software components which are considered to be external to GDS, including FrameL (the Frame Library, aka Fr), FrameCPP, metaio, fftw, expat, zlib, and readline. If these are found on your system then they will be used. If not, they can be easily built.

In past versions of GDS the Makefile in the External subdirectory was configured to automatically build the packages that needed it, and to automatically download the source code for several of those packages if the source code package was not in place. As of GDS 2.13 that has changed. The instructions for building FrameL, FrameCPP and metaio etc are still in place, but are commented out. You may find it easier to build these separately and install them. Or you can uncomment the sections in the Makefile to have those external packages built as part of one large build, as was the convention in the past.

Frame Library

The Frame Library (Fr, or sometimes FrameL) is a collection of functions for accessing and manipulating data in IGWD frame format, either as data structures or files. The Frame Library also includes standalone utility programs (FrCopy, FrDump and FrCheck) to read, write and verify frame files. The Frame Library is written in C. It should not be confused with the C++ FrameCPP library (see below).

The home page for the Frame Library is http://lappweb.in2p3.fr/virgo/FrameL. The recommended version as of June 2009 is v8r08.

FrameCPP

FrameCPP is a C++ library for accessing and manipulating data in IGWD frame format, separate from FrameL (it is instead a part of LDAS).

The FrameCPP source code is not included in the GDS source package. If FrameCPP is not already built then the External/Makefile can try to download it from the DASWG repository and build it automatically. But if this automatic build fails (as it has for me, with no reason given), then you will need to download the source package yourself, (from https://www.lsc-group.phys.uwm.edu/daswg/download/software/source/) and configure and build this first, before you build the rest of GDS.

Each version of GDS had a particular version of FrameCPP hard-coded into it: GDS 2.11.1 would download FrameCPP 1.8.235, while GDS 2.12.4 would download FrameCPP 1.10.1. If you want a different version of FrameCPP then you will have to edit External/Makefile to change the version number for FrameCPP.
When FrameCPP is built as a part of GDS a script called patchall may be run to make minor adjustments to the FrameCPP source distribution. If you are building FrameCPP by hand you probably want to apply these patches yourself. To do so, go to the External subdirectory and say
% patchall framecpp-1.9.120
using the name of the source directory for FrameCPP as the argument.

There are no patches for FrameCPP 1.10.1 in GDS 2.12.4, nor for FrameCPP 1.14.0 in GDS 2.13.1.

If you have to build GDS multiple times, as is the case when trying to get it to build on a new Linux distribution, it can be annoying to have to wait for FrameCPP to download and build each time (which can take more than 20 minutes). You can get around the download by putting a copy of the FrameCPP tarball in the External subdirectory. It will then be unpacked and built automatically. You can get around repeatedly building the same thing by going to the External subdirectory and making a tarball of the framecpp-1.xx.yyy subdirectory in it's entirety. Then untar it when needed to bypass rebuilding FrameCPP yet again. Don't forget to cd into it and do make install, to install the library and headers under the External directory. If you don't you'll get a fresh build yet again.

Don't just tar the built directory up and expect it to be unpacked by the GDS build and left alone. If the GDS build unpacks a tarball it will then do a fresh configure and build of whatever was in that tarball, even if the tarball contained an already completed build of FrameCPP.

The key to building FrameCPP seems to be to remove the configuration file lib/framecpp/src/config.h before doing anything else. Then saying

# configure --prefix=$GDS_PATH # make # make install
seems to work (still need final verification).

Additional changes may be needed to build on a particular platform. See the platform-specific release notes below for details.

metaio

The metaio package contains a library for parsing LIGO_LW table files. It has been added to 2.12.3 and later. The source code is not included with the GDS tarball. Instead, if the source tarball is not found in the External directory then it will be downloaded from https://www.lsc-group.phys.uwm.edu/daswg/download/software/source/. If that fails, or if you want to use a different version, then you can download the tarball yourself and put it in the External directory. Edit Makefile.am if the version number is different.
There is a typo in External/Makefile.am which prevents metaio from being built properly. Edit the file and change the definition of METAIOVERS to just metaio-8.2 (remove the "lib" prefix).

fftw3

This library for performing Fast Fourier Transforms (FFT's) is now available in most if not all Linux distributions, so the source code is now no longer distributed with GDS. But it will be built if the source package fftw-3.0.1.tar.gz is found in the "External" directory, provided that you uncomment the instructions to do so in Makefile.am before you run the `configure` script.

libz

Thee libz library performs file compression. Most new Linux distros have it, so it's no longer necessary to build it separately. It was included in GDS 2.11.1 but not in 2.12.3 or later. If you need to build it, uncomment the instructions to do so in Makefile.am before you run the `configure` script.

readline

The readline library provides command-line editing functionality for a user interface. It is more common now to have this included in a Linux distribution, but not always. The source code for readline was once included in the GDS source tarball, but not anymore. If you find you need it, you can build it "by hand"; Better yet, arrange to have the readline developer package installed on your system, since it's useful generally, not just for GDS.

Configuration

The proceedure for building GDS has evolved toward use of standard GNU auto-tools, but there may still be some remenants of earlier methods. Here are the steps:
  1. Before GDS 2.12, when you unpacked the source code, you had to remove or rename the existing 'configure' script, if there was one, because it was part of an earlier installation method, not a proper GNU autoconf configure script. You just had to get it out of the way so that a new one could take it's place. This does not seem to be necessary anymore.
  2. Set the GDSBUILD variable to "online". For example 
        %  export GDSBUILD=online
    This is somewhat counterintuitive, since we will be doing our analysis "off line" (using data files for input rather than direct input from the Frame Builder), but setting this to "offline" disables important parts of the code which we need, and the "online" version can also read Frame files. (The distinction between online/offline in the build proceedure is annoying, and the implementation in the Makefiles is awkward at best.)
  3. Set the ROOTSYS environment variable to point to the top level installation of ROOT. For example, using a Bourne shell, 
        %  export ROOTSYS=$HOME/opt/root
  4. Set the PKG_CONFIG_PATH environment variable to point to the library containing pre-built External libraries, such as FrameL or metaio. For example, using a Bourne shell, 
        %  export PKG_CONFIG_PATH=/usr/local/lscsoft/gds/lib
  5. Edit External/Makefile.am as appropriate to uncomment the sections to build FrameCPP, metaio and Fr, and to set the version numbers for those packages.
    See the note above about chaning the definition of METAIOVERS to just metaio-8.2 (remove the "lib" prefix) in External/Makefile.am.
  6. If you don't have the FrameCPP or metaio source code tarball thens the External/Makefile will attempt to download it using curl. If you don't have curl but do have wget (eg. Debian Sarge) then you can edit External/Makefile.am and change HTCOPY from "curl" to "wget". But you only need to do this if you want the Makefile to do the automatic download, and you probably don't.
  7. Alternatively, download the FrameCPP and/or metaio tarballs and put it in the External subdirectory and they will be built, but you won't have to wait for the download. Remember to uncomment the appropriate sections in Makefile.am (or Makefile, if you forgot before running `configure`).
  8. Run the bootstrap script, which will run automake and autoconf to create template makefiles and the configure script: 
        %  ./bootstrap
  9. Run the new configure script, with "prefix" set to the top level directory where you expect to install everything. 
        %  ./configure --prefix=$HOME/opt/lscsoft/gds
  10. There are over 60 "Monitors" in DMT, and at any one time some of them may not build correctly on some platforms. That's okay, because unless it is a monitor you are specifically interested in using then you probably don't care. Still, the build will end clean if you disable the monitors you know won't build or you don't care about. To do this, edit Monitors/Makefile, find the definition of the symbol SUBDIRS and remove the ones which won't build.

    For GDS 2.11.1 built on Fedora 4 and 5 I found that BicoMon, Cumulus and PDnMon would not build, so I removed them. Also, DuoTone would not build on Debian Sarge. You may find others (also see below). BicoMon, in particular, often does not build. You can use `make -k` to see which ones won't build then edit the Makefile to remove those and say `make` again.

  11. If you have the source package for FrameCPP already downloaded, then copy it to the Exernal subdirectory. The Makefile will unpack it and build it at the appropriate time, and won't have to get it from Caltech. (See also the notes above about saving a tarball of the build.) If you have built it already you can save it to a tarball and then unpack it later, but do not make the mistake of naming the tarball the same as the source tarball, or the Makefile will unpack it, say 'make clean' and then you have to wait while it rebuilds.

Build and install

Do not forget to build and install ROOT first, as described above, and set ROOTSYS to point to the installation.

After you have run the "configure" command (above), you can build all of GDS with the command 

   %  make
When you build GDS it builds everything, from DMT and DTT to the FrameCPP libraries and fftw3 (if not found on your system) and all the Monitors.

If there is a problem building FrameCPP it will fail without any indication as to cause:

installing FrameCPP ... this will take a while (20 mins or more) ... make[1]: *** [lib/libframecpp.so] Error 2 make[1]: Leaving directory `/sandbox/myers/LIGO/gds/gds-2.11.1/External' make: *** [all-recursive] Error 1
The workaround was to build and install FrameCPP separately, without using the build-ldas script, as described above.

Even if FrameCPP builds, It's likely something else will fail the first time around, probably one or more of the monitors. You can get past this to see how big the problems are by giving the "-k" flag to the make command: 

   %  make -k
If you find that the problem is just in one or several of the Monitors then you can edit Monitors/Makefile and remove the offending monitors from the list in SUBDIRS. Then give the `make` command again.,

Once you have built GDS cleanly you can install (in the directory specified by the --prefix flag) with the command 

    %  make install
If you are installing the software in a "system" area then you likely need to become 'root' before you give this command.

General Build Notes

Platform-specific notes

Unfortunately, because GDS depends on bringing together a wide variety of software from various sources, it can be sensitive to which flavor of Unix you happen to be using. g The following platform specific problems may or may not still be an issue. I am including even the older Unix flavors/releases on this page for reference, in case a problem resurfaces, or in hopes that it might help solve some future problem.

Red Hat Enterprise Linux 5 (RHEL5)

I'm currently working on building ROOT and GDS on the Teraport cluster at the University of Chicago. Notes so far:

Ubuntu 8.04 (Hardy Heron)

Testing on www13.i2u2.org in the I2U2 Cluster at Argonne, (as well as other machines in that cluster, all Ubuntu 8.04).

Fedora 8

Working on this now...

Fedora Core 6

Tested on my laptop, a Dell Inspiron 8200.

SunOS 5.9 (Solaris)

(This section is out of date. I have not tried building on Suns in a long time. Contact John Zweizig if you need up to date info for Suns.)

Does GDS 2.11.1 build on SunOS 5.9 (Solaris)? I'm still in the process of testing on fortress.ligo-wa.caltech.edu (which has gcc 3.3.2, and not much else). Here's what I know so far:

Scientific Linux

There is a report from Ed Daw that Scientific Linux 4 caused build problems with framecpp due to mutex locking implementation incompatibilities, while Scientific Linux 5 did not have this problem.

Fedora Core 5

Building GDS 2.11.1 on Fedora Core 5 with gcc 4.1.1 and glibc 2.4 failed while building FrameCPP 1.8.235 with the following compiler error:
cc1plus: warnings being treated as errors Compression.cc: In function 'void::FrVectZExpandShort(INT_2S*, INT_2U*, INT_8U)': Compression.cc:844: warning: overflow in implicit constant conversion Compression.cc:852: warning: dereferencing type-punned pointer will break strict-aliasing rules Compression.cc:860: warning: dereferencing type-punned pointer will break strict-aliasing rules make[5]: *** [Compression_cmn.lo] Error 1 make[5]: Leaving directory `/home/myers/LIGO/gds/gds-2.11.1/External/framecpp-1.8.235/lib/framecpp/src/Common' make[4]: *** [install] Error 2
I have not investigated further on this platform. I don't know if this is the only such error, or if this is just a result of picky compiler settings.

Debian 3.1 (Sarge)

I was able to build GDS 2.11.1 on Debian "Sarge" after making the changes below:

Red Hat Enterprise Linux 4

On a 64 bit RHEL AS 4 machine I had problems building ROOT 4.04/02g. The problem was with /usr/X11R6/lib64/libXpm.a while building the libGX11.so library. I got around this by not requiring a static build of ROOT. That is, in configuring ROOT I used the --enable-shared option.

Building GDS 2.11.1 itself went fine after removing the bad monitors mentioned above.

Fedora Core 4 (FC4)

All of the items for Fedora Core 3 (below) apply. The version of glibc in FC4 is 2.3. In addition, for building GDS 2.10.8 I found:

Fedora Core 3

Fedora Core 3 has gcc 3.3.4 and glibc 2.3.6. To build GDS 2.10.8 I had to do the following:

Red Hat 9

See John Zweizig's note on Building GDS 2.8 on Red Hat 9

References and Links:

  1. ROOT from CERN
  2. GDS Home page
  3. Data Monitoring Tools (DMT) project
  4. LSC Data Analysis Software Working Groups - note the links under "Projects"

  6. Building GDS 2.8 on Red Hat 9
  7. LIGOtools and GDS DMT on Debian GNU/Linux - this is about installing GDS via LIGOtools, not building from source.
Many thanks to John Zweizig for his patience in answering so many questions from me.

-- Eric Myers

  Copyright © 2009 by Spy Hill Research http://www.Spy-Hill.net/myers/help/ligo/GDS-build.html (served by Islay.spy-hill.com) Last modified: 06 September 2009