SPY HILL Research
Spy-Hill.net
Poughkeepsie, New York [DIR] [UP]

Adding and Updating BOINC applications

This page describes the steps required to add a new application to your BOINC project, and the steps necessary to update the application in order to release a newer version. It also discusses some problems you might encounter if you have a large number of application versions.
Last modified: 15 July 2010
Topics
Adding an Application
Updating an Application
Managment Scripts
Troubleshooting
Most BOINC projects have only one application, which does the main work of the project, though it is possible to have several different applications per project. For example, one application might generate a set of initial configurations for a simulation, and the other would then compute the simulation results from those initial configurations. Even if your project has only one primary application, you will likely need to create some test applications as you learn to use BOINC, and you will need to know how to update an application (release a new version) when the application code has been modified.

There are two separate tasks here:

  1. Creating (adding) an application on your BOINC project - this adds the application to the database for your project (but not the particular version of the application).

  2. Updating an existing application (releasing a new version). - this registers the new version in the project database, and it copies the files for the new release from the apps/ directory. The executable and any additional files associated with the application version are signed and added to the download area.
You only "add" the application once, but you "update" or "release" each time you revise the code for your application and want to distribute the newer version to all the clients (including the first time you add the application). You also need to "update" the application if you want to add a version for a new platform (Windows, Mac, or Linux). You then need to update new versions separately for each platform.

In the examples below we will "add" a new applications called "cube" to the Pirates@Home project, and then "update" to release new versions of both the "hello" and "cube" applications. Substitute your own project directory name for "pirates", and use the name of your particular application. The shell variable PROJECT is used here to indicate the top-level directory of your project. So for example for Pirates@Home I would use 

   %  setenv PROJECT /usr02/pirates
(Here and in what follows the "%" represents the Unix command prompt -- you don't type it. You do not need to be the 'root' user for any of this.)

You may find these pages from the BOINC documentation useful:

The information on those pages is fairly complete. I have just pulled it together into a single set of examples to make the process a little bit clearer.

If you are having strange problems where clients don't use the latest application version even after you have released a new version then check here. Your feeder may be "full".

Adding a new application to the Project

Adding an application to your project just puts information about the application into the database. It creates an entry in the "application" table. You will also create a directory from which new versions of the application are added to the project. After you add the application to the project you need to release the first version of your application for each platform you will support.

There are now two ways you can add an application. The original way was to edit the project.xml file and then run the xadd program. That still works. But there is now also a web form in the "ops" section of the web pages which allows you to change the settings on your project's applications and to add a new application.

Adding an application via ops/manage_apps.php

Go to the control panel (the "ops" section of the web pages) and follow the link to "Manage applications". At the bottom of the form you can add the name and description of the new application. The description is also sometimes refered to as the "user friendly name". Since it is usually shown instead of the simple name I find it useful to include the simple name in this field.

You can also use this form to "deprecate" an application which you no longer wish to use. It will then no longer show up in the listing of project applications. The form will also let you delete an application entirely. You can also use the form to enable homogeneous redundancy, which limits instances of Workunits (ie Results) to all run on the same platform.

Adding an application via xadd

This is the conventional way to add a new application, and until recently it was the only way to do it.
  1. Start from the project main directory: 
          %  cd $PROJECT

  2. Edit the file project.xml to add an item of the form: 
    	<app>
	    <name>cube</name>
	    <user_friendly_name>OpenGL cube demo</user_friendly_name>
	</app>

  3. Run the command 
           % bin/xadd
    and it will insert any new entries in the project.xml file into the database.

  4. Next, make a new directory under the project/apps directory, which is where new versions of the application will be taken from when you "release" a new version: 
          %  mkdir apps/cube
    Then whenever you want to release a new version of your application, including the first time, you put a copy of it in this directory and then run the update_versions script, as described below. This is just a staging area. The update_versions script will copy the application file(s) from here to the download area. So you are free to clean up this directory after you have run that script.

Updating an application - releasing a new version

When you have made changes to your application and it compiles cleanly and runs correctly in "standalone" mode then you are ready to release a new version for your project by updating the application version. This assumes that you have previously "added" the application to the project, as described above.

The full name of the application version has in it both the short name of the application and the name of the platform for which it is built. Some examples are given below.

To release a new version of the code you need to put the executable (and any files that might go with it) in a particular directory so that BOINC's update_versions script can find them. For a simple application which consists of a single executable you just put that executable under its full name in a directory under apps which has the short name of the application, eg. apps/cube. But more complex applications may consist of several files, such as the executable, image files for the screensaver, or data files common to all workunits. To release a version of these multi-file applications you create a directory with the full name of the application and then put all the files in that directory.

The examples below should make this process clearer. We will update (release) a new version of the simple "hello" application for the Windows platform, and then release a multi-file version of the "cube" application for Linux.

Simple single-file application:

  1. Copy the executable to the appropriate apps subdirectory using the full platform name. For the "hello, world" example program for Linux you would say something like 
            %   cp hello $PROJECT/apps/hello/hello_6.01_i686-pc-linux-gnu

  2. From the main project directory run the update_versions script and accept (commit) the change. Here is an example: 
           %  cd $PROJECT
       %  bin/update_versions
       Looking for <App#6 hello> versions in /usr02/pirates/apps/hello
         Found <App#6 hello> version 501 for <Platform#2 windows_intelx86>: hello_6.01_i686-pc-linux-gnu 
       Signing /usr02/pirates/apps/hello/hello_6.01_i686-pc-linux-gnu
       Copying hello_6.01_i686-pc-linux-gnu to /usr02/pirates/download/hello_6.01_i686-pc-linux-gnu
       Ready to commit 1 items:
	   <AppVersion#None hello 601 i686-pc-linux-gnu>
       Continue [Y/n]  y
       Committed:
	   <AppVersion#80 hello 601 i686-pc-linux-gnu>
       Done
    (Like most BOINC management commands, you cannot run this from any place, you must first cd to the main project directory.)

  3. Under normal operation the feeder re-reads the database once an hour (or more frequently when it needs to). The update_versions script will touch the "trigger" file reread_db, which should cause the feeder to automatically re-read the database immediately. If this should fail for some reason then you can stop and restart the project to force the change to register.

Multi-file applications

The steps for a multi-file application are almost the same, except that you put all the files in a directory which has the full application+platform name. As an example, the graphics demo program "cube" for Unix consists of both an executable file and a shared library which contains the graphics code. The shared library file has an extra suffix ".so". So
    Create a directory under the apps subdirectory having the full name of the application version: 
           %  cd $PROJECT
       %  mkdir apps/cube/cube_5.01_i686-pc-linux-gnu
       %  mv cube_5.01_i686-pc-linux-gnu  apps/cube/cube_5.01_i686-pc-linux-gnu/
       %  mv cube_5.01_i686-pc-linux-gnu.so  apps/cube/cube_5.01_i686-pc-linux-gnu/

  1. From the main project directory run the update_versions script and accept the change. Example: 
           %  cd $PROJECT
       %  bin/update_versions
       Looking for <App#11 cube> versions in /usr02/pirates/apps/cube
	 Found <App#11 cube> version 501 for <Platform#1 i686-pc-linux-gnu>: cube_5.01_i686-pc-linux-gnu (+ 1 bundled file(s))
       Signing /usr02/pirates/apps/cube/cube_5.01_i686-pc-linux-gnu/cube_5.01_i686-pc-linux-gnu
       Copying cube_5.01_i686-pc-linux-gnu to /usr02/pirates/download/cube_5.01_i686-pc-linux-gnu
       Copying cube_5.01_i686-pc-linux-gnu.so to /usr02/pirates/download/cube_5.01_i686-pc-linux-gnu.so
       Ready to commit 1 items:
	   <AppVersion#None cube 501 windows_intelx86>
       Continue [Y/n]  y
       Committed:
	   <AppVersion#80 cube 501 windows_intelx86>
       Done

  2. Stop and restart the project, so that the feeder can get the updated app version info, as described above.
Once you have run update_versions you can delete the files from the apps subdirectory. It is just a staging area so that the script can easily find all the files. But this is not the only thing that may need to be cleaned up...

Management Scripts

Two new pages have recently been added to the "control panel" part of the project web pages (the "ops" pages): project (the control panel).

Troubleshooting

If you are releasing many new versions of an application, as is often the case while developing a new project, or if you just have many applications for several platforms, then you need to be aware that there is a limit to the size of the app_version table in the database. Actually, the table in the database can be any size, but the feeder can only handle so many entries before it fills the area in the shared memory it uses to communicate with the scheduler. A symptom of this problem will be that clients will continue to use the old version of the application even after you have followed the steps above. You can confirm the cause of the problem by inspecting the feeder log file, which will show something like: 
  [2005/05/10 20:44:33] Executing command: feeder
  2005-05-10 20:44:34 [CRITICAL] The SCHED_SHMEM structure is too small for the app_versions table.
  Either increase the MAX_APP_VERSIONS parameter in sched_shmem.h and recompile,
  or prune old rows from the table.
  Then restart the project.
This problem is more likely to occur if you are supporting many different platforms (eg. x86 and x86-64 versions, or both Intel and PPC Mac).

The easiest way to recover is simply to use the page ops/manage_app_versions.php (in the control panel) to mark unused versions of applications as deprecated, or to delete them entirely.

  Copyright © 2010 by Spy Hill Research http://www.Spy-Hill.net/help/boinc/app_add.html (served by Islay.spy-hill.com) Last modified: 15 July 2010