Hudson Installation Procedures

Overview

Hudson is a Continuous Integration (CI) server which polls a version control repository and runs an automated build soon after a change is detected in the version control repository. Hudson organizes CI jobs into projects. Each project has its own build script and its build results are stored within Hudson's work area.

This page lists the steps necessary to configure and operate Hudson at CBIIT. The steps can be summarized as

• Prepare your project to use BDA's Hudson management scripts
• Create a CI user account on the Linux machine assigned as the Hudson host
• Configure CI user's environment
• Obtain the BDA Hudson configuration scripts

Prepare Your Project for Hudson CI

These scripts will help you install and configure Hudson automatically.

1. Obtain the BDA Hudson configuration (hudson-manager) scripts by checking out the following two files from BDA's SVN to your project's private workspace, by typing the following:
   

   svn co http://gforge.nci.nih.gov/svnroot/automation/trunk/software/hudson-manager/
   

   There are two files of note: Build File - The Hudson management build script. Installs and configures tomcat/hudson, copies a Hudson installation config to SVN and copies Hudson configuration from SVN to a Hudson installation Properties File - The Hudson management properties file

2. Add the two files to your project's SVN repository (For example, PROJECT_ROOT/tools/hudson-manager)
3. Add a config directory under the location where you stored the two files above (For example, PROJECT_ROOT/tools/hudson-manager/config). This will be used to store configurations once defined in the Hudson CI machine
   

   Remove the .svn directory from your PROJECT_ROOT/tools/hudson-manager private workspace prior to committing Hudson files to SVN.

4. Commit all new files/directories to your project's SVN repository.

Install Hudson

The following instructions assume a (nearly) blank Linux environment with only Ant, Java SDK and perhaps a database installed.

Create CI User

1. Login with your user credentials, as supplied by the CBIIT Systems Team
   

   Your user must have sudo rights in the CI box

2. Type
   

   sudo bash
   

   to obtain temporary superuser rights

3. In this step you'll create a CI user. Since Hudson will need large disk space, make sure to set user's home to a partition with enough free space. Type:
   

   df -h
   

   Select a suitable folder for the CI user's home. From the df output below, it is apparent that /usr/local will be a good home candidate for the CI user.

   -bash-3.00$ df -h
   Filesystem            Size  Used Avail Use% Mounted on
   /dev/mapper/roog-rvol 7.9G  3.8G  3.8G  51% /
   /dev/sda1              99M   23M   71M  25% /boot
   none                  1.9G     0  1.9G   0% /dev/shm
   /dev/mapper/roog-lvol  16G  247M   15G   2% /local
   /dev/mapper/roog-cvol  20G  4.3G   15G  24% /usr/local
   /dev/mapper/roog-rvol 4.0G  148M  3.6G   4% /var
   cbionfsb:/nfsb/admin  185G  176G  9.2G  96% /admfs
   cbionfsc:/nfsc/admrch  50G   30G   20G  61% /admarch
   /nfsa/home/duvallp    785G  744G   41G  95% /h1/duvallp
   /nfsa/home/duvallp    785G  744G   41G  95% /h1/duvallp
   

   1. To create the user, type:
      

      # login as your self
      sudo bash
      mkdir /local/home
      /usr/sbin/useradd -d /local/home/hudsonuser hudsonuser
      

4. To change the CI user's password, type:
   

   passwd hudsonuser
   

   and enter a password.

Configure CI User's Environment

1. Login as the CI user (the Shell account you created above (e.g. hudsonuser)
2. Add the following code to the .bash_profile file, by typing:
   

   vi .bash_profile
   

   Validate that folders listed below reflect what's in your system. If not, apply changes as necessary.

   ANT_HOME=/usr/local/ant-1.7.0
   JAVA_HOME=/usr/jdk1.5.0_10/
   GLOBUS_LOCATION=/usr/local/ws-core-4.0.3
   MAVEN_HOME=/usr/local/apache-maven-2.0.8
   HUDSON_HOME=$HOME/hudson_data
   export ANT_HOME JAVA_HOME GLOBUS_LOCATION MAVEN_HOME HUDSON_HOME
   export PATH=$ANT_HOME/bin:$JAVA_HOME/bin:$MAVEN_HOME/bin:$PATH
   # From the command line, set this to allow FireFox to run on xframebuffer for Selenium tests.
   export DISPLAY=:1.0
   

3. Source the profile to setup the environment, by typing the following from the command line.
   

   $ . .bash_profile
   

4. Log off and login. Validate the setup by typing java -version and ant -version. Both commands should return meaningful results.

Checkout Project's Hudson Management Scripts into the CI Machine

1. Login to the Hudson machine as the CI user you created above (i.e. hudsonuser)
2. cd
3. mkdir hudson
4. cd hudson
5. mkdir src
6. cd src
7. svn co http://gforge.nci.nih.gov/svnroot/*_yourproject_*/trunk/tools/hudson-manager
   

   This is a sample output generated by the svn co command [hudsonuser@xxxxxx-xxxxx src]$ svn co http://gforge.nci.nih.gov/svnroot/automation/trunk/software/hudson-manager A hudson-manager/install.properties A hudson-manager/bash_profile A hudson-manager/build.xml Checked out revision 653.

Review the Hudson Installation Properties

1. Login to the Hudson machine as the CI user you created above (i.e. hudsonuser)
2. cd
3. cd hudson/src/hudson-manager
4. Edit install.properties (original file) as necessary
   

   The following properties from install.properties may require your special attention application.base.path=${user.home}/hudson/application hudson.dir=${user.home}/hudson/data backup.dir=${user.home}/hudson/backup

   A complete list of properties is provided in the table below.

   Property	Description	Value
   application.base.path	Directory where Tomcat is extracted to	${user.home}/hudson/application
   binary.repository.url	Base Url for downloading Tomcat	http://gforge.nci.nih.gov/svnroot/commonlibrary/trunk
   tomcat.binaries.file	Tomcat file Name	jakarta-tomcat-5.0.28.zip
   tomcat.src.url	Absolute path for download file	${binary.repository.url}/techstack-2006/os-independent/${tomcat.binaries.file}
   tomcat.binaries.relative.dir	Root directory of extracted Tomcat	jakarta-tomcat-5.0.28
   tomcat.home	Tomcat Home Dir	${application.base.path}/${tomcat.binaries.relative.dir}
   tomcat.port.http	Tomcat HTTP Port	48080
   tomcat.port.ssl	Tomcat SSL port	48443
   tomcat.port.ajp	Tomcat AJP port	48009
   tomcat.port.shutdown	Tomcat Shutdown port ?	48005
   hudson.base.url	Base URL for Hudson	${binary.repository.url}/other/os-independent/hudson
   hudson.plugin.list	List of Plug-ins to download	build-timeout.hpi,checkstyle.hpi,cobertura.hpi,findbugs.hpi,hudsontracplugin.hpi,jabber.hpi,pmd.hpi,scp.hpi,svncompat13.hpi,svn-tag.hpi,violations.hpi,vmware.hpi,xvnc.hpi,findbugs.hpi
   hudson.file.name	Hudson War File	hudson.war
   hudson.dir	Hudson home dir	${user.home}/hudson/data
   backup.dir	Directory where backups will be backed up to and restored from	${user.home}/hudson/backup

Run the Hudson Installation Script

1. Login to the Hudson machine as the CI user
2. cd
3. cd hudson/src/hudson-manager
4. Type ant install
   

   If everything goes well the Ant script will download, configure and launch Hudson. You should see the following output:

   install:tomcat:configure:
   
   install:tomcat:
   
   
   
   retrieve:hudson:
        [echo] Checking for hudson hudson.war
         [get] Getting: http://gforge.nci.nih.gov/svnroot/commonlibrary/trunk/other/os-independent/hudson/hudson.war
         [get] To: /usr/local/hudsonuser/hudson/src/hudson-manager/target/working/hudson.war
        [echo] Checking for plugin checkstyle.hpi
         [get] Getting: http://gforge.nci.nih.gov/svnroot/commonlibrary/trunk/other/os-independent/hudson/checkstyle.hpi
         [get] To: /usr/local/hudsonuser/hudson/src/hudson-manager/target/working/checkstyle.hpi
        [echo] Checking for plugin findbugs.hpi
         [get] Getting: http://gforge.nci.nih.gov/svnroot/commonlibrary/trunk/other/os-independent/hudson/findbugs.hpi
         [get] To: /usr/local/hudsonuser/hudson/src/hudson-manager/target/working/findbugs.hpi
        [echo] Checking for plugin pmd.hpi
         [get] Getting: http://gforge.nci.nih.gov/svnroot/commonlibrary/trunk/other/os-independent/hudson/pmd.hpi
         [get] To: /usr/local/hudsonuser/hudson/src/hudson-manager/target/working/pmd.hpi
        [echo] Checking for plugin scp.hpi
         [get] Getting: http://gforge.nci.nih.gov/svnroot/commonlibrary/trunk/other/os-independent/hudson/scp.hpi
         [get] To: /usr/local/hudsonuser/hudson/src/hudson-manager/target/working/scp.hpi
        [echo] Checking for plugin svncompat13.hpi
         [get] Getting: http://gforge.nci.nih.gov/svnroot/commonlibrary/trunk/other/os-independent/hudson/svncompat13.hpi
         [get] To: /usr/local/hudsonuser/hudson/src/hudson-manager/target/working/svncompat13.hpi
        [echo] Checking for plugin svn-tag.hpi
         [get] Getting: http://gforge.nci.nih.gov/svnroot/commonlibrary/trunk/other/os-independent/hudson/svn-tag.hpi
         [get] To: /usr/local/hudsonuser/hudson/src/hudson-manager/target/working/svn-tag.hpi
        [echo] Checking for plugin violations.hpi
         [get] Getting: http://gforge.nci.nih.gov/svnroot/commonlibrary/trunk/other/os-independent/hudson/violations.hpi
         [get] To: /usr/local/hudsonuser/hudson/src/hudson-manager/target/working/violations.hpi
   
   install:hudson:binaries:
        [copy] Copying 1 file to /usr/local/hudsonuser/hudson/application/jakarta-tomcat-5.0.28/webapps
       [mkdir] Created dir: /usr/local/hudsonuser/hudson/data/plugins
        [copy] Copying 7 files to /usr/local/hudsonuser/hudson/data/plugins
   
   install:hudson:configure:
   
   install:hudson:
   
   start:tomcat:
        [echo] Starting Tomcat instance at /usr/local/hudsonuser/hudson/application/jakarta-tomcat-5.0.28
   
   install:
        [echo] You can access it at http://localhost:48080/hudson
   
   BUILD SUCCESSFUL
   Total time: 45 seconds
   [hudsonuser@xxxxxx-xxxxx hudson-manager]$
   

5. Verify the installation by opening a browser and navigating to http://YOUR_HUDSON_HOSTNAME:48080/hudson. You should see the Hudson dashboard with this message: Welcome to Hudson! Please create new jobs to get started.

Setting Hudson to start at boot

• Log into server as your user account
• Switch to root
  

  sudo bash
  

• cd to start about boot directory
  

  cd /etc/init.d
  

• create a file called tomcat with the following contents (vi or pico or emacs?)
  

  #!/bin/bash
  #
  # tomcat
  #
  # chkconfig: - 71 43
  # description:  Start up the Tomcat servlet engine.
  
  # Source function library.
  . /etc/init.d/functions
  
  RETVAL=$?
  CATALINA_HOME="/local/home/hudsonuser/hudson/application/apache-tomcat-5.5.20"
  
  case "$1" in
   start)
          if [ -f $CATALINA_HOME/bin/startup.sh ];
            then
       echo $"Starting Tomcat"
              /bin/su - hudsonuser $CATALINA_HOME/bin/startup.sh
          fi
   ;;
   stop)
          if [ -f $CATALINA_HOME/bin/shutdown.sh ];
            then
       echo $"Stopping Tomcat"
              /bin/su - hudsonuser $CATALINA_HOME/bin/shutdown.sh
          fi
    ;;
   *)
    echo $"Usage: $0 {start|stop}"
   exit 1
   ;;
  esac
  
  exit $RETVAL
  

• Change permisions on file
  

  chmod 750 tomcat
  

• Link your script (manually or using chkconfig) to the runlevel you want it to start at:
  

  • cd /etc/rc3.d
    ln -s ../init.d/tomcat S71tomcat
    

    or

    /sbin/chkconfig --add hudson
    /sbin/chkconfig hudson on
    

http://www.raibledesigns.com/tomcat/boot-howto.html

Create a New Hudson Job

1. Open a browser at http://YOUR_HUDSON_HOSTNAME:48080/hudson
2. Click New Job from the menu on left
3. Give a name to the new job (all lowercase, no spaces)
4. Select Build a free-style software project for Ant projects and Build a maven2 project for Maven2 projects
5. Click OK
6. In the following job configuration screen, locate the Source Code Management section and enter the project SCM properties, i.e. SVN URL, Local module, etc.
7. Similarly, modify other properties such as which artifacts to store
   

   The following is a non-exhaustive list of the job properties as they appear on the Hudson configuration page. Use this is a guide to edit your own project. Project Name Description Advanced Project Options Quiet Period - 60 (seconds) Source Code Management SVN Repository URL -http://gforge.nci.nih.gov/svnroot/_YourProject_/trunk Local module directory (optional) - projectname Build Triggers Poll SCM - Schedule * * * * * Build Periodically - Schedule 0 * * * * Build Invoke Ant Ant Version- Select version from pull down Targets - build:all static:analysis Email Notifications Recipients - yoru list Send email for every unstable build - checked Send separate emails to individuals that broke the build Publish PMD Analysis Results (checked) PMD Results - projectname/reports/pmd/*/.xml Publish CheckStyle Analysis Results (checked) PMD Results - projectname/reports/checkstyle/*/.xml Publish FindBugs Analysis Results (checked) FindBugs Results - projectname/reports/findbugs/*/.xml Report Violations (checked) Failure counts and how to set them Sunny Set this value to your target failure rate Set it to at least 1 above your current failure rate Failure counts higher than this but below stormy will show as cloudy on the dashboard This does not affect build status Stormy Set this value to your target failure rate Set it to at least 1 above your current failure rate Failure counts higher than this but below unstable will show as stormy on the dashboard This does not affect build status Unstable Set this value to the your target failure rate to break the build Set it to at least 1 above your current failure rate Failures greater than this will cause the build status show unstable (which indicates a failed build) Xml patterns checkstyle - projectname/reports/checkstyle/*/.xml findbugs - projectname/reports/findbugs/*/.xml pmd - projectname/reports/pmd/*/.xml simian - projectname/reports/simian/*/.xml

8. Click Save

Setup database

Lower Case Table names

1. ssh into the machine as your account
2. sudo bash # provide password
3. /etc/init.d/mysql.server stop # stop db
4. edit /etc/my.cnf - add 'lower-case-table-names=1' under '[mysqld]' section
5. /etc/init.d/mysql.server start # start db

Privileged user

When you receive the CI server the mysql user will not have a password set

1. Set a password on the mysql database user
   
   1. /usr/bin/mysqladmin -u mysql password 'yourpassword'
2. Verify the password
   
   1. mysql -u mysql -p # enter password from above when prompted

Configuring Hudson

• Launch browser to http://localhost:48080/hudson
• Manage Hudson System Configuration and set (as defined above)
  
  • ANT HOME - /usr/local/ant-1.7.0
  • JAVA HOME - /usr/jdk1.5.0_10/
  • Maven - /usr/local/apache-maven-2.0.8
  • CVS - copy from other user to /var/lib/hudson/.cvspass, change owner
  • Mail
    
    • SMTP server - mailfwd.nih.gov
    • Default user e-mail suffix- @mail.nih.gov
    • System Admin E-mail Adresss - buildmaster@cbv-ciweb-base.nci.nih.gov

SVN and Hudson configuration

• Create a hudson-manager area in your scm repository
• Copy Hudson configuration to svn - 'ant -Dsvn.user.name=svnusername -Dsvn.user.pass=svnpassword'
• Copy Hudson configuration from svn - 'ant -Dsvn.user.name=svnusername -Dsvn.user.pass=svnpassword deploy2hudson'

Hudson Management Files

Build File

• Build Script Targets
  
  • save-hudson-configs
    
    • init - checks that user name and password are passed in as arguments
    • copy2repo- copies files from Hudson to svn directory
    • svn-add - svn --force add files to svn (will add missing files)
    • svn-commit - commits the files to the repository
  • deploy2hudson - copies files from svn to hudson directory
  • install - wrapper target to install/configure tomcat/hudson based on properties form prop file below
    
    • install:tomcat - Wrapper target that installs and configures Tomcat
      
      • init
      • retrieve:tomcat - Retrieves Tomcat distribution from binary repository based on properties, if file is not present
      • install:tomcat:binaries - Extracts Tomcat distribution to destination directory
      • install:tomcat:configure - Configures the ports in the Tomcat installation
    • install:hudson - Wrapper target that installs and configures Hudson
      
      • init
      • retrieve:hudson - Retrieves Hudson war file and plugins (*.hpi) from binary repository
      • install:hudson:binaries - Deploys war and plugins to Tomcat and Hudson directories
      • install:hudson:configure - empty right now
    • start:tomcat - starts tomcat after installation
  • artifacts:backup - backups build artifacts and history
  • artifacts:restore - restores build artifacts from existing backup file

Usage

• Command line properties (-D_propname_=value)
  
  • svn.user.name - user to commit files to svn under
  • svn.user.pass - user's password
  • hudson.dir - directory to read or write Hudson configuration
• Usage
  

  ant -Dsvn.user.name=[svnusername] -Dsvn.user.pass=[svnpassword] [targetName]
  

Operating Hudson

Cleaning Files

• Finding Artifacts*
  
  Replace jar with whichever file type you're searching

find /builds/*/archive -type f  -name ".jar"

Reading Hudson RSS Feeds from Firefox

You can use Hudson's nifty RSS feeds to get notified of any failures or get updates of all builds. When you couple RSS feeds with Firefox's Live Bookmarks you don't even have to visit Hudson's page to get a quick status of the builds.

• Download Brief, an RSS reader plugin for Firefox, from https://addons.mozilla.org/en-US/firefox/addon/4578
• Add http://cbv-ciweb-base.nci.nih.gov:48080/hudson/job/_username_/rssAll or http://cbv-ciweb-base.nci.nih.gov:48080/hudson/job/_username_/rssFailed as Firefox Live Bookmarks
• Configure Brief to poll your Live Bookmarks.
• Get notified of new builds/failures by observing the lower-right corner of Firefox where Brief's RSS icon is located