Instructions on how to set up a new mirror for VALD3

Setting up and maintaining mirrors will be easier in VALD3. The setup should be such that the mirror servers will keep themselves updated. This is possible now the basic file structure as well as the source files for the different components are stored in a subversion repository. Once the mirror is prepared, any changes commited to the repository will be automatically be propagated to all mirrors.

Still, setting up a new mirror will require some local tweaking. Here is a step-by-step guide:

Requirements

1. Compilers for C and Fortran (at least up to Fortran90)

2. A subversion and rsync client

3. A web server that understands and executes PHP scripts (if you want to offer the web service)

Preparations

1. Find a place to install the mirror. A mirror will require a quite a lot of disk space to store all the linelists (about 50 GB, Sep. 2015).
2. If you want to run the web service, you have to make sure that your choose VALD3 to reside on a disk that is visible from your web server. If the web server runs on the same machine as where you will install VALD3 this will not be an issue.
3. Each user and mirror needs their own unique username/password combination to access the central VALD3 svn repository. If you do not have these please ask the VALD administrator for the correct credentials.

Checking out a VALD3 mirror from the repository

1. Go to the directory that you want to use as the VALD root. This normally is the home directory of your vald account, but any other place will work just fine.
2. Using the following command, you can now check-out your local copy of the VALD3 software. This will set up the necessary directory structure and files :

   svn checkout --username [username] svn://vald.astro.uu.se/VALD3

   Note that you will be asked to provide the username/password combination for the mirror (or user). Once you provided these credentials (and saved them to disk), you will not need to enter these again when working with the repository.

3. This will give you one subdirectory VALD3 that contains the entire tree. If you really want, you can move all the subdirectories up one level (including VALD3/.svn), and then remove the (now empty) VALD3 directory.

Configuring the mirror

1. If you want to have run the web service, create a symbolic link that points to the WWW/public_html directory. If you install VALD under its own user account, you want to do the following. Be aware that some web servers use non-standard names for individual WWW directories. Don't forget to check file and folder permissions.

   ln -s WWW/public_html ~

2. There are few files that require local configuration. Start with :

   cp CONFIG/valdems_local.h.template CONFIG/valdems_local.h

   Then edit CONFIG/valdems_local.h to enter your site's name, the location of the VALD3 root directory, the directory to use to store FTP requests, the URL to your FTP directory and the local path of the sendmail program.

3. Then, in a similar fashion :

   cp CONFIG/Makefile_local.inc.template CONFIG/Makefile_local.inc

   Edit CONFIG/Makefile_local.inc and choose appropriate compilers for the C and Fortran sources. There are some examples at the end of CONFIG/Makefile_local.inc.

4. Then again, take similar steps with :

   cp CONFIG/local_mirror_config.sh.template CONFIG/local_mirror_config.sh

   Edit CONFIG/local_mirror_config.sh and enter the location of the sendmail executable, as well as where the VALD mail spool file is located. Note that VALD will automatically remove the mail spool file when it is running the email-system (EMS). (Don't play around with your own inbox!) If you only want to test VALD, you can of course point to a dummy mailbox file.

5. Finally, to configure the web service, do :

   cp WWW/config/site_config_local.php.template WWW/config/site_config_local.php

   Edit WWW/config/site_config_local.php to insert your site name, the location where the vald.php script will be found your web server, the full URL of your server's root and the full path to the VALD3 root directory. Here you can also specify to which email address any requests are to be sent. Normally, this is your mirror's VALD account, but it can also be another server. (For example, if you have problems compiling your own binaries).

If you are installing VALD in a custom directory

The scripts and Makefiles assume that the VALD3 root is the same as the user's home directory. However, if you installed VALD in another (custom) directory, you will need to set the environment variable VALD_HOME to point to the appropriate directory. This variable needs to be set whenever you want to compile, run or synchronize the software. It may be tempting to include this in SOURCE/Makefile.inc, but please don't do this since it may propagate to the software repository. Why not add this definition to your .csrc file?

Compiling and installing the software

You are now ready to try to compile and install the software :

cd SOURCE
make
make install

If all works fine, you will now have a set of executables in the bin subdirectory. If you get a complaint that the Makefile_local.inc cannot be found, you have probably installed in a non-standard directory (see previous point) and you need to set the VALD_HOME environment parameter. If you encounter problems during compilation, you might need to review your choice of compiler in the CONFIG/Makefile_local.inc file.

Performing a first update

With the executables in place, you can start to download all the necessary line data and model atmospheres from the VALD3 master server. Note that this is a lot of data (currently more than 25 GB), so the download may take a while. Use the server update script.

cd SYNCHRO
bash update_this_mirror.sh

This script will also update the software sources, rebuild the binaries and send info from your mirror to the master server (statistics etc). It does no longer assume that the VALD3 root is the same as the user's home directory.

Checking functionality

1. Testing the EMS service :
   1. Insert a test request mail in your mail spool file. Make sure that your address is in the 'From:' field!

   2. Go to the EMS directory and execute the service-ems.sh script. This will process the email request, send the result to the sender address. This script will also schedule itself for execution after a delay of 20 minutes. If you don't want this, remove this at-job with at -r [job number]. You can find the job number with at -l. (If you have no at-daemon running, the operating system will complain. If this occurs, install and start this service on your machine)

2. Testing the web server :

   1. If everything is properly configured, you should be able to simply access the web server from the appropriate URL, which normally is something like: http://yourserver.org/~vald/php/vald.php. If this does not work out of the box, you may have problems with (1) PHP interpretation by the server, or (2) incorrect configuration. A good way to test your script is by executing php from the command line with php WWW/public_html/php/vald.php. This will reveal any error that might have occurred.

3. Testing synchronization :

   1. Running bash update_this_server.sh in the CONFIG directory will perform an update of your server. If you add the command-line option -m youremail@site.org, you will be informed by email if the synchronization failed.

Setting up automatic sychronization

If you have reached this point, you are ready to setup automatic synchronization of your site. This is done by setting up a cron-job that calls bash update_this_server.sh at regular intervals, typically once a night. If you use the command-line option -m youremail@site.org, you will know immediately when something is wrong with your site.

Notes: Currently (2012-02-29) statistics are not yet shared through svn.