Resgrp:comp-photo-new gdv layout

Setting up

Gaussian insists on having its executables not readable by world. The best way to achieve this is to have a properly set umask; this is a good idea anyway, as having things readable (or writable!) by others is rarely a good idea. If you need to allow others to view things you can set the permissions explicitly. One problem here is that the Gaussian makefiles use the csh, which no sane person uses as a shell. It's therefore possible that your umask is not being set for the c-shell, which results in world readable files by default. Fix this permanently by inserting this line into both ~/.bashrc and ~/.cshrc (you may have to create the latter):

 umask 077

The new Gaussian development version layout

When you first check out a version of gdv from the git repository you will notice that the code is laid out in a different format. Most of the large monolithic files have been broken down into subdirectories with their constituent subroutines split out into individual files. For example, l510.F has been replaced by a l510 directory containing each l510 subroutine in its own file. Most of the utility files have been processed similarly. This layout is primarily to help the functioning of the version control system. With the old system a modification to a subroutine within l510.F would show up only as a change to the file l510.F, with only line numbers to help you guess what subroutine had actually been altered. Moving a subroutine from a link file to a utility file would only show up as a deletion of a block of lines from one and an addition of a block of lines to the other. The new system allows us to immediately see which subroutines have been altered and to track them if they are moved or copied. When you check out the repository there are currently three scripts next to the gdv subdirectory. The new-to-old.sh and old-to-new.sh scripts convert between the two formats. See Importing your own code for details on old-to-new.sh. These will be useful when receiving new tarballs from Gaussian, or converting already written code, or when the time comes to send code back to Gaussian. The new-to-old.sh script attempts to get a distribution very close to an old form Gaussian layout, but there will be differences. Some old-style files are in an arbitrary order which the script does not attempt to recreate. Fortunately these are not files that we are likely to change. The third script, pillage.sh, takes executables, libraries, and object files from an already compiled version of Gaussian. It shouldn't matter whether the previously compiled version is new style or old-style.

Building from scratch

By default you'll want to build from scratch. This is actually quite quick. Assuming you have checked out the version you wish to compile, and have loaded the appropriate compiler and Linda modules, you should cd to the gdv directory. Do not run bldgdv, as this will start the normal Gaussian build system, which will fail.

You will need to set the symbolic link for bsd/gdv.make to point to the appropriate makefile yourself. This needs to be one of the modififed makefiles, which are prefixed with 'uber'. The easiest option is just to use uber-i386.make which uses the same options that i386.make does. The other 'uber-*.make' files are tailored to our particular systems (and therefore we may house them elsewhere in the future), such as the Sandybridge nodes or the Altix. If you want to modify an existing makefile see the comments here. Choose your 'uber' makefile (I'll use the uber-i386.make file here) and type:

 gdv$ ln -s uber-i386.make bsd/gdv.make
 gdv$

Once there is an uber makefile now set you are ready to use the new build system. To build the entirety of Gaussian type:

gdv$ make build-tools && make -j 6 util.a && make util.a && make -j 6 exe && make exe 2>&1 | tee make.log
[... lots of output ]
gdv$

The number 6 in this example instructs make to run six parallel threads while building. You can use different numbers depending on how many processors the system you're building on has, how loaded it is, and so on. I find that building in the /tmp directory with a relatively unloaded cx1 login node, I can build Gaussian in about 15 minutes. You'll notice that after each parallel invocation of make I have a serial invocation at the same time for the same target. This is because sometimes the parallel make loses track of where it is and doesn't fully make a target. The serial invocation should always correct this. Once Gaussian is built (and you can always check by seeing that make exe returns Nothing to be done for: `exe') you can also build a Linda version by typing make linda. You will need the 8.2 version of the Linda compiler in your path; I suggest this is made into a module.

Using `pillage.sh`

cd into the gdv subdirectory of your freshly checked out repository, type:

gdv$ ../pillage.sh /home/gaussian-devel/gaussiandvh13_pgi_118/gdv

(for example) wait a while and when the prompt returns you should have a fully populated and up-to-date set of binaries. pillage.sh touches all the imported libraries and executables so they are newer than the source code. This means that if you type:

gdv$ make
ln -sf browse arc
gdv$

only some trivial things get made.

It's up to you to make sure that any development you do uses a consistent compiler, Linda compiler, etc, to the pillaged binaries.

Doing it yourself

I did try to make a version of pillage.sh that linked to executables instead of copying them, and you may feel tempted to try the same. The main problem that you need to overcome is that if the libraries you are linking to are older than their corresponding .F files make will try to update them. If the libraries are not yours this will simply fail and cause make to exit. However, if the symbolically linked libraries are yours they will be updated, which is probably not what you want! If you really need to save space there are some easier things you can do. Starting from an empty repository, run bldgdv and make the build tools as above. Then copy an appropriate util.a from somewhere to your gdv directory (do not use cp -p). Finally, use make to build only the links you are planning to alter, e.g. make l510.exe. You should also alter the Makefile so that the mystuff target lists only these links. This saves space by not having any of the other links' libraries or executables. You will also not have any .o files for the util subroutines; as long as the util.a file is newer than all of the util .F files make will not try to make them. If, however, even one util .F file becomes newer than util.a make will make all of the util .o files! Now you can use the %subst keyword to Gaussian to tell it to pick up the appropriate links from this directory.

Developing

However you have built the binaries developing in the new system is very simple. Edit the files you need to change, type make, or make Linda from the gdv directory. You can also type make inside a link subdirectory, which will build just that link's executable. The makefile knows about Gaussian's dependencies, so for example if you change a subroutine in l510 then both l510.exe and l1003.exe will be re-linked (because l1003 depends on l510.a)

By default the makefile checks everything to see if it's up-to-date. As there are nearly 12,000 files in this layout this can take a few seconds, longer if the filesystem is being stressed. I find that with a hot cache on /tmp make takes less than a second to check everything. With a cold cache on /home it can take some tens of seconds. If you want to get make to only check specific executables edit the Makefile in gdv, uncomment the "mystuff" line, and change it appropriately. Note, however, that all the util files are always checked. The makefile is a tracked file in the repository, so please make sure you don't check in these trivial changes.

You do not need to edit the makefile if you add, move, or delete subroutines.

Importing your own code

If your code is in the old layout, i.e. it's all monolithic links and utilities just as in the official Gaussian distribution tarball, then use the old-to-new.sh script. First, check out the version of the development code that was the basis of your code. Let's assume your code is based on the H10 code and has not merged in changes for H11, H12P, etc.

The old-to-new.sh script assumes that you've got a gau-fsplit command in your $PATH. If you haven't, you can create one by building the build-tools, as explained above, and making sure this directory appears, however temporarily, in your $PATH. Or, you can point your path to the pre-built binaries in /home/gaussian-devel.

Now change directory to be above the gdv directory of your own code. Run the old-to-new.sh script, with an argument of the directory you want the new style to appear in (it should not exist):

tmp$ cd /path/to/my-code
my-code$ ls
gdv
my-code$ PATH=$PATH:/home/gaussian-devel/gaussiandvh10_pgi_105/gdv /tmp/testrepo/old-to-new.sh new-style
[...some output...]
my-code$ ls
gdv   new-style

Now remove the gdv directory in the newly checked out working directory and replace it with the one from new-style.

my-code$ cd /tmp/testrepo
testrepo$ rm -fr gdv
testrepo$ cp -pr /path/to/mycode/new-style/gdv .

Now you're ready to use commands like git status to see what you've changed compared to H10, git commit -a to check in your changes, git merge h13 to merge in changes up to version H13 (for example).

If you don't have the entire Gaussian source code in your directory you can get old-to-new.sh to work by making sure there's a (possibly empty) file in gdv/bsd/mdl1.F. There will be a few warnings but it will split out any files it can find in the gdv directory.

If, on the other hand, your code layout is just a few subroutines already split out, then you should just copy them to the appropriate place in the working directory.

This might seem like a pain but you'll only have to do it once for each version of the code you have!

The makefile

In modifying the Gassian build system I have attempted to balance two competing needs. The first is a strong desire to modify as little as possible of Gaussian's original makefiles. This is so that when we receive newer versions of Gaussian, with their inevitable changes to the makefiles, our modifications can be trivially merged in. The second is to have a build system that works nicely with the new file layout, which is necessary for the version control system. The compromise that I have chosen is to have a new Makefile in the gdv directory that understand such things as how to build libraries from the split util and link files, and which executables depend on which libraries, while relying on the existing Gaussian makefiles for details such as compiler invocation and flags.

Sadly it was not possible to use the existing Gaussian makefile without modifying it slightly. This modified version of bsd/i386.make is called bsd/uber-i386.make. In it the rules to make most of the libraries and executables are commented out, and a few fixes for other problems are added. If you wish to compile on another platform you will need to make an uber makefile for that platform. Base this off the differences between bsd/i386.make and bsd/uber-i386.make.

If you want to add a new link you will need to add it to the Makefile, and possibly to the uber-* makefile.

Tools and tricks

If you aren't already using these tools, you should be.

Screen

Screen is a terminal multiplexer. If you wish to have multiple terminals open on a remote system you're probably accustomed to opening multiple SSH windows from your local system. An alternative is to use screen. It allows you to have multiple windows, each with its own history and possibly executing commands, and switch between them with some simple keypresses. It also preserves the state of these windows when your connection to screen drops, allowing you to reconnect later.

It's a bit beyond the scope of this page, if you want to learn more check out the various web resources or it, such as http://www.rackaid.com/resources/linux-screen-tutorial-and-how-to/.