ChemWiki - User contributions [en]

Resgrp:comp-photo-new gdv layout

2015-11-06T18:34:26Z

Scliffor: Removed mercurial commands, some minor edits

== Setting up ==

Gaussian insists on having its executables not readable by world. The best way to achieve this is to have a properly set umask; this is a good idea anyway, as having things readable (or writable!) by others is rarely a good idea. If you need to allow others to view things you can set the permissions explicitly. One problem here is that the Gaussian makefiles use the csh, which no sane person uses as a shell. It's therefore possible that your umask is not being set for the c-shell, which results in world readable files by default. Fix this permanently by inserting this line into ''both'' <code>~/.bashrc</code> and <code>~/.cshrc</code> (you may have to create the latter):
<code>
umask 077
</code>

== The new Gaussian development version layout ==

When you first check out a version of gdv from the git repository you will notice that the code is laid out in a different format. Most of the large monolithic files have been broken down into subdirectories with their constituent subroutines split out into individual files. For example, <code>l510.F</code> has been replaced by a <code>l510</code> directory containing each <code>l510</code> subroutine in its own file. Most of the utility files have been processed similarly. This layout is primarily to help the functioning of the version control system. With the old system a modification to a subroutine within <code>l510.F</code> would show up only as a change to the file <code>l510.F</code>, with only line numbers to help you guess what subroutine had actually been altered. Moving a subroutine from a link file to a utility file would only show up as a deletion of a block of lines from one and an addition of a block of lines to the other. The new system allows us to immediately see which subroutines have been altered and to track them if they are moved or copied.
When you check out the repository there are currently three scripts next to the <code>gdv</code> subdirectory. The <code>new-to-old.sh</code> and <code>old-to-new.sh</code> scripts convert between the two formats. See [[#Importing your own code|Importing your own code]] for details on <code>old-to-new.sh</code>. These will be useful when receiving new tarballs from Gaussian, or converting already written code, or when the time comes to send code back to Gaussian. The <code>new-to-old.sh</code> script attempts to get a distribution very close to an old form Gaussian layout, but there will be differences. Some old-style files are in an arbitrary order which the script does not attempt to recreate. Fortunately these are not files that we are likely to change.
The third script, <code>pillage.sh</code>, takes executables, libraries, and object files from an already compiled version of Gaussian. It shouldn't matter whether the previously compiled version is new style or old-style.

== Building from scratch ==

By default you'll want to build from scratch. This is actually quite quick. Assuming you have checked out the version you wish to compile, and have loaded the appropriate compiler and Linda modules, you should <code>cd</code> to the <code>gdv</code> directory. Do '''not''' run <code>bldgdv</code>, as this will start the normal Gaussian build system, which will fail.

You will need to set the symbolic link for <code>bsd/gdv.make</code> to point to the appropriate makefile yourself. This needs to be one of the modififed makefiles, which are prefixed with 'uber'. The easiest option is just to use <code>uber-i386.make</code> which uses the same options that <code>i386.make</code> does. The other 'uber-*.make' files are tailored to our particular systems (and therefore we may house them elsewhere in the future), such as the Sandybridge nodes or the Altix. If you want to modify an existing makefile see the comments [[#The makefile|here]].
Choose your 'uber' makefile (I'll use the <code>uber-i386.make</code> file here) and type:
<code>
gdv$ ln -s uber-i386.make bsd/gdv.make
gdv$
</code>

Once there is an uber makefile now set you are ready to use the new build system. To build the entirety of Gaussian type:
<code>
gdv$ make build-tools && make -j 6 util.a && make util.a && make -j 6 exe && make exe 2>&1 | tee make.log
[... lots of output ]
gdv$
</code>
The number 6 in this example instructs make to run six parallel threads while building. You can use different numbers depending on how many processors the system you're building on has, how loaded it is, and so on. I find that building in the <code>/tmp</code> directory with a relatively unloaded cx1 login node, I can build Gaussian in about 15 minutes. You'll notice that after each parallel invocation of make I have a serial invocation at the same time for the same target. This is because sometimes the parallel make loses track of where it is and doesn't fully make a target. The serial invocation should always correct this.
Once Gaussian is built (and you can always check by seeing that <code>make exe</code> returns <code>Nothing to be done for: `exe'</code>) you can also build a Linda version by typing <code>make linda</code>. You will need the 8.2 version of the Linda compiler in your path; I suggest this is made into a module.

== Using <code>pillage.sh</code> ==
<code>cd</code> into the <code>gdv</code> subdirectory of your freshly checked out repository, type:
<code>
gdv$ ../pillage.sh /home/gaussian-devel/gaussiandvh13_pgi_118/gdv
</code>
(for example) wait a while and when the prompt returns you should have a fully populated and up-to-date set of binaries. <code>pillage.sh</code> touches all the imported libraries and executables so they are newer than the source code. This means that if you type:
<code>
gdv$ make
ln -sf browse arc
gdv$
</code>
only some trivial things get made.

It's up to you to make sure that any development you do uses a consistent compiler, Linda compiler, etc, to the pillaged binaries.

== Doing it yourself ==

I did try to make a version of <code>pillage.sh</code> that linked to executables instead of copying them, and you may feel tempted to try the same. The main problem that you need to overcome is that if the libraries you are linking to are older than their corresponding <code>.F</code> files make will try to update them. If the libraries are not yours this will simply fail and cause make to exit. However, if the symbolically linked libraries are yours they will be updated, which is probably not what you want!
If you really need to save space there are some easier things you can do. Starting from an empty repository, run <code>bldgdv</code> and make the build tools as above. Then copy an appropriate util.a from somewhere to your <code>gdv</code> directory (do not use <code>cp -p</code>). Finally, use make to build only the links you are planning to alter, e.g. <code>make l510.exe</code>. You should also alter the Makefile so that the mystuff target lists only these links. This saves space by not having any of the other links' libraries or executables. You will also not have any <code>.o</code> files for the util subroutines; as long as the util.a file is newer than all of the util <code>.F</code> files make will not try to make them. If, however, even one util <code>.F</code> file becomes newer than util.a make will make '''all''' of the util <code>.o</code> files!
Now you can use the <code>%subst</code> keyword to Gaussian to tell it to pick up the appropriate links from this directory.

== Developing ==

However you have built the binaries developing in the new system is very simple. Edit the files you need to change, type <code>make</code>, or <code>make Linda</code> from the <code>gdv</code> directory. You can also type make inside a link subdirectory, which will build just that link's executable. The makefile knows about Gaussian's dependencies, so for example if you change a subroutine in <code>l510</code> then both <code>l510.exe</code> and <code>l1003.exe</code> will be re-linked (because <code>l1003</code> depends on <code>l510.a</code>)

By default the makefile checks everything to see if it's up-to-date. As there are nearly 12,000 files in this layout this can take a few seconds, longer if the filesystem is being stressed. I find that with a hot cache on <code>/tmp</code> make takes less than a second to check everything. With a cold cache on <code>/home</code> it can take some tens of seconds. If you want to get make to only check specific executables edit the Makefile in <code>gdv</code>, uncomment the "mystuff" line, and change it appropriately. Note, however, that all the util files are always checked. The makefile is a tracked file in the repository, so please make sure you don't check in these trivial changes.

You do not need to edit the makefile if you add, move, or delete subroutines.

== Importing your own code ==

If your code is in the old layout, i.e. it's all monolithic links and utilities just as in the official Gaussian distribution tarball, then use the <code>old-to-new.sh</code> script. First, check out the version of the development code that was the basis of your code. Let's assume your code is based on the H10 code and has not merged in changes for H11, H12P, etc.

The <code>old-to-new.sh</code> script assumes that you've got a <code>gau-fsplit</code> command in your <code>$PATH</code>. If you haven't, you can create one by building the build-tools, as explained above, and making sure this directory appears, however temporarily, in your <code>$PATH</code>. Or, you can point your path to the pre-built binaries in <code>/home/gaussian-devel</code>.

Now change directory to be above the <code>gdv</code> directory of your own code. Run the <code>old-to-new.sh</code> script, with an argument of the directory you want the new style to appear in (it should not exist):
<code>
tmp$ cd /path/to/my-code
my-code$ ls
gdv
my-code$ PATH=$PATH:/home/gaussian-devel/gaussiandvh10_pgi_105/gdv /tmp/testrepo/old-to-new.sh new-style
[...some output...]
my-code$ ls
gdv new-style
</code>
Now remove the gdv directory in the newly checked out working directory and replace it with the one from new-style.
<code>
my-code$ cd /tmp/testrepo
testrepo$ rm -fr gdv
testrepo$ cp -pr /path/to/mycode/new-style/gdv .
</code>
Now you're ready to use commands like <code>git status</code> to see what you've changed compared to H10, <code>git commit -a</code> to check in your changes, <code>git merge h13</code> to merge in changes up to version H13 (for example).

If you don't have the entire Gaussian source code in your directory you can get <code>old-to-new.sh</code> to work by making sure there's a (possibly empty) file in <code>gdv/bsd/mdl1.F</code>. There will be a few warnings but it will split out any files it can find in the <code>gdv</code> directory.

If, on the other hand, your code layout is just a few subroutines already split out, then you should just copy them to the appropriate place in the working directory.

This might seem like a pain but you'll only have to do it once for each version of the code you have!

== The makefile ==

In modifying the Gassian build system I have attempted to balance two competing needs. The first is a strong desire to modify as little as possible of Gaussian's original makefiles. This is so that when we receive newer versions of Gaussian, with their inevitable changes to the makefiles, our modifications can be trivially merged in. The second is to have a build system that works nicely with the new file layout, which is necessary for the version control system. The compromise that I have chosen is to have a new Makefile in the <code>gdv</code> directory that understand such things as how to build libraries from the split util and link files, and which executables depend on which libraries, while relying on the existing Gaussian makefiles for details such as compiler invocation and flags.

Sadly it was not possible to use the existing Gaussian makefile without modifying it slightly. This modified version of <code>bsd/i386.make</code> is called <code>bsd/uber-i386.make</code>. In it the rules to make most of the libraries and executables are commented out, and a few fixes for other problems are added. If you wish to compile on another platform you will need to make an uber makefile for that platform. Base this off the differences between <code>bsd/i386.make</code> and <code>bsd/uber-i386.make</code>.

If you want to add a new link you will need to add it to the Makefile, and possibly to the <code>uber-*</code> makefile.

== Tools and tricks ==

If you aren't already using these tools, you should be.

=== Tags ===

One complaint that might arise in going from monolithic link files to split files is that finding the definitions of subroutines and functions becomes much more difficult. So, if you are editing <code>l510.F</code> and are in the MCSCF deck you might come to the line that calls the Davids subroutine. Finding where this subroutine is defined used to be as simple as searching for 'e Davids' (assuming it's a subroutine, and that there is only one space between subroutine and Davids, and that it's in <code>l510.F</code>, etc). How are you to find it now?

The answer is to use '''tags'''. These have been around for a few decades and are extremely useful. Firstly you must create a tags file, or files. For vi/vim you do:
<code>
testrepo/gdv$ ctags *.F */*.F */*.c
</code>
(for Emacs substitute etags for ctags). If you want to create a tags file in one of the link subdirectories just use:
<code>
testrepo/gdv/l510$ ctags *.F ../*/*.F ../*/*.c
</code>
This will create a ctags (or etags) file in the current directory. Now, start vi (or Emacs) and edit <code>l510/mcscf.F</code>. Find the line that calls the Davids subroutine and position the cursor over the word Davids. Press <code>Ctrl-]</code> (or <code>M-.</code>) and you will be immediately taken to the definition of that subroutine, even if it’s in another file. Press <code>Ctrl-T</code> (or <code>M-*</code>) and you'll be taken back to where you were. These tags stack, that is you can go into a subroutine, and from there go into another one, and so on, and still be able to back out to the beginning.

You can also start vi (vim really) with the <code>-t <tag></code> option to jump immediately to a particular tag. I don't know if emacs can do this, as I lost the will to live while browsing the 'info' documentation.

Editors other than vi and Emacs also support tags; check the documentation. If your editor doesn't support tags it's just not good enough, sorry.

=== Screen ===

Screen is a terminal multiplexer. If you wish to have multiple terminals open on a remote system you're probably accustomed to opening multiple SSH windows from your local system. An alternative is to use <code>screen</code>. It allows you to have multiple windows, each with its own history and possibly executing commands, and switch between them with some simple keypresses. It also preserves the state of these windows when your connection to screen drops, allowing you to reconnect later.

It's a bit beyond the scope of this page, if you want to learn more check out the various web resources or it, such as http://www.rackaid.com/resources/linux-screen-tutorial-and-how-to/.

Resgrp:comp-photo-version control-local-dev-master

2013-11-21T16:52:02Z

Scliffor:

This page will show the current state of the group's master dev repository <code>/home/gaussian-devel/repositories/local-dev-master</code>.
It will document what code is currently considered to be accepted as ready to use.

The format is explained [[#Formatting|here]]. You don't have to describe each revision, just
whatever the tip is at the time, and whatever might be (or have been) made into a
compiled version.
==The revisions==
The most recent revision (which should be the tip) should always be on top. Note
that the "tip" tag may appear incorrectly below. Also note that the local revision
numbers (e.g. 56 in changeset: 56:e359f0e1aa5c) may differ in other repositories. If
in doubt, use the hex number.

===changeset: 92:be0e7e296f2f===
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Sun Nov 17 22:44:39 2013 +0000
summary: Local development merged with H32

Local development merged with H32

This entailed quite a bit of editing with the onifrq code, as it
appears some of these changes have been migrated to H32, with
Gaussian Inc changes as well. I kept as much as possible of the H32
code, as this is what's already been accepted by Gau Inc.

===changeset: 73:923d6c51169a===
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Fri Jan 18 17:11:08 2013 +0000
summary: Fixed bug where ONIOM jobs would fail with:

Updated version fixing a bug where ONIOM 2nd derivative jobs would fail
with a message like:
ONIOM: calculating second derivatives.
NInts<0 in InToWP.
Error termination via Lnk1e in /home/gaussian-devel/versions/rev-a7f8ac5c6a00-i386-pg12.9/gdv/l120.exe at Fri Jan 11 18:06:43 2013.

===changeset: 72:a7f8ac5c6a00===
parent: 70:f87f732e26fb
parent: 71:69b842c84566
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Nov 21 14:12:01 2012 +0000
summary: Merged in more updates to makefiles (h23)

DEPRECATED: see revision [[#changeset: 73:923d6c51169a|73:923d6c51169a]].

This revision contains the [[Resgrg:comp-photo-version control-codes#Shaopeng|Shaopeng matrix element code]], the [[Resgrg:comp-photo-version control-codes#Seamfollowing|seam following code]],
and the [[Resgrg:comp-photo-version control-codes#ONIOM Freq|ONIOM frequency analysis code]]. They are all merged to be compatible with H23.

I have also added into the repository test jobs for each of these local changes. These are stored in
an upper directory called <code>local-tests</code>.

===changeset: 56:e359f0e1aa5c===
parent: 55:f6c5e2290dc4
parent: 53:c7f36daff923
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Oct 10 14:56:24 2012 +0100
summary: Merged Shaopeng, seamfollowing, olddirect (H23)

DEPRECATED: Do not use, likely to crash. See rev [[#changeset: 57:0c20d50c4185|57:0c20d50c4185]].

The merge of the three features. Based off H23.
===changeset: 55:f6c5e2290dc4===
parent: 54:0c83655a3822
parent: 41:c8a317a36a4e
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Oct 10 14:55:17 2012 +0100
summary: Merged olddirect code to H23

DEPRECATED: Do not use, likely to crash. See rev [[#changeset: 57:0c20d50c4185|57:0c20d50c4185]].

Simon's updates and fixes for the semi-direct matrix element code.
Based off H23.
===changeset: 53:c7f36daff923===
parent: 52:e11a2fc5b5a8
parent: 41:c8a317a36a4e
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Oct 10 12:38:05 2012 +0100
summary: Shaopeng + seamfollowing, merged to H23.

DEPRECATED: Do not use, likely to crash. See rev [[#changeset: 57:0c20d50c4185|57:0c20d50c4185]].

The Shaopeng code and seamfollowing code as below, updated to H23.
===changeset: 51:dcd851f8dc57===
parent: 50:c0a9f878ea3e
parent: 39:c90162c4fbe6
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Tue Oct 09 22:15:18 2012 +0100
summary: Updated seamfollowing code to H21

DEPRECATED: Do not use, likely to crash. See rev [[#changeset: 57:0c20d50c4185|57:0c20d50c4185]].

This is the imported seam following code updated to H21.
===changeset: 49:bda0e1259a36===
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Jul 02 15:04:35 2012 +0100
summary: Large modification and cleanup of Shaopeng code. Below for details.

This contains the [[Resgrg:comp-photo-version control-codes#Shaopeng|Shaopeng matrix element code]]
with most of Simon's modifications
and bug fixes in place. It's based off version H21
===changeset: 41:c8a317a36a4e===
tag: h23
parent: 40:cfd3ea6fcdff
parent: 25:7a8a763281d4
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:45:00 2012 +0100
summary: Merged in H23

This is official Gaussian version H23.

===changeset: 39:c90162c4fbe6===
tag: h21
parent: 38:ef0c2dff2d79
parent: 20:dd575870f775
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:44:00 2012 +0100
summary: Merged in H21

This is official Gaussian version H21.
===changeset: 37:e972407aab01===
tag: h13
parent: 36:9d36f9fe483b
parent: 17:770721638a7a
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:43:06 2012 +0100
summary: Merged in H13

This is official Gaussian version H13.
===changeset: 35:10679662e3a3===
tag: h12p
parent: 34:3ce1603d2c6c
parent: 16:427978d67ed6
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:42:11 2012 +0100
summary: Merged in H12p

This is official Gaussian version H12p.
===changeset: 33:a0e0a314db85===
tag: h11
parent: 32:3732a6888cc4
parent: 15:27066b86194e
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:39:49 2012 +0100
summary: Merged in H11

This is official Gaussian version H11.
===changeset: 31:7d8a337a7324===
tag: h10
parent: 30:86c2a0b6f977
parent: 14:62f983246ba7
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:39:11 2012 +0100
summary: Merged in H10

This is official Gaussian version H10.
===changeset: 29:56fa127c847c===
tag: h08
parent: 28:877924b27ee3
parent: 13:903ac662c7a2
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:38:04 2012 +0100
summary: Merged in H08

This is official Gaussian version H08.
===changeset: 27:a05e10e365c2===
tag: h01
parent: -1:000000000000
parent: 12:ae8da988f728
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:35:05 2012 +0100
summary: Fixed bug where code compiled with PGI >= 12 would fail with thread error.

This is official Gaussian version H01.

==Formatting==
The format is:

===changeset: 56:e359f0e1aa5c===
tag: tip
parent: 55:f6c5e2290dc4
parent: 53:c7f36daff923
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Oct 10 14:56:24 2012 +0100
summary: Merged Shaopeng, seamfollowing, olddirect (H23)

Text explaining what this revision contains,
e.g. all of the below plus merged up to H23

In raw form that's:
<code>
===changeset: 56:e359f0e1aa5c===
tag: tip
parent: 55:f6c5e2290dc4
parent: 53:c7f36daff923
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Oct 10 14:56:24 2012 +0100
summary: Merged Shaopeng, seamfollowing, olddirect (H23)

Text explaining what this revision contains,
e.g. all of the below plus merged up to H23
</code>

This is the output from <code>hg log</code> with the changeset line made into a header, a space
added to the start of each following line, and more
explanatory text. '''NB, it's important to add more explanation!'''

Resgrp:comp-photo-version control new gdv

2013-03-06T12:45:19Z

Scliffor:

==Introduction==
This page will explain how to import a new version of gdv into the version control system. For general help on
the version control system look [[Resgrp:comp-photo-version control|here]].

==How to import a new version==

I'll assume you've got a tarball from Gaussian, in their usual format.

First, get a up to date copy of the <code>gaussian-inc-versions</code> repository. Either use <code>hg clone</code> to
get a new copy or <code>hg pull</code> an existing repository. Make sure it's updated to the latest
version ''on the raw branch'' (using <code>hg update raw</code>, or <code>hg update -C raw</code> if necessary.

Next, untar the tarball somewhere and process it using the old-to-new.sh script in the repository.

<code>
$ ls
gdv new-to-old.sh old-to-new.sh pillage.sh
$ mkdir ../h29
$ cd ../h29
$ tar zxvf /work/scliffor/gaussian/gdvh29p.tgz
$ ls
gdv
</code>

(As ever, when using new-to-old.sh, you'll need to make sure you have a <code>gau-fsplit</code> in your path. I usually
just copy the binary from the latest compiled version to my <code>~/bin</code> directory.)

Now convert the code to our new [[Resgrp:comp-photo-new gdv layout|format]].

<code>
$ ../gaussian-inc-versions/old-to-new.sh new-style
doing gdv/allxc.inc
doing gdv/amber98.prm
doing gdv/amber.prm
doing gdv/archlib.F
doing gdv/arctmp.F
doing gdv/basis
doing gdv/blas-generic.F
doing gdv/bsd
...
doing gdv/view
doing gdv/vuind.inc
doing gdv/wrappers.F
doing gdv/wrmat.F
doing gdv/xcind.inc
$ ls new-style/gdv/
allxc.inc commonz.inc ertgen.inc l1002 l1112 l302 l405 l701 l904 mdarch rwfcopy.F
amber98.prm copychk.F fffcom.inc l1003 l112 l303 l502 l702 l905 mdl1 rwfdump.F
amber.prm cphfutil fhello.F l1004 l113 l305 l503 l703 l906 mdutil separ.inc
archlib crdparams.inc filecom.inc l101 l114 l306 l504 l705 l908 mm2.prm tests
arctmp.F crendx.F flddsc.inc l1014 l115 l307 l506 l709 l909 mm.F trajgen.F
basis cubegen.F fldloc.inc l102 l116 l308 l508 l715 l913 newzmatF uff.prm
blas-generic.F cubman.F formchk.F l103 l117 l309 l509 l716 l914 ntran.inc unfchk.F
bsd Default.Route.save freqchk.F l105 l118 l310 l510 l717 l915 oplsaa.prm utilam
c86dv.F demofc.F freqmem.F l106 l120 l311 l511 l718 l916 osutil util.hlp
chkchk.F dftba.prm gauopt.F l107 l121 l314 l601 l801 l918 pluck.F utilnz
chkmove.F dftbpar.prm gauoptl.F l108 l122 l315 l602 l802 l920 prparm.inc view
cktoig.F dinautil gautraj.F l109 l123 l316 l604 l804 l921 putil vuind.inc
commonb2.inc doc gdrgen.F l110 l124 l317 l607 l809 l922 putil.hlp wrappers
commonb.inc dreiding.prm ghelp.F l1101 l125 l318 l608 l810 l923 qppar.inc wrmat.F
commonlab.inc dummy.F ghelp.hlp l1102 l126 l319 l609 l811 l924 qpstat.inc xcind.inc
commonlp2.inc dummy-link.F grate.F l111 l127 l320 l610 l901 l930 rdmat.F
commonmol2.inc dummy-links.F ham506.F l1110 l202 l401 l611 l902 l9999 reform.F
commonmol.inc dummy-narch.F l1 l1111 l301 l402 l612 l903 lapack-generic.F repall.inc
</code>

It's worth having a look here to make sure nothing looks too odd. This looks OK, so we press on. Change back to the
<code>gaussian-inc-versions</code> directory, delete the <code>gdv</code> directory, and replace it with the <code>gdv</code>
directory you just created.

<code>
$ cd ../gaussian-inc-versions
$ rm -fr gdv
$ cp -pr ../h29/new-style/gdv/ .
</code>

Now we have the new code, in our format, in the repository. Our job now is to prepare a commit that will correctly represent the changes
that have been made by Gaussian.

The first task is to check that our ignore filter isn't hiding any changes we want to include. We can do this with <code>hg stat -i</code>.
If you do this in this case, you'll see that the only things being ignored are the log files of test jobs, which we don't keep in the
repository. Sometimes you'll see that Gaussian have added a library files (a .a file). If this happens add that particular file now
using <code>hg add</code>.

Next we check the output of <code>hg stat -u</code>, which shows unknown (new) files. Firstly, we're checking here to see if there are
any files that ''aren't'' being ignored that should. Examples include executables or compiled routines for which we have the source,
Gaussian outputs such as <code>.log</code or <code>.chk</code> files, temporary files (such as Emacs <code>filename~</code> files), and
so on. Anything that can be recreated from other things in the gdv directory is probably ignorable. Add suitable lines to the
<code>.hgignore</code> file as appropriate, or just delete the file if you feel an ignore entry is not needed.

In the output of <code>hg stat -u</code> in this case you'll notice that the <code>test000.com</code> files have all been renamed to <code>test0000.com</code> files. Mercurial initially reports this as a deletion (of <code>test000.com</code>) and addition
(of <code>test0000.com</code>). We can get Mercurial to recognise these as renames<ref>You may be wondering why we care about renames.
Simply put, if changes from Gaussian that include a rename or a copy must be merged with your own changes to those files
(or vice versa) that it helps mercurial to know about the renames and copies. Otherwise for a rename it only sees a file being deleted
and another file being created.</ref> instead by doing this command (the
<code>-n</code> flag makes <code>addremove</code> do a dry-run. We'll do the tests subdirectory separately, as it's probably
going to be fairly straightfoward.

<code>
$ hg addremove -s 50 -n gdv/tests | less
</code>

The <code>-s N</code> instructs Mercurial to compare files and report renames where the contents differ by no less than N%.
If you set N too low then you'll see many false positives, too high and you may miss renames. Note that the percentage
is based on the number of lines changed, so changes of even a few characters to a small file can produce quite low
similarity percentages.
If you leave <code>-s</code> off it will still report those that are 100% the same. In this case you'll see (after a while)
a huge list of lines like:

<code>
recording removal of gdv/tests/com/test370.com as rename to gdv/tests/com/test0370.com (100% similar)
...
recording removal of gdv/tests/test983d-m-ref.cube as rename to gdv/tests/test0983d-m-ref.cube (83% similar)
</code>

Now you have to engage your knowledge of Gaussian! In almost every instance here in the <code>tests</code> directory
it's clear that these are genuine renames, with some minor modifications. If in doubt, suspend the <code>addremove</code>
command and do something like:

<code>
$ hg cat gdv/tests/test983d-m-ref.cube | diff - gdv/tests/test0983d-m-ref.cube | less
</code>

which will show the nature of the changes. Here you'll see that for reasons that probably only make sense to someone not using
version control, Gaussian have very slightly altered some of the values in this <code>.cube</code> file.

You can play with the <code>-s N</code> parameter until you get a list that is only correct renames, with no false positives
(if there are some false positives they can be undone before the commit, false negatives can also be added in later). When you're happy
run the <code>addremove</code> command without the <code>-n</code> flag.

I decided to go with <code>-s 50</code>. Once the command finishes (and it can take quite a while, particularly for low values
of N) you can check what's been done with <code>hg stat -a -C</code>, which shows files that have been added, and additionally, whether
they are marked as copies. If they are copies they will appear like this:

<code>
$ hg stat -a -C
A gdv/tests/check-of
A gdv/tests/ckelap
A gdv/tests/com/test0000.com
gdv/tests/com/test000.com
A gdv/tests/com/test0001.com
gdv/tests/com/test001.com
...
A gdv/tests/com/test1036.com
gdv/tests/com/test872.com
A gdv/tests/com/test1037.com
gdv/tests/com/test872.com
A gdv/tests/com/test1038.com
...
</code>
which shows <code>gdv/tests/com/test0000.com</code> has been marked as a copy of <code>gdv/tests/com/test000.com</code>, etc.
Note that <code>gdv/tests/com/test1036.com</code> and <code>gdv/tests/com/test1037.com</code> have been marked as copies of <code>gdv/tests/com/test872.com</code>, which has already been marked as the origin of <code>gdv/tests/com/test0872.com</code>.
In certain circumstances this might be legitimate, with one file being renamed and copied (or just copied) but in this
case, using <code>hg diff</code> I decide that however it's arisen, I don't want to record it. I remove the adds and then redo them, without
marking them as renames:

<code>
$ hg revert gdv/tests/com/test1036.com gdv/tests/com/test1037.com
$ hg stat gdv/tests/com/test1036.com gdv/tests/com/test1037.com
? gdv/tests/com/test1036.com
? gdv/tests/com/test1037.com
$ hg add gdv/tests/com/test1036.com gdv/tests/com/test1037.com
$ hg stat -C gdv/tests/com/test1036.com gdv/tests/com/test1037.com
A gdv/tests/com/test1036.com
A gdv/tests/com/test1037.com
</code>

We now apply this same methodology to the remaining unknown files. We are trying to get to a situation where the output of
<code>hgstat -du</code> is empty. The <code>-u</code> flag shows files that have appeared in the <code>gdv</code> directory
but are unknown to mercurial; these are files added or renamed by Gaussian. The <code>-d</code> flag shows files
that are known to mercurial but are no longer in the <code>gdv</code> directory.

If you run <code>hg addremove</code> on the
<code>gdv</code> directory it will try re-add things in <code>tests</code>, so be careful. There's no harm in re-adding
files you've already added, but you don't want to add files you've manually removed from the list (for example,
<code>hg addremove -s 50 gdv</code> will re-mark <code>gdv/tests/com/test1036.com</code> as a copy of
<code>gdv/tests/com/test872.com</code> again. Avoid this by using the <code>-X</code> flag to add remove:

<code>
$ hg addremove -s 40 -n -X 'glob:gdv/tests/**' gdv
</code>

If I use the above command I get a long list of files being removed and added and then at the end of list of files that
are being recorded as renames. It's worth checking the list of additions and deletions, particularly scripts in the
<code>bsd</code> directory. In this case there are some scripts, but they all check out OK. The list of renames is:

<code>
...
recording removal of gdv/l718/quairc.F as rename to gdv/dinautil/quairc.F (100% similar)
recording removal of gdv/l718/quarot.F as rename to gdv/dinautil/quarot.F (100% similar)
recording removal of gdv/l718/rotirc.F as rename to gdv/dinautil/rotirc.F (100% similar)
recording removal of gdv/l120/getlfo.F as rename to gdv/utilam/getlfo.F (100% similar)
recording removal of gdv/mdutil/lshift1.F as rename to gdv/mdutil/lshft1.F (73% similar)
recording removal of gdv/l718/difrot.F as rename to gdv/dinautil/difrot.F (97% similar)
recording removal of gdv/utilam/iiabs.F as rename to gdv/utilam/ivabs.F (82% similar)
recording removal of gdv/l1/decotp.F as rename to gdv/l1/decjob.F (77% similar)
recording removal of gdv/l1/decotp.F as rename to gdv/l1/decmth.F (47% similar)
</code>

this is where some of your Gaussian intuition comes in handy. Moving files from link subdirectories
to utility subdirectories is a common and expected thing. If we inspect the <code>gdv/utilam/iiabs.F</code> to
<code>gdv/utilam/ivabs.F</code> rename we see that the files are the same apart from two one character changes.
Inspecting the final two renames shows that the file <code>gdv/l1/decotp.F</code> is indeed being split
into two new files. Therefore I go ahead with the command without the <code>-N</code> flag.

The final task before we go ahead and commit this changeset is to check the makefile for
any new targets. If these correspond to new executables we will need to add them to the
<code>.hgignore</code> list.

<code>
$ hg diff gdv/bsd/i386.make | less
</code>

shows us that there are two new links:

<code>
...
@@ -212,7 +212,7 @@
l105.exe l106.exe l107.exe l108.exe l109.exe \
l110.exe l111.exe l112.exe l113.exe l114.exe l115.exe \
l116.exe l117.exe l118.exe l120.exe l121.exe \
- l122.exe l123.exe l124.exe l125.exe l202.exe
+ l122.exe l123.exe l124.exe l125.exe l126.exe l127.exe l202.exe

exe3: l301.exe l302.exe l303.exe l305.exe l306.exe \
l307.exe l308.exe l309.exe l310.exe l311.exe \
...
</code>

and two new utilities:

<code>
...
@@ -597,6 +597,12 @@
unfchk: unfchk.o
$(RUNF77) $(FFLAGS) -o unfchk unfchk.o $(EXTOBJ) $(EXTRAS) $(GAULIB) $(LIBS)

+gdrgen: gdrgen.o
+ $(RUNF77) $(FFLAGS) -o gdrgen gdrgen.o $(EXTOBJ) $(EXTRAS) $(GAULIB) $(LIBS)
+
+trajgen: trajgen.o
+ $(RUNF77) $(FFLAGS) -o trajgen trajgen.o $(EXTOBJ) $(EXTRAS) $(GAULIB) $(LIBS)
+
rdmat: rdmat.o
$(RUNF77) $(FFLAGS) -o rdmat rdmat.o $(EXTOBJ) $(EXTRAS) $(GAULIB) $(LIBS)
...
</code>

The links are detected by a pattern in the HG ignore file a we must add the two new utilities,
so that:

<code>
$ hg diff .hgignore
diff -r 8db6c9367c60 .hgignore
--- a/.hgignore Thu Aug 02 11:17:35 2012 +0100
+++ b/.hgignore Mon Mar 04 17:58:09 2013 +0000
@@ -32,6 +32,7 @@
gdv/gauopt
gdv/gauoptl
gdv/gautraj
+gdv/gdrgen
gdv/gdv
gdv/gdvlb
gdv/ghelp
@@ -46,5 +47,6 @@
gdv/rwfdump
gdv/tags
gdv/testrt
+gdv/trajgen
gdv/unfchk
gdv/wrmat
</code>

This then,represents Gaussian as it came from Gaussian Inc converted to our new format.
Once you are satisfied commit the revision with <code>hg ci</code>. There is a preferred
format for the
log message for these raw branch commits; look at one of the previous commits using
<code>hg log -v --rev N | less</code> for an example. The log message I will use for
this particular commit is:

<code>
Import of Gaussian version H29p. See below for details.

rm -fr gdv
cp -pr ../h29p/new-style/gdv .
hg addremove -s 50 gdv/tests
hg revert gdv/tests/com/test1036.com gdv/tests/com/test1037.com
hg add gdv/tests/com/test1036.com gdv/tests/com/test1037.com
hg addremove -s 40 -X 'glob:gdv/tests/**' gdv

and added gdrgen and trajgen to .hgignore.
</code>

The idea is that someone should be able to replicate what you've done
just by referring to the log message.

Congratulations! You've just added a revision to the raw branch.
Now tag it with the version identifier:

<code>
$ hg tag h29p-raw
</code>

Next, we must merge this version into our local branch, which represents
a compilable version of Gaussian in our new format. generally this only
requires changes to our local make files, however these changes can
require considerable effort.

<code>
$ hg up local
1995 files updated, 0 files merged, 1228 files removed, 0 files unresolved
Added tags for local versions
$ hg merge raw
merging .hgignore
merging .hgtags
4 files to edit
...
</code>

At this point this guide will get a little hazy because I used vim
and you may be using some other editor. After you type
<code>hg merge raw</code> you may be presented with a graphical merge
tool, a text based merge tool, a particular mode of your favourite text editor,
a particular mode of a hated text editor, or indeed, nothing. You can customise
the tool that is used if you want. For the purposes of this guide quit out of the
tool or editor without making any changes. If you are using vim use the
<code>:cq</code> command which will cause vim to return an error to mercurial
which tells it that the merge is not finished. Again for the purposes of this guide
run <code>hg resolve -a -u</code>
to mark all the conflicting files
as unresolved.

You should now be able to see:

<code>
$ hg resolve -l
U .hgignore
U .hgtags
U gdv/bsd/uber-i386.make
U gdv/bsd/uber-sandyb.make
</code>

Let's now resolve these four files. First <code>.hgignore</code>:

<code>
$ hg resolve .hgignore
merging .hgignore
</code>

This will return immediately having automatically merged the files and
produced no conflicts. Easy enough. If you examine the file you will see
that our two new executables have been added. Next:

<code>
$ hg resolve .hgtags
merging .hgtags
</code>

For me this command starts vimdiff with windows showing the tags file as
it exists in the two-parent revisions of the merge plus another window
where I can edit the final merged version; this window will have the title
<code>gdv/bsd/.hgtags</code>, the others will contain words like "base,
other, and orig". Although the tags file
will generally look after itself I like to keep things neat so I
edit the file so that the knew tag, h29p, exists in the final version.

now to the more difficult files:

<code>
$ hg resolve gdv/bsd/uber-i386.make
</code>

again for me this opens up vimdiff. Here we have some work to do
because <code>gdv/bsd/uber-i386.make</code> has been marked as a
copy of <code>gdv/bsd/i386.make</code>, which inevitably is changed
with every new version of Gaussian. I'll go through this carefully here
the generally you'll have to use your own experience with make and
Gaussian to guide you through the process. If necessary you can quit
out of the merge tool (use <code>:cq</code> for vim based editors.
This marks the file as unresolved) and examine the changes to
<code>gdv/bsd/i386.make</code> using <code>hg diff</code>.

The first difference I see involves <code>INCDIRX</code>. However,
inspection shows that this has not changed in the new version of
<code>gdv/bsd/i386.make</code>. The next change is to the definition
of <code>DIMENSX</code>. Here we see that Gaussian have added a
flag "<code>-DSTUPID_ATLAS</code>", changed the expansion of
"<code>-DDEFMAXCOORDINFO</code>" from 16 to 32, and changed
"<code> -DSETCORE_OK</code>" to "<code> -DSETCDMP_OK</code>". These
are legitimate changes that I add to our local makefile.

The next part we come to is the part of the local makefile that says
output from set-mflags here. This is a series of compiler and other
flags that are emitted by Gaussian utility during the normal build phase.
We have to capture those flags directly into the makefile. To do this
you will need to do something like:

<code>
$ cd gdv
$ ln -s uber-i386.make bsd/gdv.make
$ make build-tools
$ PATH=$PATH:.:./bsd gdvroot=/tmp/scliffor/repos/gaussian-inc-versions/ bsd/set-mflags
GAUDIM=2500 CSIZE=2097152 CSIZEW=128 OPTOI= MMODEL='-mcmodel=medium' SPECFLAG= NKSEC=-DDEFKSEC=256 I8FLAG=-i8 R8FLAG=-r8 I8CPP1=-DI64 I8CPP2=-DP64 I8CPP3=-DPACK64 I8CPP4=-DUSE_I2 MACHTY=k8-64 GAULIBU=util.a BLAS1=bsd/libf77blas-amd64.a BLAS2=bsd/libatlas-amd64.a
</code>

that last line is space separated and not terminated with a newline.
If you paste it into a file and replace the spaces with newlines you'll
see that nothing has changed.

The next changes that we care about come in the definition of make targets
<code>exe1</code> and <code>UTILLIST</code>,
as we noticed previously Gaussian have added two new links and two new executables
<code>l126.exe</code>, <code>l127.exe</code>, <code> gdrgen</code>,
and <code>trajgen</code>. Further down we must include the commands needed
to make the two new executables.

Then there is a change to the commands needed to make <code>gau-cpp</code>
and finally there are some alterations to the lists of subroutines that need
special optimisation flags. I include all of these changes.

when you're done save the merged file and quit the merge tool. If the
merge tool or editor that you are using exits with a success code then
mercurial will mark the file as resolved.

Do the same for the <code>gdv/bsd/uber-sandyb.make</code>. The changes will be
pretty much the same.

Because there are new executables targets we must add them to our master
Makefile, <code>gdv/Makefile</code>. edit this file and you will see
that the declarations of the targets mirrors the
<code>gdb/bsd/*.make</code> files. Add <code>l126.exe</code>,
<code>l127.exe</code>, <code>gdrgen</code>,
and <code>trajgen</code> to the file. Further down, after the part of
the makefile that looks like:

<code>
.INTERMEDIATE: unfchk.o
unfchk: unfchk.o $(GAULIB)
$(RM) $@
$(MAKE) -f $(GAU_DIR)/bsd/gdv.make MAKE='$(MAKE)' $(EXTRAMAKEFLAGS) $@

</code>

copy these five lines (including the blank one) and the change all
occurrences of <code>unfchk</code> to <code>gdrgen</code>. Repeat for
<code>trajgen</code>. Finally find the lines:

<code>
.SECONDARY: $(filter-out $(ml125.o), $(patsubst %.F, %.o, $(wildcard l125/*.F)))
l125.a: $(filter-out $(ml125.o), $(patsubst %.F, %.o, $(wildcard l125/*.F)))
$(AR) rv $@ $?

</code>

copy them (again including the blank line), replacing <code>l125</code>
with <code>l126</code>. Repeat for <code>l127</code>.

After this we are hopefully left with the new version of Gaussian
in our new format with our local makefiles to build that new format.
The best way to test this is to build it! I did this and it builds
successfully. Now run the test suite (yeah right).

A quick check with <code>hg stat -u</code> will usually show some
<code>.orig</code> files. These are usually left behind by the merge
process, and can be safely removed. If anything else shows up you must decide whether
to add it to the <code>.hgignore</code> file or delete it.
You are now ready to commit this revision, so do so with an appropriate
log message, and tag it (<code>hg tag h29p-local</code>).

And now we're ready for the final merge. Update to the default branch
and do the merge:

<code>
$ hg update default
$ hg merge local
</code>

This is very much the same as the previous merge from raw to local.
What you're aiming for on the default branch is a version of Gaussian
that will build and run efficiently on the group's local systems
(namely CX1). This means that the makefiles should contain the necessary
optimisations for the CX1 architecture. It may also mean that you
may have to make changes to the code to fix runtime or compile time
bugs. For instance, when we started using a new version of the PGI
compiler it exposed a bug in Gaussian that prevented any jobs using
more than one processor.

I have made some changes to the compiler flags of both of the
<code>uber-*</code> makefiles.

It's worth compiling the code again at this point and running a few
test jobs. You can use these compiled executables in the
<code>/home/gaussian-devel/versions</code> directory.

Once you are satisfied that the code is fine push your changes back
to the master repository (no harm in making a backup of the master
beforehand). In the <code>local-dev-master</code> repository pull the
changes from the <code>gaussian-inc-versions</code> repository and let
people know that the changes are now available. They can pull and the
merge the changes as I've described here.
<references/>

Resgrp:comp-photo-version control new gdv

2013-03-04T18:52:26Z

Scliffor:

==Introduction==
This page will explain how to import a new version of gdv into the version control system. For general help on
the version control system look [[Resgrp:comp-photo-version control|here]].

==How to import a new version==

I'll assume you've got a tarball from Gaussian, in their usual format.

First, get a up to date copy of the <code>gaussian-inc-versions</code> repository. Either use <code>hg clone</code> to
get a new copy or <code>hg pull</code> an existing repository. Make sure it's updated to the latest
version ''on the raw branch'' (using <code>hg update raw</code>, or <code>hg update -C raw</code> if necessary.

Next, untar the tarball somewhere and process it using the old-to-new.sh script in the repository.

<code>
$ ls
gdv new-to-old.sh old-to-new.sh pillage.sh
$ mkdir ../h29
$ cd ../h29
$ tar zxvf /work/scliffor/gaussian/gdvh29p.tgz
$ ls
gdv
</code>

(As ever, when using new-to-old.sh, you'll need to make sure you have a <code>gau-fsplit</code> in your path. I usually
just copy the binary from the latest compiled version to my <code>~/bin</code> directory.)

Now convert the code to our new [[Resgrp:comp-photo-new gdv layout|format]].

<code>
$ ../gaussian-inc-versions/old-to-new.sh new-style
doing gdv/allxc.inc
doing gdv/amber98.prm
doing gdv/amber.prm
doing gdv/archlib.F
doing gdv/arctmp.F
doing gdv/basis
doing gdv/blas-generic.F
doing gdv/bsd
...
doing gdv/view
doing gdv/vuind.inc
doing gdv/wrappers.F
doing gdv/wrmat.F
doing gdv/xcind.inc
$ ls new-style/gdv/
allxc.inc commonz.inc ertgen.inc l1002 l1112 l302 l405 l701 l904 mdarch rwfcopy.F
amber98.prm copychk.F fffcom.inc l1003 l112 l303 l502 l702 l905 mdl1 rwfdump.F
amber.prm cphfutil fhello.F l1004 l113 l305 l503 l703 l906 mdutil separ.inc
archlib crdparams.inc filecom.inc l101 l114 l306 l504 l705 l908 mm2.prm tests
arctmp.F crendx.F flddsc.inc l1014 l115 l307 l506 l709 l909 mm.F trajgen.F
basis cubegen.F fldloc.inc l102 l116 l308 l508 l715 l913 newzmatF uff.prm
blas-generic.F cubman.F formchk.F l103 l117 l309 l509 l716 l914 ntran.inc unfchk.F
bsd Default.Route.save freqchk.F l105 l118 l310 l510 l717 l915 oplsaa.prm utilam
c86dv.F demofc.F freqmem.F l106 l120 l311 l511 l718 l916 osutil util.hlp
chkchk.F dftba.prm gauopt.F l107 l121 l314 l601 l801 l918 pluck.F utilnz
chkmove.F dftbpar.prm gauoptl.F l108 l122 l315 l602 l802 l920 prparm.inc view
cktoig.F dinautil gautraj.F l109 l123 l316 l604 l804 l921 putil vuind.inc
commonb2.inc doc gdrgen.F l110 l124 l317 l607 l809 l922 putil.hlp wrappers
commonb.inc dreiding.prm ghelp.F l1101 l125 l318 l608 l810 l923 qppar.inc wrmat.F
commonlab.inc dummy.F ghelp.hlp l1102 l126 l319 l609 l811 l924 qpstat.inc xcind.inc
commonlp2.inc dummy-link.F grate.F l111 l127 l320 l610 l901 l930 rdmat.F
commonmol2.inc dummy-links.F ham506.F l1110 l202 l401 l611 l902 l9999 reform.F
commonmol.inc dummy-narch.F l1 l1111 l301 l402 l612 l903 lapack-generic.F repall.inc
</code>

It's worth having a look here to make sure nothing looks too odd. This looks OK, so we press on. Change back to the
<code>gaussian-inc-versions</code> directory, delete the <code>gdv</code> directory, and replace it with the <code>gdv</code>
directory you just created.

<code>
$ cd ../gaussian-inc-versions
$ rm -fr gdv
$ cp -pr ../h29/new-style/gdv/ .
</code>

Now we have the new code, in our format, in the repository. Our job now is to prepare a commit that will correctly represent the changes
that have been made by Gaussian.

The first task is to check that our ignore filter isn't hiding any changes we want to include. We can do this with <code>hg stat -i</code>.
If you do this in this case, you'll see that the only things being ignored are the log files of test jobs, which we don't keep in the
repository. Sometimes you'll see that Gaussian have added a library files (a .a file). If this happens add that particular file now
using <code>hg add</code>.

Next we check the output of <code>hg stat -u</code>, which shows unknown (new) files. Firstly, we're checking here to see if there are
any files that ''aren't'' being ignored that should. Examples include executables or compiled routines for which we have the source,
Gaussian outputs such as <code>.log</code or <code>.chk</code> files, temporary files (such as Emacs <code>filename~</code> files), and
so on. Anything that can be recreated from other things in the gdv directory is probably ignorable. Add suitable lines to the
<code>.hgignore</code> file as appropriate, or just delete the file if you feel an ignore entry is not needed.

In the output of <code>hg stat -u</code> in this case you'll notice that the <code>test000.com</code> files have all been renamed to <code>test0000.com</code> files. Mercurial initially reports this as a deletion (of <code>test000.com</code>) and addition
(of <code>test0000.com</code>). We can get Mercurial to recognise these as renames<ref>You may be wondering why we care about renames.
Simply put, if changes from Gaussian that include a rename or a copy must be merged with your own changes to those files
(or vice versa) that it helps mercurial to know about the renames and copies. Otherwise for a rename it only sees a file being deleted
and another file being created.</ref> instead by doing this command (the
<code>-n</code> flag makes <code>addremove</code> do a dry-run. We'll do the tests subdirectory separately, as it's probably
going to be fairly straightfoward.

<code>
$ hg addremove -s 50 -n gdv/tests | less
</code>

The <code>-s N</code> instructs Mercurial to compare files and report renames where the contents differ by no less than N%.
If you set N too low then you'll see many false positives, too high and you may miss renames. Note that the percentage
is based on the number of lines changed, so changes of even a few characters to a small file can produce quite low
similarity percentages.
If you leave <code>-s</code> off it will still report those that are 100% the same. In this case you'll see (after a while)
a huge list of lines like:

<code>
recording removal of gdv/tests/com/test370.com as rename to gdv/tests/com/test0370.com (100% similar)
...
recording removal of gdv/tests/test983d-m-ref.cube as rename to gdv/tests/test0983d-m-ref.cube (83% similar)
</code>

Now you have to engage your knowledge of Gaussian! In almost every instance here in the <code>tests</code> directory
it's clear that these are genuine renames, with some minor modifications. If in doubt, suspend the <code>addremove</code>
command and do something like:

<code>
$ hg cat gdv/tests/test983d-m-ref.cube | diff - gdv/tests/test0983d-m-ref.cube | less
</code>

which will show the nature of the changes. Here you'll see that for reasons that probably only make sense to someone not using
version control, Gaussian have very slightly altered some of the values in this <code>.cube</code> file.

You can play with the <code>-s N</code> parameter until you get a list that is only correct renames, with no false positives
(if there are some false positives they can be undone before the commit, false negatives can also be added in later). When you're happy
run the <code>addremove</code> command without the <code>-n</code> flag.

I decided to go with <code>-s 50</code>. Once the command finishes (and it can take quite a while, particularly for low values
of N) you can check what's been done with <code>hg stat -a -C</code>, which shows files that have been added, and additionally, whether
they are marked as copies. If they are copies they will appear like this:

<code>
$ hg stat -a -C
A gdv/tests/check-of
A gdv/tests/ckelap
A gdv/tests/com/test0000.com
gdv/tests/com/test000.com
A gdv/tests/com/test0001.com
gdv/tests/com/test001.com
...
A gdv/tests/com/test1036.com
gdv/tests/com/test872.com
A gdv/tests/com/test1037.com
gdv/tests/com/test872.com
A gdv/tests/com/test1038.com
...
</code>
which shows <code>gdv/tests/com/test0000.com</code> has been marked as a copy of <code>gdv/tests/com/test000.com</code>, etc.
Note that <code>gdv/tests/com/test1036.com</code> and <code>gdv/tests/com/test1037.com</code> have been marked as copies of <code>gdv/tests/com/test872.com</code>, which has already been marked as the origin of <code>gdv/tests/com/test0872.com</code>.
In certain circumstances this might be legitimate, with one file being renamed and copied (or just copied) but in this
case, using <code>hg diff</code> I decide that however it's arisen, I don't want to record it. I remove the adds and then redo them, without
marking them as renames:

<code>
$ hg revert gdv/tests/com/test1036.com gdv/tests/com/test1037.com
$ hg stat gdv/tests/com/test1036.com gdv/tests/com/test1037.com
? gdv/tests/com/test1036.com
? gdv/tests/com/test1037.com
$ hg add gdv/tests/com/test1036.com gdv/tests/com/test1037.com
$ hg stat -C gdv/tests/com/test1036.com gdv/tests/com/test1037.com
A gdv/tests/com/test1036.com
A gdv/tests/com/test1037.com
</code>

We now apply this same methodology to the remaining unknown files. We are trying to get to a situation where the output of
<code>hgstat -du</code> is empty. The <code>-u</code> flag shows files that have appeared in the <code>gdv</code> directory
but are unknown to mercurial; these are files added or renamed by Gaussian. The <code>-d</code> flag shows files
that are known to mercurial but are no longer in the <code>gdv</code> directory.

If you run <code>hg addremove</code> on the
<code>gdv</code> directory it will try re-add things in <code>tests</code>, so be careful. There's no harm in re-adding
files you've already added, but you don't want to add files you've manually removed from the list (for example,
<code>hg addremove -s 50 gdv</code> will re-mark <code>gdv/tests/com/test1036.com</code> as a copy of
<code>gdv/tests/com/test872.com</code> again. Avoid this by using the <code>-X</code> flag to add remove:

<code>
$ hg addremove -s 40 -n -X 'glob:gdv/tests/**' gdv
</code>

If I use the above command I get a long list of files being removed and added and then at the end of list of files that
are being recorded as renames. It's worth checking the list of additions and deletions, particularly scripts in the
<code>bsd</code> directory. In this case there are some scripts, but they all check out OK. The list of renames is:

<code>
...
recording removal of gdv/l718/quairc.F as rename to gdv/dinautil/quairc.F (100% similar)
recording removal of gdv/l718/quarot.F as rename to gdv/dinautil/quarot.F (100% similar)
recording removal of gdv/l718/rotirc.F as rename to gdv/dinautil/rotirc.F (100% similar)
recording removal of gdv/l120/getlfo.F as rename to gdv/utilam/getlfo.F (100% similar)
recording removal of gdv/mdutil/lshift1.F as rename to gdv/mdutil/lshft1.F (73% similar)
recording removal of gdv/l718/difrot.F as rename to gdv/dinautil/difrot.F (97% similar)
recording removal of gdv/utilam/iiabs.F as rename to gdv/utilam/ivabs.F (82% similar)
recording removal of gdv/l1/decotp.F as rename to gdv/l1/decjob.F (77% similar)
recording removal of gdv/l1/decotp.F as rename to gdv/l1/decmth.F (47% similar)
</code>

this is where some of your Gaussian intuition comes in handy. Moving files from link subdirectories
to utility subdirectories is a common and expected thing. If we inspect the <code>gdv/utilam/iiabs.F</code> to
<code>gdv/utilam/ivabs.F</code> rename we see that the files are the same apart from two one character changes.
Inspecting the final two renames shows that the file <code>gdv/l1/decotp.F</code> is indeed being split
into two new files. Therefore I go ahead with the command without the <code>-N</code> flag.

The final task before we go ahead and commit this changeset is to check the makefile for
any new targets. If these correspond to new executables we will need to add them to the
<code>.hgignore</code> list.

<code>
$ hg diff gdv/bsd/i386.make | less
</code>

shows us that there are two new links:

<code>
...
@@ -212,7 +212,7 @@
l105.exe l106.exe l107.exe l108.exe l109.exe \
l110.exe l111.exe l112.exe l113.exe l114.exe l115.exe \
l116.exe l117.exe l118.exe l120.exe l121.exe \
- l122.exe l123.exe l124.exe l125.exe l202.exe
+ l122.exe l123.exe l124.exe l125.exe l126.exe l127.exe l202.exe

exe3: l301.exe l302.exe l303.exe l305.exe l306.exe \
l307.exe l308.exe l309.exe l310.exe l311.exe \
...
</code>

and two new utilities:

<code>
...
@@ -597,6 +597,12 @@
unfchk: unfchk.o
$(RUNF77) $(FFLAGS) -o unfchk unfchk.o $(EXTOBJ) $(EXTRAS) $(GAULIB) $(LIBS)

+gdrgen: gdrgen.o
+ $(RUNF77) $(FFLAGS) -o gdrgen gdrgen.o $(EXTOBJ) $(EXTRAS) $(GAULIB) $(LIBS)
+
+trajgen: trajgen.o
+ $(RUNF77) $(FFLAGS) -o trajgen trajgen.o $(EXTOBJ) $(EXTRAS) $(GAULIB) $(LIBS)
+
rdmat: rdmat.o
$(RUNF77) $(FFLAGS) -o rdmat rdmat.o $(EXTOBJ) $(EXTRAS) $(GAULIB) $(LIBS)
...
</code>

The links are detected by a pattern in the HG ignore file a we must add the two new utilities,
so that:

<code>
$ hg diff .hgignore
diff -r 8db6c9367c60 .hgignore
--- a/.hgignore Thu Aug 02 11:17:35 2012 +0100
+++ b/.hgignore Mon Mar 04 17:58:09 2013 +0000
@@ -32,6 +32,7 @@
gdv/gauopt
gdv/gauoptl
gdv/gautraj
+gdv/gdrgen
gdv/gdv
gdv/gdvlb
gdv/ghelp
@@ -46,5 +47,6 @@
gdv/rwfdump
gdv/tags
gdv/testrt
+gdv/trajgen
gdv/unfchk
gdv/wrmat
</code>

This then,represents Gaussian as it came from Gaussian Inc converted to our new format.
Once you are satisfied commit the revision with <code>hg ci</code>. There is a preferred
format for the
log message for these raw branch commits; look at one of the previous commits using
<code>hg log -v --rev N | less</code> for an example. The log message I will use for
this particular commit is:

<code>
Import of Gaussian version H29p. See below for details.

rm -fr gdv
cp -pr ../h29p/new-style/gdv .
hg addremove -s 50 gdv/tests
hg revert gdv/tests/com/test1036.com gdv/tests/com/test1037.com
hg add gdv/tests/com/test1036.com gdv/tests/com/test1037.com
hg addremove -s 40 -X 'glob:gdv/tests/**' gdv

and added gdrgen and trajgen to .hgignore.
</code>

The idea is that someone should be able to replicate what you've done
just by referring to the log message.

Congratulations! You've just added a revision to the raw branch.
Now tag it with the version identifier:

<code>
$ hg tag h29p-raw
</code>

Next, we must merge this version into our local branch, which represents
a compilable version of Gaussian in our new format. generally this only
requires changes to our local make files, however these changes can
require considerable effort.

<code>
$ hg up local
1995 files updated, 0 files merged, 1228 files removed, 0 files unresolved
Added tags for local versions
$ hg merge raw
merging .hgignore
merging .hgtags
4 files to edit
...
</code>

At this point this guide will get a little hazy because I used vim
and you may be using some other editor. After you type
<code>hg merge raw</code> you may be presented with a graphical merge
tool, a text based merge tool, a particular mode of your favourite text editor,
a particular mode of a hated text editor, or indeed, nothing. You can customise
the tool that is used if you want. For the purposes of this guide quit out of the
tool or editor without making any changes. If you are using vim use the
<code>:cq</code> command which will cause vim to return an error to mercurial
which tells it that the merge is not finished. Again for the purposes of this guide
run <code>hg resolve -a -u</code>
to mark all the conflicting files
as unresolved.

You should now be able to see:

<code>
$ hg resolve -l
U .hgignore
U .hgtags
U gdv/bsd/uber-i386.make
U gdv/bsd/uber-sandyb.make
</code>

Let's now resolve these four files. First <code>.hgignore</code>:

<code>
$ hg resolve .hgignore
merging .hgignore
</code>

Easy enough. If you examine the file you will see that our two
new executables have been added. Next:

<code>
$ hg resolve .hgtags
merging .hgtags
</code>

For me this command starts vimdiffwith windows showing the tags file as
it exists in the two-parent revisions of the merge plus another window
where I can edit the final merged version. Although the tags file
will generally look after itself I like to keep things neat so I
edit the file so that the knew tag, h29p, exists in the final version.

<references/>

Resgrp:comp-photo-version control new gdv

2013-02-27T18:59:30Z

Scliffor: Created page with "==Introduction== This page will explain how to import a new version of gdv into the version control system. For general help on the version control system look [[Resgrp:comp-pho..."

Resgrp:comp-photo-version control

2013-02-27T16:07:05Z

Scliffor: /* Added link to import new gdv page */

== Introduction ==

This document will teach you the basics of using mercurial, a distributed version control system (DVCS). It is particularly directed towards using mercurial with the development version of Gaussian, so the examples will use Gaussian. For more information on mercurial you can use the built-in help system <code>hg help</code>, look at the official website: http://mercurial.selenic.com/wiki/Mercurial, or check out the official O'Reilly book, the text of which is freely available at: http://hgbook.red-bean.com/.

A version control system can sometimes seem an onerous imposition unless the user understands how it is going to help them in their work, so I'll try to briefly explain. Version control is about tracking changes to data. If one has a collection of files, say a distribution of Gaussian, then one is interested in changes that have been made to those files, whether from a new distribution from Gaussian or our own modifications. We are particularly interested in cases where overlapping changes have been made. These might be updates and bug fixes from Gaussian conflicting with our own modifications to a particular link, or it might be different researchers in a group working on the same piece of code independently. A version control system will not only detect such cases but provide automated and safe methods for merging conflicting changes.

Central to the idea of a VCS is the concept of a revision or changeset. This is like a snapshot of some set of files and directories at some particular instant in their history. A repository is where a VCS stores revisions. So, using Gaussian as an example, some revisions in a particular repository might be the Gaussian source tree corresponding to Gaussian development versions H01, H08, H10, H11, etc. The data that are actually kept inside the repository are the differences, or deltas, between the revisions. This saves space compared to storing all the revisions.

== Short version ==

I've made a summary version of this document [[Resgrp:comp-photo-version control short|here]].
I've also made an [[Resgrp:comp-photo-version control example|example workflow]].
Also here is [[Resgrp:comp-photo-version control new gdv|how to import a new version of the gdv]].

== Setting up your environment (.hgrc)==
There are some one-off things you must do. Firstly, edit your login scripts (<code>.bash_profile</code> or similar) to include the command:
<code>
module load mercurial
</code>
(If you're not on an IC HPC machine mercurial is available anywhere that runs Python. It is generally available in distribution package systems, if not you can get it from http://mercurial.selenic.com/wiki/Mercurial). Note that the Redhat provided mercurial on RHEL 5 and 6 is quite old and does not, at time of writing, support the repositories I have created.

Next, create a file called <code>.hgrc</code> in your home directory and insert the following lines:
<code>
[ui]
username = Your Name <your@email.address>

[extensions]
graphlog=
</code>
(so for example, my <code>.hgrc</code> contains this line: <code>username = Simon Clifford <simon.j.clifford@gmail.com></code>).
Mercurial uses this username field to record who has changed what. (If you're not on an IC HPC machine your version of mercurial may not
have the graphlog extension. If it doesn't, don't worry, it's merely a tool for visualising changesets).

== Checking out a repository (hg clone, hg log) ==

Now let's get started and check out a repository. Once you have loaded the mercurial module you will be able to type:
<code>
tmp$ hg clone /home/gaussian-devel/example-gaussian-repo testrepo
tmp$
</code>
This will create a copy of the repository at <code>/home/gaussian-devel/example-gaussian-repo</code> and place it in a directory inside the current directory called <code>testrepo</code>. You can give a full path as the second argument, or, you can leave it off altogether in which case mercurial will clone the repository into a directory named <code>example-gaussian-repo</code>. A note here for Imperial HPC users. Copying a repository involves quite a bit of filesystem activity. I have found that on cx1 this can be quite slow on the <code>/home</code> filesystems. You may wish to try these examples in the <code>/tmp</code> filesystem which is a lot faster. Be aware, however, that this raises two potentially serious problems. The first is that <code>/tmp</code> is globally readable so before you clone you must make sure that your umask is set to <code>0077</code>. The second, of course, is that <code>/tmp</code> is periodically wiped. If you plan on working in <code>/tmp</code> there are simple ways of transferring information between repositories (<code>hg push</code> and <code>hg pull</code>) that I will cover later.

Now change into the <code>testrepo</code> directory. Inside you will see three scripts and a <code>gdv</code> directory; also some hidden files and a hidden directory. Inside the <code>gdv</code> directory you will see the Gaussian source in the [[Resgrp:comp-photo-new gdv layout|new layout]]. The hidden <code>.hg</code> directory inside the <code>testrepo</code> directory is the repository proper; you will almost never need to do anything in here. The rest of the files and directories are called the working directory; this is where you will do your work.

The first thing you might want to do is check the history of this repository to see past revisions that have been checked into it. You do this with the <code>hg log</code> command.
<code>
testrepo$ hg log | head -20
changeset: 23:1de40efd1c4b
tag: tip
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:43:13 2012 +0000
summary: Added tag h13 for changeset 6c81eb8dcbab

changeset: 22:6c81eb8dcbab
tag: h13
parent: 21:5b6ac3665a29
parent: 11:a0c273aeeb2b
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:43:08 2012 +0000
summary: Gaussian devel version H13 with our makefiles

changeset: 21:5b6ac3665a29
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:41:40 2012 +0000
summary: Added tag h12p for changeset 0ef14d7dff56
...
changeset: 1:515a93bfc3b5
branch: raw
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Sun Jan 15 11:39:10 2012 +0000
summary: Added tag h01-raw for changeset 073d6aa63ea7

changeset: 0:073d6aa63ea7
branch: raw
tag: h01-raw
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Sun Jan 15 11:39:09 2012 +0000
summary: Output from old-to-new.sh on version H01
</code>
You will see that there are various fields that may appear for each changeset. The changeset field shows two numbers separated by a colon. The second, longer, hexadecimal number is the unique identifier. A particular hex identifier will always refer to the same changeset, even in different copies of the repository. The first number is a local identifier. These local IDs refer to particular changesets in one copy of the repository; if these changesets are present in another repository they may have a different local identifier. You can use either identifier as an argument to <code>hg</code> commands. The user and date fields show who committed the change and when. The summary field shows the first line of the log entry for that revision. This means it is important to try to make the first line of your log entries a useful summary of what you have done!

To see the entry for a particular revision use the <code>-r</code> flag with a revision number, either short or long. To see more information, including the full log entry and a complete list of all files involved in the change, use the <code>-v</code> flag, e.g.:
<code>
testrepo$ hg log -r 2 -v | less
changeset: 2:9f69ed4616ad
branch: raw
tag: h08-raw
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Sun Jan 15 11:54:50 2012 +0000
files: gdv/archlib/arcvib.F gdv/archlib/brcpyw.F gdv/archlib/letset.F gdv/arctmp.F gdv/bsd/GauDiff_Compare.pm gdv/bsd/G
....
at.F gdv/utilnz/xpndnb.F gdv/utilnz/zerock.F gdv/utilnz/zerod.F gdv/wrappers/ggeev.F gdv/wrappers/lappar.F gdv/wrappers/xgetrf.F gdv/wrappers/xgetrs.F gdv/wrappers/ygeru.F gdv/wrappers/ytrsv.F
description:
Import of gdv H08. See below for details.

Result of these commands on the H01 raw rev:
rm -fr gdv
cp -pr h08/new-style-gdv .
hg add gdv/bsd/*.a
hg addremove -s 50

h08/new-style is from old-to-new.sh on freshly untarred h08.tgz
</code>
As the log entry explains, this particular revision is the change from the H01 version of Gaussian to the H08 version, so there are a lot of changed files. The description entry is intended to provide some human readable explanation of the changes. When you start committing data to the repository you should aim to at least make your log entries understandable.

As with all the commands that I will mention you can get more information by typing <code>hg help <command></code>.

== Committing changes (hg summary, hg status, hg diff) ==

Now let's make some alterations to this repository. This is quite safe; we can't break the original repository because our copy is an entirely separate clone. In fact, this is a very common working practice with mercurial. As long as the filesystem isn't slow cloning a repository can be quite quick. Indeed, cloning a repository into a destination on the same filesystem as the source makes use of hard linking which is very fast and saves space. Therefore, it is very common to clone a repository, make some alterations to it, and then decide whether to push them back to the original repository or just delete the new clone.

First, we want to know where it is we are starting from. To get a quick summary you use the <code>hg summary</code> command. This should show something like this:
<code>
testrepo$ hg summary
parent: 23:1de40efd1c4b tip
Added tag h13 for changeset 6c81eb8dcbab
branch: default
commit: (clean)
update: (current)
</code>
This shows, on the parent line, the revision that has been checked out into the working directory. Below that is the summary line from the log entry for that revision. The commit line shows if any files have been modified since the check out. The update line shows if there are any applicable newer revisions in the repository. In this example the output shows us that we checked out the revision with the local identifier 23, and unique identifier of 1de40efd1c4b, that we have modified nothing and that there are no appropriate newer revisions. This checkout occurred automatically as a part of the <code>hg clone</code> command. You can clone without checking anything out into the working directory (say to make a backup copy of the repository) by passing the <code>-U</code> flag, or you can check out a particular revision with the <code>-u REV</code> flag. See <code>hg help clone</code> for how mercurial chooses what to check out by default.

Let's say we're adding a feature to link 510. If you have something in mind go ahead and do it. For this example I have just added a few lines to the file <code>a1tdep.F</code>, and will assume that your current working directory is <code>gdv/l510</code>.

As you work you may be curious to know what it is you have changed. The command to show this is <code>hg status</code>:
<code>
testrepo$ hg status
M gdv/l510/a1tdep.F
</code>
The output shows a list of files that have been altered in some way. Here the capital M shows that the file <code>a1tdep.F</code> has been modified. By default <code>hg status</code> does not report on unchanged files. If we wish to see how <code>a1tdep.F</code> has been modified we can use the <code>hg diff</code> command:
<code>
testrepo$ hg diff a1tdep.F
diff -r 1de40efd1c4b gdv/l510/a1tdep.F
--- a/gdv/l510/a1tdep.F Wed Jan 18 14:43:13 2012 +0000
+++ b/gdv/l510/a1tdep.F Wed Jan 25 11:07:55 2012 +0000
@@ -26,6 +26,9 @@
C
C **OPTIONS FOR TDEP CODE
C
+c
+c Add super new feature
+c
iTDep=813
iTrans=822
jTDep=iTDep
</code>
This shows the output in a unified different format, refer to the man page for diff for more information.

== Adding, copying, renaming, and deleting file (hg add, hg forget, hg mv, hg rm, hg cp) ==

If you have created a new file then you must let mercurial know about its existence with the <code>hg add</code> command:
<code>
testrepo$ echo "a new subroutine" > foo.F
testrepo$ hg status
M gdv/l510/a1tdep.F
? gdv/l510/foo.F
testrepo$ hg add foo.F
testrepo$ hg status
M gdv/l510/a1tdep.F
A gdv/l510/foo.F
</code>
Here, I create a new file called <code>foo.F</code>. <code>hg status</code> shows the file with a ?, to indicate that it is unknown to the repository. I then run <code>hg add foo.F</code>, after which <code>hg status</code> shows the file with an A. This indicates that the file is scheduled to be added at the next commit. If you change your mind about the file before the next commit you can use <code>hg forget</code> to undo the add.

If you are renaming a file, including the situation where you move the file from one directory to another (e.g. from l510 to utilam) then you may use the <code>hg mv</code> command. This will actually perform the move just like the normal <code>mv</code> command. If you've already moved the file you can give <code>hg mv</code> the <code>-A</code> flag to record the rename after the fact. Similarly, the <code>hg rm</code> command removes files from the working directory and records this fact in the repository, and <code>hg cp</code> copies a file.

It is important to realise that these commands act on the working directory immediately but only affect the repository when they are committed. Also, they do not affect the repository's previous history, so removing a file and then committing that change does not affect that file's existence in previous revisions in the repository. However, <code>hg mv</code> and <code>hg cp</code> create a relationship between the source and destination files. This becomes useful when merging in changes from somewhere else. So for example, above I have made changes to <code>l510/a1tdep.F</code>. Let's say that someone else has decided to make this into a utility routine and has done the command <code>hg mv l510/a1tdep.F utilam/a1tdep.F</code>. If I choose to merge my changes with this other person's, mercurial will correctly apply the changes '''and''' move the file. If instead of <code>hg mv</code> they had used <code>hg cp</code> then my changes would be applied to both files. This means that you should only use <code>hg cp</code> when it is appropriate that changes that are recorded before the copy are applied to both files. Note that this does not mean that changes to the source file are forever applied to the destination file; this will only occur when merging a revision that does not contain a copy with a revision that does. In general, you can expect mercurial to do the right thing.

== Committing the changes (hg commit, hg tip) ==

The <code>hg commit</code> (like many <code>hg</code> commands, <code>hg</code> commit has an alias, in this case <code>hg ci</code>. I tend to use <code>hg ci</code>) command creates a new revision and records the differences between the entire working directory and the parent revision (as shown on the parent line of <code>hg summary). This will be the same as the output from <code>hg status</code>. When you run the command mercurial will open a text editor. If you wish to specify which editor is used add an <code>editor=vim</code> (for example) line to the <code>[ui]</code> part of your <code>~/.hgrc</code> file. The editor will start with a couple of empty lines and then some lines beginning with "<code>HG:</code>". These are there to give you helpful reminders of what you've changed while you write your log message and will be ignored by mercurial when you commit. If you close the editor without writing anything, or of the editor quits with an error, mercurial will abort the commit. So for our current example:
<code>
testrepo$ hg ci
... [opens vim]
HG: Enter commit message. Lines beginning with 'HG:' are removed.
HG: Leave message empty to abort commit.
HG: --
HG: user: Simon Clifford <simon.j.clifford@gmail.com>
HG: branch 'default'
HG: added gdv/l510/foo.F
HG: changed gdv/l510/a1tdep.F
~
~
</code>
For short messages you can skip the editor step by passing the <code>-m "Log message here"</code> flag.

Understanding what you are doing here is essential to knowing what to write in the log and, importantly, when to commit. There are two schools of thought. When you commit you are creating a new revision in your local repository. Since it is your repository, and as I have previously mentioned, it is very simple to create new repositories, you should commit whenever you feel like it and feel free to write cryptic log messages that only you will understand. On the other hand, you are engaged in a collaborative exercise with other people: creating what will be a single version of Gaussian that contains yours and others' modifications and Gaussian's updates. Therefore, you should only commit when your code satisfies pre-agreed group requirements, and your log message should be detailed and comprehensible to anybody who reads it, even 50 years in the future. The correct approach, of course, is to do both.

Mercurial is a distributed version control system which means that each repository is the responsibility of its owner who can feel free to commit as frequently (or infrequently) as needed. Sometimes adding a feature or removing a bug might take weeks, and you might feel that there is no point taking snapshots of the code until it works. Other modifications might proceed incrementally with naturally defined stopping points between the start and the end. Committing at these points makes perfect sense: not only does it leave a history of the modification, it provides checkpoints, versions of the code that you can retreat to without having to start again. The log messages may tersely explain what has happened since the last revision (and note there is no point listing modified, added, etc, files as this information is stored in the commit anyway) or they may contain detailed essays on how a bug arose and the steps you have taken to fix it. Information in the logs is searchable and should be thought of as both an aide memoir for yourself and a journal entry for others.

When the time comes to merge your changes into other people's repositories, you might start thinking about the quality of your commit. Does the code compile? Does it run the test suite? Can it run any test? Have you committed a test that checks the code you have added or altered? The group may decide that revisions that are being shared must answer yes to some or all of these questions. You may decide that some or all of your revisions must pass these quality checks. Or, you may be satisfied with noting in the log entry what this particular revision can or can't do (such as compile or run).

It should be said though, that if you ever think "perhaps I should check in at this point", then go for it. Revisions live in the history of your repository forever (although see <code>hg rollback</code>), but when you transfer them to other people's repositories multiple revisions can be collapsed into one, or fixed in other ways.

Returning to our example I enter a simple message. Since <code>hg log</code> only shows the first line of any log entry by default it is a good idea to make this line a summary of the rest of the message. I close the editor and the commit is done.

We can see the results in the repository with an <code>hg log</code> command (the <code>-l 2</code> flag shows the last two revisions):
<code>
testrepo$ hg log -l 2
changeset: 24:13ecff0136c2
tag: tip
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 25 17:13:00 2012 +0000
summary: A test message.

changeset: 23:1de40efd1c4b
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:43:13 2012 +0000
summary: Added tag h13 for changeset 6c81eb8dcbab
</code>
Alternatively we can use the <code>hg tip</code> command to show the most recently added repository, whether by ourselves, or by pulling from another repository (see later).

== Fixing mistakes (hg revert, hg rollback) ==

You can return one or more files, or the entire repository, to the state they were in when you last checked them out with in the <code>hg revert</code> command. Let's make an ill-advised alteration to our now committed change to <code>a1tdep.F</code>:
<code>
testrepo$ sed -i 's/super/super duper/' a1tdep.F
testrepo$ hg status
M gdv/l510/a1tdep.F
testrepo$ hg diff a1tdep.F
diff -r 13ecff0136c2 gdv/l510/a1tdep.F
--- a/gdv/l510/a1tdep.F Wed Jan 25 17:13:00 2012 +0000
+++ b/gdv/l510/a1tdep.F Wed Jan 25 17:23:42 2012 +0000
@@ -27,7 +27,7 @@
C **OPTIONS FOR TDEP CODE
C
c
-c Add super new feature
+c Add super duper new feature
c
iTDep=813
iTrans=822
testrepo$ hg revert a1tdep.F
testrepo$ hg status
? gdv/l510/a1tdep.F.orig
testrepo$ hg diff a1tdep.F
testrepo$ rm a1tdep.*
testrepo$ hg status
! gdv/l510/a1tdep.F
testrepo$
</code>
Here I use revert to undo the modifications to a file. Notice that <code>hg revert</code> leaves a copy of the modified file in <code>a1tdep.F.orig</code>. This shows up in <code>hg status</code> as an unknown file ("?"). Also notice that while trying to delete the .orig file I have accidentally deleted <code>a1tdep.F</code>, which now shows up in <code>hg status</code> as a missing file ("!"). I can revert this mistake too:
<code>
testrepo$ hg revert a1tdep.F
testrepo$ hg status
testrepo$ ls a1tdep.F
a1tdep.F
testrepo$
</code>
<code>hg revert</code> can also be used to cancel scheduled adds, removes, copies, and renames.

If you have just committed a revision and then change your mind you may be able to undo the effect with <code>hg rollback</code>. Doing this for our example gives us:
<code>
testrepo$ hg rollback
repository tip rolled back to revision 23 (undo commit)
working directory now based on revision 23
testrepo$ hg status
M gdv/l510/a1tdep.F
A gdv/l510/foo.F
testrepo$
</code>
This removes the revision we just checked in from the repository but does not alter the current working directory. If we choose, we could now use <code>hg revert</code> to restore these to their revision 23 state. There are two important caveats with <code>hg rollback</code>: it can only remove the last checked in revision, and that it is usually pointless to rollback a revision that has already been pushed or pulled (see later) into somebody else's repository, as we can only affect our repository. For the purposes of this example, I will once more check in the changes I have made:
<code>
testrepo$ hg ci -m 'A test message'
testrepo$ hg tip
changeset: 24:13ecff0136c2
tag: tip
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 25 17:38:52 2012 +0000
summary: A test message
testrepo$
</code>

== Collaboration (hg incoming, hg pull, hg outgoing, hg push, hg update, hg parent) ==

The tools described are already quite useful, and form the basis of the very earliest revision control systems. The benefits of being able to detect what you've changed, and how you've changed it, being able to undo the changes selectively, and being able to take a snapshot of your work at any point of your choosing should be apparent. The true power, however, of a modern version control system is how it mediates different streams of changes, particularly those from other users. Note that a DVCS does not solve problems of merging and so on, but provides you, the user, with tools to solve them.

Revisions in the repository form a directed acyclic graph (DAG). Every revision apart from the first has at least one parent revision and may have zero or more child revisions. A revision with no children is called a ''head''. Generally development takes place at a head of the repository: checking in a new revision onto a head creates and consumes a head. However, it is possible to add a child to any revision, possibly creating a new head. This is known as a branch, and may be named or unnamed. A revision with children may still be a branch head if it has no children on its branch.

There are various ways of managing branches. You can have a few repositories with many branches, named and unnamed. Or you can have many repositories with largely unbranched graphs; it's up to you. Since mercurial uses heads as the default targets for some of its operations the latter approach is probably best for beginners as it makes operations within each repository simpler.

This is probably best illustrated with an example. I'll make a copy of the original repository, make a different change, and then merge the changes. Change directory out of the <code>testrepo</code> repository and type:
<code>
tmp$ hg clone /home/gaussiandevel/example-gaussian-repo testmerge
updating to branch default
13427 files updated, 0 files merged, 0 files removed, 0 files unresolved
tmp$
</code>

Now we go into the new repository and make some modifications. This time I alter the files <code>a1tdep.F</code> and <code>a2tdep.F</code>, and add a file <code>bar.F</code> in the <code>l510</code> directory:
<code>
…[alterations to a1tdep.F, a2tdep.F, and bar.F]...
testmerge$ hg add gdv/l510/bar.F
testmerge$ hg diff
diff -r 1de40efd1c4b gdv/l510/a1tdep.F
--- a/gdv/l510/a1tdep.F Wed Jan 18 14:43:13 2012 +0000
+++ b/gdv/l510/a1tdep.F Thu Jan 26 12:15:13 2012 +0000
@@ -26,6 +26,9 @@
C
C **OPTIONS FOR TDEP CODE
C
+c
+c Some new feature
+c
iTDep=813
iTrans=822
jTDep=iTDep
diff -r 1de40efd1c4b gdv/l510/a2tdep.F
--- a/gdv/l510/a2tdep.F Wed Jan 18 14:43:13 2012 +0000
+++ b/gdv/l510/a2tdep.F Thu Jan 26 12:15:13 2012 +0000
@@ -16,6 +16,9 @@
C
C **DETERMINE IF WE RUN THE TIMEDEP CODE
C
+c
+c Super duper feature here too
+c
LenTst=0
CALL FILEIO(11,jTDep,LenTst,0,0)
IF (LenTst.EQ.0.OR.iopv.NE.23)RETURN
diff -r 1de40efd1c4b gdv/l510/bar.F
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/gdv/l510/bar.F Thu Jan 26 12:15:13 2012 +0000
@@ -0,0 +1,1 @@
+Stuff in here
testmerge$
testmerge$ hg ci -m 'Added second new feature'
testmerge$ hg log -l 2
changeset: 24:457db23c0b1a
tag: tip
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Thu Jan 26 12:15:54 2012 +0000
summary: Added second new feature

changeset: 23:1de40efd1c4b
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:43:13 2012 +0000
summary: Added tag h13 for changeset 6c81eb8dcbab
testmerge$
</code>
The situation now is that we have two repositories where there are revisions whose parent is revision 23 (23:1de40efd1c4b, in fact). This sort of scenario might arise because we have been working on two different features in two different repositories, or it may be that two users have been working separately. At some point you may wish to merge the work. It's up to you when and how you do this: you may wish to merge in bug fixes quite frequently, and incorporate brand-new features much less frequently. You can of course clone another repository in which to do the merge so that if it isn't satisfactory you can just delete the repository.

First we must pull the revisions from one repository to another. We use the <code>hg incoming</code> and <code>pull</code> commands to do this:
<code>
testmerge$ hg incoming ../testrepo
comparing with ../testrepo
searching for changes
changeset: 24:13ecff0136c2
tag: tip
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 25 17:39:50 2012 +0000
summary: A test message
testmerge$ hg pull ../testrepo
pulling from ../testrepo
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 2 changes to 2 files (+1 heads)
(run 'hg heads' to see heads, 'hg merge' to merge)
testmerge$ hg log -l 3
changeset: 25:13ecff0136c2
tag: tip
parent: 23:1de40efd1c4b
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 25 17:39:50 2012 +0000
summary: A test message

changeset: 24:457db23c0b1a
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Thu Jan 26 12:15:54 2012 +0000
summary: Added second new feature

changeset: 23:1de40efd1c4b
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:43:13 2012 +0000
summary: Added tag h13 for changeset 6c81eb8dcbab
testmerge$
</code>
The <code>hg incoming</code> command takes another repository as its argument and shows a list of revisions that are not present in the current repository. The <code>hg pull</code> command brings those revisions over into the current repository. The <code>hg log</code> command shows changeset 25 in the testmerge repository corresponds to number 24 in the original <code>testrepo</code> repository. Note the long ID is the same in both cases. The <code>hg pull</code> command reports that it has created a new head. We can see this clearly with the <code>hg glog</code> command (from the graphlog extension). The <code>-r 23:</code> flag tells <code>glog</code> to show revisions 23 and greater, for clarity:
<code>
testmerge$ hg glog -r 23:
o changeset: 25:13ecff0136c2
| tag: tip
| parent: 23:1de40efd1c4b
| user: Simon Clifford <simon.j.clifford@gmail.com>
| date: Wed Jan 25 17:39:50 2012 +0000
| summary: A test message
|
| @ changeset: 24:457db23c0b1a
|/ user: Simon Clifford <simon.j.clifford@gmail.com>
| date: Thu Jan 26 12:15:54 2012 +0000
| summary: Added second new feature
|
o changeset: 23:1de40efd1c4b
| user: Simon Clifford <simon.j.clifford@gmail.com>
| date: Wed Jan 18 14:43:13 2012 +0000
| summary: Added tag h13 for changeset 6c81eb8dcbab
|
testmerge$
</code>
The <code>hg push</code> command can also be used to copy revisions from one repository to another. It works much as you would expect, except that by default it does not permit the creation of new heads in the destination repository. The idea is that you tend to pull into your own repository, where you're expected to know what you're doing, while you might be pushing into someone else's repository, where creating a new head might cause confusion. <code>hg outgoing</code> is the corresponding analogue to the <code>hg incoming</code> command.

In our example we now have a situation where we have two heads. This can arise from revisions being created in different repositories and then pulled together, as shown. Alternatively you can just create new heads in one repository. A disadvantage of the single repository technique is that if you decide branch is going nowhere and elect not to merge or proceed with it, it still remains in your repository. With multiple repositories you can just delete the offending repository.

== Merging two sets of changes (hg merge, hg resolve) ==

Let's say that I decide that these two features will work together and I want to merge the two branches. I can use the <code>hg up</code> (<code>hg update</code>) command to update the current working directory to a particular revision in the repository. If no revision is specified it will update to the current branch head. The revision updated to becomes the parent (as shown by <code>hg summary</code> or <code>hg parent</code>) of the working directory. If there are modified files in the working directory the update may attempt a merge. The revision of the working directory when you start to merge is called the base of the merge. This is important if the two revisions you are merging are on different branches, because the newly merged revision will stay on the base branch. Note that the <code>hg pull</code> command does not update the working directory, so in this case it will still be at revision 24. Let's update to revision 25 and then merge the changes from revision 24:

[Which revision to merge from: in general you merge changes into whatever is your current baseline. So, if you're merging the latest updates from Gaussian (from H10 to H11 for example) into your code, you update to your code and merge in the H11 revision. If you're merging in your changes into the soon to be sent to Gaussian group development version you'd start with that and merge in the revision(s) containing your changes]
<code>
testmerge$ hg up 25
3 files updated, 0 files merged, 1 files removed, 0 files unresolved
testmerge$ hg merge 24
merging gdv/l510/a1tdep.F
warning: conflicts during merge.
merging gdv/l510/a1tdep.F failed!
2 files updated, 0 files merged, 0 files removed, 1 files unresolved
use 'hg resolve' to retry unresolved file merges or 'hg update -C .' to abandon
testmerge$
</code>
The software automatically brings in the changes to foo.F, bar.F, and a2tdep.F (as <code>hg status</code> will show). However, in our contrived example <code>a1tdep.F</code> is subject to changes from both revisions. Mercurial will attempt to merge automatically, and in this case it fails. If you edit <code>a1tdep.F</code> you will see it contains the lines:
<code>
<<<<<<< local
c Add super new feature
=======
c Some new feature
>>>>>>> other
</code>
which were inserted during the failed merge. There is also an <code>a1tdep.F.orig</code> file created as a result of the failed merge. As the output from <code>hg merge</code> says you can either now resolve this failed merge or use <code>hg update -C</code> to undo it (the <code>.orig</code> file will remain).

Since we want to resolve the merge we should edit <code>a1tdep.F</code> so that it works. At a bare minimum we will have to remove the “<code><<<<<<< local</code>”, “<code>=======</code>”, and “<code>>>>>>>> other</code>” marker lines. In a more complex situation we might have to completely rewrite this and other files. This is what I mean when I say that mercurial only provides tools to do merging. Even a completely successful merge, from mercurial's point of view, may be completely bogus code.

There are tools available to make this task easier. For example I use <code>vimdiff</code>, a part of the popular <code>vim</code> package. To enable this I have altered my <code>~/.hgrc</code> file to look like this:
<code>
[ui]
merge = vimdiff
username = Simon Clifford <simon.j.clifford@gmail.com>

[extensions]
graphlog=

[merge-tools]
vimdiff.executable = vim
vimdiff.args = -d $base $local $output $other +close +close
</code>
as per http://mercurial.selenic.com/wiki/MergingWithVim. The page http://mercurial.selenic.com/wiki/MergeProgram contains instructions for other tools, including Emacs, and graphical tools. If you edit your <code>~/.hgrc</code> file in this way then you now can type <code>hg resolve --all</code> to run your chosen tool on all unresolved files. Note that by default if your chosen tool exits without an error then <code>hg resolve</code> will regard that file as resolved. To cause <code>vim</code> to exit with an error use <code>:cq</code>. See <code>hg help resolve</code> for more details.

In the example my resolution is to have the relevant lines in <code>a1tdep.F</code> look like this:
<code>
c
c Add super new feature
c Some new feature
c
</code>

Once you have resolved all the files that mercurial thinks need fixing you should check that the final merged result makes sense. Checking that it compiles, or runs tests appropriate to both of the original revisions, for example. When you are satisfied, you should check in the merged revision:

<code>
testmerge$ hg ci -m 'Merged super and duper features'

testmerge$ hg tip

changeset: 26:857f81e59d66
tag: tip
parent: 25:13ecff0136c2
parent: 24:457db23c0b1a
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Thu Jan 26 13:10:52 2012 +0000
summary: Merged super and new features

testmerge$

</code>

You can see that the new revision has ''two'' parent revisions; a merge always consumes a head. This is clear from the output of:

<code>
testmerge$ hg glog -l 4
o changeset: 26:857f81e59d66
|\ tag: tip
| | parent: 25:13ecff0136c2
| | parent: 24:457db23c0b1a
| | user: Simon Clifford <simon.j.clifford@gmail.com>
| | date: Thu Jan 26 13:10:52 2012 +0000
| | summary: Merged super and new features
| |
| o changeset: 25:13ecff0136c2
| | parent: 23:1de40efd1c4b
| | user: Simon Clifford <simon.j.clifford@gmail.com>
| | date: Wed Jan 25 17:39:50 2012 +0000
| | summary: A test message
| |
o | changeset: 24:457db23c0b1a
|/ user: Simon Clifford <simon.j.clifford@gmail.com>
| date: Thu Jan 26 12:15:54 2012 +0000
| summary: Added second new feature
|
o changeset: 23:1de40efd1c4b
| user: Simon Clifford <simon.j.clifford@gmail.com>
| date: Wed Jan 18 14:43:13 2012 +0000
| summary: Added tag h13 for changeset 6c81eb8dcbab
|
testmerge$
</code>

Another example of merging here? Perhaps Lee's code merging in changes on official version.

== Tags and named branches (hg tag, hg tags, hg branch, hg branches) ==

It is often useful to give a name to a particular revision. This might be a changeset that corresponds to a given version of the code, or it might simply be a revision that marks a particular milestone. In the <code>example-gaussian-repo</code> you have probably noticed there are revisions in the history with descriptions like "Added tag h13 for changeset 6c81eb8dcbab". These are revisions corresponding to particular versions of the Gaussian development version, such as H01, H10, H13, etc. You can see a list of the tags in a repository by doing:
<code>
testmerge$ hg tags
tip 26:857f81e59d66
h13 22:6c81eb8dcbab
h12p 20:0ef14d7dff56
h11 18:7daba638a830
h10 16:e1d0af4e84d9
h08 14:656073c38db9
h01 12:737d1720e79a
h13-raw 10:b039f5c274e2
h12p-raw 8:522a8fe79d22
h11-raw 6:a28b789a9555
h10-raw 4:356081dab79d
h08-raw 2:9f69ed4616ad
h01-raw 0:073d6aa63ea7
testmerge$
</code>
(the <code>tip</code> tag is automatically assigned to the most recently checked in revision). You may use a tag name anywhere where you might use a revision number, so for example, to check out the changeset corresponding to the official Gaussian development release H10, you would type:
<code>
testrepo$ hg up h10
13887 files updated, 0 files merged, 0 files removed, 0 files unresolved
testrepo$
</code>

(this will look different if you have unchecked-in modifications in your working directory). You could also pass h10 to the <code>hg clone -u</code> flag, and so on.
You tag a revision with the <code>hg tag</code> command. Issue it in a working directory and it will tag the revision that is the parent of the working directory. Tags become part of the repository, that is they are shared during <code>hg pull</code> and <code>hg push</code>, so choose wisely. If you'd like to have tags that are only part of the local repository, use the <code>-l</code> flag to <code>hg tag</code>.

Tags are stored in an <code>.hgtags</code> file in the root directory of the repository. Running <code>hg tag</code> alters this file and then commits the change to the repository. Sometimes you can see conflicts in this file during merges. I normally just resolve them by keeping as much information in the merged file as possible.

The <code>hg branches</code> command can be used to show the named branches in the repository:
<code>
testrepo$ hg branches
default 24:13ecff0136c2
raw 11:a0c273aeeb2b (inactive)
testrepo$
</code>
Here I have two named branches, raw and default (the default branch is, er, the default). If you examine the full output of <code>hg glog</code>, which is quite long and I won't reproduce it here, you'll see, starting at the bottom, revisions 0 to 11 are all in the raw branch. I have created and named this branch to contain the official Gaussian development code as processed by the <code>old-to-new.sh</code> script, without any of our makefiles or other alterations. The default branch branches off from revision 1 and starts at revision 12. Its descendants, revisions 12 to 23, are also in the default branch. These were created by starting at revision 1, adding our makefiles and modifications to the build system, renaming this branch to the default branch, and then committing revision 12. I tagged revision 12 as "H01", which created revision 13. I then merged revision 3, fixed the conflicts, and committed revision 14, and so on. A more detailed guide on how to import a new official version of gdv exists in LINKY.

You can use branch names just like tags wherever mercurial expects a revision identifier. If you use a branch name mercurial will attempt to give you the revision that is most appropriate, this will usually be the newest head on that branch.

To create a new branch you type <code>hg branch <branchname></code>. This will take effect when you commit the working directory. As I've previously mentioned a simpler alternative to having branches, named or otherwise, in your repository is to have multiple repositories.

== Ignoring files ==

You may be wondering how the system will work when you start compiling files. Won't all the <code>.o</code>, <code>.a</code> and <code>.exe</code> files start showing up in the output to <code>hg status</code>? The answer is that they will not, because I've told mercurial to ignore such files. This information is stored in the <code>.hgignore</code> file in the root of the repository. It's quite long but it starts like this:
<code>
syntax: glob
gdv.make
*.o
*.lo
*.a
*.exe
*.exel
.*.swp
*~
*.log
*.log.gz
*.chk
gdv/arc
...
</code>
Any file that matches either the shell patterns or the file names in <code>.hgignore</code> is ignored by mercurial. This means it does not show up to <code>hg status</code> or <code>hg commit</code>. <code>hg add</code>, <code>hg rm</code>, and so on will ignore it unless it is explicitly mentioned on the command line. So <code>hg add gdv</code> will add all non-ignored files in the gdv directory, while <code>hg add gdv/*</code> will add all the files in the gdv directory.

The <code>.hgignore</code> file is tracked, so please only commit changes to it if you think they will be useful to everyone.
== Other commands ==

There are quite a few other commands available to mercurial, I will cover just a few here, if you're interested in becoming a power user refer to the documentation I have mentioned.

=== hg addremove ===
You can use this command when you have large numbers of files to add and remove. It also includes the <code>-s</code> flag to attempt to detect when files have been moved or copied. Using this flag does require some subtlety, as it's not helpful to mark files as moved or copied unless they have been. Also note that it is possible to inadvertently include ignored files in a mass <code>hg add</code> or <code>hg addremove</code>; check what's being added before you commit.

=== hg grep and hg locate ===
Similar to the standard UNIX grep and locate commands, but cognisant of the mercurial repository.

=== hg init ===
Creates a new mercurial repository in the current directory

=== hg serve ===
This starts a mini web server serving information about the repository. I would strongly recommend you do not use this while working on the Gaussian development version as it would be all too easy to allow any user on the system to view all of the files in your repository. I only mention it here because some of the other documentation may refer to it.

Resgrp:comp-photo-version control-local-dev-master

2013-01-21T13:59:41Z

Scliffor:

This page will show the current state of the group's master dev repository <code>/home/gaussian-devel/repositories/local-dev-master</code>.
It will document what code is currently considered to be accepted as ready to use.

The format is explained [[#Formatting|here]]. You don't have to describe each revision, just
whatever the tip is at the time, and whatever might be (or have been) made into a
compiled version.
==The revisions==
The most recent revision (which should be the tip) should always be on top. Note
that the "tip" tag may appear incorrectly below. Also note that the local revision
numbers (e.g. 56 in changeset: 56:e359f0e1aa5c) may differ in other repositories. If
in doubt, use the hex number.

===changeset: 73:923d6c51169a===
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Fri Jan 18 17:11:08 2013 +0000
summary: Fixed bug where ONIOM jobs would fail with:

Updated version fixing a bug where ONIOM 2nd derivative jobs would fail
with a message like:
ONIOM: calculating second derivatives.
NInts<0 in InToWP.
Error termination via Lnk1e in /home/gaussian-devel/versions/rev-a7f8ac5c6a00-i386-pg12.9/gdv/l120.exe at Fri Jan 11 18:06:43 2013.

===changeset: 72:a7f8ac5c6a00===
parent: 70:f87f732e26fb
parent: 71:69b842c84566
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Nov 21 14:12:01 2012 +0000
summary: Merged in more updates to makefiles (h23)

DEPRECATED: see revision [[#changeset: 73:923d6c51169a|73:923d6c51169a]].

This revision contains the [[Resgrg:comp-photo-version control-codes#Shaopeng|Shaopeng matrix element code]], the [[Resgrg:comp-photo-version control-codes#Seamfollowing|seam following code]],
and the [[Resgrg:comp-photo-version control-codes#ONIOM Freq|ONIOM frequency analysis code]]. They are all merged to be compatible with H23.

I have also added into the repository test jobs for each of these local changes. These are stored in
an upper directory called <code>local-tests</code>.

===changeset: 56:e359f0e1aa5c===
parent: 55:f6c5e2290dc4
parent: 53:c7f36daff923
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Oct 10 14:56:24 2012 +0100
summary: Merged Shaopeng, seamfollowing, olddirect (H23)

DEPRECATED: Do not use, likely to crash. See rev [[#changeset: 57:0c20d50c4185|57:0c20d50c4185]].

The merge of the three features. Based off H23.
===changeset: 55:f6c5e2290dc4===
parent: 54:0c83655a3822
parent: 41:c8a317a36a4e
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Oct 10 14:55:17 2012 +0100
summary: Merged olddirect code to H23

DEPRECATED: Do not use, likely to crash. See rev [[#changeset: 57:0c20d50c4185|57:0c20d50c4185]].

Simon's updates and fixes for the semi-direct matrix element code.
Based off H23.
===changeset: 53:c7f36daff923===
parent: 52:e11a2fc5b5a8
parent: 41:c8a317a36a4e
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Oct 10 12:38:05 2012 +0100
summary: Shaopeng + seamfollowing, merged to H23.

DEPRECATED: Do not use, likely to crash. See rev [[#changeset: 57:0c20d50c4185|57:0c20d50c4185]].

The Shaopeng code and seamfollowing code as below, updated to H23.
===changeset: 51:dcd851f8dc57===
parent: 50:c0a9f878ea3e
parent: 39:c90162c4fbe6
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Tue Oct 09 22:15:18 2012 +0100
summary: Updated seamfollowing code to H21

DEPRECATED: Do not use, likely to crash. See rev [[#changeset: 57:0c20d50c4185|57:0c20d50c4185]].

This is the imported seam following code updated to H21.
===changeset: 49:bda0e1259a36===
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Jul 02 15:04:35 2012 +0100
summary: Large modification and cleanup of Shaopeng code. Below for details.

This contains the [[Resgrg:comp-photo-version control-codes#Shaopeng|Shaopeng matrix element code]]
with most of Simon's modifications
and bug fixes in place. It's based off version H21
===changeset: 41:c8a317a36a4e===
tag: h23
parent: 40:cfd3ea6fcdff
parent: 25:7a8a763281d4
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:45:00 2012 +0100
summary: Merged in H23

This is official Gaussian version H23.

===changeset: 39:c90162c4fbe6===
tag: h21
parent: 38:ef0c2dff2d79
parent: 20:dd575870f775
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:44:00 2012 +0100
summary: Merged in H21

This is official Gaussian version H21.
===changeset: 37:e972407aab01===
tag: h13
parent: 36:9d36f9fe483b
parent: 17:770721638a7a
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:43:06 2012 +0100
summary: Merged in H13

This is official Gaussian version H13.
===changeset: 35:10679662e3a3===
tag: h12p
parent: 34:3ce1603d2c6c
parent: 16:427978d67ed6
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:42:11 2012 +0100
summary: Merged in H12p

This is official Gaussian version H12p.
===changeset: 33:a0e0a314db85===
tag: h11
parent: 32:3732a6888cc4
parent: 15:27066b86194e
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:39:49 2012 +0100
summary: Merged in H11

This is official Gaussian version H11.
===changeset: 31:7d8a337a7324===
tag: h10
parent: 30:86c2a0b6f977
parent: 14:62f983246ba7
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:39:11 2012 +0100
summary: Merged in H10

This is official Gaussian version H10.
===changeset: 29:56fa127c847c===
tag: h08
parent: 28:877924b27ee3
parent: 13:903ac662c7a2
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:38:04 2012 +0100
summary: Merged in H08

This is official Gaussian version H08.
===changeset: 27:a05e10e365c2===
tag: h01
parent: -1:000000000000
parent: 12:ae8da988f728
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:35:05 2012 +0100
summary: Fixed bug where code compiled with PGI >= 12 would fail with thread error.

This is official Gaussian version H01.

==Formatting==
The format is:

===changeset: 56:e359f0e1aa5c===
tag: tip
parent: 55:f6c5e2290dc4
parent: 53:c7f36daff923
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Oct 10 14:56:24 2012 +0100
summary: Merged Shaopeng, seamfollowing, olddirect (H23)

Text explaining what this revision contains,
e.g. all of the below plus merged up to H23

In raw form that's:
<code>
===changeset: 56:e359f0e1aa5c===
tag: tip
parent: 55:f6c5e2290dc4
parent: 53:c7f36daff923
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Oct 10 14:56:24 2012 +0100
summary: Merged Shaopeng, seamfollowing, olddirect (H23)

Text explaining what this revision contains,
e.g. all of the below plus merged up to H23
</code>

This is the output from <code>hg log</code> with the changeset line made into a header, a space
added to the start of each following line, and more
explanatory text. '''NB, it's important to add more explanation!'''

Resgrp:comp-photo-version control-local-dev-master

2013-01-21T13:58:51Z

Scliffor:

This page will show the current state of the group's master dev repository <code>/home/gaussian-devel/repositories/local-dev-master</code>.
It will document what code is currently considered to be accepted as ready to use.

The format is explained [[#Formatting|here]]. You don't have to describe each revision, just
whatever the tip is at the time, and whatever might be (or have been) made into a
compiled version.
==The revisions==
The most recent revision (which should be the tip) should always be on top. Note
that the "tip" tag may appear incorrectly below. Also note that the local revision
numbers (e.g. 56 in changeset: 56:e359f0e1aa5c) may differ in other repositories. If
in doubt, use the hex number.

===changeset: 73:923d6c51169a===
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Fri Jan 18 17:11:08 2013 +0000
summary: Fixed bug where ONIOM jobs would fail with:

Updated version fixing a bug where ONIOM 2nd derivative jobs would fail
with a message like:
ONIOM: calculating second derivatives.
NInts<0 in InToWP.
Error termination via Lnk1e in /home/gaussian-devel/versions/rev-a7f8ac5c6a00-i386-pg12.9/gdv/l120.exe at Fri Jan 11 18:06:43 2013.

===changeset: 72:a7f8ac5c6a00===
parent: 70:f87f732e26fb
parent: 71:69b842c84566
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Nov 21 14:12:01 2012 +0000
summary: Merged in more updates to makefiles (h23)

DEPRECATED: see revision [[#changeset: 73:923d6c51169a|73:923d6c51169a]].

This revision contains the [[Resgrg:comp-photo-version control-codes#Shaopeng|Shaopeng matrix element code]], the [[Resgrg:comp-photo-version control-codes#Seamfollowing|seam following code]],
and the [[Resgrg:comp-photo-version control-codes#ONIOM Freq|ONIOM frequency analysis code]]. They are all merged to be compatible with H23.

I have also added into the repository test jobs for each of these local changes. These are stored in
an upper directory called <code>local-tests</code>.

===changeset: 56:e359f0e1aa5c===
parent: 55:f6c5e2290dc4
parent: 53:c7f36daff923
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Oct 10 14:56:24 2012 +0100
summary: Merged Shaopeng, seamfollowing, olddirect (H23)

DEPRECATED: Do not use, likely to crash. See rev [[#changeset: 57:0c20d50c4185|57:0c20d50c4185]].

The merge of the three features. Based off H23.
===changeset: 55:f6c5e2290dc4===
parent: 54:0c83655a3822
parent: 41:c8a317a36a4e
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Oct 10 14:55:17 2012 +0100
summary: Merged olddirect code to H23

DEPRECATED: Do not use, likely to crash. See rev [[#changeset: 57:0c20d50c4185|57:0c20d50c4185]].

Simon's updates and fixes for the semi-direct matrix element code.
Based off H23.
===changeset: 53:c7f36daff923===
parent: 52:e11a2fc5b5a8
parent: 41:c8a317a36a4e
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Oct 10 12:38:05 2012 +0100
summary: Shaopeng + seamfollowing, merged to H23.

DEPRECATED: Do not use, likely to crash. See rev [[#changeset: 57:0c20d50c4185|57:0c20d50c4185]].

The Shaopeng code and seamfollowing code as below, updated to H23.
===changeset: 51:dcd851f8dc57===
parent: 50:c0a9f878ea3e
parent: 39:c90162c4fbe6
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Tue Oct 09 22:15:18 2012 +0100
summary: Updated seamfollowing code to H21

DEPRECATED: Do not use, likely to crash. See rev [[#changeset: 57:0c20d50c4185|57:0c20d50c4185]].

This is the imported seam following code updated to H21.
===changeset: 49:bda0e1259a36===
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Jul 02 15:04:35 2012 +0100
summary: Large modification and cleanup of Shaopeng code. Below for details.

This contains the [[Resgrg:comp-photo-version control-codes#Shaopeng|Shaopeng matrix element code]]
with most of Simon's modifications
and bug fixes in place. It's based off version H21
===changeset: 41:c8a317a36a4e===
tag: h23
parent: 40:cfd3ea6fcdff
parent: 25:7a8a763281d4
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:45:00 2012 +0100
summary: Merged in H23

This is official Gaussian version H23.

===changeset: 39:c90162c4fbe6===
tag: h21
parent: 38:ef0c2dff2d79
parent: 20:dd575870f775
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:44:00 2012 +0100
summary: Merged in H21

This is official Gaussian version H21.
===changeset: 37:e972407aab01===
tag: h13
parent: 36:9d36f9fe483b
parent: 17:770721638a7a
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:43:06 2012 +0100
summary: Merged in H13

This is official Gaussian version H13.
===changeset: 35:10679662e3a3===
tag: h12p
parent: 34:3ce1603d2c6c
parent: 16:427978d67ed6
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:42:11 2012 +0100
summary: Merged in H12p

This is official Gaussian version H12p.
===changeset: 33:a0e0a314db85===
tag: h11
parent: 32:3732a6888cc4
parent: 15:27066b86194e
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:39:49 2012 +0100
summary: Merged in H11

This is official Gaussian version H11.
===changeset: 31:7d8a337a7324===
tag: h10
parent: 30:86c2a0b6f977
parent: 14:62f983246ba7
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:39:11 2012 +0100
summary: Merged in H10

This is official Gaussian version H10.
===changeset: 29:56fa127c847c===
tag: h08
parent: 28:877924b27ee3
parent: 13:903ac662c7a2
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:38:04 2012 +0100
summary: Merged in H08

This is official Gaussian version H08.
===changeset: 27:a05e10e365c2===
tag: h01
parent: -1:000000000000
parent: 12:ae8da988f728
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:35:05 2012 +0100
summary: Fixed bug where code compiled with PGI >= 12 would fail with thread error.

This is official Gaussian version H01.

==Formatting==
The format is:

===changeset: 56:e359f0e1aa5c===
tag: tip
parent: 55:f6c5e2290dc4
parent: 53:c7f36daff923
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Oct 10 14:56:24 2012 +0100
summary: Merged Shaopeng, seamfollowing, olddirect (H23)

Text explaining what this revision contains,
e.g. all of the below plus merged up to H23

In raw form that's:
<code>
===changeset: 56:e359f0e1aa5c===
tag: tip
parent: 55:f6c5e2290dc4
parent: 53:c7f36daff923
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Oct 10 14:56:24 2012 +0100
summary: Merged Shaopeng, seamfollowing, olddirect (H23)

Text explaining what this revision contains,
e.g. all of the below plus merged up to H23
</code>

This is the output from <code>hg sum</code> with the changeset line made into a header, a space
added to the start of each following line, and more
explanatory text. '''NB, it's important to add more explanation!'''

Resgrp:comp-photo-version control-local-dev-master

2012-11-21T19:58:13Z

Scliffor:

This page will show the current state of the group's master dev repository <code>/home/gaussian-devel/repositories/local-dev-master</code>.
It will document what code is currently considered to be accepted as ready to use.

The format is explained [[#Formatting|here]]. You don't have to describe each revision, just
whatever the tip is at the time, and whatever might be (or have been) made into a
compiled version.
==The revisions==
The most recent revision (which should be the tip) should always be on top. Note
that the "tip" tag may appear incorrectly below. Also note that the local revision
numbers (e.g. 56 in changeset: 56:e359f0e1aa5c) may differ in other repositories. If
in doubt, use the hex number.

===changeset: 72:a7f8ac5c6a00===
parent: 70:f87f732e26fb
parent: 71:69b842c84566
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Nov 21 14:12:01 2012 +0000
summary: Merged in more updates to makefiles (h23)

This revision contains the [[Resgrg:comp-photo-version control-codes#Shaopeng|Shaopeng matrix element code]], the [[Resgrg:comp-photo-version control-codes#Seamfollowing|seam following code]],
and the [[Resgrg:comp-photo-version control-codes#ONIOM Freq|ONIOM frequency analysis code]]. They are all merged to be compatible with H23.

I have also added into the repository test jobs for each of these local changes. These are stored in
an upper directory called <code>local-tests</code>.

===changeset: 56:e359f0e1aa5c===
parent: 55:f6c5e2290dc4
parent: 53:c7f36daff923
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Oct 10 14:56:24 2012 +0100
summary: Merged Shaopeng, seamfollowing, olddirect (H23)

DEPRECATED: Do not use, likely to crash. See rev [[#changeset: 57:0c20d50c4185|57:0c20d50c4185]].

The merge of the three features. Based off H23.
===changeset: 55:f6c5e2290dc4===
parent: 54:0c83655a3822
parent: 41:c8a317a36a4e
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Oct 10 14:55:17 2012 +0100
summary: Merged olddirect code to H23

DEPRECATED: Do not use, likely to crash. See rev [[#changeset: 57:0c20d50c4185|57:0c20d50c4185]].

Simon's updates and fixes for the semi-direct matrix element code.
Based off H23.
===changeset: 53:c7f36daff923===
parent: 52:e11a2fc5b5a8
parent: 41:c8a317a36a4e
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Oct 10 12:38:05 2012 +0100
summary: Shaopeng + seamfollowing, merged to H23.

DEPRECATED: Do not use, likely to crash. See rev [[#changeset: 57:0c20d50c4185|57:0c20d50c4185]].

The Shaopeng code and seamfollowing code as below, updated to H23.
===changeset: 51:dcd851f8dc57===
parent: 50:c0a9f878ea3e
parent: 39:c90162c4fbe6
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Tue Oct 09 22:15:18 2012 +0100
summary: Updated seamfollowing code to H21

DEPRECATED: Do not use, likely to crash. See rev [[#changeset: 57:0c20d50c4185|57:0c20d50c4185]].

This is the imported seam following code updated to H21.
===changeset: 49:bda0e1259a36===
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Jul 02 15:04:35 2012 +0100
summary: Large modification and cleanup of Shaopeng code. Below for details.

This contains the [[Resgrg:comp-photo-version control-codes#Shaopeng|Shaopeng matrix element code]]
with most of Simon's modifications
and bug fixes in place. It's based off version H21
===changeset: 41:c8a317a36a4e===
tag: h23
parent: 40:cfd3ea6fcdff
parent: 25:7a8a763281d4
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:45:00 2012 +0100
summary: Merged in H23

This is official Gaussian version H23.

===changeset: 39:c90162c4fbe6===
tag: h21
parent: 38:ef0c2dff2d79
parent: 20:dd575870f775
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:44:00 2012 +0100
summary: Merged in H21

This is official Gaussian version H21.
===changeset: 37:e972407aab01===
tag: h13
parent: 36:9d36f9fe483b
parent: 17:770721638a7a
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:43:06 2012 +0100
summary: Merged in H13

This is official Gaussian version H13.
===changeset: 35:10679662e3a3===
tag: h12p
parent: 34:3ce1603d2c6c
parent: 16:427978d67ed6
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:42:11 2012 +0100
summary: Merged in H12p

This is official Gaussian version H12p.
===changeset: 33:a0e0a314db85===
tag: h11
parent: 32:3732a6888cc4
parent: 15:27066b86194e
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:39:49 2012 +0100
summary: Merged in H11

This is official Gaussian version H11.
===changeset: 31:7d8a337a7324===
tag: h10
parent: 30:86c2a0b6f977
parent: 14:62f983246ba7
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:39:11 2012 +0100
summary: Merged in H10

This is official Gaussian version H10.
===changeset: 29:56fa127c847c===
tag: h08
parent: 28:877924b27ee3
parent: 13:903ac662c7a2
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:38:04 2012 +0100
summary: Merged in H08

This is official Gaussian version H08.
===changeset: 27:a05e10e365c2===
tag: h01
parent: -1:000000000000
parent: 12:ae8da988f728
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:35:05 2012 +0100
summary: Fixed bug where code compiled with PGI >= 12 would fail with thread error.

This is official Gaussian version H01.

==Formatting==
The format is:

===changeset: 56:e359f0e1aa5c===
tag: tip
parent: 55:f6c5e2290dc4
parent: 53:c7f36daff923
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Oct 10 14:56:24 2012 +0100
summary: Merged Shaopeng, seamfollowing, olddirect (H23)

Text explaining what this revision contains,
e.g. all of the below plus merged up to H23

In raw form that's:
<code>
===changeset: 56:e359f0e1aa5c===
tag: tip
parent: 55:f6c5e2290dc4
parent: 53:c7f36daff923
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Oct 10 14:56:24 2012 +0100
summary: Merged Shaopeng, seamfollowing, olddirect (H23)

Text explaining what this revision contains,
e.g. all of the below plus merged up to H23
</code>

This is the output from <code>hg sum</code> with the changeset line made into a header, a space
added to the start of each following line, and more
explanatory text. '''NB, it's important to add more explanation!'''

Resgrg:comp-photo-version control-codes

2012-11-21T19:53:56Z

Scliffor: Created page with "This page explains the local modifications made to the Gaussian code by the group. Hopefully it'll also explain how to use them as well. ==The codes== ===Shaopeng=== This is a..."

This page explains the local modifications made to the Gaussian code by the group. Hopefully it'll also explain
how to use them as well.

==The codes==

===Shaopeng===
This is a replacement for the existing matrix element generator (hereafter known as the Klene code)
that is used for large (number of orbitals > 8)
CASSCF calculations. If your job shows <code>SME calculated on fly</code> then it's using this path.

The Shaopeng code computes the same matrix elements but does it faster, and in a way that allows certain compiler
optimisations to be used. The result should be an approximate 4X speedup, depending on the job and the architecture that you're running on. This speedup comes at a cost of memory. The algorithm is much more demanding of memory and this extra demand scales with the number of processors.

How to use it: by default l510 will make a decision on whether to use the Klene code path or the Shaopeng based on the problem size and the amount of memory and processors allocated. You can force one code path over the other
with the IOP(5/139=x) where:

{|
! Iop(5/139)
! Effect
|-
| 0
| Default, choose code path automatically
|-
| 1
| Force Klene code path
|-
| 2
| Force Shaopeng code path
|}

===Seamfollowing===

Dunno

===ONIOM Freq===

Dunno

Resgrp:comp-photo-version control-local-dev-master

2012-11-21T18:53:08Z

Scliffor:

This page will show the current state of the group's master dev repository <code>/home/gaussian-devel/repositories/local-dev-master</code>.
It will document what code is currently considered to be accepted as ready to use.

The format is explained [[#Formatting|here]]. You don't have to describe each revision, just
whatever the tip is at the time, and whatever might be (or have been) made into a
compiled version.
==The revisions==
The most recent revision (which should be the tip) should always be on top. Note
that the "tip" tag may appear incorrectly below. Also note that the local revision
numbers (e.g. 56 in changeset: 56:e359f0e1aa5c) may differ in other repositories. If
in doubt, use the hex number.

===changeset: 72:a7f8ac5c6a00===
parent: 70:f87f732e26fb
parent: 71:69b842c84566
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Nov 21 14:12:01 2012 +0000
summary: Merged in more updates to makefiles (h23)

This revision contains the Shaopeng matrix element code, the seam following code,
and the ONIOM frequency analysis code. They are all merged to be compatible with H23.

===changeset: 56:e359f0e1aa5c===
parent: 55:f6c5e2290dc4
parent: 53:c7f36daff923
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Oct 10 14:56:24 2012 +0100
summary: Merged Shaopeng, seamfollowing, olddirect (H23)

DEPRECATED: Do not use, likely to crash. See rev [[#changeset: 57:0c20d50c4185|57:0c20d50c4185]].

The merge of the three features. Based off H23.
===changeset: 55:f6c5e2290dc4===
parent: 54:0c83655a3822
parent: 41:c8a317a36a4e
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Oct 10 14:55:17 2012 +0100
summary: Merged olddirect code to H23

DEPRECATED: Do not use, likely to crash. See rev [[#changeset: 57:0c20d50c4185|57:0c20d50c4185]].

Simon's updates and fixes for the semi-direct matrix element code.
Based off H23.
===changeset: 53:c7f36daff923===
parent: 52:e11a2fc5b5a8
parent: 41:c8a317a36a4e
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Oct 10 12:38:05 2012 +0100
summary: Shaopeng + seamfollowing, merged to H23.

DEPRECATED: Do not use, likely to crash. See rev [[#changeset: 57:0c20d50c4185|57:0c20d50c4185]].

The Shaopeng code and seamfollowing code as below, updated to H23.
===changeset: 51:dcd851f8dc57===
parent: 50:c0a9f878ea3e
parent: 39:c90162c4fbe6
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Tue Oct 09 22:15:18 2012 +0100
summary: Updated seamfollowing code to H21

DEPRECATED: Do not use, likely to crash. See rev [[#changeset: 57:0c20d50c4185|57:0c20d50c4185]].

This is the imported seam following code updated to H21.
===changeset: 49:bda0e1259a36===
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Jul 02 15:04:35 2012 +0100
summary: Large modification and cleanup of Shaopeng code. Below for details.

This contains the Shaopeng matrix element code with most of Simon's modifications
and bug fixes in place. It's based off version H21
===changeset: 41:c8a317a36a4e===
tag: h23
parent: 40:cfd3ea6fcdff
parent: 25:7a8a763281d4
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:45:00 2012 +0100
summary: Merged in H23

This is official Gaussian version H23.

===changeset: 39:c90162c4fbe6===
tag: h21
parent: 38:ef0c2dff2d79
parent: 20:dd575870f775
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:44:00 2012 +0100
summary: Merged in H21

This is official Gaussian version H21.
===changeset: 37:e972407aab01===
tag: h13
parent: 36:9d36f9fe483b
parent: 17:770721638a7a
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:43:06 2012 +0100
summary: Merged in H13

This is official Gaussian version H13.
===changeset: 35:10679662e3a3===
tag: h12p
parent: 34:3ce1603d2c6c
parent: 16:427978d67ed6
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:42:11 2012 +0100
summary: Merged in H12p

This is official Gaussian version H12p.
===changeset: 33:a0e0a314db85===
tag: h11
parent: 32:3732a6888cc4
parent: 15:27066b86194e
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:39:49 2012 +0100
summary: Merged in H11

This is official Gaussian version H11.
===changeset: 31:7d8a337a7324===
tag: h10
parent: 30:86c2a0b6f977
parent: 14:62f983246ba7
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:39:11 2012 +0100
summary: Merged in H10

This is official Gaussian version H10.
===changeset: 29:56fa127c847c===
tag: h08
parent: 28:877924b27ee3
parent: 13:903ac662c7a2
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:38:04 2012 +0100
summary: Merged in H08

This is official Gaussian version H08.
===changeset: 27:a05e10e365c2===
tag: h01
parent: -1:000000000000
parent: 12:ae8da988f728
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:35:05 2012 +0100
summary: Fixed bug where code compiled with PGI >= 12 would fail with thread error.

This is official Gaussian version H01.

==Formatting==
The format is:

===changeset: 56:e359f0e1aa5c===
tag: tip
parent: 55:f6c5e2290dc4
parent: 53:c7f36daff923
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Oct 10 14:56:24 2012 +0100
summary: Merged Shaopeng, seamfollowing, olddirect (H23)

Text explaining what this revision contains,
e.g. all of the below plus merged up to H23

In raw form that's:
<code>
===changeset: 56:e359f0e1aa5c===
tag: tip
parent: 55:f6c5e2290dc4
parent: 53:c7f36daff923
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Oct 10 14:56:24 2012 +0100
summary: Merged Shaopeng, seamfollowing, olddirect (H23)

Text explaining what this revision contains,
e.g. all of the below plus merged up to H23
</code>

This is the output from <code>hg sum</code> with the changeset line made into a header, a space
added to the start of each following line, and more
explanatory text. '''NB, it's important to add more explanation!'''

Resgrp:comp-photo-version control-local-dev-master

2012-10-11T11:49:32Z

Scliffor:

This page will show the current state of the group's master dev repository <code>/home/gaussian-devel/repositories/local-dev-master</code>.
It will document what code is currently considered to be accepted as ready to use.

The format is explained [[#Formatting|here]]. You don't have to describe each revision, just
whatever the tip is at the time, and whatever might be (or have been) made into a
compiled version.
==The revisions==
The most recent revision (which should be the tip) should always be on top. Note
that the "tip" tag may appear incorrectly below. Also note that the local revision
numbers (e.g. 56 in changeset: 56:e359f0e1aa5c) may differ in other repositories. If
in doubt, use the hex number.

===changeset: 56:e359f0e1aa5c===
tag: tip
parent: 55:f6c5e2290dc4
parent: 53:c7f36daff923
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Oct 10 14:56:24 2012 +0100
summary: Merged Shaopeng, seamfollowing, olddirect (H23)

The merge of the three features. Based off H23.
===changeset: 55:f6c5e2290dc4===
parent: 54:0c83655a3822
parent: 41:c8a317a36a4e
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Oct 10 14:55:17 2012 +0100
summary: Merged olddirect code to H23

Simon's updates and fixes for the semi-direct matrix element code.
Based off H23.
===changeset: 53:c7f36daff923===
parent: 52:e11a2fc5b5a8
parent: 41:c8a317a36a4e
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Oct 10 12:38:05 2012 +0100
summary: Shaopeng + seamfollowing, merged to H23.

The Shaopeng code and seamfollowing code as below, updated to H23.
===changeset: 51:dcd851f8dc57===
parent: 50:c0a9f878ea3e
parent: 39:c90162c4fbe6
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Tue Oct 09 22:15:18 2012 +0100
summary: Updated seamfollowing code to H21

This is the imported seam following code updated to H21.
===changeset: 49:bda0e1259a36===
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Jul 02 15:04:35 2012 +0100
summary: Large modification and cleanup of Shaopeng code. Below for details.

This contains the Shaopeng matrix element code with most of Simon's modifications
and bug fixes in place. It's based off version H21
===changeset: 41:c8a317a36a4e===
tag: h23
parent: 40:cfd3ea6fcdff
parent: 25:7a8a763281d4
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:45:00 2012 +0100
summary: Merged in H23

This is official Gaussian version H23.

===changeset: 39:c90162c4fbe6===
tag: h21
parent: 38:ef0c2dff2d79
parent: 20:dd575870f775
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:44:00 2012 +0100
summary: Merged in H21

This is official Gaussian version H21.
===changeset: 37:e972407aab01===
tag: h13
parent: 36:9d36f9fe483b
parent: 17:770721638a7a
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:43:06 2012 +0100
summary: Merged in H13

This is official Gaussian version H13.
===changeset: 35:10679662e3a3===
tag: h12p
parent: 34:3ce1603d2c6c
parent: 16:427978d67ed6
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:42:11 2012 +0100
summary: Merged in H12p

This is official Gaussian version H12p.
===changeset: 33:a0e0a314db85===
tag: h11
parent: 32:3732a6888cc4
parent: 15:27066b86194e
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:39:49 2012 +0100
summary: Merged in H11

This is official Gaussian version H11.
===changeset: 31:7d8a337a7324===
tag: h10
parent: 30:86c2a0b6f977
parent: 14:62f983246ba7
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:39:11 2012 +0100
summary: Merged in H10

This is official Gaussian version H10.
===changeset: 29:56fa127c847c===
tag: h08
parent: 28:877924b27ee3
parent: 13:903ac662c7a2
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:38:04 2012 +0100
summary: Merged in H08

This is official Gaussian version H08.
===changeset: 27:a05e10e365c2===
tag: h01
parent: -1:000000000000
parent: 12:ae8da988f728
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:35:05 2012 +0100
summary: Fixed bug where code compiled with PGI >= 12 would fail with thread error.

This is official Gaussian version H01.

==Formatting==
The format is:

===changeset: 56:e359f0e1aa5c===
tag: tip
parent: 55:f6c5e2290dc4
parent: 53:c7f36daff923
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Oct 10 14:56:24 2012 +0100
summary: Merged Shaopeng, seamfollowing, olddirect (H23)

Text explaining what this revision contains,
e.g. all of the below plus merged up to H23

In raw form that's:
<code>
===changeset: 56:e359f0e1aa5c===
tag: tip
parent: 55:f6c5e2290dc4
parent: 53:c7f36daff923
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Oct 10 14:56:24 2012 +0100
summary: Merged Shaopeng, seamfollowing, olddirect (H23)

Text explaining what this revision contains,
e.g. all of the below plus merged up to H23
</code>

This is the output from <code>hg sum</code> with the changeset line made into a header, a space
added to the start of each following line, and more
explanatory text. '''NB, it's important to add more explanation!'''

Resgrp:comp-photo

2012-10-11T11:48:16Z

Scliffor: /* Codes */

= Computational Photochemistry Research Group Wiki, Imperial College London =

A place to write something about computing facilities (e.g. CX1),
documenting codes (Gaussian + dynamics; latest versions)
and example input files.
-[[User:Mjbear|Mjbear]] 16:16, 17 October 2008 (BST)

Please amend and update this! Log in using your IC account.

== Computing Resources ==
=== [[Resgrp:comp-photo-hpc|Using College HPC to run Gaussian: CX1]] ===
===== [[Resgrp:comp-photo-hpc-ax1|- AX1 / AX2 ]] =====
=== [[Resgrp:comp-photo-owncomp|Group-managed computers]] ===
=== [[Resgrp:utilities|Utilities]] ===

== Codes ==
=== [[Resgrp:comp-photo-gaussian|Gaussian versions]] ===
===== [[Resgrp:comp-photo-gaussian_problems|- known problems ]] =====
===Developing Gaussian===
There's a new regime!
====[[Resgrp:comp-photo-version control-local-dev-master|Local development versions]]====
A page explaining what is in the local development repository. Currently rather sparse,
you are encouraged to be much more verbose.
====[[Resgrp:comp-photo-version control|Using Mercurial]]====
A rather long manual on why and how to use the mercurial version control system,
somewhat tailored to Gaussian development.
====[[Resgrp:comp-photo-version control short|Quick guide to Mercurial]]====
A short version of the above.
====[[Resgrp:comp-photo-version control example|Example workflows]]====
A couple of examples of how one would use mercurial to develop Gaussian
====[[Resgrp:comp-photo-new gdv layout|The new layout]]====
An explanation of the new layout of the Gaussian source code necessitated
by version control. Plus some tips and tricks.
=== [[Resgrp:comp-photo-dyn|MCTDH (DD-vMCG)]] ===
=== [[Resgrp:comp-photo-onthefly|Direct CAS/RAS]] ===

== Example Calculations / Tutorials ==
=== [[Resgrp:comp-photo-benzene-tutorial|Benzene CASSCF tutorial (G03)]] ===
=== [[Resgrp:comp-photo-CHD|Seam calculations of CHD]] ===
===== [[Resgrp:comp-photo-seam|- Current Seam Frequency Code ]] =====
=== [[Resgrp:comp-photo/sh|Surface Hopping]] ===

=== [[Resgrp:comp-photo/VB|Valence-Bond analysis]] ===

=== [[Resgrp:comp-photo-visualise|GaussView, visualisation etc]] ===

=== [[ONIOM]] ===

== Calculations ==
=== [[Resgrp:comp-photo-calculations|Examples]] ===

Resgrp:comp-photo

2012-10-11T11:41:35Z

Scliffor: /* Codes */

= Computational Photochemistry Research Group Wiki, Imperial College London =

A place to write something about computing facilities (e.g. CX1),
documenting codes (Gaussian + dynamics; latest versions)
and example input files.
-[[User:Mjbear|Mjbear]] 16:16, 17 October 2008 (BST)

Please amend and update this! Log in using your IC account.

== Computing Resources ==
=== [[Resgrp:comp-photo-hpc|Using College HPC to run Gaussian: CX1]] ===
===== [[Resgrp:comp-photo-hpc-ax1|- AX1 / AX2 ]] =====
=== [[Resgrp:comp-photo-owncomp|Group-managed computers]] ===
=== [[Resgrp:utilities|Utilities]] ===

== Codes ==
=== [[Resgrp:comp-photo-gaussian|Gaussian versions]] ===
===== [[Resgrp:comp-photo-gaussian_problems|- known problems ]] =====
===Developing Gaussian===
There's a new regime!
====[[Resgrp:comp-photo-version control|Using Mercurial]]====
A rather long manual on why and how to use the mercurial version control system,
somewhat tailored to Gaussian development.
====[[Resgrp:comp-photo-version control short|Quick guide to Mercurial]]====
A short version of the above.
====[[Resgrp:comp-photo-version control example|Example workflows]]====
A couple of examples of how one would use mercurial to develop Gaussian
====[[Resgrp:comp-photo-new gdv layout|The new layout]]====
An explanation of the new layout of the Gaussian source code necessitated
by version control. Plus some tips and tricks.
=== [[Resgrp:comp-photo-dyn|MCTDH (DD-vMCG)]] ===
=== [[Resgrp:comp-photo-onthefly|Direct CAS/RAS]] ===

== Example Calculations / Tutorials ==
=== [[Resgrp:comp-photo-benzene-tutorial|Benzene CASSCF tutorial (G03)]] ===
=== [[Resgrp:comp-photo-CHD|Seam calculations of CHD]] ===
===== [[Resgrp:comp-photo-seam|- Current Seam Frequency Code ]] =====
=== [[Resgrp:comp-photo/sh|Surface Hopping]] ===

=== [[Resgrp:comp-photo/VB|Valence-Bond analysis]] ===

=== [[Resgrp:comp-photo-visualise|GaussView, visualisation etc]] ===

=== [[ONIOM]] ===

== Calculations ==
=== [[Resgrp:comp-photo-calculations|Examples]] ===

Resgrp:comp-photo

2012-10-11T11:41:00Z

Scliffor: /* Codes */

= Computational Photochemistry Research Group Wiki, Imperial College London =

A place to write something about computing facilities (e.g. CX1),
documenting codes (Gaussian + dynamics; latest versions)
and example input files.
-[[User:Mjbear|Mjbear]] 16:16, 17 October 2008 (BST)

Please amend and update this! Log in using your IC account.

== Computing Resources ==
=== [[Resgrp:comp-photo-hpc|Using College HPC to run Gaussian: CX1]] ===
===== [[Resgrp:comp-photo-hpc-ax1|- AX1 / AX2 ]] =====
=== [[Resgrp:comp-photo-owncomp|Group-managed computers]] ===
=== [[Resgrp:utilities|Utilities]] ===

== Codes ==
=== [[Resgrp:comp-photo-gaussian|Gaussian versions]] ===
===Developing Gaussian===
There's a new regime!
====[[Resgrp:comp-photo-version control|Using Mercurial]]====
A rather long manual on why and how to use the mercurial version control system,
somewhat tailored to Gaussian development.
====[[Resgrp:comp-photo-version control short|Quick guide to Mercurial]]====
A short version of the above.
====[[Resgrp:comp-photo-version control example|Example workflows]]====
A couple of examples of how one would use mercurial to develop Gaussian
====[[Resgrp:comp-photo-new gdv layout|The new layout]]====
An explanation of the new layout of the Gaussian source code necessitated
by version control. Plus some tips and tricks.
===== [[Resgrp:comp-photo-gaussian_problems|- known problems ]] =====
=== [[Resgrp:comp-photo-dyn|MCTDH (DD-vMCG)]] ===
=== [[Resgrp:comp-photo-onthefly|Direct CAS/RAS]] ===

== Example Calculations / Tutorials ==
=== [[Resgrp:comp-photo-benzene-tutorial|Benzene CASSCF tutorial (G03)]] ===
=== [[Resgrp:comp-photo-CHD|Seam calculations of CHD]] ===
===== [[Resgrp:comp-photo-seam|- Current Seam Frequency Code ]] =====
=== [[Resgrp:comp-photo/sh|Surface Hopping]] ===

=== [[Resgrp:comp-photo/VB|Valence-Bond analysis]] ===

=== [[Resgrp:comp-photo-visualise|GaussView, visualisation etc]] ===

=== [[ONIOM]] ===

== Calculations ==
=== [[Resgrp:comp-photo-calculations|Examples]] ===

Resgrp:comp-photo-version control-local-dev-master

2012-10-11T11:26:44Z

Scliffor: Created page with "This page will show the current state of the group's master dev repository (local-dev-master). It will document what code is currently considered to be accepted as ready to use. ..."

This page will show the current state of the group's master dev repository (local-dev-master).
It will document what code is currently considered to be accepted as ready to use.

The format is explained [[#Formatting|here]]. You don't have to describe each revision, just
whatever the tip is at the time, and whatever might be (or have been) made into a
compiled version.
==The revisions==
The most recent revision (which should be the tip) should always be on top. Note
that the "tip" tag may appear incorrectly below. Also note that the local revision
numbers (e.g. 56 in changeset: 56:e359f0e1aa5c) may differ in other repositories. If
in doubt, use the hex number.

===changeset: 56:e359f0e1aa5c===
tag: tip
parent: 55:f6c5e2290dc4
parent: 53:c7f36daff923
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Oct 10 14:56:24 2012 +0100
summary: Merged Shaopeng, seamfollowing, olddirect (H23)

The merge of the three features. Based off H23.
===changeset: 55:f6c5e2290dc4===
parent: 54:0c83655a3822
parent: 41:c8a317a36a4e
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Oct 10 14:55:17 2012 +0100
summary: Merged olddirect code to H23

Simon's updates and fixes for the semi-direct matrix element code.
Based off H23.
===changeset: 53:c7f36daff923===
parent: 52:e11a2fc5b5a8
parent: 41:c8a317a36a4e
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Oct 10 12:38:05 2012 +0100
summary: Shaopeng + seamfollowing, merged to H23.

The Shaopeng code and seamfollowing code as below, updated to H23.
===changeset: 51:dcd851f8dc57===
parent: 50:c0a9f878ea3e
parent: 39:c90162c4fbe6
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Tue Oct 09 22:15:18 2012 +0100
summary: Updated seamfollowing code to H21

This is the imported seam following code updated to H21.
===changeset: 49:bda0e1259a36===
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Jul 02 15:04:35 2012 +0100
summary: Large modification and cleanup of Shaopeng code. Below for details.

This contains the Shaopeng matrix element code with most of Simon's modifications
and bug fixes in place. It's based off version H21
===changeset: 41:c8a317a36a4e===
tag: h23
parent: 40:cfd3ea6fcdff
parent: 25:7a8a763281d4
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:45:00 2012 +0100
summary: Merged in H23

This is official Gaussian version H23.

===changeset: 39:c90162c4fbe6===
tag: h21
parent: 38:ef0c2dff2d79
parent: 20:dd575870f775
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:44:00 2012 +0100
summary: Merged in H21

This is official Gaussian version H21.
===changeset: 37:e972407aab01===
tag: h13
parent: 36:9d36f9fe483b
parent: 17:770721638a7a
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:43:06 2012 +0100
summary: Merged in H13

This is official Gaussian version H13.
===changeset: 35:10679662e3a3===
tag: h12p
parent: 34:3ce1603d2c6c
parent: 16:427978d67ed6
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:42:11 2012 +0100
summary: Merged in H12p

This is official Gaussian version H12p.
===changeset: 33:a0e0a314db85===
tag: h11
parent: 32:3732a6888cc4
parent: 15:27066b86194e
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:39:49 2012 +0100
summary: Merged in H11

This is official Gaussian version H11.
===changeset: 31:7d8a337a7324===
tag: h10
parent: 30:86c2a0b6f977
parent: 14:62f983246ba7
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:39:11 2012 +0100
summary: Merged in H10

This is official Gaussian version H10.
===changeset: 29:56fa127c847c===
tag: h08
parent: 28:877924b27ee3
parent: 13:903ac662c7a2
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:38:04 2012 +0100
summary: Merged in H08

This is official Gaussian version H08.
===changeset: 27:a05e10e365c2===
tag: h01
parent: -1:000000000000
parent: 12:ae8da988f728
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Mon Oct 08 17:35:05 2012 +0100
summary: Fixed bug where code compiled with PGI >= 12 would fail with thread error.

This is official Gaussian version H01.

==Formatting==
The format is:

===changeset: 56:e359f0e1aa5c===
tag: tip
parent: 55:f6c5e2290dc4
parent: 53:c7f36daff923
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Oct 10 14:56:24 2012 +0100
summary: Merged Shaopeng, seamfollowing, olddirect (H23)

Text explaining what this revision contains,
e.g. all of the below plus merged up to H23

In raw form that's:
<code>
===changeset: 56:e359f0e1aa5c===
tag: tip
parent: 55:f6c5e2290dc4
parent: 53:c7f36daff923
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Oct 10 14:56:24 2012 +0100
summary: Merged Shaopeng, seamfollowing, olddirect (H23)

Text explaining what this revision contains,
e.g. all of the below plus merged up to H23
</code>

This is the output from <code>hg sum</code> with the changeset line made into a header, a space
added to the start of each following line, and more
explanatory text. '''NB, it's important to add more explanation!'''

Resgrp:comp-photo-version control example

2012-10-11T10:48:50Z

Scliffor: Created page with "This is an example of how one might use mercurial to work on the gaussian code. ==Adding a feature you've already coded== '''You've got a set of changes that constitute some nea..."

This is an example of how one might use mercurial to work on the gaussian code.

==Adding a feature you've already coded==
'''You've got a set of changes that constitute some neat feature or bug fix, and you'''
'''want to get it checked into a repository. It's based off some older version of Gaussian and'''
'''you want to bring it up to date with the latest version.'''

First, check out a clone of the gaussian-inc-versions repository. We'll use
<code>-u h21</code> to check out a working directory with a buildable and
locally bug-fixed H21 release in it.

<code>
$ hg clone -u h21 /home/gaussian-devel/repositories/gaussian-inc-versions myrepo
13578 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ cd myrepo
</code>

The repository only goes as far back as H01 (at the moment), so if you're based off a different
release you'll have to either update it manually to h01 or work out how
to import your earlier release into a mercurial repository.

Now copy your own code over the code in the working directory. Obviously routines for
L510 go in the l510 directory and so on. Util files go in either utilam, utilnz, osutil, putil, etc. You can see what files you've added or altered using <code>hg stat</code>. If your
change requires that you ''remove'' files use <code>hg rm</code> rather than regular <code>rm</code>.
Once you're content that all your code is in the working directory you can do a:

<code>
$ hg addremove
</code>

This will mark to be added any files that you've added. Check you're not adding things like
test jobs (we need these somewhere else -- where?), temporary files, etc.

Now check in your code:

<code>
$ hg commit
</code>

An editor will open and you can write a useful log message detailing what this code does.
Be as verbose as you like, but make the first line be a useful summary. Now this repository holds
your feature. Only your repository has this change; you may decide to work on it some more,
or throw it away by deleting the repository. You may also want to update
your code to the latest Gaussian version:

<code>
$ hg merge h23
</code>
or
<code>
$ hg merge
</code>
to update to the latest version, whatever that is.

Once the merge is done and you've resolved any conflicts commit again, noting in
your log message that you've updated to a particular version:

<code>
$ hg commit -m 'Updated my feature to H23'
</code>

Again, you can share these changes with your other repositories (to combine your features)
or with other people. You can also pull them into a local-dev-master (see below) repository
to merge in other local group features.

==Getting a version to code something new into==

'''You've been told to grab the latest local development changeset and code in something new.'''

Start by cloning the latest development repository:

<code>
$ hg clone /home/gaussian-devel/repositories/local-dev-master
updating to branch default
13722 files updated, 0 files merged, 0 files removed, 0 files unresolved
</code>

This will create a clone of the repository in the current directory with the name
'local-dev-master'. You can override this if you want.

Use <code>hg log</code> and <code>hg sum</code> and check out [[Resgrp:comp-photo-version control-local-dev-master|the wiki page documenting code in this repository]] to get an idea of what
you're starting from. Generally this repository will hold features and bug fixes that the
group deems ready for prime time usage; code here is probably intended to go to Gaussian, Inc.
and is the correct starting point for new features.

As you code, commit your changes using <code>hg commit</code> at appropriate points. This is
sort of up to you, but it would be nice if we could agree to check in code that compiles.
Really though, it doesn't matter too much, as we can combine sets of revisions should we need to.

Resgrp:comp-photo-version control

2012-10-08T13:25:26Z

Scliffor: /* Short version */

== Introduction ==

This document will teach you the basics of using mercurial, a distributed version control system (DVCS). It is particularly directed towards using mercurial with the development version of Gaussian, so the examples will use Gaussian. For more information on mercurial you can use the built-in help system <code>hg help</code>, look at the official website: http://mercurial.selenic.com/wiki/Mercurial, or check out the official O'Reilly book, the text of which is freely available at: http://hgbook.red-bean.com/.

A version control system can sometimes seem an onerous imposition unless the user understands how it is going to help them in their work, so I'll try to briefly explain. Version control is about tracking changes to data. If one has a collection of files, say a distribution of Gaussian, then one is interested in changes that have been made to those files, whether from a new distribution from Gaussian or our own modifications. We are particularly interested in cases where overlapping changes have been made. These might be updates and bug fixes from Gaussian conflicting with our own modifications to a particular link, or it might be different researchers in a group working on the same piece of code independently. A version control system will not only detect such cases but provide automated and safe methods for merging conflicting changes.

Central to the idea of a VCS is the concept of a revision or changeset. This is like a snapshot of some set of files and directories at some particular instant in their history. A repository is where a VCS stores revisions. So, using Gaussian as an example, some revisions in a particular repository might be the Gaussian source tree corresponding to Gaussian development versions H01, H08, H10, H11, etc. The data that are actually kept inside the repository are the differences, or deltas, between the revisions. This saves space compared to storing all the revisions.

== Short version ==

I've made a summary version of this document [[Resgrp:comp-photo-version control short|here]].
I've also made an [[Resgrp:comp-photo-version control example|example workflow]].

== Setting up your environment (.hgrc)==
There are some one-off things you must do. Firstly, edit your login scripts (<code>.bash_profile</code> or similar) to include the command:
<code>
module load mercurial
</code>
(If you're not on an IC HPC machine mercurial is available anywhere that runs Python. It is generally available in distribution package systems, if not you can get it from http://mercurial.selenic.com/wiki/Mercurial). Note that the Redhat provided mercurial on RHEL 5 and 6 is quite old and does not, at time of writing, support the repositories I have created.

Next, create a file called <code>.hgrc</code> in your home directory and insert the following lines:
<code>
[ui]
username = Your Name <your@email.address>

[extensions]
graphlog=
</code>
(so for example, my <code>.hgrc</code> contains this line: <code>username = Simon Clifford <simon.j.clifford@gmail.com></code>).
Mercurial uses this username field to record who has changed what. (If you're not on an IC HPC machine your version of mercurial may not
have the graphlog extension. If it doesn't, don't worry, it's merely a tool for visualising changesets).

== Checking out a repository (hg clone, hg log) ==

Now let's get started and check out a repository. Once you have loaded the mercurial module you will be able to type:
<code>
tmp$ hg clone /home/gaussian-devel/example-gaussian-repo testrepo
tmp$
</code>
This will create a copy of the repository at <code>/home/gaussian-devel/example-gaussian-repo</code> and place it in a directory inside the current directory called <code>testrepo</code>. You can give a full path as the second argument, or, you can leave it off altogether in which case mercurial will clone the repository into a directory named <code>example-gaussian-repo</code>. A note here for Imperial HPC users. Copying a repository involves quite a bit of filesystem activity. I have found that on cx1 this can be quite slow on the <code>/home</code> filesystems. You may wish to try these examples in the <code>/tmp</code> filesystem which is a lot faster. Be aware, however, that this raises two potentially serious problems. The first is that <code>/tmp</code> is globally readable so before you clone you must make sure that your umask is set to <code>0077</code>. The second, of course, is that <code>/tmp</code> is periodically wiped. If you plan on working in <code>/tmp</code> there are simple ways of transferring information between repositories (<code>hg push</code> and <code>hg pull</code>) that I will cover later.

Now change into the <code>testrepo</code> directory. Inside you will see three scripts and a <code>gdv</code> directory; also some hidden files and a hidden directory. Inside the <code>gdv</code> directory you will see the Gaussian source in the [[Resgrp:comp-photo-new gdv layout|new layout]]. The hidden <code>.hg</code> directory inside the <code>testrepo</code> directory is the repository proper; you will almost never need to do anything in here. The rest of the files and directories are called the working directory; this is where you will do your work.

The first thing you might want to do is check the history of this repository to see past revisions that have been checked into it. You do this with the <code>hg log</code> command.
<code>
testrepo$ hg log | head -20
changeset: 23:1de40efd1c4b
tag: tip
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:43:13 2012 +0000
summary: Added tag h13 for changeset 6c81eb8dcbab

changeset: 22:6c81eb8dcbab
tag: h13
parent: 21:5b6ac3665a29
parent: 11:a0c273aeeb2b
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:43:08 2012 +0000
summary: Gaussian devel version H13 with our makefiles

changeset: 21:5b6ac3665a29
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:41:40 2012 +0000
summary: Added tag h12p for changeset 0ef14d7dff56
...
changeset: 1:515a93bfc3b5
branch: raw
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Sun Jan 15 11:39:10 2012 +0000
summary: Added tag h01-raw for changeset 073d6aa63ea7

changeset: 0:073d6aa63ea7
branch: raw
tag: h01-raw
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Sun Jan 15 11:39:09 2012 +0000
summary: Output from old-to-new.sh on version H01
</code>
You will see that there are various fields that may appear for each changeset. The changeset field shows two numbers separated by a colon. The second, longer, hexadecimal number is the unique identifier. A particular hex identifier will always refer to the same changeset, even in different copies of the repository. The first number is a local identifier. These local IDs refer to particular changesets in one copy of the repository; if these changesets are present in another repository they may have a different local identifier. You can use either identifier as an argument to <code>hg</code> commands. The user and date fields show who committed the change and when. The summary field shows the first line of the log entry for that revision. This means it is important to try to make the first line of your log entries a useful summary of what you have done!

To see the entry for a particular revision use the <code>-r</code> flag with a revision number, either short or long. To see more information, including the full log entry and a complete list of all files involved in the change, use the <code>-v</code> flag, e.g.:
<code>
testrepo$ hg log -r 2 -v | less
changeset: 2:9f69ed4616ad
branch: raw
tag: h08-raw
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Sun Jan 15 11:54:50 2012 +0000
files: gdv/archlib/arcvib.F gdv/archlib/brcpyw.F gdv/archlib/letset.F gdv/arctmp.F gdv/bsd/GauDiff_Compare.pm gdv/bsd/G
....
at.F gdv/utilnz/xpndnb.F gdv/utilnz/zerock.F gdv/utilnz/zerod.F gdv/wrappers/ggeev.F gdv/wrappers/lappar.F gdv/wrappers/xgetrf.F gdv/wrappers/xgetrs.F gdv/wrappers/ygeru.F gdv/wrappers/ytrsv.F
description:
Import of gdv H08. See below for details.

Result of these commands on the H01 raw rev:
rm -fr gdv
cp -pr h08/new-style-gdv .
hg add gdv/bsd/*.a
hg addremove -s 50

h08/new-style is from old-to-new.sh on freshly untarred h08.tgz
</code>
As the log entry explains, this particular revision is the change from the H01 version of Gaussian to the H08 version, so there are a lot of changed files. The description entry is intended to provide some human readable explanation of the changes. When you start committing data to the repository you should aim to at least make your log entries understandable.

As with all the commands that I will mention you can get more information by typing <code>hg help <command></code>.

== Committing changes (hg summary, hg status, hg diff) ==

Now let's make some alterations to this repository. This is quite safe; we can't break the original repository because our copy is an entirely separate clone. In fact, this is a very common working practice with mercurial. As long as the filesystem isn't slow cloning a repository can be quite quick. Indeed, cloning a repository into a destination on the same filesystem as the source makes use of hard linking which is very fast and saves space. Therefore, it is very common to clone a repository, make some alterations to it, and then decide whether to push them back to the original repository or just delete the new clone.

First, we want to know where it is we are starting from. To get a quick summary you use the <code>hg summary</code> command. This should show something like this:
<code>
testrepo$ hg summary
parent: 23:1de40efd1c4b tip
Added tag h13 for changeset 6c81eb8dcbab
branch: default
commit: (clean)
update: (current)
</code>
This shows, on the parent line, the revision that has been checked out into the working directory. Below that is the summary line from the log entry for that revision. The commit line shows if any files have been modified since the check out. The update line shows if there are any applicable newer revisions in the repository. In this example the output shows us that we checked out the revision with the local identifier 23, and unique identifier of 1de40efd1c4b, that we have modified nothing and that there are no appropriate newer revisions. This checkout occurred automatically as a part of the <code>hg clone</code> command. You can clone without checking anything out into the working directory (say to make a backup copy of the repository) by passing the <code>-U</code> flag, or you can check out a particular revision with the <code>-u REV</code> flag. See <code>hg help clone</code> for how mercurial chooses what to check out by default.

Let's say we're adding a feature to link 510. If you have something in mind go ahead and do it. For this example I have just added a few lines to the file <code>a1tdep.F</code>, and will assume that your current working directory is <code>gdv/l510</code>.

As you work you may be curious to know what it is you have changed. The command to show this is <code>hg status</code>:
<code>
testrepo$ hg status
M gdv/l510/a1tdep.F
</code>
The output shows a list of files that have been altered in some way. Here the capital M shows that the file <code>a1tdep.F</code> has been modified. By default <code>hg status</code> does not report on unchanged files. If we wish to see how <code>a1tdep.F</code> has been modified we can use the <code>hg diff</code> command:
<code>
testrepo$ hg diff a1tdep.F
diff -r 1de40efd1c4b gdv/l510/a1tdep.F
--- a/gdv/l510/a1tdep.F Wed Jan 18 14:43:13 2012 +0000
+++ b/gdv/l510/a1tdep.F Wed Jan 25 11:07:55 2012 +0000
@@ -26,6 +26,9 @@
C
C **OPTIONS FOR TDEP CODE
C
+c
+c Add super new feature
+c
iTDep=813
iTrans=822
jTDep=iTDep
</code>
This shows the output in a unified different format, refer to the man page for diff for more information.

== Adding, copying, renaming, and deleting file (hg add, hg forget, hg mv, hg rm, hg cp) ==

If you have created a new file then you must let mercurial know about its existence with the <code>hg add</code> command:
<code>
testrepo$ echo "a new subroutine" > foo.F
testrepo$ hg status
M gdv/l510/a1tdep.F
? gdv/l510/foo.F
testrepo$ hg add foo.F
testrepo$ hg status
M gdv/l510/a1tdep.F
A gdv/l510/foo.F
</code>
Here, I create a new file called <code>foo.F</code>. <code>hg status</code> shows the file with a ?, to indicate that it is unknown to the repository. I then run <code>hg add foo.F</code>, after which <code>hg status</code> shows the file with an A. This indicates that the file is scheduled to be added at the next commit. If you change your mind about the file before the next commit you can use <code>hg forget</code> to undo the add.

If you are renaming a file, including the situation where you move the file from one directory to another (e.g. from l510 to utilam) then you may use the <code>hg mv</code> command. This will actually perform the move just like the normal <code>mv</code> command. If you've already moved the file you can give <code>hg mv</code> the <code>-A</code> flag to record the rename after the fact. Similarly, the <code>hg rm</code> command removes files from the working directory and records this fact in the repository, and <code>hg cp</code> copies a file.

It is important to realise that these commands act on the working directory immediately but only affect the repository when they are committed. Also, they do not affect the repository's previous history, so removing a file and then committing that change does not affect that file's existence in previous revisions in the repository. However, <code>hg mv</code> and <code>hg cp</code> create a relationship between the source and destination files. This becomes useful when merging in changes from somewhere else. So for example, above I have made changes to <code>l510/a1tdep.F</code>. Let's say that someone else has decided to make this into a utility routine and has done the command <code>hg mv l510/a1tdep.F utilam/a1tdep.F</code>. If I choose to merge my changes with this other person's, mercurial will correctly apply the changes '''and''' move the file. If instead of <code>hg mv</code> they had used <code>hg cp</code> then my changes would be applied to both files. This means that you should only use <code>hg cp</code> when it is appropriate that changes that are recorded before the copy are applied to both files. Note that this does not mean that changes to the source file are forever applied to the destination file; this will only occur when merging a revision that does not contain a copy with a revision that does. In general, you can expect mercurial to do the right thing.

== Committing the changes (hg commit, hg tip) ==

The <code>hg commit</code> (like many <code>hg</code> commands, <code>hg</code> commit has an alias, in this case <code>hg ci</code>. I tend to use <code>hg ci</code>) command creates a new revision and records the differences between the entire working directory and the parent revision (as shown on the parent line of <code>hg summary). This will be the same as the output from <code>hg status</code>. When you run the command mercurial will open a text editor. If you wish to specify which editor is used add an <code>editor=vim</code> (for example) line to the <code>[ui]</code> part of your <code>~/.hgrc</code> file. The editor will start with a couple of empty lines and then some lines beginning with "<code>HG:</code>". These are there to give you helpful reminders of what you've changed while you write your log message and will be ignored by mercurial when you commit. If you close the editor without writing anything, or of the editor quits with an error, mercurial will abort the commit. So for our current example:
<code>
testrepo$ hg ci
... [opens vim]
HG: Enter commit message. Lines beginning with 'HG:' are removed.
HG: Leave message empty to abort commit.
HG: --
HG: user: Simon Clifford <simon.j.clifford@gmail.com>
HG: branch 'default'
HG: added gdv/l510/foo.F
HG: changed gdv/l510/a1tdep.F
~
~
</code>
For short messages you can skip the editor step by passing the <code>-m "Log message here"</code> flag.

Understanding what you are doing here is essential to knowing what to write in the log and, importantly, when to commit. There are two schools of thought. When you commit you are creating a new revision in your local repository. Since it is your repository, and as I have previously mentioned, it is very simple to create new repositories, you should commit whenever you feel like it and feel free to write cryptic log messages that only you will understand. On the other hand, you are engaged in a collaborative exercise with other people: creating what will be a single version of Gaussian that contains yours and others' modifications and Gaussian's updates. Therefore, you should only commit when your code satisfies pre-agreed group requirements, and your log message should be detailed and comprehensible to anybody who reads it, even 50 years in the future. The correct approach, of course, is to do both.

Mercurial is a distributed version control system which means that each repository is the responsibility of its owner who can feel free to commit as frequently (or infrequently) as needed. Sometimes adding a feature or removing a bug might take weeks, and you might feel that there is no point taking snapshots of the code until it works. Other modifications might proceed incrementally with naturally defined stopping points between the start and the end. Committing at these points makes perfect sense: not only does it leave a history of the modification, it provides checkpoints, versions of the code that you can retreat to without having to start again. The log messages may tersely explain what has happened since the last revision (and note there is no point listing modified, added, etc, files as this information is stored in the commit anyway) or they may contain detailed essays on how a bug arose and the steps you have taken to fix it. Information in the logs is searchable and should be thought of as both an aide memoir for yourself and a journal entry for others.

When the time comes to merge your changes into other people's repositories, you might start thinking about the quality of your commit. Does the code compile? Does it run the test suite? Can it run any test? Have you committed a test that checks the code you have added or altered? The group may decide that revisions that are being shared must answer yes to some or all of these questions. You may decide that some or all of your revisions must pass these quality checks. Or, you may be satisfied with noting in the log entry what this particular revision can or can't do (such as compile or run).

It should be said though, that if you ever think "perhaps I should check in at this point", then go for it. Revisions live in the history of your repository forever (although see <code>hg rollback</code>), but when you transfer them to other people's repositories multiple revisions can be collapsed into one, or fixed in other ways.

Returning to our example I enter a simple message. Since <code>hg log</code> only shows the first line of any log entry by default it is a good idea to make this line a summary of the rest of the message. I close the editor and the commit is done.

We can see the results in the repository with an <code>hg log</code> command (the <code>-l 2</code> flag shows the last two revisions):
<code>
testrepo$ hg log -l 2
changeset: 24:13ecff0136c2
tag: tip
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 25 17:13:00 2012 +0000
summary: A test message.

changeset: 23:1de40efd1c4b
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:43:13 2012 +0000
summary: Added tag h13 for changeset 6c81eb8dcbab
</code>
Alternatively we can use the <code>hg tip</code> command to show the most recently added repository, whether by ourselves, or by pulling from another repository (see later).

== Fixing mistakes (hg revert, hg rollback) ==

You can return one or more files, or the entire repository, to the state they were in when you last checked them out with in the <code>hg revert</code> command. Let's make an ill-advised alteration to our now committed change to <code>a1tdep.F</code>:
<code>
testrepo$ sed -i 's/super/super duper/' a1tdep.F
testrepo$ hg status
M gdv/l510/a1tdep.F
testrepo$ hg diff a1tdep.F
diff -r 13ecff0136c2 gdv/l510/a1tdep.F
--- a/gdv/l510/a1tdep.F Wed Jan 25 17:13:00 2012 +0000
+++ b/gdv/l510/a1tdep.F Wed Jan 25 17:23:42 2012 +0000
@@ -27,7 +27,7 @@
C **OPTIONS FOR TDEP CODE
C
c
-c Add super new feature
+c Add super duper new feature
c
iTDep=813
iTrans=822
testrepo$ hg revert a1tdep.F
testrepo$ hg status
? gdv/l510/a1tdep.F.orig
testrepo$ hg diff a1tdep.F
testrepo$ rm a1tdep.*
testrepo$ hg status
! gdv/l510/a1tdep.F
testrepo$
</code>
Here I use revert to undo the modifications to a file. Notice that <code>hg revert</code> leaves a copy of the modified file in <code>a1tdep.F.orig</code>. This shows up in <code>hg status</code> as an unknown file ("?"). Also notice that while trying to delete the .orig file I have accidentally deleted <code>a1tdep.F</code>, which now shows up in <code>hg status</code> as a missing file ("!"). I can revert this mistake too:
<code>
testrepo$ hg revert a1tdep.F
testrepo$ hg status
testrepo$ ls a1tdep.F
a1tdep.F
testrepo$
</code>
<code>hg revert</code> can also be used to cancel scheduled adds, removes, copies, and renames.

If you have just committed a revision and then change your mind you may be able to undo the effect with <code>hg rollback</code>. Doing this for our example gives us:
<code>
testrepo$ hg rollback
repository tip rolled back to revision 23 (undo commit)
working directory now based on revision 23
testrepo$ hg status
M gdv/l510/a1tdep.F
A gdv/l510/foo.F
testrepo$
</code>
This removes the revision we just checked in from the repository but does not alter the current working directory. If we choose, we could now use <code>hg revert</code> to restore these to their revision 23 state. There are two important caveats with <code>hg rollback</code>: it can only remove the last checked in revision, and that it is usually pointless to rollback a revision that has already been pushed or pulled (see later) into somebody else's repository, as we can only affect our repository. For the purposes of this example, I will once more check in the changes I have made:
<code>
testrepo$ hg ci -m 'A test message'
testrepo$ hg tip
changeset: 24:13ecff0136c2
tag: tip
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 25 17:38:52 2012 +0000
summary: A test message
testrepo$
</code>

== Collaboration (hg incoming, hg pull, hg outgoing, hg push, hg update, hg parent) ==

The tools described are already quite useful, and form the basis of the very earliest revision control systems. The benefits of being able to detect what you've changed, and how you've changed it, being able to undo the changes selectively, and being able to take a snapshot of your work at any point of your choosing should be apparent. The true power, however, of a modern version control system is how it mediates different streams of changes, particularly those from other users. Note that a DVCS does not solve problems of merging and so on, but provides you, the user, with tools to solve them.

Revisions in the repository form a directed acyclic graph (DAG). Every revision apart from the first has at least one parent revision and may have zero or more child revisions. A revision with no children is called a ''head''. Generally development takes place at a head of the repository: checking in a new revision onto a head creates and consumes a head. However, it is possible to add a child to any revision, possibly creating a new head. This is known as a branch, and may be named or unnamed. A revision with children may still be a branch head if it has no children on its branch.

There are various ways of managing branches. You can have a few repositories with many branches, named and unnamed. Or you can have many repositories with largely unbranched graphs; it's up to you. Since mercurial uses heads as the default targets for some of its operations the latter approach is probably best for beginners as it makes operations within each repository simpler.

This is probably best illustrated with an example. I'll make a copy of the original repository, make a different change, and then merge the changes. Change directory out of the <code>testrepo</code> repository and type:
<code>
tmp$ hg clone /home/gaussiandevel/example-gaussian-repo testmerge
updating to branch default
13427 files updated, 0 files merged, 0 files removed, 0 files unresolved
tmp$
</code>

Now we go into the new repository and make some modifications. This time I alter the files <code>a1tdep.F</code> and <code>a2tdep.F</code>, and add a file <code>bar.F</code> in the <code>l510</code> directory:
<code>
…[alterations to a1tdep.F, a2tdep.F, and bar.F]...
testmerge$ hg add gdv/l510/bar.F
testmerge$ hg diff
diff -r 1de40efd1c4b gdv/l510/a1tdep.F
--- a/gdv/l510/a1tdep.F Wed Jan 18 14:43:13 2012 +0000
+++ b/gdv/l510/a1tdep.F Thu Jan 26 12:15:13 2012 +0000
@@ -26,6 +26,9 @@
C
C **OPTIONS FOR TDEP CODE
C
+c
+c Some new feature
+c
iTDep=813
iTrans=822
jTDep=iTDep
diff -r 1de40efd1c4b gdv/l510/a2tdep.F
--- a/gdv/l510/a2tdep.F Wed Jan 18 14:43:13 2012 +0000
+++ b/gdv/l510/a2tdep.F Thu Jan 26 12:15:13 2012 +0000
@@ -16,6 +16,9 @@
C
C **DETERMINE IF WE RUN THE TIMEDEP CODE
C
+c
+c Super duper feature here too
+c
LenTst=0
CALL FILEIO(11,jTDep,LenTst,0,0)
IF (LenTst.EQ.0.OR.iopv.NE.23)RETURN
diff -r 1de40efd1c4b gdv/l510/bar.F
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/gdv/l510/bar.F Thu Jan 26 12:15:13 2012 +0000
@@ -0,0 +1,1 @@
+Stuff in here
testmerge$
testmerge$ hg ci -m 'Added second new feature'
testmerge$ hg log -l 2
changeset: 24:457db23c0b1a
tag: tip
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Thu Jan 26 12:15:54 2012 +0000
summary: Added second new feature

changeset: 23:1de40efd1c4b
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:43:13 2012 +0000
summary: Added tag h13 for changeset 6c81eb8dcbab
testmerge$
</code>
The situation now is that we have two repositories where there are revisions whose parent is revision 23 (23:1de40efd1c4b, in fact). This sort of scenario might arise because we have been working on two different features in two different repositories, or it may be that two users have been working separately. At some point you may wish to merge the work. It's up to you when and how you do this: you may wish to merge in bug fixes quite frequently, and incorporate brand-new features much less frequently. You can of course clone another repository in which to do the merge so that if it isn't satisfactory you can just delete the repository.

First we must pull the revisions from one repository to another. We use the <code>hg incoming</code> and <code>pull</code> commands to do this:
<code>
testmerge$ hg incoming ../testrepo
comparing with ../testrepo
searching for changes
changeset: 24:13ecff0136c2
tag: tip
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 25 17:39:50 2012 +0000
summary: A test message
testmerge$ hg pull ../testrepo
pulling from ../testrepo
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 2 changes to 2 files (+1 heads)
(run 'hg heads' to see heads, 'hg merge' to merge)
testmerge$ hg log -l 3
changeset: 25:13ecff0136c2
tag: tip
parent: 23:1de40efd1c4b
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 25 17:39:50 2012 +0000
summary: A test message

changeset: 24:457db23c0b1a
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Thu Jan 26 12:15:54 2012 +0000
summary: Added second new feature

changeset: 23:1de40efd1c4b
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:43:13 2012 +0000
summary: Added tag h13 for changeset 6c81eb8dcbab
testmerge$
</code>
The <code>hg incoming</code> command takes another repository as its argument and shows a list of revisions that are not present in the current repository. The <code>hg pull</code> command brings those revisions over into the current repository. The <code>hg log</code> command shows changeset 25 in the testmerge repository corresponds to number 24 in the original <code>testrepo</code> repository. Note the long ID is the same in both cases. The <code>hg pull</code> command reports that it has created a new head. We can see this clearly with the <code>hg glog</code> command (from the graphlog extension). The <code>-r 23:</code> flag tells <code>glog</code> to show revisions 23 and greater, for clarity:
<code>
testmerge$ hg glog -r 23:
o changeset: 25:13ecff0136c2
| tag: tip
| parent: 23:1de40efd1c4b
| user: Simon Clifford <simon.j.clifford@gmail.com>
| date: Wed Jan 25 17:39:50 2012 +0000
| summary: A test message
|
| @ changeset: 24:457db23c0b1a
|/ user: Simon Clifford <simon.j.clifford@gmail.com>
| date: Thu Jan 26 12:15:54 2012 +0000
| summary: Added second new feature
|
o changeset: 23:1de40efd1c4b
| user: Simon Clifford <simon.j.clifford@gmail.com>
| date: Wed Jan 18 14:43:13 2012 +0000
| summary: Added tag h13 for changeset 6c81eb8dcbab
|
testmerge$
</code>
The <code>hg push</code> command can also be used to copy revisions from one repository to another. It works much as you would expect, except that by default it does not permit the creation of new heads in the destination repository. The idea is that you tend to pull into your own repository, where you're expected to know what you're doing, while you might be pushing into someone else's repository, where creating a new head might cause confusion. <code>hg outgoing</code> is the corresponding analogue to the <code>hg incoming</code> command.

In our example we now have a situation where we have two heads. This can arise from revisions being created in different repositories and then pulled together, as shown. Alternatively you can just create new heads in one repository. A disadvantage of the single repository technique is that if you decide branch is going nowhere and elect not to merge or proceed with it, it still remains in your repository. With multiple repositories you can just delete the offending repository.

== Merging two sets of changes (hg merge, hg resolve) ==

Let's say that I decide that these two features will work together and I want to merge the two branches. I can use the <code>hg up</code> (<code>hg update</code>) command to update the current working directory to a particular revision in the repository. If no revision is specified it will update to the current branch head. The revision updated to becomes the parent (as shown by <code>hg summary</code> or <code>hg parent</code>) of the working directory. If there are modified files in the working directory the update may attempt a merge. The revision of the working directory when you start to merge is called the base of the merge. This is important if the two revisions you are merging are on different branches, because the newly merged revision will stay on the base branch. Note that the <code>hg pull</code> command does not update the working directory, so in this case it will still be at revision 24. Let's update to revision 25 and then merge the changes from revision 24:

[Which revision to merge from: in general you merge changes into whatever is your current baseline. So, if you're merging the latest updates from Gaussian (from H10 to H11 for example) into your code, you update to your code and merge in the H11 revision. If you're merging in your changes into the soon to be sent to Gaussian group development version you'd start with that and merge in the revision(s) containing your changes]
<code>
testmerge$ hg up 25
3 files updated, 0 files merged, 1 files removed, 0 files unresolved
testmerge$ hg merge 24
merging gdv/l510/a1tdep.F
warning: conflicts during merge.
merging gdv/l510/a1tdep.F failed!
2 files updated, 0 files merged, 0 files removed, 1 files unresolved
use 'hg resolve' to retry unresolved file merges or 'hg update -C .' to abandon
testmerge$
</code>
The software automatically brings in the changes to foo.F, bar.F, and a2tdep.F (as <code>hg status</code> will show). However, in our contrived example <code>a1tdep.F</code> is subject to changes from both revisions. Mercurial will attempt to merge automatically, and in this case it fails. If you edit <code>a1tdep.F</code> you will see it contains the lines:
<code>
<<<<<<< local
c Add super new feature
=======
c Some new feature
>>>>>>> other
</code>
which were inserted during the failed merge. There is also an <code>a1tdep.F.orig</code> file created as a result of the failed merge. As the output from <code>hg merge</code> says you can either now resolve this failed merge or use <code>hg update -C</code> to undo it (the <code>.orig</code> file will remain).

Since we want to resolve the merge we should edit <code>a1tdep.F</code> so that it works. At a bare minimum we will have to remove the “<code><<<<<<< local</code>”, “<code>=======</code>”, and “<code>>>>>>>> other</code>” marker lines. In a more complex situation we might have to completely rewrite this and other files. This is what I mean when I say that mercurial only provides tools to do merging. Even a completely successful merge, from mercurial's point of view, may be completely bogus code.

There are tools available to make this task easier. For example I use <code>vimdiff</code>, a part of the popular <code>vim</code> package. To enable this I have altered my <code>~/.hgrc</code> file to look like this:
<code>
[ui]
merge = vimdiff
username = Simon Clifford <simon.j.clifford@gmail.com>

[extensions]
graphlog=

[merge-tools]
vimdiff.executable = vim
vimdiff.args = -d $base $local $output $other +close +close
</code>
as per http://mercurial.selenic.com/wiki/MergingWithVim. The page http://mercurial.selenic.com/wiki/MergeProgram contains instructions for other tools, including Emacs, and graphical tools. If you edit your <code>~/.hgrc</code> file in this way then you now can type <code>hg resolve --all</code> to run your chosen tool on all unresolved files. Note that by default if your chosen tool exits without an error then <code>hg resolve</code> will regard that file as resolved. To cause <code>vim</code> to exit with an error use <code>:cq</code>. See <code>hg help resolve</code> for more details.

In the example my resolution is to have the relevant lines in <code>a1tdep.F</code> look like this:
<code>
c
c Add super new feature
c Some new feature
c
</code>

Once you have resolved all the files that mercurial thinks need fixing you should check that the final merged result makes sense. Checking that it compiles, or runs tests appropriate to both of the original revisions, for example. When you are satisfied, you should check in the merged revision:

<code>
testmerge$ hg ci -m 'Merged super and duper features'

testmerge$ hg tip

changeset: 26:857f81e59d66
tag: tip
parent: 25:13ecff0136c2
parent: 24:457db23c0b1a
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Thu Jan 26 13:10:52 2012 +0000
summary: Merged super and new features

testmerge$

</code>

You can see that the new revision has ''two'' parent revisions; a merge always consumes a head. This is clear from the output of:

<code>
testmerge$ hg glog -l 4
o changeset: 26:857f81e59d66
|\ tag: tip
| | parent: 25:13ecff0136c2
| | parent: 24:457db23c0b1a
| | user: Simon Clifford <simon.j.clifford@gmail.com>
| | date: Thu Jan 26 13:10:52 2012 +0000
| | summary: Merged super and new features
| |
| o changeset: 25:13ecff0136c2
| | parent: 23:1de40efd1c4b
| | user: Simon Clifford <simon.j.clifford@gmail.com>
| | date: Wed Jan 25 17:39:50 2012 +0000
| | summary: A test message
| |
o | changeset: 24:457db23c0b1a
|/ user: Simon Clifford <simon.j.clifford@gmail.com>
| date: Thu Jan 26 12:15:54 2012 +0000
| summary: Added second new feature
|
o changeset: 23:1de40efd1c4b
| user: Simon Clifford <simon.j.clifford@gmail.com>
| date: Wed Jan 18 14:43:13 2012 +0000
| summary: Added tag h13 for changeset 6c81eb8dcbab
|
testmerge$
</code>

Another example of merging here? Perhaps Lee's code merging in changes on official version.

== Tags and named branches (hg tag, hg tags, hg branch, hg branches) ==

It is often useful to give a name to a particular revision. This might be a changeset that corresponds to a given version of the code, or it might simply be a revision that marks a particular milestone. In the <code>example-gaussian-repo</code> you have probably noticed there are revisions in the history with descriptions like "Added tag h13 for changeset 6c81eb8dcbab". These are revisions corresponding to particular versions of the Gaussian development version, such as H01, H10, H13, etc. You can see a list of the tags in a repository by doing:
<code>
testmerge$ hg tags
tip 26:857f81e59d66
h13 22:6c81eb8dcbab
h12p 20:0ef14d7dff56
h11 18:7daba638a830
h10 16:e1d0af4e84d9
h08 14:656073c38db9
h01 12:737d1720e79a
h13-raw 10:b039f5c274e2
h12p-raw 8:522a8fe79d22
h11-raw 6:a28b789a9555
h10-raw 4:356081dab79d
h08-raw 2:9f69ed4616ad
h01-raw 0:073d6aa63ea7
testmerge$
</code>
(the <code>tip</code> tag is automatically assigned to the most recently checked in revision). You may use a tag name anywhere where you might use a revision number, so for example, to check out the changeset corresponding to the official Gaussian development release H10, you would type:
<code>
testrepo$ hg up h10
13887 files updated, 0 files merged, 0 files removed, 0 files unresolved
testrepo$
</code>

(this will look different if you have unchecked-in modifications in your working directory). You could also pass h10 to the <code>hg clone -u</code> flag, and so on.
You tag a revision with the <code>hg tag</code> command. Issue it in a working directory and it will tag the revision that is the parent of the working directory. Tags become part of the repository, that is they are shared during <code>hg pull</code> and <code>hg push</code>, so choose wisely. If you'd like to have tags that are only part of the local repository, use the <code>-l</code> flag to <code>hg tag</code>.

Tags are stored in an <code>.hgtags</code> file in the root directory of the repository. Running <code>hg tag</code> alters this file and then commits the change to the repository. Sometimes you can see conflicts in this file during merges. I normally just resolve them by keeping as much information in the merged file as possible.

The <code>hg branches</code> command can be used to show the named branches in the repository:
<code>
testrepo$ hg branches
default 24:13ecff0136c2
raw 11:a0c273aeeb2b (inactive)
testrepo$
</code>
Here I have two named branches, raw and default (the default branch is, er, the default). If you examine the full output of <code>hg glog</code>, which is quite long and I won't reproduce it here, you'll see, starting at the bottom, revisions 0 to 11 are all in the raw branch. I have created and named this branch to contain the official Gaussian development code as processed by the <code>old-to-new.sh</code> script, without any of our makefiles or other alterations. The default branch branches off from revision 1 and starts at revision 12. Its descendants, revisions 12 to 23, are also in the default branch. These were created by starting at revision 1, adding our makefiles and modifications to the build system, renaming this branch to the default branch, and then committing revision 12. I tagged revision 12 as "H01", which created revision 13. I then merged revision 3, fixed the conflicts, and committed revision 14, and so on. A more detailed guide on how to import a new official version of gdv exists in LINKY.

You can use branch names just like tags wherever mercurial expects a revision identifier. If you use a branch name mercurial will attempt to give you the revision that is most appropriate, this will usually be the newest head on that branch.

To create a new branch you type <code>hg branch <branchname></code>. This will take effect when you commit the working directory. As I've previously mentioned a simpler alternative to having branches, named or otherwise, in your repository is to have multiple repositories.

== Ignoring files ==

You may be wondering how the system will work when you start compiling files. Won't all the <code>.o</code>, <code>.a</code> and <code>.exe</code> files start showing up in the output to <code>hg status</code>? The answer is that they will not, because I've told mercurial to ignore such files. This information is stored in the <code>.hgignore</code> file in the root of the repository. It's quite long but it starts like this:
<code>
syntax: glob
gdv.make
*.o
*.lo
*.a
*.exe
*.exel
.*.swp
*~
*.log
*.log.gz
*.chk
gdv/arc
...
</code>
Any file that matches either the shell patterns or the file names in <code>.hgignore</code> is ignored by mercurial. This means it does not show up to <code>hg status</code> or <code>hg commit</code>. <code>hg add</code>, <code>hg rm</code>, and so on will ignore it unless it is explicitly mentioned on the command line. So <code>hg add gdv</code> will add all non-ignored files in the gdv directory, while <code>hg add gdv/*</code> will add all the files in the gdv directory.

The <code>.hgignore</code> file is tracked, so please only commit changes to it if you think they will be useful to everyone.
== Other commands ==

There are quite a few other commands available to mercurial, I will cover just a few here, if you're interested in becoming a power user refer to the documentation I have mentioned.

=== hg addremove ===
You can use this command when you have large numbers of files to add and remove. It also includes the <code>-s</code> flag to attempt to detect when files have been moved or copied. Using this flag does require some subtlety, as it's not helpful to mark files as moved or copied unless they have been. Also note that it is possible to inadvertently include ignored files in a mass <code>hg add</code> or <code>hg addremove</code>; check what's being added before you commit.

=== hg grep and hg locate ===
Similar to the standard UNIX grep and locate commands, but cognisant of the mercurial repository.

=== hg init ===
Creates a new mercurial repository in the current directory

=== hg serve ===
This starts a mini web server serving information about the repository. I would strongly recommend you do not use this while working on the Gaussian development version as it would be all too easy to allow any user on the system to view all of the files in your repository. I only mention it here because some of the other documentation may refer to it.

Resgrp:comp-photo-version control short

2012-10-08T13:23:48Z

Scliffor: Short summary version.

This is a short summary of [[Resgrp:comp-photo-version control|this]] page.

== Checking out a repository (hg clone)==
<code>
$ hg clone existing-repo new-repo
</code>

Makes a copy (clone) of the existing-repo and names it new-repo. If new-repo is omitted hg will put the clone in the current directory
and name it ./existing-repo. Repository specifications can be in the ssh:// or http:// formats. Useful flags: <code>-u rev</code> to update the
working directory to rev rather than the default.
<code>-U</code> to not update the working directory at all.

==Seeing the state of the repository and working directory (hg status, hg log, hg summary)==
<code>
$ hg log
</code>

Shows all commits to the repository. Useful flags: <code>-v</code> shows full log message and list of changed files for each entry.
<code>-r rev</code> shows log for rev (which can be a range).

<code>
$ hg stat
</code>

Shows altered / added / deleted files compared to working directory's parent; i.e. shows status of working directory.
Useful flags: various to limit output to only added / altered / deleted / etc files.

<code>
$ hg sum
</code>

Shows summary of working directory.

== Showing the differences (hg diff) ==
<code>
$ hg diff
$ hg diff filename
</code>

Shows difference between parent commit and working directory in 'diff' format. With no argument shows
diffs for all altered files.

==Adding, copying, renaming, and deleting files (hg add, hg forget, hg mv, hg rm, hg cp) ==
<code>
$ hg add filename
$ hg forget filename
$ hg rm filename
$ hg mv old-filename new-filename
$ hg cp old-filename new-filename
</code>

Add places a newly created file under version control (otherwise it will be listed as '?' unknown in hg stat and will not be committed).
Forget reverses an add before it's been committed.
Rm removes a file from the working directory and marks it as deleted to version control for the next commit. <code>rm -A</code> to mark a file
as deleted after you've already deleted it.
Move and copy are similar to add and forget, but allow mercurial to track changes to renamed or copied files.

==Committing the changes (hg commit)==
<code>
$ hg ci
</code>

Check in the working directory as a new changeset. User will be presented with a file to edit where they can write a useful and informative message.
Use <code>-m</code> to pass message on command line.

==Fixing mistakes (hg revert, hg rollback, hg update)==
<code>
$ hg revert filename
$ hg revert -a
</code>

Use this command to ''throw away'' changes you have made to files. The <code>-a</code> flag reverts "all" files in the working directory.
The file(s) are reverted to how they are in the parent changeset. A copy is made of each reverted file into a new file with a .orig suffix.

<code>
$ hg rollback
</code>

Undoes the last commit. This only works once: that is, you can't do an hg rollback directly after an hg rollback. There has to be an hg commit between rollbacks.
If you have pushed or pulled this commit to another repository then rolling it back locally is probably pointless.

<code>
$ hg update --clean rev
</code>

Similar to hg revert, this returns the working directory to a previous changeset rev. Note that
using hg update "without" the
<code>--clean</code> flag will do something different.

==Collaboration (hg incoming, hg pull, hg outgoing, hg push, hg update)==
<code>
$ hg incoming repository-location
$ hg outgoing repository-location
$ hg push repository-location
$ hg pull repository-location
</code>

These commands interact with other repositories. Incoming (and outgoing) shows which changesets the local (remote) repository has
that the other doesn't have. Pull (and push) actually copies the changesets into the local (remote) repository.

<code>
$ hg update rev
</code>

Update attempts to change the parent changeset of the working directory. If there are no changes
in the working directory it will be brought up to date with <code>rev</code> (the tip of the current
branch is used if rev is omitted). If there are changes then <code>hg update</code> will either
abort (if the <code>--check</code> option is used), discard the changes (if <code>--clean</code>
is used, or attempt a merge (if <code>rev</code> is an ancestor or descendant of the working
directory parent).

==Merging two sets of changes (hg merge, hg resolve)==

<code>
$ hg merge rev
</code>

Merge attempts to update the working directory with all changes between the requested revision
and the last common ancestor. Conflicts may arise in files that have changed on both sides of the merge; <code>hg merge</code> may start programs to try to resolve conflicts. Unresolved conflicts are marked in <code>hg merge</code>'s output. A working directory cannot be checked in until
all conflicts are resolved.

<code>
$ hg resolve filename
</code>

Resolve may start a tool to try to resolve conflicting files. It can also be used to
mark files as resolved (<code>-m</code> flag) or unresolved (<code>-u</code> flag).

==Tags (hg tag, hg tags==

<code>
$ hg tag tagname
$ hg tag -r rev tagname
$ hg tags
</code>

Associate <code>tagname</code> with the parent or given revision. This name can be used wherever mercurial expects a revision identifier. Tags are not local unless marked as such and will
be exported to remote repositories on push or pull. <code>hg tags</code> shows all
tags known in this repository.

=Getting help (hg help)==

<code>hg help</code> shows a list of commands, <code>hg help command</code> shows help for a
specific command or concept.

Resgrp:comp-photo-version control short

2012-10-08T13:02:53Z

Scliffor: Created page with "== Checking out a repository (hg clone)== <code> $ hg clone existing-repo new-repo </code> Makes a copy (clone) of the existing-repo and names it new-repo. If new-repo is omitte..."

== Checking out a repository (hg clone)==
<code>
$ hg clone existing-repo new-repo
</code>
Makes a copy (clone) of the existing-repo and names it new-repo. If new-repo is omitted hg will put the clone in the current directory
and name it ./existing-repo. Repository specifications can be in the ssh:// or http:// formats. Useful flags: <code>-u rev</code> to update the
working directory to rev rather than the default.
<code>-U</code> to not update the working directory at all.

==Seeing the state of the repository and working directory (hg status, hg log, hg summary)==
<code>
$ hg log
</code>
Shows all commits to the repository. Useful flags: <code>-v</code> shows full log message and list of changed files for each entry.
<code>-r rev</code> shows log for rev (which can be a range).

<code>
$ hg stat
</code>
Shows altered / added / deleted files compared to working directory's parent; i.e. shows status of working directory.
Useful flags: various to limit output to only added / altered / deleted / etc files.

<code>
$ hg sum
</code>
Shows summary of working directory.

== Showing the differences (hg diff) ==
<code>
$ hg diff
$ hg diff filename
</code>
Shows difference between parent commit and working directory in 'diff' format. With no argument shows
diffs for all altered files.

==Adding, copying, renaming, and deleting files (hg add, hg forget, hg mv, hg rm, hg cp) ==
<code>
$ hg add filename
$ hg forget filename
$ hg rm filename
$ hg mv old-filename new-filename
$ hg cp old-filename new-filename
</code>
Add places a newly created file under version control (otherwise it will be listed as '?' unknown in hg stat and will not be committed).
Forget reverses an add before it's been committed.
Rm removes a file from the working directory and marks it as deleted to version control for the next commit. <code>rm -A</code> to mark a file
as deleted after you've already deleted it.
Move and copy are similar to add and forget, but allow mercurial to track changes to renamed or copied files.

==Committing the changes (hg commit)==
<code>
$ hg ci
</code>
Check in the working directory as a new changeset. User will be presented with a file to edit where they can write a useful and informative message.
Use <code>-m</code> to pass message on command line.

==Fixing mistakes (hg revert, hg rollback, hg update)==
<code>
$ hg revert filename
$ hg revert -a
</code>
Use this command to ''throw away'' changes you have made to files. The <code>-a</code> flag reverts "all" files in the working directory.
The file(s) are reverted to how they are in the parent changeset. A copy is made of each reverted file into a new file with a .orig suffix.

<code>
$ hg rollback
</code>
Undoes the last commit. This only works once: that is, you can't do an hg rollback directly after an hg rollback. There has to be an hg commit between rollbacks.
If you have pushed or pulled this commit to another repository then rolling it back locally is probably pointless.

<code>
$ hg update --clean rev
</code>
Similar to hg revert, this returns the working directory to a previous changeset rev. Note that using hg update "without" the
<code>--clean</code> flag will do something different.

==Collaboration (hg incoming, hg pull, hg outgoing, hg push, hg update, hg parent)==
<code>
$ hg incoming repository-location
$ hg outgoing repository-location
$ hg push repository-location
$ hg pull repository-location
</code>
These commands interact with other repositories. Incoming (and outgoing) shows which changesets the local (remote) repository has
that the other doesn't have. Pull (and push) actually copies the changesets into the local (remote) repository.

<code>
$ hg update
</code>
10 Merging two sets of changes (hg merge, hg resolve)
11 Tags and named branches (hg tag, hg tags, hg branch, hg branches)
12 Ignoring files
13 Other commands
13.1 hg addremove
13.2 hg grep and hg locate
13.3 hg init
13.4 hg serve

Resgrp:comp-photo-version control

2012-10-07T15:12:42Z

Scliffor: Added link to short summary version

== Introduction ==

This document will teach you the basics of using mercurial, a distributed version control system (DVCS). It is particularly directed towards using mercurial with the development version of Gaussian, so the examples will use Gaussian. For more information on mercurial you can use the built-in help system <code>hg help</code>, look at the official website: http://mercurial.selenic.com/wiki/Mercurial, or check out the official O'Reilly book, the text of which is freely available at: http://hgbook.red-bean.com/.

A version control system can sometimes seem an onerous imposition unless the user understands how it is going to help them in their work, so I'll try to briefly explain. Version control is about tracking changes to data. If one has a collection of files, say a distribution of Gaussian, then one is interested in changes that have been made to those files, whether from a new distribution from Gaussian or our own modifications. We are particularly interested in cases where overlapping changes have been made. These might be updates and bug fixes from Gaussian conflicting with our own modifications to a particular link, or it might be different researchers in a group working on the same piece of code independently. A version control system will not only detect such cases but provide automated and safe methods for merging conflicting changes.

Central to the idea of a VCS is the concept of a revision or changeset. This is like a snapshot of some set of files and directories at some particular instant in their history. A repository is where a VCS stores revisions. So, using Gaussian as an example, some revisions in a particular repository might be the Gaussian source tree corresponding to Gaussian development versions H01, H08, H10, H11, etc. The data that are actually kept inside the repository are the differences, or deltas, between the revisions. This saves space compared to storing all the revisions.

== Short version ==

I've made a summary version of this document [[Resgrp:comp-photo-version control short|here]].

== Setting up your environment (.hgrc)==
There are some one-off things you must do. Firstly, edit your login scripts (<code>.bash_profile</code> or similar) to include the command:
<code>
module load mercurial
</code>
(If you're not on an IC HPC machine mercurial is available anywhere that runs Python. It is generally available in distribution package systems, if not you can get it from http://mercurial.selenic.com/wiki/Mercurial). Note that the Redhat provided mercurial on RHEL 5 and 6 is quite old and does not, at time of writing, support the repositories I have created.

Next, create a file called <code>.hgrc</code> in your home directory and insert the following lines:
<code>
[ui]
username = Your Name <your@email.address>

[extensions]
graphlog=
</code>
(so for example, my <code>.hgrc</code> contains this line: <code>username = Simon Clifford <simon.j.clifford@gmail.com></code>).
Mercurial uses this username field to record who has changed what. (If you're not on an IC HPC machine your version of mercurial may not
have the graphlog extension. If it doesn't, don't worry, it's merely a tool for visualising changesets).

== Checking out a repository (hg clone, hg log) ==

Now let's get started and check out a repository. Once you have loaded the mercurial module you will be able to type:
<code>
tmp$ hg clone /home/gaussian-devel/example-gaussian-repo testrepo
tmp$
</code>
This will create a copy of the repository at <code>/home/gaussian-devel/example-gaussian-repo</code> and place it in a directory inside the current directory called <code>testrepo</code>. You can give a full path as the second argument, or, you can leave it off altogether in which case mercurial will clone the repository into a directory named <code>example-gaussian-repo</code>. A note here for Imperial HPC users. Copying a repository involves quite a bit of filesystem activity. I have found that on cx1 this can be quite slow on the <code>/home</code> filesystems. You may wish to try these examples in the <code>/tmp</code> filesystem which is a lot faster. Be aware, however, that this raises two potentially serious problems. The first is that <code>/tmp</code> is globally readable so before you clone you must make sure that your umask is set to <code>0077</code>. The second, of course, is that <code>/tmp</code> is periodically wiped. If you plan on working in <code>/tmp</code> there are simple ways of transferring information between repositories (<code>hg push</code> and <code>hg pull</code>) that I will cover later.

Now change into the <code>testrepo</code> directory. Inside you will see three scripts and a <code>gdv</code> directory; also some hidden files and a hidden directory. Inside the <code>gdv</code> directory you will see the Gaussian source in the [[Resgrp:comp-photo-new gdv layout|new layout]]. The hidden <code>.hg</code> directory inside the <code>testrepo</code> directory is the repository proper; you will almost never need to do anything in here. The rest of the files and directories are called the working directory; this is where you will do your work.

The first thing you might want to do is check the history of this repository to see past revisions that have been checked into it. You do this with the <code>hg log</code> command.
<code>
testrepo$ hg log | head -20
changeset: 23:1de40efd1c4b
tag: tip
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:43:13 2012 +0000
summary: Added tag h13 for changeset 6c81eb8dcbab

changeset: 22:6c81eb8dcbab
tag: h13
parent: 21:5b6ac3665a29
parent: 11:a0c273aeeb2b
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:43:08 2012 +0000
summary: Gaussian devel version H13 with our makefiles

changeset: 21:5b6ac3665a29
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:41:40 2012 +0000
summary: Added tag h12p for changeset 0ef14d7dff56
...
changeset: 1:515a93bfc3b5
branch: raw
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Sun Jan 15 11:39:10 2012 +0000
summary: Added tag h01-raw for changeset 073d6aa63ea7

changeset: 0:073d6aa63ea7
branch: raw
tag: h01-raw
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Sun Jan 15 11:39:09 2012 +0000
summary: Output from old-to-new.sh on version H01
</code>
You will see that there are various fields that may appear for each changeset. The changeset field shows two numbers separated by a colon. The second, longer, hexadecimal number is the unique identifier. A particular hex identifier will always refer to the same changeset, even in different copies of the repository. The first number is a local identifier. These local IDs refer to particular changesets in one copy of the repository; if these changesets are present in another repository they may have a different local identifier. You can use either identifier as an argument to <code>hg</code> commands. The user and date fields show who committed the change and when. The summary field shows the first line of the log entry for that revision. This means it is important to try to make the first line of your log entries a useful summary of what you have done!

To see the entry for a particular revision use the <code>-r</code> flag with a revision number, either short or long. To see more information, including the full log entry and a complete list of all files involved in the change, use the <code>-v</code> flag, e.g.:
<code>
testrepo$ hg log -r 2 -v | less
changeset: 2:9f69ed4616ad
branch: raw
tag: h08-raw
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Sun Jan 15 11:54:50 2012 +0000
files: gdv/archlib/arcvib.F gdv/archlib/brcpyw.F gdv/archlib/letset.F gdv/arctmp.F gdv/bsd/GauDiff_Compare.pm gdv/bsd/G
....
at.F gdv/utilnz/xpndnb.F gdv/utilnz/zerock.F gdv/utilnz/zerod.F gdv/wrappers/ggeev.F gdv/wrappers/lappar.F gdv/wrappers/xgetrf.F gdv/wrappers/xgetrs.F gdv/wrappers/ygeru.F gdv/wrappers/ytrsv.F
description:
Import of gdv H08. See below for details.

Result of these commands on the H01 raw rev:
rm -fr gdv
cp -pr h08/new-style-gdv .
hg add gdv/bsd/*.a
hg addremove -s 50

h08/new-style is from old-to-new.sh on freshly untarred h08.tgz
</code>
As the log entry explains, this particular revision is the change from the H01 version of Gaussian to the H08 version, so there are a lot of changed files. The description entry is intended to provide some human readable explanation of the changes. When you start committing data to the repository you should aim to at least make your log entries understandable.

As with all the commands that I will mention you can get more information by typing <code>hg help <command></code>.

== Committing changes (hg summary, hg status, hg diff) ==

Now let's make some alterations to this repository. This is quite safe; we can't break the original repository because our copy is an entirely separate clone. In fact, this is a very common working practice with mercurial. As long as the filesystem isn't slow cloning a repository can be quite quick. Indeed, cloning a repository into a destination on the same filesystem as the source makes use of hard linking which is very fast and saves space. Therefore, it is very common to clone a repository, make some alterations to it, and then decide whether to push them back to the original repository or just delete the new clone.

First, we want to know where it is we are starting from. To get a quick summary you use the <code>hg summary</code> command. This should show something like this:
<code>
testrepo$ hg summary
parent: 23:1de40efd1c4b tip
Added tag h13 for changeset 6c81eb8dcbab
branch: default
commit: (clean)
update: (current)
</code>
This shows, on the parent line, the revision that has been checked out into the working directory. Below that is the summary line from the log entry for that revision. The commit line shows if any files have been modified since the check out. The update line shows if there are any applicable newer revisions in the repository. In this example the output shows us that we checked out the revision with the local identifier 23, and unique identifier of 1de40efd1c4b, that we have modified nothing and that there are no appropriate newer revisions. This checkout occurred automatically as a part of the <code>hg clone</code> command. You can clone without checking anything out into the working directory (say to make a backup copy of the repository) by passing the <code>-U</code> flag, or you can check out a particular revision with the <code>-u REV</code> flag. See <code>hg help clone</code> for how mercurial chooses what to check out by default.

Let's say we're adding a feature to link 510. If you have something in mind go ahead and do it. For this example I have just added a few lines to the file <code>a1tdep.F</code>, and will assume that your current working directory is <code>gdv/l510</code>.

As you work you may be curious to know what it is you have changed. The command to show this is <code>hg status</code>:
<code>
testrepo$ hg status
M gdv/l510/a1tdep.F
</code>
The output shows a list of files that have been altered in some way. Here the capital M shows that the file <code>a1tdep.F</code> has been modified. By default <code>hg status</code> does not report on unchanged files. If we wish to see how <code>a1tdep.F</code> has been modified we can use the <code>hg diff</code> command:
<code>
testrepo$ hg diff a1tdep.F
diff -r 1de40efd1c4b gdv/l510/a1tdep.F
--- a/gdv/l510/a1tdep.F Wed Jan 18 14:43:13 2012 +0000
+++ b/gdv/l510/a1tdep.F Wed Jan 25 11:07:55 2012 +0000
@@ -26,6 +26,9 @@
C
C **OPTIONS FOR TDEP CODE
C
+c
+c Add super new feature
+c
iTDep=813
iTrans=822
jTDep=iTDep
</code>
This shows the output in a unified different format, refer to the man page for diff for more information.

== Adding, copying, renaming, and deleting file (hg add, hg forget, hg mv, hg rm, hg cp) ==

If you have created a new file then you must let mercurial know about its existence with the <code>hg add</code> command:
<code>
testrepo$ echo "a new subroutine" > foo.F
testrepo$ hg status
M gdv/l510/a1tdep.F
? gdv/l510/foo.F
testrepo$ hg add foo.F
testrepo$ hg status
M gdv/l510/a1tdep.F
A gdv/l510/foo.F
</code>
Here, I create a new file called <code>foo.F</code>. <code>hg status</code> shows the file with a ?, to indicate that it is unknown to the repository. I then run <code>hg add foo.F</code>, after which <code>hg status</code> shows the file with an A. This indicates that the file is scheduled to be added at the next commit. If you change your mind about the file before the next commit you can use <code>hg forget</code> to undo the add.

If you are renaming a file, including the situation where you move the file from one directory to another (e.g. from l510 to utilam) then you may use the <code>hg mv</code> command. This will actually perform the move just like the normal <code>mv</code> command. If you've already moved the file you can give <code>hg mv</code> the <code>-A</code> flag to record the rename after the fact. Similarly, the <code>hg rm</code> command removes files from the working directory and records this fact in the repository, and <code>hg cp</code> copies a file.

It is important to realise that these commands act on the working directory immediately but only affect the repository when they are committed. Also, they do not affect the repository's previous history, so removing a file and then committing that change does not affect that file's existence in previous revisions in the repository. However, <code>hg mv</code> and <code>hg cp</code> create a relationship between the source and destination files. This becomes useful when merging in changes from somewhere else. So for example, above I have made changes to <code>l510/a1tdep.F</code>. Let's say that someone else has decided to make this into a utility routine and has done the command <code>hg mv l510/a1tdep.F utilam/a1tdep.F</code>. If I choose to merge my changes with this other person's, mercurial will correctly apply the changes '''and''' move the file. If instead of <code>hg mv</code> they had used <code>hg cp</code> then my changes would be applied to both files. This means that you should only use <code>hg cp</code> when it is appropriate that changes that are recorded before the copy are applied to both files. Note that this does not mean that changes to the source file are forever applied to the destination file; this will only occur when merging a revision that does not contain a copy with a revision that does. In general, you can expect mercurial to do the right thing.

== Committing the changes (hg commit, hg tip) ==

The <code>hg commit</code> (like many <code>hg</code> commands, <code>hg</code> commit has an alias, in this case <code>hg ci</code>. I tend to use <code>hg ci</code>) command creates a new revision and records the differences between the entire working directory and the parent revision (as shown on the parent line of <code>hg summary). This will be the same as the output from <code>hg status</code>. When you run the command mercurial will open a text editor. If you wish to specify which editor is used add an <code>editor=vim</code> (for example) line to the <code>[ui]</code> part of your <code>~/.hgrc</code> file. The editor will start with a couple of empty lines and then some lines beginning with "<code>HG:</code>". These are there to give you helpful reminders of what you've changed while you write your log message and will be ignored by mercurial when you commit. If you close the editor without writing anything, or of the editor quits with an error, mercurial will abort the commit. So for our current example:
<code>
testrepo$ hg ci
... [opens vim]
HG: Enter commit message. Lines beginning with 'HG:' are removed.
HG: Leave message empty to abort commit.
HG: --
HG: user: Simon Clifford <simon.j.clifford@gmail.com>
HG: branch 'default'
HG: added gdv/l510/foo.F
HG: changed gdv/l510/a1tdep.F
~
~
</code>
For short messages you can skip the editor step by passing the <code>-m "Log message here"</code> flag.

Understanding what you are doing here is essential to knowing what to write in the log and, importantly, when to commit. There are two schools of thought. When you commit you are creating a new revision in your local repository. Since it is your repository, and as I have previously mentioned, it is very simple to create new repositories, you should commit whenever you feel like it and feel free to write cryptic log messages that only you will understand. On the other hand, you are engaged in a collaborative exercise with other people: creating what will be a single version of Gaussian that contains yours and others' modifications and Gaussian's updates. Therefore, you should only commit when your code satisfies pre-agreed group requirements, and your log message should be detailed and comprehensible to anybody who reads it, even 50 years in the future. The correct approach, of course, is to do both.

Mercurial is a distributed version control system which means that each repository is the responsibility of its owner who can feel free to commit as frequently (or infrequently) as needed. Sometimes adding a feature or removing a bug might take weeks, and you might feel that there is no point taking snapshots of the code until it works. Other modifications might proceed incrementally with naturally defined stopping points between the start and the end. Committing at these points makes perfect sense: not only does it leave a history of the modification, it provides checkpoints, versions of the code that you can retreat to without having to start again. The log messages may tersely explain what has happened since the last revision (and note there is no point listing modified, added, etc, files as this information is stored in the commit anyway) or they may contain detailed essays on how a bug arose and the steps you have taken to fix it. Information in the logs is searchable and should be thought of as both an aide memoir for yourself and a journal entry for others.

When the time comes to merge your changes into other people's repositories, you might start thinking about the quality of your commit. Does the code compile? Does it run the test suite? Can it run any test? Have you committed a test that checks the code you have added or altered? The group may decide that revisions that are being shared must answer yes to some or all of these questions. You may decide that some or all of your revisions must pass these quality checks. Or, you may be satisfied with noting in the log entry what this particular revision can or can't do (such as compile or run).

It should be said though, that if you ever think "perhaps I should check in at this point", then go for it. Revisions live in the history of your repository forever (although see <code>hg rollback</code>), but when you transfer them to other people's repositories multiple revisions can be collapsed into one, or fixed in other ways.

Returning to our example I enter a simple message. Since <code>hg log</code> only shows the first line of any log entry by default it is a good idea to make this line a summary of the rest of the message. I close the editor and the commit is done.

We can see the results in the repository with an <code>hg log</code> command (the <code>-l 2</code> flag shows the last two revisions):
<code>
testrepo$ hg log -l 2
changeset: 24:13ecff0136c2
tag: tip
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 25 17:13:00 2012 +0000
summary: A test message.

changeset: 23:1de40efd1c4b
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:43:13 2012 +0000
summary: Added tag h13 for changeset 6c81eb8dcbab
</code>
Alternatively we can use the <code>hg tip</code> command to show the most recently added repository, whether by ourselves, or by pulling from another repository (see later).

== Fixing mistakes (hg revert, hg rollback) ==

You can return one or more files, or the entire repository, to the state they were in when you last checked them out with in the <code>hg revert</code> command. Let's make an ill-advised alteration to our now committed change to <code>a1tdep.F</code>:
<code>
testrepo$ sed -i 's/super/super duper/' a1tdep.F
testrepo$ hg status
M gdv/l510/a1tdep.F
testrepo$ hg diff a1tdep.F
diff -r 13ecff0136c2 gdv/l510/a1tdep.F
--- a/gdv/l510/a1tdep.F Wed Jan 25 17:13:00 2012 +0000
+++ b/gdv/l510/a1tdep.F Wed Jan 25 17:23:42 2012 +0000
@@ -27,7 +27,7 @@
C **OPTIONS FOR TDEP CODE
C
c
-c Add super new feature
+c Add super duper new feature
c
iTDep=813
iTrans=822
testrepo$ hg revert a1tdep.F
testrepo$ hg status
? gdv/l510/a1tdep.F.orig
testrepo$ hg diff a1tdep.F
testrepo$ rm a1tdep.*
testrepo$ hg status
! gdv/l510/a1tdep.F
testrepo$
</code>
Here I use revert to undo the modifications to a file. Notice that <code>hg revert</code> leaves a copy of the modified file in <code>a1tdep.F.orig</code>. This shows up in <code>hg status</code> as an unknown file ("?"). Also notice that while trying to delete the .orig file I have accidentally deleted <code>a1tdep.F</code>, which now shows up in <code>hg status</code> as a missing file ("!"). I can revert this mistake too:
<code>
testrepo$ hg revert a1tdep.F
testrepo$ hg status
testrepo$ ls a1tdep.F
a1tdep.F
testrepo$
</code>
<code>hg revert</code> can also be used to cancel scheduled adds, removes, copies, and renames.

If you have just committed a revision and then change your mind you may be able to undo the effect with <code>hg rollback</code>. Doing this for our example gives us:
<code>
testrepo$ hg rollback
repository tip rolled back to revision 23 (undo commit)
working directory now based on revision 23
testrepo$ hg status
M gdv/l510/a1tdep.F
A gdv/l510/foo.F
testrepo$
</code>
This removes the revision we just checked in from the repository but does not alter the current working directory. If we choose, we could now use <code>hg revert</code> to restore these to their revision 23 state. There are two important caveats with <code>hg rollback</code>: it can only remove the last checked in revision, and that it is usually pointless to rollback a revision that has already been pushed or pulled (see later) into somebody else's repository, as we can only affect our repository. For the purposes of this example, I will once more check in the changes I have made:
<code>
testrepo$ hg ci -m 'A test message'
testrepo$ hg tip
changeset: 24:13ecff0136c2
tag: tip
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 25 17:38:52 2012 +0000
summary: A test message
testrepo$
</code>

== Collaboration (hg incoming, hg pull, hg outgoing, hg push, hg update, hg parent) ==

The tools described are already quite useful, and form the basis of the very earliest revision control systems. The benefits of being able to detect what you've changed, and how you've changed it, being able to undo the changes selectively, and being able to take a snapshot of your work at any point of your choosing should be apparent. The true power, however, of a modern version control system is how it mediates different streams of changes, particularly those from other users. Note that a DVCS does not solve problems of merging and so on, but provides you, the user, with tools to solve them.

Revisions in the repository form a directed acyclic graph (DAG). Every revision apart from the first has at least one parent revision and may have zero or more child revisions. A revision with no children is called a ''head''. Generally development takes place at a head of the repository: checking in a new revision onto a head creates and consumes a head. However, it is possible to add a child to any revision, possibly creating a new head. This is known as a branch, and may be named or unnamed. A revision with children may still be a branch head if it has no children on its branch.

There are various ways of managing branches. You can have a few repositories with many branches, named and unnamed. Or you can have many repositories with largely unbranched graphs; it's up to you. Since mercurial uses heads as the default targets for some of its operations the latter approach is probably best for beginners as it makes operations within each repository simpler.

This is probably best illustrated with an example. I'll make a copy of the original repository, make a different change, and then merge the changes. Change directory out of the <code>testrepo</code> repository and type:
<code>
tmp$ hg clone /home/gaussiandevel/example-gaussian-repo testmerge
updating to branch default
13427 files updated, 0 files merged, 0 files removed, 0 files unresolved
tmp$
</code>

Now we go into the new repository and make some modifications. This time I alter the files <code>a1tdep.F</code> and <code>a2tdep.F</code>, and add a file <code>bar.F</code> in the <code>l510</code> directory:
<code>
…[alterations to a1tdep.F, a2tdep.F, and bar.F]...
testmerge$ hg add gdv/l510/bar.F
testmerge$ hg diff
diff -r 1de40efd1c4b gdv/l510/a1tdep.F
--- a/gdv/l510/a1tdep.F Wed Jan 18 14:43:13 2012 +0000
+++ b/gdv/l510/a1tdep.F Thu Jan 26 12:15:13 2012 +0000
@@ -26,6 +26,9 @@
C
C **OPTIONS FOR TDEP CODE
C
+c
+c Some new feature
+c
iTDep=813
iTrans=822
jTDep=iTDep
diff -r 1de40efd1c4b gdv/l510/a2tdep.F
--- a/gdv/l510/a2tdep.F Wed Jan 18 14:43:13 2012 +0000
+++ b/gdv/l510/a2tdep.F Thu Jan 26 12:15:13 2012 +0000
@@ -16,6 +16,9 @@
C
C **DETERMINE IF WE RUN THE TIMEDEP CODE
C
+c
+c Super duper feature here too
+c
LenTst=0
CALL FILEIO(11,jTDep,LenTst,0,0)
IF (LenTst.EQ.0.OR.iopv.NE.23)RETURN
diff -r 1de40efd1c4b gdv/l510/bar.F
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/gdv/l510/bar.F Thu Jan 26 12:15:13 2012 +0000
@@ -0,0 +1,1 @@
+Stuff in here
testmerge$
testmerge$ hg ci -m 'Added second new feature'
testmerge$ hg log -l 2
changeset: 24:457db23c0b1a
tag: tip
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Thu Jan 26 12:15:54 2012 +0000
summary: Added second new feature

changeset: 23:1de40efd1c4b
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:43:13 2012 +0000
summary: Added tag h13 for changeset 6c81eb8dcbab
testmerge$
</code>
The situation now is that we have two repositories where there are revisions whose parent is revision 23 (23:1de40efd1c4b, in fact). This sort of scenario might arise because we have been working on two different features in two different repositories, or it may be that two users have been working separately. At some point you may wish to merge the work. It's up to you when and how you do this: you may wish to merge in bug fixes quite frequently, and incorporate brand-new features much less frequently. You can of course clone another repository in which to do the merge so that if it isn't satisfactory you can just delete the repository.

First we must pull the revisions from one repository to another. We use the <code>hg incoming</code> and <code>pull</code> commands to do this:
<code>
testmerge$ hg incoming ../testrepo
comparing with ../testrepo
searching for changes
changeset: 24:13ecff0136c2
tag: tip
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 25 17:39:50 2012 +0000
summary: A test message
testmerge$ hg pull ../testrepo
pulling from ../testrepo
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 2 changes to 2 files (+1 heads)
(run 'hg heads' to see heads, 'hg merge' to merge)
testmerge$ hg log -l 3
changeset: 25:13ecff0136c2
tag: tip
parent: 23:1de40efd1c4b
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 25 17:39:50 2012 +0000
summary: A test message

changeset: 24:457db23c0b1a
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Thu Jan 26 12:15:54 2012 +0000
summary: Added second new feature

changeset: 23:1de40efd1c4b
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:43:13 2012 +0000
summary: Added tag h13 for changeset 6c81eb8dcbab
testmerge$
</code>
The <code>hg incoming</code> command takes another repository as its argument and shows a list of revisions that are not present in the current repository. The <code>hg pull</code> command brings those revisions over into the current repository. The <code>hg log</code> command shows changeset 25 in the testmerge repository corresponds to number 24 in the original <code>testrepo</code> repository. Note the long ID is the same in both cases. The <code>hg pull</code> command reports that it has created a new head. We can see this clearly with the <code>hg glog</code> command (from the graphlog extension). The <code>-r 23:</code> flag tells <code>glog</code> to show revisions 23 and greater, for clarity:
<code>
testmerge$ hg glog -r 23:
o changeset: 25:13ecff0136c2
| tag: tip
| parent: 23:1de40efd1c4b
| user: Simon Clifford <simon.j.clifford@gmail.com>
| date: Wed Jan 25 17:39:50 2012 +0000
| summary: A test message
|
| @ changeset: 24:457db23c0b1a
|/ user: Simon Clifford <simon.j.clifford@gmail.com>
| date: Thu Jan 26 12:15:54 2012 +0000
| summary: Added second new feature
|
o changeset: 23:1de40efd1c4b
| user: Simon Clifford <simon.j.clifford@gmail.com>
| date: Wed Jan 18 14:43:13 2012 +0000
| summary: Added tag h13 for changeset 6c81eb8dcbab
|
testmerge$
</code>
The <code>hg push</code> command can also be used to copy revisions from one repository to another. It works much as you would expect, except that by default it does not permit the creation of new heads in the destination repository. The idea is that you tend to pull into your own repository, where you're expected to know what you're doing, while you might be pushing into someone else's repository, where creating a new head might cause confusion. <code>hg outgoing</code> is the corresponding analogue to the <code>hg incoming</code> command.

In our example we now have a situation where we have two heads. This can arise from revisions being created in different repositories and then pulled together, as shown. Alternatively you can just create new heads in one repository. A disadvantage of the single repository technique is that if you decide branch is going nowhere and elect not to merge or proceed with it, it still remains in your repository. With multiple repositories you can just delete the offending repository.

== Merging two sets of changes (hg merge, hg resolve) ==

Let's say that I decide that these two features will work together and I want to merge the two branches. I can use the <code>hg up</code> (<code>hg update</code>) command to update the current working directory to a particular revision in the repository. If no revision is specified it will update to the current branch head. The revision updated to becomes the parent (as shown by <code>hg summary</code> or <code>hg parent</code>) of the working directory. If there are modified files in the working directory the update may attempt a merge. The revision of the working directory when you start to merge is called the base of the merge. This is important if the two revisions you are merging are on different branches, because the newly merged revision will stay on the base branch. Note that the <code>hg pull</code> command does not update the working directory, so in this case it will still be at revision 24. Let's update to revision 25 and then merge the changes from revision 24:

[Which revision to merge from: in general you merge changes into whatever is your current baseline. So, if you're merging the latest updates from Gaussian (from H10 to H11 for example) into your code, you update to your code and merge in the H11 revision. If you're merging in your changes into the soon to be sent to Gaussian group development version you'd start with that and merge in the revision(s) containing your changes]
<code>
testmerge$ hg up 25
3 files updated, 0 files merged, 1 files removed, 0 files unresolved
testmerge$ hg merge 24
merging gdv/l510/a1tdep.F
warning: conflicts during merge.
merging gdv/l510/a1tdep.F failed!
2 files updated, 0 files merged, 0 files removed, 1 files unresolved
use 'hg resolve' to retry unresolved file merges or 'hg update -C .' to abandon
testmerge$
</code>
The software automatically brings in the changes to foo.F, bar.F, and a2tdep.F (as <code>hg status</code> will show). However, in our contrived example <code>a1tdep.F</code> is subject to changes from both revisions. Mercurial will attempt to merge automatically, and in this case it fails. If you edit <code>a1tdep.F</code> you will see it contains the lines:
<code>
<<<<<<< local
c Add super new feature
=======
c Some new feature
>>>>>>> other
</code>
which were inserted during the failed merge. There is also an <code>a1tdep.F.orig</code> file created as a result of the failed merge. As the output from <code>hg merge</code> says you can either now resolve this failed merge or use <code>hg update -C</code> to undo it (the <code>.orig</code> file will remain).

Since we want to resolve the merge we should edit <code>a1tdep.F</code> so that it works. At a bare minimum we will have to remove the “<code><<<<<<< local</code>”, “<code>=======</code>”, and “<code>>>>>>>> other</code>” marker lines. In a more complex situation we might have to completely rewrite this and other files. This is what I mean when I say that mercurial only provides tools to do merging. Even a completely successful merge, from mercurial's point of view, may be completely bogus code.

There are tools available to make this task easier. For example I use <code>vimdiff</code>, a part of the popular <code>vim</code> package. To enable this I have altered my <code>~/.hgrc</code> file to look like this:
<code>
[ui]
merge = vimdiff
username = Simon Clifford <simon.j.clifford@gmail.com>

[extensions]
graphlog=

[merge-tools]
vimdiff.executable = vim
vimdiff.args = -d $base $local $output $other +close +close
</code>
as per http://mercurial.selenic.com/wiki/MergingWithVim. The page http://mercurial.selenic.com/wiki/MergeProgram contains instructions for other tools, including Emacs, and graphical tools. If you edit your <code>~/.hgrc</code> file in this way then you now can type <code>hg resolve --all</code> to run your chosen tool on all unresolved files. Note that by default if your chosen tool exits without an error then <code>hg resolve</code> will regard that file as resolved. To cause <code>vim</code> to exit with an error use <code>:cq</code>. See <code>hg help resolve</code> for more details.

In the example my resolution is to have the relevant lines in <code>a1tdep.F</code> look like this:
<code>
c
c Add super new feature
c Some new feature
c
</code>

Once you have resolved all the files that mercurial thinks need fixing you should check that the final merged result makes sense. Checking that it compiles, or runs tests appropriate to both of the original revisions, for example. When you are satisfied, you should check in the merged revision:

<code>
testmerge$ hg ci -m 'Merged super and duper features'

testmerge$ hg tip

changeset: 26:857f81e59d66
tag: tip
parent: 25:13ecff0136c2
parent: 24:457db23c0b1a
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Thu Jan 26 13:10:52 2012 +0000
summary: Merged super and new features

testmerge$

</code>

You can see that the new revision has ''two'' parent revisions; a merge always consumes a head. This is clear from the output of:

<code>
testmerge$ hg glog -l 4
o changeset: 26:857f81e59d66
|\ tag: tip
| | parent: 25:13ecff0136c2
| | parent: 24:457db23c0b1a
| | user: Simon Clifford <simon.j.clifford@gmail.com>
| | date: Thu Jan 26 13:10:52 2012 +0000
| | summary: Merged super and new features
| |
| o changeset: 25:13ecff0136c2
| | parent: 23:1de40efd1c4b
| | user: Simon Clifford <simon.j.clifford@gmail.com>
| | date: Wed Jan 25 17:39:50 2012 +0000
| | summary: A test message
| |
o | changeset: 24:457db23c0b1a
|/ user: Simon Clifford <simon.j.clifford@gmail.com>
| date: Thu Jan 26 12:15:54 2012 +0000
| summary: Added second new feature
|
o changeset: 23:1de40efd1c4b
| user: Simon Clifford <simon.j.clifford@gmail.com>
| date: Wed Jan 18 14:43:13 2012 +0000
| summary: Added tag h13 for changeset 6c81eb8dcbab
|
testmerge$
</code>

Another example of merging here? Perhaps Lee's code merging in changes on official version.

== Tags and named branches (hg tag, hg tags, hg branch, hg branches) ==

It is often useful to give a name to a particular revision. This might be a changeset that corresponds to a given version of the code, or it might simply be a revision that marks a particular milestone. In the <code>example-gaussian-repo</code> you have probably noticed there are revisions in the history with descriptions like "Added tag h13 for changeset 6c81eb8dcbab". These are revisions corresponding to particular versions of the Gaussian development version, such as H01, H10, H13, etc. You can see a list of the tags in a repository by doing:
<code>
testmerge$ hg tags
tip 26:857f81e59d66
h13 22:6c81eb8dcbab
h12p 20:0ef14d7dff56
h11 18:7daba638a830
h10 16:e1d0af4e84d9
h08 14:656073c38db9
h01 12:737d1720e79a
h13-raw 10:b039f5c274e2
h12p-raw 8:522a8fe79d22
h11-raw 6:a28b789a9555
h10-raw 4:356081dab79d
h08-raw 2:9f69ed4616ad
h01-raw 0:073d6aa63ea7
testmerge$
</code>
(the <code>tip</code> tag is automatically assigned to the most recently checked in revision). You may use a tag name anywhere where you might use a revision number, so for example, to check out the changeset corresponding to the official Gaussian development release H10, you would type:
<code>
testrepo$ hg up h10
13887 files updated, 0 files merged, 0 files removed, 0 files unresolved
testrepo$
</code>

(this will look different if you have unchecked-in modifications in your working directory). You could also pass h10 to the <code>hg clone -u</code> flag, and so on.
You tag a revision with the <code>hg tag</code> command. Issue it in a working directory and it will tag the revision that is the parent of the working directory. Tags become part of the repository, that is they are shared during <code>hg pull</code> and <code>hg push</code>, so choose wisely. If you'd like to have tags that are only part of the local repository, use the <code>-l</code> flag to <code>hg tag</code>.

Tags are stored in an <code>.hgtags</code> file in the root directory of the repository. Running <code>hg tag</code> alters this file and then commits the change to the repository. Sometimes you can see conflicts in this file during merges. I normally just resolve them by keeping as much information in the merged file as possible.

The <code>hg branches</code> command can be used to show the named branches in the repository:
<code>
testrepo$ hg branches
default 24:13ecff0136c2
raw 11:a0c273aeeb2b (inactive)
testrepo$
</code>
Here I have two named branches, raw and default (the default branch is, er, the default). If you examine the full output of <code>hg glog</code>, which is quite long and I won't reproduce it here, you'll see, starting at the bottom, revisions 0 to 11 are all in the raw branch. I have created and named this branch to contain the official Gaussian development code as processed by the <code>old-to-new.sh</code> script, without any of our makefiles or other alterations. The default branch branches off from revision 1 and starts at revision 12. Its descendants, revisions 12 to 23, are also in the default branch. These were created by starting at revision 1, adding our makefiles and modifications to the build system, renaming this branch to the default branch, and then committing revision 12. I tagged revision 12 as "H01", which created revision 13. I then merged revision 3, fixed the conflicts, and committed revision 14, and so on. A more detailed guide on how to import a new official version of gdv exists in LINKY.

You can use branch names just like tags wherever mercurial expects a revision identifier. If you use a branch name mercurial will attempt to give you the revision that is most appropriate, this will usually be the newest head on that branch.

To create a new branch you type <code>hg branch <branchname></code>. This will take effect when you commit the working directory. As I've previously mentioned a simpler alternative to having branches, named or otherwise, in your repository is to have multiple repositories.

== Ignoring files ==

You may be wondering how the system will work when you start compiling files. Won't all the <code>.o</code>, <code>.a</code> and <code>.exe</code> files start showing up in the output to <code>hg status</code>? The answer is that they will not, because I've told mercurial to ignore such files. This information is stored in the <code>.hgignore</code> file in the root of the repository. It's quite long but it starts like this:
<code>
syntax: glob
gdv.make
*.o
*.lo
*.a
*.exe
*.exel
.*.swp
*~
*.log
*.log.gz
*.chk
gdv/arc
...
</code>
Any file that matches either the shell patterns or the file names in <code>.hgignore</code> is ignored by mercurial. This means it does not show up to <code>hg status</code> or <code>hg commit</code>. <code>hg add</code>, <code>hg rm</code>, and so on will ignore it unless it is explicitly mentioned on the command line. So <code>hg add gdv</code> will add all non-ignored files in the gdv directory, while <code>hg add gdv/*</code> will add all the files in the gdv directory.

The <code>.hgignore</code> file is tracked, so please only commit changes to it if you think they will be useful to everyone.
== Other commands ==

There are quite a few other commands available to mercurial, I will cover just a few here, if you're interested in becoming a power user refer to the documentation I have mentioned.

=== hg addremove ===
You can use this command when you have large numbers of files to add and remove. It also includes the <code>-s</code> flag to attempt to detect when files have been moved or copied. Using this flag does require some subtlety, as it's not helpful to mark files as moved or copied unless they have been. Also note that it is possible to inadvertently include ignored files in a mass <code>hg add</code> or <code>hg addremove</code>; check what's being added before you commit.

=== hg grep and hg locate ===
Similar to the standard UNIX grep and locate commands, but cognisant of the mercurial repository.

=== hg init ===
Creates a new mercurial repository in the current directory

=== hg serve ===
This starts a mini web server serving information about the repository. I would strongly recommend you do not use this while working on the Gaussian development version as it would be all too easy to allow any user on the system to view all of the files in your repository. I only mention it here because some of the other documentation may refer to it.

Resgrp:comp-photo-version control

2012-03-21T15:52:34Z

Scliffor: /* Setting up your environment (.hgrc) */

== Introduction ==

This document will teach you the basics of using mercurial, a distributed version control system (DVCS). It is particularly directed towards using mercurial with the development version of Gaussian, so the examples will use Gaussian. For more information on mercurial you can use the built-in help system <code>hg help</code>, look at the official website: http://mercurial.selenic.com/wiki/Mercurial, or check out the official O'Reilly book, the text of which is freely available at: http://hgbook.red-bean.com/.

A version control system can sometimes seem an onerous imposition unless the user understands how it is going to help them in their work, so I'll try to briefly explain. Version control is about tracking changes to data. If one has a collection of files, say a distribution of Gaussian, then one is interested in changes that have been made to those files, whether from a new distribution from Gaussian or our own modifications. We are particularly interested in cases where overlapping changes have been made. These might be updates and bug fixes from Gaussian conflicting with our own modifications to a particular link, or it might be different researchers in a group working on the same piece of code independently. A version control system will not only detect such cases but provide automated and safe methods for merging conflicting changes.

Central to the idea of a VCS is the concept of a revision or changeset. This is like a snapshot of some set of files and directories at some particular instant in their history. A repository is where a VCS stores revisions. So, using Gaussian as an example, some revisions in a particular repository might be the Gaussian source tree corresponding to Gaussian development versions H01, H08, H10, H11, etc. The data that are actually kept inside the repository are the differences, or deltas, between the revisions. This saves space compared to storing all the revisions.

== Setting up your environment (.hgrc)==
There are some one-off things you must do. Firstly, edit your login scripts (<code>.bash_profile</code> or similar) to include the command:
<code>
module load mercurial
</code>
(If you're not on an IC HPC machine mercurial is available anywhere that runs Python. It is generally available in distribution package systems, if not you can get it from http://mercurial.selenic.com/wiki/Mercurial). Note that the Redhat provided mercurial on RHEL 5 and 6 is quite old and does not, at time of writing, support the repositories I have created.

Next, create a file called <code>.hgrc</code> in your home directory and insert the following lines:
<code>
[ui]
username = Your Name <your@email.address>

[extensions]
graphlog=
</code>
(so for example, my <code>.hgrc</code> contains this line: <code>username = Simon Clifford <simon.j.clifford@gmail.com></code>).
Mercurial uses this username field to record who has changed what. (If you're not on an IC HPC machine your version of mercurial may not
have the graphlog extension. If it doesn't, don't worry, it's merely a tool for visualising changesets).

== Checking out a repository (hg clone, hg log) ==

Now let's get started and check out a repository. Once you have loaded the mercurial module you will be able to type:
<code>
tmp$ hg clone /home/gaussian-devel/example-gaussian-repo testrepo
tmp$
</code>
This will create a copy of the repository at <code>/home/gaussian-devel/example-gaussian-repo</code> and place it in a directory inside the current directory called <code>testrepo</code>. You can give a full path as the second argument, or, you can leave it off altogether in which case mercurial will clone the repository into a directory named <code>example-gaussian-repo</code>. A note here for Imperial HPC users. Copying a repository involves quite a bit of filesystem activity. I have found that on cx1 this can be quite slow on the <code>/home</code> filesystems. You may wish to try these examples in the <code>/tmp</code> filesystem which is a lot faster. Be aware, however, that this raises two potentially serious problems. The first is that <code>/tmp</code> is globally readable so before you clone you must make sure that your umask is set to <code>0077</code>. The second, of course, is that <code>/tmp</code> is periodically wiped. If you plan on working in <code>/tmp</code> there are simple ways of transferring information between repositories (<code>hg push</code> and <code>hg pull</code>) that I will cover later.

Now change into the <code>testrepo</code> directory. Inside you will see three scripts and a <code>gdv</code> directory; also some hidden files and a hidden directory. Inside the <code>gdv</code> directory you will see the Gaussian source in the [[Resgrp:comp-photo-new gdv layout|new layout]]. The hidden <code>.hg</code> directory inside the <code>testrepo</code> directory is the repository proper; you will almost never need to do anything in here. The rest of the files and directories are called the working directory; this is where you will do your work.

The first thing you might want to do is check the history of this repository to see past revisions that have been checked into it. You do this with the <code>hg log</code> command.
<code>
testrepo$ hg log | head -20
changeset: 23:1de40efd1c4b
tag: tip
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:43:13 2012 +0000
summary: Added tag h13 for changeset 6c81eb8dcbab

changeset: 22:6c81eb8dcbab
tag: h13
parent: 21:5b6ac3665a29
parent: 11:a0c273aeeb2b
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:43:08 2012 +0000
summary: Gaussian devel version H13 with our makefiles

changeset: 21:5b6ac3665a29
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:41:40 2012 +0000
summary: Added tag h12p for changeset 0ef14d7dff56
...
changeset: 1:515a93bfc3b5
branch: raw
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Sun Jan 15 11:39:10 2012 +0000
summary: Added tag h01-raw for changeset 073d6aa63ea7

changeset: 0:073d6aa63ea7
branch: raw
tag: h01-raw
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Sun Jan 15 11:39:09 2012 +0000
summary: Output from old-to-new.sh on version H01
</code>
You will see that there are various fields that may appear for each changeset. The changeset field shows two numbers separated by a colon. The second, longer, hexadecimal number is the unique identifier. A particular hex identifier will always refer to the same changeset, even in different copies of the repository. The first number is a local identifier. These local IDs refer to particular changesets in one copy of the repository; if these changesets are present in another repository they may have a different local identifier. You can use either identifier as an argument to <code>hg</code> commands. The user and date fields show who committed the change and when. The summary field shows the first line of the log entry for that revision. This means it is important to try to make the first line of your log entries a useful summary of what you have done!

To see the entry for a particular revision use the <code>-r</code> flag with a revision number, either short or long. To see more information, including the full log entry and a complete list of all files involved in the change, use the <code>-v</code> flag, e.g.:
<code>
testrepo$ hg log -r 2 -v | less
changeset: 2:9f69ed4616ad
branch: raw
tag: h08-raw
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Sun Jan 15 11:54:50 2012 +0000
files: gdv/archlib/arcvib.F gdv/archlib/brcpyw.F gdv/archlib/letset.F gdv/arctmp.F gdv/bsd/GauDiff_Compare.pm gdv/bsd/G
....
at.F gdv/utilnz/xpndnb.F gdv/utilnz/zerock.F gdv/utilnz/zerod.F gdv/wrappers/ggeev.F gdv/wrappers/lappar.F gdv/wrappers/xgetrf.F gdv/wrappers/xgetrs.F gdv/wrappers/ygeru.F gdv/wrappers/ytrsv.F
description:
Import of gdv H08. See below for details.

Result of these commands on the H01 raw rev:
rm -fr gdv
cp -pr h08/new-style-gdv .
hg add gdv/bsd/*.a
hg addremove -s 50

h08/new-style is from old-to-new.sh on freshly untarred h08.tgz
</code>
As the log entry explains, this particular revision is the change from the H01 version of Gaussian to the H08 version, so there are a lot of changed files. The description entry is intended to provide some human readable explanation of the changes. When you start committing data to the repository you should aim to at least make your log entries understandable.

As with all the commands that I will mention you can get more information by typing <code>hg help <command></code>.

== Committing changes (hg summary, hg status, hg diff) ==

Now let's make some alterations to this repository. This is quite safe; we can't break the original repository because our copy is an entirely separate clone. In fact, this is a very common working practice with mercurial. As long as the filesystem isn't slow cloning a repository can be quite quick. Indeed, cloning a repository into a destination on the same filesystem as the source makes use of hard linking which is very fast and saves space. Therefore, it is very common to clone a repository, make some alterations to it, and then decide whether to push them back to the original repository or just delete the new clone.

First, we want to know where it is we are starting from. To get a quick summary you use the <code>hg summary</code> command. This should show something like this:
<code>
testrepo$ hg summary
parent: 23:1de40efd1c4b tip
Added tag h13 for changeset 6c81eb8dcbab
branch: default
commit: (clean)
update: (current)
</code>
This shows, on the parent line, the revision that has been checked out into the working directory. Below that is the summary line from the log entry for that revision. The commit line shows if any files have been modified since the check out. The update line shows if there are any applicable newer revisions in the repository. In this example the output shows us that we checked out the revision with the local identifier 23, and unique identifier of 1de40efd1c4b, that we have modified nothing and that there are no appropriate newer revisions. This checkout occurred automatically as a part of the <code>hg clone</code> command. You can clone without checking anything out into the working directory (say to make a backup copy of the repository) by passing the <code>-U</code> flag, or you can check out a particular revision with the <code>-u REV</code> flag. See <code>hg help clone</code> for how mercurial chooses what to check out by default.

Let's say we're adding a feature to link 510. If you have something in mind go ahead and do it. For this example I have just added a few lines to the file <code>a1tdep.F</code>, and will assume that your current working directory is <code>gdv/l510</code>.

As you work you may be curious to know what it is you have changed. The command to show this is <code>hg status</code>:
<code>
testrepo$ hg status
M gdv/l510/a1tdep.F
</code>
The output shows a list of files that have been altered in some way. Here the capital M shows that the file <code>a1tdep.F</code> has been modified. By default <code>hg status</code> does not report on unchanged files. If we wish to see how <code>a1tdep.F</code> has been modified we can use the <code>hg diff</code> command:
<code>
testrepo$ hg diff a1tdep.F
diff -r 1de40efd1c4b gdv/l510/a1tdep.F
--- a/gdv/l510/a1tdep.F Wed Jan 18 14:43:13 2012 +0000
+++ b/gdv/l510/a1tdep.F Wed Jan 25 11:07:55 2012 +0000
@@ -26,6 +26,9 @@
C
C **OPTIONS FOR TDEP CODE
C
+c
+c Add super new feature
+c
iTDep=813
iTrans=822
jTDep=iTDep
</code>
This shows the output in a unified different format, refer to the man page for diff for more information.

== Adding, copying, renaming, and deleting file (hg add, hg forget, hg mv, hg rm, hg cp) ==

If you have created a new file then you must let mercurial know about its existence with the <code>hg add</code> command:
<code>
testrepo$ echo "a new subroutine" > foo.F
testrepo$ hg status
M gdv/l510/a1tdep.F
? gdv/l510/foo.F
testrepo$ hg add foo.F
testrepo$ hg status
M gdv/l510/a1tdep.F
A gdv/l510/foo.F
</code>
Here, I create a new file called <code>foo.F</code>. <code>hg status</code> shows the file with a ?, to indicate that it is unknown to the repository. I then run <code>hg add foo.F</code>, after which <code>hg status</code> shows the file with an A. This indicates that the file is scheduled to be added at the next commit. If you change your mind about the file before the next commit you can use <code>hg forget</code> to undo the add.

If you are renaming a file, including the situation where you move the file from one directory to another (e.g. from l510 to utilam) then you may use the <code>hg mv</code> command. This will actually perform the move just like the normal <code>mv</code> command. If you've already moved the file you can give <code>hg mv</code> the <code>-A</code> flag to record the rename after the fact. Similarly, the <code>hg rm</code> command removes files from the working directory and records this fact in the repository, and <code>hg cp</code> copies a file.

It is important to realise that these commands act on the working directory immediately but only affect the repository when they are committed. Also, they do not affect the repository's previous history, so removing a file and then committing that change does not affect that file's existence in previous revisions in the repository. However, <code>hg mv</code> and <code>hg cp</code> create a relationship between the source and destination files. This becomes useful when merging in changes from somewhere else. So for example, above I have made changes to <code>l510/a1tdep.F</code>. Let's say that someone else has decided to make this into a utility routine and has done the command <code>hg mv l510/a1tdep.F utilam/a1tdep.F</code>. If I choose to merge my changes with this other person's, mercurial will correctly apply the changes '''and''' move the file. If instead of <code>hg mv</code> they had used <code>hg cp</code> then my changes would be applied to both files. This means that you should only use <code>hg cp</code> when it is appropriate that changes that are recorded before the copy are applied to both files. Note that this does not mean that changes to the source file are forever applied to the destination file; this will only occur when merging a revision that does not contain a copy with a revision that does. In general, you can expect mercurial to do the right thing.

== Committing the changes (hg commit, hg tip) ==

The <code>hg commit</code> (like many <code>hg</code> commands, <code>hg</code> commit has an alias, in this case <code>hg ci</code>. I tend to use <code>hg ci</code>) command creates a new revision and records the differences between the entire working directory and the parent revision (as shown on the parent line of <code>hg summary). This will be the same as the output from <code>hg status</code>. When you run the command mercurial will open a text editor. If you wish to specify which editor is used add an <code>editor=vim</code> (for example) line to the <code>[ui]</code> part of your <code>~/.hgrc</code> file. The editor will start with a couple of empty lines and then some lines beginning with "<code>HG:</code>". These are there to give you helpful reminders of what you've changed while you write your log message and will be ignored by mercurial when you commit. If you close the editor without writing anything, or of the editor quits with an error, mercurial will abort the commit. So for our current example:
<code>
testrepo$ hg ci
... [opens vim]
HG: Enter commit message. Lines beginning with 'HG:' are removed.
HG: Leave message empty to abort commit.
HG: --
HG: user: Simon Clifford <simon.j.clifford@gmail.com>
HG: branch 'default'
HG: added gdv/l510/foo.F
HG: changed gdv/l510/a1tdep.F
~
~
</code>
For short messages you can skip the editor step by passing the <code>-m "Log message here"</code> flag.

Understanding what you are doing here is essential to knowing what to write in the log and, importantly, when to commit. There are two schools of thought. When you commit you are creating a new revision in your local repository. Since it is your repository, and as I have previously mentioned, it is very simple to create new repositories, you should commit whenever you feel like it and feel free to write cryptic log messages that only you will understand. On the other hand, you are engaged in a collaborative exercise with other people: creating what will be a single version of Gaussian that contains yours and others' modifications and Gaussian's updates. Therefore, you should only commit when your code satisfies pre-agreed group requirements, and your log message should be detailed and comprehensible to anybody who reads it, even 50 years in the future. The correct approach, of course, is to do both.

Mercurial is a distributed version control system which means that each repository is the responsibility of its owner who can feel free to commit as frequently (or infrequently) as needed. Sometimes adding a feature or removing a bug might take weeks, and you might feel that there is no point taking snapshots of the code until it works. Other modifications might proceed incrementally with naturally defined stopping points between the start and the end. Committing at these points makes perfect sense: not only does it leave a history of the modification, it provides checkpoints, versions of the code that you can retreat to without having to start again. The log messages may tersely explain what has happened since the last revision (and note there is no point listing modified, added, etc, files as this information is stored in the commit anyway) or they may contain detailed essays on how a bug arose and the steps you have taken to fix it. Information in the logs is searchable and should be thought of as both an aide memoir for yourself and a journal entry for others.

When the time comes to merge your changes into other people's repositories, you might start thinking about the quality of your commit. Does the code compile? Does it run the test suite? Can it run any test? Have you committed a test that checks the code you have added or altered? The group may decide that revisions that are being shared must answer yes to some or all of these questions. You may decide that some or all of your revisions must pass these quality checks. Or, you may be satisfied with noting in the log entry what this particular revision can or can't do (such as compile or run).

It should be said though, that if you ever think "perhaps I should check in at this point", then go for it. Revisions live in the history of your repository forever (although see <code>hg rollback</code>), but when you transfer them to other people's repositories multiple revisions can be collapsed into one, or fixed in other ways.

Returning to our example I enter a simple message. Since <code>hg log</code> only shows the first line of any log entry by default it is a good idea to make this line a summary of the rest of the message. I close the editor and the commit is done.

We can see the results in the repository with an <code>hg log</code> command (the <code>-l 2</code> flag shows the last two revisions):
<code>
testrepo$ hg log -l 2
changeset: 24:13ecff0136c2
tag: tip
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 25 17:13:00 2012 +0000
summary: A test message.

changeset: 23:1de40efd1c4b
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:43:13 2012 +0000
summary: Added tag h13 for changeset 6c81eb8dcbab
</code>
Alternatively we can use the <code>hg tip</code> command to show the most recently added repository, whether by ourselves, or by pulling from another repository (see later).

== Fixing mistakes (hg revert, hg rollback) ==

You can return one or more files, or the entire repository, to the state they were in when you last checked them out with in the <code>hg revert</code> command. Let's make an ill-advised alteration to our now committed change to <code>a1tdep.F</code>:
<code>
testrepo$ sed -i 's/super/super duper/' a1tdep.F
testrepo$ hg status
M gdv/l510/a1tdep.F
testrepo$ hg diff a1tdep.F
diff -r 13ecff0136c2 gdv/l510/a1tdep.F
--- a/gdv/l510/a1tdep.F Wed Jan 25 17:13:00 2012 +0000
+++ b/gdv/l510/a1tdep.F Wed Jan 25 17:23:42 2012 +0000
@@ -27,7 +27,7 @@
C **OPTIONS FOR TDEP CODE
C
c
-c Add super new feature
+c Add super duper new feature
c
iTDep=813
iTrans=822
testrepo$ hg revert a1tdep.F
testrepo$ hg status
? gdv/l510/a1tdep.F.orig
testrepo$ hg diff a1tdep.F
testrepo$ rm a1tdep.*
testrepo$ hg status
! gdv/l510/a1tdep.F
testrepo$
</code>
Here I use revert to undo the modifications to a file. Notice that <code>hg revert</code> leaves a copy of the modified file in <code>a1tdep.F.orig</code>. This shows up in <code>hg status</code> as an unknown file ("?"). Also notice that while trying to delete the .orig file I have accidentally deleted <code>a1tdep.F</code>, which now shows up in <code>hg status</code> as a missing file ("!"). I can revert this mistake too:
<code>
testrepo$ hg revert a1tdep.F
testrepo$ hg status
testrepo$ ls a1tdep.F
a1tdep.F
testrepo$
</code>
<code>hg revert</code> can also be used to cancel scheduled adds, removes, copies, and renames.

If you have just committed a revision and then change your mind you may be able to undo the effect with <code>hg rollback</code>. Doing this for our example gives us:
<code>
testrepo$ hg rollback
repository tip rolled back to revision 23 (undo commit)
working directory now based on revision 23
testrepo$ hg status
M gdv/l510/a1tdep.F
A gdv/l510/foo.F
testrepo$
</code>
This removes the revision we just checked in from the repository but does not alter the current working directory. If we choose, we could now use <code>hg revert</code> to restore these to their revision 23 state. There are two important caveats with <code>hg rollback</code>: it can only remove the last checked in revision, and that it is usually pointless to rollback a revision that has already been pushed or pulled (see later) into somebody else's repository, as we can only affect our repository. For the purposes of this example, I will once more check in the changes I have made:
<code>
testrepo$ hg ci -m 'A test message'
testrepo$ hg tip
changeset: 24:13ecff0136c2
tag: tip
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 25 17:38:52 2012 +0000
summary: A test message
testrepo$
</code>

== Collaboration (hg incoming, hg pull, hg outgoing, hg push, hg update, hg parent) ==

The tools described are already quite useful, and form the basis of the very earliest revision control systems. The benefits of being able to detect what you've changed, and how you've changed it, being able to undo the changes selectively, and being able to take a snapshot of your work at any point of your choosing should be apparent. The true power, however, of a modern version control system is how it mediates different streams of changes, particularly those from other users. Note that a DVCS does not solve problems of merging and so on, but provides you, the user, with tools to solve them.

Revisions in the repository form a directed acyclic graph (DAG). Every revision apart from the first has at least one parent revision and may have zero or more child revisions. A revision with no children is called a ''head''. Generally development takes place at a head of the repository: checking in a new revision onto a head creates and consumes a head. However, it is possible to add a child to any revision, possibly creating a new head. This is known as a branch, and may be named or unnamed. A revision with children may still be a branch head if it has no children on its branch.

There are various ways of managing branches. You can have a few repositories with many branches, named and unnamed. Or you can have many repositories with largely unbranched graphs; it's up to you. Since mercurial uses heads as the default targets for some of its operations the latter approach is probably best for beginners as it makes operations within each repository simpler.

This is probably best illustrated with an example. I'll make a copy of the original repository, make a different change, and then merge the changes. Change directory out of the <code>testrepo</code> repository and type:
<code>
tmp$ hg clone /home/gaussiandevel/example-gaussian-repo testmerge
updating to branch default
13427 files updated, 0 files merged, 0 files removed, 0 files unresolved
tmp$
</code>

Now we go into the new repository and make some modifications. This time I alter the files <code>a1tdep.F</code> and <code>a2tdep.F</code>, and add a file <code>bar.F</code> in the <code>l510</code> directory:
<code>
…[alterations to a1tdep.F, a2tdep.F, and bar.F]...
testmerge$ hg add gdv/l510/bar.F
testmerge$ hg diff
diff -r 1de40efd1c4b gdv/l510/a1tdep.F
--- a/gdv/l510/a1tdep.F Wed Jan 18 14:43:13 2012 +0000
+++ b/gdv/l510/a1tdep.F Thu Jan 26 12:15:13 2012 +0000
@@ -26,6 +26,9 @@
C
C **OPTIONS FOR TDEP CODE
C
+c
+c Some new feature
+c
iTDep=813
iTrans=822
jTDep=iTDep
diff -r 1de40efd1c4b gdv/l510/a2tdep.F
--- a/gdv/l510/a2tdep.F Wed Jan 18 14:43:13 2012 +0000
+++ b/gdv/l510/a2tdep.F Thu Jan 26 12:15:13 2012 +0000
@@ -16,6 +16,9 @@
C
C **DETERMINE IF WE RUN THE TIMEDEP CODE
C
+c
+c Super duper feature here too
+c
LenTst=0
CALL FILEIO(11,jTDep,LenTst,0,0)
IF (LenTst.EQ.0.OR.iopv.NE.23)RETURN
diff -r 1de40efd1c4b gdv/l510/bar.F
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/gdv/l510/bar.F Thu Jan 26 12:15:13 2012 +0000
@@ -0,0 +1,1 @@
+Stuff in here
testmerge$
testmerge$ hg ci -m 'Added second new feature'
testmerge$ hg log -l 2
changeset: 24:457db23c0b1a
tag: tip
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Thu Jan 26 12:15:54 2012 +0000
summary: Added second new feature

changeset: 23:1de40efd1c4b
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:43:13 2012 +0000
summary: Added tag h13 for changeset 6c81eb8dcbab
testmerge$
</code>
The situation now is that we have two repositories where there are revisions whose parent is revision 23 (23:1de40efd1c4b, in fact). This sort of scenario might arise because we have been working on two different features in two different repositories, or it may be that two users have been working separately. At some point you may wish to merge the work. It's up to you when and how you do this: you may wish to merge in bug fixes quite frequently, and incorporate brand-new features much less frequently. You can of course clone another repository in which to do the merge so that if it isn't satisfactory you can just delete the repository.

First we must pull the revisions from one repository to another. We use the <code>hg incoming</code> and <code>pull</code> commands to do this:
<code>
testmerge$ hg incoming ../testrepo
comparing with ../testrepo
searching for changes
changeset: 24:13ecff0136c2
tag: tip
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 25 17:39:50 2012 +0000
summary: A test message
testmerge$ hg pull ../testrepo
pulling from ../testrepo
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 2 changes to 2 files (+1 heads)
(run 'hg heads' to see heads, 'hg merge' to merge)
testmerge$ hg log -l 3
changeset: 25:13ecff0136c2
tag: tip
parent: 23:1de40efd1c4b
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 25 17:39:50 2012 +0000
summary: A test message

changeset: 24:457db23c0b1a
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Thu Jan 26 12:15:54 2012 +0000
summary: Added second new feature

changeset: 23:1de40efd1c4b
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:43:13 2012 +0000
summary: Added tag h13 for changeset 6c81eb8dcbab
testmerge$
</code>
The <code>hg incoming</code> command takes another repository as its argument and shows a list of revisions that are not present in the current repository. The <code>hg pull</code> command brings those revisions over into the current repository. The <code>hg log</code> command shows changeset 25 in the testmerge repository corresponds to number 24 in the original <code>testrepo</code> repository. Note the long ID is the same in both cases. The <code>hg pull</code> command reports that it has created a new head. We can see this clearly with the <code>hg glog</code> command (from the graphlog extension). The <code>-r 23:</code> flag tells <code>glog</code> to show revisions 23 and greater, for clarity:
<code>
testmerge$ hg glog -r 23:
o changeset: 25:13ecff0136c2
| tag: tip
| parent: 23:1de40efd1c4b
| user: Simon Clifford <simon.j.clifford@gmail.com>
| date: Wed Jan 25 17:39:50 2012 +0000
| summary: A test message
|
| @ changeset: 24:457db23c0b1a
|/ user: Simon Clifford <simon.j.clifford@gmail.com>
| date: Thu Jan 26 12:15:54 2012 +0000
| summary: Added second new feature
|
o changeset: 23:1de40efd1c4b
| user: Simon Clifford <simon.j.clifford@gmail.com>
| date: Wed Jan 18 14:43:13 2012 +0000
| summary: Added tag h13 for changeset 6c81eb8dcbab
|
testmerge$
</code>
The <code>hg push</code> command can also be used to copy revisions from one repository to another. It works much as you would expect, except that by default it does not permit the creation of new heads in the destination repository. The idea is that you tend to pull into your own repository, where you're expected to know what you're doing, while you might be pushing into someone else's repository, where creating a new head might cause confusion. <code>hg outgoing</code> is the corresponding analogue to the <code>hg incoming</code> command.

In our example we now have a situation where we have two heads. This can arise from revisions being created in different repositories and then pulled together, as shown. Alternatively you can just create new heads in one repository. A disadvantage of the single repository technique is that if you decide branch is going nowhere and elect not to merge or proceed with it, it still remains in your repository. With multiple repositories you can just delete the offending repository.

== Merging two sets of changes (hg merge, hg resolve) ==

Let's say that I decide that these two features will work together and I want to merge the two branches. I can use the <code>hg up</code> (<code>hg update</code>) command to update the current working directory to a particular revision in the repository. If no revision is specified it will update to the current branch head. The revision updated to becomes the parent (as shown by <code>hg summary</code> or <code>hg parent</code>) of the working directory. If there are modified files in the working directory the update may attempt a merge. The revision of the working directory when you start to merge is called the base of the merge. This is important if the two revisions you are merging are on different branches, because the newly merged revision will stay on the base branch. Note that the <code>hg pull</code> command does not update the working directory, so in this case it will still be at revision 24. Let's update to revision 25 and then merge the changes from revision 24:

[Which revision to merge from: in general you merge changes into whatever is your current baseline. So, if you're merging the latest updates from Gaussian (from H10 to H11 for example) into your code, you update to your code and merge in the H11 revision. If you're merging in your changes into the soon to be sent to Gaussian group development version you'd start with that and merge in the revision(s) containing your changes]
<code>
testmerge$ hg up 25
3 files updated, 0 files merged, 1 files removed, 0 files unresolved
testmerge$ hg merge 24
merging gdv/l510/a1tdep.F
warning: conflicts during merge.
merging gdv/l510/a1tdep.F failed!
2 files updated, 0 files merged, 0 files removed, 1 files unresolved
use 'hg resolve' to retry unresolved file merges or 'hg update -C .' to abandon
testmerge$
</code>
The software automatically brings in the changes to foo.F, bar.F, and a2tdep.F (as <code>hg status</code> will show). However, in our contrived example <code>a1tdep.F</code> is subject to changes from both revisions. Mercurial will attempt to merge automatically, and in this case it fails. If you edit <code>a1tdep.F</code> you will see it contains the lines:
<code>
<<<<<<< local
c Add super new feature
=======
c Some new feature
>>>>>>> other
</code>
which were inserted during the failed merge. There is also an <code>a1tdep.F.orig</code> file created as a result of the failed merge. As the output from <code>hg merge</code> says you can either now resolve this failed merge or use <code>hg update -C</code> to undo it (the <code>.orig</code> file will remain).

Since we want to resolve the merge we should edit <code>a1tdep.F</code> so that it works. At a bare minimum we will have to remove the “<code><<<<<<< local</code>”, “<code>=======</code>”, and “<code>>>>>>>> other</code>” marker lines. In a more complex situation we might have to completely rewrite this and other files. This is what I mean when I say that mercurial only provides tools to do merging. Even a completely successful merge, from mercurial's point of view, may be completely bogus code.

There are tools available to make this task easier. For example I use <code>vimdiff</code>, a part of the popular <code>vim</code> package. To enable this I have altered my <code>~/.hgrc</code> file to look like this:
<code>
[ui]
merge = vimdiff
username = Simon Clifford <simon.j.clifford@gmail.com>

[extensions]
graphlog=

[merge-tools]
vimdiff.executable = vim
vimdiff.args = -d $base $local $output $other +close +close
</code>
as per http://mercurial.selenic.com/wiki/MergingWithVim. The page http://mercurial.selenic.com/wiki/MergeProgram contains instructions for other tools, including Emacs, and graphical tools. If you edit your <code>~/.hgrc</code> file in this way then you now can type <code>hg resolve --all</code> to run your chosen tool on all unresolved files. Note that by default if your chosen tool exits without an error then <code>hg resolve</code> will regard that file as resolved. To cause <code>vim</code> to exit with an error use <code>:cq</code>. See <code>hg help resolve</code> for more details.

In the example my resolution is to have the relevant lines in <code>a1tdep.F</code> look like this:
<code>
c
c Add super new feature
c Some new feature
c
</code>

Once you have resolved all the files that mercurial thinks need fixing you should check that the final merged result makes sense. Checking that it compiles, or runs tests appropriate to both of the original revisions, for example. When you are satisfied, you should check in the merged revision:

<code>
testmerge$ hg ci -m 'Merged super and duper features'

testmerge$ hg tip

changeset: 26:857f81e59d66
tag: tip
parent: 25:13ecff0136c2
parent: 24:457db23c0b1a
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Thu Jan 26 13:10:52 2012 +0000
summary: Merged super and new features

testmerge$

</code>

You can see that the new revision has ''two'' parent revisions; a merge always consumes a head. This is clear from the output of:

<code>
testmerge$ hg glog -l 4
o changeset: 26:857f81e59d66
|\ tag: tip
| | parent: 25:13ecff0136c2
| | parent: 24:457db23c0b1a
| | user: Simon Clifford <simon.j.clifford@gmail.com>
| | date: Thu Jan 26 13:10:52 2012 +0000
| | summary: Merged super and new features
| |
| o changeset: 25:13ecff0136c2
| | parent: 23:1de40efd1c4b
| | user: Simon Clifford <simon.j.clifford@gmail.com>
| | date: Wed Jan 25 17:39:50 2012 +0000
| | summary: A test message
| |
o | changeset: 24:457db23c0b1a
|/ user: Simon Clifford <simon.j.clifford@gmail.com>
| date: Thu Jan 26 12:15:54 2012 +0000
| summary: Added second new feature
|
o changeset: 23:1de40efd1c4b
| user: Simon Clifford <simon.j.clifford@gmail.com>
| date: Wed Jan 18 14:43:13 2012 +0000
| summary: Added tag h13 for changeset 6c81eb8dcbab
|
testmerge$
</code>

Another example of merging here? Perhaps Lee's code merging in changes on official version.

== Tags and named branches (hg tag, hg tags, hg branch, hg branches) ==

It is often useful to give a name to a particular revision. This might be a changeset that corresponds to a given version of the code, or it might simply be a revision that marks a particular milestone. In the <code>example-gaussian-repo</code> you have probably noticed there are revisions in the history with descriptions like "Added tag h13 for changeset 6c81eb8dcbab". These are revisions corresponding to particular versions of the Gaussian development version, such as H01, H10, H13, etc. You can see a list of the tags in a repository by doing:
<code>
testmerge$ hg tags
tip 26:857f81e59d66
h13 22:6c81eb8dcbab
h12p 20:0ef14d7dff56
h11 18:7daba638a830
h10 16:e1d0af4e84d9
h08 14:656073c38db9
h01 12:737d1720e79a
h13-raw 10:b039f5c274e2
h12p-raw 8:522a8fe79d22
h11-raw 6:a28b789a9555
h10-raw 4:356081dab79d
h08-raw 2:9f69ed4616ad
h01-raw 0:073d6aa63ea7
testmerge$
</code>
(the <code>tip</code> tag is automatically assigned to the most recently checked in revision). You may use a tag name anywhere where you might use a revision number, so for example, to check out the changeset corresponding to the official Gaussian development release H10, you would type:
<code>
testrepo$ hg up h10
13887 files updated, 0 files merged, 0 files removed, 0 files unresolved
testrepo$
</code>

(this will look different if you have unchecked-in modifications in your working directory). You could also pass h10 to the <code>hg clone -u</code> flag, and so on.
You tag a revision with the <code>hg tag</code> command. Issue it in a working directory and it will tag the revision that is the parent of the working directory. Tags become part of the repository, that is they are shared during <code>hg pull</code> and <code>hg push</code>, so choose wisely. If you'd like to have tags that are only part of the local repository, use the <code>-l</code> flag to <code>hg tag</code>.

Tags are stored in an <code>.hgtags</code> file in the root directory of the repository. Running <code>hg tag</code> alters this file and then commits the change to the repository. Sometimes you can see conflicts in this file during merges. I normally just resolve them by keeping as much information in the merged file as possible.

The <code>hg branches</code> command can be used to show the named branches in the repository:
<code>
testrepo$ hg branches
default 24:13ecff0136c2
raw 11:a0c273aeeb2b (inactive)
testrepo$
</code>
Here I have two named branches, raw and default (the default branch is, er, the default). If you examine the full output of <code>hg glog</code>, which is quite long and I won't reproduce it here, you'll see, starting at the bottom, revisions 0 to 11 are all in the raw branch. I have created and named this branch to contain the official Gaussian development code as processed by the <code>old-to-new.sh</code> script, without any of our makefiles or other alterations. The default branch branches off from revision 1 and starts at revision 12. Its descendants, revisions 12 to 23, are also in the default branch. These were created by starting at revision 1, adding our makefiles and modifications to the build system, renaming this branch to the default branch, and then committing revision 12. I tagged revision 12 as "H01", which created revision 13. I then merged revision 3, fixed the conflicts, and committed revision 14, and so on. A more detailed guide on how to import a new official version of gdv exists in LINKY.

You can use branch names just like tags wherever mercurial expects a revision identifier. If you use a branch name mercurial will attempt to give you the revision that is most appropriate, this will usually be the newest head on that branch.

To create a new branch you type <code>hg branch <branchname></code>. This will take effect when you commit the working directory. As I've previously mentioned a simpler alternative to having branches, named or otherwise, in your repository is to have multiple repositories.

== Ignoring files ==

You may be wondering how the system will work when you start compiling files. Won't all the <code>.o</code>, <code>.a</code> and <code>.exe</code> files start showing up in the output to <code>hg status</code>? The answer is that they will not, because I've told mercurial to ignore such files. This information is stored in the <code>.hgignore</code> file in the root of the repository. It's quite long but it starts like this:
<code>
syntax: glob
gdv.make
*.o
*.lo
*.a
*.exe
*.exel
.*.swp
*~
*.log
*.log.gz
*.chk
gdv/arc
...
</code>
Any file that matches either the shell patterns or the file names in <code>.hgignore</code> is ignored by mercurial. This means it does not show up to <code>hg status</code> or <code>hg commit</code>. <code>hg add</code>, <code>hg rm</code>, and so on will ignore it unless it is explicitly mentioned on the command line. So <code>hg add gdv</code> will add all non-ignored files in the gdv directory, while <code>hg add gdv/*</code> will add all the files in the gdv directory.

The <code>.hgignore</code> file is tracked, so please only commit changes to it if you think they will be useful to everyone.
== Other commands ==

There are quite a few other commands available to mercurial, I will cover just a few here, if you're interested in becoming a power user refer to the documentation I have mentioned.

=== hg addremove ===
You can use this command when you have large numbers of files to add and remove. It also includes the <code>-s</code> flag to attempt to detect when files have been moved or copied. Using this flag does require some subtlety, as it's not helpful to mark files as moved or copied unless they have been. Also note that it is possible to inadvertently include ignored files in a mass <code>hg add</code> or <code>hg addremove</code>; check what's being added before you commit.

=== hg grep and hg locate ===
Similar to the standard UNIX grep and locate commands, but cognisant of the mercurial repository.

=== hg init ===
Creates a new mercurial repository in the current directory

=== hg serve ===
This starts a mini web server serving information about the repository. I would strongly recommend you do not use this while working on the Gaussian development version as it would be all too easy to allow any user on the system to view all of the files in your repository. I only mention it here because some of the other documentation may refer to it.

Resgrp:comp-photo-new gdv layout

2012-03-20T11:50:47Z

Scliffor: /* Developing */

== Setting up ==

Gaussian insists on having its executables not readable by world. The best way to achieve this is to have a properly set umask; this is a good idea anyway, as having things readable (or writable!) by others is rarely a good idea. If you need to allow others to view things you can set the permissions explicitly. One problem here is that the Gaussian makefiles use the csh, which no sane person uses as a shell. It's therefore possible that your umask is not being set for the c-shell, which results in world readable files by default. Fix this permanently by inserting this line into ''both'' <code>~/.bashrc</code> and <code>~/.cshrc</code> (you may have to create the latter):
<code>
umask 077
</code>

== The new Gaussian development version layout ==

When you first check out a version of gdv from the [[Resgrp:comp-photo-version_control|mercurial repository]] you will notice that the code is laid out in a different format. Most of the large monolithic files have been broken down into subdirectories with their constituent subroutines split out into individual files. For example, <code>l510.F</code> has been replaced by a <code>l510</code> directory containing each <code>l510</code> subroutine in its own file. Most of the utility files have been processed similarly. This layout is primarily to help the functioning of the version control system. With the old system a modification to a subroutine within <code>l510.F</code> would show up only as a change to the file <code>l510.F</code>, with only line numbers to help you guess what subroutine had actually been altered. Moving a subroutine from a link file to a utility file would only show up as a deletion of a block of lines from one and an addition of a block of lines to the other. The new system allows us to immediately see which subroutines have been altered and to track them if they are moved or copied.
When you check out the repository there are currently three scripts next to the <code>gdv</code> subdirectory. The <code>new-to-old.sh</code> and <code>old-to-new.sh</code> scripts convert between the two formats. See [[#Importing your own code|Importing your own code]] for details on <code>old-to-new.sh</code>. These will be useful when receiving new tarballs from Gaussian, or converting already written code, or when the time comes to send code back to Gaussian. The <code>new-to-old.sh</code> script attempts to get a distribution very close to an old form Gaussian layout, but there will be differences. Some old-style files are in an arbitrary order which the script does not attempt to recreate. Fortunately these are not files that we are likely to change.
The third script, <code>pillage.sh</code>, takes executables, libraries, and object files from an already compiled version of Gaussian. It shouldn't matter whether the previously compiled version is new style or old-style.

== Building from scratch ==

If you choose not to use <code>pillage.sh</code> then you must build from scratch. This is actually quite quick. Assuming you have checked out the version you wish to compile, and have loaded the appropriate compiler and Linda modules, you should <code>cd</code> to the <code>gdv</code> directory. Do '''not''' run <code>bldgdv</code>, as this will start the normal Gaussian build system, which will fail.

You will need to set the symbolic link for <code>bsd/gdv.make</code> to point to the appropriate makefile yourself. This needs to be one of the modififed makefiles, which are prefixed with 'uber'. The easiest option is just to use <code>uber-i386.make</code> which uses the same options that <code>i386.make</code> does. The other 'uber-*.make' files are tailored to our particular systems (and therefore we may house them elsewhere in the future), such as the Sandybridge nodes or the Altix. If you want to modify an existing makefile see the comments [[#The makefile|here]].
Choose your 'uber' makefile (I'll use the <code>uber-i386.make</code> file here) and type:
<code>
gdv$ ln -s uber-i386.make bsd/gdv.make
gdv$
</code>

Once there is an uber makefile now set you are ready to use the new build system. To build the entirety of Gaussian type:
<code>
gdv$ make build-tools && make -j 6 util.a && make util.a && make -j 6 exe && make exe 2>&1 | tee make.log
[... lots of output ]
gdv$
</code>
The number 6 in this example instructs make to run six parallel threads while building. You can use different numbers depending on how many processors the system you're building on has, how loaded it is, and so on. I find that building in the <code>/tmp</code> directory with a relatively unloaded cx1 login node, I can build Gaussian in about 15 minutes. You'll notice that after each parallel invocation of make I have a serial invocation at the same time for the same target. This is because sometimes the parallel make loses track of where it is and doesn't fully make a target. The serial invocation should always correct this.
Once Gaussian is built (and you can always check by seeing that <code>make exe</code> returns <code>Nothing to be done for: `exe'</code>) you can also build a Linda version by typing <code>make linda</code>. You will need the 8.2 version of the Linda compiler in your path; I suggest this is made into a module.

== Using <code>pillage.sh</code> ==
<code>cd</code> into the <code>gdv</code> subdirectory of your freshly checked out repository, type:
<code>
gdv$ ../pillage.sh /home/gaussian-devel/gaussiandvh13_pgi_118/gdv
</code>
(for example) wait a while and when the prompt returns you should have a fully populated and up-to-date set of binaries. <code>pillage.sh</code> touches all the imported libraries and executables so they are newer than the source code. This means that if you type:
<code>
gdv$ make
ln -sf browse arc
gdv$
</code>
only some trivial things get made.

It's up to you to make sure that any development you do uses a consistent compiler, Linda compiler, etc, to the pillaged binaries.

== Doing it yourself ==

I did try to make a version of <code>pillage.sh</code> that linked to executables instead of copying them, and you may feel tempted to try the same. The main problem that you need to overcome is that if the libraries you are linking to are older than their corresponding <code>.F</code> files make will try to update them. If the libraries are not yours this will simply fail and cause make to exit. However, if the symbolically linked libraries are yours they will be updated, which is probably not what you want!
If you really need to save space there are some easier things you can do. Starting from an empty repository, run <code>bldgdv</code> and make the build tools as above. Then copy an appropriate util.a from somewhere to your <code>gdv</code> directory (do not use <code>cp -p</code>). Finally, use make to build only the links you are planning to alter, e.g. <code>make l510.exe</code>. You should also alter the Makefile so that the mystuff target lists only these links. This saves space by not having any of the other links' libraries or executables. You will also not have any <code>.o</code> files for the util subroutines; as long as the util.a file is newer than all of the util <code>.F</code> files make will not try to make them. If, however, even one util <code>.F</code> file becomes newer than util.a make will make '''all''' of the util <code>.o</code> files!
Now you can use the <code>%subst</code> keyword to Gaussian to tell it to pick up the appropriate links from this directory.

== Developing ==

However you have built the binaries developing in the new system is very simple. Edit the files you need to change, type <code>make</code>, or <code>make Linda</code> from the <code>gdv</code> directory. You can also type make inside a link subdirectory, which will build just that link's executable. The makefile knows about Gaussian's dependencies, so for example if you change a subroutine in <code>l510</code> then both <code>l510.exe</code> and <code>l1003.exe</code> will be re-linked (because <code>l1003</code> depends on <code>l510.a</code>)

By default the makefile checks everything to see if it's up-to-date. As there are nearly 12,000 files in this layout this can take a few seconds, longer if the filesystem is being stressed. I find that with a hot cache on <code>/tmp</code> make takes less than a second to check everything. With a cold cache on <code>/home</code> it can take some tens of seconds. If you want to get make to only check specific executables edit the Makefile in <code>gdv</code>, uncomment the "mystuff" line, and change it appropriately. Note, however, that all the util files are always checked. The makefile is a tracked file in the repository, so please make sure you don't check in these trivial changes.

You do not need to edit the makefile if you add, move, or delete subroutines.

== Importing your own code ==

If your code is in the old layout, i.e. it's all monolithic links and utilities just as in the official Gaussian distribution tarball, then use the <code>old-to-new.sh</code> script. First, check out the version of the development code that was the basis of your code. Let's assume you took the H10 code and have been working on that, but have not merged in changes for H11, H12P or H13.
<code>
tmp$ hg clone -u h10 /home/gaussiandevel/example-gaussian-repo /tmp/testrepo
</code>
'''or''', if you already have a repository checked out, <code>cd</code> into it and do:
<code>
tmp$ hg update h10
</code>

The <code>old-to-new.sh</code> script assumes that you've got a <code>gau-fsplit</code> command in your <code>$PATH</code>. If you haven't, you can create one by building the build-tools, as explained above, and making sure this directory appears, however temporarily, in your <code>$PATH</code>. Or, you can point your path to the pre-built binaries in <code>/home/gaussian-devel</code>.

Now change directory to be above the <code>gdv</code> directory of your own code. Run the <code>old-to-new.sh</code> script, with an argument of the directory you want the new style to appear in (it should not exist):
<code>
tmp$ cd /path/to/my-code
my-code$ ls
gdv
my-code$ PATH=$PATH:/home/gaussian-devel/gaussiandvh10_pgi_105/gdv /tmp/testrepo/old-to-new.sh new-style
[...some output...]
my-code$ ls
gdv new-style
</code>
Now remove the gdv directory in the newly checked out working directory and replace it with the one from new-style.
<code>
my-code$ cd /tmp/testrepo
testrepo$ rm -fr gdv
testrepo$ cp -pr /path/to/mycode/new-style/gdv .
</code>
Now you're ready to use commands like <code>hg status</code> to see what you've changed compared to H10, <code>hg ci</code> to check in your changes, <code>hg merge h13</code> to merge in changes up to version H13 (for example).

If you don't have the entire Gaussian source code in your directory you can get <code>old-to-new.sh</code> to work by making sure there's a (possibly empty) file in <code>gdv/bsd/mdl1.F</code>. There will be a few warnings but it will split out any files it can find in the <code>gdv</code> directory.

If you just have a few subroutines already split out, then you should just copy them to the appropriate place in the working directory.

This might seem like a pain but you'll only have to do it once for each version of the code you have!

== The makefile ==

In modifying the Gassian build system I have attempted to balance two competing needs. The first is a strong desire to modify as little as possible of Gaussian's original makefiles. This is so that when we receive newer versions of Gaussian, with their inevitable changes to the makefiles, our modifications can be trivially merged in. The second is to have a build system that works nicely with the new file layout, which is necessary for the version control system. The compromise that I have chosen is to have a new Makefile in the <code>gdv</code> directory that understand such things as how to build libraries from the split util and link files, and which executables depend on which libraries, while relying on the existing Gaussian makefiles for details such as compiler invocation and flags.

Sadly it was not possible to use the existing Gaussian makefile without modifying it slightly. This modified version of <code>bsd/i386.make</code> is called <code>bsd/uber-i386.make</code>. In it the rules to make most of the libraries and executables are commented out, and a few fixes for other problems are added. If you wish to compile on another platform you will need to make an uber makefile for that platform. Base this off the differences between <code>bsd/i386.make</code> and <code>bsd/uber-i386.make</code>.

If you want to add a new link you will need to add it to the Makefile, and possibly to the <code>uber-*</code> makefile.

== Tools and tricks ==

If you aren't already using these tools, you should be.

=== Tags ===

One complaint that might arise in going from monolithic link files to split files is that finding the definitions of subroutines and functions becomes much more difficult. So, if you are editing <code>l510.F</code> and are in the MCSCF deck you might come to the line that calls the Davids subroutine. Finding where this subroutine is defined used to be as simple as searching for 'e Davids' (assuming it's a subroutine, and that there is only one space between subroutine and Davids, and that it's in <code>l510.F</code>, etc). How are you to find it now?

The answer is to use '''tags'''. These have been around for a few decades and are extremely useful. Firstly you must create a tags file, or files. For vi/vim you do:
<code>
testrepo/gdv$ ctags *.F */*.F */*.c
</code>
(for Emacs substitute etags for ctags). If you want to create a tags file in one of the link subdirectories just use:
<code>
testrepo/gdv/l510$ ctags *.F ../*/*.F ../*/*.c
</code>
This will create a ctags (or etags) file in the current directory. Now, start vi (or Emacs) and edit <code>l510/mcscf.F</code>. Find the line that calls the Davids subroutine and position the cursor over the word Davids. Press <code>Ctrl-]</code> (or <code>M-.</code>) and you will be immediately taken to the definition of that subroutine, even if it’s in another file. Press <code>Ctrl-T</code> (or <code>M-*</code>) and you'll be taken back to where you were. These tags stack, that is you can go into a subroutine, and from there go into another one, and so on, and still be able to back out to the beginning.

You can also start vi (vim really) with the <code>-t <tag></code> option to jump immediately to a particular tag. I don't know if emacs can do this, as I lost the will to live while browsing the 'info' documentation.

Editors other than vi and Emacs also support tags; check the documentation. If your editor doesn't support tags it's just not good enough, sorry.

=== Screen ===

Screen is a terminal multiplexer. If you wish to have multiple terminals open on a remote system you're probably accustomed to opening multiple SSH windows from your local system. An alternative is to use <code>screen</code>. It allows you to have multiple windows, each with its own history and possibly executing commands, and switch between them with some simple keypresses. It also preserves the state of these windows when your connection to screen drops, allowing you to reconnect later.

It's a bit beyond the scope of this page, if you want to learn more check out the various web resources or it, such as http://www.rackaid.com/resources/linux-screen-tutorial-and-how-to/.

Resgrp:comp-photo-new gdv layout

2012-03-20T11:49:30Z

Scliffor: /* Building from scratch */

== Setting up ==

Gaussian insists on having its executables not readable by world. The best way to achieve this is to have a properly set umask; this is a good idea anyway, as having things readable (or writable!) by others is rarely a good idea. If you need to allow others to view things you can set the permissions explicitly. One problem here is that the Gaussian makefiles use the csh, which no sane person uses as a shell. It's therefore possible that your umask is not being set for the c-shell, which results in world readable files by default. Fix this permanently by inserting this line into ''both'' <code>~/.bashrc</code> and <code>~/.cshrc</code> (you may have to create the latter):
<code>
umask 077
</code>

== The new Gaussian development version layout ==

When you first check out a version of gdv from the [[Resgrp:comp-photo-version_control|mercurial repository]] you will notice that the code is laid out in a different format. Most of the large monolithic files have been broken down into subdirectories with their constituent subroutines split out into individual files. For example, <code>l510.F</code> has been replaced by a <code>l510</code> directory containing each <code>l510</code> subroutine in its own file. Most of the utility files have been processed similarly. This layout is primarily to help the functioning of the version control system. With the old system a modification to a subroutine within <code>l510.F</code> would show up only as a change to the file <code>l510.F</code>, with only line numbers to help you guess what subroutine had actually been altered. Moving a subroutine from a link file to a utility file would only show up as a deletion of a block of lines from one and an addition of a block of lines to the other. The new system allows us to immediately see which subroutines have been altered and to track them if they are moved or copied.
When you check out the repository there are currently three scripts next to the <code>gdv</code> subdirectory. The <code>new-to-old.sh</code> and <code>old-to-new.sh</code> scripts convert between the two formats. See [[#Importing your own code|Importing your own code]] for details on <code>old-to-new.sh</code>. These will be useful when receiving new tarballs from Gaussian, or converting already written code, or when the time comes to send code back to Gaussian. The <code>new-to-old.sh</code> script attempts to get a distribution very close to an old form Gaussian layout, but there will be differences. Some old-style files are in an arbitrary order which the script does not attempt to recreate. Fortunately these are not files that we are likely to change.
The third script, <code>pillage.sh</code>, takes executables, libraries, and object files from an already compiled version of Gaussian. It shouldn't matter whether the previously compiled version is new style or old-style.

== Building from scratch ==

If you choose not to use <code>pillage.sh</code> then you must build from scratch. This is actually quite quick. Assuming you have checked out the version you wish to compile, and have loaded the appropriate compiler and Linda modules, you should <code>cd</code> to the <code>gdv</code> directory. Do '''not''' run <code>bldgdv</code>, as this will start the normal Gaussian build system, which will fail.

You will need to set the symbolic link for <code>bsd/gdv.make</code> to point to the appropriate makefile yourself. This needs to be one of the modififed makefiles, which are prefixed with 'uber'. The easiest option is just to use <code>uber-i386.make</code> which uses the same options that <code>i386.make</code> does. The other 'uber-*.make' files are tailored to our particular systems (and therefore we may house them elsewhere in the future), such as the Sandybridge nodes or the Altix. If you want to modify an existing makefile see the comments [[#The makefile|here]].
Choose your 'uber' makefile (I'll use the <code>uber-i386.make</code> file here) and type:
<code>
gdv$ ln -s uber-i386.make bsd/gdv.make
gdv$
</code>

Once there is an uber makefile now set you are ready to use the new build system. To build the entirety of Gaussian type:
<code>
gdv$ make build-tools && make -j 6 util.a && make util.a && make -j 6 exe && make exe 2>&1 | tee make.log
[... lots of output ]
gdv$
</code>
The number 6 in this example instructs make to run six parallel threads while building. You can use different numbers depending on how many processors the system you're building on has, how loaded it is, and so on. I find that building in the <code>/tmp</code> directory with a relatively unloaded cx1 login node, I can build Gaussian in about 15 minutes. You'll notice that after each parallel invocation of make I have a serial invocation at the same time for the same target. This is because sometimes the parallel make loses track of where it is and doesn't fully make a target. The serial invocation should always correct this.
Once Gaussian is built (and you can always check by seeing that <code>make exe</code> returns <code>Nothing to be done for: `exe'</code>) you can also build a Linda version by typing <code>make linda</code>. You will need the 8.2 version of the Linda compiler in your path; I suggest this is made into a module.

== Using <code>pillage.sh</code> ==
<code>cd</code> into the <code>gdv</code> subdirectory of your freshly checked out repository, type:
<code>
gdv$ ../pillage.sh /home/gaussian-devel/gaussiandvh13_pgi_118/gdv
</code>
(for example) wait a while and when the prompt returns you should have a fully populated and up-to-date set of binaries. <code>pillage.sh</code> touches all the imported libraries and executables so they are newer than the source code. This means that if you type:
<code>
gdv$ make
ln -sf browse arc
gdv$
</code>
only some trivial things get made.

It's up to you to make sure that any development you do uses a consistent compiler, Linda compiler, etc, to the pillaged binaries.

== Doing it yourself ==

I did try to make a version of <code>pillage.sh</code> that linked to executables instead of copying them, and you may feel tempted to try the same. The main problem that you need to overcome is that if the libraries you are linking to are older than their corresponding <code>.F</code> files make will try to update them. If the libraries are not yours this will simply fail and cause make to exit. However, if the symbolically linked libraries are yours they will be updated, which is probably not what you want!
If you really need to save space there are some easier things you can do. Starting from an empty repository, run <code>bldgdv</code> and make the build tools as above. Then copy an appropriate util.a from somewhere to your <code>gdv</code> directory (do not use <code>cp -p</code>). Finally, use make to build only the links you are planning to alter, e.g. <code>make l510.exe</code>. You should also alter the Makefile so that the mystuff target lists only these links. This saves space by not having any of the other links' libraries or executables. You will also not have any <code>.o</code> files for the util subroutines; as long as the util.a file is newer than all of the util <code>.F</code> files make will not try to make them. If, however, even one util <code>.F</code> file becomes newer than util.a make will make '''all''' of the util <code>.o</code> files!
Now you can use the <code>%subst</code> keyword to Gaussian to tell it to pick up the appropriate links from this directory.

== Developing ==

However you have built the binaries developing in the new system is very simple. Edit the files you need to change, type <code>make</code>, or <code>make Linda</code> from the <code>gdv</code> directory. You can also type make inside a link subdirectory, which will build just that link's executable. The makefile knows about Gaussian's dependencies, so for example if you change a subroutine in <code>l510</code> then both <code>l510.exe</code> and <code>l1003.exe</code> will be re-linked.

By default the makefile checks everything to see if it's up-to-date. As there are nearly 12,000 files in this layout this can take a few seconds, longer if the filesystem is being stressed. I find that with a hot cache on <code>/tmp</code> make takes less than a second to check everything. With a cold cache on <code>/home</code> it can take some tens of seconds. If you want to get make to only check specific executables edit the Makefile in <code>gdv</code>, uncomment the "mystuff" line, and change it appropriately. Note, however, that all the util files are always checked. The makefile is a tracked file in the repository, so please make sure you don't check in these trivial changes.

You do not need to edit the makefile if you add, move, or delete subroutines.

== Importing your own code ==

If your code is in the old layout, i.e. it's all monolithic links and utilities just as in the official Gaussian distribution tarball, then use the <code>old-to-new.sh</code> script. First, check out the version of the development code that was the basis of your code. Let's assume you took the H10 code and have been working on that, but have not merged in changes for H11, H12P or H13.
<code>
tmp$ hg clone -u h10 /home/gaussiandevel/example-gaussian-repo /tmp/testrepo
</code>
'''or''', if you already have a repository checked out, <code>cd</code> into it and do:
<code>
tmp$ hg update h10
</code>

The <code>old-to-new.sh</code> script assumes that you've got a <code>gau-fsplit</code> command in your <code>$PATH</code>. If you haven't, you can create one by building the build-tools, as explained above, and making sure this directory appears, however temporarily, in your <code>$PATH</code>. Or, you can point your path to the pre-built binaries in <code>/home/gaussian-devel</code>.

Now change directory to be above the <code>gdv</code> directory of your own code. Run the <code>old-to-new.sh</code> script, with an argument of the directory you want the new style to appear in (it should not exist):
<code>
tmp$ cd /path/to/my-code
my-code$ ls
gdv
my-code$ PATH=$PATH:/home/gaussian-devel/gaussiandvh10_pgi_105/gdv /tmp/testrepo/old-to-new.sh new-style
[...some output...]
my-code$ ls
gdv new-style
</code>
Now remove the gdv directory in the newly checked out working directory and replace it with the one from new-style.
<code>
my-code$ cd /tmp/testrepo
testrepo$ rm -fr gdv
testrepo$ cp -pr /path/to/mycode/new-style/gdv .
</code>
Now you're ready to use commands like <code>hg status</code> to see what you've changed compared to H10, <code>hg ci</code> to check in your changes, <code>hg merge h13</code> to merge in changes up to version H13 (for example).

If you don't have the entire Gaussian source code in your directory you can get <code>old-to-new.sh</code> to work by making sure there's a (possibly empty) file in <code>gdv/bsd/mdl1.F</code>. There will be a few warnings but it will split out any files it can find in the <code>gdv</code> directory.

If you just have a few subroutines already split out, then you should just copy them to the appropriate place in the working directory.

This might seem like a pain but you'll only have to do it once for each version of the code you have!

== The makefile ==

In modifying the Gassian build system I have attempted to balance two competing needs. The first is a strong desire to modify as little as possible of Gaussian's original makefiles. This is so that when we receive newer versions of Gaussian, with their inevitable changes to the makefiles, our modifications can be trivially merged in. The second is to have a build system that works nicely with the new file layout, which is necessary for the version control system. The compromise that I have chosen is to have a new Makefile in the <code>gdv</code> directory that understand such things as how to build libraries from the split util and link files, and which executables depend on which libraries, while relying on the existing Gaussian makefiles for details such as compiler invocation and flags.

Sadly it was not possible to use the existing Gaussian makefile without modifying it slightly. This modified version of <code>bsd/i386.make</code> is called <code>bsd/uber-i386.make</code>. In it the rules to make most of the libraries and executables are commented out, and a few fixes for other problems are added. If you wish to compile on another platform you will need to make an uber makefile for that platform. Base this off the differences between <code>bsd/i386.make</code> and <code>bsd/uber-i386.make</code>.

If you want to add a new link you will need to add it to the Makefile, and possibly to the <code>uber-*</code> makefile.

== Tools and tricks ==

If you aren't already using these tools, you should be.

=== Tags ===

One complaint that might arise in going from monolithic link files to split files is that finding the definitions of subroutines and functions becomes much more difficult. So, if you are editing <code>l510.F</code> and are in the MCSCF deck you might come to the line that calls the Davids subroutine. Finding where this subroutine is defined used to be as simple as searching for 'e Davids' (assuming it's a subroutine, and that there is only one space between subroutine and Davids, and that it's in <code>l510.F</code>, etc). How are you to find it now?

The answer is to use '''tags'''. These have been around for a few decades and are extremely useful. Firstly you must create a tags file, or files. For vi/vim you do:
<code>
testrepo/gdv$ ctags *.F */*.F */*.c
</code>
(for Emacs substitute etags for ctags). If you want to create a tags file in one of the link subdirectories just use:
<code>
testrepo/gdv/l510$ ctags *.F ../*/*.F ../*/*.c
</code>
This will create a ctags (or etags) file in the current directory. Now, start vi (or Emacs) and edit <code>l510/mcscf.F</code>. Find the line that calls the Davids subroutine and position the cursor over the word Davids. Press <code>Ctrl-]</code> (or <code>M-.</code>) and you will be immediately taken to the definition of that subroutine, even if it’s in another file. Press <code>Ctrl-T</code> (or <code>M-*</code>) and you'll be taken back to where you were. These tags stack, that is you can go into a subroutine, and from there go into another one, and so on, and still be able to back out to the beginning.

You can also start vi (vim really) with the <code>-t <tag></code> option to jump immediately to a particular tag. I don't know if emacs can do this, as I lost the will to live while browsing the 'info' documentation.

Editors other than vi and Emacs also support tags; check the documentation. If your editor doesn't support tags it's just not good enough, sorry.

=== Screen ===

Screen is a terminal multiplexer. If you wish to have multiple terminals open on a remote system you're probably accustomed to opening multiple SSH windows from your local system. An alternative is to use <code>screen</code>. It allows you to have multiple windows, each with its own history and possibly executing commands, and switch between them with some simple keypresses. It also preserves the state of these windows when your connection to screen drops, allowing you to reconnect later.

It's a bit beyond the scope of this page, if you want to learn more check out the various web resources or it, such as http://www.rackaid.com/resources/linux-screen-tutorial-and-how-to/.

Resgrp:comp-photo-new gdv layout

2012-03-20T11:47:49Z

Scliffor:

== Setting up ==

Gaussian insists on having its executables not readable by world. The best way to achieve this is to have a properly set umask; this is a good idea anyway, as having things readable (or writable!) by others is rarely a good idea. If you need to allow others to view things you can set the permissions explicitly. One problem here is that the Gaussian makefiles use the csh, which no sane person uses as a shell. It's therefore possible that your umask is not being set for the c-shell, which results in world readable files by default. Fix this permanently by inserting this line into ''both'' <code>~/.bashrc</code> and <code>~/.cshrc</code> (you may have to create the latter):
<code>
umask 077
</code>

== The new Gaussian development version layout ==

When you first check out a version of gdv from the [[Resgrp:comp-photo-version_control|mercurial repository]] you will notice that the code is laid out in a different format. Most of the large monolithic files have been broken down into subdirectories with their constituent subroutines split out into individual files. For example, <code>l510.F</code> has been replaced by a <code>l510</code> directory containing each <code>l510</code> subroutine in its own file. Most of the utility files have been processed similarly. This layout is primarily to help the functioning of the version control system. With the old system a modification to a subroutine within <code>l510.F</code> would show up only as a change to the file <code>l510.F</code>, with only line numbers to help you guess what subroutine had actually been altered. Moving a subroutine from a link file to a utility file would only show up as a deletion of a block of lines from one and an addition of a block of lines to the other. The new system allows us to immediately see which subroutines have been altered and to track them if they are moved or copied.
When you check out the repository there are currently three scripts next to the <code>gdv</code> subdirectory. The <code>new-to-old.sh</code> and <code>old-to-new.sh</code> scripts convert between the two formats. See [[#Importing your own code|Importing your own code]] for details on <code>old-to-new.sh</code>. These will be useful when receiving new tarballs from Gaussian, or converting already written code, or when the time comes to send code back to Gaussian. The <code>new-to-old.sh</code> script attempts to get a distribution very close to an old form Gaussian layout, but there will be differences. Some old-style files are in an arbitrary order which the script does not attempt to recreate. Fortunately these are not files that we are likely to change.
The third script, <code>pillage.sh</code>, takes executables, libraries, and object files from an already compiled version of Gaussian. It shouldn't matter whether the previously compiled version is new style or old-style.

== Building from scratch ==

If you choose not to use <code>pillage.sh</code> then you must build from scratch. This is actually quite quick. Assuming you have checked out the version you wish to compile, and have loaded the appropriate compiler and Linda modules, you should <code>cd</code> to the <code>gdv</code> directory. Do '''not''' run <code>bldgdv</code>, as this will start the normal Gaussian build system, which will fail.

You will need to set the symbolic link for <code>bsd/gdv.make</code> to point to the appropriate makefile yourself. This needs to be one of the modififed makefiles, which are prefixed with 'uber'. The easiest option is just to use <code>uber-i386.make</code> which uses the same options that <code>i386.make</code> does. The other 'uber-*.make' files are tailored to our particular systems (and therefore we may house them elsewhere in the future), such as the Sandybridge nodes or the Altix. If you want to modify an existing makefile see the comments [[#The makefile|here]].
Choose your 'uber' makefile (I'll use the <code>uber-i386.make</code> file here) and type:
<code>
gdv$ ln -s uber-i386.make bsd/gdv.make
gdv$
</code>

Once there is an uber makefile now set you are now ready to use the new build system. To build the entirety of Gaussian type:
<code>
gdv$ make build-tools && make -j 6 util.a && make util.a && make -j 6 exe && make exe 2>&1 | tee make.log
[... lots of output ]
gdv$
</code>
The number 6 in this example instructs make to run six parallel threads while building. You can use different numbers depending on how many processors the system you're building on has, how loaded it is, and so on. I find that building in the <code>/tmp</code> directory with a relatively unloaded cx1 login node, I can build Gaussian in about 15 minutes. You'll notice that after each parallel invocation of make I have a serial invocation at the same time for the same target. This is because sometimes the parallel make loses track of where it is and doesn't fully make a target. The serial invocation should always correct this.
Once Gaussian is built (and you can always check by seeing that <code>make exe</code> returns <code>Nothing to be done for: `exe'</code>) you can also build a Linda version by typing <code>make linda</code>. You will need the 8.2 version of the Linda compiler in your path; I suggest this is made into a module.

== Using <code>pillage.sh</code> ==
<code>cd</code> into the <code>gdv</code> subdirectory of your freshly checked out repository, type:
<code>
gdv$ ../pillage.sh /home/gaussian-devel/gaussiandvh13_pgi_118/gdv
</code>
(for example) wait a while and when the prompt returns you should have a fully populated and up-to-date set of binaries. <code>pillage.sh</code> touches all the imported libraries and executables so they are newer than the source code. This means that if you type:
<code>
gdv$ make
ln -sf browse arc
gdv$
</code>
only some trivial things get made.

It's up to you to make sure that any development you do uses a consistent compiler, Linda compiler, etc, to the pillaged binaries.

== Doing it yourself ==

I did try to make a version of <code>pillage.sh</code> that linked to executables instead of copying them, and you may feel tempted to try the same. The main problem that you need to overcome is that if the libraries you are linking to are older than their corresponding <code>.F</code> files make will try to update them. If the libraries are not yours this will simply fail and cause make to exit. However, if the symbolically linked libraries are yours they will be updated, which is probably not what you want!
If you really need to save space there are some easier things you can do. Starting from an empty repository, run <code>bldgdv</code> and make the build tools as above. Then copy an appropriate util.a from somewhere to your <code>gdv</code> directory (do not use <code>cp -p</code>). Finally, use make to build only the links you are planning to alter, e.g. <code>make l510.exe</code>. You should also alter the Makefile so that the mystuff target lists only these links. This saves space by not having any of the other links' libraries or executables. You will also not have any <code>.o</code> files for the util subroutines; as long as the util.a file is newer than all of the util <code>.F</code> files make will not try to make them. If, however, even one util <code>.F</code> file becomes newer than util.a make will make '''all''' of the util <code>.o</code> files!
Now you can use the <code>%subst</code> keyword to Gaussian to tell it to pick up the appropriate links from this directory.

== Developing ==

However you have built the binaries developing in the new system is very simple. Edit the files you need to change, type <code>make</code>, or <code>make Linda</code> from the <code>gdv</code> directory. You can also type make inside a link subdirectory, which will build just that link's executable. The makefile knows about Gaussian's dependencies, so for example if you change a subroutine in <code>l510</code> then both <code>l510.exe</code> and <code>l1003.exe</code> will be re-linked.

By default the makefile checks everything to see if it's up-to-date. As there are nearly 12,000 files in this layout this can take a few seconds, longer if the filesystem is being stressed. I find that with a hot cache on <code>/tmp</code> make takes less than a second to check everything. With a cold cache on <code>/home</code> it can take some tens of seconds. If you want to get make to only check specific executables edit the Makefile in <code>gdv</code>, uncomment the "mystuff" line, and change it appropriately. Note, however, that all the util files are always checked. The makefile is a tracked file in the repository, so please make sure you don't check in these trivial changes.

You do not need to edit the makefile if you add, move, or delete subroutines.

== Importing your own code ==

If your code is in the old layout, i.e. it's all monolithic links and utilities just as in the official Gaussian distribution tarball, then use the <code>old-to-new.sh</code> script. First, check out the version of the development code that was the basis of your code. Let's assume you took the H10 code and have been working on that, but have not merged in changes for H11, H12P or H13.
<code>
tmp$ hg clone -u h10 /home/gaussiandevel/example-gaussian-repo /tmp/testrepo
</code>
'''or''', if you already have a repository checked out, <code>cd</code> into it and do:
<code>
tmp$ hg update h10
</code>

The <code>old-to-new.sh</code> script assumes that you've got a <code>gau-fsplit</code> command in your <code>$PATH</code>. If you haven't, you can create one by building the build-tools, as explained above, and making sure this directory appears, however temporarily, in your <code>$PATH</code>. Or, you can point your path to the pre-built binaries in <code>/home/gaussian-devel</code>.

Now change directory to be above the <code>gdv</code> directory of your own code. Run the <code>old-to-new.sh</code> script, with an argument of the directory you want the new style to appear in (it should not exist):
<code>
tmp$ cd /path/to/my-code
my-code$ ls
gdv
my-code$ PATH=$PATH:/home/gaussian-devel/gaussiandvh10_pgi_105/gdv /tmp/testrepo/old-to-new.sh new-style
[...some output...]
my-code$ ls
gdv new-style
</code>
Now remove the gdv directory in the newly checked out working directory and replace it with the one from new-style.
<code>
my-code$ cd /tmp/testrepo
testrepo$ rm -fr gdv
testrepo$ cp -pr /path/to/mycode/new-style/gdv .
</code>
Now you're ready to use commands like <code>hg status</code> to see what you've changed compared to H10, <code>hg ci</code> to check in your changes, <code>hg merge h13</code> to merge in changes up to version H13 (for example).

If you don't have the entire Gaussian source code in your directory you can get <code>old-to-new.sh</code> to work by making sure there's a (possibly empty) file in <code>gdv/bsd/mdl1.F</code>. There will be a few warnings but it will split out any files it can find in the <code>gdv</code> directory.

If you just have a few subroutines already split out, then you should just copy them to the appropriate place in the working directory.

This might seem like a pain but you'll only have to do it once for each version of the code you have!

== The makefile ==

In modifying the Gassian build system I have attempted to balance two competing needs. The first is a strong desire to modify as little as possible of Gaussian's original makefiles. This is so that when we receive newer versions of Gaussian, with their inevitable changes to the makefiles, our modifications can be trivially merged in. The second is to have a build system that works nicely with the new file layout, which is necessary for the version control system. The compromise that I have chosen is to have a new Makefile in the <code>gdv</code> directory that understand such things as how to build libraries from the split util and link files, and which executables depend on which libraries, while relying on the existing Gaussian makefiles for details such as compiler invocation and flags.

Sadly it was not possible to use the existing Gaussian makefile without modifying it slightly. This modified version of <code>bsd/i386.make</code> is called <code>bsd/uber-i386.make</code>. In it the rules to make most of the libraries and executables are commented out, and a few fixes for other problems are added. If you wish to compile on another platform you will need to make an uber makefile for that platform. Base this off the differences between <code>bsd/i386.make</code> and <code>bsd/uber-i386.make</code>.

If you want to add a new link you will need to add it to the Makefile, and possibly to the <code>uber-*</code> makefile.

== Tools and tricks ==

If you aren't already using these tools, you should be.

=== Tags ===

One complaint that might arise in going from monolithic link files to split files is that finding the definitions of subroutines and functions becomes much more difficult. So, if you are editing <code>l510.F</code> and are in the MCSCF deck you might come to the line that calls the Davids subroutine. Finding where this subroutine is defined used to be as simple as searching for 'e Davids' (assuming it's a subroutine, and that there is only one space between subroutine and Davids, and that it's in <code>l510.F</code>, etc). How are you to find it now?

The answer is to use '''tags'''. These have been around for a few decades and are extremely useful. Firstly you must create a tags file, or files. For vi/vim you do:
<code>
testrepo/gdv$ ctags *.F */*.F */*.c
</code>
(for Emacs substitute etags for ctags). If you want to create a tags file in one of the link subdirectories just use:
<code>
testrepo/gdv/l510$ ctags *.F ../*/*.F ../*/*.c
</code>
This will create a ctags (or etags) file in the current directory. Now, start vi (or Emacs) and edit <code>l510/mcscf.F</code>. Find the line that calls the Davids subroutine and position the cursor over the word Davids. Press <code>Ctrl-]</code> (or <code>M-.</code>) and you will be immediately taken to the definition of that subroutine, even if it’s in another file. Press <code>Ctrl-T</code> (or <code>M-*</code>) and you'll be taken back to where you were. These tags stack, that is you can go into a subroutine, and from there go into another one, and so on, and still be able to back out to the beginning.

You can also start vi (vim really) with the <code>-t <tag></code> option to jump immediately to a particular tag. I don't know if emacs can do this, as I lost the will to live while browsing the 'info' documentation.

Editors other than vi and Emacs also support tags; check the documentation. If your editor doesn't support tags it's just not good enough, sorry.

=== Screen ===

Screen is a terminal multiplexer. If you wish to have multiple terminals open on a remote system you're probably accustomed to opening multiple SSH windows from your local system. An alternative is to use <code>screen</code>. It allows you to have multiple windows, each with its own history and possibly executing commands, and switch between them with some simple keypresses. It also preserves the state of these windows when your connection to screen drops, allowing you to reconnect later.

It's a bit beyond the scope of this page, if you want to learn more check out the various web resources or it, such as http://www.rackaid.com/resources/linux-screen-tutorial-and-how-to/.

Resgrp:comp-photo-new gdv layout

2012-02-29T10:59:58Z

Scliffor: /* Tags */

== Setting up ==

Gaussian insists on having its executables not readable by world. The best way to achieve this is to have a properly set umask; this is a good idea anyway, as having things readable (or writable!) by others is rarely a good idea. If you need to allow others to view things you can set the permissions explicitly. One problem here is that the Gaussian makefiles use the csh, which no sane person uses as a shell. It's therefore possible that your umask is not being set for the c-shell, which results in world readable files by default. Fix this permanently by inserting this line into ''both'' <code>~/.bashrc</code> and <code>~/.cshrc</code> (you may have to create the latter):
<code>
umask 077
</code>

== The new Gaussian development version layout ==

When you first check out a version of gdv from the [[Resgrp:comp-photo-version_control|mercurial repository]] you will notice that the code is laid out in a different format. Most of the large monolithic files have been broken down into subdirectories with their constituent subroutines split out into individual files. For example, <code>l510.F</code> has been replaced by a <code>l510</code> directory containing each <code>l510</code> subroutine in its own file. Most of the utility files have been processed similarly. This layout is primarily to help the functioning of the version control system. With the old system a modification to a subroutine within <code>l510.F</code> would show up only as a change to the file <code>l510.F</code>, with only line numbers to help you guess what subroutine had actually been altered. Moving a subroutine from a link file to a utility file would only show up as a deletion of a block of lines from one and an addition of a block of lines to the other. The new system allows us to immediately see which subroutines have been altered and to track them if they are moved or copied.
When you check out the repository there are currently three scripts next to the <code>gdv</code> subdirectory. The <code>new-to-old.sh</code> and <code>old-to-new.sh</code> scripts convert between the two formats. See [[#Importing your own code|Importing your own code]] for details on <code>old-to-new.sh</code>. These will be useful when receiving new tarballs from Gaussian, or converting already written code, or when the time comes to send code back to Gaussian. The <code>new-to-old.sh</code> script attempts to get a distribution very close to an old form Gaussian layout, but there will be differences. Some old-style files are in an arbitrary order which the script does not attempt to recreate. Fortunately these are not files that we are likely to change.
The third script, <code>pillage.sh</code>, takes executables, libraries, and object files from an already compiled version of Gaussian. It shouldn't matter whether the previously compiled version is new style or old-style.
Building from scratch.
If you choose not to use <code>pillage.sh</code> then you must build from scratch. This is actually quite quick. Assuming you have checked out the version you wish to compile, and have loaded the appropriate compiler and Linda modules, you should <code>cd</code> to the <code>gdv</code> directory and type:
<code>
gdv$ bsd/bldgdv
...[lots#s of output]...
echo Currrently gdv.make points to: i386.make
Currrently gdv.make points to: i386.make
if ( -f bsd/uber-i386.make ) then
rm -f bsd/gdv.make
ln -s uber-i386.make bsd/gdv.make
echo Changing gdv.make to point to bsd/uber-i386.make
Changing gdv.make to point to bsd/uber-i386.make
endif
exit 1
gdv$
</code>
The modified <code>bldgdv</code> script terminates almost immediately. It will do the usual Gaussian setting up of the built environment, including setting the symbolic link for <code>bsd/gdv.make</code> to point to the appropriate makefile. However our modification will cause it to look for an "uber-" prefixed version of that makefile. The uber here means that there is one make file over the entire subdirectory. At the time of writing there is only one uber- prefixed makefile and that is <code>uber-i386.make</code>. The modifications to make an uber- makefile are quite simple, and are covered [[#The makefile|later]].

Provided there is an uber makefile now set you are now ready to use the new build system. To build the entirety of Gaussian type:
<code>
gdv$ make build-tools && make -j 6 util.a && make util.a && make -j 6 exe && make exe 2>&1 | tee make.log
[... lots of output ]
gdv$
</code>
The number 6 in this example instructs make to run six parallel threads while building. You can use different numbers depending on how many processors the system you're building on has, how loaded it is, and so on. I find that building in the <code>/tmp</code> directory with a relatively unloaded cx1 login node, I can build Gaussian in about 15 minutes. You'll notice that after each parallel invocation of make I have a serial invocation at the same time for the same target. This is because sometimes the parallel make loses track of where it is and doesn't fully make a target. The serial invocation should always correct this.
Once Gaussian is built (and you can always check by seeing that <code>make exe</code> returns <code>Nothing to be done for: `exe'</code>) you can also build a Linda version by typing <code>make linda</code>. You will need the 8.2 version of the Linda compiler in your path; I suggest this is made into a module.

== Using <code>pillage.sh</code> ==
<code>cd</code> into the <code>gdv</code> subdirectory of your freshly checked out repository, type:
<code>
gdv$ ../pillage.sh /home/gaussian-devel/gaussiandvh13_pgi_118/gdv
</code>
(for example) wait a while and when the prompt returns you should have a fully populated and up-to-date set of binaries. <code>pillage.sh</code> touches all the imported libraries and executables so they are newer than the source code. This means that if you type:
<code>
gdv$ make
ln -sf browse arc
gdv$
</code>
only some trivial things get made.

It's up to you to make sure that any development you do uses a consistent compiler, Linda compiler, etc, to the pillaged binaries.

== Doing it yourself ==

I did try to make a version of <code>pillage.sh</code> that linked to executables instead of copying them, and you may feel tempted to try the same. The main problem that you need to overcome is that if the libraries you are linking to are older than their corresponding <code>.F</code> files make will try to update them. If the libraries are not yours this will simply fail and cause make to exit. However, if the symbolically linked libraries are yours they will be updated, which is probably not what you want!
If you really need to save space there are some easier things you can do. Starting from an empty repository, run <code>bldgdv</code> and make the build tools as above. Then copy an appropriate util.a from somewhere to your <code>gdv</code> directory (do not use <code>cp -p</code>). Finally, use make to build only the links you are planning to alter, e.g. <code>make l510.exe</code>. You should also alter the Makefile so that the mystuff target lists only these links. This saves space by not having any of the other links' libraries or executables. You will also not have any <code>.o</code> files for the util subroutines; as long as the util.a file is newer than all of the util <code>.F</code> files make will not try to make them. If, however, even one util <code>.F</code> file becomes newer than util.a make will make '''all''' of the util <code>.o</code> files!
Now you can use the <code>%subst</code> keyword to Gaussian to tell it to pick up the appropriate links from this directory.

== Developing ==

However you have built the binaries developing in the new system is very simple. Edit the files you need to change, type <code>make</code>, or <code>make Linda</code> from the <code>gdv</code> directory. You can also type make inside a link subdirectory, which will build just that link's executable. The makefile knows about Gaussian's dependencies, so for example if you change a subroutine in <code>l510</code> then both <code>l510.exe</code> and <code>l1003.exe</code> will be re-linked.

By default the makefile checks everything to see if it's up-to-date. As there are nearly 12,000 files in this layout this can take a few seconds, longer if the filesystem is being stressed. I find that with a hot cache on <code>/tmp</code> make takes less than a second to check everything. With a cold cache on <code>/home</code> it can take some tens of seconds. If you want to get make to only check specific executables edit the Makefile in <code>gdv</code>, uncomment the "mystuff" line, and change it appropriately. Note, however, that all the util files are always checked. The makefile is a tracked file in the repository, so please make sure you don't check in these trivial changes.

You do not need to edit the makefile if you add, move, or delete subroutines.

== Importing your own code ==

If your code is in the old layout, i.e. it's all monolithic links and utilities just as in the official Gaussian distribution tarball, then use the <code>old-to-new.sh</code> script. First, check out the version of the development code that was the basis of your code. Let's assume you took the H10 code and have been working on that, but have not merged in changes for H11, H12P or H13.
<code>
tmp$ hg clone -u h10 /home/gaussiandevel/example-gaussian-repo /tmp/testrepo
</code>
'''or''', if you already have a repository checked out, <code>cd</code> into it and do:
<code>
tmp$ hg update h10
</code>

The <code>old-to-new.sh</code> script assumes that you've got a <code>gau-fsplit</code> command in your <code>$PATH</code>. If you haven't, you can create one by building the build-tools, as explained above, and making sure this directory appears, however temporarily, in your <code>$PATH</code>. Or, you can point your path to the pre-built binaries in <code>/home/gaussian-devel</code>.

Now change directory to be above the <code>gdv</code> directory of your own code. Run the <code>old-to-new.sh</code> script, with an argument of the directory you want the new style to appear in (it should not exist):
<code>
tmp$ cd /path/to/my-code
my-code$ ls
gdv
my-code$ PATH=$PATH:/home/gaussian-devel/gaussiandvh10_pgi_105/gdv /tmp/testrepo/old-to-new.sh new-style
[...some output...]
my-code$ ls
gdv new-style
</code>
Now remove the gdv directory in the newly checked out working directory and replace it with the one from new-style.
<code>
my-code$ cd /tmp/testrepo
testrepo$ rm -fr gdv
testrepo$ cp -pr /path/to/mycode/new-style/gdv .
</code>
Now you're ready to use commands like <code>hg status</code> to see what you've changed compared to H10, <code>hg ci</code> to check in your changes, <code>hg merge h13</code> to merge in changes up to version H13 (for example).

If you don't have the entire Gaussian source code in your directory you can get <code>old-to-new.sh</code> to work by making sure there's a (possibly empty) file in <code>gdv/bsd/mdl1.F</code>. There will be a few warnings but it will split out any files it can find in the <code>gdv</code> directory.

If you just have a few subroutines already split out, then you should just copy them to the appropriate place in the working directory.

This might seem like a pain but you'll only have to do it once for each version of the code you have!

== The makefile ==

In modifying the Gassian build system I have attempted to balance two competing needs. The first is a strong desire to modify as little as possible of Gaussian's original makefiles. This is so that when we receive newer versions of Gaussian, with their inevitable changes to the makefiles, our modifications can be trivially merged in. The second is to have a build system that works nicely with the new file layout, which is necessary for the version control system. The compromise that I have chosen is to have a new Makefile in the <code>gdv</code> directory that understand such things as how to build libraries from the split util and link files, and which executables depend on which libraries, while relying on the existing Gaussian makefiles for details such as compiler invocation and flags.

Sadly it was not possible to use the existing Gaussian makefile without modifying it slightly. This modified version of <code>bsd/i386.make</code> is called <code>bsd/uber-i386.make</code>. In it the rules to make most of the libraries and executables are commented out, and a few fixes for other problems are added. If you wish to compile on another platform you will need to make an uber makefile for that platform. Base this off the differences between <code>bsd/i386.make</code> and <code>bsd/uber-i386.make</code>.

If you want to add a new link you will need to add it to the Makefile, and possibly to the <code>uber-*</code> makefile.

== Tools and tricks ==

If you aren't already using these tools, you should be.

=== Tags ===

One complaint that might arise in going from monolithic link files to split files is that finding the definitions of subroutines and functions becomes much more difficult. So, if you are editing <code>l510.F</code> and are in the MCSCF deck you might come to the line that calls the Davids subroutine. Finding where this subroutine is defined used to be as simple as searching for 'e Davids' (assuming it's a subroutine, and that there is only one space between subroutine and Davids, and that it's in <code>l510.F</code>, etc). How are you to find it now?

The answer is to use '''tags'''. These have been around for a few decades and are extremely useful. Firstly you must create a tags file, or files. For vi/vim you do:
<code>
testrepo/gdv$ ctags *.F */*.F */*.c
</code>
(for Emacs substitute etags for ctags). If you want to create a tags file in one of the link subdirectories just use:
<code>
testrepo/gdv/l510$ ctags *.F ../*/*.F ../*/*.c
</code>
This will create a ctags (or etags) file in the current directory. Now, start vi (or Emacs) and edit <code>l510/mcscf.F</code>. Find the line that calls the Davids subroutine and position the cursor over the word Davids. Press <code>Ctrl-]</code> (or <code>M-.</code>) and you will be immediately taken to the definition of that subroutine, even if it’s in another file. Press <code>Ctrl-T</code> (or <code>M-*</code>) and you'll be taken back to where you were. These tags stack, that is you can go into a subroutine, and from there go into another one, and so on, and still be able to back out to the beginning.

You can also start vi (vim really) with the <code>-t <tag></code> option to jump immediately to a particular tag. I don't know if emacs can do this, as I lost the will to live while browsing the 'info' documentation.

Editors other than vi and Emacs also support tags; check the documentation. If your editor doesn't support tags it's just not good enough, sorry.

=== Screen ===

Screen is a terminal multiplexer. If you wish to have multiple terminals open on a remote system you're probably accustomed to opening multiple SSH windows from your local system. An alternative is to use <code>screen</code>. It allows you to have multiple windows, each with its own history and possibly executing commands, and switch between them with some simple keypresses. It also preserves the state of these windows when your connection to screen drops, allowing you to reconnect later.

It's a bit beyond the scope of this page, if you want to learn more check out the various web resources or it, such as http://www.rackaid.com/resources/linux-screen-tutorial-and-how-to/.

Resgrp:comp-photo-new gdv layout

2012-02-27T10:20:43Z

Scliffor:

== Setting up ==

Gaussian insists on having its executables not readable by world. The best way to achieve this is to have a properly set umask; this is a good idea anyway, as having things readable (or writable!) by others is rarely a good idea. If you need to allow others to view things you can set the permissions explicitly. One problem here is that the Gaussian makefiles use the csh, which no sane person uses as a shell. It's therefore possible that your umask is not being set for the c-shell, which results in world readable files by default. Fix this permanently by inserting this line into ''both'' <code>~/.bashrc</code> and <code>~/.cshrc</code> (you may have to create the latter):
<code>
umask 077
</code>

== The new Gaussian development version layout ==

When you first check out a version of gdv from the [[Resgrp:comp-photo-version_control|mercurial repository]] you will notice that the code is laid out in a different format. Most of the large monolithic files have been broken down into subdirectories with their constituent subroutines split out into individual files. For example, <code>l510.F</code> has been replaced by a <code>l510</code> directory containing each <code>l510</code> subroutine in its own file. Most of the utility files have been processed similarly. This layout is primarily to help the functioning of the version control system. With the old system a modification to a subroutine within <code>l510.F</code> would show up only as a change to the file <code>l510.F</code>, with only line numbers to help you guess what subroutine had actually been altered. Moving a subroutine from a link file to a utility file would only show up as a deletion of a block of lines from one and an addition of a block of lines to the other. The new system allows us to immediately see which subroutines have been altered and to track them if they are moved or copied.
When you check out the repository there are currently three scripts next to the <code>gdv</code> subdirectory. The <code>new-to-old.sh</code> and <code>old-to-new.sh</code> scripts convert between the two formats. See [[#Importing your own code|Importing your own code]] for details on <code>old-to-new.sh</code>. These will be useful when receiving new tarballs from Gaussian, or converting already written code, or when the time comes to send code back to Gaussian. The <code>new-to-old.sh</code> script attempts to get a distribution very close to an old form Gaussian layout, but there will be differences. Some old-style files are in an arbitrary order which the script does not attempt to recreate. Fortunately these are not files that we are likely to change.
The third script, <code>pillage.sh</code>, takes executables, libraries, and object files from an already compiled version of Gaussian. It shouldn't matter whether the previously compiled version is new style or old-style.
Building from scratch.
If you choose not to use <code>pillage.sh</code> then you must build from scratch. This is actually quite quick. Assuming you have checked out the version you wish to compile, and have loaded the appropriate compiler and Linda modules, you should <code>cd</code> to the <code>gdv</code> directory and type:
<code>
gdv$ bsd/bldgdv
...[lots#s of output]...
echo Currrently gdv.make points to: i386.make
Currrently gdv.make points to: i386.make
if ( -f bsd/uber-i386.make ) then
rm -f bsd/gdv.make
ln -s uber-i386.make bsd/gdv.make
echo Changing gdv.make to point to bsd/uber-i386.make
Changing gdv.make to point to bsd/uber-i386.make
endif
exit 1
gdv$
</code>
The modified <code>bldgdv</code> script terminates almost immediately. It will do the usual Gaussian setting up of the built environment, including setting the symbolic link for <code>bsd/gdv.make</code> to point to the appropriate makefile. However our modification will cause it to look for an "uber-" prefixed version of that makefile. The uber here means that there is one make file over the entire subdirectory. At the time of writing there is only one uber- prefixed makefile and that is <code>uber-i386.make</code>. The modifications to make an uber- makefile are quite simple, and are covered [[#The makefile|later]].

Provided there is an uber makefile now set you are now ready to use the new build system. To build the entirety of Gaussian type:
<code>
gdv$ make build-tools && make -j 6 util.a && make util.a && make -j 6 exe && make exe 2>&1 | tee make.log
[... lots of output ]
gdv$
</code>
The number 6 in this example instructs make to run six parallel threads while building. You can use different numbers depending on how many processors the system you're building on has, how loaded it is, and so on. I find that building in the <code>/tmp</code> directory with a relatively unloaded cx1 login node, I can build Gaussian in about 15 minutes. You'll notice that after each parallel invocation of make I have a serial invocation at the same time for the same target. This is because sometimes the parallel make loses track of where it is and doesn't fully make a target. The serial invocation should always correct this.
Once Gaussian is built (and you can always check by seeing that <code>make exe</code> returns <code>Nothing to be done for: `exe'</code>) you can also build a Linda version by typing <code>make linda</code>. You will need the 8.2 version of the Linda compiler in your path; I suggest this is made into a module.

== Using <code>pillage.sh</code> ==
<code>cd</code> into the <code>gdv</code> subdirectory of your freshly checked out repository, type:
<code>
gdv$ ../pillage.sh /home/gaussian-devel/gaussiandvh13_pgi_118/gdv
</code>
(for example) wait a while and when the prompt returns you should have a fully populated and up-to-date set of binaries. <code>pillage.sh</code> touches all the imported libraries and executables so they are newer than the source code. This means that if you type:
<code>
gdv$ make
ln -sf browse arc
gdv$
</code>
only some trivial things get made.

It's up to you to make sure that any development you do uses a consistent compiler, Linda compiler, etc, to the pillaged binaries.

== Doing it yourself ==

I did try to make a version of <code>pillage.sh</code> that linked to executables instead of copying them, and you may feel tempted to try the same. The main problem that you need to overcome is that if the libraries you are linking to are older than their corresponding <code>.F</code> files make will try to update them. If the libraries are not yours this will simply fail and cause make to exit. However, if the symbolically linked libraries are yours they will be updated, which is probably not what you want!
If you really need to save space there are some easier things you can do. Starting from an empty repository, run <code>bldgdv</code> and make the build tools as above. Then copy an appropriate util.a from somewhere to your <code>gdv</code> directory (do not use <code>cp -p</code>). Finally, use make to build only the links you are planning to alter, e.g. <code>make l510.exe</code>. You should also alter the Makefile so that the mystuff target lists only these links. This saves space by not having any of the other links' libraries or executables. You will also not have any <code>.o</code> files for the util subroutines; as long as the util.a file is newer than all of the util <code>.F</code> files make will not try to make them. If, however, even one util <code>.F</code> file becomes newer than util.a make will make '''all''' of the util <code>.o</code> files!
Now you can use the <code>%subst</code> keyword to Gaussian to tell it to pick up the appropriate links from this directory.

== Developing ==

However you have built the binaries developing in the new system is very simple. Edit the files you need to change, type <code>make</code>, or <code>make Linda</code> from the <code>gdv</code> directory. You can also type make inside a link subdirectory, which will build just that link's executable. The makefile knows about Gaussian's dependencies, so for example if you change a subroutine in <code>l510</code> then both <code>l510.exe</code> and <code>l1003.exe</code> will be re-linked.

By default the makefile checks everything to see if it's up-to-date. As there are nearly 12,000 files in this layout this can take a few seconds, longer if the filesystem is being stressed. I find that with a hot cache on <code>/tmp</code> make takes less than a second to check everything. With a cold cache on <code>/home</code> it can take some tens of seconds. If you want to get make to only check specific executables edit the Makefile in <code>gdv</code>, uncomment the "mystuff" line, and change it appropriately. Note, however, that all the util files are always checked. The makefile is a tracked file in the repository, so please make sure you don't check in these trivial changes.

You do not need to edit the makefile if you add, move, or delete subroutines.

== Importing your own code ==

If your code is in the old layout, i.e. it's all monolithic links and utilities just as in the official Gaussian distribution tarball, then use the <code>old-to-new.sh</code> script. First, check out the version of the development code that was the basis of your code. Let's assume you took the H10 code and have been working on that, but have not merged in changes for H11, H12P or H13.
<code>
tmp$ hg clone -u h10 /home/gaussiandevel/example-gaussian-repo /tmp/testrepo
</code>
'''or''', if you already have a repository checked out, <code>cd</code> into it and do:
<code>
tmp$ hg update h10
</code>

The <code>old-to-new.sh</code> script assumes that you've got a <code>gau-fsplit</code> command in your <code>$PATH</code>. If you haven't, you can create one by building the build-tools, as explained above, and making sure this directory appears, however temporarily, in your <code>$PATH</code>. Or, you can point your path to the pre-built binaries in <code>/home/gaussian-devel</code>.

Now change directory to be above the <code>gdv</code> directory of your own code. Run the <code>old-to-new.sh</code> script, with an argument of the directory you want the new style to appear in (it should not exist):
<code>
tmp$ cd /path/to/my-code
my-code$ ls
gdv
my-code$ PATH=$PATH:/home/gaussian-devel/gaussiandvh10_pgi_105/gdv /tmp/testrepo/old-to-new.sh new-style
[...some output...]
my-code$ ls
gdv new-style
</code>
Now remove the gdv directory in the newly checked out working directory and replace it with the one from new-style.
<code>
my-code$ cd /tmp/testrepo
testrepo$ rm -fr gdv
testrepo$ cp -pr /path/to/mycode/new-style/gdv .
</code>
Now you're ready to use commands like <code>hg status</code> to see what you've changed compared to H10, <code>hg ci</code> to check in your changes, <code>hg merge h13</code> to merge in changes up to version H13 (for example).

If you don't have the entire Gaussian source code in your directory you can get <code>old-to-new.sh</code> to work by making sure there's a (possibly empty) file in <code>gdv/bsd/mdl1.F</code>. There will be a few warnings but it will split out any files it can find in the <code>gdv</code> directory.

If you just have a few subroutines already split out, then you should just copy them to the appropriate place in the working directory.

This might seem like a pain but you'll only have to do it once for each version of the code you have!

== The makefile ==

In modifying the Gassian build system I have attempted to balance two competing needs. The first is a strong desire to modify as little as possible of Gaussian's original makefiles. This is so that when we receive newer versions of Gaussian, with their inevitable changes to the makefiles, our modifications can be trivially merged in. The second is to have a build system that works nicely with the new file layout, which is necessary for the version control system. The compromise that I have chosen is to have a new Makefile in the <code>gdv</code> directory that understand such things as how to build libraries from the split util and link files, and which executables depend on which libraries, while relying on the existing Gaussian makefiles for details such as compiler invocation and flags.

Sadly it was not possible to use the existing Gaussian makefile without modifying it slightly. This modified version of <code>bsd/i386.make</code> is called <code>bsd/uber-i386.make</code>. In it the rules to make most of the libraries and executables are commented out, and a few fixes for other problems are added. If you wish to compile on another platform you will need to make an uber makefile for that platform. Base this off the differences between <code>bsd/i386.make</code> and <code>bsd/uber-i386.make</code>.

If you want to add a new link you will need to add it to the Makefile, and possibly to the <code>uber-*</code> makefile.

== Tools and tricks ==

If you aren't already using these tools, you should be.

=== Tags ===

One complaint that might arise in going from monolithic link files to split files is that finding the definitions of subroutines and functions becomes much more difficult. So, if you are editing <code>l510.F</code> and are in the MCSCF deck you might come to the line that calls the Davids subroutine. Finding where this subroutine is defined used to be as simple as searching for 'e Davids' (assuming it's a subroutine, and that there is only one space between subroutine and Davids, and that it's in <code>l510.F</code>, etc). How are you to find it now?

The answer is to use '''tags'''. These have been around for a few decades and are extremely useful. Firstly you must create a tags file, or files. For vi/vim you do:
<code>
testrepo/gdv$ ctags *.F */*.F */*.c
</code>
(for Emacs substitute etags for ctags). If you want to create a tags file in one of the link subdirectories just use:
<code>
testrepo/gdv/l510$ ctags *.F ../*/*.F ../*/*.c
</code>
This will create a ctags (or etags) file in the current directory. Now, start vi (or Emacs) and edit <code>l510/mcscf.F</code>. Find the line that calls the Davids subroutine and position the cursor over the word Davids. Press <code>Ctrl-]</code> (or <code>M-.</code>) and you will be immediately taken to the definition of that subroutine, even if it’s in another file. Press <code>Ctrl-T</code> (or <code>M-*</code>) and you'll be taken back to where you were. These tags stack, that is you can go into a subroutine, and from there go into another one, and so on, and still be able to back out to the beginning.

Editors other than vi and Emacs also support tags; check the documentation. If your editor doesn't support tags it's just not good enough, sorry.

=== Screen ===

Screen is a terminal multiplexer. If you wish to have multiple terminals open on a remote system you're probably accustomed to opening multiple SSH windows from your local system. An alternative is to use <code>screen</code>. It allows you to have multiple windows, each with its own history and possibly executing commands, and switch between them with some simple keypresses. It also preserves the state of these windows when your connection to screen drops, allowing you to reconnect later.

It's a bit beyond the scope of this page, if you want to learn more check out the various web resources or it, such as http://www.rackaid.com/resources/linux-screen-tutorial-and-how-to/.

Resgrp:comp-photo-new gdv layout

2012-02-08T12:28:29Z

Scliffor:

== The new Gaussian development version layout ==

When you first check out a version of gdv from the [[Resgrp:comp-photo-version_control|mercurial repository]] you will notice that the code is laid out in a different format. Most of the large monolithic files have been broken down into subdirectories with their constituent subroutines split out into individual files. For example, <code>l510.F</code> has been replaced by a <code>l510</code> directory containing each <code>l510</code> subroutine in its own file. Most of the utility files have been processed similarly. This layout is primarily to help the functioning of the version control system. With the old system a modification to a subroutine within <code>l510.F</code> would show up only as a change to the file <code>l510.F</code>, with only line numbers to help you guess what subroutine had actually been altered. Moving a subroutine from a link file to a utility file would only show up as a deletion of a block of lines from one and an addition of a block of lines to the other. The new system allows us to immediately see which subroutines have been altered and to track them if they are moved or copied.
When you check out the repository there are currently three scripts next to the <code>gdv</code> subdirectory. The <code>new-to-old.sh</code> and <code>old-to-new.sh</code> scripts convert between the two formats. See [[#Importing your own code|Importing your own code]] for details on <code>old-to-new.sh</code>. These will be useful when receiving new tarballs from Gaussian, or converting already written code, or when the time comes to send code back to Gaussian. The <code>new-to-old.sh</code> script attempts to get a distribution very close to an old form Gaussian layout, but there will be differences. Some old-style files are in an arbitrary order which the script does not attempt to recreate. Fortunately these are not files that we are likely to change.
The third script, <code>pillage.sh</code>, takes executables, libraries, and object files from an already compiled version of Gaussian. It shouldn't matter whether the previously compiled version is new style or old-style.
Building from scratch.
If you choose not to use <code>pillage.sh</code> then you must build from scratch. This is actually quite quick. Assuming you have checked out the version you wish to compile, and have loaded the appropriate compiler and Linda modules, you should <code>cd</code> to the <code>gdv</code> directory and type:
<code>
gdv$ bsd/bldgdv
...[lots#s of output]...
echo Currrently gdv.make points to: i386.make
Currrently gdv.make points to: i386.make
if ( -f bsd/uber-i386.make ) then
rm -f bsd/gdv.make
ln -s uber-i386.make bsd/gdv.make
echo Changing gdv.make to point to bsd/uber-i386.make
Changing gdv.make to point to bsd/uber-i386.make
endif
exit 1
gdv$
</code>
The modified <code>bldgdv</code> script terminates almost immediately. It will do the usual Gaussian setting up of the built environment, including setting the symbolic link for <code>bsd/gdv.make</code> to point to the appropriate makefile. However our modification will cause it to look for an "uber-" prefixed version of that makefile. The uber here means that there is one make file over the entire subdirectory. At the time of writing there is only one uber- prefixed makefile and that is <code>uber-i386.make</code>. The modifications to make an uber- makefile are quite simple, and are covered [[#The makefile|later]].

Provided there is an uber makefile now set you are now ready to use the new build system. To build the entirety of Gaussian type:
<code>
gdv$ make build-tools && make -j 6 util.a && make util.a && make -j 6 exe && make exe 2>&1 | tee make.log
[... lots of output ]
gdv$
</code>
The number 6 in this example instructs make to run six parallel threads while building. You can use different numbers depending on how many processors the system you're building on has, how loaded it is, and so on. I find that building in the <code>/tmp</code> directory with a relatively unloaded cx1 login node, I can build Gaussian in about 15 minutes. You'll notice that after each parallel invocation of make I have a serial invocation at the same time for the same target. This is because sometimes the parallel make loses track of where it is and doesn't fully make a target. The serial invocation should always correct this.
Once Gaussian is built (and you can always check by seeing that <code>make exe</code> returns <code>Nothing to be done for: `exe'</code>) you can also build a Linda version by typing <code>make linda</code>. You will need the 8.2 version of the Linda compiler in your path; I suggest this is made into a module.

== Using <code>pillage.sh</code> ==
<code>cd</code> into the <code>gdv</code> subdirectory of your freshly checked out repository, type:
<code>
gdv$ ../pillage.sh /home/gaussian-devel/gaussiandvh13_pgi_118/gdv
</code>
(for example) wait a while and when the prompt returns you should have a fully populated and up-to-date set of binaries. <code>pillage.sh</code> touches all the imported libraries and executables so they are newer than the source code. This means that if you type:
<code>
gdv$ make
ln -sf browse arc
gdv$
</code>
only some trivial things get made.

It's up to you to make sure that any development you do uses a consistent compiler, Linda compiler, etc, to the pillaged binaries.

== Doing it yourself ==

I did try to make a version of <code>pillage.sh</code> that linked to executables instead of copying them, and you may feel tempted to try the same. The main problem that you need to overcome is that if the libraries you are linking to are older than their corresponding <code>.F</code> files make will try to update them. If the libraries are not yours this will simply fail and cause make to exit. However, if the symbolically linked libraries are yours they will be updated, which is probably not what you want!
If you really need to save space there are some easier things you can do. Starting from an empty repository, run <code>bldgdv</code> and make the build tools as above. Then copy an appropriate util.a from somewhere to your <code>gdv</code> directory (do not use <code>cp -p</code>). Finally, use make to build only the links you are planning to alter, e.g. <code>make l510.exe</code>. You should also alter the Makefile so that the mystuff target lists only these links. This saves space by not having any of the other links' libraries or executables. You will also not have any <code>.o</code> files for the util subroutines; as long as the util.a file is newer than all of the util <code>.F</code> files make will not try to make them. If, however, even one util <code>.F</code> file becomes newer than util.a make will make '''all''' of the util <code>.o</code> files!
Now you can use the <code>%subst</code> keyword to Gaussian to tell it to pick up the appropriate links from this directory.

== Developing ==

However you have built the binaries developing in the new system is very simple. Edit the files you need to change, type <code>make</code>, or <code>make Linda</code> from the <code>gdv</code> directory. You can also type make inside a link subdirectory, which will build just that link's executable. The makefile knows about Gaussian's dependencies, so for example if you change a subroutine in <code>l510</code> then both <code>l510.exe</code> and <code>l1003.exe</code> will be re-linked.

By default the makefile checks everything to see if it's up-to-date. As there are nearly 12,000 files in this layout this can take a few seconds, longer if the filesystem is being stressed. I find that with a hot cache on <code>/tmp</code> make takes less than a second to check everything. With a cold cache on <code>/home</code> it can take some tens of seconds. If you want to get make to only check specific executables edit the Makefile in <code>gdv</code>, uncomment the "mystuff" line, and change it appropriately. Note, however, that all the util files are always checked. The makefile is a tracked file in the repository, so please make sure you don't check in these trivial changes.

You do not need to edit the makefile if you add, move, or delete subroutines.

== Importing your own code ==

If your code is in the old layout, i.e. it's all monolithic links and utilities just as in the official Gaussian distribution tarball, then use the <code>old-to-new.sh</code> script. First, check out the version of the development code that was the basis of your code. Let's assume you took the H10 code and have been working on that, but have not merged in changes for H11, H12P or H13.
<code>
tmp$ hg clone -u h10 /home/gaussiandevel/example-gaussian-repo /tmp/testrepo
</code>
'''or''', if you already have a repository checked out, <code>cd</code> into it and do:
<code>
tmp$ hg update h10
</code>

The <code>old-to-new.sh</code> script assumes that you've got a <code>gau-fsplit</code> command in your <code>$PATH</code>. If you haven't, you can create one by building the build-tools, as explained above, and making sure this directory appears, however temporarily, in your <code>$PATH</code>. Or, you can point your path to the pre-built binaries in <code>/home/gaussian-devel</code>.

Now change directory to be above the <code>gdv</code> directory of your own code. Run the <code>old-to-new.sh</code> script, with an argument of the directory you want the new style to appear in (it should not exist):
<code>
tmp$ cd /path/to/my-code
my-code$ ls
gdv
my-code$ PATH=$PATH:/home/gaussian-devel/gaussiandvh10_pgi_105/gdv /tmp/testrepo/old-to-new.sh new-style
[...some output...]
my-code$ ls
gdv new-style
</code>
Now remove the gdv directory in the newly checked out working directory and replace it with the one from new-style.
<code>
my-code$ cd /tmp/testrepo
testrepo$ rm -fr gdv
testrepo$ cp -pr /path/to/mycode/new-style/gdv .
</code>
Now you're ready to use commands like <code>hg status</code> to see what you've changed compared to H10, <code>hg ci</code> to check in your changes, <code>hg merge h13</code> to merge in changes up to version H13 (for example).

If you don't have the entire Gaussian source code in your directory you can get <code>old-to-new.sh</code> to work by making sure there's a (possibly empty) file in <code>gdv/bsd/mdl1.F</code>. There will be a few warnings but it will split out any files it can find in the <code>gdv</code> directory.

If you just have a few subroutines already split out, then you should just copy them to the appropriate place in the working directory.

This might seem like a pain but you'll only have to do it once for each version of the code you have!

== The makefile ==

In modifying the Gassian build system I have attempted to balance two competing needs. The first is a strong desire to modify as little as possible of Gaussian's original makefiles. This is so that when we receive newer versions of Gaussian, with their inevitable changes to the makefiles, our modifications can be trivially merged in. The second is to have a build system that works nicely with the new file layout, which is necessary for the version control system. The compromise that I have chosen is to have a new Makefile in the <code>gdv</code> directory that understand such things as how to build libraries from the split util and link files, and which executables depend on which libraries, while relying on the existing Gaussian makefiles for details such as compiler invocation and flags.

Sadly it was not possible to use the existing Gaussian makefile without modifying it slightly. This modified version of <code>bsd/i386.make</code> is called <code>bsd/uber-i386.make</code>. In it the rules to make most of the libraries and executables are commented out, and a few fixes for other problems are added. If you wish to compile on another platform you will need to make an uber makefile for that platform. Base this off the differences between <code>bsd/i386.make</code> and <code>bsd/uber-i386.make</code>.

If you want to add a new link you will need to add it to the Makefile, and possibly to the <code>uber-*</code> makefile.

== Tools and tricks ==

If you aren't already using these tools, you should be.

=== Tags ===

One complaint that might arise in going from monolithic link files to split files is that finding the definitions of subroutines and functions becomes much more difficult. So, if you are editing <code>l510.F</code> and are in the MCSCF deck you might come to the line that calls the Davids subroutine. Finding where this subroutine is defined used to be as simple as searching for 'e Davids' (assuming it's a subroutine, and that there is only one space between subroutine and Davids, and that it's in <code>l510.F</code>, etc). How are you to find it now?

The answer is to use '''tags'''. These have been around for a few decades and are extremely useful. Firstly you must create a tags file, or files. For vi/vim you do:
<code>
testrepo/gdv$ ctags *.F */*.F */*.c
</code>
(for Emacs substitute etags for ctags). If you want to create a tags file in one of the link subdirectories just use:
<code>
testrepo/gdv/l510$ ctags *.F ../*/*.F ../*/*.c
</code>
This will create a ctags (or etags) file in the current directory. Now, start vi (or Emacs) and edit <code>l510/mcscf.F</code>. Find the line that calls the Davids subroutine and position the cursor over the word Davids. Press <code>Ctrl-]</code> (or <code>M-.</code>) and you will be immediately taken to the definition of that subroutine, even if it’s in another file. Press <code>Ctrl-T</code> (or <code>M-*</code>) and you'll be taken back to where you were. These tags stack, that is you can go into a subroutine, and from there go into another one, and so on, and still be able to back out to the beginning.

Editors other than vi and Emacs also support tags; check the documentation. If your editor doesn't support tags it's just not good enough, sorry.

=== Screen ===

Screen is a terminal multiplexer. If you wish to have multiple terminals open on a remote system you're probably accustomed to opening multiple SSH windows from your local system. An alternative is to use <code>screen</code>. It allows you to have multiple windows, each with its own history and possibly executing commands, and switch between them with some simple keypresses. It also preserves the state of these windows when your connection to screen drops, allowing you to reconnect later.

It's a bit beyond the scope of this page, if you want to learn more check out the various web resources or it, such as http://www.rackaid.com/resources/linux-screen-tutorial-and-how-to/.

Resgrp:comp-photo-new gdv layout

2012-02-07T17:08:29Z

Scliffor: Created page with "== The new Gaussian development version layout == When you first check out a version of gdv from the mercurial repository you will notice t..."

== The new Gaussian development version layout ==

When you first check out a version of gdv from the [[Resgrp:comp-photo-version_control|mercurial repository]] you will notice that the code is laid out in a different format. Most of the large monolithic files have been broken down into subdirectories with their constituent subroutines split out into individual files. For example, <code>l510.F</code> has been replaced by a <code>l510</code> directory containing each <code>l510</code> subroutine in its own file. Most of the utility files have been processed similarly. This layout is primarily to help the functioning of the version control system. With the old system a modification to a subroutine within <code>l510.F</code> would show up only as a change to the file <code>l510.F</code>, with only line numbers to help you guess what subroutine had actually been altered. Moving a subroutine from a link file to a utility file would only show up as a deletion of a block of lines from one and an addition of a block of lines to the other. The new system allows us to immediately see which subroutines have been altered and to track them if they are moved or copied.
When you check out the repository there are currently three scripts next to the <code>gdv</code> subdirectory. The <code>new-to-old.sh</code> and <code>old-to-new.sh</code> scripts convert between the two formats. These will be useful when receiving new tarballs from Gaussian, or converting already written code, or when the time comes to send code back to Gaussian. The <code>new-to-old.sh</code> script attempts to get a distribution very close to an old form Gaussian layout, but there will be differences. Some old-style files are in an arbitrary order which the script does not attempt to recreate. Fortunately these are not files that we are likely to change.
The third script, <code>pillage.sh</code>, takes executables, libraries, and object files from an already compiled version of Gaussian. It shouldn't matter whether the previously compiled version is new style or old-style.
Building from scratch.
If you choose not to use <code>pillage.sh</code> then you must build from scratch. This is actually quite quick. Assuming you have checked out the version you wish to compile, and have loaded the appropriate compiler and Linda modules, you should <code>cd</code> to the <code>gdv</code> directory and type:
<code>
gdv$ bsd/bldgdv
...[lots#s of output]...
echo Currrently gdv.make points to: i386.make
Currrently gdv.make points to: i386.make
if ( -f bsd/uber-i386.make ) then
rm -f bsd/gdv.make
ln -s uber-i386.make bsd/gdv.make
echo Changing gdv.make to point to bsd/uber-i386.make
Changing gdv.make to point to bsd/uber-i386.make
endif
exit 1
gdv$
</code>
The modified <code>bldgdv</code> script terminates almost immediately. It will do the usual Gaussian setting up of the built environment, including setting the symbolic link for <code>bsd/gdv.make</code> to point to the appropriate makefile. However our modification will cause it to look for an "uber-" prefixed version of that makefile. The uber here means that there is one make file over the entire subdirectory. At the time of writing there is only one uber- prefixed makefile and that is <code>uber-i386.make</code>. The modifications to make an uber- makefile are quite simple, and are covered [[#The makefile|later]].

Provided there is an uber makefile now set you are now ready to use the new build system. To build the entirety of Gaussian type:
<code>
gdv$ make build-tools && make -j 6 util.a && make util.a && make -j 6 exe && make exe 2>&1 | tee make.log
[... lots of output ]
gdv$
</code>
The number 6 in this example instructs make to run six parallel threads while building. You can use different numbers depending on how many processors the system you're building on has, how loaded it is, and so on. I find that building in the <code>/tmp</code> directory with a relatively unloaded cx1 login node, I can build Gaussian in about 15 minutes. You'll notice that after each parallel invocation of make I have a serial invocation at the same time for the same target. This is because sometimes the parallel make loses track of where it is and doesn't fully make a target. The serial invocation should always correct this.
Once Gaussian is built (and you can always check by seeing that <code>make exe</code> returns <code>Nothing to be done for: `exe'</code>) you can also build a Linda version by typing <code>make linda</code>. You will need the 8.2 version of the Linda compiler in your path; I suggest this is made into a module.

== Using <code>pillage.sh</code> ==
<code>cd</code> into the <code>gdv</code> subdirectory of your freshly checked out repository, type:
<code>
gdv$ ../pillage.sh /home/gaussian-devel/gaussiandvh13_pgi_118/gdv
</code>
(for example) wait a while and when the prompt returns you should have a fully populated and up-to-date set of binaries. <code>pillage.sh</code> touches all the imported libraries and executables so they are newer than the source code. This means that if you type:
<code>
gdv$ make
ln -sf browse arc
gdv$
</code>
only some trivial things get made.

It's up to you to make sure that any development you do uses a consistent compiler, Linda compiler, etc, to the pillaged binaries.

== Doing it yourself ==

I did try to make a version of <code>pillage.sh</code> that linked to executables instead of copying them, and you may feel tempted to try the same. The main problem that you need to overcome is that if the libraries you are linking to are older than their corresponding <code>.F</code> files make will try to update them. If the libraries are not yours this will simply fail and cause make to exit. However, if the symbolically linked libraries are yours they will be updated, which is probably not what you want!
If you really need to save space there are some easier things you can do. Starting from an empty repository, run <code>bldgdv</code> and make the build tools as above. Then copy an appropriate util.a from somewhere to your <code>gdv</code> directory (do not use <code>cp -p</code>). Finally, use make to build only the links you are planning to alter, e.g. <code>make l510.exe</code>. You should also alter the Makefile so that the mystuff target lists only these links. This saves space by not having any of the other links' libraries or executables. You will also not have any <code>.o</code> files for the util subroutines; as long as the util.a file is newer than all of the util <code>.F</code> files make will not try to make them. If, however, even one util <code>.F</code> file becomes newer than util.a make will make '''all''' of the util <code>.o</code> files!
Now you can use the <code>%subst</code> keyword to Gaussian to tell it to pick up the appropriate links from this directory.

== Developing ==

However you have built the binaries developing in the new system is very simple. Edit the files you need to change, type <code>make</code>, or <code>make Linda</code> from the <code>gdv</code> directory. You can also type make inside a link subdirectory, which will build just that link's executable. The makefile knows about Gaussian's dependencies, so for example if you change a subroutine in <code>l510</code> then both <code>l510.exe</code> and <code>l1003.exe</code> will be re-linked.

By default the makefile checks everything to see if it's up-to-date. As there are nearly 12,000 files in this layout this can take a few seconds, longer if the filesystem is being stressed. I find that with a hot cache on <code>/tmp</code> make takes less than a second to check everything. With a cold cache on <code>/home</code> it can take some tens of seconds. If you want to get make to only check specific executables edit the Makefile in <code>gdv</code>, uncomment the "mystuff" line, and change it appropriately. Note, however, that all the util files are always checked. The makefile is a tracked file in the repository, so please make sure you don't check in these trivial changes.

You do not need to edit the makefile if you add, move, or delete subroutines.

== The makefile ==

In modifying the Gassian build system I have attempted to balance two competing needs. The first is a strong desire to modify as little as possible of Gaussian's original makefiles. This is so that when we receive newer versions of Gaussian, with their inevitable changes to the makefiles, our modifications can be trivially merged in. The second is to have a build system that works nicely with the new file layout, which is necessary for the version control system. The compromise that I have chosen is to have a new Makefile in the <code>gdv</code> directory that understand such things as how to build libraries from the split util and link files, and which executables depend on which libraries, while relying on the existing Gaussian makefiles for details such as compiler invocation and flags.

Sadly it was not possible to use the existing Gaussian makefile without modifying it slightly. This modified version of <code>bsd/i386.make</code> is called <code>bsd/uber-i386.make</code>. In it the rules to make most of the libraries and executables are commented out, and a few fixes for other problems are added. If you wish to compile on another platform you will need to make an uber makefile for that platform. Base this off the differences between <code>bsd/i386.make</code> and <code>bsd/uber-i386.make</code>.

If you want to add a new link you will need to add it to the Makefile, and possibly to the <code>uber-*</code> makefile.

== Tools and tricks ==

If you aren't already using these tools, you should be.

=== Tags ===

One complaint that might arise in going from monolithic link files to split files is that finding the definitions of subroutines and functions becomes much more difficult. So, if you are editing <code>l510.F</code> and are in the MCSCF deck you might come to the line that calls the Davids subroutine. Finding where this subroutine is defined used to be as simple as searching for 'e Davids' (assuming it's a subroutine, and that there is only one space between subroutine and Davids, and that it's in <code>l510.F</code>, etc). How are you to find it now?

The answer is to use '''tags'''. These have been around for a few decades and are extremely useful. Firstly you must create a tags file, or files. For vi/vim you do:
<code>
testrepo/gdv$ ctags *.F */*.F */*.c
</code>
(for Emacs substitute etags for ctags). If you want to create a tags file in one of the link subdirectories just use:
<code>
testrepo/gdv/l510$ ctags *.F ../*/*.F ../*/*.c
</code>
This will create a ctags (or etags) file in the current directory. Now, start vi (or Emacs) and edit <code>l510/mcscf.F</code>. Find the line that calls the Davids subroutine and position the cursor over the word Davids. Press <code>Ctrl-]</code> (or <code>M-.</code>) and you will be immediately taken to the definition of that subroutine, even if it’s in another file. Press <code>Ctrl-T</code> (or <code>M-*</code>) and you'll be taken back to where you were. These tags stack, that is you can go into a subroutine, and from there go into another one, and so on, and still be able to back out to the beginning.

Editors other than vi and Emacs also support tags; check the documentation. If your editor doesn't support tags it's just not good enough, sorry.

=== Screen ===

Screen is a terminal multiplexer. If you wish to have multiple terminals open on a remote system you're probably accustomed to opening multiple SSH windows from your local system. An alternative is to use <code>screen</code>. It allows you to have multiple windows, each with its own history and possibly executing commands, and switch between them with some simple keypresses. It also preserves the state of these windows when your connection to screen drops, allowing you to reconnect later.

It's a bit beyond the scope of this page, if you want to learn more check out the various web resources or it, such as http://www.rackaid.com/resources/linux-screen-tutorial-and-how-to/.

Resgrp:comp-photo-version control

2012-02-07T15:40:59Z

Scliffor:

== Introduction ==

This document will teach you the basics of using mercurial, a distributed version control system (DVCS). It is particularly directed towards using mercurial with the development version of Gaussian, so the examples will use Gaussian. For more information on mercurial you can use the built-in help system <code>hg help</code>, look at the official website: http://mercurial.selenic.com/wiki/Mercurial, or check out the official O'Reilly book, the text of which is freely available at: http://hgbook.red-bean.com/.

A version control system can sometimes seem an onerous imposition unless the user understands how it is going to help them in their work, so I'll try to briefly explain. Version control is about tracking changes to data. If one has a collection of files, say a distribution of Gaussian, then one is interested in changes that have been made to those files, whether from a new distribution from Gaussian or our own modifications. We are particularly interested in cases where overlapping changes have been made. These might be updates and bug fixes from Gaussian conflicting with our own modifications to a particular link, or it might be different researchers in a group working on the same piece of code independently. A version control system will not only detect such cases but provide automated and safe methods for merging conflicting changes.

Central to the idea of a VCS is the concept of a revision or changeset. This is like a snapshot of some set of files and directories at some particular instant in their history. A repository is where a VCS stores revisions. So, using Gaussian as an example, some revisions in a particular repository might be the Gaussian source tree corresponding to Gaussian development versions H01, H08, H10, H11, etc. The data that are actually kept inside the repository are the differences, or deltas, between the revisions. This saves space compared to storing all the revisions.

== Setting up your environment (.hgrc)==
There are some one-off things you must do. Firstly, edit your login scripts (<code>.bash_profile</code> or similar) to include the command:
<code>
module load mercurial
</code>
(If you're not on an IC HPC machine mercurial is available anywhere that runs Python. It is generally available in distribution package systems, if not you can get it from http://mercurial.selenic.com/wiki/Mercurial)

Next, create a file called <code>.hgrc</code> in your home directory and insert the following lines:
<code>
[ui]
username = Your Name <your@email.address>

[extensions]
graphlog=
</code>
(so for example, my <code>.hgrc</code> contains this line: <code>username = Simon Clifford <simon.j.clifford@gmail.com></code>).
Mercurial uses this username field to record who has changed what. (If you're not on an IC HPC machine your version of mercurial may not
have the graphlog extension. If it doesn't, don't worry, it's merely a tool for visualising changesets).

== Checking out a repository (hg clone, hg log) ==

Now let's get started and check out a repository. Once you have loaded the mercurial module you will be able to type:
<code>
tmp$ hg clone /home/gaussian-devel/example-gaussian-repo testrepo
tmp$
</code>
This will create a copy of the repository at <code>/home/gaussian-devel/example-gaussian-repo</code> and place it in a directory inside the current directory called <code>testrepo</code>. You can give a full path as the second argument, or, you can leave it off altogether in which case mercurial will clone the repository into a directory named <code>example-gaussian-repo</code>. A note here for Imperial HPC users. Copying a repository involves quite a bit of filesystem activity. I have found that on cx1 this can be quite slow on the <code>/home</code> filesystems. You may wish to try these examples in the <code>/tmp</code> filesystem which is a lot faster. Be aware, however, that this raises two potentially serious problems. The first is that <code>/tmp</code> is globally readable so before you clone you must make sure that your umask is set to <code>0077</code>. The second, of course, is that <code>/tmp</code> is periodically wiped. If you plan on working in <code>/tmp</code> there are simple ways of transferring information between repositories (<code>hg push</code> and <code>hg pull</code>) that I will cover later.

Now change into the <code>testrepo</code> directory. Inside you will see three scripts and a <code>gdv</code> directory; also some hidden files and a hidden directory. Inside the <code>gdv</code> directory you will see the Gaussian source in the [[Resgrp:comp-photo-new gdv layout|new layout]]. The hidden <code>.hg</code> directory inside the <code>testrepo</code> directory is the repository proper; you will almost never need to do anything in here. The rest of the files and directories are called the working directory; this is where you will do your work.

The first thing you might want to do is check the history of this repository to see past revisions that have been checked into it. You do this with the <code>hg log</code> command.
<code>
testrepo$ hg log | head -20
changeset: 23:1de40efd1c4b
tag: tip
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:43:13 2012 +0000
summary: Added tag h13 for changeset 6c81eb8dcbab

changeset: 22:6c81eb8dcbab
tag: h13
parent: 21:5b6ac3665a29
parent: 11:a0c273aeeb2b
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:43:08 2012 +0000
summary: Gaussian devel version H13 with our makefiles

changeset: 21:5b6ac3665a29
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:41:40 2012 +0000
summary: Added tag h12p for changeset 0ef14d7dff56
...
changeset: 1:515a93bfc3b5
branch: raw
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Sun Jan 15 11:39:10 2012 +0000
summary: Added tag h01-raw for changeset 073d6aa63ea7

changeset: 0:073d6aa63ea7
branch: raw
tag: h01-raw
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Sun Jan 15 11:39:09 2012 +0000
summary: Output from old-to-new.sh on version H01
</code>
You will see that there are various fields that may appear for each changeset. The changeset field shows two numbers separated by a colon. The second, longer, hexadecimal number is the unique identifier. A particular hex identifier will always refer to the same changeset, even in different copies of the repository. The first number is a local identifier. These local IDs refer to particular changesets in one copy of the repository; if these changesets are present in another repository they may have a different local identifier. You can use either identifier as an argument to <code>hg</code> commands. The user and date fields show who committed the change and when. The summary field shows the first line of the log entry for that revision. This means it is important to try to make the first line of your log entries a useful summary of what you have done!

To see the entry for a particular revision use the <code>-r</code> flag with a revision number, either short or long. To see more information, including the full log entry and a complete list of all files involved in the change, use the <code>-v</code> flag, e.g.:
<code>
testrepo$ hg log -r 2 -v | less
changeset: 2:9f69ed4616ad
branch: raw
tag: h08-raw
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Sun Jan 15 11:54:50 2012 +0000
files: gdv/archlib/arcvib.F gdv/archlib/brcpyw.F gdv/archlib/letset.F gdv/arctmp.F gdv/bsd/GauDiff_Compare.pm gdv/bsd/G
....
at.F gdv/utilnz/xpndnb.F gdv/utilnz/zerock.F gdv/utilnz/zerod.F gdv/wrappers/ggeev.F gdv/wrappers/lappar.F gdv/wrappers/xgetrf.F gdv/wrappers/xgetrs.F gdv/wrappers/ygeru.F gdv/wrappers/ytrsv.F
description:
Import of gdv H08. See below for details.

Result of these commands on the H01 raw rev:
rm -fr gdv
cp -pr h08/new-style-gdv .
hg add gdv/bsd/*.a
hg addremove -s 50

h08/new-style is from old-to-new.sh on freshly untarred h08.tgz
</code>
As the log entry explains, this particular revision is the change from the H01 version of Gaussian to the H08 version, so there are a lot of changed files. The description entry is intended to provide some human readable explanation of the changes. When you start committing data to the repository you should aim to at least make your log entries understandable.

As with all the commands that I will mention you can get more information by typing <code>hg help <command></code>.

== Committing changes (hg summary, hg status, hg diff) ==

Now let's make some alterations to this repository. This is quite safe; we can't break the original repository because our copy is an entirely separate clone. In fact, this is a very common working practice with mercurial. As long as the filesystem isn't slow cloning a repository can be quite quick. Indeed, cloning a repository into a destination on the same filesystem as the source makes use of hard linking which is very fast and saves space. Therefore, it is very common to clone a repository, make some alterations to it, and then decide whether to push them back to the original repository or just delete the new clone.

First, we want to know where it is we are starting from. To get a quick summary you use the <code>hg summary</code> command. This should show something like this:
<code>
testrepo$ hg summary
parent: 23:1de40efd1c4b tip
Added tag h13 for changeset 6c81eb8dcbab
branch: default
commit: (clean)
update: (current)
</code>
This shows, on the parent line, the revision that has been checked out into the working directory. Below that is the summary line from the log entry for that revision. The commit line shows if any files have been modified since the check out. The update line shows if there are any applicable newer revisions in the repository. In this example the output shows us that we checked out the revision with the local identifier 23, and unique identifier of 1de40efd1c4b, that we have modified nothing and that there are no appropriate newer revisions. This checkout occurred automatically as a part of the <code>hg clone</code> command. You can clone without checking anything out into the working directory (say to make a backup copy of the repository) by passing the <code>-U</code> flag, or you can check out a particular revision with the <code>-u REV</code> flag. See <code>hg help clone</code> for how mercurial chooses what to check out by default.

Let's say we're adding a feature to link 510. If you have something in mind go ahead and do it. For this example I have just added a few lines to the file <code>a1tdep.F</code>, and will assume that your current working directory is <code>gdv/l510</code>.

As you work you may be curious to know what it is you have changed. The command to show this is <code>hg status</code>:
<code>
testrepo$ hg status
M gdv/l510/a1tdep.F
</code>
The output shows a list of files that have been altered in some way. Here the capital M shows that the file <code>a1tdep.F</code> has been modified. By default <code>hg status</code> does not report on unchanged files. If we wish to see how <code>a1tdep.F</code> has been modified we can use the <code>hg diff</code> command:
<code>
testrepo$ hg diff a1tdep.F
diff -r 1de40efd1c4b gdv/l510/a1tdep.F
--- a/gdv/l510/a1tdep.F Wed Jan 18 14:43:13 2012 +0000
+++ b/gdv/l510/a1tdep.F Wed Jan 25 11:07:55 2012 +0000
@@ -26,6 +26,9 @@
C
C **OPTIONS FOR TDEP CODE
C
+c
+c Add super new feature
+c
iTDep=813
iTrans=822
jTDep=iTDep
</code>
This shows the output in a unified different format, refer to the man page for diff for more information.

== Adding, copying, renaming, and deleting file (hg add, hg forget, hg mv, hg rm, hg cp) ==

If you have created a new file then you must let mercurial know about its existence with the <code>hg add</code> command:
<code>
testrepo$ echo "a new subroutine" > foo.F
testrepo$ hg status
M gdv/l510/a1tdep.F
? gdv/l510/foo.F
testrepo$ hg add foo.F
testrepo$ hg status
M gdv/l510/a1tdep.F
A gdv/l510/foo.F
</code>
Here, I create a new file called <code>foo.F</code>. <code>hg status</code> shows the file with a ?, to indicate that it is unknown to the repository. I then run <code>hg add foo.F</code>, after which <code>hg status</code> shows the file with an A. This indicates that the file is scheduled to be added at the next commit. If you change your mind about the file before the next commit you can use <code>hg forget</code> to undo the add.

If you are renaming a file, including the situation where you move the file from one directory to another (e.g. from l510 to utilam) then you may use the <code>hg mv</code> command. This will actually perform the move just like the normal <code>mv</code> command. If you've already moved the file you can give <code>hg mv</code> the <code>-A</code> flag to record the rename after the fact. Similarly, the <code>hg rm</code> command removes files from the working directory and records this fact in the repository, and <code>hg cp</code> copies a file.

It is important to realise that these commands act on the working directory immediately but only affect the repository when they are committed. Also, they do not affect the repository's previous history, so removing a file and then committing that change does not affect that file's existence in previous revisions in the repository. However, <code>hg mv</code> and <code>hg cp</code> create a relationship between the source and destination files. This becomes useful when merging in changes from somewhere else. So for example, above I have made changes to <code>l510/a1tdep.F</code>. Let's say that someone else has decided to make this into a utility routine and has done the command <code>hg mv l510/a1tdep.F utilam/a1tdep.F</code>. If I choose to merge my changes with this other person's, mercurial will correctly apply the changes '''and''' move the file. If instead of <code>hg mv</code> they had used <code>hg cp</code> then my changes would be applied to both files. This means that you should only use <code>hg cp</code> when it is appropriate that changes that are recorded before the copy are applied to both files. Note that this does not mean that changes to the source file are forever applied to the destination file; this will only occur when merging a revision that does not contain a copy with a revision that does. In general, you can expect mercurial to do the right thing.

== Committing the changes (hg commit, hg tip) ==

The <code>hg commit</code> (like many <code>hg</code> commands, <code>hg</code> commit has an alias, in this case <code>hg ci</code>. I tend to use <code>hg ci</code>) command creates a new revision and records the differences between the entire working directory and the parent revision (as shown on the parent line of <code>hg summary). This will be the same as the output from <code>hg status</code>. When you run the command mercurial will open a text editor. If you wish to specify which editor is used add an <code>editor=vim</code> (for example) line to the <code>[ui]</code> part of your <code>~/.hgrc</code> file. The editor will start with a couple of empty lines and then some lines beginning with "<code>HG:</code>". These are there to give you helpful reminders of what you've changed while you write your log message and will be ignored by mercurial when you commit. If you close the editor without writing anything, or of the editor quits with an error, mercurial will abort the commit. So for our current example:
<code>
testrepo$ hg ci
... [opens vim]
HG: Enter commit message. Lines beginning with 'HG:' are removed.
HG: Leave message empty to abort commit.
HG: --
HG: user: Simon Clifford <simon.j.clifford@gmail.com>
HG: branch 'default'
HG: added gdv/l510/foo.F
HG: changed gdv/l510/a1tdep.F
~
~
</code>
For short messages you can skip the editor step by passing the <code>-m "Log message here"</code> flag.

Understanding what you are doing here is essential to knowing what to write in the log and, importantly, when to commit. There are two schools of thought. When you commit you are creating a new revision in your local repository. Since it is your repository, and as I have previously mentioned, it is very simple to create new repositories, you should commit whenever you feel like it and feel free to write cryptic log messages that only you will understand. On the other hand, you are engaged in a collaborative exercise with other people: creating what will be a single version of Gaussian that contains yours and others' modifications and Gaussian's updates. Therefore, you should only commit when your code satisfies pre-agreed group requirements, and your log message should be detailed and comprehensible to anybody who reads it, even 50 years in the future. The correct approach, of course, is to do both.

Mercurial is a distributed version control system which means that each repository is the responsibility of its owner who can feel free to commit as frequently (or infrequently) as needed. Sometimes adding a feature or removing a bug might take weeks, and you might feel that there is no point taking snapshots of the code until it works. Other modifications might proceed incrementally with naturally defined stopping points between the start and the end. Committing at these points makes perfect sense: not only does it leave a history of the modification, it provides checkpoints, versions of the code that you can retreat to without having to start again. The log messages may tersely explain what has happened since the last revision (and note there is no point listing modified, added, etc, files as this information is stored in the commit anyway) or they may contain detailed essays on how a bug arose and the steps you have taken to fix it. Information in the logs is searchable and should be thought of as both an aide memoir for yourself and a journal entry for others.

When the time comes to merge your changes into other people's repositories, you might start thinking about the quality of your commit. Does the code compile? Does it run the test suite? Can it run any test? Have you committed a test that checks the code you have added or altered? The group may decide that revisions that are being shared must answer yes to some or all of these questions. You may decide that some or all of your revisions must pass these quality checks. Or, you may be satisfied with noting in the log entry what this particular revision can or can't do (such as compile or run).

It should be said though, that if you ever think "perhaps I should check in at this point", then go for it. Revisions live in the history of your repository forever (although see <code>hg rollback</code>), but when you transfer them to other people's repositories multiple revisions can be collapsed into one, or fixed in other ways.

Returning to our example I enter a simple message. Since <code>hg log</code> only shows the first line of any log entry by default it is a good idea to make this line a summary of the rest of the message. I close the editor and the commit is done.

We can see the results in the repository with an <code>hg log</code> command (the <code>-l 2</code> flag shows the last two revisions):
<code>
testrepo$ hg log -l 2
changeset: 24:13ecff0136c2
tag: tip
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 25 17:13:00 2012 +0000
summary: A test message.

changeset: 23:1de40efd1c4b
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:43:13 2012 +0000
summary: Added tag h13 for changeset 6c81eb8dcbab
</code>
Alternatively we can use the <code>hg tip</code> command to show the most recently added repository, whether by ourselves, or by pulling from another repository (see later).

== Fixing mistakes (hg revert, hg rollback) ==

You can return one or more files, or the entire repository, to the state they were in when you last checked them out with in the <code>hg revert</code> command. Let's make an ill-advised alteration to our now committed change to <code>a1tdep.F</code>:
<code>
testrepo$ sed -i 's/super/super duper/' a1tdep.F
testrepo$ hg status
M gdv/l510/a1tdep.F
testrepo$ hg diff a1tdep.F
diff -r 13ecff0136c2 gdv/l510/a1tdep.F
--- a/gdv/l510/a1tdep.F Wed Jan 25 17:13:00 2012 +0000
+++ b/gdv/l510/a1tdep.F Wed Jan 25 17:23:42 2012 +0000
@@ -27,7 +27,7 @@
C **OPTIONS FOR TDEP CODE
C
c
-c Add super new feature
+c Add super duper new feature
c
iTDep=813
iTrans=822
testrepo$ hg revert a1tdep.F
testrepo$ hg status
? gdv/l510/a1tdep.F.orig
testrepo$ hg diff a1tdep.F
testrepo$ rm a1tdep.*
testrepo$ hg status
! gdv/l510/a1tdep.F
testrepo$
</code>
Here I use revert to undo the modifications to a file. Notice that <code>hg revert</code> leaves a copy of the modified file in <code>a1tdep.F.orig</code>. This shows up in <code>hg status</code> as an unknown file ("?"). Also notice that while trying to delete the .orig file I have accidentally deleted <code>a1tdep.F</code>, which now shows up in <code>hg status</code> as a missing file ("!"). I can revert this mistake too:
<code>
testrepo$ hg revert a1tdep.F
testrepo$ hg status
testrepo$ ls a1tdep.F
a1tdep.F
testrepo$
</code>
<code>hg revert</code> can also be used to cancel scheduled adds, removes, copies, and renames.

If you have just committed a revision and then change your mind you may be able to undo the effect with <code>hg rollback</code>. Doing this for our example gives us:
<code>
testrepo$ hg rollback
repository tip rolled back to revision 23 (undo commit)
working directory now based on revision 23
testrepo$ hg status
M gdv/l510/a1tdep.F
A gdv/l510/foo.F
testrepo$
</code>
This removes the revision we just checked in from the repository but does not alter the current working directory. If we choose, we could now use <code>hg revert</code> to restore these to their revision 23 state. There are two important caveats with <code>hg rollback</code>: it can only remove the last checked in revision, and that it is usually pointless to rollback a revision that has already been pushed or pulled (see later) into somebody else's repository, as we can only affect our repository. For the purposes of this example, I will once more check in the changes I have made:
<code>
testrepo$ hg ci -m 'A test message'
testrepo$ hg tip
changeset: 24:13ecff0136c2
tag: tip
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 25 17:38:52 2012 +0000
summary: A test message
testrepo$
</code>

== Collaboration (hg incoming, hg pull, hg outgoing, hg push, hg update, hg parent) ==

The tools described are already quite useful, and form the basis of the very earliest revision control systems. The benefits of being able to detect what you've changed, and how you've changed it, being able to undo the changes selectively, and being able to take a snapshot of your work at any point of your choosing should be apparent. The true power, however, of a modern version control system is how it mediates different streams of changes, particularly those from other users. Note that a DVCS does not solve problems of merging and so on, but provides you, the user, with tools to solve them.

Revisions in the repository form a directed acyclic graph (DAG). Every revision apart from the first has at least one parent revision and may have zero or more child revisions. A revision with no children is called a ''head''. Generally development takes place at a head of the repository: checking in a new revision onto a head creates and consumes a head. However, it is possible to add a child to any revision, possibly creating a new head. This is known as a branch, and may be named or unnamed. A revision with children may still be a branch head if it has no children on its branch.

There are various ways of managing branches. You can have a few repositories with many branches, named and unnamed. Or you can have many repositories with largely unbranched graphs; it's up to you. Since mercurial uses heads as the default targets for some of its operations the latter approach is probably best for beginners as it makes operations within each repository simpler.

This is probably best illustrated with an example. I'll make a copy of the original repository, make a different change, and then merge the changes. Change directory out of the <code>testrepo</code> repository and type:
<code>
tmp$ hg clone /home/gaussiandevel/example-gaussian-repo testmerge
updating to branch default
13427 files updated, 0 files merged, 0 files removed, 0 files unresolved
tmp$
</code>

Now we go into the new repository and make some modifications. This time I alter the files <code>a1tdep.F</code> and <code>a2tdep.F</code>, and add a file <code>bar.F</code> in the <code>l510</code> directory:
<code>
…[alterations to a1tdep.F, a2tdep.F, and bar.F]...
testmerge$ hg add gdv/l510/bar.F
testmerge$ hg diff
diff -r 1de40efd1c4b gdv/l510/a1tdep.F
--- a/gdv/l510/a1tdep.F Wed Jan 18 14:43:13 2012 +0000
+++ b/gdv/l510/a1tdep.F Thu Jan 26 12:15:13 2012 +0000
@@ -26,6 +26,9 @@
C
C **OPTIONS FOR TDEP CODE
C
+c
+c Some new feature
+c
iTDep=813
iTrans=822
jTDep=iTDep
diff -r 1de40efd1c4b gdv/l510/a2tdep.F
--- a/gdv/l510/a2tdep.F Wed Jan 18 14:43:13 2012 +0000
+++ b/gdv/l510/a2tdep.F Thu Jan 26 12:15:13 2012 +0000
@@ -16,6 +16,9 @@
C
C **DETERMINE IF WE RUN THE TIMEDEP CODE
C
+c
+c Super duper feature here too
+c
LenTst=0
CALL FILEIO(11,jTDep,LenTst,0,0)
IF (LenTst.EQ.0.OR.iopv.NE.23)RETURN
diff -r 1de40efd1c4b gdv/l510/bar.F
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/gdv/l510/bar.F Thu Jan 26 12:15:13 2012 +0000
@@ -0,0 +1,1 @@
+Stuff in here
testmerge$
testmerge$ hg ci -m 'Added second new feature'
testmerge$ hg log -l 2
changeset: 24:457db23c0b1a
tag: tip
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Thu Jan 26 12:15:54 2012 +0000
summary: Added second new feature

changeset: 23:1de40efd1c4b
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:43:13 2012 +0000
summary: Added tag h13 for changeset 6c81eb8dcbab
testmerge$
</code>
The situation now is that we have two repositories where there are revisions whose parent is revision 23 (23:1de40efd1c4b, in fact). This sort of scenario might arise because we have been working on two different features in two different repositories, or it may be that two users have been working separately. At some point you may wish to merge the work. It's up to you when and how you do this: you may wish to merge in bug fixes quite frequently, and incorporate brand-new features much less frequently. You can of course clone another repository in which to do the merge so that if it isn't satisfactory you can just delete the repository.

First we must pull the revisions from one repository to another. We use the <code>hg incoming</code> and <code>pull</code> commands to do this:
<code>
testmerge$ hg incoming ../testrepo
comparing with ../testrepo
searching for changes
changeset: 24:13ecff0136c2
tag: tip
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 25 17:39:50 2012 +0000
summary: A test message
testmerge$ hg pull ../testrepo
pulling from ../testrepo
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 2 changes to 2 files (+1 heads)
(run 'hg heads' to see heads, 'hg merge' to merge)
testmerge$ hg log -l 3
changeset: 25:13ecff0136c2
tag: tip
parent: 23:1de40efd1c4b
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 25 17:39:50 2012 +0000
summary: A test message

changeset: 24:457db23c0b1a
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Thu Jan 26 12:15:54 2012 +0000
summary: Added second new feature

changeset: 23:1de40efd1c4b
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:43:13 2012 +0000
summary: Added tag h13 for changeset 6c81eb8dcbab
testmerge$
</code>
The <code>hg incoming</code> command takes another repository as its argument and shows a list of revisions that are not present in the current repository. The <code>hg pull</code> command brings those revisions over into the current repository. The <code>hg log</code> command shows changeset 25 in the testmerge repository corresponds to number 24 in the original <code>testrepo</code> repository. Note the long ID is the same in both cases. The <code>hg pull</code> command reports that it has created a new head. We can see this clearly with the <code>hg glog</code> command (from the graphlog extension). The <code>-r 23:</code> flag tells <code>glog</code> to show revisions 23 and greater, for clarity:
<code>
testmerge$ hg glog -r 23:
o changeset: 25:13ecff0136c2
| tag: tip
| parent: 23:1de40efd1c4b
| user: Simon Clifford <simon.j.clifford@gmail.com>
| date: Wed Jan 25 17:39:50 2012 +0000
| summary: A test message
|
| @ changeset: 24:457db23c0b1a
|/ user: Simon Clifford <simon.j.clifford@gmail.com>
| date: Thu Jan 26 12:15:54 2012 +0000
| summary: Added second new feature
|
o changeset: 23:1de40efd1c4b
| user: Simon Clifford <simon.j.clifford@gmail.com>
| date: Wed Jan 18 14:43:13 2012 +0000
| summary: Added tag h13 for changeset 6c81eb8dcbab
|
testmerge$
</code>
The <code>hg push</code> command can also be used to copy revisions from one repository to another. It works much as you would expect, except that by default it does not permit the creation of new heads in the destination repository. The idea is that you tend to pull into your own repository, where you're expected to know what you're doing, while you might be pushing into someone else's repository, where creating a new head might cause confusion. <code>hg outgoing</code> is the corresponding analogue to the <code>hg incoming</code> command.

In our example we now have a situation where we have two heads. This can arise from revisions being created in different repositories and then pulled together, as shown. Alternatively you can just create new heads in one repository. A disadvantage of the single repository technique is that if you decide branch is going nowhere and elect not to merge or proceed with it, it still remains in your repository. With multiple repositories you can just delete the offending repository.

== Merging two sets of changes (hg merge, hg resolve) ==

Let's say that I decide that these two features will work together and I want to merge the two branches. I can use the <code>hg up</code> (<code>hg update</code>) command to update the current working directory to a particular revision in the repository. If no revision is specified it will update to the current branch head. The revision updated to becomes the parent (as shown by <code>hg summary</code> or <code>hg parent</code>) of the working directory. If there are modified files in the working directory the update may attempt a merge. The revision of the working directory when you start to merge is called the base of the merge. This is important if the two revisions you are merging are on different branches, because the newly merged revision will stay on the base branch. Note that the <code>hg pull</code> command does not update the working directory, so in this case it will still be at revision 24. Let's update to revision 25 and then merge the changes from revision 24:

[Which revision to merge from: in general you merge changes into whatever is your current baseline. So, if you're merging the latest updates from Gaussian (from H10 to H11 for example) into your code, you update to your code and merge in the H11 revision. If you're merging in your changes into the soon to be sent to Gaussian group development version you'd start with that and merge in the revision(s) containing your changes]
<code>
testmerge$ hg up 25
3 files updated, 0 files merged, 1 files removed, 0 files unresolved
testmerge$ hg merge 24
merging gdv/l510/a1tdep.F
warning: conflicts during merge.
merging gdv/l510/a1tdep.F failed!
2 files updated, 0 files merged, 0 files removed, 1 files unresolved
use 'hg resolve' to retry unresolved file merges or 'hg update -C .' to abandon
testmerge$
</code>
The software automatically brings in the changes to foo.F, bar.F, and a2tdep.F (as <code>hg status</code> will show). However, in our contrived example <code>a1tdep.F</code> is subject to changes from both revisions. Mercurial will attempt to merge automatically, and in this case it fails. If you edit <code>a1tdep.F</code> you will see it contains the lines:
<code>
<<<<<<< local
c Add super new feature
=======
c Some new feature
>>>>>>> other
</code>
which were inserted during the failed merge. There is also an <code>a1tdep.F.orig</code> file created as a result of the failed merge. As the output from <code>hg merge</code> says you can either now resolve this failed merge or use <code>hg update -C</code> to undo it (the <code>.orig</code> file will remain).

Since we want to resolve the merge we should edit <code>a1tdep.F</code> so that it works. At a bare minimum we will have to remove the “<code><<<<<<< local</code>”, “<code>=======</code>”, and “<code>>>>>>>> other</code>” marker lines. In a more complex situation we might have to completely rewrite this and other files. This is what I mean when I say that mercurial only provides tools to do merging. Even a completely successful merge, from mercurial's point of view, may be completely bogus code.

There are tools available to make this task easier. For example I use <code>vimdiff</code>, a part of the popular <code>vim</code> package. To enable this I have altered my <code>~/.hgrc</code> file to look like this:
<code>
[ui]
merge = vimdiff
username = Simon Clifford <simon.j.clifford@gmail.com>

[extensions]
graphlog=

[merge-tools]
vimdiff.executable = vim
vimdiff.args = -d $base $local $output $other +close +close
</code>
as per http://mercurial.selenic.com/wiki/MergingWithVim. The page http://mercurial.selenic.com/wiki/MergeProgram contains instructions for other tools, including Emacs, and graphical tools. If you edit your <code>~/.hgrc</code> file in this way then you now can type <code>hg resolve --all</code> to run your chosen tool on all unresolved files. Note that by default if your chosen tool exits without an error then <code>hg resolve</code> will regard that file as resolved. To cause <code>vim</code> to exit with an error use <code>:cq</code>. See <code>hg help resolve</code> for more details.

In the example my resolution is to have the relevant lines in <code>a1tdep.F</code> look like this:
<code>
c
c Add super new feature
c Some new feature
c
</code>

Once you have resolved all the files that mercurial thinks need fixing you should check that the final merged result makes sense. Checking that it compiles, or runs tests appropriate to both of the original revisions, for example. When you are satisfied, you should check in the merged revision:

<code>
testmerge$ hg ci -m 'Merged super and duper features'

testmerge$ hg tip

changeset: 26:857f81e59d66
tag: tip
parent: 25:13ecff0136c2
parent: 24:457db23c0b1a
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Thu Jan 26 13:10:52 2012 +0000
summary: Merged super and new features

testmerge$

</code>

You can see that the new revision has ''two'' parent revisions; a merge always consumes a head. This is clear from the output of:

<code>
testmerge$ hg glog -l 4
o changeset: 26:857f81e59d66
|\ tag: tip
| | parent: 25:13ecff0136c2
| | parent: 24:457db23c0b1a
| | user: Simon Clifford <simon.j.clifford@gmail.com>
| | date: Thu Jan 26 13:10:52 2012 +0000
| | summary: Merged super and new features
| |
| o changeset: 25:13ecff0136c2
| | parent: 23:1de40efd1c4b
| | user: Simon Clifford <simon.j.clifford@gmail.com>
| | date: Wed Jan 25 17:39:50 2012 +0000
| | summary: A test message
| |
o | changeset: 24:457db23c0b1a
|/ user: Simon Clifford <simon.j.clifford@gmail.com>
| date: Thu Jan 26 12:15:54 2012 +0000
| summary: Added second new feature
|
o changeset: 23:1de40efd1c4b
| user: Simon Clifford <simon.j.clifford@gmail.com>
| date: Wed Jan 18 14:43:13 2012 +0000
| summary: Added tag h13 for changeset 6c81eb8dcbab
|
testmerge$
</code>

Another example of merging here? Perhaps Lee's code merging in changes on official version.

== Tags and named branches (hg tag, hg tags, hg branch, hg branches) ==

It is often useful to give a name to a particular revision. This might be a changeset that corresponds to a given version of the code, or it might simply be a revision that marks a particular milestone. In the <code>example-gaussian-repo</code> you have probably noticed there are revisions in the history with descriptions like "Added tag h13 for changeset 6c81eb8dcbab". These are revisions corresponding to particular versions of the Gaussian development version, such as H01, H10, H13, etc. You can see a list of the tags in a repository by doing:
<code>
testmerge$ hg tags
tip 26:857f81e59d66
h13 22:6c81eb8dcbab
h12p 20:0ef14d7dff56
h11 18:7daba638a830
h10 16:e1d0af4e84d9
h08 14:656073c38db9
h01 12:737d1720e79a
h13-raw 10:b039f5c274e2
h12p-raw 8:522a8fe79d22
h11-raw 6:a28b789a9555
h10-raw 4:356081dab79d
h08-raw 2:9f69ed4616ad
h01-raw 0:073d6aa63ea7
testmerge$
</code>
(the <code>tip</code> tag is automatically assigned to the most recently checked in revision). You may use a tag name anywhere where you might use a revision number, so for example, to check out the changeset corresponding to the official Gaussian development release H10, you would type:
<code>
testrepo$ hg up h10
13887 files updated, 0 files merged, 0 files removed, 0 files unresolved
testrepo$
</code>

(this will look different if you have unchecked-in modifications in your working directory). You could also pass h10 to the <code>hg clone -u</code> flag, and so on.
You tag a revision with the <code>hg tag</code> command. Issue it in a working directory and it will tag the revision that is the parent of the working directory. Tags become part of the repository, that is they are shared during <code>hg pull</code> and <code>hg push</code>, so choose wisely. If you'd like to have tags that are only part of the local repository, use the <code>-l</code> flag to <code>hg tag</code>.

Tags are stored in an <code>.hgtags</code> file in the root directory of the repository. Running <code>hg tag</code> alters this file and then commits the change to the repository. Sometimes you can see conflicts in this file during merges. I normally just resolve them by keeping as much information in the merged file as possible.

The <code>hg branches</code> command can be used to show the named branches in the repository:
<code>
testrepo$ hg branches
default 24:13ecff0136c2
raw 11:a0c273aeeb2b (inactive)
testrepo$
</code>
Here I have two named branches, raw and default (the default branch is, er, the default). If you examine the full output of <code>hg glog</code>, which is quite long and I won't reproduce it here, you'll see, starting at the bottom, revisions 0 to 11 are all in the raw branch. I have created and named this branch to contain the official Gaussian development code as processed by the <code>old-to-new.sh</code> script, without any of our makefiles or other alterations. The default branch branches off from revision 1 and starts at revision 12. Its descendants, revisions 12 to 23, are also in the default branch. These were created by starting at revision 1, adding our makefiles and modifications to the build system, renaming this branch to the default branch, and then committing revision 12. I tagged revision 12 as "H01", which created revision 13. I then merged revision 3, fixed the conflicts, and committed revision 14, and so on. A more detailed guide on how to import a new official version of gdv exists in LINKY.

You can use branch names just like tags wherever mercurial expects a revision identifier. If you use a branch name mercurial will attempt to give you the revision that is most appropriate, this will usually be the newest head on that branch.

To create a new branch you type <code>hg branch <branchname></code>. This will take effect when you commit the working directory. As I've previously mentioned a simpler alternative to having branches, named or otherwise, in your repository is to have multiple repositories.

== Ignoring files ==

You may be wondering how the system will work when you start compiling files. Won't all the <code>.o</code>, <code>.a</code> and <code>.exe</code> files start showing up in the output to <code>hg status</code>? The answer is that they will not, because I've told mercurial to ignore such files. This information is stored in the <code>.hgignore</code> file in the root of the repository. It's quite long but it starts like this:
<code>
syntax: glob
gdv.make
*.o
*.lo
*.a
*.exe
*.exel
.*.swp
*~
*.log
*.log.gz
*.chk
gdv/arc
...
</code>
Any file that matches either the shell patterns or the file names in <code>.hgignore</code> is ignored by mercurial. This means it does not show up to <code>hg status</code> or <code>hg commit</code>. <code>hg add</code>, <code>hg rm</code>, and so on will ignore it unless it is explicitly mentioned on the command line. So <code>hg add gdv</code> will add all non-ignored files in the gdv directory, while <code>hg add gdv/*</code> will add all the files in the gdv directory.

The <code>.hgignore</code> file is tracked, so please only commit changes to it if you think they will be useful to everyone.
== Other commands ==

There are quite a few other commands available to mercurial, I will cover just a few here, if you're interested in becoming a power user refer to the documentation I have mentioned.

=== hg addremove ===
You can use this command when you have large numbers of files to add and remove. It also includes the <code>-s</code> flag to attempt to detect when files have been moved or copied. Using this flag does require some subtlety, as it's not helpful to mark files as moved or copied unless they have been. Also note that it is possible to inadvertently include ignored files in a mass <code>hg add</code> or <code>hg addremove</code>; check what's being added before you commit.

=== hg grep and hg locate ===
Similar to the standard UNIX grep and locate commands, but cognisant of the mercurial repository.

=== hg init ===
Creates a new mercurial repository in the current directory

=== hg serve ===
This starts a mini web server serving information about the repository. I would strongly recommend you do not use this while working on the Gaussian development version as it would be all too easy to allow any user on the system to view all of the files in your repository. I only mention it here because some of the other documentation may refer to it.

Resgrp:comp-photo-version control

2012-02-06T22:18:54Z

Scliffor:

== Introduction ==

This document will teach you the basics of using mercurial, a distributed version control system (DVCS). It is particularly directed towards using mercurial with the development version of Gaussian, so the examples will use Gaussian. For more information on mercurial you can use the built-in help system <code>hg help</code>, look at the official website: http://mercurial.selenic.com/wiki/Mercurial, or check out the official O'Reilly book, the text of which is freely available at: http://hgbook.red-bean.com/.

A version control system can sometimes seem an onerous imposition unless the user understands how it is going to help them in their work, so I'll try to briefly explain. Version control is about tracking changes to data. If one has a collection of files, say a distribution of Gaussian, then one is interested in changes that have been made to those files, whether from a new distribution from Gaussian or our own modifications. We are particularly interested in cases where overlapping changes have been made. These might be updates and bug fixes from Gaussian conflicting with our own modifications to a particular link, or it might be different researchers in a group working on the same piece of code independently. A version control system will not only detect such cases but provide automated and safe methods for merging conflicting changes.

Central to the idea of a VCS is the concept of a revision or changeset. This is like a snapshot of some set of files and directories at some particular instant in their history. A repository is where a VCS stores revisions. So, using Gaussian as an example, some revisions in a particular repository might be the Gaussian source tree corresponding to Gaussian development versions H01, H08, H10, H11, etc. The data that are actually kept inside the repository are the differences, or deltas, between the revisions. This saves space compared to storing all the revisions.

== Setting up your environment (.hgrc)==
There are some one-off things you must do. Firstly, edit your login scripts (<code>.bash_profile</code> or similar) to include the command:
<code>
module load mercurial
</code>
(If you're not on an IC HPC machine mercurial is available anywhere that runs Python. It is generally available in distribution package systems, if not you can get it from http://mercurial.selenic.com/wiki/Mercurial)

Next, create a file called <code>.hgrc</code> in your home directory and insert the following lines:
<code>
[ui]
username = Your Name <your@email.address>

[extensions]
graphlog=
</code>
(so for example, my <code>.hgrc</code> contains this line: <code>username = Simon Clifford <simon.j.clifford@gmail.com></code>).
Mercurial uses this username field to record who has changed what. (If you're not on an IC HPC machine your version of mercurial may not
have the graphlog extension. If it doesn't, don't worry, it's merely a tool for visualising changesets).

== Checking out a repository (hg clone, hg log) ==

Now let's get started and check out a repository. Once you have loaded the mercurial module you will be able to type:
<code>
tmp$ hg clone /home/gaussian-devel/example-gaussian-repo testrepo
tmp$
</code>
This will create a copy of the repository at <code>/home/gaussian-devel/example-gaussian-repo</code> and place it in a directory inside the current directory called <code>testrepo</code>. You can give a full path as the second argument, or, you can leave it off altogether in which case mercurial will clone the repository into a directory named <code>example-gaussian-repo</code>. A note here for Imperial HPC users. Copying a repository involves quite a bit of filesystem activity. I have found that on cx1 this can be quite slow on the <code>/home</code> filesystems. You may wish to try these examples in the <code>/tmp</code> filesystem which is a lot faster. Be aware, however, that this raises two potentially serious problems. The first is that <code>/tmp</code> is globally readable so before you clone you must make sure that your umask is set to <code>0077</code>. The second, of course, is that <code>/tmp</code> is periodically wiped. If you plan on working in <code>/tmp</code> there are simple ways of transferring information between repositories (<code>hg push</code> and <code>hg pull</code>) that I will cover later.

Now change into the <code>testrepo</code> directory. Inside you will see three scripts and a <code>gdv</code> directory; also some hidden files and a hidden directory. Inside the <code>gdv</code> directory you will see the Gaussian source in the [[Resgrp:comp-photo-new gdv layout|new layout]]. The hidden <code>.hg</code> directory inside the <code>testrepo</code> directory is the repository proper; you will almost never need to do anything in here. The rest of the files and directories are called the working directory; this is where you will do your work.

The first thing you might want to do is check the history of this repository to see past revisions that have been checked into it. You do this with the <code>hg log</code> command.
<code>
testrepo$ hg log | head -20
changeset: 23:1de40efd1c4b
tag: tip
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:43:13 2012 +0000
summary: Added tag h13 for changeset 6c81eb8dcbab

changeset: 22:6c81eb8dcbab
tag: h13
parent: 21:5b6ac3665a29
parent: 11:a0c273aeeb2b
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:43:08 2012 +0000
summary: Gaussian devel version H13 with our makefiles

changeset: 21:5b6ac3665a29
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:41:40 2012 +0000
summary: Added tag h12p for changeset 0ef14d7dff56
...
changeset: 1:515a93bfc3b5
branch: raw
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Sun Jan 15 11:39:10 2012 +0000
summary: Added tag h01-raw for changeset 073d6aa63ea7

changeset: 0:073d6aa63ea7
branch: raw
tag: h01-raw
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Sun Jan 15 11:39:09 2012 +0000
summary: Output from old-to-new.sh on version H01
</code>
You will see that there are various fields that may appear for each changeset. The changeset field shows two numbers separated by a colon. The second, longer, hexadecimal number is the unique identifier. A particular hex identifier will always refer to the same changeset, even in different copies of the repository. The first number is a local identifier. These local IDs refer to particular changesets in one copy of the repository; if these changesets are present in another repository they may have a different local identifier. You can use either identifier as an argument to <code>hg</code> commands. The user and date fields show who committed the change and when. The summary field shows the first line of the log entry for that revision. This means it is important to try to make the first line of your log entries a useful summary of what you have done!

To see the entry for a particular revision use the <code>-r</code> flag with a revision number, either short or long. To see more information, including the full log entry and a complete list of all files involved in the change, use the <code>-v</code> flag, e.g.:
<code>
testrepo$ hg log -r 2 -v | less
changeset: 2:9f69ed4616ad
branch: raw
tag: h08-raw
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Sun Jan 15 11:54:50 2012 +0000
files: gdv/archlib/arcvib.F gdv/archlib/brcpyw.F gdv/archlib/letset.F gdv/arctmp.F gdv/bsd/GauDiff_Compare.pm gdv/bsd/G
....
at.F gdv/utilnz/xpndnb.F gdv/utilnz/zerock.F gdv/utilnz/zerod.F gdv/wrappers/ggeev.F gdv/wrappers/lappar.F gdv/wrappers/xgetrf.F gdv/wrappers/xgetrs.F gdv/wrappers/ygeru.F gdv/wrappers/ytrsv.F
description:
Import of gdv H08. See below for details.

Result of these commands on the H01 raw rev:
rm -fr gdv
cp -pr h08/new-style-gdv .
hg add gdv/bsd/*.a
hg addremove -s 50

h08/new-style is from old-to-new.sh on freshly untarred h08.tgz
</code>
As the log entry explains, this particular revision is the change from the H01 version of Gaussian to the H08 version, so there are a lot of changed files. The description entry is intended to provide some human readable explanation of the changes. When you start committing data to the repository you should aim to at least make your log entries understandable.

As with all the commands that I will mention you can get more information by typing <code>hg help <command></code>.

== Committing changes (hg summary, hg status, hg diff) ==

Now let's make some alterations to this repository. This is quite safe; we can't break the original repository because our copy is an entirely separate clone. In fact, this is a very common working practice with mercurial. As long as the filesystem isn't slow cloning a repository can be quite quick. Indeed, cloning a repository into a destination on the same filesystem as the source makes use of hard linking which is very fast and saves space. Therefore, it is very common to clone a repository, make some alterations to it, and then decide whether to push them back to the original repository or just delete the new clone.

First, we want to know where it is we are starting from. To get a quick summary you use the <code>hg summary</code> command. This should show something like this:
<code>
testrepo$ hg summary
parent: 23:1de40efd1c4b tip
Added tag h13 for changeset 6c81eb8dcbab
branch: default
commit: (clean)
update: (current)
</code>
This shows, on the parent line, the revision that has been checked out into the working directory. Below that is the summary line from the log entry for that revision. The commit line shows if any files have been modified since the check out. The update line shows if there are any applicable newer revisions in the repository. In this example the output shows us that we checked out the revision with the local identifier 23, and unique identifier of 1de40efd1c4b, that we have modified nothing and that there are no appropriate newer revisions. This checkout occurred automatically as a part of the <code>hg clone</code> command. You can clone without checking anything out into the working directory (say to make a backup copy of the repository) by passing the <code>-U</code> flag, or you can check out a particular revision with the <code>-u REV</code> flag. See <code>hg help clone</code> for how mercurial chooses what to check out by default.

Let's say we're adding a feature to link 510. If you have something in mind go ahead and do it. For this example I have just added a few lines to the file <code>a1tdep.F</code>, and will assume that your current working directory is <code>gdv/l510</code>.

As you work you may be curious to know what it is you have changed. The command to show this is <code>hg status</code>:
<code>
testrepo$ hg status
M gdv/l510/a1tdep.F
</code>
The output shows a list of files that have been altered in some way. Here the capital M shows that the file <code>a1tdep.F</code> has been modified. By default <code>hg status</code> does not report on unchanged files. If we wish to see how <code>a1tdep.F</code> has been modified we can use the <code>hg diff</code> command:
<code>
testrepo$ hg diff a1tdep.F
diff -r 1de40efd1c4b gdv/l510/a1tdep.F
--- a/gdv/l510/a1tdep.F Wed Jan 18 14:43:13 2012 +0000
+++ b/gdv/l510/a1tdep.F Wed Jan 25 11:07:55 2012 +0000
@@ -26,6 +26,9 @@
C
C **OPTIONS FOR TDEP CODE
C
+c
+c Add super new feature
+c
iTDep=813
iTrans=822
jTDep=iTDep
</code>
This shows the output in a unified different format, refer to the man page for diff for more information.

== Adding, copying, renaming, and deleting file (hg add, hg forget, hg mv, hg rm, hg cp) ==

If you have created a new file then you must let mercurial know about its existence with the <code>hg add</code> command:
<code>
testrepo$ echo "a new subroutine" > foo.F
testrepo$ hg status
M gdv/l510/a1tdep.F
? gdv/l510/foo.F
testrepo$ hg add foo.F
testrepo$ hg status
M gdv/l510/a1tdep.F
A gdv/l510/foo.F
</code>
Here, I create a new file called <code>foo.F</code>. <code>hg status</code> shows the file with a ?, to indicate that it is unknown to the repository. I then run <code>hg add foo.F</code>, after which <code>hg status</code> shows the file with an A. This indicates that the file is scheduled to be added at the next commit. If you change your mind about the file before the next commit you can use <code>hg forget</code> to undo the add.

If you are renaming a file, including the situation where you move the file from one directory to another (e.g. from l510 to utilam) then you may use the <code>hg mv</code> command. This will actually perform the move just like the normal <code>mv</code> command. If you've already moved the file you can give <code>hg mv</code> the <code>-A</code> flag to record the rename after the fact. Similarly, the <code>hg rm</code> command removes files from the working directory and records this fact in the repository, and <code>hg cp</code> copies a file.

It is important to realise that these commands act on the working directory immediately but only affect the repository when they are committed. Also, they do not affect the repository's previous history, so removing a file and then committing that change does not affect that file's existence in previous revisions in the repository. However, <code>hg mv</code> and <code>hg cp</code> create a relationship between the source and destination files. This becomes useful when merging in changes from somewhere else. So for example, above I have made changes to <code>l510/a1tdep.F</code>. Let's say that someone else has decided to make this into a utility routine and has done the command <code>hg mv l510/a1tdep.F utilam/a1tdep.F</code>. If I choose to merge my changes with this other person's, mercurial will correctly apply the changes '''and''' move the file. If instead of <code>hg mv</code> they had used <code>hg cp</code> then my changes would be applied to both files. This means that you should only use <code>hg cp</code> when it is appropriate that changes that are recorded before the copy are applied to both files. Note that this does not mean that changes to the source file are forever applied to the destination file; this will only occur when merging a revision that does not contain a copy with a revision that does. In general, you can expect mercurial to do the right thing.

== Committing the changes (hg commit, hg tip) ==

The <code>hg commit</code> (like many <code>hg</code> commands, <code>hg</code> commit has an alias, in this case <code>hg ci</code>. I tend to use <code>hg ci</code>) command creates a new revision and records the differences between the entire working directory and the parent revision (as shown on the parent line of <code>hg summary). This will be the same as the output from <code>hg status</code>. When you run the command mercurial will open a text editor. If you wish to specify which editor is used add an <code>editor=vim</code> (for example) line to the <code>[ui]</code> part of your <code>~/.hgrc</code> file. The editor will start with a couple of empty lines and then some lines beginning with "<code>HG:</code>". These are there to give you helpful reminders of what you've changed while you write your log message and will be ignored by mercurial when you commit. If you close the editor without writing anything, or of the editor quits with an error, mercurial will abort the commit. So for our current example:
<code>
testrepo$ hg ci
... [opens vim]
HG: Enter commit message. Lines beginning with 'HG:' are removed.
HG: Leave message empty to abort commit.
HG: --
HG: user: Simon Clifford <simon.j.clifford@gmail.com>
HG: branch 'default'
HG: added gdv/l510/foo.F
HG: changed gdv/l510/a1tdep.F
~
~
</code>
For short messages you can skip the editor step by passing the <code>-m "Log message here"</code> flag.

Understanding what you are doing here is essential to knowing what to write in the log and, importantly, when to commit. There are two schools of thought. When you commit you are creating a new revision in your local repository. Since it is your repository, and as I have previously mentioned, it is very simple to create new repositories, you should commit whenever you feel like it and feel free to write cryptic log messages that only you will understand. On the other hand, you are engaged in a collaborative exercise with other people: creating what will be a single version of Gaussian that contains yours and others' modifications and Gaussian's updates. Therefore, you should only commit when your code satisfies pre-agreed group requirements, and your log message should be detailed and comprehensible to anybody who reads it, even 50 years in the future. The correct approach, of course, is to do both.

Mercurial is a distributed version control system which means that each repository is the responsibility of its owner who can feel free to commit as frequently (or infrequently) as needed. Sometimes adding a feature or removing a bug might take weeks, and you might feel that there is no point taking snapshots of the code until it works. Other modifications might proceed incrementally with naturally defined stopping points between the start and the end. Committing at these points makes perfect sense: not only does it leave a history of the modification, it provides checkpoints, versions of the code that you can retreat to without having to start again. The log messages may tersely explain what has happened since the last revision (and note there is no point listing modified, added, etc, files as this information is stored in the commit anyway) or they may contain detailed essays on how a bug arose and the steps you have taken to fix it. Information in the logs is searchable and should be thought of as both an aide memoir for yourself and a journal entry for others.

When the time comes to merge your changes into other people's repositories, you might start thinking about the quality of your commit. Does the code compile? Does it run the test suite? Can it run any test? Have you committed a test that checks the code you have added or altered? The group may decide that revisions that are being shared must answer yes to some or all of these questions. You may decide that some or all of your revisions must pass these quality checks. Or, you may be satisfied with noting in the log entry what this particular revision can or can't do (such as compile or run).

It should be said though, that if you ever think "perhaps I should check in at this point", then go for it. Revisions live in the history of your repository forever (although see <code>hg rollback</code>), but when you transfer them to other people's repositories multiple revisions can be collapsed into one, or fixed in other ways.

Returning to our example I enter a simple message. Since <code>hg log</code> only shows the first line of any log entry by default it is a good idea to make this line a summary of the rest of the message. I close the editor and the commit is done.

We can see the results in the repository with an <code>hg log</code> command (the <code>-l 2</code> flag shows the last two revisions):
<code>
testrepo$ hg log -l 2
changeset: 24:13ecff0136c2
tag: tip
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 25 17:13:00 2012 +0000
summary: A test message.

changeset: 23:1de40efd1c4b
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:43:13 2012 +0000
summary: Added tag h13 for changeset 6c81eb8dcbab
</code>
Alternatively we can use the <code>hg tip</code> command to show the most recently added repository, whether by ourselves, or by pulling from another repository (see later).

== Fixing mistakes (hg revert, hg rollback) ==

You can return one or more files, or the entire repository, to the state they were in when you last checked them out with in the <code>hg revert</code> command. Let's make an ill-advised alteration to our now committed change to <code>a1tdep.F</code>:
<code>
testrepo$ sed -i 's/super/super duper/' a1tdep.F
testrepo$ hg status
M gdv/l510/a1tdep.F
testrepo$ hg diff a1tdep.F
diff -r 13ecff0136c2 gdv/l510/a1tdep.F
--- a/gdv/l510/a1tdep.F Wed Jan 25 17:13:00 2012 +0000
+++ b/gdv/l510/a1tdep.F Wed Jan 25 17:23:42 2012 +0000
@@ -27,7 +27,7 @@
C **OPTIONS FOR TDEP CODE
C
c
-c Add super new feature
+c Add super duper new feature
c
iTDep=813
iTrans=822
testrepo$ hg revert a1tdep.F
testrepo$ hg status
? gdv/l510/a1tdep.F.orig
testrepo$ hg diff a1tdep.F
testrepo$ rm a1tdep.*
testrepo$ hg status
! gdv/l510/a1tdep.F
testrepo$
</code>
Here I use revert to undo the modifications to a file. Notice that <code>hg revert</code> leaves a copy of the modified file in <code>a1tdep.F.orig</code>. This shows up in <code>hg status</code> as an unknown file ("?"). Also notice that while trying to delete the .orig file I have accidentally deleted <code>a1tdep.F</code>, which now shows up in <code>hg status</code> as a missing file ("!"). I can revert this mistake too:
<code>
testrepo$ hg revert a1tdep.F
testrepo$ hg status
testrepo$ ls a1tdep.F
a1tdep.F
testrepo$
</code>
<code>hg revert</code> can also be used to cancel scheduled adds, removes, copies, and renames.

If you have just committed a revision and then change your mind you may be able to undo the effect with <code>hg rollback</code>. Doing this for our example gives us:
<code>
testrepo$ hg rollback
repository tip rolled back to revision 23 (undo commit)
working directory now based on revision 23
testrepo$ hg status
M gdv/l510/a1tdep.F
A gdv/l510/foo.F
testrepo$
</code>
This removes the revision we just checked in from the repository but does not alter the current working directory. If we choose, we could now use <code>hg revert</code> to restore these to their revision 23 state. There are two important caveats with <code>hg rollback</code>: it can only remove the last checked in revision, and that it is usually pointless to rollback a revision that has already been pushed or pulled (see later) into somebody else's repository, as we can only affect our repository. For the purposes of this example, I will once more check in the changes I have made:
<code>
testrepo$ hg ci -m 'A test message'
testrepo$ hg tip
changeset: 24:13ecff0136c2
tag: tip
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 25 17:38:52 2012 +0000
summary: A test message
testrepo$
</code>

== Collaboration (hg incoming, hg pull, hg outgoing, hg push, hg update, hg parent) ==

The tools described are already quite useful, and form the basis of the very earliest revision control systems. The benefits of being able to detect what you've changed, and how you've changed it, being able to undo the changes selectively, and being able to take a snapshot of your work at any point of your choosing should be apparent. The true power, however, of a modern version control system is how it mediates different streams of changes, particularly those from other users. Note that a DVCS does not solve problems of merging and so on, but provides you, the user, with tools to solve them.

Revisions in the repository form a directed acyclic graph (DAG). Every revision apart from the first has at least one parent revision and may have zero or more child revisions. A revision with no children is called a ''head''. Generally development takes place at a head of the repository: checking in a new revision onto a head creates and consumes a head. However, it is possible to add a child to any revision, possibly creating a new head. This is known as a branch, and may be named or unnamed. A revision with children may still be a branch head if it has no children on its branch.

There are various ways of managing branches. You can have a few repositories with many branches, named and unnamed. Or you can have many repositories with largely unbranched graphs; it's up to you. Since mercurial uses heads as the default targets for some of its operations the latter approach is probably best for beginners as it makes operations within each repository simpler.

This is probably best illustrated with an example. I'll make a copy of the original repository, make a different change, and then merge the changes. Change directory out of the <code>testrepo</code> repository and type:
<code>
tmp$ hg clone /home/gaussiandevel/example-gaussian-repo testmerge
updating to branch default
13427 files updated, 0 files merged, 0 files removed, 0 files unresolved
tmp$
</code>

Now we go into the new repository and make some modifications. This time I alter the files <code>a1tdep.F</code> and <code>a2tdep.F</code>, and add a file <code>bar.F</code> in the <code>l510</code> directory:
<code>
…[alterations to a1tdep.F, a2tdep.F, and bar.F]...
testmerge$ hg add gdv/l510/bar.F
testmerge$ hg diff
diff -r 1de40efd1c4b gdv/l510/a1tdep.F
--- a/gdv/l510/a1tdep.F Wed Jan 18 14:43:13 2012 +0000
+++ b/gdv/l510/a1tdep.F Thu Jan 26 12:15:13 2012 +0000
@@ -26,6 +26,9 @@
C
C **OPTIONS FOR TDEP CODE
C
+c
+c Some new feature
+c
iTDep=813
iTrans=822
jTDep=iTDep
diff -r 1de40efd1c4b gdv/l510/a2tdep.F
--- a/gdv/l510/a2tdep.F Wed Jan 18 14:43:13 2012 +0000
+++ b/gdv/l510/a2tdep.F Thu Jan 26 12:15:13 2012 +0000
@@ -16,6 +16,9 @@
C
C **DETERMINE IF WE RUN THE TIMEDEP CODE
C
+c
+c Super duper feature here too
+c
LenTst=0
CALL FILEIO(11,jTDep,LenTst,0,0)
IF (LenTst.EQ.0.OR.iopv.NE.23)RETURN
diff -r 1de40efd1c4b gdv/l510/bar.F
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/gdv/l510/bar.F Thu Jan 26 12:15:13 2012 +0000
@@ -0,0 +1,1 @@
+Stuff in here
testmerge$
testmerge$ hg ci -m 'Added second new feature'
testmerge$ hg log -l 2
changeset: 24:457db23c0b1a
tag: tip
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Thu Jan 26 12:15:54 2012 +0000
summary: Added second new feature

changeset: 23:1de40efd1c4b
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:43:13 2012 +0000
summary: Added tag h13 for changeset 6c81eb8dcbab
testmerge$
</code>
The situation now is that we have two repositories where there are revisions whose parent is revision 23 (23:1de40efd1c4b, in fact). This sort of scenario might arise because we have been working on two different features in two different repositories, or it may be that two users have been working separately. At some point you may wish to merge the work. It's up to you when and how you do this: you may wish to merge in bug fixes quite frequently, and incorporate brand-new features much less frequently. You can of course clone another repository in which to do the merge so that if it isn't satisfactory you can just delete the repository.

First we must pull the revisions from one repository to another. We use the <code>hg incoming</code> and <code>pull</code> commands to do this:
<code>
testmerge$ hg incoming ../testrepo
comparing with ../testrepo
searching for changes
changeset: 24:13ecff0136c2
tag: tip
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 25 17:39:50 2012 +0000
summary: A test message
testmerge$ hg pull ../testrepo
pulling from ../testrepo
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 2 changes to 2 files (+1 heads)
(run 'hg heads' to see heads, 'hg merge' to merge)
testmerge$ hg log -l 3
changeset: 25:13ecff0136c2
tag: tip
parent: 23:1de40efd1c4b
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 25 17:39:50 2012 +0000
summary: A test message

changeset: 24:457db23c0b1a
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Thu Jan 26 12:15:54 2012 +0000
summary: Added second new feature

changeset: 23:1de40efd1c4b
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:43:13 2012 +0000
summary: Added tag h13 for changeset 6c81eb8dcbab
testmerge$
</code>
The <code>hg incoming</code> command takes another repository as its argument and shows a list of revisions that are not present in the current repository. The <code>hg pull</code> command brings those revisions over into the current repository. The <code>hg log</code> command shows changeset 25 in the testmerge repository corresponds to number 24 in the original <code>testrepo</code> repository. Note the long ID is the same in both cases. The <code>hg pull</code> command reports that it has created a new head. We can see this clearly with the <code>hg glog</code> command (from the graphlog extension). The <code>-r 23:</code> flag tells <code>glog</code> to show revisions 23 and greater, for clarity:
<code>
testmerge$ hg glog -r 23:
o changeset: 25:13ecff0136c2
| tag: tip
| parent: 23:1de40efd1c4b
| user: Simon Clifford <simon.j.clifford@gmail.com>
| date: Wed Jan 25 17:39:50 2012 +0000
| summary: A test message
|
| @ changeset: 24:457db23c0b1a
|/ user: Simon Clifford <simon.j.clifford@gmail.com>
| date: Thu Jan 26 12:15:54 2012 +0000
| summary: Added second new feature
|
o changeset: 23:1de40efd1c4b
| user: Simon Clifford <simon.j.clifford@gmail.com>
| date: Wed Jan 18 14:43:13 2012 +0000
| summary: Added tag h13 for changeset 6c81eb8dcbab
|
testmerge$
</code>
The <code>hg push</code> command can also be used to copy revisions from one repository to another. It works much as you would expect, except that by default it does not permit the creation of new heads in the destination repository. The idea is that you tend to pull into your own repository, where you're expected to know what you're doing, while you might be pushing into someone else's repository, where creating a new head might cause confusion. <code>hg outgoing</code> is the corresponding analogue to the <code>hg incoming</code> command.

In our example we now have a situation where we have two heads. This can arise from revisions being created in different repositories and then pulled together, as shown. Alternatively you can just create new heads in one repository. A disadvantage of the single repository technique is that if you decide branch is going nowhere and elect not to merge or proceed with it, it still remains in your repository. With multiple repositories you can just delete the offending repository.

== Merging two sets of changes (hg merge, hg resolve) ==

Let's say that I decide that these two features will work together and I want to merge the two branches. I can use the <code>hg up</code> (<code>hg update</code>) command to update the current working directory to a particular revision in the repository. If no revision is specified it will update to the current branch head. The revision updated to becomes the parent (as shown by <code>hg summary</code> or <code>hg parent</code>) of the working directory. If there are modified files in the working directory the update may attempt a merge. The revision of the working directory when you start to merge is called the base of the merge. This is important if the two revisions you are merging are on different branches, because the newly merged revision will stay on the base branch. Note that the <code>hg pull</code> command does not update the working directory, so in this case it will still be at revision 24. Let's update to revision 25 and then merge the changes from revision 24:

[Which revision to merge from: in general you merge changes into whatever is your current baseline. So, if you're merging the latest updates from Gaussian (from H10 to H11 for example) into your code, you update to your code and merge in the H11 revision. If you're merging in your changes into the soon to be sent to Gaussian group development version you'd start with that and merge in the revision(s) containing your changes]
<code>
testmerge$ hg up 25
3 files updated, 0 files merged, 1 files removed, 0 files unresolved
testmerge$ hg merge 24
merging gdv/l510/a1tdep.F
warning: conflicts during merge.
merging gdv/l510/a1tdep.F failed!
2 files updated, 0 files merged, 0 files removed, 1 files unresolved
use 'hg resolve' to retry unresolved file merges or 'hg update -C .' to abandon
testmerge$
</code>
The software automatically brings in the changes to foo.F, bar.F, and a2tdep.F (as <code>hg status</code> will show). However, in our contrived example <code>a1tdep.F</code> is subject to changes from both revisions. Mercurial will attempt to merge automatically, and in this case it fails. If you edit <code>a1tdep.F</code> you will see it contains the lines:
<code>
<<<<<<< local
c Add super new feature
=======
c Some new feature
>>>>>>> other
</code>
which were inserted during the failed merge. There is also an <code>a1tdep.F.orig</code> file created as a result of the failed merge. As the output from <code>hg merge</code> says you can either now resolve this failed merge or use <code>hg update -C</code> to undo it (the <code>.orig</code> file will remain).

Since we want to resolve the merge we should edit <code>a1tdep.F</code> so that it works. At a bare minimum we will have to remove the “<code><<<<<<< local</code>”, “<code>=======</code>”, and “<code>>>>>>>> other</code>” marker lines. In a more complex situation we might have to completely rewrite this and other files. This is what I mean when I say that mercurial only provides tools to do merging. Even a completely successful merge, from mercurial's point of view, may be completely bogus code.

There are tools available to make this task easier. For example I use <code>vimdiff</code>, a part of the popular <code>vim</code> package. To enable this I have altered my <code>~/.hgrc</code> file to look like this:
<code>
[ui]
merge = vimdiff
username = Simon Clifford <simon.j.clifford@gmail.com>

[extensions]
graphlog=

[merge-tools]
vimdiff.executable = vim
vimdiff.args = -d $base $local $output $other +close +close
</code>
as per http://mercurial.selenic.com/wiki/MergingWithVim. The page http://mercurial.selenic.com/wiki/MergeProgram contains instructions for other tools, including Emacs, and graphical tools. If you edit your <code>~/.hgrc</code> file in this way then you now can type <code>hg resolve --all</code> to run your chosen tool on all unresolved files. Note that by default if your chosen tool exits without an error then <code>hg resolve</code> will regard that file as resolved. To cause <code>vim</code> to exit with an error use <code>:cq</code>. See <code>hg help resolve</code> for more details.

In the example my resolution is to have the relevant lines in <code>a1tdep.F</code> look like this:
<code>
c
c Add super new feature
c Some new feature
c
</code>

== Tags and named branches ==

== Other commands ==

Resgrp:comp-photo-version control

2012-02-05T21:32:58Z

Scliffor:

== Introduction. ==

This document will teach you the basics of using mercurial, a distributed version control system (DVCS). It is particularly directed towards using mercurial with the development version of Gaussian, so the examples will use Gaussian. For more information on mercurial you can use the built-in help system <code>hg help</code>, look at the official website: <http://mercurial.selenic.com/wiki/Mercurial>, or check out the official O'Reilly book, the text of which is freely available at: <http://hgbook.red-bean.com/>.

A version control system can sometimes seem an onerous imposition unless the user understands how it is going to help them in their work, so I'll try to briefly explain. Version control is about tracking changes to data. If one has a collection of files, say a distribution of Gaussian, then one is interested in changes that have been made to those files, whether from a new distribution from Gaussian or our own modifications. We are particularly interested in cases where overlapping changes have been made. These might be updates and bug fixes from Gaussian conflicting with our own modifications to a particular link, or it might be different researchers in a group working on the same piece of code independently. A version control system will not only detect such cases but provide automated and safe methods of merging conflicting changes.

Central to the idea of a VCS is the concept of a revision or changeset. This is like a snapshot of some set of files and directories at some particular instant in their history. A repository is where a VCS stores revisions. So, using Gaussian as an example, some revisions in a particular repository might be the Gaussian source tree corresponding to Gaussian development versions H01, H08, H10, H11, etc. The data that are actually kept inside the repository are the differences, or deltas, between the revisions. This saves space compared to storing all the revisions.

== Setting up your environment (for hpc.ic). ==
There are some one-off things you must do. Firstly, edit your login scripts (<code>.bash_profile</code> or similar) to include the line:
<code>
module load mercurial
</code>
(If you're not on an IC HPC machine mercurial is available anywhere that runs Python. It is generally available in distribution package systems, if not you can get it from <http://mercurial.selenic.com/wiki/Mercurial>)

Next, create a file called <code>.hgrc</code> in your home directory and insert the following lines:
<code>
[ui]
username = Your Name <your@email.address>

[extensions]
graphlog=
</code>
(so for example, my <code>.hgrc</code> contains this line: <code>username = Simon Clifford <simon.j.clifford@gmail.com></code>).
Mercurial uses this username field to record who has changed what.

== Checking out a repository. ==

Now let's get started and check out a repository. Once you have loaded the mercurial module you will be able to type:
<code>
tmp$ hg clone /home/gaussian-devel/official-gaussian-version testrepo
tmp$
</code>
This will create a copy of the repository at <code>/home/gaussian-devel/official-gaussian-version</code> and place it in a directory inside the current directory called <code>testrepo</code>. You can give a full path as the second argument, or, you can leave it off altogether in which case mercurial will clone the repository into a directory named <code>official-gaussian-version</code>. A note here for Imperial HPC users. Copying a repository involves quite a bit of filesystem activity. I have found that on cx1 this can be quite slow on the /home filesystems. You may wish to try these examples in the <code>/tmp</code> filesystem which is a lot faster. Be aware, however, that there are two potentially serious problems. The first is that <code>/tmp</code> is globally readable so before you clone you must make sure that your umask is set to <code>0077</code>. The second, of course, is that <code>/tmp</code> is periodically wiped. If you plan on working in <code>/tmp</code> there are simple ways of transferring information between repositories (<code>hg push</code> and <code>hg pull</code>) that I will cover later.

Change into the <code>testrepo</code> directory. Inside you will see three scripts and a <code>gdv</code> directory; also some hidden files and a hidden directory. Inside the <code>gdv</code> directory you will see the Gaussian source in the new layout. The hidden <code>.hg</code> directory inside the <code>testrepo</code> directory is the repository proper; you will almost never need to do anything in here. The rest of the files and directories are called the working directory; this is where you will do your work.

The first thing you might want to do is check the history of this repository to see past revisions that have been checked into it. You do this with the <code>hg log</code> command.
<code>
testrepo$ hg log | head -20
changeset: 23:1de40efd1c4b
tag: tip
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:43:13 2012 +0000
summary: Added tag h13 for changeset 6c81eb8dcbab

changeset: 22:6c81eb8dcbab
tag: h13
parent: 21:5b6ac3665a29
parent: 11:a0c273aeeb2b
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:43:08 2012 +0000
summary: Gaussian devel version H13 with our makefiles

changeset: 21:5b6ac3665a29
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:41:40 2012 +0000
summary: Added tag h12p for changeset 0ef14d7dff56
...
changeset: 1:515a93bfc3b5
branch: raw
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Sun Jan 15 11:39:10 2012 +0000
summary: Added tag h01-raw for changeset 073d6aa63ea7

changeset: 0:073d6aa63ea7
branch: raw
tag: h01-raw
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Sun Jan 15 11:39:09 2012 +0000
summary: Output from old-to-new.sh on version H01
</code>
You will see that there are various fields that may appear for each changeset. The changeset field shows two numbers separated by a colon. The second, longer, hexadecimal number is the unique identifier. A particular hex identifier will always refer to the same changeset, even in different copies of the repository. The first number is a local identifier. These local IDs refer to particular changesets in one copy of the repository; if these changesets are present in another repository they may have a different local identifier. You can use either identifier as an argument to <code>hg</code> commands. The user and date fields show who committed the change and when. The summary field shows the first line of the log entry for that revision. This means it is important to try to make the first line of your log entries a useful summary of what you have done!

To see the log entry for a particular revision use the <code>-r</code> flag with a revision number, either short or long. To see more information, including the full log entry and a complete list of all files involved in the change, use the <code>-v</code> flag, e.g.:
<code>
testrepo$ hg log -r 2 -v | less
changeset: 2:9f69ed4616ad
branch: raw
tag: h08-raw
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Sun Jan 15 11:54:50 2012 +0000
files: gdv/archlib/arcvib.F gdv/archlib/brcpyw.F gdv/archlib/letset.F gdv/arctmp.F gdv/bsd/GauDiff_Compare.pm gdv/bsd/G
....
at.F gdv/utilnz/xpndnb.F gdv/utilnz/zerock.F gdv/utilnz/zerod.F gdv/wrappers/ggeev.F gdv/wrappers/lappar.F gdv/wrappers/xgetrf.F gdv/wrappers/xgetrs.F gdv/wrappers/ygeru.F gdv/wrappers/ytrsv.F
description:
Import of gdv H08. See below for details.

Result of these commands on the H01 raw rev:
rm -fr gdv
cp -pr h08/new-style-gdv .
hg add gdv/bsd/*.a
hg addremove -s 50

h08/new-style is from old-to-new.sh on freshly untarred h08.tgz
</code>
As the log entry explains, this particular revision is the change from the H01 version of Gaussian to the H08 version, so there are a lot of changed files. The full log entry is intended to provide some human readable explanation of the changes. When you start committing data to the repository you should aim to at least make your log entries understandable.

As with all the commands that I will mention you can get more information by typing <code>hg help</code> command.

== Committing changes. ==

Now let's make some alterations to this repository. This is quite safe; we can't break the original repository because our copy is an entirely separate clone. In fact, this is a very common working practice with mercurial. As long as the filesystem isn't slow cloning a repository can be quite quick. Indeed, cloning a repository into a destination on the same filesystem as the source makes use of hard linking which is very fast and saves space. Therefore, it is very common to clone a repository, make some alterations to it, and then decide whether to push them back to the original repository or just delete the new clone.

First, we want to know where it is we are starting from. To get a quick summary you use the <code>hg summary</code> command. This should show something like this:
<code>
testrepo$ hg summary
parent: 23:1de40efd1c4b tip
Added tag h13 for changeset 6c81eb8dcbab
branch: default
commit: (clean)
update: (current)
</code>
This shows, on the parent line, the revision that has been checked out into the working directory. Below that is the summary line from the log entry for that revision. The commit line shows if any files have been modified since the check out. The update line shows if there are any applicable newer revisions in the repository. In this example the output shows us that we checked out the revision with the local identifier 23, and unique identifier of 1de40efd1c4b, that we have modified nothing and that there are no appropriate newer revisions. This checkout occurred automatically as a part of the <code>hg clone</code> command. You can clone without checking anything out into the working directory (say to make a backup copy of the repository) by passing the <code>-U</code> flag, or you can check out a particular revision with the <code>-u REV</code> flag. See <code>hg help clone</code> for how mercurial chooses what to check out by default.

Let's say we're adding a feature to link 510. If you have something in mind go ahead and do it. For this example I have just added a few lines to the file <code>a1tdep.F</code>, and will assume that your current working directory is <code>gdv/l510</code>.

As you work you may be curious to know what it is you have changed. The command to show this is <code>hg status</code>:
<code>
testrepo$ hg status
M gdv/l510/a1tdep.F
</code>
The output shows a list of files that have been altered in some way. Here the capital M shows that the file <code>a1tdep.F</code> has been modified. By default <code>hg status</code> does not report on unchanged files. If we wish to see how <code>a1tdep.F</code> has been modified we can use the <code>hg diff</code> command:
<code>
testrepo$ hg diff a1tdep.F
diff -r 1de40efd1c4b gdv/l510/a1tdep.F
--- a/gdv/l510/a1tdep.F Wed Jan 18 14:43:13 2012 +0000
+++ b/gdv/l510/a1tdep.F Wed Jan 25 11:07:55 2012 +0000
@@ -26,6 +26,9 @@
C
C **OPTIONS FOR TDEP CODE
C
+c
+c Add super new feature
+c
iTDep=813
iTrans=822
jTDep=iTDep
</code>
This shows the output in a unified different format, refer to the man page for diff for more information.

== Adding, copying, renaming, and deleting files. ==

If you have created a new file then you must let mercurial know about its existence with the <code>hg add</code> command:
<code>
testrepo$ echo "a new subroutine" > foo.F
testrepo$ hg status
M gdv/l510/a1tdep.F
? gdv/l510/foo.F
testrepo$ hg add foo.F
testrepo$ hg status
M gdv/l510/a1tdep.F
A gdv/l510/foo.F
</code>
Here, I create a new file called <code>foo.F</code>. <code>hg status</code> shows the file with a ?, to indicate that it is unknown to the repository. I then run <code>hg add foo.F</code>, after which <code>hg status</code> shows the file with an A. This indicates that the file is scheduled to be added at the next commit. If you change your mind about the file before the next commit you can use <code>hg forget</code> to undo the add.

If you are renaming a file, including the situation where you move the file from one directory to another (e.g. from l510 to utilam) then you may use the <code>hg mv</code> command. This will actually perform the move just like the normal <code>mv</code> command. If you've already moved the file you can give <code>hg mv</code> the <code>-A</code> flag to record the rename after the fact. Similarly, the <code>hg rm</code> command removes files from the working directory and records this fact in the repository, and <code>hg cp</code> copies a file.

It is important to realise that these commands act on the working directory immediately but only affect the repository when they are committed. Also, they do not affect the repository's previous history, so removing a file and then committing that change does not affect that file's existence in previous revisions in the repository. However, <code>hg mv</code> and <code>hg cp</code> create a relationship between the source and destination files. This becomes useful when merging in changes from somewhere else. So for example, above I have made changes to <code>l510/a1tdep.F</code>. Let's say that someone else has decided to make this into a utility routine and has done the command <code>hg mv l510/a1tdep.F utilam/a1tdep.F</code>. If I choose to merge my changes with this other person's, mercurial will correctly apply the changes '''and''' move the file. If instead of <code>hg mv</code> they had used <code>hg cp</code> then my changes would be applied to both files. This means that you should only use <code>hg cp</code> when it is appropriate that changes that are recorded before the copy are applied to both files. Note that this does not mean that changes to the source file are forever applied to the destination file; this will only occur when merging a revision that does not contain a copy with a revision that does. In general, you can expect mercurial to do the right thing.

== Committing the changes. ==

The <code>hg commit</code> (like many <code>hg</code> commands, <code>hg</code> commit has an alias, in this case <code>hg ci</code>. I tend to use <code>hg ci</code>) command creates a new revision and records the differences between the entire working directory and the parent revision (as shown on the parent line of <code>hg summary). This will be the same as the output from <code>hg status</code>. When you run the command mercurial will open a text editor. If you wish to specify which editor is used add an <code>editor=vim</code> (for example) line to the <code>[ui]</code> part of your <code>~/.hgrc</code> file. The editor will start with a couple of empty lines and then some lines beginning with "<code>HG:</code>". These are there to give you helpful reminders of what you've changed while you write your log message and will be ignored by mercurial when you commit. If you close the editor without writing anything, or of the editor quits with an error, mercurial will abort the commit. So for our current example:
<code>
testrepo$ hg ci
... [opens vim]
HG: Enter commit message. Lines beginning with 'HG:' are removed.
HG: Leave message empty to abort commit.
HG: --
HG: user: Simon Clifford <simon.j.clifford@gmail.com>
HG: branch 'default'
HG: added gdv/l510/foo.F
HG: changed gdv/l510/a1tdep.F
~
~
</code>
For short messages you can skip the editor step by passing the <code>-m "Log message here"</code> flag.

Understanding what you are doing here is essential to knowing what to write in the log and, importantly, when to commit. There are two schools of thought. When you commit you are creating a new revision in your local repository. Since it is your repository, and as I have previously mentioned, it is very simple to create new repositories, you should commit whenever you feel like it and feel free to write cryptic log messages that only you will understand. On the other hand, you are engaged in a collaborative exercise with other people: creating what will be a single version of Gaussian that contains yours and others' modifications and Gaussian's updates. Therefore, you should only commit when your code satisfies pre-agreed group requirements, and your log message should be detailed and comprehensible to anybody who reads it, even 50 years in the future. The correct approach, of course, is to do both.

Mercurial is a distributed version control system which means that each repository is the responsibility of its owner who can feel free to commit as frequently (or infrequently) as needed. Sometimes adding a feature or removing a bug might take weeks, and you might feel that there is no point taking snapshots of the code until it works. Other modifications might proceed incrementally with naturally defined stopping points between the start and the end. Committing at these points makes perfect sense: not only does it leave a history of the modification, it provides checkpoints, versions of the code that you can retreat to without having to start again. The log messages may tersely explain what has happened since the last revision (and note there is no point listing modified, added, etc, files as this information is stored in the commit anyway) or they may contain detailed essays on how a bug arose and the steps you have taken to fix it. Information in the logs is searchable and should be thought of as both an aide memoir for yourself and a journal entry for others.

When the time comes to merge your changes into other people's repositories, you might start thinking about the quality of your commit. Does the code compile? Does it run the test suite? Can it run any test? Have you committed a test that checks the code you have added or altered? The group may decide that revisions that are being shared must answer yes to some or all of these questions. You may decide that some or all of your revisions must pass these quality checks. Or, you may be satisfied with noting in the log entry what this particular revision can or can't do (such as compile or run).

It should be said though, that if you ever think "perhaps I should check in at this point", then go for it. Revisions live in the history of your repository forever (although see <code>hg rollback</code>), but when you transfer them to other people's repositories multiple revisions can be collapsed into one, or fixed in other ways.

Returning to our example I enter a simple message. Since <code>hg log</code> only shows the first line of any log entry by default it is a good idea to make this line a summary of the rest of the message. I close the editor and the commit is done.

We can see the results in the repository with an <code>hg log</code> command (the <code>-l 2</code> flag shows the last two revisions):
<code>
testrepo$ hg log -l 2
changeset: 24:13ecff0136c2
tag: tip
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 25 17:13:00 2012 +0000
summary: A test message.

changeset: 23:1de40efd1c4b
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:43:13 2012 +0000
summary: Added tag h13 for changeset 6c81eb8dcbab
</code>
Alternatively we can use the <code>hg tip</code> command to show the most recently added repository, whether by ourselves, or by pulling from another repository (see later).

== Fixing mistakes. ==

You can return one or more files, or the entire repository, to the state they were in when you last checked them out with in the <code>hg revert</code> command. Let's make an ill-advised alteration to our now committed change to <code>a1tdep.F</code>:
<code>
testrepo$ sed -i 's/super/super duper/' a1tdep.F
testrepo$ hg status
M gdv/l510/a1tdep.F
testrepo$ hg diff a1tdep.F
diff -r 13ecff0136c2 gdv/l510/a1tdep.F
--- a/gdv/l510/a1tdep.F Wed Jan 25 17:13:00 2012 +0000
+++ b/gdv/l510/a1tdep.F Wed Jan 25 17:23:42 2012 +0000
@@ -27,7 +27,7 @@
C **OPTIONS FOR TDEP CODE
C
c
-c Add super new feature
+c Add super duper new feature
c
iTDep=813
iTrans=822
testrepo$ hg revert a1tdep.F
testrepo$ hg status
? gdv/l510/a1tdep.F.orig
testrepo$ hg diff a1tdep.F
testrepo$ rm a1tdep.*
testrepo$ hg status
! gdv/l510/a1tdep.F
testrepo$
</code>
Here I use revert to undo the modifications to a file. Notice that <code>hg revert</code> leaves a copy of the modified file in <code>a1tdep.F.orig</code>. This shows up in <code>hg status</code> as an unknown file ("?"). Also notice that while trying to delete the .orig file I have accidentally deleted <code>a1tdep.F</code>, which now shows up in <code>hg status</code> as a missing file ("!"). I can revert this mistake too:
<code>
testrepo$ hg revert a1tdep.F
testrepo$ hg status
testrepo$ ls a1tdep.F
a1tdep.F
testrepo$
</code>
<code>hg revert</code> can also be used to cancel scheduled adds, removes, copies, and renames.

If you have just committed a revision and then change your mind you may be able to undo the effect with <code>hg rollback</code>. Doing this for our example gives us:
<code>
testrepo$ hg rollback
repository tip rolled back to revision 23 (undo commit)
working directory now based on revision 23
testrepo$ hg status
M gdv/l510/a1tdep.F
A gdv/l510/foo.F
testrepo$
</code>
This removes the revision we just checked in from the repository but does not alter the current working directory. If we choose, we could now use <code>hg revert</code> to restore these to their revision 23 state. There are two important caveats with <code>hg rollback</code>: it can only remove the last checked in revision, and that it is usually pointless to rollback a revision that has already been pushed or pulled (see later) into somebody else's repository, as we can only affect our repository. For the purposes of this example, I will once more check in the changes I have made:
<code>
testrepo$ hg ci -m 'A test message'
testrepo$ hg tip
changeset: 24:13ecff0136c2
tag: tip
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 25 17:38:52 2012 +0000
summary: A test message
testrepo$
</code>

== Collaboration. ==

The tools described are already quite useful, and form the basis of the very earliest revision control systems. The benefits of being able to detect what you've changed, and how you've changed it, being able to undo the changes selectively, and being able to take a snapshot of your work at any point of your choosing should be apparent. The true power, however, of a modern version control system is how it mediates different streams of changes, particularly those from other users. Note that a DVCS does not solve problems of merging and so on, but provides you, the user, with tools to solve them.

Revisions in the repository form a directed acyclic graph (DAG). Every revision apart from the first has at least one parent revision and may have zero or more child revisions. A revision with no children is called a "head". Generally development takes place at a head of the repository: checking in a new revision onto a head creates and consumes a head. However, it is possible to add a child to any revision, possibly creating a new head. This is known as a branch, and may be named or unnamed. A revision with children may still be a branch head if it has no children on its branch.

There are various ways of managing branches. You can have a few repositories with many branches, named and unnamed. Or you can have many repositories with largely unbranched graphs; it's up to you. Since mercurial uses heads as the default targets for some of its operations the latter approach is probably best for beginners as it makes operations within each repository simpler.

This is probably best illustrated with an example. Change directory out of the <code>testrepo</code> repository and type:
<code>
tmp$ hg clone -r 23 testrepo testmerge
updating to branch default
13427 files updated, 0 files merged, 0 files removed, 0 files unresolved
tmp$
</code>
This will clone a copy of <code>testrepo</code> with revision 23 and all its ancestors. This checks out everything except our test revision and is equivalent to cloning a fresh copy of the gaussian-official-version repository; this way is probably faster.

Now we go into the new repository and make some modifications. This time I alter the files <code>a1tdep.F</code> and <code>a2tdep.F</code>, and add a file <code>bar.F</code> in the <code>l510</code> directory:
<code>
…[some alterations]...
testmerge$ hg add gdv/l510/bar.F
testmerge$ hg diff
diff -r 1de40efd1c4b gdv/l510/a1tdep.F
--- a/gdv/l510/a1tdep.F Wed Jan 18 14:43:13 2012 +0000
+++ b/gdv/l510/a1tdep.F Thu Jan 26 12:15:13 2012 +0000
@@ -26,6 +26,9 @@
C
C **OPTIONS FOR TDEP CODE
C
+c
+c Some new feature
+c
iTDep=813
iTrans=822
jTDep=iTDep
diff -r 1de40efd1c4b gdv/l510/a2tdep.F
--- a/gdv/l510/a2tdep.F Wed Jan 18 14:43:13 2012 +0000
+++ b/gdv/l510/a2tdep.F Thu Jan 26 12:15:13 2012 +0000
@@ -16,6 +16,9 @@
C
C **DETERMINE IF WE RUN THE TIMEDEP CODE
C
+c
+c Super duper feature here too
+c
LenTst=0
CALL FILEIO(11,jTDep,LenTst,0,0)
IF (LenTst.EQ.0.OR.iopv.NE.23)RETURN
diff -r 1de40efd1c4b gdv/l510/bar.F
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/gdv/l510/bar.F Thu Jan 26 12:15:13 2012 +0000
@@ -0,0 +1,1 @@
+Stuff in here
testmerge$
testmerge$ hg ci -m 'Added second new feature'
testmerge$ hg log -l 2
changeset: 24:457db23c0b1a
tag: tip
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Thu Jan 26 12:15:54 2012 +0000
summary: Added second new feature

changeset: 23:1de40efd1c4b
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:43:13 2012 +0000
summary: Added tag h13 for changeset 6c81eb8dcbab
testmerge$
</code>
The situation now is that we have two repositories where there are revisions whose parent is revision 23 (23:1de40efd1c4b, in fact). This sort of scenario might arise because we have been working on two different features in two different repositories, or it may be that two users have been working separately. At some point you may wish to merge the work. It's up to you when and how you do this: you may wish to merge in bug fixes quite frequently, and incorporate brand-new features much less frequently. You can of course clone another repository in which to do the merge so that if it isn't satisfactory you can just delete the repository.

First we must pull the revisions from one repository to another. We use the <code>hg incoming</code> and <code>pull</code> commands to do this:
<code>
testmerge$ hg incoming ../testrepo
comparing with ../testrepo
searching for changes
changeset: 24:13ecff0136c2
tag: tip
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 25 17:39:50 2012 +0000
summary: A test message
testmerge$ hg pull ../testrepo
pulling from ../testrepo
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 2 changes to 2 files (+1 heads)
(run 'hg heads' to see heads, 'hg merge' to merge)
testmerge$ hg log -l 3
changeset: 25:13ecff0136c2
tag: tip
parent: 23:1de40efd1c4b
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 25 17:39:50 2012 +0000
summary: A test message

changeset: 24:457db23c0b1a
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Thu Jan 26 12:15:54 2012 +0000
summary: Added second new feature

changeset: 23:1de40efd1c4b
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:43:13 2012 +0000
summary: Added tag h13 for changeset 6c81eb8dcbab
testmerge$
</code>
The <code>hg incoming</code> command takes another repository as its argument and shows a list of revisions that are not present in the current repository. The <code>hg pull</code> command brings those revisions over into the current repository. The <code>hg log</code> command shows changeset 25 in the testmerge repository corresponds to number 24 in the original <code>testrepo</code> repository. Note the long ID is the same in both cases. The <code>hg pull</code> command reports that it has created a new head. We can see this clearly with the <code>hg glog</code> command (from the graphlog extension). The <code>-r 23:</code> flag tells <code>glog</code> to show revisions 23 and greater, for clarity:
<code>
testmerge$ hg glog -r 23:
o changeset: 25:13ecff0136c2
| tag: tip
| parent: 23:1de40efd1c4b
| user: Simon Clifford <simon.j.clifford@gmail.com>
| date: Wed Jan 25 17:39:50 2012 +0000
| summary: A test message
|
| @ changeset: 24:457db23c0b1a
|/ user: Simon Clifford <simon.j.clifford@gmail.com>
| date: Thu Jan 26 12:15:54 2012 +0000
| summary: Added second new feature
|
o changeset: 23:1de40efd1c4b
| user: Simon Clifford <simon.j.clifford@gmail.com>
| date: Wed Jan 18 14:43:13 2012 +0000
| summary: Added tag h13 for changeset 6c81eb8dcbab
|
testmerge$
</code>
The <code>hg push</code> command can also be used to copy revisions from one repository to another. It works much as you would expect, except that by default it does not permit the creation of new heads in the destination repository. The idea is that you tend to pull into your own repository, where you're expected to know what you're doing, while you might be pushing into someone else's repository, where creating a new head might cause confusion. <code>hg outgoing</code> is the corresponding analogue to the <code>hg incoming</code> command.

In our example we now have a situation where we have two heads. This can arise from revisions being created in different repositories and then pulled together, as shown. Alternatively you can just create new heads in one repository. A disadvantage of the single repository technique is that if you decide branch is going nowhere and elect not to merge or proceed with it, it still remains in your repository. With multiple repositories you can just delete the offending repository.

Let's say that I decide that these two features will work together and I want to merge the two branches. I can use the <code>hg up</code> (<code>hg update</code>) command to update the current working directory to a particular revision in the repository. If no revision is specified it will update to the current branch head. The revision updated to becomes the parent (as shown by <code>hg summary</code> or <code>hg parent</code>) of the working directory. If there are modified files in the working directory the update may attempt a merge. The revision of the working directory when you start to merge is called the base of the merge. This is important if the two revisions you are merging are on different branches, because the newly merged revision will stay on the base branch. Note that the <code>hg pull</code> command does not update the working directory, so in this case it will still be at revision 24. Let's update to revision 25 and then merge the changes from revision 24:

[Which revision to merge from: in general you merge changes into whatever is your current baseline. So, if you're merging the latest updates from Gaussian (from H10 to H11 for example) into your code, you update to your code and merge in the H11 revision. If you're merging in your changes into the soon to be sent to Gaussian group development version you'd start with that and merge in the revision(s) containing your changes]
<code>
testmerge$ hg up 25
3 files updated, 0 files merged, 1 files removed, 0 files unresolved
testmerge$ hg merge 24
merging gdv/l510/a1tdep.F
warning: conflicts during merge.
merging gdv/l510/a1tdep.F failed!
2 files updated, 0 files merged, 0 files removed, 1 files unresolved
use 'hg resolve' to retry unresolved file merges or 'hg update -C .' to abandon
testmerge$
</code>
The software automatically brings in the changes to foo.F, bar.F, and a2tdep.F (as <code>hg status</code> will show). However, in our deliberately contrived example <code>a1tdep.F</code> is subject to changes from both revisions. Mercurial will attempt to merge automatically, and in this case it fails. If you edit <code>a1tdep.F</code> you will see it contains the lines:
<code>
<<<<<<< local
c Add super new feature
=======
c Some new feature
>>>>>>> other
</code>
which were inserted during the failed merge. There is also an <code>a1tdep.F.orig</code> file created as a result of the failed merge. As the output from <code>hg merge</code> says you can either now resolve this failed merge or use <code>hg update -C</code> to undo it (the <code>.orig</code> file will remain).

Since we want to resolve the merge we could edit <code>a1tdep.F</code> so that it works. This would certainly involve removing the “<code><<<<<<< local</code>”, “<code>=======</code>”, and “<code>>>>>>>> other</code>” lines, but might involve a complete rewrite of this and other files. This is what I mean when I say that mercurial only provides tools to do merging. Even a completely successful merge, from mercurial's point of view, may be completely bogus code. However, there are tools available to make this task easier. For example I use <code>vimdiff</code>, a part of the popular <code>vim</code> package. To enable this I have altered my <code>~/.hgrc</code> file to look like this:
<code>
[ui]
merge = vimdiff
username = Simon Clifford <simon.j.clifford@gmail.com>

[extensions]
graphlog=

[merge-tools]
vimdiff.executable = vim
vimdiff.args = -d $base $local $output $other +close +close
</code>
as per <http://mercurial.selenic.com/wiki/MergingWithVim>. The page <http://mercurial.selenic.com/wiki/MergeProgram> contains instructions for other tools, including Emacs, and graphical tools. If you edit your <code>~/.hgrc</code> file in this way then you now can type <code>hg resolve --all</code> to run your chosen tool on all unresolved files. Note that by default if your chosen tool exits without an error then mercurial will regard that file as resolved. To cause <code>vim</code> to exit with an error use <code>:cq</code>. See <code>hg help resolve</code> for more details.

In the example my resolution is to have the offending lines in <code>a1tdep.F</code> look like this:
<code>
c
c Add super new feature
c Some new feature
c
</code>

Resgrp:comp-photo-version control

2012-02-04T22:58:03Z

Scliffor:

== Introduction. ==

This document will teach you the basics of using mercurial, a distributed version control system (DVCS). It is particularly directed towards using mercurial with the development version of Gaussian, so the examples will use Gaussian. For more information on mercurial you can use the built-in help system "hg help", look at the official website: <http://mercurial.selenic.com/wiki/Mercurial>, or check out the official O'Reilly book, the text of which is freely available at: <http://hgbook.red-bean.com/>.

A version control system can sometimes seem an onerous imposition unless the user understands how it is going to help them in their work, so I'll try to briefly explain. Version control is about tracking changes to data. If one has a collection of files, say a distribution of Gaussian, then one is interested in changes that have been made to those files, whether from a new distribution from Gaussian or our own modifications. We are particularly interested in cases where overlapping changes have been made. These might be updates and bug fixes from Gaussian conflicting with our own modifications to a particular link, or it might be different researchers in a group working on the same piece of code independently. A version control system will not only detect such cases but provide automated and safe methods of merging conflicting changes.

Central to the idea of a VCS is the concept of a revision or changeset. This is like a snapshot of some set of files and directories at some particular instant in their history. A repository is where a VCS stores revisions. So, using Gaussian as an example, some revisions in a particular repository might be the Gaussian source tree corresponding to Gaussian development versions H01, H08, H10, H11, etc. The data that are actually kept inside the repository are the differences, or deltas, between the revisions. This saves space compared to storing all the revisions.

== Setting up your environment (for hpc.ic). ==

There are some one-off things you must do. Firstly, edit your login scripts (.bash_profile or similar) to include the line:

<code>
module load mercurial
</code>

(If you're not on an IC HPC machine mercurial is available anywhere that runs Python. It is generally available in distribution package systems, if not you can get it from <http://mercurial.selenic.com/wiki/Mercurial>)

Next, create a file called .hgrc in your home directory and insert the following lines:

<code>
[ui]
username = Your Name <your@email.address>

[extensions]
graphlog=
</code>
(so for example, my <code>.hgrc</code contains this line: <code>username = Simon Clifford <simon.j.clifford@gmail.com></code>).
Mercurial uses this username field to record who has changed what.

== Checking out a repository. ==

Now let's get started and check out a repository. Once you have loaded the mercurial module you will be able to type:
<code>
tmp$ hg clone /home/gaussian-devel/official-gaussian-version testrepo
tmp$
</code>
This will create a copy of the repository at <code>/home/gaussian-devel/official-gaussian-version</code> and place it in a directory inside the current directory called testrepo. You can give a full path as the second argument, or, you can leave it off altogether in which case mercurial will clone the repository into a directory named official-gaussian-version. A note here for Imperial HPC users. Copying a repository involves quite a bit of filesystem activity. I have found that on cx1 this can be quite slow on the /home filesystems. You may wish to try these examples in the /tmp filesystem which is a lot faster. Be aware, however, that there are two potentially serious problems. The first is that /tmp is globally readable so before you clone you must make sure that your umask is set to 0077. The second, of course, is that /tmp is periodically wiped. If you plan on working in /tmp there are simple ways of transferring information between repositories (hg push and hg pull) that I will cover later.

Change into the testrepo directory. Inside you will see three scripts and a gdv directory; also some hidden files and a hidden directory. Inside the gdv directory you will see the Gaussian source in the new layout. The hidden .hg directory inside the testrepo directory is the repository proper; you will almost never need to do anything in here. The rest of the files and directories are called the working directory; this is where you will do your work.

The first thing you might want to do is check the history of this repository to see pat revisions that have been checked into it. You do this with the hg log command.

<code>
testrepo$ hg log | head -20
changeset: 23:1de40efd1c4b
tag: tip
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:43:13 2012 +0000
summary: Added tag h13 for changeset 6c81eb8dcbab

changeset: 22:6c81eb8dcbab
tag: h13
parent: 21:5b6ac3665a29
parent: 11:a0c273aeeb2b
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:43:08 2012 +0000
summary: Gaussian devel version H13 with our makefiles

changeset: 21:5b6ac3665a29
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:41:40 2012 +0000
summary: Added tag h12p for changeset 0ef14d7dff56
...
changeset: 1:515a93bfc3b5
branch: raw
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Sun Jan 15 11:39:10 2012 +0000
summary: Added tag h01-raw for changeset 073d6aa63ea7

changeset: 0:073d6aa63ea7
branch: raw
tag: h01-raw
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Sun Jan 15 11:39:09 2012 +0000
summary: Output from old-to-new.sh on version H01
</code>
You will see that there are various fields that may appear for each changeset. The changeset field shows two numbers separated by a colon. The second, longer, hexadecimal number is the unique identifier. A particular hex identifier will always refer to the same changeset, even in different copies of the repository. The first number is a local identifier. These local IDs refer to particular changesets in one copy of the repository; if these changesets are present in another repository they may have a different local identifier. You can use either identifier as an argument to hg commands. The user and date fields show who committed the change and when. The summary field shows the first line of the log entry for that revision. This means it is important to try to make the first line of your log entries a useful summary of what you have done!

To see the log entry for a particular revision use the -r flag with a revision number, either short or long. To see more information, including the full log entry and a complete list of all files involved in the change, use the -v flag, e.g.:
<code>
testrepo$ hg log -r 2 -v | less
changeset: 2:9f69ed4616ad
branch: raw
tag: h08-raw
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Sun Jan 15 11:54:50 2012 +0000
files: gdv/archlib/arcvib.F gdv/archlib/brcpyw.F gdv/archlib/letset.F gdv/arctmp.F gdv/bsd/GauDiff_Compare.pm gdv/bsd/G
....
at.F gdv/utilnz/xpndnb.F gdv/utilnz/zerock.F gdv/utilnz/zerod.F gdv/wrappers/ggeev.F gdv/wrappers/lappar.F gdv/wrappers/xgetrf.F gdv/wrappers/xgetrs.F gdv/wrappers/ygeru.F gdv/wrappers/ytrsv.F
description:
Import of gdv H08. See below for details.

Result of these commands on the H01 raw rev:
rm -fr gdv
cp -pr h08/new-style-gdv .
hg add gdv/bsd/*.a
hg addremove -s 50

h08/new-style is from old-to-new.sh on freshly untarred h08.tgz
</code>
As the log entry explains, this particular revision is the change from the H01 version of Gaussian to the H08 version, so there are a lot of changed files. The full log entry is intended to provide some human readable explanation of the changes. When you start committing data to the repository you should aim to at least make your log entries understandable.

As with all the commands that I will mention you can get more information by typing hg help command.

== Committing changes. ==

Now let's make some alterations to this repository. This is quite safe; we can't break the original repository because our copy is an entirely separate clone. In fact, this is a very common working practice with mercurial. As long as the filesystem isn't slow cloning repository can be quite quick. Indeed, cloning repository into a destination on the same filesystem as the source makes use of hard linking which is very fast and saves space. Therefore, it is very common to clone a repository, make some alterations to it, and then decide whether to push them back to the original repository or just delete the new clone.

First, we want to know where it is we are starting from. To get a quick summary you use the "hg summary" command. This should show something like this:
<code>
testrepo$ hg summary
parent: 23:1de40efd1c4b tip
Added tag h13 for changeset 6c81eb8dcbab
branch: default
commit: (clean)
update: (current)
</code>
This shows, on the parent line, the revision that has been checked out into the working directory. Below that is the summary line from the log entry for that revision. The commit line shows if any files have been modified since the check out. The update line shows if there are any applicable newer revisions in the repository. In this example the output shows us that we checked out the revision with the local identifier 23, and unique identifier of 1de40efd1c4b, that we have modified nothing and that there are no appropriate newer revisions. This checkout occurred automatically as a part of the hg clone command. You can clone without checking anything out into the working directory (say to make a backup copy of the repository) by passing the -U flag, or you can check out a particular revision with the -u REV flag. See hg help clone for how hg chooses what to check out by default.

Let's say we're adding a feature to link 510. If you have something in mind go ahead and do it. For this example I have just added a few lines to the file a1tdep.F, and will assume that your current working directory is gdv/l510.

As you work you may be curious to know what it is you have changed. The command to show this is hg status:
<code>
testrepo$ hg status
M gdv/l510/a1tdep.F
</code>
The output shows a list of files that have been altered in some way. Here the capital M shows that the file a1tdep.F has been modified. By default hg status does not report on unchanged files. If we wish to see how a1tdep.F has been modified we can use the hg diff command:
<code>
testrepo$ hg diff a1tdep.F
diff -r 1de40efd1c4b gdv/l510/a1tdep.F
--- a/gdv/l510/a1tdep.F Wed Jan 18 14:43:13 2012 +0000
+++ b/gdv/l510/a1tdep.F Wed Jan 25 11:07:55 2012 +0000
@@ -26,6 +26,9 @@
C
C **OPTIONS FOR TDEP CODE
C
+c
+c Add super new feature
+c
iTDep=813
iTrans=822
jTDep=iTDep
</code>
This shows the output in a unified different format, refer to the man page for diff for more information.

== Adding, copying, renaming, and deleting files. ==

If you have created a new file then you must let mercurial know about its existence with the hg add command:
<code>
testrepo$ echo "a new subroutine" > foo.F
testrepo$ hg status
M gdv/l510/a1tdep.F
? gdv/l510/foo.F
testrepo$ hg add foo.F
testrepo$ hg status
M gdv/l510/a1tdep.F
A gdv/l510/foo.F
</code>
Here, I create a new file called foo.F. hg status shows the file with a ?, to indicate that it is unknown to the repository. I then run "hg add foo.F", after which hg status shows the file with an A. This indicates that the file is scheduled to be added at the next commit. If you change your mind about the file before the next commit you can use hg forget to undo the add.

If you are renaming a file, including the situation where you move the file from one directory to another (e.g. from l510 to utilam) then you may use the hg mv command. This will actually perform the move just like the normal mv command. If you've already moved the file you can give hg mv the -A flag to record the rename after the fact. Similarly, the hg rm command removes files from the working directory and records this fact in the repository, and hg cp copies a file.

It is important to realise that these commands (and hg copy) act on the working directory immediately but only affect the repository when they are committed. Also, they do not affect the repository's previous history, so removing a file and then committing that change does not affect that file's existence in previous revisions in the repository. However, hg mv and hg cp create a relationship between the source and destination files. This becomes useful when merging in changes from somewhere else. So for example, above I have made changes to l510/a1tdep.F. Let's say that someone else has decided to make this into a utility routine and has done the command hg mv l510/a1tdep.F utilam/a1tdep.F. If I choose to merge my changes with this other person's, mercurial will correctly apply the changes AND move the file. If instead of hg mv they had used hg cp then my changes would be applied to both files. This means that you should only use hg cp when it is appropriate that changes that are recorded before the copy are applied to both files. Note that this does not mean that changes to the source file are forever applied to the destination file; this will only occur when merging a revision that does not contain a copy with a revision that does. In general, you can expect mercurial to do the right thing.

== Committing the changes. ==

The hg commit (like many hg commands, hg commit has an alias, in this case hg ci. I tend to use hg ci) command creates a new revision and records the differences between the entire working directory and the parent revision (as shown on the parent line of hg summary). This will be the same as the output from hg status. When you run the command mercurial will open a text editor. If you wish to specify which editor is used add an editor=vim (for example) line to the [ui] part of your ~/.hgrc file. The editor will start with a couple of empty lines and then some lines beginning with "HG:". These are there to give you helpful reminders of what you've changed while you write your log message and will be ignored by mercurial when you commit. If you close the editor without writing anything, or of the editor quits with an error, mercurial will abort the commit. So for our current example:
<code>
testrepo$ hg ci
... [opens vim]
HG: Enter commit message. Lines beginning with 'HG:' are removed.
HG: Leave message empty to abort commit.
HG: --
HG: user: Simon Clifford <simon.j.clifford@gmail.com>
HG: branch 'default'
HG: added gdv/l510/foo.F
HG: changed gdv/l510/a1tdep.F
~
~
</code>
For short messages you can skip the editor step by passing the -m "Log message here" flag.

Understanding what you are doing here is essential to knowing what to write in the log and, importantly, when to commit. There are two schools of thought. When you commit you are creating a new revision in your local repository. Since it is your repository, and as I have previously mentioned, it is very simple to create new repositories, you should commit whenever you feel like it and feel free to write cryptic log messages that only you will understand. On the other hand, you are engaged in a collaborative exercise with other people: creating what will be a single version of Gaussian that contains yours and others' modifications and Gaussian's updates. Therefore, you should only commit when your code satisfies pre-agreed group requirements, and your log message should be detailed and comprehensible to anybody who reads it, even 50 years in the future. The correct approach, of course, is to do both.

Mercurial is a distributed version control system which means that each repository is the responsibility of its owner who can feel free to commit as frequently (or infrequently) as needed. Sometimes adding a feature or removing a bug might take weeks, and you might feel that there is no point taking snapshots of the code until it works. Other modifications might proceed incrementally with naturally defined stopping points between the start and the end. Committing at these points makes perfect sense: not only does it leave a history of the modification, it provides checkpoints, versions of the code that you can retreat to without having to start again. The log messages may tersely explain what has happened since the last revision (and note there is no point listing modified, added, etc, files as this information is stored in the commit anyway) or they may contain detailed essays on how a bug arose and the steps you have taken to fix it. Information in the logs is searchable and should be thought of as both an aide memoir for yourself and a journal entry for others.

When the time comes to merge your changes into other people's repositories, you might start thinking about the quality of your commit. Does the code compile? Does it run the test suite? Can it run any test? Have you committed a test that checks the code you have added or altered? The group may decide that revisions that are being shared must answer yes to some or all of these questions. You may decide that some or all of your revisions must pass these quality checks. Or, you may be satisfied with noting in the log entry what this particular revision can or can't do (such as compile or run).

It should be said though, that if you ever think "perhaps I should check in at this point", then go for it. Revisions live in the history of your repository forever (although see hg rollback), but when you transfer them to other people's repositories multiple revisions can be collapsed into one, or fixed in other ways.

Returning to our example I enter a simple message. Since hg log only shows the first line of any log entry by default it is a good idea to make this line a summary of the rest of the message. I close the editor and the commit is done.

We can see the results in the repository with an hg log command (the -l 2 flag shows the last five revisions):
<code>
testrepo$ hg log -l 2
changeset: 24:13ecff0136c2
tag: tip
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 25 17:13:00 2012 +0000
summary: A test message.

changeset: 23:1de40efd1c4b
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:43:13 2012 +0000
summary: Added tag h13 for changeset 6c81eb8dcbab
</code>
Alternatively we can use the hg tip command to show the most recently added repository, whether by ourselves, or by pulling from another repository (see later).

== Fixing mistakes. ==

You can return one or more files, or the entire repository, to the state they were in when you last checked them out with in the hg revert command. Let's make an ill-advised alteration to our now committed change to a1tdep.F:
<code>
testrepo$ sed -i 's/super/super duper/' a1tdep.F
testrepo$ hg status
M gdv/l510/a1tdep.F
testrepo$ hg diff a1tdep.F
diff -r 13ecff0136c2 gdv/l510/a1tdep.F
--- a/gdv/l510/a1tdep.F Wed Jan 25 17:13:00 2012 +0000
+++ b/gdv/l510/a1tdep.F Wed Jan 25 17:23:42 2012 +0000
@@ -27,7 +27,7 @@
C **OPTIONS FOR TDEP CODE
C
c
-c Add super new feature
+c Add super duper new feature
c
iTDep=813
iTrans=822
testrepo$ hg revert a1tdep.F
testrepo$ hg status
? gdv/l510/a1tdep.F.orig
testrepo$ hg diff a1tdep.F
testrepo$ rm a1tdep.*
testrepo$ hg status
! gdv/l510/a1tdep.F
testrepo$
</code>
Here I use revert to undo the modifications to a file. Notice that hg revert leaves a copy of the modified file in a1tdep.F.orig. This shows up in hg status as an unknown file ("?"). Also notice that while trying to delete the .orig file I have accidentally deleted a1tdep.F, which now shows up in hg status as a missing file ("!"). I can revert this mistake too:
<code>
testrepo$ hg revert a1tdep.F
testrepo$ hg status
testrepo$ ls a1tdep.F
a1tdep.F
testrepo$
</code>
hg revert can also be used to cancel scheduled adds, removes, copies, and renames.

If you have just committed a revision and then change your mind you may be able to undo the effect with hg rollback. Doing this for our example gives us:
<code>
testrepo$ hg rollback
repository tip rolled back to revision 23 (undo commit)
working directory now based on revision 23
testrepo$ hg status
M gdv/l510/a1tdep.F
A gdv/l510/foo.F
testrepo$
</code>
This removes the revision we just checked in from the repository but does not alter the current working directory. If we choose, we could now use hg revert to restore these to their revision 23 state. There are two important caveats with hg rollback: it can only remove the last checked in revision, and that it is usually pointless to rollback a revision that has already been pushed or pulled (see later) into somebody else's repository, as we can only affect our repository. For the purposes of this example, I will once more check in the changes I have made:
<code>
testrepo$ hg ci -m 'A test message'
testrepo$ hg tip
changeset: 24:13ecff0136c2
tag: tip
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 25 17:38:52 2012 +0000
summary: A test message
testrepo$
</code>

== Collaboration. ==

The tools described are already quite useful, and form the basis of the very earliest revision control systems. The benefits of being able to detect what you've changed, and how you've changed it, being able to undo the changes selectively, and being able to take a snapshot of your work at any point of your choosing should be apparent. The true power, however, of a modern version control system is how it mediates different streams of changes, particularly those from other users. Note that a DVCS does not solve problems of merging and so on, but provides you, the user, with tools to solve them.

Revisions in the repository form a directed acyclic graph (DAG). Every revision apart from the first has at least one parent revision and may have zero or more child revisions. A revision with no children is called a "head". Generally development takes place at a head of the repository: checking in a new revision onto a head creates and consumes a head. However, it is possible to add a child to any revision, possibly creating a new head. This is known as a branch, and may be named or unnamed. A revision with children may still be a branch head if it has no children on its branch.

There are various ways of managing branches. You can have a few repositories with many branches, named and unnamed. Or you can have many repositories with largely unbranched graphs; it's up to you. Since mercurial uses heads as the default targets for some of its operations the latter approach is probably best for beginners as it makes operations within each repository simpler.

This is probably best illustrated with an example. Change directory out of the testrepo repository and type:
<code>
tmp$ hg clone -r 23 testrepo testmerge
updating to branch default
13427 files updated, 0 files merged, 0 files removed, 0 files unresolved
tmp$
</code>
This will clone a copy of testrepo with revision 23 and all its ancestors. This checks out everything except our test revision and is equivalent to cloning a fresh copy of the gaussian-official-version repository; this way is probably faster.

Now we go into the new repository and make some modifications. This time I alter the files a1tdep.F and a2tdep.F, and add a file bar.F in the l510 directory:
<code>
…[some alterations]...
testmerge$ hg add gdv/l510/bar.F
testmerge$ hg diff
diff -r 1de40efd1c4b gdv/l510/a1tdep.F
--- a/gdv/l510/a1tdep.F Wed Jan 18 14:43:13 2012 +0000
+++ b/gdv/l510/a1tdep.F Thu Jan 26 12:15:13 2012 +0000
@@ -26,6 +26,9 @@
C
C **OPTIONS FOR TDEP CODE
C
+c
+c Some new feature
+c
iTDep=813
iTrans=822
jTDep=iTDep
diff -r 1de40efd1c4b gdv/l510/a2tdep.F
--- a/gdv/l510/a2tdep.F Wed Jan 18 14:43:13 2012 +0000
+++ b/gdv/l510/a2tdep.F Thu Jan 26 12:15:13 2012 +0000
@@ -16,6 +16,9 @@
C
C **DETERMINE IF WE RUN THE TIMEDEP CODE
C
+c
+c Super duper feature here too
+c
LenTst=0
CALL FILEIO(11,jTDep,LenTst,0,0)
IF (LenTst.EQ.0.OR.iopv.NE.23)RETURN
diff -r 1de40efd1c4b gdv/l510/bar.F
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/gdv/l510/bar.F Thu Jan 26 12:15:13 2012 +0000
@@ -0,0 +1,1 @@
+Stuff in here
testmerge$
testmerge$ hg ci -m 'Added second new feature'
testmerge$ hg log -l 2
changeset: 24:457db23c0b1a
tag: tip
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Thu Jan 26 12:15:54 2012 +0000
summary: Added second new feature

changeset: 23:1de40efd1c4b
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:43:13 2012 +0000
summary: Added tag h13 for changeset 6c81eb8dcbab
testmerge$
</code>
The situation now is that we have two repositories where there are revisions whose parent is revision 23 (23:1de40efd1c4b, in fact). This sort of scenario might arise because we have working on two different features in two different repositories, or it may be that two users have been working separately. At some point you may wish to merge the work. It's up to you when and how you do this: you may wish to merge in bug fixes quite frequently, and incorporate brand-new features much less frequently. You can of course clone another repository in which to do the merge so that if it isn't satisfactory you can just delete the repository.

First we must pull the revisions from one repository to another. We use the hg incoming and pull commands to do this:
<code>
testmerge$ hg incoming ../testrepo
comparing with ../testrepo
searching for changes
changeset: 24:13ecff0136c2
tag: tip
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 25 17:39:50 2012 +0000
summary: A test message
testmerge$ hg pull ../testrepo
pulling from ../testrepo
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 2 changes to 2 files (+1 heads)
(run 'hg heads' to see heads, 'hg merge' to merge)
testmerge$ hg log -l 3
changeset: 25:13ecff0136c2
tag: tip
parent: 23:1de40efd1c4b
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 25 17:39:50 2012 +0000
summary: A test message

changeset: 24:457db23c0b1a
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Thu Jan 26 12:15:54 2012 +0000
summary: Added second new feature

changeset: 23:1de40efd1c4b
user: Simon Clifford <simon.j.clifford@gmail.com>
date: Wed Jan 18 14:43:13 2012 +0000
summary: Added tag h13 for changeset 6c81eb8dcbab
testmerge$
</code>
The hg incoming command takes another repository as its argument and shows a list of revisions that are not present in the current repository. The hg pull command brings those revisions over into the current repository. The hg log command shows changeset 25 in the testmerge repository corresponds to number 24 in the original testrepo repository. Note the long ID is the same in both cases. The hg pull command reports that it has created a new head. We can see this clearly with the hg glog command (from the graphlog extension). The "-r 23:" flag tells glog to show revisions 23 and greater, for clarity:
<code>
testmerge$ hg glog -r 23:
o changeset: 25:13ecff0136c2
| tag: tip
| parent: 23:1de40efd1c4b
| user: Simon Clifford <simon.j.clifford@gmail.com>
| date: Wed Jan 25 17:39:50 2012 +0000
| summary: A test message
|
| @ changeset: 24:457db23c0b1a
|/ user: Simon Clifford <simon.j.clifford@gmail.com>
| date: Thu Jan 26 12:15:54 2012 +0000
| summary: Added second new feature
|
o changeset: 23:1de40efd1c4b
| user: Simon Clifford <simon.j.clifford@gmail.com>
| date: Wed Jan 18 14:43:13 2012 +0000
| summary: Added tag h13 for changeset 6c81eb8dcbab
|
testmerge$
</code>
The hg push command can also be used to copy revisions from one repository to another. It works much as you would expect, except that by default it does not permit the creation of new heads in the destination repository. The idea is that you tend to pull into your own repository, where you're expected to know what you're doing, while you might be pushing into someone else's repository, where creating a new head might cause confusion. hg outgoing is the corresponding analogue to the hg incoming command.

In our example we now have a situation where we have two heads. This can arise from revisions being created in different repositories and then pulled together, as shown. Alternatively you can just create new heads in one repository. A disadvantage of the single repository technique is that if you decide branch is going nowhere and elect not to merge or proceed with it, it still remains in your repository. With multiple repositories you can just delete the offending repository.

Let's say that I decide that these two features will work together and I want to merge the two branches. I can use the hg up (hg update) command to update the current working directory to a particular revision in the repository. If no revision is specified it will update to the current branch head. The revision updated to becomes the parent (as shown by hg summary or hg parent) of the working directory. If there are modified files in the working directory the update may attempt a merge. The revision of the working directory when you start to merge is called the base of the merge. This is important if the two revisions you are merging are on different branches, because the newly merged revision will stay on the base branch. Note that the hg pull command does not update the working directory, so in this case it will still be at revision 24. Let's update to revision 25 and then merge the changes from revision 24:

[Which revision to merge from: in general you merge changes into whatever is your current baseline. So, if you're merging the latest updates from Gaussian (from H10 to H11 for example) into your code, you update to your code and merge in the H11 revision. If you're merging in your changes into the soon to be sent to Gaussian group development version you'd start with that and merge in the revision(s) containing your changes]
<code>
testmerge$ hg up 25
3 files updated, 0 files merged, 1 files removed, 0 files unresolved
testmerge$ hg merge 24
merging gdv/l510/a1tdep.F
warning: conflicts during merge.
merging gdv/l510/a1tdep.F failed!
2 files updated, 0 files merged, 0 files removed, 1 files unresolved
use 'hg resolve' to retry unresolved file merges or 'hg update -C .' to abandon
testmerge$
</code>
The software automatically brings in the changes to foo.F, bar.F, and a2tdep.F (as hg status will show). However, in our deliberately contrived example a1tdep.F is subject to changes from both revisions. Mercurial will attempt to merge automatically, and in this case it fails. If you edit a1tdep.F you will see it contains the lines:
<code>
<<<<<<< local
c Add super new feature
=======
c Some new feature
>>>>>>> other
</code>
which were inserted during the failed merge. There is also an a1tdep.F.orig file created as a result of the failed merge. As the output from hg merge says you can either now resolve this failed merge or use hg update -C to undo it (the .orig file will remain).

Since we want to resolve the merge we could edit a1tdep.F so that it works. This would certainly involve removing the “<<<<<<< local”, “=======”, and “>>>>>>> other” lines, but might involve a complete rewrite of this and other files. This is what I mean when I say that mercurial only provides tools to do merging. Even a completely successful merge, from mercurial's point of view, may be completely bogus code. However, there are tools available to make this task easier. For example I use vimdiff, a part of the popular vim package. To enable this I have altered my ~/.hgrc file to look like this:
<code>
[ui]
merge = vimdiff
username = Simon Clifford <simon.j.clifford@gmail.com>

[extensions]
graphlog=

[merge-tools]
vimdiff.executable = vim
vimdiff.args = -d $base $local $output $other +close +close
</code>
as per http://mercurial.selenic.com/wiki/MergingWithVim. The page http://mercurial.selenic.com/wiki/MergeProgram contains instructions for other tools, including Emacs, and graphical tools. If you edit your ~/.hgrc file in this way then you now can type hg resolve --all to run your chosen tool on all unresolved files. Note that by default if your chosen tool exits without an error then mercurial will regard that file as resolved. To cause vim to exit with an error use :cq. See hg help resolve for more details.

In the example my resolution is to have the offending lines in a1tdep.F look like this:
<code>
c
c Add super new feature
c Some new feature
c
</code>

Resgrp:comp-photo-version control

2012-02-04T22:07:44Z

Scliffor: Created page with "La la la."

La la la.