ChemWiki - User contributions [en]

Mod:Hunt Research Group/OTRSH Funct

2018-04-03T12:49:27Z

Rf614: /* Running The Code */

==Links to files==
Zip Archive: [[Media:OTRSH_hpc_ver_V2_Mar2018.zip]]

==What's an OTRSH Functional?==
It is an optimally tuned range separated hybrid functional (OTRSH). This is a class of functionals that are useful for ccalculating electronic properties such as UV-Vis spectra (especially if charge-transfer excitations are important) or photoelectron spectra. Before talking about OTRSH functionals, it is useful to first talk about how standard range separated hybrid functionals work, and why they were created.

===The Problem===
Standard DFT has problems describing long range interactions, due to the form of the exchange energy in standard functionals. Actual (exact) Exchange energy scales with distance as <math>\frac{1}{r_{12}}</math> where <math>r_{12}</math> is the distance between two electrons. However, standard pure DFT exchange functionals depend directly on the electron density, which decays exponentially with distance from an atomic center (i.e. DFT exchange decays faster than actual exchange). This means that standard DFT functionals are poor at describing long-range interactions (charge transfer excitations in DFT is an example). Range seperated hybrid functionals try to fix this by using Hartree-Fock exchange energy (which is exact) at large distances (long-range) while keeping DFT exchange energy at short range.

===One Solution - Normal Range Separated Hybrids===
A range separated hydrbrid functional divides the exchange energy into two terms - short range exchange (<math> E_{X}^{SR}</math>) and long range exchange (<math> E_{X}^{LR} </math>). The exchange energy at any one distance is a weighted combination of short-range and long range exchange, and can essentially be described with three variables; <math>\alpha</math>, <math>\beta</math> and <math>\mu</math> (See picture below).

[[File:Frac_HF_Exchange_mu.png|200px|center|Description of the Exchange energy at varying distances in a range-separated hybrid functtional]]

The equations for total, short range (DFT) and long-range (HF) exchange are:

<math>
E_{X}^{Total} = E_{X}^{SR} + E_{X}^{LR}
</math>

<math>
E_{X}^{SR} = [1-(\alpha + \beta{}.\text{erf}(\mu r_{12})] E_{X}^{DFT}(r_{12})
</math>

<math>
E_{X}^{LR} = [\alpha + \beta{}.\text{erf}(\mu{}r_{12})] E_{X}^{HF}(r_{12})
</math>

where erf is the error function. <math>\alpha</math> represents the amount of HF exchange at short range. For example, <math>\alpha=0.2</math> would mean 20% HF exchange was used at <math>r_{12}=0</math>. <math>\alpha + \beta </math> represents the amount of HF exchange at long range (i.e as <math>r_{12}</math> approaches infinity). A value of <math>\alpha + \beta =1</math> means that 100% HF exchange is employed at long range, which should give the correct long-range behavior for the exchange energy. <math>\mu</math> is the range seperation parameter, its value determines how quickly we change from short-range to long-range exchange expresssions. The higher <math>\mu</math> is, the faster (shorter distances) we start using the long-range exchange energy expression.

A problem with range separated hybrids is in picking the correct value for <math>\mu</math> (i.e when should we stop using DFT exchange?). This is often determined empirically by varying <math>\mu</math> for a test data set, and seeing which <math>\mu</math> value leads to a functional that best reproduces an experimental property (e.g. atomic ionisation energies). This has the same problem as any kind of empirical fitting - its not clear how well the functional will work outside the test set of molecules/properties. Also this empirical tuning of parameters can lead to unphysical values. For example, CAM-B3LYP uses <math>\alpha + \beta =0.65</math>, meaning it definitely has the wrong long-range behavior.

===OTRSH Functionals===

OTRSH functionals provide a non-empirical method for choosing <math>\mu</math>; <math>\mu</math> is "tuned" on a per-calculation basis. The <math>\mu</math> value is chosen so that the calculated ionisation energy matches the negative of the HOMO energy, i.e.:

<math>
IP_{Calc} = -E_{HOMO}
</math>

<math>
IP_{Calc} = E_{N-1} - E_{N}
</math>

Where <math>E_{N-1}</math> and <math>E_{N}</math> represent the energy of the N-1 and N electron system (both in the N electron system geometry). Note that all values in the equations above can be obtained from two single-point energy calculations. Also note that <math>IP_{Calc} = -E_{HOMO}</math> is true for the exact (unknown) KS-DFT functional, hence it is a sensible thing to try to reproduce in a functional.

To summarise an OTRSH functional is one whereby <math>\mu</math> is chosen to minimise <math>
IP_{Calc} + E_{HOMO}</math>, hence the <math>\mu</math> value will be different for every calculation you carry out. Sometimes the IP and HOMO energy of the N-1 electron system are also used to tune <math>\mu</math>, but unfortunately thats not possible the code on this page.

==Running The Code==

===Quick Start Guide (Using the test file)===
Once you download the .zip archive, copy the files to somewhere on your hpc path [Note: make sure you dont miss the .rng_OTRSH_V2 file]. Personally i use "/home/rf614/bin" as the location. To use the gf_OTRSH_V2 function straight away you'll need to put them in $HOME/bin. "echo $HOME" while on the hpc to find where your home directory is if your not sure.

Move the test file ('H2O_SPE_6311plgdp_GP.com') to whereever you like in your work directory, and navigate to this folder. Then type either of the following commands (assuming you put the scripts in ~/bin):

<pre>
qsub -v in=H2O_SPE_6311plgdp_GP -N test ~/bin/.rng_OTRSH_V2
gf_OTRSH_V2 H2O_SPE_6311plgdp_GP.com
</pre>

If you saved the files somewhere else then simply modify "~/bin/.rng_OTRSH" to whatevers appropriate for the top command. For details on how the gf command works see this wiki [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:Hunt_Research_Group/pimpQSUB link].

===Quick Start Guide - The output files===
Assuming you ran the job above you will now have a loada output files in your current directory, i.e.:

H2O_SPE_6311plgdp_GP_gsstate_mu_opt.*

H2O_SPE_6311plgdp_GP_ionstate_mu_opt.*

H2O_SPE_6311plgdp_GP_OTRSH.out

The '''H2O_SPE_6311plgdp_GP_OTRSH.out''' file is essentially a progress file for how the tuning process is going. Once the calculation is over its main purpose is really to keep track of the options you used to create the OTRSH functional. Its a comma delimited file, and the headings should be pretty self-explanatory (at least they will be if you read the above section on "What is an OTRSH")

The '''H2O_SPE_6311plgdp_GP_gsstate_mu_opt'''* files are for single energy calculations using the OTRSH functional tuned for this particular system. If your goal was to calculate ground state properties with an OTRSH then these files might be all you need. If you want to do something other than a single-point energy calculation, then just add the relevant keywords (e.g TD) to the H2O_SPE_6311plgdp_GP_gsstate_mu_opt.com file - this file contains all the keywords needed to define the OTRSH functional.

The '''H2O_SPE_6311plgdp_GP_ionstate_mu_opt'''* files are the single point energy calculations (using the OTRSH) for the ionised system. You probably have no need for these.

You will also have a new folder named '''temp_files_H2O_SPE_6311plgdp_GP'''. At every tested value for <math>\mu</math>, the .log files for the ground and ionised state are written to this folder. The purpose of this folder is to check nothings going wrong during the calculation. If something stops the code extracting scf energies for each <math>\mu</math> value, then it might just loop infinitely (or at least for a loooooong time). The code '''SHOULD''' catch these sorts of errors and kill the job, but if it runs a long time without any change in the *.out file then make sure the *.log files in this folder are ok. The secondary purpose of this folder is therefore to tell you WHY your job got killed, if it got killed (an error message should be given in the .out file as well but....)

===Quick Note on Creating Input Files===

If you want to run the code, with default options, on your own systems then theres not much to say. Simply prepare a normal '''SINGLE POINT ENERGY''' *.com file for your system, but remove the functional from the input line. Also dont use the "#p" command for excess printing - this will break the OTRSH code (sorry, the code essentially uses the # for a search + replace to insert a functional at each step of the tuning process). For example, this input line is fine(and used in the example file):
<pre>
# 6-311+g(d,p) geom=connectivity int=ultrafine scf=conver=9
</pre>

Make sure you use a '''single point energy calculation input file''', using a keyword like "freq" wont throw any errors, but you'll be waiting a really long time for your job to finish.

===Understanding (And Modifying) The Default Options===

The easily changeable variables in this code are stored in the .rng_OTRSH_V2 file. They are (in order of "likelyhood of needing altering"):

<pre>
es_mult=2
alpha_val=0
beta_val=0
mu_lower=0.1
mu_upper=0.2
max_step=0.03
min_step=0.001
xc_funct="LC-wPBE"
</pre>

Its likely you'll only ever need/want to change the '''es_mult''' here. This is the multiplicity of the ionised state, and is obviously equal to 2 if you have a closed shell system. If you use this code on an open shell system you need to set es_mult to the appropriate value for the ionised state. Personally i create seperate .rng files for each value of multiplicity i use, to save me editing the .rng_OTRSH file too often.

'''alpha_val''' corresponds to the fraction HF exchange at short range. '''Setting this to zero means use the default value for the functional'''.Its plausible that raising this to say alpha_val=0.2 might improve your results in some cases, so it might be worth trying the functional with two different alpha values if your trying to match an experimental spectrum or have something else you can validate with.

'''beta_val''' - as stated above "alpha_val + beta_val" corresponds to the fraction HF exchange at long range. '''Setting this to zero means use the default value for the functional'''. If alpha_val and beta_val are both zero, then only the <math>\mu</math> value should differ between the base functional and the OTRSH

'''mu_lower''' and '''mu_upper''' control the initial range of <math>\mu</math> values to try out. If you have a good idea what your final <math>\mu</math> will be, then settings mu_lower and mu_upper to nearby values should speed up the optimisation process. However, there is '''NO NEED''' for the final mu_val to be between these initial search values, it will just take longer if its far away from this range.

'''max_step''' and '''min_step''' correspond to the maximum and minimum change in <math>\mu</math> between optimisation steps. Note that i think the minimum possible min_step=0.0001, so dont set it lower than that or it could cause serious problems (it'll lead to a gradient of zero between two mu values, i have no idea how the optimiser will react to that but it wont be good). Theres probably '''NEVER''' a need to mess with these values. If the optimiser isnt efficient enough, changing these values wont help much. Writing a better optimiser (the current ones pretty terrible by the way,but it works so.....) would probably be easier than working out the best values for these numbers.

'''LC-wPBE''' corresponds to the underlying functional to use. The functional this code generates is essentially LC-wPBE with a modified <math>\mu</math> value. The underlying DFT exchange/ DFT correlation functionals are the same as those used in LC-wPBE. The underlying functional needs to be a range separated one (i think). If you change it, make sure you can reproduce the base functional with the code (e.g. run the code, change mu to that of the base functional and see if the results are the same).

Finally, the convergence criteria for the OTRSH (the IP_HOMO - E_HOMO value) is set to 0.01 eV by default, and controlled by the IP_conver_thres variable in optimiser.py. I guess its possible you may want to relax that criteria in certain circumstances (obviously it would speed up the optimisation and lower the accuracy).

==Further Reading/Links to relevant Papers==
CAM-B3LYP paper - doi:'''10.1016/j.cplett.2004.06.011'''

Early (first?) OTRSH paper - doi:'''10.1021/ja8087482'''

Review on using OTRSH functional to calculate bandgaps (2012) - doi:'''10.1021/ct2009363'''

Use of OTRSH functionals for Ground state properties (2013) - doi:'''10.1063/1.4807325'''

Mod:Hunt Research Group/OTRSH Funct

2018-04-03T12:41:14Z

Rf614: /* Links to files */

==Links to files==
Zip Archive: [[Media:OTRSH_hpc_ver_V2_Mar2018.zip]]

==What's an OTRSH Functional?==
It is an optimally tuned range separated hybrid functional (OTRSH). This is a class of functionals that are useful for ccalculating electronic properties such as UV-Vis spectra (especially if charge-transfer excitations are important) or photoelectron spectra. Before talking about OTRSH functionals, it is useful to first talk about how standard range separated hybrid functionals work, and why they were created.

===The Problem===
Standard DFT has problems describing long range interactions, due to the form of the exchange energy in standard functionals. Actual (exact) Exchange energy scales with distance as <math>\frac{1}{r_{12}}</math> where <math>r_{12}</math> is the distance between two electrons. However, standard pure DFT exchange functionals depend directly on the electron density, which decays exponentially with distance from an atomic center (i.e. DFT exchange decays faster than actual exchange). This means that standard DFT functionals are poor at describing long-range interactions (charge transfer excitations in DFT is an example). Range seperated hybrid functionals try to fix this by using Hartree-Fock exchange energy (which is exact) at large distances (long-range) while keeping DFT exchange energy at short range.

===One Solution - Normal Range Separated Hybrids===
A range separated hydrbrid functional divides the exchange energy into two terms - short range exchange (<math> E_{X}^{SR}</math>) and long range exchange (<math> E_{X}^{LR} </math>). The exchange energy at any one distance is a weighted combination of short-range and long range exchange, and can essentially be described with three variables; <math>\alpha</math>, <math>\beta</math> and <math>\mu</math> (See picture below).

[[File:Frac_HF_Exchange_mu.png|200px|center|Description of the Exchange energy at varying distances in a range-separated hybrid functtional]]

The equations for total, short range (DFT) and long-range (HF) exchange are:

<math>
E_{X}^{Total} = E_{X}^{SR} + E_{X}^{LR}
</math>

<math>
E_{X}^{SR} = [1-(\alpha + \beta{}.\text{erf}(\mu r_{12})] E_{X}^{DFT}(r_{12})
</math>

<math>
E_{X}^{LR} = [\alpha + \beta{}.\text{erf}(\mu{}r_{12})] E_{X}^{HF}(r_{12})
</math>

where erf is the error function. <math>\alpha</math> represents the amount of HF exchange at short range. For example, <math>\alpha=0.2</math> would mean 20% HF exchange was used at <math>r_{12}=0</math>. <math>\alpha + \beta </math> represents the amount of HF exchange at long range (i.e as <math>r_{12}</math> approaches infinity). A value of <math>\alpha + \beta =1</math> means that 100% HF exchange is employed at long range, which should give the correct long-range behavior for the exchange energy. <math>\mu</math> is the range seperation parameter, its value determines how quickly we change from short-range to long-range exchange expresssions. The higher <math>\mu</math> is, the faster (shorter distances) we start using the long-range exchange energy expression.

A problem with range separated hybrids is in picking the correct value for <math>\mu</math> (i.e when should we stop using DFT exchange?). This is often determined empirically by varying <math>\mu</math> for a test data set, and seeing which <math>\mu</math> value leads to a functional that best reproduces an experimental property (e.g. atomic ionisation energies). This has the same problem as any kind of empirical fitting - its not clear how well the functional will work outside the test set of molecules/properties. Also this empirical tuning of parameters can lead to unphysical values. For example, CAM-B3LYP uses <math>\alpha + \beta =0.65</math>, meaning it definitely has the wrong long-range behavior.

===OTRSH Functionals===

OTRSH functionals provide a non-empirical method for choosing <math>\mu</math>; <math>\mu</math> is "tuned" on a per-calculation basis. The <math>\mu</math> value is chosen so that the calculated ionisation energy matches the negative of the HOMO energy, i.e.:

<math>
IP_{Calc} = -E_{HOMO}
</math>

<math>
IP_{Calc} = E_{N-1} - E_{N}
</math>

Where <math>E_{N-1}</math> and <math>E_{N}</math> represent the energy of the N-1 and N electron system (both in the N electron system geometry). Note that all values in the equations above can be obtained from two single-point energy calculations. Also note that <math>IP_{Calc} = -E_{HOMO}</math> is true for the exact (unknown) KS-DFT functional, hence it is a sensible thing to try to reproduce in a functional.

To summarise an OTRSH functional is one whereby <math>\mu</math> is chosen to minimise <math>
IP_{Calc} + E_{HOMO}</math>, hence the <math>\mu</math> value will be different for every calculation you carry out. Sometimes the IP and HOMO energy of the N-1 electron system are also used to tune <math>\mu</math>, but unfortunately thats not possible the code on this page.

==Running The Code==

===Quick Start Guide (Using the test file)===
Once you download the .zip archive, copy the files to somewhere on your hpc path [Note: make sure you dont miss the .rng_OTRSH file]. Personally i use "/home/rf614/bin" as the location. To use the gf_OTRSH function straight away you'll need to put them in $HOME/bin. "echo $HOME" while on the hpc to find where your home directory is if your not sure.

Move the test file ('H2O_SPE_6311plgdp_GP_B3LYP_builtin.com') to whereever you like in your work directory, and navigate to this folder. Then type either of the following commands (assuming you put the scripts in ~/bin):

<pre>
qsub -v in=H2O_SPE_6311plgdp_GP_B3LYP_builtin -N test ~/bin/.rng_OTRSH
gf_OTRSH H2O_SPE_6311plgdp_GP_B3LYP_builtin.com
</pre>

If you saved the files somewhere else then simply modify "~/bin/.rng_OTRSH" to whatevers appropriate for the top command. For details on how the gf command works see this wiki [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:Hunt_Research_Group/pimpQSUB link].

===Quick Start Guide - The output files===
Assuming you ran the job above you will now have a loada output files in your current directory, i.e.:

H2O_SPE_6311plgdp_GP_B3LYP_builtin_gsstate_mu_opt.*

H2O_SPE_6311plgdp_GP_B3LYP_builtin_ionstate_mu_opt.*

H2O_SPE_6311plgdp_GP_B3LYP_builtin_OTRSH.out

The '''H2O_SPE_6311plgdp_GP_B3LYP_builtin_OTRSH.out''' file is essentially a progress file for how the tuning process is going. Once the calculation is over its main purpose is really to keep track of the options you used to create the OTRSH functional. Its a comma delimited file, and the headings should be pretty self-explanatory (at least they will be if you read the above section on "What is an OTRSH")

The '''H2O_SPE_6311plgdp_GP_B3LYP_builtin_gsstate_mu_opt'''* files are for single energy calculations using the OTRSH functional tuned for this particular system. If your goal was to calculate ground state properties with an OTRSH then these files might be all you need. If you want to do something other than a single-point energy calculation, then just add the relevant keywords (e.g TD) to the H2O_SPE_6311plgdp_GP_B3LYP_builtin_gsstate_mu_opt.com file - this file contains all the keywords needed to define the OTRSH functional.

The '''H2O_SPE_6311plgdp_GP_B3LYP_builtin_ionstate_mu_opt'''* files are the single point energy calculations (using the OTRSH) for the ionised system. You probably have no need for these.

You will also have a new folder named '''temp_files_H2O_SPE_6311plgdp_GP_B3LYP_builtin'''. At every tested value for <math>\mu</math>, the .log files for the ground and ionised state are written to this folder. The purpose of this folder is to check nothings going wrong during the calculation. If something stops the code extracting scf energies for each <math>\mu</math> value, then it might just loop infinitely (or at least for a loooooong time). The code '''SHOULD''' catch these sorts of errors and kill the job, but if it runs a long time without any change in the *.out file then make sure the *.log files in this folder are ok. The secondary purpose of this folder is therefore to tell you WHY your job got killed, if it got killed (an error message should be given in the .out file as well but....)

===Quick Note on Creating Input Files===

If you want to run the code, with default options, on your own systems then theres not much to say. Simply prepare a normal '''SINGLE POINT ENERGY''' *.com file for your system, but remove the functional from the input line. Also dont use the "#p" command for excess printing - this will break the OTRSH code (sorry, the code essentially uses the # for a search + replace to insert a functional at each step of the tuning process). For example, this input line is fine(and used in the example file):
<pre>
# 6-311+g(d,p) geom=connectivity int=ultrafine scf=conver=9
</pre>

Make sure you use a '''single point energy calculation input file''', using a keyword like "freq" wont throw any errors, but you'll be waiting a really long time for your job to finish.

===Understanding (And Modifying) The Default Options===

The easily changeable variables in this code are stored in the .rng_OTRSH file. They are (in order of "likelyhood of needing altering"):

<pre>
es_mult=2
alpha_val=0
beta_val=1
mu_lower=0.1
mu_upper=0.2
max_step=0.03
min_step=0.001
P5_val=0.81
xc_funct="CAM-B3LYP"
</pre>

Its likely you'll only ever need/want to change the '''es_mult''' here. This is the multiplicity of the ionised state, and is obviously equal to 2 if you have a closed shell system. If you use this code on an open shell system you need to set es_mult to the appropriate value for the ionised state. Personally i create seperate .rng files for each value of multiplicity i use, to save me editing the .rng_OTRSH file too often.

'''alpha_val''' corresponds to the fraction HF exchange at short range. Its plausible that raising this to say alpha_val=0.2 might improve your results in some cases, so it might be worth trying the functional with two different alpha values if your trying to match an experimental spectrum or have something else you can validate with. I personally set it to zero since i think its the least arbitrary value it can be.

'''beta_val''' - as stated above "alpha_val + beta_val" corresponds to the fraction HF exchange at long range. So if you '''change alpha_val, you need to change beta_val too''', otherwise you may end up with >100% HF exchange at long range. I'd advise you keep alpha_val + beta_val =1.0, since that should give you the correct long-range potential.

'''mu_lower''' and '''mu_upper''' control the initial range of <math>\mu</math> values to try out. If you have a good idea what your final <math>\mu</math> will be, then settings mu_lower and mu_upper to nearby values should speed up the optimisation process. However, there is '''NO NEED''' for the final mu_val to be between these initial search values, it will just take longer if its far away from this range.

'''max_step''' and '''min_step''' correspond to the maximum and minimum change in <math>\mu</math> between optimisation steps. Note that i think the minimum possible min_step=0.0001, so dont set it lower than that or it could cause serious problems (it'll lead to a gradient of zero between two mu values, i have no idea how the optimiser will react to that but it wont be good). Theres probably '''NEVER''' a need to mess with these values. If the optimiser isnt efficient enough, changing these values wont help much. Writing a better optimiser (the current ones pretty terrible by the way,but it works so.....) would probably be easier than working out the best values for these numbers.

'''P5_val''' corresponds to a parameter in the DFT functional controlling the amount of non-local correlation to use. Its the same P5 parameter that appears on the gaussian 09/16 guide to "user-defined DFT functionals". Just leave it alone unless you REALLY know what your doing.

'''CAM-B3LYP''' corresponds to the underlying functional to use. The functional this code generates is essentially CAM-B3LYP with modified <math>\alpha</math>,<math>\beta</math> and <math>\mu</math> parameters. The underlying DFT exchange/ DFT correlation functionals are the same as those used in CAM-B3LYP. I've never tried modifying this, it would be a massive pain. If you want to try then i suggest you use a range-separated functional as your base and start by passing the relevant arguments to the "init_OTRSH.py" functional, and seeing if you can replicate the original functional (e.g if you want to use wB97XD as your base functional, try and reproduce reslts of that functional by passing relevant options to init_OTRSH.py.

Finally, the convergence criteria for the OTRSH (the IP_HOMO - E_HOMO value) is set to 0.01 eV by default, and controlled by the IP_conver_thres variable in optimiser.py. I guess its possible you may want to relax that criteria in certain circumstances (obviously it would speed up the optimisation and lower the accuracy).

==Further Reading/Links to relevant Papers==
CAM-B3LYP paper - doi:'''10.1016/j.cplett.2004.06.011'''

Early (first?) OTRSH paper - doi:'''10.1021/ja8087482'''

Review on using OTRSH functional to calculate bandgaps (2012) - doi:'''10.1021/ct2009363'''

Use of OTRSH functionals for Ground state properties (2013) - doi:'''10.1063/1.4807325'''

File:OTRSH hpc ver V2 Mar2018.zip

2018-04-03T12:39:18Z

Rf614: Script to generate OTRSH functionals

Script to generate OTRSH functionals

Mod:Hunt Research Group/Some G09 Parsers

2017-07-29T16:08:21Z

Rf614:

== Introduction ==

This page has some parsers i've made for some different types of gaussian calculation. Meaning they are scripts which essentially pull important information from output files and store them in convenient objects. They can be useful for using as part of other scripts, for example the TDDFT matlab class is a really good starting point for writing code to generate UV-Vis spectra from gaussian 09 TD-DFT calculations. If you have no interest in coding then these parsers are totally useless to you.

Also since this is a page about parsers, i should probably mention cclib - this provides python parsers for log files from a range of computational chemistry software (including gaussian 09)[link: https://github.com/cclib/cclib]

== Links To Files ==

python molecule class parser [[Media:Mol Class Parser Jul2017 V0pt5.zip]]

matlab NBO file parser [[Media:G09 NBO Parser Matlab.zip]]

matlab AIM .sumviz file parser [[Media:AIM sumviz parser matlab.zip]]

matlab TDDFT file parser [[Media:TDDFT Classes.zip]]

== Rough description of Matlab Parsers ==

All the matlab parsers are called in the same way, by calling the relevant function with a full filepath as the argument. Assuming you set the variable "fname" to the full path of the desired file:

NBO parser: "NBO_test = NBO_E2_Class_V2(fname) "

AIM parser: "AIM_test = Sumviz_File_Obj_V2(fname)"

TDDFT parser: "TDDFT_test = TDDFT_Output_Class_V1_2014a(fname)"

Both NBO and AIM parsers also have static methods which allow you to create an object/instance for each relevant file in a given folder (these objects get stored in an array). All the parsers have various (possibly) useful methods which are documented within the code. For example, the AIM parser has a method which returns the sum of bond critical point densities for all BCPS between two user defined fragments.

== Rough description of molecule_class (Python parser) ==

This code is written in python3.5 (3.4 should work, not sure about anything earlier) and uses numpy. It pulls some basic information from *.com/*.log/*.fchk files, and is meant to be used on frequency files with a single structure (not sure if there are any issues with using it on an optimisation, ive never tried). The information it extracts depends on the type of file it reads, but in all cases a geometry is extracted. Documentation is mainly within the code, though some of its capabilities can be seen by testing the simple scripts i bundled with it (see below).

As well as Molecule_Class_V0pt5 the zip folder contains a couple of simple scripts utilising the parser, and some example files. To test the simple scripts make sure they have executable permissions ("chmod u+x filename"), are in the same folder as Molecule_Class_V0pt5 and are in your system path ("PATH=$PATH:<dir>", replace the word <dir> with the path for the folder containing the scripts.

Then navigate to the example_files directory. In terminal type:

"get_rel_energies.py ."

A file called "Rel_Energies.txt" will have been written in the current folder. This is a comma-delimited file with conformer names + there relative energies (in kJ mol).

To run the second script type:

"Min_Distances_Two_Frags.py . 1-3 4-18"

The first argument is the folder to look for .log files. The second two arguments define sub-fragments for your geometry. "1-3" corresponds to atoms 1,2,3. The script (with these arguments) finds the shortest distance between the two fragments "atoms 1-3" and "atoms 4-18", and outputs the distance in angstroms and also the identity of atoms involved.

Mod:Hunt Research Group/Some G09 Parsers

2017-07-29T16:00:04Z

Rf614: /* Links To Files */

== Introduction ==

This page has some parsers i've made for some different types of gaussian calculation. Meaning they are scripts which essentially pull important information from output files and store them in convenient objects. They can be useful for using as part of other scripts, for example the TDDFT matlab class is a really good starting point for writing code to generate UV-Vis spectra from gaussian 09 TD-DFT calculations. If you have no interest in coding then these parsers are totally useless to you.

Also since this is a page about parsers, i should probably mention cclib - this provides python parsers for log files from a range of computational chemistry software (including gaussian 09)[link: https://github.com/cclib/cclib]

== Links To Files ==

python molecule class parser [[Media:Mol Class Parser Jul2017 V0pt5.zip]]

matlab NBO file parser [[Media:G09 NBO Parser Matlab.zip]]

matlab AIM .sumviz file parser [[Media:AIM sumviz parser matlab.zip]]

matlab TDDFT file parser [[Media:TDDFT Classes.zip]]

== Rough description of Matlab Parsers ==

== Rough description of molecule_class (Python parser) ==

This code is written in python3.5 (3.4 should work, not sure about anything earlier) and uses numpy. It pulls some basic information from *.com/*.log/*.fchk files, and is meant to be used on frequency files with a single structure (not sure if there are any issues with using it on an optimisation, ive never tried). The information it extracts depends on the type of file it reads, but in all cases a geometry is extracted. Documentation is mainly within the code, though some of its capabilities can be seen by testing the simple scripts i bundled with it (see below).

As well as Molecule_Class_V0pt5 the zip folder contains a couple of simple scripts utilising the parser, and some example files. To test the simple scripts make sure they have executable permissions ("chmod u+x filename"), are in the same folder as Molecule_Class_V0pt5 and are in your system path ("PATH=$PATH:<dir>", replace the word <dir> with the path for the folder containing the scripts.

Then navigate to the example_files directory. In terminal type:

"get_rel_energies.py ."

A file called "Rel_Energies.txt" will have been written in the current folder. This is a comma-delimited file with conformer names + there relative energies (in kJ mol).

To run the second script type:

"Min_Distances_Two_Frags.py . 1-3 4-18"

The first argument is the folder to look for .log files. The second two arguments define sub-fragments for your geometry. "1-3" corresponds to atoms 1,2,3. The script (with these arguments) finds the shortest distance between the two fragments "atoms 1-3" and "atoms 4-18", and outputs the distance in angstroms and also the identity of atoms involved.

Mod:Hunt Research Group/Some G09 Parsers

2017-07-29T15:59:43Z

Rf614:

== Introduction ==

This page has some parsers i've made for some different types of gaussian calculation. Meaning they are scripts which essentially pull important information from output files and store them in convenient objects. They can be useful for using as part of other scripts, for example the TDDFT matlab class is a really good starting point for writing code to generate UV-Vis spectra from gaussian 09 TD-DFT calculations. If you have no interest in coding then these parsers are totally useless to you.

Also since this is a page about parsers, i should probably mention cclib - this provides python parsers for log files from a range of computational chemistry software (including gaussian 09)[link: https://github.com/cclib/cclib]

== Links To Files ==

python molecule class parser [[Media:Mol Class Parser Jul2017 V0pt5.zip]]

matlab NBO file parser [[Media:G09 NBO Parser Matlab.zip]]

matlab AIM .sumviz file parser [[Media:AIM sumviz parser matlab.zip]]

matlab TDDFT file parser [[Media:TDDFT Classes]]

== Rough description of Matlab Parsers ==

== Rough description of molecule_class (Python parser) ==

This code is written in python3.5 (3.4 should work, not sure about anything earlier) and uses numpy. It pulls some basic information from *.com/*.log/*.fchk files, and is meant to be used on frequency files with a single structure (not sure if there are any issues with using it on an optimisation, ive never tried). The information it extracts depends on the type of file it reads, but in all cases a geometry is extracted. Documentation is mainly within the code, though some of its capabilities can be seen by testing the simple scripts i bundled with it (see below).

As well as Molecule_Class_V0pt5 the zip folder contains a couple of simple scripts utilising the parser, and some example files. To test the simple scripts make sure they have executable permissions ("chmod u+x filename"), are in the same folder as Molecule_Class_V0pt5 and are in your system path ("PATH=$PATH:<dir>", replace the word <dir> with the path for the folder containing the scripts.

Then navigate to the example_files directory. In terminal type:

"get_rel_energies.py ."

A file called "Rel_Energies.txt" will have been written in the current folder. This is a comma-delimited file with conformer names + there relative energies (in kJ mol).

To run the second script type:

"Min_Distances_Two_Frags.py . 1-3 4-18"

The first argument is the folder to look for .log files. The second two arguments define sub-fragments for your geometry. "1-3" corresponds to atoms 1,2,3. The script (with these arguments) finds the shortest distance between the two fragments "atoms 1-3" and "atoms 4-18", and outputs the distance in angstroms and also the identity of atoms involved.

File:TDDFT Classes.zip

2017-07-29T15:59:29Z

Rf614: matlab parser for TD-DFT files from gaussian 09

matlab parser for TD-DFT files from gaussian 09

File:AIM sumviz parser matlab.zip

2017-07-29T15:58:00Z

Rf614: Matlab code to parse .sumviz files (from AIMAll)

Matlab code to parse .sumviz files (from AIMAll)

File:G09 NBO Parser Matlab.zip

2017-07-29T15:54:59Z

Rf614: Parser for g09 NBO files, written in matlab 2014a.

Parser for g09 NBO files, written in matlab 2014a.

Mod:Hunt Research Group/Some G09 Parsers

2017-07-27T20:28:22Z

Rf614: /* Rough description of molecule_class (Python parser) */

== Introduction ==

This page has some parsers i've made for some different types of gaussian calculation. Meaning they are scripts which essentially pull important information from output files and store them in convenient objects. They can be useful for using as part of other scripts, for example the TDDFT matlab class is a really good starting point for writing code to generate UV-Vis spectra from gaussian 09 TD-DFT calculations. If you have no interest in coding then these parsers are totally useless to you.

Also since this is a page about parsers, i should probably mention cclib - this provides python parsers for log files from a range of computational chemistry software (including gaussian 09)[link: https://github.com/cclib/cclib]

== Links To Files ==

python molecule class parser [[Media:Mol Class Parser Jul2017 V0pt5.zip]]

== Rough description of Matlab Parsers ==

== Rough description of molecule_class (Python parser) ==

This code is written in python3.5 (3.4 should work, not sure about anything earlier) and uses numpy. It pulls some basic information from *.com/*.log/*.fchk files, and is meant to be used on frequency files with a single structure (not sure if there are any issues with using it on an optimisation, ive never tried). The information it extracts depends on the type of file it reads, but in all cases a geometry is extracted. Documentation is mainly within the code, though some of its capabilities can be seen by testing the simple scripts i bundled with it (see below).

As well as Molecule_Class_V0pt5 the zip folder contains a couple of simple scripts utilising the parser, and some example files. To test the simple scripts make sure they have executable permissions ("chmod u+x filename"), are in the same folder as Molecule_Class_V0pt5 and are in your system path ("PATH=$PATH:<dir>", replace the word <dir> with the path for the folder containing the scripts.

Then navigate to the example_files directory. In terminal type:

"get_rel_energies.py ."

A file called "Rel_Energies.txt" will have been written in the current folder. This is a comma-delimited file with conformer names + there relative energies (in kJ mol).

To run the second script type:

"Min_Distances_Two_Frags.py . 1-3 4-18"

The first argument is the folder to look for .log files. The second two arguments define sub-fragments for your geometry. "1-3" corresponds to atoms 1,2,3. The script (with these arguments) finds the shortest distance between the two fragments "atoms 1-3" and "atoms 4-18", and outputs the distance in angstroms and also the identity of atoms involved.

Mod:Hunt Research Group/Some G09 Parsers

2017-07-27T20:14:49Z

Rf614: /* Introduction */

Mod:Hunt Research Group

2017-07-27T20:14:25Z

Rf614: /* Codes to Help Analysis */

==Hunt Group Wiki==

Back to the main [http://www.ch.ic.ac.uk/hunt web-page]
===HPC Resources===
#Computing resources available in the chemistry department [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/computing_resources link]
#HPC servers and run scripts [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/hpc link]
#Setting up a connection to HPC if you have a PC [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/hpc_connections link]
#How to fix Windows files under UNIX [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/Windowsfiles link]
#How to make ssh more comfortable [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/pimpSSH link]
#How to make qsub more comfortable (gfunc) [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/pimpQSUB link]
#How to set up a SSH keypair [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/SSHkeyfile link]
#How to use gaussview directly on the HPC [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/gview link]
#How to comfortably search through old BASH commands [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/searchbash link]
#How to connect to HPC directory on desktop for file transfers - MacFusion [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/hpc_Directory_on_desktop link]
#How to set up cx2 [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/cx2 link]

===Visualisation===
*'''density based visualisation'''
#download [http://aim.tkgristmill.com AIMALL]

*'''ESPs'''
#Instructions for visualizing electrostatic potentials (Gaussview)[https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/electrostatic_potentials link]
#Electrostatic Potentials II (Molden) [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/electrostatic_potentials_2 link]
*'''VMD: a molecular dynamics visualisation package'''
#Download VMD [//wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/download_vmd link]
#Quick reminder [https://www.ch.ic.ac.uk/wiki/index.php/Talk:Mod:Hunt_Research_Group/VMDReminder link]
#Tricks and tips [https://www.ch.ic.ac.uk/wiki/index.php/Talk:Mod:Hunt_Research_Group/VMDTips link]
#Changing the graphical representation of your structures [https://www.ch.ic.ac.uk/wiki/index.php/Talk:Mod:Hunt_Research_Group/vmd link]
#Basic visualisation of a trajectory [https://www.ch.ic.ac.uk/wiki/index.php/Talk:Mod:Hunt_Research_Group/VisualisingyourSimulation link]
#How to turn a Gaussian optimization into a VMD movie [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/VMDmovie link]
#Using scripts in VMD [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/VmdScripts link]
#Dealing with periodic boundaries and bonding (under construction) [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/VmdScriptsPeriodic link]
#Dealing with bonding (under construction) [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/VmdBonding link]
#Overlapping two structures [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/VmdVisual link]
*'''JMol'''
#Visualising MOs using Jmol [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:basic_jmol_instructions link]
#Surfaces (Solvent-Accessible and Connolly) in Jmol [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/jmolsurfaces link]
*'''EMO Code'''
#How to use Ling's emo plot code[https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/emoplot link]
#How to plot EMOs [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/emo link]
*'''PyGauss'''
#Python API for analysis of Gaussian compuations [https://pygauss.readthedocs.org - Documentation]
*'''MD Post processing'''
#Code to Recentre DL_PLOY HISTORY file [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/recentre_xyz.py link]
#Link to the code to convert the DL_POLY HISTORY file to the multi-frame XYZ file[https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/his2xyz.py link]

*'''SDFs'''
#How to generate SDFs [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/sdfs_generate link]

===Gaussian General===
#We are starting a database of common errors encountered when running Gaussian jobs [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/gaussian_errors link]
# Here is an already existing database of common errors [https://www.ace-net.ca/wiki/Gaussian_Error_Messages link]
# [http://www.ch.ic.ac.uk/hunt/g03_man/index.htm G03 Manual]
#How to run NBO5.9 on the HPC [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/NBO5.9 link]
#How to include dispersion [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/dispersion link]
#Basic ONIOM (Mechanical Embedding) [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/basiconiom link]
#M0n and DFT-D [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/DFTD link]
#IL ONIOM clusters [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/oniomclusers link]
#Molecular volume calculations [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:Hunt_Research_Group/molecular_volume link]
#problems with scf convergence [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/scf_convergence link]
#Using a z-matrix [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/z-matrix link]
#generating natural transition orbitals [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/nto link]
#Using solvent models [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/solvent link]
#Using SMD on ILs [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:Hunt_Research_Group:_Using_SMD_on_ILs link]
#computing excited state polarisabilities [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:Hunt_Research_Group:_ES_alpha link]
#computing deuterated and/or anharmonic spectra [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:Hunt_Research_Group:_Danharm link]
#manipulating checkpoint files [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:Hunt_Research_Group:usingchkfiles link]
#AimAll with pseudo potentials [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:Hunt_Research_Group:aim_pseudopotentials link]
#Script to pull thermodynamic data and low frequencies from log files [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:Hunt_Research_Group:freq_script link]
#General procedure for locating transition state structures [[link]]

===ADF General===
#Submission script [http://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/ADF_sricpt link]

===Codes to Help Analysis===
# Extract E2 Values (From NBO Calculations) [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/NBO_Matlab_Code link]
# Calculate pDoS/XP spectra code (under construction) [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/Calc_XPS_Code link]
# Codes to extract frequency data from gaussian .log files and generate vibrational spectra [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:Hunt_Research_Group:frequency_spectrum_script link]
# Optimally Tuned Range Seperated Hybrid Functionals [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:Hunt_Research_Group/OTRSH_Funct link]
# Some G09 Parsers [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:Hunt_Research_Group/Some_G09_Parsers link]

===Setup and Running Ab-Initio MD Simulations===
#CPMD: Car-Parrinello Molecular Dynamics [https://www.ch.ic.ac.uk/wiki/index.php/Talk:Mod:Hunt_Research_Group/cpmd link]
#How to run CPMD to study aqueous solutions [https://www.ch.ic.ac.uk/wiki/index.php/Talk:Mod:Hunt_Research_Group/cpmd_water link]
#How to run CP2K [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/cp2k_how link]
#[bmim]Cl using CPMD [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/bmimCl_cpmd link]
#[bmim]Cl using CP2K [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/bmimCl_cp2k link]
#mman using CPMD and Gaussian [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/mman link]
#[emim]SCN using CP2K[https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/emimscn link]
#CP2K Donts [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/cp2k link]

===Setup and Running Classical MD Simulations===
#DLPOLY a MD simulation package, Installation on an IMac (old needs to be updated) [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/dlpoly_install link]
#DL_POLY FAQs [http://www.stfc.ac.uk/cse/DL_POLY/ccp1gui/38621.aspx] from DL_POLY webpage.
#Installing Packmol
#Getting started: generating a solvated structure and "relaxing" it [https://www.ch.ic.ac.uk/wiki/index.php/Talk:Mod:Hunt_Research_Group/Starting_MD link]
#Equilibration and production simulations [https://www.ch.ic.ac.uk/wiki/index.php/Talk:Mod:Hunt_Research_Group/EquilibrationandProduction link]
#How to equilibrate an MD run[https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/equilibration link]
#Getting the Force Field [https://www.ch.ic.ac.uk/wiki/index.php/Talk:Mod:Hunt_Research_Group/Wheretostart link]
#Choosing an Ensemble [https://www.ch.ic.ac.uk/wiki/index.php/Talk:Mod:Hunt_Research_Group/Ensembles link]
#Molten Salt Simulations [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:Hunt_Research_Group/MoltenSaltSimulation link]
#Common Errors [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:Hunt_Research_Group/CommonErrors link]
#Voids in ILs[https://www.ch.ic.ac.uk/wiki/index.php/Talk:Mod:Hunt_Research_Group/voids link]
#Equilibration of [bmim][BF4] and [bmim][NO3][https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/BmimBF4_equilibration link]
#Summary of discussions with Ruth[https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/Aug09QtoRuth link]

===Running QM/MM Simulations in ChemShell===
#ChemShell official website which contains the manual and a tutorial [http://www.stfc.ac.uk/CSE/randd/ccg/36254.aspx link]
#Introduction to ChemShell - Copper in water [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/ChemShell_Introduction link]
#Defining the system: Cu2+ and its first 2 solvation shells [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/ChemShell_System_Aqeuous_Cu(II) link]
#Defining the force field parameters [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/ChemShell_Force_Field_Parameters_Aqueous_Cu(II) link]
#Single point QM/MM energy calculation [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/QMMM_SP_Aqeuous_Cu(II) link]
#QM/MM Optimisation [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/QMMM_OPT_Aqeuous_Cu(II) link]
#QM/MM Molecular Dynamics [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/QMMM_MD_Aqeuous_Cu(II) link]
#Using MolCluster [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/MolCluster link]
#Running ChemShell [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/ChemShell link]
#Explaining ChemShell files [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/ChemShell_files link]
#Step By Step [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/Chemshell_Step_By_Step link]

===Research Notes===
#Cl- in water [https://www.ch.ic.ac.uk/wiki/index.php/Talk:Mod:Hunt_Research_Group/wannier_centre link]
#The use of Legendre time correlation functions to study reorientational dynamics in liquids[https://www.ch.ic.ac.uk/wiki/index.php/Talk:Mod:Hunt_Research_Group/legendre link]
#Functional for ILs using CPMD [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/IL_cpmd_functional link]
#Solving the angular part of the Schrödinger equation for a hydrogen atom [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/angular_schrodinger link] (notes by Vincent)
#Systematic conformational scan for ion-pair dimers [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/ion_pair_scan link]
#Obtaining NBO, ESP, and RESP charges [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:Hunt_Research_Group/Charges link]

===Installing and using other packages===
#How to install POLYRATE [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/polyrate link]
#How to install Geomview [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/geomview link]
#XMGRACE, gfortran, c compilers for Lion [http://hpc.sourceforge.net/]

===Admin Stuff===
#Not used to writing a wiki, make your test runs [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/testing on this page]
#How to set-up new macs [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/mac_setup link]
#How to set-up remote desktop [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/mac_remote link]
#[https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/calendar Calendar]
#How to switch the printer HP CP3525dn duplex on and off [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/printing link]

Mod:Hunt Research Group/Some G09 Parsers

2017-07-27T20:13:57Z

Rf614:

File:Mol Class Parser Jul2017 V0pt5.zip

2017-07-27T20:11:20Z

Rf614: basic parser for gaussian 09 .fchk/.log/.com files

basic parser for gaussian 09 .fchk/.log/.com files

Mod:Hunt Research Group/Some G09 Parsers

2017-07-27T20:09:50Z

Rf614:

Mod:Hunt Research Group/Some G09 Parsers

2017-07-27T20:02:09Z

Rf614: Created page with " == Introduction == == Links To Files == == Rough description of Matlab Parsers == == Rough description of molecule_class (Python parser) =="

== Introduction ==

== Links To Files ==

== Rough description of Matlab Parsers ==

== Rough description of molecule_class (Python parser) ==

Talk:Mod:Hunt Research Group/Calc XPS Code

2017-07-27T16:33:32Z

Rf614:

==Links to Files==
Open Shell Version: [[Media:Gelius_open_shell_code_Aug2016.zip]]

Closed Shell Version: [[Media:Gelius_closed_shell_code_Aug2016.zip]]

Plot Atomic Contribs Script: [[Media:Plot Atomic Contribs V3.m]]

*MLinpt.txt File parser(Matlab): [[Media:ML inpt att2.m]]

==Introduction==
The purpose of this code is to calculate partial density of states (pDoS) plots and Valence band x-ray photoelectron (XP) spectra from .fchk files. There is both an open shell and a closed shell version of the code, they work in exactly the same way.

All code was written using python 2.7 (with numpy), Multiwfn version 3.3.8, Matlab version 2014a and a cygwin(bash) shell on windows 8. It should work on windows or linux if the correct options are set. It might not work on a mac currently due to Multiwfn not working properly (I haven't personally tested it though).

I've now also uploaded a parser for *MLinpt.txt files. Documentation is in the file itself, this is a totally non-essential code but can make writing scripts that rely on *MLinpt.txt files easier.(e.g scripts to extract HOMO energies).

==Some Theory==

===Mulliken Population Analysis===

A pretty key part to this code is the breakdown of molecular orbitals (MOs) in atomic orbitals (AOs, which are essentially basis functions). This is carried out in multiwfn using Mulliken population analysis, which i'll briefly summarise due to its importance.

The basic question is "For MO X, how many electrons come from atomic orbital Y?". Note im going to be using "basis functions" instead of "atomic orbitals" from this point, in terms of DFT calculations we interpret a basis function centered on atom X as an atomic orbital of atom X.We'll define <math>\psi</math> as the wavefunction for a '''single MO containing a single electron'''. Next we write <math>\psi</math> as the linear combination of individual basis functions (<math>\phi_{i}</math>):

<math>
\psi = {{\sum\limits_{i} c_{i} \phi_{i} }}
</math>

Since we want a number of electrons, we're interested in the formula for the electron density (<math>\rho</math>) of this MO:

<math>
\rho = \langle{} \Psi | \Psi \rangle{}
</math>

<math>
\langle{}\Psi | \Psi \rangle{} = ({{\sum\limits_{i} c_{i} \phi_{i} }})({{\sum\limits_{i} c_{i} \phi_{i} }})
</math>

<math>
\langle{}\Psi | \Psi \rangle{} = ({{\sum\limits_{i} c_{i}^{2} \phi_{i}^{2} }}) + 2 {{ \sum\limits_{i>j}}} {{\sum\limits_{j} c_{i}c_{j} \langle{}\phi_{j} | \phi_{i} \rangle{} }}
</math>

If these formulae are tricky to follow, try doing it for a system with just two basis functions.
Note that in the third line the indice ''j'' runs over the same coefficients and basis functions as the indice ''i'', its just used to show that the <math>{{ \sum\limits_{i>j}}} {{\sum\limits_{j} c_{i}c_{j} \langle{}\phi_{j} | \phi_{i} \rangle{} }}</math> term only contains products between 2 different basis functions/co-efficients.
Hence, we now have a formula for the MO density which is purely in terms of (known) basis functions and (known) coefficients. Now its just a case of working out how we should interpret these values.

Firstly note that <math>\phi_{i}^{2}=1</math> (since we use normalised basis functions) hence:

<math>
{{\sum\limits_{i} c_{i}^{2} \phi_{i}^{2} }} = {{\sum\limits_{i} c_{i}^{2}}}
</math>

The values of c_{i} in this expression are easy to interpret, they represent the fractional contribution of AO ''i'' to the wavefunction. The <math>2 {{ \sum\limits_{i>j}}} {{\sum\limits_{j} c_{i}c_{j} \langle{}\phi_{j} | \phi_{i} \rangle{} }}</math> term is more difficult to interpret, each expression in the series can be thought of as the number of electrons shared between two basis functions. i.e. the following term would represent the number of electrons shared between basis function 1 and 2:

<math>
n_{shared} = 2 c_{1}c_{2} \langle{}\phi_{1} | \phi_{2} \rangle{}
</math>

However, we need to split this "shared" electron density between the two contributing basis functions. In Mulliken Population analysis "shared" electron density is divided equally between the two contributing basis functions. So the total electron density assigned to one basis function can be written as:

<math>
n_{i} = c_{i}^{2} + {{\sum\limits_{j \neq i} c_{i}c_{j} \langle{}\phi_{j} | \phi_{i} \rangle{} }}
</math>

<math>n_{i}</math> represents the '''fractional''' contribution of ''i'' to the wavefunction. Hence if you had a 2-electron MO, you'd simply multiply <math>n_{i}</math> by 2.

===What the code actually does (roughly/briefly)===

Step 1 - The first step is to run a Mulliken population analysis on the *.fchk file. This leads to a file containing the energy/AO composition of each MO in the system (the *MLinpt.txt file). Subsequent steps vary depending on if you want partial density of states (pDoS) plots or calculated valence spectra.

If '''calculating pDoS''' (not XPS spectra):

For each type of atomic orbital present do the following:

- For each MO, center a gaussian-lorentzian function at that MO energy. Multiply the intensity by the contribution of the current AO to the current MO.

- Sum all the resultant peaks, this generates the pDOS the current atomic orbital

Once all pDoS are generated, the total DoS could be created by summing them all. In practice the code probably just calculates the total DoS first, by just placing a gaussian-lorenztian with area=1 around each MO energy.

If '''calculating XPS spectra''':

For each type of atomic orbital present do the following:

- Multiply the fractional contributions of this AO to each MO by the AO cross-section. This value will be reffered to as the "intensity" This cross-section depends on both the AO type and the photon energy. Default values in this code are stored in tables in X_Section_Library, and custom/different values can easily be added.

- For each MO, center a gaussian-lorentzian function at that MO energy. This area of this function is equal to the intensity calculated in the step above.

Sum all the resultants peaks, this generates the contribution of the current AO to the valence XPS spectrum
Once this is repeated for each AO type, sum all those individual contributions to generate the total XPS valence spectrum.

=="Installation" Instructions==

All instructions will be given for the open shell version of the code. However, the process is the same except whenever these instructions say to edit the "gen_gelius_input_opensh.sh", just do the corresponding edit on the gen_gelius_input.sh file. The Matlab scripts are EXACTLY the same for both open shell/closed shell codes.

0) Make sure you have multiwfn (http://sobereva.com/multiwfn/) saved somewhere, python 2.x and numpy installed.
:a)On ubuntu Multiwfn needs an extra library, type the following line:
sudo apt-get install libmotif-dev

1)Put the following files (all the .py/.sh files) somewhere in system path (e.g i use /usr/local/bin):
:a)Format_Multiwfn_Output_alpha.py (Format_Multiwfn_Output.py for closed shell)
:b)Format_Multilwfn_Output_beta.py
:c)Gen_Multiwfn_Input_Comms.py
:d)Gen_Multiwfn_Input_Comms_beta.py
:e)Merge_Fragment_Files.py
:f)gen_gelius_input_opensh.sh

Make sure all files (including Multiwfn) have executable persmission (navigate to the folder and do "chmod u+x filename")

2)In the gen_gelius_input_opensh.sh change the current lines accordingly:
:a) multiwfn_folder = 'C:/Users/Richard/Documents/Multiwfn_Installed' to the directory where you store multiwfn and associated files.
:b) "cygwin=1” to “cygwin=0” if your not using a Cygwin shell.
:c) “old_sed=1” to “old_sed=0” if your version of sed allows the –i option (edit in place) to be used. I don’t think the standard version on mac does(meaning set old_sed=1),standard version of Cygwin does though.
:d) If on linux "cp $multiwfn_folder/Multiwfn.exe $curr_folder" to "cp $multiwfn_folder/Multiwfn $curr_folder" . Do the same for the line where multiwfn is deleted from current folder("rm Multiwfn.exe").

3)Put the following files somewhere in matlabs path (e.g your matlab folder in documents)
:a) Get_X_Section.m
:b) Gen_Gelius_Spectrum.m

The Get_X_Section REALLY does need to be in matlabs path (unless you want to edit some code in Gen_Gelius_Spectrum.m). If you know what your doing in matlab feel free to put Gen_Gelius_Spectrum.m wherever you want.

4)Put the X_Section_Library folder wherever you want.

5) In Get_X_Section.m modify the following line:
:"Lib_Path ='C:\Users\Richard\Documents\X_Section_Library' " to the path where you put the X_Section_Library folder in.

6) OPTIONAL: Add the code in vim_template_script to your vimrc (c+p it somewhere in the file). "vim ~/.vimrc" to access your vimrc. You can then bring up a template input file in vim by using ":call Gelius_Input()"
Since i forgot to put the relevant text file in to the code folders, just c+p the below into your vimrc to get this to work.

<pre>
function! Gelius_Input()
exe 'normal aCoSCN4_freq_AllNCS_B3LYP_6311plgdp_Tetrahed_BMIMSCNSMD_Abr.fchk\rNumb_Fragments=10\r1a=\r1t=a\r1l=TDOS\r2a=\r2t=\r2l=\r3a=\r3t=\r3l=\r4a=\r4t=\r4l=\r5a=\r5t=\r5l=\r6a=\r6t=\r6l=\r7a=\r7t=\r7l=\r8a=\r8t=\r8l=\r9a=\r9t=\r9l=\r10a=\r10t=\r10l=\rM_factor=1.0\rLink_File='
exe '%s/\\r/\r/g'
endfunction
</pre>

==Running The Code==
====Using the Example files====
1) Navigate (in your shell) to the folder "Example_files". e.g for me i ran:

"cd /cygdrive/c/Users/Richard/Documents/Random_Coding/Gelius_Spectrum_Automated_Jan_2016/open_shell_code/Example_files"

2) Run this command in console(for closed shell use gen_gelius_input.sh):

"gen_gelius_input_opensh.sh"

Command 2) will have created a folder for each .fchk file in Example_files/. Therefore you should now have folders:
:CoSCN4_freq_AllNCS_B3LYP_6311plgdp_Tetrahed_BMIMSCNSMD_Abr
:other_file
Within these folders are tons of output files. The most important ones are those ending in "MLinpt.txt" - these will be the input files for step 3.

The script will run on all .fchk files in the current folder, when you do this for real you need to make sure that all the fchks are for the same system (e.g a water molecule) and that the atomic numbering is the same in all files (view-->labels in gaussview gives you what im calling atomic numbering).

If anything went wrong try revisiting the installation instructions.

3) Everything from now on is done using matlab. Theres a file called "pathname_file" in the Example_files folder which was created by the gen_gelius_input_opensh.sh script. Copy its full contents to clipboard and in MATLAB terminal type:

<pre>
plist=<CTRL-V>
(<CTRL-V> means paste, rather than type out CTRL-V)
</pre>

In matlab terms, plist now contains a cell array where each entry is the full path to a folder. The folder paths are those for the folders created in step 2). Hence to do this manually for just one of the two folders you'd type something like:
<pre>
plist={'C:\Users\Richard\Documents\Random_Coding\Gelius_Spectrum_Automated_Jan_2016\open_shell_code\Example_files\other_file'}
</pre>

4) To create either an XPS spectrum or pDoS plots we use the Gen_Gelius_Spectrum script. This takes the following three arguments IN THE FOLLOWING ORDER:
:a) the plist variable we just made
:b) the photon energy we want to use (any negative number requests a pDoS rather than an XPS spectrum)
:c) the FWHM of the peaks to place around each MO energy (units = eV)

So to create a pDoS plot you could type the following into the matlab terminal(Try this one first):
<pre>
Gen_Gelius_Spectrum(plist,-1,1)
</pre>
To create an XPS spectrum taken at photon energy of 1486eV (standard lab-based value) type the following:
<pre>
Gen_Gelius_Spectrum(plist,1486,1)
</pre>

If you get an error like "Undefined function 'Gen_Gelius_Spectrum' for input arguments of type 'cell'.", then your Gen_Gelius_Spectrum script probably isnt in your matlab path. (see installation instructions, or google a bit)

If your error free then well done. Sub-folders should have been created in "other_file" and "CoSCN4_freq_AllNCS_B3LYP_6311plgdp_Tetrahed_BMIMSCNSMD_Abr", which in turn will contain a loada output files. See the output files section of the wiki for an explanation of what these are.

====Creating your own input files====
It can be named anything (except a substring of the fchk file name). The extension must be .test.

If you followed the optional part above and made a vimscript, then a template for the input file can be brought up in vim by typing ":call Gelius_Input()"

An example input file is shown below. Each "fragment" (e.g cation,anion, alkyl group etc.) should have its own input file.
<pre>
other_file.fchk
Numb_Fragments=4
1a=10-13
1t=a
1l=TDOS
2a=10-13
2t=s
2l=S3s
3a=10-13
3t=p
3l=S3p
4a=10-13
4t=d+
4l=S3d
M_factor=1.0
Link_File=all_frags.test
</pre>
The input file can be divided into parts as follows:

1) The top line of the input file tells the code the name of the checkpoint file to use in the calculation. In practice you can write ANYTHING on this top line if using the gen_gelius_input*.sh script, as the script itself replaces the first line (whatevers there) with the names of all .fchks in the current folder. But SOMETHING must be written there.

2)'''Numb_Fragments''' is the number of sub-fragments (max 10) that you want to break your pDoS down into (e.g fragment 2 corresponds to all Sulfur 3s atomic orbitals in the example above). Code will break if you get this wrong.

3) '''1a/1t/1l''' instruct the code what to use as the total density of states. Hence 1a must be set to ALL the atoms your interested in for this fragment, 1t must be set to "a" and 1l should be set to TDOS (not sure it matters what this is set to, but to be on the safe side just use TDOS). The purpose of this fragment is essentially error checking.

4) '''2a/3a/*a''' : a stands for atoms here, so the value of a corresponds to the labels of the atoms you want to be included in the sub-fragment. These labels are the same integers you get in gaussview when you go to view-->labels. Examples of valid options may be "1,4,5-7" or "1,3,5" (basically comma seperated list of integers)

5) '''2t/3t/*t''': t stands for "type of AO".This is a comma seperated list with possible arguments (any combination) "a,s,p,d,f,g". "a" corresponds to all types of AO centered of the atoms specified with the 2a/3a/*a line. s means s-type atomc orbitals (0 angular momentum), p means p-type atomic orbitals (1 angular momentum) etc. To get all AOs of a certain angular momentum and higher (up to g-type AOs) add a plus to the end of the relevant letter. e.g "p+" says "include all AOs with angular momemtnum greater than or equal to 1 in this sub-fragment")
'''DON'T''' use "a" with any other letter, and dont do anything else that will lead to a type of AO being requested twice (e.g p+,d)

6) '''2l/3l/*l''': l stands for label. If you dont care about calculating an XPS spectrum you can essentially set it to whatever you want. This tells the matlab script what cross section to use for calculating the XPS spectrum, hence it must match (case insensitive) a filename in the X_Section_Library folder. e.g S3s tells matlab to get the S3S.txt file when looking for what cross-section to use.

7) '''M_factor=1.0''', you really never need to modify this but make sure its present. It's purpose is to tell matlab to multiple the pDoS of this fragment (that this input file corresponds to) by the value M_factor. The purpose is for systems where the real stoichiometry differs from that in your calculation. For example if i have a mixture of 99 water:1 anion but only a calculation on 1 water molecule mixed with 1 anion i could set M_factor for the water fragment input file to M_factor=99.0 to estimate (badly) what the actual DoS of the mixture would look like.

8) '''Link_File=*'''. This options for if you have a fragment (e.g an anion) but want it broken into more than 10 sub-fragments (e.g this will be true for any molecule with >3 elements). In this case make 2 files for that fragment, and in the second file set Link_File to the name of the first file. In the example above, this input file will essentially be merged with "all_frags.test", and only an output file for all_frags.test will be generated.

====More complex input file options====
====Gen_Gelius_Spectrum.m====
This code has the following options that can be changed (all are explained by comments in the .m file):
<pre>1)min_e/max_e
</pre>
Matlab ignores any orbitals with energy below min_e or above max_e when generating the XPS spectrum. Note if you want to change this to include core orbitals or very high energy virtual orbitals then you'll also have to edit Gen_Multiwfn_Input_Comms.py to set the range in multiwfn to the one you want.(Theres currently no variable in that .py code to do that, it just uses the defaults in multiwfn. But the function you'll need to edit in Gen_Multiwfn_Input_Comms.py is Output_Header.
<pre>2)min_e_plotted/max_e_plotted
</pre>
These refer to the minimum/maximum binding energies to calculate spectral intensity for. The range should probably be slightly wider than the min_e/max_e range really, though thats not the default setting.
<pre>3)step_size
</pre>
The value (in eV) between calculated data points in the output spectra. For example, if set to 0.001eV then a data point will be calculated for 0eV,0.001eV,0.002eV etc. Lower values=larger files/slower runtimes.
<pre>4)lorentz_fraction
</pre>
Value between 0 and 1 that determines the % Lorentzian character of the lineshape used in the broadening function(the remaining % is set to gaussian). By broadening function i essentially mean the peak thats placed around every MO energy in generating these spectra. lorentz_fraction=0 will give you a pure gaussian peak, while lorentz_fraction=1 gives a pure Lorentzian peak.
<pre>5)savetext/figsave/savejpeg/all_contribs_output
</pre>
These all control whether a certain type of output file is saved, with variable names that should be self-explanatory.Values of 1 mean save the file, other values mean dont save. set savetext=0 to not save any .txt output files, figsave=0 to not save any .fig output, savejpeg=0 to not save any .jpeg outputs and all_contribs_output=0 to not save the .txt file with a full breakdown of the spectrum. Leaving them all as 1 (except maybe the savejpeg) is strongly recommended.

====Plot_Atomic_Contribs_V3====

This is an optional script that plots certain breakdowns of your calculated DoS/XPS spectrum (i.e you run it AFTER Gen_Gelius_Spectrum) . Its not too hard to do the same thing in excel using the *ALL_Contribs* output file (see output file sections) but this is a lot faster. There are three types of breakdown this function can do (see pictures below):

1) Atomic breakdown with fragments kept seperate (merge_spd=1,merge_fragments=0)

2) Atomic orbital breakdown (merge_spd=0,merge_fragments=1)

3) Atomic breakdown (merge_spd=1,merge_fragments=1)

The function in its uploaded form takes a single argument , though there are two variables in the actual code that you need to mess with (merge_spd and merge_fragments).

So for example, to run this on the output file from the "Using the Example files" i typed into the matlab terminal:

Plot_Atomic_Contribs_V3('C:\Users\Richard\Desktop\Open_Shell_Code_Some_Stuff\Example_files\other_file\DoS_1ev_0pt3l')

The argument needs to be a folder containing the output files from the Gen_Gelius_Spectrum script. You'll then get a saved graph with the breakdown you asked for and also a .txt file containing the breakdown in a comma delimited format. The output file names record the options you used to make the script (e.g "Plotted_Atomic_Contribs_spd0_frag1.txt" means merge_spd was set to 0 and merge_fragments was set to 1).

[[File:Plotted Atomic Contribs spd1 frag1.jpg|thumbnail|center| merge_spd=1,merge_fragments=1 ]]

[[File:Plotted_Atomic_Contribs_spd0_frag1.jpg|thumbnail|center| merge_spd=0,merge_fragments=1 ]]

[[File:Plotted Atomic Contribs spd1 frag0.jpg|thumbnail|center| merge_spd=1,merge_fragments=0 ]]

==Output Files==

The first "output file" is the *MCRF*.txt file, which just contains the commands to build the required fragment in multiwfn.

===Files From Multiwfn===
Most people should be able to completely ignore these files. But in case your curious:

'''1) *DOS_curve*.txt''' Essentially a DoS plot. 1st column is energies in au, second column is TDOS of the fragment. Not really sure what the 3rd column is. No particular use for this file, but it gets auto-generated by multiwfn and i dont go out my way to delete it.

'''2) *DOS_line*.txt''' Data to create a stick plot (where each MO is a stick). Plot column 1 vs column 2 as a line graph and you get an axis with sticks representing the MOs. 4th column onwards represent contribution of the various sub-fragments to each "stick". This is the file that gets used to generate the *ML_inpt.txt file.

'''3) *DOS_frag*.txt ''' This file contains the sub-fragment info in a way multiwfn understands. If its renamed to DOS_frag.txt then multiwfn can import it to get the fragments (therefore can be useful for debugging, no further purpose).

===MLinpt.txt File===

Creation of the *ML_inpt.txt files is essentially the point of the gen_gelius_input* script. These are primarily input files for matlab, but they do have useful information in an easy to read format. If you import the file into excel and COMMA DELIMIT ONLY (i.e untick the space delimit option) you can get something like below(there are more decimal places in the actual file):

<pre>
#M_FACTOR=1.0,
#Energy(eV) TDOS CO4S CO3P CO3D N2S N2P N2D C2S C2P C2D S3S S3P S3D
[LINES DELETED]
-24.5 0.9 0.0 0.0 0.0 0.5 0.1 0.0 0.2 0.1 0.0 0.1 0.0 0.0
-24.4 1.0 0.0 0.0 0.0 0.5 0.1 0.0 0.3 0.1 0.0 0.0 0.0 0.0
-24.4 1.0 0.0 0.0 0.0 0.5 0.1 0.0 0.3 0.1 0.0 0.0 0.0 0.0
-24.4 1.0 0.0 0.0 0.0 0.5 0.1 0.0 0.3 0.1 0.0 0.0 0.0 0.0
-19.8 0.3 0.0 0.0 0.0 0.1 -0.1 0.0 0.2 0.1 0.0 0.7 0.1 0.0
-19.7 0.3 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.1 0.0 0.7 0.1 0.0
[LINES DELETED]
</pre>

The shown file is "all_frags_alpha_MLinpt.txt", which is generated from the example files in the open shell code. As this actually comes from merging two MLinpt.txt files (gen_gelius_input.sh does this part silently) the TDOS in this case doesnt mean anything.

The first column lists the energy of each MO in eV (1 ev ~27.212 au). For any row, columns 3(CO4S)-->end (S3D) list the fractional contributions of each type of atmoic orbital to the MO in that row. For example the MO with energy -19.7 eV is mainly from S3s type atomic orbitals. These fractions were determiend by multiwfn by using Mulliken population analysis.

All the fractional contributions in the file get multiplied by the M_FACTOR value (as explained above in the Creating your own input files section). Therefore the value of M_FACTOR can easily be set after the MLinpt.txt file is created, as opposed to within the initial input file.

===Formatted_UMOs===

The default for the gen_gelius_input* script (i.e the settings for the version i uploaded) is to put only OCCUPIED MO energies/breakdowns into the *MLinpt.txt file. If you want all MOs (I really DONT recommend this) in the MLinpt.txt file then change all_orbs='cheese' to all_orbs='ALLORBS' at the top of the gen_gelius_input* script.

Otherwise you will get a *Formatted_UMOs* file for every MLinpt.txt file you get. The format of the file is identical to the MLinpt.txt format except the MO at the top of the file corresponds to the LUMO.

WARNING: If you use the link_files feature, then the output formatted_UMOs files DO NOT get merged. This shouldnt matter as you should never really use the formatted UMOs for anything.

===Output From Matlab Code ===

The matlab code firstly will create a new folder with a name reflecting the input arguments. The format is hv_FWHM_%lorenztian where hv is the photon energy, FWHM is full width half maximum of the broadeing function and %lorenztian is the amount of lorenztian mixed in the broadening function. For example the earlier command (from using example files section) "Gen_Gelius_Spectrum(plist,-1,1)" generates the folder "DoS_1ev_0pt3l". DoS = Density of states, and means we did not use photoionisation cross sections. The 0pt3l part means our broadening function was 30% lorenztian and 70% Gaussian.

Two .fig files will have been produced from this code, these contain matlab graphs of the output. These .fig files follow the naming ("filename" is the name of the original fchk file):

1) filename_DoS.fig contains the total DoS plot.

2) filename_DoS_frag_contribs.fig contains the DoS AND the pDoS of each of the fragments. For closed shell code you get 1 fragment per input file (assuming you dont use link_files), but for the open shell code each input file leads to two fragments - one for alpha electrons and one for beta electrons.

[[File:Other_file_DoS.jpeg|thumb|center| 1) Total Density of States for a Co(SCN)4 fragment with filename other_file_DoS.fig]]

[[File:Other file DoS frag contribs.jpeg|thumb|center| 2) Contribtuions of alpha and beta electrons to the Total Density of States for Co(SCN)4, filename other_file_DoS_frag_contribs.fig]]

The filename_DoS.fig plot data is output in a .txt file of the same name, while each fragment in filename_DoS_frag_contribs.fig has its own .txt file. For the example above the two fragments have names "all_frags_alpha_other_file_DoS" and "all_frags_beta_other_file_DoS". The start of the names are taken from the name of the original input file "all_frags.test".

The final file output is the "ALL_Contribs_other_file_DoS.txt" file. This(as the name implies) contains all contributions to the DoS spectra in a comma delimited file. The Plot_Atomic_Contribs*.m code can be run on this file to generate plots with a few different breakdowns (e.g atomic breakdowns) - see the relevant section of this wiki (not done yet....). The file will look something like below (commas replaced with tabs):

<pre>
#all_frags_alpha all_frags_beta
#Sub fragments in each= 12 12
#Energy(eV) CO4S CO3P CO3D N2S N2P N2D C2S C2P C2D [COLUMNS REMOVED]
[ROWS REMOVED,numbers rounded]
7.5 0.00 0.01 0.31 0.04 0.13 0.00 0.00 0.11 0.00 [COLUMNS REMOVED]
7.51 0.00 0.01 0.33 0.04 0.13 0.00 0.00 0.11 0.00 [COLUMNS REMOVED]
7.52 0.00 0.01 0.34 0.04 0.13 0.00 0.00 0.11 0.00 [COLUMNS REMOVED]
7.53 0.00 0.01 0.35 0.04 0.14 0.00 0.00 0.12 0.00 [COLUMNS REMOVED]
7.54 0.00 0.01 0.36 0.04 0.14 0.00 0.00 0.12 0.00 [COLUMNS REMOVED]
7.55 0.00 0.02 0.37 0.04 0.14 0.00 0.00 0.12 0.00 [COLUMNS REMOVED]
7.56 0.00 0.02 0.39 0.05 0.14 0.00 0.00 0.13 0.00 [COLUMNS REMOVED]
[ROWS REMOVED]
</pre>

Hopefully the format is mostly intuitive, the top two rows contain information on what the data is, and the rest contain the data. The top row contains (originally comma delimited) a list of all "fragments" names, which is the number of input files for the closed shell code and 2x the number of input files for the open shell code. The second row contains the number of sub-fragments (which should be atomic orbitals) in each fragment. In the example above each fragment had 12 types of atomic orbital.

So for the example above column 2-14 shows contributions from "all_frags_alpha" while column 15-27 shows contributions from "all_frags_beta" (column 1 lists energies).

==Bugs/Things to Watch For==
===Fragment Names===
Make sure your fragment names arent a substring of your .fchk file names. For example an input file called "Cl.test" wont work for a file called "BMIM_Cl.fchk". However, "Cl_frag.test" will work. Therefore personally i add _frag to the end of every input file name. The other way to sort this is to set linked_files=0 in gen_gelius_input_opensh.sh, sincce the link files part of the code is where this bug comes from.

===Fragment Error Checking===
Unless you manually turn it off (The "error=" option in gen_gelius_input_opensh.sh script) the code tries to check your input file for errors. Specifically it tests whether the TDOS in an input_file really is the sum of the other sub-fragments in the input(.test) file. E.g it will trigger if you request only Sulfur p-type and d-type AOs (but no s-type AOs) in the input file, or if you request all sulfur p-type AOs twice. If the error check triggers then it BREAKS the input file for matlab by putting an error msg in the first line of the '''*MLinpt.txt file'''. An example of what the error looks like is:
<pre>
“Fragment Sum incorrect on row 19 for fchk file xxxx.fchk”
</pre>

If you *MLinpt.txt files arent working in matlab then this is probably the reason.

===Pathlengths===
This code makes a lot of sub-folders based on the name of .fchk files. If you have really long .fchk file names its likely that you'll end up with some files which exceed the 256 character limit for a pathlength on a windows computer. In matlab 2014 this will cause the code to exit with an error, saying it cant save the .fig files. In newer matlab versions it may just save the output files , but you wont be able to open them on windows if the path length is greater than 256 characters. Ways to avoid this are to shorten .fchk file names where possible, or (less practical) run the code on a folder as close as possible to root (i.e a folder with a short a pathlength as possible).

Talk:Mod:Hunt Research Group/Calc XPS Code

2017-07-27T16:29:32Z

Rf614: /* Links to Files */

==Links to Files==
Open Shell Version: [[Media:Gelius_open_shell_code_Aug2016.zip]]

Closed Shell Version: [[Media:Gelius_closed_shell_code_Aug2016.zip]]

Plot Atomic Contribs Script: [[Media:Plot Atomic Contribs V3.m]]

*MLinpt.txt File parser(Matlab): [[Media:ML inpt att2.m]]

==Introduction==
The purpose of this code is to calculate partial density of states (pDoS) plots and Valence band x-ray photoelectron (XP) spectra from .fchk files. There is both an open shell and a closed shell version of the code, they work in exactly the same way.

All code was written using python 2.7 (with numpy), Multiwfn version 3.3.8, Matlab version 2014a and a cygwin(bash) shell on windows 8. It should work on windows or linux if the correct options are set. It might not work on a mac currently due to Multiwfn not working properly (I haven't personally tested it though).

==Some Theory==

===Mulliken Population Analysis===

A pretty key part to this code is the breakdown of molecular orbitals (MOs) in atomic orbitals (AOs, which are essentially basis functions). This is carried out in multiwfn using Mulliken population analysis, which i'll briefly summarise due to its importance.

The basic question is "For MO X, how many electrons come from atomic orbital Y?". Note im going to be using "basis functions" instead of "atomic orbitals" from this point, in terms of DFT calculations we interpret a basis function centered on atom X as an atomic orbital of atom X.We'll define <math>\psi</math> as the wavefunction for a '''single MO containing a single electron'''. Next we write <math>\psi</math> as the linear combination of individual basis functions (<math>\phi_{i}</math>):

<math>
\psi = {{\sum\limits_{i} c_{i} \phi_{i} }}
</math>

Since we want a number of electrons, we're interested in the formula for the electron density (<math>\rho</math>) of this MO:

<math>
\rho = \langle{} \Psi | \Psi \rangle{}
</math>

<math>
\langle{}\Psi | \Psi \rangle{} = ({{\sum\limits_{i} c_{i} \phi_{i} }})({{\sum\limits_{i} c_{i} \phi_{i} }})
</math>

<math>
\langle{}\Psi | \Psi \rangle{} = ({{\sum\limits_{i} c_{i}^{2} \phi_{i}^{2} }}) + 2 {{ \sum\limits_{i>j}}} {{\sum\limits_{j} c_{i}c_{j} \langle{}\phi_{j} | \phi_{i} \rangle{} }}
</math>

If these formulae are tricky to follow, try doing it for a system with just two basis functions.
Note that in the third line the indice ''j'' runs over the same coefficients and basis functions as the indice ''i'', its just used to show that the <math>{{ \sum\limits_{i>j}}} {{\sum\limits_{j} c_{i}c_{j} \langle{}\phi_{j} | \phi_{i} \rangle{} }}</math> term only contains products between 2 different basis functions/co-efficients.
Hence, we now have a formula for the MO density which is purely in terms of (known) basis functions and (known) coefficients. Now its just a case of working out how we should interpret these values.

Firstly note that <math>\phi_{i}^{2}=1</math> (since we use normalised basis functions) hence:

<math>
{{\sum\limits_{i} c_{i}^{2} \phi_{i}^{2} }} = {{\sum\limits_{i} c_{i}^{2}}}
</math>

The values of c_{i} in this expression are easy to interpret, they represent the fractional contribution of AO ''i'' to the wavefunction. The <math>2 {{ \sum\limits_{i>j}}} {{\sum\limits_{j} c_{i}c_{j} \langle{}\phi_{j} | \phi_{i} \rangle{} }}</math> term is more difficult to interpret, each expression in the series can be thought of as the number of electrons shared between two basis functions. i.e. the following term would represent the number of electrons shared between basis function 1 and 2:

<math>
n_{shared} = 2 c_{1}c_{2} \langle{}\phi_{1} | \phi_{2} \rangle{}
</math>

However, we need to split this "shared" electron density between the two contributing basis functions. In Mulliken Population analysis "shared" electron density is divided equally between the two contributing basis functions. So the total electron density assigned to one basis function can be written as:

<math>
n_{i} = c_{i}^{2} + {{\sum\limits_{j \neq i} c_{i}c_{j} \langle{}\phi_{j} | \phi_{i} \rangle{} }}
</math>

<math>n_{i}</math> represents the '''fractional''' contribution of ''i'' to the wavefunction. Hence if you had a 2-electron MO, you'd simply multiply <math>n_{i}</math> by 2.

===What the code actually does (roughly/briefly)===

Step 1 - The first step is to run a Mulliken population analysis on the *.fchk file. This leads to a file containing the energy/AO composition of each MO in the system (the *MLinpt.txt file). Subsequent steps vary depending on if you want partial density of states (pDoS) plots or calculated valence spectra.

If '''calculating pDoS''' (not XPS spectra):

For each type of atomic orbital present do the following:

- For each MO, center a gaussian-lorentzian function at that MO energy. Multiply the intensity by the contribution of the current AO to the current MO.

- Sum all the resultant peaks, this generates the pDOS the current atomic orbital

Once all pDoS are generated, the total DoS could be created by summing them all. In practice the code probably just calculates the total DoS first, by just placing a gaussian-lorenztian with area=1 around each MO energy.

If '''calculating XPS spectra''':

For each type of atomic orbital present do the following:

- Multiply the fractional contributions of this AO to each MO by the AO cross-section. This value will be reffered to as the "intensity" This cross-section depends on both the AO type and the photon energy. Default values in this code are stored in tables in X_Section_Library, and custom/different values can easily be added.

- For each MO, center a gaussian-lorentzian function at that MO energy. This area of this function is equal to the intensity calculated in the step above.

Sum all the resultants peaks, this generates the contribution of the current AO to the valence XPS spectrum
Once this is repeated for each AO type, sum all those individual contributions to generate the total XPS valence spectrum.

=="Installation" Instructions==

All instructions will be given for the open shell version of the code. However, the process is the same except whenever these instructions say to edit the "gen_gelius_input_opensh.sh", just do the corresponding edit on the gen_gelius_input.sh file. The Matlab scripts are EXACTLY the same for both open shell/closed shell codes.

0) Make sure you have multiwfn (http://sobereva.com/multiwfn/) saved somewhere, python 2.x and numpy installed.
:a)On ubuntu Multiwfn needs an extra library, type the following line:
sudo apt-get install libmotif-dev

1)Put the following files (all the .py/.sh files) somewhere in system path (e.g i use /usr/local/bin):
:a)Format_Multiwfn_Output_alpha.py (Format_Multiwfn_Output.py for closed shell)
:b)Format_Multilwfn_Output_beta.py
:c)Gen_Multiwfn_Input_Comms.py
:d)Gen_Multiwfn_Input_Comms_beta.py
:e)Merge_Fragment_Files.py
:f)gen_gelius_input_opensh.sh

Make sure all files (including Multiwfn) have executable persmission (navigate to the folder and do "chmod u+x filename")

2)In the gen_gelius_input_opensh.sh change the current lines accordingly:
:a) multiwfn_folder = 'C:/Users/Richard/Documents/Multiwfn_Installed' to the directory where you store multiwfn and associated files.
:b) "cygwin=1” to “cygwin=0” if your not using a Cygwin shell.
:c) “old_sed=1” to “old_sed=0” if your version of sed allows the –i option (edit in place) to be used. I don’t think the standard version on mac does(meaning set old_sed=1),standard version of Cygwin does though.
:d) If on linux "cp $multiwfn_folder/Multiwfn.exe $curr_folder" to "cp $multiwfn_folder/Multiwfn $curr_folder" . Do the same for the line where multiwfn is deleted from current folder("rm Multiwfn.exe").

3)Put the following files somewhere in matlabs path (e.g your matlab folder in documents)
:a) Get_X_Section.m
:b) Gen_Gelius_Spectrum.m

The Get_X_Section REALLY does need to be in matlabs path (unless you want to edit some code in Gen_Gelius_Spectrum.m). If you know what your doing in matlab feel free to put Gen_Gelius_Spectrum.m wherever you want.

4)Put the X_Section_Library folder wherever you want.

5) In Get_X_Section.m modify the following line:
:"Lib_Path ='C:\Users\Richard\Documents\X_Section_Library' " to the path where you put the X_Section_Library folder in.

6) OPTIONAL: Add the code in vim_template_script to your vimrc (c+p it somewhere in the file). "vim ~/.vimrc" to access your vimrc. You can then bring up a template input file in vim by using ":call Gelius_Input()"
Since i forgot to put the relevant text file in to the code folders, just c+p the below into your vimrc to get this to work.

<pre>
function! Gelius_Input()
exe 'normal aCoSCN4_freq_AllNCS_B3LYP_6311plgdp_Tetrahed_BMIMSCNSMD_Abr.fchk\rNumb_Fragments=10\r1a=\r1t=a\r1l=TDOS\r2a=\r2t=\r2l=\r3a=\r3t=\r3l=\r4a=\r4t=\r4l=\r5a=\r5t=\r5l=\r6a=\r6t=\r6l=\r7a=\r7t=\r7l=\r8a=\r8t=\r8l=\r9a=\r9t=\r9l=\r10a=\r10t=\r10l=\rM_factor=1.0\rLink_File='
exe '%s/\\r/\r/g'
endfunction
</pre>

==Running The Code==
====Using the Example files====
1) Navigate (in your shell) to the folder "Example_files". e.g for me i ran:

"cd /cygdrive/c/Users/Richard/Documents/Random_Coding/Gelius_Spectrum_Automated_Jan_2016/open_shell_code/Example_files"

2) Run this command in console(for closed shell use gen_gelius_input.sh):

"gen_gelius_input_opensh.sh"

Command 2) will have created a folder for each .fchk file in Example_files/. Therefore you should now have folders:
:CoSCN4_freq_AllNCS_B3LYP_6311plgdp_Tetrahed_BMIMSCNSMD_Abr
:other_file
Within these folders are tons of output files. The most important ones are those ending in "MLinpt.txt" - these will be the input files for step 3.

The script will run on all .fchk files in the current folder, when you do this for real you need to make sure that all the fchks are for the same system (e.g a water molecule) and that the atomic numbering is the same in all files (view-->labels in gaussview gives you what im calling atomic numbering).

If anything went wrong try revisiting the installation instructions.

3) Everything from now on is done using matlab. Theres a file called "pathname_file" in the Example_files folder which was created by the gen_gelius_input_opensh.sh script. Copy its full contents to clipboard and in MATLAB terminal type:

<pre>
plist=<CTRL-V>
(<CTRL-V> means paste, rather than type out CTRL-V)
</pre>

In matlab terms, plist now contains a cell array where each entry is the full path to a folder. The folder paths are those for the folders created in step 2). Hence to do this manually for just one of the two folders you'd type something like:
<pre>
plist={'C:\Users\Richard\Documents\Random_Coding\Gelius_Spectrum_Automated_Jan_2016\open_shell_code\Example_files\other_file'}
</pre>

4) To create either an XPS spectrum or pDoS plots we use the Gen_Gelius_Spectrum script. This takes the following three arguments IN THE FOLLOWING ORDER:
:a) the plist variable we just made
:b) the photon energy we want to use (any negative number requests a pDoS rather than an XPS spectrum)
:c) the FWHM of the peaks to place around each MO energy (units = eV)

So to create a pDoS plot you could type the following into the matlab terminal(Try this one first):
<pre>
Gen_Gelius_Spectrum(plist,-1,1)
</pre>
To create an XPS spectrum taken at photon energy of 1486eV (standard lab-based value) type the following:
<pre>
Gen_Gelius_Spectrum(plist,1486,1)
</pre>

If you get an error like "Undefined function 'Gen_Gelius_Spectrum' for input arguments of type 'cell'.", then your Gen_Gelius_Spectrum script probably isnt in your matlab path. (see installation instructions, or google a bit)

If your error free then well done. Sub-folders should have been created in "other_file" and "CoSCN4_freq_AllNCS_B3LYP_6311plgdp_Tetrahed_BMIMSCNSMD_Abr", which in turn will contain a loada output files. See the output files section of the wiki for an explanation of what these are.

====Creating your own input files====
It can be named anything (except a substring of the fchk file name). The extension must be .test.

If you followed the optional part above and made a vimscript, then a template for the input file can be brought up in vim by typing ":call Gelius_Input()"

An example input file is shown below. Each "fragment" (e.g cation,anion, alkyl group etc.) should have its own input file.
<pre>
other_file.fchk
Numb_Fragments=4
1a=10-13
1t=a
1l=TDOS
2a=10-13
2t=s
2l=S3s
3a=10-13
3t=p
3l=S3p
4a=10-13
4t=d+
4l=S3d
M_factor=1.0
Link_File=all_frags.test
</pre>
The input file can be divided into parts as follows:

1) The top line of the input file tells the code the name of the checkpoint file to use in the calculation. In practice you can write ANYTHING on this top line if using the gen_gelius_input*.sh script, as the script itself replaces the first line (whatevers there) with the names of all .fchks in the current folder. But SOMETHING must be written there.

2)'''Numb_Fragments''' is the number of sub-fragments (max 10) that you want to break your pDoS down into (e.g fragment 2 corresponds to all Sulfur 3s atomic orbitals in the example above). Code will break if you get this wrong.

3) '''1a/1t/1l''' instruct the code what to use as the total density of states. Hence 1a must be set to ALL the atoms your interested in for this fragment, 1t must be set to "a" and 1l should be set to TDOS (not sure it matters what this is set to, but to be on the safe side just use TDOS). The purpose of this fragment is essentially error checking.

4) '''2a/3a/*a''' : a stands for atoms here, so the value of a corresponds to the labels of the atoms you want to be included in the sub-fragment. These labels are the same integers you get in gaussview when you go to view-->labels. Examples of valid options may be "1,4,5-7" or "1,3,5" (basically comma seperated list of integers)

5) '''2t/3t/*t''': t stands for "type of AO".This is a comma seperated list with possible arguments (any combination) "a,s,p,d,f,g". "a" corresponds to all types of AO centered of the atoms specified with the 2a/3a/*a line. s means s-type atomc orbitals (0 angular momentum), p means p-type atomic orbitals (1 angular momentum) etc. To get all AOs of a certain angular momentum and higher (up to g-type AOs) add a plus to the end of the relevant letter. e.g "p+" says "include all AOs with angular momemtnum greater than or equal to 1 in this sub-fragment")
'''DON'T''' use "a" with any other letter, and dont do anything else that will lead to a type of AO being requested twice (e.g p+,d)

6) '''2l/3l/*l''': l stands for label. If you dont care about calculating an XPS spectrum you can essentially set it to whatever you want. This tells the matlab script what cross section to use for calculating the XPS spectrum, hence it must match (case insensitive) a filename in the X_Section_Library folder. e.g S3s tells matlab to get the S3S.txt file when looking for what cross-section to use.

7) '''M_factor=1.0''', you really never need to modify this but make sure its present. It's purpose is to tell matlab to multiple the pDoS of this fragment (that this input file corresponds to) by the value M_factor. The purpose is for systems where the real stoichiometry differs from that in your calculation. For example if i have a mixture of 99 water:1 anion but only a calculation on 1 water molecule mixed with 1 anion i could set M_factor for the water fragment input file to M_factor=99.0 to estimate (badly) what the actual DoS of the mixture would look like.

8) '''Link_File=*'''. This options for if you have a fragment (e.g an anion) but want it broken into more than 10 sub-fragments (e.g this will be true for any molecule with >3 elements). In this case make 2 files for that fragment, and in the second file set Link_File to the name of the first file. In the example above, this input file will essentially be merged with "all_frags.test", and only an output file for all_frags.test will be generated.

====More complex input file options====
====Gen_Gelius_Spectrum.m====
This code has the following options that can be changed (all are explained by comments in the .m file):
<pre>1)min_e/max_e
</pre>
Matlab ignores any orbitals with energy below min_e or above max_e when generating the XPS spectrum. Note if you want to change this to include core orbitals or very high energy virtual orbitals then you'll also have to edit Gen_Multiwfn_Input_Comms.py to set the range in multiwfn to the one you want.(Theres currently no variable in that .py code to do that, it just uses the defaults in multiwfn. But the function you'll need to edit in Gen_Multiwfn_Input_Comms.py is Output_Header.
<pre>2)min_e_plotted/max_e_plotted
</pre>
These refer to the minimum/maximum binding energies to calculate spectral intensity for. The range should probably be slightly wider than the min_e/max_e range really, though thats not the default setting.
<pre>3)step_size
</pre>
The value (in eV) between calculated data points in the output spectra. For example, if set to 0.001eV then a data point will be calculated for 0eV,0.001eV,0.002eV etc. Lower values=larger files/slower runtimes.
<pre>4)lorentz_fraction
</pre>
Value between 0 and 1 that determines the % Lorentzian character of the lineshape used in the broadening function(the remaining % is set to gaussian). By broadening function i essentially mean the peak thats placed around every MO energy in generating these spectra. lorentz_fraction=0 will give you a pure gaussian peak, while lorentz_fraction=1 gives a pure Lorentzian peak.
<pre>5)savetext/figsave/savejpeg/all_contribs_output
</pre>
These all control whether a certain type of output file is saved, with variable names that should be self-explanatory.Values of 1 mean save the file, other values mean dont save. set savetext=0 to not save any .txt output files, figsave=0 to not save any .fig output, savejpeg=0 to not save any .jpeg outputs and all_contribs_output=0 to not save the .txt file with a full breakdown of the spectrum. Leaving them all as 1 (except maybe the savejpeg) is strongly recommended.

====Plot_Atomic_Contribs_V3====

This is an optional script that plots certain breakdowns of your calculated DoS/XPS spectrum (i.e you run it AFTER Gen_Gelius_Spectrum) . Its not too hard to do the same thing in excel using the *ALL_Contribs* output file (see output file sections) but this is a lot faster. There are three types of breakdown this function can do (see pictures below):

1) Atomic breakdown with fragments kept seperate (merge_spd=1,merge_fragments=0)

2) Atomic orbital breakdown (merge_spd=0,merge_fragments=1)

3) Atomic breakdown (merge_spd=1,merge_fragments=1)

The function in its uploaded form takes a single argument , though there are two variables in the actual code that you need to mess with (merge_spd and merge_fragments).

So for example, to run this on the output file from the "Using the Example files" i typed into the matlab terminal:

Plot_Atomic_Contribs_V3('C:\Users\Richard\Desktop\Open_Shell_Code_Some_Stuff\Example_files\other_file\DoS_1ev_0pt3l')

The argument needs to be a folder containing the output files from the Gen_Gelius_Spectrum script. You'll then get a saved graph with the breakdown you asked for and also a .txt file containing the breakdown in a comma delimited format. The output file names record the options you used to make the script (e.g "Plotted_Atomic_Contribs_spd0_frag1.txt" means merge_spd was set to 0 and merge_fragments was set to 1).

[[File:Plotted Atomic Contribs spd1 frag1.jpg|thumbnail|center| merge_spd=1,merge_fragments=1 ]]

[[File:Plotted_Atomic_Contribs_spd0_frag1.jpg|thumbnail|center| merge_spd=0,merge_fragments=1 ]]

[[File:Plotted Atomic Contribs spd1 frag0.jpg|thumbnail|center| merge_spd=1,merge_fragments=0 ]]

==Output Files==

The first "output file" is the *MCRF*.txt file, which just contains the commands to build the required fragment in multiwfn.

===Files From Multiwfn===
Most people should be able to completely ignore these files. But in case your curious:

'''1) *DOS_curve*.txt''' Essentially a DoS plot. 1st column is energies in au, second column is TDOS of the fragment. Not really sure what the 3rd column is. No particular use for this file, but it gets auto-generated by multiwfn and i dont go out my way to delete it.

'''2) *DOS_line*.txt''' Data to create a stick plot (where each MO is a stick). Plot column 1 vs column 2 as a line graph and you get an axis with sticks representing the MOs. 4th column onwards represent contribution of the various sub-fragments to each "stick". This is the file that gets used to generate the *ML_inpt.txt file.

'''3) *DOS_frag*.txt ''' This file contains the sub-fragment info in a way multiwfn understands. If its renamed to DOS_frag.txt then multiwfn can import it to get the fragments (therefore can be useful for debugging, no further purpose).

===MLinpt.txt File===

Creation of the *ML_inpt.txt files is essentially the point of the gen_gelius_input* script. These are primarily input files for matlab, but they do have useful information in an easy to read format. If you import the file into excel and COMMA DELIMIT ONLY (i.e untick the space delimit option) you can get something like below(there are more decimal places in the actual file):

<pre>
#M_FACTOR=1.0,
#Energy(eV) TDOS CO4S CO3P CO3D N2S N2P N2D C2S C2P C2D S3S S3P S3D
[LINES DELETED]
-24.5 0.9 0.0 0.0 0.0 0.5 0.1 0.0 0.2 0.1 0.0 0.1 0.0 0.0
-24.4 1.0 0.0 0.0 0.0 0.5 0.1 0.0 0.3 0.1 0.0 0.0 0.0 0.0
-24.4 1.0 0.0 0.0 0.0 0.5 0.1 0.0 0.3 0.1 0.0 0.0 0.0 0.0
-24.4 1.0 0.0 0.0 0.0 0.5 0.1 0.0 0.3 0.1 0.0 0.0 0.0 0.0
-19.8 0.3 0.0 0.0 0.0 0.1 -0.1 0.0 0.2 0.1 0.0 0.7 0.1 0.0
-19.7 0.3 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.1 0.0 0.7 0.1 0.0
[LINES DELETED]
</pre>

The shown file is "all_frags_alpha_MLinpt.txt", which is generated from the example files in the open shell code. As this actually comes from merging two MLinpt.txt files (gen_gelius_input.sh does this part silently) the TDOS in this case doesnt mean anything.

The first column lists the energy of each MO in eV (1 ev ~27.212 au). For any row, columns 3(CO4S)-->end (S3D) list the fractional contributions of each type of atmoic orbital to the MO in that row. For example the MO with energy -19.7 eV is mainly from S3s type atomic orbitals. These fractions were determiend by multiwfn by using Mulliken population analysis.

All the fractional contributions in the file get multiplied by the M_FACTOR value (as explained above in the Creating your own input files section). Therefore the value of M_FACTOR can easily be set after the MLinpt.txt file is created, as opposed to within the initial input file.

===Formatted_UMOs===

The default for the gen_gelius_input* script (i.e the settings for the version i uploaded) is to put only OCCUPIED MO energies/breakdowns into the *MLinpt.txt file. If you want all MOs (I really DONT recommend this) in the MLinpt.txt file then change all_orbs='cheese' to all_orbs='ALLORBS' at the top of the gen_gelius_input* script.

Otherwise you will get a *Formatted_UMOs* file for every MLinpt.txt file you get. The format of the file is identical to the MLinpt.txt format except the MO at the top of the file corresponds to the LUMO.

WARNING: If you use the link_files feature, then the output formatted_UMOs files DO NOT get merged. This shouldnt matter as you should never really use the formatted UMOs for anything.

===Output From Matlab Code ===

The matlab code firstly will create a new folder with a name reflecting the input arguments. The format is hv_FWHM_%lorenztian where hv is the photon energy, FWHM is full width half maximum of the broadeing function and %lorenztian is the amount of lorenztian mixed in the broadening function. For example the earlier command (from using example files section) "Gen_Gelius_Spectrum(plist,-1,1)" generates the folder "DoS_1ev_0pt3l". DoS = Density of states, and means we did not use photoionisation cross sections. The 0pt3l part means our broadening function was 30% lorenztian and 70% Gaussian.

Two .fig files will have been produced from this code, these contain matlab graphs of the output. These .fig files follow the naming ("filename" is the name of the original fchk file):

1) filename_DoS.fig contains the total DoS plot.

2) filename_DoS_frag_contribs.fig contains the DoS AND the pDoS of each of the fragments. For closed shell code you get 1 fragment per input file (assuming you dont use link_files), but for the open shell code each input file leads to two fragments - one for alpha electrons and one for beta electrons.

[[File:Other_file_DoS.jpeg|thumb|center| 1) Total Density of States for a Co(SCN)4 fragment with filename other_file_DoS.fig]]

[[File:Other file DoS frag contribs.jpeg|thumb|center| 2) Contribtuions of alpha and beta electrons to the Total Density of States for Co(SCN)4, filename other_file_DoS_frag_contribs.fig]]

The filename_DoS.fig plot data is output in a .txt file of the same name, while each fragment in filename_DoS_frag_contribs.fig has its own .txt file. For the example above the two fragments have names "all_frags_alpha_other_file_DoS" and "all_frags_beta_other_file_DoS". The start of the names are taken from the name of the original input file "all_frags.test".

The final file output is the "ALL_Contribs_other_file_DoS.txt" file. This(as the name implies) contains all contributions to the DoS spectra in a comma delimited file. The Plot_Atomic_Contribs*.m code can be run on this file to generate plots with a few different breakdowns (e.g atomic breakdowns) - see the relevant section of this wiki (not done yet....). The file will look something like below (commas replaced with tabs):

<pre>
#all_frags_alpha all_frags_beta
#Sub fragments in each= 12 12
#Energy(eV) CO4S CO3P CO3D N2S N2P N2D C2S C2P C2D [COLUMNS REMOVED]
[ROWS REMOVED,numbers rounded]
7.5 0.00 0.01 0.31 0.04 0.13 0.00 0.00 0.11 0.00 [COLUMNS REMOVED]
7.51 0.00 0.01 0.33 0.04 0.13 0.00 0.00 0.11 0.00 [COLUMNS REMOVED]
7.52 0.00 0.01 0.34 0.04 0.13 0.00 0.00 0.11 0.00 [COLUMNS REMOVED]
7.53 0.00 0.01 0.35 0.04 0.14 0.00 0.00 0.12 0.00 [COLUMNS REMOVED]
7.54 0.00 0.01 0.36 0.04 0.14 0.00 0.00 0.12 0.00 [COLUMNS REMOVED]
7.55 0.00 0.02 0.37 0.04 0.14 0.00 0.00 0.12 0.00 [COLUMNS REMOVED]
7.56 0.00 0.02 0.39 0.05 0.14 0.00 0.00 0.13 0.00 [COLUMNS REMOVED]
[ROWS REMOVED]
</pre>

Hopefully the format is mostly intuitive, the top two rows contain information on what the data is, and the rest contain the data. The top row contains (originally comma delimited) a list of all "fragments" names, which is the number of input files for the closed shell code and 2x the number of input files for the open shell code. The second row contains the number of sub-fragments (which should be atomic orbitals) in each fragment. In the example above each fragment had 12 types of atomic orbital.

So for the example above column 2-14 shows contributions from "all_frags_alpha" while column 15-27 shows contributions from "all_frags_beta" (column 1 lists energies).

==Bugs/Things to Watch For==
===Fragment Names===
Make sure your fragment names arent a substring of your .fchk file names. For example an input file called "Cl.test" wont work for a file called "BMIM_Cl.fchk". However, "Cl_frag.test" will work. Therefore personally i add _frag to the end of every input file name. The other way to sort this is to set linked_files=0 in gen_gelius_input_opensh.sh, sincce the link files part of the code is where this bug comes from.

===Fragment Error Checking===
Unless you manually turn it off (The "error=" option in gen_gelius_input_opensh.sh script) the code tries to check your input file for errors. Specifically it tests whether the TDOS in an input_file really is the sum of the other sub-fragments in the input(.test) file. E.g it will trigger if you request only Sulfur p-type and d-type AOs (but no s-type AOs) in the input file, or if you request all sulfur p-type AOs twice. If the error check triggers then it BREAKS the input file for matlab by putting an error msg in the first line of the '''*MLinpt.txt file'''. An example of what the error looks like is:
<pre>
“Fragment Sum incorrect on row 19 for fchk file xxxx.fchk”
</pre>

If you *MLinpt.txt files arent working in matlab then this is probably the reason.

===Pathlengths===
This code makes a lot of sub-folders based on the name of .fchk files. If you have really long .fchk file names its likely that you'll end up with some files which exceed the 256 character limit for a pathlength on a windows computer. In matlab 2014 this will cause the code to exit with an error, saying it cant save the .fig files. In newer matlab versions it may just save the output files , but you wont be able to open them on windows if the path length is greater than 256 characters. Ways to avoid this are to shorten .fchk file names where possible, or (less practical) run the code on a folder as close as possible to root (i.e a folder with a short a pathlength as possible).

File:ML inpt att2.m

2017-07-27T16:28:48Z

Rf614: Matlab parser for *MLinpt.txt files

Matlab parser for *MLinpt.txt files

File:Plot Atomic Contribs V3.m

2017-07-27T14:10:27Z

Rf614: Rf614 uploaded a new version of File:Plot Atomic Contribs V3.m

Matlab script used with code to generate pDoS/Valence XPS spectra

File:Gelius closed shell code Aug2016.zip

2017-07-25T13:27:36Z

Rf614: Rf614 uploaded a new version of File:Gelius closed shell code Aug2016.zip

All files needed to calculate pDoS/XPS spectra for closed shell .fchk gaussian files.

File:Gelius open shell code Aug2016.zip

2017-07-25T13:27:08Z

Rf614: Rf614 uploaded a new version of File:Gelius open shell code Aug2016.zip

All code required to calculate XP spectra/pDoS from a gaussian .fchk file.

Talk:Mod:Hunt Research Group/Calc XPS Code

2017-07-25T13:25:17Z

Rf614: /* Some Theory */

==Links to Files==
Open Shell Version: [[Media:Gelius_open_shell_code_Aug2016.zip]]

Closed Shell Version: [[Media:Gelius_closed_shell_code_Aug2016.zip]]

Plot Atomic Contribs Script: [[Media:Plot Atomic Contribs V3.m]]

==Introduction==
The purpose of this code is to calculate partial density of states (pDoS) plots and Valence band x-ray photoelectron (XP) spectra from .fchk files. There is both an open shell and a closed shell version of the code, they work in exactly the same way.

All code was written using python 2.7 (with numpy), Multiwfn version 3.3.8, Matlab version 2014a and a cygwin(bash) shell on windows 8. It should work on windows or linux if the correct options are set. It might not work on a mac currently due to Multiwfn not working properly (I haven't personally tested it though).

==Some Theory==

===Mulliken Population Analysis===

A pretty key part to this code is the breakdown of molecular orbitals (MOs) in atomic orbitals (AOs, which are essentially basis functions). This is carried out in multiwfn using Mulliken population analysis, which i'll briefly summarise due to its importance.

The basic question is "For MO X, how many electrons come from atomic orbital Y?". Note im going to be using "basis functions" instead of "atomic orbitals" from this point, in terms of DFT calculations we interpret a basis function centered on atom X as an atomic orbital of atom X.We'll define <math>\psi</math> as the wavefunction for a '''single MO containing a single electron'''. Next we write <math>\psi</math> as the linear combination of individual basis functions (<math>\phi_{i}</math>):

<math>
\psi = {{\sum\limits_{i} c_{i} \phi_{i} }}
</math>

Since we want a number of electrons, we're interested in the formula for the electron density (<math>\rho</math>) of this MO:

<math>
\rho = \langle{} \Psi | \Psi \rangle{}
</math>

<math>
\langle{}\Psi | \Psi \rangle{} = ({{\sum\limits_{i} c_{i} \phi_{i} }})({{\sum\limits_{i} c_{i} \phi_{i} }})
</math>

<math>
\langle{}\Psi | \Psi \rangle{} = ({{\sum\limits_{i} c_{i}^{2} \phi_{i}^{2} }}) + 2 {{ \sum\limits_{i>j}}} {{\sum\limits_{j} c_{i}c_{j} \langle{}\phi_{j} | \phi_{i} \rangle{} }}
</math>

If these formulae are tricky to follow, try doing it for a system with just two basis functions.
Note that in the third line the indice ''j'' runs over the same coefficients and basis functions as the indice ''i'', its just used to show that the <math>{{ \sum\limits_{i>j}}} {{\sum\limits_{j} c_{i}c_{j} \langle{}\phi_{j} | \phi_{i} \rangle{} }}</math> term only contains products between 2 different basis functions/co-efficients.
Hence, we now have a formula for the MO density which is purely in terms of (known) basis functions and (known) coefficients. Now its just a case of working out how we should interpret these values.

Firstly note that <math>\phi_{i}^{2}=1</math> (since we use normalised basis functions) hence:

<math>
{{\sum\limits_{i} c_{i}^{2} \phi_{i}^{2} }} = {{\sum\limits_{i} c_{i}^{2}}}
</math>

The values of c_{i} in this expression are easy to interpret, they represent the fractional contribution of AO ''i'' to the wavefunction. The <math>2 {{ \sum\limits_{i>j}}} {{\sum\limits_{j} c_{i}c_{j} \langle{}\phi_{j} | \phi_{i} \rangle{} }}</math> term is more difficult to interpret, each expression in the series can be thought of as the number of electrons shared between two basis functions. i.e. the following term would represent the number of electrons shared between basis function 1 and 2:

<math>
n_{shared} = 2 c_{1}c_{2} \langle{}\phi_{1} | \phi_{2} \rangle{}
</math>

However, we need to split this "shared" electron density between the two contributing basis functions. In Mulliken Population analysis "shared" electron density is divided equally between the two contributing basis functions. So the total electron density assigned to one basis function can be written as:

<math>
n_{i} = c_{i}^{2} + {{\sum\limits_{j \neq i} c_{i}c_{j} \langle{}\phi_{j} | \phi_{i} \rangle{} }}
</math>

<math>n_{i}</math> represents the '''fractional''' contribution of ''i'' to the wavefunction. Hence if you had a 2-electron MO, you'd simply multiply <math>n_{i}</math> by 2.

===What the code actually does (roughly/briefly)===

Step 1 - The first step is to run a Mulliken population analysis on the *.fchk file. This leads to a file containing the energy/AO composition of each MO in the system (the *MLinpt.txt file). Subsequent steps vary depending on if you want partial density of states (pDoS) plots or calculated valence spectra.

If '''calculating pDoS''' (not XPS spectra):

For each type of atomic orbital present do the following:

- For each MO, center a gaussian-lorentzian function at that MO energy. Multiply the intensity by the contribution of the current AO to the current MO.

- Sum all the resultant peaks, this generates the pDOS the current atomic orbital

Once all pDoS are generated, the total DoS could be created by summing them all. In practice the code probably just calculates the total DoS first, by just placing a gaussian-lorenztian with area=1 around each MO energy.

If '''calculating XPS spectra''':

For each type of atomic orbital present do the following:

- Multiply the fractional contributions of this AO to each MO by the AO cross-section. This value will be reffered to as the "intensity" This cross-section depends on both the AO type and the photon energy. Default values in this code are stored in tables in X_Section_Library, and custom/different values can easily be added.

- For each MO, center a gaussian-lorentzian function at that MO energy. This area of this function is equal to the intensity calculated in the step above.

Sum all the resultants peaks, this generates the contribution of the current AO to the valence XPS spectrum
Once this is repeated for each AO type, sum all those individual contributions to generate the total XPS valence spectrum.

=="Installation" Instructions==

All instructions will be given for the open shell version of the code. However, the process is the same except whenever these instructions say to edit the "gen_gelius_input_opensh.sh", just do the corresponding edit on the gen_gelius_input.sh file. The Matlab scripts are EXACTLY the same for both open shell/closed shell codes.

0) Make sure you have multiwfn (https://multiwfn.codeplex.com) saved somewhere, python 2.x and numpy installed.

1)Put the following files (all the .py/.sh files) somewhere in system path (e.g i use /usr/local/bin):
:a)Format_Multiwfn_Output_alpha.py (Format_Multiwfn_Output.py for closed shell)
:b)Format_Multilwfn_Output_beta.py
:c)Gen_Multiwfn_Input_Comms.py
:d)Gen_Multiwfn_Input_Comms_beta.py
:e)Merge_Fragment_Files.py
:f)gen_gelius_input_opensh.sh

Make sure all files have executable persmission (navigate to the folder and do "chmod u+x filename")

2)In the gen_gelius_input_opensh.sh change the current lines accordingly:
:a) multiwfn_folder = 'C:/Users/Richard/Documents/Multiwfn_Installed' to the directory where you store multiwfn and associated files.
:b) "cygwin=1” to “cygwin=0” if your not using a Cygwin shell.
:c) “old_sed=1” to “old_sed=0” if your version of sed allows the –i option (edit in place) to be used. I don’t think the standard version on mac does(meaning set old_sed=1),standard version of Cygwin does though.
:d) POSSIBLY "cp $multiwfn_folder/Multiwfn.exe $curr_folder" to "cp $multiwfn_folder/Multiwfn $curr_folder" if on linux(should be really obvious if this is required). Do the same for the line where multiwfn is deleted from current folder("rm Multiwfn.exe").

3)Put the following files somewhere in matlabs path (e.g your matlab folder in documents)
:a) Get_X_Section.m
:b) Gen_Gelius_Spectrum.m

The Get_X_Section REALLY does need to be in matlabs path (unless you want to edit some code in Gen_Gelius_Spectrum.m). If you know what your doing in matlab feel free to put Gen_Gelius_Spectrum.m wherever you want.

4)Put the X_Section_Library folder wherever you want.

5) In Get_X_Section.m modify the following line:
:"Lib_Path ='C:\Users\Richard\Documents\X_Section_Library' " to the path where you put the X_Section_Library folder in.

6) OPTIONAL: Add the code in vim_template_script to your vimrc (c+p it somewhere in the file). "vim ~/.vimrc" to access your vimrc. You can then bring up a template input file in vim by using ":call Gelius_Input()"
Since i forgot to put the relevant text file in to the code folders, just c+p the below into your vimrc to get this to work.

<pre>
function! Gelius_Input()
exe 'normal aCoSCN4_freq_AllNCS_B3LYP_6311plgdp_Tetrahed_BMIMSCNSMD_Abr.fchk\rNumb_Fragments=10\r1a=\r1t=a\r1l=TDOS\r2a=\r2t=\r2l=\r3a=\r3t=\r3l=\r4a=\r4t=\r4l=\r5a=\r5t=\r5l=\r6a=\r6t=\r6l=\r7a=\r7t=\r7l=\r8a=\r8t=\r8l=\r9a=\r9t=\r9l=\r10a=\r10t=\r10l=\rM_factor=1.0\rLink_File='
exe '%s/\\r/\r/g'
endfunction
</pre>

==Running The Code==
====Using the Example files====
1) Navigate (in your shell) to the folder "Example_files". e.g for me i ran:

"cd /cygdrive/c/Users/Richard/Documents/Random_Coding/Gelius_Spectrum_Automated_Jan_2016/open_shell_code/Example_files"

2) Run this command in console(for closed shell use gen_gelius_input.sh):

"gen_gelius_input_opensh.sh"

Command 2) will have created a folder for each .fchk file in Example_files/. Therefore you should now have folders:
:CoSCN4_freq_AllNCS_B3LYP_6311plgdp_Tetrahed_BMIMSCNSMD_Abr
:other_file
Within these folders are tons of output files. The most important ones are those ending in "MLinpt.txt" - these will be the input files for step 3.

The script will run on all .fchk files in the current folder, when you do this for real you need to make sure that all the fchks are for the same system (e.g a water molecule) and that the atomic numbering is the same in all files (view-->labels in gaussview gives you what im calling atomic numbering).

If anything went wrong try revisiting the installation instructions.

3) Everything from now on is done using matlab. Theres a file called "pathname_file" in the Example_files folder which was created by the gen_gelius_input_opensh.sh script. Copy its full contents to clipboard and in MATLAB terminal type:

<pre>
plist=<CTRL-V>
(<CTRL-V> means paste, rather than type out CTRL-V)
</pre>

In matlab terms, plist now contains a cell array where each entry is the full path to a folder. The folder paths are those for the folders created in step 2). Hence to do this manually for just one of the two folders you'd type something like:
<pre>
plist={'C:\Users\Richard\Documents\Random_Coding\Gelius_Spectrum_Automated_Jan_2016\open_shell_code\Example_files\other_file'}
</pre>

4) To create either an XPS spectrum or pDoS plots we use the Gen_Gelius_Spectrum script. This takes the following three arguments IN THE FOLLOWING ORDER:
:a) the plist variable we just made
:b) the photon energy we want to use (any negative number requests a pDoS rather than an XPS spectrum)
:c) the FWHM of the peaks to place around each MO energy (units = eV)

So to create a pDoS plot you could type the following into the matlab terminal(Try this one first):
<pre>
Gen_Gelius_Spectrum(plist,-1,1)
</pre>
To create an XPS spectrum taken at photon energy of 1486eV (standard lab-based value) type the following:
<pre>
Gen_Gelius_Spectrum(plist,1486,1)
</pre>

If you get an error like "Undefined function 'Gen_Gelius_Spectrum' for input arguments of type 'cell'.", then your Gen_Gelius_Spectrum script probably isnt in your matlab path. (see installation instructions, or google a bit)

If your error free then well done. Sub-folders should have been created in "other_file" and "CoSCN4_freq_AllNCS_B3LYP_6311plgdp_Tetrahed_BMIMSCNSMD_Abr", which in turn will contain a loada output files. See the output files section of the wiki for an explanation of what these are.

====Creating your own input files====
If you followed the optional part above and made a vimscript, then a template for the input file can be brought up in vim by typing ":call Gelius_Input()"

An example input file is shown below. Each "fragment" (e.g cation,anion, alkyl group etc.) should have its own input file.
<pre>
other_file.fchk
Numb_Fragments=4
1a=10-13
1t=a
1l=TDOS
2a=10-13
2t=s
2l=S3s
3a=10-13
3t=p
3l=S3p
4a=10-13
4t=d+
4l=S3d
M_factor=1.0
Link_File=all_frags.test
</pre>
The input file can be divided into parts as follows:

1) The top line of the input file tells the code the name of the checkpoint file to use in the calculation. In practice you can write ANYTHING on this top line if using the gen_gelius_input*.sh script, as the script itself replaces the first line (whatevers there) with the names of all .fchks in the current folder. But SOMETHING must be written there.

2)'''Numb_Fragments''' is the number of sub-fragments (max 10) that you want to break your pDoS down into (e.g fragment 2 corresponds to all Sulfur 3s atomic orbitals in the example above). Code will break if you get this wrong.

3) '''1a/1t/1l''' instruct the code what to use as the total density of states. Hence 1a must be set to ALL the atoms your interested in for this fragment, 1t must be set to "a" and 1l should be set to TDOS (not sure it matters what this is set to, but to be on the safe side just use TDOS). The purpose of this fragment is essentially error checking.

4) '''2a/3a/*a''' : a stands for atoms here, so the value of a corresponds to the labels of the atoms you want to be included in the sub-fragment. These labels are the same integers you get in gaussview when you go to view-->labels. Examples of valid options may be "1,4,5-7" or "1,3,5" (basically comma seperated list of integers)

5) '''2t/3t/*t''': t stands for "type of AO".This is a comma seperated list with possible arguments (any combination) "a,s,p,d,f,g". "a" corresponds to all types of AO centered of the atoms specified with the 2a/3a/*a line. s means s-type atomc orbitals (0 angular momentum), p means p-type atomic orbitals (1 angular momentum) etc. To get all AOs of a certain angular momentum and higher (up to g-type AOs) add a plus to the end of the relevant letter. e.g "p+" says "include all AOs with angular momemtnum greater than or equal to 1 in this sub-fragment")
'''DON'T''' use "a" with any other letter, and dont do anything else that will lead to a type of AO being requested twice (e.g p+,d)

6) '''2l/3l/*l''': l stands for label. If you dont care about calculating an XPS spectrum you can essentially set it to whatever you want. This tells the matlab script what cross section to use for calculating the XPS spectrum, hence it must match (case insensitive) a filename in the X_Section_Library folder. e.g S3s tells matlab to get the S3S.txt file when looking for what cross-section to use.

7) '''M_factor=1.0''', you really never need to modify this but make sure its present. It's purpose is to tell matlab to multiple the pDoS of this fragment (that this input file corresponds to) by the value M_factor. The purpose is for systems where the real stoichiometry differs from that in your calculation. For example if i have a mixture of 99 water:1 anion but only a calculation on 1 water molecule mixed with 1 anion i could set M_factor for the water fragment input file to M_factor=99.0 to estimate (badly) what the actual DoS of the mixture would look like.

8) '''Link_File=*'''. This options for if you have a fragment (e.g an anion) but want it broken into more than 10 sub-fragments (e.g this will be true for any molecule with >3 elements). In this case make 2 files for that fragment, and in the second file set Link_File to the name of the first file. In the example above, this input file will essentially be merged with "all_frags.test", and only an output file for all_frags.test will be generated.

====More complex input file options====
====Gen_Gelius_Spectrum.m====
This code has the following options that can be changed (all are explained by comments in the .m file):
<pre>1)min_e/max_e
</pre>
Matlab ignores any orbitals with energy below min_e or above max_e when generating the XPS spectrum. Note if you want to change this to include core orbitals or very high energy virtual orbitals then you'll also have to edit Gen_Multiwfn_Input_Comms.py to set the range in multiwfn to the one you want.(Theres currently no variable in that .py code to do that, it just uses the defaults in multiwfn. But the function you'll need to edit in Gen_Multiwfn_Input_Comms.py is Output_Header.
<pre>2)min_e_plotted/max_e_plotted
</pre>
These refer to the minimum/maximum binding energies to calculate spectral intensity for. The range should probably be slightly wider than the min_e/max_e range really, though thats not the default setting.
<pre>3)step_size
</pre>
The value (in eV) between calculated data points in the output spectra. For example, if set to 0.001eV then a data point will be calculated for 0eV,0.001eV,0.002eV etc. Lower values=larger files/slower runtimes.
<pre>4)lorentz_fraction
</pre>
Value between 0 and 1 that determines the % Lorentzian character of the lineshape used in the broadening function(the remaining % is set to gaussian). By broadening function i essentially mean the peak thats placed around every MO energy in generating these spectra. lorentz_fraction=0 will give you a pure gaussian peak, while lorentz_fraction=1 gives a pure Lorentzian peak.
<pre>5)savetext/figsave/savejpeg/all_contribs_output
</pre>
These all control whether a certain type of output file is saved, with variable names that should be self-explanatory.Values of 1 mean save the file, other values mean dont save. set savetext=0 to not save any .txt output files, figsave=0 to not save any .fig output, savejpeg=0 to not save any .jpeg outputs and all_contribs_output=0 to not save the .txt file with a full breakdown of the spectrum. Leaving them all as 1 (except maybe the savejpeg) is strongly recommended.

====Plot_Atomic_Contribs_V3====

This is an optional script that plots certain breakdowns of your calculated DoS/XPS spectrum (i.e you run it AFTER Gen_Gelius_Spectrum) . Its not too hard to do the same thing in excel using the *ALL_Contribs* output file (see output file sections) but this is a lot faster. There are three types of breakdown this function can do (see pictures below):

1) Atomic breakdown with fragments kept seperate (merge_spd=1,merge_fragments=0)

2) Atomic orbital breakdown (merge_spd=0,merge_fragments=1)

3) Atomic breakdown (merge_spd=1,merge_fragments=1)

The function in its uploaded form takes a single argument , though there are two variables in the actual code that you need to mess with (merge_spd and merge_fragments).

So for example, to run this on the output file from the "Using the Example files" i typed into the matlab terminal:

Plot_Atomic_Contribs_V3('C:\Users\Richard\Desktop\Open_Shell_Code_Some_Stuff\Example_files\other_file\DoS_1ev_0pt3l')

The argument needs to be a folder containing the output files from the Gen_Gelius_Spectrum script. You'll then get a saved graph with the breakdown you asked for and also a .txt file containing the breakdown in a comma delimited format. The output file names record the options you used to make the script (e.g "Plotted_Atomic_Contribs_spd0_frag1.txt" means merge_spd was set to 0 and merge_fragments was set to 1).

[[File:Plotted Atomic Contribs spd1 frag1.jpg|thumbnail|center| merge_spd=1,merge_fragments=1 ]]

[[File:Plotted_Atomic_Contribs_spd0_frag1.jpg|thumbnail|center| merge_spd=0,merge_fragments=1 ]]

[[File:Plotted Atomic Contribs spd1 frag0.jpg|thumbnail|center| merge_spd=1,merge_fragments=0 ]]

==Output Files==

The first "output file" is the *MCRF*.txt file, which just contains the commands to build the required fragment in multiwfn.

===Files From Multiwfn===
Most people should be able to completely ignore these files. But in case your curious:

'''1) *DOS_curve*.txt''' Essentially a DoS plot. 1st column is energies in au, second column is TDOS of the fragment. Not really sure what the 3rd column is. No particular use for this file, but it gets auto-generated by multiwfn and i dont go out my way to delete it.

'''2) *DOS_line*.txt''' Data to create a stick plot (where each MO is a stick). Plot column 1 vs column 2 as a line graph and you get an axis with sticks representing the MOs. 4th column onwards represent contribution of the various sub-fragments to each "stick". This is the file that gets used to generate the *ML_inpt.txt file.

'''3) *DOS_frag*.txt ''' This file contains the sub-fragment info in a way multiwfn understands. If its renamed to DOS_frag.txt then multiwfn can import it to get the fragments (therefore can be useful for debugging, no further purpose).

===MLinpt.txt File===

Creation of the *ML_inpt.txt files is essentially the point of the gen_gelius_input* script. These are primarily input files for matlab, but they do have useful information in an easy to read format. If you import the file into excel and COMMA DELIMIT ONLY (i.e untick the space delimit option) you can get something like below(there are more decimal places in the actual file):

<pre>
#M_FACTOR=1.0,
#Energy(eV) TDOS CO4S CO3P CO3D N2S N2P N2D C2S C2P C2D S3S S3P S3D
[LINES DELETED]
-24.5 0.9 0.0 0.0 0.0 0.5 0.1 0.0 0.2 0.1 0.0 0.1 0.0 0.0
-24.4 1.0 0.0 0.0 0.0 0.5 0.1 0.0 0.3 0.1 0.0 0.0 0.0 0.0
-24.4 1.0 0.0 0.0 0.0 0.5 0.1 0.0 0.3 0.1 0.0 0.0 0.0 0.0
-24.4 1.0 0.0 0.0 0.0 0.5 0.1 0.0 0.3 0.1 0.0 0.0 0.0 0.0
-19.8 0.3 0.0 0.0 0.0 0.1 -0.1 0.0 0.2 0.1 0.0 0.7 0.1 0.0
-19.7 0.3 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.1 0.0 0.7 0.1 0.0
[LINES DELETED]
</pre>

The shown file is "all_frags_alpha_MLinpt.txt", which is generated from the example files in the open shell code. As this actually comes from merging two MLinpt.txt files (gen_gelius_input.sh does this part silently) the TDOS in this case doesnt mean anything.

The first column lists the energy of each MO in eV (1 ev ~27.212 au). For any row, columns 3(CO4S)-->end (S3D) list the fractional contributions of each type of atmoic orbital to the MO in that row. For example the MO with energy -19.7 eV is mainly from S3s type atomic orbitals. These fractions were determiend by multiwfn by using Mulliken population analysis.

All the fractional contributions in the file get multiplied by the M_FACTOR value (as explained above in the Creating your own input files section). Therefore the value of M_FACTOR can easily be set after the MLinpt.txt file is created, as opposed to within the initial input file.

===Formatted_UMOs===

The default for the gen_gelius_input* script (i.e the settings for the version i uploaded) is to put only OCCUPIED MO energies/breakdowns into the *MLinpt.txt file. If you want all MOs (I really DONT recommend this) in the MLinpt.txt file then change all_orbs='cheese' to all_orbs='ALLORBS' at the top of the gen_gelius_input* script.

Otherwise you will get a *Formatted_UMOs* file for every MLinpt.txt file you get. The format of the file is identical to the MLinpt.txt format except the MO at the top of the file corresponds to the LUMO.

WARNING: If you use the link_files feature, then the output formatted_UMOs files DO NOT get merged. This shouldnt matter as you should never really use the formatted UMOs for anything.

===Output From Matlab Code ===

The matlab code firstly will create a new folder with a name reflecting the input arguments. The format is hv_FWHM_%lorenztian where hv is the photon energy, FWHM is full width half maximum of the broadeing function and %lorenztian is the amount of lorenztian mixed in the broadening function. For example the earlier command (from using example files section) "Gen_Gelius_Spectrum(plist,-1,1)" generates the folder "DoS_1ev_0pt3l". DoS = Density of states, and means we did not use photoionisation cross sections. The 0pt3l part means our broadening function was 30% lorenztian and 70% Gaussian.

Two .fig files will have been produced from this code, these contain matlab graphs of the output. These .fig files follow the naming ("filename" is the name of the original fchk file):

1) filename_DoS.fig contains the total DoS plot.

2) filename_DoS_frag_contribs.fig contains the DoS AND the pDoS of each of the fragments. For closed shell code you get 1 fragment per input file (assuming you dont use link_files), but for the open shell code each input file leads to two fragments - one for alpha electrons and one for beta electrons.

[[File:Other_file_DoS.jpeg|thumb|center| 1) Total Density of States for a Co(SCN)4 fragment with filename other_file_DoS.fig]]

[[File:Other file DoS frag contribs.jpeg|thumb|center| 2) Contribtuions of alpha and beta electrons to the Total Density of States for Co(SCN)4, filename other_file_DoS_frag_contribs.fig]]

The filename_DoS.fig plot data is output in a .txt file of the same name, while each fragment in filename_DoS_frag_contribs.fig has its own .txt file. For the example above the two fragments have names "all_frags_alpha_other_file_DoS" and "all_frags_beta_other_file_DoS". The start of the names are taken from the name of the original input file "all_frags.test".

The final file output is the "ALL_Contribs_other_file_DoS.txt" file. This(as the name implies) contains all contributions to the DoS spectra in a comma delimited file. The Plot_Atomic_Contribs*.m code can be run on this file to generate plots with a few different breakdowns (e.g atomic breakdowns) - see the relevant section of this wiki (not done yet....). The file will look something like below (commas replaced with tabs):

<pre>
#all_frags_alpha all_frags_beta
#Sub fragments in each= 12 12
#Energy(eV) CO4S CO3P CO3D N2S N2P N2D C2S C2P C2D [COLUMNS REMOVED]
[ROWS REMOVED,numbers rounded]
7.5 0.00 0.01 0.31 0.04 0.13 0.00 0.00 0.11 0.00 [COLUMNS REMOVED]
7.51 0.00 0.01 0.33 0.04 0.13 0.00 0.00 0.11 0.00 [COLUMNS REMOVED]
7.52 0.00 0.01 0.34 0.04 0.13 0.00 0.00 0.11 0.00 [COLUMNS REMOVED]
7.53 0.00 0.01 0.35 0.04 0.14 0.00 0.00 0.12 0.00 [COLUMNS REMOVED]
7.54 0.00 0.01 0.36 0.04 0.14 0.00 0.00 0.12 0.00 [COLUMNS REMOVED]
7.55 0.00 0.02 0.37 0.04 0.14 0.00 0.00 0.12 0.00 [COLUMNS REMOVED]
7.56 0.00 0.02 0.39 0.05 0.14 0.00 0.00 0.13 0.00 [COLUMNS REMOVED]
[ROWS REMOVED]
</pre>

Hopefully the format is mostly intuitive, the top two rows contain information on what the data is, and the rest contain the data. The top row contains (originally comma delimited) a list of all "fragments" names, which is the number of input files for the closed shell code and 2x the number of input files for the open shell code. The second row contains the number of sub-fragments (which should be atomic orbitals) in each fragment. In the example above each fragment had 12 types of atomic orbital.

So for the example above column 2-14 shows contributions from "all_frags_alpha" while column 15-27 shows contributions from "all_frags_beta" (column 1 lists energies).

==Bugs/Things to Watch For==
===Fragment Names===
Make sure your fragment names arent a substring of your .fchk file names. For example an input file called "Cl.test" wont work for a file called "BMIM_Cl.fchk". However, "Cl_frag.test" will work. Therefore personally i add _frag to the end of every input file name. The other way to sort this is to set linked_files=0 in gen_gelius_input_opensh.sh, sincce the link files part of the code is where this bug comes from.

===Fragment Error Checking===
Unless you manually turn it off (The "error=" option in gen_gelius_input_opensh.sh script) the code tries to check your input file for errors. Specifically it tests whether the TDOS in an input_file really is the sum of the other sub-fragments in the input(.test) file. E.g it will trigger if you request only Sulfur p-type and d-type AOs (but no s-type AOs) in the input file, or if you request all sulfur p-type AOs twice. If the error check triggers then it BREAKS the input file for matlab by putting an error msg in the first line of the '''*MLinpt.txt file'''. An example of what the error looks like is:
<pre>
“Fragment Sum incorrect on row 19 for fchk file xxxx.fchk”
</pre>

If you *MLinpt.txt files arent working in matlab then this is probably the reason.

===Pathlengths===
This code makes a lot of sub-folders based on the name of .fchk files. If you have really long .fchk file names its likely that you'll end up with some files which exceed the 256 character limit for a pathlength on a windows computer. In matlab 2014 this will cause the code to exit with an error, saying it cant save the .fig files. In newer matlab versions it may just save the output files , but you wont be able to open them on windows if the path length is greater than 256 characters. Ways to avoid this are to shorten .fchk file names where possible, or (less practical) run the code on a folder as close as possible to root (i.e a folder with a short a pathlength as possible).

Talk:Mod:Hunt Research Group/Calc XPS Code

2017-07-25T10:09:32Z

Rf614: /* Links to Files */

==Links to Files==
Open Shell Version: [[Media:Gelius_open_shell_code_Aug2016.zip]]

Closed Shell Version: [[Media:Gelius_closed_shell_code_Aug2016.zip]]

Plot Atomic Contribs Script: [[Media:Plot Atomic Contribs V3.m]]

==Introduction==
The purpose of this code is to calculate partial density of states (pDoS) plots and Valence band x-ray photoelectron (XP) spectra from .fchk files. There is both an open shell and a closed shell version of the code, they work in exactly the same way.

All code was written using python 2.7 (with numpy), Multiwfn version 3.3.8, Matlab version 2014a and a cygwin(bash) shell on windows 8. It should work on windows or linux if the correct options are set. It might not work on a mac currently due to Multiwfn not working properly (I haven't personally tested it though).

==Some Theory==

===Mulliken Population Analysis===

A pretty key part to this code is the breakdown of molecular orbitals (MOs) in atomic orbitals (AOs, which are essentially basis functions). This is carried out in multiwfn using Mulliken population analysis, which i'll briefly summarise due to its importance.

The basic question is "For MO X, how many electrons come from atomic orbital Y?". Note im going to be using "basis functions" instead of "atomic orbitals" from this point, in terms of DFT calculations we interpret a basis function centered on atom X as an atomic orbital of atom X.We'll define <math>\psi</math> as the wavefunction for a '''single MO containing a single electron'''. Next we write <math>\psi</math> as the linear combination of individual basis functions (<math>\phi_{i}</math>):

<math>
\psi = {{\sum\limits_{i} c_{i} \phi_{i} }}
</math>

Since we want a number of electrons, we're interested in the formula for the electron density (<math>\rho</math>) of this MO:

<math>
\rho = \langle{} \Psi | \Psi \rangle{}
</math>

<math>
\langle{}\Psi | \Psi \rangle{} = ({{\sum\limits_{i} c_{i} \phi_{i} }})({{\sum\limits_{i} c_{i} \phi_{i} }})
</math>

<math>
\langle{}\Psi | \Psi \rangle{} = ({{\sum\limits_{i} c_{i}^{2} \phi_{i}^{2} }}) + 2 {{ \sum\limits_{i>j}}} {{\sum\limits_{j} c_{i}c_{j} \langle{}\phi_{j} | \phi_{i} \rangle{} }}
</math>

If these formulae are tricky to follow, try doing it for a system with just two basis functions.
Note that in the third line the indice ''j'' runs over the same coefficients and basis functions as the indice ''i'', its just used to show that the <math>{{ \sum\limits_{i>j}}} {{\sum\limits_{j} c_{i}c_{j} \langle{}\phi_{j} | \phi_{i} \rangle{} }}</math> term only contains products between 2 different basis functions/co-efficients.
Hence, we now have a formula for the MO density which is purely in terms of (known) basis functions and (known) coefficients. Now its just a case of working out how we should interpret these values.

Firstly note that <math>\phi_{i}^{2}=1</math> (since we use normalised basis functions) hence:

<math>
{{\sum\limits_{i} c_{i}^{2} \phi_{i}^{2} }} = {{\sum\limits_{i} c_{i}^{2}}}
</math>

The values of c_{i} in this expression are easy to interpret, they represent the fractional contribution of AO ''i'' to the wavefunction. The <math>2 {{ \sum\limits_{i>j}}} {{\sum\limits_{j} c_{i}c_{j} \langle{}\phi_{j} | \phi_{i} \rangle{} }}</math> term is more difficult to interpret, each expression in the series can be thought of as the number of electrons shared between two basis functions. i.e. the following term would represent the number of electrons shared between basis function 1 and 2:

<math>
n_{shared} = 2 c_{1}c_{2} \langle{}\phi_{1} | \phi_{2} \rangle{}
</math>

However, we need to split this "shared" electron density between the two contributing basis functions. In Mulliken Population analysis "shared" electron density is divided equally between the two contributing basis functions. So the total electron density assigned to one basis function can be written as:

<math>
n_{i} = c_{i}^{2} + {{\sum\limits_{i \neq j} c_{i}c_{j} \langle{}\phi_{j} | \phi_{i} \rangle{} }}
</math>

<math>n_{i}</math> represents the '''fractional''' contribution of ''i'' to the wavefunction. Hence if you had a 2-electron MO, you'd simply multiply <math>n_{i}</math> by 2.

===What the code actually does (roughly/briefly)===

Step 1 - The first step is to run a Mulliken population analysis on the *.fchk file. This leads to a file containing the energy/AO composition of each MO in the system (the *MLinpt.txt file). Subsequent steps vary depending on if you want partial density of states (pDoS) plots or calculated valence spectra.

If '''calculating pDoS''' (not XPS spectra):

For each type of atomic orbital present do the following:

- For each MO, center a gaussian-lorentzian function at that MO energy. Multiply the intensity by the contribution of the current AO to the current MO.

- Sum all the resultant peaks, this generates the pDOS the current atomic orbital

Once all pDoS are generated, the total DoS could be created by summing them all. In practice the code probably just calculates the total DoS first, by just placing a gaussian-lorenztian with area=1 around each MO energy.

If '''calculating XPS spectra''':

For each type of atomic orbital present do the following:

- Multiply the fractional contributions of this AO to each MO by the AO cross-section. This value will be reffered to as the "intensity" This cross-section depends on both the AO type and the photon energy. Default values in this code are stored in tables in X_Section_Library, and custom/different values can easily be added.

- For each MO, center a gaussian-lorentzian function at that MO energy. This area of this function is equal to the intensity calculated in the step above.

Sum all the resultants peaks, this generates the contribution of the current AO to the valence XPS spectrum
Once this is repeated for each AO type, sum all those individual contributions to generate the total XPS valence spectrum.

=="Installation" Instructions==

All instructions will be given for the open shell version of the code. However, the process is the same except whenever these instructions say to edit the "gen_gelius_input_opensh.sh", just do the corresponding edit on the gen_gelius_input.sh file. The Matlab scripts are EXACTLY the same for both open shell/closed shell codes.

0) Make sure you have multiwfn (https://multiwfn.codeplex.com) saved somewhere, python 2.x and numpy installed.

1)Put the following files (all the .py/.sh files) somewhere in system path (e.g i use /usr/local/bin):
:a)Format_Multiwfn_Output_alpha.py (Format_Multiwfn_Output.py for closed shell)
:b)Format_Multilwfn_Output_beta.py
:c)Gen_Multiwfn_Input_Comms.py
:d)Gen_Multiwfn_Input_Comms_beta.py
:e)Merge_Fragment_Files.py
:f)gen_gelius_input_opensh.sh

Make sure all files have executable persmission (navigate to the folder and do "chmod u+x filename")

2)In the gen_gelius_input_opensh.sh change the current lines accordingly:
:a) multiwfn_folder = 'C:/Users/Richard/Documents/Multiwfn_Installed' to the directory where you store multiwfn and associated files.
:b) "cygwin=1” to “cygwin=0” if your not using a Cygwin shell.
:c) “old_sed=1” to “old_sed=0” if your version of sed allows the –i option (edit in place) to be used. I don’t think the standard version on mac does(meaning set old_sed=1),standard version of Cygwin does though.
:d) POSSIBLY "cp $multiwfn_folder/Multiwfn.exe $curr_folder" to "cp $multiwfn_folder/Multiwfn $curr_folder" if on linux(should be really obvious if this is required). Do the same for the line where multiwfn is deleted from current folder("rm Multiwfn.exe").

3)Put the following files somewhere in matlabs path (e.g your matlab folder in documents)
:a) Get_X_Section.m
:b) Gen_Gelius_Spectrum.m

The Get_X_Section REALLY does need to be in matlabs path (unless you want to edit some code in Gen_Gelius_Spectrum.m). If you know what your doing in matlab feel free to put Gen_Gelius_Spectrum.m wherever you want.

4)Put the X_Section_Library folder wherever you want.

5) In Get_X_Section.m modify the following line:
:"Lib_Path ='C:\Users\Richard\Documents\X_Section_Library' " to the path where you put the X_Section_Library folder in.

6) OPTIONAL: Add the code in vim_template_script to your vimrc (c+p it somewhere in the file). "vim ~/.vimrc" to access your vimrc. You can then bring up a template input file in vim by using ":call Gelius_Input()"
Since i forgot to put the relevant text file in to the code folders, just c+p the below into your vimrc to get this to work.

<pre>
function! Gelius_Input()
exe 'normal aCoSCN4_freq_AllNCS_B3LYP_6311plgdp_Tetrahed_BMIMSCNSMD_Abr.fchk\rNumb_Fragments=10\r1a=\r1t=a\r1l=TDOS\r2a=\r2t=\r2l=\r3a=\r3t=\r3l=\r4a=\r4t=\r4l=\r5a=\r5t=\r5l=\r6a=\r6t=\r6l=\r7a=\r7t=\r7l=\r8a=\r8t=\r8l=\r9a=\r9t=\r9l=\r10a=\r10t=\r10l=\rM_factor=1.0\rLink_File='
exe '%s/\\r/\r/g'
endfunction
</pre>

==Running The Code==
====Using the Example files====
1) Navigate (in your shell) to the folder "Example_files". e.g for me i ran:

"cd /cygdrive/c/Users/Richard/Documents/Random_Coding/Gelius_Spectrum_Automated_Jan_2016/open_shell_code/Example_files"

2) Run this command in console(for closed shell use gen_gelius_input.sh):

"gen_gelius_input_opensh.sh"

Command 2) will have created a folder for each .fchk file in Example_files/. Therefore you should now have folders:
:CoSCN4_freq_AllNCS_B3LYP_6311plgdp_Tetrahed_BMIMSCNSMD_Abr
:other_file
Within these folders are tons of output files. The most important ones are those ending in "MLinpt.txt" - these will be the input files for step 3.

The script will run on all .fchk files in the current folder, when you do this for real you need to make sure that all the fchks are for the same system (e.g a water molecule) and that the atomic numbering is the same in all files (view-->labels in gaussview gives you what im calling atomic numbering).

If anything went wrong try revisiting the installation instructions.

3) Everything from now on is done using matlab. Theres a file called "pathname_file" in the Example_files folder which was created by the gen_gelius_input_opensh.sh script. Copy its full contents to clipboard and in MATLAB terminal type:

<pre>
plist=<CTRL-V>
(<CTRL-V> means paste, rather than type out CTRL-V)
</pre>

In matlab terms, plist now contains a cell array where each entry is the full path to a folder. The folder paths are those for the folders created in step 2). Hence to do this manually for just one of the two folders you'd type something like:
<pre>
plist={'C:\Users\Richard\Documents\Random_Coding\Gelius_Spectrum_Automated_Jan_2016\open_shell_code\Example_files\other_file'}
</pre>

4) To create either an XPS spectrum or pDoS plots we use the Gen_Gelius_Spectrum script. This takes the following three arguments IN THE FOLLOWING ORDER:
:a) the plist variable we just made
:b) the photon energy we want to use (any negative number requests a pDoS rather than an XPS spectrum)
:c) the FWHM of the peaks to place around each MO energy (units = eV)

So to create a pDoS plot you could type the following into the matlab terminal(Try this one first):
<pre>
Gen_Gelius_Spectrum(plist,-1,1)
</pre>
To create an XPS spectrum taken at photon energy of 1486eV (standard lab-based value) type the following:
<pre>
Gen_Gelius_Spectrum(plist,1486,1)
</pre>

If you get an error like "Undefined function 'Gen_Gelius_Spectrum' for input arguments of type 'cell'.", then your Gen_Gelius_Spectrum script probably isnt in your matlab path. (see installation instructions, or google a bit)

If your error free then well done. Sub-folders should have been created in "other_file" and "CoSCN4_freq_AllNCS_B3LYP_6311plgdp_Tetrahed_BMIMSCNSMD_Abr", which in turn will contain a loada output files. See the output files section of the wiki for an explanation of what these are.

====Creating your own input files====
If you followed the optional part above and made a vimscript, then a template for the input file can be brought up in vim by typing ":call Gelius_Input()"

An example input file is shown below. Each "fragment" (e.g cation,anion, alkyl group etc.) should have its own input file.
<pre>
other_file.fchk
Numb_Fragments=4
1a=10-13
1t=a
1l=TDOS
2a=10-13
2t=s
2l=S3s
3a=10-13
3t=p
3l=S3p
4a=10-13
4t=d+
4l=S3d
M_factor=1.0
Link_File=all_frags.test
</pre>
The input file can be divided into parts as follows:

1) The top line of the input file tells the code the name of the checkpoint file to use in the calculation. In practice you can write ANYTHING on this top line if using the gen_gelius_input*.sh script, as the script itself replaces the first line (whatevers there) with the names of all .fchks in the current folder. But SOMETHING must be written there.

2)'''Numb_Fragments''' is the number of sub-fragments (max 10) that you want to break your pDoS down into (e.g fragment 2 corresponds to all Sulfur 3s atomic orbitals in the example above). Code will break if you get this wrong.

3) '''1a/1t/1l''' instruct the code what to use as the total density of states. Hence 1a must be set to ALL the atoms your interested in for this fragment, 1t must be set to "a" and 1l should be set to TDOS (not sure it matters what this is set to, but to be on the safe side just use TDOS). The purpose of this fragment is essentially error checking.

4) '''2a/3a/*a''' : a stands for atoms here, so the value of a corresponds to the labels of the atoms you want to be included in the sub-fragment. These labels are the same integers you get in gaussview when you go to view-->labels. Examples of valid options may be "1,4,5-7" or "1,3,5" (basically comma seperated list of integers)

5) '''2t/3t/*t''': t stands for "type of AO".This is a comma seperated list with possible arguments (any combination) "a,s,p,d,f,g". "a" corresponds to all types of AO centered of the atoms specified with the 2a/3a/*a line. s means s-type atomc orbitals (0 angular momentum), p means p-type atomic orbitals (1 angular momentum) etc. To get all AOs of a certain angular momentum and higher (up to g-type AOs) add a plus to the end of the relevant letter. e.g "p+" says "include all AOs with angular momemtnum greater than or equal to 1 in this sub-fragment")
'''DON'T''' use "a" with any other letter, and dont do anything else that will lead to a type of AO being requested twice (e.g p+,d)

6) '''2l/3l/*l''': l stands for label. If you dont care about calculating an XPS spectrum you can essentially set it to whatever you want. This tells the matlab script what cross section to use for calculating the XPS spectrum, hence it must match (case insensitive) a filename in the X_Section_Library folder. e.g S3s tells matlab to get the S3S.txt file when looking for what cross-section to use.

7) '''M_factor=1.0''', you really never need to modify this but make sure its present. It's purpose is to tell matlab to multiple the pDoS of this fragment (that this input file corresponds to) by the value M_factor. The purpose is for systems where the real stoichiometry differs from that in your calculation. For example if i have a mixture of 99 water:1 anion but only a calculation on 1 water molecule mixed with 1 anion i could set M_factor for the water fragment input file to M_factor=99.0 to estimate (badly) what the actual DoS of the mixture would look like.

8) '''Link_File=*'''. This options for if you have a fragment (e.g an anion) but want it broken into more than 10 sub-fragments (e.g this will be true for any molecule with >3 elements). In this case make 2 files for that fragment, and in the second file set Link_File to the name of the first file. In the example above, this input file will essentially be merged with "all_frags.test", and only an output file for all_frags.test will be generated.

====More complex input file options====
====Gen_Gelius_Spectrum.m====
This code has the following options that can be changed (all are explained by comments in the .m file):
<pre>1)min_e/max_e
</pre>
Matlab ignores any orbitals with energy below min_e or above max_e when generating the XPS spectrum. Note if you want to change this to include core orbitals or very high energy virtual orbitals then you'll also have to edit Gen_Multiwfn_Input_Comms.py to set the range in multiwfn to the one you want.(Theres currently no variable in that .py code to do that, it just uses the defaults in multiwfn. But the function you'll need to edit in Gen_Multiwfn_Input_Comms.py is Output_Header.
<pre>2)min_e_plotted/max_e_plotted
</pre>
These refer to the minimum/maximum binding energies to calculate spectral intensity for. The range should probably be slightly wider than the min_e/max_e range really, though thats not the default setting.
<pre>3)step_size
</pre>
The value (in eV) between calculated data points in the output spectra. For example, if set to 0.001eV then a data point will be calculated for 0eV,0.001eV,0.002eV etc. Lower values=larger files/slower runtimes.
<pre>4)lorentz_fraction
</pre>
Value between 0 and 1 that determines the % Lorentzian character of the lineshape used in the broadening function(the remaining % is set to gaussian). By broadening function i essentially mean the peak thats placed around every MO energy in generating these spectra. lorentz_fraction=0 will give you a pure gaussian peak, while lorentz_fraction=1 gives a pure Lorentzian peak.
<pre>5)savetext/figsave/savejpeg/all_contribs_output
</pre>
These all control whether a certain type of output file is saved, with variable names that should be self-explanatory.Values of 1 mean save the file, other values mean dont save. set savetext=0 to not save any .txt output files, figsave=0 to not save any .fig output, savejpeg=0 to not save any .jpeg outputs and all_contribs_output=0 to not save the .txt file with a full breakdown of the spectrum. Leaving them all as 1 (except maybe the savejpeg) is strongly recommended.

====Plot_Atomic_Contribs_V3====

This is an optional script that plots certain breakdowns of your calculated DoS/XPS spectrum (i.e you run it AFTER Gen_Gelius_Spectrum) . Its not too hard to do the same thing in excel using the *ALL_Contribs* output file (see output file sections) but this is a lot faster. There are three types of breakdown this function can do (see pictures below):

1) Atomic breakdown with fragments kept seperate (merge_spd=1,merge_fragments=0)

2) Atomic orbital breakdown (merge_spd=0,merge_fragments=1)

3) Atomic breakdown (merge_spd=1,merge_fragments=1)

The function in its uploaded form takes a single argument , though there are two variables in the actual code that you need to mess with (merge_spd and merge_fragments).

So for example, to run this on the output file from the "Using the Example files" i typed into the matlab terminal:

Plot_Atomic_Contribs_V3('C:\Users\Richard\Desktop\Open_Shell_Code_Some_Stuff\Example_files\other_file\DoS_1ev_0pt3l')

The argument needs to be a folder containing the output files from the Gen_Gelius_Spectrum script. You'll then get a saved graph with the breakdown you asked for and also a .txt file containing the breakdown in a comma delimited format. The output file names record the options you used to make the script (e.g "Plotted_Atomic_Contribs_spd0_frag1.txt" means merge_spd was set to 0 and merge_fragments was set to 1).

[[File:Plotted Atomic Contribs spd1 frag1.jpg|thumbnail|center| merge_spd=1,merge_fragments=1 ]]

[[File:Plotted_Atomic_Contribs_spd0_frag1.jpg|thumbnail|center| merge_spd=0,merge_fragments=1 ]]

[[File:Plotted Atomic Contribs spd1 frag0.jpg|thumbnail|center| merge_spd=1,merge_fragments=0 ]]

==Output Files==

The first "output file" is the *MCRF*.txt file, which just contains the commands to build the required fragment in multiwfn.

===Files From Multiwfn===
Most people should be able to completely ignore these files. But in case your curious:

'''1) *DOS_curve*.txt''' Essentially a DoS plot. 1st column is energies in au, second column is TDOS of the fragment. Not really sure what the 3rd column is. No particular use for this file, but it gets auto-generated by multiwfn and i dont go out my way to delete it.

'''2) *DOS_line*.txt''' Data to create a stick plot (where each MO is a stick). Plot column 1 vs column 2 as a line graph and you get an axis with sticks representing the MOs. 4th column onwards represent contribution of the various sub-fragments to each "stick". This is the file that gets used to generate the *ML_inpt.txt file.

'''3) *DOS_frag*.txt ''' This file contains the sub-fragment info in a way multiwfn understands. If its renamed to DOS_frag.txt then multiwfn can import it to get the fragments (therefore can be useful for debugging, no further purpose).

===MLinpt.txt File===

Creation of the *ML_inpt.txt files is essentially the point of the gen_gelius_input* script. These are primarily input files for matlab, but they do have useful information in an easy to read format. If you import the file into excel and COMMA DELIMIT ONLY (i.e untick the space delimit option) you can get something like below(there are more decimal places in the actual file):

<pre>
#M_FACTOR=1.0,
#Energy(eV) TDOS CO4S CO3P CO3D N2S N2P N2D C2S C2P C2D S3S S3P S3D
[LINES DELETED]
-24.5 0.9 0.0 0.0 0.0 0.5 0.1 0.0 0.2 0.1 0.0 0.1 0.0 0.0
-24.4 1.0 0.0 0.0 0.0 0.5 0.1 0.0 0.3 0.1 0.0 0.0 0.0 0.0
-24.4 1.0 0.0 0.0 0.0 0.5 0.1 0.0 0.3 0.1 0.0 0.0 0.0 0.0
-24.4 1.0 0.0 0.0 0.0 0.5 0.1 0.0 0.3 0.1 0.0 0.0 0.0 0.0
-19.8 0.3 0.0 0.0 0.0 0.1 -0.1 0.0 0.2 0.1 0.0 0.7 0.1 0.0
-19.7 0.3 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.1 0.0 0.7 0.1 0.0
[LINES DELETED]
</pre>

The shown file is "all_frags_alpha_MLinpt.txt", which is generated from the example files in the open shell code. As this actually comes from merging two MLinpt.txt files (gen_gelius_input.sh does this part silently) the TDOS in this case doesnt mean anything.

The first column lists the energy of each MO in eV (1 ev ~27.212 au). For any row, columns 3(CO4S)-->end (S3D) list the fractional contributions of each type of atmoic orbital to the MO in that row. For example the MO with energy -19.7 eV is mainly from S3s type atomic orbitals. These fractions were determiend by multiwfn by using Mulliken population analysis.

All the fractional contributions in the file get multiplied by the M_FACTOR value (as explained above in the Creating your own input files section). Therefore the value of M_FACTOR can easily be set after the MLinpt.txt file is created, as opposed to within the initial input file.

===Formatted_UMOs===

The default for the gen_gelius_input* script (i.e the settings for the version i uploaded) is to put only OCCUPIED MO energies/breakdowns into the *MLinpt.txt file. If you want all MOs (I really DONT recommend this) in the MLinpt.txt file then change all_orbs='cheese' to all_orbs='ALLORBS' at the top of the gen_gelius_input* script.

Otherwise you will get a *Formatted_UMOs* file for every MLinpt.txt file you get. The format of the file is identical to the MLinpt.txt format except the MO at the top of the file corresponds to the LUMO.

WARNING: If you use the link_files feature, then the output formatted_UMOs files DO NOT get merged. This shouldnt matter as you should never really use the formatted UMOs for anything.

===Output From Matlab Code ===

The matlab code firstly will create a new folder with a name reflecting the input arguments. The format is hv_FWHM_%lorenztian where hv is the photon energy, FWHM is full width half maximum of the broadeing function and %lorenztian is the amount of lorenztian mixed in the broadening function. For example the earlier command (from using example files section) "Gen_Gelius_Spectrum(plist,-1,1)" generates the folder "DoS_1ev_0pt3l". DoS = Density of states, and means we did not use photoionisation cross sections. The 0pt3l part means our broadening function was 30% lorenztian and 70% Gaussian.

Two .fig files will have been produced from this code, these contain matlab graphs of the output. These .fig files follow the naming ("filename" is the name of the original fchk file):

1) filename_DoS.fig contains the total DoS plot.

2) filename_DoS_frag_contribs.fig contains the DoS AND the pDoS of each of the fragments. For closed shell code you get 1 fragment per input file (assuming you dont use link_files), but for the open shell code each input file leads to two fragments - one for alpha electrons and one for beta electrons.

[[File:Other_file_DoS.jpeg|thumb|center| 1) Total Density of States for a Co(SCN)4 fragment with filename other_file_DoS.fig]]

[[File:Other file DoS frag contribs.jpeg|thumb|center| 2) Contribtuions of alpha and beta electrons to the Total Density of States for Co(SCN)4, filename other_file_DoS_frag_contribs.fig]]

The filename_DoS.fig plot data is output in a .txt file of the same name, while each fragment in filename_DoS_frag_contribs.fig has its own .txt file. For the example above the two fragments have names "all_frags_alpha_other_file_DoS" and "all_frags_beta_other_file_DoS". The start of the names are taken from the name of the original input file "all_frags.test".

The final file output is the "ALL_Contribs_other_file_DoS.txt" file. This(as the name implies) contains all contributions to the DoS spectra in a comma delimited file. The Plot_Atomic_Contribs*.m code can be run on this file to generate plots with a few different breakdowns (e.g atomic breakdowns) - see the relevant section of this wiki (not done yet....). The file will look something like below (commas replaced with tabs):

<pre>
#all_frags_alpha all_frags_beta
#Sub fragments in each= 12 12
#Energy(eV) CO4S CO3P CO3D N2S N2P N2D C2S C2P C2D [COLUMNS REMOVED]
[ROWS REMOVED,numbers rounded]
7.5 0.00 0.01 0.31 0.04 0.13 0.00 0.00 0.11 0.00 [COLUMNS REMOVED]
7.51 0.00 0.01 0.33 0.04 0.13 0.00 0.00 0.11 0.00 [COLUMNS REMOVED]
7.52 0.00 0.01 0.34 0.04 0.13 0.00 0.00 0.11 0.00 [COLUMNS REMOVED]
7.53 0.00 0.01 0.35 0.04 0.14 0.00 0.00 0.12 0.00 [COLUMNS REMOVED]
7.54 0.00 0.01 0.36 0.04 0.14 0.00 0.00 0.12 0.00 [COLUMNS REMOVED]
7.55 0.00 0.02 0.37 0.04 0.14 0.00 0.00 0.12 0.00 [COLUMNS REMOVED]
7.56 0.00 0.02 0.39 0.05 0.14 0.00 0.00 0.13 0.00 [COLUMNS REMOVED]
[ROWS REMOVED]
</pre>

Hopefully the format is mostly intuitive, the top two rows contain information on what the data is, and the rest contain the data. The top row contains (originally comma delimited) a list of all "fragments" names, which is the number of input files for the closed shell code and 2x the number of input files for the open shell code. The second row contains the number of sub-fragments (which should be atomic orbitals) in each fragment. In the example above each fragment had 12 types of atomic orbital.

So for the example above column 2-14 shows contributions from "all_frags_alpha" while column 15-27 shows contributions from "all_frags_beta" (column 1 lists energies).

==Bugs/Things to Watch For==
===Fragment Names===
Make sure your fragment names arent a substring of your .fchk file names. For example an input file called "Cl.test" wont work for a file called "BMIM_Cl.fchk". However, "Cl_frag.test" will work. Therefore personally i add _frag to the end of every input file name. The other way to sort this is to set linked_files=0 in gen_gelius_input_opensh.sh, sincce the link files part of the code is where this bug comes from.

===Fragment Error Checking===
Unless you manually turn it off (The "error=" option in gen_gelius_input_opensh.sh script) the code tries to check your input file for errors. Specifically it tests whether the TDOS in an input_file really is the sum of the other sub-fragments in the input(.test) file. E.g it will trigger if you request only Sulfur p-type and d-type AOs (but no s-type AOs) in the input file, or if you request all sulfur p-type AOs twice. If the error check triggers then it BREAKS the input file for matlab by putting an error msg in the first line of the '''*MLinpt.txt file'''. An example of what the error looks like is:
<pre>
“Fragment Sum incorrect on row 19 for fchk file xxxx.fchk”
</pre>

If you *MLinpt.txt files arent working in matlab then this is probably the reason.

===Pathlengths===
This code makes a lot of sub-folders based on the name of .fchk files. If you have really long .fchk file names its likely that you'll end up with some files which exceed the 256 character limit for a pathlength on a windows computer. In matlab 2014 this will cause the code to exit with an error, saying it cant save the .fig files. In newer matlab versions it may just save the output files , but you wont be able to open them on windows if the path length is greater than 256 characters. Ways to avoid this are to shorten .fchk file names where possible, or (less practical) run the code on a folder as close as possible to root (i.e a folder with a short a pathlength as possible).

File:Plot Atomic Contribs V3.m

2017-07-25T10:08:55Z

Rf614: Matlab script used with code to generate pDoS/Valence XPS spectra

Matlab script used with code to generate pDoS/Valence XPS spectra

File:Gelius open shell code Aug2016.zip

2017-07-24T15:58:59Z

Rf614: Rf614 uploaded a new version of File:Gelius open shell code Aug2016.zip

All code required to calculate XP spectra/pDoS from a gaussian .fchk file.

File:Gelius closed shell code Aug2016.zip

2017-07-24T15:58:18Z

Rf614: Rf614 uploaded a new version of File:Gelius closed shell code Aug2016.zip

All files needed to calculate pDoS/XPS spectra for closed shell .fchk gaussian files.

Talk:Mod:Hunt Research Group/Calc XPS Code

2017-06-07T14:52:29Z

Rf614: /* Some Theory */

==Links to Files==
Open Shell Version: [[Media:Gelius_open_shell_code_Aug2016.zip]]

Closed Shell Version: [[Media:Gelius_closed_shell_code_Aug2016.zip]]

==Introduction==
The purpose of this code is to calculate partial density of states (pDoS) plots and Valence band x-ray photoelectron (XP) spectra from .fchk files. There is both an open shell and a closed shell version of the code, they work in exactly the same way.

All code was written using python 2.7 (with numpy), Multiwfn version 3.3.8, Matlab version 2014a and a cygwin(bash) shell on windows 8. It should work on windows or linux if the correct options are set. It might not work on a mac currently due to Multiwfn not working properly (I haven't personally tested it though).

==Some Theory==

===Mulliken Population Analysis===

A pretty key part to this code is the breakdown of molecular orbitals (MOs) in atomic orbitals (AOs, which are essentially basis functions). This is carried out in multiwfn using Mulliken population analysis, which i'll briefly summarise due to its importance.

The basic question is "For MO X, how many electrons come from atomic orbital Y?". Note im going to be using "basis functions" instead of "atomic orbitals" from this point, in terms of DFT calculations we interpret a basis function centered on atom X as an atomic orbital of atom X.We'll define <math>\psi</math> as the wavefunction for a '''single MO containing a single electron'''. Next we write <math>\psi</math> as the linear combination of individual basis functions (<math>\phi_{i}</math>):

<math>
\psi = {{\sum\limits_{i} c_{i} \phi_{i} }}
</math>

Since we want a number of electrons, we're interested in the formula for the electron density (<math>\rho</math>) of this MO:

<math>
\rho = \langle{} \Psi | \Psi \rangle{}
</math>

<math>
\langle{}\Psi | \Psi \rangle{} = ({{\sum\limits_{i} c_{i} \phi_{i} }})({{\sum\limits_{i} c_{i} \phi_{i} }})
</math>

<math>
\langle{}\Psi | \Psi \rangle{} = ({{\sum\limits_{i} c_{i}^{2} \phi_{i}^{2} }}) + 2 {{ \sum\limits_{i>j}}} {{\sum\limits_{j} c_{i}c_{j} \langle{}\phi_{j} | \phi_{i} \rangle{} }}
</math>

If these formulae are tricky to follow, try doing it for a system with just two basis functions.
Note that in the third line the indice ''j'' runs over the same coefficients and basis functions as the indice ''i'', its just used to show that the <math>{{ \sum\limits_{i>j}}} {{\sum\limits_{j} c_{i}c_{j} \langle{}\phi_{j} | \phi_{i} \rangle{} }}</math> term only contains products between 2 different basis functions/co-efficients.
Hence, we now have a formula for the MO density which is purely in terms of (known) basis functions and (known) coefficients. Now its just a case of working out how we should interpret these values.

Firstly note that <math>\phi_{i}^{2}=1</math> (since we use normalised basis functions) hence:

<math>
{{\sum\limits_{i} c_{i}^{2} \phi_{i}^{2} }} = {{\sum\limits_{i} c_{i}^{2}}}
</math>

The values of c_{i} in this expression are easy to interpret, they represent the fractional contribution of AO ''i'' to the wavefunction. The <math>2 {{ \sum\limits_{i>j}}} {{\sum\limits_{j} c_{i}c_{j} \langle{}\phi_{j} | \phi_{i} \rangle{} }}</math> term is more difficult to interpret, each expression in the series can be thought of as the number of electrons shared between two basis functions. i.e. the following term would represent the number of electrons shared between basis function 1 and 2:

<math>
n_{shared} = 2 c_{1}c_{2} \langle{}\phi_{1} | \phi_{2} \rangle{}
</math>

However, we need to split this "shared" electron density between the two contributing basis functions. In Mulliken Population analysis "shared" electron density is divided equally between the two contributing basis functions. So the total electron density assigned to one basis function can be written as:

<math>
n_{i} = c_{i}^{2} + {{\sum\limits_{i \neq j} c_{i}c_{j} \langle{}\phi_{j} | \phi_{i} \rangle{} }}
</math>

<math>n_{i}</math> represents the '''fractional''' contribution of ''i'' to the wavefunction. Hence if you had a 2-electron MO, you'd simply multiply <math>n_{i}</math> by 2.

===What the code actually does (roughly/briefly)===

Step 1 - The first step is to run a Mulliken population analysis on the *.fchk file. This leads to a file containing the energy/AO composition of each MO in the system (the *MLinpt.txt file). Subsequent steps vary depending on if you want partial density of states (pDoS) plots or calculated valence spectra.

If '''calculating pDoS''' (not XPS spectra):

For each type of atomic orbital present do the following:

- For each MO, center a gaussian-lorentzian function at that MO energy. Multiply the intensity by the contribution of the current AO to the current MO.

- Sum all the resultant peaks, this generates the pDOS the current atomic orbital

Once all pDoS are generated, the total DoS could be created by summing them all. In practice the code probably just calculates the total DoS first, by just placing a gaussian-lorenztian with area=1 around each MO energy.

If '''calculating XPS spectra''':

For each type of atomic orbital present do the following:

- Multiply the fractional contributions of this AO to each MO by the AO cross-section. This value will be reffered to as the "intensity" This cross-section depends on both the AO type and the photon energy. Default values in this code are stored in tables in X_Section_Library, and custom/different values can easily be added.

- For each MO, center a gaussian-lorentzian function at that MO energy. This area of this function is equal to the intensity calculated in the step above.

Sum all the resultants peaks, this generates the contribution of the current AO to the valence XPS spectrum
Once this is repeated for each AO type, sum all those individual contributions to generate the total XPS valence spectrum.

=="Installation" Instructions==

All instructions will be given for the open shell version of the code. However, the process is the same except whenever these instructions say to edit the "gen_gelius_input_opensh.sh", just do the corresponding edit on the gen_gelius_input.sh file. The Matlab scripts are EXACTLY the same for both open shell/closed shell codes.

0) Make sure you have multiwfn (https://multiwfn.codeplex.com) saved somewhere, python 2.x and numpy installed.

1)Put the following files (all the .py/.sh files) somewhere in system path (e.g i use /usr/local/bin):
:a)Format_Multiwfn_Output_alpha.py (Format_Multiwfn_Output.py for closed shell)
:b)Format_Multilwfn_Output_beta.py
:c)Gen_Multiwfn_Input_Comms.py
:d)Gen_Multiwfn_Input_Comms_beta.py
:e)Merge_Fragment_Files.py
:f)gen_gelius_input_opensh.sh

Make sure all files have executable persmission (navigate to the folder and do "chmod u+x filename")

2)In the gen_gelius_input_opensh.sh change the current lines accordingly:
:a) multiwfn_folder = 'C:/Users/Richard/Documents/Multiwfn_Installed' to the directory where you store multiwfn and associated files.
:b) "cygwin=1” to “cygwin=0” if your not using a Cygwin shell.
:c) “old_sed=1” to “old_sed=0” if your version of sed allows the –i option (edit in place) to be used. I don’t think the standard version on mac does(meaning set old_sed=1),standard version of Cygwin does though.
:d) POSSIBLY "cp $multiwfn_folder/Multiwfn.exe $curr_folder" to "cp $multiwfn_folder/Multiwfn $curr_folder" if on linux(should be really obvious if this is required). Do the same for the line where multiwfn is deleted from current folder("rm Multiwfn.exe").

3)Put the following files somewhere in matlabs path (e.g your matlab folder in documents)
:a) Get_X_Section.m
:b) Gen_Gelius_Spectrum.m

The Get_X_Section REALLY does need to be in matlabs path (unless you want to edit some code in Gen_Gelius_Spectrum.m). If you know what your doing in matlab feel free to put Gen_Gelius_Spectrum.m wherever you want.

4)Put the X_Section_Library folder wherever you want.

5) In Get_X_Section.m modify the following line:
:"Lib_Path ='C:\Users\Richard\Documents\X_Section_Library' " to the path where you put the X_Section_Library folder in.

6) OPTIONAL: Add the code in vim_template_script to your vimrc (c+p it somewhere in the file). "vim ~/.vimrc" to access your vimrc. You can then bring up a template input file in vim by using ":call Gelius_Input()"
Since i forgot to put the relevant text file in to the code folders, just c+p the below into your vimrc to get this to work.

<pre>
function! Gelius_Input()
exe 'normal aCoSCN4_freq_AllNCS_B3LYP_6311plgdp_Tetrahed_BMIMSCNSMD_Abr.fchk\rNumb_Fragments=10\r1a=\r1t=a\r1l=TDOS\r2a=\r2t=\r2l=\r3a=\r3t=\r3l=\r4a=\r4t=\r4l=\r5a=\r5t=\r5l=\r6a=\r6t=\r6l=\r7a=\r7t=\r7l=\r8a=\r8t=\r8l=\r9a=\r9t=\r9l=\r10a=\r10t=\r10l=\rM_factor=1.0\rLink_File='
exe '%s/\\r/\r/g'
endfunction
</pre>

==Running The Code==
====Using the Example files====
1) Navigate (in your shell) to the folder "Example_files". e.g for me i ran:

"cd /cygdrive/c/Users/Richard/Documents/Random_Coding/Gelius_Spectrum_Automated_Jan_2016/open_shell_code/Example_files"

2) Run this command in console(for closed shell use gen_gelius_input.sh):

"gen_gelius_input_opensh.sh"

Command 2) will have created a folder for each .fchk file in Example_files/. Therefore you should now have folders:
:CoSCN4_freq_AllNCS_B3LYP_6311plgdp_Tetrahed_BMIMSCNSMD_Abr
:other_file
Within these folders are tons of output files. The most important ones are those ending in "MLinpt.txt" - these will be the input files for step 3.

The script will run on all .fchk files in the current folder, when you do this for real you need to make sure that all the fchks are for the same system (e.g a water molecule) and that the atomic numbering is the same in all files (view-->labels in gaussview gives you what im calling atomic numbering).

If anything went wrong try revisiting the installation instructions.

3) Everything from now on is done using matlab. Theres a file called "pathname_file" in the Example_files folder which was created by the gen_gelius_input_opensh.sh script. Copy its full contents to clipboard and in MATLAB terminal type:

<pre>
plist=<CTRL-V>
(<CTRL-V> means paste, rather than type out CTRL-V)
</pre>

In matlab terms, plist now contains a cell array where each entry is the full path to a folder. The folder paths are those for the folders created in step 2). Hence to do this manually for just one of the two folders you'd type something like:
<pre>
plist={'C:\Users\Richard\Documents\Random_Coding\Gelius_Spectrum_Automated_Jan_2016\open_shell_code\Example_files\other_file'}
</pre>

4) To create either an XPS spectrum or pDoS plots we use the Gen_Gelius_Spectrum script. This takes the following three arguments IN THE FOLLOWING ORDER:
:a) the plist variable we just made
:b) the photon energy we want to use (any negative number requests a pDoS rather than an XPS spectrum)
:c) the FWHM of the peaks to place around each MO energy (units = eV)

So to create a pDoS plot you could type the following into the matlab terminal(Try this one first):
<pre>
Gen_Gelius_Spectrum(plist,-1,1)
</pre>
To create an XPS spectrum taken at photon energy of 1486eV (standard lab-based value) type the following:
<pre>
Gen_Gelius_Spectrum(plist,1486,1)
</pre>

If you get an error like "Undefined function 'Gen_Gelius_Spectrum' for input arguments of type 'cell'.", then your Gen_Gelius_Spectrum script probably isnt in your matlab path. (see installation instructions, or google a bit)

If your error free then well done. Sub-folders should have been created in "other_file" and "CoSCN4_freq_AllNCS_B3LYP_6311plgdp_Tetrahed_BMIMSCNSMD_Abr", which in turn will contain a loada output files. See the output files section of the wiki for an explanation of what these are.

====Creating your own input files====
If you followed the optional part above and made a vimscript, then a template for the input file can be brought up in vim by typing ":call Gelius_Input()"

An example input file is shown below. Each "fragment" (e.g cation,anion, alkyl group etc.) should have its own input file.
<pre>
other_file.fchk
Numb_Fragments=4
1a=10-13
1t=a
1l=TDOS
2a=10-13
2t=s
2l=S3s
3a=10-13
3t=p
3l=S3p
4a=10-13
4t=d+
4l=S3d
M_factor=1.0
Link_File=all_frags.test
</pre>
The input file can be divided into parts as follows:

1) The top line of the input file tells the code the name of the checkpoint file to use in the calculation. In practice you can write ANYTHING on this top line if using the gen_gelius_input*.sh script, as the script itself replaces the first line (whatevers there) with the names of all .fchks in the current folder. But SOMETHING must be written there.

2)'''Numb_Fragments''' is the number of sub-fragments (max 10) that you want to break your pDoS down into (e.g fragment 2 corresponds to all Sulfur 3s atomic orbitals in the example above). Code will break if you get this wrong.

3) '''1a/1t/1l''' instruct the code what to use as the total density of states. Hence 1a must be set to ALL the atoms your interested in for this fragment, 1t must be set to "a" and 1l should be set to TDOS (not sure it matters what this is set to, but to be on the safe side just use TDOS). The purpose of this fragment is essentially error checking.

4) '''2a/3a/*a''' : a stands for atoms here, so the value of a corresponds to the labels of the atoms you want to be included in the sub-fragment. These labels are the same integers you get in gaussview when you go to view-->labels. Examples of valid options may be "1,4,5-7" or "1,3,5" (basically comma seperated list of integers)

5) '''2t/3t/*t''': t stands for "type of AO".This is a comma seperated list with possible arguments (any combination) "a,s,p,d,f,g". "a" corresponds to all types of AO centered of the atoms specified with the 2a/3a/*a line. s means s-type atomc orbitals (0 angular momentum), p means p-type atomic orbitals (1 angular momentum) etc. To get all AOs of a certain angular momentum and higher (up to g-type AOs) add a plus to the end of the relevant letter. e.g "p+" says "include all AOs with angular momemtnum greater than or equal to 1 in this sub-fragment")
'''DON'T''' use "a" with any other letter, and dont do anything else that will lead to a type of AO being requested twice (e.g p+,d)

6) '''2l/3l/*l''': l stands for label. If you dont care about calculating an XPS spectrum you can essentially set it to whatever you want. This tells the matlab script what cross section to use for calculating the XPS spectrum, hence it must match (case insensitive) a filename in the X_Section_Library folder. e.g S3s tells matlab to get the S3S.txt file when looking for what cross-section to use.

7) '''M_factor=1.0''', you really never need to modify this but make sure its present. It's purpose is to tell matlab to multiple the pDoS of this fragment (that this input file corresponds to) by the value M_factor. The purpose is for systems where the real stoichiometry differs from that in your calculation. For example if i have a mixture of 99 water:1 anion but only a calculation on 1 water molecule mixed with 1 anion i could set M_factor for the water fragment input file to M_factor=99.0 to estimate (badly) what the actual DoS of the mixture would look like.

8) '''Link_File=*'''. This options for if you have a fragment (e.g an anion) but want it broken into more than 10 sub-fragments (e.g this will be true for any molecule with >3 elements). In this case make 2 files for that fragment, and in the second file set Link_File to the name of the first file. In the example above, this input file will essentially be merged with "all_frags.test", and only an output file for all_frags.test will be generated.

====More complex input file options====
====Gen_Gelius_Spectrum.m====
This code has the following options that can be changed (all are explained by comments in the .m file):
<pre>1)min_e/max_e
</pre>
Matlab ignores any orbitals with energy below min_e or above max_e when generating the XPS spectrum. Note if you want to change this to include core orbitals or very high energy virtual orbitals then you'll also have to edit Gen_Multiwfn_Input_Comms.py to set the range in multiwfn to the one you want.(Theres currently no variable in that .py code to do that, it just uses the defaults in multiwfn. But the function you'll need to edit in Gen_Multiwfn_Input_Comms.py is Output_Header.
<pre>2)min_e_plotted/max_e_plotted
</pre>
These refer to the minimum/maximum binding energies to calculate spectral intensity for. The range should probably be slightly wider than the min_e/max_e range really, though thats not the default setting.
<pre>3)step_size
</pre>
The value (in eV) between calculated data points in the output spectra. For example, if set to 0.001eV then a data point will be calculated for 0eV,0.001eV,0.002eV etc. Lower values=larger files/slower runtimes.
<pre>4)lorentz_fraction
</pre>
Value between 0 and 1 that determines the % Lorentzian character of the lineshape used in the broadening function(the remaining % is set to gaussian). By broadening function i essentially mean the peak thats placed around every MO energy in generating these spectra. lorentz_fraction=0 will give you a pure gaussian peak, while lorentz_fraction=1 gives a pure Lorentzian peak.
<pre>5)savetext/figsave/savejpeg/all_contribs_output
</pre>
These all control whether a certain type of output file is saved, with variable names that should be self-explanatory.Values of 1 mean save the file, other values mean dont save. set savetext=0 to not save any .txt output files, figsave=0 to not save any .fig output, savejpeg=0 to not save any .jpeg outputs and all_contribs_output=0 to not save the .txt file with a full breakdown of the spectrum. Leaving them all as 1 (except maybe the savejpeg) is strongly recommended.

====Plot_Atomic_Contribs_V3====

This is an optional script that plots certain breakdowns of your calculated DoS/XPS spectrum (i.e you run it AFTER Gen_Gelius_Spectrum) . Its not too hard to do the same thing in excel using the *ALL_Contribs* output file (see output file sections) but this is a lot faster. There are three types of breakdown this function can do (see pictures below):

1) Atomic breakdown with fragments kept seperate (merge_spd=1,merge_fragments=0)

2) Atomic orbital breakdown (merge_spd=0,merge_fragments=1)

3) Atomic breakdown (merge_spd=1,merge_fragments=1)

The function in its uploaded form takes a single argument , though there are two variables in the actual code that you need to mess with (merge_spd and merge_fragments).

So for example, to run this on the output file from the "Using the Example files" i typed into the matlab terminal:

Plot_Atomic_Contribs_V3('C:\Users\Richard\Desktop\Open_Shell_Code_Some_Stuff\Example_files\other_file\DoS_1ev_0pt3l')

The argument needs to be a folder containing the output files from the Gen_Gelius_Spectrum script. You'll then get a saved graph with the breakdown you asked for and also a .txt file containing the breakdown in a comma delimited format. The output file names record the options you used to make the script (e.g "Plotted_Atomic_Contribs_spd0_frag1.txt" means merge_spd was set to 0 and merge_fragments was set to 1).

[[File:Plotted Atomic Contribs spd1 frag1.jpg|thumbnail|center| merge_spd=1,merge_fragments=1 ]]

[[File:Plotted_Atomic_Contribs_spd0_frag1.jpg|thumbnail|center| merge_spd=0,merge_fragments=1 ]]

[[File:Plotted Atomic Contribs spd1 frag0.jpg|thumbnail|center| merge_spd=1,merge_fragments=0 ]]

==Output Files==

The first "output file" is the *MCRF*.txt file, which just contains the commands to build the required fragment in multiwfn.

===Files From Multiwfn===
Most people should be able to completely ignore these files. But in case your curious:

'''1) *DOS_curve*.txt''' Essentially a DoS plot. 1st column is energies in au, second column is TDOS of the fragment. Not really sure what the 3rd column is. No particular use for this file, but it gets auto-generated by multiwfn and i dont go out my way to delete it.

'''2) *DOS_line*.txt''' Data to create a stick plot (where each MO is a stick). Plot column 1 vs column 2 as a line graph and you get an axis with sticks representing the MOs. 4th column onwards represent contribution of the various sub-fragments to each "stick". This is the file that gets used to generate the *ML_inpt.txt file.

'''3) *DOS_frag*.txt ''' This file contains the sub-fragment info in a way multiwfn understands. If its renamed to DOS_frag.txt then multiwfn can import it to get the fragments (therefore can be useful for debugging, no further purpose).

===MLinpt.txt File===

Creation of the *ML_inpt.txt files is essentially the point of the gen_gelius_input* script. These are primarily input files for matlab, but they do have useful information in an easy to read format. If you import the file into excel and COMMA DELIMIT ONLY (i.e untick the space delimit option) you can get something like below(there are more decimal places in the actual file):

<pre>
#M_FACTOR=1.0,
#Energy(eV) TDOS CO4S CO3P CO3D N2S N2P N2D C2S C2P C2D S3S S3P S3D
[LINES DELETED]
-24.5 0.9 0.0 0.0 0.0 0.5 0.1 0.0 0.2 0.1 0.0 0.1 0.0 0.0
-24.4 1.0 0.0 0.0 0.0 0.5 0.1 0.0 0.3 0.1 0.0 0.0 0.0 0.0
-24.4 1.0 0.0 0.0 0.0 0.5 0.1 0.0 0.3 0.1 0.0 0.0 0.0 0.0
-24.4 1.0 0.0 0.0 0.0 0.5 0.1 0.0 0.3 0.1 0.0 0.0 0.0 0.0
-19.8 0.3 0.0 0.0 0.0 0.1 -0.1 0.0 0.2 0.1 0.0 0.7 0.1 0.0
-19.7 0.3 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.1 0.0 0.7 0.1 0.0
[LINES DELETED]
</pre>

The shown file is "all_frags_alpha_MLinpt.txt", which is generated from the example files in the open shell code. As this actually comes from merging two MLinpt.txt files (gen_gelius_input.sh does this part silently) the TDOS in this case doesnt mean anything.

The first column lists the energy of each MO in eV (1 ev ~27.212 au). For any row, columns 3(CO4S)-->end (S3D) list the fractional contributions of each type of atmoic orbital to the MO in that row. For example the MO with energy -19.7 eV is mainly from S3s type atomic orbitals. These fractions were determiend by multiwfn by using Mulliken population analysis.

All the fractional contributions in the file get multiplied by the M_FACTOR value (as explained above in the Creating your own input files section). Therefore the value of M_FACTOR can easily be set after the MLinpt.txt file is created, as opposed to within the initial input file.

===Formatted_UMOs===

The default for the gen_gelius_input* script (i.e the settings for the version i uploaded) is to put only OCCUPIED MO energies/breakdowns into the *MLinpt.txt file. If you want all MOs (I really DONT recommend this) in the MLinpt.txt file then change all_orbs='cheese' to all_orbs='ALLORBS' at the top of the gen_gelius_input* script.

Otherwise you will get a *Formatted_UMOs* file for every MLinpt.txt file you get. The format of the file is identical to the MLinpt.txt format except the MO at the top of the file corresponds to the LUMO.

WARNING: If you use the link_files feature, then the output formatted_UMOs files DO NOT get merged. This shouldnt matter as you should never really use the formatted UMOs for anything.

===Output From Matlab Code ===

The matlab code firstly will create a new folder with a name reflecting the input arguments. The format is hv_FWHM_%lorenztian where hv is the photon energy, FWHM is full width half maximum of the broadeing function and %lorenztian is the amount of lorenztian mixed in the broadening function. For example the earlier command (from using example files section) "Gen_Gelius_Spectrum(plist,-1,1)" generates the folder "DoS_1ev_0pt3l". DoS = Density of states, and means we did not use photoionisation cross sections. The 0pt3l part means our broadening function was 30% lorenztian and 70% Gaussian.

Two .fig files will have been produced from this code, these contain matlab graphs of the output. These .fig files follow the naming ("filename" is the name of the original fchk file):

1) filename_DoS.fig contains the total DoS plot.

2) filename_DoS_frag_contribs.fig contains the DoS AND the pDoS of each of the fragments. For closed shell code you get 1 fragment per input file (assuming you dont use link_files), but for the open shell code each input file leads to two fragments - one for alpha electrons and one for beta electrons.

[[File:Other_file_DoS.jpeg|thumb|center| 1) Total Density of States for a Co(SCN)4 fragment with filename other_file_DoS.fig]]

[[File:Other file DoS frag contribs.jpeg|thumb|center| 2) Contribtuions of alpha and beta electrons to the Total Density of States for Co(SCN)4, filename other_file_DoS_frag_contribs.fig]]

The filename_DoS.fig plot data is output in a .txt file of the same name, while each fragment in filename_DoS_frag_contribs.fig has its own .txt file. For the example above the two fragments have names "all_frags_alpha_other_file_DoS" and "all_frags_beta_other_file_DoS". The start of the names are taken from the name of the original input file "all_frags.test".

The final file output is the "ALL_Contribs_other_file_DoS.txt" file. This(as the name implies) contains all contributions to the DoS spectra in a comma delimited file. The Plot_Atomic_Contribs*.m code can be run on this file to generate plots with a few different breakdowns (e.g atomic breakdowns) - see the relevant section of this wiki (not done yet....). The file will look something like below (commas replaced with tabs):

<pre>
#all_frags_alpha all_frags_beta
#Sub fragments in each= 12 12
#Energy(eV) CO4S CO3P CO3D N2S N2P N2D C2S C2P C2D [COLUMNS REMOVED]
[ROWS REMOVED,numbers rounded]
7.5 0.00 0.01 0.31 0.04 0.13 0.00 0.00 0.11 0.00 [COLUMNS REMOVED]
7.51 0.00 0.01 0.33 0.04 0.13 0.00 0.00 0.11 0.00 [COLUMNS REMOVED]
7.52 0.00 0.01 0.34 0.04 0.13 0.00 0.00 0.11 0.00 [COLUMNS REMOVED]
7.53 0.00 0.01 0.35 0.04 0.14 0.00 0.00 0.12 0.00 [COLUMNS REMOVED]
7.54 0.00 0.01 0.36 0.04 0.14 0.00 0.00 0.12 0.00 [COLUMNS REMOVED]
7.55 0.00 0.02 0.37 0.04 0.14 0.00 0.00 0.12 0.00 [COLUMNS REMOVED]
7.56 0.00 0.02 0.39 0.05 0.14 0.00 0.00 0.13 0.00 [COLUMNS REMOVED]
[ROWS REMOVED]
</pre>

Hopefully the format is mostly intuitive, the top two rows contain information on what the data is, and the rest contain the data. The top row contains (originally comma delimited) a list of all "fragments" names, which is the number of input files for the closed shell code and 2x the number of input files for the open shell code. The second row contains the number of sub-fragments (which should be atomic orbitals) in each fragment. In the example above each fragment had 12 types of atomic orbital.

So for the example above column 2-14 shows contributions from "all_frags_alpha" while column 15-27 shows contributions from "all_frags_beta" (column 1 lists energies).

==Bugs/Things to Watch For==
===Fragment Names===
Make sure your fragment names arent a substring of your .fchk file names. For example an input file called "Cl.test" wont work for a file called "BMIM_Cl.fchk". However, "Cl_frag.test" will work. Therefore personally i add _frag to the end of every input file name. The other way to sort this is to set linked_files=0 in gen_gelius_input_opensh.sh, sincce the link files part of the code is where this bug comes from.

===Fragment Error Checking===
Unless you manually turn it off (The "error=" option in gen_gelius_input_opensh.sh script) the code tries to check your input file for errors. Specifically it tests whether the TDOS in an input_file really is the sum of the other sub-fragments in the input(.test) file. E.g it will trigger if you request only Sulfur p-type and d-type AOs (but no s-type AOs) in the input file, or if you request all sulfur p-type AOs twice. If the error check triggers then it BREAKS the input file for matlab by putting an error msg in the first line of the '''*MLinpt.txt file'''. An example of what the error looks like is:
<pre>
“Fragment Sum incorrect on row 19 for fchk file xxxx.fchk”
</pre>

If you *MLinpt.txt files arent working in matlab then this is probably the reason.

===Pathlengths===
This code makes a lot of sub-folders based on the name of .fchk files. If you have really long .fchk file names its likely that you'll end up with some files which exceed the 256 character limit for a pathlength on a windows computer. In matlab 2014 this will cause the code to exit with an error, saying it cant save the .fig files. In newer matlab versions it may just save the output files , but you wont be able to open them on windows if the path length is greater than 256 characters. Ways to avoid this are to shorten .fchk file names where possible, or (less practical) run the code on a folder as close as possible to root (i.e a folder with a short a pathlength as possible).

Talk:Mod:Hunt Research Group/Calc XPS Code

2017-06-07T14:29:41Z

Rf614: /* Some Theory */

==Links to Files==
Open Shell Version: [[Media:Gelius_open_shell_code_Aug2016.zip]]

Closed Shell Version: [[Media:Gelius_closed_shell_code_Aug2016.zip]]

==Introduction==
The purpose of this code is to calculate partial density of states (pDoS) plots and Valence band x-ray photoelectron (XP) spectra from .fchk files. There is both an open shell and a closed shell version of the code, they work in exactly the same way.

All code was written using python 2.7 (with numpy), Multiwfn version 3.3.8, Matlab version 2014a and a cygwin(bash) shell on windows 8. It should work on windows or linux if the correct options are set. It might not work on a mac currently due to Multiwfn not working properly (I haven't personally tested it though).

==Some Theory==

===Mulliken Population Analysis===

A pretty key part to this code is the breakdown of molecular orbitals (MOs) in atomic orbitals (AOs, which are essentially basis functions). This is carried out in multiwfn using Mulliken population analysis, which i'll briefly summarise due to its importance.

The basic question is "For MO X, how many electrons come from atomic orbital Y?". Note im going to be using "basis functions" instead of "atomic orbitals" from this point, in terms of DFT calculations we interpret a basis function centered on atom X as an atomic orbital of atom X.We'll define <math>\psi</math> as the wavefunction for a '''single MO containing a single electron'''. Next we write <math>\psi</math> as the linear combination of individual basis functions (<math>\phi_{i}</math>):

<math>
\psi = {{\sum\limits_{i} c_{i} \phi_{i} }}
</math>

Since we want a number of electrons, we're interested in the formula for the electron density (<math>\rho</math>) of this MO:

<math>
\rho = \langle{} \Psi | \Psi \rangle{}
</math>

<math>
\langle{}\Psi | \Psi \rangle{} = ({{\sum\limits_{i} c_{i} \phi_{i} }})({{\sum\limits_{i} c_{i} \phi_{i} }})
</math>

<math>
\langle{}\Psi | \Psi \rangle{} = ({{\sum\limits_{i} c_{i}^{2} \phi_{i}^{2} }}) + 2 {{ \sum\limits_{i>j}}} {{\sum\limits_{j} c_{i}c_{j} \langle{}\phi_{j} | \phi_{i} \rangle{} }}
</math>

If these formulae are tricky to follow, try doing it for a system with just two basis functions.
Note that in the third line the indice ''j'' runs over the same coefficients and basis functions as the indice ''i'', its just used to show that the <math>{{ \sum\limits_{i>j}}} {{\sum\limits_{j} c_{i}c_{j} \langle{}\phi_{j} | \phi_{i} \rangle{} }}</math> term only contains products between 2 different basis functions/co-efficients.
Hence, we now have a formula for the MO density which is purely in terms of (known) basis functions and (known) coefficients. Now its just a case of working out how we should interpret these values.

Firstly note that <math>\phi_{i}^{2}=1</math> (since we use normalised basis functions) hence:

<math>
{{\sum\limits_{i} c_{i}^{2} \phi_{i}^{2} }} = {{\sum\limits_{i} c_{i}^{2}}}
</math>

The values of c_{i} in this expression are easy to interpret, they represent the fractional contribution of AO ''i'' to the wavefunction. The <math>2 {{ \sum\limits_{i>j}}} {{\sum\limits_{j} c_{i}c_{j} \langle{}\phi_{j} | \phi_{i} \rangle{} }}</math> term is more difficult to interpret, each expression in the series can be thought of as the number of electrons shared between two basis functions. i.e. the following term would represent the number of electrons shared between basis function 1 and 2:

<math>
n_{shared} = 2 c_{1}c_{2} \langle{}\phi_{1} | \phi_{2} \rangle{}
</math>

However, we need to split this "shared" electron density between the two contributing basis functions. In Mulliken Population analysis "shared" electron density is divided equally between the two contributing basis functions. So the total electron density assigned to one basis function can be written as:

<math>
n_{i} = c_{i}^{2} + {{\sum\limits_{i \neq j} c_{i}c_{j} \langle{}\phi_{j} | \phi_{i} \rangle{} }}
</math>

<math>n_{i}</math> represents the '''fractional''' contribution of ''i'' to the wavefunction. Hence if you had a 2-electron MO, you'd simply multiply <math>n_{i}</math> by 2.

===What the code actually does (roughly/briefly)===

=="Installation" Instructions==

All instructions will be given for the open shell version of the code. However, the process is the same except whenever these instructions say to edit the "gen_gelius_input_opensh.sh", just do the corresponding edit on the gen_gelius_input.sh file. The Matlab scripts are EXACTLY the same for both open shell/closed shell codes.

0) Make sure you have multiwfn (https://multiwfn.codeplex.com) saved somewhere, python 2.x and numpy installed.

1)Put the following files (all the .py/.sh files) somewhere in system path (e.g i use /usr/local/bin):
:a)Format_Multiwfn_Output_alpha.py (Format_Multiwfn_Output.py for closed shell)
:b)Format_Multilwfn_Output_beta.py
:c)Gen_Multiwfn_Input_Comms.py
:d)Gen_Multiwfn_Input_Comms_beta.py
:e)Merge_Fragment_Files.py
:f)gen_gelius_input_opensh.sh

Make sure all files have executable persmission (navigate to the folder and do "chmod u+x filename")

2)In the gen_gelius_input_opensh.sh change the current lines accordingly:
:a) multiwfn_folder = 'C:/Users/Richard/Documents/Multiwfn_Installed' to the directory where you store multiwfn and associated files.
:b) "cygwin=1” to “cygwin=0” if your not using a Cygwin shell.
:c) “old_sed=1” to “old_sed=0” if your version of sed allows the –i option (edit in place) to be used. I don’t think the standard version on mac does(meaning set old_sed=1),standard version of Cygwin does though.
:d) POSSIBLY "cp $multiwfn_folder/Multiwfn.exe $curr_folder" to "cp $multiwfn_folder/Multiwfn $curr_folder" if on linux(should be really obvious if this is required). Do the same for the line where multiwfn is deleted from current folder("rm Multiwfn.exe").

3)Put the following files somewhere in matlabs path (e.g your matlab folder in documents)
:a) Get_X_Section.m
:b) Gen_Gelius_Spectrum.m

The Get_X_Section REALLY does need to be in matlabs path (unless you want to edit some code in Gen_Gelius_Spectrum.m). If you know what your doing in matlab feel free to put Gen_Gelius_Spectrum.m wherever you want.

4)Put the X_Section_Library folder wherever you want.

5) In Get_X_Section.m modify the following line:
:"Lib_Path ='C:\Users\Richard\Documents\X_Section_Library' " to the path where you put the X_Section_Library folder in.

6) OPTIONAL: Add the code in vim_template_script to your vimrc (c+p it somewhere in the file). "vim ~/.vimrc" to access your vimrc. You can then bring up a template input file in vim by using ":call Gelius_Input()"
Since i forgot to put the relevant text file in to the code folders, just c+p the below into your vimrc to get this to work.

<pre>
function! Gelius_Input()
exe 'normal aCoSCN4_freq_AllNCS_B3LYP_6311plgdp_Tetrahed_BMIMSCNSMD_Abr.fchk\rNumb_Fragments=10\r1a=\r1t=a\r1l=TDOS\r2a=\r2t=\r2l=\r3a=\r3t=\r3l=\r4a=\r4t=\r4l=\r5a=\r5t=\r5l=\r6a=\r6t=\r6l=\r7a=\r7t=\r7l=\r8a=\r8t=\r8l=\r9a=\r9t=\r9l=\r10a=\r10t=\r10l=\rM_factor=1.0\rLink_File='
exe '%s/\\r/\r/g'
endfunction
</pre>

==Running The Code==
====Using the Example files====
1) Navigate (in your shell) to the folder "Example_files". e.g for me i ran:

"cd /cygdrive/c/Users/Richard/Documents/Random_Coding/Gelius_Spectrum_Automated_Jan_2016/open_shell_code/Example_files"

2) Run this command in console(for closed shell use gen_gelius_input.sh):

"gen_gelius_input_opensh.sh"

Command 2) will have created a folder for each .fchk file in Example_files/. Therefore you should now have folders:
:CoSCN4_freq_AllNCS_B3LYP_6311plgdp_Tetrahed_BMIMSCNSMD_Abr
:other_file
Within these folders are tons of output files. The most important ones are those ending in "MLinpt.txt" - these will be the input files for step 3.

The script will run on all .fchk files in the current folder, when you do this for real you need to make sure that all the fchks are for the same system (e.g a water molecule) and that the atomic numbering is the same in all files (view-->labels in gaussview gives you what im calling atomic numbering).

If anything went wrong try revisiting the installation instructions.

3) Everything from now on is done using matlab. Theres a file called "pathname_file" in the Example_files folder which was created by the gen_gelius_input_opensh.sh script. Copy its full contents to clipboard and in MATLAB terminal type:

<pre>
plist=<CTRL-V>
(<CTRL-V> means paste, rather than type out CTRL-V)
</pre>

In matlab terms, plist now contains a cell array where each entry is the full path to a folder. The folder paths are those for the folders created in step 2). Hence to do this manually for just one of the two folders you'd type something like:
<pre>
plist={'C:\Users\Richard\Documents\Random_Coding\Gelius_Spectrum_Automated_Jan_2016\open_shell_code\Example_files\other_file'}
</pre>

4) To create either an XPS spectrum or pDoS plots we use the Gen_Gelius_Spectrum script. This takes the following three arguments IN THE FOLLOWING ORDER:
:a) the plist variable we just made
:b) the photon energy we want to use (any negative number requests a pDoS rather than an XPS spectrum)
:c) the FWHM of the peaks to place around each MO energy (units = eV)

So to create a pDoS plot you could type the following into the matlab terminal(Try this one first):
<pre>
Gen_Gelius_Spectrum(plist,-1,1)
</pre>
To create an XPS spectrum taken at photon energy of 1486eV (standard lab-based value) type the following:
<pre>
Gen_Gelius_Spectrum(plist,1486,1)
</pre>

If you get an error like "Undefined function 'Gen_Gelius_Spectrum' for input arguments of type 'cell'.", then your Gen_Gelius_Spectrum script probably isnt in your matlab path. (see installation instructions, or google a bit)

If your error free then well done. Sub-folders should have been created in "other_file" and "CoSCN4_freq_AllNCS_B3LYP_6311plgdp_Tetrahed_BMIMSCNSMD_Abr", which in turn will contain a loada output files. See the output files section of the wiki for an explanation of what these are.

====Creating your own input files====
If you followed the optional part above and made a vimscript, then a template for the input file can be brought up in vim by typing ":call Gelius_Input()"

An example input file is shown below. Each "fragment" (e.g cation,anion, alkyl group etc.) should have its own input file.
<pre>
other_file.fchk
Numb_Fragments=4
1a=10-13
1t=a
1l=TDOS
2a=10-13
2t=s
2l=S3s
3a=10-13
3t=p
3l=S3p
4a=10-13
4t=d+
4l=S3d
M_factor=1.0
Link_File=all_frags.test
</pre>
The input file can be divided into parts as follows:

1) The top line of the input file tells the code the name of the checkpoint file to use in the calculation. In practice you can write ANYTHING on this top line if using the gen_gelius_input*.sh script, as the script itself replaces the first line (whatevers there) with the names of all .fchks in the current folder. But SOMETHING must be written there.

2)'''Numb_Fragments''' is the number of sub-fragments (max 10) that you want to break your pDoS down into (e.g fragment 2 corresponds to all Sulfur 3s atomic orbitals in the example above). Code will break if you get this wrong.

3) '''1a/1t/1l''' instruct the code what to use as the total density of states. Hence 1a must be set to ALL the atoms your interested in for this fragment, 1t must be set to "a" and 1l should be set to TDOS (not sure it matters what this is set to, but to be on the safe side just use TDOS). The purpose of this fragment is essentially error checking.

4) '''2a/3a/*a''' : a stands for atoms here, so the value of a corresponds to the labels of the atoms you want to be included in the sub-fragment. These labels are the same integers you get in gaussview when you go to view-->labels. Examples of valid options may be "1,4,5-7" or "1,3,5" (basically comma seperated list of integers)

5) '''2t/3t/*t''': t stands for "type of AO".This is a comma seperated list with possible arguments (any combination) "a,s,p,d,f,g". "a" corresponds to all types of AO centered of the atoms specified with the 2a/3a/*a line. s means s-type atomc orbitals (0 angular momentum), p means p-type atomic orbitals (1 angular momentum) etc. To get all AOs of a certain angular momentum and higher (up to g-type AOs) add a plus to the end of the relevant letter. e.g "p+" says "include all AOs with angular momemtnum greater than or equal to 1 in this sub-fragment")
'''DON'T''' use "a" with any other letter, and dont do anything else that will lead to a type of AO being requested twice (e.g p+,d)

6) '''2l/3l/*l''': l stands for label. If you dont care about calculating an XPS spectrum you can essentially set it to whatever you want. This tells the matlab script what cross section to use for calculating the XPS spectrum, hence it must match (case insensitive) a filename in the X_Section_Library folder. e.g S3s tells matlab to get the S3S.txt file when looking for what cross-section to use.

7) '''M_factor=1.0''', you really never need to modify this but make sure its present. It's purpose is to tell matlab to multiple the pDoS of this fragment (that this input file corresponds to) by the value M_factor. The purpose is for systems where the real stoichiometry differs from that in your calculation. For example if i have a mixture of 99 water:1 anion but only a calculation on 1 water molecule mixed with 1 anion i could set M_factor for the water fragment input file to M_factor=99.0 to estimate (badly) what the actual DoS of the mixture would look like.

8) '''Link_File=*'''. This options for if you have a fragment (e.g an anion) but want it broken into more than 10 sub-fragments (e.g this will be true for any molecule with >3 elements). In this case make 2 files for that fragment, and in the second file set Link_File to the name of the first file. In the example above, this input file will essentially be merged with "all_frags.test", and only an output file for all_frags.test will be generated.

====More complex input file options====
====Gen_Gelius_Spectrum.m====
This code has the following options that can be changed (all are explained by comments in the .m file):
<pre>1)min_e/max_e
</pre>
Matlab ignores any orbitals with energy below min_e or above max_e when generating the XPS spectrum. Note if you want to change this to include core orbitals or very high energy virtual orbitals then you'll also have to edit Gen_Multiwfn_Input_Comms.py to set the range in multiwfn to the one you want.(Theres currently no variable in that .py code to do that, it just uses the defaults in multiwfn. But the function you'll need to edit in Gen_Multiwfn_Input_Comms.py is Output_Header.
<pre>2)min_e_plotted/max_e_plotted
</pre>
These refer to the minimum/maximum binding energies to calculate spectral intensity for. The range should probably be slightly wider than the min_e/max_e range really, though thats not the default setting.
<pre>3)step_size
</pre>
The value (in eV) between calculated data points in the output spectra. For example, if set to 0.001eV then a data point will be calculated for 0eV,0.001eV,0.002eV etc. Lower values=larger files/slower runtimes.
<pre>4)lorentz_fraction
</pre>
Value between 0 and 1 that determines the % Lorentzian character of the lineshape used in the broadening function(the remaining % is set to gaussian). By broadening function i essentially mean the peak thats placed around every MO energy in generating these spectra. lorentz_fraction=0 will give you a pure gaussian peak, while lorentz_fraction=1 gives a pure Lorentzian peak.
<pre>5)savetext/figsave/savejpeg/all_contribs_output
</pre>
These all control whether a certain type of output file is saved, with variable names that should be self-explanatory.Values of 1 mean save the file, other values mean dont save. set savetext=0 to not save any .txt output files, figsave=0 to not save any .fig output, savejpeg=0 to not save any .jpeg outputs and all_contribs_output=0 to not save the .txt file with a full breakdown of the spectrum. Leaving them all as 1 (except maybe the savejpeg) is strongly recommended.

====Plot_Atomic_Contribs_V3====

This is an optional script that plots certain breakdowns of your calculated DoS/XPS spectrum (i.e you run it AFTER Gen_Gelius_Spectrum) . Its not too hard to do the same thing in excel using the *ALL_Contribs* output file (see output file sections) but this is a lot faster. There are three types of breakdown this function can do (see pictures below):

1) Atomic breakdown with fragments kept seperate (merge_spd=1,merge_fragments=0)

2) Atomic orbital breakdown (merge_spd=0,merge_fragments=1)

3) Atomic breakdown (merge_spd=1,merge_fragments=1)

The function in its uploaded form takes a single argument , though there are two variables in the actual code that you need to mess with (merge_spd and merge_fragments).

So for example, to run this on the output file from the "Using the Example files" i typed into the matlab terminal:

Plot_Atomic_Contribs_V3('C:\Users\Richard\Desktop\Open_Shell_Code_Some_Stuff\Example_files\other_file\DoS_1ev_0pt3l')

The argument needs to be a folder containing the output files from the Gen_Gelius_Spectrum script. You'll then get a saved graph with the breakdown you asked for and also a .txt file containing the breakdown in a comma delimited format. The output file names record the options you used to make the script (e.g "Plotted_Atomic_Contribs_spd0_frag1.txt" means merge_spd was set to 0 and merge_fragments was set to 1).

[[File:Plotted Atomic Contribs spd1 frag1.jpg|thumbnail|center| merge_spd=1,merge_fragments=1 ]]

[[File:Plotted_Atomic_Contribs_spd0_frag1.jpg|thumbnail|center| merge_spd=0,merge_fragments=1 ]]

[[File:Plotted Atomic Contribs spd1 frag0.jpg|thumbnail|center| merge_spd=1,merge_fragments=0 ]]

==Output Files==

The first "output file" is the *MCRF*.txt file, which just contains the commands to build the required fragment in multiwfn.

===Files From Multiwfn===
Most people should be able to completely ignore these files. But in case your curious:

'''1) *DOS_curve*.txt''' Essentially a DoS plot. 1st column is energies in au, second column is TDOS of the fragment. Not really sure what the 3rd column is. No particular use for this file, but it gets auto-generated by multiwfn and i dont go out my way to delete it.

'''2) *DOS_line*.txt''' Data to create a stick plot (where each MO is a stick). Plot column 1 vs column 2 as a line graph and you get an axis with sticks representing the MOs. 4th column onwards represent contribution of the various sub-fragments to each "stick". This is the file that gets used to generate the *ML_inpt.txt file.

'''3) *DOS_frag*.txt ''' This file contains the sub-fragment info in a way multiwfn understands. If its renamed to DOS_frag.txt then multiwfn can import it to get the fragments (therefore can be useful for debugging, no further purpose).

===MLinpt.txt File===

Creation of the *ML_inpt.txt files is essentially the point of the gen_gelius_input* script. These are primarily input files for matlab, but they do have useful information in an easy to read format. If you import the file into excel and COMMA DELIMIT ONLY (i.e untick the space delimit option) you can get something like below(there are more decimal places in the actual file):

<pre>
#M_FACTOR=1.0,
#Energy(eV) TDOS CO4S CO3P CO3D N2S N2P N2D C2S C2P C2D S3S S3P S3D
[LINES DELETED]
-24.5 0.9 0.0 0.0 0.0 0.5 0.1 0.0 0.2 0.1 0.0 0.1 0.0 0.0
-24.4 1.0 0.0 0.0 0.0 0.5 0.1 0.0 0.3 0.1 0.0 0.0 0.0 0.0
-24.4 1.0 0.0 0.0 0.0 0.5 0.1 0.0 0.3 0.1 0.0 0.0 0.0 0.0
-24.4 1.0 0.0 0.0 0.0 0.5 0.1 0.0 0.3 0.1 0.0 0.0 0.0 0.0
-19.8 0.3 0.0 0.0 0.0 0.1 -0.1 0.0 0.2 0.1 0.0 0.7 0.1 0.0
-19.7 0.3 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.1 0.0 0.7 0.1 0.0
[LINES DELETED]
</pre>

The shown file is "all_frags_alpha_MLinpt.txt", which is generated from the example files in the open shell code. As this actually comes from merging two MLinpt.txt files (gen_gelius_input.sh does this part silently) the TDOS in this case doesnt mean anything.

The first column lists the energy of each MO in eV (1 ev ~27.212 au). For any row, columns 3(CO4S)-->end (S3D) list the fractional contributions of each type of atmoic orbital to the MO in that row. For example the MO with energy -19.7 eV is mainly from S3s type atomic orbitals. These fractions were determiend by multiwfn by using Mulliken population analysis.

All the fractional contributions in the file get multiplied by the M_FACTOR value (as explained above in the Creating your own input files section). Therefore the value of M_FACTOR can easily be set after the MLinpt.txt file is created, as opposed to within the initial input file.

===Formatted_UMOs===

The default for the gen_gelius_input* script (i.e the settings for the version i uploaded) is to put only OCCUPIED MO energies/breakdowns into the *MLinpt.txt file. If you want all MOs (I really DONT recommend this) in the MLinpt.txt file then change all_orbs='cheese' to all_orbs='ALLORBS' at the top of the gen_gelius_input* script.

Otherwise you will get a *Formatted_UMOs* file for every MLinpt.txt file you get. The format of the file is identical to the MLinpt.txt format except the MO at the top of the file corresponds to the LUMO.

WARNING: If you use the link_files feature, then the output formatted_UMOs files DO NOT get merged. This shouldnt matter as you should never really use the formatted UMOs for anything.

===Output From Matlab Code ===

The matlab code firstly will create a new folder with a name reflecting the input arguments. The format is hv_FWHM_%lorenztian where hv is the photon energy, FWHM is full width half maximum of the broadeing function and %lorenztian is the amount of lorenztian mixed in the broadening function. For example the earlier command (from using example files section) "Gen_Gelius_Spectrum(plist,-1,1)" generates the folder "DoS_1ev_0pt3l". DoS = Density of states, and means we did not use photoionisation cross sections. The 0pt3l part means our broadening function was 30% lorenztian and 70% Gaussian.

Two .fig files will have been produced from this code, these contain matlab graphs of the output. These .fig files follow the naming ("filename" is the name of the original fchk file):

1) filename_DoS.fig contains the total DoS plot.

2) filename_DoS_frag_contribs.fig contains the DoS AND the pDoS of each of the fragments. For closed shell code you get 1 fragment per input file (assuming you dont use link_files), but for the open shell code each input file leads to two fragments - one for alpha electrons and one for beta electrons.

[[File:Other_file_DoS.jpeg|thumb|center| 1) Total Density of States for a Co(SCN)4 fragment with filename other_file_DoS.fig]]

[[File:Other file DoS frag contribs.jpeg|thumb|center| 2) Contribtuions of alpha and beta electrons to the Total Density of States for Co(SCN)4, filename other_file_DoS_frag_contribs.fig]]

The filename_DoS.fig plot data is output in a .txt file of the same name, while each fragment in filename_DoS_frag_contribs.fig has its own .txt file. For the example above the two fragments have names "all_frags_alpha_other_file_DoS" and "all_frags_beta_other_file_DoS". The start of the names are taken from the name of the original input file "all_frags.test".

The final file output is the "ALL_Contribs_other_file_DoS.txt" file. This(as the name implies) contains all contributions to the DoS spectra in a comma delimited file. The Plot_Atomic_Contribs*.m code can be run on this file to generate plots with a few different breakdowns (e.g atomic breakdowns) - see the relevant section of this wiki (not done yet....). The file will look something like below (commas replaced with tabs):

<pre>
#all_frags_alpha all_frags_beta
#Sub fragments in each= 12 12
#Energy(eV) CO4S CO3P CO3D N2S N2P N2D C2S C2P C2D [COLUMNS REMOVED]
[ROWS REMOVED,numbers rounded]
7.5 0.00 0.01 0.31 0.04 0.13 0.00 0.00 0.11 0.00 [COLUMNS REMOVED]
7.51 0.00 0.01 0.33 0.04 0.13 0.00 0.00 0.11 0.00 [COLUMNS REMOVED]
7.52 0.00 0.01 0.34 0.04 0.13 0.00 0.00 0.11 0.00 [COLUMNS REMOVED]
7.53 0.00 0.01 0.35 0.04 0.14 0.00 0.00 0.12 0.00 [COLUMNS REMOVED]
7.54 0.00 0.01 0.36 0.04 0.14 0.00 0.00 0.12 0.00 [COLUMNS REMOVED]
7.55 0.00 0.02 0.37 0.04 0.14 0.00 0.00 0.12 0.00 [COLUMNS REMOVED]
7.56 0.00 0.02 0.39 0.05 0.14 0.00 0.00 0.13 0.00 [COLUMNS REMOVED]
[ROWS REMOVED]
</pre>

Hopefully the format is mostly intuitive, the top two rows contain information on what the data is, and the rest contain the data. The top row contains (originally comma delimited) a list of all "fragments" names, which is the number of input files for the closed shell code and 2x the number of input files for the open shell code. The second row contains the number of sub-fragments (which should be atomic orbitals) in each fragment. In the example above each fragment had 12 types of atomic orbital.

So for the example above column 2-14 shows contributions from "all_frags_alpha" while column 15-27 shows contributions from "all_frags_beta" (column 1 lists energies).

==Bugs/Things to Watch For==
===Fragment Names===
Make sure your fragment names arent a substring of your .fchk file names. For example an input file called "Cl.test" wont work for a file called "BMIM_Cl.fchk". However, "Cl_frag.test" will work. Therefore personally i add _frag to the end of every input file name. The other way to sort this is to set linked_files=0 in gen_gelius_input_opensh.sh, sincce the link files part of the code is where this bug comes from.

===Fragment Error Checking===
Unless you manually turn it off (The "error=" option in gen_gelius_input_opensh.sh script) the code tries to check your input file for errors. Specifically it tests whether the TDOS in an input_file really is the sum of the other sub-fragments in the input(.test) file. E.g it will trigger if you request only Sulfur p-type and d-type AOs (but no s-type AOs) in the input file, or if you request all sulfur p-type AOs twice. If the error check triggers then it BREAKS the input file for matlab by putting an error msg in the first line of the '''*MLinpt.txt file'''. An example of what the error looks like is:
<pre>
“Fragment Sum incorrect on row 19 for fchk file xxxx.fchk”
</pre>

If you *MLinpt.txt files arent working in matlab then this is probably the reason.

===Pathlengths===
This code makes a lot of sub-folders based on the name of .fchk files. If you have really long .fchk file names its likely that you'll end up with some files which exceed the 256 character limit for a pathlength on a windows computer. In matlab 2014 this will cause the code to exit with an error, saying it cant save the .fig files. In newer matlab versions it may just save the output files , but you wont be able to open them on windows if the path length is greater than 256 characters. Ways to avoid this are to shorten .fchk file names where possible, or (less practical) run the code on a folder as close as possible to root (i.e a folder with a short a pathlength as possible).

Talk:Mod:Hunt Research Group/Calc XPS Code

2017-06-07T14:28:39Z

Rf614: /* Mulliken Population Analysis */

==Links to Files==
Open Shell Version: [[Media:Gelius_open_shell_code_Aug2016.zip]]

Closed Shell Version: [[Media:Gelius_closed_shell_code_Aug2016.zip]]

==Introduction==
The purpose of this code is to calculate partial density of states (pDoS) plots and Valence band x-ray photoelectron (XP) spectra from .fchk files. There is both an open shell and a closed shell version of the code, they work in exactly the same way.

All code was written using python 2.7 (with numpy), Multiwfn version 3.3.8, Matlab version 2014a and a cygwin(bash) shell on windows 8. It should work on windows or linux if the correct options are set. It might not work on a mac currently due to Multiwfn not working properly (I haven't personally tested it though).

==Some Theory==

===Mulliken Population Analysis===

A pretty key part to this code is the breakdown of molecular orbitals (MOs) in atomic orbitals (AOs, which are essentially basis functions). This is carried out in multiwfn using Mulliken population analysis, which i'll briefly summarise due to its importance.

The basic question is "For MO X, how many electrons come from atomic orbital Y?". Note im going to be using "basis functions" instead of "atomic orbitals" from this point, in terms of DFT calculations we interpret a basis function centered on atom X as an atomic orbital of atom X.We'll define <math>\psi</math> as the wavefunction for a '''single MO containing a single electron'''. Next we write <math>\psi</math> as the linear combination of individual basis functions (<math>\phi_{i}</math>):

<math>
\psi = {{\sum\limits_{i} c_{i} \phi_{i} }}
</math>

Since we want a number of electrons, we're interested in the formula for the electron density (<math>\rho</math>) of this MO:

<math>
\rho = \langle{} \Psi | \Psi \rangle{}
</math>

<math>
\langle{}\Psi | \Psi \rangle{} = ({{\sum\limits_{i} c_{i} \phi_{i} }})({{\sum\limits_{i} c_{i} \phi_{i} }})
</math>

<math>
\langle{}\Psi | \Psi \rangle{} = ({{\sum\limits_{i} c_{i}^{2} \phi_{i}^{2} }}) + 2 {{ \sum\limits_{i>j}}} {{\sum\limits_{j} c_{i}c_{j} \langle{}\phi_{j} | \phi_{i} \rangle{} }}
</math>

If these formulae are tricky to follow, try doing it for a system with just two basis functions.
Note that in the third line the indice ''j'' runs over the same coefficients and basis functions as the indice ''i'', its just used to show that the <math>{{ \sum\limits_{i>j}}} {{\sum\limits_{j} c_{i}c_{j} \langle{}\phi_{j} | \phi_{i} \rangle{} }}</math> term only contains products between 2 different basis functions/co-efficients.
Hence, we now have a formula for the MO density which is purely in terms of (known) basis functions and (known) coefficients. Now its just a case of working out how we should interpret these values.

Firstly note that <math>\phi_{i}^{2}=1</math> (since we use normalised basis functions) hence:

<math>
{{\sum\limits_{i} c_{i}^{2} \phi_{i}^{2} }} = {{\sum\limits_{i} c_{i}^{2}}}
</math>

The values of c_{i} in this expression are easy to interpret, they represent the fractional contribution of AO ''i'' to the wavefunction. The <math>2 {{ \sum\limits_{i>j}}} {{\sum\limits_{j} c_{i}c_{j} \langle{}\phi_{j} | \phi_{i} \rangle{} }}</math> term is more difficult to interpret, each expression in the series can be thought of as the number of electrons shared between two basis functions. i.e. the following term would represent the number of electrons shared between basis function 1 and 2:

<math>
n_{shared} = 2 c_{1}c_{2} \langle{}\phi_{1} | \phi_{2} \rangle{}
</math>

However, we need to split this "shared" electron density between the two contributing basis functions. In Mulliken Population analysis "shared" electron density is divided equally between the two contributing basis functions. So the total electron density assigned to one basis function can be written as:

<math>
n_{i} = c_{i} + {{\sum\limits_{i \neq j} c_{i}c_{j} \langle{}\phi_{j} | \phi_{i} \rangle{} }}
</math>

<math>n_{i}</math> represents the '''fractional''' contribution of ''i'' to the wavefunction. Hence if you had a 2-electron MO, you'd simply multiply <math>n_{i}</math> by 2.

===What the code actually does (roughly/briefly)===

=="Installation" Instructions==

All instructions will be given for the open shell version of the code. However, the process is the same except whenever these instructions say to edit the "gen_gelius_input_opensh.sh", just do the corresponding edit on the gen_gelius_input.sh file. The Matlab scripts are EXACTLY the same for both open shell/closed shell codes.

0) Make sure you have multiwfn (https://multiwfn.codeplex.com) saved somewhere, python 2.x and numpy installed.

1)Put the following files (all the .py/.sh files) somewhere in system path (e.g i use /usr/local/bin):
:a)Format_Multiwfn_Output_alpha.py (Format_Multiwfn_Output.py for closed shell)
:b)Format_Multilwfn_Output_beta.py
:c)Gen_Multiwfn_Input_Comms.py
:d)Gen_Multiwfn_Input_Comms_beta.py
:e)Merge_Fragment_Files.py
:f)gen_gelius_input_opensh.sh

Make sure all files have executable persmission (navigate to the folder and do "chmod u+x filename")

2)In the gen_gelius_input_opensh.sh change the current lines accordingly:
:a) multiwfn_folder = 'C:/Users/Richard/Documents/Multiwfn_Installed' to the directory where you store multiwfn and associated files.
:b) "cygwin=1” to “cygwin=0” if your not using a Cygwin shell.
:c) “old_sed=1” to “old_sed=0” if your version of sed allows the –i option (edit in place) to be used. I don’t think the standard version on mac does(meaning set old_sed=1),standard version of Cygwin does though.
:d) POSSIBLY "cp $multiwfn_folder/Multiwfn.exe $curr_folder" to "cp $multiwfn_folder/Multiwfn $curr_folder" if on linux(should be really obvious if this is required). Do the same for the line where multiwfn is deleted from current folder("rm Multiwfn.exe").

3)Put the following files somewhere in matlabs path (e.g your matlab folder in documents)
:a) Get_X_Section.m
:b) Gen_Gelius_Spectrum.m

The Get_X_Section REALLY does need to be in matlabs path (unless you want to edit some code in Gen_Gelius_Spectrum.m). If you know what your doing in matlab feel free to put Gen_Gelius_Spectrum.m wherever you want.

4)Put the X_Section_Library folder wherever you want.

5) In Get_X_Section.m modify the following line:
:"Lib_Path ='C:\Users\Richard\Documents\X_Section_Library' " to the path where you put the X_Section_Library folder in.

6) OPTIONAL: Add the code in vim_template_script to your vimrc (c+p it somewhere in the file). "vim ~/.vimrc" to access your vimrc. You can then bring up a template input file in vim by using ":call Gelius_Input()"
Since i forgot to put the relevant text file in to the code folders, just c+p the below into your vimrc to get this to work.

<pre>
function! Gelius_Input()
exe 'normal aCoSCN4_freq_AllNCS_B3LYP_6311plgdp_Tetrahed_BMIMSCNSMD_Abr.fchk\rNumb_Fragments=10\r1a=\r1t=a\r1l=TDOS\r2a=\r2t=\r2l=\r3a=\r3t=\r3l=\r4a=\r4t=\r4l=\r5a=\r5t=\r5l=\r6a=\r6t=\r6l=\r7a=\r7t=\r7l=\r8a=\r8t=\r8l=\r9a=\r9t=\r9l=\r10a=\r10t=\r10l=\rM_factor=1.0\rLink_File='
exe '%s/\\r/\r/g'
endfunction
</pre>

==Running The Code==
====Using the Example files====
1) Navigate (in your shell) to the folder "Example_files". e.g for me i ran:

"cd /cygdrive/c/Users/Richard/Documents/Random_Coding/Gelius_Spectrum_Automated_Jan_2016/open_shell_code/Example_files"

2) Run this command in console(for closed shell use gen_gelius_input.sh):

"gen_gelius_input_opensh.sh"

Command 2) will have created a folder for each .fchk file in Example_files/. Therefore you should now have folders:
:CoSCN4_freq_AllNCS_B3LYP_6311plgdp_Tetrahed_BMIMSCNSMD_Abr
:other_file
Within these folders are tons of output files. The most important ones are those ending in "MLinpt.txt" - these will be the input files for step 3.

The script will run on all .fchk files in the current folder, when you do this for real you need to make sure that all the fchks are for the same system (e.g a water molecule) and that the atomic numbering is the same in all files (view-->labels in gaussview gives you what im calling atomic numbering).

If anything went wrong try revisiting the installation instructions.

3) Everything from now on is done using matlab. Theres a file called "pathname_file" in the Example_files folder which was created by the gen_gelius_input_opensh.sh script. Copy its full contents to clipboard and in MATLAB terminal type:

<pre>
plist=<CTRL-V>
(<CTRL-V> means paste, rather than type out CTRL-V)
</pre>

In matlab terms, plist now contains a cell array where each entry is the full path to a folder. The folder paths are those for the folders created in step 2). Hence to do this manually for just one of the two folders you'd type something like:
<pre>
plist={'C:\Users\Richard\Documents\Random_Coding\Gelius_Spectrum_Automated_Jan_2016\open_shell_code\Example_files\other_file'}
</pre>

4) To create either an XPS spectrum or pDoS plots we use the Gen_Gelius_Spectrum script. This takes the following three arguments IN THE FOLLOWING ORDER:
:a) the plist variable we just made
:b) the photon energy we want to use (any negative number requests a pDoS rather than an XPS spectrum)
:c) the FWHM of the peaks to place around each MO energy (units = eV)

So to create a pDoS plot you could type the following into the matlab terminal(Try this one first):
<pre>
Gen_Gelius_Spectrum(plist,-1,1)
</pre>
To create an XPS spectrum taken at photon energy of 1486eV (standard lab-based value) type the following:
<pre>
Gen_Gelius_Spectrum(plist,1486,1)
</pre>

If you get an error like "Undefined function 'Gen_Gelius_Spectrum' for input arguments of type 'cell'.", then your Gen_Gelius_Spectrum script probably isnt in your matlab path. (see installation instructions, or google a bit)

If your error free then well done. Sub-folders should have been created in "other_file" and "CoSCN4_freq_AllNCS_B3LYP_6311plgdp_Tetrahed_BMIMSCNSMD_Abr", which in turn will contain a loada output files. See the output files section of the wiki for an explanation of what these are.

====Creating your own input files====
If you followed the optional part above and made a vimscript, then a template for the input file can be brought up in vim by typing ":call Gelius_Input()"

An example input file is shown below. Each "fragment" (e.g cation,anion, alkyl group etc.) should have its own input file.
<pre>
other_file.fchk
Numb_Fragments=4
1a=10-13
1t=a
1l=TDOS
2a=10-13
2t=s
2l=S3s
3a=10-13
3t=p
3l=S3p
4a=10-13
4t=d+
4l=S3d
M_factor=1.0
Link_File=all_frags.test
</pre>
The input file can be divided into parts as follows:

1) The top line of the input file tells the code the name of the checkpoint file to use in the calculation. In practice you can write ANYTHING on this top line if using the gen_gelius_input*.sh script, as the script itself replaces the first line (whatevers there) with the names of all .fchks in the current folder. But SOMETHING must be written there.

2)'''Numb_Fragments''' is the number of sub-fragments (max 10) that you want to break your pDoS down into (e.g fragment 2 corresponds to all Sulfur 3s atomic orbitals in the example above). Code will break if you get this wrong.

3) '''1a/1t/1l''' instruct the code what to use as the total density of states. Hence 1a must be set to ALL the atoms your interested in for this fragment, 1t must be set to "a" and 1l should be set to TDOS (not sure it matters what this is set to, but to be on the safe side just use TDOS). The purpose of this fragment is essentially error checking.

4) '''2a/3a/*a''' : a stands for atoms here, so the value of a corresponds to the labels of the atoms you want to be included in the sub-fragment. These labels are the same integers you get in gaussview when you go to view-->labels. Examples of valid options may be "1,4,5-7" or "1,3,5" (basically comma seperated list of integers)

5) '''2t/3t/*t''': t stands for "type of AO".This is a comma seperated list with possible arguments (any combination) "a,s,p,d,f,g". "a" corresponds to all types of AO centered of the atoms specified with the 2a/3a/*a line. s means s-type atomc orbitals (0 angular momentum), p means p-type atomic orbitals (1 angular momentum) etc. To get all AOs of a certain angular momentum and higher (up to g-type AOs) add a plus to the end of the relevant letter. e.g "p+" says "include all AOs with angular momemtnum greater than or equal to 1 in this sub-fragment")
'''DON'T''' use "a" with any other letter, and dont do anything else that will lead to a type of AO being requested twice (e.g p+,d)

6) '''2l/3l/*l''': l stands for label. If you dont care about calculating an XPS spectrum you can essentially set it to whatever you want. This tells the matlab script what cross section to use for calculating the XPS spectrum, hence it must match (case insensitive) a filename in the X_Section_Library folder. e.g S3s tells matlab to get the S3S.txt file when looking for what cross-section to use.

7) '''M_factor=1.0''', you really never need to modify this but make sure its present. It's purpose is to tell matlab to multiple the pDoS of this fragment (that this input file corresponds to) by the value M_factor. The purpose is for systems where the real stoichiometry differs from that in your calculation. For example if i have a mixture of 99 water:1 anion but only a calculation on 1 water molecule mixed with 1 anion i could set M_factor for the water fragment input file to M_factor=99.0 to estimate (badly) what the actual DoS of the mixture would look like.

8) '''Link_File=*'''. This options for if you have a fragment (e.g an anion) but want it broken into more than 10 sub-fragments (e.g this will be true for any molecule with >3 elements). In this case make 2 files for that fragment, and in the second file set Link_File to the name of the first file. In the example above, this input file will essentially be merged with "all_frags.test", and only an output file for all_frags.test will be generated.

====More complex input file options====
====Gen_Gelius_Spectrum.m====
This code has the following options that can be changed (all are explained by comments in the .m file):
<pre>1)min_e/max_e
</pre>
Matlab ignores any orbitals with energy below min_e or above max_e when generating the XPS spectrum. Note if you want to change this to include core orbitals or very high energy virtual orbitals then you'll also have to edit Gen_Multiwfn_Input_Comms.py to set the range in multiwfn to the one you want.(Theres currently no variable in that .py code to do that, it just uses the defaults in multiwfn. But the function you'll need to edit in Gen_Multiwfn_Input_Comms.py is Output_Header.
<pre>2)min_e_plotted/max_e_plotted
</pre>
These refer to the minimum/maximum binding energies to calculate spectral intensity for. The range should probably be slightly wider than the min_e/max_e range really, though thats not the default setting.
<pre>3)step_size
</pre>
The value (in eV) between calculated data points in the output spectra. For example, if set to 0.001eV then a data point will be calculated for 0eV,0.001eV,0.002eV etc. Lower values=larger files/slower runtimes.
<pre>4)lorentz_fraction
</pre>
Value between 0 and 1 that determines the % Lorentzian character of the lineshape used in the broadening function(the remaining % is set to gaussian). By broadening function i essentially mean the peak thats placed around every MO energy in generating these spectra. lorentz_fraction=0 will give you a pure gaussian peak, while lorentz_fraction=1 gives a pure Lorentzian peak.
<pre>5)savetext/figsave/savejpeg/all_contribs_output
</pre>
These all control whether a certain type of output file is saved, with variable names that should be self-explanatory.Values of 1 mean save the file, other values mean dont save. set savetext=0 to not save any .txt output files, figsave=0 to not save any .fig output, savejpeg=0 to not save any .jpeg outputs and all_contribs_output=0 to not save the .txt file with a full breakdown of the spectrum. Leaving them all as 1 (except maybe the savejpeg) is strongly recommended.

====Plot_Atomic_Contribs_V3====

This is an optional script that plots certain breakdowns of your calculated DoS/XPS spectrum (i.e you run it AFTER Gen_Gelius_Spectrum) . Its not too hard to do the same thing in excel using the *ALL_Contribs* output file (see output file sections) but this is a lot faster. There are three types of breakdown this function can do (see pictures below):

1) Atomic breakdown with fragments kept seperate (merge_spd=1,merge_fragments=0)

2) Atomic orbital breakdown (merge_spd=0,merge_fragments=1)

3) Atomic breakdown (merge_spd=1,merge_fragments=1)

The function in its uploaded form takes a single argument , though there are two variables in the actual code that you need to mess with (merge_spd and merge_fragments).

So for example, to run this on the output file from the "Using the Example files" i typed into the matlab terminal:

Plot_Atomic_Contribs_V3('C:\Users\Richard\Desktop\Open_Shell_Code_Some_Stuff\Example_files\other_file\DoS_1ev_0pt3l')

The argument needs to be a folder containing the output files from the Gen_Gelius_Spectrum script. You'll then get a saved graph with the breakdown you asked for and also a .txt file containing the breakdown in a comma delimited format. The output file names record the options you used to make the script (e.g "Plotted_Atomic_Contribs_spd0_frag1.txt" means merge_spd was set to 0 and merge_fragments was set to 1).

[[File:Plotted Atomic Contribs spd1 frag1.jpg|thumbnail|center| merge_spd=1,merge_fragments=1 ]]

[[File:Plotted_Atomic_Contribs_spd0_frag1.jpg|thumbnail|center| merge_spd=0,merge_fragments=1 ]]

[[File:Plotted Atomic Contribs spd1 frag0.jpg|thumbnail|center| merge_spd=1,merge_fragments=0 ]]

==Output Files==

The first "output file" is the *MCRF*.txt file, which just contains the commands to build the required fragment in multiwfn.

===Files From Multiwfn===
Most people should be able to completely ignore these files. But in case your curious:

'''1) *DOS_curve*.txt''' Essentially a DoS plot. 1st column is energies in au, second column is TDOS of the fragment. Not really sure what the 3rd column is. No particular use for this file, but it gets auto-generated by multiwfn and i dont go out my way to delete it.

'''2) *DOS_line*.txt''' Data to create a stick plot (where each MO is a stick). Plot column 1 vs column 2 as a line graph and you get an axis with sticks representing the MOs. 4th column onwards represent contribution of the various sub-fragments to each "stick". This is the file that gets used to generate the *ML_inpt.txt file.

'''3) *DOS_frag*.txt ''' This file contains the sub-fragment info in a way multiwfn understands. If its renamed to DOS_frag.txt then multiwfn can import it to get the fragments (therefore can be useful for debugging, no further purpose).

===MLinpt.txt File===

Creation of the *ML_inpt.txt files is essentially the point of the gen_gelius_input* script. These are primarily input files for matlab, but they do have useful information in an easy to read format. If you import the file into excel and COMMA DELIMIT ONLY (i.e untick the space delimit option) you can get something like below(there are more decimal places in the actual file):

<pre>
#M_FACTOR=1.0,
#Energy(eV) TDOS CO4S CO3P CO3D N2S N2P N2D C2S C2P C2D S3S S3P S3D
[LINES DELETED]
-24.5 0.9 0.0 0.0 0.0 0.5 0.1 0.0 0.2 0.1 0.0 0.1 0.0 0.0
-24.4 1.0 0.0 0.0 0.0 0.5 0.1 0.0 0.3 0.1 0.0 0.0 0.0 0.0
-24.4 1.0 0.0 0.0 0.0 0.5 0.1 0.0 0.3 0.1 0.0 0.0 0.0 0.0
-24.4 1.0 0.0 0.0 0.0 0.5 0.1 0.0 0.3 0.1 0.0 0.0 0.0 0.0
-19.8 0.3 0.0 0.0 0.0 0.1 -0.1 0.0 0.2 0.1 0.0 0.7 0.1 0.0
-19.7 0.3 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.1 0.0 0.7 0.1 0.0
[LINES DELETED]
</pre>

The shown file is "all_frags_alpha_MLinpt.txt", which is generated from the example files in the open shell code. As this actually comes from merging two MLinpt.txt files (gen_gelius_input.sh does this part silently) the TDOS in this case doesnt mean anything.

The first column lists the energy of each MO in eV (1 ev ~27.212 au). For any row, columns 3(CO4S)-->end (S3D) list the fractional contributions of each type of atmoic orbital to the MO in that row. For example the MO with energy -19.7 eV is mainly from S3s type atomic orbitals. These fractions were determiend by multiwfn by using Mulliken population analysis.

All the fractional contributions in the file get multiplied by the M_FACTOR value (as explained above in the Creating your own input files section). Therefore the value of M_FACTOR can easily be set after the MLinpt.txt file is created, as opposed to within the initial input file.

===Formatted_UMOs===

The default for the gen_gelius_input* script (i.e the settings for the version i uploaded) is to put only OCCUPIED MO energies/breakdowns into the *MLinpt.txt file. If you want all MOs (I really DONT recommend this) in the MLinpt.txt file then change all_orbs='cheese' to all_orbs='ALLORBS' at the top of the gen_gelius_input* script.

Otherwise you will get a *Formatted_UMOs* file for every MLinpt.txt file you get. The format of the file is identical to the MLinpt.txt format except the MO at the top of the file corresponds to the LUMO.

WARNING: If you use the link_files feature, then the output formatted_UMOs files DO NOT get merged. This shouldnt matter as you should never really use the formatted UMOs for anything.

===Output From Matlab Code ===

The matlab code firstly will create a new folder with a name reflecting the input arguments. The format is hv_FWHM_%lorenztian where hv is the photon energy, FWHM is full width half maximum of the broadeing function and %lorenztian is the amount of lorenztian mixed in the broadening function. For example the earlier command (from using example files section) "Gen_Gelius_Spectrum(plist,-1,1)" generates the folder "DoS_1ev_0pt3l". DoS = Density of states, and means we did not use photoionisation cross sections. The 0pt3l part means our broadening function was 30% lorenztian and 70% Gaussian.

Two .fig files will have been produced from this code, these contain matlab graphs of the output. These .fig files follow the naming ("filename" is the name of the original fchk file):

1) filename_DoS.fig contains the total DoS plot.

2) filename_DoS_frag_contribs.fig contains the DoS AND the pDoS of each of the fragments. For closed shell code you get 1 fragment per input file (assuming you dont use link_files), but for the open shell code each input file leads to two fragments - one for alpha electrons and one for beta electrons.

[[File:Other_file_DoS.jpeg|thumb|center| 1) Total Density of States for a Co(SCN)4 fragment with filename other_file_DoS.fig]]

[[File:Other file DoS frag contribs.jpeg|thumb|center| 2) Contribtuions of alpha and beta electrons to the Total Density of States for Co(SCN)4, filename other_file_DoS_frag_contribs.fig]]

The filename_DoS.fig plot data is output in a .txt file of the same name, while each fragment in filename_DoS_frag_contribs.fig has its own .txt file. For the example above the two fragments have names "all_frags_alpha_other_file_DoS" and "all_frags_beta_other_file_DoS". The start of the names are taken from the name of the original input file "all_frags.test".

The final file output is the "ALL_Contribs_other_file_DoS.txt" file. This(as the name implies) contains all contributions to the DoS spectra in a comma delimited file. The Plot_Atomic_Contribs*.m code can be run on this file to generate plots with a few different breakdowns (e.g atomic breakdowns) - see the relevant section of this wiki (not done yet....). The file will look something like below (commas replaced with tabs):

<pre>
#all_frags_alpha all_frags_beta
#Sub fragments in each= 12 12
#Energy(eV) CO4S CO3P CO3D N2S N2P N2D C2S C2P C2D [COLUMNS REMOVED]
[ROWS REMOVED,numbers rounded]
7.5 0.00 0.01 0.31 0.04 0.13 0.00 0.00 0.11 0.00 [COLUMNS REMOVED]
7.51 0.00 0.01 0.33 0.04 0.13 0.00 0.00 0.11 0.00 [COLUMNS REMOVED]
7.52 0.00 0.01 0.34 0.04 0.13 0.00 0.00 0.11 0.00 [COLUMNS REMOVED]
7.53 0.00 0.01 0.35 0.04 0.14 0.00 0.00 0.12 0.00 [COLUMNS REMOVED]
7.54 0.00 0.01 0.36 0.04 0.14 0.00 0.00 0.12 0.00 [COLUMNS REMOVED]
7.55 0.00 0.02 0.37 0.04 0.14 0.00 0.00 0.12 0.00 [COLUMNS REMOVED]
7.56 0.00 0.02 0.39 0.05 0.14 0.00 0.00 0.13 0.00 [COLUMNS REMOVED]
[ROWS REMOVED]
</pre>

Hopefully the format is mostly intuitive, the top two rows contain information on what the data is, and the rest contain the data. The top row contains (originally comma delimited) a list of all "fragments" names, which is the number of input files for the closed shell code and 2x the number of input files for the open shell code. The second row contains the number of sub-fragments (which should be atomic orbitals) in each fragment. In the example above each fragment had 12 types of atomic orbital.

So for the example above column 2-14 shows contributions from "all_frags_alpha" while column 15-27 shows contributions from "all_frags_beta" (column 1 lists energies).

==Bugs/Things to Watch For==
===Fragment Names===
Make sure your fragment names arent a substring of your .fchk file names. For example an input file called "Cl.test" wont work for a file called "BMIM_Cl.fchk". However, "Cl_frag.test" will work. Therefore personally i add _frag to the end of every input file name. The other way to sort this is to set linked_files=0 in gen_gelius_input_opensh.sh, sincce the link files part of the code is where this bug comes from.

===Fragment Error Checking===
Unless you manually turn it off (The "error=" option in gen_gelius_input_opensh.sh script) the code tries to check your input file for errors. Specifically it tests whether the TDOS in an input_file really is the sum of the other sub-fragments in the input(.test) file. E.g it will trigger if you request only Sulfur p-type and d-type AOs (but no s-type AOs) in the input file, or if you request all sulfur p-type AOs twice. If the error check triggers then it BREAKS the input file for matlab by putting an error msg in the first line of the '''*MLinpt.txt file'''. An example of what the error looks like is:
<pre>
“Fragment Sum incorrect on row 19 for fchk file xxxx.fchk”
</pre>

If you *MLinpt.txt files arent working in matlab then this is probably the reason.

===Pathlengths===
This code makes a lot of sub-folders based on the name of .fchk files. If you have really long .fchk file names its likely that you'll end up with some files which exceed the 256 character limit for a pathlength on a windows computer. In matlab 2014 this will cause the code to exit with an error, saying it cant save the .fig files. In newer matlab versions it may just save the output files , but you wont be able to open them on windows if the path length is greater than 256 characters. Ways to avoid this are to shorten .fchk file names where possible, or (less practical) run the code on a folder as close as possible to root (i.e a folder with a short a pathlength as possible).

Talk:Mod:Hunt Research Group/Calc XPS Code

2017-06-07T14:27:02Z

Rf614: /* Some Theory */

==Links to Files==
Open Shell Version: [[Media:Gelius_open_shell_code_Aug2016.zip]]

Closed Shell Version: [[Media:Gelius_closed_shell_code_Aug2016.zip]]

==Introduction==
The purpose of this code is to calculate partial density of states (pDoS) plots and Valence band x-ray photoelectron (XP) spectra from .fchk files. There is both an open shell and a closed shell version of the code, they work in exactly the same way.

All code was written using python 2.7 (with numpy), Multiwfn version 3.3.8, Matlab version 2014a and a cygwin(bash) shell on windows 8. It should work on windows or linux if the correct options are set. It might not work on a mac currently due to Multiwfn not working properly (I haven't personally tested it though).

==Some Theory==

===Mulliken Population Analysis===

A pretty key part to this code is the breakdown of molecular orbitals (MOs) in atomic orbitals (AOs, which are essentially basis functions). This is carried out in multiwfn using Mulliken population analysis, which i'll briefly summarise due to its importance.

The basic question is "For MO X, how many electrons come from atomic orbital Y?". Hence we'll define <math>\psi</math> as the wavefunction for a '''single MO containing a single electron'''. Next we write <math>\psi</math> as the linear combination of individual basis functions (<math>\phi_{i}</math>):

<math>
\psi = {{\sum\limits_{i} c_{i} \phi_{i} }}
</math>

Since we want a number of electrons, we're interested in the formula for the electron density (<math>\rho</math>) of this MO:

<math>
\rho = \langle{} \Psi | \Psi \rangle{}
</math>

<math>
\langle{}\Psi | \Psi \rangle{} = ({{\sum\limits_{i} c_{i} \phi_{i} }})({{\sum\limits_{i} c_{i} \phi_{i} }})
</math>

<math>
\langle{}\Psi | \Psi \rangle{} = ({{\sum\limits_{i} c_{i}^{2} \phi_{i}^{2} }}) + 2 {{ \sum\limits_{i>j}}} {{\sum\limits_{j} c_{i}c_{j} \langle{}\phi_{j} | \phi_{i} \rangle{} }}
</math>

If these formulae are tricky to follow, try doing it for a system with just two basis functions.
Note that in the third line the indice ''j'' runs over the same coefficients and basis functions as the indice ''i'', its just used to show that the <math>{{ \sum\limits_{i>j}}} {{\sum\limits_{j} c_{i}c_{j} \langle{}\phi_{j} | \phi_{i} \rangle{} }}</math> term only contains products between 2 different basis functions/co-efficients.
Hence, we now have a formula for the MO density which is purely in terms of (known) basis functions and (known) coefficients. Now its just a case of working out how we should interpret these values.

Firstly note that <math>\phi_{i}^{2}=1</math> (since we use normalised basis functions) hence:

<math>
{{\sum\limits_{i} c_{i}^{2} \phi_{i}^{2} }} = {{\sum\limits_{i} c_{i}^{2}}}
</math>

The values of c_{i} in this expression are easy to interpret, they represent the fractional contribution of AO ''i'' to the wavefunction. The <math>2 {{ \sum\limits_{i>j}}} {{\sum\limits_{j} c_{i}c_{j} \langle{}\phi_{j} | \phi_{i} \rangle{} }}</math> term is more difficult to interpret, each expression in the series can be thought of as the number of electrons shared between two basis functions. i.e. the following term would represent the number of electrons shared between basis function 1 and 2:

<math>
n_{shared} = 2 c_{1}c_{2} \langle{}\phi_{1} | \phi_{2} \rangle{}
</math>

However, we need to split this "shared" electron density between the two contributing basis functions. In Mulliken Population analysis "shared" electron density is divided equally between the two contributing basis functions. So the total electron density assigned to one basis function can be written as:

<math>
n_{i} = c_{i} + {{\sum\limits_{i \neq j} c_{i}c_{j} \langle{}\phi_{j} | \phi_{i} \rangle{} }}
</math>

<math>n_{i}</math> represents the '''fractional''' contribution of ''i'' to the wavefunction. Hence if you had a 2-electron MO, you'd simply multiply <math>n_{i}</math> by 2.

===What the code actually does (roughly/briefly)===

=="Installation" Instructions==

All instructions will be given for the open shell version of the code. However, the process is the same except whenever these instructions say to edit the "gen_gelius_input_opensh.sh", just do the corresponding edit on the gen_gelius_input.sh file. The Matlab scripts are EXACTLY the same for both open shell/closed shell codes.

0) Make sure you have multiwfn (https://multiwfn.codeplex.com) saved somewhere, python 2.x and numpy installed.

1)Put the following files (all the .py/.sh files) somewhere in system path (e.g i use /usr/local/bin):
:a)Format_Multiwfn_Output_alpha.py (Format_Multiwfn_Output.py for closed shell)
:b)Format_Multilwfn_Output_beta.py
:c)Gen_Multiwfn_Input_Comms.py
:d)Gen_Multiwfn_Input_Comms_beta.py
:e)Merge_Fragment_Files.py
:f)gen_gelius_input_opensh.sh

Make sure all files have executable persmission (navigate to the folder and do "chmod u+x filename")

2)In the gen_gelius_input_opensh.sh change the current lines accordingly:
:a) multiwfn_folder = 'C:/Users/Richard/Documents/Multiwfn_Installed' to the directory where you store multiwfn and associated files.
:b) "cygwin=1” to “cygwin=0” if your not using a Cygwin shell.
:c) “old_sed=1” to “old_sed=0” if your version of sed allows the –i option (edit in place) to be used. I don’t think the standard version on mac does(meaning set old_sed=1),standard version of Cygwin does though.
:d) POSSIBLY "cp $multiwfn_folder/Multiwfn.exe $curr_folder" to "cp $multiwfn_folder/Multiwfn $curr_folder" if on linux(should be really obvious if this is required). Do the same for the line where multiwfn is deleted from current folder("rm Multiwfn.exe").

3)Put the following files somewhere in matlabs path (e.g your matlab folder in documents)
:a) Get_X_Section.m
:b) Gen_Gelius_Spectrum.m

The Get_X_Section REALLY does need to be in matlabs path (unless you want to edit some code in Gen_Gelius_Spectrum.m). If you know what your doing in matlab feel free to put Gen_Gelius_Spectrum.m wherever you want.

4)Put the X_Section_Library folder wherever you want.

5) In Get_X_Section.m modify the following line:
:"Lib_Path ='C:\Users\Richard\Documents\X_Section_Library' " to the path where you put the X_Section_Library folder in.

6) OPTIONAL: Add the code in vim_template_script to your vimrc (c+p it somewhere in the file). "vim ~/.vimrc" to access your vimrc. You can then bring up a template input file in vim by using ":call Gelius_Input()"
Since i forgot to put the relevant text file in to the code folders, just c+p the below into your vimrc to get this to work.

<pre>
function! Gelius_Input()
exe 'normal aCoSCN4_freq_AllNCS_B3LYP_6311plgdp_Tetrahed_BMIMSCNSMD_Abr.fchk\rNumb_Fragments=10\r1a=\r1t=a\r1l=TDOS\r2a=\r2t=\r2l=\r3a=\r3t=\r3l=\r4a=\r4t=\r4l=\r5a=\r5t=\r5l=\r6a=\r6t=\r6l=\r7a=\r7t=\r7l=\r8a=\r8t=\r8l=\r9a=\r9t=\r9l=\r10a=\r10t=\r10l=\rM_factor=1.0\rLink_File='
exe '%s/\\r/\r/g'
endfunction
</pre>

==Running The Code==
====Using the Example files====
1) Navigate (in your shell) to the folder "Example_files". e.g for me i ran:

"cd /cygdrive/c/Users/Richard/Documents/Random_Coding/Gelius_Spectrum_Automated_Jan_2016/open_shell_code/Example_files"

2) Run this command in console(for closed shell use gen_gelius_input.sh):

"gen_gelius_input_opensh.sh"

Command 2) will have created a folder for each .fchk file in Example_files/. Therefore you should now have folders:
:CoSCN4_freq_AllNCS_B3LYP_6311plgdp_Tetrahed_BMIMSCNSMD_Abr
:other_file
Within these folders are tons of output files. The most important ones are those ending in "MLinpt.txt" - these will be the input files for step 3.

The script will run on all .fchk files in the current folder, when you do this for real you need to make sure that all the fchks are for the same system (e.g a water molecule) and that the atomic numbering is the same in all files (view-->labels in gaussview gives you what im calling atomic numbering).

If anything went wrong try revisiting the installation instructions.

3) Everything from now on is done using matlab. Theres a file called "pathname_file" in the Example_files folder which was created by the gen_gelius_input_opensh.sh script. Copy its full contents to clipboard and in MATLAB terminal type:

<pre>
plist=<CTRL-V>
(<CTRL-V> means paste, rather than type out CTRL-V)
</pre>

In matlab terms, plist now contains a cell array where each entry is the full path to a folder. The folder paths are those for the folders created in step 2). Hence to do this manually for just one of the two folders you'd type something like:
<pre>
plist={'C:\Users\Richard\Documents\Random_Coding\Gelius_Spectrum_Automated_Jan_2016\open_shell_code\Example_files\other_file'}
</pre>

4) To create either an XPS spectrum or pDoS plots we use the Gen_Gelius_Spectrum script. This takes the following three arguments IN THE FOLLOWING ORDER:
:a) the plist variable we just made
:b) the photon energy we want to use (any negative number requests a pDoS rather than an XPS spectrum)
:c) the FWHM of the peaks to place around each MO energy (units = eV)

So to create a pDoS plot you could type the following into the matlab terminal(Try this one first):
<pre>
Gen_Gelius_Spectrum(plist,-1,1)
</pre>
To create an XPS spectrum taken at photon energy of 1486eV (standard lab-based value) type the following:
<pre>
Gen_Gelius_Spectrum(plist,1486,1)
</pre>

If you get an error like "Undefined function 'Gen_Gelius_Spectrum' for input arguments of type 'cell'.", then your Gen_Gelius_Spectrum script probably isnt in your matlab path. (see installation instructions, or google a bit)

If your error free then well done. Sub-folders should have been created in "other_file" and "CoSCN4_freq_AllNCS_B3LYP_6311plgdp_Tetrahed_BMIMSCNSMD_Abr", which in turn will contain a loada output files. See the output files section of the wiki for an explanation of what these are.

====Creating your own input files====
If you followed the optional part above and made a vimscript, then a template for the input file can be brought up in vim by typing ":call Gelius_Input()"

An example input file is shown below. Each "fragment" (e.g cation,anion, alkyl group etc.) should have its own input file.
<pre>
other_file.fchk
Numb_Fragments=4
1a=10-13
1t=a
1l=TDOS
2a=10-13
2t=s
2l=S3s
3a=10-13
3t=p
3l=S3p
4a=10-13
4t=d+
4l=S3d
M_factor=1.0
Link_File=all_frags.test
</pre>
The input file can be divided into parts as follows:

1) The top line of the input file tells the code the name of the checkpoint file to use in the calculation. In practice you can write ANYTHING on this top line if using the gen_gelius_input*.sh script, as the script itself replaces the first line (whatevers there) with the names of all .fchks in the current folder. But SOMETHING must be written there.

2)'''Numb_Fragments''' is the number of sub-fragments (max 10) that you want to break your pDoS down into (e.g fragment 2 corresponds to all Sulfur 3s atomic orbitals in the example above). Code will break if you get this wrong.

3) '''1a/1t/1l''' instruct the code what to use as the total density of states. Hence 1a must be set to ALL the atoms your interested in for this fragment, 1t must be set to "a" and 1l should be set to TDOS (not sure it matters what this is set to, but to be on the safe side just use TDOS). The purpose of this fragment is essentially error checking.

4) '''2a/3a/*a''' : a stands for atoms here, so the value of a corresponds to the labels of the atoms you want to be included in the sub-fragment. These labels are the same integers you get in gaussview when you go to view-->labels. Examples of valid options may be "1,4,5-7" or "1,3,5" (basically comma seperated list of integers)

5) '''2t/3t/*t''': t stands for "type of AO".This is a comma seperated list with possible arguments (any combination) "a,s,p,d,f,g". "a" corresponds to all types of AO centered of the atoms specified with the 2a/3a/*a line. s means s-type atomc orbitals (0 angular momentum), p means p-type atomic orbitals (1 angular momentum) etc. To get all AOs of a certain angular momentum and higher (up to g-type AOs) add a plus to the end of the relevant letter. e.g "p+" says "include all AOs with angular momemtnum greater than or equal to 1 in this sub-fragment")
'''DON'T''' use "a" with any other letter, and dont do anything else that will lead to a type of AO being requested twice (e.g p+,d)

6) '''2l/3l/*l''': l stands for label. If you dont care about calculating an XPS spectrum you can essentially set it to whatever you want. This tells the matlab script what cross section to use for calculating the XPS spectrum, hence it must match (case insensitive) a filename in the X_Section_Library folder. e.g S3s tells matlab to get the S3S.txt file when looking for what cross-section to use.

7) '''M_factor=1.0''', you really never need to modify this but make sure its present. It's purpose is to tell matlab to multiple the pDoS of this fragment (that this input file corresponds to) by the value M_factor. The purpose is for systems where the real stoichiometry differs from that in your calculation. For example if i have a mixture of 99 water:1 anion but only a calculation on 1 water molecule mixed with 1 anion i could set M_factor for the water fragment input file to M_factor=99.0 to estimate (badly) what the actual DoS of the mixture would look like.

8) '''Link_File=*'''. This options for if you have a fragment (e.g an anion) but want it broken into more than 10 sub-fragments (e.g this will be true for any molecule with >3 elements). In this case make 2 files for that fragment, and in the second file set Link_File to the name of the first file. In the example above, this input file will essentially be merged with "all_frags.test", and only an output file for all_frags.test will be generated.

====More complex input file options====
====Gen_Gelius_Spectrum.m====
This code has the following options that can be changed (all are explained by comments in the .m file):
<pre>1)min_e/max_e
</pre>
Matlab ignores any orbitals with energy below min_e or above max_e when generating the XPS spectrum. Note if you want to change this to include core orbitals or very high energy virtual orbitals then you'll also have to edit Gen_Multiwfn_Input_Comms.py to set the range in multiwfn to the one you want.(Theres currently no variable in that .py code to do that, it just uses the defaults in multiwfn. But the function you'll need to edit in Gen_Multiwfn_Input_Comms.py is Output_Header.
<pre>2)min_e_plotted/max_e_plotted
</pre>
These refer to the minimum/maximum binding energies to calculate spectral intensity for. The range should probably be slightly wider than the min_e/max_e range really, though thats not the default setting.
<pre>3)step_size
</pre>
The value (in eV) between calculated data points in the output spectra. For example, if set to 0.001eV then a data point will be calculated for 0eV,0.001eV,0.002eV etc. Lower values=larger files/slower runtimes.
<pre>4)lorentz_fraction
</pre>
Value between 0 and 1 that determines the % Lorentzian character of the lineshape used in the broadening function(the remaining % is set to gaussian). By broadening function i essentially mean the peak thats placed around every MO energy in generating these spectra. lorentz_fraction=0 will give you a pure gaussian peak, while lorentz_fraction=1 gives a pure Lorentzian peak.
<pre>5)savetext/figsave/savejpeg/all_contribs_output
</pre>
These all control whether a certain type of output file is saved, with variable names that should be self-explanatory.Values of 1 mean save the file, other values mean dont save. set savetext=0 to not save any .txt output files, figsave=0 to not save any .fig output, savejpeg=0 to not save any .jpeg outputs and all_contribs_output=0 to not save the .txt file with a full breakdown of the spectrum. Leaving them all as 1 (except maybe the savejpeg) is strongly recommended.

====Plot_Atomic_Contribs_V3====

This is an optional script that plots certain breakdowns of your calculated DoS/XPS spectrum (i.e you run it AFTER Gen_Gelius_Spectrum) . Its not too hard to do the same thing in excel using the *ALL_Contribs* output file (see output file sections) but this is a lot faster. There are three types of breakdown this function can do (see pictures below):

1) Atomic breakdown with fragments kept seperate (merge_spd=1,merge_fragments=0)

2) Atomic orbital breakdown (merge_spd=0,merge_fragments=1)

3) Atomic breakdown (merge_spd=1,merge_fragments=1)

The function in its uploaded form takes a single argument , though there are two variables in the actual code that you need to mess with (merge_spd and merge_fragments).

So for example, to run this on the output file from the "Using the Example files" i typed into the matlab terminal:

Plot_Atomic_Contribs_V3('C:\Users\Richard\Desktop\Open_Shell_Code_Some_Stuff\Example_files\other_file\DoS_1ev_0pt3l')

The argument needs to be a folder containing the output files from the Gen_Gelius_Spectrum script. You'll then get a saved graph with the breakdown you asked for and also a .txt file containing the breakdown in a comma delimited format. The output file names record the options you used to make the script (e.g "Plotted_Atomic_Contribs_spd0_frag1.txt" means merge_spd was set to 0 and merge_fragments was set to 1).

[[File:Plotted Atomic Contribs spd1 frag1.jpg|thumbnail|center| merge_spd=1,merge_fragments=1 ]]

[[File:Plotted_Atomic_Contribs_spd0_frag1.jpg|thumbnail|center| merge_spd=0,merge_fragments=1 ]]

[[File:Plotted Atomic Contribs spd1 frag0.jpg|thumbnail|center| merge_spd=1,merge_fragments=0 ]]

==Output Files==

The first "output file" is the *MCRF*.txt file, which just contains the commands to build the required fragment in multiwfn.

===Files From Multiwfn===
Most people should be able to completely ignore these files. But in case your curious:

'''1) *DOS_curve*.txt''' Essentially a DoS plot. 1st column is energies in au, second column is TDOS of the fragment. Not really sure what the 3rd column is. No particular use for this file, but it gets auto-generated by multiwfn and i dont go out my way to delete it.

'''2) *DOS_line*.txt''' Data to create a stick plot (where each MO is a stick). Plot column 1 vs column 2 as a line graph and you get an axis with sticks representing the MOs. 4th column onwards represent contribution of the various sub-fragments to each "stick". This is the file that gets used to generate the *ML_inpt.txt file.

'''3) *DOS_frag*.txt ''' This file contains the sub-fragment info in a way multiwfn understands. If its renamed to DOS_frag.txt then multiwfn can import it to get the fragments (therefore can be useful for debugging, no further purpose).

===MLinpt.txt File===

Creation of the *ML_inpt.txt files is essentially the point of the gen_gelius_input* script. These are primarily input files for matlab, but they do have useful information in an easy to read format. If you import the file into excel and COMMA DELIMIT ONLY (i.e untick the space delimit option) you can get something like below(there are more decimal places in the actual file):

<pre>
#M_FACTOR=1.0,
#Energy(eV) TDOS CO4S CO3P CO3D N2S N2P N2D C2S C2P C2D S3S S3P S3D
[LINES DELETED]
-24.5 0.9 0.0 0.0 0.0 0.5 0.1 0.0 0.2 0.1 0.0 0.1 0.0 0.0
-24.4 1.0 0.0 0.0 0.0 0.5 0.1 0.0 0.3 0.1 0.0 0.0 0.0 0.0
-24.4 1.0 0.0 0.0 0.0 0.5 0.1 0.0 0.3 0.1 0.0 0.0 0.0 0.0
-24.4 1.0 0.0 0.0 0.0 0.5 0.1 0.0 0.3 0.1 0.0 0.0 0.0 0.0
-19.8 0.3 0.0 0.0 0.0 0.1 -0.1 0.0 0.2 0.1 0.0 0.7 0.1 0.0
-19.7 0.3 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.1 0.0 0.7 0.1 0.0
[LINES DELETED]
</pre>

The shown file is "all_frags_alpha_MLinpt.txt", which is generated from the example files in the open shell code. As this actually comes from merging two MLinpt.txt files (gen_gelius_input.sh does this part silently) the TDOS in this case doesnt mean anything.

The first column lists the energy of each MO in eV (1 ev ~27.212 au). For any row, columns 3(CO4S)-->end (S3D) list the fractional contributions of each type of atmoic orbital to the MO in that row. For example the MO with energy -19.7 eV is mainly from S3s type atomic orbitals. These fractions were determiend by multiwfn by using Mulliken population analysis.

All the fractional contributions in the file get multiplied by the M_FACTOR value (as explained above in the Creating your own input files section). Therefore the value of M_FACTOR can easily be set after the MLinpt.txt file is created, as opposed to within the initial input file.

===Formatted_UMOs===

The default for the gen_gelius_input* script (i.e the settings for the version i uploaded) is to put only OCCUPIED MO energies/breakdowns into the *MLinpt.txt file. If you want all MOs (I really DONT recommend this) in the MLinpt.txt file then change all_orbs='cheese' to all_orbs='ALLORBS' at the top of the gen_gelius_input* script.

Otherwise you will get a *Formatted_UMOs* file for every MLinpt.txt file you get. The format of the file is identical to the MLinpt.txt format except the MO at the top of the file corresponds to the LUMO.

WARNING: If you use the link_files feature, then the output formatted_UMOs files DO NOT get merged. This shouldnt matter as you should never really use the formatted UMOs for anything.

===Output From Matlab Code ===

The matlab code firstly will create a new folder with a name reflecting the input arguments. The format is hv_FWHM_%lorenztian where hv is the photon energy, FWHM is full width half maximum of the broadeing function and %lorenztian is the amount of lorenztian mixed in the broadening function. For example the earlier command (from using example files section) "Gen_Gelius_Spectrum(plist,-1,1)" generates the folder "DoS_1ev_0pt3l". DoS = Density of states, and means we did not use photoionisation cross sections. The 0pt3l part means our broadening function was 30% lorenztian and 70% Gaussian.

Two .fig files will have been produced from this code, these contain matlab graphs of the output. These .fig files follow the naming ("filename" is the name of the original fchk file):

1) filename_DoS.fig contains the total DoS plot.

2) filename_DoS_frag_contribs.fig contains the DoS AND the pDoS of each of the fragments. For closed shell code you get 1 fragment per input file (assuming you dont use link_files), but for the open shell code each input file leads to two fragments - one for alpha electrons and one for beta electrons.

[[File:Other_file_DoS.jpeg|thumb|center| 1) Total Density of States for a Co(SCN)4 fragment with filename other_file_DoS.fig]]

[[File:Other file DoS frag contribs.jpeg|thumb|center| 2) Contribtuions of alpha and beta electrons to the Total Density of States for Co(SCN)4, filename other_file_DoS_frag_contribs.fig]]

The filename_DoS.fig plot data is output in a .txt file of the same name, while each fragment in filename_DoS_frag_contribs.fig has its own .txt file. For the example above the two fragments have names "all_frags_alpha_other_file_DoS" and "all_frags_beta_other_file_DoS". The start of the names are taken from the name of the original input file "all_frags.test".

The final file output is the "ALL_Contribs_other_file_DoS.txt" file. This(as the name implies) contains all contributions to the DoS spectra in a comma delimited file. The Plot_Atomic_Contribs*.m code can be run on this file to generate plots with a few different breakdowns (e.g atomic breakdowns) - see the relevant section of this wiki (not done yet....). The file will look something like below (commas replaced with tabs):

<pre>
#all_frags_alpha all_frags_beta
#Sub fragments in each= 12 12
#Energy(eV) CO4S CO3P CO3D N2S N2P N2D C2S C2P C2D [COLUMNS REMOVED]
[ROWS REMOVED,numbers rounded]
7.5 0.00 0.01 0.31 0.04 0.13 0.00 0.00 0.11 0.00 [COLUMNS REMOVED]
7.51 0.00 0.01 0.33 0.04 0.13 0.00 0.00 0.11 0.00 [COLUMNS REMOVED]
7.52 0.00 0.01 0.34 0.04 0.13 0.00 0.00 0.11 0.00 [COLUMNS REMOVED]
7.53 0.00 0.01 0.35 0.04 0.14 0.00 0.00 0.12 0.00 [COLUMNS REMOVED]
7.54 0.00 0.01 0.36 0.04 0.14 0.00 0.00 0.12 0.00 [COLUMNS REMOVED]
7.55 0.00 0.02 0.37 0.04 0.14 0.00 0.00 0.12 0.00 [COLUMNS REMOVED]
7.56 0.00 0.02 0.39 0.05 0.14 0.00 0.00 0.13 0.00 [COLUMNS REMOVED]
[ROWS REMOVED]
</pre>

Hopefully the format is mostly intuitive, the top two rows contain information on what the data is, and the rest contain the data. The top row contains (originally comma delimited) a list of all "fragments" names, which is the number of input files for the closed shell code and 2x the number of input files for the open shell code. The second row contains the number of sub-fragments (which should be atomic orbitals) in each fragment. In the example above each fragment had 12 types of atomic orbital.

So for the example above column 2-14 shows contributions from "all_frags_alpha" while column 15-27 shows contributions from "all_frags_beta" (column 1 lists energies).

==Bugs/Things to Watch For==
===Fragment Names===
Make sure your fragment names arent a substring of your .fchk file names. For example an input file called "Cl.test" wont work for a file called "BMIM_Cl.fchk". However, "Cl_frag.test" will work. Therefore personally i add _frag to the end of every input file name. The other way to sort this is to set linked_files=0 in gen_gelius_input_opensh.sh, sincce the link files part of the code is where this bug comes from.

===Fragment Error Checking===
Unless you manually turn it off (The "error=" option in gen_gelius_input_opensh.sh script) the code tries to check your input file for errors. Specifically it tests whether the TDOS in an input_file really is the sum of the other sub-fragments in the input(.test) file. E.g it will trigger if you request only Sulfur p-type and d-type AOs (but no s-type AOs) in the input file, or if you request all sulfur p-type AOs twice. If the error check triggers then it BREAKS the input file for matlab by putting an error msg in the first line of the '''*MLinpt.txt file'''. An example of what the error looks like is:
<pre>
“Fragment Sum incorrect on row 19 for fchk file xxxx.fchk”
</pre>

If you *MLinpt.txt files arent working in matlab then this is probably the reason.

===Pathlengths===
This code makes a lot of sub-folders based on the name of .fchk files. If you have really long .fchk file names its likely that you'll end up with some files which exceed the 256 character limit for a pathlength on a windows computer. In matlab 2014 this will cause the code to exit with an error, saying it cant save the .fig files. In newer matlab versions it may just save the output files , but you wont be able to open them on windows if the path length is greater than 256 characters. Ways to avoid this are to shorten .fchk file names where possible, or (less practical) run the code on a folder as close as possible to root (i.e a folder with a short a pathlength as possible).

Talk:Mod:Hunt Research Group/Calc XPS Code

2017-06-07T13:56:54Z

Rf614:

==Links to Files==
Open Shell Version: [[Media:Gelius_open_shell_code_Aug2016.zip]]

Closed Shell Version: [[Media:Gelius_closed_shell_code_Aug2016.zip]]

==Introduction==
The purpose of this code is to calculate partial density of states (pDoS) plots and Valence band x-ray photoelectron (XP) spectra from .fchk files. There is both an open shell and a closed shell version of the code, they work in exactly the same way.

All code was written using python 2.7 (with numpy), Multiwfn version 3.3.8, Matlab version 2014a and a cygwin(bash) shell on windows 8. It should work on windows or linux if the correct options are set. It might not work on a mac currently due to Multiwfn not working properly (I haven't personally tested it though).

==Some Theory==

===Mulliken Population Analysis===

A pretty key part to this code is the breakdown of molecular orbitals (MOs) in atomic orbitals (AOs, which are essentially basis functions). This is carried out in multiwfn using Mulliken population analysis, which i'll briefly summarise due to its importance.

The basic question is "For MO X, how many electrons come from atomic orbital Y?". Hence we'll define <math>\psi</math> as the wavefunction for a single MO. Next we write <math>\psi</math> as the linear combination of individual basis functions (<math>\phi_{i}</math>):

<math>
\psi = {{\sum\limits_{i} c_{i} \phi_{i} }}
</math>

Since we want a number of electrons, we're interested in the formula for the electron density:
<math>
\langle{}\Psi | \Psi \rangle{}
</math>

===What the code actually does (roughly/briefly)===

=="Installation" Instructions==

All instructions will be given for the open shell version of the code. However, the process is the same except whenever these instructions say to edit the "gen_gelius_input_opensh.sh", just do the corresponding edit on the gen_gelius_input.sh file. The Matlab scripts are EXACTLY the same for both open shell/closed shell codes.

0) Make sure you have multiwfn (https://multiwfn.codeplex.com) saved somewhere, python 2.x and numpy installed.

1)Put the following files (all the .py/.sh files) somewhere in system path (e.g i use /usr/local/bin):
:a)Format_Multiwfn_Output_alpha.py (Format_Multiwfn_Output.py for closed shell)
:b)Format_Multilwfn_Output_beta.py
:c)Gen_Multiwfn_Input_Comms.py
:d)Gen_Multiwfn_Input_Comms_beta.py
:e)Merge_Fragment_Files.py
:f)gen_gelius_input_opensh.sh

Make sure all files have executable persmission (navigate to the folder and do "chmod u+x filename")

2)In the gen_gelius_input_opensh.sh change the current lines accordingly:
:a) multiwfn_folder = 'C:/Users/Richard/Documents/Multiwfn_Installed' to the directory where you store multiwfn and associated files.
:b) "cygwin=1” to “cygwin=0” if your not using a Cygwin shell.
:c) “old_sed=1” to “old_sed=0” if your version of sed allows the –i option (edit in place) to be used. I don’t think the standard version on mac does(meaning set old_sed=1),standard version of Cygwin does though.
:d) POSSIBLY "cp $multiwfn_folder/Multiwfn.exe $curr_folder" to "cp $multiwfn_folder/Multiwfn $curr_folder" if on linux(should be really obvious if this is required). Do the same for the line where multiwfn is deleted from current folder("rm Multiwfn.exe").

3)Put the following files somewhere in matlabs path (e.g your matlab folder in documents)
:a) Get_X_Section.m
:b) Gen_Gelius_Spectrum.m

The Get_X_Section REALLY does need to be in matlabs path (unless you want to edit some code in Gen_Gelius_Spectrum.m). If you know what your doing in matlab feel free to put Gen_Gelius_Spectrum.m wherever you want.

4)Put the X_Section_Library folder wherever you want.

5) In Get_X_Section.m modify the following line:
:"Lib_Path ='C:\Users\Richard\Documents\X_Section_Library' " to the path where you put the X_Section_Library folder in.

6) OPTIONAL: Add the code in vim_template_script to your vimrc (c+p it somewhere in the file). "vim ~/.vimrc" to access your vimrc. You can then bring up a template input file in vim by using ":call Gelius_Input()"
Since i forgot to put the relevant text file in to the code folders, just c+p the below into your vimrc to get this to work.

<pre>
function! Gelius_Input()
exe 'normal aCoSCN4_freq_AllNCS_B3LYP_6311plgdp_Tetrahed_BMIMSCNSMD_Abr.fchk\rNumb_Fragments=10\r1a=\r1t=a\r1l=TDOS\r2a=\r2t=\r2l=\r3a=\r3t=\r3l=\r4a=\r4t=\r4l=\r5a=\r5t=\r5l=\r6a=\r6t=\r6l=\r7a=\r7t=\r7l=\r8a=\r8t=\r8l=\r9a=\r9t=\r9l=\r10a=\r10t=\r10l=\rM_factor=1.0\rLink_File='
exe '%s/\\r/\r/g'
endfunction
</pre>

==Running The Code==
====Using the Example files====
1) Navigate (in your shell) to the folder "Example_files". e.g for me i ran:

"cd /cygdrive/c/Users/Richard/Documents/Random_Coding/Gelius_Spectrum_Automated_Jan_2016/open_shell_code/Example_files"

2) Run this command in console(for closed shell use gen_gelius_input.sh):

"gen_gelius_input_opensh.sh"

Command 2) will have created a folder for each .fchk file in Example_files/. Therefore you should now have folders:
:CoSCN4_freq_AllNCS_B3LYP_6311plgdp_Tetrahed_BMIMSCNSMD_Abr
:other_file
Within these folders are tons of output files. The most important ones are those ending in "MLinpt.txt" - these will be the input files for step 3.

The script will run on all .fchk files in the current folder, when you do this for real you need to make sure that all the fchks are for the same system (e.g a water molecule) and that the atomic numbering is the same in all files (view-->labels in gaussview gives you what im calling atomic numbering).

If anything went wrong try revisiting the installation instructions.

3) Everything from now on is done using matlab. Theres a file called "pathname_file" in the Example_files folder which was created by the gen_gelius_input_opensh.sh script. Copy its full contents to clipboard and in MATLAB terminal type:

<pre>
plist=<CTRL-V>
(<CTRL-V> means paste, rather than type out CTRL-V)
</pre>

In matlab terms, plist now contains a cell array where each entry is the full path to a folder. The folder paths are those for the folders created in step 2). Hence to do this manually for just one of the two folders you'd type something like:
<pre>
plist={'C:\Users\Richard\Documents\Random_Coding\Gelius_Spectrum_Automated_Jan_2016\open_shell_code\Example_files\other_file'}
</pre>

4) To create either an XPS spectrum or pDoS plots we use the Gen_Gelius_Spectrum script. This takes the following three arguments IN THE FOLLOWING ORDER:
:a) the plist variable we just made
:b) the photon energy we want to use (any negative number requests a pDoS rather than an XPS spectrum)
:c) the FWHM of the peaks to place around each MO energy (units = eV)

So to create a pDoS plot you could type the following into the matlab terminal(Try this one first):
<pre>
Gen_Gelius_Spectrum(plist,-1,1)
</pre>
To create an XPS spectrum taken at photon energy of 1486eV (standard lab-based value) type the following:
<pre>
Gen_Gelius_Spectrum(plist,1486,1)
</pre>

If you get an error like "Undefined function 'Gen_Gelius_Spectrum' for input arguments of type 'cell'.", then your Gen_Gelius_Spectrum script probably isnt in your matlab path. (see installation instructions, or google a bit)

If your error free then well done. Sub-folders should have been created in "other_file" and "CoSCN4_freq_AllNCS_B3LYP_6311plgdp_Tetrahed_BMIMSCNSMD_Abr", which in turn will contain a loada output files. See the output files section of the wiki for an explanation of what these are.

====Creating your own input files====
If you followed the optional part above and made a vimscript, then a template for the input file can be brought up in vim by typing ":call Gelius_Input()"

An example input file is shown below. Each "fragment" (e.g cation,anion, alkyl group etc.) should have its own input file.
<pre>
other_file.fchk
Numb_Fragments=4
1a=10-13
1t=a
1l=TDOS
2a=10-13
2t=s
2l=S3s
3a=10-13
3t=p
3l=S3p
4a=10-13
4t=d+
4l=S3d
M_factor=1.0
Link_File=all_frags.test
</pre>
The input file can be divided into parts as follows:

1) The top line of the input file tells the code the name of the checkpoint file to use in the calculation. In practice you can write ANYTHING on this top line if using the gen_gelius_input*.sh script, as the script itself replaces the first line (whatevers there) with the names of all .fchks in the current folder. But SOMETHING must be written there.

2)'''Numb_Fragments''' is the number of sub-fragments (max 10) that you want to break your pDoS down into (e.g fragment 2 corresponds to all Sulfur 3s atomic orbitals in the example above). Code will break if you get this wrong.

3) '''1a/1t/1l''' instruct the code what to use as the total density of states. Hence 1a must be set to ALL the atoms your interested in for this fragment, 1t must be set to "a" and 1l should be set to TDOS (not sure it matters what this is set to, but to be on the safe side just use TDOS). The purpose of this fragment is essentially error checking.

4) '''2a/3a/*a''' : a stands for atoms here, so the value of a corresponds to the labels of the atoms you want to be included in the sub-fragment. These labels are the same integers you get in gaussview when you go to view-->labels. Examples of valid options may be "1,4,5-7" or "1,3,5" (basically comma seperated list of integers)

5) '''2t/3t/*t''': t stands for "type of AO".This is a comma seperated list with possible arguments (any combination) "a,s,p,d,f,g". "a" corresponds to all types of AO centered of the atoms specified with the 2a/3a/*a line. s means s-type atomc orbitals (0 angular momentum), p means p-type atomic orbitals (1 angular momentum) etc. To get all AOs of a certain angular momentum and higher (up to g-type AOs) add a plus to the end of the relevant letter. e.g "p+" says "include all AOs with angular momemtnum greater than or equal to 1 in this sub-fragment")
'''DON'T''' use "a" with any other letter, and dont do anything else that will lead to a type of AO being requested twice (e.g p+,d)

6) '''2l/3l/*l''': l stands for label. If you dont care about calculating an XPS spectrum you can essentially set it to whatever you want. This tells the matlab script what cross section to use for calculating the XPS spectrum, hence it must match (case insensitive) a filename in the X_Section_Library folder. e.g S3s tells matlab to get the S3S.txt file when looking for what cross-section to use.

7) '''M_factor=1.0''', you really never need to modify this but make sure its present. It's purpose is to tell matlab to multiple the pDoS of this fragment (that this input file corresponds to) by the value M_factor. The purpose is for systems where the real stoichiometry differs from that in your calculation. For example if i have a mixture of 99 water:1 anion but only a calculation on 1 water molecule mixed with 1 anion i could set M_factor for the water fragment input file to M_factor=99.0 to estimate (badly) what the actual DoS of the mixture would look like.

8) '''Link_File=*'''. This options for if you have a fragment (e.g an anion) but want it broken into more than 10 sub-fragments (e.g this will be true for any molecule with >3 elements). In this case make 2 files for that fragment, and in the second file set Link_File to the name of the first file. In the example above, this input file will essentially be merged with "all_frags.test", and only an output file for all_frags.test will be generated.

====More complex input file options====
====Gen_Gelius_Spectrum.m====
This code has the following options that can be changed (all are explained by comments in the .m file):
<pre>1)min_e/max_e
</pre>
Matlab ignores any orbitals with energy below min_e or above max_e when generating the XPS spectrum. Note if you want to change this to include core orbitals or very high energy virtual orbitals then you'll also have to edit Gen_Multiwfn_Input_Comms.py to set the range in multiwfn to the one you want.(Theres currently no variable in that .py code to do that, it just uses the defaults in multiwfn. But the function you'll need to edit in Gen_Multiwfn_Input_Comms.py is Output_Header.
<pre>2)min_e_plotted/max_e_plotted
</pre>
These refer to the minimum/maximum binding energies to calculate spectral intensity for. The range should probably be slightly wider than the min_e/max_e range really, though thats not the default setting.
<pre>3)step_size
</pre>
The value (in eV) between calculated data points in the output spectra. For example, if set to 0.001eV then a data point will be calculated for 0eV,0.001eV,0.002eV etc. Lower values=larger files/slower runtimes.
<pre>4)lorentz_fraction
</pre>
Value between 0 and 1 that determines the % Lorentzian character of the lineshape used in the broadening function(the remaining % is set to gaussian). By broadening function i essentially mean the peak thats placed around every MO energy in generating these spectra. lorentz_fraction=0 will give you a pure gaussian peak, while lorentz_fraction=1 gives a pure Lorentzian peak.
<pre>5)savetext/figsave/savejpeg/all_contribs_output
</pre>
These all control whether a certain type of output file is saved, with variable names that should be self-explanatory.Values of 1 mean save the file, other values mean dont save. set savetext=0 to not save any .txt output files, figsave=0 to not save any .fig output, savejpeg=0 to not save any .jpeg outputs and all_contribs_output=0 to not save the .txt file with a full breakdown of the spectrum. Leaving them all as 1 (except maybe the savejpeg) is strongly recommended.

====Plot_Atomic_Contribs_V3====

This is an optional script that plots certain breakdowns of your calculated DoS/XPS spectrum (i.e you run it AFTER Gen_Gelius_Spectrum) . Its not too hard to do the same thing in excel using the *ALL_Contribs* output file (see output file sections) but this is a lot faster. There are three types of breakdown this function can do (see pictures below):

1) Atomic breakdown with fragments kept seperate (merge_spd=1,merge_fragments=0)

2) Atomic orbital breakdown (merge_spd=0,merge_fragments=1)

3) Atomic breakdown (merge_spd=1,merge_fragments=1)

The function in its uploaded form takes a single argument , though there are two variables in the actual code that you need to mess with (merge_spd and merge_fragments).

So for example, to run this on the output file from the "Using the Example files" i typed into the matlab terminal:

Plot_Atomic_Contribs_V3('C:\Users\Richard\Desktop\Open_Shell_Code_Some_Stuff\Example_files\other_file\DoS_1ev_0pt3l')

The argument needs to be a folder containing the output files from the Gen_Gelius_Spectrum script. You'll then get a saved graph with the breakdown you asked for and also a .txt file containing the breakdown in a comma delimited format. The output file names record the options you used to make the script (e.g "Plotted_Atomic_Contribs_spd0_frag1.txt" means merge_spd was set to 0 and merge_fragments was set to 1).

[[File:Plotted Atomic Contribs spd1 frag1.jpg|thumbnail|center| merge_spd=1,merge_fragments=1 ]]

[[File:Plotted_Atomic_Contribs_spd0_frag1.jpg|thumbnail|center| merge_spd=0,merge_fragments=1 ]]

[[File:Plotted Atomic Contribs spd1 frag0.jpg|thumbnail|center| merge_spd=1,merge_fragments=0 ]]

==Output Files==

The first "output file" is the *MCRF*.txt file, which just contains the commands to build the required fragment in multiwfn.

===Files From Multiwfn===
Most people should be able to completely ignore these files. But in case your curious:

'''1) *DOS_curve*.txt''' Essentially a DoS plot. 1st column is energies in au, second column is TDOS of the fragment. Not really sure what the 3rd column is. No particular use for this file, but it gets auto-generated by multiwfn and i dont go out my way to delete it.

'''2) *DOS_line*.txt''' Data to create a stick plot (where each MO is a stick). Plot column 1 vs column 2 as a line graph and you get an axis with sticks representing the MOs. 4th column onwards represent contribution of the various sub-fragments to each "stick". This is the file that gets used to generate the *ML_inpt.txt file.

'''3) *DOS_frag*.txt ''' This file contains the sub-fragment info in a way multiwfn understands. If its renamed to DOS_frag.txt then multiwfn can import it to get the fragments (therefore can be useful for debugging, no further purpose).

===MLinpt.txt File===

Creation of the *ML_inpt.txt files is essentially the point of the gen_gelius_input* script. These are primarily input files for matlab, but they do have useful information in an easy to read format. If you import the file into excel and COMMA DELIMIT ONLY (i.e untick the space delimit option) you can get something like below(there are more decimal places in the actual file):

<pre>
#M_FACTOR=1.0,
#Energy(eV) TDOS CO4S CO3P CO3D N2S N2P N2D C2S C2P C2D S3S S3P S3D
[LINES DELETED]
-24.5 0.9 0.0 0.0 0.0 0.5 0.1 0.0 0.2 0.1 0.0 0.1 0.0 0.0
-24.4 1.0 0.0 0.0 0.0 0.5 0.1 0.0 0.3 0.1 0.0 0.0 0.0 0.0
-24.4 1.0 0.0 0.0 0.0 0.5 0.1 0.0 0.3 0.1 0.0 0.0 0.0 0.0
-24.4 1.0 0.0 0.0 0.0 0.5 0.1 0.0 0.3 0.1 0.0 0.0 0.0 0.0
-19.8 0.3 0.0 0.0 0.0 0.1 -0.1 0.0 0.2 0.1 0.0 0.7 0.1 0.0
-19.7 0.3 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.1 0.0 0.7 0.1 0.0
[LINES DELETED]
</pre>

The shown file is "all_frags_alpha_MLinpt.txt", which is generated from the example files in the open shell code. As this actually comes from merging two MLinpt.txt files (gen_gelius_input.sh does this part silently) the TDOS in this case doesnt mean anything.

The first column lists the energy of each MO in eV (1 ev ~27.212 au). For any row, columns 3(CO4S)-->end (S3D) list the fractional contributions of each type of atmoic orbital to the MO in that row. For example the MO with energy -19.7 eV is mainly from S3s type atomic orbitals. These fractions were determiend by multiwfn by using Mulliken population analysis.

All the fractional contributions in the file get multiplied by the M_FACTOR value (as explained above in the Creating your own input files section). Therefore the value of M_FACTOR can easily be set after the MLinpt.txt file is created, as opposed to within the initial input file.

===Formatted_UMOs===

The default for the gen_gelius_input* script (i.e the settings for the version i uploaded) is to put only OCCUPIED MO energies/breakdowns into the *MLinpt.txt file. If you want all MOs (I really DONT recommend this) in the MLinpt.txt file then change all_orbs='cheese' to all_orbs='ALLORBS' at the top of the gen_gelius_input* script.

Otherwise you will get a *Formatted_UMOs* file for every MLinpt.txt file you get. The format of the file is identical to the MLinpt.txt format except the MO at the top of the file corresponds to the LUMO.

WARNING: If you use the link_files feature, then the output formatted_UMOs files DO NOT get merged. This shouldnt matter as you should never really use the formatted UMOs for anything.

===Output From Matlab Code ===

The matlab code firstly will create a new folder with a name reflecting the input arguments. The format is hv_FWHM_%lorenztian where hv is the photon energy, FWHM is full width half maximum of the broadeing function and %lorenztian is the amount of lorenztian mixed in the broadening function. For example the earlier command (from using example files section) "Gen_Gelius_Spectrum(plist,-1,1)" generates the folder "DoS_1ev_0pt3l". DoS = Density of states, and means we did not use photoionisation cross sections. The 0pt3l part means our broadening function was 30% lorenztian and 70% Gaussian.

Two .fig files will have been produced from this code, these contain matlab graphs of the output. These .fig files follow the naming ("filename" is the name of the original fchk file):

1) filename_DoS.fig contains the total DoS plot.

2) filename_DoS_frag_contribs.fig contains the DoS AND the pDoS of each of the fragments. For closed shell code you get 1 fragment per input file (assuming you dont use link_files), but for the open shell code each input file leads to two fragments - one for alpha electrons and one for beta electrons.

[[File:Other_file_DoS.jpeg|thumb|center| 1) Total Density of States for a Co(SCN)4 fragment with filename other_file_DoS.fig]]

[[File:Other file DoS frag contribs.jpeg|thumb|center| 2) Contribtuions of alpha and beta electrons to the Total Density of States for Co(SCN)4, filename other_file_DoS_frag_contribs.fig]]

The filename_DoS.fig plot data is output in a .txt file of the same name, while each fragment in filename_DoS_frag_contribs.fig has its own .txt file. For the example above the two fragments have names "all_frags_alpha_other_file_DoS" and "all_frags_beta_other_file_DoS". The start of the names are taken from the name of the original input file "all_frags.test".

The final file output is the "ALL_Contribs_other_file_DoS.txt" file. This(as the name implies) contains all contributions to the DoS spectra in a comma delimited file. The Plot_Atomic_Contribs*.m code can be run on this file to generate plots with a few different breakdowns (e.g atomic breakdowns) - see the relevant section of this wiki (not done yet....). The file will look something like below (commas replaced with tabs):

<pre>
#all_frags_alpha all_frags_beta
#Sub fragments in each= 12 12
#Energy(eV) CO4S CO3P CO3D N2S N2P N2D C2S C2P C2D [COLUMNS REMOVED]
[ROWS REMOVED,numbers rounded]
7.5 0.00 0.01 0.31 0.04 0.13 0.00 0.00 0.11 0.00 [COLUMNS REMOVED]
7.51 0.00 0.01 0.33 0.04 0.13 0.00 0.00 0.11 0.00 [COLUMNS REMOVED]
7.52 0.00 0.01 0.34 0.04 0.13 0.00 0.00 0.11 0.00 [COLUMNS REMOVED]
7.53 0.00 0.01 0.35 0.04 0.14 0.00 0.00 0.12 0.00 [COLUMNS REMOVED]
7.54 0.00 0.01 0.36 0.04 0.14 0.00 0.00 0.12 0.00 [COLUMNS REMOVED]
7.55 0.00 0.02 0.37 0.04 0.14 0.00 0.00 0.12 0.00 [COLUMNS REMOVED]
7.56 0.00 0.02 0.39 0.05 0.14 0.00 0.00 0.13 0.00 [COLUMNS REMOVED]
[ROWS REMOVED]
</pre>

Hopefully the format is mostly intuitive, the top two rows contain information on what the data is, and the rest contain the data. The top row contains (originally comma delimited) a list of all "fragments" names, which is the number of input files for the closed shell code and 2x the number of input files for the open shell code. The second row contains the number of sub-fragments (which should be atomic orbitals) in each fragment. In the example above each fragment had 12 types of atomic orbital.

So for the example above column 2-14 shows contributions from "all_frags_alpha" while column 15-27 shows contributions from "all_frags_beta" (column 1 lists energies).

==Bugs/Things to Watch For==
===Fragment Names===
Make sure your fragment names arent a substring of your .fchk file names. For example an input file called "Cl.test" wont work for a file called "BMIM_Cl.fchk". However, "Cl_frag.test" will work. Therefore personally i add _frag to the end of every input file name. The other way to sort this is to set linked_files=0 in gen_gelius_input_opensh.sh, sincce the link files part of the code is where this bug comes from.

===Fragment Error Checking===
Unless you manually turn it off (The "error=" option in gen_gelius_input_opensh.sh script) the code tries to check your input file for errors. Specifically it tests whether the TDOS in an input_file really is the sum of the other sub-fragments in the input(.test) file. E.g it will trigger if you request only Sulfur p-type and d-type AOs (but no s-type AOs) in the input file, or if you request all sulfur p-type AOs twice. If the error check triggers then it BREAKS the input file for matlab by putting an error msg in the first line of the '''*MLinpt.txt file'''. An example of what the error looks like is:
<pre>
“Fragment Sum incorrect on row 19 for fchk file xxxx.fchk”
</pre>

If you *MLinpt.txt files arent working in matlab then this is probably the reason.

===Pathlengths===
This code makes a lot of sub-folders based on the name of .fchk files. If you have really long .fchk file names its likely that you'll end up with some files which exceed the 256 character limit for a pathlength on a windows computer. In matlab 2014 this will cause the code to exit with an error, saying it cant save the .fig files. In newer matlab versions it may just save the output files , but you wont be able to open them on windows if the path length is greater than 256 characters. Ways to avoid this are to shorten .fchk file names where possible, or (less practical) run the code on a folder as close as possible to root (i.e a folder with a short a pathlength as possible).

Mod:Hunt Research Group/OTRSH Funct

2017-06-07T13:38:52Z

Rf614:

==Links to files==
Zip Archive: [[Media:OTRSH_hpc_ver_June3rd_2017.zip]]

==What's an OTRSH Functional?==
It is an optimally tuned range separated hybrid functional (OTRSH). This is a class of functionals that are useful for ccalculating electronic properties such as UV-Vis spectra (especially if charge-transfer excitations are important) or photoelectron spectra. Before talking about OTRSH functionals, it is useful to first talk about how standard range separated hybrid functionals work, and why they were created.

===The Problem===
Standard DFT has problems describing long range interactions, due to the form of the exchange energy in standard functionals. Actual (exact) Exchange energy scales with distance as <math>\frac{1}{r_{12}}</math> where <math>r_{12}</math> is the distance between two electrons. However, standard pure DFT exchange functionals depend directly on the electron density, which decays exponentially with distance from an atomic center (i.e. DFT exchange decays faster than actual exchange). This means that standard DFT functionals are poor at describing long-range interactions (charge transfer excitations in DFT is an example). Range seperated hybrid functionals try to fix this by using Hartree-Fock exchange energy (which is exact) at large distances (long-range) while keeping DFT exchange energy at short range.

===One Solution - Normal Range Separated Hybrids===
A range separated hydrbrid functional divides the exchange energy into two terms - short range exchange (<math> E_{X}^{SR}</math>) and long range exchange (<math> E_{X}^{LR} </math>). The exchange energy at any one distance is a weighted combination of short-range and long range exchange, and can essentially be described with three variables; <math>\alpha</math>, <math>\beta</math> and <math>\mu</math> (See picture below).

[[File:Frac_HF_Exchange_mu.png|200px|center|Description of the Exchange energy at varying distances in a range-separated hybrid functtional]]

The equations for total, short range (DFT) and long-range (HF) exchange are:

<math>
E_{X}^{Total} = E_{X}^{SR} + E_{X}^{LR}
</math>

<math>
E_{X}^{SR} = [1-(\alpha + \beta{}.\text{erf}(\mu r_{12})] E_{X}^{DFT}(r_{12})
</math>

<math>
E_{X}^{LR} = [\alpha + \beta{}.\text{erf}(\mu{}r_{12})] E_{X}^{HF}(r_{12})
</math>

where erf is the error function. <math>\alpha</math> represents the amount of HF exchange at short range. For example, <math>\alpha=0.2</math> would mean 20% HF exchange was used at <math>r_{12}=0</math>. <math>\alpha + \beta </math> represents the amount of HF exchange at long range (i.e as <math>r_{12}</math> approaches infinity). A value of <math>\alpha + \beta =1</math> means that 100% HF exchange is employed at long range, which should give the correct long-range behavior for the exchange energy. <math>\mu</math> is the range seperation parameter, its value determines how quickly we change from short-range to long-range exchange expresssions. The higher <math>\mu</math> is, the faster (shorter distances) we start using the long-range exchange energy expression.

A problem with range separated hybrids is in picking the correct value for <math>\mu</math> (i.e when should we stop using DFT exchange?). This is often determined empirically by varying <math>\mu</math> for a test data set, and seeing which <math>\mu</math> value leads to a functional that best reproduces an experimental property (e.g. atomic ionisation energies). This has the same problem as any kind of empirical fitting - its not clear how well the functional will work outside the test set of molecules/properties. Also this empirical tuning of parameters can lead to unphysical values. For example, CAM-B3LYP uses <math>\alpha + \beta =0.65</math>, meaning it definitely has the wrong long-range behavior.

===OTRSH Functionals===

OTRSH functionals provide a non-empirical method for choosing <math>\mu</math>; <math>\mu</math> is "tuned" on a per-calculation basis. The <math>\mu</math> value is chosen so that the calculated ionisation energy matches the negative of the HOMO energy, i.e.:

<math>
IP_{Calc} = -E_{HOMO}
</math>

<math>
IP_{Calc} = E_{N-1} - E_{N}
</math>

Where <math>E_{N-1}</math> and <math>E_{N}</math> represent the energy of the N-1 and N electron system (both in the N electron system geometry). Note that all values in the equations above can be obtained from two single-point energy calculations. Also note that <math>IP_{Calc} = -E_{HOMO}</math> is true for the exact (unknown) KS-DFT functional, hence it is a sensible thing to try to reproduce in a functional.

To summarise an OTRSH functional is one whereby <math>\mu</math> is chosen to minimise <math>
IP_{Calc} + E_{HOMO}</math>, hence the <math>\mu</math> value will be different for every calculation you carry out. Sometimes the IP and HOMO energy of the N-1 electron system are also used to tune <math>\mu</math>, but unfortunately thats not possible the code on this page.

==Running The Code==

===Quick Start Guide (Using the test file)===
Once you download the .zip archive, copy the files to somewhere on your hpc path [Note: make sure you dont miss the .rng_OTRSH file]. Personally i use "/home/rf614/bin" as the location. To use the gf_OTRSH function straight away you'll need to put them in $HOME/bin. "echo $HOME" while on the hpc to find where your home directory is if your not sure.

Move the test file ('H2O_SPE_6311plgdp_GP_B3LYP_builtin.com') to whereever you like in your work directory, and navigate to this folder. Then type either of the following commands (assuming you put the scripts in ~/bin):

<pre>
qsub -v in=H2O_SPE_6311plgdp_GP_B3LYP_builtin -N test ~/bin/.rng_OTRSH
gf_OTRSH H2O_SPE_6311plgdp_GP_B3LYP_builtin.com
</pre>

If you saved the files somewhere else then simply modify "~/bin/.rng_OTRSH" to whatevers appropriate for the top command. For details on how the gf command works see this wiki [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:Hunt_Research_Group/pimpQSUB link].

===Quick Start Guide - The output files===
Assuming you ran the job above you will now have a loada output files in your current directory, i.e.:

H2O_SPE_6311plgdp_GP_B3LYP_builtin_gsstate_mu_opt.*

H2O_SPE_6311plgdp_GP_B3LYP_builtin_ionstate_mu_opt.*

H2O_SPE_6311plgdp_GP_B3LYP_builtin_OTRSH.out

The '''H2O_SPE_6311plgdp_GP_B3LYP_builtin_OTRSH.out''' file is essentially a progress file for how the tuning process is going. Once the calculation is over its main purpose is really to keep track of the options you used to create the OTRSH functional. Its a comma delimited file, and the headings should be pretty self-explanatory (at least they will be if you read the above section on "What is an OTRSH")

The '''H2O_SPE_6311plgdp_GP_B3LYP_builtin_gsstate_mu_opt'''* files are for single energy calculations using the OTRSH functional tuned for this particular system. If your goal was to calculate ground state properties with an OTRSH then these files might be all you need. If you want to do something other than a single-point energy calculation, then just add the relevant keywords (e.g TD) to the H2O_SPE_6311plgdp_GP_B3LYP_builtin_gsstate_mu_opt.com file - this file contains all the keywords needed to define the OTRSH functional.

The '''H2O_SPE_6311plgdp_GP_B3LYP_builtin_ionstate_mu_opt'''* files are the single point energy calculations (using the OTRSH) for the ionised system. You probably have no need for these.

You will also have a new folder named '''temp_files_H2O_SPE_6311plgdp_GP_B3LYP_builtin'''. At every tested value for <math>\mu</math>, the .log files for the ground and ionised state are written to this folder. The purpose of this folder is to check nothings going wrong during the calculation. If something stops the code extracting scf energies for each <math>\mu</math> value, then it might just loop infinitely (or at least for a loooooong time). The code '''SHOULD''' catch these sorts of errors and kill the job, but if it runs a long time without any change in the *.out file then make sure the *.log files in this folder are ok. The secondary purpose of this folder is therefore to tell you WHY your job got killed, if it got killed (an error message should be given in the .out file as well but....)

===Quick Note on Creating Input Files===

If you want to run the code, with default options, on your own systems then theres not much to say. Simply prepare a normal '''SINGLE POINT ENERGY''' *.com file for your system, but remove the functional from the input line. Also dont use the "#p" command for excess printing - this will break the OTRSH code (sorry, the code essentially uses the # for a search + replace to insert a functional at each step of the tuning process). For example, this input line is fine(and used in the example file):
<pre>
# 6-311+g(d,p) geom=connectivity int=ultrafine scf=conver=9
</pre>

Make sure you use a '''single point energy calculation input file''', using a keyword like "freq" wont throw any errors, but you'll be waiting a really long time for your job to finish.

===Understanding (And Modifying) The Default Options===

The easily changeable variables in this code are stored in the .rng_OTRSH file. They are (in order of "likelyhood of needing altering"):

<pre>
es_mult=2
alpha_val=0
beta_val=1
mu_lower=0.1
mu_upper=0.2
max_step=0.03
min_step=0.001
P5_val=0.81
xc_funct="CAM-B3LYP"
</pre>

Its likely you'll only ever need/want to change the '''es_mult''' here. This is the multiplicity of the ionised state, and is obviously equal to 2 if you have a closed shell system. If you use this code on an open shell system you need to set es_mult to the appropriate value for the ionised state. Personally i create seperate .rng files for each value of multiplicity i use, to save me editing the .rng_OTRSH file too often.

'''alpha_val''' corresponds to the fraction HF exchange at short range. Its plausible that raising this to say alpha_val=0.2 might improve your results in some cases, so it might be worth trying the functional with two different alpha values if your trying to match an experimental spectrum or have something else you can validate with. I personally set it to zero since i think its the least arbitrary value it can be.

'''beta_val''' - as stated above "alpha_val + beta_val" corresponds to the fraction HF exchange at long range. So if you '''change alpha_val, you need to change beta_val too''', otherwise you may end up with >100% HF exchange at long range. I'd advise you keep alpha_val + beta_val =1.0, since that should give you the correct long-range potential.

'''mu_lower''' and '''mu_upper''' control the initial range of <math>\mu</math> values to try out. If you have a good idea what your final <math>\mu</math> will be, then settings mu_lower and mu_upper to nearby values should speed up the optimisation process. However, there is '''NO NEED''' for the final mu_val to be between these initial search values, it will just take longer if its far away from this range.

'''max_step''' and '''min_step''' correspond to the maximum and minimum change in <math>\mu</math> between optimisation steps. Note that i think the minimum possible min_step=0.0001, so dont set it lower than that or it could cause serious problems (it'll lead to a gradient of zero between two mu values, i have no idea how the optimiser will react to that but it wont be good). Theres probably '''NEVER''' a need to mess with these values. If the optimiser isnt efficient enough, changing these values wont help much. Writing a better optimiser (the current ones pretty terrible by the way,but it works so.....) would probably be easier than working out the best values for these numbers.

'''P5_val''' corresponds to a parameter in the DFT functional controlling the amount of non-local correlation to use. Its the same P5 parameter that appears on the gaussian 09/16 guide to "user-defined DFT functionals". Just leave it alone unless you REALLY know what your doing.

'''CAM-B3LYP''' corresponds to the underlying functional to use. The functional this code generates is essentially CAM-B3LYP with modified <math>\alpha</math>,<math>\beta</math> and <math>\mu</math> parameters. The underlying DFT exchange/ DFT correlation functionals are the same as those used in CAM-B3LYP. I've never tried modifying this, it would be a massive pain. If you want to try then i suggest you use a range-separated functional as your base and start by passing the relevant arguments to the "init_OTRSH.py" functional, and seeing if you can replicate the original functional (e.g if you want to use wB97XD as your base functional, try and reproduce reslts of that functional by passing relevant options to init_OTRSH.py.

Finally, the convergence criteria for the OTRSH (the IP_HOMO - E_HOMO value) is set to 0.01 eV by default, and controlled by the IP_conver_thres variable in optimiser.py. I guess its possible you may want to relax that criteria in certain circumstances (obviously it would speed up the optimisation and lower the accuracy).

==Further Reading/Links to relevant Papers==
CAM-B3LYP paper - doi:'''10.1016/j.cplett.2004.06.011'''

Early (first?) OTRSH paper - doi:'''10.1021/ja8087482'''

Review on using OTRSH functional to calculate bandgaps (2012) - doi:'''10.1021/ct2009363'''

Use of OTRSH functionals for Ground state properties (2013) - doi:'''10.1063/1.4807325'''

Mod:Hunt Research Group/OTRSH Funct

2017-06-03T15:55:37Z

Rf614: /* What is an OTRSH Functional? */

==Links to files==
Zip Archive: [[Media:OTRSH_hpc_ver_June3rd_2017.zip]]

==What's an OTRSH Functional?==
It is an optimally tuned range separated hybrid functional (OTRSH). This is a class of functionals that are useful for ccalculating electronic properties such as UV-Vis spectra (especially if charge-transfer excitations are important) or photoelectron spectra. Before talking about OTRSH functionals, it is useful to first talk about how standard range separated hybrid functionals work, and why they were created.

===The Problem===
Standard DFT has problems describing long range interactions, due to the form of the exchange energy in standard functionals. Actual (exact) Exchange energy scales with distance as <math>\frac{1}{r_{12}}</math> where <math>r_{12}</math> is the distance between two electrons. However, standard pure DFT exchange functionals depend directly on the electron density, which decays exponentially with distance from an atomic center (i.e. DFT exchange decays faster than actual exchange). This means that standard DFT functionals are poor at describing long-range interactions (charge transfer excitations in DFT is an example). Range seperated hybrid functionals try to fix this by using Hartree-Fock exchange energy (which is exact) at large distances (long-range) while keeping DFT exchange energy at short range.

===One Solution - Normal Range Separated Hybrids===
A range separated hydrbrid functional divides the exchange energy into two terms - short range exchange (<math> E_{X}^{SR}</math>) and long range exchange (<math> E_{X}^{LR} </math>). The exchange energy at any one distance is a weighted combination of short-range and long range exchange, and can essentially be described with three variables; <math>\alpha</math>, <math>\beta</math> and <math>\mu</math> (See picture below).

[[File:Frac_HF_Exchange_mu.png|200px|center|Description of the Exchange energy at varying distances in a range-separated hybrid functtional]]

The equations for total, short range (DFT) and long-range (HF) exchange are:

<math>
E_{X}^{Total} = E_{X}^{SR} + E_{X}^{LR}
</math>

<math>
E_{X}^{SR} = [1-(\alpha + \beta{}.\text{erf}(\mu r_{12})] E_{X}^{DFT}(r_{12})
</math>

<math>
E_{X}^{LR} = [\alpha + \beta{}.\text{erf}(\mu{}r_{12})] E_{X}^{HF}(r_{12})
</math>

where erf is the error function. <math>\alpha</math> represents the amount of HF exchange at short range. For example, <math>\alpha=0.2</math> would mean 20% HF exchange was used at <math>r_{12}=0</math>. <math>\alpha + \beta </math> represents the amount of HF exchange at long range (i.e as <math>r_{12}</math> approaches infinity). A value of <math>\alpha + \beta =1</math> means that 100% HF exchange is employed at long range, which should give the correct long-range behavior for the exchange energy. <math>\mu</math> is the range seperation parameter, its value determines how quickly we change from short-range to long-range exchange expresssions. The higher <math>\mu</math> is, the faster (shorter distances) we start using the long-range exchange energy expression.

A problem with range separated hybrids is in picking the correct value for <math>\mu</math> (i.e when should we stop using DFT exchange?). This is often determined empirically by varying <math>\mu</math> for a test data set, and seeing which <math>\mu</math> value leads to a functional that best reproduces an experimental property (e.g. atomic ionisation energies). This has the same problem as any kind of empirical fitting - its not clear how well the functional will work outside the test set of molecules/properties. Also this empirical tuning of parameters can lead to unphysical values. For example, CAM-B3LYP uses <math>\alpha + \beta =0.65</math>, meaning it definitely has the wrong long-range behavior.

===OTRSH Functionals===

OTRSH functionals provide a non-empirical method for choosing <math>\mu</math>; <math>\mu</math> is "tuned" on a per-calculation basis. The <math>\mu</math> value is chosen so that the calculated ionisation energy matches the negative of the HOMO energy, i.e.:

<math>
IP_{Calc} = -E_{HOMO}
</math>

<math>
IP_{Calc} = E_{N-1} - E_{N}
</math>

Where <math>E_{N-1}</math> and <math>E_{N}</math> represent the energy of the N-1 and N electron system (both in the N electron system geometry). Note that all values in the equations above can be obtained from two single-point energy calculations. Also note that <math>IP_{Calc} = -E_{HOMO}</math> is true for the exact (unknown) KS-DFT functional, hence it is a sensible thing to try to reproduce in a functional.

To summarise an OTRSH functional is one whereby <math>\mu</math> is chosen to minimise <math>
IP_{Calc} + E_{HOMO}</math>, hence the <math>\mu</math> value will be different for every calculation you carry out. Sometimes the IP and HOMO energy of the N-1 electron system are also used to tune <math>\mu</math>, but unfortunately thats not possible the code on this page.

==What it does(Roughly)==

==Running The Code==

===Quick Start Guide (Using the test file)===
Once you download the .zip archive, copy the files to somewhere on your hpc path [Note: make sure you dont miss the .rng_OTRSH file]. Personally i use "/home/rf614/bin" as the location. To use the gf_OTRSH function straight away you'll need to put them in $HOME/bin. "echo $HOME" while on the hpc to find where your home directory is if your not sure.

Move the test file ('H2O_SPE_6311plgdp_GP_B3LYP_builtin.com') to whereever you like in your work directory, and navigate to this folder. Then type either of the following commands (assuming you put the scripts in ~/bin):

<pre>
qsub -v in=H2O_SPE_6311plgdp_GP_B3LYP_builtin -N test ~/bin/.rng_OTRSH
gf_OTRSH H2O_SPE_6311plgdp_GP_B3LYP_builtin.com
</pre>

If you saved the files somewhere else then simply modify "~/bin/.rng_OTRSH" to whatevers appropriate for the top command. For details on how the gf command works see this wiki [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:Hunt_Research_Group/pimpQSUB link].

===Quick Start Guide - The output files===
Assuming you ran the job above you will now have a loada output files in your current directory, i.e.:

H2O_SPE_6311plgdp_GP_B3LYP_builtin_gsstate_mu_opt.*

H2O_SPE_6311plgdp_GP_B3LYP_builtin_ionstate_mu_opt.*

H2O_SPE_6311plgdp_GP_B3LYP_builtin_OTRSH.out

The '''H2O_SPE_6311plgdp_GP_B3LYP_builtin_OTRSH.out''' file is essentially a progress file for how the tuning process is going. Once the calculation is over its main purpose is really to keep track of the options you used to create the OTRSH functional. Its a comma delimited file, and the headings should be pretty self-explanatory (at least they will be if you read the above section on "What is an OTRSH")

The '''H2O_SPE_6311plgdp_GP_B3LYP_builtin_gsstate_mu_opt'''* files are for single energy calculations using the OTRSH functional tuned for this particular system. If your goal was to calculate ground state properties with an OTRSH then these files might be all you need. If you want to do something other than a single-point energy calculation, then just add the relevant keywords (e.g TD) to the H2O_SPE_6311plgdp_GP_B3LYP_builtin_gsstate_mu_opt.com file - this file contains all the keywords needed to define the OTRSH functional.

The '''H2O_SPE_6311plgdp_GP_B3LYP_builtin_ionstate_mu_opt'''* files are the single point energy calculations (using the OTRSH) for the ionised system. You probably have no need for these.

You will also have a new folder named '''temp_files_H2O_SPE_6311plgdp_GP_B3LYP_builtin'''. At every tested value for <math>\mu</math>, the .log files for the ground and ionised state are written to this folder. The purpose of this folder is to check nothings going wrong during the calculation. If something stops the code extracting scf energies for each <math>\mu</math> value, then it might just loop infinitely (or at least for a loooooong time). The code '''SHOULD''' catch these sorts of errors and kill the job, but if it runs a long time without any change in the *.out file then make sure the *.log files in this folder are ok. The secondary purpose of this folder is therefore to tell you WHY your job got killed, if it got killed (an error message should be given in the .out file as well but....)

===Quick Note on Creating Input Files===

If you want to run the code, with default options, on your own systems then theres not much to say. Simply prepare a normal '''SINGLE POINT ENERGY''' *.com file for your system, but remove the functional from the input line. Also dont use the "#p" command for excess printing - this will break the OTRSH code (sorry, the code essentially uses the # for a search + replace to insert a functional at each step of the tuning process). For example, this input line is fine(and used in the example file):
<pre>
# 6-311+g(d,p) geom=connectivity int=ultrafine scf=conver=9
</pre>

Make sure you use a '''single point energy calculation input file''', using a keyword like "freq" wont throw any errors, but you'll be waiting a really long time for your job to finish.

===Understanding (And Modifying) The Default Options===

The easily changeable variables in this code are stored in the .rng_OTRSH file. They are (in order of "likelyhood of needing altering"):

<pre>
es_mult=2
alpha_val=0
beta_val=1
mu_lower=0.1
mu_upper=0.2
max_step=0.03
min_step=0.001
P5_val=0.81
xc_funct="CAM-B3LYP"
</pre>

Its likely you'll only ever need/want to change the '''es_mult''' here. This is the multiplicity of the ionised state, and is obviously equal to 2 if you have a closed shell system. If you use this code on an open shell system you need to set es_mult to the appropriate value for the ionised state. Personally i create seperate .rng files for each value of multiplicity i use, to save me editing the .rng_OTRSH file too often.

'''alpha_val''' corresponds to the fraction HF exchange at short range. Its plausible that raising this to say alpha_val=0.2 might improve your results in some cases, so it might be worth trying the functional with two different alpha values if your trying to match an experimental spectrum or have something else you can validate with. I personally set it to zero since i think its the least arbitrary value it can be.

'''beta_val''' - as stated above "alpha_val + beta_val" corresponds to the fraction HF exchange at long range. So if you '''change alpha_val, you need to change beta_val too''', otherwise you may end up with >100% HF exchange at long range. I'd advise you keep alpha_val + beta_val =1.0, since that should give you the correct long-range potential.

'''mu_lower''' and '''mu_upper''' control the initial range of <math>\mu</math> values to try out. If you have a good idea what your final <math>\mu</math> will be, then settings mu_lower and mu_upper to nearby values should speed up the optimisation process. However, there is '''NO NEED''' for the final mu_val to be between these initial search values, it will just take longer if its far away from this range.

'''max_step''' and '''min_step''' correspond to the maximum and minimum change in <math>\mu</math> between optimisation steps. Note that i think the minimum possible min_step=0.0001, so dont set it lower than that or it could cause serious problems (it'll lead to a gradient of zero between two mu values, i have no idea how the optimiser will react to that but it wont be good). Theres probably '''NEVER''' a need to mess with these values. If the optimiser isnt efficient enough, changing these values wont help much. Writing a better optimiser (the current ones pretty terrible by the way,but it works so.....) would probably be easier than working out the best values for these numbers.

'''P5_val''' corresponds to a parameter in the DFT functional controlling the amount of non-local correlation to use. Its the same P5 parameter that appears on the gaussian 09/16 guide to "user-defined DFT functionals". Just leave it alone unless you REALLY know what your doing.

'''CAM-B3LYP''' corresponds to the underlying functional to use. The functional this code generates is essentially CAM-B3LYP with modified <math>\alpha</math>,<math>\beta</math> and <math>\mu</math> parameters. The underlying DFT exchange/ DFT correlation functionals are the same as those used in CAM-B3LYP. I've never tried modifying this, it would be a massive pain. If you want to try then i suggest you use a range-separated functional as your base and start by passing the relevant arguments to the "init_OTRSH.py" functional, and seeing if you can replicate the original functional (e.g if you want to use wB97XD as your base functional, try and reproduce reslts of that functional by passing relevant options to init_OTRSH.py.

Finally, the convergence criteria for the OTRSH (the IP_HOMO - E_HOMO value) is set to 0.01 eV by default, and controlled by the IP_conver_thres variable in optimiser.py. I guess its possible you may want to relax that criteria in certain circumstances (obviously it would speed up the optimisation and lower the accuracy).

==Further Reading/Links to relevant Papers==

Mod:Hunt Research Group/OTRSH Funct

2017-06-03T15:55:21Z

Rf614: /* What is an OTRSH Functional */

==Links to files==
Zip Archive: [[Media:OTRSH_hpc_ver_June3rd_2017.zip]]

==What is an OTRSH Functional?==
It is an optimally tuned range separated hybrid functional (OTRSH). This is a class of functionals that are useful for ccalculating electronic properties such as UV-Vis spectra (especially if charge-transfer excitations are important) or photoelectron spectra. Before talking about OTRSH functionals, it is useful to first talk about how standard range separated hybrid functionals work, and why they were created.

===The Problem===
Standard DFT has problems describing long range interactions, due to the form of the exchange energy in standard functionals. Actual (exact) Exchange energy scales with distance as <math>\frac{1}{r_{12}}</math> where <math>r_{12}</math> is the distance between two electrons. However, standard pure DFT exchange functionals depend directly on the electron density, which decays exponentially with distance from an atomic center (i.e. DFT exchange decays faster than actual exchange). This means that standard DFT functionals are poor at describing long-range interactions (charge transfer excitations in DFT is an example). Range seperated hybrid functionals try to fix this by using Hartree-Fock exchange energy (which is exact) at large distances (long-range) while keeping DFT exchange energy at short range.

===One Solution - Normal Range Separated Hybrids===
A range separated hydrbrid functional divides the exchange energy into two terms - short range exchange (<math> E_{X}^{SR}</math>) and long range exchange (<math> E_{X}^{LR} </math>). The exchange energy at any one distance is a weighted combination of short-range and long range exchange, and can essentially be described with three variables; <math>\alpha</math>, <math>\beta</math> and <math>\mu</math> (See picture below).

[[File:Frac_HF_Exchange_mu.png|200px|center|Description of the Exchange energy at varying distances in a range-separated hybrid functtional]]

The equations for total, short range (DFT) and long-range (HF) exchange are:

<math>
E_{X}^{Total} = E_{X}^{SR} + E_{X}^{LR}
</math>

<math>
E_{X}^{SR} = [1-(\alpha + \beta{}.\text{erf}(\mu r_{12})] E_{X}^{DFT}(r_{12})
</math>

<math>
E_{X}^{LR} = [\alpha + \beta{}.\text{erf}(\mu{}r_{12})] E_{X}^{HF}(r_{12})
</math>

where erf is the error function. <math>\alpha</math> represents the amount of HF exchange at short range. For example, <math>\alpha=0.2</math> would mean 20% HF exchange was used at <math>r_{12}=0</math>. <math>\alpha + \beta </math> represents the amount of HF exchange at long range (i.e as <math>r_{12}</math> approaches infinity). A value of <math>\alpha + \beta =1</math> means that 100% HF exchange is employed at long range, which should give the correct long-range behavior for the exchange energy. <math>\mu</math> is the range seperation parameter, its value determines how quickly we change from short-range to long-range exchange expresssions. The higher <math>\mu</math> is, the faster (shorter distances) we start using the long-range exchange energy expression.

A problem with range separated hybrids is in picking the correct value for <math>\mu</math> (i.e when should we stop using DFT exchange?). This is often determined empirically by varying <math>\mu</math> for a test data set, and seeing which <math>\mu</math> value leads to a functional that best reproduces an experimental property (e.g. atomic ionisation energies). This has the same problem as any kind of empirical fitting - its not clear how well the functional will work outside the test set of molecules/properties. Also this empirical tuning of parameters can lead to unphysical values. For example, CAM-B3LYP uses <math>\alpha + \beta =0.65</math>, meaning it definitely has the wrong long-range behavior.

===OTRSH Functionals===

OTRSH functionals provide a non-empirical method for choosing <math>\mu</math>; <math>\mu</math> is "tuned" on a per-calculation basis. The <math>\mu</math> value is chosen so that the calculated ionisation energy matches the negative of the HOMO energy, i.e.:

<math>
IP_{Calc} = -E_{HOMO}
</math>

<math>
IP_{Calc} = E_{N-1} - E_{N}
</math>

Where <math>E_{N-1}</math> and <math>E_{N}</math> represent the energy of the N-1 and N electron system (both in the N electron system geometry). Note that all values in the equations above can be obtained from two single-point energy calculations. Also note that <math>IP_{Calc} = -E_{HOMO}</math> is true for the exact (unknown) KS-DFT functional, hence it is a sensible thing to try to reproduce in a functional.

To summarise an OTRSH functional is one whereby <math>\mu</math> is chosen to minimise <math>
IP_{Calc} + E_{HOMO}</math>, hence the <math>\mu</math> value will be different for every calculation you carry out. Sometimes the IP and HOMO energy of the N-1 electron system are also used to tune <math>\mu</math>, but unfortunately thats not possible the code on this page.

==What it does(Roughly)==

==Running The Code==

===Quick Start Guide (Using the test file)===
Once you download the .zip archive, copy the files to somewhere on your hpc path [Note: make sure you dont miss the .rng_OTRSH file]. Personally i use "/home/rf614/bin" as the location. To use the gf_OTRSH function straight away you'll need to put them in $HOME/bin. "echo $HOME" while on the hpc to find where your home directory is if your not sure.

Move the test file ('H2O_SPE_6311plgdp_GP_B3LYP_builtin.com') to whereever you like in your work directory, and navigate to this folder. Then type either of the following commands (assuming you put the scripts in ~/bin):

<pre>
qsub -v in=H2O_SPE_6311plgdp_GP_B3LYP_builtin -N test ~/bin/.rng_OTRSH
gf_OTRSH H2O_SPE_6311plgdp_GP_B3LYP_builtin.com
</pre>

If you saved the files somewhere else then simply modify "~/bin/.rng_OTRSH" to whatevers appropriate for the top command. For details on how the gf command works see this wiki [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:Hunt_Research_Group/pimpQSUB link].

===Quick Start Guide - The output files===
Assuming you ran the job above you will now have a loada output files in your current directory, i.e.:

H2O_SPE_6311plgdp_GP_B3LYP_builtin_gsstate_mu_opt.*

H2O_SPE_6311plgdp_GP_B3LYP_builtin_ionstate_mu_opt.*

H2O_SPE_6311plgdp_GP_B3LYP_builtin_OTRSH.out

The '''H2O_SPE_6311plgdp_GP_B3LYP_builtin_OTRSH.out''' file is essentially a progress file for how the tuning process is going. Once the calculation is over its main purpose is really to keep track of the options you used to create the OTRSH functional. Its a comma delimited file, and the headings should be pretty self-explanatory (at least they will be if you read the above section on "What is an OTRSH")

The '''H2O_SPE_6311plgdp_GP_B3LYP_builtin_gsstate_mu_opt'''* files are for single energy calculations using the OTRSH functional tuned for this particular system. If your goal was to calculate ground state properties with an OTRSH then these files might be all you need. If you want to do something other than a single-point energy calculation, then just add the relevant keywords (e.g TD) to the H2O_SPE_6311plgdp_GP_B3LYP_builtin_gsstate_mu_opt.com file - this file contains all the keywords needed to define the OTRSH functional.

The '''H2O_SPE_6311plgdp_GP_B3LYP_builtin_ionstate_mu_opt'''* files are the single point energy calculations (using the OTRSH) for the ionised system. You probably have no need for these.

You will also have a new folder named '''temp_files_H2O_SPE_6311plgdp_GP_B3LYP_builtin'''. At every tested value for <math>\mu</math>, the .log files for the ground and ionised state are written to this folder. The purpose of this folder is to check nothings going wrong during the calculation. If something stops the code extracting scf energies for each <math>\mu</math> value, then it might just loop infinitely (or at least for a loooooong time). The code '''SHOULD''' catch these sorts of errors and kill the job, but if it runs a long time without any change in the *.out file then make sure the *.log files in this folder are ok. The secondary purpose of this folder is therefore to tell you WHY your job got killed, if it got killed (an error message should be given in the .out file as well but....)

===Quick Note on Creating Input Files===

If you want to run the code, with default options, on your own systems then theres not much to say. Simply prepare a normal '''SINGLE POINT ENERGY''' *.com file for your system, but remove the functional from the input line. Also dont use the "#p" command for excess printing - this will break the OTRSH code (sorry, the code essentially uses the # for a search + replace to insert a functional at each step of the tuning process). For example, this input line is fine(and used in the example file):
<pre>
# 6-311+g(d,p) geom=connectivity int=ultrafine scf=conver=9
</pre>

Make sure you use a '''single point energy calculation input file''', using a keyword like "freq" wont throw any errors, but you'll be waiting a really long time for your job to finish.

===Understanding (And Modifying) The Default Options===

The easily changeable variables in this code are stored in the .rng_OTRSH file. They are (in order of "likelyhood of needing altering"):

<pre>
es_mult=2
alpha_val=0
beta_val=1
mu_lower=0.1
mu_upper=0.2
max_step=0.03
min_step=0.001
P5_val=0.81
xc_funct="CAM-B3LYP"
</pre>

Its likely you'll only ever need/want to change the '''es_mult''' here. This is the multiplicity of the ionised state, and is obviously equal to 2 if you have a closed shell system. If you use this code on an open shell system you need to set es_mult to the appropriate value for the ionised state. Personally i create seperate .rng files for each value of multiplicity i use, to save me editing the .rng_OTRSH file too often.

'''alpha_val''' corresponds to the fraction HF exchange at short range. Its plausible that raising this to say alpha_val=0.2 might improve your results in some cases, so it might be worth trying the functional with two different alpha values if your trying to match an experimental spectrum or have something else you can validate with. I personally set it to zero since i think its the least arbitrary value it can be.

'''beta_val''' - as stated above "alpha_val + beta_val" corresponds to the fraction HF exchange at long range. So if you '''change alpha_val, you need to change beta_val too''', otherwise you may end up with >100% HF exchange at long range. I'd advise you keep alpha_val + beta_val =1.0, since that should give you the correct long-range potential.

'''mu_lower''' and '''mu_upper''' control the initial range of <math>\mu</math> values to try out. If you have a good idea what your final <math>\mu</math> will be, then settings mu_lower and mu_upper to nearby values should speed up the optimisation process. However, there is '''NO NEED''' for the final mu_val to be between these initial search values, it will just take longer if its far away from this range.

'''max_step''' and '''min_step''' correspond to the maximum and minimum change in <math>\mu</math> between optimisation steps. Note that i think the minimum possible min_step=0.0001, so dont set it lower than that or it could cause serious problems (it'll lead to a gradient of zero between two mu values, i have no idea how the optimiser will react to that but it wont be good). Theres probably '''NEVER''' a need to mess with these values. If the optimiser isnt efficient enough, changing these values wont help much. Writing a better optimiser (the current ones pretty terrible by the way,but it works so.....) would probably be easier than working out the best values for these numbers.

'''P5_val''' corresponds to a parameter in the DFT functional controlling the amount of non-local correlation to use. Its the same P5 parameter that appears on the gaussian 09/16 guide to "user-defined DFT functionals". Just leave it alone unless you REALLY know what your doing.

'''CAM-B3LYP''' corresponds to the underlying functional to use. The functional this code generates is essentially CAM-B3LYP with modified <math>\alpha</math>,<math>\beta</math> and <math>\mu</math> parameters. The underlying DFT exchange/ DFT correlation functionals are the same as those used in CAM-B3LYP. I've never tried modifying this, it would be a massive pain. If you want to try then i suggest you use a range-separated functional as your base and start by passing the relevant arguments to the "init_OTRSH.py" functional, and seeing if you can replicate the original functional (e.g if you want to use wB97XD as your base functional, try and reproduce reslts of that functional by passing relevant options to init_OTRSH.py.

Finally, the convergence criteria for the OTRSH (the IP_HOMO - E_HOMO value) is set to 0.01 eV by default, and controlled by the IP_conver_thres variable in optimiser.py. I guess its possible you may want to relax that criteria in certain circumstances (obviously it would speed up the optimisation and lower the accuracy).

==Further Reading/Links to relevant Papers==

Mod:Hunt Research Group/OTRSH Funct

2017-06-03T15:54:45Z

Rf614: /* Running The Code */

==Links to files==
Zip Archive: [[Media:OTRSH_hpc_ver_June3rd_2017.zip]]

==What is an OTRSH Functional==
It is an optimally tuned range separated hybrid functional (OTRSH). This is a class of functionals that are useful for ccalculating electronic properties such as UV-Vis spectra (especially if charge-transfer excitations are important) or photoelectron spectra. Before talking about OTRSH functionals, it is useful to first talk about how standard range separated hybrid functionals work, and why they were created.

===The Problem===
Standard DFT has problems describing long range interactions, due to the form of the exchange energy in standard functionals. Actual (exact) Exchange energy scales with distance as <math>\frac{1}{r_{12}}</math> where <math>r_{12}</math> is the distance between two electrons. However, standard pure DFT exchange functionals depend directly on the electron density, which decays exponentially with distance from an atomic center (i.e. DFT exchange decays faster than actual exchange). This means that standard DFT functionals are poor at describing long-range interactions (charge transfer excitations in DFT is an example). Range seperated hybrid functionals try to fix this by using Hartree-Fock exchange energy (which is exact) at large distances (long-range) while keeping DFT exchange energy at short range.

===One Solution - Normal Range Separated Hybrids===
A range separated hydrbrid functional divides the exchange energy into two terms - short range exchange (<math> E_{X}^{SR}</math>) and long range exchange (<math> E_{X}^{LR} </math>). The exchange energy at any one distance is a weighted combination of short-range and long range exchange, and can essentially be described with three variables; <math>\alpha</math>, <math>\beta</math> and <math>\mu</math> (See picture below).

[[File:Frac_HF_Exchange_mu.png|200px|center|Description of the Exchange energy at varying distances in a range-separated hybrid functtional]]

The equations for total, short range (DFT) and long-range (HF) exchange are:

<math>
E_{X}^{Total} = E_{X}^{SR} + E_{X}^{LR}
</math>

<math>
E_{X}^{SR} = [1-(\alpha + \beta{}.\text{erf}(\mu r_{12})] E_{X}^{DFT}(r_{12})
</math>

<math>
E_{X}^{LR} = [\alpha + \beta{}.\text{erf}(\mu{}r_{12})] E_{X}^{HF}(r_{12})
</math>

where erf is the error function. <math>\alpha</math> represents the amount of HF exchange at short range. For example, <math>\alpha=0.2</math> would mean 20% HF exchange was used at <math>r_{12}=0</math>. <math>\alpha + \beta </math> represents the amount of HF exchange at long range (i.e as <math>r_{12}</math> approaches infinity). A value of <math>\alpha + \beta =1</math> means that 100% HF exchange is employed at long range, which should give the correct long-range behavior for the exchange energy. <math>\mu</math> is the range seperation parameter, its value determines how quickly we change from short-range to long-range exchange expresssions. The higher <math>\mu</math> is, the faster (shorter distances) we start using the long-range exchange energy expression.

A problem with range separated hybrids is in picking the correct value for <math>\mu</math> (i.e when should we stop using DFT exchange?). This is often determined empirically by varying <math>\mu</math> for a test data set, and seeing which <math>\mu</math> value leads to a functional that best reproduces an experimental property (e.g. atomic ionisation energies). This has the same problem as any kind of empirical fitting - its not clear how well the functional will work outside the test set of molecules/properties. Also this empirical tuning of parameters can lead to unphysical values. For example, CAM-B3LYP uses <math>\alpha + \beta =0.65</math>, meaning it definitely has the wrong long-range behavior.

===OTRSH Functionals===

OTRSH functionals provide a non-empirical method for choosing <math>\mu</math>; <math>\mu</math> is "tuned" on a per-calculation basis. The <math>\mu</math> value is chosen so that the calculated ionisation energy matches the negative of the HOMO energy, i.e.:

<math>
IP_{Calc} = -E_{HOMO}
</math>

<math>
IP_{Calc} = E_{N-1} - E_{N}
</math>

Where <math>E_{N-1}</math> and <math>E_{N}</math> represent the energy of the N-1 and N electron system (both in the N electron system geometry). Note that all values in the equations above can be obtained from two single-point energy calculations. Also note that <math>IP_{Calc} = -E_{HOMO}</math> is true for the exact (unknown) KS-DFT functional, hence it is a sensible thing to try to reproduce in a functional.

To summarise an OTRSH functional is one whereby <math>\mu</math> is chosen to minimise <math>
IP_{Calc} + E_{HOMO}</math>, hence the <math>\mu</math> value will be different for every calculation you carry out. Sometimes the IP and HOMO energy of the N-1 electron system are also used to tune <math>\mu</math>, but unfortunately thats not possible the code on this page.

==What it does(Roughly)==

==Running The Code==

===Quick Start Guide (Using the test file)===
Once you download the .zip archive, copy the files to somewhere on your hpc path [Note: make sure you dont miss the .rng_OTRSH file]. Personally i use "/home/rf614/bin" as the location. To use the gf_OTRSH function straight away you'll need to put them in $HOME/bin. "echo $HOME" while on the hpc to find where your home directory is if your not sure.

Move the test file ('H2O_SPE_6311plgdp_GP_B3LYP_builtin.com') to whereever you like in your work directory, and navigate to this folder. Then type either of the following commands (assuming you put the scripts in ~/bin):

<pre>
qsub -v in=H2O_SPE_6311plgdp_GP_B3LYP_builtin -N test ~/bin/.rng_OTRSH
gf_OTRSH H2O_SPE_6311plgdp_GP_B3LYP_builtin.com
</pre>

If you saved the files somewhere else then simply modify "~/bin/.rng_OTRSH" to whatevers appropriate for the top command. For details on how the gf command works see this wiki [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:Hunt_Research_Group/pimpQSUB link].

===Quick Start Guide - The output files===
Assuming you ran the job above you will now have a loada output files in your current directory, i.e.:

H2O_SPE_6311plgdp_GP_B3LYP_builtin_gsstate_mu_opt.*

H2O_SPE_6311plgdp_GP_B3LYP_builtin_ionstate_mu_opt.*

H2O_SPE_6311plgdp_GP_B3LYP_builtin_OTRSH.out

The '''H2O_SPE_6311plgdp_GP_B3LYP_builtin_OTRSH.out''' file is essentially a progress file for how the tuning process is going. Once the calculation is over its main purpose is really to keep track of the options you used to create the OTRSH functional. Its a comma delimited file, and the headings should be pretty self-explanatory (at least they will be if you read the above section on "What is an OTRSH")

The '''H2O_SPE_6311plgdp_GP_B3LYP_builtin_gsstate_mu_opt'''* files are for single energy calculations using the OTRSH functional tuned for this particular system. If your goal was to calculate ground state properties with an OTRSH then these files might be all you need. If you want to do something other than a single-point energy calculation, then just add the relevant keywords (e.g TD) to the H2O_SPE_6311plgdp_GP_B3LYP_builtin_gsstate_mu_opt.com file - this file contains all the keywords needed to define the OTRSH functional.

The '''H2O_SPE_6311plgdp_GP_B3LYP_builtin_ionstate_mu_opt'''* files are the single point energy calculations (using the OTRSH) for the ionised system. You probably have no need for these.

You will also have a new folder named '''temp_files_H2O_SPE_6311plgdp_GP_B3LYP_builtin'''. At every tested value for <math>\mu</math>, the .log files for the ground and ionised state are written to this folder. The purpose of this folder is to check nothings going wrong during the calculation. If something stops the code extracting scf energies for each <math>\mu</math> value, then it might just loop infinitely (or at least for a loooooong time). The code '''SHOULD''' catch these sorts of errors and kill the job, but if it runs a long time without any change in the *.out file then make sure the *.log files in this folder are ok. The secondary purpose of this folder is therefore to tell you WHY your job got killed, if it got killed (an error message should be given in the .out file as well but....)

===Quick Note on Creating Input Files===

If you want to run the code, with default options, on your own systems then theres not much to say. Simply prepare a normal '''SINGLE POINT ENERGY''' *.com file for your system, but remove the functional from the input line. Also dont use the "#p" command for excess printing - this will break the OTRSH code (sorry, the code essentially uses the # for a search + replace to insert a functional at each step of the tuning process). For example, this input line is fine(and used in the example file):
<pre>
# 6-311+g(d,p) geom=connectivity int=ultrafine scf=conver=9
</pre>

Make sure you use a '''single point energy calculation input file''', using a keyword like "freq" wont throw any errors, but you'll be waiting a really long time for your job to finish.

===Understanding (And Modifying) The Default Options===

The easily changeable variables in this code are stored in the .rng_OTRSH file. They are (in order of "likelyhood of needing altering"):

<pre>
es_mult=2
alpha_val=0
beta_val=1
mu_lower=0.1
mu_upper=0.2
max_step=0.03
min_step=0.001
P5_val=0.81
xc_funct="CAM-B3LYP"
</pre>

Its likely you'll only ever need/want to change the '''es_mult''' here. This is the multiplicity of the ionised state, and is obviously equal to 2 if you have a closed shell system. If you use this code on an open shell system you need to set es_mult to the appropriate value for the ionised state. Personally i create seperate .rng files for each value of multiplicity i use, to save me editing the .rng_OTRSH file too often.

'''alpha_val''' corresponds to the fraction HF exchange at short range. Its plausible that raising this to say alpha_val=0.2 might improve your results in some cases, so it might be worth trying the functional with two different alpha values if your trying to match an experimental spectrum or have something else you can validate with. I personally set it to zero since i think its the least arbitrary value it can be.

'''beta_val''' - as stated above "alpha_val + beta_val" corresponds to the fraction HF exchange at long range. So if you '''change alpha_val, you need to change beta_val too''', otherwise you may end up with >100% HF exchange at long range. I'd advise you keep alpha_val + beta_val =1.0, since that should give you the correct long-range potential.

'''mu_lower''' and '''mu_upper''' control the initial range of <math>\mu</math> values to try out. If you have a good idea what your final <math>\mu</math> will be, then settings mu_lower and mu_upper to nearby values should speed up the optimisation process. However, there is '''NO NEED''' for the final mu_val to be between these initial search values, it will just take longer if its far away from this range.

'''max_step''' and '''min_step''' correspond to the maximum and minimum change in <math>\mu</math> between optimisation steps. Note that i think the minimum possible min_step=0.0001, so dont set it lower than that or it could cause serious problems (it'll lead to a gradient of zero between two mu values, i have no idea how the optimiser will react to that but it wont be good). Theres probably '''NEVER''' a need to mess with these values. If the optimiser isnt efficient enough, changing these values wont help much. Writing a better optimiser (the current ones pretty terrible by the way,but it works so.....) would probably be easier than working out the best values for these numbers.

'''P5_val''' corresponds to a parameter in the DFT functional controlling the amount of non-local correlation to use. Its the same P5 parameter that appears on the gaussian 09/16 guide to "user-defined DFT functionals". Just leave it alone unless you REALLY know what your doing.

'''CAM-B3LYP''' corresponds to the underlying functional to use. The functional this code generates is essentially CAM-B3LYP with modified <math>\alpha</math>,<math>\beta</math> and <math>\mu</math> parameters. The underlying DFT exchange/ DFT correlation functionals are the same as those used in CAM-B3LYP. I've never tried modifying this, it would be a massive pain. If you want to try then i suggest you use a range-separated functional as your base and start by passing the relevant arguments to the "init_OTRSH.py" functional, and seeing if you can replicate the original functional (e.g if you want to use wB97XD as your base functional, try and reproduce reslts of that functional by passing relevant options to init_OTRSH.py.

Finally, the convergence criteria for the OTRSH (the IP_HOMO - E_HOMO value) is set to 0.01 eV by default, and controlled by the IP_conver_thres variable in optimiser.py. I guess its possible you may want to relax that criteria in certain circumstances (obviously it would speed up the optimisation and lower the accuracy).

==Further Reading/Links to relevant Papers==

Mod:Hunt Research Group/OTRSH Funct

2017-06-03T15:21:26Z

Rf614: /* Quick Start Guide - The output files */

Mod:Hunt Research Group/OTRSH Funct

2017-06-03T15:14:39Z

Rf614: /* Running The Code */

Mod:Hunt Research Group/OTRSH Funct

2017-06-03T14:43:35Z

Rf614: /* Links to files */

File:OTRSH hpc ver June3rd 2017.zip

2017-06-03T14:43:02Z

Rf614:

Mod:Hunt Research Group/OTRSH Funct

2017-06-03T12:06:20Z

Rf614:

==Links to files==

==What is an OTRSH Functional==
It is an optimally tuned range separated hybrid functional (OTRSH). This is a class of functionals that are useful for ccalculating electronic properties such as UV-Vis spectra (especially if charge-transfer excitations are important) or photoelectron spectra. Before talking about OTRSH functionals, it is useful to first talk about how standard range separated hybrid functionals work, and why they were created.

===The Problem===
Standard DFT has problems describing long range interactions, due to the form of the exchange energy in standard functionals. Actual (exact) Exchange energy scales with distance as <math>\frac{1}{r_{12}}</math> where <math>r_{12}</math> is the distance between two electrons. However, standard pure DFT exchange functionals depend directly on the electron density, which decays exponentially with distance from an atomic center (i.e. DFT exchange decays faster than actual exchange). This means that standard DFT functionals are poor at describing long-range interactions (charge transfer excitations in DFT is an example). Range seperated hybrid functionals try to fix this by using Hartree-Fock exchange energy (which is exact) at large distances (long-range) while keeping DFT exchange energy at short range.

===One Solution - Normal Range Separated Hybrids===
A range separated hydrbrid functional divides the exchange energy into two terms - short range exchange (<math> E_{X}^{SR}</math>) and long range exchange (<math> E_{X}^{LR} </math>). The exchange energy at any one distance is a weighted combination of short-range and long range exchange, and can essentially be described with three variables; <math>\alpha</math>, <math>\beta</math> and <math>\mu</math> (See picture below).

[[File:Frac_HF_Exchange_mu.png|200px|center|Description of the Exchange energy at varying distances in a range-separated hybrid functtional]]

The equations for total, short range (DFT) and long-range (HF) exchange are:

<math>
E_{X}^{Total} = E_{X}^{SR} + E_{X}^{LR}
</math>

<math>
E_{X}^{SR} = [1-(\alpha + \beta{}.\text{erf}(\mu r_{12})] E_{X}^{DFT}(r_{12})
</math>

<math>
E_{X}^{LR} = [\alpha + \beta{}.\text{erf}(\mu{}r_{12})] E_{X}^{HF}(r_{12})
</math>

where erf is the error function. <math>\alpha</math> represents the amount of HF exchange at short range. For example, <math>\alpha=0.2</math> would mean 20% HF exchange was used at <math>r_{12}=0</math>. <math>\alpha + \beta </math> represents the amount of HF exchange at long range (i.e as <math>r_{12}</math> approaches infinity). A value of <math>\alpha + \beta =1</math> means that 100% HF exchange is employed at long range, which should give the correct long-range behavior for the exchange energy. <math>\mu</math> is the range seperation parameter, its value determines how quickly we change from short-range to long-range exchange expresssions. The higher <math>\mu</math> is, the faster (shorter distances) we start using the long-range exchange energy expression.

A problem with range separated hybrids is in picking the correct value for <math>\mu</math> (i.e when should we stop using DFT exchange?). This is often determined empirically by varying <math>\mu</math> for a test data set, and seeing which <math>\mu</math> value leads to a functional that best reproduces an experimental property (e.g. atomic ionisation energies). This has the same problem as any kind of empirical fitting - its not clear how well the functional will work outside the test set of molecules/properties. Also this empirical tuning of parameters can lead to unphysical values. For example, CAM-B3LYP uses <math>\alpha + \beta =0.65</math>, meaning it definitely has the wrong long-range behavior.

===OTRSH Functionals===

OTRSH functionals provide a non-empirical method for choosing <math>\mu</math>; <math>\mu</math> is "tuned" on a per-calculation basis. The <math>\mu</math> value is chosen so that the calculated ionisation energy matches the negative of the HOMO energy, i.e.:

<math>
IP_{Calc} = -E_{HOMO}
</math>

<math>
IP_{Calc} = E_{N-1} - E_{N}
</math>

Where <math>E_{N-1}</math> and <math>E_{N}</math> represent the energy of the N-1 and N electron system (both in the N electron system geometry). Note that all values in the equations above can be obtained from two single-point energy calculations. Also note that <math>IP_{Calc} = -E_{HOMO}</math> is true for the exact (unknown) KS-DFT functional, hence it is a sensible thing to try to reproduce in a functional.

To summarise an OTRSH functional is one whereby <math>\mu</math> is chosen to minimise <math>
IP_{Calc} + E_{HOMO}</math>, hence the <math>\mu</math> value will be different for every calculation you carry out. Sometimes the IP and HOMO energy of the N-1 electron system are also used to tune <math>\mu</math>, but unfortunately thats not possible the code on this page.

==What it does(Roughly)==

==Running The Code==

==Further Reading/Links to relevant Papers==

Mod:Hunt Research Group/OTRSH Funct

2017-06-03T12:05:43Z

Rf614: /* What is an OTRSH Functional */

Mod:Hunt Research Group

2017-06-02T17:31:23Z

Rf614: /* Codes to Help Analysis */

==Hunt Group Wiki==

Back to the main [http://www.ch.ic.ac.uk/hunt web-page]
===HPC Resources===
#Computing resources available in the chemistry department [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/computing_resources link]
#HPC servers and run scripts [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/hpc link]
#Setting up a connection to HPC if you have a PC [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/hpc_connections link]
#How to fix Windows files under UNIX [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/Windowsfiles link]
#How to make ssh more comfortable [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/pimpSSH link]
#How to make qsub more comfortable (gfunc) [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/pimpQSUB link]
#How to set up a SSH keypair [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/SSHkeyfile link]
#How to use gaussview directly on the HPC [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/gview link]
#How to comfortably search through old BASH commands [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/searchbash link]
#How to connect to HPC directory on desktop for file transfers - MacFusion [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/hpc_Directory_on_desktop link]
#How to set up cx2 [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/cx2 link]

===Visualisation===
*'''density based visualisation'''
#download [http://aim.tkgristmill.com AIMALL]

*'''ESPs'''
#Instructions for visualizing electrostatic potentials (Gaussview)[https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/electrostatic_potentials link]
#Electrostatic Potentials II (Molden) [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/electrostatic_potentials_2 link]
*'''VMD: a molecular dynamics visualisation package'''
#Download VMD [//wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/download_vmd link]
#Quick reminder [https://www.ch.ic.ac.uk/wiki/index.php/Talk:Mod:Hunt_Research_Group/VMDReminder link]
#Tricks and tips [https://www.ch.ic.ac.uk/wiki/index.php/Talk:Mod:Hunt_Research_Group/VMDTips link]
#Changing the graphical representation of your structures [https://www.ch.ic.ac.uk/wiki/index.php/Talk:Mod:Hunt_Research_Group/vmd link]
#Basic visualisation of a trajectory [https://www.ch.ic.ac.uk/wiki/index.php/Talk:Mod:Hunt_Research_Group/VisualisingyourSimulation link]
#How to turn a Gaussian optimization into a VMD movie [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/VMDmovie link]
#Using scripts in VMD [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/VmdScripts link]
#Dealing with periodic boundaries and bonding (under construction) [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/VmdScriptsPeriodic link]
#Dealing with bonding (under construction) [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/VmdBonding link]
#Overlapping two structures [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/VmdVisual link]
*'''JMol'''
#Visualising MOs using Jmol [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:basic_jmol_instructions link]
#Surfaces (Solvent-Accessible and Connolly) in Jmol [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/jmolsurfaces link]
*'''EMO Code'''
#How to use Ling's emo plot code[https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/emoplot link]
#How to plot EMOs [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/emo link]
*'''PyGauss'''
#Python API for analysis of Gaussian compuations [https://pygauss.readthedocs.org - Documentation]
*'''MD Post processing'''
#Code to Recentre DL_PLOY HISTORY file [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/recentre_xyz.py link]
#Link to the code to convert the DL_POLY HISTORY file to the multi-frame XYZ file[https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/his2xyz.py link]

*'''SDFs'''
#How to generate SDFs [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/sdfs_generate link]

===Gaussian General===
#We are starting a database of common errors encountered when running Gaussian jobs [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/gaussian_errors link]
# Here is an already existing database of common errors [https://www.ace-net.ca/wiki/Gaussian_Error_Messages link]
# [http://www.ch.ic.ac.uk/hunt/g03_man/index.htm G03 Manual]
#How to run NBO5.9 on the HPC [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/NBO5.9 link]
#How to include dispersion [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/dispersion link]
#Basic ONIOM (Mechanical Embedding) [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/basiconiom link]
#M0n and DFT-D [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/DFTD link]
#IL ONIOM clusters [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/oniomclusers link]
#Molecular volume calculations [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:Hunt_Research_Group/molecular_volume link]
#problems with scf convergence [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/scf_convergence link]
#Using a z-matrix [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/z-matrix link]
#generating natural transition orbitals [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/nto link]
#Using solvent models [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/solvent link]
#Using SMD on ILs [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:Hunt_Research_Group:_Using_SMD_on_ILs link]
#computing excited state polarisabilities [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:Hunt_Research_Group:_ES_alpha link]
#computing deuterated and/or anharmonic spectra [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:Hunt_Research_Group:_Danharm link]
#manipulating checkpoint files [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:Hunt_Research_Group:usingchkfiles link]
#AimAll with pseudo potentials [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:Hunt_Research_Group:aim_pseudopotentials link]
#Script to pull thermodynamic data and low frequencies from log files [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:Hunt_Research_Group:freq_script link]
#General procedure for locating transition state structures [[link]]

===ADF General===
#Submission script [http://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/ADF_sricpt link]

===Codes to Help Analysis===
# Extract E2 Values (From NBO Calculations) [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/NBO_Matlab_Code link]
# Calculate pDoS/XP spectra code (under construction) [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/Calc_XPS_Code link]
# Codes to extract frequency data from gaussian .log files and generate vibrational spectra [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:Hunt_Research_Group:frequency_spectrum_script link]
# Optimally Tuned Range Seperated Hybrid Functionals [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:Hunt_Research_Group/OTRSH_Funct link]

===Setup and Running Ab-Initio MD Simulations===
#CPMD: Car-Parrinello Molecular Dynamics [https://www.ch.ic.ac.uk/wiki/index.php/Talk:Mod:Hunt_Research_Group/cpmd link]
#How to run CPMD to study aqueous solutions [https://www.ch.ic.ac.uk/wiki/index.php/Talk:Mod:Hunt_Research_Group/cpmd_water link]
#How to run CP2K [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/cp2k_how link]
#[bmim]Cl using CPMD [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/bmimCl_cpmd link]
#[bmim]Cl using CP2K [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/bmimCl_cp2k link]
#mman using CPMD and Gaussian [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/mman link]
#[emim]SCN using CP2K[https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/emimscn link]
#CP2K Donts [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/cp2k link]

===Setup and Running Classical MD Simulations===
#DLPOLY a MD simulation package, Installation on an IMac (old needs to be updated) [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/dlpoly_install link]
#DL_POLY FAQs [http://www.stfc.ac.uk/cse/DL_POLY/ccp1gui/38621.aspx] from DL_POLY webpage.
#Installing Packmol
#Getting started: generating a solvated structure and "relaxing" it [https://www.ch.ic.ac.uk/wiki/index.php/Talk:Mod:Hunt_Research_Group/Starting_MD link]
#Equilibration and production simulations [https://www.ch.ic.ac.uk/wiki/index.php/Talk:Mod:Hunt_Research_Group/EquilibrationandProduction link]
#How to equilibrate an MD run[https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/equilibration link]
#Getting the Force Field [https://www.ch.ic.ac.uk/wiki/index.php/Talk:Mod:Hunt_Research_Group/Wheretostart link]
#Choosing an Ensemble [https://www.ch.ic.ac.uk/wiki/index.php/Talk:Mod:Hunt_Research_Group/Ensembles link]
#Molten Salt Simulations [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:Hunt_Research_Group/MoltenSaltSimulation link]
#Common Errors [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:Hunt_Research_Group/CommonErrors link]
#Voids in ILs[https://www.ch.ic.ac.uk/wiki/index.php/Talk:Mod:Hunt_Research_Group/voids link]
#Equilibration of [bmim][BF4] and [bmim][NO3][https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/BmimBF4_equilibration link]
#Summary of discussions with Ruth[https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/Aug09QtoRuth link]

===Running QM/MM Simulations in ChemShell===
#ChemShell official website which contains the manual and a tutorial [http://www.stfc.ac.uk/CSE/randd/ccg/36254.aspx link]
#Introduction to ChemShell - Copper in water [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/ChemShell_Introduction link]
#Defining the system: Cu2+ and its first 2 solvation shells [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/ChemShell_System_Aqeuous_Cu(II) link]
#Defining the force field parameters [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/ChemShell_Force_Field_Parameters_Aqueous_Cu(II) link]
#Single point QM/MM energy calculation [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/QMMM_SP_Aqeuous_Cu(II) link]
#QM/MM Optimisation [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/QMMM_OPT_Aqeuous_Cu(II) link]
#QM/MM Molecular Dynamics [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/QMMM_MD_Aqeuous_Cu(II) link]
#Using MolCluster [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/MolCluster link]
#Running ChemShell [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/ChemShell link]

===Research Notes===
#Cl- in water [https://www.ch.ic.ac.uk/wiki/index.php/Talk:Mod:Hunt_Research_Group/wannier_centre link]
#The use of Legendre time correlation functions to study reorientational dynamics in liquids[https://www.ch.ic.ac.uk/wiki/index.php/Talk:Mod:Hunt_Research_Group/legendre link]
#Functional for ILs using CPMD [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/IL_cpmd_functional link]
#Solving the angular part of the Schrödinger equation for a hydrogen atom [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/angular_schrodinger link] (notes by Vincent)
#Systematic conformational scan for ion-pair dimers [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/ion_pair_scan link]

===Installing and using other packages===
#How to install POLYRATE [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/polyrate link]
#How to install Geomview [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/geomview link]
#XMGRACE, gfortran, c compilers for Lion [http://hpc.sourceforge.net/]

===Admin Stuff===
#Not used to writing a wiki, make your test runs [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/testing on this page]
#How to set-up new macs [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/mac_setup link]
#How to set-up remote desktop [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/mac_remote link]
#[https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/calendar Calendar]
#How to switch the printer HP CP3525dn duplex on and off [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/printing link]

Mod:Hunt Research Group/OTRSH Funct

2017-06-02T15:21:11Z

Rf614: /* What is an OTRSH Functional */

File:Frac HF Exchange mu.png

2017-06-02T15:11:56Z

Rf614:

Mod:Hunt Research Group/OTRSH Funct

2017-06-02T15:10:02Z

Rf614: /* What is an OTRSH Functional */

Mod:Hunt Research Group/OTRSH Funct

2017-06-02T14:53:39Z

Rf614: /* What is an OTRSH Functional */

==Links to files==

==What is an OTRSH Functional==
It is an optimally tuned range separated hybrid functional.

<math>
E_{X}^{SR} = [1-(\alpha + \beta{}.\text{erf}(\mu r_{12})] E_{X}^{DFT}(r_{12})
</math>

<math>
E_{X}^{LR} = [\alpha + \beta{}.\text{erf}(\mu{}r_{12})] E_{X}^{HF}(r_{12})
</math>

==What it does(Roughly)==

==Running The Code==

Mod:Hunt Research Group/OTRSH Funct

2017-06-02T14:47:08Z

Rf614:

==Links to files==

==What is an OTRSH Functional==
It is an optimally tuned range separated hybrid functional.

==What it does(Roughly)==

==Running The Code==

Mod:Hunt Research Group/OTRSH Funct

2017-06-02T14:44:32Z

Rf614: Created page with "text"

text

Mod:Hunt Research Group

2017-06-02T14:40:18Z

Rf614: /* Codes to Help Analysis */

==Hunt Group Wiki==

Back to the main [http://www.ch.ic.ac.uk/hunt web-page]
===HPC Resources===
#Computing resources available in the chemistry department [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/computing_resources link]
#HPC servers and run scripts [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/hpc link]
#Setting up a connection to HPC if you have a PC [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/hpc_connections link]
#How to fix Windows files under UNIX [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/Windowsfiles link]
#How to make ssh more comfortable [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/pimpSSH link]
#How to make qsub more comfortable (gfunc) [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/pimpQSUB link]
#How to set up a SSH keypair [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/SSHkeyfile link]
#How to use gaussview directly on the HPC [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/gview link]
#How to comfortably search through old BASH commands [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/searchbash link]
#How to connect to HPC directory on desktop for file transfers - MacFusion [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/hpc_Directory_on_desktop link]
#How to set up cx2 [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/cx2 link]

===Visualisation===
*'''density based visualisation'''
#download [http://aim.tkgristmill.com AIMALL]

*'''ESPs'''
#Instructions for visualizing electrostatic potentials (Gaussview)[https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/electrostatic_potentials link]
#Electrostatic Potentials II (Molden) [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/electrostatic_potentials_2 link]
*'''VMD: a molecular dynamics visualisation package'''
#Download VMD [//wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/download_vmd link]
#Quick reminder [https://www.ch.ic.ac.uk/wiki/index.php/Talk:Mod:Hunt_Research_Group/VMDReminder link]
#Tricks and tips [https://www.ch.ic.ac.uk/wiki/index.php/Talk:Mod:Hunt_Research_Group/VMDTips link]
#Changing the graphical representation of your structures [https://www.ch.ic.ac.uk/wiki/index.php/Talk:Mod:Hunt_Research_Group/vmd link]
#Basic visualisation of a trajectory [https://www.ch.ic.ac.uk/wiki/index.php/Talk:Mod:Hunt_Research_Group/VisualisingyourSimulation link]
#How to turn a Gaussian optimization into a VMD movie [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/VMDmovie link]
#Using scripts in VMD [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/VmdScripts link]
#Dealing with periodic boundaries and bonding (under construction) [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/VmdScriptsPeriodic link]
#Dealing with bonding (under construction) [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/VmdBonding link]
#Overlapping two structures [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/VmdVisual link]
*'''JMol'''
#Visualising MOs using Jmol [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:basic_jmol_instructions link]
#Surfaces (Solvent-Accessible and Connolly) in Jmol [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/jmolsurfaces link]
*'''EMO Code'''
#How to use Ling's emo plot code[https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/emoplot link]
#How to plot EMOs [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/emo link]
*'''PyGauss'''
#Python API for analysis of Gaussian compuations [https://pygauss.readthedocs.org - Documentation]
*'''MD Post processing'''
#Code to Recentre DL_PLOY HISTORY file [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/recentre_xyz.py link]
#Link to the code to convert the DL_POLY HISTORY file to the multi-frame XYZ file[https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/his2xyz.py link]

*'''SDFs'''
#How to generate SDFs [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/sdfs_generate link]

===Gaussian General===
#We are starting a database of common errors encountered when running Gaussian jobs [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/gaussian_errors link]
# Here is an already existing database of common errors [https://www.ace-net.ca/wiki/Gaussian_Error_Messages link]
# [http://www.ch.ic.ac.uk/hunt/g03_man/index.htm G03 Manual]
#How to run NBO5.9 on the HPC [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/NBO5.9 link]
#How to include dispersion [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/dispersion link]
#Basic ONIOM (Mechanical Embedding) [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/basiconiom link]
#M0n and DFT-D [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/DFTD link]
#IL ONIOM clusters [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/oniomclusers link]
#Molecular volume calculations [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:Hunt_Research_Group/molecular_volume link]
#problems with scf convergence [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/scf_convergence link]
#Using a z-matrix [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/z-matrix link]
#generating natural transition orbitals [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/nto link]
#Using solvent models [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/solvent link]
#Using SMD on ILs [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:Hunt_Research_Group:_Using_SMD_on_ILs link]
#computing excited state polarisabilities [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:Hunt_Research_Group:_ES_alpha link]
#computing deuterated and/or anharmonic spectra [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:Hunt_Research_Group:_Danharm link]
#manipulating checkpoint files [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:Hunt_Research_Group:usingchkfiles link]
#AimAll with pseudo potentials [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:Hunt_Research_Group:aim_pseudopotentials link]
#Script to pull thermodynamic data and low frequencies from log files [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:Hunt_Research_Group:freq_script link]
#General procedure for locating transition state structures [[link]]

===ADF General===
#Submission script [http://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/ADF_sricpt link]

===Codes to Help Analysis===
# Extract E2 Values (From NBO Calculations) [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/NBO_Matlab_Code link]
# Calculate pDoS/XP spectra code (under construction) [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/Calc_XPS_Code link]
# Codes to extract frequency data from gaussian .log files and generate vibrational spectra [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:Hunt_Research_Group:frequency_spectrum_script link]
# Optimally Tuned Range Seperated Hybrid Functionals [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/OTRSH_Funct link]

===Setup and Running Ab-Initio MD Simulations===
#CPMD: Car-Parrinello Molecular Dynamics [https://www.ch.ic.ac.uk/wiki/index.php/Talk:Mod:Hunt_Research_Group/cpmd link]
#How to run CPMD to study aqueous solutions [https://www.ch.ic.ac.uk/wiki/index.php/Talk:Mod:Hunt_Research_Group/cpmd_water link]
#How to run CP2K [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/cp2k_how link]
#[bmim]Cl using CPMD [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/bmimCl_cpmd link]
#[bmim]Cl using CP2K [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/bmimCl_cp2k link]
#mman using CPMD and Gaussian [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/mman link]
#[emim]SCN using CP2K[https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/emimscn link]
#CP2K Donts [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/cp2k link]

===Setup and Running Classical MD Simulations===
#DLPOLY a MD simulation package, Installation on an IMac (old needs to be updated) [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/dlpoly_install link]
#DL_POLY FAQs [http://www.stfc.ac.uk/cse/DL_POLY/ccp1gui/38621.aspx] from DL_POLY webpage.
#Installing Packmol
#Getting started: generating a solvated structure and "relaxing" it [https://www.ch.ic.ac.uk/wiki/index.php/Talk:Mod:Hunt_Research_Group/Starting_MD link]
#Equilibration and production simulations [https://www.ch.ic.ac.uk/wiki/index.php/Talk:Mod:Hunt_Research_Group/EquilibrationandProduction link]
#How to equilibrate an MD run[https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/equilibration link]
#Getting the Force Field [https://www.ch.ic.ac.uk/wiki/index.php/Talk:Mod:Hunt_Research_Group/Wheretostart link]
#Choosing an Ensemble [https://www.ch.ic.ac.uk/wiki/index.php/Talk:Mod:Hunt_Research_Group/Ensembles link]
#Molten Salt Simulations [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:Hunt_Research_Group/MoltenSaltSimulation link]
#Common Errors [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Mod:Hunt_Research_Group/CommonErrors link]
#Voids in ILs[https://www.ch.ic.ac.uk/wiki/index.php/Talk:Mod:Hunt_Research_Group/voids link]
#Equilibration of [bmim][BF4] and [bmim][NO3][https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/BmimBF4_equilibration link]
#Summary of discussions with Ruth[https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/Aug09QtoRuth link]

===Running QM/MM Simulations in ChemShell===
#ChemShell official website which contains the manual and a tutorial [http://www.stfc.ac.uk/CSE/randd/ccg/36254.aspx link]
#Introduction to ChemShell - Copper in water [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/ChemShell_Introduction link]
#Defining the system: Cu2+ and its first 2 solvation shells [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/ChemShell_System_Aqeuous_Cu(II) link]
#Defining the force field parameters [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/ChemShell_Force_Field_Parameters_Aqueous_Cu(II) link]
#Single point QM/MM energy calculation [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/QMMM_SP_Aqeuous_Cu(II) link]
#QM/MM Optimisation [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/QMMM_OPT_Aqeuous_Cu(II) link]
#QM/MM Molecular Dynamics [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/QMMM_MD_Aqeuous_Cu(II) link]
#Using MolCluster [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/MolCluster link]
#Running ChemShell [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/ChemShell link]

===Research Notes===
#Cl- in water [https://www.ch.ic.ac.uk/wiki/index.php/Talk:Mod:Hunt_Research_Group/wannier_centre link]
#The use of Legendre time correlation functions to study reorientational dynamics in liquids[https://www.ch.ic.ac.uk/wiki/index.php/Talk:Mod:Hunt_Research_Group/legendre link]
#Functional for ILs using CPMD [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/IL_cpmd_functional link]
#Solving the angular part of the Schrödinger equation for a hydrogen atom [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/angular_schrodinger link] (notes by Vincent)
#Systematic conformational scan for ion-pair dimers [https://wiki.ch.ic.ac.uk/wiki/index.php?title=Talk:Mod:Hunt_Research_Group/ion_pair_scan link]

===Installing and using other packages===
#How to install POLYRATE [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/polyrate link]
#How to install Geomview [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/geomview link]
#XMGRACE, gfortran, c compilers for Lion [http://hpc.sourceforge.net/]

===Admin Stuff===
#Not used to writing a wiki, make your test runs [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/testing on this page]
#How to set-up new macs [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/mac_setup link]
#How to set-up remote desktop [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/mac_remote link]
#[https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/calendar Calendar]
#How to switch the printer HP CP3525dn duplex on and off [https://www.ch.ic.ac.uk/wiki/index.php/Mod:Hunt_Research_Group/printing link]