Difference between revisions of "Gaussian"

From Cncz
Jump to navigation Jump to search
(Initial draft - better than nothing.)
(added SOP from Jana's mail)
 
(24 intermediate revisions by the same user not shown)
Line 1: Line 1:
 +
 
[nl]
 
[nl]
 
bekijk Engels version. Mijn nederlands is noch niet goed.
 
bekijk Engels version. Mijn nederlands is noch niet goed.
Line 7: Line 8:
 
* Being member of the qm, qmorgchem or qmteaching and gaussian groups (ask Jan for access).
 
* Being member of the qm, qmorgchem or qmteaching and gaussian groups (ask Jan for access).
 
* [[Vpn]] (when working from home).
 
* [[Vpn]] (when working from home).
* SSH client (linux usually has [https://www.openssh.com/ OpenSSH], for instance [https://www.putty.org/ Putty] for Windows - [https://www.youtube.com/watch?v=EBfj6o__p-s guide]).
+
* SSH client (linux usually has [https://www.openssh.com/ OpenSSH], windows - for instance [https://mobaxterm.mobatek.net/ MobaXterm] or [https://www.putty.org/ Putty]).
* X11 client (linux usually has [https://www.openssh.com/ OpenSSH], for instance [https://sourceforge.net/projects/xming/ xming] for Windows, [https://www.youtube.com/watch?v=EBfj6o__p-s guide]).
+
* X11 client (linux usually has [https://www.openssh.com/ OpenSSH], windows - for instance [https://mobaxterm.mobatek.net/ MobaXterm] or [https://sourceforge.net/projects/xming/ xming]).
 +
* (optional) [https://winscp.net/eng/download.php WinSCP]
  
==Setting-up the environment==
+
==Video guides==
you must be within university network or connected through VPN.
+
[https://youtu.be/krr8xk7LCCA GUIDE] how to setup environment for Gaussian calculations in windows.
  
====Tedious, but explained====
+
==Setting-up the environment (to be done once)==
To work effectively with the cluster its wise to edit your '''~/.profile''' file so you can easily execute Gaussian calculations and do few other tweaks. In following section I will explain both what to do and why to do it. Lest start with a '''~/.profile'''. '''~/.profile''' is a file which sets up working environment when connecting for an interactive session. The recommended version looks like this:
+
To submit calculations easily you need to set-up the environment first.
  
export PATH=$PATH:/vol/qmorgchem/scripts
+
===Automated script===
  module load Gaussian-16
+
You must be within university network or connected through VPN.
umask 027
+
connect through SSH to lilo.science.ru.nl (in putty that is the hostname). Your username is your science login. After logging-in, execute '''/vol/qmorgchem/scripts/gaussify''' script (cnorris is the username in this example):
 +
  cnorris@lilo7:~$ /vol/qmorgchem/scripts/gaussify
  
* '''[https://linux.die.net/man/1/bash export] PATH=...''' - PATH is a variable where folders which contains executable files are linked - by this line you ensure that /vol/qmorgchem/scripts is added to PATH so you can execute scripts stored there - for instance a script for submitting a calculation.
+
If you're more experienced user, you can setup the things on your own.
* '''[https://linux.die.net/man/1/module module]''' - this line ensures that you modify your environment to be able to run Gaussian jobs properly.
 
* '''[https://linux.die.net/man/2/umask umask 027]''' gives permissions to newly created files - after the profile is active, this umask ensures that you have all the rights to work with your files, people in the same groups can view the files, but cannot edit them and people from other groups does not have permission to inspect the files. More about umask is on [https://en.wikipedia.org/wiki/Umask wiki], especially octal codes gets thing a bit more clear.
 
  
After you change the '''~/.profile''' file accordingly and you log-out and log-in you should be able to get the calculations running.
+
===Explanation of the automated script===
 +
The bash script sets-up few things on your behalf. What it does is:
  
To get things more smooth it is also recommended to create shortcuts in your home directory.
+
====~/.profile file editation====
 +
to see your ~/.profile content, execute:
 +
cnorris@lilo7:~$ cat ~/.profile
  
* Log-out and log-in if you've not done so after modifying your profile.
+
you should see something like this:
* Then execute following commands:
+
export PATH=$PATH:/vol/qmorgchem/scripts
 +
module load Gaussian-16
 +
umask 027
  
cnorris@lilo7:~$ mkdir -p /vol/qmorgchem/"$USER"
+
This file stores some of your server settings, the script:
 +
* Adds ('''[https://linux.die.net/man/1/bash export]''' PATH=... line) /vol/qmorgchem/scripts in your $PATH variable - by doing this you can easily execute scripts located in /vol/qmorgchem/scripts folder such as the important '''G16''' script.
 +
* Automatically loads Gaussian-16 module ('''[https://linux.die.net/man/1/module module]''' load... line) which enables you to use commands like '''cubegen''' and similar.
 +
* Changes the '''[https://linux.die.net/man/2/umask umask]''' to 027 - that means that files newly created from the console can be readed by people from qmorgchem, but '''cannot be changed''' by them. If you want to keep the files changeable by other people from qmorgchem, then remove this line from your ~/.profile file.
  
if you're a memeber of the qmorgchem group:
+
====Folders and shortcuts creation====
cnorris@lilo7:~$ mkdir -p /vol/qmorgchem-old/"$USER"
+
Script creates your folder in qmorgchem disk. If you are member of the qmorgchem group it creates your folder also on the qmorgchem-old disk.
cnorris@lilo7:~$ chown $USER:qmorgchem /vol/qmorgchem/"$USER"
+
It creates shortcuts (named symlinks in linux) to those folders within your $HOME directory by '''ln -s''' commands.
cnorris@lilo7:~$ chown $USER:qmorgchem /vol/qmorgchem-old/"$USER"
 
  
By this you created your own directory on the qmorgchem and optionally on the qmorgchem-old disks. By optional chown you changed the group to which this folder belongs.
+
To see those folder changes, execute:
 +
cnorris@lilo7:/vol$ ls -lh ~
  
'''USER''' is just another environment variable, to get what it does run following command:
+
You should then see something like this:
  cnorris@lilo7:~$ echo $USER
+
total 24K
 +
drwxr-s--- 2 cnorris cnorris 4.0K Apr  1 13:37 '$RECYCLE.BIN'
 +
-rw-r----- 1 cnorris cnorris  402 Apr  1 13:37  desktop.ini
 +
lrwxrwxrwx 1 cnorris cnorris  28 Apr  1 13:37  old -> /vol/qmorgchem-old/cnorris/
 +
lrwxrwxrwx 1 cnorris cnorris  15 Apr  1 13:37  others -> /vol/qmorgchem/
 +
  lrwxrwxrwx 1 cnorris cnorris   24 Apr  1 13:37  running -> /vol/qmorgchem/cnorris/
  
hopefully this clears the things a bit.
+
ls is a command which lists directory content, ~ is a shorthand for your $HOME directory (/home/cnorris in case of cnorris user), -lh are arguments: l stands for "long" and h stands for "human-readable." You clearly see that shortcut named "old" points to your folder at qmorgchem-old disk and "running" points to your folder at qmorgchem disk. Please use qmorgchem-old for inactive projects and do not store '''.chk''' files there.
  
Now its time to create shortcuts - they're called symlinks in linux.
+
==Checking calculations, submitting jobs, checking queues==
  
cnorris@lilo7:~$ ln -s /vol/qmorgchem/"$USER"/ ~/running
+
===Queues===
 +
Each computational job must be submitted to a queue. Currently we have two queues - '''orgfast''' and '''orglong.'''
  
and optionally (if you're part of qmorgchem):
+
* '''orgfast''' - Queue should be used for standard jobs (e.g., coptimization of geometries, harmonic frequency calculations, excited state spectra calculations). Please use '''max 4 cpus''' and '''maxi 12 GB of ram''' in this queue.
 +
* '''orglong''' - Queue for more demanding jobs (typically days or weeks of computational time). This queue has a lower priority; jobs will be running only if the given number of processors is free and not requested in the orgfast queue. Please use '''max 12 cpus''' and '''max 32 GB of ram''' in this queue.
  
  cnorris@lilo7:~$ ln -s /vol/qmorgchem-org/"$USER"/ ~/old
+
===Submitting jobs, checking queues===
 +
There are currently two queues available on the cluster: '''orgfast''' for small and quick jobs and '''orglong''' for bigger and more-demanding jobs. We are currently using [[Slurm]] as our job queuing manager.
  
ln is a command to link things together and -s parameter tells it that it should create a shortcut.
+
'''To submit a job and doing anything with the running jobs you MUST be on slurm20'''
  
====Fast & Dirty====
+
'''Basic Commands''':
Login to lilo.science.ru.nl or slurm20.science.ru.nl, then execute:
+
* '''G16''' ''$job_filename'' ''$queue'' - for instance '''G16 co2capcture.gjf orgfast''' - submits a calculation
 +
* '''squeue''' - checks queue
 +
* '''scancel $jobid''' - cancels a job
 +
* '''scontrol show jobid $jobid''' - show details of a job with job number $jobid (good for getting a full name)
  
cnorris@lilo7:~$ /vol/qmorgchem/scripts/gaussify
+
Once calculation is running a '''.log''' file will appear in the folder where is the input file.
  
'''ẂARNING''' this will change the access rights to your newly created files, if you do not like it, comment (add # at start) the line starting with '''umask''' in '''~/.profile'''.
+
==== Putty ====
 +
Connect to slurm20.science.ru.nl - details are the same as for the abovementioned for lilo.science.ru.nl. Change directory to your active directory where computations are being made (cd running - this brings you through shortcut to your folder on the qmorgchem disk).
  
logout and login after executing the command so the changes can take place.
+
To submit a job, type:
 +
cnorris@slurm20:~$ G16 $job_filename $queue
  
==Checking calculations, submitting jobs, checking queues==
+
command where $job_filename is filename of your Gaussian input and $queue is either orgfast or orglong.
  
===submitting jobs, checking queues===
+
==== WinSCP ====
There are currently two queues available on the cluster: '''orgfast''' for small and quick jobs and '''orglong''' for bigger and more-demanding jobs. We are currently using [[Slurm]] as our job queuing manager.
 
  
To submit a job, type:
+
connect through SFTP to slurm20.science.ru.nl and then open folder running and then type:
 
  cnorris@slurm20:~$ G16 $job_filename $queue
 
  cnorris@slurm20:~$ G16 $job_filename $queue
  
 
command where $job_filename is filename of your Gaussian input and $queue is either orgfast or orglong.
 
command where $job_filename is filename of your Gaussian input and $queue is either orgfast or orglong.
  
'''To submit a job and doing anything with the running jobs you MUST be on slurm20'''
+
For clarity, see the pictures below:
 
+
<gallery>
To check current queues, type:
+
Winscp1.png|opening console
cnorris@slurm20:~$ squeue
+
Winscp2.png|logging in
 
+
Winscp3.png|submitting
Once calculation is running a '''.log''' file will start to appear in the folder where is the input file.
+
</gallery>
  
 
===Checking calculations===
 
===Checking calculations===
once calculations are finished or you want to take a look, you can either use '''molden''' (its installed on the cluster) or you can connect to the disks.
+
once calculations are finished or you want to take a look, you can either use '''molden''' (its installed on the cluster - you can run it when you have xming on and you are connected on lilo) or you can connect to the disks and check the files from your windows molecule viewer.
  
 
===Connecting to disks - windows===
 
===Connecting to disks - windows===
Line 90: Line 111:
 
for qmorgchem-old, the link is:
 
for qmorgchem-old, the link is:
 
  \\qmorgchem-old-srv.science.ru.nl\qmorgchem-old\
 
  \\qmorgchem-old-srv.science.ru.nl\qmorgchem-old\
 +
 +
===Connecting to disks - Linux===
 +
create some file in your homedir - for example lilofs:
 +
mkdir ~/lilofs
 +
 +
install sshfs if you do not have it yet and then:
 +
sshfs lilo.science.ru.nl:/ ~/lilofs
 +
 +
the disks are then located in ~/lilofs/vol , I will leave creating appropriate symlinks as a homework excercise - its analogous to commands mentioned above.
 +
 +
==Standard operation procedure, tips==
 +
* If the orglong queue is full and you have quick jobs, run them at orgfast
 +
* Pay attention when setting-up memory allocation, use of excessive memory prevents other jobs from running. Memory is shared. Do not use memory that you do not need.
 +
* Do calculations in optimization steps
 +
* reading force constants (RCFC) speeds-up the initial optimization step.
 +
 +
==== optimization steps ====
 +
This section is an example of the optimization steps. Basis sets and levels of theory are case-specific and thus may vary.
 +
 +
* semiempirical (if applicable, AM1, PM6)
 +
* cheap DFT (pure DFT, fe: B97D/sto-3g, M06L/midix)
 +
* expensive DFT (hybrid, fe: B3Lyp)
 +
 +
 +
==Issues==
 +
===Open===
 +
==== Failing calculations after submission ====
 +
'''error note:''' ''galloc:  could not allocate memory.''
 +
For now there is an issue that sometimes calculation fail while submitting a job. If this happens to you, please record a job number and send it to Jan. Do not delete .error, .out and .log files of the failed job, they are important for debugging this issue.
  
 
==Disclaimer==
 
==Disclaimer==
This guide is a draft, if you have any suggestions how to improve the readability, understandability or how to make it better do not be shy and write Jan.
+
If you have any issue to report please write Jan.
 +
 
 +
This guide is a draft, if you have any suggestions how to improve the readability, understandability or any idea how to make it better, please do not be shy and write Jan.
 +
[/en]

Latest revision as of 11:44, 13 December 2021

Prerequisities

  • Science (CNCZ) account.
  • Being member of the qm, qmorgchem or qmteaching and gaussian groups (ask Jan for access).
  • Vpn (when working from home).
  • SSH client (linux usually has OpenSSH, windows - for instance MobaXterm or Putty).
  • X11 client (linux usually has OpenSSH, windows - for instance MobaXterm or xming).
  • (optional) WinSCP

Video guides

GUIDE how to setup environment for Gaussian calculations in windows.

Setting-up the environment (to be done once)

To submit calculations easily you need to set-up the environment first.

Automated script

You must be within university network or connected through VPN. connect through SSH to lilo.science.ru.nl (in putty that is the hostname). Your username is your science login. After logging-in, execute /vol/qmorgchem/scripts/gaussify script (cnorris is the username in this example):

cnorris@lilo7:~$ /vol/qmorgchem/scripts/gaussify

If you're more experienced user, you can setup the things on your own.

Explanation of the automated script

The bash script sets-up few things on your behalf. What it does is:

~/.profile file editation

to see your ~/.profile content, execute:

cnorris@lilo7:~$ cat ~/.profile

you should see something like this:

export PATH=$PATH:/vol/qmorgchem/scripts
module load Gaussian-16
umask 027

This file stores some of your server settings, the script:

  • Adds (export PATH=... line) /vol/qmorgchem/scripts in your $PATH variable - by doing this you can easily execute scripts located in /vol/qmorgchem/scripts folder such as the important G16 script.
  • Automatically loads Gaussian-16 module (module load... line) which enables you to use commands like cubegen and similar.
  • Changes the umask to 027 - that means that files newly created from the console can be readed by people from qmorgchem, but cannot be changed by them. If you want to keep the files changeable by other people from qmorgchem, then remove this line from your ~/.profile file.

Folders and shortcuts creation

Script creates your folder in qmorgchem disk. If you are member of the qmorgchem group it creates your folder also on the qmorgchem-old disk. It creates shortcuts (named symlinks in linux) to those folders within your $HOME directory by ln -s commands.

To see those folder changes, execute:

cnorris@lilo7:/vol$ ls -lh ~

You should then see something like this:

total 24K 
drwxr-s--- 2 cnorris cnorris 4.0K Apr  1 13:37 '$RECYCLE.BIN'
-rw-r----- 1 cnorris cnorris  402 Apr  1 13:37  desktop.ini
lrwxrwxrwx 1 cnorris cnorris   28 Apr  1 13:37  old -> /vol/qmorgchem-old/cnorris/
lrwxrwxrwx 1 cnorris cnorris   15 Apr  1 13:37  others -> /vol/qmorgchem/
lrwxrwxrwx 1 cnorris cnorris   24 Apr  1 13:37  running -> /vol/qmorgchem/cnorris/

ls is a command which lists directory content, ~ is a shorthand for your $HOME directory (/home/cnorris in case of cnorris user), -lh are arguments: l stands for "long" and h stands for "human-readable." You clearly see that shortcut named "old" points to your folder at qmorgchem-old disk and "running" points to your folder at qmorgchem disk. Please use qmorgchem-old for inactive projects and do not store .chk files there.

Checking calculations, submitting jobs, checking queues

Queues

Each computational job must be submitted to a queue. Currently we have two queues - orgfast and orglong.

  • orgfast - Queue should be used for standard jobs (e.g., coptimization of geometries, harmonic frequency calculations, excited state spectra calculations). Please use max 4 cpus and maxi 12 GB of ram in this queue.
  • orglong - Queue for more demanding jobs (typically days or weeks of computational time). This queue has a lower priority; jobs will be running only if the given number of processors is free and not requested in the orgfast queue. Please use max 12 cpus and max 32 GB of ram in this queue.

Submitting jobs, checking queues

There are currently two queues available on the cluster: orgfast for small and quick jobs and orglong for bigger and more-demanding jobs. We are currently using Slurm as our job queuing manager.

To submit a job and doing anything with the running jobs you MUST be on slurm20

Basic Commands:

  • G16 $job_filename $queue - for instance G16 co2capcture.gjf orgfast - submits a calculation
  • squeue - checks queue
  • scancel $jobid - cancels a job
  • scontrol show jobid $jobid - show details of a job with job number $jobid (good for getting a full name)

Once calculation is running a .log file will appear in the folder where is the input file.

Putty

Connect to slurm20.science.ru.nl - details are the same as for the abovementioned for lilo.science.ru.nl. Change directory to your active directory where computations are being made (cd running - this brings you through shortcut to your folder on the qmorgchem disk).

To submit a job, type:

cnorris@slurm20:~$ G16 $job_filename $queue

command where $job_filename is filename of your Gaussian input and $queue is either orgfast or orglong.

WinSCP

connect through SFTP to slurm20.science.ru.nl and then open folder running and then type:

cnorris@slurm20:~$ G16 $job_filename $queue

command where $job_filename is filename of your Gaussian input and $queue is either orgfast or orglong.

For clarity, see the pictures below:

Checking calculations

once calculations are finished or you want to take a look, you can either use molden (its installed on the cluster - you can run it when you have xming on and you are connected on lilo) or you can connect to the disks and check the files from your windows molecule viewer.

Connecting to disks - windows

Open file explorer and into navigation bar type:

\\qmorgchem-srv.science.ru.nl\qmorgchem\

for qmorgchem-old, the link is:

\\qmorgchem-old-srv.science.ru.nl\qmorgchem-old\

Connecting to disks - Linux

create some file in your homedir - for example lilofs:

mkdir ~/lilofs

install sshfs if you do not have it yet and then:

sshfs lilo.science.ru.nl:/ ~/lilofs

the disks are then located in ~/lilofs/vol , I will leave creating appropriate symlinks as a homework excercise - its analogous to commands mentioned above.

Standard operation procedure, tips

  • If the orglong queue is full and you have quick jobs, run them at orgfast
  • Pay attention when setting-up memory allocation, use of excessive memory prevents other jobs from running. Memory is shared. Do not use memory that you do not need.
  • Do calculations in optimization steps
  • reading force constants (RCFC) speeds-up the initial optimization step.

optimization steps

This section is an example of the optimization steps. Basis sets and levels of theory are case-specific and thus may vary.

  • semiempirical (if applicable, AM1, PM6)
  • cheap DFT (pure DFT, fe: B97D/sto-3g, M06L/midix)
  • expensive DFT (hybrid, fe: B3Lyp)


Issues

Open

Failing calculations after submission

error note: galloc: could not allocate memory. For now there is an issue that sometimes calculation fail while submitting a job. If this happens to you, please record a job number and send it to Jan. Do not delete .error, .out and .log files of the failed job, they are important for debugging this issue.

Disclaimer

If you have any issue to report please write Jan.

This guide is a draft, if you have any suggestions how to improve the readability, understandability or any idea how to make it better, please do not be shy and write Jan.