Gwave (PSU)¶
Key information | |
---|---|
Home page | |
Account sign-up | https://grouper.ligo.org/ldg/ |
Support | Open a Help Desk ticket |
Login hosts¶
Hostname | Description | Memory | CPU model | cores/CPU | total threads |
---|---|---|---|---|---|
ligo01.gwave.ics.psu.edu | Production submit node | 192GB | 2 x 2.6GHz Xeon Gold 6126 | 12 | 48 |
ligo02.gwave.ics.psu.edu | Production submit node | 192GB | 2 x 2.6GHz Xeon Gold 6126 | 12 | 48 |
For details on how to connect to these machines, please see Access to the LIGO Data Grid.
Execute hosts¶
Hostname | Memory | CPU model | cores/CPU | total threads |
---|---|---|---|---|
comp-ex-0{001-184} | 192GB | 2 x 2.6GHz Xeon Gold 6126 | 12 | 48 |
Functional Architecture¶
Provided Services and Dashboards¶
Service | URL |
---|---|
user webspace | https://ldas-jobs.gwave.ics.psu.edu/~albert.einstein/ |
cluster utilization dashboards | https://foreman01.ext.gwave.ics.psu.edu/grafana/dashboards |
disk usage map | https://foreman01.ext.gwave.ics.psu.edu/diskusage/ |
OSG jobs on GWave | https://gracc.opensciencegrid.org/d/000000079/site-summary?orgId=1&var-site=PSU%20LIGO |
User Information¶
Important Directories¶
/ligo/home/ligo.org/albert.einstein/
: regular files, analysis output, etc./ligo/home/ligo.org/albert.einstein/public_html/
: files viewable by LIGO-authtenticated users. See below for setting up file sharing/ligo/software/ligo.org/albert.einstein/
: for software builds/cvmfs
: CVMFS data files served from a local NFS Aliencache and remote Stashcaches
Data Access¶
Frame files are provided by CVMFS via repositories in the /cvmfs
directory. To access proprietary data, you must first create an active proxy via ligo-proxy-init user.name
. You must do this prior to sourcing environments or running analyses that require proprietary data.
Sharing LIGO Files¶
Files in your home directory may be shared by using the /ligo/home/ligo.org/albert.einstein/public_html/
folder. To do this, you must set up permissions for both public_html
and your home folder:
chmod 775 /ligo/home/ligo.org/albert.einstein/public_html
chmod 711 /ligo/home/ligo.org/albert.einstein/
This allows your files to be viewable by LIGO-authenticated users at https://ldas-jobs.gwave.ics.psu.edu/~albert.einstein/ (replacing albert.einstein
with your LIGO credentials).
Building Software¶
When compiling software, use the /ligo/software/
directory instead of the /ligo/home/
directory.
Conda Environments
By default, user conda environments are stored in /ligo/home/
, so this must be changed via:
ENV_DIR='conda-envs' # this is the folder name where you want to keep your environments
mkdir -p /ligo/software/ligo.org/${USER}/${ENV_DIR}
conda config --add envs_dirs /ligo/software/ligo.org/${USER}/${ENV_DIR}
Use conda activate
to load environments as usual. Verify your setup with conda info --env
; output should be similar to this:
$ conda info --env
# conda environments:
#
test-env * /ligo/software/ligo.org/albert.einstein/<conda-envs>/envs/test-env
To shorten the long path that prefixes the terminal prompt, enter:
conda config --set env_prompt '({name})'
After building software or moving environments to the /ligo/software
directory, reference that path in your HTCondor submit and DAG files in place of your /ligo/home
directory.
Other Tips¶
SSH keys¶
To SSH to GWave without entering your password each time, copy your public SSH key using:
ssh-copy-id -i ~/.ssh/<your_pubkey.pub> albert.einstein@ligo01.gwave.ics.psu.edu
You will be prompted for your password to copy the file. Afterwards, you will be able to SSH without re-entering your password.