- Problem

I recently noted one of the systems we are simulating suffered from decreasing performance as the simulation progressed. Further investigation revealed it is not the actual performance of the simulation that was lower, but the fact that PLUMED needed increasingly large amounts of time to pre-process the HILLS file for each replica.

This is not an issue for a short-running simulation but for one where the simulation time runs into several μs (such as Coarse-grained simulations), it can completely hinder the trajectory from progressing past a certain point in time.

- Solution

Comparisons with other systems and similar issues mentioned in the issues of the PLUMED github repository, as well as in the PLUMED mailing list, pointed in the direction of a grid-related setting being the cause of the delay, specifically, the fineness/coarseness of the grid. 

The two files below (CORRECT PLUMED INPUT FILE & OLD PLUMED INPUT FILE) highlight a different way of specifying the grid density.

In the `old` file, the grid spacing was manually specified with the GRID_SPACING argument in the METAD block.

In the `correct` file, that line is absent from the file. In the absence of a GRID_SPACING or a GRID_BIN argument, PLUMED is going to use a grid spacing value equal to 1/5 of the Collective Variable (CV) width, for each CV, espectively.

Further testing is required but this value appears to be a robust default and has solved this issue in this particular instance.


CORRECT PLUMED INPUT FILE

C1: RMSD REFERENCE=rmsd_reference.pdb TYPE=OPTIMAL

COM1: CENTER ATOMS=1-122
COM2: CENTER ATOMS=123-244

D1: DISTANCE ATOMS=COM1,COM2

METAD ...
ARG=C1,D1
SIGMA=0.4,0.8
HEIGHT=0.005
PACE=100
LABEL=meta
BIASFACTOR=2.0
TEMP=300
GRID_MIN=0,1
GRID_MAX=4.5,6
... METAD
UPPER_WALLS ARG=C1 AT=4 KAPPA=300.0 EXP=2 EPS=1 OFFSET=0 LABEL=uwall
UPPER_WALLS ARG=D1 AT=5.5 KAPPA=300.0 EXP=2 EPS=1 OFFSET=0 LABEL=u2wall

# monitor the two variables and the metadynamics bias potential
PRINT STRIDE=10000 ARG=C1,D1,meta.bias FILE=COLVAR
​


OLD INPUT PLUMED FILE (do not use!)

C1: RMSD REFERENCE=rmsd_reference.pdb TYPE=OPTIMAL

COM1: CENTER ATOMS=1-122
COM2: CENTER ATOMS=123-244

D1: DISTANCE ATOMS=COM1,COM2

METAD ...
ARG=C1,D1
SIGMA=0.4,0.8
HEIGHT=0.005
PACE=100
LABEL=meta
BIASFACTOR=2.0
TEMP=300
GRID_MIN=0,1
GRID_MAX=4.5,6
GRID_SPACING=0.01,0.01
... METAD
UPPER_WALLS ARG=C1 AT=4 KAPPA=300.0 EXP=2 EPS=1 OFFSET=0 LABEL=uwall
UPPER_WALLS ARG=D1 AT=5.5 KAPPA=300.0 EXP=2 EPS=1 OFFSET=0 LABEL=u2wall

# monitor the two variables and the metadynamics bias potential
PRINT STRIDE=10000 ARG=C1,D1,meta.bias FILE=COLVAR

- Problem

Files/folders might become corrupted when transferring between computers or
even disks, in a way that is non obvious to your code so it won't throw a
warning, but might still affect the validity of your data.

- Solution

This problem can occur when transferring files between one cluster and another
but it can also occur at any point when file transfer is taking place, i.e.

 * Transfer file from internal to external disk drive
 * Transfer file from one internal disk drive to another internal disk drive
 * Transfer file from one computer to another over the network

It can even take place when no operations are being performed on the file at
all. This is called bit rot and data centers that specialise in archival and
where data integrity is of high importance, employ specialised hardware and
software to detect and correct it.

For our purposes, what we can do is focus on best practices when downloading
or uploading files from or to a location. This boils down to two things:

(1) Instead of transferring multiple small files and folders it is better to
    transfer a single item instead.

    This can be achieved with a command like

    `tar -czf directory.tgz directory` if we are interested in transferring
    a single directory but can, of course, accomodate as many folders as we
    require.

    For the next step we need a way of generating a unique "identity" for the
    tgz archive. For this we can use a checksum. One way of computing one can
    be seen in the command below:

    `md5sum directory.tgz`

    This will print a string of alphanumeric characters (the aforementioned
    "identity" of the file) followed by white-space and the filename. The
    output of the command can be stored in a file for easier comparison. After
    transferring the file to the destination we can run the `md5sum` command
    there as well and verify the hashes are identical.

    An added benefit of transferring data in a single archive is that it is
    faster, as our file transfer program of choice (e.g. `scp`, or `rsync`)
    only needs to negotiate a single connection.

(2) Alternatively, assuming it doesn't make sense to bundle our data in a
    single archive, we can run `md5sum` on all files to be transferred and
    compare all of the checksums before and after the transfer. This can be
    achieved in many ways but one command that would do the trick is:

    `find -L . -type f | xargs md5sum | sort -Vk2`

    This can be run from a location that contains all of the files you want to
    transfer. An short explanation about the various flags follows:

    `-L`: This instructs `find` to follow symlinks
    `.`: This instructs `find` to search in the current directory
    `-type f`: This instructs `find` to only identify files

    `xargs md5sum`: Run `md5sum` on all detected files

    `sort -Vk2`: Sort the results lexicographically by file name to avoid
                 differences in the default sorting order due to location
                 settings

    The two files produced by the above command (before and after transfer),
    can be compared to ensure the transferred files are identical.

​Panos