Some Common Questions and Answers

Below are a selection of common questions but please feel free to email the developers if you have a quesiton not covered below. If you find a bug in the code, please submit this on the github page.

Q. When I compile cpl-library, I get the following error message:

A. gcc version does not support the c++ 14 standard (which is turned on with the -std=c++14 flag) see https://gcc.gnu.org/projects/cxx1y.html. The version of gcc from your linux distribution package manager should be fine if you have a recent kernal (e.g. Ubuntu 14.04) For older kernals, such as Ubuntu 12.04 you can get later version from the ubuntu-toolchain-r repo as folllows:

or build the latest gcc from source. Version 5.2.0 of gcc is known to work correctly but we are currently working on backwards compatibility to older versions of the c++ standard.

Q. Why am I getting some strange assertion fault or other error from the internals of mpi

A. Different version of mpi do not play well together, openmpi and mpich will be very unlikely to work together and even different version of the same package are non guaranteed to interoperate. For example, if you have compiled one code (e.g. you CFD code) with mpich 3.0 while cpl-library and your MD code are both complied with mpich 3.2, you could get unexpected behaviour when they try to communicate. In fact, even if your versions of mpi are the same but are compiled with different compilers this can cause compatibility problems at the linking stage. Call

and check that the versions of libmpich, (as well as libgcc, libgfortran, libstdc++, etc) are consistent. The build/run may work with mixed libraries but this can lead to some very odd and difficult to debug errors.
You are therefore strongly advised to install gcc-5 and gfortran-5 (or later) compilers, use these to build the latest version of mpi and then build cpl-library and the two codes you plan to couple with this version of mpi.

Q. When you exchange data with cpl_send(), for a specified region, do you have to fill the asend array with the data of the whole region (the entire overlap region) or just the data of the portion of the processor?

A. The idea is to allocate a 4D buffer array based on EXTENTS, copy data into the buffer and pass. Optional limits can also be set when calling cpl_send/cpl_rec This goes as follows:

1. Allocate a buffer array to the size of the number of OVERLAP cells on a processor as follows:
   
   1. Get extents from CPL_proc_extents which takes the minimum limit based on the total number of cells, cells on the calling processor and cells in the overlap region, i.e.
      min(globalcell_limit,
      processorcell_limit,
      overlapcell_limit))
      
   2. Define array size, e.g. in x: nclx = extents(2)-extents(1)+1
      
   3. Allocate memory/array to size of all cells on processor,
      e.g. allocate(A(nrecs, nclx, ncly, nclz))
      
2. If you pass array A into cpl_send/cpl_recv it will send everything on that processor in the overlap region (the EXTENTS). However, often you want to only pass a subsection of the overlapping processor cells (e.g. the CFD halo or MD constraint region). To do this:
   
   1. You load values in the region you want to send (although you could load whatever you want into the send buffer as you only send the limits passed to send).
      
   2. You then specify the limits of the array to use in cpl_send through
      icmin_send, icmax_send,
      jcmin_send, jcmax_send,
      kcmin_send, kcmax_send.
      These can be obtained from CPL_get( icmin_cnst = icmin_send ) to specify the constraint region passed from CFD to MD, hardwired based on the section of the array you want to send (e.g. a fixed value of -1 for the CFD halo) or the minimum of the overlap such as cpl_get(jcmin_olap=icmin_send).
      
3. The cell limits to send/recv must be identical and consistent in both codes that exchange.
   

Note that all indexing for limits specified in send/recv/gather/scatter (e.g. icmin_send) are in the coupled grid global coordinates.

An example may make it clearer:
With a domain with global cell limits in the form
[xmin xmax ymin ymax zmin zmax] = [1 6 1 2 1 1]
split into 3 by 3 by 1 processors, the extents called on each processor would look like:

Where V is void and should be checked and exchange skipped on these processors (although no harm if you don't). Working through the steps 1., 2. and 3. above

1. On processor 1,2 (Fortran numbering)
   
   1. the CPL_proc_extents() would return extents = [3 4 1 2 1 1].
      
   2. nclx = 3-4+1; ncly = 2-1+1; nclz = 1-1+1
      
   3. allocate(A(1,2,2,1))
      
2. If we pass cpl_send(A) this will send all 4 values on proc 1,2. To specify constraint limits (e.g. if only cell 1 in y was constrained)
   
   1. CPL_get( jcmin_cnst) is 1 and CPL_get( jcmax_cnst) is also 1
      
   2. CPL_send(A, jcmin_send=1, icmax_send=1) will send all x and z but limit in y to just 1
      
   Note that all indexing for limits specified in send/recv/gather/scatter (e.g. icmin_send) are in the coupled grid global coordinates, e.g. run from [1 6 1 2 1 1] and so the cpl_send/recv only does something if it has that bit of the global domain. This may seem confusing as the allocated array are based on the size in local cell coordinates (size [1,2,1,2,1,1] so 4 cells). The alternative would require the user to allocate global arrays of size 12 on each processor, which would not scale well for large system sizes.
   
3. On the recieving side you need to match the recv with the same global limits jcmin_recv/jcmax_recv and allocate an array based on extent. which is the number of cells on the CFD processor, assumed to be equal or greater than the coupled MD processor. CPL_get( jcmin_cnst) is 1 and CPL_get( jcmax_cnst) is also 1 CPL_recv(A, jcmin_recv=1, jcmax_recv=1) The limits for extents should then be handled correctly by the cpl_send/recv.
   

Q. When I attempt to compile cpl-library with 'make', I get the following error message:

A. The error is related to differences in versions of the gfortran compiler. The line should be: C_LOC(r_md_f(1)). This has been fixed in the latest version.