Decomp

Decomp

Determines the domain decomposition and periodicity. The Decomp block determines how the simulation is broken up into domains for parallel processing and which directions are periodic.

If the domain boundaries lead to a specification of a number of processors that is not found at runtime, Vorpal will reconfigure the domain decomposition to match the runtime number of processors. That is, if you do not specify a decomposition, Vorpal will try to calculate an appropriate domain decomposition for you.

The decomposition algorithm used by Vorpal proceeds by first doing a prime factorization of the number of processors. It then uses that list of factors to divide up each dimension using the largest remaining factor on the dimension with the largest number of cells.

For example, for a simulation using 30 processors and a domain size of 200 x 100 x 100, Vorpal will perform the following calculations:

30 = 5*3*2

The x direction has the largest number of cells (200) and 5 is the largest factor of the number of processors, so we divide x into 5 regions, each

40x100x100

With the x direction done, y is the now the direction with the largest number of cells (100), so Vorpal uses the next largest factor to divide it into 3 sections:

40x{33 or 34}x100

Finally, Vorpal partitions z with the remaining factor is 2, dividing it into 2 pieces.

40x{33 or 34}x50.

That is the size of our final domains.

In practice, for parallelism, Vorpal simulations should have approximately 20 - 40 cells in each dimension. Otherwise, the amount of messaging per computation increases and the benefits of using multiple processors is outweighed by the cost of the communication between them. The 20 - 40 rule depends on the complexity of the calculations being done. With a large number of particles per cell (20 or more) you could use approximately 20 cells in each direction.

The decomposition may affect what features can be used; for example, guard cells require a domain of at least four cells in every direction. If the number of processors and dimensions of the simulation chosen cause this requirement to be violated, you will see errors in the simulation.

Decomp Parameters

kind (string)

(deprecated in v8.0) regular only allowed at present. Version 8.0+ should simply omit the kind parameter.

periodicDirs (integervector)

Directions to have periodic boundary conditions.

decomp0 (integervector)

Domain boundaries along direction 0.

decomp1 (integervector)

Domain boundaries along direction 1.

decomp2 (integervector)

Domain boundaries along direction 2.

Example of Default Decomposition Generation

In this example, Vorpal uses the aforementioned algorithm (prime factorization) to automatically generate a decomposition for a parallel simulation. The y- and z- directions will be periodic as indicated by use of periodicDirs.

<Decomp decomp>
   periodicDirs = [1 2]
 </Decomp>

Example of User-specified Decomposition

In this example, assuming there is a 200 x 200 mesh grid as described in the section on Grid cell specification, the user requests domain boundaries in the x-direction at 50, 100, and 150, and in the y-direction for 100. This will produce 8 domains, requiring that 8 processors be used for the simulation. If a different number of processors is actually used, Vorpal will default to the auto-generated (prime factorization) decomposition as discussed in the general description of Decomp, above.

<Decomp  decomp>
    periodicDirs = [1 2]
    decomp0 = [50 100 150]
    decomp1 = [100]
</Decomp>