Domain decomposition Sample Clauses
Domain decomposition. The computation time of a SimpleX calculation is independent of the number of sources, it is therefore sufficient to have a domain decomposition that assigns every processor an approx- imately equal number of grid points. Dividing space into equal volumes and assigning each volume to a processor is not sufficient because the number of points in each volume may dif- fer dramatically due to the adaptive grid. We therefore use a domain decomposition based on the space-filling ▇▇▇▇▇▇▇ curve, which is also employed in other methods without a regular grid (▇▇▇▇▇▇▇▇ & Bertschinger 2005; ▇▇▇▇▇▇▇▇ 2005, 2010). The ▇▇▇▇▇▇▇ curve is a fractal that com- pletely fills a cubic rectangular volume. A ▇▇▇▇▇▇▇ curve is uniquely defined by its order m and its dimensionality d, filling every cell of a d-dimensional cube of length 2m. The following properties of the ▇▇▇▇▇▇▇ curve are beneficial when using it for domain decomposition: • Locality: points that are close along the 1D ▇▇▇▇▇▇▇ curve are in general also close in 3D space. • Compactness: a set of cells defined by a continuous section of the ▇▇▇▇▇▇▇ curve has a small surface to volume ratio. • Self-similarity: the ▇▇▇▇▇▇▇ curve can be extended to arbitrarily large size. The first two properties minimise the number of communications between processors, while the third property ensures that we can use an arbitrarily large number of cells to determine the domain decomposition. The first step in the domain decomposition is dividing the domain into 2md equal, regular cells, where d is again the dimension and m the order of the ▇▇▇▇▇▇▇ curve. We then step through the cells along the ▇▇▇▇▇▇▇ curve, counting the number of grid points inside each cell until the number of grid points equals the total number of grid points divided by the number of processors. In this way, every processor approximately holds an equal number of grid points, thus dividing the work load evenly, while the necessary communications between processors are minimal due to the locality and compactness property of the ▇▇▇▇▇▇▇ curve.
Domain decomposition. The typical application case of the simulation code will be blood flow through complex irregular and sparse artery geometries. Different domain decomposition approaches (quad-tree cubic subdivision, rectangular subdivision, graph-partitioning) have been analyzed in respect of their memory usage and load-balancing in such cases. The tight coupling between the Eulerian LBM and the Lagrangian particles and their technical implications has been taken into account here. Though it made a complete re-implementation of the LBM datastructures in form of sparse linearized arrays with neighborhood lists and the inverse location mapping (index to coordinate) necessary, graph-partitioning using METIS has been chosen. It takes only actual domain elements (LBM lattice nodes) into account and minimizes the size of subdomain interfaces with the fine granularity of actual LBM lattice nodes. Despite the increased complexity, memory footprint is minimized by not allocating memory for space outside the flow domain. METIS has been shown to produce excellent partitionings with minimal interface and good load- balance for related application cases (Axner et al, J. Comput. Phys. 227, 4895, 2008).
Domain decomposition. A simulation setup workflow has been developed that takes a b/w bitmap (in PPM format) filename and the number of subdomains as input arguments for a single call of pasumo_MkCase. It creates a partitioning with METIS, identifies inner domain nodes and nodes shared between each pair of subdomains at their interface, as well as domain neighbors and neighbors of each lattice node list. Enumeration of fluid nodes is such that “inner” nodes (those that don't need to take part in communication) are contiguous in memory. Same applies to all interface nodes, at inner and outer interfaces separately, so that there communication can be achieved through send/receive of a contiguous chunk of memory without the need of serialization/deserialization later in the simulation.