Parallel multithreaded deduplication of data sequences in nuclear structure calculations.

This article focuses on the development and evaluation of three parallel multithreaded algorithms for deduplication of data sequences in high performance computing (HPC) applications, specifically within nuclear structure calculations using the symmetry-adapted no-core shell model (SA-NCSM). The proposed algorithms—MTSK and MTIK, which utilize concurrent hash tables, and MTPS, based on parallel sorting—are designed to efficiently identify and remove redundant sequences arising naturally in SA-NCSM basis data, thereby reducing memory usage and computational overhead. Experimental results on large-scale SA-NCSM problems demonstrate that the MTSK algorithm generally achieves the best performance and scalability on multicore architectures, while MTPS offers a hash table-free alternative suitable for environments where portability is prioritized. The study highlights trade-offs among the algorithms related to memory access patterns, concurrency support, and applicability depending on available data structures and hardware configurations.