Performance enhancement of the Ozaki Scheme on integer matrix multiplication unit.

This article focuses on improving the Ozaki scheme, a highly accurate matrix multiplication algorithm that achieves high precision by decomposing a high-precision matrix product into multiple lower-precision matrix multiplications and higher-precision accumulations. The study proposes new implementation methods leveraging NVIDIA's INT8 Tensor Cores to accelerate the accumulation step and introduces an alternative splitting method based on rounding to nearest floating-point arithmetic, enhancing both performance and accuracy compared to the original Ozaki scheme implementation (ozIMMU). Numerical experiments on NVIDIA GeForce RTX 4090 and GH200 Grace Hopper Superchip demonstrate that the proposed methods reduce the accumulation overhead significantly, achieve up to 1.6-fold speedup, and require fewer slices for comparable accuracy, making them well-suited for next-generation architectures emphasizing low-precision high-throughput computations. The paper also provides a detailed rounding error analysis confirming that the proposed approaches maintain rigorous error bounds while improving computational efficiency.