Checklist 1. I have searched related issues but cannot get the expected help. 2. The bug has not been fixed in the latest version. 3. Please note that if the bug-related issue you submitted lacks c ...

Besides, we extend the FP16 Tensor-Cores-based QR factorization to accommodate FP32 and FP64 on FP16 and INT8 Tensor Cores, respectively. Additionally, to address the issue of orthogonality loss in ...

Optimal rank selection is an important issue in tensor decomposition problems, especially for Tensor Train (TT) and Tensor Ring (TR) (also known as Tensor Chain) decompositions. In this paper, a new ...

To fully harness the power of LUT-based mpGEMM, we introduce LUT Tensor Core, a software-hardware co-design optimized for low-bit LLM inference. Specifically, we introduce software-based operator ...