SageAttention2 Technical Report: Accurate 4 Bit Attention for Plug-and-play Inference Acceleration.

Mon, 01 Jan 2024 00:00:00 +0100

Jintao Zhang, Haofeng Huang, Pengle Zhang, Jia Wei, Jun Zhu, Jianfei Chen:
SageAttention2 Technical Report: Accurate 4 Bit Attention for Plug-and-play Inference Acceleration. CoRR abs/2411.10958 (2024)