Collective Store (Warp Shfl) Row Major (NN) Col Major (TN) SGEMM FP32/TF32 ...

I have also implemented FlashAttention-2 using pure MMA PTX instructions, which supports features such as Multi-Stages, Tile MMA, Tile Warp and Collective Store ...