o 0 i@sddlmZddlZdejdedejfddZ  dd ejjd ejd ejd ejd eejdedeede ejdffddZ dS))OptionalN hidden_statesn_repreturncCs^|j\}}}}|dkr |S|dddddddddf|||||}||||||S)z This is the equivalent of torch.repeat_interleave(x, dim=1, repeats=n_rep). The hidden states go from (batch, num_key_value_heads, seqlen, head_dim) to (batch, num_attention_heads, seqlen, head_dim) N)shapeexpandZreshape)rrbatchZnum_key_value_headsslenZhead_dimr p/home/app/PaddleOCR-VL-test/.venv_paddleocr/lib/python3.10/site-packages/transformers/integrations/sdpa_paged.py repeat_kvs 0r modulequerykeyvalueattention_maskdropoutscalingc Ks|dd}|dur*|j|||jfi|\}}|ddd}|ddd}t|dr;t||j}t||j}|} |}|}|}t j j j |||| ||dd} | dd} | dfS)Ncacherrnum_key_value_groupsF)Z attn_maskZ dropout_pscaleZ is_causal) popupdateZ layer_idxZ transposeZ unsqueezehasattrr r contiguoustorchnnZ functionalZscaled_dot_product_attention) rrrrrrrkwargsrZ causal_maskZ attn_outputr r r sdpa_attention_paged_forwards.     r!)rN) typingrrZTensorintr rModulefloattupler!r r r r s,