o )i@slddlmZddlmZddlZddlmZddlmZddl m Z ddl m Z ee ZGd d d e ZdS) )Iterable)UnionN)SymInt) init_logger)is_func)VllmInductorPassc@szeZdZdZdejjfddZdee e ejj fdee e e ffddZ d e e ejj fd e e e fd efd d ZdS)NoOpEliminationPassa This is an inductor pass that removes redundant reshape/slice operations. It is required for RMSNorm-quant fusion to work properly. That's because apply_fp8_linear adds a reshape, which is redundant in the 2D-case. Additionally, torch internal no-op elimination pass does not handle certain slice variants. Cases handled: 1. A chain of reshapes is equivalent to the last reshape called on the base tensor (input of the first reshape). 2. A reshape that produces the shape of the input is redundant 3. A slice that produces the shape of the input is redundant Example graph 1: mul_1: "f16[s0, 4096]" = ... view_1: "f16[s0, 128, 32]" = torch.reshape(mul_1, [-1, 128, 32]) view_2: "f16[s0, 4096]" = torch.reshape(view_2, [-1, 4096]) view_3: "f16[s0, 128, 32]" = torch.reshape(view_3, [-1, 128, 32]) Can be replaced with: mul_1: "f16[s0, 4096]" = ... view_3: "f16[s0, 128, 32]" = ... Example graph 2: getitem_1: "f16[s0, 4096]" = ... view_1: "f16[s0, 4096]" = torch.reshape(getitem_1, [-1, 4096]) at = auto_functionalized(static_scaled_fp8_quant, input = view_1, ...) out: "f8e4m3fn[s0, 4096]" = at[1] Can be replaced with: getitem_1: "f16[s0, 4096]" = ... at = auto_functionalized(static_scaled_fp8_quant, input = getitem_1, ...) out: "f8e4m3fn[s0, 4096]" = at[1] Example graph 3: arg0: "s0" = SymInt(s0) scaled_mm: "f16[s0, 4096]" = ... slice_1: "f16[s0, 4096]" = torch.slice(scaled_mm, -1, 0, arg0) at = auto_functionalized(fused_add_rms_norm, input = slice_1, ...) out: "f16[s0, 4096]" = torch.slice_scatter(scaled_mm, at[1], 0, 0, arg0) Can be replaced with: arg0: "s0" = SymInt(s0) scaled_mm: "f16[s0, 4096]" = ... at = auto_functionalized(fused_add_rms_norm, input = scaled_mm, ...) out: "f16[s0, 4096]" = at[1] TODO(luka): This is currently tested in test_fusion, but separate tests could be good. graphcCs|||dd}|jD]}t|tjjjjrv|j d}t|tjjjjrA| d|j dt |j dkrA| ||d7}|j dd\}}|jdj}t |t |krYq|ddkraq|||ru||| ||d7}qt|tjjjjr|j dd\}}}} |jdj}||} |dkr|| | r||| ||d7}qt|tjjjjr|j dd\} } }}} | jdj} | jdj}||}| |kr|dkr|| |r|| | ||d7}qtd |||d |dS) NZbefore_noop_eliminationrrvalz$Removed %s no-op reshapes and slicesZafter_noop_elimination)beginZ dump_graphnodesrtorchopsZatenZreshapedefaultargsZ update_arglenZusersZ erase_nodemetashapecountall_dims_equivalentZreplace_all_uses_withsliceZTensordims_equivalentZ slice_scatterloggerdebugZ end_and_log)selfr rnodeinputrZ input_shapeZ dim_indexstartendi_dimbaseviewZ base_shapeZ view_shapeZview_dimr'm/home/app/PaddleOCR-VL-test/.venv_paddleocr/lib/python3.10/site-packages/vllm/compilation/noop_elimination.py__call__Fs\                   zNoOpEliminationPass.__call__dimsi_dimscstfddt||DS)Nc3s |] \}}||VqdS)N)r).0sZi_srr'r( s z:NoOpEliminationPass.all_dims_equivalent..)allzip)rr*r+r'r.r(rs z'NoOpEliminationPass.all_dims_equivalentdimr$returncCs0||ks|dkr dSt|tjjo|jd|kS)a This function checks if two dimensions are equivalent. :param dim: The dimension arg to reshape/slice :param i_dim: The corresponding dimension in the input tensor :return: Are the dimensions equivalent? There are three cases in which the dimensions are equivalent: 1. The dimensions are equal (both integers) 2. The reshape dimension is -1 (i.e. inferred) 3. The dimensions both correspond to the same SymInt While case 2 does not guarantee the dimensions are equal, they are equal if all other dimensions are equal. In case 3, the reshape dimension is a torch.fx.Node, and its value is a SymInt. That value is equal to the input dimension. r Tr ) isinstancerfxNoder)rr2r$r'r'r(rsz#NoOpEliminationPass.dims_equivalentN)__name__ __module__ __qualname____doc__rr5ZGraphr)rrintr6rrboolrr'r'r'r(r s3A  r )collections.abcrtypingrZtorch.fxrrZ vllm.loggerrZfx_utilsrZvllm_inductor_passrr7rr r'r'r'r(s