o )i@@sddlZddlZddlmZddlmZddlmZddlmZddl m Z ddl m Z ddlmZdd lmZGd d d ejZdS) N)abstractmethod)Iterable)fx)auto_functionalized)pattern_matcher) OpOverload)Node) find_auto_fnc@seZdZdZdejfddZeddZe de e j fdd Z e de jfd d Zde j fd d ZddZde j deedee j dffddZdede fddZdS)MultiOutputMatchz This class provides utilities to process multi-output matches and manually insert replacements. This is necessary because the automatic replacement for multi-output matches is broken: https://github.com/pytorch/pytorch/issues/137280 matchcCs ||_dSN)r )selfr ro/home/app/PaddleOCR-VL-test/.venv_paddleocr/lib/python3.10/site-packages/vllm/compilation/multi_output_match.py__init__s zMultiOutputMatch.__init__cCst)ae Process a multi-output match and manually insert the replacement. This method should: 1. Insert the replacement nodes after the last node in the match. 2. Rebind the users of nodes in the match to use the new nodes. 3. Set meta["val"] for de-functionalization. The result of an auto-functionalized node is a tuple of tensors. The first element is the return value of the function, usually None. The remaining elements are the mutated args of the function. All auto-functionalized nodes must contain a proper meta["val"], as it is used by de-functionalization. meta["val"] has to contain the value of the node (tuple of tensors) that would be returned by the functionalized node during tracing. Existing nodes in the graph all have this property set, but we have to set it manually for new nodes we insert. Example: # op schema: foo(a: Tensor!, b: Tensor, c: Tensor!) -> None at = auto_functionalized(torch.ops._C.foo.default, a, b, c) # at.meta["val"] = (None, a, c) )NotImplementedErrorr rrrprocessszMultiOutputMatch.processreturncC|jjSr )r nodesrrrrr;zMultiOutputMatch.nodescCrr )r graphrrrrr?rzMultiOutputMatch.graphcCs t|j|S)zY Find the first auto_functionalized node with the given op in the match. )r r)r oprrrr Cs zMultiOutputMatch.find_auto_fncCs6t|jjD] }||jjvrnqtd|j|S)z Insert nodes after the last node in the match. This is done to avoid use-before-definition errors after inserting replacement nodes. zNo nodes in graph)reversedrrr ValueErrorinserting_after)r Zlast_node_in_matchrrrinserting_after_matchIs   z&MultiOutputMatch.inserting_after_match tuple_nodeindices.csHjtfdd|DWdS1swYdS)a* Insert operator.getitem nodes to extract elements from a tuple node. :param tuple_node: The tuple node to extract elements from. :param indices: The indices of the elements to extract. :return: Tuple of the new getitem nodes, corresponding to the indices. c3s$|] }jtj|fVqdSr )r call_functionoperatorgetitem).0idxr rrr ds  z3MultiOutputMatch.insert_getitems..N)rrtuple)r rrrr%rinsert_getitemsZs  $z MultiOutputMatch.insert_getitemsrcCs|jjt|f|dS)zR Insert an auto_functionalized node with the given op and kwargs. )kwargs)rr r)r rr)rrrinsert_auto_fnhs zMultiOutputMatch.insert_auto_fnN)__name__ __module__ __qualname____doc__pmMatchrrrpropertylistrrrZGraphrr rrintr'r(rr*rrrrr s"   r )abcr!rcollections.abcrZtorchrZ*torch._higher_order_ops.auto_functionalizerZtorch._inductorrr/Z torch._opsrZtorch.fxrZvllm.compilation.fx_utilsr ABCr rrrrs