o p i @sXddlZddlZddlZddlZddlmZddlmZddlmZm Z ddl m Z m Z m Z mZmZddlmZmZmZddlmZmZddlmZdd lmZmZdd lmZd d lmZe e!Z"ed dZ#ddZ$ddZ%ddZ&ddZ' dUddZ(e)e)ddfddZ*ddZ+dejdejfd d!Z,d"d#Z- dVd$edejd%e.d&e/fd'd(Z0d)d*Z1d+d,Z2d-d.Z3    dWd/d0Z4d$edejdejd%e.d&e/f d1d2Z5d$edejd%e.d3e6d4e/d&e/f d5d6Z7d$edejd%e.fd7d8Z8d9d:Z9d;d<Z:d=d>Z;d?d@ZdEdFZ?dGdHZ@dIdJZAdKdLZBdMdNZCdOdPZDdQdRZEdVdSdTZFdS)XN)pir) ir_backward) ValueDictValueSet)call_decomp_rulecall_decomp_vjp decomp_ops_contain_unused_outputhas_decomp_rulehas_decomp_vjp)pir_chunk_id_guardpir_op_name_guardpir_op_role_guard)Block Operation)signature_safe_contextmanager) DebugPrintauto_recompute)core)registerc csJt}z|s tddVW|stddSdS|s$tdww)NTF)rZ_is_all_prim_enabledZ_set_prim_all_enabled)Z prim_staterb/home/app/PaddleOCR-VL/.venv_paddleocr/lib/python3.10/site-packages/paddle/decomposition/decomp.py prim_guard,s  rcCs:t|tjr |fSt|tjrt|Stdt|dS)NzType z is not supported.) isinstancerValuetypingSequencetuple TypeErrortype)xsrrr_build_tensor_tuple8s  r!cCst|t|ks Jg}t|D]B\}}t||tjrM|tvr5|t|vr5|ddus4Jnt|dkrCt|dtjsEJ||dq||q|S)Nrr) len enumeraterrrnamerkeysappend) orig_outs decomp_outsopresidxvaluerrr_analyse_decomp_results@s  r-csd}g}D]-}|}|r0|}t|tr*||kr*dd|D}||q|dq|kr?|fS|fddD}t |S)z For standard api of operator, its inputs should keep consistent with organization of its inputs and attrs. Args: op (Operator): The target operator. builtin.combinecSg|]}|qSrsource).0itemrrr cz1_prepare_python_api_arguments..Ncsg|]}|qSr)attrsr2xr)rrr4os) operandsr1 initializedget_defining_oprrr$r&Zget_attr_namesr)r)combine_op_nameinputsr8inputprev_opZ api_argumentsrr9r_prepare_python_api_argumentsRs      rAc Csd}g}|D]X}|}|r`|}t|trG||krG|D] }|j}d|vrEt d|jd|ddSq%q|j}d|vr`t d|jd|ddSqdS)Nr.z;Decomp op does not support dynamic shape -1, but got shape z in inputs of op  Tz in op ) r:r1r;r<rrr$shapewarningswarn)r)r=r>r8r?r@r3rDrrr_check_prim_dynamicss4    rGc CsDt|t|ksJd|dt|dt|t||D]\}}|dus*|dur<|tjvrtjd|B}t|tsJt|tsJt||||||S)a Search nonbasic ops which have be registered composite rules and replace them with primitive ops. The operators in blacklist will be excluded from program when decomposed into primitives, and only the operators in whitelist will be decomposed. The priority of blacklist is higher than whitelist, it means an operator both in blacklist and whitelist will not be decomposed. The finally set that will be decomposed is: (block.ops & ops have decomposite rule & whitelist) - blacklist Note: All variables must be contained inside the given program. Args: program (Program): The program to be processed. src_vars (list[Value]): In program, once some operator is decomposed, its vars will be replaced by new ones. This argument means some vars will be used later and corresponding vars will be returned for later usage. blacklist (frozenset): The Operators that will be exclude when decomposed into primitives. whitelist (frozenset): Only the operators in whitelist will be decomposed into primitives. start_index (int): The start index of decomposed operator in global block, default 0; end_index (int): The end index of decomposed operator in global block, default -1 means all ops will be composed. start_index and end_index follow the principle of left closed and right open, that is [start_index, end_index). Returns: dst_vars (list): A list contains all vars which replace origin ones in src_vars. Zforward_blacklist)r prim_configrintZsinking_decomp)programZsrc_vars blacklist whitelist start_indexZ end_indexrrr decomposes  rTcCsV|}|}||krdSt|D]}||||s(dSqdS)zAcheck whether the inputs of two builtins.combine ops are the sameFT)r< num_operandsrangeoperand_sourceis_same)Zinput1Zinput2Zbuiltin_combine_op1Zbuiltin_combine_op2irrr_check_combine_inputss rZfwd_opbwd_opcCs2|dus|d|krdS|}dd|D}t|t|ks)Jdg}t|D]\}}d|vr>|||q/dd|D}|}dd|D} d d g} |D];} | r| d krd} | D] } t | | rxd } nqm| s~dSq[| t |vs| t |vs| | vsdSq[d S) z3check whether the bwd_op is corresponding to fwd_opN_gradFcSr/rr0r7rrrr4r5z_check_op..z1backward op names do not match backward op inputscSr/rr0r7rrrr4r5cS4g|]}|r|dkr|qSr.r1r;r<r$r7rrrr4 zpd_op.full_int_arrayz pd_op.fullr.T) r$get_input_namesr:r"r#r&resultsr;r<rZr)r[r\bwd_op_input_names bwd_inputsZfwd_op_related_inputs_outputsr+r$ fwd_inputs fwd_outputsfwd_vec_inputsZinserted_op_name_listoperandin_fwd vec_inputrrr _check_opsJ   rlc Cs\|}gd}t|D]\}}||vr+||}||vr+||}|}|Sq dS)N)out_gradZOut_gradZ loss_grad)rbr#rir1r<) r\grad_var_to_varrdZ out_grad_namer+Z input_namermoutr[rrr _get_fwd_op0srpblockrnreturncCstj|j|}|}t|}t|}|p|}|rt |r, WddS|dur6t |nt |t |} |rLt |} t || |} nt|| } t||| t||| d|tvrtt|D]} | t|vr|| | | qmn|tvr|d| dn|| |||durd} |D] }|rd} nq| r||d}| dfWdSt|dfWdS1swYdS)a Decompose the forward op into a list of primitive ops. Args: block (Block): the block to which the forward op belongs. fwd_op (pir.Operation): the forward op to be decomposed. grad_var_to_var (dict): a dict obtained from distributed processing, which maps the backward grad variable to its corresponding forward variable. prev_op (pir.Operation): the previous op of fwd_op in the block. If prev_op is builtin.combine, insertion point when decomposing fwd_op will be set to prev_op. Returns: new_outputs (tuple(Value)): the new outputs after decomposing. has_decomposed: whether the forward op has been successfully decomposed. NNF)r'rKrTF)rr program_guardrPr$rcrZget_decomp_ruler rGset_insertion_pointrArr-r!rM_upgrade_grad_var_to_varrr%rVr"replace_all_uses_with remove_op has_one_user)rqr[rnr@rJr'Z decom_ruleZhas_sink_decomp_rulelowerZ input_argsr(rKr+rxr3rrr_decomp_fwd_op=sd           >$r{cszg}|D]4}|r2|dkr2|fddtdD}||q||gq|S)Nr.csg|]}|qSr)rW)r2rYZbuiltin_combine_oprrr4sz#_prepare_inputs..r)r:r1r;r<r$rVrUr&)r[Z new_inputsr?Z new_inputrr|r_prepare_inputss      r}cCsl|}|}t|t|ksJddd|D}|}t|t|ks-Jddd|D}dd|D}g}g} t|D]I\} } | rw| dkrwd} |D] } t | | rfd } nq[| sv| | g| || qG| t |vs| t |vs| | g| || qGg}d }|D]}|d | vr| |||d 7}q| t gq|S) Nz7forward op output names do not match forward op outputscSr/rr0r7rrrr4r5z)_prepare_grad_outputs..z7backward op input names do not match backward op inputscSr/rr0r7rrrr4r5cSr^r_r`r7rrrr4rar.FTrr]r)rcZget_output_namesr"r:rbr#r;r<r$rZr&rr fake_value)r[r\rgZfwd_output_namesreZbwd_input_namesrfrh grad_outputsZgrad_output_namesrY bwd_inputrjrkZnew_grad_outputsindexZfwd_output_namerrr_prepare_grad_outputssV       rcCsRg}t|D] \}}|rdgt||}n dgt||}||q|S)NFT)r#r;r"r&)rfZ bwd_outputsZstop_gradientsr+Z bwd_outputZ stop_gradientrrr_prepare_stop_gradientss rc Cs|dusJd|dur&|dur&t|D]\}}||vr%|||||<q|durK|durM|D]\}}t|D]\} } || rI|| ||<q:q2dSdSdS)Nz"grad_var_to_var should not be None)r#popitemsrX) rn orig_grads new_gradsr'rKr+ grad_inputgrad_varvarrYZorin_varrrrrvs"   rvcCst|}dd|D}t||}t||}|j|}t|j} t|||||} t|j} | | } | dkrO|jd | krO| |jddS|jd dkrd|jd d gg} g} | D]}|ddur~|d r~| |dqh| tqht| t|ksJd t||| d |}t| | D]}||j|||d7}q|| | |t| d fS) z Decompose the backward op into a list of primitive ops. If forward op has composite vjp rules (including custom vjp), call call_vjp() to get a list of primitive operators in backward graph, then replace backward op. cSsg|]}|gqSrr)r2Z fwd_outputrrrr4sz(_decomp_bwd_with_vjp..rrBrs builtin.splitrNzNresults of original backward op do not match results of decomposed backward oprrT)r}rcrropsrr"rZcall_vjpr$rxrir1r;r&rr~rvrVmove_oprwr)rqr[r\rn fwd_inputs_ fwd_outputs_Z grad_outputs_Zstop_gradients_ bwd_op_idxbefore_num_opsnew_grad_inputs after_num_opsZnum_appended_opsr*r insert_idxrYrrr_decomp_bwd_with_vjpsD           rrffwd_outputs_after_decomposecsDdurtddd|D}|}tfdd|D}tfdd|D}tfdd|D} |j|} t|j} t|| |} t|j} g}d }t |D]\}}| rj| | ||d 7}qV| t qVt||d | }t| | D]}||j|||d 7}q||||d }t||fS) aV Decompose the backward op into a list of primitive ops. If forward op has no composite vjp rules, and forward op has been decomposed to a list of primitive operators in forward graph previously, call grad() for the decomposed forward subgraph to get a list of primitive operators in backward graph, then replace backward op. Nz=To decompose backward op, please decompose forward op firstlycSr/rr0r7rrrr4Gr5z+_decomp_bwd_without_vjp..c3s,|]}|tvs|tvs|VqdSN)r)r2r)rfrrr Is  z*_decomp_bwd_without_vjp..c3s|]}|VqdSrr)r2Z grad_outputrnrrrQs c3s |] }|r|VqdSr)r;)r2rrrrrTs rrrT) RuntimeErrorr:rcrrrr"rZgradr#r;r&rr~rvrVrrwrx)rqr\rnrfrreZ grad_inputsrrrrrrrr*Zinput_grads_idxr+rrrYZhas_decomposedr)rfrrnr_decomp_bwd_without_vjp4sJ           rc Cst||}t||st|ddSt|st|r dSt||||\}}|sHdd|D}t|||\}}|rHt |||||\}}||fS)a3 Decompose a backward op in pir program. Get the corresponding forward op according to grad_var_to_var firstly, then (1) try to decompose backward op by calling _decompose_bwd_with_vjp, if forward op has composite vjp rules (including custom vjp), _decompose_bwd_with_vjp will call call_vjp() to get a list of primitive operators in backward graph, then replace backward op successfully and return True; (2) when _decompose_bwd_with_vjp return False, means there is no composite vjp rules, try to decompose forward op firstly by calling _decomp_fwd_op firstly and get corresponding primitive operators in backward graph by calling _decompose_bwd_without_vjp secondly, then replace backward op successfully and return True; (3) if the backward op is still not decomposed by the above two steps, returns False. Args: block (Block): the block to which the backward op belongs. bwd_op (pir.Operation): the backward op to be decomposed. grad_var_to_var (dict): a dict obtained from distributed processing, which maps the backward grad variable to its corresponding forward variable. Return: new_input_grads (tuple(Value)): new results of backward op after decomposing. has_decomposed: whether the backward op has been successfully decomposed. zM can not be decomposed due to the mismatch between forward op and backward oprscSr/rr0r7rrrr4r5z"_decomp_bwd_op..) rprlloggerdebugr$rGrr:r{r) rqr\rnr[rbwd_has_decomposedrfZnew_fwd_outputsfwd_has_decomposedrrr_decomp_bwd_op{sN   rcCsRg}|}|jD]}|ds|dr&||vr&||q |S)Nr]Z_grad_) global_blockrr$endswithr&) pir_programbwd_opsrr)rrr_get_all_bwd_opss    rcCsxg}t}t}||||tdtdtjj dd}tj ddidtjjj _ |||S)NTFLAGS_enable_pir_api) rZ_is_fwd_prim_enabledZ_is_bwd_prim_enabledr&_set_prim_forward_enabled_set_prim_backward_enabledpaddlebase frameworkZ get_flags set_flags global_var _use_pir_api_)stateZprev_fwd_prim_stateZprev_bwd_prim_stateZprev_pir_api_flagrrr_set_prim_states        rcCsXt|dks Jdt|dt|dtjd|di|dtjjj_ dS)NzEstate should contain fwd_prim_state, bwd_prim_state and pir_api_staterrr) r"rrrrrrrrr)rrrr_reset_prim_statesrc Cst}|D]\}}||vr||vrt||dkr8t||dkr8||d}||d}|||<qg}g}t||dkrN|||dn3t||dkrl||ddkrl|||dntdt||D] } |||| qut||dkr|||dn6t||dkr||ddkr|||dn||d} | dr|||dt|dksJdtdt|D] } |d||| <qq|S)zbtranslate grad_var_to_var (mapping VarDesc->VarDesc) to pir_grad_var_to_var (mapping Value->Value)rrrz builtin.slicerB_z#translate pir_grad_var_to_var error) rrr%r"r&r<r$rVr) param_mappingrnpir_grad_var_to_varrrZ new_grad_varZnew_varZ new_grad_varsZnew_varsrYZlast_oprrr_translate_gradvartovar_to_pirsD   rc Cstjj|7t|}g}|j}|D]!}|}||vr6t|||\}}|s6||vr6| |qWdn1sAwYt d|dS)z2Traverse and decompose all backward OPs in programNz.Following backward ops can not be decomposed: ) rrrrtrrrr$rr&rr) rrrZundecomposed_bwd_opsrr) bwd_op_namerrrrr_decomp_bwd_programs(    rc Cstjj|X|j}t|}ddg}g}d}|D]<}|}||vrM||vrDt||||\} } | sC||vrC| |n ||vrM| ||dkrU|nd}qWdn1sbwYt d|dS)z1Traverse and decompose all forward OPs in programz pd_op.stackz pd_op.squeezeNr.z-Following forward ops can not be decomposed: ) rrrrtrrrr$r{r&rr) rrrrZ black_fwd_opsZundecomposed_fwd_opsr@r)Z fwd_op_namerrrrr_decomp_fwd_program0s:     rc CsFt|gtjd}|}d}tjj||j}|D]}|}| dd|vr/qd}t |r t s@t |r@d}|st |xt|jct|jNt||}d} t|} tt|D]0} || r|| } | dkrd} t| | | | | | | || qjWdn1swYWdn1swYWdn1swY| st|| |} t||| || |||dkr|}|dur d}|D] }|rd}nq|r||d}qWdn 1swYtj|dS) z{ Decompose all non-primitive ops into primitive ops in a pir program. It may contain forward ops and backward ops. Zbackward_blacklistN.rBFTrr.)rTrrNrrrrtrr$splitr Z_enable_prim_dynamic_shaperGr r Zop_roler Zchunk_idrurcrrVr"ryZ first_useownerrMrwrxr-Z set_insertion_point_to_block_end)rrQrqZpre_combine_oprr)rZ skip_decompr'Z is_next_splitr(rYZnext_oprKrxr3rrrdecompose_dist_programSs               ;rcCs0t}t||}t||t||t||S)az Decompose all PHI ops into prim ops in a pir program. Args: pir_program (Program): the program to be decomposed param_mapping (dict): a map of program variables to pir program values grad_var_to_var (dict): a dict obtained from distributed processing, which maps the backward grad variable to its corresponding forward variable. )rrrrr)rrrnrrrrrdecompose_pir_programs   rcCsTg}|jD] }|dkr||d|dkr'||dq|S)Nz pd_op.datarzbuiltin.parameter)rrr$r&rcrrcr)rrr"get_inputs_from_data_and_parameters  rcCs8g}|jD]}|dkr||dq|S)Nz pd_op.fetchr)rrr$r&rir1rrrrget_outputs_from_fetch_ops  rcCs@g}t}|D]\}}|||<q |D] }|||q|Sr)rrr&)outputsrrcZ var2grad_varkvoutputrrrget_grad_var_for_lists rcs0fdd}g}|D] }|||q |S)Nc0tjD] \}}||kr|SqtdNzop not found in programr#rrrr)r+Zop_iterrPrrgetIdx z'get_defining_op_indices..getIdx)r&r<)rPZ output_valuesrrcrrrrget_defining_op_indicess rcs:fdd}g}jD] }||r|||q |S)Ncrrrrrrrrrz#get_forward_op_idxs..getIdx)rrr&)rPis_forward_op_funcrrcr)rrrget_forward_op_idxss rc Cstdtd|t|}t|}d}t|rtt||}td}|tj |dur@z tt ||}Wn | dY|dkrK| d|S|d}t |||g||\}}|S)NzStart Recompute Pir Program:zBefore Recompute: rBzauto-recomputezNo Forward Ops Found!zSkip Auto Recompute!r) rrrr"maxrlogging getLoggersetLevelINFOrinfor) rrr>rZfwd_op_end_idxrZbackward_op_start_idxrPrrrrauto_recompute_pir_programs:      r)NNr)NNNN)GrrrErrZpaddle.autogradrZpaddle.autograd.backward_utilsrrZpaddle.base.corerrrr r Zpaddle.base.frameworkr r r Zpaddle.base.libpaddle.pirrrZpaddle.base.wrapped_decoratorrZpaddle.decomposition.recomputerrZpaddle.frameworkrrr__name__rrr!r-rArGrM frozensetrTrZrlrpdictrr{r}rrrvrlistrrrrrrrrrrrrrrrrrrrrs         ! ; ' 2 Q9   ; G H  .#J