o + iQI@sddlZddlZddlZddlZddlZddlmZddlm Z ddl m Z m Z m Z mZddlmZmZmZmZmZddlmZd Zd Ze eejd d ZGd ddZdS)N) quant_layers) get_logger)_get_input_name_index_get_op_input_var_names_get_op_output_var_names_get_output_name_index) fuse_utils ptq_config ptq_hooks ptq_quantizerutils) PTQRegistry.pdmodelz .pdiparamsz&%(asctime)s-%(levelname)s: %(message)s)fmtcseZdZdZejffdd ZdddZddd Zd d Z d d Z ddZ ddZ ddZ ddZddZeddZeddZZS) ImperativePTQz, Static post training quantization. cs$tt|tjs J||_dS)aQ Constructor. Args: quant_config(PTQConfig): the config of post training quantization. The config has weight_quantizer and activation_quantizer. In default, the weight_quantizer is PerChannelAbsmaxQuantizer and the activation_quantizer is KLQuantizer. N)super__init__ isinstancer Z PTQConfig _quant_config)self quant_config __class__n/home/app/PaddleOCR-VL-test/.venv_paddleocr/lib/python3.10/site-packages/paddle/quantization/imperative/ptq.pyr/s  zImperativePTQ.__init__FNc Cst|tjjs Jd|st|}|r|t||}| D]8\}}t |rZt |rZ||sZt|j}t |rCd|_||_tj}||} | |_|jj| jddq"|S)a Add quant config and hook to the target layer. Args: model(paddle.nn.Layer): The model to be quantized. inplace(bool): Whether apply quantization to the input model. Default: False. fuse(bool): Whether to fuse layers. Default: False. fuse_list(list): The layers' names to be fused. For example, "fuse_list = [["conv1", "bn1"], ["conv2", "bn2"]]". A TypeError would be raised if "fuse" was set as True but "fuse_list" was None. Default: None. Return quantized_model(paddle.nn.Layer): The quantized model. 2The model must be the instance of paddle.nn.Layer.TF)last)rpaddlennLayercopydeepcopyevalr Z fuse_layersnamed_sublayersrZis_supported_layerrZ is_leaf_layer_is_skip_layerris_simulated_quant_layerenable_in_act_quantizerr Zquant_forward_post_hookZregister_forward_post_hookquant_hook_handleZ_forward_post_hooks move_to_endZ_hook_id) rmodelZinplaceZfuseZ fuse_listnamelayerrhookr)rrrquantize?s6     zImperativePTQ.quantizec sRt|tjjs Jd||tjjd|||d|d}tr)d}tt }tj }tj |}t j|} t j|} | t} | t} tj j| || | d\} }||||d}| durrd}n| dr| d d d }n| }t j| |}fd d | D}tj j||||d|rtdSdS)a 1. Convert the quantized model 2. Call jit.save to save the inference model 3. Post process the inference model. Args: model (Layer): The model to be saved. path (str): The path prefix to save model. The format is ``dirname/file_prefix`` or ``file_prefix``. input_spec (list[InputSpec|Tensor], optional): Describes the input of the saved model's forward method, which can be described by InputSpec or example Tensor. If None, all input variables of the original Layer's forward method would be the inputs of the saved model. Default None. **config (dict, optional): Other save configuration options for compatibility. We do not recommend using these configurations, they may be removed in the future. If not necessary, DO NOT use them. Default None. The following options are currently supported: (1) output_spec (list[Tensor]): Selects the output targets of the saved model. By default, all return variables of original Layer's forward method are kept as the output of the saved model. If the provided ``output_spec`` list is not all output variables, the saved model will be pruned according to the given ``output_spec`` list. Returns: None r)r-path input_specFT) path_prefixexecutormodel_filenameparams_filenameNr+r.r rcsg|] }|qSr)Z global_blockvar).0r,Z infer_programrr sz6ImperativePTQ.save_quantized_model..)r3programr)rrr r!_convertZjitsaveZin_dynamic_modeZ enable_staticZCPUPlaceZstaticZ global_scopeExecutorosr0dirnamebasenameINFER_MODEL_SUFFIXINFER_PARAMS_SUFFIXZload_inference_model _clean_up_gather_input_thresholds_remove_scale_opendswithrsplitjoinZsave_inference_modelcloneZdisable_static)rr+r0r1configZis_dynamic_modeZplacescopeexer@rAr4r5Zfeed_target_namesZ fetch_targetsZ model_namer2Z feed_varsrr9rsave_quantized_modelosb          z"ImperativePTQ.save_quantized_modelcCsj|D]\}}||r|jjq|||D]\}}||r-|||jq||dS)a  Convert the quantized model. Args: model(paddle.nn.Layer): The quantized model. inplace(bool): Whether apply conversion to the input model. Default: False. Returns: None N)r%_is_quant_layerrr)remove_cal_thresholds_save_output_thresholds_wrap_simulated_layers)rr+r, sub_layerrrrr<s    zImperativePTQ._convertcCst|tjjs Jdd}d}|D] \}}||r |d7}q|D]D\}}||ri|d7}|ddkrDtd|d|d|j}|j rO|j |j t |ri|jf}|j|||j q%dS) z Calculate the thresholds of inputs and outputs. Args: model(paddle.nn.Layer): The quantized model. Returns: None 8The input model must be the instance of paddle.nn.Layer.rr z Process the z / z layerN)rrr r!r%rO_loggerinforr(in_act_quantizerZcal_thresholdsout_act_quantizerrr'weight wt_quantizerZ sample_data)rr+Z total_numZcur_numr,rTrweightsrrrrQs0        zImperativePTQ._cal_thresholdscCst|tjjs Jdt|}|j}|jj}t |dksJt |dkrC|dt dd}| ||di| d|didSt d|ddt |dS) z Save the output thresholds to the layer. Args: sub_layer(paddle.nn.Layer): The quantized layer. quant_config(PTQConfig): the quant config for the layer. Returns: None rUr r _threshold out_thresholdzoutput_thresholds shape of z need to be 1, but received N)rrr r!r layer_info output_namesrZ thresholdslenstrZ _set_op_attrsrWwarning)rrTrr`raZoutput_thresholdsZ save_namerrrrR s   z%ImperativePTQ._save_output_thresholdscCst|tjjs Jd|D]\}}||rt|r|j}|j dus'J|j }|j }d}t j D]\}} t|| rCd|}nq4|dusJJt|tjrSd} nd} | d|j|jd} tj||fi| } t| d spJt| jd sxJt|jd krtj|jd gtjd } | jj| t| dsJt| jd sJt|jd ksJ|jd }t|trtj|tjd }n tj|gtjd }| jj||| |t ||\}}t!||| qdS)z Replace conv2d and linear with the quantized layers, and save thresholds into the fake layers. Args: model(paddle.nn.Layer): The model to be quantized. Returns: None rUTNZ QuantizedZabs_maxZchannel_wise_abs_maxZmoving_average_abs_max)weight_quantize_typeZactivation_quantize_typeZ weight_bitsZactivation_bits_fake_quant_input_scaler r)Zdtype_fake_quant_weight)"rrr r!r%rOrr'rr(r\rYrZlayer_name_mapitemsr ZAbsmaxQuantizerZ quant_bitsr__dict__hasattrrgrcrbnparrayZfloat32rh set_valuerilistrRZfind_parent_layer_and_sub_namesetattr)rr+r,rTrr\rYZquant_layer_namekeyvaluerfkwargsZ quant_layerZinput_thresholdZweight_thresholdZ parent_layerZsub_namerrrrS%s~        z$ImperativePTQ._wrap_simulated_layersc Cs(t|D]}t|D]}t|j|}|durq d|jvs#|jdkrO|dd}t||}t|}t ||\}} | |t | d|| ddq t |D]<} | |krZqSt || \}} |t | d}||soqS||} t ||\}} |t | d}| || | ddqSq qdS) z Get and save input thresholds from the front ops. Args: program(Program): the input infer program. scope(Scope): the corresponding scope for the program. Returns: None Nquantize_dequantizemoving_average_abs_max_scaleZOutScalerr^with_quant_attrT)rprogram_all_opsrZfind_previous_opblocktypeoutputZload_variable_dataZfp_numpy_to_naiver _set_attrrdrrhas_attrattr) rr;rLop in_var_nameZ previous_op attr_nameZ in_thresholdargnameindex out_var_name thresholdrrrrEvs@         z&ImperativePTQ._gather_input_thresholdsc Csdd}t|D]v}d|jvr |jD] }d|vr||qq |jdvr|jdkr,dnd}||d }t|j|}t|d ksI|d jd krJq |d }t ||\} } | t | d} t ||dd \} } | t | d} |||| | |||d d q d S)z Remove useless thresholds which are added in jit.save. Args: program(Program): the input infer program. Returns: None cSsn||r1||r3||||kr5||}||||||||dddSdSdSdS)NrwT)r}r~ _remove_attrr|)rnext_op old_attr_name new_attr_namerrrr_helpers    z(ImperativePTQ._clean_up.._helperrur^)conv2dmatmulrOutputOutrr Zelementwise_addr_N) rrxrzZ attr_namesrr{ find_next_opsryrcrrd) rr;rrrZarg_namernext_opsrrrrrrrrrDs2     zImperativePTQ._clean_upcCs^t|D]'}|jdkr,|dd}|dd}t|j|}|D]}|||q#qdS)z= Remove the moving_average_abs_max_scale op. rvXrrN)rrxrzinputr{rryZ _rename_input)rr;rrrrrrrrrFs zImperativePTQ._remove_scale_opcCst|do |jduS)N skip_quantT)rlrr-rrrr&szImperativePTQ._is_skip_layercCs t|dS)Nr)rlrrrrrOs zImperativePTQ._is_quant_layer)FFN)N)__name__ __module__ __qualname____doc__r Zdefault_ptq_configrr/rNr<rQrRrSrErDrF staticmethodr&rO __classcell__rrrrr*s   0^$Q-0 r)r"loggingr?numpyrmrZpaddle.nn.quantrZstatic.log_helperrZstatic.quantization.utilsrrrrr r r r rZ ptq_registryrrBrCrINFOrWrrrrrs