o + it@s>ddlmZddlZddlmZddlZddlmZmZddlm Z m Z m Z ddl m Z mZddlmZerTdd lmZdd lmZdd lmZdd lmZmZmZmZgZ   dEdFddZ        dGdHd,d-Z    dIdJd0d1Z     2    dKdLd6d7Z 8 9 :  dMdNd@dAZ     dOdPdCdDZ dS)Q) annotationsN) TYPE_CHECKING)_C_opsin_dynamic_mode)in_dygraph_modein_dynamic_or_pir_mode in_pir_mode) check_typecheck_variable_and_dtype) LayerHelper)Sequence)Literal)Tensor) DataLayout1D DataLayout2D DataLayout3D DataLayoutND-q=xrpfloataxisintepsilonname str | Nonereturnc Csptr!tjdg||jd}t|t|||dd}|t||StrFtjdg||jd}t|t|||dd}tj |t|||dSt |dtt fdt |dt dt |d gd dt |jdkru|d kru|d krutd ||t|d|d}tdit}|j|jd}|jdd|id|i|d|jj|jd}tjdg||jd}tj |t|||dS)aL Normalize ``x`` along dimension ``axis`` using :math:`L_p` norm. This layer computes .. math:: y = \frac{x}{ \max\left( \lvert \lvert x \rvert \rvert_p, epsilon\right) } .. math:: \lvert \lvert x \rvert \rvert_p = \left( \sum_i {\lvert x_i \rvert^p} \right)^{1/p} where, :math:`\sum_i{\lvert x_i \rvert^p}` is calculated along the ``axis`` dimension. Parameters: x (Tensor): The input tensor could be N-D tensor, and the input data type could be float32 or float64. p (float|int, optional): The exponent value in the norm formulation. Default: 2. axis (int, optional): The axis on which to apply normalization. If `axis < 0`, the dimension to normalization is `x.ndim + axis`. -1 is the last dimension. epsilon (float, optional): Small float added to denominator to avoid dividing by zero. Default is 1e-12. name (str|None, optional): Name for the operation (optional, default is None). For more information, please refer to :ref:`api_guide_Name`. Returns: Tensor, the output has the same shape and data type with ``x``. Examples: .. code-block:: python >>> import paddle >>> import paddle.nn.functional as F >>> paddle.disable_static() >>> x = paddle.arange(6, dtype="float32").reshape([2,3]) >>> y = F.normalize(x) >>> print(y) Tensor(shape=[2, 3], dtype=float32, place=Place(cpu), stop_gradient=True, [[0. , 0.44721359, 0.89442718], [0.42426404, 0.56568539, 0.70710671]]) >>> y = F.normalize(x, p=1.5) >>> print(y) Tensor(shape=[2, 3], dtype=float32, place=Place(cpu), stop_gradient=True, [[0. , 0.40862012, 0.81724024], [0.35684016, 0.47578689, 0.59473360]]) >>> y = F.normalize(x, axis=0) >>> print(y) Tensor(shape=[2, 3], dtype=float32, place=Place(cpu), stop_gradient=True, [[0. , 0.24253564, 0.37139067], [1. , 0.97014254, 0.92847669]]) r)shapeZ fill_valuedtypeTFrr normalizerr)float16float32float64rzAAxis must be 0 or -1 when x is a 1-D tensor, but received axis = )rZporderZkeepdimrp_normr!XZOuttypeinputsoutputsattrsN)r()rpaddlefullr!rr(rmaximumrdivider rr lenr ValueErrorr locals"create_variable_for_type_inference append_opblockZ create_var) rrrrrepsoutr/helperr=e/home/app/PaddleOCR-VL-test/.venv_paddleocr/lib/python3.10/site-packages/paddle/nn/functional/norm.pyr#.s<; r#F?h㈵>NCHW running_mean running_varweight Tensor | Nonebiastrainingboolmomentum data_formatruse_global_stats bool | Nonec  Cst|jdks Jd|} |} gd} || vrtd||ddkr&dnd}| d ur2| } d }n| }trPt|||||| |||| | \}}}}}}|Strkt|||||| |||| | \}}}}}}|St|d gd d ||| |d | |d}|g|g|gd}|r|g|d<|r|g|d<t dit }ddl m }||j dvr|j nd}|j|dd}|j|dd}||j }|g|g|g|g|gd}|s|r|j|j dd}|g|d<|jd|||d||S)aQ Applies Batch Normalization as described in the paper Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift . nn.functional.batch_norm is used for nn.BatchNorm1D, nn.BatchNorm2D, nn.BatchNorm3D. Please use above API for BatchNorm. Parameters: x(Tensor): input value. It's data type should be float32, float64. running_mean(Tensor): running mean. running_var(Tensor): running variance. weight(Tensor, optional): The weight tensor of batch_norm. Default: None. bias(Tensor, optional): The bias tensor of batch_norm. Default: None. epsilon(float, optional): The small value added to the variance to prevent division by zero. Default: 1e-5. training(bool, optional): True means train mode which compute by batch data and track global mean and var during train period. False means inference mode which compute by global mean and var which calculated by train period. Default False. momentum(float, optional): The value used for the moving_mean and moving_var computation. Default: 0.9. data_format(str, optional): Specify the input data format, may be "NC", "NCL", "NCHW", "NCDHW", "NLC", "NHWC" or "NDHWC", where `N` is batch size, `C` is the number of the feature map, `D` is the depth of the feature, `H` is the height of the feature map, `W` is the width of the feature map, `L` is the length of the feature map. Default "NCHW". use_global_stats(bool|None, optional): Whether to use global mean and variance. If set to False, use the statistics of one mini-batch, if set to True, use the global statistics, if set to None, use global statistics in the test phase and use the statistics of one mini-batch in the training phase. Default: None. name(str|None, optional): Name for the BatchNorm, default is None. For more information, please refer to :ref:`api_guide_Name`.. Returns: None Examples: .. code-block:: python >>> import paddle >>> x = paddle.arange(12, dtype="float32").reshape([2, 1, 2, 3]) >>> print(x) Tensor(shape=[2, 1, 2, 3], dtype=float32, place=Place(cpu), stop_gradient=True, [[[[0. , 1. , 2. ], [3. , 4. , 5. ]]], [[[6. , 7. , 8. ], [9. , 10., 11.]]]]) >>> running_mean = paddle.to_tensor([0], dtype="float32") >>> running_variance = paddle.to_tensor([1], dtype="float32") >>> weight = paddle.to_tensor([2], dtype="float32") >>> bias = paddle.to_tensor([1], dtype="float32") >>> batch_norm_out = paddle.nn.functional.batch_norm(x, running_mean, ... running_variance, weight, bias) >>> print(batch_norm_out) Tensor(shape=[2, 1, 2, 3], dtype=float32, place=Place(cpu), stop_gradient=True, [[[[1. , 2.99998999 , 4.99997997 ], [6.99996948 , 8.99995995 , 10.99994946]]], [[[12.99993896, 14.99992943, 16.99991989], [18.99990845, 20.99989891, 22.99988937]]]]) rzinput dim must be larger than 1)ZNCNCLrANCDHWNLCNHWCNDHWCz\data_format must be one of 'NC', 'NCL', 'NCHW', 'NCDHW', 'NLC', 'NHWC', 'NDHWC' but receive rCrArPNFinput)r$uint16r%r&Z BatchNorm)rIrZis_test data_layoutZfuse_with_relurKtrainable_statistics)r*MeanVarianceScaleBias batch_normr convert_dtype)r$rTr%Tr!Z stop_gradient)YZMeanOutZ VarianceOut SavedMean SavedVarianceZ ReserveSpacer+)r[)r4r r5rrr[rZ batch_norm_r r r6paddle.base.data_feederr]r!r7r8append_activation)rrBrCrDrFrGrIrrJrKrmean_out variance_outZtrue_data_formatrVZbatch_norm_out_t1t2t3Zt4r/r-r<r] param_dtype saved_meansaved_variancer.Z reserve_spacer=r=r>r[s=          r[normalized_shapeint | Sequence[int]cCst|j}t|}t|tjr|g}nt|trt|}n t|ts&tdt|}||} ||ks=tj || d|sUt |} td| d| dddt |t rct ||||| } | St|dgdd i} |g| d <|rz|g| d <|r|g| d <|| d } tdit}ddlm}||jdkr|jnd}|j|dd}|j|dd}||j}|jd| |||d|| d d||S)a" nn.LayerNorm is recommended. For more information, please refer to :ref:`api_paddle_nn_LayerNorm` . Parameters: x(Tensor): Input Tensor. It's data type should be bfloat16, float16, float32, float64. normalized_shape(int|list|tuple): Input shape from an expected input of size :math:`[*, normalized_shape[0], normalized_shape[1], ..., normalized_shape[-1]]`. If it is a single integer, this module will normalize over the last dimension which is expected to be of that specific size. weight(Tensor, optional): The weight tensor of layer_norm. Default: None. bias(Tensor, optional): The bias tensor of layer_norm. Default: None. epsilon(float, optional): The small value added to the variance to prevent division by zero. Default: 1e-05. name(str, optional): Name for the LayerNorm, default is None. For more information, please refer to :ref:`api_guide_Name` . Returns: None Examples: .. code-block:: python >>> import paddle >>> paddle.seed(2023) >>> x = paddle.rand((2, 2, 2, 3)) >>> layer_norm_out = paddle.nn.functional.layer_norm(x, x.shape[1:]) >>> print(layer_norm_out) Tensor(shape=[2, 2, 2, 3], dtype=float32, place=Place(cpu), stop_gradient=True, [[[[ 0.87799639, -0.32706568, -1.23529339], [ 1.01540327, -0.66222906, -0.72354043]], [[ 1.24183702, 0.45458138, -0.33506915], [ 0.41468468, 1.26852870, -1.98983312]]], [[[ 0.02837803, 1.27684665, -0.90110683], [-0.94709367, -0.15110941, -1.16546965]], [[-0.82010198, 0.11218392, -0.86506516], [ 1.09489357, 0.19107464, 2.14656854]]]]) z@`normalized_shape` should be int, list of ints or tuple of ints.NzGiven normalized_shape is z , expected input with shape [*, rz, but got input shape rS)rTr$r%r&Z LayerNormr*rYrZ)rbegin_norm_axis layer_normrr\r$r%Tr^r_rWrXr+)rp)listr r4 isinstancenumbersIntegraltupler5r0utilsZ is_same_shapestrrrrpr r r6rbr]r!r7r8rc)rrmrDrFrrZ input_shapeZ input_ndimZnormalized_ndimroZstr_normalized_shaper;r-r/r<r]rjrdreZlayer_norm_outr=r=r>rp@s /               rpTuse_input_statsr:%Literal['NC', 'NCL', 'NCHW', 'NCDHW']c Cstr t||||} | St|dgdd|||d} |r*|r*|g|g|gd} nd|gi} td it} | j|jdd }| j|jdd }| |j}|g|g|gd }| jd| || d |S)a It is recommended to use :ref:`api_paddle_nn_InstanceNorm1D` , :ref:`api_paddle_nn_InstanceNorm2D` , :ref:`api_paddle_nn_InstanceNorm3D` to call this method internally. Parameters: x (Tensor): Input Tensor. It's data type should be float32, float64. running_mean (Tensor, optional): running mean. Default None. Obsolete (that is, no longer usable). running_var (Tensor, optional): running variance. Default None. Obsolete (that is, no longer usable). weight (Tensor, optional): The weight tensor of instance_norm. Default: None. If its value is None, this parameter will be initialized by one. bias (Tensor, optional): The bias tensor of instance_norm. Default: None. If its value is None, this parameter will be initialized by zero. eps (float, optional): A value added to the denominator for numerical stability. Default is 1e-5. momentum (float, optional): The value used for the moving_mean and moving_var computation. Default: 0.9. use_input_stats (bool, optional): Default True. Obsolete (that is, no longer usable). data_format (str, optional): Specify the input data format, may be "NC", "NCL", "NCHW" or "NCDHW". Default "NCHW". name (str|None, optional): Name for the InstanceNorm, default is None. For more information, please refer to :ref:`api_guide_Name`.. Returns: None. Examples: .. code-block:: python >>> import paddle >>> paddle.seed(2023) >>> x = paddle.rand((2, 2, 2, 3)) >>> instance_norm_out = paddle.nn.functional.instance_norm(x) >>> print(instance_norm_out) Tensor(shape=[2, 2, 2, 3], dtype=float32, place=Place(cpu), stop_gradient=True, [[[[ 1.25768495, -0.18054862, -1.26451230], [ 1.42167914, -0.58056390, -0.65373862]], [[ 0.95882601, 0.25075224, -0.45947552], [ 0.21486834, 0.98283297, -1.94780385]]], [[[ 0.40697321, 1.90885782, -0.71117985], [-0.76650119, 0.19105314, -1.02920341]], [[-1.06926346, -0.18710862, -1.11180890], [ 0.74275863, -0.11246002, 1.73788261]]]]) rS)r%r&r$rTZ InstanceNorm)rrIrJ)r*rYrZr* instance_normTr^)r_r`rar+N)r{) rrr{r r r6r7r!r8)rrBrCrDrFryrIr:rJrr;r/r-r<rkrlZinstance_norm_outr.r=r=r>r{s@5  r{-C6???sizealphabetak*DataLayout1D | DataLayout2D | DataLayout3Dc Csts t|dddgd|dvrtd||j}t|}|dkr*td|d t|D]\} } | d ksD| d krDtd | d | q.|d dkrMdnd} d dlm} | dd|ddd} tj t ||dd}| sd d |d|ddg}|df}|d d|d|dt | |d|dg}d d d d |d|ddg}|ddf}n:|d|ddd d g}d|f}|d d|dt | |d|d |d g}|d|ddd d d d g}dd|f}|dkrtj j j||d}tj j j||dd}tj|dd}n&tj||d}tj j j||dd}tj j j||dd}ttj|dd|}tj|||d}t||}tj|||d}|S)a Local Response Normalization performs a type of "lateral inhibition" by normalizing over local input regions. For more information, please refer to `ImageNet Classification with Deep Convolutional Neural Networks `_ The formula is as follows: .. math:: Output(i, x, y) = Input(i, x, y) / \left(k + \alpha \sum\limits^{\min(C-1, i + size/2)}_{j = \max(0, i - size/2)}(Input(j, x, y))^2\right)^{\beta} In the above equation: - :math:`size` : The number of channels to sum over. - :math:`k` : The offset (avoid being divided by 0). - :math:`\\alpha` : The scaling parameter. - :math:`\\beta` : The exponent parameter. Args: x (Tensor): The input 3-D/4-D/5-D tensor. The data type is float16 or float32. size (int): The number of channels to sum over. alpha (float, optional): The scaling parameter, positive. Default:1e-4 beta (float, optional): The exponent, positive. Default:0.75 k (float, optional): An offset, positive. Default: 1.0 data_format (str, optional): Specify the data format of the input, and the data format of the output will be consistent with that of the input. An optional string from: If x is 3-D Tensor, the string could be `"NCL"` or `"NLC"` . When it is `"NCL"`, the data is stored in the order of: `[batch_size, input_channels, feature_length]`. If x is 4-D Tensor, the string could be `"NCHW"`, `"NHWC"`. When it is `"NCHW"`, the data is stored in the order of: `[batch_size, input_channels, input_height, input_width]`. If x is 5-D Tensor, the string could be `"NCDHW"`, `"NDHWC"` . When it is `"NCDHW"`, the data is stored in the order of: `[batch_size, input_channels, input_depth, input_height, input_width]`. name (str|None, optional): Name for the operation (optional, default is None). For more information, please refer to :ref:`api_guide_Name`. Returns: A tensor storing the transformation result with the same shape and data type as input. Examples: .. code-block:: python >>> import paddle >>> x = paddle.rand(shape=(3, 3, 112, 112), dtype="float32") >>> y = paddle.nn.functional.local_response_norm(x, size=5) >>> print(y.shape) [3, 3, 112, 112] rr$r%local_response_norm)rMrOrArPrNrQzNdata_format should be in one of [NCL, NCHW, NCDHW, NLC, NHWC, NDHWC], but got r z4Expected 3D or higher dimensionality input, but got z dimensionsrzCExpected every dim's size to be larger than 0, but the size of the z -th dim is r'rRTF)reducecSs||S)Nr=)ryr=r=r>qsz%local_response_norm..rN)rr)pad)Z kernel_sizeZstride)r rN)rrJ)scalerFr")rr r5r r4 enumerate functoolsrr0Z unsqueezemultiplyrnnZ functionalrZ avg_pool2dZsqueezeZreshapeZ avg_pool3drpowr3)rrrrrrJrsizesdimiszZ channel_lastrZ sum_sizesdivZ pad4d_shapeZ pool2d_shapeZ reshape_shapeZ pad5d_shapeZ pool3d_shaperesr=r=r>rs<       r num_groupsc Cs|dvr td||ddkrdnd}tr!t||||||Stdit}|j|jdd }|j|jdd } d |i} |d urE|| d <|d urM|| d <|j|jd} |jd| | || d|||dd| | S)a nn.GroupNorm is recommended. For more information, please refer to :ref:`api_paddle_nn_GroupNorm` . Parameters: x(Tensor): Input Tensor with shape: attr:`(batch, num_features, *)`. num_groups(int): The number of groups that divided from channels. epsilon(float, optional): The small value added to the variance to prevent division by zero. Default: 1e-05. weight(Tensor, optional): The weight Tensor of group_norm, with shape: attr:`[num_channels]`. Default: None. bias(Tensor, optional): The bias Tensor of group_norm, with shape: attr:`[num_channels]`. Default: None. data_format(str, optional): Specify the input data format. Support "NCL", "NCHW", "NCDHW", "NLC", "NHWC" or "NDHWC". Default: "NCHW". name(str|None, optional): Name for the GroupNorm, default is None. For more information, please refer to :ref:`api_guide_Name`.. Returns: Tensor, the output has the same shape with ``x``. Examples: .. code-block:: python >>> import paddle >>> paddle.seed(100) >>> x = paddle.arange(48, dtype="float32").reshape((2, 6, 2, 2)) >>> group_norm_out = paddle.nn.functional.group_norm(x, num_groups=6) >>> print(group_norm_out) Tensor(shape=[2, 6, 2, 2], dtype=float32, place=Place(cpu), stop_gradient=True, [[[[-1.34163547, -0.44721183], [ 0.44721183, 1.34163547]], [[-1.34163547, -0.44721183], [ 0.44721183, 1.34163547]], [[-1.34163547, -0.44721183], [ 0.44721183, 1.34163547]], [[-1.34163547, -0.44721183], [ 0.44721183, 1.34163547]], [[-1.34163547, -0.44721183], [ 0.44721183, 1.34163547]], [[-1.34163547, -0.44721183], [ 0.44721183, 1.34163547]]], [[[-1.34163547, -0.44721183], [ 0.44721183, 1.34163547]], [[-1.34163547, -0.44721183], [ 0.44721183, 1.34163547]], [[-1.34163547, -0.44721183], [ 0.44721183, 1.34163547]], [[-1.34163547, -0.44721183], [ 0.44721183, 1.34163547]], [[-1.34163547, -0.44721183], [ 0.44721183, 1.34163547]], [[-1.34163547, -0.44721183], [ 0.44721183, 1.34163547]]]]) )rMrArNrOrPrQzunsupported data layout:rrRrArP group_normTr^r*NrZrYr)rq)rgroupsrUr+)r) r5rrrr r6r7r!r8rc) rrrrDrFrJrr<rdrer-Zgroup_norm_outr=r=r>rsP?   r)rrrN) rrrrrrrrrrrr)NNFr?r@rANN)rBrrCrrDrErFrErGrHrIrrrrJrrKrLrrrr)NNr@N)rrrmrnrDrErFrErrrrrr) NNNNTr?r@rAN)rrrBrErCrErDrErFrEryrHrIrr:rrJrzrrrr)r|r}r~rAN)rrrrrrrrrrrJrrrrr)r@NNrAN)rrrrrrrDrErFrErJrrrrr)! __future__rrttypingrr0rrZpaddle.base.frameworkrrrZbase.data_feederr r Zbase.layer_helperr collections.abcr rrZpaddle._typingrrrr__all__r#r[rpr{rrr=r=r=r>sr       d 6 { c