o + i38@sddlmZddlZddlmZddlZddlmZddlmZm Z m Z ddl m Z m Z mZdd lmZmZer>dd lmZgZGd d d eZGd ddeZGdddeZdS)) annotationsN) TYPE_CHECKING)_C_ops)core framework unique_name)_current_expected_placein_dygraph_mode in_pir_mode) Initializercalculate_gain) _NonLinearitycs>eZdZdZ      ddfdd Z dd ddZZS)!MSRAInitializera]Implements the MSRA initializer a.k.a. Kaiming Initializer This class implements the weight initialization from the paper `Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification `_ by Kaiming He, Xiangyu Zhang, Shaoqing Ren and Jian Sun. This is a robust initialization method that particularly considers the rectifier nonlinearities. In case of Uniform distribution, the range is [-x, x], where .. math:: x = gain \times \sqrt{\frac{3}{fan\_in}} In case of Normal distribution, the mean is 0 and the standard deviation is .. math:: \frac{gain}{\sqrt{{fan\_in}}} Args: uniform (bool, optional): whether to use uniform or normal distribution. Default is True. fan_in (float32|None, optional): fan_in (in_features) of trainable Tensor, If None, it will be inferred automatically. If you don't want to use in_features of the Tensor, you can set the value of 'fan_in' smartly by yourself. Default is None. seed (int32, optional): random seed. Default is 0. negative_slope (float, optional): negative_slope (only used with leaky_relu). Default is 0.0. nonlinearity(str, optional): the non-linear function. Default is relu. mode(str, optional): the mode of initialization, can be 'fan_in' or 'fan_out'. When set to 'fan_in', the fan_in parameter is used for initialization. When set to 'fan_out', the out_features of trainable Tensor will be used. Default is 'fan_in'. Note: It is recommended to set fan_in to None for most cases. TNrrelufan_inuniformbool float | Noneseedintnegative_slopefloat nonlinearityrmodestrreturnNonecs|dusJ|dus Jt||_||_||_||_||_||_|jdvr1td|jd|jdkr?|jdurAtddSdS)zConstructor for MSRAInitializerN)rfan_outzWThe mode of KaimingNormal/KaimingUniform should be 'fan_in' or 'fan_out', but received .rzdThe mode of KaimingNormal/KaimingUniform is 'fan_out', but fan_in is set. Please set fan_in to None.) super__init___uniform_fan_in_seed_negative_slope _nonlinearity_mode ValueError)selfrrrrrr __class__i/home/app/PaddleOCR-VL-test/.venv_paddleocr/lib/python3.10/site-packages/paddle/nn/initializer/kaiming.pyr"Hs(   zMSRAInitializer.__init__var paddle.Tensorblockpaddle.pir.Block | Nonepaddle.Tensor | Nonec Cst|tjr|rJd||}t|tjtjjtjj j fs#Jt|tj tjj fs/J| |\}}|j dkrE|jdurB|n|j}|j dkrL|}|jdkrV|jj|_|j}|t jjjksj|t jjjkr|jst jjj}|jtdd|jdg|j|t jjjd d }n|t j j!t j j"fvr|jst j j#}|}n|}|}t$r|jrt%|j&|j'} | t()d t*|} t+,|j|| | |jt-}nt%|j&|j'} | t()t*|} t-} t+.|jd | |j|| }|t jjjks|t jjjt j j!t j j"fvr|jst+/||} | 0|dS|0|dSt1ru|jr?t%|j&|j'} | t()d t*|} t+,|j|| | |jt-}nt%|j&|j'} | t()t*|} t-} t+.|jd | |j|| }|t j j!t j j"fvrs|jsst+/||S|S|jrt%|j&|j'} | t()d t*|} |j2d id|i|jt3|| | |jddd}n%t%|j&|j'} | t()t*|} |j2dd|i|jt3|d | |jddd}|t jjjks|t jjjkr|js|j2dd|id|i|j|dd||_4|S)a\Initialize the input tensor with MSRA initialization. Args: var(Tensor): Tensor that needs to be initialized. block(Block|None, optional): The block in which initialization ops should be added. Used in static graph only, default None. Returns: The initialization op. zDCurrently, kaiming initializer not support lazy init for dist param.rNrrr Z masra_inittmpF)nameshapedtypetypeZ persistableg@Zuniform_randomZOut)r6r7minmaxrT)r8inputsoutputsattrs stop_gradientZgaussian_random)r6r7meanstdr)r8r=r>r?castX)Zin_dtype out_dtype)r8r<r=r>)5 isinstancerZEagerParamBaseZis_distZ _check_blockVariablepaddleZpirValuerZ ParameterMetaZBlockZ _compute_fansr(r$r%programZ random_seedr7ZVarDescZVarTypeZFP16ZBF16r#ZFP32Z create_varrgeneratejoinr5r6Z DENSE_TENSORZDataTypeZFLOAT16ZBFLOAT16ZFLOAT32r rr'r&mathsqrtrrrr ZgaussianrBZ_share_underline_tensor_tor Z append_oprop)r*r/r1Zf_inZf_outrZ origin_dtyperDZout_varZgainlimitrAZplaceZvar_tmprNr-r-r.forwardfs                zMSRAInitializer.forward)TNrrrr)rrrrrrrrrrrrrr)N)r/r0r1r2rr3)__name__ __module__ __qualname____doc__r"rP __classcell__r-r-r+r.r&s#rc,eZdZdZ    ddfdd ZZS) KaimingNormala_Implements the Kaiming Normal initializer This class implements the weight initialization from the paper `Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification `_ by Kaiming He, Xiangyu Zhang, Shaoqing Ren and Jian Sun. This is a robust initialization method that particularly considers the rectifier nonlinearities. In case of Normal distribution, the mean is 0 and the standard deviation is .. math:: \frac{gain}{\sqrt{{fan\_in}}} Args: fan_in (float32|None, optional): fan_in (in_features) of trainable Tensor, If None, it will be inferred automatically. If you don't want to use in_features of the Tensor, you can set the value of 'fan_in' smartly by yourself. Default is None. negative_slope (float, optional): negative_slope (only used with leaky_relu). Default is 0.0. nonlinearity(str, optional): the non-linear function. Default is relu. mode(str, optional): the mode of initialization, can be 'fan_in' or 'fan_out'. When set to 'fan_in', the fan_in parameter is used for initialization. When set to 'fan_out', the out_features of trainable Tensor will be used. Default is 'fan_in'. Note: It is recommended to set fan_in to None for most cases. Examples: .. code-block:: python >>> import paddle >>> import paddle.nn as nn >>> linear = nn.Linear(2, 4, weight_attr=nn.initializer.KaimingNormal()) >>> data = paddle.rand([30, 10, 2], dtype='float32') >>> res = linear(data) Nr9rrrrrrrrrrctjd|d|||ddS)NFrrrrrrrr!r"r*rrrrr+r-r.r"< zKaimingNormal.__init__Nr9rr rrrrrrrrrrrQrRrSrTr"rUr-r-r+r.rWs'rWcrV)KaimingUniformaYImplements the Kaiming Uniform initializer This class implements the weight initialization from the paper `Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification `_ by Kaiming He, Xiangyu Zhang, Shaoqing Ren and Jian Sun. This is a robust initialization method that particularly considers the rectifier nonlinearities. In case of Uniform distribution, the range is [-x, x], where .. math:: x = gain \times \sqrt{\frac{3}{fan\_in}} Args: fan_in (float32|None, optional): fan_in (in_features) of trainable Tensor, If None, it will be inferred automatically. If you don't want to use in_features of the Tensor, you can set the value of 'fan_in' smartly by yourself. Default is None. negative_slope (float, optional): negative_slope (only used with leaky_relu). Default is 0.0. nonlinearity(str, optional): the non-linear function. Default is relu. mode(str, optional): the mode of initialization, can be 'fan_in' or 'fan_out'. When set to 'fan_in', the fan_in parameter is used for initialization. When set to 'fan_out', the out_features of trainable Tensor will be used. Default is 'fan_in'. Note: It is recommended to set fan_in to None for most cases. Examples: .. code-block:: python >>> import paddle >>> import paddle.nn as nn >>> linear = nn.Linear(2, 4, weight_attr=nn.initializer.KaimingUniform()) >>> data = paddle.rand([30, 10, 2], dtype='float32') >>> res = linear(data) Nr9rrrrrrrrrrcrX)NTrrYrZr[r+r-r.r"rr\zKaimingUniform.__init__r]r^r_r-r-r+r.r`Ms&r`) __future__rrLtypingrrGrbaserrrZbase.frameworkr r r Z initializerr rr__all__rrWr`r-r-r-r.s    q7