o + i8@sddlmZddlZddlmZmZddlmZmZmZddl m Z ddl m Z m Z mZdd lmZdd lmZgZGd d d eZGd ddeZGdddeZGdddeZdS)) annotationsN)_C_opspir)core framework unique_name)check_variable_and_dtype)_current_expected_placein_dygraph_mode in_pir_mode) Initializer)lazy_init_helpercs4eZdZdZ ddfd d Z ddddZZS)NormalInitializeraAImplements the Random Normal(Gaussian) distribution initializer Args: loc (float|complex, optional): mean of the normal distribution. Default is 0.0. scale (float, optional): standard deviation of the normal distribution. Default is 1.0. seed (int, optional): random seed. Default is 0. ?rlocfloatscaleseedintreturnNonecs|dusJ|dus J|dusJt||_||_||_t|jtrC|jj|jjkr>> import paddle >>> data = paddle.ones(shape=[3, 1, 2], dtype='float32') >>> weight_attr = paddle.framework.ParamAttr( ... name="linear_weight", ... initializer=paddle.nn.initializer.Normal(mean=0.0, std=2.0)) >>> bias_attr = paddle.framework.ParamAttr( ... name="linear_bias", ... initializer=paddle.nn.initializer.Normal(mean=0.0, std=2.0)) >>> # doctest: +SKIP('name has been used') >>> linear = paddle.nn.Linear(2, 2, weight_attr=weight_attr, bias_attr=bias_attr) >>> print(linear.weight) Parameter containing: Tensor(shape=[2, 2], dtype=float32, place=Place(cpu), stop_gradient=False, [[ 2.1973135 -2.2697184], [-1.9104223 -1.0541488]]) >>> print(linear.bias) Parameter containing: Tensor(shape=[2], dtype=float32, place=Place(cpu), stop_gradient=False, [ 0.7885926 -0.74719954]) >>> res = linear(data) >>> print(res) Tensor(shape=[3, 1, 2], dtype=float32, place=Place(cpu), stop_gradient=False, [[[ 1.0754838 -4.071067 ]], [[ 1.0754838 -4.071067 ]], [[ 1.0754838 -4.071067 ]]]) rrNr1rr2name str | Nonerrcs6|dusJd|dusJdtj||dddS)Nmean should not be Nonestd should not be Noner)rrrrr)r$r1r2rIr%r'r(rszNormal.__init__)rrN)r1rr2rrIrJrrrCrDrErFrrGr'r'r%r(rHs+rHcs<eZdZdZ     ddfdd Z ddddZZS)TruncatedNormalInitializera|Implements the Random TruncatedNormal(Gaussian) distribution initializer Note: It is better to set `a <= mean <= b`. If `mean < a - 2*std` or `mean > b + 2*std`, the distribution of values may be incorrect. Args: loc (float, optional): Mean of the normal distribution. Default is :math:`0.0`. scale (float, optional): Standard deviation of the normal distribution. Default is :math:`1.0`. seed (int, optional): random seed. Default is 0. a (float, optional): The minimum cutoff value. Default is -2.0. b (float, optional): The maximum cutoff value. Default is 2.0. rrr@rrrrrabrrcsh|dusJ|dus J|dusJ|dusJ|dusJt||_||_||_||_||_dSr@)rrrrr_a_b)r$rrrrRrSr%r'r(rs       z#TruncatedNormalInitializer.__init__Nr)r*r+r,r-c Cs&||}tjrtjtjjjtj j f}n tjtjj tjjjf}t ||s'Jt |tj tj fs2J|jdkr<|jj|_|jtjjjtjjjfvrgtjjj}|jtdd|jdg|j|tjjjdd}n|j}|}trt |j|j!|j"|j|j#|j$|t%}|jtjjjtjjjfvrt&||j}|'|dS|'|dSt(rt |j|j!|j"|j|j#|j$|t%}|jtjjjtjjjfvrt&||j}|'||S|j)dd|i|j||j!|j"|j|j#|j$d d d }|jtjjjtjjjfvr|j)d d |id|i|j|jdd||_*|S)adInitialize the input tensor with TruncatedNormal distribution. Args: var(Tensor): Tensor that needs to be initialized. block(Block|None, optional): The block in which initialization ops should be added. Used in static graph only, default None. Returns: The initialization op r.truncated_gaussian_randomtmpF)rIr/r0r4Z persistableNr.)r/r0r1r2rrRrSTr3castX)Zin_dtype out_dtype)r4Zinputsr5r6)+r7rstaterVariablepaddlerrZ ParameterMetaeagerZTensorValuerr8rr9r:r0ZVarDescZVarTypeZFP16ZBF16ZFP32Z create_varrgeneratejoinrIr/Z DENSE_TENSORr rrWrrrTrUr rYr;r r<r=)r$r)r+expectedr[r>Zvar_tmpr=r'r'r(r?s            z"TruncatedNormalInitializer.forward)rrrrPrQ) rrrrrrrRrrSrrrr@rArBr'r'r%r(rOsrOcs.eZdZdZ     ddfdd ZZS)TruncatedNormalaThe truncated normal distribution (Gaussian distribution) initializer. Note: It is better to set `a <= mean <= b`. If `mean < a - 2*std` or `mean > b + 2*std`, the distribution of values may be incorrect. Args: mean (float, optional): Mean of the normal distribution. Default is :math:`0.0`. std (float, optional): Standard deviation of the normal distribution. Default is :math:`1.0`. a (float, optional): The minimum cutoff value. Default is -2.0. b (float, optional): The maximum cutoff value. Default is 2.0. name (str|None, optional): For details, please refer to :ref:`api_guide_Name`. Generally, no setting is required. Default: None. Returns: A parameter initialized by truncated normal distribution (Gaussian distribution). Examples: .. code-block:: python >>> import paddle >>> data = paddle.ones(shape=[3, 1, 2], dtype='float32') >>> weight_attr = paddle.framework.ParamAttr( ... name="linear_weight", ... initializer=paddle.nn.initializer.TruncatedNormal(mean=0.0, std=2.0)) >>> bias_attr = paddle.framework.ParamAttr( ... name="linear_bias", ... initializer=paddle.nn.initializer.TruncatedNormal(mean=0.0, std=2.0)) >>> # doctest: +SKIP('name has been used') >>> linear = paddle.nn.Linear(2, 2, weight_attr=weight_attr, bias_attr=bias_attr) >>> print(linear.weight) Parameter containing: Tensor(shape=[2, 2], dtype=float32, place=Place(cpu), stop_gradient=False, [[-1.0981836 1.4140984], [ 3.1390522 -2.8266568]]) >>> print(linear.bias) Parameter containing: Tensor(shape=[2], dtype=float32, place=Place(cpu), stop_gradient=False, [ -2.1546738 -1.6570673]) >>> res = linear(data) >>> print(res) Tensor(shape=[3, 1, 2], dtype=float32, place=Place(cpu), stop_gradient=False, [[[-0.11380529 -3.0696259 ]], [[-0.11380529 -3.0696259 ]], [[-0.11380529 -3.0696259 ]]]) rrrPrQNr1rr2rRrSrIrJrrcsZ|dusJd|dusJd|dusJd|dus Jdtj||d||ddS)NrKrLza should not be Nonezb should not be Noner)rrrrRrSrM)r$r1r2rRrSrIr%r'r(rs zTruncatedNormal.__init__)rrrPrQN) r1rr2rrRrrSrrIrJrrrNr'r'r%r(rdXs1rd) __future__rr^rrbaserrrZbase.data_feederr Zbase.frameworkr r r Z initializerrZ lazy_initr__all__rrHrOrdr'r'r'r(s    i3