o ñp iÑã@sRddlZddlmZddlmZddlmZmZddlm Z d dd„Z d d „Z dS) éN)Úfleet)Úin_dynamic_modeé)ÚHeterParallelOptimizerÚHybridParallelOptimizer)ÚloggercCsªtj}||_|dur|jrt d¡t |¡|_i|_|  ¡dkrS|jj sMt ||j |jƒ}|jj djr:d|_d|_|jj djrKd|_|jrKJdƒ‚|St||jƒS|S)aÔ Optimizer for distributed training. For the distributed training, this method would rebuild a new instance of DistributedOptimizer. Which has basic Optimizer function and special features for distributed training. Args: optimizer(Optimizer): The executor to run for init server. strategy(DistributedStrategy): Extra properties for distributed optimizer. It is recommended to use DistributedStrategy in fleet.init(). The strategy here is for compatibility. If the strategy in fleet.distributed_optimizer() is not None, then it will overwrite the DistributedStrategy in fleet.init(), which will take effect in distributed training. Returns: Fleet: instance of fleet. Examples: .. code-block:: python >>> import paddle >>> import paddle.distributed.fleet as fleet >>> fleet.init(is_collective=True) >>> strategy = fleet.DistributedStrategy() >>> linear = paddle.nn.Linear(10, 10) >>> optimizer = paddle.optimizer.SGD(learning_rate=0.001, parameters=linear.parameters()) >>> optimizer = fleet.distributed_optimizer(optimizer, strategy=strategy) Na$It is recommended to use DistributedStrategy in fleet_env.init(). The strategy here is only for compatibility. If the strategy in fleet_env.distributed_optimizer() is not None, then it will overwrite the DistributedStrategy in fleet_env.init(), which will take effect in distributed training.rZ pp_configsFz7sep parallel can not coexist with sharding_comm_overlap)rZuser_defined_optimizerZ_is_collectiverÚwarningÚcopyÚdeepcopyZ_user_defined_strategyÚ_contextZ worker_numZheter_ccl_moderZ_hcgZhybrid_configsZdp_comm_overlapZ _dp_enableZ _sep_enableZsharding_comm_overlapZ_sharding_enabler)Z optimizerZstrategyZ fleet_envZhp_optim©r úi/home/app/PaddleOCR-VL/.venv_paddleocr/lib/python3.10/site-packages/paddle/distributed/fleet/optimizer.pyÚ_dygraph_distributed_optimizersFÿ   ÿÿþÿþÿÿrcOs&tƒr t|i|¤ŽStjj|i|¤ŽS©N)rrrÚdistributed_optimizer)ÚargsÚkwargsr r r r`srr) r Zpaddle.distributedrZpaddle.frameworkrZmeta_optimizersrrZutils.log_utilrrrr r r r Ús     H