o ü* iYã@s¦ddlmZddlmZmZddlZddlZddlmZddl m Z ddl m Z m Z erBddlmZdd lmZdd lmZed ƒZ  d d!dd„Z d"d#dd„ZdS)$é)Ú annotations)Ú TYPE_CHECKINGÚTypeVarN)Ú framework)Ústreamé)Úconvert_object_to_tensorÚconvert_tensor_to_object)ÚTensor)Útask)ÚGroupÚ_TTÚ tensor_listú list[Tensor]Útensorr Úgroupú Group | NoneÚsync_opÚboolÚreturnú task | NonecCst ||||¡S)aÊ Gather tensors from all participators and all get the result. As shown below, one process is started with a GPU and the data of this process is represented by its group rank. Through the all_gather operator, each GPU will have data from all GPUs. .. image:: https://githubraw.cdn.bcebos.com/PaddlePaddle/docs/develop/docs/api/paddle/distributed/img/allgather.png :width: 800 :alt: all_gather :align: center Args: tensor_list (list): A list of output Tensors. Every element in the list must be a Tensor whose data type should be float16, float32, float64, int32, int64, int8, uint8, bool, bfloat16, complex64 or complex128. tensor (Tensor): The Tensor to send. Its data type should be float16, float32, float64, int32, int64, int8, uint8, bool, bfloat16, complex64 or complex128. group (Group|None, optional): The group instance return by new_group or None for global default group. sync_op (bool, optional): Whether this op is a sync op. The default value is True. Returns: None. Examples: .. code-block:: python >>> # doctest: +REQUIRES(env: DISTRIBUTED) >>> import paddle >>> import paddle.distributed as dist >>> dist.init_parallel_env() >>> tensor_list = [] # type: ignore >>> if dist.get_rank() == 0: ... data = paddle.to_tensor([[4, 5, 6], [4, 5, 6]]) >>> else: ... data = paddle.to_tensor([[1, 2, 3], [1, 2, 3]]) >>> dist.all_gather(tensor_list, data) >>> print(tensor_list) >>> # [[[4, 5, 6], [4, 5, 6]], [[1, 2, 3], [1, 2, 3]]] (2 GPUs) )rÚ all_gather)rrrr©rúw/home/app/PaddleOCR-VL-test/.venv_paddleocr/lib/python3.10/site-packages/paddle/distributed/communication/all_gather.pyr&s.rÚ object_listúlist[_T]Úobjr ÚNonec Cs–t ¡sJdƒ‚t|ƒ\}}g}t|||ƒtt|ƒ ¡ƒ}| ¡}t  ||g¡}t   |¡}g} t| ||ƒt | ƒD]\} }|  t||| ƒ¡q:dS)aH Gather picklable objects from all participators and all get the result. Similar to all_gather(), but python object can be passed in. Args: object_list (list): A list of output object. The datatype of every element in the list is same as the input obj. obj (Any): The picklable object to send. group (Group): The group instance return by new_group or None for global default group. Returns: None. Warning: This API only supports the dygraph mode. Examples: .. code-block:: python >>> # doctest: +REQUIRES(env: DISTRIBUTED) >>> import paddle >>> import paddle.distributed as dist >>> dist.init_parallel_env() >>> object_list = [] # type: ignore >>> if dist.get_rank() == 0: ... obj = {"foo": [1, 2, 3]} >>> else: ... obj = {"bar": [4, 5, 6]} >>> dist.all_gather_object(object_list, obj) >>> print(object_list) >>> # [{'foo': [1, 2, 3]}, {'bar': [4, 5, 6]}] (2 GPUs) z4all_gather_object doesn't support static graph mode.N)rZin_dynamic_moderrÚintÚmaxÚitemÚnumpyÚnpÚresizeÚpaddleZ to_tensorÚ enumerateÚappendr ) rrrrZ len_of_tensorZlist_len_of_tensorZmax_len_of_tensorZ numpy_dataZ input_tensorrÚirrrÚall_gather_objectWs" #ÿ     ÿÿr()NT) rrrr rrrrrr)N)rrrr rr rr)Ú __future__rÚtypingrrr!r"r$rZ paddle.distributed.communicationrZserialization_utilsrr r Zpaddle.base.corer Z&paddle.distributed.communication.groupr r rr(rrrrÚs"       ü2ÿ