o * iH@s<ddlZddlmZmZgZ d ddZddZdd ZdS) N) get_clusterloggerc std}|dus Jdtd}|dusJdtd}|dus'Jdttd}|dus6Jd |d }t|}t|}|d krZ|d |krZtd |d |dtd} | dur|} |dkrzttdd} |t|kr| |krtd| d| } Wnty} z t | WYd} ~ nd} ~ ww| durd} t t | | t|} g} |D]| fdd| Dqn'| d }g} ||t|ksJt |D]}| ||||d|qt d|d|d|d| t||| ||\}}||j|fS)za args_node_ips:string, device_mode:DeviceMode(Int), device_per_proc:list, args_port: int PADDLE_TRAINERSNz"PADDLE_TRAINERS should not be NonePOD_IPzPOD_IP should not be NonePADDLE_TRAINER_IDz$PADDLE_TRAINER_ID should not be NoneTRAINER_PORTS_NUMz$TRAINER_PORTS_NUM should not be None,z 127.0.0.1zPlease NOTE: When using paddlecloud, cluster_node_ips is automatically got from PADDLE_TRAINERS(multi nodes) or POD_IP(single node).Your input cluster_node_ips: z doesn't equals to IPs: z from paddlecloud environment.ZDISTRIBUTED_TRAINER_ENDPOINTSZ PADDLE_PORTzUse Cloud specified port:.rcsg|] }d|qS):).0portiprp/home/app/PaddleOCR-VL-test/.venv_paddleocr/lib/python3.10/site-packages/paddle/distributed/fleet/cloud_utils.py Msz%get_cloud_cluster..zparsed from args: node_ips:z node_ip:z node_rank:z trainer_endpoints:)osgetenvintsplitlenjoinrwarning ExceptionprintlistrangeappenddebugrZpods)Z args_node_ipsZ device_modeZdevices_per_procZ args_portnode_ipsnode_ip node_rankpaddle_ports_numZ num_nodesZtrainer_endpointsZ started_portZ paddle_porteZportsZtrainer_endpoints_oriiZclusterZpodrrrget_cloud_clusters~         r(cCsPtd}td}td}td}|dus$|dus$|dus$|dur&dSdS)NrrrrFT)rr)r"r#r$r%rrruse_paddlecloudds    r)cCsttddS)NZPADDLE_TRAINERS_NUM1)rrrrrrrget_trainers_numtsr+)r)rZ%paddle.distributed.fleet.launch_utilsrr__all__r(r)r+rrrrs N