o 1 i@s ddlZddlmZmZmZmZmZmZddlm Z ddl m Z m Z ddl mZmZeeZedZedZeeee e gee feee e e gee fedee ffZeGd d d ZeGd d d eZeGd ddeZdeeefdefddZdS)N)AnyCallableIterableOptionalTypeVarUnion) TaskContext)BlockUserDefinedFunction) DeveloperAPI PublicAPITU.c@s eZdZdS)ComputeStrategyN)__name__ __module__ __qualname__rrf/home/app/PaddleOCR-VL-test/.venv_paddleocr/lib/python3.10/site-packages/ray/data/_internal/compute.pyrsrc@sFeZdZdZ d deefddZdedefdd Z de fd d Z dS) TaskPoolStrategya;Specify the task-based compute strategy for a Dataset transform. TaskPoolStrategy executes dataset transformations using Ray tasks that are scheduled through a pool. Provide ``size`` to cap the number of concurrent tasks; leave it unset to allow Ray Data to scale the task count automatically. NsizecCs$|dur |dkr td|||_dS)zConstruct TaskPoolStrategy for a Dataset transform. Args: size: Specify the maximum size of the task pool. Nz`size` must be >= 1) ValueErrorr)selfrrrr__init__)s  zTaskPoolStrategy.__init__otherreturncCs(t|tr |j|jkp|dko|jduS)Ntasks) isinstancerrrrrrr__eq__7szTaskPoolStrategy.__eq__cCsd|jdS)NzTaskPoolStrategy(size=))rrrrr__repr__<szTaskPoolStrategy.__repr__N rrr__doc__rintrrboolr strr#rrrrrs  rc @speZdZdZdddddddeedeedeedeedeef d d Zd ed efd dZ d e fddZ dS)ActorPoolStrategya{Specify the actor-based compute strategy for a Dataset transform. ActorPoolStrategy specifies that an autoscaling pool of actors should be used for a given Dataset transform. This is useful for stateful setup of callable classes. For a fixed-sized pool of size ``n``, use ``ActorPoolStrategy(size=n)``. To autoscale from ``m`` to ``n`` actors, use ``ActorPoolStrategy(min_size=m, max_size=n)``. To autoscale from ``m`` to ``n`` actors, with an initial size of ``initial``, use ``ActorPoolStrategy(min_size=m, max_size=n, initial_size=initial)``. To increase opportunities for pipelining task dependency prefetching with computation and avoiding actor startup delays, set max_tasks_in_flight_per_actor to 2 or greater; to try to decrease the delay due to queueing of tasks on the worker actors, set max_tasks_in_flight_per_actor to 1. N)rmin_sizemax_size initial_sizemax_tasks_in_flight_per_actorrr+r,r-r.cCs6|dur#|dkr td||dus|dus|durtd|}|}|}|dur0|dkr0td||durD|dur:d}||krDtd|||durQ|dkrQtd||pTd|_|p[td|_|dur||jkrrtd |d |jd |jtdkr||jkrtd |d |jd |p|j|_||_d |_d|_dS)aaConstruct ActorPoolStrategy for a Dataset transform. Args: size: Specify a fixed size actor pool of this size. It is an error to specify both `size` and `min_size` or `max_size`. min_size: The minimum size of the actor pool. max_size: The maximum size of the actor pool. initial_size: The initial number of actors to start with. If not specified, defaults to min_size. Must be between min_size and max_size. max_tasks_in_flight_per_actor: The maximum number of tasks to concurrently send to a single actor worker. Increasing this will increase opportunities for pipelining task dependency prefetching with computation and avoiding actor startup delays, but will also increase queueing delay. Nrzsize must be >= 1zMmin_size, max_size, and initial_size cannot be set at the same time as `size`zmin_size must be >= 1zmin_size must be <= max_sizez1max_tasks_in_flight_per_actor must be >= 1, got: infzinitial_size (z) must be >= min_size (r!z) must be <= max_size (rg?)rr+floatr,r-r. num_workersready_to_total_workers_ratio)rrr+r,r-r.rrrrVsL       zActorPoolStrategy.__init__rrcCs:t|to|j|jko|j|jko|j|jko|j|jkSr$)rr*r+r,r-r.rrrrr s    zActorPoolStrategy.__eq__c Cs6d|jd|jd|jd|jd|jd|jd S)NzActorPoolStrategy(min_size=z , max_size=z, initial_size=z , max_tasks_in_flight_per_actor=z )num_workers=z, ready_to_total_workers_ratio=r!)r+r,r-r.r1r2r"rrrr#s zActorPoolStrategy.__repr__r%rrrrr*@s( E r* compute_specrcCsTt|ttfstd|d|r|dkrtS|dkrtSt|tr&|Std)NzXIn Ray 2.5, the compute spec must be either TaskPoolStrategy or ActorPoolStrategy, was: .rZactorsz;compute must be one of [`tasks`, `actors`, ComputeStrategy])rrr*rr)r3rrr get_computes  r5)loggingtypingrrrrrrZ'ray.data._internal.execution.interfacesrZray.data.blockr r Zray.util.annotationsr r getLoggerrloggerr rZBlockTransformrrr*r)r5rrrrs*     n