o )iW@sVddlmZmZddlmZddlZddlmZedddddZ eGd d d Z dS) ) dataclassfield)OptionalN) InputBatchggg?) temperaturemin_ptop_ktop_pc @s.eZdZUdZejed<dZejed<dZejed<dZ ejed<dZ e ed<dZ e ed <dZ e ed <dZdZdZdZed d d Zeeeed<dZedd d Zeeeeefed<dZdZedd d Zeeejfed<edeeejffddZ e! dde"dedej#de ddf ddZ$dS)TPUSupportedSamplingMetadataNrrrr T all_greedyFlogprobs no_penaltiescCtSNlistrrg/home/app/PaddleOCR-VL-test/.venv_paddleocr/lib/python3.10/site-packages/vllm/v1/sample/tpu/metadata.py/z%TPUSupportedSamplingMetadata.)default_factoryoutput_token_idscCrrrrrrrr4r logit_biascCrr)dictrrrrr;r _generatorsreturncCs|jSr)r)selfrrr generators=sz'TPUSupportedSamplingMetadata.generators input_batchpadded_num_reqs xla_devicegenerate_params_if_all_greedycs|jr|jdknd}|jdur|dur|d|dS|jdtjdtjffdd }||jtd ||jtd ||jtd ||j td ||jd  ||j|j d  ||jd  ||jd  ||dS)a/ Copy sampling tensors slices from `input_batch` to on device tensors. `InputBatch._make_sampling_metadata` causes recompilation on XLA as it slices dynamic shapes on device tensors. This impl moves the dynamic ops to CPU and produces tensors of fixed `padded_num_reqs` size. Args: input_batch: The input batch containing sampling parameters. padded_num_reqs: The padded number of requests. xla_device: The XLA device. generate_params_if_all_greedy: If True, generate sampling parameters even if all requests are greedy. this is useful for cases where we want to pre-compile a graph with sampling parameters, even if they are not strictly needed for greedy decoding. rFT)r r cpu_tensorrcs||<dSrr)r"Zfill_valnum_reqsrrr fill_slicecszATPUSupportedSamplingMetadata.from_input_batch..fill_slicerrrr N)rr r rrr ) Zmax_num_logprobsr r$torchTensorZtemperature_cpu_tensorDEFAULT_SAMPLING_PARAMSZmin_p_cpu_tensorZtop_k_cpu_tensorZtop_p_cpu_tensorto)clsrrr r!Zneeds_logprobsr%rr#rfrom_input_batchBsH    z-TPUSupportedSamplingMetadata.from_input_batch)F)%__name__ __module__ __qualname__rr&r'__annotations__rrr r boolr r Zprompt_token_idsZfrequency_penaltiesZpresence_penaltiesZrepetition_penaltiesrrrintZ min_tokensrrrfloatZallowed_token_ids_maskZbad_words_token_idsr Generatorpropertyr classmethodrZdevicer+rrrrr sL      r ) dataclassesrrtypingrr&Zvllm.v1.worker.tpu_input_batchrrr(r rrrrs