o )iJ@sddlZddlmZddlmZddlmZddlmZddl m Z m Z m Z ddl mZmZddlmZmZdd lmZmZeeZedZeGd d d ZdS) N)Iterable) dataclass)Optional) init_logger)LogprobPromptLogprobsSampleLogprobs) AnyTokenizerconvert_ids_list_to_tokens)EngineCoreOutputEngineCoreRequest) LogprobsListsLogprobsTensorsc@seZdZUeeed<eeed<eeed<eeed<ee ed<ee ed<e deede ddfd d Z d e dd fd dZdedd fddZdeefddZedeedee deeede de dee eff ddZdedd fddZd S)LogprobsProcessor tokenizerlogprobsprompt_logprobscumulative_logprob num_logprobsnum_prompt_logprobsrequestreturncCsX|jdusJ|jj}|jj}|||durdnd|durdng|dur%dndg||dS)Ng)rrrrrr)Zsampling_paramsrr)clsrrrrrc/home/app/PaddleOCR-VL-test/.venv_paddleocr/lib/python3.10/site-packages/vllm/v1/engine/logprobs.pyfrom_new_request#sz"LogprobsProcessor.from_new_requestlogprobs_listsNc Cs|jdusJ|jdusJ|jdusJ|\}}}t|||D]+\}}}|jdur,tnt|j|}|d} |j| 7_|j||||||jq dS)aUpdate with sample logprobs from EngineCore. Outer lists are only of len > 1 if EngineCore made >1 tokens in prior step (e.g. in spec decoding). Args: logprobs_lists: the lists of logprob tokens, logprobs, and ranks. Nr) rrrziprNONESr append_make_logprob_dict) selfrZ token_ids_lstZ logprobs_lstZ ranks_lstrankr token_idsdecoded_tokensZsampled_token_logprobrrr_update_sample_logprobs6s,   z)LogprobsProcessor._update_sample_logprobsprompt_logprobs_tensorsc Cs|jdusJ|jdusJ|\}}}|jdurdn t|j|}|j\}}|}|} |}t|D]*} | |} | |} |durItn|| | } |j | | | || | || |jq9dS)zUpdate with prompt logprobs from EngineCore. Args: prompt_logprobs_tensors: tuple containing the prompt logprobs tensors. N) rrrr flattentolistshaperangerrr )r!r&r#rranksr$Znum_prompt_tokensrZprompt_token_ranksrposoffsetZ offset_endZdecoded_tokens_for_posrrr_update_prompt_logprobs\s6      z)LogprobsProcessor._update_prompt_logprobscCs|j}|rg|_|S)aPop and return all request prompt logprobs The logprobs processor aggregates prompt chunk logprobs over one or more prefill chunks. This method returns all prompt logprobs at once and then forgets them. Ensures correct RequestOutputKind.DELTA semantics wherein all prompt logprobs are returned at once at the end of prefill. Returns: None if prompt logprobs are disabled for this request. List of all prompt logprobs, otherwise. )r)r!Zplprrrpop_prompt_logprobssz%LogprobsProcessor.pop_prompt_logprobslogprob_token_idsr$r"cCsD|dkrt|}td|d}t|f|}ddt||||DS)aMake a Logprob dictionary for a position. Args: logprobs: list of log probabilities logprob_token_ids: list of top token ids decoded_tokens: list of decoded top tokens rank: rank of the sampled token num_logprobs: number of logprobs requested by the user (in addition to sampled logprob) Returns: dict[token id, Logprob] cSs$i|]\}}}}|t|||dqS))logprobr"Z decoded_token)r).0Ztoken_idr3r"tokenrrr s z8LogprobsProcessor._make_logprob_dict..)lenr* itertoolschainr)rr0r$r"rZ topk_ranksr+rrrr sz$LogprobsProcessor._make_logprob_dictoutputcCs4|jdur ||j|jdur||jdSdS)N)Z new_logprobsr%Znew_prompt_logprobs_tensorsr.)r!r:rrrupdate_from_outputs   z$LogprobsProcessor.update_from_output)__name__ __module__ __qualname__rr __annotations__rrfloatint classmethodr rr r%rr.r/ staticmethodlistrstrdictrr r r;rrrrrsJ       & /   &r)r8collections.abcr dataclassesrtypingrZ vllm.loggerrZ vllm.sequencerrrZ)vllm.transformers_utils.detokenizer_utilsr r Zvllm.v1.enginer r Zvllm.v1.outputsr rr<loggerrepeatrrrrrrs