o )i5b@sdZddlZddlmZddlmZmZddlmZddl m Z m Z m Z m Z ddlZddlmZddlmZdd lmZdd lmZeeZd Zd ZGd ddeZeGdddZGdddeZGdddejdddZGdddejdddZ dS)z(Sampling parameters for text generation.N) dataclass)EnumIntEnum)cached_property) AnnotatedAnyOptionalUnion) BaseModel) init_logger)LogitsProcessor) AnyTokenizergh㈵>g{Gz?c@eZdZdZdZdZdS) SamplingTyperN)__name__ __module__ __qualname__GREEDYRANDOM RANDOM_SEEDrr`/home/app/PaddleOCR-VL-test/.venv_paddleocr/lib/python3.10/site-packages/vllm/sampling_params.pyrsrc@sNeZdZUdZdZeeeefe d<dZ eee d<dZ ee ee d<dZ eee d<dZeee d< dZeee d<d Zee d <d Zee d <d Zee d <d Zee d <dZeee d<dZeee d<e        ddeeeeefdeedee edeedeedeedeedeededfddZddZdS)GuidedDecodingParamsz:z5GuidedDecodingParams.from_optional..rrrrrr r%r&)all isinstancer typeZmodel_json_schemarr-rrr from_optional/s   z"GuidedDecodingParams.from_optionalcCsLt|jdu|jdu|jdu|jdu|jdug}|dkr$td|jdS)z1Validate that some fields are mutually exclusive.NrzIYou can only use one kind of guided decoding but multiple are specified: )sumrrrrr ValueError__dict__)selfZ guide_countrrr __post_init__Ksz"GuidedDecodingParams.__post_init__)NNNNNNNN)rrr__doc__rrr strdict__annotations__rrlistrrboolr r!r"r#r$r%r& staticmethodr r1r6rrrrrsX       rc@r)RequestOutputKindrrrN)rrr CUMULATIVEDELTAZ FINAL_ONLYrrrrr>Wsr>c?@sxeZdZUdZdZeed< dZeeed< dZ eeed<dZ e ed< dZ e ed < d Z e ed < d Ze ed < d Ze ed < dZeed< dZe ed< dZeeed< dZeeeeefed< dZeeeed< dZeed< dZeeed< dZeed< dZeeed< dZeeed< dZeed< dZeed< dZ eed< dZ!ee"ed< dZ#eed < dZ$ee%ee&j'dd!fed"< e(j)Z*e(ed#<dZ+eed$<e&j,e-d%Z.e-eed&<dZ/ee0ed'< dZ1ee2ee fed(< dZ3eeeed)< dZ4ee2ee"fed*< dZ5eeeed+< dZ6eeeeed,<e7ddddd d d ddddddddddddddddde(j)ddddfdeedeedee d ee d ee d ee d ee dede deedeeeeefdeeed+eeed ededeededeedeededededeee8d"ee%ee&j'dd!fd#e(d'ee0d(eee2ee fe2ee ffd)eeed*ee2ee"fd-dfd-dfd;d<Z?e@d-eAfd=d>ZBeCd-e-efd?d@ZDeCd-eeeefdAdBZEdIdCdDZFd-efdEdFZGdS)JSamplingParamsaSampling parameters for text generation. Overall, we follow the sampling parameters from the OpenAI text completion API (https://platform.openai.com/docs/api-reference/completions/create). In addition, we support beam search, which is not supported by OpenAI. rnNbest_of_real_npresence_penaltyfrequency_penalty?repetition_penalty temperaturetop_prtop_kmin_pseedstopstop_token_idsF ignore_eos max_tokens min_tokenslogprobsprompt_logprobsT detokenizeskip_special_tokensspaces_between_special_tokenslogits_processorsinclude_stop_str_in_output)getruncate_prompt_tokens output_kindoutput_text_buffer_length)default_factory_all_stop_token_idsguided_decoding logit_biasallowed_token_ids extra_args bad_words_bad_words_token_idsr'cCs|dur dd|D}td#id|durdn|d|d|dur#dn|d|dur,dn|d |dur5d n|d |dur>d n|d |durGd nF|d |d|d| d| d| d| d| d|d|d|d|d|d|d|d|d|d|d|d|d |d!|d"|Sd |d|d| d| d| d| d| d|d|d|d|d|d|d|d|d|d|d|d|d |d!|d"|S)$NcSs&i|]\}}t|tdtd|qS)gY@gY)intminmax)r)tokenZbiasrrr sz0SamplingParams.from_optional..rBrrCrFrErGrIrHrJrKrLrMrNrOrPrfr[rQrSrTrUrVrWrXrYrZr]r^rbrcrdrer)itemsrA)rBrCrFrGrIrJrKrLrMrNrOrPrfr[rQrSrTrUrVrWrXrYrZr]r^rbrcrdrerrrr1s!           zSamplingParams.from_optionalcCsX|jr!|j|jkrtd|jd|jd|js!|j|_|j|_d|jkr,tkr>nntd|jttt|jt|_|j dkrFd|_ |j durOg|_ n t |j t rZ|j g|_ |j durbg|_ |jdurjg|_|jdurrd|_|jdurzd|_|j r|jstd d |j Dd|_||jtkrd |_d|_d |_||j|j dS) N2best_of must be greater than or equal to n, got n= and best_of=.rzstemperature %s is less than %s, which may cause numerical errors nan or inf in tensors. We have maxed it out to %s.Trcss|]}t|VqdSr()len)r)srrrr+Tr,z/SamplingParams.__post_init__..rHrE)rCrBr3rDrJ _MAX_TEMPloggerwarningrjrNrOr/r8rPrfrUrVr[r_ _verify_args _SAMPLING_EPSrKrLrM_verify_greedy_samplingraupdater5rrrr6&sR           zSamplingParams.__post_init__cCsrt|jtstdt|j|jdkrtd|jd|jdurSt|jts3tdt|j|jdkr@td|j|j|jkrStd|jd|jdd |jkr^d kshntd |jdd |jkrsd ks}ntd |jd|jd krtd|jd|j d krtd|j dd |j krdksntd|j d|j dkrtd|j dt|j tst dt|j j d |jkrdksntd|jd|jdur|jdkrtd|jd|jdkrtd|jd|jdur|j|jkrtd|jd|jd|jdur9|jdkr9|jdkr9td|jd|jdurN|jdkrNtd|jd|jdurb|jdkrbtd|jt|jtskJtdd|jDstd |jdt|jtsJtd!d|jDrtd"|jr|jstd#|j|jkr|jtjkrtd$dSdS)%Nz!n must be an int, but is of type rzn must be at least 1, got rpz best_of must be an integer, got z best_of must be at least 1, got rnrogg@z)presence_penalty must be in [-2, 2], got z*frequency_penalty must be in [-2, 2], got rEz2repetition_penalty must be greater than zero, got z&temperature must be non-negative, got rHztop_p must be in (0, 1], got rqz.top_k must be 0 (disable), or at least 1, got ztop_k must be an integer, got zmin_p must be in [0, 1], got z#max_tokens must be at least 1, got rz3min_tokens must be greater than or equal to 0, got z4min_tokens must be less than or equal to max_tokens=z, got z)logprobs must be non-negative or -1, got z*prompt_logprobs must be non-negative, got z)truncate_prompt_tokens must be >= 1, got css|]}t|tVqdSr()r/rh)r)Zst_idrrrr+sz.SamplingParams._verify_args..z/stop_token_ids must contain only integers, got css|]}| VqdSr(r)r)Zstop_strrrrr+sz$stop cannot contain an empty string.zYstop strings are only supported when detokenize is True. Set detokenize=True to use stop.z-best_of must equal n to use output_kind=DELTA)r/rBrhr3r0rCrFrGrIrJrKrL TypeErrorrrMrSrTrUrVr]rPr;r.rOanyrWrDr^r>r@r{rrrrwbs                      zSamplingParams._verify_argscCs |jdkrtd|jddS)Nrz,n must be 1 when using greedy sampling, got rp)rBr3r{rrrrys  z&SamplingParams._verify_greedy_samplinggeneration_configmodel_eos_token_idcCs|dur |j||d}dur@t|tr|hnt|}|dur(|||rB|j||jsD||j t ||_ dSdSdSdS)z=Update if there are non-default values from generation_configNZ eos_token_id) raaddgetr/rhsetdiscardrzrQrPr;)r5r~rZeos_idsrrrupdate_from_generation_configs    z,SamplingParams.update_from_generation_config tokenizercs|jsdSg|_|jD]:}dD]5}|rdnd}||}j|dd}|r>|rD|d|jddkrDt|t|jdkrD|j|qq fdd |jD}t|dkrhtd jd d |d jddS)N)FT F)textZadd_special_tokensrrqcs,g|]}|D] }|dks|jkr|qqS)r) max_token_id)r)bad_words_token_idsZtoken_idrrr sz8SamplingParams.update_from_tokenizer..zThe model vocabulary size is rz2, but the following tokens were specified as bad: zF. All token id values should be integers satisfying: 0 <= token_id <= rp)rfrglstripencoderrappendr3r)r5rZbad_wordZadd_prefix_spaceprefixpromptZprompt_token_idsZinvalid_token_idsrrrupdate_from_tokenizersD       z$SamplingParams.update_from_tokenizercCs&|jtkrtjS|jdurtjStjSr()rJrxrrrNrrr{rrr sampling_types  zSamplingParams.sampling_typecC|jSr()rar{rrrall_stop_token_idssz!SamplingParams.all_stop_token_idscCrr()rgr{rrrrsz"SamplingParams.bad_words_token_idscCs,|jdurdndd|jD}tj||dS)a[Deep copy, but maybe not the LogitsProcessor objects. LogitsProcessor objects may contain an arbitrary, nontrivial amount of data that is expensive to copy. However, if not copied, the processor needs to support parallel decoding for multiple sequences See https://github.com/vllm-project/vllm/issues/3087 NcSs(i|]}t|t|dr|n|qS)clone)idhasattrr)r)Zlprrrrl sz(SamplingParams.clone..)memo)rZcopydeepcopy)r5Zlogit_processor_refsrrrrs zSamplingParams.clonecCs"dgd|jd|jd|jd|jd|jd|jd|jd |jd |j d |j d |j d |j d|j d|jd|jd|jd|jd|jd|jd|jd|jd|jd|jdS)NrzSamplingParams(n=z, presence_penalty=z, frequency_penalty=z, repetition_penalty=z, temperature=z, top_p=z, top_k=z, min_p=z, seed=z, stop=z, stop_token_ids=z , bad_words=z, include_stop_str_in_output=z , ignore_eos=z , max_tokens=z , min_tokens=z , logprobs=z, prompt_logprobs=z, skip_special_tokens=z , spaces_between_special_tokens=z, truncate_prompt_tokens=z, guided_decoding=z , extra_args=))joinrBrFrGrIrJrKrLrMrNrOrPrfr[rQrSrTrUrVrXrYr]rbrer{rrr__repr__s\      zSamplingParams.__repr__)r'Nr()r'rA)Hrrrr7rBrhr:rCrrDrFfloatrGrIrJrKrLrMrNrOr r8r;rPrQr<rSrTrUrVrWrXrYrZrr[r]rmsgspecZMetar>r?r^r_fieldrrarbrrcr9rdrerfrgr=r r1r6rwryrr rrrrpropertyrrrrrrrrrA`s`                          K < N  $ rAT)Z omit_defaultsr9c@sReZdZUdZeed<eed<dZeed<dZe ed<dZ e ed <dZ eed <d S) BeamSearchParamsz+Beam search parameters for text generation.Z beam_widthrSFrQrErJrHlength_penaltyr[N) rrrr7rhr:rQr<rJrrr[rrrrr/s    r)!r7r dataclassesrenumrr functoolsrtypingrrrr rZpydanticr Z vllm.loggerr Zvllm.logits_processr Z!vllm.transformers_utils.tokenizerr rrurxrtrrr>StructrArrrrrs>      8  R