o o i)@sddlZddlmZmZmZmZmZerddlZGdddeZGdddZ e Z GdddZ e Z d e d ed gd ffd d Zded ed gd ffddZded ed gd ffddZGdddZeZdS)N) TYPE_CHECKINGCallableOptionalProtocolTuplec@s0eZdZUeed<        d dd Zd S) Samplersamplesnext_token_logitstorch.DoubleTensorsequence_weightsrngtorch.GeneratorreturncCsdSN)selfr r r rr]/home/app/PaddleOCR-VL-test/.venv_paddleocr/lib/python3.10/site-packages/outlines/samplers.py__call__ szSampler.__call__N)r r r r r r rr )__name__ __module__ __qualname__int__annotations__rrrrrrs rc@s.eZdZdZddZ      d dd Zd S) GreedySampleraGreedy Sampling algorithm. Greedy sampling consists in choosing the token with the largest likelihood at every step. We don't allow more than one sample. We could attribute this a meaning, for instance the k-th sample represents the k-th most likely token. In which case it would be equivalent to beam search without the sequence weights. Attributes ---------- samples The number of samples taken for each input sequence. cCs d|_dS)Nr)rrrr__init__% zGreedySampler.__init__r r r rc Cs`ddl}|jjj|dd}|j|ddd}|j|jd|jd}|||d| }|||fS) aCall the greedy sampler. Parameters ---------- next_token_logits A tensor of shape ``(n_seqs, vocab_size,)`` that represents the probability distribution of the next token over the vocabulary. sequence_weights A tensor of shape ``(n_seqs,)`` that represents the cumulative weight of each sequence. rng A random number generator. Returns ------- A tuple with an array that contains the ids of the sampled tokens of shape ``(n_seqs, 1)``, an array that contains the ancestors of each sampled id of shape ``(n_seqs,)`` and an array that contains the updated cumulative weights of each sequence of shape ``(n_seqs,)``. rNdimT)r Zkeepdimdevicer) torchnn functional log_softmaxZargmaxarangeshaper"gathersqueeze) rr r _r#logprobsnext_token_ids ancestorsweightsrrrr(s  zGreedySampler.__call__N)r r r r rr )rrr__doc__rrrrrrrsrc @sbeZdZdZ ddddddedeedeedeefd d Zd d d d dddedfddZ dS)MultinomialSamplera<Multinomial sampling algorithm. Multinomial sampling consists in randomly sampling the next token assuming its distribution is a Categorical distribution parametrized by the next-token logits. Attributes ---------- samples The number of samples taken for each input sequence. rN)top_ktop_p temperaturerr2r3r4cCsp||_||_||_||_g|_|dur|jt|n |dur(|jt||dur6|jt|dSdSr) rr2r3r4logits_processorsappendkeep_top_k_logitskeep_top_p_logitsrescale_logits)rrr2r3r4rrrrbszMultinomialSampler.__init__r r r r r rr r r c Csddl}|}|jD]}||}q |jjj|dd}|j|d|d}|jjj|dd} |j|jd|j d} || | d| } || | fS)a Call the multinomial sampler. Parameters ---------- next_token_logits A tensor of shape ``(n_seqs, vocab_size,)`` that represents the probability distribution of the next token over the vocabulary. sequence_weights A tensor of shape ``(n_seqs,)`` that represents the cumulative weight of each sequence. rng A random number generator. Returns ------- A tuple with an array that contains the ids of the sampled tokens of shape ``(n_seqs, 1)``, an array that contains the ancestors of each sampled id of shape ``(n_seqs,)`` and an array that contains the updated cumulative weights of each sequence of shape ``(n_seqs,)``. rNrrr)Z num_samples generatorr!) r#r5r$r%softmax multinomialr&r'r(r"r)r*) rr r r r#Zaltered_next_token_logitsZlogit_processorZprobsr-r,r.r/rrrrxs    zMultinomialSampler.__call__r) rrrr0rrfloatrrrrrrrr1Ss2 r1kr torch.TensorcsJddlttr dkrtdddjdjffdd }|S) zBuild a function that masks logits values smaller than the top `k` ones. Parameters ---------- k The ranking below which logit values are replaced by `-math.inf`. rNrz.`k` must be a strictly positive integers, got instead.logitsrcs8t|d}|||ddk}||tj S)Nrr).rN)minsizetopk masked_fillmathinf)rCZ num_to_keepmask_idxr@r#rrlogits_processorsz+keep_top_k_logits..logits_processor)r# isinstancer ValueErrorTensor)r@rLrrKrr7s  r7pcsHddldks dkrtdddjdjffd d }|S) aBuild a function that masks the lowest probability tokens whose cumulative probability is below a certain threshold. Parameters ---------- p The value of the threshold. We keep the highest probability tokens whose cumulative distribution is greater than or equal to `p` and mask the others. Its value must be between 0 (excluded) and 1 (included). rNg?zO`p` must be a floating point number between 0 (excluded) and 1 (included), got rBrCrcsXj|dd\}}jjj|ddjdd}|dk}|d||}||tj S)NF)Z descendingrrr) sortr$r%r<ZcumsumZscatterrGrHrI)rCZ sorted_logitsZ sorted_idxZcumulative_probabiltiesZsorted_masked_idxrJrPr#rrrLs z+keep_top_p_logits..logits_processor)r#rNrO)rPrLrrSrr8s   r8r4csDttr dkrtdddkrtdd fdd }|S) zBuild a function that rescales the token probabilities exponentially. Parameters ---------- temperature The value by which we rescale the logits. rQzE`temperature` must be a strictly positive floating point number, got rBzFPlease use the greedy sampler instead of setting the temperature to 0.rCrArcs|Srr)rCr4rrrLsz(rescale_logits..logits_processorN)rCrArrA)rMr?rN)r4rLrrTrr9s  r9c@s:eZdZdZddefddZddddd ed fd d Zd S)BeamSearchSamplerzBeam Search sampling algorithm. Attributes ---------- samples The number of samples taken for each input sequence. Equivalent to the number of beams. rbeamscCs ||_dSrr)rrVrrrrrzBeamSearchSampler.__init__r r r rr:c Csddl}|jjj|dd}||d|}|jd|j}|jd}|||j|}| |dkr>|ddd|f}|j ||jdddd\}} |j | |dd } | |} |j d||j|j|j d d} | | } | |j|} ||j|}| |j|d} | | |fS) a Call the beam search sampler. Parameters ---------- next_token_logits A tensor of shape ``(n_seqs, vocab_size,)`` that represents the probability distribution of the next token over the vocabulary. sequence_weights A tensor of shape ``(n_seqs,)`` that represents the cumulative weight of each sequence. rng A random number generator. Returns ------- A tuple with an array that contains the ids of the sampled tokens of shape ``(n_seqs, 1)``, an array that contains the ancestors of each sampled id of shape ``(n_seqs,)`` and an array that contains the updated cumulative weights of each sequence of shape ``(n_seqs,)``. rNrrrT)r Zlargestsortedfloor)Z rounding_moder!)r#r$r%r&Z unsqueezeZ expand_asr(rviewallrFdivr'r") rr r r+r#r,r/Z batch_sizeZ vocab_sizeindicesr.r-Zfirst_batch_idxrrrrs.    zBeamSearchSampler.__call__Nr>)rrrr0rrrrrrrrrUs rU)rHtypingrrrrrr#rrZgreedyr1r=rr7r?r8r9rUZ beam_searchrrrrs <R M