o )i>@sddlmZddlmZmZddlZddlmZddl m Z m Z ddl m Z ddlmZmZe eZedGd d d eZdS) )Sequence)OptionalUnionN)PreTrainedTokenizerBase)ChatCompletionRequest DeltaMessage) init_logger)ReasoningParserReasoningParserManagerZgranitecs$eZdZdZdeffdd Zdededee ee effdd Z d ed ed ed e e de e de e de edffddZdedefddZdedefddZd ed edefddZd eded edefddZd ededed ede def dd Zd edee ee e e effd!d"ZZS)#GraniteReasoningParserz Reasoning parser for IBM Granite. IBM granite models currently use "Here is my thought process:" and "Here is my response:" to separate its thinking / response outputs. tokenizercsrt|d|_d|_t|jd|jdtj|_ddg|_ddg|_ d |_ d |_ t d d |jD|_ dS) Nz&(?:Here's|Here is) my thought process:z(?:Here's|Here is) my response:z(.*?)z(.*)zHere's my thought process:zHere is my thought process:zHere's my response:zHere is my response::ZHerecss|]}t|VqdSN)len.0 think_startrs/home/app/PaddleOCR-VL-test/.venv_paddleocr/lib/python3.10/site-packages/vllm/reasoning/granite_reasoning_parser.py 4s z2GraniteReasoningParser.__init__..)super__init__Zthink_start_exprZresponse_start_exprrecompileDOTALLreasoning_regexvalid_think_startsvalid_response_startsseq_boundary_endseq_boundary_startmaxZlongest_think_start)selfr  __class__rrrs zGraniteReasoningParser.__init__ model_outputrequestreturncCs8|j|}|s d|fS|d\}}|s|dfS||fS)aExtract the reasoning content & content sections, respectively. If the sequence doesn't match what we expect, i.e., the model generates something else, all content is considered non-reasoning content. Args: model_output (str): Output of the model to be parsed. request (ChatCompletionRequest): Request being processed. Returns: tuple[Optional[str], Optional[str]]: Tuple pair containing the reasoning content and non-reasoning content. Nr)rfindall)r!r$r%re_matchreasoning_contentresponse_contentrrrextract_reasoning_content7s  z0GraniteReasoningParser.extract_reasoning_content previous_text current_text delta_textprevious_token_idscurrent_token_idsdelta_token_idsNc Csh||\}}} |s|||} n| s||||} n|dus!J|||| ||} | js2| js2dS| S)aExtract the reasoning content / content emitted by granite models; If the sequence doesn't match what we expect, i.e., the model generates something else, all content is considered non-reasoning content. NOTE: Granite models do not use a special token to start their reasoning and response sections; instead they have token sequences, e.g., Here is my thought process: Foo Here is my response: Bar This increases the complexity of correctly handling streams, since we need to watch for specific sequences and correctly parse them without dropping content that is potentially overlapping & spanning multiple delta messages. Args: previous_text (str): Previous text outside of this delta message. current_text (str): Previous text + delta text. delta_text (str): Text to consider and parse content from. previous_token_ids (Sequence[int]): Token IDs of previous_text. current_token_ids (Sequence[int]): Token IDs of current_text. delta_token_ids (Sequence[int]): Token IDs of delta_text. Returns: Union[DeltaMessage, None] DeltaMessage with either reasoning content or content, or None. N)_get_content_sections+_get_delta_message_with_no_reasoning_bounds*_get_delta_message_with_no_response_bounds#_get_delta_message_with_both_boundscontentr)) r!r,r-r.r/r0r1r)Z resp_seq_lenr6Z delta_messagerrr#extract_reasoning_content_streamingNs&#   z:GraniteReasoningParser.extract_reasoning_content_streamingtextctfdd|jDS)zCheck if a text matches one of the possible start reasoning seqs. Args: text (str): Text to check for leading substr. Returns: bool: True if any of the possible reasoning start seqs match. c3|]}|VqdSr startswithrr8rrr  zDGraniteReasoningParser._is_reasoning_start_substr..)anyrr!r8rr=r_is_reasoning_start_substr z1GraniteReasoningParser._is_reasoning_start_substrcr9)zCheck if a text matches one of the possible start response seqs. Args: text (str): Text to check for leading substr. Returns: bool: True if any of the possible response start seqs match. c3r:rr;rresponse_startr=rrrr>zCGraniteReasoningParser._is_response_start_substr..)r?rr@rr=r_is_response_start_substrrBz0GraniteReasoningParser._is_response_start_substrcCs\t|t|}||}||d|}|r |s td|dS|r(tdddStd|dS)a^Parse the delta message when the current text has not yet completed its start of reasoning sequence. Args: current_text (str): The full previous + delta text. delta_text (str): Text to consider and parse content from. Returns: DeltaMessage: Message containing the parsed content. Nr)r6)rrAr)r!r-r.Zprev_longest_lengthZ is_substrZ was_substrrrrr3s    zBGraniteReasoningParser._get_delta_message_with_no_reasoning_boundsr)c s0tfdd|jD}|dus|rtdddS|dt| }|||j}||j}|dkr<|||dnd}|dkrK||dnd} |dkrZ|||dnd} | rdtdddS|sx| rrt|d|ddSt|ddS| r||d|d|}t|ddSt||d|ddS)a=Parse the delta message when the current text has both reasoning content with no (response) content. NOTE that we may have overlapping tokens with the start of reasoning / start of response sequences on either side of the delta text. Args: current_text (str): The full previous + delta text. reasoning_content (str): reasoning content from current_text. delta_text (str): Text to consider and parse content from. Returns: DeltaMessage: Message containing the parsed content. c3s|]}|VqdSr)endswithrCr-rrrr>zTGraniteReasoningParser._get_delta_message_with_no_response_bounds..NrFrF)r?rrrrfindrrE) r!r-r)r.Zends_with_start_response_seqr,Zprev_idxZ delta_idxZprev_was_substrZdelta_continues_substrZdelta_new_substrrrHrr4s`            zAGraniteReasoningParser._get_delta_message_with_no_response_boundsr*response_seq_lenc Cs|t| d}t|t||}|dkrd}n$t||t|d} t|t|} | | } | dkr8d} || |}t||dS)aBParse the delta message when the current text has both reasoning content and normal (response) content. Args: delta_text (str): Text to consider and parse content from. reasoning_content (str): reasoning content from current_text. response_content (str): response content from current_text. current_text (str): The full previous + delta text. response_seq_len(str): Len of the complete response sequence used. Returns: DeltaMessage: Message containing the parsed content. NrrF)rr) r!r.r)r*r-rJZ delta_contentZreasoning_end_idxZdelta_reasoning_contentZstart_reasoning_content_idxZ delta_offsetZ start_offsetrrrr5 s8 z:GraniteReasoningParser._get_delta_message_with_both_boundsc sd}d}d}fddt|D}|D]]}|||}|dur8jD]}||ddkr6|d}|d}nq"q|spjD]2} |t|  dd| ddkro|t| } ||| } ||dd} | t| | fSq=q|r~|s~||dddfSdS) aParse the text to extract the reasoning content / content if we have them. Args: current_text (str): The full previous + delta text. Returns: tuple[Optional[str], Optional[int], Optional[str]]: Tuple of len 3 containing the reasoning content, the length of the response seq (if there is one) and the non-reasoning content. rNFcsg|] \}}|jkr|qSr)r)ridxcharr!rr Ks  z@GraniteReasoningParser._get_content_sections..rK)NNN) enumeraterrr) r!r-Zcurrent_chunk_startZstart_reasoning_contentZparsed_contentZdelimiter_idxsZcurrent_chunk_endZ current_chunkrrDZend_reasoning_contentr)r*rrNrr2:sP        z,GraniteReasoningParser._get_content_sections)__name__ __module__ __qualname____doc__rrstrrtuplerr+rintrrr7boolrArEr3r4r5r2 __classcell__rrr"rr sz   ;   # G -r )collections.abcrtypingrrregexrZ transformersrZ vllm.entrypoints.openai.protocolrrZ vllm.loggerrZvllm.reasoningr r rRloggerZregister_moduler rrrrs