o ưio@sdZddlmZmZmZmZmZmZmZm Z m Z m Z m Z m Z ddlZddlmZddlmZmZddlmZmZddlmZddlmZmZdd lmZdd lm Z m!Z!dd l"m#Z#Gd d d eZ$GdddeZ%dS)z Transformation logic from OpenAI /v1/chat/completion format to Mistral's /chat/completion format. Why separate file? Make it easy to see how transformation works Docs - https://docs.mistral.ai/api/ ) Any AsyncIterator CoroutineIteratorListLiteralOptionalTupleUnioncastget_type_hintsoverloadN)Logging)3handle_messages_with_content_list_to_str_conversionstrip_none_values_from_message)OpenAIGPTConfig$OpenAIChatCompletionStreamingHandler)get_secret_str)MistralThinkingBlockMistralToolCallMessage)AllMessageValues) ModelResponseModelResponseStream) convert_to_model_response_objectcseZdZUdZdZeeed<dZeeed<dZ eeed<dZ ee ed<dZ ee ded<dZeeed <dZeeed <dZeeed <dZeeee fed <         dSdeedeedeedee dee dd eed eed eed eeee fd dfddZefddZded eefddZded efddZed efddZdedededed ef ddZdeedeed eeeeffd d!Z e!d"ee"ded#e d$d e#e$e$ee"ffd%d&Z%e! 'dTd"ee"ded#e d'd ee"fd(d&Z% 'dTd"ee"ded#ed eee"e#e$e$ee"ffffd)d& Z%d"ee"ded ee"ffd*d+ Z&d"ee"ded ee"ffd,d- Z'd"ee"d ee"fd.d/Z(d"ee"ded ee"fd0d1Z)ede d e fd2d3Z*ed4e"d e"fd5d6Z+ed4e"d e"fd7d8Z,ed4e"d efd9d:Z-ed;ed efde/d efd?d@Z0ed;ed efdAdBZ1ded"ee"dedCedDed ef fdEdF Z2  dUdedGe3j4dHe5dIe6dJed"ee"dedCedKe$deedLeed e5fdMdNZ7 'dTdOee8ee9ee5fdPedLeefdQdRZ:Z;S)V MistralConfiga Reference: https://docs.mistral.ai/api/ The class `MistralConfig` provides configuration for the Mistral's Chat API interface. Below are the parameters: - `temperature` (number or null): Defines the sampling temperature to use, varying between 0 and 2. API Default - 0.7. - `top_p` (number or null): An alternative to sampling with temperature, used for nucleus sampling. API Default - 1. - `max_tokens` (integer or null): This optional parameter helps to set the maximum number of tokens to generate in the chat completion. API Default - null. - `tools` (list or null): A list of available tools for the model. Use this to specify functions for which the model can generate JSON inputs. - `tool_choice` (string - 'auto'/'any'/'none' or null): Specifies if/how functions are called. If set to none the model won't call a function and will generate a message instead. If set to auto the model can choose to either generate a message or call a function. If set to any the model is forced to call a function. Default - 'auto'. - `stop` (string or array of strings): Stop generation if this token is detected. Or if one of these tokens is detected when providing an array - `random_seed` (integer or null): The seed to use for random sampling. If set, different calls will generate deterministic results. - `safe_prompt` (boolean): Whether to inject a safety prompt before all conversations. API Default - 'false'. - `response_format` (object or null): An object specifying the format that the model must output. Setting to { "type": "json_object" } enables JSON mode, which guarantees the message the model generates is in JSON. When using JSON mode you MUST also instruct the model to produce JSON yourself with a system or a user message. N temperaturetop_p max_tokenstools)autoanynone tool_choice random_seed safe_promptresponse_formatstopreturnc Cs>t} | D]\} } | dkr| durt|j| | q dS)Nself)localscopyitemssetattr __class__) r(rrrrr"r#r$r%r&Zlocals_keyvaluer0_/home/app/Keep/.python/lib/python3.10/site-packages/litellm/llms/mistral/chat/transformation.py__init__Ms zMistralConfig.__init__cs tSN)super get_config)clsr-r0r1r5^s zMistralConfig.get_configmodelcCs&gd}d|vr|ddg|S)N) streamrrrmax_completion_tokensrr"seedr&r%parallel_tool_calls magistralthinkingreasoning_effort)lowerextend)r(r8Zsupported_paramsr0r0r1get_supported_openai_paramsbs z)MistralConfig.get_supported_openai_paramscCs$|dks|dkr |S|dkrdSdS)Nrr!requiredr r0)r(r"r0r0r1_map_tool_choicews zMistralConfig._map_tool_choicecCsdS)z Returns the system prompt for Mistral reasoning models. Based on Mistral's documentation: https://huggingface.co/mistralai/Magistral-Small-2506 Mistral recommends the following system prompt for reasoning: ay [SYSTEM_PROMPT]system_prompt A user will ask you to solve a task. You should first draft your thinking process (inner monologue) until you have derived the final answer. Afterwards, write a self-contained summary of your thoughts (i.e. your summary should be succinct but contain all the critical steps you needed to reach the conclusion). You should use Markdown to format your response. Write both your thoughts and summary in the same language as the task posed by the user. NEVER use oxed{} in your response. Your thinking process must follow the template below: Your thoughts or/and draft, like working through an exercise on scratch paper. Be as casual and as long as you want until you are confident to generate a correct answer. Here, provide a concise summary that reflects your reasoning and presents a clear final answer to the user. Don't mention that this is a summary. Problem: [/SYSTEM_PROMPT][INST]user_message[/INST] reasoning_traces assistant_response[INST]user_message[/INST] r0r0r0r0r1$_get_mistral_reasoning_system_promptsz2MistralConfig._get_mistral_reasoning_system_promptnon_default_paramsoptional_params drop_paramscCs"|D]\}}|dkr||d<|dkr||d<|dkr#|||d<|dkr/|dur/||d<|dkr7||d<|dkr?||d<|dkrG||d<|d krXt|trX|j|d |d <|d krbd |i|d <|dkrj||d<|dkrxd|vrxd|d<|dkrd|vrd|d<|dkr||d<q|S)Nrr:rr9Trrr&r")r"r;r#Z extra_bodyr%r?r=_add_reasoning_promptr>r<)r+_clean_tool_schema_for_mistral isinstancestrrDr@)r(rFrGr8rHparamr/r0r0r1map_openai_paramss>  zMistralConfig.map_openai_paramsapi_baseapi_keycCsF|ptdpd}|dur|ds|d}|ptdptd}||fS)NZMISTRAL_AZURE_API_BASEzhttps://api.mistral.ai/v1z/v1ZMISTRAL_AZURE_API_KEYZMISTRAL_API_KEY)rendswith)r(rOrPZdynamic_api_keyr0r0r1$_get_openai_compatible_provider_infos z2MistralConfig._get_openai_compatible_provider_infomessagesis_asyncTcCdSr3r0r(rSr8rTr0r0r1_transform_messagessz!MistralConfig._transform_messagesFcCrUr3r0rVr0r0r1rWscs|D]+}|d}|r-t|tr-tdd|Dr-|r#|||S|||}|Sqt|}g}|D]}t|}t |}t |rHq6t |}| |q6|r\t ||dSt ||dS)a - handles scenario where content is list and not string - content list is just text, and no images - if image passed in, then just return as is (user-intended) - if `name` is passed, then drop it for mistral API: https://github.com/BerriAI/litellm/issues/6696 Motivation: mistral api doesn't support content as a list. The above statement is not valid now. Need to plan to remove all the #1,2,3 Mistral API supports content as a list. contentcss|] }|ddvVqdS)type)Z image_urlfileNget.0cr0r0r1 z4MistralConfig._transform_messages..TF)r\rKlistr _transform_messages_async_transform_messages_syncrr_handle_name_in_message_handle_tool_call_message_is_empty_assistant_messagerappendr4rW)r(rSr8rTm_content_blockZ new_messagesr7r0r1rWs*      cs&t||dIdH}||}|S)zV Handle modification of messages for Mistral API in an async context. TNr4rW_handle_message_with_filer(rSr8r7r0r1rcs z'MistralConfig._transform_messages_asynccst||d}||}|S)zL Handle modification of messages for Mistral API in a sync context. Frkrmr7r0r1rd#s z&MistralConfig._transform_messages_synccCsz|D]8}|d}|r:t|tr:tdd|Dr:dd|D}|D]}|did}|r9||d<|ddq"q|S) zZ Mistral API supports only 'file_id' in message content with type 'file'. rXcs|] }|ddkVqdS)rYrZNr[r]r0r0r1r`8raz:MistralConfig._handle_message_with_file..cSsg|] }|ddkr|qS)rYrZr[r]r0r0r1 ;sz;MistralConfig._handle_message_with_file..rZfile_idN)r\rKrbr pop)r(rSrirj file_contentsZ file_contentrpr0r0r1rl/s  z'MistralConfig._handle_message_with_filec Cs|dds|Stdd|D}|rct|D]J\}}|ddkra|dd}|}t|tr9|d |}nt|trId |d d g|}n |d t|}tti|d|i||<nqnttd|d } | g|}| dd |S)zn Add reasoning system prompt for Mistral magistral models when reasoning_effort is specified. rIFcsrn)rolesystemNr[)r^msgr0r0r1r`NrazGMistralConfig._add_reasoning_system_prompt_if_needed..rsrtrXz text)rYrw)rsrXN) r\r enumeraterErKrLrbr rrq) r(rSrGZhas_system_messageiruZexisting_contentZreasoning_promptZ new_contentZreasoning_messager0r0r1&_add_reasoning_system_prompt_if_neededDs@        z4MistralConfig._add_reasoning_system_prompt_if_neededcCsB|s|Sddl}ddlm}ddlm}||}|||d}|S)a Clean tool schemas to remove fields that cause issues with Mistral API. Removes: - $id and $schema fields (cause grammar validation errors) - additionalProperties=False (causes OpenAI API schema errors) - strict field (not supported by Mistral) Args: tools: List of tool definitions max_depth: Maximum recursion depth for schema cleaning (default: 10) Returns: Cleaned tools list rN)DEFAULT_MAX_RECURSE_DEPTH)_remove_json_schema_refs) max_depth)r*Zlitellm.constantsr{ litellm.utilsr|deepcopy)r6rr*r{r|Z cleaned_toolsr0r0r1rJys   z,MistralConfig._clean_tool_schema_for_mistralmessagecCsX|d}|dur*|ddkr|dd|St|tr*t|dkr*|dd|S)z Mistral API only supports `name` in tool messages If role == tool, then we keep `name` if it's not an empty string Otherwise, we drop `name` nameNrsZtoolr)r\rqrKrLlenstrip)r6r_namer0r0r1res    z%MistralConfig._handle_name_in_messagecCsZ|d}g}|dur+t|tr+|D]}t|dd|dd}||q||d<|S)zc Mistral API only supports tool_calls in Messages in `MistralToolCallMessage` spec Z tool_callsNidfunction)rrYr)r\rKrbrrh)r6rZ _tool_callsZmistral_tool_callsZ_toolZ_tool_call_messager0r0r1rfs  z'MistralConfig._handle_tool_call_messagecCsbddlm}t|}d}|D]}|dkr.||dur.|dkr)||dkr)qd}|Sq|S) zQ Mistral API does not support empty string in assistant content. r)ChatCompletionAssistantMessageTrsNrXrvF)litellm.types.llms.openairr keysr\)r6rrZset_keysZall_expected_values_are_emptyr.r0r0r1rgs  z)MistralConfig._is_empty_assistant_message response_datacCsT|dr(t|ddkr(|dD]}|dr'|dddkr'd|dd<q|S)a Handle Mistral-specific behavior where empty string content should be converted to None. Mistral API sometimes returns empty string content ('') instead of null, which can cause issues with downstream processing. Args: response_data: The raw response data from Mistral API Returns: dict: The response data with empty string content converted to None choicesrrrXrvN)r\r)rchoicer0r0r1_handle_empty_content_responses   z,MistralConfig._handle_empty_content_responsethinking_blockscCsddd|dDS)zG Convert Mistral thinking blocks to reasoning content.  cSsg|]}|ddqS)rwrvr[)r^blockr0r0r1roszNMistralConfig._convert_thinking_block_to_reasoning_content..r>)join)rr0r0r1,_convert_thinking_block_to_reasoning_contentsz:MistralConfig._convert_thinking_block_to_reasoning_contentc Cs|dryt|ddkry|dD]g}|drx|ddrx|dd}t|trxd}d}|D]9}|ddkr\|dg}g}|D]}|ddkrU||ddqCd |}q0|ddkri|dd}q0||dd<|rx||dd <q|S) z Handle Mistral's content list format and extract thinking content. Map mistral's content list to string and extract thinking blocks: - Thinking block -> reasoning_content field - Text block -> content field rrrrXrvrYr>rwrreasoning_content)r\rrKrbrhr) rrrXZthinking_contentZ text_contentrrZthinking_textsthinking_blockr0r0r1&_handle_content_list_to_str_conversions4         z4MistralConfig._handle_content_list_to_str_conversionlitellm_paramsheaderscs:d|vr|ddr|||}tj|||||dS)a Transform the overall request to be sent to the API. For magistral models, adds reasoning system prompt when reasoning_effort is specified. Returns: dict: The transformed request. Sent as the body of the API call. r=rIF)r8rSrGrr)r@r\rzr4transform_request)r(r8rSrGrrr7r0r1rszMistralConfig.transform_request raw_responsemodel_response logging_obj request_dataencoding json_modec Cs\|j|jd|j|jd<|} || } || } ttt | |d|jit |jd} | S)z Transform the raw response from Mistral API. Handles Mistral-specific behavior like converting empty string content to None and extracting thinking content from content lists. )original_responseresponse_headersr)Zresponse_objectZmodel_response_objectZ hidden_paramsZ_response_headers) Z post_callrwrZmodel_call_detailsjsonrrr rrdict)r(r8rrrrrSrGrrrPrrZfinal_response_objr0r0r1transform_response<s    z MistralConfig.transform_responsestreaming_response sync_streamcCst|||dS)N)rrr)MistralChatResponseIterator)r(rrrr0r0r1get_model_response_iteratorcs z)MistralConfig.get_model_response_iterator) NNNNNNNNN)F)NN)<__name__ __module__ __qualname____doc__rrint__annotations__rrrrbr"rr#r$boolr%rr&r rLr2 classmethodr5rrBrD staticmethodrErNr rRr rrrrWrcrdrlrzrJrerfrgrrrrrhttpxResponserLiteLLMLoggingObjrrrr __classcell__r0r0r7r1r*sx      +    *    5!  (+     +rc sVeZdZdedeffdd Zedeedee e eee e ffddZ Z S)rchunkr'csz>|dgD]5}|di}|d}t|tr<||\}}}||d<|r0||d<||d<q|dd|ddqWntyMt|YSwt|S)NrdeltarXrr)r\rKrb_normalize_content_blocksrq Exceptionr4 chunk_parser)r(rrrrXnormalized_textrrr7r0r1rqs,        z(MistralChatResponseIterator.chunk_parsercontent_blocksc Csg}g}g}|D]K}|d}|dkrF|dg}g}|D]}|ddkr/||ddqd|} | rE|| |d| ddq|dkrS||ddq|r[d|nd} |rdd|nd} | || fS) zl Convert Mistral magistral content blocks into OpenAI-compatible content + thinking_blocks. rYr>rwrvZmistral)rYr> signatureNr)r\rhr) rZ text_segmentsrZreasoning_segmentsrZ block_typeZmistral_thinkingZthinking_text_partsrZ thinking_textrrr0r0r1rs8     z5MistralChatResponseIterator._normalize_content_blocks) rrrrrrrrr rrLrrr0r0r7r1rpsr)&rtypingrrrrrrrr r r r r rZ*litellm.litellm_core_utils.litellm_loggingrrZ8litellm.litellm_core_utils.prompt_templates.common_utilsrrZ+litellm.llms.openai.chat.gpt_transformationrrZlitellm.secret_managers.mainrZlitellm.types.llms.mistralrrrrZlitellm.types.utilsrrr~rrrr0r0r0r1s"8    J