o ưi@s^ddlZddlZddlmZmZmZmZmZmZm Z m Z ddl m Z ddl mZddlmZddlmZddlmZmZddlmZdd lmZmZmZmZmZmZmZm Z erdd l!m"Z"dd l#m$Z%dd l&m'Z'dd l(m)Z*ddl+m,Z,ddl-m.Z.m/Z/m0Z0ddl1m2Z2e e%efZ$neZ$eZ*eZ,eZ.eZ/eZ0eZ3eZ4eZ2e5dej6Z7GdddZ8dS)N) TYPE_CHECKINGAnyAsyncGeneratorDictListOptionalTupleUnion) BaseModelverbose_logger)/DEFAULT_MAX_RECURSE_DEPTH_SENSITIVE_DATA_MASKER) ArgillaItem)AllMessageValuesChatCompletionRequest) PromptSpec)AdapterCompletionStreamWrapper CallTypesCallTypesLiteralLLMResponseTypes ModelResponseModelResponseStreamStandardCallbackDynamicParamsStandardLoggingPayload) HTTPException)Span) DualCache)Logging)UserAPIKeyAuth)MCPPostCallResponseObjectMCPPreCallRequestObjectMCPPreCallResponseObject)PreRoutingHookResponsezOdata:(?:application|image|audio|video)/[a-zA-Z0-9.+-]+;base64,[A-Za-z0-9+/=\s]+c @seZdZ  ddededdfddZedd eedeefd d Z d d Z ddZ ddZ ddZ ddZddZddZdedededeefddZdd Zd!d"Z      ddedeed#ed$eed%eed&ed'ed(eed)eeed*eed+eed,eed-eedeeeeeffd.d/Z     ddedeed#ed$eed%eed&ed(eed*eed+eed,eed-eedeeeeeffd0d1Z   dded2edeeeeefd3eeeefd4eedeef d5d6Z  dded7edeeed2eed8ee!deef d9d:Z"deee#fd;ee$deefded8ee!deefd?d@Z&d>edeefdAdBZ'dCedDe(d;ee$dee(fdEdFZ)dCedGe#d;ee$dee#fdHdIZ*dJe+dKeedefdLdMZ,dKededNe+fdOdPZ-dKededNe+fdQdRZ.dee/fdSdTZ0dDe1dee2fdUdVZ3dWe#dee4fdXdYZ5dZe6d[ee7dee6fd\d]Z8 d^e9d_d`daed;e:deee+eeff dbdcZ; ddaed^e9dDe#ddeeeefdeeeeff dedfZ< ddCedNe+d^e9dgeededhf didjZ=daed^e9dDe(de#fdkdlZ>dedme#d;edeee#ffdndoZ?dedme#d;edeee#ffdpdqZ@daed^e9d;e:de#fdrdsZAd^e9dDede#fdtduZBd^e9dDe#dCedeCeDdffdvdwZEdxdyZFdzd{ZGd|d}ZHd~dZIdeJdeeJfddZKdDe#dedeed)eeededededeeeffddZLd)ededeedDe#de#dedddedede#fddZMdDe#dedeed)eeededededeeeffddZNd)ededeedDe#dedddedede#fddZOde7fddZPde7dededdfddZQdededefddZR dd2eedeefddZSdedefddZTdedeefddZUd efddZVeWfdddeddfddZXeWfdddeddfddZYdeWfde#dedede#fddZZde#defddZ[eWfdee#dedeeee#ffddZ\dS) CustomLoggerFTturn_off_message_loggingmessage_loggingreturnNcKs||_||_dS)a Args: turn_off_message_logging: bool - if True, the message logging will be turned off. Message and response will be redacted from StandardLoggingPayload. message_logging: bool - deprecated param, use `turn_off_message_logging` instead N)r%r$)selfr$r%kwargsr)Y/home/app/Keep/.python/lib/python3.10/site-packages/litellm/integrations/custom_logger.py__init__Es zCustomLogger.__init__ callback_namecCs|durgS|}ddi}|||}zddlm}Wn ty'gYSw|}t||d}|dur7gSt|dd}|sAgSt|S)a2 Return the environment variables associated with a given callback name as defined in the proxy callback registry. Args: callback_name: The name of the callback to look up. Returns: List[str]: A list of required environment variable names. NZ langfuse_otelZlangfuser) AllCallbacksZlitellm_callback_params)lowergetlitellm.proxy._typesr- Exceptiongetattrlist)r,Znormalized_nameZ alias_map lookup_namer- callbacksZ callback_infoparamsr)r)r*get_callback_env_varsUs&     z"CustomLogger.get_callback_env_varscCdSNr)r'modelmessagesr(r)r)r*log_pre_api_call{zCustomLogger.log_pre_api_callcCr8r9r)r'r( response_obj start_timeend_timer)r)r*log_post_api_call~r>zCustomLogger.log_post_api_callcCr8r9r)r?r)r)r*log_stream_eventr>zCustomLogger.log_stream_eventcCr8r9r)r?r)r)r*log_success_eventr>zCustomLogger.log_success_eventcCr8r9r)r?r)r)r*log_failure_eventr>zCustomLogger.log_failure_eventcdSr9r)r?r)r)r*async_log_stream_eventz#CustomLogger.async_log_stream_eventcrGr9r)r:r)r)r*async_log_pre_api_callrIz#CustomLogger.async_log_pre_api_callr;r<r(cdS)a Hook called before making the API request to allow modifying request parameters. This is specifically designed for modifying the request before it's sent to the provider. Unlike async_log_pre_api_call (which is for logging), this hook is meant for transformations. Args: model: The model name messages: The messages list kwargs: The request parameters (tools, stream, temperature, etc.) Returns: Optional[Dict]: Modified kwargs to use for the request, or None if no modifications Example: ```python async def async_pre_request_hook(self, model, messages, kwargs): # Convert native tools to standard format if kwargs.get("tools"): kwargs["tools"] = convert_tools(kwargs["tools"]) return kwargs ``` Nr)r:r)r)r*async_pre_request_hooksz#CustomLogger.async_pre_request_hookcrGr9r)r?r)r)r*async_log_success_eventrIz$CustomLogger.async_log_success_eventcrGr9r)r?r)r)r*async_log_failure_eventrIz$CustomLogger.async_log_failure_eventnon_default_params prompt_idprompt_variablesdynamic_callback_paramslitellm_logging_obj prompt_spectools prompt_labelprompt_versionignore_prompt_manager_model%ignore_prompt_manager_optional_paramscs |||fSax Returns: - model: str - the model to use (can be pulled from prompt management tool) - messages: List[AllMessageValues] - the messages to use (can be pulled from prompt management tool) - non_default_params: dict - update with any optional params (e.g. temperature, max_tokens, etc.) to use (can be pulled from prompt management tool) r))r'r;r<rOrPrQrRrSrTrUrVrWrXrYr)r)r* async_get_chat_completion_prompts z-CustomLogger.async_get_chat_completion_promptc Cs |||fSrZr)) r'r;r<rOrPrQrRrTrVrWrXrYr)r)r*get_chat_completion_prompts z'CustomLogger.get_chat_completion_promptrequest_kwargsinputspecific_deploymentcrK)z This hook is called before the routing decision is made. Used for the litellm auto-router to modify the request before the routing decision is made. Nr))r'r;r]r<r^r_r)r)r*async_pre_routing_hooks z#CustomLogger.async_pre_routing_hookhealthy_deploymentsparent_otel_spancs|Sr9r))r'r;rar<r]rbr)r)r*async_filter_deploymentsz%CustomLogger.async_filter_deployments call_typecrK)a Allow modifying the request just before it's sent to the deployment. Use this instead of 'async_pre_call_hook' when you need to modify the request AFTER a deployment is selected, but BEFORE the request is sent. Used in managed_files.py Nr))r'r(rer)r)r*async_pre_call_deployment_hooks z+CustomLogger.async_pre_call_deployment_hook deploymentcrGr9r))r'rgrbr)r)r*async_pre_call_checkz!CustomLogger.async_pre_call_checkcCr8r9r))r'rgr)r)r*pre_call_checkr>zCustomLogger.pre_call_check request_dataresponsecrK)zh Allow modifying / reviewing the response just after it's received from the deployment. Nr))r'rkrlrer)r)r*'async_post_call_success_deployment_hook z4CustomLogger.async_post_call_success_deployment_hookresponse_chunkcrK)z Allow modifying streaming chunks just before they're returned to the user. This is called for each streaming chunk in the response. Nr))r'rkrorer)r)r*)async_post_call_streaming_deployment_hook!s z6CustomLogger.async_post_call_streaming_deployment_hook exceptionoriginal_model_groupcrGr9r))r'rqrrr(r)r)r* log_model_group_rate_limit_error/riz-CustomLogger.log_model_group_rate_limit_errororiginal_exceptioncrGr9r)r'rrr(rtr)r)r*log_success_fallback_event4riz'CustomLogger.log_success_fallback_eventcrGr9r)rur)r)r*log_failure_fallback_event9riz'CustomLogger.log_failure_fallback_eventcCdS)zt Translates the input params, from the provider's native format to the litellm.completion() format. Nr))r'r(r)r)r*!translate_completion_input_params@z.CustomLogger.translate_completion_input_paramscCrx)z\ Translates the output params, from the OpenAI format to the custom format. Nr))r'rlr)r)r*"translate_completion_output_paramsHrzz/CustomLogger.translate_completion_output_paramscompletion_streamcCrx)z^ Translates the streaming chunk, from the OpenAI format to the custom format. Nr))r'r|r)r)r*,translate_completion_output_params_streamingPrzz9CustomLogger.translate_completion_output_params_streaming logged_itemstandard_logging_payloadcs td)z - Decide if the result should be logged to Argilla. - Modify the result before logging to Argilla. - Return None if the result should not be logged to Argilla. z"async_dataset_hook not implemented)NotImplementedError)r'r~rr)r)r*async_dataset_hookZs zCustomLogger.async_dataset_hookuser_api_key_dictcacherdatacrGr9r))r'rrrrer)r)r*async_pre_call_hookkrnz CustomLogger.async_pre_call_hookrequest_headerscrK)af Called after an LLM API call (success or failure) to allow injecting custom HTTP response headers. Args: - data: dict - The request data. - user_api_key_dict: UserAPIKeyAuth - The user API key dictionary. - response: Any - The response object (None for failure cases). - request_headers: Optional[Dict[str, str]] - The original request headers. Returns: - Optional[Dict[str, str]]: A dictionary of headers to inject into the HTTP response. Return None to not inject any headers. Nr))r'rrrlrr)r)r*%async_post_call_response_headers_hookvz2CustomLogger.async_post_call_response_headers_hook traceback_strrcrK)au Called after an LLM API call fails. Can return or raise HTTPException to transform error responses. Args: - request_data: dict - The request data. - original_exception: Exception - The original exception that occurred. - user_api_key_dict: UserAPIKeyAuth - The user API key dictionary. - traceback_str: Optional[str] - The traceback string. Returns: - Optional[HTTPException]: Return an HTTPException to transform the error response sent to the client. Return None to use the original exception. Nr))r'rkrtrrr)r)r*async_post_call_failure_hookrz)CustomLogger.async_post_call_failure_hookcrGr9r))r'rrrlr)r)r*async_post_call_success_hookz)CustomLogger.async_post_call_success_hookresultcs ||fSzUFor masking logged request/response. Return a modified version of the request/result.r)r'r(rrer)r)r*async_logging_hookszCustomLogger.async_logging_hookcCs||fSrr)rr)r)r* logging_hookszCustomLogger.logging_hookcrGr9r))r'rrrer)r)r*async_moderation_hookrz"CustomLogger.async_moderation_hookcrGr9r))r'rrlr)r)r*async_post_call_streaming_hooksz+CustomLogger.async_post_call_streaming_hookcCs |2z 3dHW}|Vq6dSr9r))r'rrlrkitemr)r)r*'async_post_call_streaming_iterator_hooksz4CustomLogger.async_post_call_streaming_iterator_hookcCs\z||d<||d<d|d<|||d|WdSty-|dtYdSwNr;r<Z pre_api_calllog_event_typez$Custom Logger - model call details: Custom Logger Error - r1 traceback format_excr'r;r<r( print_verbose callback_funcr)r)r*log_input_events zCustomLogger.log_input_eventcsdz||d<||d<d|d<||IdH|d|WdSty1|dtYdSwrrrr)r)r*async_log_input_events  z"CustomLogger.async_log_input_eventcCsDzd|d<|||||WdSty!|dtYdSwNZ post_api_callrrrr'r(r@rArBrrr)r)r* log_events zCustomLogger.log_eventcsLzd|d<|||||IdHWdSty%|dtYdSwrrrr)r)r*async_log_events zCustomLogger.async_log_eventr@crK)z This log gets called after the MCP tool call is made. Useful if you want to modiy the standard logging payload after the MCP tool call is made. Nr)r?r)r)r*async_post_mcp_tool_call_hookrdz*CustomLogger.async_post_mcp_tool_call_hookstreamcustom_llm_providerc difS)a Hook to determine if agentic loop should be executed. Called after receiving response from model, before returning to user. USE CASE: Enables transparent server-side tool execution for models that don't natively support server-side tools. User makes ONE API call and gets back the final answer - the agentic loop happens transparently on the server. Example use cases: - WebSearch: Intercept WebSearch tool calls for Bedrock/Claude, execute litellm.search(), return final answer with search results - Code execution: Execute code in sandboxed environment, return results - Database queries: Execute queries server-side, return data to model - API calls: Make external API calls and inject responses back into context Flow: 1. User calls litellm.messages.acreate(tools=[...]) 2. Model responds with tool_use 3. THIS HOOK checks if tool should run server-side 4. If True, async_run_agentic_loop executes the tool 5. User receives final answer (never sees intermediate tool_use) Args: response: Response from model (AnthropicMessagesResponse or AsyncIterator) model: Model name messages: Original messages sent to model tools: List of tool definitions from request stream: Whether response is streaming custom_llm_provider: Provider name (e.g., "bedrock", "anthropic") kwargs: Additional request parameters Returns: (should_run, tools): should_run: True if agentic loop should execute tools: Dict with tool_calls and metadata for execution Example: # Detect WebSearch tool call if has_websearch_tool_use(response): return True, { "tool_calls": extract_tool_calls(response), "tool_type": "websearch" } return False, {} Fr)r'rlr;r<rUrrr(r)r)r*async_should_run_agentic_loops8z*CustomLogger.async_should_run_agentic_loop"anthropic_messages_provider_config*anthropic_messages_optional_request_params logging_objLiteLLMLoggingObjc rK)aI Hook to execute agentic loop based on context from should_run hook. Called only if async_messages_should_run_agentic_loop returns True. USE CASE: Execute server-side tools and orchestrate the agentic loop to return a complete answer to the user in a single API call. What to do here: 1. Extract tool calls from tools dict 2. Execute the tools (litellm.search, code execution, DB queries, etc.) 3. Build assistant message with tool_use blocks 4. Build user message with tool_result blocks containing results 5. Make follow-up litellm.messages.acreate() call with results 6. Return the final response Args: tools: Dict from async_should_run_agentic_loop Contains tool_calls and metadata model: Model name messages: Original messages sent to model response: Original response from model (with tool_use) anthropic_messages_provider_config: Provider config for making requests anthropic_messages_optional_request_params: Request parameters (tools, etc.) logging_obj: LiteLLM logging object stream: Whether response is streaming kwargs: Additional request parameters Returns: Final response after executing agentic loop (AnthropicMessagesResponse with final answer) Example: # Extract tool calls tool_calls = agentic_context["tool_calls"] # Execute searches in parallel search_results = await asyncio.gather( *[litellm.asearch(tc["input"]["query"]) for tc in tool_calls] ) # Build messages with tool results assistant_msg = {"role": "assistant", "content": [...tool_use blocks...]} user_msg = {"role": "user", "content": [...tool_result blocks...]} # Make follow-up request from litellm.anthropic_interface import messages final_response = await messages.acreate( model=model, messages=messages + [assistant_msg, user_msg], max_tokens=anthropic_messages_optional_request_params.get("max_tokens"), **anthropic_messages_optional_request_params ) return final_response Nr)) r'rUr;r<rlrrrrr(r)r)r*async_run_agentic_loopVsDz#CustomLogger.async_run_agentic_loopcr)zW Hook to determine if chat completion agentic loop should be executed. Fr)rr)r)r*-async_should_run_chat_completion_agentic_loops z:CustomLogger.async_should_run_chat_completion_agentic_loopoptional_paramsc rK)ze Hook to execute chat completion agentic loop based on context from should_run hook. Nr)) r'rUr;r<rlrrrr(r)r)r*&async_run_chat_completion_agentic_loopsz3CustomLogger.async_run_chat_completion_agentic_loopstandard_logging_objectcCs*d}gd}|D] }|j|||dqdS)a Truncate error strings and message content in logging payload Some loggers like DataDog/ GCS Bucket have a limit on the size of the payload. (1MB) This function truncates the error string and the message content if they exceed a certain length. i')Z error_strr<rl)r field_name max_lengthN)_truncate_field)r'rZMAX_STR_LENGTHZfields_to_truncatefieldr)r)r*)truncate_standard_logging_payload_contents z6CustomLogger.truncate_standard_logging_payload_contentrrcCs@||}|rt|}t||kr|j||d||<dSdSdS)aP Helper function to truncate a field in the logging payload This converts the field to a string and then truncates it if it exceeds the max length. Why convert to string ? 1. User was sending a poorly formatted list for `messages` field, we could not predict where they would send content - Converting to string and then truncating the logged content catches this 2. We want to avoid modifying the original `messages`, `response`, and `error_str` in the logging payload since these are in kwargs and could be returned to the user )textrN)r/strlen_truncate_text)r'rrrZ field_valueZ str_valuer)r)r*rs  zCustomLogger._truncate_fieldrcCs t||kr|d|dS|S)z&Truncate text if it exceeds max_lengthNzC...truncated by litellm, this logger does not support large content)r)r'rrr)r)r*rs  zCustomLogger._truncate_textcCs,ddlm}m}|durdS||vr|S|S)z Select the metadata field to use for logging 1. If `litellm_metadata` is in the request kwargs, use it 2. Otherwise, use `metadata` r)LITELLM_METADATA_FIELDOLD_LITELLM_METADATA_FIELDN)litellm.constantsrr)r'r]rrr)r)r*_select_metadata_fields  z#CustomLogger._select_metadata_fieldmodel_call_detailscCsddl}ddlm}ddlm}m}m}t|dd}t|dd}|dur(|s(|S||} |d} | dur7| S|| } |rI|D] } | | vrH| | =q?|rd } d |pQgvrd| d durd|| d g| d <d |phgvr| d dur| d }t|t rd |vrddlm }||}t|d t r|d D]&}t|t rd|vrt|dt r|dD]}t|t rd|vr| |d<qq|| d <n|||| d dgd}|}|| d <| | d<| S)a Redacts or excludes fields from StandardLoggingPayload before callbacks receive it. This method handles two features: 1. turn_off_message_logging: When True, redacts messages and responses 2. standard_logging_payload_excluded_fields: Removes specified fields entirely Return a modified copy of the provided logging payload. This is useful for logging payloads that contain sensitive information. rN)copy)ChoicesMessagerr$FZ(standard_logging_payload_excluded_fieldsrzredacted-by-litellmr<contentrloutput)deepcopyrr)message)choices) litellmrrrrr2r/Z model_dump isinstancedictrr3)r'rrrrrrr$Zexcluded_fieldsZmodel_call_details_copyrZstandard_logging_object_copyrZ redacted_strrlrZ response_copyZ output_itemZ content_itemZmodel_responseZmodel_response_dictr)r)r*7redact_standard_logging_payload_from_model_call_detailssl          zDCustomLogger.redact_standard_logging_payload_from_model_call_details object_keycrK)z_ Get the proxy server request from cold storage using the object key directly. Nr))r'rr)r)r*:get_proxy_server_request_from_cold_storage_with_object_keygszGCustomLogger.get_proxy_server_request_from_cold_storage_with_object_keyc Csz7ddl}ddlm}|j}|D]}t|dr+|d||j|dWdSq|d|dWdSty]}zddlm}|d |d t |WYd}~dSd}~ww) z Handle callback logging failures by incrementing Prometheus metrics. Call this method in exception handlers within your callback when logging fails. rNr "increment_callback_logging_failurez)Incrementing callback failure metric for )r,zENo callback with increment_callback_logging_failure method found for z2. Ensure 'prometheus' is in your callbacks config.z%Error in handle_callback_failure for z: ) rlitellm._loggingr Zlogging_callback_managerZ_get_all_callbackshasattrdebugrr1r)r'r,rr Z all_callbacksZ callback_objer)r)r*handle_callback_failureps,       z$CustomLogger.handle_callback_failurepayloadr max_depthcs|dg}t|tr|ng}tdt|d|r&|j||d|d<d}|dgp/gD]}t|trH|dg}t|trH|t|7}q0td|d|S u Removes or redacts base64-encoded file data (e.g., PDFs, images, audio) from messages and responses before sending to SQS. Behavior: • Drop entries with a 'file' key. • Drop entries with type == 'file' or any non-text type. • Keep untyped or text content. • Recursively redact inline base64 blobs in *any* string field, at any depth. r<z%[CustomLogger] Stripping base64 from z messages)r<rrrz0[CustomLogger] Completed base64 strip; retained z content itemsr/rr3r rr_process_messagesrr'rrZ raw_messagesr<Z total_itemsmrr)r)r*_strip_base64_from_messagess*       z(CustomLogger._strip_base64_from_messagescCs|dg}t|tr |ng}tdt|d|r%|j||d|d<d}|dgp.gD]}t|trG|dg}t|trG|t|7}q/td|d|Srrrr)r)r* _strip_base64_from_messages_syncs(       z-CustomLogger._strip_base64_from_messages_syncrvaluedepthcskrtdddSt|tr.t|r,td|dddtd|S|St|tr>fd d |DSt|t rPfd d | DS|S) z\Recursively redact inline base64 from any nested structure with a max recursion depth limit.z#[CustomLogger] Max recursion depth z reached while redacting base64z[MAX_DEPTH_REACHED]z.[CustomLogger] Redacted inline base64 string: N(z...z[BASE64_REDACTED]cs g|] }j|ddqS)rrr_redact_base64).0vrrr'r)r* sz/CustomLogger._redact_base64..cs&i|]\}}|j|ddqSrr)rkrrr)r* sz/CustomLogger._redact_base64..) r warningrr_BASE64_INLINE_PATTERNsearchrsubr3ritems)r'rrrr)rr*rs*      zCustomLogger._redact_base64rcCs8t|tsdSd|vr dS|d}t|to|dk S)z4Return True if this content item should be retained.TfileFtyper)rrr/r)r'rctyper)r)r*_should_keep_contents  z!CustomLogger._should_keep_contentc Csg}|D]T}t|ts q|d}t|tr2g}|D]}|j|dr,||j||dq||d<n |j||d|d<t|D]\}} |dkrR|j| |d||<qA||q|S)Nrr)rr)rrr/r3rappendrr) r'r<rZfiltered_messagesmsgcontentscleanedckeyvalr)r)r*rs.        zCustomLogger._process_messages)FTr9)NNNNFF)NNNFF)NNF)NN)]__name__ __module__ __qualname__boolr+ staticmethodrrrr7r=rCrDrErFrHrJrrLrMrNrrrrrintrr[r\r r"r`rrcrrrfrhrjrrmrpr1rsrvrwrryrr r{rr}rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr rrrrrr)r)r)r*r#Csp %                                                  :   F         b  # ) ) # r#)9rertypingrrrrrrrr Zpydanticr rr rr Z"litellm.types.integrations.argillarZlitellm.types.llms.openairrZ"litellm.types.prompts.init_promptsrZlitellm.types.utilsrrrrrrrrZfastapirZopentelemetry.tracer_SpanZlitellm.caching.cachingrZ*litellm.litellm_core_utils.litellm_loggingrrr0rZlitellm.types.mcprr r!Zlitellm.types.routerr"ZMCPDuringCallRequestObjectZMCPDuringCallResponseObjectcompile MULTILINErr#r)r)r)r*sB(    (