o ưi@sdZddlZddlZddlmZmZmZmZmZm Z m Z ddl Z ddl m Z ddlmZddlmZddlmZddlmZmZmZmZdd lmZdd lmZdd lmZGd d d eZ dS)z WebSearch Interception Handler CustomLogger that intercepts WebSearch tool calls for models that don't natively support web search (e.g., Bedrock/Claude) and executes them server-side using litellm router's search tools. N)AnyDictListOptionalTupleUnioncast)verbose_logger)messages)LITELLM_WEB_SEARCH_TOOL_NAME) CustomLogger)get_litellm_web_search_tool"get_litellm_web_search_tool_openaiis_web_search_tool"is_web_search_tool_chat_completion)WebSearchTransformation)WebSearchInterceptionConfig) LlmProviderscsteZdZdZ  d8deeeeefdeeffdd Z de ee fdee d ee fd d Z ed ed dfd dZdedee de d ee fddZde dedee deee dedede d eee ffddZde dedee deee dedede d eee ffddZde dedee de de de de dede d e fddZde dedee de d e de dede d e fd!d"Zed e de d efd#d$Zede d e fd%d&Zdedee d'ee d(ee de de dede d e fd)d*Zd+ed efd,d-Z .d9dedee d'ee d e de dede d/ed e fd0d1Zd efd2d3Zed4e ee fd5e ee fd dfd6d7Z Z!S):WebSearchInterceptionLoggerab CustomLogger that intercepts WebSearch tool calls for models that don't natively support web search. Implements agentic loop: 1. Detects WebSearch tool_use in model response 2. Executes litellm.asearch() for each query using router's search tools 3. Makes follow-up request with search results 4. Returns final response Nenabled_providerssearch_tool_namecs@t|durtjjg|_ndd|D|_||_d|_dS)a Args: enabled_providers: List of LLM providers to enable interception for. Use LlmProviders enum values (e.g., [LlmProviders.BEDROCK]) If None or empty list, enables for ALL providers. Default: None (all providers enabled) search_tool_name: Name of search tool configured in router's search_tools. If None, will attempt to use first available search tool. NcSs g|] }t|tr |jn|qS) isinstancervalue).0prrj/home/app/Keep/.python/lib/python3.10/site-packages/litellm/integrations/websearch_interception/handler.py @sz8WebSearchInterceptionLogger.__init__..F)super__init__rZBEDROCKrrrZ_request_has_websearch)selfrr __class__rrr-s  z$WebSearchInterceptionLogger.__init__kwargs call_typereturnc s |ddp|didd}|s/ztj|ddd\}}}}Wn ty.d}Ynw||jvr6dS|d}|s?dStdd |D}|sLdStd g}|D]*}t|rzt } | | td |d d d|dddt qU| |qU||d<|S)aE Pre-call hook to convert native Anthropic web_search tools to regular tools. This prevents Bedrock from trying to execute web search server-side (which fails). Instead, we convert it to a regular tool so the model returns tool_use blocks that we can intercept and execute ourselves. custom_llm_providerlitellm_paramsmodel)r)Ntoolscs|]}t|VqdSNrrtrrr bzMWebSearchInterceptionLogger.async_pre_call_deployment_hook..zMWebSearchInterception: Converting native web_search tools to LiteLLM standard!WebSearchInterception: Converted nameunknown (type=typenone) to ) getlitellmZget_llm_provider Exceptionranyr debugrrappendr ) r r#r$r&_r* has_websearchconverted_toolstoolZconverted_toolrrrasync_pre_call_deployment_hookGsD        z:WebSearchInterceptionLogger.async_pre_call_deployment_hookconfigc Csp|dd}|dd}d}|dur2g}|D]}z t|}||Wqty1||Yqw|||dS)a Initialize WebSearchInterceptionLogger from proxy config.yaml parameters. Args: config: Configuration dictionary from litellm_settings.websearch_interception_params Returns: Configured WebSearchInterceptionLogger instance Example: From proxy_config.yaml: litellm_settings: websearch_interception_params: enabled_providers: ["bedrock"] search_tool_name: "my-perplexity-search" Usage: config = litellm_settings.get("websearch_interception_params", {}) logger = WebSearchInterceptionLogger.from_config_yaml(config) rNr)rr)r9rr> ValueError)clsrDZenabled_providers_strrrproviderZ provider_enumrrrfrom_config_yaml~s   z,WebSearchInterceptionLogger.from_config_yamlr)r c sF|didd}td|d|jpd|jdur1||jvr1td|d |jdS|d }|s:dStd d |D}|sGdStd |g}|D]*}t|rxt} || td|ddd|dddtqS||qS||d <tddd|D|drtdd|d<d|d<|S)a Pre-request hook to convert native web search tools to LiteLLM standard. This hook is called before the API request is made, allowing us to: 1. Detect native web search tools (web_search_20250305, etc.) 2. Convert them to LiteLLM standard format (litellm_web_search) 3. Convert stream=True to stream=False for interception This prevents providers like Bedrock from trying to execute web search natively (which fails), and ensures our agentic loop can intercept tool_use. Returns: Modified kwargs dict with converted tools, or None if no modifications needed r(r&r'zEWebSearchInterception: Pre-request hook called - custom_llm_provider=z - enabled_providers=ALLNz+WebSearchInterception: Skipping - provider z not in r*csr+r,r-r.rrrr0r1zEWebSearchInterceptionLogger.async_pre_request_hook..z?WebSearchInterception: Pre-request hook triggered for provider=r2r3r4r5r6r7r8z/WebSearchInterception: Tools after conversion: cSsg|]}|dqS)r3)r9r.rrrrszFWebSearchInterceptionLogger.async_pre_request_hook..streamz=WebSearchInterception: Converting stream=True to stream=FalseFTZ(_websearch_interception_converted_stream) r9r r=rr<rr r>r ) r r)r r#r&r*r@rArBZ standard_toolrrrasync_pre_request_hooksb      z2WebSearchInterceptionLogger.async_pre_request_hookresponser*rJr&cstd|d|tdt||jdur1||jvr1td|d|jddifStd d |p8gD}|sGtd difStj||d d \} } | s\tddifStdt| dg} t|t ru| dg} nt |dgp|g} | D]H} t| t r| d}nt | dd}|dvrt| t r| | qd|i}|dkrt | dd|d<t | dd|d<nt | dd|d<| |q| rtdt| d| d|d | d}d|fS)z Check if WebSearch tool interception is needed for Anthropic Messages API. This is the legacy method for Anthropic-style responses. For chat completions, use async_should_run_chat_completion_agentic_loop instead. z-WebSearchInterception: Hook called! provider= , stream=&WebSearchInterception: Response type: N)WebSearchInterception: Skipping provider  (not in enabled list: )Fcsr+r,r-r.rrrr0r1zLWebSearchInterceptionLogger.async_should_run_agentic_loop..z4WebSearchInterception: No web search tool in requestZ anthropicrLrJresponse_formatzAWebSearchInterception: No WebSearch tool_use detected in response WebSearchInterception: Detected / WebSearch tool call(s), executing agentic loopcontentr6)thinkingZredacted_thinkingrWr' signaturedataz!WebSearchInterception: Extracted z thinking block(s) from response websearch) tool_calls tool_typerGrSthinking_blocksT) r r=r6rr<rtransform_requestlenrdictr9getattrr>)r rLr)r r*rJr&r#has_websearch_toolshould_interceptr[r]rVblockZ block_typeZthinking_block_dict tools_dictrrrasync_should_run_agentic_loops|         z9WebSearchInterceptionLogger.async_should_run_agentic_loopc std|d|tdt||jdur1||jvr1td|d|jddifStd d |p8gD}|sGtd difStj||d d \} } | s\tddifStdt| d| d|d d} d| fS)z Check if WebSearch tool interception is needed for Chat Completions API. Similar to async_should_run_agentic_loop but for OpenAI-style chat completions. z=WebSearchInterception: Chat completion hook called! provider=rMrNNrOrPrQFcsr+r,)rr.rrrr0}r1z\WebSearchInterceptionLogger.async_should_run_chat_completion_agentic_loop..z thinking.budget_tokens``. If the constraint is violated, auto-adjust to ``budget_tokens + 1024``. max_tokensirW budget_tokensNrzvWebSearchInterception: max_tokens=%s <= thinking.budget_tokens=%s, adjusting to %s to satisfy Anthropic API constraint) r9rr`intfloatmathisfiniteceilr r=)ror#rrZthinking_paramrsadjustedrrr_resolve_max_tokenss*     z/WebSearchInterceptionLogger._resolve_max_tokenscsdhfdd|DS)uBuild kwargs for the follow-up call, excluding internal keys. ``litellm_logging_obj`` MUST be excluded so the follow-up call creates its own ``Logging`` instance via ``function_setup``. Reusing the initial call's logging object triggers the dedup flag (``has_logged_async_success``) which silently prevents the initial call's spend from being recorded — the root cause of the SpendLog / AWS billing mismatch. litellm_logging_objc(i|]\}}|ds|vr||qS_websearch_interception startswithrkvZ_internal_keysrr zHWebSearchInterceptionLogger._prepare_followup_kwargs..)items)r#rrr_prepare_followup_kwargss z4WebSearchInterceptionLogger._prepare_followup_kwargsr[r]c  sg} |D]/} | dd} | r"td| d| || qtd| dd| |qtdt| d tj| d d iId H} g} t | D]C\}}t |t rrt d |dt || dt |qQt |t r| tt |qQtdt|d|| t |qQtj|| |d\}}||tt|g}tdtdt|td|t|dd p|dd}|}z\|||}td|ddd|D}||}|d ur|jdi}|d|}td|tjd#|||d||Id H}td t|td!||WSt yG}ztd"||t|t| t |d }~ww)$z3Execute litellm.search() and make follow-up requestinputquery1WebSearchInterception: Queuing search for query=''!WebSearchInterception: Tool call id has no query!WebSearchInterception: Executing  search(es) in parallelreturn_exceptionsTNWebSearchInterception: Search  failed with error: Search failed: .WebSearchInterception: Unexpected result type at index )r[search_resultsr]zCWebSearchInterception: Making follow-up request with search results1WebSearchInterception: Follow-up messages count: z3WebSearchInterception: Last message (tool_result): Zlitellm_call_idr4z(WebSearchInterception: Using max_tokens=z for follow-up requestcSsi|] \}}|dkr||qS)rrrrrrrrv zEWebSearchInterceptionLogger._execute_agentic_loop..Zagentic_loop_paramsr))WebSearchInterception: Using model name: )rrr r)CWebSearchInterception: Follow-up request completed, response type: z'WebSearchInterception: Final response: zaWebSearchInterception: Follow-up request failed [call_id=%s model=%s messages=%d searches=%d]: %sr)r9r r=r>_execute_search_create_empty_search_resultr_asynciogather enumeraterr;errorstrrr6rtransform_responserrarzrrZmodel_call_detailsanthropic_messagesZacreate exception)r r)r r[r]rjrkrJr# search_tasks tool_callrrfinal_search_resultsiresultassistant_messageZ user_messagefollow_up_messagesZ_call_idfull_model_namerrZ"optional_params_without_max_tokenskwargs_for_followupZagentic_paramsfinal_responseerrrrms               z1WebSearchInterceptionLogger._execute_agentic_looprc szzddlm}Wntytdd}Ynwd}|durvt|drvjrZfdd|jD}|rP|d}|did }td jd |d n td jd|sv|jrv|jd}|did }td|d |sd}td|d td|d|d t j ||dIdH}t |}td|dt |d|WSty} ztd|dt| d} ~ ww)z7Execute a single web search using router's search toolsr) llm_routerz~WebSearchInterception: Could not import llm_router from proxy_server, falling back to direct litellm.asearch() with perplexityN search_toolscs g|] }|djkr|qS)r)r9r)rrBr rrrs z?WebSearchInterceptionLogger._execute_search..r(search_providerz*WebSearchInterception: Found search tool 'z' with provider 'rz$WebSearchInterception: Search tool 'zD' not found in router, falling back to first available or perplexityzHWebSearchInterception: Using first available search tool with provider 'Z perplexityzUWebSearchInterception: No search tools configured in router, using default provider 'z-WebSearchInterception: Executing search for 'z' using provider ')rrz-WebSearchInterception: Search completed for 'z', got z charsz*WebSearchInterception: Search failed for 'z': )Zlitellm.proxy.proxy_serverr ImportErrorr r=hasattrrrr9r:ZasearchrZformat_search_responser_r;rr) r rrrZmatching_toolsZ search_toolZ first_toolrZsearch_result_textrrrrrst        z+WebSearchInterceptionLogger._execute_searchrgrSc  sg} |D][} d} d| vrt| dtr| dd} nd| vr9| d} t| tr9| di} t| tr9| d} | rMtd| d| || qtd| d d | |qtd t| d t j | d diIdH}g}t |D]C\}}t|t rt d|dt||dt|q}t|tr|tt|q}tdt|d||t|q}tj|||d\}}|dkr||gttt|}n ||tt|g}tdtdt|z`hdfdd|D}|}d|vr!|d}||s!d|vr!|d|}td||d}dd|D}tjd#|||d ||IdH}td!t||WSt yl}z td"t|d}~ww)$zCExecute litellm.search() and make follow-up chat completion requestNrrfunction argumentsrrrrrrrrTrrrrr)r[rrSrgzSWebSearchInterception: Making follow-up chat completion request with search resultsr>r~model_alias_mapr{r& acompletionstream_responsecustom_prompt_dictcr|r}rrZinternal_paramsrrrErzUWebSearchInterceptionLogger._execute_chat_completion_agentic_loop..r&/rr*cSsi|] \}}|dvr||qS)>Z extra_bodyrrr*rrrrrrr]r)r)r r*rz1WebSearchInterception: Follow-up request failed: r)rr`r9r r=r>rrr_rrrr;rrrr6rrrrrrr:rr)r r)r r[rorkrJr#rSrrrfuncargsrrrrrZtool_messages_or_userrrrr&Z tools_paramZoptional_params_cleanrrrrrrps                  zAWebSearchInterceptionLogger._execute_chat_completion_agentic_loopcsdS)zs$