o ưi$@slddlmZddlmZmZmZmZmZmZm Z ddl Z ddl m Z ddl mZddlmZddlmZer?ddlmZeZneZd ed ed efd d Zdeed ed eeeeeeffddZdeededeededededed efddZdededefddZdededefddZ deeed efdd Z!de eeeeeeffd efd!d"Z"dS)#)Enum) TYPE_CHECKINGAnyDictListOptionalTupleUnionN)verbose_router_logger) CustomLogger) add_fallback_headers_to_response)LiteLLMParamsTypedDict)Router model_group fallback_keyreturncCsVtjD]%}t|tr|j}n|}||dr(||dd}||kr(dSqdS)z Handles wildcard routing scenario where fallbacks set like: [{"gpt-3.5-turbo": ["claude-3-haiku"]}] but model_group is like: "openai/gpt-3.5-turbo" Returns: - True if the stripped model group == fallback_key /TF)litellmZ provider_list isinstancervalue startswithreplace)rrprovider _providerZstripped_model_grouprc/home/app/Keep/.python/lib/python3.10/site-packages/litellm/router_utils/fallback_event_handlers.py_check_stripped_model_groups r fallbackscCsd}d}d}t|D]H\}}t|trGt|d|kr#||}n0t|t|ddr:|t|d}q t|ddkrF|}q t|trR||g}q |durk|dura|}||fS|durk||d}||fS)a) Returns: - fallback_model_group: List[str] of fallback model groups. example: ["gpt-4", "gpt-3.5-turbo"] - generic_fallback_idx: int of the index of the generic fallback in the fallbacks list. Checks: - exact match - stripped model group match - generic fallback Nr)rr*) enumeraterdictlistkeysrstrpop)rrZgeneric_fallback_idxZstripped_model_fallbackfallback_model_groupidxitemrrrget_fallback_model_group-s2     r)argslitellm_routerr&original_model_grouporiginal_exception max_fallbacksfallback_depthc s,||kr||}|D]} | |krq zc|j||d}td| t| tr,| |d<n t| tr6|| |did|ddi|d}||d<||d <|j |i|IdH} td t | |d } t |||d IdH| WSt y} z| }t |||d IdHWYd} ~ q d} ~ ww|) a Loops through all the fallback model groups and calls kwargs["original_function"] with the arguments and keyword arguments provided. If the call is successful, it logs the success and returns the response. If the call fails, it logs the failure and continues to the next fallback model group. If all fallback model groups fail, it raises the most recent exception. Args: litellm_router: The litellm router instance. *args: Positional arguments. fallback_model_group: List[str] of fallback model groups. example: ["gpt-4", "gpt-3.5-turbo"] original_model_group: The original model group. example: "gpt-3.5-turbo" original_exception: The original exception. **kwargs: Keyword arguments. Returns: The response from the successful fallback model group. Raises: The most recent exception if all fallback model groups fail. )kwargsezFalling back to model_group = modelmetadatarNr/r.zSuccessful fallback b/w models.)responseZattempted_fallbacksr,r0r-)Z log_retryr inforr$r!update setdefaultgetZasync_function_with_fallbacksr log_success_fallback_event Exceptionlog_failure_fallback_event) r+r&r,r-r.r/r*r0Zerror_from_fallbacksZmgr5r1rrrrun_async_fallbackUs\        r>r0c ntjt}|D]+}z |j|||dIdHWq ty4}ztdt|WYd}~q d}~wwdS)a Log a successful fallback event to all registered callbacks. Uses LoggingCallbackManager.get_custom_loggers_for_type() to get deduplicated CustomLogger instances from all callback lists. Args: original_model_group (str): The original model group before fallback. kwargs (dict): kwargs for the request Note: Errors during logging are caught and reported but do not interrupt the process. r6Nz%Error in log_success_fallback_event: ) rlogging_callback_managerget_custom_loggers_for_typer r;r<r errorr$r,r0r-Zcustom_loggersZ_callback_custom_loggerr1rrrr;$ r;c r?)a Log a failed fallback event to all registered callbacks. Uses LoggingCallbackManager.get_custom_loggers_for_type() to get deduplicated CustomLogger instances from all callback lists. Args: original_model_group (str): The original model group before fallback. kwargs (dict): kwargs for the request Note: Errors during logging are caught and reported but do not interrupt the process. r6Nz%Error in log_failure_fallback_event: ) rr@rAr r=r<r rBr$rCrrrr=rDr=cCst|dust|trt|dkrdStdd|DrdStdd|Dr8tjD] }||dvr7dSq*dS)ar Checks if the fallbacks list is a list of strings or a list of dictionaries. If - List[str]: e.g. ["claude-3-haiku", "openai/o-1"] - List[Dict[, Any]]: e.g. [{"model": "claude-3-haiku", "messages": [{"role": "user", "content": "Hey, how's it going?"}]}] If [{"gpt-3.5-turbo": ["claude-3-haiku"]}] then standard format. NrFcs|]}t|tVqdSN)rr$.0r(rrr z6_check_non_standard_fallback_format..TcsrErF)rr!rGrrrrIrJ)rr"lenallr __annotations__r#)rkeyrrr#_check_non_standard_fallback_formats rOcCsdSrFr)rrrrr run_non_standard_fallback_formatsrP)#enumrtypingrrrrrrr rZlitellm._loggingr Z"litellm.integrations.custom_loggerr Z/litellm.router_utils.add_retry_fallback_headersr Zlitellm.types.routerr Zlitellm.routerrZ_RouterZ LitellmRouterr$boolrintr)r<r>r!r;r=rOrPrrrrsp $      ( O " "