o ưi"@spdZddlZddlZddlmZddlmZmZmZmZddl m Z ddl m Z GdddZ Gd d d ZdS) zA Class to handle llm wildcard routing and regex pattern matching N)Match)DictListOptionalTuple)get_llm_provider)verbose_router_loggerc @s\eZdZededeeeffddZedeee efde eee effddZ dS) PatternUtilspatternreturncs*gd}ttfdd|Df}|S)z Calculate pattern specificity based on length and complexity. Args: pattern: Regex pattern to analyze Returns: Tuple of (length, complexity) for sorting ) *+?\^$|()c3s|]}|VqdSN)count).0charr e/home/app/Keep/.python/lib/python3.10/site-packages/litellm/router_utils/pattern_match_deployments.py s  z=PatternUtils.calculate_pattern_specificity..)lensum)r Zcomplexity_charsZret_valrrrcalculate_pattern_specificitys  z*PatternUtils.calculate_pattern_specificitypatternscCst|ddddS)z Cached property for patterns sorted by specificity. Returns: Sorted list of pattern-deployment tuples cSst|dS)Nr)r r)xrrr/sz.PatternUtils.sorted_patterns..T)keyreverse)sorteditemsr rrrsorted_patterns#s  zPatternUtils.sorted_patternsN) __name__ __module__ __qualname__ staticmethodstrrintrrrr(rrrrr sr c @seZdZdZddZdedefddZdedefd d Zd e d e ede efd dZ dde ede e ede e efddZ ed e dedefddZ ddede ede e efddZ ddede ede efddZdS)PatternMatchRouterz Class to handle llm wildcard routing and regex pattern matching doc: https://docs.litellm.ai/docs/proxy/configs#provider-specific-wildcard-routing This class will store a mapping for regex pattern: List[Deployments] cCs i|_dSrr'selfrrr__init__=s zPatternMatchRouter.__init__r llm_deploymentcCs2||}||jvrg|j|<|j||dS)z Add a regex pattern and the corresponding llm deployments to the patterns Args: pattern: str llm_deployment: str or List[str] N)_pattern_to_regexr append)r1r r3regexrrr add_pattern@s  zPatternMatchRouter.add_patternr cCst|ddS)a0 Convert a wildcard pattern to a regex pattern example: pattern: openai/* regex: openai/.* pattern: openai/fo::*::static::* regex: openai/fo::.*::static::.* Args: pattern: str Returns: str: regex pattern z\*z(.*))reescapereplace)r1r rrrr4Nsz$PatternMatchRouter._pattern_to_regexmatched_pattern deploymentscCsDg}|D]}t|}tj||ddd|dd<||q|S)NZlitellm_paramsmodel)r; litellm_deployment_litellm_model)copydeepcopyr/set_deployment_model_namer5)r1r;r<Znew_deploymentsZ deploymentZnew_deploymentrrr#_return_pattern_matched_deploymentsfs   z6PatternMatchRouter._return_pattern_matched_deploymentsNrequestfiltered_model_namesc szA|durWdStj}|durfdd|Dng}|D]\}}|dur,||vr,qt||}|r>j||dWSqWdSty^}ztdt |WYd}~dSd}~ww)a Route a requested model to the corresponding llm deployments based on the regex pattern loop through all the patterns and find the matching pattern if a pattern is found, return the corresponding llm deployments if no pattern is found, return None Args: request: str - the received model name from the user (can be a wildcard route). If none, No deployments will be returned. filtered_model_names: Optional[List[str]] - if provided, only return deployments that match the filtered_model_names Returns: Optional[List[Deployment]]: llm deployments Ncsg|]}|qSr)r4)rmr0rr sz,PatternMatchRouter.route..)r;r<z#Error in PatternMatchRouter.route: ) r r(r r8matchrB Exceptionrdebugr-) r1rCrDr(Zregex_filtered_model_namesr Zllm_deployments pattern_matcherr0rroutevs2    zPatternMatchRouter.router>cCsLd|vr|S|d}|}t||kr|jS|D] }|d|d}q|S)a Set the model name for the matched pattern llm deployment E.g.: Case 1: model_name: llmengine/* (can be any regex pattern or wildcard pattern) litellm_params: model: openai/* if model_name = "llmengine/foo" -> model = "openai/foo" Case 2: model_name: llmengine/fo::*::static::* litellm_params: model: openai/fo::*::static::* if model_name = "llmengine/foo::bar::static::baz" -> model = "openai/foo::bar::static::baz" Case 3: model_name: *meta.llama3* litellm_params: model: bedrock/meta.llama3* if model_name = "hello-world-meta.llama3-70b" -> model = "bedrock/meta.llama3-70b" r )rgroupsrstringr:)r;r>Zwildcard_countZdynamic_segmentssegmentrrrrAs!  z,PatternMatchRouter.set_deployment_model_namer=custom_llm_providercCsP|durz t|d\}}}}Wn tyYnw||p'||d|S)z Check if a pattern exists for the given model and custom llm provider Args: model: str custom_llm_provider: Optional[str] Returns: bool: True if pattern exists, False otherwise N)r=/)rrHrL)r1r=rQ_rrr get_patterns  zPatternMatchRouter.get_patterncCs|||}|r |SgS)z Get the deployments by pattern Args: model: str custom_llm_provider: Optional[str] Returns: List[Dict]: llm deployments matching the pattern )rT)r1r=rQrJrrrget_deployments_by_patterns z-PatternMatchRouter.get_deployments_by_patternr)r)r*r+__doc__r2r-rr7r4rrrBrrLr,rArTrUrrrrr/4sX    * 5  r/)rVr?r8rtypingrrrrZ1litellm.litellm_core_utils.get_llm_provider_logicrZlitellm._loggingrr r/rrrrs   &