o ưins @s8ddlZddlZddlZddlmZddlmZmZmZm Z ddl m Z m Z m Z ddlmZmZddlmZddlmZddlTdd lmZ dYd e d eed eefddZ dYdeeed e d eed e eeeffddZded efddZdeded efddZdededed efddZdedededeed ef d d!Z ded"edeeded ef d#d$Z!d e d%ed&efd'd(Z"d)efd*d+Z#d e d efd,d-Z$ed.d/d&ed efd0d1Z%d e d efd2d3Z&d4ed efd5d6Z'd7e(fd8d9Z)d:e*d ee+ee(ffd;d<Z,d:e*d ee+ee(ffd=d>Z-d:e*d?e.d@dAe.dBd ee+ee(ffdCdDZ/d:e*d ee+ee(ffdEdFZ0d:e*d ee+ee(ffdGdHZ1d&ed efdIdJZ2dKdLZ3d eefdMdNZ4dOeed eefdPdQZ5 dZdedOeed eefdRdSZ6d%ed&ed ee7eeeffdTdUZ8dVed efdWdXZ9dS)[N) lru_cache)AnyListOptionalTuple) HTTPExceptionRequeststatus)Router provider_list)verbose_proxy_logger)STANDARD_CUSTOMER_ID_HEADERS)*)#CONFIGURABLE_CLIENTSIDE_AUTH_PARAMSFrequestuse_x_forwarded_forreturncCsBd}|durd|jvr|jd}|S|jdur|jj}|Sd}|S)NTzx-forwarded-for)headersclienthost)rr client_iprT/home/app/Keep/.python/lib/python3.10/site-packages/litellm/proxy/auth/auth_utils.py_get_request_ip_addresss  r allowed_ipscCs0|durdSt||d}||vrd|fSd|fS)z) Returns if ip is allowed or not N)TN)rrFT)r)rrrrrrr_check_valid_ipsr request_bodycCsNd}|d}|dur dSd|vsd|vsd|vsd|vrdSd|vr%d SdS) zh if 'api_base' in request body. Check if complete credentials given. Prevent malicious attacks. NmodelFZ sagemakerZbedrockZ vertex_aiZvertex_ai_betaapi_keyT)get)rZ given_modelrrrcheck_complete_credentials5s r!request_body_value regex_strcCst||s ||kr dSdS)zP Check if request_body_value matches the regex_str or is equal to param TF)rematchr"r#rrrcheck_regex_or_str_matchNsr'param#configurable_clientside_auth_paramscCsZ|durdS|D]"}t|tr||krdSt|tr*|dkr*t||ddr*dSqdS)zd Check if param is a str or dict and if request_body_value is in the list of allowed values NFTapi_baser&) isinstancestrDictr')r(r"r)itemrrr_is_param_allowedWs  r/r llm_routercCs~|durdS|j|d}|dur&|dddtvr&|j|dddd}|dur,dS|dus5|jdur7dSt|||jdS)z Check if model is allowed to use configurable client-side params - get matching model - check if 'clientside_configurable_parameters' is set for model - NF)Z model_group/r)r(r"r))Zget_model_group_infosplitr r)r/)rr(r"r0Z model_inforrr5_allow_model_level_clientside_configurable_parametersos"  r4general_settingscCsjddg}|D],}||vr2t|ds2|ddurdSt|||||ddur*dStd|dqdS) u Check if the request body is safe. A malicious user can set the api_base to their own domain and invoke POST /chat/completions to intercept and steal the OpenAI API key. Relevant issue: https://huntr.com/bounties/4001e1a2-7b7a-4776-a3ae-e6692ec3d997 r*base_url)rZallow_client_side_credentialsT)rr(r"r0zRejected Request: z is not allowed in request body. Enable with `general_settings::allow_client_side_credentials` on proxy config.yaml. Relevant Issue: https://huntr.com/bounties/4001e1a2-7b7a-4776-a3ae-e6692ec3d997)r!r r4 ValueError)rr5r0rZ banned_paramsr(rrris_request_body_safes.  r8 request_dataroutec sddlm}m}m}t|dIdHt||||dddt|dd|d d |d \}}|s=tt j d |d dd|vrm|d}|durSt dt jj||vrot d|d|tt j d|dddSdS)ax 1. Checks if request size is under max_request_size_mb (if set) 2. Check if request body is safe (example user has not set api_base in request body) 3. Check if IP address is allowed (if set) 4. Check if request route is an allowed route on the proxy (if set) Returns: - True Raises: - HTTPException if request fails initial auth checks r)r5r0 premium_user)rNrr)rr5r0rrrF)rrrzAccess forbidden: IP address z not allowed.) status_codedetailZallowed_routesTz=Trying to set allowed_routes. This is an Enterprise feature. zRoute z not in allowed_routes=zAccess forbidden: Route z not allowed)litellm.proxy.proxy_serverr5r0r;check_if_request_size_is_safer8r rrr ZHTTP_403_FORBIDDENr errorCommonProxyErrorsnot_premium_uservalue) rr9r:r5r0r;Z is_valid_ipZ passed_in_ipZ_allowed_routesrrrpre_db_read_auth_checkssJ      rD current_routec Csddlm}ddlm}m}z.|durWdS|durWdS|dg}||vr*WdS|D] }|j||dr9WdSq,WdStyY}zt d t |WYd}~dSd}~ww) ay Helper to check if the user defined public_routes on config.yaml Parameters: - current_route: str - the route the user is trying to call Returns: - bool - True if the route is defined in public_routes - bool - False if the route is not defined in public_routes Supports wildcard patterns (e.g., "/api/*" matches "/api/users", "/api/users/123") In order to use this the litellm config.yaml should have the following in general_settings: ```yaml general_settings: master_key: sk-1234 public_routes: ["LiteLLMRoutes.public_routes", "/spend/calculate", "/api/*"] ``` r) RouteChecksr5r;TFNZ public_routes)r:patternz"route_in_additonal_public_routes: ) Zlitellm.proxy.auth.route_checksrFr>r5r;r Z_route_matches_wildcard_pattern Exceptionr r@r,)rErFr5r;Zroutes_definedZ route_patternerrr route_in_additonal_public_routess,  rKc Csz"t|dr|jj|jjr|jjt|jjddWS|jjWStyG}ztdt |d|jj|jjWYd}~Sd}~ww)z Helper to get the route from the request remove base url from path if set e.g. `/genai/chat/completions` -> `/chat/completions r6r2Nzerror on get_request_route: z!, defaulting to request.url.path=) hasattrurlpath startswithr6lenrIr debugr,)rrJrrrget_request_route&s rR)maxsizecCs8gd}|D]\}}t|||}||kr|Sq|S)a Normalize request routes by replacing dynamic path parameters with placeholders. This prevents high cardinality in Prometheus metrics by collapsing routes like: - /v1/responses/1234567890 -> /v1/responses/{response_id} - /v1/threads/thread_123 -> /v1/threads/{thread_id} Args: route: The request route path Returns: Normalized route with dynamic parameters replaced by placeholders Examples: >>> normalize_request_route("/v1/responses/abc123") '/v1/responses/{response_id}' >>> normalize_request_route("/v1/responses/abc123/cancel") '/v1/responses/{response_id}/cancel' >>> normalize_request_route("/chat/completions") '/chat/completions' ))z3^(/(?:openai/)?v1/responses)/([^/]+)(/input_items)$\1/{response_id}\3)z.^(/(?:openai/)?v1/responses)/([^/]+)(/cancel)$rU)z%^(/(?:openai/)?v1/responses)/([^/]+)$\1/{response_id})z$^(/responses)/([^/]+)(/input_items)$rU)z^(/responses)/([^/]+)(/cancel)$rU)z^(/responses)/([^/]+)$rV)zB^(/(?:openai/)?v1/threads)/([^/]+)(/runs)/([^/]+)(/steps)/([^/]+)$z%\1/{thread_id}\3/{run_id}\5/{step_id})z:^(/(?:openai/)?v1/threads)/([^/]+)(/runs)/([^/]+)(/steps)$\1/{thread_id}\3/{run_id}\5)z;^(/(?:openai/)?v1/threads)/([^/]+)(/runs)/([^/]+)(/cancel)$rW)zH^(/(?:openai/)?v1/threads)/([^/]+)(/runs)/([^/]+)(/submit_tool_outputs)$rW)z2^(/(?:openai/)?v1/threads)/([^/]+)(/runs)/([^/]+)$z\1/{thread_id}\3/{run_id})z*^(/(?:openai/)?v1/threads)/([^/]+)(/runs)$\1/{thread_id}\3)z6^(/(?:openai/)?v1/threads)/([^/]+)(/messages)/([^/]+)$z\1/{thread_id}\3/{message_id})z.^(/(?:openai/)?v1/threads)/([^/]+)(/messages)$rX)z#^(/(?:openai/)?v1/threads)/([^/]+)$z\1/{thread_id})z9^(/(?:openai/)?v1/vector_stores)/([^/]+)(/files)/([^/]+)$z \1/{vector_store_id}\3/{file_id})z1^(/(?:openai/)?v1/vector_stores)/([^/]+)(/files)$\1/{vector_store_id}\3)z@^(/(?:openai/)?v1/vector_stores)/([^/]+)(/file_batches)/([^/]+)$z!\1/{vector_store_id}\3/{batch_id})z8^(/(?:openai/)?v1/vector_stores)/([^/]+)(/file_batches)$rY)z)^(/(?:openai/)?v1/vector_stores)/([^/]+)$z\1/{vector_store_id})z&^(/(?:openai/)?v1/assistants)/([^/]+)$z\1/{assistant_id})z+^(/(?:openai/)?v1/files)/([^/]+)(/content)$z\1/{file_id}\3)z!^(/(?:openai/)?v1/files)/([^/]+)$z \1/{file_id})z,^(/(?:openai/)?v1/batches)/([^/]+)(/cancel)$z\1/{batch_id}\3)z#^(/(?:openai/)?v1/batches)/([^/]+)$z \1/{batch_id})z5^(/(?:openai/)?v1/fine_tuning/jobs)/([^/]+)(/events)$\1/{fine_tuning_job_id}\3)z5^(/(?:openai/)?v1/fine_tuning/jobs)/([^/]+)(/cancel)$rZ)z:^(/(?:openai/)?v1/fine_tuning/jobs)/([^/]+)(/checkpoints)$rZ)z,^(/(?:openai/)?v1/fine_tuning/jobs)/([^/]+)$z\1/{fine_tuning_job_id})z"^(/(?:openai/)?v1/models)/([^/]+)$z \1/{model})r$sub)r:patternsrH replacement normalizedrrrnormalize_request_route;s 1r_c sddlm}m}|dd}|dur|dur#tdtjjdS|j d}|rSt |}t |d}t d |||krQt d |d |d tjjd dddS|IdH}t|}t |d} t d| | |krt d | d |d tjjd dddS)a Enterprise Only: - Checks if the request size is within the limit Args: request (Request): The incoming request. Returns: bool: True if the request size is within the limit Raises: ProxyException: If the request size is too large rrGmax_request_size_mbNTzPusing max_request_size_mb - not checking - this is an enterprise only feature. content-length bytes_valuez"content_length request size in MB=z+Request size is too large. Request size is  MB. Max size is  MBmessagetypecoder(z request body request size in MB=)r>r5r;r r warningrArBrCrint bytes_to_mbrQProxyExceptionProxyErrorTypesbad_request_errorbodyrP) rr5r;r`content_length header_sizeZheader_size_mbrqZ body_sizeZrequest_size_mbrrrr?sL     r?responsecsddlm}m}|dd}|durH|dur#tdtjjdSt t |d}t d|||krHt d |d |d tjjd d ddS)a  Enterprise Only: - Checks if the response size is within the limit Args: response (Any): The response to check. Returns: bool: True if the response size is within the limit Raises: ProxyException: If the response size is too large rrGmax_response_size_mbNTzQusing max_response_size_mb - not checking - this is an enterprise only feature. rbzresponse size in MB=z-Response size is too large. Response size is rdrerfrarg)r>r5r;r r rkrArBrCrmsys getsizeofrQrnrorp)rtr5r;ruZresponse_size_mbrrrcheck_response_size_is_safes&  rxrccCs|dS)z' Helper to convert bytes to MB irrbrrrrmsrmuser_api_key_dictcCz|jr |jd}|r |S|jr2i}|jD]\}}t|tr-|ddur-|d||<q|r2|S|jr;|jdSdS)z Get the model rpm limit for a given api key. Priority order (returns first found): 1. Key metadata (model_rpm_limit) 2. Key model_max_budget (rpm_limit per model) 3. Team metadata (model_rpm_limit) model_rpm_limitZ rpm_limitNmetadatar Zmodel_max_budgetitemsr+dict team_metadata)ryresultr{rbudgetrrrget_key_model_rpm_limit   rcCrz)z Get the model tpm limit for a given api key. Priority order (returns first found): 1. Key metadata (model_tpm_limit) 2. Key model_max_budget (tpm_limit per model) 3. Team metadata (model_tpm_limit) model_tpm_limitZ tpm_limitNr|)ryrrrrrrrget_key_model_tpm_limitrrmetadata_accessor_key)rZorganization_metadatarate_limit_key)r{rcCst||r t|||SdSN)getattrr )ryrrrrr"get_model_rate_limit_from_metadata@s rcC|jr |jdSdS)Nr{rr ryrrrget_team_model_rpm_limitJ rcCr)Nrrrrrrget_team_model_tpm_limitRrrcCs"dg}|D] }||vrdSqdS)Nz vertex-aiTFr)r:Z%PROVIDER_SPECIFIC_PASS_THROUGH_ROUTESprefixrrris_pass_through_provider_routeZsrcCs@tdd}tdd}tdd}|dup|dup|du}|S)z Check if the user has set up single sign-on (SSO) by verifying the presence of Microsoft client ID, Google client ID or generic client ID and UI username environment variables. Returns a boolean indicating whether SSO has been set up. ZMICROSOFT_CLIENT_IDNZGOOGLE_CLIENT_IDZGENERIC_CLIENT_ID)osgetenv)Zmicrosoft_client_idZgoogle_client_idZgeneric_client_idZ sso_setuprrr_has_user_setup_ssogs   rcCsz|sdSt|tr |n|g}|D]*}t|tsq|d}|d}|dus(|s)qt|ttjkr:|SqdS)zDReturn the header_name mapped to CUSTOMER role, if any (dict-based).NZlitellm_user_role header_name)r+listrr r,lowerZLitellmUserRolesZCUSTOMER)user_id_mappingr~r.Zrolerrrr%get_customer_user_header_from_mappingys    rrequest_headerscCsd|durdStD]'}|D] \}}||kr.|dur"t|nd}|r.|SqqdS)av Check standard customer ID headers for a customer/end-user ID. This enables tools like Claude Code to pass customer IDs via ANTHROPIC_CUSTOM_HEADERS. No configuration required - these headers are always checked. Args: request_headers: The request headers dict Returns: The customer ID if found in standard headers, None otherwise Nr)r r~rr,strip)rZstandard_headerr header_value user_id_strrrr&_get_customer_id_from_standard_headerss rcCshddlm}t|d}|dur|S|durcd}|dd}|r#t|}|s9d}||}t|tr9|dkr9|}t|trc|D] \}} | | krb| } | durXt| nd} | rb| SqBd|vru|dduru|d} t| S|d} t| t r| d}|durt|S|d }t|t r|d }|durt|S|d dur|d } t| SdS) Nr)r5)rZuser_header_mappingsZuser_header_nameruserlitellm_metadatar}Zuser_idZsafety_identifier) r>r5rr rr+r,rr~rr)rrr5Z customer_idZcustom_header_name_to_checkrZuser_id_header_config_keyrCrrZuser_id_from_headerrZuser_from_body_user_fieldrZuser_from_litellm_metadata metadata_dictZuser_id_from_metadata_fieldrrr!get_end_user_id_from_request_bodysX           rcCs|dp |d}|dur'|d}t|dkr |d}ndd|D}|dur8td|}|r8|d}|durP|d sPt d |}|rP|d}|durh|d sht d |}|rh|d}|dur|d rt d |}|r|d}|S) NrZtarget_model_names,r2rcSsg|]}|qSr)r).0mrrr sz*get_model_from_request..z/openai/deployments/([^/]+)z/vertexz /(?:v1beta|beta)/models/([^:]+):z^/models/([^:]+):z/models/([^:]+)) r r3rPrr$r%grouprrOsearch)r9r:rZ model_namesr%Z google_matchZ vertex_matchrrrget_model_from_requests.          rrcCsd|ddS)Nzsk-...r)rrrrabbreviate_api_key sr)Fr):rr$rv functoolsrtypingrrrrZfastapirrr Zlitellmr r Zlitellm._loggingr Zlitellm.constantsr Zlitellm.proxy._typesZlitellm.types.routerrboolr,rrrr!r'r/r4r8rDrKrRr_r?rxrlrmZUserAPIKeyAuthr-rrLiteralrrrrrrrrUnionrrrrrrs          ! ' >1R@( ! !       M /