o ưi8@sddlmZddlmZmZmZmZmZmZmZm Z m Z ddl m Z ddl mZddlmZddlmZmZmZmZddlmZddlmZdd lmZmZmZmZmZm Z zdd l!m"Z"Wn e#yid Z"Ynwerrdd l$m%Z&eZ'Gd dde(Z)GdddeZ*ddZ+d S))datetime) TYPE_CHECKINGAnyDictListLiteralOptionalTypeUnionget_args)verbose_logger) DualCache) CustomLogger)DynamicGuardrailParamsGuardrailEventHooks LitellmParamsMode)AllMessageValues)GuardrailConfigModel) CallTypesGenericGuardrailAPIInputsGuardrailStatusGuardrailTracingDetailLLMResponseTypes#StandardLoggingGuardrailInformation) HTTPExceptionN)LoggingcsTeZdZdZ  d dededeeefdeedeeeeff fdd ZZ S) ModifyResponseExceptiona Exception raised when a guardrail wants to modify the response. This exception carries the synthetic response that should be returned to the user instead of calling the LLM or instead of the LLM's response. It should be caught by the proxy and returned with a 200 status code. This is a base exception that all guardrails can use to replace responses, allowing violation messages to be returned as successful responses rather than errors. Nmessagemodel request_dataguardrail_namedetection_infocs2||_||_||_||_|pi|_t|dS)a Initialize the modify response exception. Args: message: The violation message to return to the user model: The model that was being called request_data: The original request data guardrail_name: Name of the guardrail that raised this exception detection_info: Additional detection metadata (scores, rules, etc.) N)rrr r!r"super__init__)selfrrr r!r" __class__\/home/app/Keep/.python/lib/python3.10/site-packages/litellm/integrations/custom_guardrail.pyr$9s  z ModifyResponseException.__init__)NN) __name__ __module__ __qualname____doc__strrrrr$ __classcell__r(r(r&r)r,s rcseZdZ          d_deedeeedeeeeeefde de de d eed ee d eed eeffd d Z d`dedee ee fdefddZ d`dede ee fdee ee fddfddZedeedfddZdeeeeeefdeeddfddZdedee fdd Zd!e de fd"d#Zdedeeeee eefffd$d%Zd&eeeee eeffde fd'd(Zd)e ee fd*eedeefd+d,Zded-ed*eedeefd.d/Zd0ede fd1d2Zd0ede fd3d4Zdedefd5d6Z de fd7d8Z!       dad9ee"eeeefded:e#d;ee$dee ee fd?eed0eed@ee%ddfdAdBZ& d`dCe'dedDe(dEdFedGde'f dHdIZ)     dbd-ee ded;ee$dcSs<|D]}t|tr t|}||vrtd|d|qdS)N Event hook % is not in the supported event hooks ) isinstancer.r ValueError)r2r1hookr(r(r)5_validate_event_hook_list_is_in_supported_event_hookss zcCustomGuardrail._validate_event_hook.._validate_event_hook_list_is_in_supported_event_hooksrJrK) r rrr.rLlistrtagsvaluesextendappendr<rM)r%r2r1rOZtag_values_flatv default_listr(r(r)r:sP          z$CustomGuardrail._validate_event_hookdatacCs:d|vr|dS|dp|di}d|vr|dSdS)zI Returns True if the global guardrail should be disabled disable_global_guardraillitellm_metadatametadataFrGr%rWrZr(r(r)get_disable_global_guardrails z,CustomGuardrail.get_disable_global_guardrailresultc CsV|durdSz tt}t||WSty*}zdt|vr%WYd}~dSd}~ww)a Check if result is a valid LLMResponseTypes instance. Safely handles TypedDict types which don't support isinstance checks. For non-LiteLLM responses (like passthrough httpx.Response), returns True to allow them through. NF TypedDictT)r rrL TypeErrorr.)r%r^Zresponse_typesrCr(r(r)_is_valid_response_type s  z'CustomGuardrail._is_valid_response_typecCs4d|vr|dS|dp|di}|dpgS)zN Returns the guardrail(s) to be run from the metadata or root guardrailsrYrZr[r\r(r(r)get_guardrail_from_metadata#sz+CustomGuardrail.get_guardrail_from_metadatarequested_guardrailscCsD|D]}t|tr|j|vrdSqt|tr|j|krdSqdS)NTF)rLdictr!r.)r%rdZ _guardrailr(r(r)%_guardrail_is_in_requested_guardrails/s    z5CustomGuardrail._guardrail_is_in_requested_guardrailsr; call_typec sddlm}|d}|dust|ts|S|j|tjddur#|S|tj ks-|tj krh|j ||d|d|d|d |d d t ||j pLd d IdH}|durht|trh|d}|durh||d<|S)NrUserAPIKeyAuthrbrW event_typeTuser_api_key_user_iduser_api_key_team_iduser_api_key_end_user_iduser_api_key_hashuser_api_key_request_routeZuser_idZteam_idZ end_user_idZapi_keyZ request_route acompletion)user_api_key_dictcacherWrgmessages)litellm.proxy._typesrirGrLrPshould_run_guardrailrpre_callr completionrrasync_pre_call_hookdcvaluere)r%r;rgrilitellm_guardrailsr^Zresult_messagesr(r(r)async_pre_call_deployment_hook=s:    z.CustomGuardrail.async_pre_call_deployment_hookresponsec sddlm}|d}|dust|ts|S|j|tjddur#|S|j||d|d|d |d |d d ||d IdH}| |sK|S|S)zh Allow modifying / reviewing the response just after it's received from the deployment. rrhrbNrjTrlrmrnrorprq)rsrWr) rvrirGrLrPrwr post_callasync_post_call_success_hookra)r%r rrgrir}r^r(r(r)'async_post_call_success_deployment_hookes2  z7CustomGuardrail.async_post_call_success_deployment_hookrkcCs"||}||}td|j||j||j|jdurQ|durQ||rOt|jt rMzddl m }Wn t y>t dw| ||j|}|durM|SdSdS|jr`||s`|jdkr`dS||sgdSt|jt rzddl m }Wn t yt dw| ||j|}|dur|SdS) zO Returns True if the guardrail should be run on the event_type zinside should_run_guardrail for guardrail=%s event_type= %s guardrail_supported_event_hooks= %s requested_guardrails= %s self.default_on= %sTr)EnterpriseCustomGuardrailHelperzuSetting tag-based guardrails is only available in litellm-enterprise. You must be a premium user to use this feature.NFZ logging_only)rcr]r debugr!r2r3_event_hook_is_event_typerLrZ0litellm_enterprise.integrations.custom_guardrailr ImportErrorZ_should_run_if_mode_by_tagrfr|)r%rWrkrdrXrr^r(r(r)rwsb         z$CustomGuardrail.should_run_guardrailcCs|jdurdSt|jtr|j|jvSt|jtrS|jjD]}t|tr/|j|vr.dSq|j|kr7dSq|jjrQt|jjtrG|jjn|jjg}|j|vSdS|j|jkS)a Returns True if the event_hook is the same as the event_type eg. if `self.event_hook == "pre_call" and event_type == "pre_call"` -> then True eg. if `self.event_hook == "pre_call" and event_type == "post_call"` -> then False NTF)r2rLrPr|rrQrRr<)r%rkZ tag_valuerVr(r(r)rs*           z)CustomGuardrail._event_hook_is_event_typecCs||}|D]=}t|trD|j|vrDtdi||j}|di}|dur@t|tr<|rcsLd}||}|durg||<dSt|tr|dS|g||<dS)NZ&standard_logging_guardrail_information)rGrLrPrT)rkeyexistingZslgr(r)_append_guardrail_info]s  zjCustomGuardrail.add_standard_logging_guardrail_information_to_request_data.._append_guardrail_inforZrYr() rLrAr.litellm.types.utilsrr2rreZ model_dumpZ'litellm.litellm_core_utils.core_helpersrrr!)r%rr rrrrrrrkrrrrZclean_guardrail_responserr(rr):add_standard_logging_guardrail_information_to_request_data"sD       zJCustomGuardrail.add_standard_logging_guardrail_information_to_request_datainputs input_type)requestr logging_objLiteLLMLoggingObjcs|S)a Apply your guardrail logic to the given inputs Args: inputs: Dictionary containing: - texts: List of texts to apply the guardrail to - images: Optional list of images to apply the guardrail to - tool_calls: Optional list of tool calls to apply the guardrail to request_data: The request data dictionary - containing user api key metadata (e.g. user_id, team_id, etc.) input_type: The type of input to apply the guardrail to - "request" or "response" logging_obj: Optional logging object for tracking the guardrail execution Any of the custom guardrails can override this method to provide custom guardrail logic Returns the texts with the guardrail applied and the images with the guardrail applied (if any) Raises: Exception: - If the guardrail raises an exception r()r%rr rrr(r(r)apply_guardrailtszCustomGuardrail.apply_guardrailoriginal_inputsc Csd|durin|}|durt|tr|||rd}nd}td||j||d||||d|S) Add StandardLoggingGuardrailInformation to the request data This gets logged on downsteam Langfuse, DataDog, etc. NmaskZallowzGuardrail response: successrr rrrrrk)rLre_inputs_were_modifiedr rr) r%rr rrrrkrrr(r(r)_process_responses"  z!CustomGuardrail._process_responserCcCs2t|trdStdurt|tr|jdkrdSdS)ak Returns True if the exception represents an intentional guardrail block (this was logged previously as an API failure - guardrail_failed_to_respond). Guardrails signal intentional blocks by raising: - HTTPException with status 400 (content policy violation) - ModifyResponseException (passthrough mode violation) TNiF)rLrr status_code)rCr(r(r)_is_guardrail_interventions  z*CustomGuardrail._is_guardrail_interventionc CsB||rdnd}|}d|jjvrd}|j|||||||d|)rZguardrail_intervenedZguardrail_failed_to_respondZCustomCodeGuardrailZdenyr)rr'r*r) r%rCr rrrrkrrr(r(r)_process_errors"  zCustomGuardrail._process_errorcCsHt|t|B}|D]}||}||}||kr!dSqdS)z Compare original inputs with response to determine if content was modified. Returns True if the inputs were modified (mask scenario), False otherwise (allow scenario). TF)setrrG)r%rrall_keysroriginal_valueZresponse_valuer(r(r)rs  z%CustomGuardrail._inputs_were_modifiedcontent_string mask_string start_index end_indexcCsFd|kr|krt|ks|S|S|d||||dS)zS Mask the content in the string between the start and end indices. rN)len)r%rrrrr(r(r)mask_content_in_strings z&CustomGuardrail.mask_content_in_stringlitellm_paramscCs&t|D] \}}t|||qdS)z@ Update the guardrails litellm params in memory N)varsitemssetattr)r%rrr|r(r(r)update_in_memory_litellm_paramssz/CustomGuardrail.update_in_memory_litellm_paramscCs|dus|dur dS|tjjks|tjjks|tjjkr!|dS|tjjks-|tjjkrRddlm }ddl m }|d}|durDdS|j ||d}|t t|SdS)zG Returns the messages for the given call type and data Nrur)cast) LiteLLMCompletionResponsesConfiginput)rZresponses_api_request)rryr|rrZanthropic_messagesrG responsesZ aresponsestypingrZBlitellm.responses.litellm_completion_transformation.transformationrZ)transform_responses_api_input_to_messagesrr)r%rgrWrrZ input_datarur(r(r)%get_guardrails_messages_for_call_type!s&         z5CustomGuardrail.get_guardrails_messages_for_call_type) NNNFFFNNNNN)NNNNNNN)NNNNN)NNNN)4r*r+r,rr.rrr rboolintr$rrrDrH staticmethodr rIr:rer]rarrcrfrr~rrrwrrrrArfloatrrrrrrrrrrrrrrr/r(r(r&r)r0Ss      1   , 4      ( * ?*    W " * $  r0csvddl}ddldtdttfdd|fdd|fd d |fd d }|S) a Decorator to add standard logging guardrail information to any function Add this decorator to ensure your guardrail response is logged to DataDog, OTEL, s3, GCS etc. Logs for: - pre_call - during_call - post_call rN func_namer>cSs.|dkrtjS|dkrtjS|dvrtjSdS)z2Infer the actual event type from the function namerzZasync_moderation_hook)rZasync_post_call_streaming_hookN)rrxZ during_callr)rr(r(r)$_infer_event_type_from_function_name^szGlog_guardrail_information.._infer_event_type_from_function_namec st}|d}|dp|dpi}j}d}jdkr*d|vr*|d}z#|i|IdH}|j|||tt|||dWStyv}z|j|||tt||dWYd}~Sd}~ww)NrrWr rr)rr rrrrkr)rCr rrrrk) rnowrGr*r timestamp total_secondsrAr argsr;rr%r rkrrrCrfuncr(r) async_wrapperms<     z0log_guardrail_information..async_wrapperc st}|d}|dp|dpi}j}d}jdkr)d|vr)|d}z|i|}|j||t|||dWStyb}z|j||t||dWYd}~Sd}~ww)NrrWr rr)rr rrkr)rCr rrk)rrrGr*rrrArrrr(r) sync_wrappers2  z/log_guardrail_information..sync_wrappercs&r |i|S|i|Sr)iscoroutinefunction)rr;)rrinspectrr(r)wrappers z*log_guardrail_information..wrapper) functoolsrr.rrwraps)rrrr()rrrrrr)log_guardrail_informationPs   r),rrrrrrrrr r r Zlitellm._loggingr Zlitellm.cachingr Z"litellm.integrations.custom_loggerrZlitellm.types.guardrailsrrrrZlitellm.types.llms.openairZ3litellm.types.proxy.guardrails.guardrail_hooks.baserrrrrrrrZfastapi.exceptionsrrZ*litellm.litellm_core_utils.litellm_loggingrrr{rArr0rr(r(r(r)s4 ,        '