o ưiL@sdZddlZddlZddlmZmZmZmZmZddl m Z ddl m Z ddl mZddlmZddlmZmZdd lmZmZGd d d eZeZddlZejedS) a Skills Injection Hook for LiteLLM Proxy Main hook that orchestrates skill processing: - Fetches skills from LiteLLM DB - Injects SKILL.md content into system prompt - Adds litellm_code_execution tool for automatic code execution - Handles agentic loop internally when litellm_code_execution is called For non-Anthropic models (e.g., Bedrock, OpenAI, etc.): - Skills are converted to OpenAI-style tools - Skill file content (SKILL.md) is extracted and injected into the system prompt - litellm_code_execution tool is added - when model calls it, LiteLLM handles execution automatically and returns final response with file_ids Usage: # Simple - LiteLLM handles everything automatically via proxy # The container parameter triggers the SkillsInjectionHook response = await litellm.acompletion( model="gpt-4o-mini", messages=[{"role": "user", "content": "Create a bouncing ball GIF"}], container={"skills": [{"skill_id": "litellm:skill_abc123"}]}, ) # Response includes file_ids for generated files N)AnyDictListOptionalUnion)verbose_proxy_logger) DualCache) CustomLogger)SkillPromptInjectionHandler)LiteLLM_SkillsTableUserAPIKeyAuth) CallTypesCallTypesLiteralcseZdZdZfddZdedededede e e e eff d d Z d1ded eed edefddZded eedefddZde de efddZde defddZdedede ede efddZdedeee effddZdededee efdefdd Zd!e dee efd"ed#eee efde f d$d%Zd&e d'ee efdee efd"ed#eee efde f d(d)Zdededee efdefd*d+Zd,edee efd"ed#eee efde f d-d.Zded#eee efdefd/d0Z Z!S)2SkillsInjectionHooka Pre/Post-call hook that processes skills from container.skills parameter. Pre-call (async_pre_call_hook): - Skills with 'litellm:' prefix are fetched from LiteLLM DB - For Anthropic models: native skills pass through, LiteLLM skills converted to tools - For non-Anthropic models: LiteLLM skills are converted to tools + execute_code tool Post-call (async_post_call_success_deployment_hook): - If response has litellm_code_execution tool call, automatically execute code - Continue conversation loop until model gives final response - Return response with generated files inline This hook is called automatically by litellm during completion calls. c sPddlm}m}||_t|_|d||_|d||_t j di|dS)Nr)DEFAULT_MAX_ITERATIONSDEFAULT_SANDBOX_TIMEOUTmax_iterationssandbox_timeout) Z+litellm.llms.litellm_proxy.skills.constantsrrZoptional_paramsr prompt_handlergetrrsuper__init__)selfkwargsrr __class__r^/home/app/Keep/.python/lib/python3.10/site-packages/litellm/proxy/hooks/litellm_skills/main.pyr:s zSkillsInjectionHook.__init__user_api_key_dictcachedata call_typereturnc s|dvr|S|d}|rt|ts|S|d}|r!t|ts#|Stdt|dg}g}|D]2} t| tsrKrDrr.r/) rr r)r8rErFrGr5rHrIrJrKrrr_process_non_anthropic_modelsF             z0SkillsInjectionHook._process_non_anthropic_modelr'c s^zddlm}||IdHWSty.}ztd|d|WYd}~dSd}~ww)z Fetch a skill from the LiteLLM database. Args: skill_id: The skill ID (without 'litellm:' prefix) Returns: LiteLLM_SkillsTable or None if not found r)LiteLLMSkillsHandlerNz*SkillsInjectionHook: Error fetching skill z: )Z)litellm.llms.litellm_proxy.skills.handlerrMZfetch_skill_from_db Exceptionrr3)rr'rMerrrr1s z(SkillsInjectionHook._fetch_skill_from_dbmodelcCsVzddlm}||d\}}}}|dkWSty*d|vp'|dYSw)z Check if the model is an Anthropic model using get_llm_provider. Args: model: The model name/identifier Returns: True if Anthropic model, False otherwise r)get_llm_provider)rP anthropicZclaudez anthropic/)Z1litellm.litellm_core_utils.get_llm_provider_logicrQrNlowerr0)rrPrQ_Zcustom_llm_providerrrr_is_anthropic_model&s   z'SkillsInjectionHook._is_anthropic_model request_dataresponsecsddlm}|dp i}|dpi}|dp|d}|s#dS|dp-|di}i} |D]} | | q4| sEtddS||} | sNdSd } | D]} | d d }||jj kse| d rid } nqR| sndSt d|j ||| dIdHS)a Post-call hook to handle automatic code execution. Handles both OpenAI format (response.choices) and Anthropic/messages API format (response["content"]). If the response contains a tool call (litellm_code_execution or skill tool): 1. Execute the code in sandbox 2. Add result to messages 3. Make another LLM call 4. Repeat until model gives final response 5. Return modified response with generated files rLiteLLMInternalToolsr:metadatar<Nr;z>SkillsInjectionHook: No skill files found, cannot execute codeFnamer(skill_Tz@SkillsInjectionHook: Detected tool call, starting execution loop)r rWrI) r>rYrvaluesupdaterr3_extract_tool_callsCODE_EXECUTIONvaluer0r._execute_code_loop_messages_api)rrVrWr!rYr:rZZcode_exec_enabledZskill_files_by_idrFZ files_dict tool_callsZhas_executable_tooltc tool_namerrr'async_post_call_success_deployment_hook;sP         z;SkillsInjectionHook.async_post_call_success_deployment_hookc Cs0g}d}t|tr|dg}nt|dr|j}|ra|D]D}t|tr>|ddkr>||d|d|didqt|dr`t|dddkr`|t|ddt|ddt|didq|st|d r|jr|jd j}t|d r|j r|j D]}||j |j j |j j rt|j j nidq||S) z8Extract tool calls from response, handling both formats.NrHtypetool_useidr[input)rir[rjchoicesrrc)r+r,rhasattrrHr2getattrrkmessagercrifunctionr[ argumentsjsonloads)rrWrcrHblockmsgrdrrrr_s>          z'SkillsInjectionHook._extract_tool_callsrIc sddl}ddlm}ddlm}|durtddS|dd}t|dg}|d g} |d d } ||j d } g} |} t |j D]3}| | }t | trW| d nt| d d}t | trh| dgnt| dg}g}|psgD]2}t |tr||qtt|dr||qtt|dr|t|jqt|dt|dqtd|d}|||dks|std|ddt| d|| | Sg}|D]N}|dd}|dd}|di}||jjkr|dd}|||| | IdH}n|dr||||| | IdH}nd|d }|d!||d"q|d#|dtd$|d%z"|jj||| | d&IdH} | durUtd'||| WSWqEtyy}ztd(|||| WYd}~Sd}~wwt d)|j d*|| | S)+z Execute the code execution loop for messages API (Anthropic format). Returns the final response with generated files inline. rNrXSkillsSandboxExecutorz?SkillsInjectionHook: Response is None, cannot execute code looprPr(messagesr8 max_tokenstimeout stop_reasonrH model_dump__dict__text)rgr assistantrolerHrhz*SkillsInjectionHook: Loop completed after  iterations,  files generatedr[rirjcoder\Tool 'z ' not handled tool_result)rgZ tool_use_idrHuser/SkillsInjectionHook: Making LLM call iteration rPrwr8rxz+SkillsInjectionHook: LLM call returned Nonez&SkillsInjectionHook: LLM call failed: %SkillsInjectionHook: Max iterations ( ) reached)!litellmr>rY2litellm.llms.litellm_proxy.skills.sandbox_executorrvrerrorrr-rrangerr_r+r,rmr2rlr}r~strr.r/_attach_files_to_responser`ra _execute_coder0_execute_skill_toolrRacreaterNr3)rr rWrIrrYrvrPrwr8rxexecutorgenerated_filescurrent_response iterationrcr|Z raw_contentZcontent_blocksrsZ assistant_msgZ tool_resultsrdreZtool_id tool_inputrresultrOrrrrbs        "                     z3SkillsInjectionHook._execute_code_loop_messages_apirrrc szYtdt|d|j||d}|ddpd}|drH|dD]"}||d|d|d tt|d d |d |d7}q%|d rV|d |d 7}|pYdWStys}z dt |WYd}~Sd}~ww)z1Execute code in sandbox and return result string.%SkillsInjectionHook: Executing code ( chars)rrIoutputr(filesr[ mime_typecontent_base64r[rrsizez Generated file: rz Error: zCode executed successfullyCode execution failed: N) rr.r/executerr2base64 b64decoderNr) rrrIrr exec_resultrfrOrrrr&s*    z!SkillsInjectionHook._execute_codererc sdd|D}d}|D]}d|vs"d|vs"d|vr&|}nq|s/|r/|d}|rE|dd d d } d | d | d} nd} || |||IdHS)zKExecute a skill tool by generating and running code based on skill content.cSs$g|]}|dr|ds|qS)r9z __init__.py)rB).0prrr Rs$z;SkillsInjectionHook._execute_skill_tool..NbuilderZcreator generatorr/.r9r(zX # Auto-generated code to execute skill import sys sys.path.insert(0, '/sandbox') from z] import * # Try to find and use a Builder/Creator class import inspect module = __import__('aE', fromlist=['']) for name, obj in inspect.getmembers(module): if inspect.isclass(obj) and name != 'object': try: instance = obj() # Try common methods if hasattr(instance, 'create'): result = instance.create() elif hasattr(instance, 'build'): result = instance.build() elif hasattr(instance, 'generate'): result = instance.generate() elif hasattr(instance, 'save'): instance.save('output.gif') print(f'Used {name} class') break except Exception as e: print(f'Error with {name}: {e}') continue # List generated files import os for f in os.listdir('.'): if f.endswith(('.gif', '.png', '.jpg')): print(f'Generated: {f}') z+ print('No executable skill module found') )rArSreplacer) rrerrIrrZpython_modulesZ main_modulemodZ import_pathrrrrrGs& $ &z'SkillsInjectionHook._execute_skill_toolc sddl}ddlm}ddlm}|dd}t|dg}|dg} thd fd d |D} ||j d } g} |} t |j D]}| j dj }| j dj}d |jd}|jrfdd|jD|d<|||dksr|jstd|ddt| d|| | S|jD])}|jj}||jjkr|j||| | dIdH}nd|d}|d|j|dqtd|d|jj||| | dddIdH} qEtd |j d!|| | S)"z Execute the code execution loop until model gives final response. Returns the final response with generated files inline. rNrXrurPr(rwr8>rwr%rPr:rZr8csi|] \}}|vr||qSrr)rkvZ_EXCLUDED_ACOMPLETION_KEYSrr s z:SkillsInjectionHook._execute_code_loop..rzrrcSs(g|]}|jd|jj|jjddqS)ro)r[rp)rirgro)riror[rp)rrdrrrrs z:SkillsInjectionHook._execute_code_loop..rcz9SkillsInjectionHook: Code execution loop completed after rrr) tool_callrIrrrz' not handled automaticallyZtool)rZ tool_call_idrHrrrxryrrr)rr>rYrrvrr- frozensetitemsrrrrkrnZ finish_reasonrHrcr2rr.r/rror[r`ra_execute_code_toolrirRrr3)rr rWrIrrYrvrPrwr8rrrrrZassistant_messager|Zassistant_msg_dictrrerrrr_execute_code_loops|                  z&SkillsInjectionHook._execute_code_looprc sFzt|jj}|dd}tdt|d|j||d}|ddp(d}|drq|d7}|dD]:} t | d } | | d | d | d t| d |d | d dt| d7}td| d dt| dq6|dr|d|d7}|WSt y} zt d| dt| WYd} ~ Sd} ~ ww)zDExecute a litellm_code_execution tool call and return result string.rr(rrrrrz Generated files:rr[rrz - z (z bytes)z$SkillsInjectionHook: Generated file rz Error: z,SkillsInjectionHook: Code execution failed: rN)rqrrrorprrr.r/rrrr2rNrr) rrrIrrargsrrrrZ file_contentrOrrrrsL     z&SkillsInjectionHook._execute_code_toolcCs|s|St|tr||d<tdt|d|Sz||_Wn ty(Ynwt|dr;|jdur6i|_||jd<tdt|d|S)z Attach generated files to the response object. Files are added to response._litellm_generated_files for easy access. For dict responses, files are added as a key. _litellm_generated_fileszSkillsInjectionHook: Attached z files to dict response model_extraNz files to response) r+r,rr.r/rAttributeErrorrlr)rrWrrrrr7s*       z-SkillsInjectionHook._attach_files_to_response)T)"__name__ __module__ __qualname____doc__rr rr,rrrrNrr6rr boolr4rLr1rUrr rfrr_bytesrbrrrrr __classcell__rrrrr)s   F G C M(  v  !   I  r  5r)rrrqtypingrrrrrZlitellm._loggingrZlitellm.caching.cachingrZ"litellm.integrations.custom_loggerr Z2litellm.llms.litellm_proxy.skills.prompt_injectionr Zlitellm.proxy._typesr r Zlitellm.types.utilsr rrZskills_injection_hookrZlogging_callback_managerZadd_litellm_callbackrrrrs(    >