o ưia1 @sdZddlZddlmZmZddlmZmZmZmZddl m Z ddl m Z ddl mZddlmZmZGd d d Z dd ed ed eeedefddZdS)z> Response Polling Handler for Background Responses with Cache N)datetimetimezone)AnyDictListOptional)verbose_proxy_logger)uuid4) RedisCache)ResponsesAPIResponseResponsesAPIStatusc+@seZdZdZdZdZd1deedefdd Z e d e fd d Z e d e d e fddZe de d e fddZde dee efd efddZ                   d2de deedeedeedeedeedeedeedeedee dee d eed!eed"eed#ee d$eed%ee d&ee d'ee d(ee d df*d)d*Zde d eee effd+d,Zde d e fd-d.Zde d e fd/d0ZdS)3ResponsePollingHandlerz0Handles polling-based responses with Redis cachezlitellm:polling:response:Z litellm_poll_N redis_cachettlcCs||_||_dSN)rr)selfrrre/home/app/Keep/.python/lib/python3.10/site-packages/litellm/proxy/response_polling/polling_handler.py__init__s zResponsePollingHandler.__init__returncCs|jtS)z4Generate a unique UUID for polling with clear prefix)POLLING_ID_PREFIXr )clsrrrgenerate_polling_idsz*ResponsePollingHandler.generate_polling_id response_idcCs ||jS)z&Check if a response_id is a polling ID) startswithr)rrrrr is_polling_ids z$ResponsePollingHandler.is_polling_id polling_idcCs|j|S)z$Get Redis cache key for a polling ID)CACHE_KEY_PREFIX)rrrrr get_cache_key"sz$ResponsePollingHandler.get_cache_key request_datac stttj}t|dd|g|didd}||}|j r>|j j || |j dIdHt d|d|j d |S) a Create initial state in Redis for a polling request Uses OpenAI ResponsesAPIResponse object: https://platform.openai.com/docs/api-reference/responses/object Args: polling_id: Unique identifier for this polling request request_data: Original request data Returns: ResponsesAPIResponse object following OpenAI spec responseZqueuedmetadataN)idobjectstatusZ created_atoutputr"usagekeyvaluerz"Created initial polling state for z with TTL=s)intrnowrutc timestampr getrrasync_set_cacheZmodel_dump_jsonrrdebug)rrr Zcreated_timestampr! cache_keyrrrcreate_initial_state's,   z+ResponsePollingHandler.create_initial_stater%r'errorincomplete_details reasoning tool_choicetoolsr&model instructions temperaturetop_pmax_output_tokensprevious_response_idtext truncationparallel_tool_callsuserstorecs|jsdS||}|j|IdH}|s td|dSt|}|r+||d<| dur3| |d<|r9||d<|rCd|d<||d<|rI||d<|durQ||d <|durY||d <|dura||d <| duri| |d <| durq| |d <| dury| |d<| dur| |d<|dur||d<|dur||d<|dur||d<|dur||d<|dur||d<|dur||d<|dur||d<|jj|t||j dIdHt | dg}t d|d|dd|dS)a# Update the polling state in Redis Uses OpenAI Response object format with native status types: https://platform.openai.com/docs/api-reference/responses/object Args: polling_id: Unique identifier for this polling request status: OpenAI ResponsesAPIStatus value usage: Usage information error: Error dict (automatically sets status to "failed") incomplete_details: Details for incomplete responses reasoning: Reasoning configuration from response.completed tool_choice: Tool choice configuration from response.completed tools: Tools list from response.completed output: Full output list to replace current output model: Model identifier instructions: System instructions temperature: Sampling temperature top_p: Nucleus sampling parameter max_output_tokens: Maximum output tokens previous_response_id: ID of previous response in conversation text: Text configuration truncation: Truncation setting parallel_tool_calls: Whether parallel tool calls are enabled user: User identifier store: Whether to store the response Nz&No cached state found for polling_id: r%r&r'failedr5r6r7r8r9r:r;r<r=r>r?r@rArBrCrDr(zUpdated polling state for z : status=z, output_items=) rrasync_get_cacherwarningjsonloadsr1dumpsrlenr0r2)rrr%r'r5r6r7r8r9r&r:r;r<r=r>r?r@rArBrCrDr3 cached_statestateZ output_countrrr update_stateUsv4   z#ResponsePollingHandler.update_statecs:|jsdS||}|j|IdH}|rt|SdS)z$Get current polling state from RedisN)rrrFrHrI)rrr3rLrrr get_states  z ResponsePollingHandler.get_statecs|j|ddIdHdS)zx Cancel a polling request Following OpenAI Response object format for cancelled status cancelled)rr%NT)rN)rrrrrcancel_pollings  z%ResponsePollingHandler.cancel_pollingcs,|jsdS||}|j|IdHdS)z#Delete a polling request from cacheFNT)rrZasync_delete_cache)rrr3rrrdelete_pollings  z%ResponsePollingHandler.delete_polling)Nr)NNNNNNNNNNNNNNNNNNN)__name__ __module__ __qualname____doc__rrrr r,r classmethodstrrboolrrrrr r4r listfloatrNrOrQrRrrrrr s  1       r background_moder:native_background_moderc CsF|r|r|sdS|r||vrtd|ddS|dkrdSt|trd|vr7|dd}||vr5dSdS|durzG|j|g}|D]:}|j|} | d i} | d } | sj| d d } d| vrj| dd} | r| |vrtd |d| WdSqEWdSty} ztd|d| WYd} ~ dSd} ~ wwdS)a Determine if polling via cache should be used for a request. Args: background_mode: Whether background=true was set in the request polling_via_cache_enabled: Config value - False, "all", or list of providers redis_cache: Redis cache instance (required for polling) model: Model name from the request (e.g., "gpt-5" or "openai/gpt-4o") llm_router: LiteLLM router instance for looking up model deployments native_background_mode: List of model names that should use native provider background mode instead of polling via cache Returns: True if polling should be used, False otherwise FzModel z> is in native_background_mode list, skipping polling via cacheallT/rNlitellm_paramsZcustom_llm_providerr:zPolling enabled for model=z , provider=z%Could not resolve provider for model z: ) rr2 isinstancerZsplitZ model_name_to_deployment_indicesr0Z model_list Exception)r\Zpolling_via_cache_enabledrr:Z llm_routerr]providerindicesidxZdeployment_dictr`Z dep_providerZ dep_modelerrrshould_use_polling_for_requestsT         rir)rVrHrrtypingrrrrZlitellm._loggingrZ litellm._uuidr Zlitellm.caching.redis_cacher Zlitellm.types.llms.openair r r rYrXrirrrrs(   w