o ưi@sNddlZddlmZmZmZmZmZmZddlZddl m Z GdddZ dS)N)AnyDictIteratorListOptionalUnion)UnauthorizedErrorc@s2eZdZddedeefddZdeeeffddZ        dd ed eeeefd ee d ee dee dee dee dee deede de eee fejffddZ       dd ed eeeefd ee d ee dee dee dee dee deedeeee ffddZdS) ChatClientNbase_urlapi_keycCs|d|_||_dS)a Initialize the ChatClient. Args: base_url (str): The base URL of the LiteLLM proxy server (e.g., "http://localhost:8000") api_key (Optional[str]): API key for authentication. If provided, it will be sent as a Bearer token. /N)rstrip _base_url_api_key)selfr r rP/home/app/Keep/.python/lib/python3.10/site-packages/litellm/proxy/client/chat.py__init__ s  zChatClient.__init__returncCs"ddi}|jrd|j|d<|S)z Get the headers for API requests, including authorization if api_key is set. Returns: Dict[str, str]: Headers to use for API requests z Content-Typezapplication/jsonzBearer Authorization)r)rheadersrrr _get_headersszChatClient._get_headersFmodelmessages temperaturetop_pn max_tokenspresence_penaltyfrequency_penaltyuserreturn_requestc  Cs|jd} ||d} |dur|| d<|dur|| d<|dur#|| d<|dur+|| d<|dur3|| d<|dur;|| d <| durC| | d <tjd | || d } | rR| St}z|| }||WStj j y}z |j j d krzt |d}~ww)a Create a chat completion. Args: model (str): The model to use for completion messages (List[Dict[str, str]]): The messages to generate a completion for temperature (Optional[float]): Sampling temperature between 0 and 2 top_p (Optional[float]): Nucleus sampling parameter between 0 and 1 n (Optional[int]): Number of completions to generate max_tokens (Optional[int]): Maximum number of tokens to generate presence_penalty (Optional[float]): Presence penalty between -2.0 and 2.0 frequency_penalty (Optional[float]): Frequency penalty between -2.0 and 2.0 user (Optional[str]): Unique identifier for the end user return_request (bool): If True, returns the prepared request object instead of executing it Returns: Union[Dict[str, Any], requests.Request]: Either the completion response from the server or a prepared request object if return_request is True Raises: UnauthorizedError: If the request fails with a 401 status code requests.exceptions.RequestException: If the request fails with any other error /chat/completions)rrNrrrrrr r!POST)rjson)rrequestsRequestrSessionsendprepareraise_for_statusr% exceptions HTTPErrorresponse status_coder )rrrrrrrrr r!r"urldatarequestsessionr/errr completions!s< $   zChatClient.completionsc  csZ|jd} ||dd} |dur|| d<|dur|| d<|dur%|| d<|dur-|| d<|dur5|| d <|dur=|| d <| durE| | d <t} zJ| j| || dd } | | D]3}|r|d }|dr|dd}| dkr{WdSz t |}|VWq]t j yYq]wq]WdStj jy}z |jjdkrt|d}~ww)a Create a streaming chat completion. Args: model (str): The model to use for completion messages (List[Dict[str, str]]): The messages to generate a completion for temperature (Optional[float]): Sampling temperature between 0 and 2 top_p (Optional[float]): Nucleus sampling parameter between 0 and 1 n (Optional[int]): Number of completions to generate max_tokens (Optional[int]): Maximum number of tokens to generate presence_penalty (Optional[float]): Presence penalty between -2.0 and 2.0 frequency_penalty (Optional[float]): Frequency penalty between -2.0 and 2.0 user (Optional[str]): Unique identifier for the end user Yields: Dict[str, Any]: Streaming response chunks from the server Raises: UnauthorizedError: If the request fails with a 401 status code requests.exceptions.RequestException: If the request fails with any other error r#T)rrstreamNrrrrrr r!)rr%r7zutf-8zdata: z[DONE]r&)rr'r)postrr, iter_linesdecode startswithstripr%loadsJSONDecodeErrorr-r.r/r0r )rrrrrrrrr r!r1r2r4r/lineZdata_strchunkr5rrrcompletions_streamjsd !        zChatClient.completions_stream)N)NNNNNNNF)NNNNNNN)__name__ __module__ __qualname__strrrrrrfloatintboolrrr'r(r6rrBrrrrr s|     M   r ) r%typingrrrrrrr'r-r r rrrrs