o
ưi @ s d dl Z d dlmZ d dlmZmZmZ d dlZd dlm Z dZ
G dd de Z ed Z g dZ
d efd
d Zed d
dedefddZdS ) N) lru_cache)LiteralOptionalUnion)BaseLLMExceptionzhttps://huggingface.coc
sL e Zd Z ddeej deej deeeje f f fdd
Z
Z S )HuggingFaceErrorNrequestresponseheadersc s t j|||||d d S )N) status_codemessager r r
)super__init__)selfr r r r r
__class__ \/home/app/Keep/.python/lib/python3.10/site-packages/litellm/llms/huggingface/common_utils.pyr
s
zHuggingFaceError.__init__)NNN) __name__
__module__ __qualname__r httpxRequestResponser ZHeadersdictr
__classcell__r r r r r s r )ztext-generation-inferenceZconversationalztext-classificationztext-generationgenerated_textc C sh g d}|D ]+}| |r| |dd} | |r1| ddd |ddd ddddd } q| S )z
Parse the output text to remove any special characters. In our current approach we just check for ChatML tokens.
Initial issue that prompted this - https://github.com/BerriAI/litellm/issues/763
)z
<|assistant|>z
<|system|>z<|user|>zz N)strip
startswithreplaceendswith)r Zchat_template_tokenstokenr r r
output_parser- s
,r% )maxsizemodelreturnc
C s ddi}t drdt d |d< t d| }ddgi}z!tj|||d }| | d}|d
u r>td | |W S tjyq } z$t |d r]t
|j d
d}t
|j di }nd}i }t dt
| ||dd
}~ww )aT
Fetch provider mappings for a model from the Hugging Face Hub.
Args:
model: The model identifier (e.g., 'meta-llama/Llama-2-7b')
Returns:
dict: The inference provider mapping for the model
Raises:
ValueError: If no provider mapping is found
HuggingFaceError: If the API request fails
Acceptzapplication/jsonZHUGGINGFACE_API_KEYzBearer
Authorizationz /api/models/expandZinferenceProviderMapping)r
paramsNz$No provider mapping found for model r r i r
z"Failed to fetch provider mapping: )r r r
)osgetenv
HF_HUB_URLr getraise_for_statusjson
ValueError HTTPErrorhasattrgetattrr r str)r( r
pathr- r Zprovider_mappinger r r r !_fetch_inference_provider_mapping< s2
r; )r. functoolsr typingr r r r Z)litellm.llms.base_llm.chat.transformationr r0 r Zhf_tasksZ hf_task_listr8 r% r r; r r r r s