o yq i${@s(UdZddlZddlZddlZddlZddlZddlZddlZddlZddl m Z ddl m Z ddl mZddlmZmZmZmZmZddlZddlmZdd lmZdd lmZmZmZmZmZm Z m!Z!d d lm"Z"d d l#m$Z$d dl%m&Z&e"'e(Z)dZ*dZ+ej,dej-dZ.dej/ddfddZ0dej/ddfddZ1dej2ddfddZ3dej4fddZ5dej6fddZ7egej4fZ8egej6fZ9e:Z;e5ae9e=d!<da?eej4e=d"<d#e8ddfd$d%Z@d&e9ddfd'd(ZAdej4fd)d*ZBdej6fd+d,ZCdVd-d.ZDeEeDeFed/rejGeDd0d1d d2ejHejIfe jJd3d4d5e&d6eKd7eLd8eMd9eMd:eeNeOePeNeOd;ffdd?ZRd1d d2ejHejIfe jJd@d5e&d6eKd7eLd8eMd9eMd:eeNeOePeNeOd;ffdContains utilities to handle HTTP requests in huggingface_hub.N)contextmanager) HTTPStatus)quote)AnyCallable GeneratorOptionalUnion)OfflineModeIsEnabled) constants)BadRequestErrorDisabledRepoErrorGatedRepoErrorHfHubHTTPErrorRemoteEntryNotFoundErrorRepositoryNotFoundErrorRevisionNotFoundError)logging) SliceFileObj) HTTP_METHOD_TzX-Amzn-Trace-Idz x-request-ida  # staging or production endpoint ^https://[^/]+ ( # on /api/repo_type/repo_id /api/(models|datasets|spaces)/(.+) | # or /repo_id/resolve/revision/... /(.+)/resolve/(.+) ) )flagsrequestreturnc Cstjr td|jdt|jvr |jtptt |jt<|jt}t d||j |j|jddutjrBt dt||S)z Event hook that will be used to make HTTP requests to the Hugging Face Hub. What it does: - Block requests if offline mode is enabled - Add a request ID to the request headers - Log the request if debug mode is enabled z Cannot reach za: offline mode is enabled. To disable it, please unset the `HF_HUB_OFFLINE` environment variable.z%Request %s: %s %s (authenticated: %s) authorizationNzSend: %s)r ZHF_HUB_OFFLINEr urlX_AMZN_TRACE_IDheadersget X_REQUEST_IDstruuiduuid4loggerdebugmethodZHF_DEBUG_curlify)r request_idr)b/home/app/PaddleOCR-VL/.venv_paddleocr/lib/python3.10/site-packages/huggingface_hub/utils/_http.pyhf_request_event_hookIs"    r+cs t|S)z3 Async version of `hf_request_event_hook`. )r+)rr)r)r*async_hf_request_event_hookjsr,responsecsd|jdkr,d|jvr.z t|jd}Wn tyYdSw|dkr0|IdHdSdSdSdS)NzContent-lengthi@B) status_coderint ValueErrorZaread)r-lengthr)r)r*async_hf_response_event_hookqs   r3cCs"tjdtgidtjtjdddS)zQ Factory function to create a `httpx.Client` with the default transport. rTN@writeZ event_hooksZfollow_redirectstimeout)httpxClientr+Timeoutr DEFAULT_REQUEST_TIMEOUTr)r)r)r*default_client_factorys r=cCs&tjtgtgddtjtjdddS)zV Factory function to create a `httpx.AsyncClient` with the default transport. )rr-Tr4r5r7)r9 AsyncClientr,r3r;r r<r)r)r)r*default_async_client_factorys  r?_GLOBAL_CLIENT_FACTORY_GLOBAL_ASYNC_CLIENT_FACTORY_GLOBAL_CLIENTclient_factorycCs4tt|aWddS1swYdS)a Set the HTTP client factory to be used by `huggingface_hub`. The client factory is a method that returns a `httpx.Client` object. On the first call to [`get_client`] the client factory will be used to create a new `httpx.Client` object that will be shared between all calls made by `huggingface_hub`. This can be useful if you are running your scripts in a specific environment requiring custom configuration (e.g. custom proxy or certifications). Use [`get_client`] to get a correctly configured `httpx.Client`. N) _CLIENT_LOCK close_sessionr@)rCr)r)r*set_client_factorys "rFasync_client_factorycCs|adS)a Set the HTTP async client factory to be used by `huggingface_hub`. The async client factory is a method that returns a `httpx.AsyncClient` object. This can be useful if you are running your scripts in a specific environment requiring custom configuration (e.g. custom proxy or certifications). Use [`get_async_client`] to get a correctly configured `httpx.AsyncClient`. Contrary to the `httpx.Client` that is shared between all calls made by `huggingface_hub`, the `httpx.AsyncClient` is not shared. It is recommended to use an async context manager to ensure the client is properly closed when the context is exited. NrA)rGr)r)r*set_async_client_factorysrIcCs8tdurt taWdtS1swYtS)a Get a `httpx.Client` object, using the transport factory from the user. This client is shared between all calls made by `huggingface_hub`. Therefore you should not close it manually. Use [`set_client_factory`] to customize the `httpx.Client`. N)rBrDr@r)r)r)r* get_sessions  rJcCstS)a Return a `httpx.AsyncClient` object, using the transport factory from the user. Use [`set_async_client_factory`] to customize the `httpx.AsyncClient`. Contrary to the `httpx.Client` that is shared between all calls made by `huggingface_hub`, the `httpx.AsyncClient` is not shared. It is recommended to use an async context manager to ensure the client is properly closed when the context is exited. rHr)r)r)r*get_async_sessions rKc CsZt}da|dur+z|WdSty*}ztd|WYd}~dSd}~wwdS)z Close the global `httpx.Client` used by `huggingface_hub`. If a Client is closed, it will be recreated on the next call to [`get_session`]. Can be useful if e.g. an SSL certificate has been updated. NzError closing client: )rBclose Exceptionr$warning)clienter)r)r*rEs rEregister_at_fork)after_in_childF) max_retriesbase_wait_time max_wait_timeretry_on_exceptionsretry_on_status_codesstreamr&rrUrVrWrX.rYrZc +st|tr |f}ttrfd|} d} d|vr+t|dtjtfr+|d} t} d7za| dur?|d| dt j dt ffdd } |r~| j dd |} | | sn| V WdWdSWdn1sxwYn| j dd |} | | s| VWdSWn/|y}z#td |d d t|t jrtkr|WYd}~nd}~wwtd| dddt| t|| d} q/)zfInternal implementation of HTTP backoff logic shared between `http_backoff` and `http_stream_backoff`.rNdataTrr-rcsD|jvrdStd|jddkr t|dSdS)zNHandle response and return True if should retry, False if should return/yield.Fz HTTP Error z thrown while requesting  T)r/r$rNhf_raise_for_status)r-rUr&Znb_triesrYrr)r* _should_retry&s z)_http_backoff_base.._should_retry)r&r'z' thrown while requesting r\z Retrying in z s [Retry /z].r r)) isinstancetyper0ioIOBasertellrJseekr9ResponseboolrZrr$rNZ ConnectErrorrEtimesleepmin)r&rrUrVrWrXrYrZkwargsZ sleep_timeZio_obj_initial_posrOr_r-errr)r^r*_http_backoff_basesX       ro)rUrVrWrXrYc Ks$ttd|||||||dd|S)a Wrapper around httpx to retry calls on an endpoint, with exponential backoff. Endpoint call is retried on exceptions (ex: connection timeout, proxy error,...) and/or on specific status codes (ex: service unavailable). If the call failed more than `max_retries`, the exception is thrown or `raise_for_status` is called on the response object. Re-implement mechanisms from the `backoff` library to avoid adding an external dependencies to `hugging_face_hub`. See https://github.com/litl/backoff. Args: method (`Literal["GET", "OPTIONS", "HEAD", "POST", "PUT", "PATCH", "DELETE"]`): HTTP method to perform. url (`str`): The URL of the resource to fetch. max_retries (`int`, *optional*, defaults to `5`): Maximum number of retries, defaults to 5 (no retries). base_wait_time (`float`, *optional*, defaults to `1`): Duration (in seconds) to wait before retrying the first time. Wait time between retries then grows exponentially, capped by `max_wait_time`. max_wait_time (`float`, *optional*, defaults to `8`): Maximum duration (in seconds) to wait before retrying. retry_on_exceptions (`type[Exception]` or `tuple[type[Exception]]`, *optional*): Define which exceptions must be caught to retry the request. Can be a single type or a tuple of types. By default, retry on `httpx.TimeoutException` and `httpx.NetworkError`. retry_on_status_codes (`int` or `tuple[int]`, *optional*, defaults to `503`): Define on which status codes the request must be retried. By default, only HTTP 503 Service Unavailable is retried. **kwargs (`dict`, *optional*): kwargs to pass to `httpx.request`. Example: ``` >>> from huggingface_hub.utils import http_backoff # Same usage as "httpx.request". >>> response = http_backoff("GET", "https://www.google.com") >>> response.raise_for_status() # If you expect a Gateway Timeout from time to time >>> http_backoff("PUT", upload_url, data=data, retry_on_status_codes=504) >>> response.raise_for_status() ``` > [!WARNING] > When using `requests` it is possible to stream data by passing an iterator to the > `data` argument. On http backoff this is a problem as the iterator is not reset > after a failed call. This issue is mitigated for file objects or any IO streams > by saving the initial position of the cursor (with `data.tell()`) and resetting the > cursor between each call (with `data.seek()`). For arbitrary iterators, http backoff > will fail. If this is a hard constraint for you, please let us know by opening an > issue on [Github](https://github.com/huggingface/huggingface_hub). Fr&rrUrVrWrXrYrZNr))nextror&rrUrVrWrXrYrmr)r)r* http_backoffQsD rsc ks,td|||||||dd|EdHdS)ah Wrapper around httpx to retry calls on an endpoint, with exponential backoff. Endpoint call is retried on exceptions (ex: connection timeout, proxy error,...) and/or on specific status codes (ex: service unavailable). If the call failed more than `max_retries`, the exception is thrown or `raise_for_status` is called on the response object. Re-implement mechanisms from the `backoff` library to avoid adding an external dependencies to `hugging_face_hub`. See https://github.com/litl/backoff. Args: method (`Literal["GET", "OPTIONS", "HEAD", "POST", "PUT", "PATCH", "DELETE"]`): HTTP method to perform. url (`str`): The URL of the resource to fetch. max_retries (`int`, *optional*, defaults to `5`): Maximum number of retries, defaults to 5 (no retries). base_wait_time (`float`, *optional*, defaults to `1`): Duration (in seconds) to wait before retrying the first time. Wait time between retries then grows exponentially, capped by `max_wait_time`. max_wait_time (`float`, *optional*, defaults to `8`): Maximum duration (in seconds) to wait before retrying. retry_on_exceptions (`type[Exception]` or `tuple[type[Exception]]`, *optional*): Define which exceptions must be caught to retry the request. Can be a single type or a tuple of types. By default, retry on `httpx.Timeout` and `httpx.NetworkError`. retry_on_status_codes (`int` or `tuple[int]`, *optional*, defaults to `503`): Define on which status codes the request must be retried. By default, only HTTP 503 Service Unavailable is retried. **kwargs (`dict`, *optional*): kwargs to pass to `httpx.request`. Example: ``` >>> from huggingface_hub.utils import http_stream_backoff # Same usage as "httpx.stream". >>> with http_stream_backoff("GET", "https://www.google.com") as response: ... for chunk in response.iter_bytes(): ... print(chunk) # If you expect a Gateway Timeout from time to time >>> with http_stream_backoff("PUT", upload_url, data=data, retry_on_status_codes=504) as response: ... response.raise_for_status() ``` When using `httpx` it is possible to stream data by passing an iterator to the `data` argument. On http backoff this is a problem as the iterator is not reset after a failed call. This issue is mitigated for file objects or any IO streams by saving the initial position of the cursor (with `data.tell()`) and resetting the cursor between each call (with `data.seek()`). For arbitrary iterators, http backoff will fail. If this is a hard constraint for you, please let us know by opening an issue on [Github](https://github.com/huggingface/huggingface_hub). TrpNr))rorrr)r)r*http_stream_backoffsI rtendpointcCsD|r|dntj}|tjtjfvr |tj|}|tj|}|S)zReplace the default endpoint in a URL by a custom one. This is useful when using a proxy and the Hugging Face Hub returns a URL with the default endpoint. ra)rstripr ZENDPOINTZ_HF_DEFAULT_ENDPOINTZ_HF_DEFAULT_STAGING_ENDPOINTreplace)rrur)r)r*fix_hf_endpoint_in_urls rx endpoint_namec Csxz|WdStjy;}z%|jddkr WYd}~dS|jd}|jd}|dkrF|jddd |jd }tt||||d kr`|jddd |jd }tt ||||d krz|jddd|jd }tt ||||dkr|jddd|jd dd}tt ||||dks|jdkr|dkr|j dur|j jdurt t|j jdur|jddd|jd d}tt||||jdkr|durd|dnd}tt||||jdkr d|jd|d d|jd d}tt||||jdkr.|j jd }|d!|d"|jd#d }tt|||ttt|||d}~ww)$a Internal version of `response.raise_for_status()` that will refine a potential HTTPError. Raised exception will be an instance of [`~errors.HfHubHTTPError`]. This helper is meant to be the unique method to raise_for_status when making a call to the Hugging Face Hub. Args: response (`Response`): Response from the server. endpoint_name (`str`, *optional*): Name of the endpoint that has been called. If provided, the error message will be more complete. > [!WARNING] > Raises when the request has failed: > > - [`~utils.RepositoryNotFoundError`] > If the repository to download from cannot be found. This may be because it > doesn't exist, because `repo_type` is not set correctly, or because the repo > is `private` and you do not have access. > - [`~utils.GatedRepoError`] > If the repository exists but is gated and the user is not on the authorized > list. > - [`~utils.RevisionNotFoundError`] > If the repository exists but the revision couldn't be found. > - [`~utils.EntryNotFoundError`] > If the repository exists but the entry (e.g. the requested file) couldn't be > find. > - [`~utils.BadRequestError`] > If request failed with a HTTP 400 BadRequest error. > - [`~utils.HfHubHTTPError`] > If request failed for a reason not listed above. dNz X-Error-CodeX-Error-MessageZRevisionNotFoundz Client Error. zRevision Not Found for url: .Z EntryNotFoundzEntry Not Found for url: Z GatedRepoz!Cannot access gated repo for url z$Access to this resource is disabled.z!Cannot access repository for url  Z RepoNotFoundiz+Invalid credentials in Authorization headerzRepository Not Found for url: z Please make sure you specified the correct `repo_id` and `repo_type`. If you are trying to access a private or gated repo, make sure you are authenticated. For more details, see https://huggingface.co/docs/huggingface_hub/authenticationr.z Bad request for z endpoint:z Bad request:iz Forbidden: z Cannot access content at: z2 Make sure your token has the correct permissions.iRangez. Requested range: z. Content-Range: z Content-Range)raise_for_statusr9ZHTTPStatusErrorr/rrr_formatrrrrrREPO_API_REGEXsearchr!rr r)r-ryrPZ error_code error_messagemessageZ range_headerr)r)r*r]s~!             r] error_typecustom_messagec Cs*g}|jd}|dur||z[z|}Wn tjy8z ||}Wn ty5i}YnwYnw|d}|durRt|t rM| |n|||d}|durk|D] }d|vrj||dq]Wn tj y|jdd}|j rd| vr||j Ynwdd |D}t t|}d |} |} | r| | vrd |vr| d | 7} n| d | 7} t|jtd} | rd | d } nt|jtd} | rd| d } | r | | vr d | vr| d } | d| | | | d} n| | 7} || || pddS)Nr|errorerrorsrz Content-TypehtmlcSs$g|]}t|rt|qSr))r!strip).0liner)r)r* s$z_format..rr}z (Request ID: )z (Amzn Trace ID: )r-server_message)rrappendjsonr9ZResponseNotReadread RuntimeErrorrblistextendJSONDecodeErrortextlowerdictfromkeysjoinr!r rindexr)rrr-Z server_errorsZ from_headersr[rr content_typerZfinal_error_messager(Zrequest_id_messageZ newline_indexr)r)r*rxsl                rcCsdd|jfg}t|jD]\}}|dkrd}|d|d|fg7}qd}|jdurF|jjdd d }t|d krE|dd d }n|jdurMd }|dur\|d| ddfg7}|d|j fg7}g}|D]\}}|rw| t t ||r| t t |qhd|S)zConvert a `httpx.Request` into a curl command (str). Used for debug purposes only. Implementation vendored from https://github.com/ofw/curlify/blob/master/curlify.py. MIT License Copyright (c) 2016 Egor. )ZcurlNz-Xrzz-Hz: Nzutf-8ignore)riz ... [truncated]zz-drrr\)r&sortedritemsrcontentdecodelenrZrwrrrr!r)rpartskvbodyZ flat_partsr)r)r*r's4       r'z%^\s*bytes\s*=\s*(\d*)\s*-\s*(\d*)\s*$original_range resume_sizecCs|sd|dSd|vrtd|dt|}|s#td|d|\}}|sN|s5td|dt||}d|}|d krLtd |d|St|}||}|rrt|}d|d|}||krptd |d|Sd|dS) zB Adjust HTTP Range header to account for resume position. zbytes=-,zMultiple ranges detected - z, not supported yet.zInvalid range format - r~zbytes=-rzEmpty new range - )r1 RANGE_REGEXmatchrgroupsr0)rrrstartendZ new_suffixZ new_range new_startr)r)r*_adjust_range_headers2      r)rN)N)\__doc__atexitrdrosre threadingrjr" contextlibrhttprshlexrtypingrrrrr r9Zhuggingface_hub.errorsr rr rr rrrrrrrZ_lfsrZ_typingrZ get_logger__name__r$rr compileVERBOSErRequestr+r,rhr3r:r=r>r?ZCLIENT_FACTORY_TZASYNC_CLIENT_FACTORY_TLockrDr@__annotations__rArBrFrIrJrKrEregisterhasattrrQZTimeoutExceptionZ NetworkErrorSERVICE_UNAVAILABLEr!r0floatrcrMtuplerirorsrtrxr]rr' IGNORECASErrr)r)r)r*s     $    !           Y  S  U qV)"