o |q iP@sjddlZddlZddlZddlmZddlmZddlm Z m Z m Z m Z m Z ddlmZddlmZddlmZddlmZmZdd lmZmZdd lmZmZmZmZmZm Z d Z!Gd d d e"Z#Gddde$eZ%GdddZ&Gddde&Z'Gddde'Z(Gddde&Z)Gddde&Z*Gddde&Z+Gddde+Z,GdddeZ-Gd d!d!ee-Z.dS)"N)abstractmethod)Enum)AnyDictListMappingOptional)urlparse)CallbackManagerForLLMRun)BaseLLM) Generation LLMResult)convert_to_secret_strget_from_dict_or_env) BaseModel ConfigDict SecretStrfield_validatormodel_validator validator2c @sVeZdZdZdefdededededdf d d Z dd ed e e d e defddZ dS)AzureMLEndpointClientz AzureML Managed Endpoint client. endpoint_urlendpoint_api_keydeployment_nametimeoutreturnNcCs,|r|std||_||_||_||_dS)zInitialize the class.zXA key/token and REST endpoint should be provided to invoke the endpointN) ValueErrorrrrr)selfrrrrr p/home/app/PaddleOCR-VL/.venv_paddleocr/lib/python3.10/site-packages/langchain_community/llms/azureml_endpoint.py__init__s zAzureMLEndpointClient.__init__body run_managerkwargscKs\dd|jd}|jdkr|j|d<tj|j||}tjj||d|jd}| }|S)zcall.application/jsonzBearer )z Content-Type Authorizationrzazureml-model-deploymentr)r) rrurllibrequestRequestrurlopengetrread)rr#r$r%headersreqresponseresultr r r!call.s   zAzureMLEndpointClient.callN) __name__ __module__ __qualname____doc__DEFAULT_TIMEOUTstrintr"bytesrr rr2r r r r!rs2 rc@seZdZdZdZdZdZdS)AzureMLEndpointApiTypezAzure ML endpoints API types. Use `dedicated` for models deployed in hosted infrastructure (also known as Online Endpoints in Azure Machine Learning), or `serverless` for models deployed as a service with a pay-as-you-go billing or PTU. dedicatedrealtime serverlessN)r4r5r6r7r=r>r?r r r r!r<Hs r<c @seZdZUdZ dZeeed< dZeeed< dZ eed<e dedefd d Z e de efd d Zejfded ededefddZeejfdededefddZdS)ContentFormatterBasezZTransform request and response of AzureML endpoint to match with required schema. r& content_typeacceptszError while formatting response payload for chat model of type `{api_type}`. Are you using the right formatter for the deployed model and endpoint type?format_error_msgpromptrcCs6dddddddd}|D] \}}|||}q|S) z*Escapes any special characters in `prompt`z\\z\"z\bz\fz\nz\rz\t)\"    )itemsreplace)rDZ escape_mapZescape_sequenceZescaped_sequencer r r!escape_special_characterss z.ContentFormatterBase.escape_special_characterscCtjgS)zSupported APIs for the given formatter. Azure ML supports deploying models using different hosting methods. Each method may have a different API structure.r<r=rr r r!supported_api_typessz(ContentFormatterBase.supported_api_types model_kwargsapi_typecCst)zFormats the request body according to the input schema of the model. Returns bytes or seekable file like object in the format specified in the content_type request header. )NotImplementedError)rrDrSrTr r r!format_request_payloads z+ContentFormatterBase.format_request_payloadoutputcCdS)zFormats the response body according to the output schema of the model. Returns the data type that is received from the response. Nr )rrWrTr r r!format_response_payloadsz,ContentFormatterBase.format_response_payloadN)r4r5r6r7rArr9__annotations__rBrC staticmethodrNpropertyrr<rRr=rrrVrr;r rYr r r r!r@Ts@     r@c@VeZdZdZedeefddZdede dede fdd Z d e dede fd d Z d S)GPT2ContentFormatterzContent handler for GPT2rcCrOr3rPrQr r r!rRz(GPT2ContentFormatter.supported_api_typesrDrSrTcC2t|}tdd|dgi|d}t|S)N input_stringrFZinputs parametersr@rNjsondumpsr9encoderrDrSrTrequest_payloadr r r!rV  z+GPT2ContentFormatter.format_request_payloadrWc CsTz t|dd}Wntttfy$}z t|jj|d|d}~wwt|dS)Nr0rTtext reloadsKeyError IndexError TypeErrorrrCformatr rrWrTchoiceer r r!rYs z,GPT2ContentFormatter.format_response_payloadNr4r5r6r7r\rr<rRr9rr;rVr rYr r r r!r^(  r^c0eZdZUdZdZeed<dfdd ZZS)OSSContentFormatterz`Deprecated: Kept for backwards compatibility Content handler for LLMs from the OSS catalog.Ncontent_formatterrcttddS)Nz`OSSContentFormatter` will be deprecated in the future. Please use `GPT2ContentFormatter` instead. superr"warningswarnrQ __class__r r!r" zOSSContentFormatter.__init__rN r4r5r6r7r|rrZr" __classcell__r r rr!r{  r{c@r])HFContentFormatterz6Content handler for LLMs from the HuggingFace catalog.rcCrOr3rPrQr r r!rRr_z&HFContentFormatter.supported_api_typesrDrSrTcCs.t|td|dg|d}t|S)NrFrbrdrhr r r!rVs   z)HFContentFormatter.format_request_payloadrWc CsXz t|ddd}Wntttfy&}z t|jj|d|d}~wwt|dS)Nrrkgenerated_textrlrmrorur r r!rYs z*HFContentFormatter.format_response_payloadNrxr r r r!rryrc@r])DollyContentFormatterz*Content handler for the Dolly-v2-12b modelrcCrOr3rPrQr r r!rRr_z)DollyContentFormatter.supported_api_typesrDrSrTcCr`)NrarF) input_datarcrdrhr r r!rVrjz,DollyContentFormatter.format_request_payloadrWc CsPz t|d}Wntttfy"}z t|jj|d|d}~wwt|dS)Nrrlrmrorur r r!rYs z-DollyContentFormatter.format_response_payloadNrxr r r r!rryrc@r])CustomOpenAIContentFormatterzAContent formatter for models that use the OpenAI like API scheme.rcCs tjtjgSr3)r<r=r?rQr r r!rRs z0CustomOpenAIContentFormatter.supported_api_typesrDrSrTcCsrt|}|tjtjfvrtdd|dg|di}n|tjkr,td|i|}ntd|dt |S)z/Formats the request according to the chosen apirrF)rarcrD `api_type` # is not supported by this formatter) r@rNr<r=r>rerfr?rr9rgrhr r r!rV"s$     z3CustomOpenAIContentFormatter.format_request_payloadrWc Cs|tjtjfvr2z t|dd}Wntttfy,}z t|j j |d|d}~wwt |dS|tj kryzt|dd}t |tsJtdWntttfyd}z t|j j |d|d}~wwt |dt|d |d d d Std |d)zFormats responserrkrlNrmchoicesziEndpoint response is not well formed for a chat model. Expected `dict` but `{type(choice)}` was received.rn finish_reasonlogprobs)rr)rnZgeneration_inforr)r<r=r>rerprqrrrsrrCrtr r? isinstancedictstripr,rur r r!rY;s>    z4CustomOpenAIContentFormatter.format_response_payloadNrxr r r r!rs( rcrz)LlamaContentFormatterzNDeprecated: Kept for backwards compatibility Content formatter for Llama.Nr|rcr})Nz`LlamaContentFormatter` will be deprecated in the future. Please use `CustomOpenAIContentFormatter` instead. r~rQrr r!r"drzLlamaContentFormatter.__init__rrr r rr!r]rrc@s@eZdZUdZdZeed< ejZ eed< e dZ e ed< dZ eed< eZeed< dZeed <d Zeed <dZeed < dZeeed < eddZeddededefddZed dededefddZ e!dddededefddZ"eddededefddZ#ed dd dedede$fd!d"Z%dS)#AzureMLBaseEndpointz Azure ML Online Endpoint models.rrendpoint_api_typerrrN http_client max_retriesr|rSr )Zprotected_namespacesbefore)modevaluesrcCsdtt|dd|d<t|dd|d<t|ddd|d<t|dd tj|d<t|d d tt|d <|S) NrZAZUREML_ENDPOINT_API_KEYrZAZUREML_ENDPOINT_URLrZAZUREML_DEPLOYMENT_NAMErrZAZUREML_ENDPOINT_API_TYPErZAZUREML_TIMEOUT)rrr<r=r9r8)clsrr r r!validate_environs, z$AzureMLBaseEndpoint.validate_environ field_valuecCs:|d}||jvrtdt|d|jd|d|S)z>Validate that content formatter is supported by endpoint type.rzContent formatter fz8 is not supported by this endpoint. Supported types are z but endpoint is .)r,rRrtype)rrrrr r r!validate_content_formatters   z.AzureMLBaseEndpoint.validate_content_formatteraftervaluecCs:|dr |dd}t|}|jr|jdkrtd|S)z'Validate that endpoint url is complete./Nz`endpoint_url` should contain the full invocation URL including `/score` for `endpoint_api_type='dedicated'` or `/completions` or `/models/chat/completions` for `endpoint_api_type='serverless'`)endswithr pathr)rrurlr r r!validate_endpoint_urls  z)AzureMLBaseEndpoint.validate_endpoint_urlcCsTt|d}|tjks|tjkr|jdkstd|tjkr(|jdvr(td|S)zBValidate that endpoint api type is compatible with the URL format.rz/scorezEndpoints of type `dedicated` should follow the format `https://..inference.ml.azure.com/score`. If your endpoint URL ends with `/completions` or`/models/chat/completions`,use `endpoint_api_type='serverless'` instead.)z /completionsz/models/chat/completionszEndpoints of type `serverless` should follow the format `https://..inference.ml.azure.com/completions` or `https://..inference.ml.azure.com/models/chat/completions`)r r,r<r=r>rrr?)rrrrr r r!validate_endpoint_api_types     z.AzureMLBaseEndpoint.validate_endpoint_api_typeT)alwayscCs@|d}|d}|d}|dt}t||||}|S)z?Validate that api key and python package exists in environment.rrrr)r,r8rZget_secret_value)rrrrZ endpoint_keyrrrr r r!validate_clients    z#AzureMLBaseEndpoint.validate_client)&r4r5r6r7rr9rZr<r=rrrrrr8rr:rrrr|rSrrrZ model_configr classmethodrrrr@rrrrrrr r r r!rmsV          rc @speZdZdZedeeeffddZedefddZ  dde ed e e ed e e d ede f d d ZdS)AzureMLOnlineEndpointaAzure ML Online Endpoint models. Example: .. code-block:: python azure_llm = AzureMLOnlineEndpoint( endpoint_url="https://..inference.ml.azure.com/score", endpoint_api_type=AzureMLApiType.dedicated, endpoint_api_key="my-api-key", timeout=120, content_formatter=content_formatter, ) rcCs |jpi}id|jid|iS)zGet the identifying parameters.rrS)rSr)r _model_kwargsr r r!_identifying_paramss z)AzureMLOnlineEndpoint._identifying_paramscCrX)zReturn type of llm.Zazureml_endpointr rQr r r! _llm_typeszAzureMLOnlineEndpoint._llm_typeNpromptsstopr$r%c Ksv|jpi}|||r||d<g}|D]!}|j|||j}|jj||d} |j| |j} || gqt |dS)anRun the LLM on the given prompts. Args: prompts: The prompt to pass into the model. stop: Optional list of stop words to use when generating. Returns: The string generated by the model. Example: .. code-block:: python response = azureml_model.invoke("Tell me a joke.") r)r#r$) generations) rSupdater|rVrrr2rYappendr ) rrrr$r%rrrDriZresponse_payloadrr r r! _generates"   zAzureMLOnlineEndpoint._generate)NN)r4r5r6r7r\rr9rrrrrr r rr r r r!rs&  r)/reurllib.requestr(rabcrenumrtypingrrrrr urllib.parser Z langchain_core.callbacks.managerr Z#langchain_core.language_models.llmsr Zlangchain_core.outputsr r Zlangchain_core.utilsrrZpydanticrrrrrrr8objectrr9r<r@r^r{rrrrrrr r r r!s2       / `B