o 0 i*< @sddlZddlZddlZddlmZddlmZmZmZm Z GdddeZ zddl m Z ddl mZmZmZdZWn eyEd ZYnwdd d Z ddd dd d eeeeee eeegeffffddZeeZeGdddZdS)N)Enum)AnyCallableOptionalUnionc@s,eZdZdZdZdZdZdZdZdZ dZ d S) RequestStatusz5Status of a generation request through its lifecycle.pendingZ prefillingZprefilling_splitZsplit_pending_remainderZdecodingfinishedfailedN) __name__ __module__ __qualname____doc__PENDING PREFILLINGPREFILLING_SPLITZSPLIT_PENDING_REMAINDERDECODINGFINISHEDZFAILEDrrf/home/app/PaddleOCR-VL-test/.venv_paddleocr/lib/python3.10/site-packages/transformers/utils/metrics.pyrsr)metrics)Status StatusCode get_tracerTFcstsddSfdd}|S)a Decorator that attaches a tracer to a class. This decorator should be applied to classes that need OpenTelemetry tracing. It adds a tracer attribute to the class instance that can be used by the traced decorator. Args: tracer_name_template: Optional template string for the tracer name. If provided, it should contain {module} which will be replaced with the class's full module path and {class_name} for the class name. If None, a default naming scheme will be used where: - If the module already starts with "transformers.", it will use that directly - Otherwise, it will prepend "transformers." to the module name Returns: Class decorator function cSs|SNr)clsrrr0szattach_tracer..cs*jtfdd}|_S)Ncsr|g|Ri|j}j}dur+|dr"|d|}nd|d|}nj||d}t||_dS)N transformers..)module class_name)r r startswithformatrtracer)selfargskwargs module_namer Z tracer_name)r original_inittracer_name_templaterrinit_with_tracer5s z:attach_tracer..decorator..init_with_tracer)__init__ functoolswraps)rr*r))rr(r decorator2s z attach_tracer..decorator)_has_opentelemetry)r)r/rr.r attach_tracers r1) span_name standaloneadditional_attributesr4cs$fdd}|dur|S||S)a Decorator to trace function calls with OpenTelemetry. Can be used as @traced or @traced(span_name="custom_name") Args: func: The function to trace span_name: Optional custom name for the span (defaults to function name) standalone: If True, creates a parentless span additional_attributes: Optional list of additional attributes to set on the span. Each item is a tuple of (instance_attribute_name, span_attribute_key, value_or_transform_function) where: - instance_attribute_name: Name of the attribute to get from the class instance - span_attribute_key: Key to use when setting the attribute on the span - value_or_transform_function: Either a raw value to use directly, or a function to transform the attribute value before setting it on the span Returns: Decorated function with tracing cs(tsStfdd}|S)Nc s,|rtdrjdur|dnd}|du}|r!t|dr!|j}n tdjdj}p1j}r7|jn|j}||}|dj|dj|d||rt |D]*\}} t | t t t tfsj| durv|d |t | qY|d |t t| qY|r|D]*\} } t | t t t tfs| dur|d | t | q|d | t t| qr|rވD]"} | \} }}t|| rt|| }t|r||}n|}|||qz|i|}|WWdSty }z|ttj||d}~ww1swYdS) N__self__rr#rrz function.namezfunction.modulezfunction.is_methodzargs.zkwargs.)hasattrr5r#rr r Z start_spanZstart_as_current_spanZ set_attribute enumerate isinstancestrintfloatbooltypeitemsgetattrcallable ExceptionZ set_statusrrERRORZrecord_exception)r%r&instanceZ is_methodr#nameZspan_fnspaniargkeyvalueZ attr_configZinstance_attribute_nameZspan_attribute_keyZvalue_or_transform_functionZattribute_valueZtransformed_valueresulte)r4funcr2r3rrwrapperlsT$          z*traced..decorator..wrapper)r0r,r-)rLrMr4r2r3)rLrr/hs 0ztraced..decoratorNr)rLr2r3r4r/rrNrtracedLs7rOc@seZdZdZdefddZddZedede d d fd d Z ed e d d fddZ edddZ ededed d fddZedede d d fddZd S)ContinuousBatchProcessorMetricsz0Metrics collection for ContinuousBatchProcessor.max_batch_tokenscCs||_|dS)zInitialize metrics for continuous batch processor. Args: max_batch_tokens: Maximum number of tokens in a batch N)rQ_setup_metrics)r$rQrrrr+s z(ContinuousBatchProcessorMetrics.__init__cCsts tddStd|_gd}|jjddd|d|_|jjd d d d |_ |jjd dd d |_ gd}|jjddd|d|_ |jjdddd |_ |jj dddd |_|jj dddd |_gd}|jjddd|d|_|jjddd d |_|jjd!d"d d |_dS)#zIInitialize OpenTelemetry metrics and tracing if the library is available.zIOpenTelemetry is not installed. Metrics and tracing will not be recorded.Nz2transformers.generation.continuous_batch_processor) 2Kdi,i'Zttft_millisecondsz#Time to first token in millisecondsms)rD descriptionunitZ#explicit_bucket_boundaries_advisoryZactive_requests_countz3Number of active requests currently being processedrequests)rDr`raZwaiting_requests_countz*Number of requests waiting to be processed) rUrWrZr[r\r]r^i Ni0ui`Zrequest_latency_millisecondsz9End-to-end latency for completed requests in millisecondsZdecode_prefill_ratioz3Ratio of decode tokens to prefill tokens in a batchratioZprefill_tokens_processedz"Number of prefill tokens processedtokensZdecode_tokens_processedz!Number of decode tokens processed) rS(rU<FPZ_brWZbatch_fill_percentagez5Percentage of max_batch_tokens utilized in each batchpercentZkv_cache_free_memory_bytesz/Free memory of the PagedAttentionCache in bytesbytesZkv_cache_memory_bytesz0Memory usage of the PagedAttentionCache in bytes)r0loggerinforZ get_meterZmeterZcreate_histogramttft_histogramZ create_gaugeactive_requests_gaugewaiting_requests_gaugerequest_latency_histogramdecode_prefill_ratio_gaugeZcreate_counterprefill_tokens_counterdecode_tokens_counterbatch_fill_percentage_histogramkv_cache_free_memory_gaugekv_cache_memory_gauge)r$Z ttft_bucketsZlatency_bucketsZbatch_fill_bucketsrrrrRsx   z.ContinuousBatchProcessorMetrics._setup_metrics created_time request_idreturnNc C|tsdSt|d}z|j|td|d|ddWdSty=}ztd|WYd}~dSd}~ww)zRecord Time to First Token (TTFT). Args: created_time: The time the request was created request_id: The ID of the request N@@zRecorded TTFT for request : .2fr_zFailed to record TTFT metric: )r0timertrecordrrdebugrAwarning)r$r~rZttft_msrKrrrrecord_ttft_metrics  z2ContinuousBatchProcessorMetrics.record_ttft_metricrequests_in_batchc Cs&tr|sdSd}d}|D]}|jtjkr|d7}q |jtjtjfvr)|t|j7}q ||}zI|dkr9|j ||dkrC|j ||dkrQ||}|j |||j d}|j|td|d|d|dd |d |j d WdSty}ztd |WYd}~dSd}~ww) zRecord metrics about the batch composition including decode/prefill ratio and batch fill percentage. Args: requests_in_batch: List of request states in the current batch NrgY@zBatch metrics: z decode tokens, z prefill tokens, batch fill: rz% (/)z Failed to record batch metrics: )r0statusrrrrlenZ prompt_idsryaddrzrxsetrQr{rrrrrAr) r$rZ decode_tokensZprefill_tokensstateZtotal_batch_tokensrdZfill_percentagerKrrrrecord_batch_metricssD       z4ContinuousBatchProcessorMetrics.record_batch_metricsc CstsdSzT|j|j}||jj}dt|j|j|}|}|j |}||}||}|j ||j |t d|ddd|d|j d||j d d d WdStys} zt d | WYd} ~ dSd} ~ ww) a&Record memory usage of the PagedAttentionCache without GPU synchronization. This calculates the theoretical memory usage based on cache configuration and the number of blocks currently in use. Args: cache: The PagedAttentionCache object to measure NzKV Cache memory: irzMB, Used blocks: rz (rWz.1fz%)z*Failed to record KV cache memory metrics: )r0Zhead_dimZnum_key_value_headsZdtypeitemsizerZ key_cache block_sizeZget_num_free_blocksZ num_blocksr}rr|rrrrAr) r$cacheZ page_sizeZpage_mem_in_bytesZblock_mem_in_bytesZ free_blocksZ used_blocksZused_memory_bytesZfree_memory_bytesrKrrrrecord_kv_cache_memory_metricsBs2       z>ContinuousBatchProcessorMetrics.record_kv_cache_memory_metricsactive_requestswaiting_requestsc CsvtsdSz|j||j|td|d|dWdSty:}ztd|WYd}~dSd}~ww)zRecord metrics about active and waiting requests. Args: active_requests: Number of active requests waiting_requests: Number of waiting requests NzQueue metrics: z active requests, z waiting requestsz Failed to record queue metrics: )r0rurrvrrrrAr)r$rrrKrrrrecord_queue_metricsjs  z4ContinuousBatchProcessorMetrics.record_queue_metricsc Cr)zRecord metrics about a completed request. Args: created_time: The time the request was created request_id: The ID of the request Nrz Recorded request completion for rrr_z,Failed to record request completion metric: )r0rrwrrrrrAr)r$r~rZ latency_msrKrrrrecord_request_completion|s  z9ContinuousBatchProcessorMetrics.record_request_completion)rN)r r r rr:r+rRrOr;r9rlistrrrrrrrrrPs Q( 'rPr)r,loggingrenumrtypingrrrrrZ opentelemetryrZopentelemetry.tracerrrr0 ImportErrorr1rtupler9rO getLoggerr rrrPrrrrs2    0& X