o ưid+@s< UddlZddlZddlmZddlmZmZmZmZm Z m Z m Z m Z ddl mZddlmZddlZddlZddlmZddlmZmZddlmZdd lmZdd lmZmZmZmZm Z m!Z!m"Z"m#Z#dd l$m%Z&dd l'm%Z(dd l)m%Z*dd l+m,Z,dd l-m%Z.dd l/m%Z0dd l1m%Z2dd l3m%Z4dd l5m%Z6dd l7m%Z8dd l9m:Z;dd l9m%Z<dd l=m%Z>ddl?m@Z@ddlAmBZCdd lAm%ZDddlAmEZFdd lGm%ZHddlImJZJddlKmLZLddlMmNZNmOZOmPZPmQZQmRZRmSZSmTZTmUZUddlVmWZWmXZXddlYmZZZm[Z[m\Z\m]Z]m^Z^m_Z_m`Z`maZambZbmcZcddldmeZemfZfmgZgmhZhmiZimjZjmkZkmlZlmmZmmnZnmoZoer5ddlpmqZrneZreseejtjueejvjuhZweseejxjueejyjueejzjueej{juhZ|eseej}jueej~juhZeseejjueejjuhZeseejjueejjuhZeseejjueejjuhZeejjuZeejjuZ   ddedede ede efde ede e eeff dd Zd!ed"e ededede ef d#d$Zd%e ebdefd&d'Z (           )    dd!ededede ed"e ed*e ed+e ed,e ed-e ede efde ed.e ed/e ebd0e eWd1eZd2ed3e ed4e ed5e ede eeff(d6d7Z%dd8efd9d:Zd;edefdd?Z    dd!e ed8e ed@e edAe ed"e edBe ede efdCdDZeedEd!edefdFdGZd8ede efdHdIZdJdKdKddLZeedM<dNe ede efdOdPZd8ede ebfdQdRZdSdTZd1e eZd8ede eZfdUdVZdWed"e ede eeeffdXdYZdWed"e ede eeeeffdZd[Z       dd\e erd]ed^ed_ed`edae edbe edce edde edee edfe edge eddfdhdiZddd(gd(dddddddddddddddddfd!e edjedke ed1e eZdle edme edne ede efde edoe edAe ed@e edpe e_dqe edBe ed\e erd3e edef$drdsZdte eefde efdudvZ    (     ddwe eiegehemeleNeXeUe[ePee,f d!ed"e ed1edxdoedye ed@e edAe edzedpe e_dqe edBe ed\e erd3e edefd{d|Z dd!ed"e ed4e ede eeffd}d~Zd!e ed"ed4ecde eeffddZ dd!ed"e ede eWde eeffddZd!ed"e edede eeffddZ   dd!ed"e edme edne edle edoe edefddZ  dd!eded"e ede e^def ddZ  ddebd!ed"e ede e^de eeff ddZGdddZGdddeZdeQdebd"edqedef ddZdS)N) lru_cache) TYPE_CHECKINGAnyListLiteralOptionalTupleUnioncast)Response) BaseModel)verbose_logger)DEFAULT_MAX_LRU_CACHE_SIZE&DEFAULT_REPLICATE_GPU_PRICE_PER_SECOND)StandardBuiltInToolCostTracking)&TranscriptionUsageObjectTransformation)CostCalculatorUtils_generic_cost_per_character_get_service_tier_cost_key_parse_prompt_tokens_detailscalculate_cost_componentgeneric_cost_per_tokenget_billable_input_tokensselect_cost_metric_for_modelcost_per_token)SearchResponse)cost_per_second)get_model_params_and_category)cost_per_character) cost_router)ResponseAPILoggingUtils)LiteLLMSendMessageResponse)HttpxBinaryResponseContentImageGenerationRequestQualityOpenAIModerationResponseOpenAIRealtimeStreamList&OpenAIRealtimeStreamResponseBaseObject!OpenAIRealtimeStreamSessionEventsResponseAPIUsageResponsesAPIResponse)RerankBilledUnitsRerankResponse) CallTypesLiteral"LiteLLMRealtimeStreamLoggingObject LlmProvidersLlmProvidersSet ModelInfoStandardBuiltInToolsParams TranscriptionUsageDurationObjectTranscriptionUsageTokensObjectUsageVectorStoreSearchResponse) CallTypes CostPerTokenEmbeddingResponse ImageResponse ModelResponseModelResponseStreamProviderConfigManagerTextCompletionResponseTranscriptionResponse_cached_get_model_info_helper token_counter)Logging prompt_tokenscompletion_tokensresponse_time_mscustom_cost_per_tokencustom_cost_per_secondreturncCs\|dur |dur dS|dur|d|}|d|}||fS|dur,||d}d|fSdS)zFInternal helper function for calculating cost, if custom pricing givenNinput_cost_per_tokenoutput_cost_per_tokenr)rDrErFrGrH input_cost output_costrMrMN/home/app/Keep/.python/lib/python3.10/site-packages/litellm/cost_calculator.py%_cost_per_token_custom_pricing_helpers    rQmodelcustom_llm_providerc Cs|sdSz'd}|dkrddlm}||}|r&t|dr)|j|||dWSWdSWdStyF}ztd|WYd}~dSd}~ww)a Calculate additional costs beyond standard token costs. This function delegates to provider-specific config classes to calculate any additional costs like routing fees, infrastructure costs, etc. Args: model: The model name custom_llm_provider: The provider name (optional) prompt_tokens: Number of prompt tokens completion_tokens: Number of completion tokens Returns: Optional dictionary with cost names and amounts, or None if no additional costs Nazure_air)AzureFoundryModelInfocalculate_additional_costs)rRrDrEz$Error calculating additional costs: )Z"litellm.llms.azure_ai.common_utilsrUZget_azure_ai_config_for_modelhasattrrV Exceptionr debug)rRrSrDrEZ config_classrUerMrMrP_get_additional_costss,    r[ usage_blockcCs|durdSt|ddp d}t|ddpd}t|dd}|dur:t|ddp'd}t|ddp/d}|dks8|dkr:dS|dkpA|dkS) NFrDrrEprompt_tokens_detailsZ audio_tokensZ text_tokensT)getattr)r\Zprompt_tokens_valZcompletion_tokens_valZprompt_detailsZaudio_token_countZtext_token_countrMrMrP&_transcription_usage_has_token_detailss r_ completionprompt_characterscompletion_characterscache_creation_input_tokenscache_read_input_tokensnumber_of_queries usage_objectrerank_billed_units call_type!audio_transcription_file_duration service_tierresponse request_modelc( Csr|durtd| dur| }n t|||||| d}t|||| | d}|dur/|d|dfSd}d}tj}|}|durW|d|}|durV|d|d|}||vrV|}n tj|d\}}}}|}|dd}t|dkrt|d}n|} ||vr~|}n ||vr|}n||vr|}|d ks|d krtj||d }t |}d } d }!|d kr|durt d ||||t |||dddd\}"}#|"dus|#durt d |"|#|||||"} |#}!| |!fS|dkrt ||||d\} }!| |!fS|dks|dkrt|||dS|dks|dkrt||tt|dS|dks|dkr"t|||dS|dks8|dks8|tjks8|tjkr?t|||dS|dksI|dkr\t|rUt|||d St|||d!S|d"ksf|d#krdd$lm}$|$||| psd|rt|d%r|jd&Sdd&S|d'krt|||d(}%|%d krt|||||d)S|%dkrt ||||d*SdS|d+krt!||d,S|d-krt"|||d S|d.krt|||d S|d/krt#||d,S|d0krt$||d,S|d1krt%|||d2S|d3krt&|||d S|d4kr t'||d,S|d5krt(||d,S|d6kr t)||d,S|d7kr+t*||d,S|d8krddur|durt01d?||'/d>d||'d>|d@}|'/dAddur|durt01dB||'/dAd||'dA|d@}t01dC|||||fS)Dar Calculates the cost per token for a given model, prompt tokens, and completion tokens. Parameters: model (str): The name of the model to use. Default is "" prompt_tokens (int): The number of tokens in the prompt. completion_tokens (int): The number of tokens in the completion. response_time (float): The amount of time, in milliseconds, it took the call to complete. prompt_characters (float): The number of characters in the prompt. Used for vertex ai cost calculation. completion_characters (float): The number of characters in the completion response. Used for vertex ai cost calculation. custom_llm_provider (str): The llm provider to whom the call was made (see init.py for full list) custom_cost_per_token: Optional[CostPerToken]: the cost per input + output token for the llm api call. custom_cost_per_second: Optional[float]: the cost per second for the llm api call. call_type: Optional[str]: the call type Returns: tuple: A tuple containing the cost in USD dollars for prompt tokens and completion tokens, respectively. Nz"Invalid arg. Model cannot be none.)rDrE total_tokensrdre)rDrErFrHrGr/rRspeechaspeechrRrSrCrzvprompt_characters must be provided for tts calls. prompt_characters={}, model={}, custom_llm_provider={}, call_type={})rRrSrbrcZcustom_prompt_costZcustom_completion_costzcost for tts call is None. prompt_cost={}, completion_cost={}, model={}, custom_llm_provider={}, prompt_characters={}, completion_characters={}r)rRusagerSrkarerankrerank)rRrS billed_unitsZavector_store_searchZvector_store_search)rRrSrlZocrZaocraretrieve_batchretrieve_batch)rurRrSatranscription transcription)rRrurkrRrSdurationsearchasearchsearch_provider_cost_per_query_hidden_paramsrRrSrfoptional_params vertex_ai)rRrSri)rRrSrbrcru)rRrSrurkZ anthropic)rRruZbedrockopenaiZ databricksZ fireworks_aiazure)rRrurFgeminiZdeepseekZ perplexityZxaiZlemonadeZ dashscoperrT)rRrurFrmrJrKZinput_cost_per_secondz;For model=%s - input_cost_per_second: %s; response time: %srLoutput_cost_per_secondz standard pricing (service_tier = None) ON_DEMAND_PRIORITY -> priority pricing (service_tier = "priority") FLEX / BATCH -> batch/flex pricing (service_tier = "flex") N)rrupper)rrkrMrMrP!_map_traffic_type_to_service_tiers rcCsttttttft|tr|dn t|dddd}|dur#dSt|tr*|S|durBt|ts8t|trBt |rBt |St |rSt ttttf|St|tr_tdi|St|trmtdi|Stdt|d|dS)NrurcSsdSNrM)xrMrMrPsz#_get_usage_object..zUnknown usage object type: z , usage_obj: rM)r r r5r)rr rrr^r!_is_response_api_usage+_transform_response_api_usage_to_chat_usageris_transcription_usage_object$transform_transcription_usage_objectr3r4 model_dumpr rYtype)r usage_objrMrMrP_get_usage_objectsP     rcCs t|tjpt|tpt|S)z3Returns True if the usage obj is a known Usage type)rrr5r)rrrrMrMrP_is_known_usage_objectss rcCs|dur|S|dur dSt|tst|trdSt|trdSt|tr&dSt|tr-dSt|tr4dSt|tr;dSt|trBdSt|t rIdS|S) Nra embeddingr|rrrwimage_generationtext_completion send_message) rr;r<r9r?r#r,r:r>r")rirrMrMrP_infer_call_type!s.        r base_costc Cs|}d}d}|r?|tjvr?tj|}||}||}ttjr:td|dd|d|dd|dd|dd |||fS|||fS) a Apply provider-specific cost discount from module-level config. Args: base_cost: The base cost before discount custom_llm_provider: The LLM provider name Returns: Tuple of (final_cost, discount_percent, discount_amount) rCzApplied dz% discount to : $.6f -> $z (saved $))rcost_discount_configr isEnabledForloggingDEBUGrY)rrS original_costdiscount_percentdiscount_amount final_costrMrMrP_apply_cost_discount@s&    rcCs|}d}d}d}d}|r(|tjvr(tj|}ttjr'td|d|n/dtjvrAtjd}ttjr@td|nttjrWtd|dttj|durt |t t frkt |}||}n#t |t rd |vrt |d }|||7}d |vrt |d }||7}||}ttjrtd |pdd |d d|d d|dkr|dndd|d d|d d ||||fS||||fS)aC Apply provider-specific or global cost margin from module-level config. Args: base_cost: The base cost before margin (after discount if applicable) custom_llm_provider: The LLM provider name Returns: Tuple of (final_cost, margin_percent, margin_fixed_amount, margin_total_amount) rCNz*Found provider-specific margin config for z: globalzUsing global margin config: z"No margin config found. Provider: z, Available configs: percentageZ fixed_amountzApplied margin to rrrz (margin: rrz% + $z = $r) rcost_margin_configr rrrrYlistkeysrintfloatr)rrSrmargin_percentmargin_fixed_amountmargin_total_amountZ margin_configrrMrMrP_apply_cost_marginbsf                rlitellm_logging_objrr'cost_for_built_in_tools_cost_usd_dollartotal_cost_usd_dollaradditional_costsrrrrrrc Csn|durdSz|j||||||||| | | d WdSty6} ztdt| WYd} ~ dSd} ~ ww)ar Helper function to store cost breakdown in the logging object. Args: litellm_logging_obj: The logging object to store breakdown in prompt_tokens_cost_usd_dollar: Cost of input tokens completion_tokens_cost_usd_dollar: Cost of completion tokens (includes reasoning if applicable) cost_for_built_in_tools_cost_usd_dollar: Cost of built-in tools total_cost_usd_dollar: Total cost of request additional_costs: Free-form additional costs dict (e.g., {"azure_model_router_flat_cost": 0.00014}) original_cost: Cost before discount discount_percent: Discount percentage applied (0.05 = 5%) discount_amount: Discount amount in USD margin_percent: Margin percentage applied (0.10 = 10%) margin_fixed_amount: Fixed margin amount in USD margin_total_amount: Total margin added in USD N) rNrO total_costrrrrrrrrzError storing cost breakdown: )Zset_cost_breakdownrXr rYr) rrrrrrrrrrrrZbreakdown_errorrMrMrP$_store_cost_breakdown_in_logging_objs* rmessagesrsizequalitynrstandard_built_in_tools_paramslitellm_model_namecJCsV zt||pd}|dks|dkr&|dur&t|tr&t|dkr&|dkr&d}d}d}d}d}d}d}d}t|d }d}|durH|durH|d }|durf|durft|tr\t|d d}n t|trf|d }|dur|durt|trzt|d d}n t|tr|d }t ||||||d }|t |g} |dur| |t | D]r\}!}zDt tjrt d ||durt|tst|trt|tr|d i}"nt|d i}"t|"trt|"dst|d tjdWi|"|"duri}#n t|"tr|"}#n|"}#t|#rt|#}#nt|#r.ttttt f|#}$|$dur-|$}#n|#}#|#dd}|#dd}|#dd}|#dd}d|#vre|#dikre|#dre|#di}%|%dd}t|dd}t|dd}&|&dur|&d|p}d}|&d|}|dur|&dpi}'|'d}(|(rt!|(}n0|durt"d|d|t|dkrt#||d}n t|dkrt#||d}t#||d}|t$vrddl%m&})|)j'|d WWS|durt"d|d||dur"z tj(|d!\}}}*}*Wnt)y!}+zt d"*t|+WYd}+~+nd}+~+wwt+,|r?t|t-r?t+j.|||| | | ||d#WWS|t/vrd},|rh|durht|d$d}-|-durh|-d%ipai}.|.d&d},t|d d}"|dur|"rt|"tr|"d'd}/nt|"d'd}/|/durdd(l0m1}0|0||/||,d)WWSt2|d||,d)WWS|t3vrtj4j5|d*}n|t6vrt|dipi}1|1d+t|d,d}n|t7vr|dur t|t8r |j9}2|2dur|2d-ipi}3ni}3t:|3d.|3d/d0}|3d.p d1}4|4}n|t;vrdd2lr0t|7}6n|7dur7d1}6|p;d4}8|rId5|8vrI|d6}8|5|8||6|d7\}9}:|9|:};|;}t@|;|d8\};}?}@}AtA||9|:d|;|<|=|>|?|@|Ad9 |;WWS|tBkrt|tCr|dus|durt"d:*||tD|jE|||d;WWS|tFkrdd|vs|d>krtJ|tK|d?}n|tjLvsd@|vr|tjMvrtN||WWS|durt"d|d||dur<|dAkrnd}=d}>tjXrt@|;|d8\};}?}@}And}?d}@d}A|durtA||F|G|I|;|<|H|=|>|?|@|AdT |;WWSt)y}+zt dU*|t|+|!t| d1kr |+WYd}+~+qd}+~+wwt)dV*| t)y*}+z|+d}+~+ww)Xa Calculate the cost of a given completion call fot GPT-3.5-turbo, llama2, any litellm supported llm. Parameters: completion_response (litellm.ModelResponses): [Required] The response received from a LiteLLM completion request. [OPTIONAL PARAMS] model (str): Optional. The name of the language model used in the completion calls prompt (str): Optional. The input prompt passed to the llm completion (str): Optional. The output completion text from the llm total_time (float, int): Optional. (Only used for Replicate LLMs) The total time used for the request in seconds custom_cost_per_token: Optional[CostPerToken]: the cost per input + output token for the llm api call. custom_cost_per_second: Optional[float]: the cost per second for the llm api call. Returns: float: The cost in USD dollars for the completion based on the provided parameters. Exceptions: Raises exception if model not in the litellm model cost map. Register model, via custom pricing or PR - https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json Note: - If completion_response is provided, the function extracts token information and the model name from it. - If completion_response is not provided, the function calculates token counts based on the model and input text. - The cost is calculated based on the model, prompt tokens, and completion tokens. - For certain models containing "togethercomputer" in the name, prices are based on the model size. - For un-mapped Replicate models, the cost is calculated based on the total time used for the request. raaimage_generationrNrrzdall-e-2rCrrk)rRrrSrrrz*selected model name for cost calculation: rurrDrErdrer]Z cached_tokensZ _response_msrrSrZprovider_specific_fieldsrz[Model is None and does not exist in passed completion_response. Passed completion_response=z, model=)rRr)rRtext)A2ACostCalculator)rrqzXlitellm.cost_calculator.py::completion_cost() - Error inferring custom_llm_provider - {})rRrSrrrrrriZlitellm_paramsmetadatarduration_seconds)video_generation_cost)rRrrSr)rZaudio_transcription_durationr~rx search_unitsrn)r rnrorqueryr`rpz/searchr)rrS) rrrrrrrrrrrzusage object and custom_llm_provider must be provided for realtime stream cost calculation. Got cost_per_token_usage_object={}, custom_llm_provider={})resultscombined_usage_objectrSr)MCPCostCalculatorZtogethercomputerZ together_ai)riZ replicaterr)datari)Z response_objrRrFrHrGrbrcrgrirjrhrlrmrT)rRrSrDrE)rRresponse_objectrurrS) rrrrrrrrrrrrzXlitellm.cost_calculator.py::completion_cost() - Error calculating cost for model={} - {}z?Unable to calculat cost for received potential model names - {}rM)Yrrrrrrr r^rrrappend enumerater rrrrYrsetattrrr5rr!rrrrrr r r3r4rrrA_A2A_CALL_TYPESZ$litellm.a2a_protocol.cost_calculatorrZcalculate_a2a_costrrXrrZ_call_type_has_image_responser:Z&route_image_generation_cost_calculator_VIDEO_CALL_TYPES$litellm.llms.openai.cost_calculationr default_video_cost_calculator_SPEECH_CALL_TYPESutilsZ_count_characters_TRANSCRIPTION_CALL_TYPES_RERANK_CALL_TYPESr,metar+_SEARCH_CALL_TYPESrrrrrr_AREALTIME_CALL_TYPEr.'handle_realtime_stream_cost_calculationr _MCP_CALL_TYPEZ6litellm.proxy._experimental.mcp_server.cost_calculatorrZcalculate_mcp_tool_call_costrr7Zreplicate_modelsrrZget_formatted_promptr;Zget_response_stringrRrr[rZget_cost_for_built_in_toolsrr)JrrRpromptrrarrirSrrrrrGrHrrrrrrrrkrDrbrErcrdrerjZcost_per_token_usage_objectrhZselected_modelpotential_model_namesidxrZ_usageZtr_usager]rZprovider_specificZraw_traffic_typerrrZZ_video_model_infoZ_litellm_params _metadatarr Z_hiddenZmeta_objrxr rrfr Z search_modelrZcompletion_cost_resultZ _final_costrrrrrrrZ prompt_stringZcompletion_stringZrequest_model_for_costrrrZcost_for_built_in_toolsrMrMrPrs8                                                                                        rrcCsTt|tr |}n|}|di}|r(d|vr(|d}|dur"dSt|dSdS)Nadditional_headersz$llm_provider-x-litellm-response-cost)rr rrr)rZ_hidden_params_dictr%rrMrMrP$get_response_cost_from_hidden_params-s    r&r)rZ aembeddingraZ acompletionZatext_completionrrrZ moderationZ amoderationr{r|rsrrrwrvrr cache_hitr!cCsz>d}|dur|durd}|WSt|tr+t|dr+||jd<t|j}|dur+|WSt||||||||| | | | | d }|WStyL}z|d}~ww)z7 Returns - float or None: cost of response rCNTrr) rrRrirSrrrr!rrrrrk)rr rWrr&rrX)rrRrSrirr'rrr!rrrrrkrZprovider_response_costrZrMrMrPresponse_cost_calculatorAsB5  r(cCsddlm}|dust||stdt||jdur!td|jj}|dur-tdz tj||d}Wn t yAd}Ynwd}|durO| d pNd}||}|dfS) aJ Args: model: str - model name custom_llm_provider: Optional[str] - custom LLM provider response: Optional[Any] - response object Returns: Tuple[float, float]: cost of OCR processing (Parent function requires a tuple, so we return a tuple. Cost is only in the first element.) r) OCRResponseNz.response must be of type OCRResponse got type=zOCR response usage_info is Nonez$OCR response pages_processed is NonertrCocr_cost_per_page) Z(litellm.llms.base_llm.ocr.transformationr)rrrZ usage_infopages_processedrrrXr)rRrSrlr)r+rr*Ztotal_ocr_processing_costrMrMrPrs,    rcCsnd}|durd}|durd|vrtj|d\}}}}tjt||d}|dur1td|dS|j|dS) zB Returns - float or None: cost of vector store search Nrrprq)providerapi_typez)Vector store search is not supported for rCrC)rl)rrr=Z!get_provider_vector_stores_configr/r rYZcalculate_vector_store_cost)rRrSrlr-rconfigrMrMrPrs& rrxc Cstj||d\}}}}z*tj|dgt|d}z tj||d}Wn ty+d}Ynw|j||||dWStyC}z|d}~ww)zI Returns - float or None: cost of response OR none if error. rtN)rRZapi_baseZpresent_version_paramsr,)rRrSrxr)rrr=Zget_provider_rerank_configr/rrXZcalculate_rerank_cost)rRrSrxrr/rrZrMrMrPrs6  rr~cCst|||dS)Nr})rr}rMrMrPtranscription_costsr0ro 1024-x-1024cCs|pd}d|vrd|vr|ddn|}tt|d\}}|d|} d} |rD||drD||dd} |d|d| } |rM|d| n| } tjjd| } t d| d| |d|dd } |rx|d| n| }d}| | | || || g}|D]}|dur|t j vrt j |}nq|durt d |d |vr|d dur|d |Sd |vr|d dur|d |||St d |d|)a Default image cost calculator for image generation Args: model (str): Model name image_response (ImageResponse): Response from image generation quality (Optional[str]): Image quality setting n (Optional[int]): Number of images generated size (Optional[str]): Image size (e.g. "1024x1024" or "1024-x-1024") Returns: float: Cost in USD for the image generation Raises: Exception: If model pricing not found in cost map r1rz-x-rpNr`zLooking up cost for models: z, z,Model not found in cost map. Tried checking Zinput_cost_per_imageZinput_cost_per_pixelz'No pricing information found for model z. Tried checking ) replacemaprr startswithr$ZHIGHvaluer rYrrrX)rRrSrrrrZsize_strheightwidthbase_model_name&model_name_without_custom_llm_providerZmodel_name_with_qualityZmodel_name_with_v2_qualitymodel_without_providerZ#model_with_quality_without_provider cost_infomodels_to_check_modelrMrMrPdefault_image_cost_calculatorsf       r?rrc Cs4d}|dur t|}n_|}d}|r)||dr)||dd}|d|}td||dd}||||g}|D]} | durR| tjvrRtj| }nq@|durj|rj|d|} | tjvrjtj| }|durutd|| d} | dur| |S| d} | dur| |St d |d d S) aN Default video cost calculator for video generation Args: model (str): Model name duration_seconds (float): Duration of the generated video in seconds custom_llm_provider (Optional[str]): Custom LLM provider model_info (Optional[ModelInfo]): Deployment-level model info containing custom video pricing. When provided, used before falling back to the global litellm.model_cost lookup. Returns: float: Cost in USD for the video generation Raises: Exception: If model pricing not found in cost map Nrpr`z!Looking up cost for video model: r2z&Model not found in cost map for model=Z output_cost_per_video_per_secondrz*No cost information found for video model z<. Please add pricing to model_prices_and_context_window.jsonrC) rr5r3r rYrrrrXrinfo) rRrrSrr<r9r:r;r=r>Zprefixed_modelZvideo_cost_per_secondrrMrMrPr}sR          rrucCstj||d\}}}}td|||dur+z tj||d}Wn ty*d}Ynw|s/dS|d}|d}|d}|d}d } d } |rO|j|} n!|rpt||d } t |} | d } t d d} | t || | d 7} |r{|j |} | | fS|r|j |d } | | fS) aF Calculate the cost of a batch job. Args: model_info: Optional deployment-level model info containing custom batch pricing (e.g. input_cost_per_token_batches). When provided, skips the global litellm.get_model_info() lookup so that deployment-specific pricing is used. rtzBCalculating batch cost per token. model=%s, custom_llm_provider=%sNr.input_cost_per_token_batchesrJoutput_cost_per_token_batchesrKrCZcache_hit_tokensZcache_read_input_token_cost) rrr rYrrXrrDrrrrrE)rurRrSrrrArJrBrKZtotal_prompt_costZtotal_completion_costdetailsZcache_read_tokensZcache_read_cost_keyrMrMrPrs\         rc@s&eZdZedeedefddZdS)BaseTokenUsageProcessor usage_objectsrIc Csddlm}m}m}|}|D]}t|D]4}|dsItt||sIt||d}t||d}|durIt|t t frIt|t t frIt ||||qt |dr|j rt |drZ|j s^||_ t|j jD];}t |j |r|dstt|j |st|j |dpd}t|j |dpd}|durt|t t frt |j |||qdt |dr|jrt |dr|js||_t|jjD].}|dstt|j|st|j|d}t|j|d}|dur|durt |j|||qq|S)zs Combine multiple Usage objects into a single Usage object, checking model keys for nested values. r)CompletionTokensDetailsWrapperPromptTokensDetailsWrapperr5rNr]completion_tokens_details)litellm.types.utilsrGrHr5dirr5callabler^rrrrrWr]rZ model_fieldsrI) rFrGrHr5combinedruattrZ current_valnew_valrMrMrPcombine_usage_objectss~        z-BaseTokenUsageProcessor.combine_usage_objectsN)__name__ __module__ __qualname__ staticmethodrr5rPrMrMrMrPrEsrEc@sVeZdZededeefddZededefddZededede fdd Z d S) RealtimeAPITokenUsageProcessorr rIcCsJtttdd|D}g}|D]}t|ddi}||q|S)z< Collect usage from realtime stream results cSsg|] }|ddkr|qS)rz response.donerM).0resultrMrMrP zsz]RealtimeAPITokenUsageProcessor.collect_usage_from_realtime_stream_results..rlru)r rr'r!rrr)r Zresponse_done_eventsrFrWrgrMrMrP*collect_usage_from_realtime_stream_resultsqs  zIRealtimeAPITokenUsageProcessor.collect_usage_from_realtime_stream_resultscCst|}t|}|S)zH Collect and combine usage from realtime stream results )rUrYrP)r Zcollected_usage_objectsr rMrMrP6collect_and_combine_usage_from_realtime_stream_resultsszURealtimeAPITokenUsageProcessor.collect_and_combine_usage_from_realtime_stream_resultsrucCs t||dS)Nrur )r.r[rMrMrPcreate_logging_realtime_objectsz=RealtimeAPITokenUsageProcessor.create_logging_realtime_objectN) rQrRrSrTr&rr5rYrZr.r\rMrMrMrPrUps*  rUr r c Csd}g}|D]}|ddkrtt|ddd}||q||d}d}|D]%} z| dur3Wq*t| ||d\} } Wn tyFYq*w|| 7}|| 7}||} | S)z Handles the cost calculation for realtime stream responses. Pick the 'response.done' events. Calculate total cost across all 'response.done' events. Args: results: A list of OpenAIRealtimeStreamBaseObject objects Nrzsession.createdsessionrRrC)rRrurS)r r(rrrrX) r r rSrZreceived_modelr"rWrJrKZ model_nameZ_input_cost_per_tokenZ_output_cost_per_tokenrrMrMrPrs>    r)rrrCNN)r`rrrCNNNNrrNNNNNrarCNNN)rCr)NNNN)NNNNNNN) NNNr`NNNNN)NNror1N)NN)rr functoolsrtypingrrrrrrr r Zhttpxr Zpydanticr rZlitellm._loggingr Zlitellm.constantsrrZ@litellm.litellm_core_utils.llm_cost_calc.tool_call_cost_trackingrZDlitellm.litellm_core_utils.llm_cost_calc.usage_object_transformationrZ.litellm.litellm_core_utils.llm_cost_calc.utilsrrrrrrrrZ'litellm.llms.anthropic.cost_calculationrrZ#litellm.llms.azure.cost_calculationrZ%litellm.llms.azure_ai.cost_calculatorrZ+litellm.llms.base_llm.search.transformationrZ%litellm.llms.bedrock.cost_calculationrZ'litellm.llms.databricks.cost_calculatorrZ%litellm.llms.deepseek.cost_calculatorrZ)litellm.llms.fireworks_ai.cost_calculatorrZ#litellm.llms.gemini.cost_calculatorrZ%litellm.llms.lemonade.cost_calculatorrrrrrZ'litellm.llms.perplexity.cost_calculatorrZ(litellm.llms.together_ai.cost_calculatorrZ&litellm.llms.vertex_ai.cost_calculatorrrrr rZ litellm.llms.xai.cost_calculatorrZlitellm.responses.utilsr!Zlitellm.types.agentsr"Zlitellm.types.llms.openair#r$r%r&r'r(r)r*Zlitellm.types.rerankr+r,rJr-r.r/r0r1r2r3r4r5r6Z litellm.utilsr7r8r9r:r;r<r=r>r?r@rAZ*litellm.litellm_core_utils.litellm_loggingrBZLitellmLoggingObject frozensetZ asend_messager6rrZ create_videoZ acreate_videoZ video_remixZ avideo_remixrrrrsrr{r|rrwrvrrrrZ arealtimerZ call_mcp_toolr rrQrrrr[boolr_rrrrrrr__annotations__rrrrrrrrr&r(rrrr0r?rrrErUrrMrMrMrPs  (     (                   ( 04   .      Q   D  0    " I     9   T  :%&'()*+,-/0 Z  0  %  &    h V  HX1