o ưiC@sdZddlZddlZddlmZmZmZmZddl m Z ddl mZddl m ZddlmZddlmZmZmZmZmZmZmZmZmZmZmZmZmZdd lm Z m!Z!m"Z"dd l#m$Z$Gd d d eZ%dS) z This file contains the transformation logic for Bedrock Nova Sonic realtime API. Transforms between OpenAI Realtime API format and Bedrock Nova Sonic format. N)AnyListOptionalUnion)verbose_logger)uuid)Logging)BaseRealtimeConfig) OpenAIRealtimeContentPartDoneOpenAIRealtimeDoneEventOpenAIRealtimeEventsOpenAIRealtimeOutputItemDoneOpenAIRealtimeResponseAudioDone&OpenAIRealtimeResponseContentPartAddedOpenAIRealtimeResponseDelta OpenAIRealtimeResponseDoneObjectOpenAIRealtimeResponseTextDone&OpenAIRealtimeStreamResponseBaseObject+OpenAIRealtimeStreamResponseOutputItemAddedOpenAIRealtimeStreamSession!OpenAIRealtimeStreamSessionEvents)ALL_DELTA_TYPESRealtimeResponseTransformInputRealtimeResponseTypedDict)get_empty_usagec@s,eZdZdZddZ dEdededeedefd d Z dEd eededeedefd d Z de fddZ dEdedee edefddZ de ede efddZdFdede defddZdede efddZdede efddZdede efdd Zdede efd!d"Zdede efd#d$Zdede efd%d&Z dEd'eded(eede efd)d*Zd+eded,edefd-d.Zd+ed/eed0eed1eedee eeeeeeeeeff d2d3Zd+ed0eed/eed4ee edee eee eff d5d6Z d+ed0eed/eede efd7d8Z!d+ed0eed/eed9eed4ee edee eee eff d:d;Z"d+ed/eed1eedee eeeeeeeffdd?Z$dede efd@dAZ%d'e&ee'fded,edBe(de)f dCdDZ*dS)GBedrockRealtimeConfigz>Configuration for Bedrock Nova Sonic realtime transformations.cCstt|_tt|_tt|_d|_d|_d|_d|_ d|_ d|_ d|_ d|_ d |_d |_d |_d|_d|_d|_d |_d |_d |_dS) Nig?gffffff?]Zmatthewbase64ZSPEECHz audio/lpcm> text/plain)struuid_libuuid4 prompt_nameZ content_nameaudio_content_name max_tokenstop_p temperatureoutput_sample_rate_hertzoutput_sample_size_bitsoutput_channel_countvoice_idoutput_encodingoutput_audio_typeoutput_media_typeinput_sample_rate_hertzinput_sample_size_bitsinput_channel_countinput_encodinginput_audio_typeinput_media_typetext_media_typeselfr:c/home/app/Keep/.python/lib/python3.10/site-packages/litellm/llms/bedrock/realtime/transformation.py__init__)s( zBedrockRealtimeConfig.__init__Nheadersmodelapi_keyreturncCs|S)z@Validate environment - no special validation needed for Bedrock.r:)r9r=r>r?r:r:r;validate_environmentIsz*BedrockRealtimeConfig.validate_environmentapi_basecCs|pdS)z6Get complete URL - handled by aws_sdk_bedrock_runtime.r:)r9rBr>r?r:r:r;get_complete_urlOsz&BedrockRealtimeConfig.get_complete_urlcCsdS)z'Bedrock requires session configuration.Tr:r8r:r:r;requires_session_configurationUsz4BedrockRealtimeConfig.requires_session_configurationtoolsc Csddd|j|j|jdiii}|jd|ji|j|j|j|j|j |j |j dd}|r9ddi|d <d | |i|d <dd |ii}t ||d S)z Create initial session configuration for Bedrock Nova Sonic. Args: model: Model ID tools: Optional list of tool definitions Returns JSON string with session start and prompt start events. event sessionStartinferenceConfigurationZ maxTokensZtopPr) mediaTyperKsampleRateHertzsampleSizeBits channelCountZvoiceIdencoding audioType promptNameZtextOutputConfigurationZaudioOutputConfigurationapplication/jsontoolUseOutputConfigurationrFtoolConfiguration promptStart) session_start prompt_start)r'r(r)r%r7r0r*r+r,r-r.r/"_transform_tools_to_bedrock_formatjsondumps)r9r>rFrXprompt_start_configrYr:r:r;session_configuration_requestYs<    z3BedrockRealtimeConfig.session_configuration_requestc Cshg}|D]-}|ddkr1|di}d|dd|dddt|diid i}||q|S) z Transform OpenAI tool format to Bedrock tool format. Args: tools: List of OpenAI format tools Returns: List of Bedrock format tools typefunctionZtoolSpecnamerC descriptionr[ parameters)rarbZ inputSchema)getr[r\append)r9rFZ bedrock_toolsZtoolr`Z bedrock_toolr:r:r;rZs     z8BedrockRealtimeConfig._transform_tools_to_bedrock_formatT audio_format is_outputcCs,|dkr |rdSdS|dvrdS|rdSdS)a" Map OpenAI audio format to sample rate. Args: audio_format: OpenAI audio format (pcm16, g711_ulaw, g711_alaw) is_output: Whether this is for output (True) or input (False) Returns: Sample rate in Hz Zpcm16rr )Z g711_ulawZ g711_alawi@r:)r9rfrgr:r:r; _map_audio_format_to_sample_rates   z6BedrockRealtimeConfig._map_audio_format_to_sample_rate json_messagec Cstdg}|di}d|vr|d|_d|vr|d|_d|vr(|d|_d|vr8|d}|j|dd|_d |vrH|d }|j|d d|_d |vrQ|d |_d |vrZ|d |_d dd|j|j |jdiii}| t ||j d|ji|j|j|j|j|j|j|jdd}|d}|rddi|d<d||i|d<d d|ii} | t | |d} | rtt} d d|j | dd dd|jidii} | t | d d|j | | dii} | t | d d |j | d!ii}| t ||S)"z Transform session.update event to Bedrock session configuration. Args: json_message: OpenAI session.update message Returns: List of Bedrock format messages (JSON strings) zHandling session.updatesessionZmax_response_output_tokensr)ZvoiceZoutput_audio_formatT)rgZinput_audio_formatFr*r1rGrHrIrJrKrLrRrFrTrUrVrW instructions contentStartTEXTZSYSTEMrS contentNamer_ interactiveroletextInputConfiguration textInputrSrocontent contentEndrSro)rdebugrdr'r)r-rhr*r1r(rer[r\r%r7r0r+r,r.r/rZr"r#r$)r9rimessagesZsession_configZ output_formatZ input_formatrXr]rFrYrktext_content_nametext_content_start text_inputtext_content_endr:r:r;transform_session_update_events               z4BedrockRealtimeConfig.transform_session_update_eventcCstdg}t|ds3dd|j|jddd|j|j|j|j|j |j dd ii}| t |d|_|d d }dd |j|j|d ii}| t ||S)z Transform input_audio_buffer.append event to Bedrock audio input. Args: json_message: OpenAI input_audio_buffer.append message Returns: List of Bedrock format messages (JSON strings) z"Handling input_audio_buffer.append_audio_content_startedrGrlZAUDIOTUSER)rKrMrNrOrQrP)rSror_rprqZaudioInputConfigurationaudiorCZ audioInputrt)rrxhasattrr%r&r6r1r2r3r5r4rer[r\rrd)r9riryZaudio_content_startZ audio_dataZ audio_eventr:r:r;)transform_input_audio_buffer_append_event<sB    z?BedrockRealtimeConfig.transform_input_audio_buffer_append_eventcCsLtdg}t|dr$dd|j|jdii}|t|t|d|S)z Transform input_audio_buffer.commit event to Bedrock audio content end. Args: json_message: OpenAI input_audio_buffer.commit message Returns: List of Bedrock format messages (JSON strings) z"Handling input_audio_buffer.commitrrGrvrw) rrxrr%r&rer[r\delattr)r9riryZaudio_content_endr:r:r;)transform_input_audio_buffer_commit_eventps   z?BedrockRealtimeConfig.transform_input_audio_buffer_commit_eventc Cstdg}|di}|d}|dkr||S|dkry|dg}|D]Q}|ddkrxtt}dd |j|d d d d |jidii}| t |dd|j||dddii} | t | dd|j|dii} | t | q'|S)z Transform conversation.item.create event to Bedrock text input or tool result. Args: json_message: OpenAI conversation.item.create message Returns: List of Bedrock format messages (JSON strings) z!Handling conversation.item.createitemr_function_call_outputmessageruZ input_textrGrlrmTrrKrnrstextrCrtrvrw) rrxrd4transform_conversation_item_create_tool_result_eventr"r#r$r%r7rer[r\) r9riryrZ item_typeruZ content_partrzr{r|r}r:r:r;(transform_conversation_item_create_eventsV        z>BedrockRealtimeConfig.transform_conversation_item_create_eventcCtdgS)z Transform response.create event to Bedrock format. Args: json_message: OpenAI response.create message Returns: List of Bedrock format messages (JSON strings) zHandling response.createrrxr9rir:r:r;transform_response_create_event z5BedrockRealtimeConfig.transform_response_create_eventcCr)z Transform response.cancel event to Bedrock format. Args: json_message: OpenAI response.cancel message Returns: List of Bedrock format messages (JSON strings) zHandling response.cancelrrr:r:r;transform_response_cancel_eventrz5BedrockRealtimeConfig.transform_response_cancel_eventrr^c Cszt|}Wntjytd|ddgYSw|d}|dkr.||S|dkr7||S|dkr@||S|dkrI| |S|d krR| |S|d kr[| |Std |gS) aE Transform OpenAI realtime request to Bedrock Nova Sonic format. Args: message: OpenAI format message (JSON string) model: Model ID session_configuration_request: Previous session config Returns: List of Bedrock format messages (JSON strings) Invalid JSON message: Nr_zsession.updatezinput_audio_buffer.appendzinput_audio_buffer.commitzconversation.item.createzresponse.createzresponse.cancelzUnknown message type: ) r[loadsJSONDecodeErrorrwarningrdr~rrrrr)r9rr>r^riZ message_typer:r:r;transform_realtime_requests*       z0BedrockRealtimeConfig.transform_realtime_requestrG logging_objcCsLtdt|jddgd}|durt|tr||d<td|ttdS) a  Transform Bedrock sessionStart event to OpenAI session.created. Args: event: Bedrock sessionStart event model: Model ID logging_obj: Logging object Returns: OpenAI session.created event zHandling sessionStartrr)idZ modalitiesNr>zsession.created)r_rjevent_id) rrxrZlitellm_trace_id isinstancer"rrr$)r9rGr>rrjr:r:r;transform_session_start_events  z3BedrockRealtimeConfig.transform_session_start_eventcurrent_response_idcurrent_output_item_idcurrent_conversation_idc Cs.|d}|d}|dkrg|||dfStd|s"dt}|s+dt}|s4dt}|d d }|d kr@d nd }g} td dtd|dg|dd} | | td|d|ddddgdd} | | tddddt||d krd ddnd dd|d} | | | ||||fS)a Transform Bedrock contentStart event to OpenAI response events. Args: event: Bedrock contentStart event current_response_id: Current response ID current_output_item_id: Current output item ID current_conversation_id: Current conversation ID Returns: Tuple of (events, response_id, output_item_id, conversation_id, delta_type) rlrqZ ASSISTANTNzHandling ASSISTANT contentStartZresp_Zitem_Zconv_r_rmrrzresponse.createdevent_realtime.responseZ in_progress)objectrstatusoutputconversation_idr_rresponsezresponse.output_item.addedr realtime.itemr assistantrrr_rrqru)r_ response_id output_indexrzresponse.content_part.addedrCr_rr_Z transcript)r_ content_indexrritem_idpartr) rdrrxrr$rrerr) r9rGrrrZ content_startrq content_typecurrent_delta_typereturned_messagesZresponse_createdZoutput_item_addedZcontent_part_addedr:r:r;transform_content_start_event6sp         z3BedrockRealtimeConfig.transform_content_start_eventcurrent_delta_chunksc Csjtd|ddd}|r|sg|fStdddt|d||d}|d ur+g}|||g|fS) a{ Transform Bedrock textOutput event to OpenAI response.text.delta. Args: event: Bedrock textOutput event current_output_item_id: Current output item ID current_response_id: Current response ID current_delta_chunks: Current delta chunks Returns: Tuple of (events, updated_delta_chunks) zHandling textOutput textOutputrurCzresponse.text.deltarrr_rrrrrdeltaN)rrxrdrrr$re)r9rGrrrZ text_contentZ text_deltar:r:r;transform_text_output_events"     z1BedrockRealtimeConfig.transform_text_output_eventc CsLtd|ddd}|r|sgStdddt|d||d}|gS) a5 Transform Bedrock audioOutput event to OpenAI response.audio.delta. Args: event: Bedrock audioOutput event current_output_item_id: Current output item ID current_response_id: Current response ID Returns: List of OpenAI events zHandling audioOutput audioOutputrurCzresponse.audio.deltarrr)rrxrdrrr$)r9rGrrZ audio_contentZ audio_deltar:r:r;transform_audio_output_events   z2BedrockRealtimeConfig.transform_audio_output_eventrc CsH|d}td||r|sg|fSg}|dkrVd}|r(ddd|D}tddd t|d||d } || td dd t|dd|d |d } || n0|dkrtddd t|d|d} || td dd t|dddd|d } || t dd td||ddddgdd} || |dfS)a Transform Bedrock contentEnd event to OpenAI response done events. Args: event: Bedrock contentEnd event current_output_item_id: Current output item ID current_response_id: Current response ID current_delta_type: Current delta type (text or audio) current_delta_chunks: Current delta chunks Returns: Tuple of (events, reset_delta_chunks) rvzHandling contentEnd: rrCcSsg|]}|ddqS)rrC)rd).0chunkr:r:r; szEBedrockRealtimeConfig.transform_content_end_event..zresponse.text.donerr)r_rrrrrrzresponse.content_part.doner)r_rrrrrrrzresponse.audio.done)r_rrrrrrzresponse.output_item.donerr completedrr)r_rrrrN) rrxjoinrrr$rer rr ) r9rGrrrrZ content_endrZaccumulated_textZ text_doneZcontent_part_doneZ audio_doneZoutput_item_doner:r:r;transform_content_end_events           z1BedrockRealtimeConfig.transform_content_end_eventc Cshtd|r |sgdddfSt}tddttd|dg||j|j|j ddd }|gdddfS) ac Transform Bedrock promptEnd event to OpenAI response.done. Args: event: Bedrock promptEnd event current_response_id: Current response ID current_conversation_id: Current conversation ID Returns: Tuple of (events, reset_output_item_id, reset_response_id, reset_delta_type) zHandling promptEndNz response.donerrr) prompt_tokenscompletion_tokens total_tokens)rrrrrusager) rrxrr rr$rrrr)r9rGrrZ usage_objZ response_doner:r:r;transform_prompt_end_eventPs(   z0BedrockRealtimeConfig.transform_prompt_end_eventc Cstd|d}|r |sgddfSi}d|vr9zt|dtr't|dn|d}Wn tjy8i}Ynw|dd}|dd}ddlm }d d t ||d||t |d } |t | g||fS) aA Transform Bedrock toolUse event to OpenAI format. Args: event: Bedrock toolUse event current_output_item_id: Current output item ID current_response_id: Current response ID Returns: Tuple of (events, tool_call_id, tool_name) for tracking zHandling toolUsetoolUserCinput toolUseIdZtoolNamer)castz%response.function_call_arguments.doner)r_rrrrcall_idra arguments)rrxrr"r[rrrdtypingrrr$r\r ) r9rGrrZtool_useZ tool_input tool_call_id tool_namerZfunction_call_eventr:r:r;transform_tool_use_event{s0  (     z.BedrockRealtimeConfig.transform_tool_use_eventc Cstdg}|di}|ddkrqtt}|dd}|dd}dd |j|d d d |d d diddii}|t |dd|j|t |trN|nt |dii}|t |dd|j|dii} |t | |S)a Transform conversation.item.create with tool result to Bedrock format. Args: json_message: OpenAI conversation.item.create message with tool result Returns: List of Bedrock format messages (JSON strings) z1Handling conversation.item.create for tool resultrr_rrrCrrGrlFZTOOLrmrKr!)rr_rr)rSrorpr_rqZtoolResultInputConfigurationZ toolResultrtrvrw) rrxrdr"r#r$r%rer[r\r) r9riryrZtool_content_namerrZtool_content_startZ tool_resultZtool_content_endr:r:r;rsR      zJBedrockRealtimeConfig.transform_conversation_item_create_tool_result_event!realtime_response_transform_inputc CsDzt|}WnHtjyOt|tr|ddjdddn|dd}td|g|d|d|d |d |d |d |d dYSw|d}|d}|d } |d } |d } |d } g} |di}d|vr| |||}| |t ddi} nd|vr| |||| \}}}} } | |nod|vr||||| \}} | |n[d|vr||||}| |nJd|vr||||| | \}} | |n5d|vr||||\}}}| |td|d|dnd|vr|||| \}}}} | || ||| | |d | | dS)ab Transform Bedrock Nova Sonic response to OpenAI realtime format. Args: message: Bedrock format message (JSON string) model: Model ID logging_obj: Logging object realtime_response_transform_input: Current state Returns: Transformed response with updated state Nrzutf-8replace)errorsrrrrrcurrent_item_chunksrr^)rrrrrrrr^rGrHZ configuredTrlrrrvrzTool use event: z (ID: )Z promptEnd)r[rrrbytesdecoderrrdrrer\rextendrrrrrxr)r9rr>rrriZmessage_previewrrrrrr^rrGZsession_createdeventsrrr:r:r;transform_realtime_responses,           z1BedrockRealtimeConfig.transform_realtime_response)N)T)+__name__ __module__ __qualname____doc__r<dictr"rrArDboolrErr^rZintrhr~rrrrrrLiteLLMLoggingObjrrtupler rrrrrrrrrrrrrrr:r:r:r;r&s !   44E *   e  * #  h + 2A r)&rr[rr#rrrrrZlitellm._loggingrZ litellm._uuidZ*litellm.litellm_core_utils.litellm_loggingrrZ-litellm.llms.base_llm.realtime.transformationr Zlitellm.types.llms.openair r r r rrrrrrrrrZlitellm.types.realtimerrrZ litellm.utilsrrr:r:r:r;s    <