o ưiL @sdZddlZddlmZmZmZmZddlZddlm Z m Z m Z m Z m Z mZddlmZddlZddlmZddlTddlmZmZdd lmZmZmZe Z  d"d ed eeeeefd eed eeeffddZ  d"d edeeefded eeeeefd eed df ddZ de d eeeefeeeeefeeeeffddZ!ej"de egedgdej"de egedgde efde de defddZ#ej"de egedgdej"de egedgde efde de defd d!Z$dS)#z RAG Endpoints for LiteLLM Proxy. Provides: - /rag/ingest: All-in-one document ingestion pipeline (Upload -> Chunk -> Embed -> Vector Store) - /rag/query: RAG query pipeline (Search -> Rerank -> LLM Completion) N)AnyDictOptionalTuple) APIRouterDepends HTTPExceptionRequestResponsestatus)ORJSONResponse)verbose_proxy_logger)*)UserAPIKeyAuthuser_api_key_auth)_read_request_body_safe_get_request_headers get_form_dataresponse file_datafile_urlreturnc Csddlm}m}d}t|dr|d}nt|dr|j}d}d}d}|rC|d}t|dkr5t|dnd}t|dkrA|dnd}|||||jd} |durX|| d <|dur`|| d <| S) a Build a file metadata entry for storing in vector_store_metadata. Args: response: The response from litellm.aingest containing file_id file_data: Optional tuple of (filename, content, content_type) file_url: Optional URL if file was ingested from URL Returns: Dictionary with file metadata (file_id, filename, file_url, ingested_at, etc.) r)datetimetimezoneNgetfile_id)rfilenamerZ ingested_at file_size content_type) rrhasattrrrlennowutc isoformat) rrrrrrrrr file_entryr'\/home/app/Keep/.python/lib/python3.10/site-packages/litellm/proxy/rag_endpoints/endpoints.py_build_file_metadata_entrys.   r)ingest_optionsuser_api_key_dictc sddlm}t|dr|d}nt|dr|j}n tdt|dS|dus0t|t s7tddS|di}|d } |d i} | d } | d } i} d dh}| D]\}}||vrl|durl|| |<q\t |||d }z|j j jd|idIdH}|durtd|dd|gi}| pd|dd}| pd}||| pd||||| r| nd|j|jd IdHtd|dWdStd|d|jpi}t|t rddl}||}|dg}||||d<ddlm}|j j jd|id||idIdHtd|dp|dd d!|d"WdStyA}ztd#|d$|WYd}~dSd}~ww)%a Helper function to save a newly created vector store from RAG ingest to the database. This function: - Extracts vector store ID and config from the ingest response - Checks if the vector store already exists in the database - Creates a new database entry if it doesn't exist - Adds the vector store to the registry - Tracks team_id and user_id for access control Args: response: The response from litellm.aingest() ingest_options: The ingest options containing vector store config prisma_client: The Prisma database client user_api_key_dict: User API key authentication info r)create_vector_store_in_dbrvector_store_idz6Unable to extract vector_store_id from response type: Nz?Vector store ID is None or not a string, skipping database save vector_storecustom_llm_providerlitellm_vector_store_paramsvector_store_namevector_store_description)rrr)wherez"Saving newly created vector store z to databaseingested_fileszRAG Vector Store - zCreated via RAG ingest endpointZopenai) r-r/ prisma_clientr1r2vector_store_metadataZlitellm_paramsteam_iduser_idz Vector store z saved to database successfullyz+ already exists, appending file to metadata) safe_dumpsr7)r3dataz Added file rrUnknownz to vector store z metadatazFailed to save vector store z to database: )Z9litellm.proxy.vector_store_endpoints.management_endpointsr,r!rr-r warningtype isinstancestritemsr)dbZ litellm_managedvectorstorestableZ find_uniqueinfor8r9r7jsonloadsappendZlitellm.proxy.utilsr:update Exception exception)rr*r6r+rrr,r-Zvector_store_configr/r0Zcustom_vector_store_nameZcustom_vector_store_descriptionZprovider_specific_paramsZ excluded_keyskeyvaluer&Zexisting_vector_storeZinitial_metadatar1r2Zexisting_metadatarDr4r:Zdb_errorr'r'r((_save_vector_store_to_db_from_rag_ingestQs                         $ rLrequestc st|}|dd}d}d}d}i}d|vrWt|IdH}|d}|dur:t|dr:|IdH} |j| |jf}|d} | rVt| } | di}| d }| d }n[t |IdH} | di}| d }| d }| d}|rt |t r|d } |d }|d d}| r|rz t |} | | |f}Wnty}z tddd|idd}~ww|dur|dur|durtdddidd|vrtdddid||||fS)z Parse RAG ingest request. Supports: - Form: file + request JSON in form field - JSON body for URL-based ingestion Returns: Tuple of (ingest_options, file_data, file_url, file_id) z content-typeNzmultipart/form-datafilereadrMr*rrrcontentr zapplication/octet-streamerrorzInvalid base64 content:  status_codedetailz'Must provide file, file_url, or file_idr.z8ingest_options must contain 'vector_store' configuration)rrrr!rPrr orjsonrErr?dictbase64 b64decoderHr)rMheadersr rrrr*Z form_dataZfile_objZ file_contentZrequest_json_str request_datar;rZ content_b64er'r'r(parse_rag_ingest_requestsf                 r^z/v1/rag/ingestZrag) dependenciesresponse_classtagsz /rag/ingestfastapi_responsec sddlm}m}m}m}m}m}zt|IdH\} } } } |jt j j kr6| di ds6t tjddidi} || |||||d IdH} td | tjd| | | | |d | IdH}td |dud |dudt||dur|durt|| ||| | dIdH|WStd|dud |du|WSt yty}ztd|t ddt|idd}~ww)a RAG Ingest endpoint - all-in-one document ingestion pipeline. Supports form upload (for files) or JSON body (for URLs). ## Form upload (for files): ```bash curl -X POST "http://localhost:4000/v1/rag/ingest" \ -H "Authorization: Bearer sk-1234" \ -F file="@document.pdf" \ -F 'ingest_options={"vector_store": {"custom_llm_provider": "openai"}}' ``` ## JSON body (for URLs): ```bash curl -X POST "http://localhost:4000/v1/rag/ingest" \ -H "Authorization: Bearer sk-1234" \ -H "Content-Type: application/json" \ -d '{ "file_url": "https://example.com/document.pdf", "ingest_options": {"vector_store": {"custom_llm_provider": "openai"}} }' ``` ## Bedrock: ```bash curl -X POST "http://localhost:4000/v1/rag/ingest" \ -H "Authorization: Bearer sk-1234" \ -F file="@document.pdf" \ -F 'ingest_options={"vector_store": {"custom_llm_provider": "bedrock"}}' ``` r)add_litellm_data_to_requestgeneral_settings llm_routerr6 proxy_configversionNr.r-rSzinternal_user_viewer role can only ingest files to an existing vector store. Provide 'vector_store_id' in ingest_options.vector_store.rTr;rMrdr+rgrfzRAG Ingest - options: )r*rrrrouterz>RAG Ingest - Checking database save conditions: prisma_client=z , response=z, response_type=)rr*r6r+rrz&Skipping database save: prisma_client=zRAG Ingest failed: r')litellm.proxy.proxy_serverrcrdrer6rfrgr^Z user_roleZLitellmUserRolesZINTERNAL_USER_VIEW_ONLYrKrrr ZHTTP_403_FORBIDDENr debuglitellmZaingestr>rLr=rHrIr@)rMrbr+rcrdrer6rfrgr*rrrr\rr]r'r'r( rag_ingest*st 1          rnz /v1/rag/queryz /rag/queryc s^ddlm}m}m}m}m}zzt|IdH}|d} |d} |d} |d} |dd } | s;td d d id | sEtd d did | sOtd d did d| vr[td d did i}|||||||dIdH}t d| d| t j d| | | | | |d|IdH}|WStyt y}zt d|tdd t|id d}~ww)a RAG Query endpoint - search vector store, optionally rerank, and generate LLM response. This endpoint: 1. Extracts the query from the last user message 2. Searches the vector store for relevant context 3. Optionally reranks the results 4. Generates an LLM response with the retrieved context ## Example Request: ```bash curl -X POST "http://localhost:4000/v1/rag/query" \ -H "Authorization: Bearer sk-1234" \ -H "Content-Type: application/json" \ -d '{ "model": "gpt-4o-mini", "messages": [{"role": "user", "content": "What is LiteLLM?"}], "retrieval_config": { "vector_store_id": "vs_abc123", "custom_llm_provider": "openai", "top_k": 5 } }' ``` ## With Reranking: ```bash curl -X POST "http://localhost:4000/v1/rag/query" \ -H "Authorization: Bearer sk-1234" \ -H "Content-Type: application/json" \ -d '{ "model": "gpt-4o-mini", "messages": [{"role": "user", "content": "What is LiteLLM?"}], "retrieval_config": { "vector_store_id": "vs_abc123", "custom_llm_provider": "openai", "top_k": 10 }, "rerank": { "enabled": true, "model": "cohere/rerank-english-v3.0", "top_n": 3 } }' ``` r)rcrdrerfrgNmodelmessagesretrieval_configrerankstreamFrRrSzmodel is requiredrTzmessages is requiredzretrieval_config is requiredr-z/retrieval_config must contain 'vector_store_id'rhzRAG Query - model: z, retrieval_config: )rorprqrrrsrizRAG Query failed: rjr')rkrcrdrerfrgrrrr rlrmZaqueryrHrIr@)rMrbr+rcrdrerfrgr;rorprqrrrsr\rr]r'r'r( rag_querys|?          rt)NN)%__doc__rYtypingrrrrrWZfastapirrrr r r Zfastapi.responsesr rmZlitellm._loggingr Zlitellm.proxy._typesZ$litellm.proxy.auth.user_api_key_authrrZ-litellm.proxy.common_utils.http_parsing_utilsrrrrir@bytesr)rLr^postrnrtr'r'r'r(s     9   , O u