o |q i@sfddlZddlZddlmZmZmZmZmZmZddl m Z ddl m Z e eZGddde ZdS)N) AsyncIteratorDictIteratorListOptionalSequence)Document) BaseLoaderc@seZdZdZddddddedededeed eeed eeed ed dfd dZ d e e fddZ d e e fddZd ee fddZd e e fddZded e fddZd eefddZ d"dedeeded efd d!ZdS)# MongodbLoaderzLoad MongoDB documents.NT)filter_criteria field_namesmetadata_names!include_db_collection_in_metadataconnection_stringdb_namecollection_namer r r rreturnc Cszddlm}Wnty} ztd| d} ~ ww|s td|s&td|s,td|||_||_||_|p:g|_|p?i|_|pDg|_ ||_ |j ||_ |j ||_dS)a Initializes the MongoDB loader with necessary database connection details and configurations. Args: connection_string (str): MongoDB connection URI. db_name (str):Name of the database to connect to. collection_name (str): Name of the collection to fetch documents from. filter_criteria (Optional[Dict]): MongoDB filter criteria for querying documents. field_names (Optional[Sequence[str]]): List of field names to retrieve from documents. metadata_names (Optional[Sequence[str]]): Additional metadata fields to extract from documents. include_db_collection_in_metadata (bool): Flag to include database and collection names in metadata. Raises: ImportError: If the motor library is not installed. ValueError: If any necessary argument is missing. r)AsyncIOMotorClientzBCannot import from motor, please install with `pip install motor`.Nz#connection_string must be provided.zdb_name must be provided.z!collection_name must be provided.)Zmotor.motor_asyncior ImportError ValueErrorclientrrr r r rZ get_databasedbZget_collection collection) selfrrrr r r rrers/home/app/PaddleOCR-VL/.venv_paddleocr/lib/python3.10/site-packages/langchain_community/document_loaders/mongodb.py__init__s2     zMongodbLoader.__init__cCst|S)aLoad data into Document objects. Attention: This implementation starts an asyncio event loop which will only work if running in a sync env. In an async env, it should fail since there is already an event loop running. This code should be updated to kick off the event loop from a separate thread if running within an async context. )asynciorunaload)rrrrloadJs zMongodbLoader.loadccsnzt}Wntyt}t|Ynw|} z ||}|VWn ty5YdSwq)aA lazy loader for MongoDB documents. Attention: This implementation starts an asyncio event loop which will only work if running in a sync env. In an async env, it should fail since there is already an event loop running. This code should be updated to kick off the event loop from a separate thread if running within an async context. Yields: Document: A document from the MongoDB collection. TN) rget_running_loop RuntimeErrornew_event_loopset_event_loop alazy_loadrun_until_complete __anext__StopAsyncIteration)rZ event_loopasync_generatordocumentrrr lazy_loadXs     zMongodbLoader.lazy_loadcCs:|}|j|j|2z 3dHW}||Vq 6dS)zAsynchronously yields Document objects one at a time. Yields: Document: A document from the MongoDB collection. N)_construct_projectionrfindr _process_document)r projectiondocrrrr&vs zMongodbLoader.alazy_loadcs~g}|j|jIdH}|}|j|j|2z3dHW}|||q6t||kr=t dt|d|d|S)z0Asynchronously loads data into Document objects.Nz6Only partial collection of documents returned. Loaded z docs, expected .) rZcount_documentsr r-r.appendr/lenloggerwarning)rresultZ total_docsr0r1rrrr s  zMongodbLoader.aloadr1cCsz|j||jdd}|jr||j|jd|jdur3|j||jdd}dd|D}d|}nt |}t ||dS) zProcess a single MongoDB document into a Document object. Args: doc: The MongoDB document dictionary to process into a Document object. )default)ZdatabaserNcSsg|]}t|qSr)str).0valuerrr sz3MongodbLoader._process_document.. )Z page_contentmetadata) _extract_fieldsr rupdaterrr valuesjoinr:r)rr1r?fieldsZtextstextrrrr/s   zMongodbLoader._process_documentcCs:t|jpg}t|jp g}||}|rdd|DSdS)zuConstructs the projection dictionary for MongoDB query based on the specified field names and metadata names.cSsi|]}|dqS)r)r;fieldrrr sz7MongodbLoader._construct_projection..N)listr r )rr r Z all_fieldsrrrr-sz#MongodbLoader._construct_projectionr8r+rDr9c CsVi}|pgD]"}|}|dD]}|||}||krnq|dd}|||<q|S)zAExtracts and returns values for specified fields from a document.r2_)splitgetreplace) rr+rDr9Z extractedrGr<keyZnew_field_namerrrr@s    zMongodbLoader._extract_fields)r8)__name__ __module__ __qualname____doc__r:rrrboolrrrr!rr,rr&r r/r-r@rrrrr sN    ;  r )rloggingtypingrrrrrrZlangchain_core.documentsrZ)langchain_community.document_loaders.baser getLoggerrOr5r rrrrs