o rq i@F@s|ddlZddlZddlmZmZddlmZddlZddlm Z ddl m Z m Z m Z ddlmZeZdZGdd d ZdS) N)ListOptional)urlparse)tqdm)MODELSCOPE_URL_SCHEMEencode_image_to_base64 get_endpoint) get_loggerzihttps://modelscope.cn/models/modelscope/modelscope_aigc_default_logo/resolve/master/aigc_default_logo.pngc@seZdZdZhdZhdZhdZ       d/d ed ed ed ed eedeedee edeedee edee efddZ ddZ ddZ ddZ ddZd0dedeefdd Zdd!ddd"d#eed$ed%eed&eed'df d(d)Zd'efd*d+Zed,efd-d.ZdS)1 AigcModela[ Helper class to encapsulate AIGC-specific model creation parameters. This class can be initialized directly with parameters, or loaded from a JSON configuration file using the `from_json_file` classmethod. Example of direct initialization: >>> aigc_model = AigcModel( ... aigc_type='Checkpoint', ... base_model_type='SD_XL', ... model_path='/path/to/your/model.safetensors' ... base_model_id='AI-ModelScope/FLUX.1-dev' ... ) Example of loading from a JSON file: `config.json`: { "model_path": "/path/to/your/model.safetensors", "aigc_type": "Checkpoint", "base_model_type": "SD_XL", "base_model_id": "AI-ModelScope/FLUX.1-dev" } >>> aigc_model = AigcModel.from_json_file('config.json') >ZLoRAZ CheckpointZVAE>ZSD_2_1ZSD_XLZWAN_VIDEO_2_2_TI2V_5_BZSD_1_5UNKNOWNZWAN_VIDEO_2_1_I2V_14_BZWAN_VIDEO_2_1_14_BZWAN_VIDEO_2_2_T2V_A_14_BZSD_2ZWAN_VIDEO_2_2_I2V_A_14_BZFLUX_1ZQWEN_IMAGE_20_BZWAN_VIDEO_2_1_T2V_14_BZWAN_VIDEO_2_1_T2V_1_3_BZWAN_VIDEO_2_1_FLF2V_14_BZSD_3>zcharacter-strongzarchitectural-designZ commodityzclassic-painting-styleZfoodz other-styleszautomobile-trafficZclothingzflat-abstractionz sci-fi-mechaZsceneryz main-strongZplantze-commerce-designz toy-figurinesz cg-fantasywomanmanzillustration-designzart-style-strongZ3dzother-functionszpicture-control construction dimensionZ photographyZanimalzhand-drawn-styleZartwarezcharacter-enhancementZlogov1.0this is an aigc modelN aigc_typebase_model_type model_path base_model_idtag description cover_images path_in_repo trigger_words official_tagsc  Cs6||_||_||_||_||_|dursg} |D]W} t| tri| dsc| dsc| dsczt| } | | t dt j | Wqttfyb}zt d| || | WYd}~qd}~ww| | q| | q| |_ntg|_||_||_| |_||| r| |_|nd|_|dS)a Initializes the AigcModel helper. Args: model_path (str): The path of checkpoint/LoRA weight file or folder. aigc_type (str): AIGC model type. Recommended: 'Checkpoint', 'LoRA', 'VAE'. base_model_type (str): Vision foundation model type. Recommended values are in BASE_MODEL_TYPES. tag (str, optional): Tag for the AIGC model. Defaults to 'v1.0'. description (str, optional): Model description. Defaults to 'this is an aigc model'. cover_images (List[str], optional): List of cover image URLs. base_model_id (str, optional): Base model name. e.g., 'AI-ModelScope/FLUX.1-dev'. path_in_repo (str, optional): Path in the repository. trigger_words (List[str], optional): Trigger words for the AIGC Lora model. official_tags (List[str], optional): Official tags for the AIGC model. Defaults to None. Nzhttp://zhttps://zdata:z#Converted local image to base64: %sz2Failed to process local image %s: %s. Using as-is.)rrrrr isinstancestr startswithrappendloggerinfoospathbasenameFileNotFoundError ValueErrorwarningrDEFAULT_AIGC_COVER_IMAGErrr_validate_aigc_type_validate_base_model_typer_validate_official_tags_process_model_path)selfrrrrrrrrrrZprocessed_cover_imagesimgZ processed_imger1`/home/app/PaddleOCR-VL/.venv_paddleocr/lib/python3.10/site-packages/modelscope/hub/utils/aigc.py__init__HsT        zAigcModel.__init__cC>|j|jvrdt|j}td|jd|ddSdS)z?Validate aigc_type and provide a warning for unsupported types., zUnsupported aigc_type: "z". Recommended values: z1. Custom values are allowed but may cause issues.N)r AIGC_TYPESjoinsortedr!r(r.Zsupported_typesr1r1r2r*s zAigcModel._validate_aigc_typecCr4)zCValidate base_model_type and provide warning for unsupported types.r5zYour base_model_type: "z," may not be supported. Recommended values: z2. Custom values are allowed but may cause issues. N)rBASE_MODEL_TYPESr7r8r!r(r9r1r1r2r+s  z#AigcModel._validate_base_model_typecsXfddjD}|r*dj}ddd|D}td|d|dd Sd S) z@Validate official tags and provide warning for unsupported tags.csh|] }|jvr|qSr1) OFFICIAL_TAGS.0rr.r1r2 s z4AigcModel._validate_official_tags..r5css|] }d|dVqdS)"Nr1r<r1r1r2 sz4AigcModel._validate_official_tags..z Your tag(s): z+ may not be supported. Recommended values: z. N)rr7r;r!r()r.Z invalid_tagssupported_tagsZinvalid_tags_strr1r>r2r,s  z!AigcModel._validate_official_tagsc sLddlm}tjj_tjjstdjtjjr2j}t dtj |ntj jrt j}t|dkrNtdjdfdd|D}hd }|rmd d |D}||rmtd gd }d}|D]M}ttjjd|} | r| d}t d|dtj |t| dkrt d|dtj |t d|ddd| ddDnqu|sfddt jD} | rtjj| d}t dtj |t d| ntdjdntdj|r$t d|||} tj |_| d_| d _|_dSdS)!z0Process model_path to extract weight informationr) get_file_hashzModel path does not exist: zUsing file: %szDirectory is empty: zW. Please place at least one model file at the top level (e.g., .safetensors/.pth/.bin).c(g|]}tjtjj|r|qSr1r#r$isfiler7rr=namer>r1r2 z1AigcModel._process_model_path..>z readme.mdzconfiguration.jsonz.gitattributescSsh|]}|qSr1)lowerrGr1r1r2r?sz0AigcModel._process_model_path..zTop-level directory contains only [.gitattributes, configuration.json, README.md]. Please place additional model files at the top level (e.g., .safetensors/.pth/.bin).)z .safetensorsz.pthz.binN*zFound z file: %sz Multiple z$ files found, using: %s for metadatazOther z files: %scSsg|]}tj|qSr1)r#r$r%r=fr1r1r2rIscrDr1rErNr>r1r2rIrJz:No safetensors/pth/bin files found, using: %s for metadatazAvailable files: %szNo files found in directory: zI. AIGC models require at least one model file (.safetensors recommended).z(Model path must be a file or directory: z!Computing hash and size for %s...Z file_hash file_size)Zmodelscope.utils.file_utilsrCr#r$ expanduserrexistsr'rFr!r"r%isdirlistdirlenissubsetglobr7r(weight_filename weight_sha256 weight_size target_file) r.rCr[Z top_entriesZ top_filesZplaceholder_names normalizedZfile_extensionsextfiles all_files hash_infor1r>r2r-s                   zAigcModel._process_model_pathmodel_idtokenc Cstd|zNtj|jr"td|j|j||j|j|ddn*tj|jrLtd|j |j |j |jr?|jd|j n|j ||d|j dd td |Wd St yr}zt d |t d WYd}~dSd}~ww)z!Upload model files to repository.zUploading model to %s...zUploading directory: %sz"Upload model folder for AIGC model)repo_idZ folder_pathrrbcommit_messagezUploading file: %s/zUpload z for AIGC model)Zpath_or_fileobjrrcrbrdz!Successfully uploaded model to %sTz#Warning: Failed to upload model: %sz9You may need to upload the model manually after creation.NF)r!r"r#r$rSrZ upload_folderrrFr[Z upload_filerX Exceptionr()r.apirarbr0r1r1r2upload_to_repo s@    zAigcModel.upload_to_repoi,)cookiestimeoutheadersendpointrirjrkrlreturnc Cs|pt}t|jd}|dstd|n td|d}|d}t|ddp2|j}t j t j |}t j |sJtd ||rPt|nd}|durZtd |d d |d it j |} ddtfdd} t| ddddd)} t|d} tj||| | | |d} Wdn1swYWdn1swYz| }Wntjjy| YdSwt|tr|ddur|dd}td|dSdS)aPre-upload aigc model weights to the LFS server. Server may require the sha256 of weights to be registered before creation. This method streams the weight file so the sha gets registered. Args: cookies: Optional requests-style cookies (CookieJar/dict). If provided, preferred. timeout: Request timeout seconds. headers: Optional headers. zwww.prezlfs.zpre-lfs.zpre.z/api/v1/models/aigc/weightsr[Nz Pre-upload expects a file, got: z)Token does not exist, please login first.Cookiez m_session_id=Z m_session_id chunk_sizecss, ||}|s dS|t||VqN)readupdaterU)Z file_objectpbarrqZckr1r1r2read_in_chunksSs z3AigcModel.preupload_weights..read_in_chunksBTz[Pre-uploading] )totalunitZ unit_scaleZ dynamic_ncolsZdescrb)rkdatarjZSuccessFMessagez unknown errorzPre-upload failed: )rp)rrstriphostnamelstriprrgetattrrr#r$abspathrQrFr'dictrtgetsizeintropenrequestsputjson exceptionsJSONDecodeErrorraise_for_statusrget RuntimeError)r.rirjrkrlZ endpoint_hostbase_urlurl file_pathrPrvrurOrrespmsgr1r1r2preupload_weights-s^         zAigcModel.preupload_weightsc Cs6|j|j|j|j|j|j|j|j|j|j |j |j d S)zDConverts the AIGC parameters to a dictionary suitable for API calls. rrrrrrrrXrYrZrrrr>r1r1r2to_dicttszAigcModel.to_dict json_pathcCsddl}tj|}tj|std|t|ddd }||}Wdn1s.wYgd}|D]}||vrGtd|d q9|d i|S) z Creates an AigcModel instance from a JSON configuration file. Args: json_path (str): The path to the JSON configuration file. Returns: AigcModel: An instance of the AigcModel. rNzJSON config file not found at: rzutf-8)encoding)rrrrz(Missing required field in JSON config: ''r1) rr#r$rQrRr&rloadr')clsrrrOconfigZrequired_fieldsfieldr1r1r2from_json_files"    zAigcModel.from_json_file)rrrNrNNrr)__name__ __module__ __qualname____doc__r6r:r;rrrr3r*r+r,r-rhobjectrrrr classmethodrr1r1r1r2r sp    L  Z# Gr )rWr#typingrr urllib.parserrZ tqdm.autorZmodelscope.hub.utils.utilsrrrZmodelscope.utils.loggerr r!r)r r1r1r1r2s