o rq i2@sddlZddlZddlmZddlmZmZmZddlm Z m Z m Z ddl m Z ddlmZddlmZe ZGd d d ZGd d d ZdS) N)Optional)GitErrorInvalidParameterNotLoginException)DEFAULT_DATASET_REVISIONDEFAULT_REPOSITORY_REVISIONMASTER_MODEL_BRANCH) get_logger)GitCommandWrapper get_endpointc @seZdZdZeddfdededeedeedeef dd Zd d Zd d Z d%dedefddZ defddZ eedfdedeedeedee fddZ efdeded eefd!d"Zefdeded eefd#d$ZdS)& Repositoryz8A local representation of the model git repository. N model_dir clone_fromrevision auth_tokengit_pathc Cs ||_tj||_tj||_|sd}t|ddlm }|r%||_ n| |_ t }| s6tdt ||_tj|jdd||} t|jr`|} |j| } | r`| | kr`dS|j|j|j | |j|| rw||j|j|j|j|j r|j|j|j dSdS)a"Instantiate a Repository object by cloning the remote ModelScopeHub repo Args: model_dir (str): The model root directory. clone_from (str): model id in ModelScope-hub from which git clone revision (str, optional): revision of the model you want to clone from. Can be any of a branch, tag or commit hash auth_token (str, optional): token obtained when calling `HubApi.login()`. Usually you can safely ignore the parameter as the token is already saved when you login the first time, if None, we will use saved token. git_path (str, optional): The git command line path, if None, we use 'git' Raises: InvalidParameter: revision is None. 0a non-default value of revision cannot be empty.rModelScopeConfigz)git lfs is not installed, please install.Texist_okN)rospathdirnamemodel_base_dirbasenamemodel_repo_namermodelscope.hub.apirr get_tokenr Zis_lfs_installedloggererror git_wrappermakedirs_get_model_id_urllistdir_get_remote_urlremove_token_from_urlcloneZgit_lfs_install add_user_infoconfig_auth_token) selfrrrrrerr_msgrr#url remote_urlr0`/home/app/PaddleOCR-VL/.venv_paddleocr/lib/python3.10/site-packages/modelscope/hub/repository.py__init__s@          zRepository.__init__cCstd|d}|S)N/.gitr )r,Zmodel_idr.r0r0r1r%QszRepository._get_model_id_urlcC.z |j|j}W|Styd}Y|SwN)r#get_repo_remote_urlrrr,remoter0r0r1r'U zRepository._get_remote_urloriginmasterr9branchcCs|jj|j||ddS)zPull remote branch Args: remote (str, optional): The remote name. Defaults to 'origin'. branch (str, optional): The remote branch. Defaults to 'master'. )r9r=N)r#pullr)r,r9r=r0r0r1r>\szRepository.pullfile_name_suffixcCs"td|tj|jdfdS)zAdd file suffix to lfs list. Args: file_name_suffix (str): The file name suffix. examples '*.safetensors' z5printf ' %s filter=lfs diff=lfs merge=lfs -text '>>%sz.gitattributesN)rsystemrjoinr)r,r?r0r0r1 add_lfs_typees zRepository.add_lfs_typeFcommit_message local_branch remote_branchforcecCstjdtdd |dust|tsd}t|t|ts!td|js(td|j |j |j|j |j |j|j |j }|j j|j dd|j |j ||j j|j |j|||d dS NzThis function is deprecated and will be removed in future versions. Please use git command directly or use HubApi().upload_folder instead) stacklevelz commit_message must be provided!zforce must be boolz'Must login to push, please login first.T) all_files)repo_dirtokenr.rDrE)warningswarnDeprecationWarning isinstancestrrboolrrr#r+rr*rrr7addcommitpush)r,rCrDrErFmsgr.r0r0r1rUps6    zRepository.pushtag_namemessagerefcCsR|dus|dkrd}t||dus|dkrd}t||jj|j|||ddS)aCreate a new tag. Args: tag_name (str): The name of the tag message (str): The tag message. ref (str, optional): The tag reference, can be commit id or branch. Raises: InvalidParameter: no commit message. NzFWe use tag-based revision, therefore tag_name cannot be None or empty.z=We use annotated tag, therefore message cannot None or empty.)rKrWrXrY)rr#tagr)r,rWrXrYrVr0r0r1r[s zRepository.tagcCs$|||||jj|j|ddS)zCreate tag and push to remote Args: tag_name (str): The name of the tag message (str): The tag message. ref (str, optional): The tag ref, can be commit id or branch. Defaults to MASTER_MODEL_BRANCH. )rKrWN)r[r#Zpush_tagr)r,rWrXrYr0r0r1 tag_and_pushs zRepository.tag_and_push)r;r<)__name__ __module__ __qualname____doc__rrQrr2r%r'r>rBrRrUrr[r\r0r0r0r1rs` ;   2 rc @seZdZdZeddfdededeedeedeef dd Zd efd d Zed fdedeedee fddZ ddZ ddZ dS)DatasetRepositoryzEA local representation of the dataset (metadata) git repository. N repo_work_dir dataset_idrrrcCs||_|r t|tsd}t||d|_|jsd}t|tj|j|_ tj |j|_ |s7d}t|||_ ddl m}|rF||_n||_t||_tj|jdd|j|d |_d S) au Instantiate a Dataset Repository object by cloning the remote ModelScope dataset repo Args: repo_work_dir (str): The dataset repo root directory. dataset_id (str): dataset id in ModelScope from which git clone revision (str, optional): revision of the dataset you want to clone from. Can be any of a branch, tag or commit hash auth_token (str, optional): token obtained when calling `HubApi.login()`. Usually you can safely ignore the parameter as the token is already saved when you login the first time, if None, we will use saved token. git_path (str, optional): The git command line path, if None, we use 'git' Raises: InvalidParameter: parameter invalid. z"dataset_work_dir must be provided!r3z%dataset_work_dir can not be root dir!rrrTr)rcN)rcrPrQrrstriprbrrr repo_base_dirr repo_namerrrrr r r#r$ _get_repo_urlrepo_url)r,rbrcrrrr-rr0r0r1r2s*    zDatasetRepository.__init__returncCsht|jr|}|j|}|r||jkrdStd |j|j |j |j |j|j |j|jS)NrZzCloning repo from {} )rr&rbr'r#r(rhr!infoformatr)rerrfr)r,r/r0r0r1r)s   zDatasetRepository.cloneFrCr=rFcCstjdtdd |dust|tsd}t|t|ts!td|js(td|j |j |j|j |j |j|}|j |}|j |j |j j|j dd|j |j ||j j|j |j|||d dSrG)rMrNrOrPrQrrRrrr#r+rbr*rerfr'r(r>rSrTrU)r,rCr=rFrVr/r0r0r1rU s6   zDatasetRepository.pushcCstd|dS)Nz /datasets/r4r )r,rcr0r0r1rg<szDatasetRepository._get_repo_urlcCr5r6)r#r7rbrr8r0r0r1r'?r:z!DatasetRepository._get_remote_url) r]r^r_r`rrQrr2r)rRrUrgr'r0r0r0r1ras8 / 2 ra)rrMtypingrZmodelscope.hub.errorsrrrZmodelscope.utils.constantrrrZmodelscope.utils.loggerr gitr Z utils.utilsr r!rrar0r0r0r1s    8