o rq i @sZddlZddlmZddlmZmZddlZddl m Z m Z ddl m Z GdddZdS)N)abstractmethod)ListUnion) read_configsnapshot_download)Configc@sveZdZdefddZedeeeeeefdeefddZ defd d Z e d efd d Z e dddZdS)InferFrameworkmodel_id_or_dircKstj|r ||_nt||_||}t|j}|d}|dur)|p(||}tj|jd}tj |rLt |}|d}|durL|pK||}|sUt d|dS)z Args: model_id_or_dir(`str`): The model id of the modelhub or a local dir containing model files. z model.typeNz config.json model_typez"Model accelerating not supported: ) ospathexistsZ model_dirrmodel_type_supportedrZsafe_getjoinisfiler from_file ValueError)selfr kwargsZmodel_supportedconfigr config_filerk/home/app/PaddleOCR-VL/.venv_paddleocr/lib/python3.10/site-packages/modelscope/pipelines/accelerate/base.py__init__ s.        zInferFramework.__init__promptsreturncKdS)z Args: prompts(`Union[List[str], List[List[int]]]`): The string batch or the token list batch to input to the model. Returns: The answers in list according to the input prompt batch. Nr)rrrrrr__call__)s zInferFramework.__call__r cCr)NFr)rr rrrr5sz#InferFramework.model_type_supported major_versioncCstj\}}||kS)z%Check the GPU compatibility. )torchcudaZget_device_capability)rmajor_rrrcheck_gpu_compatibility8sz&InferFramework.check_gpu_compatibilityvllmcKs<|dkrddlm}||fi|}||_|Std|)aInstantiate the model wrapped by an accelerate framework. Args: model_id_or_dir(`str`): The model id of the modelhub or a local dir containing model files. framework(`str`): The framework to use. Returns: The wrapped model. r$)VllmzFramework not supported: )r$r&Z llm_frameworkr)clsr Z frameworkrr&r$rrrfrom_pretrained?s  zInferFramework.from_pretrainedN)r$)__name__ __module__ __qualname__strrrrrintrr staticmethodr# classmethodr(rrrrr s  r)Zos.pathr abcrtypingrrZ torch.cudarZ modelscoperrZmodelscope.utils.configrrrrrrs