o )io$ @sUddlZddlmZmZmZddlZddlZddlm Z m Z m Z m Z ddl mZddlmZddlmZddlmZddlmZmZmZmZmZdd lmZmZmZmZm Z m!Z!m"Z"m#Z#m$Z$dd l%m&Z&erydd l'm(Z(dd l)m*Z*dd l+m,Z,ee-Z.e$eee!e e#e"eeeeeeh Z/e0e1ee2d< d/dej3de4dede5deedej3f ddZ6 d/ddddde4dedeedef ddZ7dej3de8dej3dej3fdd Z9 d/d!e8d"ed#de:e8e;e;ffd$d%Zd,e8de8fd-d.Z?dS)0N) TYPE_CHECKINGOptionalUnion)EntryNotFoundErrorHfHubHTTPErrorHFValidationErrorRepositoryNotFoundError)nn)PretrainedConfig) LoRAConfig) init_logger)#ColumnParallelLinearWithShardedLoRA)MergedColumnParallelLinearWithShardedLoRA&MergedQKVParallelLinearWithShardedLoRA QKVParallelLinearWithShardedLoRA RowParallelLinearWithShardedLoRA) BaseLayerWithLoRAColumnParallelLinearWithLoRALogitsProcessorWithLoRA"MergedColumnParallelLinearWithLoRAMergedQKVParallelLinearWithLoRAQKVParallelLinearWithLoRAReplicatedLinearWithLoRARowParallelLinearWithLoRAVocabParallelEmbeddingWithLoRA) LinearBase)LogitsProcessor)ParallelLMHead) WeightsMapper_all_lora_classeslayer max_loras lora_configpacked_modules_list model_configreturncCs>tD]}|j||||dr||}|||||Sq|S)N)Z source_layerr"r#r$)rZcan_replace_layercreate_lora_weights)r r!r"r#r$Zlora_clsZinstance_layerr'[/home/app/PaddleOCR-VL-test/.venv_paddleocr/lib/python3.10/site-packages/vllm/lora/utils.py from_layer=sr)rlm_headrcCs0t||j|jj|jj|}|||||SN)rZ embedding_dimweightZdtypeZdeviceZget_sharded_to_full_mappingr&)r r*r!r"r$retr'r'r(from_layer_logits_processorOs  r.model module_name new_modulecCs<|d|ddd}|dd}t||||S)z1Replace a submodule in a model with a new module..N)Z get_submodulejoinsplitsetattr)r/r0r1parent target_namer'r'r(replace_submodule]s r9nameweights_mapperrcCs|dr|dd}|r||n|}d|}n |r ||n|}|dr)dnd}|d}|ddkrT|dd ksB|dd krTd||d}||dd kd fS|dd ks`|dd krrd||d}||dd kd fS|ddkrd||d}|d dfSt|d)aParse the name of lora weights. args: name: the name of the fine-tuned LoRA, e.g. base_model.model.dense1.weight weights_mapper: maps the name of weight, e.g. `model.` -> `language_model.model.`, return: tuple(module_name, is_lora_a): module_name: the name of the module, e.g. model.dense1, is_lora_a whether the tensor is lora_a or lora_b. is_bias whether the tensor is lora bias. zbase_model.model.rr2r3r,Zlora_AZlora_BFZlora_embedding_AZlora_embedding_BZbiasTz is unsupported LoRA weight) startswithreplaceZ _map_namer5r4 ValueError)r:r; start_indexpartsnew_namer'r'r(parse_fine_tuned_lora_namefs$       rE load_modulesexpected_lora_modulescCsTdd}dd}t|tsdS||r(td|}|r(|dd}|||SdS) a PEFT supports passing `target_modules` in the form of regular expressions, such as `model.*(q_proj|k_proj|v_proj)$`. This function is mainly used to determine whether the suffix in the regular expression is present in the `expected_lora_modules`. cSs(zt|WdStjyYdSw)NTF)recompileerror)patternr'r'r(is_valid_regexs  z/is_regex_target_modules..is_valid_regexcSst|t|Sr+)setissubset)Zsub_listZ full_listr'r'r( is_subsetsz*is_regex_target_modules..is_subsetFz \((.*?)\)\$?$|) isinstancestrrHsearchgroupr5)rFrGrLrOmatchsuffixr'r'r(is_regex_target_moduless    rXcCsht}|D](\}}t|dd}|dur|D]}||qt|tfr/||ddqt|S)z2 In vLLM, all linear layers support LoRA. embedding_modulesNr2r3)rMZ named_modulesgetattraddrRrr5list)r/Zsupported_lora_modulesr:modulerYr'r'r(get_supported_lora_moduless   r^ lora_pathc Csztj|r|S|drtj|Stj|rtj|Sz tj|d}W|St t t t fy<t d|YSw)a' Resolves the given lora_path to an absolute local path. If the lora_path is identified as a Hugging Face model identifier, it will download the model and return the local snapshot path. Otherwise, it treats the lora_path as a local file path and converts it to an absolute path. Parameters: lora_path (str): The path to the lora model, which can be an absolute path, a relative path, or a Hugging Face model identifier. Returns: str: The resolved absolute local path to the lora model. ~)Zrepo_idz'Error downloading the HuggingFace model)ospathisabsr? expanduserexistsabspathhuggingface_hubZsnapshot_downloadrrrrlogger exception)r_Zlocal_snapshot_pathr'r'r(get_adapter_absolute_paths"       rjr+)@ratypingrrrrgregexrHZhuggingface_hub.utilsrrrrZtorchr Z transformersr Z vllm.configr Z vllm.loggerr Zvllm.lora.fully_sharded_layersr rrrrZvllm.lora.layersrrrrrrrrrZ!vllm.model_executor.layers.linearrZ+vllm.model_executor.layers.logits_processorrZ3vllm.model_executor.layers.vocab_parallel_embeddingrZ vllm.model_executor.models.utilsr__name__rhrrMtype__annotations__Moduleintr\r)r.rSr9tupleboolrErXr^rjr'r'r'r(s     ,           3