o o i! @sddlZddlmZmZmZmZmZmZmZddl m Z er>ddl m Z ddlmZddl mZddlmZmZddlmZGdd d Ziidd fd ed ed edeedef ddZdS)N) TYPE_CHECKING GeneratorIteratorListOptionalTupleUnion)TransformerTokenizer)PreTrainedTokenizer)GenerationParametersSamplingParameters)OutlinesLogitsProcessorc@seZdZdZ    dddZdeeeefd d d d d efddZdeeeefd d d d d e efddZ ddde e de e deddd e eee fddff ddZdS)MLXLMz& Represents an `mlx_lm` model model nn.Module tokenizerr cCs||_||_t|j|_dS)N)r mlx_tokenizerr Z _tokenizerr)selfrrra/home/app/PaddleOCR-VL-test/.venv_paddleocr/lib/python3.10/site-packages/outlines/models/mlxlm.py__init__s  zMLXLM.__init__promptsgeneration_parametersr sampling_parametersr returncCs|||||}dt|S)N)streamjoinlist)rrrlogits_processorrstreamerrrrgenerateszMLXLM.generateccs4ddlm}t|\}}}t|\} } } } } |dur!td}t|ts*td| dkr2td| dkr:td| durBtd |durJtd |durRtd | | | |d }||j |}|j j }| t |j|fi|t|D]\\}}}||jjkrn |||jVqx||jVdS) aGenerate text using `mlx_lm`. Arguments --------- prompts A prompt or list of prompts. generation_parameters An instance of `GenerationParameters` that contains the prompt, the maximum number of tokens, stop sequences and seed. All the arguments to `SequenceGeneratorAdapter`'s `__cal__` method. logits_processor The logits processor to use when generating text. sampling_parameters An instance of `SamplingParameters`, a dataclass that contains the name of the sampler to use and related parameters as available in Outlines. Returns ------- The generated text. rNgeAz6The `mlx-lm` library does not support batch inference.Z beam_searchz2The `mlx-lm` library does not support Beam Search.r zrcs|}dks dkrj|dd}n,dkr9dur.dkr.dkr.j|}nj|d}ntd d |d|f}||fS) NgZgreedy)ZaxisZ multinomialrr?r zInvalid mlx-lm sampler: ``)ZsoftmaxZargmaxZ sample_utilsZtop_p_samplingrandomZ categorical ValueError)r@Zsoftmax_logitsr:r;mlx_lmr7r%r9r$rrsamples  z#MLXLM.generate_step..sampleT)cacherAr ) r&r'rFrfloatmodelsrHZmake_prompt_cacherZreshapeitemappend)rr=r#r$r%r rGrHZunprocessed_input_idsZ generated_idsr@Z logits_1dZnew_token_singler;Z new_tokenrrErr3|s* &      zMLXLM.generate_step)rrrr )__name__ __module__ __qualname____doc__rrr,rr"rrrrIrrr*r3rrrrrsL     QrF model_nametokenizer_config model_config adapter_pathlazyc Csdz ddlm}ddl}Wn tytdw|js!td|j|||||d\}}t||S)aInstantiate a model from the `mlx_lm` library and its tokenizer. Signature adapted from https://github.com/ml-explore/mlx-examples/blob/4872727/llms/mlx_lm/utils.py#L422 Parameters ---------- Args: path_or_hf_repo (Path): The path or the huggingface repository to load the model from. tokenizer_config (dict, optional): Configuration parameters specifically for the tokenizer. Defaults to an empty dictionary. model_config(dict, optional): Configuration parameters specifically for the model. Defaults to an empty dictionary. adapter_path (str, optional): Path to the LoRA adapters. If provided, applies LoRA layers to the model. Default: ``None``. lazy (bool): If False eval the model parameters to make sure they are loaded in memory before returning, otherwise they will be loaded when needed. Default: ``False`` Returns ------- A `MLXLM` model instance. rNzKThe `mlx_lm` library needs to be installed in order to use `mlx_lm` models.z5You cannot use `mlx_lm` without Apple Silicon (Metal))rRrSrTrU) r&r'rF ImportErrorZmetalZ is_available RuntimeErrorloadr) rQrRrSrTrUr7rFrrrrrmlxlms$     rY)r(typingrrrrrrrZ transformersr r&r'r7Zmlx.nnnnr Zoutlines.generate.apir r Zoutlines.processorsrrr,dictboolrYrrrrs4$     6