o 1 i3@sddlZddlZddlZddlmZmZmZddlZddl Z ddl Z ddl m Z ddl mZmZddlmZmZmZerBddlmZeeZe d>deded eefd d Zed?d ddefddZed@defddZed@defddZGdddZ GdddZ!GdddZ"eddZ#ed@defd d!Z$Gd"d#d#Z%Gd$d%d%Z&Gd&d'd'Z'ed@defd(d)Z(d*d+Z)e ,  -    dAd.eded/ed0ed1ed eed2eed3eed4efd5d6Z*e d7d8d9d:dBd;eedefd/s z#convert_to_tensor..z framework=z( not supported in `convert_to_tensor()`!)Zray.rllib.utils.torch_utilsr try_import_tftreeZ map_structureNotImplementedError)r r rr_rrrrs    rconfigr num_gpus_requestedcCs|jdkrHt\}}|dkrCddlm}|}t|dkr,tjjtjjj kr,|dS|j |j ks=Jd|j d| |j S| dStd|jd ) aReturns a single device (CPU or some GPU) depending on a config. Args: config: An AlgorithmConfig to extract information from about the device to use. num_gpus_requested: The number of GPUs actually requested. This may be the value of `config.num_gpus_per_env_runner` when for example calling this function from an EnvRunner. Returns: A single device (or name) given `config` and `num_gpus_requested`. rr) get_devicesrzlocal_gpu_idx z+ is not a valid GPU ID or is not available.cpuz`framework_str` z not supported!)Z framework_strtry_import_torchZray.air._internal.torch_utilsr!lenrayZ_privateZworker_modeZ WORKER_MODEZ local_gpu_idxcudaZ device_countrr)rr rrr!Zdevicesrrr get_device5s       r(FerrorcCsTdtjvr tddSz ddl}ddl}W||fSty)|r&tdYdSw)a*Tries importing JAX and FLAX and returns both modules (or Nones). Args: error: Whether to raise an error if JAX/FLAX cannot be imported. Returns: Tuple containing the jax- and the flax modules. Raises: ImportError: If error=True and JAX is not installed. ZRLLIB_TEST_NO_JAX_IMPORTz$Not importing JAX for test purposes.)NNrNzCould not import JAX! RLlib requires you to install at least one deep-learning framework: `pip install [torch|tensorflow|jax]`.)osenvironloggerwarningflaxjax ImportError)r)r.r/rrrtry_import_jaxis   r1cCst}dtjvrtdd|dfSdtjvrdtjd<d}dtjvr+tjd}d}nzd dl}WntyE|r>td d|dfYSwz |j j }|j |j j |s]|||j |j jWn tyq|}Ynwt|d szd }n d |jddvrdnd }|||fS)aTries importing tf and returns the module (or None). Args: error: Whether to raise an error if tf cannot be imported. Returns: Tuple containing 1) tf1.x module (either from tf2.x.compat.v1 OR as tf1.x). 2) tf module (resulting from `import tensorflow`). Either tf1.x or 2.x. 3) The actually installed tf version as int: 1 or 2. Raises: ImportError: If error=True and tf is not installed. RLLIB_TEST_NO_TF_IMPORTz*Not importing TensorFlow for test purposesNZTF_CPP_MIN_LOG_LEVEL3F tensorflowTrzCould not import TensorFlow! RLlib requires you to install at least one deep-learning framework: `pip install [torch|tensorflow|jax]`. __version__rz2.)_TFStubr*r+r,r-sysmodulesr4r0compatv1loggingZ set_verbosityERRORZdisable_v2_behaviorZenable_resource_variablesWARNAttributeErrorhasattrr5)r)Ztf_stubZ was_imported tf_moduleZ tf1_moduleversionrrrrsB             rc@seZdZdddZddZdS)r7returnNcCs t|_dSr) _KerasStubZkerasselfrrr__init__s z_TFStub.__init__cCsdS)NFrrErrr__bool__sz_TFStub.__bool__rCN)__name__ __module__ __qualname__rGrHrrrrr7s  r7c@seZdZdddZdS)rDrCNcCs t|_dSr)_FakeTfClassStubZModelrErrrrGs z_KerasStub.__init__rIrJrKrLrGrrrrrDsrDc@eZdZddZdS)rMcOtd)Nz:Could not import `tensorflow`. Try pip install tensorflow.r0rFakwrrrrGz_FakeTfClassStub.__init__NrNrrrrrM rMcsfdd}|S)zqConditional decorator for @tf.function. Use @tf_function(tf) instead to avoid errors if tf is not installed.csdusr |S|Sr)Zexecuting_eagerlyfunction)funcrArr decorators ztf_function..decoratorr)rArZrrYr tf_functions r[c CsVdtjvr tddSzddl}|WSty*}z |r|WYd}~dSd}~ww)zTries importing tfp and returns the module (or None). Args: error: Whether to raise an error if tfp cannot be imported. Returns: The tfp module. Raises: ImportError: If error=True and tfp is not installed. r2z7Not importing TensorFlow Probability for test purposes.Nr)r*r+r,r-Ztensorflow_probabilityr0)r)tfperrrtry_import_tfps r^c@rO)_NNStubcOsd|_t|_t|_dSr)Z functional_FakeTorchClassStubModule _ParallelStubZparallelrRrrrrGs z_NNStub.__init__NrNrrrrr_rVr_c@rO)r`cOrP)Nz0Could not import `torch`. Try pip install torch.rQrRrrrrGrUz_FakeTorchClassStub.__init__NrNrrrrr`rVr`c@rO)rbcOst|_t|_dSr)r`Z DataParallelZDistributedDataParallelrRrrrrGs z_ParallelStub.__init__NrNrrrrrbrVrbcCs^dtjvr tdtSzddl}ddlm}||fWSty.|r)tdtYSw)a&Tries importing torch and returns the module (or None). Args: error: Whether to raise an error if torch cannot be imported. Returns: Tuple consisting of the torch- AND torch.nn modules. Raises: ImportError: If error=True and PyTorch is not installed. ZRLLIB_TEST_NO_TORCH_IMPORTz(Not importing PyTorch for test purposes.rNzCould not import PyTorch! RLlib requires you to install at least one deep-learning framework: `pip install [torch|tensorflow|jax]`.) r*r+r,r- _torch_stubsrZtorch.nnnnr0)r)rrdrrrr#"s     r#cCst}d|fSr)r_)rdrrrrcBsrcrunnamed-variablevalue trainabletf_name torch_tensorshapedtyperCc Cs|dvr9ddl}|pt|dt|tr|jn t|tr|jnd}|jjj |f|||d|dur3iSd|iS|dkr|durt \} } t|t j sQt |}| |} || jt jfvrc| } n|| jt jfvrp| } n || jt jfvr|| } |r| |} || _| S|S) aCreates a tf variable, a torch tensor, or a python primitive. Args: value: The initial value to use. In the non-tf case, this will be returned as is. In the tf case, this could be a tf-Initializer object. framework: One of "tf", "torch", or None. trainable: Whether the generated variable should be trainable (tf)/require_grad (torch) or not (default: False). tf_name: For framework="tf": An optional name for the tf.Variable. torch_tensor: For framework="torch": Whether to actually create a torch.tensor, or just a python value (default). device: An optional torch device to use for the created torch tensor. shape: An optional shape to use iff `value` does not have any (e.g. if it's an initializer w/o explicit value). dtype: An optional dtype to use iff `value` does not have any (e.g. if it's an initializer w/o explicit value). This should always be a numpy dtype (e.g. np.float32, np.int64). Returns: A framework-specific variable (tf.Variable, torch.tensor, or python primitive). )rrrNrk)Z initializerrkrgrjrT)r4getattr isinstancefloatZfloat32intZint32r:r; get_variabler#npZndarrayarrayZ from_numpyZfloat64doubletoZ requires_grad) rfr rgrhrirrjrkrrrZvar_rrrrpGsL$         rpz+rllib/utils/framework.py::get_activation_fnz(rllib/models/utils.py::get_activation_fnT)oldnewr)namecCsdSrr)rwr rrrget_activation_fnsrxr)r)F)rFreFNNN)Nr),r<r*r8typingrrrnumpyrqrr%Zray._common.deprecationrZray.rllib.utils.annotationsrrZray.rllib.utils.typingrr r Z%ray.rllib.algorithms.algorithm_configr getLoggerrJr,strrror(boolr1rr7rDrMr[r^r_r`rbr#rcrprxrrrrs    3B    J