o
ij @ s d dl mZ d dlZd dlmZ d dlmZ d dlmZmZ d dl m
Z
m Z d dl m
Z
d dlmZmZ d d lmZ d d
lmZmZ erZd d l mZ d d lmZmZ d d
lmZm Z ddd4ddZ d5d6d!d"Z d7d$d%Z d8d(d)Z d9d2d3Z!dS ): ) annotationsN)datetime)Path)
TYPE_CHECKINGAny)_DELTALAKE_AVAILABLE deltalake)eprint)NullTime)
unpack_dtypes)POLARS_STORAGE_CONFIG_KEYS_get_path_scheme)
DeltaTable) DataFrameDataType)
SchemaDictStorageOptionsDictT)strict table_uri
str | Pathr boolreturnstrc C s0 t t| d u rt| |}|S | }|S )N)r r r
expanduserresolve)r r resolved_uri r M/home/app/Keep/.python/lib/python3.10/site-packages/polars/io/delta/_utils.py_resolve_delta_lake_uri s r
table_pathstr | Path | DeltaTableversionint | str | datetime | Nonestorage_optionsStorageOptionsDict | Nonedelta_table_optionsdict[str, Any] | Nonedeltalake.DeltaTablec C s t |durdd | D }t| tjr-t|du|du|dugr+tjdtdd | S |du r3i }t | }t|t
t fsLtj|f||d|}|S tj| fd|i|}| | |S ) z
Initialize a Delta lake table for use in read and scan operations.
Notes
-----
Make sure to install deltalake>=0.8.0. Read the documentation
`here `_.
Nc S s i | ] \}}|t vr||qS r )r
).0kvr r r
1 s
z)_get_delta_lake_table..zWhen supplying a DeltaTable directly, `version`, `storage_options`, and `delta_table_options` are ignored.
To silence this warning, don't supply those parameters. )
stacklevel)r" r$ r$ )
_check_if_delta_availableitems
isinstancer r anywarningswarnRuntimeWarningr r r Zload_as_version)r r" r$ r& r Zdl_tblr r r _get_delta_lake_table sP
r6 Nonec C s t sd} t| d S )Nz=deltalake is not installed
Please run: pip install deltalake)r ModuleNotFoundError)msgr r r r/ [ s r/ dtypeslist[DataType]c C s2 t | }tth}||@ }rd|}t|d S )Nz+dataframe contains unsupported data types: )r r r
TypeError)r: Z
schema_dtypesZunsupported_typesoverlapr9 r r r _check_for_unsupported_typesa s
r> add_actions_dfr filter_columns list[str]schemar verboseDataFrame | Nonec
C sf dd l }d| vr|rtd d S d| d i}d| v r%| d j jddni }d| v r6| d j jddni }d | v rG| d j jddni }|D ]_} || }
d u re|jd g| d
d| j }
|| } d u r}|jd g|| d
d| j } || } d u r|jd g|| d
d| j } |
|| d < | || d < | || d
< qK|j
|| j dS )Nr Z num_recordszDscan_delta: statistics load failed: 'num_records' column not presentlenZ
null_countT)Z as_seriesminmax)ZdtypeZ_nc_min_max)height) polarsr structZunnestto_dictgetZSeriesZget_index_typeZnew_from_indexrJ r )
r? r@ rB rC ploutZnull_count_colsZmin_colsZmax_colsZcol_nameZcol_ncZcol_minZcol_maxr r r 0_extract_table_statistics_from_delta_add_actionsm sJ rQ )r r r r r r )NNN)
r r! r" r# r$ r% r& r' r r( )r r7 )r: r; r r7 )
r? r r@ rA rB r rC r r rD )"
__future__r r3 r pathlibr typingr r Zpolars._dependenciesr r Zpolars._utils.loggingr Zpolars.datatypesr
r Zpolars.datatypes.convertr Zpolars.io.cloud._utilsr
r r rK r r Zpolars._typingr r r r6 r/ r> rQ r r r r s,
<