B a@sPddlZddlZddlZddlZddlZddlZddlZddlm Z ddlm Z ddlm Z ddlm Z ddlm Z ddlmZddlmZdd lmZeejZd Zd Zd Zd ZdZdZdZdZeeeeefZeeefZeeefZeeeZ e j!ej"#e j!ej$#e j!ej%#e j!ej&#ee Z'e(e e(dddZ)e(e e(dddZ*e(e e(dddZ+e e,e-dddZ.e(e-dddZ/e(e e(dd d!Z0e e,e e(d"fe e(d"fd#d$d%Z1e e,e e(d"fdd&d'Z2e(e e(d"fdd(d)Z3e4d*ej5ej6BZ7e4d+Z8e(e(d,d-d.Z9e(e e(d/d0d1Z:dS)2N)IO)List)Optional)Set)Tuple) extensions) interpreters)licenses directorysymlinksocketfile executableznon-executabletextbinary)pathreturnc Cs>yt|}Wn&ttfk r4t|dYnX|j}t|rLthSt|r\t hSt |rlt hSt h}t |tj}|r|tn |tttj|}t|dkr||n*|rt|}t|dkr|t|dt|@st|r |tn |tt|@s(t|t|@s:t||S)Nz does not exist.r)oslstatOSError ValueErrorst_modestatS_ISDIR DIRECTORYS_ISLNKSYMLINKS_ISSOCKSOCKETFILEaccessX_OKadd EXECUTABLENON_EXECUTABLEtags_from_filenamerbasenamelenupdateparse_shebang_from_filetags_from_interpreter ENCODING_TAGS file_is_textTEXTBINARYAssertionError MODE_TAGS)rsrmodetagsrtshebangr65/tmp/pip-unpacked-wheel-jv6j9gop/identify/identify.pytags_from_path)s<            r8cCstj|\}}tj|\}}t}x4|g|dD] }|tjkr8|tj|Pq8Wt|dkr|dd }|tj kr|tj |n|tj kr|tj ||S)N.r) rrsplitsplitextsetrNAMESr(r'lower EXTENSIONSEXTENSIONS_NEED_BINARY_CHECK)r_filenameextretpartr6r6r7r%Ws    r%) interpreterrcCsD|d\}}}x,|r<|tjkr*tj|S|d\}}}qWtS)N/r9) rpartitionr INTERPRETERSr=)rGrBr6r6r7r*ms   r*)bytesiorc CsLtddddddddgttd d ttd d }t|d d| S)zReturn whether the first KB of contents seems to be binary. This is roughly based on libmagic's binary/text detection: https://github.com/file/file/blob/df74b09b9027676088c797528edcaae5a9ce9ad0/src/encoding.c#L203-L228   iN) bytearrayrangeboolread translate)rKZ text_charsr6r6r7is_textzs$r]c Cs8tj|st|dt|d }t|SQRXdS)Nz does not exist.rb)rrlexistsropenr])rfr6r6r7r,s  r,)linercCs(y t|Stk r"|SXdS)N)shlexr;r)rbr6r6r7_shebang_splits rd.)rKcmdrcCsx|ddkr|}y|d}Wntk r:|SXx|D]}|tkrB|SqBWtt|}x4t|ddD] \}}|dkrqx||df}qxWqW|S)Ns#!zUTF-8z-ir:) r[readlinedecodeUnicodeDecodeError printabletuplerdstrip enumerate)rKreZ next_line_bZ next_linecZ line_tokensitokenr6r6r7_parse_nix_shebangs rrcCs|ddkrdS|}y|d}Wntk r<dSXx|D]}|tkrDdSqDWtt|}|r|ddkr|dd}|d krt||S|S) z8Parse the shebang from a file opened for reading binary.rfs#!r6zUTF-8rz /usr/bin/envr:N)z nix-shell) r[rhrirjrkrlrdrmrr)rKZ first_line_b first_linerorer6r6r7 parse_shebangs    rtc Cstj|st|dt|tjs,dSyt|d }t|SQRXWn4tk r~}z|j t j krldSWdd}~XYnXdS)z$Parse the shebang given a file path.z does not exist.r6r^N) rrr_rr r!r`rtrerrnoEINVAL)rraer6r6r7r)s   r)z^\s*(Copyright|\(C\)) .*$z\s+)srcCs td|}td|}|S)N ) COPYRIGHT_REsubWS_RErm)rxr6r6r7 _norm_licenses  r~)rCrc Csddl}t|dd}|}WdQRXt|}tj}d}xhtjD]^\}}t|} || kr^|S|rtt |t | t |dkrqB| || } | |krB| }|}qBW|r|t |dkr|SdSdS)aReturn the spdx id for the license contained in `filename`. If no license is detected, returns `None`. spdx: https://spdx.org/licenses/ licenses from choosealicense.com: https://github.com/choosealicense.com Approximate algorithm: 1. strip copyright line 2. normalize whitespace (replace all whitespace with a single space) 3. check exact text match with existing licenses 4. failing that use edit distance rNzUTF-8)encodingryg?) editdistance_sr`r[r~sysmaxsizer ZLICENSESabsr'Zdistance) rCrracontentsZnormZ min_edit_distZmin_edit_dist_spdxZspdxrZ norm_licenseZ edit_distr6r6r7 license_ids&$ r);ruos.pathrrercrstringrtypingrrrrridentifyrrZidentify.vendorr frozensetrkrrrrr#r$r-r.Z TYPE_TAGSr0r+Z _ALL_TAGSr(r@valuesrAr>rJZALL_TAGSstrr8r%r*bytesrZr]r,rdrrrtr)compileI MULTILINEr{r}r~rr6r6r6r7s^            .