a &0_!^@s>ddlmZddlZddlZddlZddlmZddlmZddl m Z ddl m Z m Z mZmZmZmZmZmZd4d d Zd d Zd dZd5ddZddZddZd6ddZddZddZddZGdddeZdZdZ d Z!d!Z"d"d#Z#d$d%Z$ej%d&d'Z&d(d)Z'd*d+Z(d,d-Z)ej%d.d/Z*d0d1Z+d2d3Z,dS)7)absolute_importN)_)open) repository) cacheutilerror narrowspecphasespycompatscmutilstoreutilFc Cs|j}|j}d}|jr,d|jdgvr,d}|r8|s8dS|rD|sDdSt|rPdS|jrZdS|j}|durr|d}|szdSt }|dr| dn||d }|s|jj t d dSt |d }||j}|r|jj t d d t||jj t ddS|}d|fS)aWhether it is possible to perform a streaming clone as part of pull. ``bundle2`` will cause the function to consider stream clone through bundle2 and only through bundle2. Returns a tuple of (supported, requirements). ``supported`` is True if streaming clone is supported and False otherwise. ``requirements`` is a set of repo requirements from the remote, or ``None`` if stream clone isn't supported. Fsv2sstreamT)FNNsstream-preferredsrevlogv1s streamreqss=warning: stream clone requested but server has them disabled ,sGwarning: stream clone requested but client is missing requirements: %s , sQ(see https://www.mercurial-scm.org/wiki/MissingRequirement for more information) )reporemoteZ canusebundle2Zremotebundle2capsgetlenZheadsZstreamclonerequestedcapablesetadduiwarnrsplitsupportedformatsjoinsorted) pullopZbundle2rrZbundle2supportedZstreamrequested requirementsZ streamreqs missingreqsr!7/usr/lib/python3/dist-packages/mercurial/streamclone.pycanperformstreamclones`        r#c Csddlm}t|\}}|s dS|j}|j}d}|drr| }|di}Wdn1sh0Y|j t d| }|di}Wdn1s0Y| } z t | } Wn"tytt d| Yn0| dkr tt dn2| d kr&tt d n| d kr>tt d | } ztt | d d\} } Wn(ttfytt d| Yn0|nt||| | ||j|jB|_||j |j|j|j_t||r|j |||!Wdn1s0YdS)zPossibly perform a legacy stream clone operation. Legacy stream clones are performed as part of pull but before all other operations. A legacy stream clone will not be performed if a bundle2 stream clone is supported. r localrepoNs branchmapsstreaming all changes s stream_out'unexpected response from remote server:soperation forbidden by servers$locking the remote repository failedrs%the server sent an unknown error code )"r%r#rrrZcommandexecutorZ callcommandresultrstatusrreadlineint ValueErrorr ResponseErrorAbortmapr TypeErrorlock consumev1rrresolvestorevfsoptionsfeaturessvfsoptionsr writereporequirementsZ _branchcachesreplace invalidate) rr% supportedrrrZ rbranchmapefplZresp filecount bytecountr!r!r"maybeperformlegacystreamclonezsZ    . .           rBcCsFtj|jvrdS|jjdddds&dSt|}|rB|jddSdS)z5Whether streaming clones are allowed from the server.Fsservers uncompressedT)Z untrustedsuncompressedallowsecret)rZREPO_FEATURE_STREAM_CLONEr6rZ configboolr Z hassecret)rZsecretr!r!r"allowservergenerations  rCcCs |j|SN)r walk)rmatcherr!r!r"_walkstreamfilessrGcsgd}JjdtD]$\}}}|r&||f||7}q&Wdn1s`0Yjdt|fjjjfdd}t||fS)aEmit content for version 1 of a streaming clone. This returns a 3-tuple of (file count, byte size, data iterator). The data iterator consists of N entries for each file being transferred. Each file entry starts as a line with the file name and integer size delimited by a null byte. The raw file data follows. Following the raw file data is the next file entry, or EOF. When used on the wire protocol, an additional line indicating protocol success will be prepended to the stream. This function is not responsible for adding it. This function will obtain a repository lock to ensure a consistent view of the store is captured. It therefore may raise LockError. r scanning Ns%d files, %d bytes to transfer c 3sD]\}}r$jd||fdt||fV|ddd@}|dkr^||Vntj||dD] }|VqlWdq1s0YqdS)Nssending %s (%d bytes) s%s%d rbF)Z auditpathlimit)rdebugr Z encodedirreadr filechunkiter)namesizer>chunk debugflagentriesrr7r!r"emitrevlogdatas z"generatev1..emitrevlogdata)r3rrMrGappendrr7rT)rZ total_bytesrPenamerQrVr!rSr" generatev1s  (rYccsjt|sdVdSzt|\}}}Wntjy@dVYdS0dVd||fV|D] }|VqZdS)aEmit content for version 1 of streaming clone suitable for the wire. This is the data output from ``generatev1()`` with 2 header lines. The first line indicates overall success. The 2nd contains the file count and byte size of payload. The success line contains "0" for success, "1" for stream generation not allowed, and "2" for error locking the repository (possibly indicating a permissions error for the server process). s1 Ns2 s0 s%d %d )rCrYrZ LockError)rr@rAitrRr!r!r"generatev1wireprotos r[UNcsDdkrtdjj@}dt|fdd}||fS)a@Emit content for version 1 of a stream clone bundle. The first 4 bytes of the output ("HGS1") denote this as stream clone bundle version 1. The next 2 bytes indicate the compression type. Only "UN" is currently supported. The next 16 bytes are two 64-bit big endian unsigned integers indicating file count and byte count, respectively. The next 2 bytes is a 16-bit big endian unsigned short declaring the length of the requirements string, including a trailing . The following N bytes are the requirements string, which is ASCII containing a comma-delimited list of repo requirements that are needed to support the data. The remaining content is the output of ``generatev1()`` (which may be compressed in the future). Returns a tuple of (requirements, data generator). r\s.we do not support the compression argument yetrc3sdVVt\}}}jtd||ftd||VtdtdVdVdksnJjjtd|td d }|d |D]}|j t|d |Vq| dS) NsHGS1swriting %d bytes for %d files >QQ>Hrr\bundlebytestotalZunitrstep) rYrr+rstructZpackr makeprogressupdate incrementcomplete)r@rArZprogressrR compressionrrequiresr!r"genKs$   zgeneratebundlev1..gen)r.rrrr)rrmrror!rlr"generatebundlev1/s  rpc Cs||jtd|t|f|jjtd|tdd}|dt}| d:|j j |j|dt |D]}|}z|dd\}} t| } Wn&ttfyttd |Yn0|jjr|jd |t| ft|} |j | d d d @} tj|| dD] } |jt| d| | qWdq1sV0YqWdn1sx0Y|jd dWdn1s0Yt|} | dkrd} ||jtdt|| t|| fWdn1s0YdS)a Apply the contents from version 1 of a streaming clone file handle. This takes the output from "stream_out" and applies it to the specified repository. Like "stream_out," the status line added by the wire protocol is not handled by this function. !%d files to transfer, %s of data clonerarbr)Z expectedcountr_rr&sadding %s (%s) wT)ZbackgroundcloserKrdNZclearfilecacheMbP?(transferred %s in %.1f seconds (%s/sec) )r3rr+rrrArgrhtimer transactionr7backgroundclosingr Zxranger,rr-r.r2rr/rTrMr Z decodedirrOrirwriter;rj)rr>r@rArkstartir?rPrQpathofprRelapsedr!r!r"r4isX      P,   r4cCs|d}|dkr$ttd|td|d\}}td|dd}||}|dsrttd t|d d }|||fS) Nr'r\s<only uncompressed stream clone bundles are supported; got %sr]r^rr_s@malformed stream clone bundle: requirements not properly encodedr) rNrr0rrfZunpackendswithrrstripr)r>rmr@rAZ requireslenrnrr!r!r"readbundle1headers(   rcCs`t|rttdt|\}}}||j}|rNttddt|t||||dS)zApply the content from a stream clone bundle version 1. We assume the 4 byte header has been read and validated and the file handle is at the 2 byte compression identifier. s2cannot apply stream clone bundle on non-empty repo4unable to apply stream clone: unsupported format: %srN) rrr0rrrrrr4)rr>r@rArr r!r!r" applybundlev1s  rc@s eZdZdZddZddZdS)streamcloneapplierzClass to manage applying streaming clone bundles. We need to wrap ``applybundlev1()`` in a dedicated type to enable bundle readers to perform bundle type-specific functionality. cCs ||_dSrD)_fh)selfZfhr!r!r"__init__szstreamcloneapplier.__init__cCs t||jSrD)rr)rrr!r!r"applyszstreamcloneapplier.applyN)__name__ __module__ __qualname____doc__rrr!r!r!r"rsrsccCsg}|s|d|S)z!list snapshot file from the stores phaseroots)Z publishingrW)rfnamesr!r!r"_walkstreamfullstorefiless rcCs2|\}}}}|tkr|S|||||||fS)z actually copy the snapshot files) _filefullr)entrycopyvfsmapsrcrPftypedatar!r!r" _filterfulls rc #sJgz*fdd}|VWD]}t|qnD]}t|q40dS)z(return a function to temporary copy filecs4t\}}t||tj||dd|S)NT)Zhardlink)r ZmkstemposcloserWrZ copyfiles)rfdZdstfilesr!r"rs    zmaketempcopies..copyN)rZ tryunlink)rtmpr!rr"maketempcopies s rcCs&t|jt|ji}|j|vs"J|S)z$make a (src -> vfs) map for the repo) _srcstorer7 _srccachecachevfsvfsvalues)rrr!r!r"_makemap s rc #sxt||jjtd|tdd}|dt.|fdd|D}dVd}|D]\}}}}|} |Vtt|V|t kr| |} |} n|t krt |d} t | j } zht| V|V| d kr| | f} ntj| | d } | D] } |t| 7}||| VqW| q`| 0q`Wdn1sJ0YWdn1sj0YdS) zactually emit the stream bundler`rarbrcsg|]}t|qSr!)r).0r=rrr!r" 6z_emit2..NrIrJrK)rrrgrrhrrZ uvarintencoder _fileappendrrfstatst_sizerNrOr)rrU totalfilesizerkseenrrPrrrr>rQchunksrRr!rr"_emit2-s<       rc Csr|Hg}d}d}|s |r0t|j||}|jdt||D](\}}} | rF|t|t | f|| 7}qFt |D]4}|j |rx||j |j7}|t|tdfqx|r|j dr||j dj7}|tdtdft|D]4}|j |r||j |j7}|t|tdfqt|||} t| } | dusDJWdn1sZ0Yt||| fS)aEmit content for version 2 of a streaming clone. the data stream consists the following entries: 1) A char representing the file destination (eg: store or cache) 2) A varint containing the length of the filename 3) A varint containing the length of file data 4) N bytes containing the filename (the internal, store-agnostic form) 5) N bytes containing the file data Returns a 3-tuple of (file count, file size, data iterator). rNrHsobsstore)r3r matchrootrrMrGrWrrrr7existslstatrrrZ cachetocopyrrrnextr) rZincludesZexcludesZincludeobsmarkersrUrrFrPrXrQrfirstr!r!r" generatev2Ss4       .rc gsv|d}|dd}|J|rNt|dVWdqT1sB0YndVWdn1sh0YdS)Nrr)nested)ctxsthisrestr!r!r"rs  &rc sjtd|t|ft}jjtd|tdd}|dt } dfdd| D}t |t |D]}t|d} || } t|} t|} t|| } jjrjd | | t| f| | d @}tj|| d D] }|jt|d ||q Wd q1sB0YqWd n1sd0YjddWd n1s0Yt|}|dkrd}jtdt|j|t|j|f|Wd n1s0Yd S)zApply the contents from a version 2 streaming clone. Data is read from an object that only needs to provide a ``read(size)`` method. rqrrrarbrc3s|]}|jVqdSrD)ryr)rrrr!r" rzconsumev2..rsadding [%s] %s (%s) rsrKrdNTrtrurv)r3rr+rrrArwrgrhrrxrrrangeZ readexactlyZuvarintdecodestreamrTrMrOrirrzr;posrj)rr>r@filesizer{rkrrr|rrZnamelenZdatalenrPr~rRrr!rr" consumev2sX          P,   rcsddlm}fdd|D}|r>ttddt|t|||t|j j B_ | j j j j_tdS)Nrr$csg|]}|jvr|qSr!)r<)rrrr!r"rrz!applybundlev2..rr)r)r%rr0rrrrrrrr5rr6r7r8r r9)rr>r@rrr%r r!rr" applybundlev2s     r)F)N)r\)-Z __future__r contextlibrrfZi18nrr rZ interfacesrr)rrr r r r rr#rBrCrGrYr[rpr4rrobjectrrrrrrrcontextmanagerrrrrrrrr!r!r!r"sB    ( ]M 6 :K  &. <