B qnd7*@sddlZddlmZddlmZmZmZmZGdddZGdddej Z e e j Z Gdd d ej ZGd d d ZGd d d eZGdddeZGdddeZGdddeZGdddeZGdddeZGdddZGdddZd!eeeeeddd ZdS)"N)chain)ListSetTupleTypec@sFeZdZeeeedddZddZddddZd d Zd d Z d S)Entity)sent_idstartendtagcCs||_||_||_||_dS)N)rr r r )selfrr r r r 2/tmp/pip-unpacked-wheel-a5vokgmp/seqeval/scheme.py__init__szEntity.__init__cCsd|j|j|j|jS)Nz({}, {}, {}, {}))formatrr r r )r r r r__repr__szEntity.__repr__)othercCs||kS)N)to_tuple)r rr r r__eq__sz Entity.__eq__cCs t|S)N)hashr)r r r r__hash__szEntity.__hash__cCs|j|j|j|jfS)N)rr r r )r r r rrszEntity.to_tupleN) __name__ __module__ __qualname__intstrrrrrrr r r rrs rc@s`eZdZeZeZeZeZeZ eZ eZ eeBeBeBe Be Be BZ dS)PrefixN) rrrenumautoIOBESULANYr r r rrsrc@s$eZdZeZeZeeBZdS)TagN)rrrrrSAMEDIFFr&r r r rr')sr'c@seZdZdZdZdZdZdeeedddZ ddZ d d Z dd d d Z dd ddZ dd ddZdedddZdeeeeefdddZdS)TokenNF-)tokensuffix delimitercCsT||_|rt|dn t|d|_|r4|ddn |dd}||pLd|_dS)Nr_)r,Prefixesprefixstripr )r r,r-r.r r r rr5szToken.__init__cCs|jS)N)r,)r r r rr;szToken.__repr__cCs8|j|jkr4t|jdd}d}t||j|dS)z+Check whether the prefix is allowed or not.zPrefix.z5Invalid token is found: {}. Allowed prefixes are: {}.T)r3allowed_prefixrreplace ValueErrorrr,)r Zallowed_prefixesmessager r ris_valid>s  zToken.is_valid)prevcCs|||jS)z6Check whether the current token is the start of chunk.)check_patternsstart_patterns)r r;r r ris_startFszToken.is_startcCs|||jS)z3Check whether the current token is inside of chunk.)r<inside_patterns)r r;r r r is_insideJszToken.is_insidecCs|||jS)z5Check whether the previous token is the end of chunk.)r< end_patterns)r r;r r ris_endNsz Token.is_end)r;condcCsF|tjkrdS|j|jkr(|tjkr(dS|j|jkrB|tjkrBdSdS)z)Check whether the tag pattern is matched.TF)r'r&r r(r))r r;rCr r r check_tagRs zToken.check_tag)r;patternscCs<x6|D].\}}}|j|kr|j|kr|||rdSqWdS)z.Check whether the prefix patterns are matched.TF)r3rD)r r;rEZ prev_prefixZcurrent_prefixZtag_condr r rr<\s zToken.check_patterns)Fr+)rrrr6r=r?rArboolrrr:r>r@rBr'rDrrrr<r r r rr*/s r*c@seZdZejejBejBZejejej fejejej fejejej fejejej fejejej fhZ ejejej fejejej fhZ ejejej fejejej fejejej fejejej fejejej fejejej fhZdS)IOB1N)rrrrrr r!r6r'r&r)r(r=r?rAr r r rrGdsrGc@seZdZejejBejBZejejej fejejej fejejej fejejej fhZ ejejej fejejej fhZ ejejej fejejej fejejej fejejej fejejej fhZdS)IOE1N)rrrrrr r"r6r'r&r)r(r=r?rAr r r rrH{srHc@seZdZejejBejBZejeje jfhZ ejeje j fejeje j fhZ ejeje jfejeje j fejeje jfejeje jfejeje j fejeje jfhZdS)IOB2N)rrrrrr r!r6r&r'r=r(r?r)rAr r r rrIsrIc@seZdZejejBejBZejejej fejejej fejejej fejejej fejejej fejejej fhZ ejejej fejejej fhZ ejej ej fhZdS)IOE2N)rrrrrr r"r6r'r&r)r=r(r?rAr r r rrJsrJc@seZdZejejBejBejBejBZ ej eje j fej eje j fhZ ejeje j fejeje j fejeje j fejeje j fhZejej e j fejej e j fhZdS)IOBESN)rrrrrr r!r"r#r6r&r'r=r(r?rAr r r rrKsrKc@seZdZejejBejBejBejBZ ej eje j fej eje j fhZ ejeje j fejeje j fejeje j fejeje j fhZejej e j fejej e j fhZdS)BILOUN)rrrrr!rr%r r$r6r&r'r=r(r?rAr r r rrLsrLc@sVeZdZdeeeeeeedddZ e ddZ eed d d Z ed d dZ dS)TokensFr+N)tokensschemer-r.rcsBdd|_fdd|D|_|j|jg|_||_dS)Nr )r-r.csg|]}|dqS))r-r.r ).0r,)r.rOr-r r sz#Tokens.__init__..) outside_tokenrNextended_tokensr)r rNrOr-r.rr )r.rOr-rrszTokens.__init__cCsd}g}|j}x|t|jkr|j|}|||rz|j|d|d}||rtt|j|||j d}| ||}n|d7}|j|d}qW|S)zExtract entities from tokens. Returns: list: list of Entity. Example: >>> tokens = Tokens(['B-PER', 'I-PER', 'O', 'B-LOC'], IOB2) >>> tokens.entities [('PER', 0, 2), ('LOC', 3, 4)] rr0)r r;)rr r r ) rRlenrSr:r>_forward_is_endrrr append)r ientitiesr;r,r entityr r rrYs     zTokens.entities)r r;cCsDx4t|j|d|D]\}}||r.|}q|SqWt|jdS)Nr0) enumeraterSr@rTrN)r r r;rXr,r r rrUs  zTokens._forward)rXcCs"|j|}|j|d}||S)Nr0)rSrB)r rXr,r;r r rrV s zTokens._is_end)Fr+N)rrrrrrr*rFrrpropertyrYrUrVr r r rrMs  rMc@sHeZdZd eeeeeeedddZedddZ e d d Z d S) EntitiesFr+) sequencesrOr-r.cs fddt|D|_dS)Nc s$g|]\}}t||djqS))rOr-r.r)rMrY)rPrseq)r.rOr-r rrQsz%Entities.__init__..)r[rY)r r^rOr-r.r )r.rOr-rrszEntities.__init__)tag_namecsfddt|jD}|S)Ncsh|]}|jkr|qSr )r )rPrZ)r`r r sz"Entities.filter..)rrY)r r`rYr )r`rfilterszEntities.filtercCsddt|jD}|S)NcSsh|] }|jqSr )r )rPrZr r rrasz'Entities.unique_tags..)rrY)r tagsr r r unique_tagsszEntities.unique_tagsN)Fr+) rrrrrrr*rFrrbr\rdr r r rr]s"r]Fr+)r^r-r.c Cst}d}x\|D]T}xN|D]F}yt|||d}||jWqtk r^t||YqXqWqWtjtj tj htjtj htj tj htj hg}tjtj tj htjtj htj tj htj hg}tjtj tj tj tj htjtj tj tj htjtj tj tj htj tj tj tj htjtj tj htj tj tj htj tj tj htj tj htj hg } tjtj tj tj tjhtjtj tj tjhtjtj tj tj htj tj tj tjhtjtj tj htj tj tjhtj tj tj htj tj htjhg } ||krtS||krtS|| krtS|| kr tSt||dS)zuDetects scheme automatically. auto_detect supports the following schemes: - IOB2 - IOE2 - IOBES z This scheme is not supported: {})r-r.N)setr*addr3KeyErrorr8rrrr r!r"r#r%r$rIrJrKrL) r^r-r.prefixes error_messagerNr,Zallowed_iob2_prefixesZallowed_ioe2_prefixesZallowed_iobes_prefixesZallowed_bilou_prefixesr r r auto_detect#sX                rj)Fr+)r itertoolsrtypingrrrrrFlagrdict __members__r2r'r*rGrHrIrJrKrLrMr]rrFrjr r r rs   54