fa ddlZddlZddlZddlZddlmZddlmZddlm Z ejdZ GddeZ dZ d d Zy) N)MutableMapping)cached_property) url_to_fszfsspec.mappingceZdZdZddZedZdZddZdZ dZ d Z d Z dd Z dd Zd ZdZdZdZdZdZy)FSMapaWrap a FileSystem instance as a mutable wrapping. The keys of the mapping become files under the given root, and the values (which must be bytes) the contents of those files. Parameters ---------- root: string prefix for all the files fs: FileSystem instance check: bool (=True) performs a touch at the location, to check for write access. Examples -------- >>> fs = FileSystem(**parameters) # doctest: +SKIP >>> d = FSMap('my-data/path/', fs) # doctest: +SKIP or, more likely >>> d = fs.get_mapper('my-data/path/') >>> d['loc1'] = b'Hello World' # doctest: +SKIP >>> list(d.keys()) # doctest: +SKIP ['loc1'] >>> d['loc1'] # doctest: +SKIP b'Hello World' Nc*||_|j||_|jtj|ddd|_|t ttf}||_ ||_ ||_ |r6|jj|s|jj||rg|jj|std|d|jj|dz|jj!|dzyy)NxzPath z9 does not exist. Create with the ``create=True`` keywordz/a)fs_strip_protocolroot posixpathjoin_root_key_to_strFileNotFoundErrorIsADirectoryErrorNotADirectoryErrormissing_exceptionscheckcreateexistsmkdir ValueErrortouchrm)selfr r rrrs W/var/lib/jenkins/workspace/mettalog/venv/lib/python3.12/site-packages/fsspec/mapping.py__init__zFSMap.__init__)s&&t, " 2 29>>$3L Mcr R  %!!""  #5  77>>$' d# 77>>$' D6"89 GGMM$+ & GGJJtd{ # cJddlm}||j|jS)z@dirfs instance that can be used with the same keys as the mapper) DirFileSystem)pathr )implementations.dirfsr"rr )rr"s rdirfsz FSMap.dirfsBs 9$"7"7DGGDDrctjd|j |jj |jd|jj |jy#YyxYw)z0Remove all keys below root - empties out mappingzClear mapping at %sTN)loggerinfor r rrrs rclearz FSMap.clearIsN )4995  GGJJtyy$ ' GGMM$)) $  s A A..A2c |Dcgc]}|j|}}|dk(r|nd} |jj||}t|tr|d|i}|jDcic](\}}|t||j r t n|*}}}t||D cic]#\} } |dk(st|| ts| || %c} } Scc}w#|j $r }t |d}~wwxYwcc}}wcc} } w)aFetch multiple items from the store If the backend is async-able, this might proceed concurrently Parameters ---------- keys: list(str) They keys to be fetched on_error : "raise", "omit", "return" If raise, an underlying exception will be raised (converted to KeyError if the type is in self.missing_exceptions); if omit, keys with exception will simply not be included in the output; if "return", all keys are included in the output, but the value will be bytes or an exception instance. Returns ------- dict(key, bytes|exception) raisereturn)on_errorrN) _key_to_strr cat isinstancebytesrKeyErroritemszip BaseException) rkeysr.kkeys2oeoutevkeyk2s rgetitemszFSMap.getitemsRs(/33!!!$33!W,X( "''++eb+1C#u%Qxo   1 jD,C,CD ! K  tU+ R8#:c"g}+M RL  4 && " ! "  s(C4C 0-C>/(D C;/C66C;c|jDcic] \}}|j|t|"}}}|jj |ycc}}w)zSet the values of multiple items in the store Parameters ---------- values_dict: dict(str, bytes) N)r4r/ maybe_convertr pipe)r values_dictr8r=valuess rsetitemszFSMap.setitemsxsQEPDUDUDWXDAq$""1%}Q'77XX  VYs%Ac~|jj|Dcgc]}|j|c}ycc}w)z#Remove multiple keys from the storeN)r rr/)rr7r8s rdelitemszFSMap.delitemss+  6AD$$Q'676s:ct|ts@tjdtt|t r t |}t|}|j|jdS)zGenerate full path for the keyzAfrom fsspec 2023.5 onward FSMap non-str keys will raise TypeError/) r1strwarningswarnDeprecationWarninglisttuplerrstrip)rr>s rr/zFSMap._key_to_strs]#s# MMS" #t$Cjc(C''(.55c::rcP|t|jdjdS)zStrip path of to leave key nameNrJ)lenr lstrip)rss r _str_to_keyzFSMap._str_to_keys#TYY!"))#..rc|j|} |jj|}|S#|j$r}||cYd}~St ||d}~wwxYw)z Retrieve dataN)r/r r0rr3)rr>defaultr8resultexcs r __getitem__zFSMap.__getitem__s^   S ! )WW[[^F  && )"3-S ( )s 0AAA AAcR|j||} ||=|S#t$rY|SwxYw)zPop data)r[r3)rr>rXrYs rpopz FSMap.pops@!!#w/ S      s  &&c|j|}|jj|jj|d|jj |t |y)zStore value in keyT)exist_okN)r/r mkdirs_parent pipe_filerB)rr>values r __setitem__zFSMap.__setitem__sLs# twws+d; #}U34rc`fdjjjDS)Nc3@K|]}j|ywN)rV).0r rs r z!FSMap.__iter__..sE  #Es)r findr r)s`r__iter__zFSMap.__iter__sETWW\\$))-DEErc^t|jj|jSrg)rSr rjr r)s r__len__z FSMap.__len__s477<< *++rc |jj|j|y#t$r }t|d}~wwxYw)z Remove keyN)r rr/ Exceptionr3)rr>rZs r __delitem__zFSMap.__delitem__s: $ GGJJt'', - $ # $s*- A=AcZ|j|}|jj|S)zDoes key exist in mapping?)r/r isfile)rr>r#s r __contains__zFSMap.__contains__s%$ww~~d##rcXt|j|jdd|jffS)NF)rr r rr)s r __reduce__zFSMap.__reduce__s&tyy$''5%9P9PQQQr)FFN)r,rg)__name__ __module__ __qualname____doc__rrr%r*r@rFrHr/rVr[r]rdrkrmrprsrurrrr sk6$2EE $ L8 ;/ 5 F,$$ Rrrct|tjs t|drIt|dr)|jjdvr|j d}t t|}|S)N __array__dtypeMmint64)r1arrayhasattrr}kindviewr2 memoryview)rcs rrBrBsX%% )D 5' "u{{'7'74'?JJw'Ej'( Lrc Lt|fi|\}}||n|}t|||||S)aCreate key-value interface for given URL and options The URL will be of the form "protocol://location" and point to the root of the mapper required. All keys will be file-names below this location, and their values the contents of each key. Also accepts compound URLs like zip::s3://bucket/file.zip , see ``fsspec.open``. Parameters ---------- url: str Root URL of mapping check: bool Whether to attempt to read from the location before instantiation, to check that the mapping does exist create: bool Whether to make the directory corresponding to the root before instantiating missing_exceptions: None or tuple If given, these exception types will be regarded as missing keys and return KeyError when trying to read data. By default, you get (FileNotFoundError, IsADirectoryError, NotADirectoryError) alternate_root: None or str In cases of complex URLs, the parser may fail to pick the correct part for the mapper root, so this arg can override Returns ------- ``FSMap`` instance, the dict-like key-value store. )r)rr) urlrrralternate_rootkwargsr urlpathr s r get_mapperrs8NC*6*KB+7>WD r5&=O PPr)FFNN)rloggingrrLcollections.abcr functoolsr fsspec.corer getLoggerr'rrBrrzrrrsZ *%!   + ,wRNwRt   )Qr