U c(5@s<ddlmZddlmZddlZddlmZddlm Z ddl m Z ddl Z ddlmZmZddlmZdd lmZmZmZmZdd lmZmZmZmZejejejej ej!ej"ej"d Z#ejej$dfej"ej%e fejej&dfejej&dfej ej&dfej'ej%d fej!ej(dfiZ)ej&d ej$dej%diZ*GdddeZ+dS)) annotations)AnyN) infer_dtype)iNaT)cache_readonly)is_categorical_dtypeis_string_dtype) PandasBuffer)Column ColumnBuffersColumnNullType DtypeKind) ArrowCTypes EndiannessNoBufferPresentdtype_to_arrow_c_fmt)iufbUMmzThis column is non-nullablezThis column uses NaN as nullz!This column uses a sentinel valuec@seZdZdZd.ddddddZd d d d Zed d d dZedd ddZ dd ddZ eddZ eddZ ed d ddZ edd ddZd d ddZd/d d!d"d#Zd$d d%d&Zd'd d(d)Zd'd d*d+Zd'd d,d-ZdS)0 PandasColumna A column object, with only the methods and properties required by the interchange protocol defined. A column can contain one or more chunks. Each chunk can contain up to three buffers - a data buffer, a mask buffer (depending on null representation), and an offsets buffer (if variable-size binary; e.g., variable-length strings). Note: this Column object can only be produced by ``__dataframe__``, so doesn't need its own version or ``__column__`` protocol. Tz pd.SeriesboolNone)column allow_copyreturncCs0t|tjs tdt|d||_||_dS)zu Note: doesn't deal with extension arrays yet, just assume a regular Series/ndarray for now. zColumns of type  not handled yetN) isinstancepdSeriesNotImplementedErrortype_col _allow_copy)selfrrr)B/tmp/pip-unpacked-wheel-g7fro6k3/pandas/core/interchange/column.py__init__Hs zPandasColumn.__init__int)rcCs|jjS)z2 Size of the column, in elements. )r&sizer(r)r)r*r-TszPandasColumn.sizecCsdS)z7 Offset of first element. Always zero. rr)r.r)r)r*offsetZszPandasColumn.offsetztuple[DtypeKind, int, str, str]cCs|jj}t|r>|jjj}||j\}}}}tj||tj fSt |rrt |jdkrhtj dt |tj fStdn ||SdS)Nstringz.Non-string object dtypes are not supported yet)r&dtypervaluescodes_dtype_from_pandasdtyper CATEGORICALrNATIVErrSTRINGrr$)r(r2r4_ZbitwidthZc_arrow_dtype_f_strr)r)r*r2bs.   zPandasColumn.dtypecCs>t|jd}|dkr&td|d||jdt||jfS)z/ See `self.dtype` for details. N Data type z& not supported by interchange protocolr1) _NP_KINDSgetkind ValueErroritemsizer byteorder)r(r2r=r)r)r*r5sz$PandasColumn._dtype_from_pandasdtypecCs:|jdtjkstd|jjjdtt |jjj dS)a: If the dtype is categorical, there are two options: - There are only values in the data buffer. - There is a separate non-categorical Column encoding for categorical values. Raises TypeError if the dtype is not categorical Content of returned dict: - "is_ordered" : bool, whether the ordering of dictionary indices is semantically meaningful. - "is_dictionary" : bool, whether a dictionary-style mapping of categorical values to other objects exists - "categories" : Column representing the (implicit) mapping of indices to category values (e.g. an array of cat1, cat2, ...). None if not a dictionary-style categorical. rzCdescribe_categorical only works on a column with categorical dtype!T)Z is_orderedZ is_dictionary categories) r2r r6 TypeErrorr&catZorderedrr"r#rAr.r)r)r*describe_categoricalsz!PandasColumn.describe_categoricalcCsH|jd}zt|\}}Wn$tk r>td|dYnX||fS)Nrr:z not yet supported)r2_NULL_DESCRIPTIONKeyErrorr$)r(r=nullvaluer)r)r* describe_nulls  zPandasColumn.describe_nullcCs|jS)zB Number of null elements. Should always be known. )r&Zisnasumitemr.r)r)r* null_countszPandasColumn.null_countzdict[str, pd.Index]cCs d|jjiS)z8 Store specific metadata of the column. z pandas.index)r&indexr.r)r)r*metadataszPandasColumn.metadatacCsdS)zE Return the number of chunks the column consists of. r)r.r)r)r* num_chunksszPandasColumn.num_chunksNz int | None)n_chunksccsr|rh|dkrht|j}||}||dkr2|d7}td|||D]"}t|jj||||jVqBn|VdS)zy Return an iterator yielding the chunks. See `DataFrame.get_chunks` for details on ``n_chunks``. rOrN)lenr&rangerZilocr')r(rQr-stepstartr)r)r* get_chunkss    zPandasColumn.get_chunksr cCs`|ddd}z||d<Wntk r4YnXz||d<Wntk rZYnX|S)a` Return a dictionary containing the underlying buffers. The returned dictionary has the following contents: - "data": a two-element tuple whose first element is a buffer containing the data and whose second element is the data buffer's associated dtype. - "validity": a two-element tuple whose first element is a buffer containing mask values indicating missing data and whose second element is the mask value buffer's associated dtype. None if the null representation is not a bit or byte mask. - "offsets": a two-element tuple whose first element is a buffer containing the offset values for variable-size binary data (e.g., variable-length strings) and whose second element is the offsets buffer's associated dtype. None if the data buffer does not have an associated offsets buffer. N)datavalidityoffsetsrXrY)_get_data_buffer_get_validity_bufferr_get_offsets_buffer)r(buffersr)r)r* get_buffersszPandasColumn.get_buffersztuple[PandasBuffer, Any]cCs|jdtjtjtjtjtjfkr>t|j |j d}|j}n|jdtj krt|jj j }t||j d}||j}n|jdtjkr|j }t}|D] }t|tr||jddqttj|dd}tjdtjtjf}ntd|jjd ||fS) zZ Return the buffer containing the data and the buffer's associated dtype. r)rutf-8encodingZuint8)r2r1r:r )r2r INTUINTFLOATBOOLDATETIMEr r&to_numpyr'r6r3_codesr5r8 bytearrayr!strextendencodenpZ frombufferrrr7r$)r(bufferr2r4bufrobjr)r)r*rZs6   zPandasColumn._get_data_bufferc Cs|j\}}|jdtjkr|j}|dk}| }tjt|ftj d}t |D]\}}t |t rf|n|||<qPt |}tjdtjtjf} || fSzt|d} Wntk rtdYnXt| dS)z Return the buffer containing the mask values indicating missing data and the buffer's associated dtype. Raises NoBufferPresent if null representation is not a bit or byte mask. rshaper2r1z! so does not have a separate maskzSee self.describe_nullN)rIr2r r8r&rgrmzerosrRZbool_ enumerater!rjr rerrr7_NO_VALIDITY_BUFFERrFr$r) r(rGinvalidroZvalidmaskrrprnr2msgr)r)r*r[-s   z!PandasColumn._get_validity_bufferc Cs|jdtjkr|j}d}tjt|dftjd}t |D]6\}}t |t rj|j dd}|t|7}|||d<q@t |}tjdtjtjf}ntd||fS)a Return the buffer containing the offset values for variable-size binary data (e.g., variable-length strings) and the buffer's associated dtype. Raises NoBufferPresent if the data buffer does not have an associated offsets buffer. rrOrqr_r`@zJThis column has a fixed-length dtype so it does not have an offsets buffer)r2r r8r&rgrmrsrRZint64rtr!rjrlr rbrZINT64rr7r) r(r3ptrrYrvrrnr2r)r)r*r\Ss&    z PandasColumn._get_offsets_buffer)T)N)__name__ __module__ __qualname____doc__r+r-propertyr/rr2r5rDrIrLrNrPrVr^rZr[r\r)r)r)r*r<s,    %-&r), __future__rtypingrZnumpyrmZpandas._libs.librZpandas._libs.tslibsrZpandas.util._decoratorsrZpandasr"Zpandas.api.typesrrZpandas.core.interchange.bufferr Z*pandas.core.interchange.dataframe_protocolr r r r Zpandas.core.interchange.utilsrrrrrbrcrdrer8rfr;ZUSE_NANZ USE_SENTINELZ NON_NULLABLEr6Z USE_BYTEMASKrErurr)r)r)r*sR