From a39857e349df63501ff4062cf4ff8965e8b84cac Mon Sep 17 00:00:00 2001 From: CoprDistGit Date: Mon, 15 May 2023 05:22:23 +0000 Subject: automatic import of python-typeddfs --- .gitignore | 1 + python-typeddfs.spec | 850 +++++++++++++++++++++++++++++++++++++++++++++++++++ sources | 1 + 3 files changed, 852 insertions(+) create mode 100644 python-typeddfs.spec create mode 100644 sources diff --git a/.gitignore b/.gitignore index e69de29..ed35750 100644 --- a/.gitignore +++ b/.gitignore @@ -0,0 +1 @@ +/typeddfs-0.16.5.tar.gz diff --git a/python-typeddfs.spec b/python-typeddfs.spec new file mode 100644 index 0000000..f65b6f3 --- /dev/null +++ b/python-typeddfs.spec @@ -0,0 +1,850 @@ +%global _empty_manifest_terminate_build 0 +Name: python-typeddfs +Version: 0.16.5 +Release: 1 +Summary: Pandas DataFrame subclasses that enforce structure and can self-organize. +License: Apache-2.0 +URL: https://github.com/dmyersturnbull/typed-dfs +Source0: https://mirrors.nju.edu.cn/pypi/web/packages/df/69/23b4c90de17493d82dcd65f094b636dc389fc3cd99342a109678bb06e101/typeddfs-0.16.5.tar.gz +BuildArch: noarch + +Requires: python3-natsort +Requires: python3-orjson +Requires: python3-pandas +Requires: python3-regex +Requires: python3-tabulate +Requires: python3-wcwidth +Requires: python3-pyarrow +Requires: python3-lxml +Requires: python3-openpyxl +Requires: python3-pyxlsb +Requires: python3-html5lib +Requires: python3-beautifulsoup4 +Requires: python3-tomlkit + +%description +# Typed DataFrames + +[![Version status](https://img.shields.io/pypi/status/typeddfs?label=status)](https://pypi.org/project/typeddfs) +[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) +[![Python version compatibility](https://img.shields.io/pypi/pyversions/typeddfs?label=Python)](https://pypi.org/project/typeddfs) +[![Version on GitHub](https://img.shields.io/github/v/release/dmyersturnbull/typed-dfs?include_prereleases&label=GitHub)](https://github.com/dmyersturnbull/typed-dfs/releases) +[![Version on PyPi](https://img.shields.io/pypi/v/typeddfs?label=PyPi)](https://pypi.org/project/typeddfs) +[![Build (Actions)](https://img.shields.io/github/workflow/status/dmyersturnbull/typed-dfs/Build%20&%20test?label=Tests)](https://github.com/dmyersturnbull/typed-dfs/actions) +[![Coverage (coveralls)](https://coveralls.io/repos/github/dmyersturnbull/typed-dfs/badge.svg?branch=main&service=github)](https://coveralls.io/github/dmyersturnbull/typed-dfs?branch=main) +[![Documentation status](https://readthedocs.org/projects/typed-dfs/badge)](https://typed-dfs.readthedocs.io/en/stable/) +[![Maintainability](https://api.codeclimate.com/v1/badges/6b804351b6ba5e7694af/maintainability)](https://codeclimate.com/github/dmyersturnbull/typed-dfs/maintainability) +[![Scrutinizer Code Quality](https://scrutinizer-ci.com/g/dmyersturnbull/typed-dfs/badges/quality-score.png?b=main)](https://scrutinizer-ci.com/g/dmyersturnbull/typed-dfs/?branch=main) +[![Created with Tyrannosaurus](https://img.shields.io/badge/Created_with-Tyrannosaurus-0000ff.svg)](https://github.com/dmyersturnbull/tyrannosaurus) + +Pandas DataFrame subclasses that self-organize and serialize robustly. + +```python +Film = TypedDfs.typed("Film").require("name", "studio", "year").build() +df = Film.read_csv("file.csv") +assert df.columns.tolist() == ["name", "studio", "year"] +type(df) # Film +``` + +Your types remember how to be read, +including columns, dtypes, indices, and custom requirements. +No index_cols=, header=, set_index, or astype needed. + +**Read and write any format:** + +```python +path = input("input file? [.csv/.tsv/.tab/.json/.xml.bz2/.feather/.snappy.h5/...]") +df = Film.read_file(path) +df.write_file("output.snappy") +``` + +**Need dataclasses?** + +```python +instances = df.to_dataclass_instances() +Film.from_dataclass_instances(instances) +``` + +**Save metadata?** + +```python +df = df.set_attrs(dataset="piano") +df.write_file("df.csv", attrs=True) +df = Film.read_file("df.csv", attrs=True) +print(df.attrs) # e.g. {"dataset": "piano") +``` + +**Make dirs? Don’t overwrite?** + +```python +df.write_file("df.csv", mkdirs=True, overwrite=False) +``` + +**Write / verify checksums?** + +```python +df.write_file("df.csv", file_hash=True) +df = Film.read_file("df.csv", file_hash=True) # fails if wrong +``` + +**Get example datasets?** + +```python +print(ExampleDfs.penguins().df) +# species island bill_length_mm ... flipper_length_mm body_mass_g sex +# 0 Adelie Torgersen 39.1 ... 181.0 3750.0 MALE +``` + +**Pretty-print the obvious way?** + +```python +df.pretty_print(to="all_data.md.zip") +wiki_txt = df.pretty_print(fmt="mediawiki") +``` + +All standard DataFrame methods remain available. +Use `.of(df)` to convert to your type, or `.vanilla()` for a plain DataFrame. + +**[Read the docs πŸ“š](https://typed-dfs.readthedocs.io/en/stable/)** for more info and examples. + +### πŸ› Pandas serialization bugs fixed + +Pandas has several issues with serialization. + +
+See: Fixed issues +Depending on the format and columns, these issues occur: + +- columns being silently added or dropped, +- errors on either read or write of empty DataFrames, +- the inability to use DataFrames with indices in Feather, +- writing to Parquet failing with half-precision, +- lingering partially written files on error, +- the buggy xlrd being preferred by read_excel, +- the buggy odfpy also being preferred, +- writing a file and reading it back results in a different DataFrame, +- you can’t write fixed-width format, +- and the platform text encoding being used rather than utf-8. +- invalid JSON is written via the built-in json library + +
+ +### 🎁 Other features + +See more in the [guided walkthrough ✏️](https://typed-dfs.readthedocs.io/en/latest/guide.html) + +
+See: Short feature list + +- Dtype-aware natural sorting +- UTF-8 by default +- Near-atomicity of read/write +- Matrix-like typed dataframes and methods (e.g. `matrix.is_symmetric()`) +- DataFrame-compatible frozen, hashable, ordered collections (dict, list, and set) +- Serialize JSON robustly, preserving NaN, inf, βˆ’inf, enums, timezones, complex numbers, etc. +- Serialize more formats like TOML and INI +- Interpreting paths and formats (e.g. `FileFormat.split("dir/myfile.csv.gz").compression # gz`) +- Generate good CLI help text for input DataFrames +- Parse/verify/add/update/delete files in a .shasum-like file + +
+ +### πŸ’” Limitations + +
+See: List of limitations + +- Multi-level columns are not yet supported. +- Columns and index levels cannot share names. +- Duplicate column names are not supported. (These are strange anyway.) +- A typed DF cannot have columns "level_0", "index", or "Unnamed: 0". +- `inplace` is forbidden in some functions; avoid it or use `.vanilla()`. + +
+ +### πŸ”Œ Serialization support + +TypedDfs provides the methods `read_file` and `write_file`, which guess the format from the +filename extension. For example, this will convert a gzipped, tab-delimited file to Feather: + +```python +TastyDf = typeddfs.typed("TastyDf").build() +TastyDf.read_file("myfile.tab.gz").write_file("myfile.feather") +``` + +Pandas does most of the serialization, but some formats require extra packages. +Typed-dfs specifies [extras](https://python-poetry.org/docs/pyproject/#extras) +to help you get required packages and with compatible versions. + +Here are the extras: + +- `feather`: [Feather](https://arrow.apache.org/docs/python/feather.html) (uses: pyarrow) +- `parquet`: [Parquet (e.g. .snappy)](https://github.com/apache/parquet-format) (uses: pyarrow) +- `xml` (uses: lxml) +- `excel`: Excel and LibreOffice .xlsx/.ods/.xls, etc. (uses: openpyxl, defusedxml) +- `toml`: [TOML](https://toml.io/en/) (uses: tomlkit) +- `html` (uses: html5lib, beautifulsoup4) +- `xlsb`: rare binary Excel file (uses: pyxlsb) +- [HDF5](https://www.hdfgroup.org/solutions/hdf5/) _{no extra provided}_ (_use:_ `tables`) + +For example, for Feather and TOML support use: `typeddfs[feather,toml]` +As a shorthand for all formats, use `typeddfs[all]`. + +### πŸ“Š Serialization in-depth + +
+See: Full table + +| format | packages | extra | sanity | speed | file sizes | +| ----------- | ---------------------------- | --------- | ------ | ----- | ---------- | +| Feather | `pyarrow` | `feather` | +++ | ++++ | +++ | +| Parquet | `pyarrow` or `fastparquet` † | `parquet` | ++ | +++ | ++++ | +| csv/tsv | none | none | ++ | βˆ’βˆ’ | βˆ’βˆ’ | +| flexwf ‑ | none | none | ++ | βˆ’βˆ’ | βˆ’βˆ’ | +| .fwf | none | none | + | βˆ’βˆ’ | βˆ’βˆ’ | +| json | none | none | βˆ’βˆ’ | βˆ’βˆ’βˆ’ | βˆ’βˆ’βˆ’ | +| xml | `lxml` | `xml` | βˆ’ | βˆ’βˆ’βˆ’ | βˆ’βˆ’βˆ’ | +| .properties | none | none | βˆ’βˆ’ | βˆ’βˆ’ | βˆ’βˆ’ | +| toml | `tomlkit` | `toml` | βˆ’βˆ’ | βˆ’βˆ’ | βˆ’βˆ’ | +| INI | none | none | βˆ’βˆ’βˆ’ | βˆ’βˆ’ | βˆ’βˆ’ | +| .lines | none | none | ++ | βˆ’βˆ’ | βˆ’βˆ’ | +| .npy | none | none | βˆ’ | + | +++ | +| .npz | none | none | βˆ’ | + | +++ | +| .html | `html5lib,beautifulsoup4` | `html` | βˆ’βˆ’ | βˆ’βˆ’βˆ’ | βˆ’βˆ’βˆ’ | +| pickle | none | none | βˆ’βˆ’ | βˆ’βˆ’βˆ’ | βˆ’βˆ’βˆ’ | +| XLSX | `openpyxl,defusedxml` | `excel` | + | βˆ’βˆ’ | + | +| ODS | `openpyxl,defusedxml` | `excel` | + | βˆ’βˆ’ | + | +| XLS | `openpyxl,defusedxml` | `excel` | βˆ’βˆ’ | βˆ’βˆ’ | + | +| XLSB | `pyxlsb` | `xlsb` | βˆ’βˆ’ | βˆ’βˆ’ | ++ | +| HDF5 | `tables` | `hdf5` | βˆ’βˆ’ | βˆ’ | ++ | + +**⚠ Note:** The `hdf5` extra is currently disabled. + +
+ +
+See: serialization notes + +- † `fastparquet` can be used instead. It is slower but much smaller. +- Parquet only supports str, float64, float32, int64, int32, and bool. + Other numeric types are automatically converted during write. +- ‑ `.flexwf` is fixed-width with optional delimiters. +- JSON has inconsistent handling of `None`. ([orjson](https://github.com/ijl/orjson) is more consistent). +- XML requires Pandas 1.3+. +- Not all JSON, XML, TOML, and HDF5 files can be read. +- .ini and .properties can only be written with exactly 2 columns + index levels: + a key and a value. INI keys are in the form `section.name`. +- .lines can only be written with exactly 1 column or index level. +- .npy and .npz only serialize numpy objects. + They are not supported in `read_file` and `write_file`. +- .html is not supported in `read_file` and `write_file`. +- Pickle is insecure and not recommended. +- Pandas supports odfpy for ODS and xlrd for XLS. In fact, it prefers those. + However, they are very buggy; openpyxl is much better. +- XLSM, XLTX, XLTM, XLS, and XLSB files can contain macros, which Microsoft Excel will ingest. +- XLS is a deprecated format. +- XLSB is not fully supported in Pandas. +- HDF may not work on all platforms yet due to a + [tables issue](https://github.com/PyTables/PyTables/issues/854). + +Feather offers massively better performance over CSV, gzipped CSV, and HDF5 +in read speed, write speed, memory overhead, and compression ratios. +Parquet typically results in smaller file sizes than Feather at some cost in speed. +Feather is the preferred format for most cases. + +
+ +### πŸ”’ Security + +Refer to the [security policy](https://github.com/dmyersturnbull/typed-dfs/blob/main/SECURITY.md). + +### πŸ“ Extra notes + +
+See: Pinned versions + +Dependencies in the extras only have version minimums, not maximums. +For example, typed-dfs requires pyarrow >= 4. +[natsort](https://github.com/SethMMorton/natsort) is also only assigned a minimum version number. +This means that the result of typed-df’s `sort_natural` could change. +To fix this, pin natsort to a specific major version; +e.g. `natsort = "^8"` with [Poetry](https://python-poetry.org/) or `natsort>=8,<9` with pip. + +
+ +### 🍁 Contributing + +Typed-Dfs is licensed under the [Apache License, version 2.0](https://www.apache.org/licenses/LICENSE-2.0). +[New issues](https://github.com/dmyersturnbull/typed-dfs/issues) and pull requests are welcome. +Please refer to the [contributing guide](https://github.com/dmyersturnbull/typed-dfs/blob/main/CONTRIBUTING.md). +Generated with [Tyrannosaurus](https://github.com/dmyersturnbull/tyrannosaurus). + + +%package -n python3-typeddfs +Summary: Pandas DataFrame subclasses that enforce structure and can self-organize. +Provides: python-typeddfs +BuildRequires: python3-devel +BuildRequires: python3-setuptools +BuildRequires: python3-pip +%description -n python3-typeddfs +# Typed DataFrames + +[![Version status](https://img.shields.io/pypi/status/typeddfs?label=status)](https://pypi.org/project/typeddfs) +[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) +[![Python version compatibility](https://img.shields.io/pypi/pyversions/typeddfs?label=Python)](https://pypi.org/project/typeddfs) +[![Version on GitHub](https://img.shields.io/github/v/release/dmyersturnbull/typed-dfs?include_prereleases&label=GitHub)](https://github.com/dmyersturnbull/typed-dfs/releases) +[![Version on PyPi](https://img.shields.io/pypi/v/typeddfs?label=PyPi)](https://pypi.org/project/typeddfs) +[![Build (Actions)](https://img.shields.io/github/workflow/status/dmyersturnbull/typed-dfs/Build%20&%20test?label=Tests)](https://github.com/dmyersturnbull/typed-dfs/actions) +[![Coverage (coveralls)](https://coveralls.io/repos/github/dmyersturnbull/typed-dfs/badge.svg?branch=main&service=github)](https://coveralls.io/github/dmyersturnbull/typed-dfs?branch=main) +[![Documentation status](https://readthedocs.org/projects/typed-dfs/badge)](https://typed-dfs.readthedocs.io/en/stable/) +[![Maintainability](https://api.codeclimate.com/v1/badges/6b804351b6ba5e7694af/maintainability)](https://codeclimate.com/github/dmyersturnbull/typed-dfs/maintainability) +[![Scrutinizer Code Quality](https://scrutinizer-ci.com/g/dmyersturnbull/typed-dfs/badges/quality-score.png?b=main)](https://scrutinizer-ci.com/g/dmyersturnbull/typed-dfs/?branch=main) +[![Created with Tyrannosaurus](https://img.shields.io/badge/Created_with-Tyrannosaurus-0000ff.svg)](https://github.com/dmyersturnbull/tyrannosaurus) + +Pandas DataFrame subclasses that self-organize and serialize robustly. + +```python +Film = TypedDfs.typed("Film").require("name", "studio", "year").build() +df = Film.read_csv("file.csv") +assert df.columns.tolist() == ["name", "studio", "year"] +type(df) # Film +``` + +Your types remember how to be read, +including columns, dtypes, indices, and custom requirements. +No index_cols=, header=, set_index, or astype needed. + +**Read and write any format:** + +```python +path = input("input file? [.csv/.tsv/.tab/.json/.xml.bz2/.feather/.snappy.h5/...]") +df = Film.read_file(path) +df.write_file("output.snappy") +``` + +**Need dataclasses?** + +```python +instances = df.to_dataclass_instances() +Film.from_dataclass_instances(instances) +``` + +**Save metadata?** + +```python +df = df.set_attrs(dataset="piano") +df.write_file("df.csv", attrs=True) +df = Film.read_file("df.csv", attrs=True) +print(df.attrs) # e.g. {"dataset": "piano") +``` + +**Make dirs? Don’t overwrite?** + +```python +df.write_file("df.csv", mkdirs=True, overwrite=False) +``` + +**Write / verify checksums?** + +```python +df.write_file("df.csv", file_hash=True) +df = Film.read_file("df.csv", file_hash=True) # fails if wrong +``` + +**Get example datasets?** + +```python +print(ExampleDfs.penguins().df) +# species island bill_length_mm ... flipper_length_mm body_mass_g sex +# 0 Adelie Torgersen 39.1 ... 181.0 3750.0 MALE +``` + +**Pretty-print the obvious way?** + +```python +df.pretty_print(to="all_data.md.zip") +wiki_txt = df.pretty_print(fmt="mediawiki") +``` + +All standard DataFrame methods remain available. +Use `.of(df)` to convert to your type, or `.vanilla()` for a plain DataFrame. + +**[Read the docs πŸ“š](https://typed-dfs.readthedocs.io/en/stable/)** for more info and examples. + +### πŸ› Pandas serialization bugs fixed + +Pandas has several issues with serialization. + +
+See: Fixed issues +Depending on the format and columns, these issues occur: + +- columns being silently added or dropped, +- errors on either read or write of empty DataFrames, +- the inability to use DataFrames with indices in Feather, +- writing to Parquet failing with half-precision, +- lingering partially written files on error, +- the buggy xlrd being preferred by read_excel, +- the buggy odfpy also being preferred, +- writing a file and reading it back results in a different DataFrame, +- you can’t write fixed-width format, +- and the platform text encoding being used rather than utf-8. +- invalid JSON is written via the built-in json library + +
+ +### 🎁 Other features + +See more in the [guided walkthrough ✏️](https://typed-dfs.readthedocs.io/en/latest/guide.html) + +
+See: Short feature list + +- Dtype-aware natural sorting +- UTF-8 by default +- Near-atomicity of read/write +- Matrix-like typed dataframes and methods (e.g. `matrix.is_symmetric()`) +- DataFrame-compatible frozen, hashable, ordered collections (dict, list, and set) +- Serialize JSON robustly, preserving NaN, inf, βˆ’inf, enums, timezones, complex numbers, etc. +- Serialize more formats like TOML and INI +- Interpreting paths and formats (e.g. `FileFormat.split("dir/myfile.csv.gz").compression # gz`) +- Generate good CLI help text for input DataFrames +- Parse/verify/add/update/delete files in a .shasum-like file + +
+ +### πŸ’” Limitations + +
+See: List of limitations + +- Multi-level columns are not yet supported. +- Columns and index levels cannot share names. +- Duplicate column names are not supported. (These are strange anyway.) +- A typed DF cannot have columns "level_0", "index", or "Unnamed: 0". +- `inplace` is forbidden in some functions; avoid it or use `.vanilla()`. + +
+ +### πŸ”Œ Serialization support + +TypedDfs provides the methods `read_file` and `write_file`, which guess the format from the +filename extension. For example, this will convert a gzipped, tab-delimited file to Feather: + +```python +TastyDf = typeddfs.typed("TastyDf").build() +TastyDf.read_file("myfile.tab.gz").write_file("myfile.feather") +``` + +Pandas does most of the serialization, but some formats require extra packages. +Typed-dfs specifies [extras](https://python-poetry.org/docs/pyproject/#extras) +to help you get required packages and with compatible versions. + +Here are the extras: + +- `feather`: [Feather](https://arrow.apache.org/docs/python/feather.html) (uses: pyarrow) +- `parquet`: [Parquet (e.g. .snappy)](https://github.com/apache/parquet-format) (uses: pyarrow) +- `xml` (uses: lxml) +- `excel`: Excel and LibreOffice .xlsx/.ods/.xls, etc. (uses: openpyxl, defusedxml) +- `toml`: [TOML](https://toml.io/en/) (uses: tomlkit) +- `html` (uses: html5lib, beautifulsoup4) +- `xlsb`: rare binary Excel file (uses: pyxlsb) +- [HDF5](https://www.hdfgroup.org/solutions/hdf5/) _{no extra provided}_ (_use:_ `tables`) + +For example, for Feather and TOML support use: `typeddfs[feather,toml]` +As a shorthand for all formats, use `typeddfs[all]`. + +### πŸ“Š Serialization in-depth + +
+See: Full table + +| format | packages | extra | sanity | speed | file sizes | +| ----------- | ---------------------------- | --------- | ------ | ----- | ---------- | +| Feather | `pyarrow` | `feather` | +++ | ++++ | +++ | +| Parquet | `pyarrow` or `fastparquet` † | `parquet` | ++ | +++ | ++++ | +| csv/tsv | none | none | ++ | βˆ’βˆ’ | βˆ’βˆ’ | +| flexwf ‑ | none | none | ++ | βˆ’βˆ’ | βˆ’βˆ’ | +| .fwf | none | none | + | βˆ’βˆ’ | βˆ’βˆ’ | +| json | none | none | βˆ’βˆ’ | βˆ’βˆ’βˆ’ | βˆ’βˆ’βˆ’ | +| xml | `lxml` | `xml` | βˆ’ | βˆ’βˆ’βˆ’ | βˆ’βˆ’βˆ’ | +| .properties | none | none | βˆ’βˆ’ | βˆ’βˆ’ | βˆ’βˆ’ | +| toml | `tomlkit` | `toml` | βˆ’βˆ’ | βˆ’βˆ’ | βˆ’βˆ’ | +| INI | none | none | βˆ’βˆ’βˆ’ | βˆ’βˆ’ | βˆ’βˆ’ | +| .lines | none | none | ++ | βˆ’βˆ’ | βˆ’βˆ’ | +| .npy | none | none | βˆ’ | + | +++ | +| .npz | none | none | βˆ’ | + | +++ | +| .html | `html5lib,beautifulsoup4` | `html` | βˆ’βˆ’ | βˆ’βˆ’βˆ’ | βˆ’βˆ’βˆ’ | +| pickle | none | none | βˆ’βˆ’ | βˆ’βˆ’βˆ’ | βˆ’βˆ’βˆ’ | +| XLSX | `openpyxl,defusedxml` | `excel` | + | βˆ’βˆ’ | + | +| ODS | `openpyxl,defusedxml` | `excel` | + | βˆ’βˆ’ | + | +| XLS | `openpyxl,defusedxml` | `excel` | βˆ’βˆ’ | βˆ’βˆ’ | + | +| XLSB | `pyxlsb` | `xlsb` | βˆ’βˆ’ | βˆ’βˆ’ | ++ | +| HDF5 | `tables` | `hdf5` | βˆ’βˆ’ | βˆ’ | ++ | + +**⚠ Note:** The `hdf5` extra is currently disabled. + +
+ +
+See: serialization notes + +- † `fastparquet` can be used instead. It is slower but much smaller. +- Parquet only supports str, float64, float32, int64, int32, and bool. + Other numeric types are automatically converted during write. +- ‑ `.flexwf` is fixed-width with optional delimiters. +- JSON has inconsistent handling of `None`. ([orjson](https://github.com/ijl/orjson) is more consistent). +- XML requires Pandas 1.3+. +- Not all JSON, XML, TOML, and HDF5 files can be read. +- .ini and .properties can only be written with exactly 2 columns + index levels: + a key and a value. INI keys are in the form `section.name`. +- .lines can only be written with exactly 1 column or index level. +- .npy and .npz only serialize numpy objects. + They are not supported in `read_file` and `write_file`. +- .html is not supported in `read_file` and `write_file`. +- Pickle is insecure and not recommended. +- Pandas supports odfpy for ODS and xlrd for XLS. In fact, it prefers those. + However, they are very buggy; openpyxl is much better. +- XLSM, XLTX, XLTM, XLS, and XLSB files can contain macros, which Microsoft Excel will ingest. +- XLS is a deprecated format. +- XLSB is not fully supported in Pandas. +- HDF may not work on all platforms yet due to a + [tables issue](https://github.com/PyTables/PyTables/issues/854). + +Feather offers massively better performance over CSV, gzipped CSV, and HDF5 +in read speed, write speed, memory overhead, and compression ratios. +Parquet typically results in smaller file sizes than Feather at some cost in speed. +Feather is the preferred format for most cases. + +
+ +### πŸ”’ Security + +Refer to the [security policy](https://github.com/dmyersturnbull/typed-dfs/blob/main/SECURITY.md). + +### πŸ“ Extra notes + +
+See: Pinned versions + +Dependencies in the extras only have version minimums, not maximums. +For example, typed-dfs requires pyarrow >= 4. +[natsort](https://github.com/SethMMorton/natsort) is also only assigned a minimum version number. +This means that the result of typed-df’s `sort_natural` could change. +To fix this, pin natsort to a specific major version; +e.g. `natsort = "^8"` with [Poetry](https://python-poetry.org/) or `natsort>=8,<9` with pip. + +
+ +### 🍁 Contributing + +Typed-Dfs is licensed under the [Apache License, version 2.0](https://www.apache.org/licenses/LICENSE-2.0). +[New issues](https://github.com/dmyersturnbull/typed-dfs/issues) and pull requests are welcome. +Please refer to the [contributing guide](https://github.com/dmyersturnbull/typed-dfs/blob/main/CONTRIBUTING.md). +Generated with [Tyrannosaurus](https://github.com/dmyersturnbull/tyrannosaurus). + + +%package help +Summary: Development documents and examples for typeddfs +Provides: python3-typeddfs-doc +%description help +# Typed DataFrames + +[![Version status](https://img.shields.io/pypi/status/typeddfs?label=status)](https://pypi.org/project/typeddfs) +[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) +[![Python version compatibility](https://img.shields.io/pypi/pyversions/typeddfs?label=Python)](https://pypi.org/project/typeddfs) +[![Version on GitHub](https://img.shields.io/github/v/release/dmyersturnbull/typed-dfs?include_prereleases&label=GitHub)](https://github.com/dmyersturnbull/typed-dfs/releases) +[![Version on PyPi](https://img.shields.io/pypi/v/typeddfs?label=PyPi)](https://pypi.org/project/typeddfs) +[![Build (Actions)](https://img.shields.io/github/workflow/status/dmyersturnbull/typed-dfs/Build%20&%20test?label=Tests)](https://github.com/dmyersturnbull/typed-dfs/actions) +[![Coverage (coveralls)](https://coveralls.io/repos/github/dmyersturnbull/typed-dfs/badge.svg?branch=main&service=github)](https://coveralls.io/github/dmyersturnbull/typed-dfs?branch=main) +[![Documentation status](https://readthedocs.org/projects/typed-dfs/badge)](https://typed-dfs.readthedocs.io/en/stable/) +[![Maintainability](https://api.codeclimate.com/v1/badges/6b804351b6ba5e7694af/maintainability)](https://codeclimate.com/github/dmyersturnbull/typed-dfs/maintainability) +[![Scrutinizer Code Quality](https://scrutinizer-ci.com/g/dmyersturnbull/typed-dfs/badges/quality-score.png?b=main)](https://scrutinizer-ci.com/g/dmyersturnbull/typed-dfs/?branch=main) +[![Created with Tyrannosaurus](https://img.shields.io/badge/Created_with-Tyrannosaurus-0000ff.svg)](https://github.com/dmyersturnbull/tyrannosaurus) + +Pandas DataFrame subclasses that self-organize and serialize robustly. + +```python +Film = TypedDfs.typed("Film").require("name", "studio", "year").build() +df = Film.read_csv("file.csv") +assert df.columns.tolist() == ["name", "studio", "year"] +type(df) # Film +``` + +Your types remember how to be read, +including columns, dtypes, indices, and custom requirements. +No index_cols=, header=, set_index, or astype needed. + +**Read and write any format:** + +```python +path = input("input file? [.csv/.tsv/.tab/.json/.xml.bz2/.feather/.snappy.h5/...]") +df = Film.read_file(path) +df.write_file("output.snappy") +``` + +**Need dataclasses?** + +```python +instances = df.to_dataclass_instances() +Film.from_dataclass_instances(instances) +``` + +**Save metadata?** + +```python +df = df.set_attrs(dataset="piano") +df.write_file("df.csv", attrs=True) +df = Film.read_file("df.csv", attrs=True) +print(df.attrs) # e.g. {"dataset": "piano") +``` + +**Make dirs? Don’t overwrite?** + +```python +df.write_file("df.csv", mkdirs=True, overwrite=False) +``` + +**Write / verify checksums?** + +```python +df.write_file("df.csv", file_hash=True) +df = Film.read_file("df.csv", file_hash=True) # fails if wrong +``` + +**Get example datasets?** + +```python +print(ExampleDfs.penguins().df) +# species island bill_length_mm ... flipper_length_mm body_mass_g sex +# 0 Adelie Torgersen 39.1 ... 181.0 3750.0 MALE +``` + +**Pretty-print the obvious way?** + +```python +df.pretty_print(to="all_data.md.zip") +wiki_txt = df.pretty_print(fmt="mediawiki") +``` + +All standard DataFrame methods remain available. +Use `.of(df)` to convert to your type, or `.vanilla()` for a plain DataFrame. + +**[Read the docs πŸ“š](https://typed-dfs.readthedocs.io/en/stable/)** for more info and examples. + +### πŸ› Pandas serialization bugs fixed + +Pandas has several issues with serialization. + +
+See: Fixed issues +Depending on the format and columns, these issues occur: + +- columns being silently added or dropped, +- errors on either read or write of empty DataFrames, +- the inability to use DataFrames with indices in Feather, +- writing to Parquet failing with half-precision, +- lingering partially written files on error, +- the buggy xlrd being preferred by read_excel, +- the buggy odfpy also being preferred, +- writing a file and reading it back results in a different DataFrame, +- you can’t write fixed-width format, +- and the platform text encoding being used rather than utf-8. +- invalid JSON is written via the built-in json library + +
+ +### 🎁 Other features + +See more in the [guided walkthrough ✏️](https://typed-dfs.readthedocs.io/en/latest/guide.html) + +
+See: Short feature list + +- Dtype-aware natural sorting +- UTF-8 by default +- Near-atomicity of read/write +- Matrix-like typed dataframes and methods (e.g. `matrix.is_symmetric()`) +- DataFrame-compatible frozen, hashable, ordered collections (dict, list, and set) +- Serialize JSON robustly, preserving NaN, inf, βˆ’inf, enums, timezones, complex numbers, etc. +- Serialize more formats like TOML and INI +- Interpreting paths and formats (e.g. `FileFormat.split("dir/myfile.csv.gz").compression # gz`) +- Generate good CLI help text for input DataFrames +- Parse/verify/add/update/delete files in a .shasum-like file + +
+ +### πŸ’” Limitations + +
+See: List of limitations + +- Multi-level columns are not yet supported. +- Columns and index levels cannot share names. +- Duplicate column names are not supported. (These are strange anyway.) +- A typed DF cannot have columns "level_0", "index", or "Unnamed: 0". +- `inplace` is forbidden in some functions; avoid it or use `.vanilla()`. + +
+ +### πŸ”Œ Serialization support + +TypedDfs provides the methods `read_file` and `write_file`, which guess the format from the +filename extension. For example, this will convert a gzipped, tab-delimited file to Feather: + +```python +TastyDf = typeddfs.typed("TastyDf").build() +TastyDf.read_file("myfile.tab.gz").write_file("myfile.feather") +``` + +Pandas does most of the serialization, but some formats require extra packages. +Typed-dfs specifies [extras](https://python-poetry.org/docs/pyproject/#extras) +to help you get required packages and with compatible versions. + +Here are the extras: + +- `feather`: [Feather](https://arrow.apache.org/docs/python/feather.html) (uses: pyarrow) +- `parquet`: [Parquet (e.g. .snappy)](https://github.com/apache/parquet-format) (uses: pyarrow) +- `xml` (uses: lxml) +- `excel`: Excel and LibreOffice .xlsx/.ods/.xls, etc. (uses: openpyxl, defusedxml) +- `toml`: [TOML](https://toml.io/en/) (uses: tomlkit) +- `html` (uses: html5lib, beautifulsoup4) +- `xlsb`: rare binary Excel file (uses: pyxlsb) +- [HDF5](https://www.hdfgroup.org/solutions/hdf5/) _{no extra provided}_ (_use:_ `tables`) + +For example, for Feather and TOML support use: `typeddfs[feather,toml]` +As a shorthand for all formats, use `typeddfs[all]`. + +### πŸ“Š Serialization in-depth + +
+See: Full table + +| format | packages | extra | sanity | speed | file sizes | +| ----------- | ---------------------------- | --------- | ------ | ----- | ---------- | +| Feather | `pyarrow` | `feather` | +++ | ++++ | +++ | +| Parquet | `pyarrow` or `fastparquet` † | `parquet` | ++ | +++ | ++++ | +| csv/tsv | none | none | ++ | βˆ’βˆ’ | βˆ’βˆ’ | +| flexwf ‑ | none | none | ++ | βˆ’βˆ’ | βˆ’βˆ’ | +| .fwf | none | none | + | βˆ’βˆ’ | βˆ’βˆ’ | +| json | none | none | βˆ’βˆ’ | βˆ’βˆ’βˆ’ | βˆ’βˆ’βˆ’ | +| xml | `lxml` | `xml` | βˆ’ | βˆ’βˆ’βˆ’ | βˆ’βˆ’βˆ’ | +| .properties | none | none | βˆ’βˆ’ | βˆ’βˆ’ | βˆ’βˆ’ | +| toml | `tomlkit` | `toml` | βˆ’βˆ’ | βˆ’βˆ’ | βˆ’βˆ’ | +| INI | none | none | βˆ’βˆ’βˆ’ | βˆ’βˆ’ | βˆ’βˆ’ | +| .lines | none | none | ++ | βˆ’βˆ’ | βˆ’βˆ’ | +| .npy | none | none | βˆ’ | + | +++ | +| .npz | none | none | βˆ’ | + | +++ | +| .html | `html5lib,beautifulsoup4` | `html` | βˆ’βˆ’ | βˆ’βˆ’βˆ’ | βˆ’βˆ’βˆ’ | +| pickle | none | none | βˆ’βˆ’ | βˆ’βˆ’βˆ’ | βˆ’βˆ’βˆ’ | +| XLSX | `openpyxl,defusedxml` | `excel` | + | βˆ’βˆ’ | + | +| ODS | `openpyxl,defusedxml` | `excel` | + | βˆ’βˆ’ | + | +| XLS | `openpyxl,defusedxml` | `excel` | βˆ’βˆ’ | βˆ’βˆ’ | + | +| XLSB | `pyxlsb` | `xlsb` | βˆ’βˆ’ | βˆ’βˆ’ | ++ | +| HDF5 | `tables` | `hdf5` | βˆ’βˆ’ | βˆ’ | ++ | + +**⚠ Note:** The `hdf5` extra is currently disabled. + +
+ +
+See: serialization notes + +- † `fastparquet` can be used instead. It is slower but much smaller. +- Parquet only supports str, float64, float32, int64, int32, and bool. + Other numeric types are automatically converted during write. +- ‑ `.flexwf` is fixed-width with optional delimiters. +- JSON has inconsistent handling of `None`. ([orjson](https://github.com/ijl/orjson) is more consistent). +- XML requires Pandas 1.3+. +- Not all JSON, XML, TOML, and HDF5 files can be read. +- .ini and .properties can only be written with exactly 2 columns + index levels: + a key and a value. INI keys are in the form `section.name`. +- .lines can only be written with exactly 1 column or index level. +- .npy and .npz only serialize numpy objects. + They are not supported in `read_file` and `write_file`. +- .html is not supported in `read_file` and `write_file`. +- Pickle is insecure and not recommended. +- Pandas supports odfpy for ODS and xlrd for XLS. In fact, it prefers those. + However, they are very buggy; openpyxl is much better. +- XLSM, XLTX, XLTM, XLS, and XLSB files can contain macros, which Microsoft Excel will ingest. +- XLS is a deprecated format. +- XLSB is not fully supported in Pandas. +- HDF may not work on all platforms yet due to a + [tables issue](https://github.com/PyTables/PyTables/issues/854). + +Feather offers massively better performance over CSV, gzipped CSV, and HDF5 +in read speed, write speed, memory overhead, and compression ratios. +Parquet typically results in smaller file sizes than Feather at some cost in speed. +Feather is the preferred format for most cases. + +
+ +### πŸ”’ Security + +Refer to the [security policy](https://github.com/dmyersturnbull/typed-dfs/blob/main/SECURITY.md). + +### πŸ“ Extra notes + +
+See: Pinned versions + +Dependencies in the extras only have version minimums, not maximums. +For example, typed-dfs requires pyarrow >= 4. +[natsort](https://github.com/SethMMorton/natsort) is also only assigned a minimum version number. +This means that the result of typed-df’s `sort_natural` could change. +To fix this, pin natsort to a specific major version; +e.g. `natsort = "^8"` with [Poetry](https://python-poetry.org/) or `natsort>=8,<9` with pip. + +
+ +### 🍁 Contributing + +Typed-Dfs is licensed under the [Apache License, version 2.0](https://www.apache.org/licenses/LICENSE-2.0). +[New issues](https://github.com/dmyersturnbull/typed-dfs/issues) and pull requests are welcome. +Please refer to the [contributing guide](https://github.com/dmyersturnbull/typed-dfs/blob/main/CONTRIBUTING.md). +Generated with [Tyrannosaurus](https://github.com/dmyersturnbull/tyrannosaurus). + + +%prep +%autosetup -n typeddfs-0.16.5 + +%build +%py3_build + +%install +%py3_install +install -d -m755 %{buildroot}/%{_pkgdocdir} +if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi +if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi +if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi +if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi +pushd %{buildroot} +if [ -d usr/lib ]; then + find usr/lib -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/lib64 ]; then + find usr/lib64 -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/bin ]; then + find usr/bin -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/sbin ]; then + find usr/sbin -type f -printf "/%h/%f\n" >> filelist.lst +fi +touch doclist.lst +if [ -d usr/share/man ]; then + find usr/share/man -type f -printf "/%h/%f.gz\n" >> doclist.lst +fi +popd +mv %{buildroot}/filelist.lst . +mv %{buildroot}/doclist.lst . + +%files -n python3-typeddfs -f filelist.lst +%dir %{python3_sitelib}/* + +%files help -f doclist.lst +%{_docdir}/* + +%changelog +* Mon May 15 2023 Python_Bot - 0.16.5-1 +- Package Spec generated diff --git a/sources b/sources new file mode 100644 index 0000000..362bf72 --- /dev/null +++ b/sources @@ -0,0 +1 @@ +f97a009aa64c13e52bc8e2307dbb1d64 typeddfs-0.16.5.tar.gz -- cgit v1.2.3