diff options
| author | CoprDistGit <infra@openeuler.org> | 2023-05-15 05:22:23 +0000 |
|---|---|---|
| committer | CoprDistGit <infra@openeuler.org> | 2023-05-15 05:22:23 +0000 |
| commit | a39857e349df63501ff4062cf4ff8965e8b84cac (patch) | |
| tree | 373d80bd796d225fc7397ef0b07ba795ad2f2952 | |
| parent | 1c6d237238606f347c3e6793f70f04951b6a243c (diff) | |
automatic import of python-typeddfs
| -rw-r--r-- | .gitignore | 1 | ||||
| -rw-r--r-- | python-typeddfs.spec | 850 | ||||
| -rw-r--r-- | sources | 1 |
3 files changed, 852 insertions, 0 deletions
@@ -0,0 +1 @@ +/typeddfs-0.16.5.tar.gz diff --git a/python-typeddfs.spec b/python-typeddfs.spec new file mode 100644 index 0000000..f65b6f3 --- /dev/null +++ b/python-typeddfs.spec @@ -0,0 +1,850 @@ +%global _empty_manifest_terminate_build 0 +Name: python-typeddfs +Version: 0.16.5 +Release: 1 +Summary: Pandas DataFrame subclasses that enforce structure and can self-organize. +License: Apache-2.0 +URL: https://github.com/dmyersturnbull/typed-dfs +Source0: https://mirrors.nju.edu.cn/pypi/web/packages/df/69/23b4c90de17493d82dcd65f094b636dc389fc3cd99342a109678bb06e101/typeddfs-0.16.5.tar.gz +BuildArch: noarch + +Requires: python3-natsort +Requires: python3-orjson +Requires: python3-pandas +Requires: python3-regex +Requires: python3-tabulate +Requires: python3-wcwidth +Requires: python3-pyarrow +Requires: python3-lxml +Requires: python3-openpyxl +Requires: python3-pyxlsb +Requires: python3-html5lib +Requires: python3-beautifulsoup4 +Requires: python3-tomlkit + +%description +# Typed DataFrames + +[](https://pypi.org/project/typeddfs) +[](https://opensource.org/licenses/Apache-2.0) +[](https://pypi.org/project/typeddfs) +[](https://github.com/dmyersturnbull/typed-dfs/releases) +[](https://pypi.org/project/typeddfs) +[](https://github.com/dmyersturnbull/typed-dfs/actions) +[](https://coveralls.io/github/dmyersturnbull/typed-dfs?branch=main) +[](https://typed-dfs.readthedocs.io/en/stable/) +[](https://codeclimate.com/github/dmyersturnbull/typed-dfs/maintainability) +[](https://scrutinizer-ci.com/g/dmyersturnbull/typed-dfs/?branch=main) +[](https://github.com/dmyersturnbull/tyrannosaurus) + +Pandas DataFrame subclasses that self-organize and serialize robustly. + +```python +Film = TypedDfs.typed("Film").require("name", "studio", "year").build() +df = Film.read_csv("file.csv") +assert df.columns.tolist() == ["name", "studio", "year"] +type(df) # Film +``` + +Your types remember how to be read, +including columns, dtypes, indices, and custom requirements. +No index_cols=, header=, set_index, or astype needed. + +**Read and write any format:** + +```python +path = input("input file? [.csv/.tsv/.tab/.json/.xml.bz2/.feather/.snappy.h5/...]") +df = Film.read_file(path) +df.write_file("output.snappy") +``` + +**Need dataclasses?** + +```python +instances = df.to_dataclass_instances() +Film.from_dataclass_instances(instances) +``` + +**Save metadata?** + +```python +df = df.set_attrs(dataset="piano") +df.write_file("df.csv", attrs=True) +df = Film.read_file("df.csv", attrs=True) +print(df.attrs) # e.g. {"dataset": "piano") +``` + +**Make dirs? Donβt overwrite?** + +```python +df.write_file("df.csv", mkdirs=True, overwrite=False) +``` + +**Write / verify checksums?** + +```python +df.write_file("df.csv", file_hash=True) +df = Film.read_file("df.csv", file_hash=True) # fails if wrong +``` + +**Get example datasets?** + +```python +print(ExampleDfs.penguins().df) +# species island bill_length_mm ... flipper_length_mm body_mass_g sex +# 0 Adelie Torgersen 39.1 ... 181.0 3750.0 MALE +``` + +**Pretty-print the obvious way?** + +```python +df.pretty_print(to="all_data.md.zip") +wiki_txt = df.pretty_print(fmt="mediawiki") +``` + +All standard DataFrame methods remain available. +Use `.of(df)` to convert to your type, or `.vanilla()` for a plain DataFrame. + +**[Read the docs π](https://typed-dfs.readthedocs.io/en/stable/)** for more info and examples. + +### π Pandas serialization bugs fixed + +Pandas has several issues with serialization. + +<details> +<summary><em>See: Fixed issues</em></summary> +Depending on the format and columns, these issues occur: + +- columns being silently added or dropped, +- errors on either read or write of empty DataFrames, +- the inability to use DataFrames with indices in Feather, +- writing to Parquet failing with half-precision, +- lingering partially written files on error, +- the buggy xlrd being preferred by read_excel, +- the buggy odfpy also being preferred, +- writing a file and reading it back results in a different DataFrame, +- you canβt write fixed-width format, +- and the platform text encoding being used rather than utf-8. +- invalid JSON is written via the built-in json library + +</details> + +### π Other features + +See more in the [guided walkthrough βοΈ](https://typed-dfs.readthedocs.io/en/latest/guide.html) + +<details> +<summary><em>See: Short feature list</em></summary> + +- Dtype-aware natural sorting +- UTF-8 by default +- Near-atomicity of read/write +- Matrix-like typed dataframes and methods (e.g. `matrix.is_symmetric()`) +- DataFrame-compatible frozen, hashable, ordered collections (dict, list, and set) +- Serialize JSON robustly, preserving NaN, inf, βinf, enums, timezones, complex numbers, etc. +- Serialize more formats like TOML and INI +- Interpreting paths and formats (e.g. `FileFormat.split("dir/myfile.csv.gz").compression # gz`) +- Generate good CLI help text for input DataFrames +- Parse/verify/add/update/delete files in a .shasum-like file + +</details> + +### π Limitations + +<details> +<summary><em>See: List of limitations</em></summary> + +- Multi-level columns are not yet supported. +- Columns and index levels cannot share names. +- Duplicate column names are not supported. (These are strange anyway.) +- A typed DF cannot have columns "level_0", "index", or "Unnamed: 0". +- `inplace` is forbidden in some functions; avoid it or use `.vanilla()`. + +</details> + +### π Serialization support + +TypedDfs provides the methods `read_file` and `write_file`, which guess the format from the +filename extension. For example, this will convert a gzipped, tab-delimited file to Feather: + +```python +TastyDf = typeddfs.typed("TastyDf").build() +TastyDf.read_file("myfile.tab.gz").write_file("myfile.feather") +``` + +Pandas does most of the serialization, but some formats require extra packages. +Typed-dfs specifies [extras](https://python-poetry.org/docs/pyproject/#extras) +to help you get required packages and with compatible versions. + +Here are the extras: + +- `feather`: [Feather](https://arrow.apache.org/docs/python/feather.html) (uses: pyarrow) +- `parquet`: [Parquet (e.g. .snappy)](https://github.com/apache/parquet-format) (uses: pyarrow) +- `xml` (uses: lxml) +- `excel`: Excel and LibreOffice .xlsx/.ods/.xls, etc. (uses: openpyxl, defusedxml) +- `toml`: [TOML](https://toml.io/en/) (uses: tomlkit) +- `html` (uses: html5lib, beautifulsoup4) +- `xlsb`: rare binary Excel file (uses: pyxlsb) +- [HDF5](https://www.hdfgroup.org/solutions/hdf5/) _{no extra provided}_ (_use:_ `tables`) + +For example, for Feather and TOML support use: `typeddfs[feather,toml]` +As a shorthand for all formats, use `typeddfs[all]`. + +### π Serialization in-depth + +<details> +<summary><em>See: Full table</em></summary> + +| format | packages | extra | sanity | speed | file sizes | +| ----------- | ---------------------------- | --------- | ------ | ----- | ---------- | +| Feather | `pyarrow` | `feather` | +++ | ++++ | +++ | +| Parquet | `pyarrow` or `fastparquet` β | `parquet` | ++ | +++ | ++++ | +| csv/tsv | none | none | ++ | ββ | ββ | +| flexwf β‘ | none | none | ++ | ββ | ββ | +| .fwf | none | none | + | ββ | ββ | +| json | none | none | ββ | βββ | βββ | +| xml | `lxml` | `xml` | β | βββ | βββ | +| .properties | none | none | ββ | ββ | ββ | +| toml | `tomlkit` | `toml` | ββ | ββ | ββ | +| INI | none | none | βββ | ββ | ββ | +| .lines | none | none | ++ | ββ | ββ | +| .npy | none | none | β | + | +++ | +| .npz | none | none | β | + | +++ | +| .html | `html5lib,beautifulsoup4` | `html` | ββ | βββ | βββ | +| pickle | none | none | ββ | βββ | βββ | +| XLSX | `openpyxl,defusedxml` | `excel` | + | ββ | + | +| ODS | `openpyxl,defusedxml` | `excel` | + | ββ | + | +| XLS | `openpyxl,defusedxml` | `excel` | ββ | ββ | + | +| XLSB | `pyxlsb` | `xlsb` | ββ | ββ | ++ | +| HDF5 | `tables` | `hdf5` | ββ | β | ++ | + +**β Note:** The `hdf5` extra is currently disabled. + +</details> + +<details> +<summary><em>See: serialization notes</em></summary> + +- β `fastparquet` can be used instead. It is slower but much smaller. +- Parquet only supports str, float64, float32, int64, int32, and bool. + Other numeric types are automatically converted during write. +- β‘ `.flexwf` is fixed-width with optional delimiters. +- JSON has inconsistent handling of `None`. ([orjson](https://github.com/ijl/orjson) is more consistent). +- XML requires Pandas 1.3+. +- Not all JSON, XML, TOML, and HDF5 files can be read. +- .ini and .properties can only be written with exactly 2 columns + index levels: + a key and a value. INI keys are in the form `section.name`. +- .lines can only be written with exactly 1 column or index level. +- .npy and .npz only serialize numpy objects. + They are not supported in `read_file` and `write_file`. +- .html is not supported in `read_file` and `write_file`. +- Pickle is insecure and not recommended. +- Pandas supports odfpy for ODS and xlrd for XLS. In fact, it prefers those. + However, they are very buggy; openpyxl is much better. +- XLSM, XLTX, XLTM, XLS, and XLSB files can contain macros, which Microsoft Excel will ingest. +- XLS is a deprecated format. +- XLSB is not fully supported in Pandas. +- HDF may not work on all platforms yet due to a + [tables issue](https://github.com/PyTables/PyTables/issues/854). + +Feather offers massively better performance over CSV, gzipped CSV, and HDF5 +in read speed, write speed, memory overhead, and compression ratios. +Parquet typically results in smaller file sizes than Feather at some cost in speed. +Feather is the preferred format for most cases. + +</details> + +### π Security + +Refer to the [security policy](https://github.com/dmyersturnbull/typed-dfs/blob/main/SECURITY.md). + +### π Extra notes + +<details> +<summary><em>See: Pinned versions</em></summary> + +Dependencies in the extras only have version minimums, not maximums. +For example, typed-dfs requires pyarrow >= 4. +[natsort](https://github.com/SethMMorton/natsort) is also only assigned a minimum version number. +This means that the result of typed-dfβs `sort_natural` could change. +To fix this, pin natsort to a specific major version; +e.g. `natsort = "^8"` with [Poetry](https://python-poetry.org/) or `natsort>=8,<9` with pip. + +</details> + +### π Contributing + +Typed-Dfs is licensed under the [Apache License, version 2.0](https://www.apache.org/licenses/LICENSE-2.0). +[New issues](https://github.com/dmyersturnbull/typed-dfs/issues) and pull requests are welcome. +Please refer to the [contributing guide](https://github.com/dmyersturnbull/typed-dfs/blob/main/CONTRIBUTING.md). +Generated with [Tyrannosaurus](https://github.com/dmyersturnbull/tyrannosaurus). + + +%package -n python3-typeddfs +Summary: Pandas DataFrame subclasses that enforce structure and can self-organize. +Provides: python-typeddfs +BuildRequires: python3-devel +BuildRequires: python3-setuptools +BuildRequires: python3-pip +%description -n python3-typeddfs +# Typed DataFrames + +[](https://pypi.org/project/typeddfs) +[](https://opensource.org/licenses/Apache-2.0) +[](https://pypi.org/project/typeddfs) +[](https://github.com/dmyersturnbull/typed-dfs/releases) +[](https://pypi.org/project/typeddfs) +[](https://github.com/dmyersturnbull/typed-dfs/actions) +[](https://coveralls.io/github/dmyersturnbull/typed-dfs?branch=main) +[](https://typed-dfs.readthedocs.io/en/stable/) +[](https://codeclimate.com/github/dmyersturnbull/typed-dfs/maintainability) +[](https://scrutinizer-ci.com/g/dmyersturnbull/typed-dfs/?branch=main) +[](https://github.com/dmyersturnbull/tyrannosaurus) + +Pandas DataFrame subclasses that self-organize and serialize robustly. + +```python +Film = TypedDfs.typed("Film").require("name", "studio", "year").build() +df = Film.read_csv("file.csv") +assert df.columns.tolist() == ["name", "studio", "year"] +type(df) # Film +``` + +Your types remember how to be read, +including columns, dtypes, indices, and custom requirements. +No index_cols=, header=, set_index, or astype needed. + +**Read and write any format:** + +```python +path = input("input file? [.csv/.tsv/.tab/.json/.xml.bz2/.feather/.snappy.h5/...]") +df = Film.read_file(path) +df.write_file("output.snappy") +``` + +**Need dataclasses?** + +```python +instances = df.to_dataclass_instances() +Film.from_dataclass_instances(instances) +``` + +**Save metadata?** + +```python +df = df.set_attrs(dataset="piano") +df.write_file("df.csv", attrs=True) +df = Film.read_file("df.csv", attrs=True) +print(df.attrs) # e.g. {"dataset": "piano") +``` + +**Make dirs? Donβt overwrite?** + +```python +df.write_file("df.csv", mkdirs=True, overwrite=False) +``` + +**Write / verify checksums?** + +```python +df.write_file("df.csv", file_hash=True) +df = Film.read_file("df.csv", file_hash=True) # fails if wrong +``` + +**Get example datasets?** + +```python +print(ExampleDfs.penguins().df) +# species island bill_length_mm ... flipper_length_mm body_mass_g sex +# 0 Adelie Torgersen 39.1 ... 181.0 3750.0 MALE +``` + +**Pretty-print the obvious way?** + +```python +df.pretty_print(to="all_data.md.zip") +wiki_txt = df.pretty_print(fmt="mediawiki") +``` + +All standard DataFrame methods remain available. +Use `.of(df)` to convert to your type, or `.vanilla()` for a plain DataFrame. + +**[Read the docs π](https://typed-dfs.readthedocs.io/en/stable/)** for more info and examples. + +### π Pandas serialization bugs fixed + +Pandas has several issues with serialization. + +<details> +<summary><em>See: Fixed issues</em></summary> +Depending on the format and columns, these issues occur: + +- columns being silently added or dropped, +- errors on either read or write of empty DataFrames, +- the inability to use DataFrames with indices in Feather, +- writing to Parquet failing with half-precision, +- lingering partially written files on error, +- the buggy xlrd being preferred by read_excel, +- the buggy odfpy also being preferred, +- writing a file and reading it back results in a different DataFrame, +- you canβt write fixed-width format, +- and the platform text encoding being used rather than utf-8. +- invalid JSON is written via the built-in json library + +</details> + +### π Other features + +See more in the [guided walkthrough βοΈ](https://typed-dfs.readthedocs.io/en/latest/guide.html) + +<details> +<summary><em>See: Short feature list</em></summary> + +- Dtype-aware natural sorting +- UTF-8 by default +- Near-atomicity of read/write +- Matrix-like typed dataframes and methods (e.g. `matrix.is_symmetric()`) +- DataFrame-compatible frozen, hashable, ordered collections (dict, list, and set) +- Serialize JSON robustly, preserving NaN, inf, βinf, enums, timezones, complex numbers, etc. +- Serialize more formats like TOML and INI +- Interpreting paths and formats (e.g. `FileFormat.split("dir/myfile.csv.gz").compression # gz`) +- Generate good CLI help text for input DataFrames +- Parse/verify/add/update/delete files in a .shasum-like file + +</details> + +### π Limitations + +<details> +<summary><em>See: List of limitations</em></summary> + +- Multi-level columns are not yet supported. +- Columns and index levels cannot share names. +- Duplicate column names are not supported. (These are strange anyway.) +- A typed DF cannot have columns "level_0", "index", or "Unnamed: 0". +- `inplace` is forbidden in some functions; avoid it or use `.vanilla()`. + +</details> + +### π Serialization support + +TypedDfs provides the methods `read_file` and `write_file`, which guess the format from the +filename extension. For example, this will convert a gzipped, tab-delimited file to Feather: + +```python +TastyDf = typeddfs.typed("TastyDf").build() +TastyDf.read_file("myfile.tab.gz").write_file("myfile.feather") +``` + +Pandas does most of the serialization, but some formats require extra packages. +Typed-dfs specifies [extras](https://python-poetry.org/docs/pyproject/#extras) +to help you get required packages and with compatible versions. + +Here are the extras: + +- `feather`: [Feather](https://arrow.apache.org/docs/python/feather.html) (uses: pyarrow) +- `parquet`: [Parquet (e.g. .snappy)](https://github.com/apache/parquet-format) (uses: pyarrow) +- `xml` (uses: lxml) +- `excel`: Excel and LibreOffice .xlsx/.ods/.xls, etc. (uses: openpyxl, defusedxml) +- `toml`: [TOML](https://toml.io/en/) (uses: tomlkit) +- `html` (uses: html5lib, beautifulsoup4) +- `xlsb`: rare binary Excel file (uses: pyxlsb) +- [HDF5](https://www.hdfgroup.org/solutions/hdf5/) _{no extra provided}_ (_use:_ `tables`) + +For example, for Feather and TOML support use: `typeddfs[feather,toml]` +As a shorthand for all formats, use `typeddfs[all]`. + +### π Serialization in-depth + +<details> +<summary><em>See: Full table</em></summary> + +| format | packages | extra | sanity | speed | file sizes | +| ----------- | ---------------------------- | --------- | ------ | ----- | ---------- | +| Feather | `pyarrow` | `feather` | +++ | ++++ | +++ | +| Parquet | `pyarrow` or `fastparquet` β | `parquet` | ++ | +++ | ++++ | +| csv/tsv | none | none | ++ | ββ | ββ | +| flexwf β‘ | none | none | ++ | ββ | ββ | +| .fwf | none | none | + | ββ | ββ | +| json | none | none | ββ | βββ | βββ | +| xml | `lxml` | `xml` | β | βββ | βββ | +| .properties | none | none | ββ | ββ | ββ | +| toml | `tomlkit` | `toml` | ββ | ββ | ββ | +| INI | none | none | βββ | ββ | ββ | +| .lines | none | none | ++ | ββ | ββ | +| .npy | none | none | β | + | +++ | +| .npz | none | none | β | + | +++ | +| .html | `html5lib,beautifulsoup4` | `html` | ββ | βββ | βββ | +| pickle | none | none | ββ | βββ | βββ | +| XLSX | `openpyxl,defusedxml` | `excel` | + | ββ | + | +| ODS | `openpyxl,defusedxml` | `excel` | + | ββ | + | +| XLS | `openpyxl,defusedxml` | `excel` | ββ | ββ | + | +| XLSB | `pyxlsb` | `xlsb` | ββ | ββ | ++ | +| HDF5 | `tables` | `hdf5` | ββ | β | ++ | + +**β Note:** The `hdf5` extra is currently disabled. + +</details> + +<details> +<summary><em>See: serialization notes</em></summary> + +- β `fastparquet` can be used instead. It is slower but much smaller. +- Parquet only supports str, float64, float32, int64, int32, and bool. + Other numeric types are automatically converted during write. +- β‘ `.flexwf` is fixed-width with optional delimiters. +- JSON has inconsistent handling of `None`. ([orjson](https://github.com/ijl/orjson) is more consistent). +- XML requires Pandas 1.3+. +- Not all JSON, XML, TOML, and HDF5 files can be read. +- .ini and .properties can only be written with exactly 2 columns + index levels: + a key and a value. INI keys are in the form `section.name`. +- .lines can only be written with exactly 1 column or index level. +- .npy and .npz only serialize numpy objects. + They are not supported in `read_file` and `write_file`. +- .html is not supported in `read_file` and `write_file`. +- Pickle is insecure and not recommended. +- Pandas supports odfpy for ODS and xlrd for XLS. In fact, it prefers those. + However, they are very buggy; openpyxl is much better. +- XLSM, XLTX, XLTM, XLS, and XLSB files can contain macros, which Microsoft Excel will ingest. +- XLS is a deprecated format. +- XLSB is not fully supported in Pandas. +- HDF may not work on all platforms yet due to a + [tables issue](https://github.com/PyTables/PyTables/issues/854). + +Feather offers massively better performance over CSV, gzipped CSV, and HDF5 +in read speed, write speed, memory overhead, and compression ratios. +Parquet typically results in smaller file sizes than Feather at some cost in speed. +Feather is the preferred format for most cases. + +</details> + +### π Security + +Refer to the [security policy](https://github.com/dmyersturnbull/typed-dfs/blob/main/SECURITY.md). + +### π Extra notes + +<details> +<summary><em>See: Pinned versions</em></summary> + +Dependencies in the extras only have version minimums, not maximums. +For example, typed-dfs requires pyarrow >= 4. +[natsort](https://github.com/SethMMorton/natsort) is also only assigned a minimum version number. +This means that the result of typed-dfβs `sort_natural` could change. +To fix this, pin natsort to a specific major version; +e.g. `natsort = "^8"` with [Poetry](https://python-poetry.org/) or `natsort>=8,<9` with pip. + +</details> + +### π Contributing + +Typed-Dfs is licensed under the [Apache License, version 2.0](https://www.apache.org/licenses/LICENSE-2.0). +[New issues](https://github.com/dmyersturnbull/typed-dfs/issues) and pull requests are welcome. +Please refer to the [contributing guide](https://github.com/dmyersturnbull/typed-dfs/blob/main/CONTRIBUTING.md). +Generated with [Tyrannosaurus](https://github.com/dmyersturnbull/tyrannosaurus). + + +%package help +Summary: Development documents and examples for typeddfs +Provides: python3-typeddfs-doc +%description help +# Typed DataFrames + +[](https://pypi.org/project/typeddfs) +[](https://opensource.org/licenses/Apache-2.0) +[](https://pypi.org/project/typeddfs) +[](https://github.com/dmyersturnbull/typed-dfs/releases) +[](https://pypi.org/project/typeddfs) +[](https://github.com/dmyersturnbull/typed-dfs/actions) +[](https://coveralls.io/github/dmyersturnbull/typed-dfs?branch=main) +[](https://typed-dfs.readthedocs.io/en/stable/) +[](https://codeclimate.com/github/dmyersturnbull/typed-dfs/maintainability) +[](https://scrutinizer-ci.com/g/dmyersturnbull/typed-dfs/?branch=main) +[](https://github.com/dmyersturnbull/tyrannosaurus) + +Pandas DataFrame subclasses that self-organize and serialize robustly. + +```python +Film = TypedDfs.typed("Film").require("name", "studio", "year").build() +df = Film.read_csv("file.csv") +assert df.columns.tolist() == ["name", "studio", "year"] +type(df) # Film +``` + +Your types remember how to be read, +including columns, dtypes, indices, and custom requirements. +No index_cols=, header=, set_index, or astype needed. + +**Read and write any format:** + +```python +path = input("input file? [.csv/.tsv/.tab/.json/.xml.bz2/.feather/.snappy.h5/...]") +df = Film.read_file(path) +df.write_file("output.snappy") +``` + +**Need dataclasses?** + +```python +instances = df.to_dataclass_instances() +Film.from_dataclass_instances(instances) +``` + +**Save metadata?** + +```python +df = df.set_attrs(dataset="piano") +df.write_file("df.csv", attrs=True) +df = Film.read_file("df.csv", attrs=True) +print(df.attrs) # e.g. {"dataset": "piano") +``` + +**Make dirs? Donβt overwrite?** + +```python +df.write_file("df.csv", mkdirs=True, overwrite=False) +``` + +**Write / verify checksums?** + +```python +df.write_file("df.csv", file_hash=True) +df = Film.read_file("df.csv", file_hash=True) # fails if wrong +``` + +**Get example datasets?** + +```python +print(ExampleDfs.penguins().df) +# species island bill_length_mm ... flipper_length_mm body_mass_g sex +# 0 Adelie Torgersen 39.1 ... 181.0 3750.0 MALE +``` + +**Pretty-print the obvious way?** + +```python +df.pretty_print(to="all_data.md.zip") +wiki_txt = df.pretty_print(fmt="mediawiki") +``` + +All standard DataFrame methods remain available. +Use `.of(df)` to convert to your type, or `.vanilla()` for a plain DataFrame. + +**[Read the docs π](https://typed-dfs.readthedocs.io/en/stable/)** for more info and examples. + +### π Pandas serialization bugs fixed + +Pandas has several issues with serialization. + +<details> +<summary><em>See: Fixed issues</em></summary> +Depending on the format and columns, these issues occur: + +- columns being silently added or dropped, +- errors on either read or write of empty DataFrames, +- the inability to use DataFrames with indices in Feather, +- writing to Parquet failing with half-precision, +- lingering partially written files on error, +- the buggy xlrd being preferred by read_excel, +- the buggy odfpy also being preferred, +- writing a file and reading it back results in a different DataFrame, +- you canβt write fixed-width format, +- and the platform text encoding being used rather than utf-8. +- invalid JSON is written via the built-in json library + +</details> + +### π Other features + +See more in the [guided walkthrough βοΈ](https://typed-dfs.readthedocs.io/en/latest/guide.html) + +<details> +<summary><em>See: Short feature list</em></summary> + +- Dtype-aware natural sorting +- UTF-8 by default +- Near-atomicity of read/write +- Matrix-like typed dataframes and methods (e.g. `matrix.is_symmetric()`) +- DataFrame-compatible frozen, hashable, ordered collections (dict, list, and set) +- Serialize JSON robustly, preserving NaN, inf, βinf, enums, timezones, complex numbers, etc. +- Serialize more formats like TOML and INI +- Interpreting paths and formats (e.g. `FileFormat.split("dir/myfile.csv.gz").compression # gz`) +- Generate good CLI help text for input DataFrames +- Parse/verify/add/update/delete files in a .shasum-like file + +</details> + +### π Limitations + +<details> +<summary><em>See: List of limitations</em></summary> + +- Multi-level columns are not yet supported. +- Columns and index levels cannot share names. +- Duplicate column names are not supported. (These are strange anyway.) +- A typed DF cannot have columns "level_0", "index", or "Unnamed: 0". +- `inplace` is forbidden in some functions; avoid it or use `.vanilla()`. + +</details> + +### π Serialization support + +TypedDfs provides the methods `read_file` and `write_file`, which guess the format from the +filename extension. For example, this will convert a gzipped, tab-delimited file to Feather: + +```python +TastyDf = typeddfs.typed("TastyDf").build() +TastyDf.read_file("myfile.tab.gz").write_file("myfile.feather") +``` + +Pandas does most of the serialization, but some formats require extra packages. +Typed-dfs specifies [extras](https://python-poetry.org/docs/pyproject/#extras) +to help you get required packages and with compatible versions. + +Here are the extras: + +- `feather`: [Feather](https://arrow.apache.org/docs/python/feather.html) (uses: pyarrow) +- `parquet`: [Parquet (e.g. .snappy)](https://github.com/apache/parquet-format) (uses: pyarrow) +- `xml` (uses: lxml) +- `excel`: Excel and LibreOffice .xlsx/.ods/.xls, etc. (uses: openpyxl, defusedxml) +- `toml`: [TOML](https://toml.io/en/) (uses: tomlkit) +- `html` (uses: html5lib, beautifulsoup4) +- `xlsb`: rare binary Excel file (uses: pyxlsb) +- [HDF5](https://www.hdfgroup.org/solutions/hdf5/) _{no extra provided}_ (_use:_ `tables`) + +For example, for Feather and TOML support use: `typeddfs[feather,toml]` +As a shorthand for all formats, use `typeddfs[all]`. + +### π Serialization in-depth + +<details> +<summary><em>See: Full table</em></summary> + +| format | packages | extra | sanity | speed | file sizes | +| ----------- | ---------------------------- | --------- | ------ | ----- | ---------- | +| Feather | `pyarrow` | `feather` | +++ | ++++ | +++ | +| Parquet | `pyarrow` or `fastparquet` β | `parquet` | ++ | +++ | ++++ | +| csv/tsv | none | none | ++ | ββ | ββ | +| flexwf β‘ | none | none | ++ | ββ | ββ | +| .fwf | none | none | + | ββ | ββ | +| json | none | none | ββ | βββ | βββ | +| xml | `lxml` | `xml` | β | βββ | βββ | +| .properties | none | none | ββ | ββ | ββ | +| toml | `tomlkit` | `toml` | ββ | ββ | ββ | +| INI | none | none | βββ | ββ | ββ | +| .lines | none | none | ++ | ββ | ββ | +| .npy | none | none | β | + | +++ | +| .npz | none | none | β | + | +++ | +| .html | `html5lib,beautifulsoup4` | `html` | ββ | βββ | βββ | +| pickle | none | none | ββ | βββ | βββ | +| XLSX | `openpyxl,defusedxml` | `excel` | + | ββ | + | +| ODS | `openpyxl,defusedxml` | `excel` | + | ββ | + | +| XLS | `openpyxl,defusedxml` | `excel` | ββ | ββ | + | +| XLSB | `pyxlsb` | `xlsb` | ββ | ββ | ++ | +| HDF5 | `tables` | `hdf5` | ββ | β | ++ | + +**β Note:** The `hdf5` extra is currently disabled. + +</details> + +<details> +<summary><em>See: serialization notes</em></summary> + +- β `fastparquet` can be used instead. It is slower but much smaller. +- Parquet only supports str, float64, float32, int64, int32, and bool. + Other numeric types are automatically converted during write. +- β‘ `.flexwf` is fixed-width with optional delimiters. +- JSON has inconsistent handling of `None`. ([orjson](https://github.com/ijl/orjson) is more consistent). +- XML requires Pandas 1.3+. +- Not all JSON, XML, TOML, and HDF5 files can be read. +- .ini and .properties can only be written with exactly 2 columns + index levels: + a key and a value. INI keys are in the form `section.name`. +- .lines can only be written with exactly 1 column or index level. +- .npy and .npz only serialize numpy objects. + They are not supported in `read_file` and `write_file`. +- .html is not supported in `read_file` and `write_file`. +- Pickle is insecure and not recommended. +- Pandas supports odfpy for ODS and xlrd for XLS. In fact, it prefers those. + However, they are very buggy; openpyxl is much better. +- XLSM, XLTX, XLTM, XLS, and XLSB files can contain macros, which Microsoft Excel will ingest. +- XLS is a deprecated format. +- XLSB is not fully supported in Pandas. +- HDF may not work on all platforms yet due to a + [tables issue](https://github.com/PyTables/PyTables/issues/854). + +Feather offers massively better performance over CSV, gzipped CSV, and HDF5 +in read speed, write speed, memory overhead, and compression ratios. +Parquet typically results in smaller file sizes than Feather at some cost in speed. +Feather is the preferred format for most cases. + +</details> + +### π Security + +Refer to the [security policy](https://github.com/dmyersturnbull/typed-dfs/blob/main/SECURITY.md). + +### π Extra notes + +<details> +<summary><em>See: Pinned versions</em></summary> + +Dependencies in the extras only have version minimums, not maximums. +For example, typed-dfs requires pyarrow >= 4. +[natsort](https://github.com/SethMMorton/natsort) is also only assigned a minimum version number. +This means that the result of typed-dfβs `sort_natural` could change. +To fix this, pin natsort to a specific major version; +e.g. `natsort = "^8"` with [Poetry](https://python-poetry.org/) or `natsort>=8,<9` with pip. + +</details> + +### π Contributing + +Typed-Dfs is licensed under the [Apache License, version 2.0](https://www.apache.org/licenses/LICENSE-2.0). +[New issues](https://github.com/dmyersturnbull/typed-dfs/issues) and pull requests are welcome. +Please refer to the [contributing guide](https://github.com/dmyersturnbull/typed-dfs/blob/main/CONTRIBUTING.md). +Generated with [Tyrannosaurus](https://github.com/dmyersturnbull/tyrannosaurus). + + +%prep +%autosetup -n typeddfs-0.16.5 + +%build +%py3_build + +%install +%py3_install +install -d -m755 %{buildroot}/%{_pkgdocdir} +if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi +if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi +if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi +if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi +pushd %{buildroot} +if [ -d usr/lib ]; then + find usr/lib -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/lib64 ]; then + find usr/lib64 -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/bin ]; then + find usr/bin -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/sbin ]; then + find usr/sbin -type f -printf "/%h/%f\n" >> filelist.lst +fi +touch doclist.lst +if [ -d usr/share/man ]; then + find usr/share/man -type f -printf "/%h/%f.gz\n" >> doclist.lst +fi +popd +mv %{buildroot}/filelist.lst . +mv %{buildroot}/doclist.lst . + +%files -n python3-typeddfs -f filelist.lst +%dir %{python3_sitelib}/* + +%files help -f doclist.lst +%{_docdir}/* + +%changelog +* Mon May 15 2023 Python_Bot <Python_Bot@openeuler.org> - 0.16.5-1 +- Package Spec generated @@ -0,0 +1 @@ +f97a009aa64c13e52bc8e2307dbb1d64 typeddfs-0.16.5.tar.gz |