%global _empty_manifest_terminate_build 0
Name: python-typeddfs
Version: 0.16.5
Release: 1
Summary: Pandas DataFrame subclasses that enforce structure and can self-organize.
License: Apache-2.0
URL: https://github.com/dmyersturnbull/typed-dfs
Source0: https://mirrors.aliyun.com/pypi/web/packages/df/69/23b4c90de17493d82dcd65f094b636dc389fc3cd99342a109678bb06e101/typeddfs-0.16.5.tar.gz
BuildArch: noarch
Requires: python3-natsort
Requires: python3-orjson
Requires: python3-pandas
Requires: python3-regex
Requires: python3-tabulate
Requires: python3-wcwidth
Requires: python3-pyarrow
Requires: python3-lxml
Requires: python3-openpyxl
Requires: python3-pyxlsb
Requires: python3-html5lib
Requires: python3-beautifulsoup4
Requires: python3-tomlkit
%description
# Typed DataFrames
[](https://pypi.org/project/typeddfs)
[](https://opensource.org/licenses/Apache-2.0)
[](https://pypi.org/project/typeddfs)
[](https://github.com/dmyersturnbull/typed-dfs/releases)
[](https://pypi.org/project/typeddfs)
[](https://github.com/dmyersturnbull/typed-dfs/actions)
[](https://coveralls.io/github/dmyersturnbull/typed-dfs?branch=main)
[](https://typed-dfs.readthedocs.io/en/stable/)
[](https://codeclimate.com/github/dmyersturnbull/typed-dfs/maintainability)
[](https://scrutinizer-ci.com/g/dmyersturnbull/typed-dfs/?branch=main)
[](https://github.com/dmyersturnbull/tyrannosaurus)
Pandas DataFrame subclasses that self-organize and serialize robustly.
```python
Film = TypedDfs.typed("Film").require("name", "studio", "year").build()
df = Film.read_csv("file.csv")
assert df.columns.tolist() == ["name", "studio", "year"]
type(df) # Film
```
Your types remember how to be read,
including columns, dtypes, indices, and custom requirements.
No index_cols=, header=, set_index, or astype needed.
**Read and write any format:**
```python
path = input("input file? [.csv/.tsv/.tab/.json/.xml.bz2/.feather/.snappy.h5/...]")
df = Film.read_file(path)
df.write_file("output.snappy")
```
**Need dataclasses?**
```python
instances = df.to_dataclass_instances()
Film.from_dataclass_instances(instances)
```
**Save metadata?**
```python
df = df.set_attrs(dataset="piano")
df.write_file("df.csv", attrs=True)
df = Film.read_file("df.csv", attrs=True)
print(df.attrs) # e.g. {"dataset": "piano")
```
**Make dirs? Donβt overwrite?**
```python
df.write_file("df.csv", mkdirs=True, overwrite=False)
```
**Write / verify checksums?**
```python
df.write_file("df.csv", file_hash=True)
df = Film.read_file("df.csv", file_hash=True) # fails if wrong
```
**Get example datasets?**
```python
print(ExampleDfs.penguins().df)
# species island bill_length_mm ... flipper_length_mm body_mass_g sex
# 0 Adelie Torgersen 39.1 ... 181.0 3750.0 MALE
```
**Pretty-print the obvious way?**
```python
df.pretty_print(to="all_data.md.zip")
wiki_txt = df.pretty_print(fmt="mediawiki")
```
All standard DataFrame methods remain available.
Use `.of(df)` to convert to your type, or `.vanilla()` for a plain DataFrame.
**[Read the docs π](https://typed-dfs.readthedocs.io/en/stable/)** for more info and examples.
### π Pandas serialization bugs fixed
Pandas has several issues with serialization.
See: Fixed issues
Depending on the format and columns, these issues occur:
- columns being silently added or dropped,
- errors on either read or write of empty DataFrames,
- the inability to use DataFrames with indices in Feather,
- writing to Parquet failing with half-precision,
- lingering partially written files on error,
- the buggy xlrd being preferred by read_excel,
- the buggy odfpy also being preferred,
- writing a file and reading it back results in a different DataFrame,
- you canβt write fixed-width format,
- and the platform text encoding being used rather than utf-8.
- invalid JSON is written via the built-in json library
### π Other features
See more in the [guided walkthrough βοΈ](https://typed-dfs.readthedocs.io/en/latest/guide.html)
See: Short feature list
- Dtype-aware natural sorting
- UTF-8 by default
- Near-atomicity of read/write
- Matrix-like typed dataframes and methods (e.g. `matrix.is_symmetric()`)
- DataFrame-compatible frozen, hashable, ordered collections (dict, list, and set)
- Serialize JSON robustly, preserving NaN, inf, βinf, enums, timezones, complex numbers, etc.
- Serialize more formats like TOML and INI
- Interpreting paths and formats (e.g. `FileFormat.split("dir/myfile.csv.gz").compression # gz`)
- Generate good CLI help text for input DataFrames
- Parse/verify/add/update/delete files in a .shasum-like file
### π Limitations
See: List of limitations
- Multi-level columns are not yet supported.
- Columns and index levels cannot share names.
- Duplicate column names are not supported. (These are strange anyway.)
- A typed DF cannot have columns "level_0", "index", or "Unnamed: 0".
- `inplace` is forbidden in some functions; avoid it or use `.vanilla()`.
### π Serialization support
TypedDfs provides the methods `read_file` and `write_file`, which guess the format from the
filename extension. For example, this will convert a gzipped, tab-delimited file to Feather:
```python
TastyDf = typeddfs.typed("TastyDf").build()
TastyDf.read_file("myfile.tab.gz").write_file("myfile.feather")
```
Pandas does most of the serialization, but some formats require extra packages.
Typed-dfs specifies [extras](https://python-poetry.org/docs/pyproject/#extras)
to help you get required packages and with compatible versions.
Here are the extras:
- `feather`: [Feather](https://arrow.apache.org/docs/python/feather.html) (uses: pyarrow)
- `parquet`: [Parquet (e.g. .snappy)](https://github.com/apache/parquet-format) (uses: pyarrow)
- `xml` (uses: lxml)
- `excel`: Excel and LibreOffice .xlsx/.ods/.xls, etc. (uses: openpyxl, defusedxml)
- `toml`: [TOML](https://toml.io/en/) (uses: tomlkit)
- `html` (uses: html5lib, beautifulsoup4)
- `xlsb`: rare binary Excel file (uses: pyxlsb)
- [HDF5](https://www.hdfgroup.org/solutions/hdf5/) _{no extra provided}_ (_use:_ `tables`)
For example, for Feather and TOML support use: `typeddfs[feather,toml]`
As a shorthand for all formats, use `typeddfs[all]`.
### π Serialization in-depth
See: Full table
| format | packages | extra | sanity | speed | file sizes |
| ----------- | ---------------------------- | --------- | ------ | ----- | ---------- |
| Feather | `pyarrow` | `feather` | +++ | ++++ | +++ |
| Parquet | `pyarrow` or `fastparquet` β | `parquet` | ++ | +++ | ++++ |
| csv/tsv | none | none | ++ | ββ | ββ |
| flexwf β‘ | none | none | ++ | ββ | ββ |
| .fwf | none | none | + | ββ | ββ |
| json | none | none | ββ | βββ | βββ |
| xml | `lxml` | `xml` | β | βββ | βββ |
| .properties | none | none | ββ | ββ | ββ |
| toml | `tomlkit` | `toml` | ββ | ββ | ββ |
| INI | none | none | βββ | ββ | ββ |
| .lines | none | none | ++ | ββ | ββ |
| .npy | none | none | β | + | +++ |
| .npz | none | none | β | + | +++ |
| .html | `html5lib,beautifulsoup4` | `html` | ββ | βββ | βββ |
| pickle | none | none | ββ | βββ | βββ |
| XLSX | `openpyxl,defusedxml` | `excel` | + | ββ | + |
| ODS | `openpyxl,defusedxml` | `excel` | + | ββ | + |
| XLS | `openpyxl,defusedxml` | `excel` | ββ | ββ | + |
| XLSB | `pyxlsb` | `xlsb` | ββ | ββ | ++ |
| HDF5 | `tables` | `hdf5` | ββ | β | ++ |
**β Note:** The `hdf5` extra is currently disabled.
See: serialization notes
- β `fastparquet` can be used instead. It is slower but much smaller.
- Parquet only supports str, float64, float32, int64, int32, and bool.
Other numeric types are automatically converted during write.
- β‘ `.flexwf` is fixed-width with optional delimiters.
- JSON has inconsistent handling of `None`. ([orjson](https://github.com/ijl/orjson) is more consistent).
- XML requires Pandas 1.3+.
- Not all JSON, XML, TOML, and HDF5 files can be read.
- .ini and .properties can only be written with exactly 2 columns + index levels:
a key and a value. INI keys are in the form `section.name`.
- .lines can only be written with exactly 1 column or index level.
- .npy and .npz only serialize numpy objects.
They are not supported in `read_file` and `write_file`.
- .html is not supported in `read_file` and `write_file`.
- Pickle is insecure and not recommended.
- Pandas supports odfpy for ODS and xlrd for XLS. In fact, it prefers those.
However, they are very buggy; openpyxl is much better.
- XLSM, XLTX, XLTM, XLS, and XLSB files can contain macros, which Microsoft Excel will ingest.
- XLS is a deprecated format.
- XLSB is not fully supported in Pandas.
- HDF may not work on all platforms yet due to a
[tables issue](https://github.com/PyTables/PyTables/issues/854).
Feather offers massively better performance over CSV, gzipped CSV, and HDF5
in read speed, write speed, memory overhead, and compression ratios.
Parquet typically results in smaller file sizes than Feather at some cost in speed.
Feather is the preferred format for most cases.
### π Security
Refer to the [security policy](https://github.com/dmyersturnbull/typed-dfs/blob/main/SECURITY.md).
### π Extra notes
See: Pinned versions
Dependencies in the extras only have version minimums, not maximums.
For example, typed-dfs requires pyarrow >= 4.
[natsort](https://github.com/SethMMorton/natsort) is also only assigned a minimum version number.
This means that the result of typed-dfβs `sort_natural` could change.
To fix this, pin natsort to a specific major version;
e.g. `natsort = "^8"` with [Poetry](https://python-poetry.org/) or `natsort>=8,<9` with pip.
### π Contributing
Typed-Dfs is licensed under the [Apache License, version 2.0](https://www.apache.org/licenses/LICENSE-2.0).
[New issues](https://github.com/dmyersturnbull/typed-dfs/issues) and pull requests are welcome.
Please refer to the [contributing guide](https://github.com/dmyersturnbull/typed-dfs/blob/main/CONTRIBUTING.md).
Generated with [Tyrannosaurus](https://github.com/dmyersturnbull/tyrannosaurus).
%package -n python3-typeddfs
Summary: Pandas DataFrame subclasses that enforce structure and can self-organize.
Provides: python-typeddfs
BuildRequires: python3-devel
BuildRequires: python3-setuptools
BuildRequires: python3-pip
%description -n python3-typeddfs
# Typed DataFrames
[](https://pypi.org/project/typeddfs)
[](https://opensource.org/licenses/Apache-2.0)
[](https://pypi.org/project/typeddfs)
[](https://github.com/dmyersturnbull/typed-dfs/releases)
[](https://pypi.org/project/typeddfs)
[](https://github.com/dmyersturnbull/typed-dfs/actions)
[](https://coveralls.io/github/dmyersturnbull/typed-dfs?branch=main)
[](https://typed-dfs.readthedocs.io/en/stable/)
[](https://codeclimate.com/github/dmyersturnbull/typed-dfs/maintainability)
[](https://scrutinizer-ci.com/g/dmyersturnbull/typed-dfs/?branch=main)
[](https://github.com/dmyersturnbull/tyrannosaurus)
Pandas DataFrame subclasses that self-organize and serialize robustly.
```python
Film = TypedDfs.typed("Film").require("name", "studio", "year").build()
df = Film.read_csv("file.csv")
assert df.columns.tolist() == ["name", "studio", "year"]
type(df) # Film
```
Your types remember how to be read,
including columns, dtypes, indices, and custom requirements.
No index_cols=, header=, set_index, or astype needed.
**Read and write any format:**
```python
path = input("input file? [.csv/.tsv/.tab/.json/.xml.bz2/.feather/.snappy.h5/...]")
df = Film.read_file(path)
df.write_file("output.snappy")
```
**Need dataclasses?**
```python
instances = df.to_dataclass_instances()
Film.from_dataclass_instances(instances)
```
**Save metadata?**
```python
df = df.set_attrs(dataset="piano")
df.write_file("df.csv", attrs=True)
df = Film.read_file("df.csv", attrs=True)
print(df.attrs) # e.g. {"dataset": "piano")
```
**Make dirs? Donβt overwrite?**
```python
df.write_file("df.csv", mkdirs=True, overwrite=False)
```
**Write / verify checksums?**
```python
df.write_file("df.csv", file_hash=True)
df = Film.read_file("df.csv", file_hash=True) # fails if wrong
```
**Get example datasets?**
```python
print(ExampleDfs.penguins().df)
# species island bill_length_mm ... flipper_length_mm body_mass_g sex
# 0 Adelie Torgersen 39.1 ... 181.0 3750.0 MALE
```
**Pretty-print the obvious way?**
```python
df.pretty_print(to="all_data.md.zip")
wiki_txt = df.pretty_print(fmt="mediawiki")
```
All standard DataFrame methods remain available.
Use `.of(df)` to convert to your type, or `.vanilla()` for a plain DataFrame.
**[Read the docs π](https://typed-dfs.readthedocs.io/en/stable/)** for more info and examples.
### π Pandas serialization bugs fixed
Pandas has several issues with serialization.
See: Fixed issues
Depending on the format and columns, these issues occur:
- columns being silently added or dropped,
- errors on either read or write of empty DataFrames,
- the inability to use DataFrames with indices in Feather,
- writing to Parquet failing with half-precision,
- lingering partially written files on error,
- the buggy xlrd being preferred by read_excel,
- the buggy odfpy also being preferred,
- writing a file and reading it back results in a different DataFrame,
- you canβt write fixed-width format,
- and the platform text encoding being used rather than utf-8.
- invalid JSON is written via the built-in json library
### π Other features
See more in the [guided walkthrough βοΈ](https://typed-dfs.readthedocs.io/en/latest/guide.html)
See: Short feature list
- Dtype-aware natural sorting
- UTF-8 by default
- Near-atomicity of read/write
- Matrix-like typed dataframes and methods (e.g. `matrix.is_symmetric()`)
- DataFrame-compatible frozen, hashable, ordered collections (dict, list, and set)
- Serialize JSON robustly, preserving NaN, inf, βinf, enums, timezones, complex numbers, etc.
- Serialize more formats like TOML and INI
- Interpreting paths and formats (e.g. `FileFormat.split("dir/myfile.csv.gz").compression # gz`)
- Generate good CLI help text for input DataFrames
- Parse/verify/add/update/delete files in a .shasum-like file
### π Limitations
See: List of limitations
- Multi-level columns are not yet supported.
- Columns and index levels cannot share names.
- Duplicate column names are not supported. (These are strange anyway.)
- A typed DF cannot have columns "level_0", "index", or "Unnamed: 0".
- `inplace` is forbidden in some functions; avoid it or use `.vanilla()`.
### π Serialization support
TypedDfs provides the methods `read_file` and `write_file`, which guess the format from the
filename extension. For example, this will convert a gzipped, tab-delimited file to Feather:
```python
TastyDf = typeddfs.typed("TastyDf").build()
TastyDf.read_file("myfile.tab.gz").write_file("myfile.feather")
```
Pandas does most of the serialization, but some formats require extra packages.
Typed-dfs specifies [extras](https://python-poetry.org/docs/pyproject/#extras)
to help you get required packages and with compatible versions.
Here are the extras:
- `feather`: [Feather](https://arrow.apache.org/docs/python/feather.html) (uses: pyarrow)
- `parquet`: [Parquet (e.g. .snappy)](https://github.com/apache/parquet-format) (uses: pyarrow)
- `xml` (uses: lxml)
- `excel`: Excel and LibreOffice .xlsx/.ods/.xls, etc. (uses: openpyxl, defusedxml)
- `toml`: [TOML](https://toml.io/en/) (uses: tomlkit)
- `html` (uses: html5lib, beautifulsoup4)
- `xlsb`: rare binary Excel file (uses: pyxlsb)
- [HDF5](https://www.hdfgroup.org/solutions/hdf5/) _{no extra provided}_ (_use:_ `tables`)
For example, for Feather and TOML support use: `typeddfs[feather,toml]`
As a shorthand for all formats, use `typeddfs[all]`.
### π Serialization in-depth
See: Full table
| format | packages | extra | sanity | speed | file sizes |
| ----------- | ---------------------------- | --------- | ------ | ----- | ---------- |
| Feather | `pyarrow` | `feather` | +++ | ++++ | +++ |
| Parquet | `pyarrow` or `fastparquet` β | `parquet` | ++ | +++ | ++++ |
| csv/tsv | none | none | ++ | ββ | ββ |
| flexwf β‘ | none | none | ++ | ββ | ββ |
| .fwf | none | none | + | ββ | ββ |
| json | none | none | ββ | βββ | βββ |
| xml | `lxml` | `xml` | β | βββ | βββ |
| .properties | none | none | ββ | ββ | ββ |
| toml | `tomlkit` | `toml` | ββ | ββ | ββ |
| INI | none | none | βββ | ββ | ββ |
| .lines | none | none | ++ | ββ | ββ |
| .npy | none | none | β | + | +++ |
| .npz | none | none | β | + | +++ |
| .html | `html5lib,beautifulsoup4` | `html` | ββ | βββ | βββ |
| pickle | none | none | ββ | βββ | βββ |
| XLSX | `openpyxl,defusedxml` | `excel` | + | ββ | + |
| ODS | `openpyxl,defusedxml` | `excel` | + | ββ | + |
| XLS | `openpyxl,defusedxml` | `excel` | ββ | ββ | + |
| XLSB | `pyxlsb` | `xlsb` | ββ | ββ | ++ |
| HDF5 | `tables` | `hdf5` | ββ | β | ++ |
**β Note:** The `hdf5` extra is currently disabled.
See: serialization notes
- β `fastparquet` can be used instead. It is slower but much smaller.
- Parquet only supports str, float64, float32, int64, int32, and bool.
Other numeric types are automatically converted during write.
- β‘ `.flexwf` is fixed-width with optional delimiters.
- JSON has inconsistent handling of `None`. ([orjson](https://github.com/ijl/orjson) is more consistent).
- XML requires Pandas 1.3+.
- Not all JSON, XML, TOML, and HDF5 files can be read.
- .ini and .properties can only be written with exactly 2 columns + index levels:
a key and a value. INI keys are in the form `section.name`.
- .lines can only be written with exactly 1 column or index level.
- .npy and .npz only serialize numpy objects.
They are not supported in `read_file` and `write_file`.
- .html is not supported in `read_file` and `write_file`.
- Pickle is insecure and not recommended.
- Pandas supports odfpy for ODS and xlrd for XLS. In fact, it prefers those.
However, they are very buggy; openpyxl is much better.
- XLSM, XLTX, XLTM, XLS, and XLSB files can contain macros, which Microsoft Excel will ingest.
- XLS is a deprecated format.
- XLSB is not fully supported in Pandas.
- HDF may not work on all platforms yet due to a
[tables issue](https://github.com/PyTables/PyTables/issues/854).
Feather offers massively better performance over CSV, gzipped CSV, and HDF5
in read speed, write speed, memory overhead, and compression ratios.
Parquet typically results in smaller file sizes than Feather at some cost in speed.
Feather is the preferred format for most cases.
### π Security
Refer to the [security policy](https://github.com/dmyersturnbull/typed-dfs/blob/main/SECURITY.md).
### π Extra notes
See: Pinned versions
Dependencies in the extras only have version minimums, not maximums.
For example, typed-dfs requires pyarrow >= 4.
[natsort](https://github.com/SethMMorton/natsort) is also only assigned a minimum version number.
This means that the result of typed-dfβs `sort_natural` could change.
To fix this, pin natsort to a specific major version;
e.g. `natsort = "^8"` with [Poetry](https://python-poetry.org/) or `natsort>=8,<9` with pip.
### π Contributing
Typed-Dfs is licensed under the [Apache License, version 2.0](https://www.apache.org/licenses/LICENSE-2.0).
[New issues](https://github.com/dmyersturnbull/typed-dfs/issues) and pull requests are welcome.
Please refer to the [contributing guide](https://github.com/dmyersturnbull/typed-dfs/blob/main/CONTRIBUTING.md).
Generated with [Tyrannosaurus](https://github.com/dmyersturnbull/tyrannosaurus).
%package help
Summary: Development documents and examples for typeddfs
Provides: python3-typeddfs-doc
%description help
# Typed DataFrames
[](https://pypi.org/project/typeddfs)
[](https://opensource.org/licenses/Apache-2.0)
[](https://pypi.org/project/typeddfs)
[](https://github.com/dmyersturnbull/typed-dfs/releases)
[](https://pypi.org/project/typeddfs)
[](https://github.com/dmyersturnbull/typed-dfs/actions)
[](https://coveralls.io/github/dmyersturnbull/typed-dfs?branch=main)
[](https://typed-dfs.readthedocs.io/en/stable/)
[](https://codeclimate.com/github/dmyersturnbull/typed-dfs/maintainability)
[](https://scrutinizer-ci.com/g/dmyersturnbull/typed-dfs/?branch=main)
[](https://github.com/dmyersturnbull/tyrannosaurus)
Pandas DataFrame subclasses that self-organize and serialize robustly.
```python
Film = TypedDfs.typed("Film").require("name", "studio", "year").build()
df = Film.read_csv("file.csv")
assert df.columns.tolist() == ["name", "studio", "year"]
type(df) # Film
```
Your types remember how to be read,
including columns, dtypes, indices, and custom requirements.
No index_cols=, header=, set_index, or astype needed.
**Read and write any format:**
```python
path = input("input file? [.csv/.tsv/.tab/.json/.xml.bz2/.feather/.snappy.h5/...]")
df = Film.read_file(path)
df.write_file("output.snappy")
```
**Need dataclasses?**
```python
instances = df.to_dataclass_instances()
Film.from_dataclass_instances(instances)
```
**Save metadata?**
```python
df = df.set_attrs(dataset="piano")
df.write_file("df.csv", attrs=True)
df = Film.read_file("df.csv", attrs=True)
print(df.attrs) # e.g. {"dataset": "piano")
```
**Make dirs? Donβt overwrite?**
```python
df.write_file("df.csv", mkdirs=True, overwrite=False)
```
**Write / verify checksums?**
```python
df.write_file("df.csv", file_hash=True)
df = Film.read_file("df.csv", file_hash=True) # fails if wrong
```
**Get example datasets?**
```python
print(ExampleDfs.penguins().df)
# species island bill_length_mm ... flipper_length_mm body_mass_g sex
# 0 Adelie Torgersen 39.1 ... 181.0 3750.0 MALE
```
**Pretty-print the obvious way?**
```python
df.pretty_print(to="all_data.md.zip")
wiki_txt = df.pretty_print(fmt="mediawiki")
```
All standard DataFrame methods remain available.
Use `.of(df)` to convert to your type, or `.vanilla()` for a plain DataFrame.
**[Read the docs π](https://typed-dfs.readthedocs.io/en/stable/)** for more info and examples.
### π Pandas serialization bugs fixed
Pandas has several issues with serialization.
See: Fixed issues
Depending on the format and columns, these issues occur:
- columns being silently added or dropped,
- errors on either read or write of empty DataFrames,
- the inability to use DataFrames with indices in Feather,
- writing to Parquet failing with half-precision,
- lingering partially written files on error,
- the buggy xlrd being preferred by read_excel,
- the buggy odfpy also being preferred,
- writing a file and reading it back results in a different DataFrame,
- you canβt write fixed-width format,
- and the platform text encoding being used rather than utf-8.
- invalid JSON is written via the built-in json library
### π Other features
See more in the [guided walkthrough βοΈ](https://typed-dfs.readthedocs.io/en/latest/guide.html)
See: Short feature list
- Dtype-aware natural sorting
- UTF-8 by default
- Near-atomicity of read/write
- Matrix-like typed dataframes and methods (e.g. `matrix.is_symmetric()`)
- DataFrame-compatible frozen, hashable, ordered collections (dict, list, and set)
- Serialize JSON robustly, preserving NaN, inf, βinf, enums, timezones, complex numbers, etc.
- Serialize more formats like TOML and INI
- Interpreting paths and formats (e.g. `FileFormat.split("dir/myfile.csv.gz").compression # gz`)
- Generate good CLI help text for input DataFrames
- Parse/verify/add/update/delete files in a .shasum-like file
### π Limitations
See: List of limitations
- Multi-level columns are not yet supported.
- Columns and index levels cannot share names.
- Duplicate column names are not supported. (These are strange anyway.)
- A typed DF cannot have columns "level_0", "index", or "Unnamed: 0".
- `inplace` is forbidden in some functions; avoid it or use `.vanilla()`.
### π Serialization support
TypedDfs provides the methods `read_file` and `write_file`, which guess the format from the
filename extension. For example, this will convert a gzipped, tab-delimited file to Feather:
```python
TastyDf = typeddfs.typed("TastyDf").build()
TastyDf.read_file("myfile.tab.gz").write_file("myfile.feather")
```
Pandas does most of the serialization, but some formats require extra packages.
Typed-dfs specifies [extras](https://python-poetry.org/docs/pyproject/#extras)
to help you get required packages and with compatible versions.
Here are the extras:
- `feather`: [Feather](https://arrow.apache.org/docs/python/feather.html) (uses: pyarrow)
- `parquet`: [Parquet (e.g. .snappy)](https://github.com/apache/parquet-format) (uses: pyarrow)
- `xml` (uses: lxml)
- `excel`: Excel and LibreOffice .xlsx/.ods/.xls, etc. (uses: openpyxl, defusedxml)
- `toml`: [TOML](https://toml.io/en/) (uses: tomlkit)
- `html` (uses: html5lib, beautifulsoup4)
- `xlsb`: rare binary Excel file (uses: pyxlsb)
- [HDF5](https://www.hdfgroup.org/solutions/hdf5/) _{no extra provided}_ (_use:_ `tables`)
For example, for Feather and TOML support use: `typeddfs[feather,toml]`
As a shorthand for all formats, use `typeddfs[all]`.
### π Serialization in-depth
See: Full table
| format | packages | extra | sanity | speed | file sizes |
| ----------- | ---------------------------- | --------- | ------ | ----- | ---------- |
| Feather | `pyarrow` | `feather` | +++ | ++++ | +++ |
| Parquet | `pyarrow` or `fastparquet` β | `parquet` | ++ | +++ | ++++ |
| csv/tsv | none | none | ++ | ββ | ββ |
| flexwf β‘ | none | none | ++ | ββ | ββ |
| .fwf | none | none | + | ββ | ββ |
| json | none | none | ββ | βββ | βββ |
| xml | `lxml` | `xml` | β | βββ | βββ |
| .properties | none | none | ββ | ββ | ββ |
| toml | `tomlkit` | `toml` | ββ | ββ | ββ |
| INI | none | none | βββ | ββ | ββ |
| .lines | none | none | ++ | ββ | ββ |
| .npy | none | none | β | + | +++ |
| .npz | none | none | β | + | +++ |
| .html | `html5lib,beautifulsoup4` | `html` | ββ | βββ | βββ |
| pickle | none | none | ββ | βββ | βββ |
| XLSX | `openpyxl,defusedxml` | `excel` | + | ββ | + |
| ODS | `openpyxl,defusedxml` | `excel` | + | ββ | + |
| XLS | `openpyxl,defusedxml` | `excel` | ββ | ββ | + |
| XLSB | `pyxlsb` | `xlsb` | ββ | ββ | ++ |
| HDF5 | `tables` | `hdf5` | ββ | β | ++ |
**β Note:** The `hdf5` extra is currently disabled.
See: serialization notes
- β `fastparquet` can be used instead. It is slower but much smaller.
- Parquet only supports str, float64, float32, int64, int32, and bool.
Other numeric types are automatically converted during write.
- β‘ `.flexwf` is fixed-width with optional delimiters.
- JSON has inconsistent handling of `None`. ([orjson](https://github.com/ijl/orjson) is more consistent).
- XML requires Pandas 1.3+.
- Not all JSON, XML, TOML, and HDF5 files can be read.
- .ini and .properties can only be written with exactly 2 columns + index levels:
a key and a value. INI keys are in the form `section.name`.
- .lines can only be written with exactly 1 column or index level.
- .npy and .npz only serialize numpy objects.
They are not supported in `read_file` and `write_file`.
- .html is not supported in `read_file` and `write_file`.
- Pickle is insecure and not recommended.
- Pandas supports odfpy for ODS and xlrd for XLS. In fact, it prefers those.
However, they are very buggy; openpyxl is much better.
- XLSM, XLTX, XLTM, XLS, and XLSB files can contain macros, which Microsoft Excel will ingest.
- XLS is a deprecated format.
- XLSB is not fully supported in Pandas.
- HDF may not work on all platforms yet due to a
[tables issue](https://github.com/PyTables/PyTables/issues/854).
Feather offers massively better performance over CSV, gzipped CSV, and HDF5
in read speed, write speed, memory overhead, and compression ratios.
Parquet typically results in smaller file sizes than Feather at some cost in speed.
Feather is the preferred format for most cases.
### π Security
Refer to the [security policy](https://github.com/dmyersturnbull/typed-dfs/blob/main/SECURITY.md).
### π Extra notes
See: Pinned versions
Dependencies in the extras only have version minimums, not maximums.
For example, typed-dfs requires pyarrow >= 4.
[natsort](https://github.com/SethMMorton/natsort) is also only assigned a minimum version number.
This means that the result of typed-dfβs `sort_natural` could change.
To fix this, pin natsort to a specific major version;
e.g. `natsort = "^8"` with [Poetry](https://python-poetry.org/) or `natsort>=8,<9` with pip.
### π Contributing
Typed-Dfs is licensed under the [Apache License, version 2.0](https://www.apache.org/licenses/LICENSE-2.0).
[New issues](https://github.com/dmyersturnbull/typed-dfs/issues) and pull requests are welcome.
Please refer to the [contributing guide](https://github.com/dmyersturnbull/typed-dfs/blob/main/CONTRIBUTING.md).
Generated with [Tyrannosaurus](https://github.com/dmyersturnbull/tyrannosaurus).
%prep
%autosetup -n typeddfs-0.16.5
%build
%py3_build
%install
%py3_install
install -d -m755 %{buildroot}/%{_pkgdocdir}
if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi
if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi
if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi
if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi
pushd %{buildroot}
if [ -d usr/lib ]; then
find usr/lib -type f -printf "\"/%h/%f\"\n" >> filelist.lst
fi
if [ -d usr/lib64 ]; then
find usr/lib64 -type f -printf "\"/%h/%f\"\n" >> filelist.lst
fi
if [ -d usr/bin ]; then
find usr/bin -type f -printf "\"/%h/%f\"\n" >> filelist.lst
fi
if [ -d usr/sbin ]; then
find usr/sbin -type f -printf "\"/%h/%f\"\n" >> filelist.lst
fi
touch doclist.lst
if [ -d usr/share/man ]; then
find usr/share/man -type f -printf "\"/%h/%f.gz\"\n" >> doclist.lst
fi
popd
mv %{buildroot}/filelist.lst .
mv %{buildroot}/doclist.lst .
%files -n python3-typeddfs -f filelist.lst
%dir %{python3_sitelib}/*
%files help -f doclist.lst
%{_docdir}/*
%changelog
* Thu Jun 08 2023 Python_Bot - 0.16.5-1
- Package Spec generated