%global _empty_manifest_terminate_build 0
Name: python-satip
Version: 2.11.6
Release: 1
Summary: Satip provides the functionality necessary for
License: MIT
URL: https://pypi.org/project/satip/
Source0: https://mirrors.nju.edu.cn/pypi/web/packages/77/f9/64469de1bc4ea119305ab312cbaedfd6fd26b4ee2573fc1673664a3c65d8/satip-2.11.6.tar.gz
BuildArch: noarch
Requires: python3-affine
Requires: python3-aiobotocore
Requires: python3-aiohttp
Requires: python3-aioitertools
Requires: python3-aiosignal
Requires: python3-alembic
Requires: python3-appdirs
Requires: python3-asciitree
Requires: python3-async-timeout
Requires: python3-attrs
Requires: python3-bokeh
Requires: python3-botocore
Requires: python3-Bottleneck
Requires: python3-brotlipy
Requires: python3-cached-property
Requires: python3-Cartopy
Requires: python3-certifi
Requires: python3-cffi
Requires: python3-cftime
Requires: python3-charset-normalizer
Requires: python3-click
Requires: python3-click-plugins
Requires: python3-cligj
Requires: python3-cloudpickle
Requires: python3-colorama
Requires: python3-configobj
Requires: python3-cryptography
Requires: python3-cycler
Requires: python3-cytoolz
Requires: python3-dask
Requires: python3-distributed
Requires: python3-docutils
Requires: python3-donfig
Requires: python3-eccodes-python
Requires: python3-entrypoints
Requires: python3-eumdac
Requires: python3-fasteners
Requires: python3-freezegun
Requires: python3-frozenlist
Requires: python3-fsspec
Requires: python3-h5netcdf
Requires: python3-h5py
Requires: python3-HeapDict
Requires: python3-idna
Requires: python3-imagecodecs
Requires: python3-importlib-metadata
Requires: python3-Jinja2
Requires: python3-jmespath
Requires: python3-kiwisolver
Requires: python3-locket
Requires: python3-lz4
Requires: python3-Mako
Requires: python3-MarkupSafe
Requires: python3-matplotlib
Requires: python3-msgpack
Requires: python3-multidict
Requires: python3-munkres
Requires: python3-netCDF4
Requires: python3-nowcasting-datamodel
Requires: python3-numcodecs
Requires: python3-numpy
Requires: python3-packaging
Requires: python3-pandas
Requires: python3-partd
Requires: python3-Pillow
Requires: python3-pip
Requires: python3-pooch
Requires: python3-psutil
Requires: python3-psycopg2-binary
Requires: python3-pycparser
Requires: python3-pydantic
Requires: python3-pykdtree
Requires: python3-pyOpenSSL
Requires: python3-pyorbital
Requires: python3-pyparsing
Requires: python3-pyproj
Requires: python3-pyresample
Requires: python3-pyshp
Requires: python3-PySocks
Requires: python3-pyspectral
Requires: python3-dateutil
Requires: python3-geotiepoints
Requires: python3-pytz
Requires: python3-PyYAML
Requires: python3-rasterio
Requires: python3-requests
Requires: python3-s3fs
Requires: python3-satpy
Requires: python3-scipy
Requires: python3-setuptools
Requires: python3-shapely
Requires: python3-six
Requires: python3-snuggs
Requires: python3-sortedcontainers
Requires: python3-SQLAlchemy
Requires: python3-structlog
Requires: python3-tblib
Requires: python3-toolz
Requires: python3-tornado
Requires: python3-tqdm
Requires: python3-trollimage
Requires: python3-trollsift
Requires: python3-typing-extensions
Requires: python3-urllib3
Requires: python3-wheel
Requires: python3-wrapt
Requires: python3-xarray
Requires: python3-yarl
Requires: python3-zarr
Requires: python3-zict
Requires: python3-zipp
Requires: python3-ocf-blosc2
%description
# Satip
[](#contributors-)
[](https://badge.fury.io/py/satip)
[](https://codecov.io/gh/openclimatefix/Satip)
> Satip is a library for satellite image processing, and provides all of the functionality necessary for retrieving, and storing EUMETSAT data
### Installation
To install the `satip` library please run:
```bash
pip install satip
```
Or if you're working in the development environment you can run the following from the directory root:
```bash
pip install -e .
```
#### Conda
Or, if you want to use `conda` from the a cloned Satip repository:
```bash
conda env create -f environment.yml
conda activate satip
pip install -e .
```
If you plan to work on the development of Satip then also consider installing these development tools:
```bash
conda install pytest flake8 jedi mypy black pre-commit
pre-commit install
```
## Operation
### Getting your own API key
In order to contribute to development or just test-run some scripts, you will need your own Eumetsat-API-key. Please follow these steps:
1. Go to https://eoportal.eumetsat.int and register an account.
2. You can log in and got to https://data.eumetsat.int/ to check available data services. From there go to your profile and choose the option "API key" or go to https://api.eumetsat.int/api-key/ directly.
3. Please make sure that you added the key and secret to your user's environment variables.
### Downloading EUMETSAT Data
The following command will download the last 2 hours of RSS imagery into NetCDF files at the specified location
```bash
python satip/app.py --api-key= --api-secret= --save-dr="/path/to/saving/files/" --history="2 hours"
```
To download more historical data, the command below will download the native files, compress with bz2, and save into a subdirectory.
```bash
python satip/get_raw_eumetsat_data.py --user-key= --user-secret=
```
### Converting Native files to Zarr
`scripts/convert_native_to_zarr.py` converts EUMETSAT `.nat` files to Zarr datasets, using very mild lossy [JPEG-XL](https://en.wikipedia.org/wiki/JPEG_XL) compression. (JPEG-XL is the "new kid on the block" of image compression algorithms). JPEG-XL makes the files about a quarter the size of the equivalent `bz2` compressed files, whilst the images are visually indistinguishable. JPEG-XL cannot represent NaNs so NaNs. JPEG-XL understands float32 values in the range `[0, 1]`. NaNs are encoded as the value `0.025`. All "real" values are in the range `[0.075, 1]`. We leave a gap between "NaNs" and "real values" because there is very slight "ringing" around areas of constant value (see [this comment for more details](https://github.com/openclimatefix/Satip/issues/67#issuecomment-1036456502)). Use `satip.jpeg_xl_float_with_nans.JpegXlFloatWithNaNs` to decode the satellite data. This class will reconstruct the NaNs and rescale the data to the range `[0, 1]`.
## Running in Production
The live service uses `app.py` as the entrypoint for running the live data download for OCF's forecasting service, and has a few configuration options, configurable by command line argument or environment variable.
`--api-key` or `API_KEY` is the EUMETSAT API key
`--api-secret` or `API_SECRET` is the EUMETSAT API secret
`--save-dir` or `SAVE_DIR` is the top level directory to save the output files, a `latest` subfolder will be added to that directory to contain the latest data
`--history` or `HISTORY` is the amount of history timesteps to use in the `latest.zarr` files
`--db-url` or `DB_URL` is the URL to the database to save to when a run has finished
`--use-rescaler` or `USE_RESCALER` tells whether to rescale the satellite data to between 0 and 1 or not when saving to disk. Primarily used as backwards compatibility for the current production models, all new training and production Zarrs should use the rescaled data.
## Testing
To run tests, simply run ```pytest .``` from the root of the repository. To generate the test plots, run ```python scripts/generate_test_plots.py```.
## Contributors ✨
Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/docs/en/emoji-key)):
This project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification. Contributions of any kind welcome!
%package -n python3-satip
Summary: Satip provides the functionality necessary for
Provides: python-satip
BuildRequires: python3-devel
BuildRequires: python3-setuptools
BuildRequires: python3-pip
%description -n python3-satip
# Satip
[](#contributors-)
[](https://badge.fury.io/py/satip)
[](https://codecov.io/gh/openclimatefix/Satip)
> Satip is a library for satellite image processing, and provides all of the functionality necessary for retrieving, and storing EUMETSAT data
### Installation
To install the `satip` library please run:
```bash
pip install satip
```
Or if you're working in the development environment you can run the following from the directory root:
```bash
pip install -e .
```
#### Conda
Or, if you want to use `conda` from the a cloned Satip repository:
```bash
conda env create -f environment.yml
conda activate satip
pip install -e .
```
If you plan to work on the development of Satip then also consider installing these development tools:
```bash
conda install pytest flake8 jedi mypy black pre-commit
pre-commit install
```
## Operation
### Getting your own API key
In order to contribute to development or just test-run some scripts, you will need your own Eumetsat-API-key. Please follow these steps:
1. Go to https://eoportal.eumetsat.int and register an account.
2. You can log in and got to https://data.eumetsat.int/ to check available data services. From there go to your profile and choose the option "API key" or go to https://api.eumetsat.int/api-key/ directly.
3. Please make sure that you added the key and secret to your user's environment variables.
### Downloading EUMETSAT Data
The following command will download the last 2 hours of RSS imagery into NetCDF files at the specified location
```bash
python satip/app.py --api-key= --api-secret= --save-dr="/path/to/saving/files/" --history="2 hours"
```
To download more historical data, the command below will download the native files, compress with bz2, and save into a subdirectory.
```bash
python satip/get_raw_eumetsat_data.py --user-key= --user-secret=
```
### Converting Native files to Zarr
`scripts/convert_native_to_zarr.py` converts EUMETSAT `.nat` files to Zarr datasets, using very mild lossy [JPEG-XL](https://en.wikipedia.org/wiki/JPEG_XL) compression. (JPEG-XL is the "new kid on the block" of image compression algorithms). JPEG-XL makes the files about a quarter the size of the equivalent `bz2` compressed files, whilst the images are visually indistinguishable. JPEG-XL cannot represent NaNs so NaNs. JPEG-XL understands float32 values in the range `[0, 1]`. NaNs are encoded as the value `0.025`. All "real" values are in the range `[0.075, 1]`. We leave a gap between "NaNs" and "real values" because there is very slight "ringing" around areas of constant value (see [this comment for more details](https://github.com/openclimatefix/Satip/issues/67#issuecomment-1036456502)). Use `satip.jpeg_xl_float_with_nans.JpegXlFloatWithNaNs` to decode the satellite data. This class will reconstruct the NaNs and rescale the data to the range `[0, 1]`.
## Running in Production
The live service uses `app.py` as the entrypoint for running the live data download for OCF's forecasting service, and has a few configuration options, configurable by command line argument or environment variable.
`--api-key` or `API_KEY` is the EUMETSAT API key
`--api-secret` or `API_SECRET` is the EUMETSAT API secret
`--save-dir` or `SAVE_DIR` is the top level directory to save the output files, a `latest` subfolder will be added to that directory to contain the latest data
`--history` or `HISTORY` is the amount of history timesteps to use in the `latest.zarr` files
`--db-url` or `DB_URL` is the URL to the database to save to when a run has finished
`--use-rescaler` or `USE_RESCALER` tells whether to rescale the satellite data to between 0 and 1 or not when saving to disk. Primarily used as backwards compatibility for the current production models, all new training and production Zarrs should use the rescaled data.
## Testing
To run tests, simply run ```pytest .``` from the root of the repository. To generate the test plots, run ```python scripts/generate_test_plots.py```.
## Contributors ✨
Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/docs/en/emoji-key)):
This project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification. Contributions of any kind welcome!
%package help
Summary: Development documents and examples for satip
Provides: python3-satip-doc
%description help
# Satip
[](#contributors-)
[](https://badge.fury.io/py/satip)
[](https://codecov.io/gh/openclimatefix/Satip)
> Satip is a library for satellite image processing, and provides all of the functionality necessary for retrieving, and storing EUMETSAT data
### Installation
To install the `satip` library please run:
```bash
pip install satip
```
Or if you're working in the development environment you can run the following from the directory root:
```bash
pip install -e .
```
#### Conda
Or, if you want to use `conda` from the a cloned Satip repository:
```bash
conda env create -f environment.yml
conda activate satip
pip install -e .
```
If you plan to work on the development of Satip then also consider installing these development tools:
```bash
conda install pytest flake8 jedi mypy black pre-commit
pre-commit install
```
## Operation
### Getting your own API key
In order to contribute to development or just test-run some scripts, you will need your own Eumetsat-API-key. Please follow these steps:
1. Go to https://eoportal.eumetsat.int and register an account.
2. You can log in and got to https://data.eumetsat.int/ to check available data services. From there go to your profile and choose the option "API key" or go to https://api.eumetsat.int/api-key/ directly.
3. Please make sure that you added the key and secret to your user's environment variables.
### Downloading EUMETSAT Data
The following command will download the last 2 hours of RSS imagery into NetCDF files at the specified location
```bash
python satip/app.py --api-key= --api-secret= --save-dr="/path/to/saving/files/" --history="2 hours"
```
To download more historical data, the command below will download the native files, compress with bz2, and save into a subdirectory.
```bash
python satip/get_raw_eumetsat_data.py --user-key= --user-secret=
```
### Converting Native files to Zarr
`scripts/convert_native_to_zarr.py` converts EUMETSAT `.nat` files to Zarr datasets, using very mild lossy [JPEG-XL](https://en.wikipedia.org/wiki/JPEG_XL) compression. (JPEG-XL is the "new kid on the block" of image compression algorithms). JPEG-XL makes the files about a quarter the size of the equivalent `bz2` compressed files, whilst the images are visually indistinguishable. JPEG-XL cannot represent NaNs so NaNs. JPEG-XL understands float32 values in the range `[0, 1]`. NaNs are encoded as the value `0.025`. All "real" values are in the range `[0.075, 1]`. We leave a gap between "NaNs" and "real values" because there is very slight "ringing" around areas of constant value (see [this comment for more details](https://github.com/openclimatefix/Satip/issues/67#issuecomment-1036456502)). Use `satip.jpeg_xl_float_with_nans.JpegXlFloatWithNaNs` to decode the satellite data. This class will reconstruct the NaNs and rescale the data to the range `[0, 1]`.
## Running in Production
The live service uses `app.py` as the entrypoint for running the live data download for OCF's forecasting service, and has a few configuration options, configurable by command line argument or environment variable.
`--api-key` or `API_KEY` is the EUMETSAT API key
`--api-secret` or `API_SECRET` is the EUMETSAT API secret
`--save-dir` or `SAVE_DIR` is the top level directory to save the output files, a `latest` subfolder will be added to that directory to contain the latest data
`--history` or `HISTORY` is the amount of history timesteps to use in the `latest.zarr` files
`--db-url` or `DB_URL` is the URL to the database to save to when a run has finished
`--use-rescaler` or `USE_RESCALER` tells whether to rescale the satellite data to between 0 and 1 or not when saving to disk. Primarily used as backwards compatibility for the current production models, all new training and production Zarrs should use the rescaled data.
## Testing
To run tests, simply run ```pytest .``` from the root of the repository. To generate the test plots, run ```python scripts/generate_test_plots.py```.
## Contributors ✨
Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/docs/en/emoji-key)):
This project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification. Contributions of any kind welcome!
%prep
%autosetup -n satip-2.11.6
%build
%py3_build
%install
%py3_install
install -d -m755 %{buildroot}/%{_pkgdocdir}
if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi
if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi
if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi
if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi
pushd %{buildroot}
if [ -d usr/lib ]; then
find usr/lib -type f -printf "/%h/%f\n" >> filelist.lst
fi
if [ -d usr/lib64 ]; then
find usr/lib64 -type f -printf "/%h/%f\n" >> filelist.lst
fi
if [ -d usr/bin ]; then
find usr/bin -type f -printf "/%h/%f\n" >> filelist.lst
fi
if [ -d usr/sbin ]; then
find usr/sbin -type f -printf "/%h/%f\n" >> filelist.lst
fi
touch doclist.lst
if [ -d usr/share/man ]; then
find usr/share/man -type f -printf "/%h/%f.gz\n" >> doclist.lst
fi
popd
mv %{buildroot}/filelist.lst .
mv %{buildroot}/doclist.lst .
%files -n python3-satip -f filelist.lst
%dir %{python3_sitelib}/*
%files help -f doclist.lst
%{_docdir}/*
%changelog
* Tue Apr 25 2023 Python_Bot - 2.11.6-1
- Package Spec generated