From 6bbae8f8b572fbef640d2af6fe60a72287e7f555 Mon Sep 17 00:00:00 2001 From: CoprDistGit Date: Tue, 11 Apr 2023 23:43:19 +0000 Subject: automatic import of python-satip --- python-satip.spec | 532 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 532 insertions(+) create mode 100644 python-satip.spec (limited to 'python-satip.spec') diff --git a/python-satip.spec b/python-satip.spec new file mode 100644 index 0000000..a4e6352 --- /dev/null +++ b/python-satip.spec @@ -0,0 +1,532 @@ +%global _empty_manifest_terminate_build 0 +Name: python-satip +Version: 2.11.4 +Release: 1 +Summary: Satip provides the functionality necessary for +License: MIT +URL: https://pypi.org/project/satip/ +Source0: https://mirrors.nju.edu.cn/pypi/web/packages/70/8a/36c87e90a5f7797a673eee0ca4c5ff3cd8423440329c6dd5a3b914898801/satip-2.11.4.tar.gz +BuildArch: noarch + +Requires: python3-affine +Requires: python3-aiobotocore +Requires: python3-aiohttp +Requires: python3-aioitertools +Requires: python3-aiosignal +Requires: python3-alembic +Requires: python3-appdirs +Requires: python3-asciitree +Requires: python3-async-timeout +Requires: python3-attrs +Requires: python3-bokeh +Requires: python3-botocore +Requires: python3-Bottleneck +Requires: python3-brotlipy +Requires: python3-cached-property +Requires: python3-Cartopy +Requires: python3-certifi +Requires: python3-cffi +Requires: python3-cftime +Requires: python3-charset-normalizer +Requires: python3-click +Requires: python3-click-plugins +Requires: python3-cligj +Requires: python3-cloudpickle +Requires: python3-colorama +Requires: python3-configobj +Requires: python3-cryptography +Requires: python3-cycler +Requires: python3-cytoolz +Requires: python3-dask +Requires: python3-distributed +Requires: python3-docutils +Requires: python3-donfig +Requires: python3-eccodes-python +Requires: python3-entrypoints +Requires: python3-eumdac +Requires: python3-fasteners +Requires: python3-freezegun +Requires: python3-frozenlist +Requires: python3-fsspec +Requires: python3-h5netcdf +Requires: python3-h5py +Requires: python3-HeapDict +Requires: python3-idna +Requires: python3-imagecodecs +Requires: python3-importlib-metadata +Requires: python3-Jinja2 +Requires: python3-jmespath +Requires: python3-kiwisolver +Requires: python3-locket +Requires: python3-lz4 +Requires: python3-Mako +Requires: python3-MarkupSafe +Requires: python3-matplotlib +Requires: python3-msgpack +Requires: python3-multidict +Requires: python3-munkres +Requires: python3-netCDF4 +Requires: python3-nowcasting-datamodel +Requires: python3-numcodecs +Requires: python3-numpy +Requires: python3-packaging +Requires: python3-pandas +Requires: python3-partd +Requires: python3-Pillow +Requires: python3-pip +Requires: python3-pooch +Requires: python3-psutil +Requires: python3-psycopg2-binary +Requires: python3-pycparser +Requires: python3-pydantic +Requires: python3-pykdtree +Requires: python3-pyOpenSSL +Requires: python3-pyorbital +Requires: python3-pyparsing +Requires: python3-pyproj +Requires: python3-pyresample +Requires: python3-pyshp +Requires: python3-PySocks +Requires: python3-pyspectral +Requires: python3-dateutil +Requires: python3-geotiepoints +Requires: python3-pytz +Requires: python3-PyYAML +Requires: python3-rasterio +Requires: python3-requests +Requires: python3-s3fs +Requires: python3-satpy +Requires: python3-scipy +Requires: python3-setuptools +Requires: python3-shapely +Requires: python3-six +Requires: python3-snuggs +Requires: python3-sortedcontainers +Requires: python3-SQLAlchemy +Requires: python3-structlog +Requires: python3-tblib +Requires: python3-toolz +Requires: python3-tornado +Requires: python3-tqdm +Requires: python3-trollimage +Requires: python3-trollsift +Requires: python3-typing-extensions +Requires: python3-urllib3 +Requires: python3-wheel +Requires: python3-wrapt +Requires: python3-xarray +Requires: python3-yarl +Requires: python3-zarr +Requires: python3-zict +Requires: python3-zipp +Requires: python3-ocf-blosc2 + +%description +# Satip + +[![All Contributors](https://img.shields.io/badge/all_contributors-6-orange.svg?style=flat-square)](#contributors-) + + +[![PyPI version](https://badge.fury.io/py/satip.svg)](https://badge.fury.io/py/satip) + +[![codecov](https://codecov.io/gh/openclimatefix/Satip/branch/main/graph/badge.svg?token=GTQDR2ZZ2S)](https://codecov.io/gh/openclimatefix/Satip) + +> Satip is a library for satellite image processing, and provides all of the functionality necessary for retrieving, and storing EUMETSAT data + +
+ +### Installation + +To install the `satip` library please run: + +```bash +pip install satip +``` + +Or if you're working in the development environment you can run the following from the directory root: + +```bash +pip install -e . +``` + +#### Conda + +Or, if you want to use `conda` from the a cloned Satip repository: + +```bash +conda env create -f environment.yml +conda activate satip +pip install -e . +``` + +If you plan to work on the development of Satip then also consider installing these development tools: + +```bash +conda install pytest flake8 jedi mypy black pre-commit +pre-commit install +``` + +## Operation + +### Getting your own API key + +In order to contribute to development or just test-run some scripts, you will need your own Eumetsat-API-key. Please follow these steps: + +1. Go to https://eoportal.eumetsat.int and register an account. +2. You can log in and got to https://data.eumetsat.int/ to check available data services. From there go to your profile and choose the option "API key" or go to https://api.eumetsat.int/api-key/ directly. +3. Please make sure that you added the key and secret to your user's environment variables. + +### Downloading EUMETSAT Data + +The following command will download the last 2 hours of RSS imagery into NetCDF files at the specified location + +```bash +python satip/app.py --api-key= --api-secret= --save-dr="/path/to/saving/files/" --history="2 hours" +``` + +To download more historical data, the command below will download the native files, compress with bz2, and save into a subdirectory. + +```bash +python satip/get_raw_eumetsat_data.py --user-key= --user-secret= +``` + +### Converting Native files to Zarr +`scripts/convert_native_to_zarr.py` converts EUMETSAT `.nat` files to Zarr datasets, using very mild lossy [JPEG-XL](https://en.wikipedia.org/wiki/JPEG_XL) compression. (JPEG-XL is the "new kid on the block" of image compression algorithms). JPEG-XL makes the files about a quarter the size of the equivalent `bz2` compressed files, whilst the images are visually indistinguishable. JPEG-XL cannot represent NaNs so NaNs. JPEG-XL understands float32 values in the range `[0, 1]`. NaNs are encoded as the value `0.025`. All "real" values are in the range `[0.075, 1]`. We leave a gap between "NaNs" and "real values" because there is very slight "ringing" around areas of constant value (see [this comment for more details](https://github.com/openclimatefix/Satip/issues/67#issuecomment-1036456502)). Use `satip.jpeg_xl_float_with_nans.JpegXlFloatWithNaNs` to decode the satellite data. This class will reconstruct the NaNs and rescale the data to the range `[0, 1]`. + + +## Running in Production + +The live service uses `app.py` as the entrypoint for running the live data download for OCF's forecasting service, and has a few configuration options, configurable by command line argument or environment variable. + +`--api-key` or `API_KEY` is the EUMETSAT API key + +`--api-secret` or `API_SECRET` is the EUMETSAT API secret + +`--save-dir` or `SAVE_DIR` is the top level directory to save the output files, a `latest` subfolder will be added to that directory to contain the latest data + +`--history` or `HISTORY` is the amount of history timesteps to use in the `latest.zarr` files + +`--db-url` or `DB_URL` is the URL to the database to save to when a run has finished + +`--use-rescaler` or `USE_RESCALER` tells whether to rescale the satellite data to between 0 and 1 or not when saving to disk. Primarily used as backwards compatibility for the current production models, all new training and production Zarrs should use the rescaled data. + +## Testing + +To run tests, simply run ```pytest .``` from the root of the repository. To generate the test plots, run ```python scripts/generate_test_plots.py```. + +## Contributors ✨ + +Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/docs/en/emoji-key)): + + + + + + + + + + + + + +

Jacob Bieker
💻
Jack Kelly
💻
Ayrton Bourn
💻
Laurence Watson
💻
Notger Heinz
📖
Peter Dudfield
📖
+ + + + + + +This project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification. Contributions of any kind welcome! + + +%package -n python3-satip +Summary: Satip provides the functionality necessary for +Provides: python-satip +BuildRequires: python3-devel +BuildRequires: python3-setuptools +BuildRequires: python3-pip +%description -n python3-satip +# Satip + +[![All Contributors](https://img.shields.io/badge/all_contributors-6-orange.svg?style=flat-square)](#contributors-) + + +[![PyPI version](https://badge.fury.io/py/satip.svg)](https://badge.fury.io/py/satip) + +[![codecov](https://codecov.io/gh/openclimatefix/Satip/branch/main/graph/badge.svg?token=GTQDR2ZZ2S)](https://codecov.io/gh/openclimatefix/Satip) + +> Satip is a library for satellite image processing, and provides all of the functionality necessary for retrieving, and storing EUMETSAT data + +
+ +### Installation + +To install the `satip` library please run: + +```bash +pip install satip +``` + +Or if you're working in the development environment you can run the following from the directory root: + +```bash +pip install -e . +``` + +#### Conda + +Or, if you want to use `conda` from the a cloned Satip repository: + +```bash +conda env create -f environment.yml +conda activate satip +pip install -e . +``` + +If you plan to work on the development of Satip then also consider installing these development tools: + +```bash +conda install pytest flake8 jedi mypy black pre-commit +pre-commit install +``` + +## Operation + +### Getting your own API key + +In order to contribute to development or just test-run some scripts, you will need your own Eumetsat-API-key. Please follow these steps: + +1. Go to https://eoportal.eumetsat.int and register an account. +2. You can log in and got to https://data.eumetsat.int/ to check available data services. From there go to your profile and choose the option "API key" or go to https://api.eumetsat.int/api-key/ directly. +3. Please make sure that you added the key and secret to your user's environment variables. + +### Downloading EUMETSAT Data + +The following command will download the last 2 hours of RSS imagery into NetCDF files at the specified location + +```bash +python satip/app.py --api-key= --api-secret= --save-dr="/path/to/saving/files/" --history="2 hours" +``` + +To download more historical data, the command below will download the native files, compress with bz2, and save into a subdirectory. + +```bash +python satip/get_raw_eumetsat_data.py --user-key= --user-secret= +``` + +### Converting Native files to Zarr +`scripts/convert_native_to_zarr.py` converts EUMETSAT `.nat` files to Zarr datasets, using very mild lossy [JPEG-XL](https://en.wikipedia.org/wiki/JPEG_XL) compression. (JPEG-XL is the "new kid on the block" of image compression algorithms). JPEG-XL makes the files about a quarter the size of the equivalent `bz2` compressed files, whilst the images are visually indistinguishable. JPEG-XL cannot represent NaNs so NaNs. JPEG-XL understands float32 values in the range `[0, 1]`. NaNs are encoded as the value `0.025`. All "real" values are in the range `[0.075, 1]`. We leave a gap between "NaNs" and "real values" because there is very slight "ringing" around areas of constant value (see [this comment for more details](https://github.com/openclimatefix/Satip/issues/67#issuecomment-1036456502)). Use `satip.jpeg_xl_float_with_nans.JpegXlFloatWithNaNs` to decode the satellite data. This class will reconstruct the NaNs and rescale the data to the range `[0, 1]`. + + +## Running in Production + +The live service uses `app.py` as the entrypoint for running the live data download for OCF's forecasting service, and has a few configuration options, configurable by command line argument or environment variable. + +`--api-key` or `API_KEY` is the EUMETSAT API key + +`--api-secret` or `API_SECRET` is the EUMETSAT API secret + +`--save-dir` or `SAVE_DIR` is the top level directory to save the output files, a `latest` subfolder will be added to that directory to contain the latest data + +`--history` or `HISTORY` is the amount of history timesteps to use in the `latest.zarr` files + +`--db-url` or `DB_URL` is the URL to the database to save to when a run has finished + +`--use-rescaler` or `USE_RESCALER` tells whether to rescale the satellite data to between 0 and 1 or not when saving to disk. Primarily used as backwards compatibility for the current production models, all new training and production Zarrs should use the rescaled data. + +## Testing + +To run tests, simply run ```pytest .``` from the root of the repository. To generate the test plots, run ```python scripts/generate_test_plots.py```. + +## Contributors ✨ + +Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/docs/en/emoji-key)): + + + + + + + + + + + + + +

Jacob Bieker
💻
Jack Kelly
💻
Ayrton Bourn
💻
Laurence Watson
💻
Notger Heinz
📖
Peter Dudfield
📖
+ + + + + + +This project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification. Contributions of any kind welcome! + + +%package help +Summary: Development documents and examples for satip +Provides: python3-satip-doc +%description help +# Satip + +[![All Contributors](https://img.shields.io/badge/all_contributors-6-orange.svg?style=flat-square)](#contributors-) + + +[![PyPI version](https://badge.fury.io/py/satip.svg)](https://badge.fury.io/py/satip) + +[![codecov](https://codecov.io/gh/openclimatefix/Satip/branch/main/graph/badge.svg?token=GTQDR2ZZ2S)](https://codecov.io/gh/openclimatefix/Satip) + +> Satip is a library for satellite image processing, and provides all of the functionality necessary for retrieving, and storing EUMETSAT data + +
+ +### Installation + +To install the `satip` library please run: + +```bash +pip install satip +``` + +Or if you're working in the development environment you can run the following from the directory root: + +```bash +pip install -e . +``` + +#### Conda + +Or, if you want to use `conda` from the a cloned Satip repository: + +```bash +conda env create -f environment.yml +conda activate satip +pip install -e . +``` + +If you plan to work on the development of Satip then also consider installing these development tools: + +```bash +conda install pytest flake8 jedi mypy black pre-commit +pre-commit install +``` + +## Operation + +### Getting your own API key + +In order to contribute to development or just test-run some scripts, you will need your own Eumetsat-API-key. Please follow these steps: + +1. Go to https://eoportal.eumetsat.int and register an account. +2. You can log in and got to https://data.eumetsat.int/ to check available data services. From there go to your profile and choose the option "API key" or go to https://api.eumetsat.int/api-key/ directly. +3. Please make sure that you added the key and secret to your user's environment variables. + +### Downloading EUMETSAT Data + +The following command will download the last 2 hours of RSS imagery into NetCDF files at the specified location + +```bash +python satip/app.py --api-key= --api-secret= --save-dr="/path/to/saving/files/" --history="2 hours" +``` + +To download more historical data, the command below will download the native files, compress with bz2, and save into a subdirectory. + +```bash +python satip/get_raw_eumetsat_data.py --user-key= --user-secret= +``` + +### Converting Native files to Zarr +`scripts/convert_native_to_zarr.py` converts EUMETSAT `.nat` files to Zarr datasets, using very mild lossy [JPEG-XL](https://en.wikipedia.org/wiki/JPEG_XL) compression. (JPEG-XL is the "new kid on the block" of image compression algorithms). JPEG-XL makes the files about a quarter the size of the equivalent `bz2` compressed files, whilst the images are visually indistinguishable. JPEG-XL cannot represent NaNs so NaNs. JPEG-XL understands float32 values in the range `[0, 1]`. NaNs are encoded as the value `0.025`. All "real" values are in the range `[0.075, 1]`. We leave a gap between "NaNs" and "real values" because there is very slight "ringing" around areas of constant value (see [this comment for more details](https://github.com/openclimatefix/Satip/issues/67#issuecomment-1036456502)). Use `satip.jpeg_xl_float_with_nans.JpegXlFloatWithNaNs` to decode the satellite data. This class will reconstruct the NaNs and rescale the data to the range `[0, 1]`. + + +## Running in Production + +The live service uses `app.py` as the entrypoint for running the live data download for OCF's forecasting service, and has a few configuration options, configurable by command line argument or environment variable. + +`--api-key` or `API_KEY` is the EUMETSAT API key + +`--api-secret` or `API_SECRET` is the EUMETSAT API secret + +`--save-dir` or `SAVE_DIR` is the top level directory to save the output files, a `latest` subfolder will be added to that directory to contain the latest data + +`--history` or `HISTORY` is the amount of history timesteps to use in the `latest.zarr` files + +`--db-url` or `DB_URL` is the URL to the database to save to when a run has finished + +`--use-rescaler` or `USE_RESCALER` tells whether to rescale the satellite data to between 0 and 1 or not when saving to disk. Primarily used as backwards compatibility for the current production models, all new training and production Zarrs should use the rescaled data. + +## Testing + +To run tests, simply run ```pytest .``` from the root of the repository. To generate the test plots, run ```python scripts/generate_test_plots.py```. + +## Contributors ✨ + +Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/docs/en/emoji-key)): + + + + + + + + + + + + + +

Jacob Bieker
💻
Jack Kelly
💻
Ayrton Bourn
💻
Laurence Watson
💻
Notger Heinz
📖
Peter Dudfield
📖
+ + + + + + +This project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification. Contributions of any kind welcome! + + +%prep +%autosetup -n satip-2.11.4 + +%build +%py3_build + +%install +%py3_install +install -d -m755 %{buildroot}/%{_pkgdocdir} +if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi +if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi +if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi +if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi +pushd %{buildroot} +if [ -d usr/lib ]; then + find usr/lib -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/lib64 ]; then + find usr/lib64 -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/bin ]; then + find usr/bin -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/sbin ]; then + find usr/sbin -type f -printf "/%h/%f\n" >> filelist.lst +fi +touch doclist.lst +if [ -d usr/share/man ]; then + find usr/share/man -type f -printf "/%h/%f.gz\n" >> doclist.lst +fi +popd +mv %{buildroot}/filelist.lst . +mv %{buildroot}/doclist.lst . + +%files -n python3-satip -f filelist.lst +%dir %{python3_sitelib}/* + +%files help -f doclist.lst +%{_docdir}/* + +%changelog +* Tue Apr 11 2023 Python_Bot - 2.11.4-1 +- Package Spec generated -- cgit v1.2.3