diff options
| -rw-r--r-- | .gitignore | 1 | ||||
| -rw-r--r-- | python-ckanext-dcor-depot.spec | 229 | ||||
| -rw-r--r-- | sources | 1 |
3 files changed, 231 insertions, 0 deletions
@@ -0,0 +1 @@ +/ckanext-dcor_depot-0.11.0.tar.gz diff --git a/python-ckanext-dcor-depot.spec b/python-ckanext-dcor-depot.spec new file mode 100644 index 0000000..79f7355 --- /dev/null +++ b/python-ckanext-dcor-depot.spec @@ -0,0 +1,229 @@ +%global _empty_manifest_terminate_build 0 +Name: python-ckanext-dcor-depot +Version: 0.11.0 +Release: 1 +Summary: Manages data storage for CKAN/DCOR (import, symlink, etc.) +License: AGPLv3+ +URL: https://github.com/DCOR-dev/ckanext-dcor_depot +Source0: https://mirrors.nju.edu.cn/pypi/web/packages/1c/75/54b3065125dd70a3950760d2bbd76e84633c8e30eb592bdc5a6426b33d2f/ckanext-dcor_depot-0.11.0.tar.gz +BuildArch: noarch + +Requires: python3-click +Requires: python3-dclab +Requires: python3-dcor-shared +Requires: python3-h5py +Requires: python3-html2text +Requires: python3-numpy +Requires: python3-requests + +%description +|PyPI Version| |Build Status| |Coverage Status| +This plugin manages how data are stored in DCOR. There are two types of +files in DCOR: +1. Resources uploaded by users, imported from figshare, or + imported from a data archive +2. Ancillary files that are generated upon resource creation, such as + condensed DC data, preview images (see + `ckanext-dc_view <https://github.com/DCOR-dev/ckanext-dc_view>`_). +This plugin implements: +- Data storage management. All resources uploaded by a user are moved + to ``/data/users-HOSTNAME/USERNAME-ORGNAME/PK/ID/PKGNAME_RESID_RESNAME`` + and symlinks are created in ``/data/ckan-HOSTNAME/resources/RES/OUR/CEID``. + CKAN itself will not notice this. The idea is to have a filesystem overview + about the datasets of each user. +- Import datasets from figshare. Existing datasets from figshare are + downloaded to the ``/data/depots/figshare`` directory and, upon resource + creation, symlinked there from ``/data/ckan-HOSTNAME/resources/RES/OUR/CEID`` + (Note that this is an exemption of the data storage management described + above). When running the following command, the "figshare-import" organization + is created and the datasets listed in ``figshare_dois.txt`` are added to CKAN: + ckan import-figshare +- Populate an internal depot from RT-DC data stored in tar archives. This + is part of an effort to have automated imports of RT-DC data from other + sources. The idea is to move experimental data to the DCOR server in + tar archives and DCOR can then populate the internal depot with it. + The location of the internal depot is ``/data/depots/internal/`` + and it follows a very specific directory structure + ``201X/2019-08/20/2019-08-20_1126_c083de*`` where the path is generated + from the acquisition date, time, and part of the hash (``c083de``) of + the original data file. According to this scheme, all files with the + same path stem belong to one dataset: + - ``2019-08-20_1126_c083de.sha256sums`` a file containing SHA256 sums + - ``2019-08-20_1126_c083de_v1.rtdc`` the actual measurement + - ``2019-08-20_1126_c083de_v1_condensed.rtdc`` the condensed dataset + - ``2019-08-20_1126_c083de_ad1_m001_bg.png`` an ancillary image + - ``2019-08-20_1126_c083de_ad2_m002_bg.png`` another ancillary image + - ... + ckan depotize-archive +- Import datasets from the internal depot. The previous command + ``depotize-archive`` just populates the depot directory structure. + To make the datasets available in CKAN, this step must be performed: + ckan import-internal +- Upgrade datasets from the internal depot. If you find issues or have to add + new versions of an .rtdc resource, simply create the `_v2.rtdc` files and + run this command: + ckan upgrade-internal +- CLI for symlinking datasets that have failed to symlink before: + ckan run-jobs-dcor-depot +- CLI for appending a resource to a dataset + ckan append-resource +Please make sure that the necessary file permissions are given in ``/data``. + +%package -n python3-ckanext-dcor-depot +Summary: Manages data storage for CKAN/DCOR (import, symlink, etc.) +Provides: python-ckanext-dcor-depot +BuildRequires: python3-devel +BuildRequires: python3-setuptools +BuildRequires: python3-pip +%description -n python3-ckanext-dcor-depot +|PyPI Version| |Build Status| |Coverage Status| +This plugin manages how data are stored in DCOR. There are two types of +files in DCOR: +1. Resources uploaded by users, imported from figshare, or + imported from a data archive +2. Ancillary files that are generated upon resource creation, such as + condensed DC data, preview images (see + `ckanext-dc_view <https://github.com/DCOR-dev/ckanext-dc_view>`_). +This plugin implements: +- Data storage management. All resources uploaded by a user are moved + to ``/data/users-HOSTNAME/USERNAME-ORGNAME/PK/ID/PKGNAME_RESID_RESNAME`` + and symlinks are created in ``/data/ckan-HOSTNAME/resources/RES/OUR/CEID``. + CKAN itself will not notice this. The idea is to have a filesystem overview + about the datasets of each user. +- Import datasets from figshare. Existing datasets from figshare are + downloaded to the ``/data/depots/figshare`` directory and, upon resource + creation, symlinked there from ``/data/ckan-HOSTNAME/resources/RES/OUR/CEID`` + (Note that this is an exemption of the data storage management described + above). When running the following command, the "figshare-import" organization + is created and the datasets listed in ``figshare_dois.txt`` are added to CKAN: + ckan import-figshare +- Populate an internal depot from RT-DC data stored in tar archives. This + is part of an effort to have automated imports of RT-DC data from other + sources. The idea is to move experimental data to the DCOR server in + tar archives and DCOR can then populate the internal depot with it. + The location of the internal depot is ``/data/depots/internal/`` + and it follows a very specific directory structure + ``201X/2019-08/20/2019-08-20_1126_c083de*`` where the path is generated + from the acquisition date, time, and part of the hash (``c083de``) of + the original data file. According to this scheme, all files with the + same path stem belong to one dataset: + - ``2019-08-20_1126_c083de.sha256sums`` a file containing SHA256 sums + - ``2019-08-20_1126_c083de_v1.rtdc`` the actual measurement + - ``2019-08-20_1126_c083de_v1_condensed.rtdc`` the condensed dataset + - ``2019-08-20_1126_c083de_ad1_m001_bg.png`` an ancillary image + - ``2019-08-20_1126_c083de_ad2_m002_bg.png`` another ancillary image + - ... + ckan depotize-archive +- Import datasets from the internal depot. The previous command + ``depotize-archive`` just populates the depot directory structure. + To make the datasets available in CKAN, this step must be performed: + ckan import-internal +- Upgrade datasets from the internal depot. If you find issues or have to add + new versions of an .rtdc resource, simply create the `_v2.rtdc` files and + run this command: + ckan upgrade-internal +- CLI for symlinking datasets that have failed to symlink before: + ckan run-jobs-dcor-depot +- CLI for appending a resource to a dataset + ckan append-resource +Please make sure that the necessary file permissions are given in ``/data``. + +%package help +Summary: Development documents and examples for ckanext-dcor-depot +Provides: python3-ckanext-dcor-depot-doc +%description help +|PyPI Version| |Build Status| |Coverage Status| +This plugin manages how data are stored in DCOR. There are two types of +files in DCOR: +1. Resources uploaded by users, imported from figshare, or + imported from a data archive +2. Ancillary files that are generated upon resource creation, such as + condensed DC data, preview images (see + `ckanext-dc_view <https://github.com/DCOR-dev/ckanext-dc_view>`_). +This plugin implements: +- Data storage management. All resources uploaded by a user are moved + to ``/data/users-HOSTNAME/USERNAME-ORGNAME/PK/ID/PKGNAME_RESID_RESNAME`` + and symlinks are created in ``/data/ckan-HOSTNAME/resources/RES/OUR/CEID``. + CKAN itself will not notice this. The idea is to have a filesystem overview + about the datasets of each user. +- Import datasets from figshare. Existing datasets from figshare are + downloaded to the ``/data/depots/figshare`` directory and, upon resource + creation, symlinked there from ``/data/ckan-HOSTNAME/resources/RES/OUR/CEID`` + (Note that this is an exemption of the data storage management described + above). When running the following command, the "figshare-import" organization + is created and the datasets listed in ``figshare_dois.txt`` are added to CKAN: + ckan import-figshare +- Populate an internal depot from RT-DC data stored in tar archives. This + is part of an effort to have automated imports of RT-DC data from other + sources. The idea is to move experimental data to the DCOR server in + tar archives and DCOR can then populate the internal depot with it. + The location of the internal depot is ``/data/depots/internal/`` + and it follows a very specific directory structure + ``201X/2019-08/20/2019-08-20_1126_c083de*`` where the path is generated + from the acquisition date, time, and part of the hash (``c083de``) of + the original data file. According to this scheme, all files with the + same path stem belong to one dataset: + - ``2019-08-20_1126_c083de.sha256sums`` a file containing SHA256 sums + - ``2019-08-20_1126_c083de_v1.rtdc`` the actual measurement + - ``2019-08-20_1126_c083de_v1_condensed.rtdc`` the condensed dataset + - ``2019-08-20_1126_c083de_ad1_m001_bg.png`` an ancillary image + - ``2019-08-20_1126_c083de_ad2_m002_bg.png`` another ancillary image + - ... + ckan depotize-archive +- Import datasets from the internal depot. The previous command + ``depotize-archive`` just populates the depot directory structure. + To make the datasets available in CKAN, this step must be performed: + ckan import-internal +- Upgrade datasets from the internal depot. If you find issues or have to add + new versions of an .rtdc resource, simply create the `_v2.rtdc` files and + run this command: + ckan upgrade-internal +- CLI for symlinking datasets that have failed to symlink before: + ckan run-jobs-dcor-depot +- CLI for appending a resource to a dataset + ckan append-resource +Please make sure that the necessary file permissions are given in ``/data``. + +%prep +%autosetup -n ckanext-dcor-depot-0.11.0 + +%build +%py3_build + +%install +%py3_install +install -d -m755 %{buildroot}/%{_pkgdocdir} +if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi +if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi +if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi +if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi +pushd %{buildroot} +if [ -d usr/lib ]; then + find usr/lib -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/lib64 ]; then + find usr/lib64 -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/bin ]; then + find usr/bin -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/sbin ]; then + find usr/sbin -type f -printf "/%h/%f\n" >> filelist.lst +fi +touch doclist.lst +if [ -d usr/share/man ]; then + find usr/share/man -type f -printf "/%h/%f.gz\n" >> doclist.lst +fi +popd +mv %{buildroot}/filelist.lst . +mv %{buildroot}/doclist.lst . + +%files -n python3-ckanext-dcor-depot -f filelist.lst +%dir %{python3_sitelib}/* + +%files help -f doclist.lst +%{_docdir}/* + +%changelog +* Fri May 05 2023 Python_Bot <Python_Bot@openeuler.org> - 0.11.0-1 +- Package Spec generated @@ -0,0 +1 @@ +b36d7c7a61f0c8d737c73437431d665a ckanext-dcor_depot-0.11.0.tar.gz |