summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorCoprDistGit <infra@openeuler.org>2023-05-05 14:21:40 +0000
committerCoprDistGit <infra@openeuler.org>2023-05-05 14:21:40 +0000
commit9a70a20d821c801c5ebd40bd80113cb9c8c1eb06 (patch)
treeeb6a2be198904067f6b905d1890111142b4583de
parent59355cd1e86a6c66f581fa0484d63f0f59ede1b7 (diff)
automatic import of python-ckanext-dcor-depotopeneuler20.03
-rw-r--r--.gitignore1
-rw-r--r--python-ckanext-dcor-depot.spec229
-rw-r--r--sources1
3 files changed, 231 insertions, 0 deletions
diff --git a/.gitignore b/.gitignore
index e69de29..162f08f 100644
--- a/.gitignore
+++ b/.gitignore
@@ -0,0 +1 @@
+/ckanext-dcor_depot-0.11.0.tar.gz
diff --git a/python-ckanext-dcor-depot.spec b/python-ckanext-dcor-depot.spec
new file mode 100644
index 0000000..79f7355
--- /dev/null
+++ b/python-ckanext-dcor-depot.spec
@@ -0,0 +1,229 @@
+%global _empty_manifest_terminate_build 0
+Name: python-ckanext-dcor-depot
+Version: 0.11.0
+Release: 1
+Summary: Manages data storage for CKAN/DCOR (import, symlink, etc.)
+License: AGPLv3+
+URL: https://github.com/DCOR-dev/ckanext-dcor_depot
+Source0: https://mirrors.nju.edu.cn/pypi/web/packages/1c/75/54b3065125dd70a3950760d2bbd76e84633c8e30eb592bdc5a6426b33d2f/ckanext-dcor_depot-0.11.0.tar.gz
+BuildArch: noarch
+
+Requires: python3-click
+Requires: python3-dclab
+Requires: python3-dcor-shared
+Requires: python3-h5py
+Requires: python3-html2text
+Requires: python3-numpy
+Requires: python3-requests
+
+%description
+|PyPI Version| |Build Status| |Coverage Status|
+This plugin manages how data are stored in DCOR. There are two types of
+files in DCOR:
+1. Resources uploaded by users, imported from figshare, or
+ imported from a data archive
+2. Ancillary files that are generated upon resource creation, such as
+ condensed DC data, preview images (see
+ `ckanext-dc_view <https://github.com/DCOR-dev/ckanext-dc_view>`_).
+This plugin implements:
+- Data storage management. All resources uploaded by a user are moved
+ to ``/data/users-HOSTNAME/USERNAME-ORGNAME/PK/ID/PKGNAME_RESID_RESNAME``
+ and symlinks are created in ``/data/ckan-HOSTNAME/resources/RES/OUR/CEID``.
+ CKAN itself will not notice this. The idea is to have a filesystem overview
+ about the datasets of each user.
+- Import datasets from figshare. Existing datasets from figshare are
+ downloaded to the ``/data/depots/figshare`` directory and, upon resource
+ creation, symlinked there from ``/data/ckan-HOSTNAME/resources/RES/OUR/CEID``
+ (Note that this is an exemption of the data storage management described
+ above). When running the following command, the "figshare-import" organization
+ is created and the datasets listed in ``figshare_dois.txt`` are added to CKAN:
+ ckan import-figshare
+- Populate an internal depot from RT-DC data stored in tar archives. This
+ is part of an effort to have automated imports of RT-DC data from other
+ sources. The idea is to move experimental data to the DCOR server in
+ tar archives and DCOR can then populate the internal depot with it.
+ The location of the internal depot is ``/data/depots/internal/``
+ and it follows a very specific directory structure
+ ``201X/2019-08/20/2019-08-20_1126_c083de*`` where the path is generated
+ from the acquisition date, time, and part of the hash (``c083de``) of
+ the original data file. According to this scheme, all files with the
+ same path stem belong to one dataset:
+ - ``2019-08-20_1126_c083de.sha256sums`` a file containing SHA256 sums
+ - ``2019-08-20_1126_c083de_v1.rtdc`` the actual measurement
+ - ``2019-08-20_1126_c083de_v1_condensed.rtdc`` the condensed dataset
+ - ``2019-08-20_1126_c083de_ad1_m001_bg.png`` an ancillary image
+ - ``2019-08-20_1126_c083de_ad2_m002_bg.png`` another ancillary image
+ - ...
+ ckan depotize-archive
+- Import datasets from the internal depot. The previous command
+ ``depotize-archive`` just populates the depot directory structure.
+ To make the datasets available in CKAN, this step must be performed:
+ ckan import-internal
+- Upgrade datasets from the internal depot. If you find issues or have to add
+ new versions of an .rtdc resource, simply create the `_v2.rtdc` files and
+ run this command:
+ ckan upgrade-internal
+- CLI for symlinking datasets that have failed to symlink before:
+ ckan run-jobs-dcor-depot
+- CLI for appending a resource to a dataset
+ ckan append-resource
+Please make sure that the necessary file permissions are given in ``/data``.
+
+%package -n python3-ckanext-dcor-depot
+Summary: Manages data storage for CKAN/DCOR (import, symlink, etc.)
+Provides: python-ckanext-dcor-depot
+BuildRequires: python3-devel
+BuildRequires: python3-setuptools
+BuildRequires: python3-pip
+%description -n python3-ckanext-dcor-depot
+|PyPI Version| |Build Status| |Coverage Status|
+This plugin manages how data are stored in DCOR. There are two types of
+files in DCOR:
+1. Resources uploaded by users, imported from figshare, or
+ imported from a data archive
+2. Ancillary files that are generated upon resource creation, such as
+ condensed DC data, preview images (see
+ `ckanext-dc_view <https://github.com/DCOR-dev/ckanext-dc_view>`_).
+This plugin implements:
+- Data storage management. All resources uploaded by a user are moved
+ to ``/data/users-HOSTNAME/USERNAME-ORGNAME/PK/ID/PKGNAME_RESID_RESNAME``
+ and symlinks are created in ``/data/ckan-HOSTNAME/resources/RES/OUR/CEID``.
+ CKAN itself will not notice this. The idea is to have a filesystem overview
+ about the datasets of each user.
+- Import datasets from figshare. Existing datasets from figshare are
+ downloaded to the ``/data/depots/figshare`` directory and, upon resource
+ creation, symlinked there from ``/data/ckan-HOSTNAME/resources/RES/OUR/CEID``
+ (Note that this is an exemption of the data storage management described
+ above). When running the following command, the "figshare-import" organization
+ is created and the datasets listed in ``figshare_dois.txt`` are added to CKAN:
+ ckan import-figshare
+- Populate an internal depot from RT-DC data stored in tar archives. This
+ is part of an effort to have automated imports of RT-DC data from other
+ sources. The idea is to move experimental data to the DCOR server in
+ tar archives and DCOR can then populate the internal depot with it.
+ The location of the internal depot is ``/data/depots/internal/``
+ and it follows a very specific directory structure
+ ``201X/2019-08/20/2019-08-20_1126_c083de*`` where the path is generated
+ from the acquisition date, time, and part of the hash (``c083de``) of
+ the original data file. According to this scheme, all files with the
+ same path stem belong to one dataset:
+ - ``2019-08-20_1126_c083de.sha256sums`` a file containing SHA256 sums
+ - ``2019-08-20_1126_c083de_v1.rtdc`` the actual measurement
+ - ``2019-08-20_1126_c083de_v1_condensed.rtdc`` the condensed dataset
+ - ``2019-08-20_1126_c083de_ad1_m001_bg.png`` an ancillary image
+ - ``2019-08-20_1126_c083de_ad2_m002_bg.png`` another ancillary image
+ - ...
+ ckan depotize-archive
+- Import datasets from the internal depot. The previous command
+ ``depotize-archive`` just populates the depot directory structure.
+ To make the datasets available in CKAN, this step must be performed:
+ ckan import-internal
+- Upgrade datasets from the internal depot. If you find issues or have to add
+ new versions of an .rtdc resource, simply create the `_v2.rtdc` files and
+ run this command:
+ ckan upgrade-internal
+- CLI for symlinking datasets that have failed to symlink before:
+ ckan run-jobs-dcor-depot
+- CLI for appending a resource to a dataset
+ ckan append-resource
+Please make sure that the necessary file permissions are given in ``/data``.
+
+%package help
+Summary: Development documents and examples for ckanext-dcor-depot
+Provides: python3-ckanext-dcor-depot-doc
+%description help
+|PyPI Version| |Build Status| |Coverage Status|
+This plugin manages how data are stored in DCOR. There are two types of
+files in DCOR:
+1. Resources uploaded by users, imported from figshare, or
+ imported from a data archive
+2. Ancillary files that are generated upon resource creation, such as
+ condensed DC data, preview images (see
+ `ckanext-dc_view <https://github.com/DCOR-dev/ckanext-dc_view>`_).
+This plugin implements:
+- Data storage management. All resources uploaded by a user are moved
+ to ``/data/users-HOSTNAME/USERNAME-ORGNAME/PK/ID/PKGNAME_RESID_RESNAME``
+ and symlinks are created in ``/data/ckan-HOSTNAME/resources/RES/OUR/CEID``.
+ CKAN itself will not notice this. The idea is to have a filesystem overview
+ about the datasets of each user.
+- Import datasets from figshare. Existing datasets from figshare are
+ downloaded to the ``/data/depots/figshare`` directory and, upon resource
+ creation, symlinked there from ``/data/ckan-HOSTNAME/resources/RES/OUR/CEID``
+ (Note that this is an exemption of the data storage management described
+ above). When running the following command, the "figshare-import" organization
+ is created and the datasets listed in ``figshare_dois.txt`` are added to CKAN:
+ ckan import-figshare
+- Populate an internal depot from RT-DC data stored in tar archives. This
+ is part of an effort to have automated imports of RT-DC data from other
+ sources. The idea is to move experimental data to the DCOR server in
+ tar archives and DCOR can then populate the internal depot with it.
+ The location of the internal depot is ``/data/depots/internal/``
+ and it follows a very specific directory structure
+ ``201X/2019-08/20/2019-08-20_1126_c083de*`` where the path is generated
+ from the acquisition date, time, and part of the hash (``c083de``) of
+ the original data file. According to this scheme, all files with the
+ same path stem belong to one dataset:
+ - ``2019-08-20_1126_c083de.sha256sums`` a file containing SHA256 sums
+ - ``2019-08-20_1126_c083de_v1.rtdc`` the actual measurement
+ - ``2019-08-20_1126_c083de_v1_condensed.rtdc`` the condensed dataset
+ - ``2019-08-20_1126_c083de_ad1_m001_bg.png`` an ancillary image
+ - ``2019-08-20_1126_c083de_ad2_m002_bg.png`` another ancillary image
+ - ...
+ ckan depotize-archive
+- Import datasets from the internal depot. The previous command
+ ``depotize-archive`` just populates the depot directory structure.
+ To make the datasets available in CKAN, this step must be performed:
+ ckan import-internal
+- Upgrade datasets from the internal depot. If you find issues or have to add
+ new versions of an .rtdc resource, simply create the `_v2.rtdc` files and
+ run this command:
+ ckan upgrade-internal
+- CLI for symlinking datasets that have failed to symlink before:
+ ckan run-jobs-dcor-depot
+- CLI for appending a resource to a dataset
+ ckan append-resource
+Please make sure that the necessary file permissions are given in ``/data``.
+
+%prep
+%autosetup -n ckanext-dcor-depot-0.11.0
+
+%build
+%py3_build
+
+%install
+%py3_install
+install -d -m755 %{buildroot}/%{_pkgdocdir}
+if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi
+if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi
+if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi
+if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi
+pushd %{buildroot}
+if [ -d usr/lib ]; then
+ find usr/lib -type f -printf "/%h/%f\n" >> filelist.lst
+fi
+if [ -d usr/lib64 ]; then
+ find usr/lib64 -type f -printf "/%h/%f\n" >> filelist.lst
+fi
+if [ -d usr/bin ]; then
+ find usr/bin -type f -printf "/%h/%f\n" >> filelist.lst
+fi
+if [ -d usr/sbin ]; then
+ find usr/sbin -type f -printf "/%h/%f\n" >> filelist.lst
+fi
+touch doclist.lst
+if [ -d usr/share/man ]; then
+ find usr/share/man -type f -printf "/%h/%f.gz\n" >> doclist.lst
+fi
+popd
+mv %{buildroot}/filelist.lst .
+mv %{buildroot}/doclist.lst .
+
+%files -n python3-ckanext-dcor-depot -f filelist.lst
+%dir %{python3_sitelib}/*
+
+%files help -f doclist.lst
+%{_docdir}/*
+
+%changelog
+* Fri May 05 2023 Python_Bot <Python_Bot@openeuler.org> - 0.11.0-1
+- Package Spec generated
diff --git a/sources b/sources
new file mode 100644
index 0000000..8695d57
--- /dev/null
+++ b/sources
@@ -0,0 +1 @@
+b36d7c7a61f0c8d737c73437431d665a ckanext-dcor_depot-0.11.0.tar.gz