1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
|
%global _empty_manifest_terminate_build 0
Name: python-fastdownload
Version: 0.0.7
Release: 1
Summary: A general purpose data downloading library.
License: Apache Software License 2.0
URL: https://github.com/fastai/fastdownload/tree/master/
Source0: https://mirrors.nju.edu.cn/pypi/web/packages/08/be/d2c2e8dc81aa88316ed27f1bd707440a83a7420c35e67c0b143fe81aeca9/fastdownload-0.0.7.tar.gz
BuildArch: noarch
Requires: python3-fastprogress
Requires: python3-fastcore
%description
# fastdownload
> Easily download, verify, and extract archives
If you have datasets or other archives that you want to make available to your users, and ensure they always have the latest versions and that they are downloaded correctly, `fastdownload` can help.
## Install
Using pip:
pip install fastdownload
...or using conda:
conda install -c fastai fastdownload
## What's this about?
The situation where you might want to use `fastdownload` is where you have one or more URLs pointing at some archives you want to make available, and you want to ensure that your users download those archives correctly, have the latest version, and that it's as easy as possible for them to access the information in those archives.
Your user just calls a single method, `FastDownload.get`, passing the URL required, and the URL will be downloaded and extracted to the directories you choose. The path to the extracted file is returned. If that URL has already been downloaded, then the cached archive or contents will be used automatically. However, if that size or hash of the archive is different to what it should be, then the user will be informed, and a new version will be downloaded.
In the future, you may want to update one or more of your archives. When you do so, `fastdownload` will ensure your users have the latest version, by checking their downloaded archives against your updated file size and hash information.
For instance, `fastai` uses `fastdownload` to provide access to datasets for deep learning. `fastai` users can download and extract them with a single command, using the return value to access the files. The files are automatically placed in appropriate subdirectories of a `.fastai` folder in the user's homedir. If a dataset is updated, users are informed the next time they use the dataset, and the latest version is automatically downloaded and extracted for them.
## Usage: downloading files
When your users download an archive, `fastdownload` will automatically save it to a directory, check if the size and hash matches, and extract the contents. Minimal usage for downloading and extracting is:
```python
from fastdownload import FastDownload
d = FastDownload()
path = d.get('https://...')
```
After this, `path` will contain the path where the extracted files are located. By default, archives are saved to `{base}/archive`, and extracted to `{base}/data`. `{base}` defaults to `~/.fastdownload`. If there is more than one file or folder in the root of the downloaded archive, then a new folder is created in `data` for the contents.
Instead of `get`, use `download` to download the URL without extracting it, or `extract` to extract the URL without downloading it (assuming it's already been downloaded to the `archive` directory). All of these methods accept a `force` parameter which will download/extract the archive even if it's already present.
You can change any or all of the `base`, `archive`, and `data` paths by passing them to `FastDownload`:
```python
d = FastDownload(base='~/.mypath', archive='downloaded', data='extracted')
```
You can remove the cached archive file and/or the extracted contents with `rm`:
```python
d.rm('https://...')
```
## Usage: making archives available to download
`fastdownload` will add a file `download_checks.py` to your Python module which contains file sizes and hashes for your archives. The file is located in the same directory as a module you choose, e.g.:
```python
d = FastDownload(module=fastai.some_module)
```
Then use `update` to create or update the size and hash for a URL:
```python
d.update('https://...')
```
You will now find there is a file called `download_checks.py` in the same directory where `fastai.some_module` is located, which contains a Python dict with the URL, size, and hash for this file. If you've downloaded this file before to your `archive` path then it will be used, instead of downloading a new copy. Use `get(force=True)` first to download a new copy if even you have it in your archive.
## Config file
If there is a file called `config.ini` in your `base` directory, then keys `archive` and `data` will be used as the default values for `FastDownload`. The file should be in [configparser](https://docs.python.org/3/library/configparser.html) format. Here's a sample `config.ini`:
```
[DEFAULT]
archive = downloaded
data = extracted
```
If there is no ini file present, one will be automatically created for for you using the details you pass to `FastDownload`.
You can add any additional key/value pairs to the config file that you want. When you call `FastDownload.get` pass `extract_key` to use a key other than `data` for choosing a location to extract to.
%package -n python3-fastdownload
Summary: A general purpose data downloading library.
Provides: python-fastdownload
BuildRequires: python3-devel
BuildRequires: python3-setuptools
BuildRequires: python3-pip
%description -n python3-fastdownload
# fastdownload
> Easily download, verify, and extract archives
If you have datasets or other archives that you want to make available to your users, and ensure they always have the latest versions and that they are downloaded correctly, `fastdownload` can help.
## Install
Using pip:
pip install fastdownload
...or using conda:
conda install -c fastai fastdownload
## What's this about?
The situation where you might want to use `fastdownload` is where you have one or more URLs pointing at some archives you want to make available, and you want to ensure that your users download those archives correctly, have the latest version, and that it's as easy as possible for them to access the information in those archives.
Your user just calls a single method, `FastDownload.get`, passing the URL required, and the URL will be downloaded and extracted to the directories you choose. The path to the extracted file is returned. If that URL has already been downloaded, then the cached archive or contents will be used automatically. However, if that size or hash of the archive is different to what it should be, then the user will be informed, and a new version will be downloaded.
In the future, you may want to update one or more of your archives. When you do so, `fastdownload` will ensure your users have the latest version, by checking their downloaded archives against your updated file size and hash information.
For instance, `fastai` uses `fastdownload` to provide access to datasets for deep learning. `fastai` users can download and extract them with a single command, using the return value to access the files. The files are automatically placed in appropriate subdirectories of a `.fastai` folder in the user's homedir. If a dataset is updated, users are informed the next time they use the dataset, and the latest version is automatically downloaded and extracted for them.
## Usage: downloading files
When your users download an archive, `fastdownload` will automatically save it to a directory, check if the size and hash matches, and extract the contents. Minimal usage for downloading and extracting is:
```python
from fastdownload import FastDownload
d = FastDownload()
path = d.get('https://...')
```
After this, `path` will contain the path where the extracted files are located. By default, archives are saved to `{base}/archive`, and extracted to `{base}/data`. `{base}` defaults to `~/.fastdownload`. If there is more than one file or folder in the root of the downloaded archive, then a new folder is created in `data` for the contents.
Instead of `get`, use `download` to download the URL without extracting it, or `extract` to extract the URL without downloading it (assuming it's already been downloaded to the `archive` directory). All of these methods accept a `force` parameter which will download/extract the archive even if it's already present.
You can change any or all of the `base`, `archive`, and `data` paths by passing them to `FastDownload`:
```python
d = FastDownload(base='~/.mypath', archive='downloaded', data='extracted')
```
You can remove the cached archive file and/or the extracted contents with `rm`:
```python
d.rm('https://...')
```
## Usage: making archives available to download
`fastdownload` will add a file `download_checks.py` to your Python module which contains file sizes and hashes for your archives. The file is located in the same directory as a module you choose, e.g.:
```python
d = FastDownload(module=fastai.some_module)
```
Then use `update` to create or update the size and hash for a URL:
```python
d.update('https://...')
```
You will now find there is a file called `download_checks.py` in the same directory where `fastai.some_module` is located, which contains a Python dict with the URL, size, and hash for this file. If you've downloaded this file before to your `archive` path then it will be used, instead of downloading a new copy. Use `get(force=True)` first to download a new copy if even you have it in your archive.
## Config file
If there is a file called `config.ini` in your `base` directory, then keys `archive` and `data` will be used as the default values for `FastDownload`. The file should be in [configparser](https://docs.python.org/3/library/configparser.html) format. Here's a sample `config.ini`:
```
[DEFAULT]
archive = downloaded
data = extracted
```
If there is no ini file present, one will be automatically created for for you using the details you pass to `FastDownload`.
You can add any additional key/value pairs to the config file that you want. When you call `FastDownload.get` pass `extract_key` to use a key other than `data` for choosing a location to extract to.
%package help
Summary: Development documents and examples for fastdownload
Provides: python3-fastdownload-doc
%description help
# fastdownload
> Easily download, verify, and extract archives
If you have datasets or other archives that you want to make available to your users, and ensure they always have the latest versions and that they are downloaded correctly, `fastdownload` can help.
## Install
Using pip:
pip install fastdownload
...or using conda:
conda install -c fastai fastdownload
## What's this about?
The situation where you might want to use `fastdownload` is where you have one or more URLs pointing at some archives you want to make available, and you want to ensure that your users download those archives correctly, have the latest version, and that it's as easy as possible for them to access the information in those archives.
Your user just calls a single method, `FastDownload.get`, passing the URL required, and the URL will be downloaded and extracted to the directories you choose. The path to the extracted file is returned. If that URL has already been downloaded, then the cached archive or contents will be used automatically. However, if that size or hash of the archive is different to what it should be, then the user will be informed, and a new version will be downloaded.
In the future, you may want to update one or more of your archives. When you do so, `fastdownload` will ensure your users have the latest version, by checking their downloaded archives against your updated file size and hash information.
For instance, `fastai` uses `fastdownload` to provide access to datasets for deep learning. `fastai` users can download and extract them with a single command, using the return value to access the files. The files are automatically placed in appropriate subdirectories of a `.fastai` folder in the user's homedir. If a dataset is updated, users are informed the next time they use the dataset, and the latest version is automatically downloaded and extracted for them.
## Usage: downloading files
When your users download an archive, `fastdownload` will automatically save it to a directory, check if the size and hash matches, and extract the contents. Minimal usage for downloading and extracting is:
```python
from fastdownload import FastDownload
d = FastDownload()
path = d.get('https://...')
```
After this, `path` will contain the path where the extracted files are located. By default, archives are saved to `{base}/archive`, and extracted to `{base}/data`. `{base}` defaults to `~/.fastdownload`. If there is more than one file or folder in the root of the downloaded archive, then a new folder is created in `data` for the contents.
Instead of `get`, use `download` to download the URL without extracting it, or `extract` to extract the URL without downloading it (assuming it's already been downloaded to the `archive` directory). All of these methods accept a `force` parameter which will download/extract the archive even if it's already present.
You can change any or all of the `base`, `archive`, and `data` paths by passing them to `FastDownload`:
```python
d = FastDownload(base='~/.mypath', archive='downloaded', data='extracted')
```
You can remove the cached archive file and/or the extracted contents with `rm`:
```python
d.rm('https://...')
```
## Usage: making archives available to download
`fastdownload` will add a file `download_checks.py` to your Python module which contains file sizes and hashes for your archives. The file is located in the same directory as a module you choose, e.g.:
```python
d = FastDownload(module=fastai.some_module)
```
Then use `update` to create or update the size and hash for a URL:
```python
d.update('https://...')
```
You will now find there is a file called `download_checks.py` in the same directory where `fastai.some_module` is located, which contains a Python dict with the URL, size, and hash for this file. If you've downloaded this file before to your `archive` path then it will be used, instead of downloading a new copy. Use `get(force=True)` first to download a new copy if even you have it in your archive.
## Config file
If there is a file called `config.ini` in your `base` directory, then keys `archive` and `data` will be used as the default values for `FastDownload`. The file should be in [configparser](https://docs.python.org/3/library/configparser.html) format. Here's a sample `config.ini`:
```
[DEFAULT]
archive = downloaded
data = extracted
```
If there is no ini file present, one will be automatically created for for you using the details you pass to `FastDownload`.
You can add any additional key/value pairs to the config file that you want. When you call `FastDownload.get` pass `extract_key` to use a key other than `data` for choosing a location to extract to.
%prep
%autosetup -n fastdownload-0.0.7
%build
%py3_build
%install
%py3_install
install -d -m755 %{buildroot}/%{_pkgdocdir}
if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi
if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi
if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi
if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi
pushd %{buildroot}
if [ -d usr/lib ]; then
find usr/lib -type f -printf "/%h/%f\n" >> filelist.lst
fi
if [ -d usr/lib64 ]; then
find usr/lib64 -type f -printf "/%h/%f\n" >> filelist.lst
fi
if [ -d usr/bin ]; then
find usr/bin -type f -printf "/%h/%f\n" >> filelist.lst
fi
if [ -d usr/sbin ]; then
find usr/sbin -type f -printf "/%h/%f\n" >> filelist.lst
fi
touch doclist.lst
if [ -d usr/share/man ]; then
find usr/share/man -type f -printf "/%h/%f.gz\n" >> doclist.lst
fi
popd
mv %{buildroot}/filelist.lst .
mv %{buildroot}/doclist.lst .
%files -n python3-fastdownload -f filelist.lst
%dir %{python3_sitelib}/*
%files help -f doclist.lst
%{_docdir}/*
%changelog
* Fri Apr 21 2023 Python_Bot <Python_Bot@openeuler.org> - 0.0.7-1
- Package Spec generated
|