diff options
| -rw-r--r-- | .gitignore | 1 | ||||
| -rw-r--r-- | python-ncluster.spec | 369 | ||||
| -rw-r--r-- | sources | 1 |
3 files changed, 371 insertions, 0 deletions
@@ -0,0 +1 @@ +/ncluster-0.1.91.tar.gz diff --git a/python-ncluster.spec b/python-ncluster.spec new file mode 100644 index 0000000..281657e --- /dev/null +++ b/python-ncluster.spec @@ -0,0 +1,369 @@ +%global _empty_manifest_terminate_build 0 +Name: python-ncluster +Version: 0.1.91 +Release: 1 +Summary: Lightweight interface to launching jobs in the cloud +License: MIT License +URL: https://github.com/yaroslavvb/ncluster +Source0: https://mirrors.nju.edu.cn/pypi/web/packages/c0/ee/cf0fd84a012568310556416ee150e167725ce01ba78c3744e6b1ccad7fce/ncluster-0.1.91.tar.gz +BuildArch: noarch + +Requires: python3-botocore +Requires: python3-boto3 +Requires: python3-boto3-type-annotations +Requires: python3-cryptography +Requires: python3-paramiko +Requires: python3-portalocker +Requires: python3-portpicker +Requires: python3-pytz +Requires: python3-wandb + +%description +# ncluster +By Yaroslav Bulatov, Andrew Shaw, Ben Mann +https://github.com/cybertronai/ncluster + +Ncluster provides Python API to do the following things: +- Allocate AWS machine +- Upload file to machine +- Run command on machine +- Download file from machine + +IE + +``` +import ncluster +task = ncluster.make_task(instance_type='p2.xlarge') +task.upload('myscript.py') +task.run('python myscript.py > out') +task.download('out') +``` + +Necessary AWS infrastructure is created on demand using defaults optimal for fast prototyping. IE, your machines are preconfigured for passwordless SSH, can access each other over all interfaces, and have a persistent file system mounted under /ncluster. Commands are executed in a remote tmux session so you can take over the environment at any time and continue from your terminal. + + +## Installation +Install pip, tmux, Python 3.6 (see below), and [write down](https://docs.google.com/document/d/1Z8lCZVWXs7XORbiNmBAsBDtouV3KwrtH8-UL5M-zHus/edit) your AWS security keys, then + +``` +pip install -r https://raw.githubusercontent.com/yaroslavvb/ncluster/master/requirements.txt +pip install -U ncluster +export AWS_ACCESS_KEY_ID=AKIAIBATdf343 +export AWS_SECRET_ACCESS_KEY=z7yKEP/RhO3Olk343aiP +export AWS_DEFAULT_REGION=us-east-1 +``` + + + +## Command-line tools + +``` +ncluster +ncluster ls +ncluster hosts +ncluster ls +ncluster ls <substring> +ncluster ssh # connects to latest instance +ncluster ssh <substring> # connects to latest instance containing <substring> +ncluster ssh \'<exact match>\' +ncluster mosh <substring> +ncluster kill <substring> # terminates matching instances +ncluster kill \'<exact match>\' +ncluster stop <substring> # stops matching instances +ncluster start <substring> # starts matching stopped instances +ncluster nano # starts a tiny instance +ncluster keys # information on enabling SSH access for your team-members + +ncluster ssh_ # like ssh but works on dumb terminals +ncluster ls +ncluster cat <fn> +ncluster cmd "some command to run remotely on AWS" + +ncluster efs # gives EFS info such as the mount command + +nsync -m gpubox +nsync -m gpubox -d transformer-xl + +nsync -d {target directory} -m {machine name substring} + +nsync -m gpubox # syncs . to ~ on gpubox +nsync -d transformer-xl -m 4gpubox # syncs . to ~/transformer-xl on 4gpubox + + +{substring} selects the most recently launched instances whose name contains the substring. Empty string is a valid substring. Skipping -t will sync to ~ on remote machine. Sync seems to be 1 way (from local -> remote) +``` + +## Docs +- Some out-of-date docs with more info [docs](https://docs.google.com/document/d/178ITRCAkboHoOEZFnz9XvOsc8lXik6Acz_DS_V1u8hY/edit?usp=sharing) + +### Extra +An example of installing pip/tmux/python 3.6 on MacOS + +1. Download Anaconda distribution following https://conda.io/docs/user-guide/install/index.html +2. Install tmux through homebrew: https://brew.sh/, then `brew install tmux` + +Then + +``` +conda create -n new python=3.6 -y +conda activate new +``` + +Extra Deps: +``` +brew install fswatch +``` + + + + +%package -n python3-ncluster +Summary: Lightweight interface to launching jobs in the cloud +Provides: python-ncluster +BuildRequires: python3-devel +BuildRequires: python3-setuptools +BuildRequires: python3-pip +%description -n python3-ncluster +# ncluster +By Yaroslav Bulatov, Andrew Shaw, Ben Mann +https://github.com/cybertronai/ncluster + +Ncluster provides Python API to do the following things: +- Allocate AWS machine +- Upload file to machine +- Run command on machine +- Download file from machine + +IE + +``` +import ncluster +task = ncluster.make_task(instance_type='p2.xlarge') +task.upload('myscript.py') +task.run('python myscript.py > out') +task.download('out') +``` + +Necessary AWS infrastructure is created on demand using defaults optimal for fast prototyping. IE, your machines are preconfigured for passwordless SSH, can access each other over all interfaces, and have a persistent file system mounted under /ncluster. Commands are executed in a remote tmux session so you can take over the environment at any time and continue from your terminal. + + +## Installation +Install pip, tmux, Python 3.6 (see below), and [write down](https://docs.google.com/document/d/1Z8lCZVWXs7XORbiNmBAsBDtouV3KwrtH8-UL5M-zHus/edit) your AWS security keys, then + +``` +pip install -r https://raw.githubusercontent.com/yaroslavvb/ncluster/master/requirements.txt +pip install -U ncluster +export AWS_ACCESS_KEY_ID=AKIAIBATdf343 +export AWS_SECRET_ACCESS_KEY=z7yKEP/RhO3Olk343aiP +export AWS_DEFAULT_REGION=us-east-1 +``` + + + +## Command-line tools + +``` +ncluster +ncluster ls +ncluster hosts +ncluster ls +ncluster ls <substring> +ncluster ssh # connects to latest instance +ncluster ssh <substring> # connects to latest instance containing <substring> +ncluster ssh \'<exact match>\' +ncluster mosh <substring> +ncluster kill <substring> # terminates matching instances +ncluster kill \'<exact match>\' +ncluster stop <substring> # stops matching instances +ncluster start <substring> # starts matching stopped instances +ncluster nano # starts a tiny instance +ncluster keys # information on enabling SSH access for your team-members + +ncluster ssh_ # like ssh but works on dumb terminals +ncluster ls +ncluster cat <fn> +ncluster cmd "some command to run remotely on AWS" + +ncluster efs # gives EFS info such as the mount command + +nsync -m gpubox +nsync -m gpubox -d transformer-xl + +nsync -d {target directory} -m {machine name substring} + +nsync -m gpubox # syncs . to ~ on gpubox +nsync -d transformer-xl -m 4gpubox # syncs . to ~/transformer-xl on 4gpubox + + +{substring} selects the most recently launched instances whose name contains the substring. Empty string is a valid substring. Skipping -t will sync to ~ on remote machine. Sync seems to be 1 way (from local -> remote) +``` + +## Docs +- Some out-of-date docs with more info [docs](https://docs.google.com/document/d/178ITRCAkboHoOEZFnz9XvOsc8lXik6Acz_DS_V1u8hY/edit?usp=sharing) + +### Extra +An example of installing pip/tmux/python 3.6 on MacOS + +1. Download Anaconda distribution following https://conda.io/docs/user-guide/install/index.html +2. Install tmux through homebrew: https://brew.sh/, then `brew install tmux` + +Then + +``` +conda create -n new python=3.6 -y +conda activate new +``` + +Extra Deps: +``` +brew install fswatch +``` + + + + +%package help +Summary: Development documents and examples for ncluster +Provides: python3-ncluster-doc +%description help +# ncluster +By Yaroslav Bulatov, Andrew Shaw, Ben Mann +https://github.com/cybertronai/ncluster + +Ncluster provides Python API to do the following things: +- Allocate AWS machine +- Upload file to machine +- Run command on machine +- Download file from machine + +IE + +``` +import ncluster +task = ncluster.make_task(instance_type='p2.xlarge') +task.upload('myscript.py') +task.run('python myscript.py > out') +task.download('out') +``` + +Necessary AWS infrastructure is created on demand using defaults optimal for fast prototyping. IE, your machines are preconfigured for passwordless SSH, can access each other over all interfaces, and have a persistent file system mounted under /ncluster. Commands are executed in a remote tmux session so you can take over the environment at any time and continue from your terminal. + + +## Installation +Install pip, tmux, Python 3.6 (see below), and [write down](https://docs.google.com/document/d/1Z8lCZVWXs7XORbiNmBAsBDtouV3KwrtH8-UL5M-zHus/edit) your AWS security keys, then + +``` +pip install -r https://raw.githubusercontent.com/yaroslavvb/ncluster/master/requirements.txt +pip install -U ncluster +export AWS_ACCESS_KEY_ID=AKIAIBATdf343 +export AWS_SECRET_ACCESS_KEY=z7yKEP/RhO3Olk343aiP +export AWS_DEFAULT_REGION=us-east-1 +``` + + + +## Command-line tools + +``` +ncluster +ncluster ls +ncluster hosts +ncluster ls +ncluster ls <substring> +ncluster ssh # connects to latest instance +ncluster ssh <substring> # connects to latest instance containing <substring> +ncluster ssh \'<exact match>\' +ncluster mosh <substring> +ncluster kill <substring> # terminates matching instances +ncluster kill \'<exact match>\' +ncluster stop <substring> # stops matching instances +ncluster start <substring> # starts matching stopped instances +ncluster nano # starts a tiny instance +ncluster keys # information on enabling SSH access for your team-members + +ncluster ssh_ # like ssh but works on dumb terminals +ncluster ls +ncluster cat <fn> +ncluster cmd "some command to run remotely on AWS" + +ncluster efs # gives EFS info such as the mount command + +nsync -m gpubox +nsync -m gpubox -d transformer-xl + +nsync -d {target directory} -m {machine name substring} + +nsync -m gpubox # syncs . to ~ on gpubox +nsync -d transformer-xl -m 4gpubox # syncs . to ~/transformer-xl on 4gpubox + + +{substring} selects the most recently launched instances whose name contains the substring. Empty string is a valid substring. Skipping -t will sync to ~ on remote machine. Sync seems to be 1 way (from local -> remote) +``` + +## Docs +- Some out-of-date docs with more info [docs](https://docs.google.com/document/d/178ITRCAkboHoOEZFnz9XvOsc8lXik6Acz_DS_V1u8hY/edit?usp=sharing) + +### Extra +An example of installing pip/tmux/python 3.6 on MacOS + +1. Download Anaconda distribution following https://conda.io/docs/user-guide/install/index.html +2. Install tmux through homebrew: https://brew.sh/, then `brew install tmux` + +Then + +``` +conda create -n new python=3.6 -y +conda activate new +``` + +Extra Deps: +``` +brew install fswatch +``` + + + + +%prep +%autosetup -n ncluster-0.1.91 + +%build +%py3_build + +%install +%py3_install +install -d -m755 %{buildroot}/%{_pkgdocdir} +if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi +if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi +if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi +if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi +pushd %{buildroot} +if [ -d usr/lib ]; then + find usr/lib -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/lib64 ]; then + find usr/lib64 -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/bin ]; then + find usr/bin -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/sbin ]; then + find usr/sbin -type f -printf "/%h/%f\n" >> filelist.lst +fi +touch doclist.lst +if [ -d usr/share/man ]; then + find usr/share/man -type f -printf "/%h/%f.gz\n" >> doclist.lst +fi +popd +mv %{buildroot}/filelist.lst . +mv %{buildroot}/doclist.lst . + +%files -n python3-ncluster -f filelist.lst +%dir %{python3_sitelib}/* + +%files help -f doclist.lst +%{_docdir}/* + +%changelog +* Mon May 15 2023 Python_Bot <Python_Bot@openeuler.org> - 0.1.91-1 +- Package Spec generated @@ -0,0 +1 @@ +03b74336d088b4df1f6d7fcf192462b7 ncluster-0.1.91.tar.gz |