%global _empty_manifest_terminate_build 0 Name: python-ncluster Version: 0.1.91 Release: 1 Summary: Lightweight interface to launching jobs in the cloud License: MIT License URL: https://github.com/yaroslavvb/ncluster Source0: https://mirrors.nju.edu.cn/pypi/web/packages/c0/ee/cf0fd84a012568310556416ee150e167725ce01ba78c3744e6b1ccad7fce/ncluster-0.1.91.tar.gz BuildArch: noarch Requires: python3-botocore Requires: python3-boto3 Requires: python3-boto3-type-annotations Requires: python3-cryptography Requires: python3-paramiko Requires: python3-portalocker Requires: python3-portpicker Requires: python3-pytz Requires: python3-wandb %description # ncluster By Yaroslav Bulatov, Andrew Shaw, Ben Mann https://github.com/cybertronai/ncluster Ncluster provides Python API to do the following things: - Allocate AWS machine - Upload file to machine - Run command on machine - Download file from machine IE ``` import ncluster task = ncluster.make_task(instance_type='p2.xlarge') task.upload('myscript.py') task.run('python myscript.py > out') task.download('out') ``` Necessary AWS infrastructure is created on demand using defaults optimal for fast prototyping. IE, your machines are preconfigured for passwordless SSH, can access each other over all interfaces, and have a persistent file system mounted under /ncluster. Commands are executed in a remote tmux session so you can take over the environment at any time and continue from your terminal. ## Installation Install pip, tmux, Python 3.6 (see below), and [write down](https://docs.google.com/document/d/1Z8lCZVWXs7XORbiNmBAsBDtouV3KwrtH8-UL5M-zHus/edit) your AWS security keys, then ``` pip install -r https://raw.githubusercontent.com/yaroslavvb/ncluster/master/requirements.txt pip install -U ncluster export AWS_ACCESS_KEY_ID=AKIAIBATdf343 export AWS_SECRET_ACCESS_KEY=z7yKEP/RhO3Olk343aiP export AWS_DEFAULT_REGION=us-east-1 ``` ## Command-line tools ``` ncluster ncluster ls ncluster hosts ncluster ls ncluster ls ncluster ssh # connects to latest instance ncluster ssh # connects to latest instance containing ncluster ssh \'\' ncluster mosh ncluster kill # terminates matching instances ncluster kill \'\' ncluster stop # stops matching instances ncluster start # starts matching stopped instances ncluster nano # starts a tiny instance ncluster keys # information on enabling SSH access for your team-members ncluster ssh_ # like ssh but works on dumb terminals ncluster ls ncluster cat ncluster cmd "some command to run remotely on AWS" ncluster efs # gives EFS info such as the mount command nsync -m gpubox nsync -m gpubox -d transformer-xl nsync -d {target directory} -m {machine name substring} nsync -m gpubox # syncs . to ~ on gpubox nsync -d transformer-xl -m 4gpubox # syncs . to ~/transformer-xl on 4gpubox {substring} selects the most recently launched instances whose name contains the substring. Empty string is a valid substring. Skipping -t will sync to ~ on remote machine. Sync seems to be 1 way (from local -> remote) ``` ## Docs - Some out-of-date docs with more info [docs](https://docs.google.com/document/d/178ITRCAkboHoOEZFnz9XvOsc8lXik6Acz_DS_V1u8hY/edit?usp=sharing) ### Extra An example of installing pip/tmux/python 3.6 on MacOS 1. Download Anaconda distribution following https://conda.io/docs/user-guide/install/index.html 2. Install tmux through homebrew: https://brew.sh/, then `brew install tmux` Then ``` conda create -n new python=3.6 -y conda activate new ``` Extra Deps: ``` brew install fswatch ``` %package -n python3-ncluster Summary: Lightweight interface to launching jobs in the cloud Provides: python-ncluster BuildRequires: python3-devel BuildRequires: python3-setuptools BuildRequires: python3-pip %description -n python3-ncluster # ncluster By Yaroslav Bulatov, Andrew Shaw, Ben Mann https://github.com/cybertronai/ncluster Ncluster provides Python API to do the following things: - Allocate AWS machine - Upload file to machine - Run command on machine - Download file from machine IE ``` import ncluster task = ncluster.make_task(instance_type='p2.xlarge') task.upload('myscript.py') task.run('python myscript.py > out') task.download('out') ``` Necessary AWS infrastructure is created on demand using defaults optimal for fast prototyping. IE, your machines are preconfigured for passwordless SSH, can access each other over all interfaces, and have a persistent file system mounted under /ncluster. Commands are executed in a remote tmux session so you can take over the environment at any time and continue from your terminal. ## Installation Install pip, tmux, Python 3.6 (see below), and [write down](https://docs.google.com/document/d/1Z8lCZVWXs7XORbiNmBAsBDtouV3KwrtH8-UL5M-zHus/edit) your AWS security keys, then ``` pip install -r https://raw.githubusercontent.com/yaroslavvb/ncluster/master/requirements.txt pip install -U ncluster export AWS_ACCESS_KEY_ID=AKIAIBATdf343 export AWS_SECRET_ACCESS_KEY=z7yKEP/RhO3Olk343aiP export AWS_DEFAULT_REGION=us-east-1 ``` ## Command-line tools ``` ncluster ncluster ls ncluster hosts ncluster ls ncluster ls ncluster ssh # connects to latest instance ncluster ssh # connects to latest instance containing ncluster ssh \'\' ncluster mosh ncluster kill # terminates matching instances ncluster kill \'\' ncluster stop # stops matching instances ncluster start # starts matching stopped instances ncluster nano # starts a tiny instance ncluster keys # information on enabling SSH access for your team-members ncluster ssh_ # like ssh but works on dumb terminals ncluster ls ncluster cat ncluster cmd "some command to run remotely on AWS" ncluster efs # gives EFS info such as the mount command nsync -m gpubox nsync -m gpubox -d transformer-xl nsync -d {target directory} -m {machine name substring} nsync -m gpubox # syncs . to ~ on gpubox nsync -d transformer-xl -m 4gpubox # syncs . to ~/transformer-xl on 4gpubox {substring} selects the most recently launched instances whose name contains the substring. Empty string is a valid substring. Skipping -t will sync to ~ on remote machine. Sync seems to be 1 way (from local -> remote) ``` ## Docs - Some out-of-date docs with more info [docs](https://docs.google.com/document/d/178ITRCAkboHoOEZFnz9XvOsc8lXik6Acz_DS_V1u8hY/edit?usp=sharing) ### Extra An example of installing pip/tmux/python 3.6 on MacOS 1. Download Anaconda distribution following https://conda.io/docs/user-guide/install/index.html 2. Install tmux through homebrew: https://brew.sh/, then `brew install tmux` Then ``` conda create -n new python=3.6 -y conda activate new ``` Extra Deps: ``` brew install fswatch ``` %package help Summary: Development documents and examples for ncluster Provides: python3-ncluster-doc %description help # ncluster By Yaroslav Bulatov, Andrew Shaw, Ben Mann https://github.com/cybertronai/ncluster Ncluster provides Python API to do the following things: - Allocate AWS machine - Upload file to machine - Run command on machine - Download file from machine IE ``` import ncluster task = ncluster.make_task(instance_type='p2.xlarge') task.upload('myscript.py') task.run('python myscript.py > out') task.download('out') ``` Necessary AWS infrastructure is created on demand using defaults optimal for fast prototyping. IE, your machines are preconfigured for passwordless SSH, can access each other over all interfaces, and have a persistent file system mounted under /ncluster. Commands are executed in a remote tmux session so you can take over the environment at any time and continue from your terminal. ## Installation Install pip, tmux, Python 3.6 (see below), and [write down](https://docs.google.com/document/d/1Z8lCZVWXs7XORbiNmBAsBDtouV3KwrtH8-UL5M-zHus/edit) your AWS security keys, then ``` pip install -r https://raw.githubusercontent.com/yaroslavvb/ncluster/master/requirements.txt pip install -U ncluster export AWS_ACCESS_KEY_ID=AKIAIBATdf343 export AWS_SECRET_ACCESS_KEY=z7yKEP/RhO3Olk343aiP export AWS_DEFAULT_REGION=us-east-1 ``` ## Command-line tools ``` ncluster ncluster ls ncluster hosts ncluster ls ncluster ls ncluster ssh # connects to latest instance ncluster ssh # connects to latest instance containing ncluster ssh \'\' ncluster mosh ncluster kill # terminates matching instances ncluster kill \'\' ncluster stop # stops matching instances ncluster start # starts matching stopped instances ncluster nano # starts a tiny instance ncluster keys # information on enabling SSH access for your team-members ncluster ssh_ # like ssh but works on dumb terminals ncluster ls ncluster cat ncluster cmd "some command to run remotely on AWS" ncluster efs # gives EFS info such as the mount command nsync -m gpubox nsync -m gpubox -d transformer-xl nsync -d {target directory} -m {machine name substring} nsync -m gpubox # syncs . to ~ on gpubox nsync -d transformer-xl -m 4gpubox # syncs . to ~/transformer-xl on 4gpubox {substring} selects the most recently launched instances whose name contains the substring. Empty string is a valid substring. Skipping -t will sync to ~ on remote machine. Sync seems to be 1 way (from local -> remote) ``` ## Docs - Some out-of-date docs with more info [docs](https://docs.google.com/document/d/178ITRCAkboHoOEZFnz9XvOsc8lXik6Acz_DS_V1u8hY/edit?usp=sharing) ### Extra An example of installing pip/tmux/python 3.6 on MacOS 1. Download Anaconda distribution following https://conda.io/docs/user-guide/install/index.html 2. Install tmux through homebrew: https://brew.sh/, then `brew install tmux` Then ``` conda create -n new python=3.6 -y conda activate new ``` Extra Deps: ``` brew install fswatch ``` %prep %autosetup -n ncluster-0.1.91 %build %py3_build %install %py3_install install -d -m755 %{buildroot}/%{_pkgdocdir} if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi pushd %{buildroot} if [ -d usr/lib ]; then find usr/lib -type f -printf "/%h/%f\n" >> filelist.lst fi if [ -d usr/lib64 ]; then find usr/lib64 -type f -printf "/%h/%f\n" >> filelist.lst fi if [ -d usr/bin ]; then find usr/bin -type f -printf "/%h/%f\n" >> filelist.lst fi if [ -d usr/sbin ]; then find usr/sbin -type f -printf "/%h/%f\n" >> filelist.lst fi touch doclist.lst if [ -d usr/share/man ]; then find usr/share/man -type f -printf "/%h/%f.gz\n" >> doclist.lst fi popd mv %{buildroot}/filelist.lst . mv %{buildroot}/doclist.lst . %files -n python3-ncluster -f filelist.lst %dir %{python3_sitelib}/* %files help -f doclist.lst %{_docdir}/* %changelog * Tue May 30 2023 Python_Bot - 0.1.91-1 - Package Spec generated