%global _empty_manifest_terminate_build 0
Name: python-trainml
Version: 0.4.16
Release: 1
Summary: trainML client SDK and command line utilities
License: MIT
URL: https://github.com/trainML/trainml-cli
Source0: https://mirrors.aliyun.com/pypi/web/packages/93/cb/dc018c77bb3299c0743ba5e10d8d0786bf05dfb2aa3f8abe8e0a91fc9ac5/trainml-0.4.16.tar.gz
BuildArch: noarch
Requires: python3-aiodocker
Requires: python3-aiohttp
Requires: python3-boto3
Requires: python3-Click
Requires: python3-jose[cryptography]
Requires: python3-requests
%description
# trainML Python SDK and Command Line Tools
Provides programmatic access to [trainML platform](https://app.trainml.ai).
## Installation
Python 3.8 or above is required.
```
pip install trainml
```
## Authentication
### Prerequisites
You must have a valid [trainML account](https://app.trainml.ai). On the [account settings page](https://app.trainml.ai/account/settings) click the `Create` button in the `API Keys` section. This will automatically download a `credentials.json` file. This file can only be generated once per API key. Treat this file as a password, as anyone with access to your API key will have the ability to create and control resources in your trainML account. You can deactivate any API key by clicking the `Remove` button.
> Creating resources on the trainML platform requires a non-zero credit balance. To purchase credits or sign-up for automatic credit top-ups, visit the [billing page](https://app.trainml.ai/account/billing).
### Methods
#### Credentials File
The easiest way to authenticate is to place the credentials file downloaded into the `.trainml` folder of your home directory and ensure only you have access to it. From the directory that the `credentials.json` file was downloaded, run the following command:
```
mkdir -p ~/.trainml
mv credentials.json ~/.trainml/credentials.json
chmod 600 ~/.trainml/credentials.json
```
#### Environment Variables
You can also use environment variables `TRAINML_USER` and `TRAINML_KEY` and set them to their respective values from the `credentials.json` file.
```
export TRAINML_USER=<'user' field from credentials.json>
export TRAINML_KEY=<'key' field from credentials.json>
python create_job.py
```
Environment variables will override any credentials stored in `~/.trainml/credentials.json`
#### Runtime Variables
API credentials can also be passed directly to the TrainML object constructor at runtime.
```
import trainml
trainml = trainml.TrainML(user="user field from credentials.json",key="key field from credentials.json>")
await trainml.jobs.create(...)
```
Passing credentials to the TrainML constructor will override all other methods for setting credentials.
## Configuration
By default, all operations using the trainML SDK/CLI will use the Personal [project](https://docs.trainml.ai/reference/projects) for trainML account the API keys were generated from. To change the active project, run the configure command:
```
trainml configure
```
This command will output the currently configured active project (`UNSET` defaults to `Personal`) and allows you to specify any project you have access to as the new active project.
```
Current Active Project: Personal
Select Active Project: (My Other Project, Personal, Project Shared With Me) [Personal]:
```
Once you select a project, it will store the results of your selection in the `config.json` file in the `TRAINML_CONFIG_DIR` folder (`~/.trainml` by default). Once the active project is set, all subsequent operations will use the selected project.
This setting can also be overridden at runtime using the environment variable `TRAINML_PROJECT`:
```
TRAINML_PROJECT= python create_job.py
```
or by instantiating the trainml client with the `project` keyword argument:
```
import trainml
trainml = trainml.TrainML(project="PROJECT ID")
await trainml.jobs.create(...)
```
> You must specify the project ID (not name) when using the runtime options. The project ID can be found by running `trainml project list`.
## Usage
### Python SDK
The trainML SDK utilizes the [asyncio library](https://docs.python.org/3/library/asyncio.html) to ease the concurrent execution of long running tasks. An example of how to create a dataset from an S3 bucket and immediately run a training job on that dataset is the following:
```
from trainml.trainml import TrainML
import asyncio
trainml_client = TrainML()
# Create the dataset
dataset = asyncio.run(
trainml_client.datasets.create(
name="Example Dataset",
source_type="aws",
source_uri="s3://trainml-examples/data/cifar10",
)
)
print(dataset)
# Watch the log output, attach will return when data transfer is complete
asyncio.run(dataset.attach())
# Create the job
job = asyncio.run(
trainml_client.jobs.create(
name="Example Training Job",
type="training",
gpu_type="GTX 1060",
gpu_count=1,
disk_size=10,
workers=[
"PYTHONPATH=$PYTHONPATH:$TRAINML_MODEL_PATH python -m official.vision.image_classification.resnet_cifar_main --num_gpus=1 --data_dir=$TRAINML_DATA_PATH --model_dir=$TRAINML_OUTPUT_PATH --enable_checkpoint_and_export=True --train_epochs=10 --batch_size=1024",
],
data=dict(
datasets=[dict(id=dataset.id, type="existing")],
output_uri="s3://trainml-examples/output/resnet_cifar10",
output_type="aws",
),
model=dict(git_uri="git@github.com:trainML/test-private.git"),
)
)
print(job)
# Watch the log output, attach will return when the training job stops
asyncio.run(job.attach())
# Cleanup job and dataset
asyncio.run(job.remove())
asyncio.run(dataset.remove())
```
See more examples in the [examples folder](examples)
### Command Line Interface
The command line interface is rooted in the `trainml` command. To see the available options, run:
```
trainml --help
```
To list all jobs:
```
trainml job list
```
To list all datasets:
```
trainml dataset list
```
To connect to a job that requires the [connection capability](https://docs.trainml.ai/reference/connection-capability):
```
trainml job connect
```
To watch the realtime job logs:
```
trainml job attach
```
To create and open a notebook job:
```
trainml job create notebook "My Notebook Job"
```
To create a multi-GPU notebook job on a specific GPU type with larger scratch directory space:
```
trainml job create notebook --gpu-type "RTX 3090" --gpu-count 4 --disk-size 50 "My Notebook Job"
```
To run the model training code in the `train.py` file in your local `~/model-code` directory on the training data in your local `~/data` directory:
```
trainml job create training --model-dir ~/model-code --data-dir ~/data "My Training Job" "python train.py"
```
Stop a job by job ID:
```
trainml job stop fe52527c-1f4b-468f-b57d-86db864cc089
```
Stop a job by name:
```
trainml job stop "My Notebook Job"
```
Restart a notebook job:
```
trainml job start "My Notebook Job"
```
Remove a job by job ID:
```
trainml job remove fe52527c-1f4b-468f-b57d-86db864cc089
```
%package -n python3-trainml
Summary: trainML client SDK and command line utilities
Provides: python-trainml
BuildRequires: python3-devel
BuildRequires: python3-setuptools
BuildRequires: python3-pip
%description -n python3-trainml
# trainML Python SDK and Command Line Tools
Provides programmatic access to [trainML platform](https://app.trainml.ai).
## Installation
Python 3.8 or above is required.
```
pip install trainml
```
## Authentication
### Prerequisites
You must have a valid [trainML account](https://app.trainml.ai). On the [account settings page](https://app.trainml.ai/account/settings) click the `Create` button in the `API Keys` section. This will automatically download a `credentials.json` file. This file can only be generated once per API key. Treat this file as a password, as anyone with access to your API key will have the ability to create and control resources in your trainML account. You can deactivate any API key by clicking the `Remove` button.
> Creating resources on the trainML platform requires a non-zero credit balance. To purchase credits or sign-up for automatic credit top-ups, visit the [billing page](https://app.trainml.ai/account/billing).
### Methods
#### Credentials File
The easiest way to authenticate is to place the credentials file downloaded into the `.trainml` folder of your home directory and ensure only you have access to it. From the directory that the `credentials.json` file was downloaded, run the following command:
```
mkdir -p ~/.trainml
mv credentials.json ~/.trainml/credentials.json
chmod 600 ~/.trainml/credentials.json
```
#### Environment Variables
You can also use environment variables `TRAINML_USER` and `TRAINML_KEY` and set them to their respective values from the `credentials.json` file.
```
export TRAINML_USER=<'user' field from credentials.json>
export TRAINML_KEY=<'key' field from credentials.json>
python create_job.py
```
Environment variables will override any credentials stored in `~/.trainml/credentials.json`
#### Runtime Variables
API credentials can also be passed directly to the TrainML object constructor at runtime.
```
import trainml
trainml = trainml.TrainML(user="user field from credentials.json",key="key field from credentials.json>")
await trainml.jobs.create(...)
```
Passing credentials to the TrainML constructor will override all other methods for setting credentials.
## Configuration
By default, all operations using the trainML SDK/CLI will use the Personal [project](https://docs.trainml.ai/reference/projects) for trainML account the API keys were generated from. To change the active project, run the configure command:
```
trainml configure
```
This command will output the currently configured active project (`UNSET` defaults to `Personal`) and allows you to specify any project you have access to as the new active project.
```
Current Active Project: Personal
Select Active Project: (My Other Project, Personal, Project Shared With Me) [Personal]:
```
Once you select a project, it will store the results of your selection in the `config.json` file in the `TRAINML_CONFIG_DIR` folder (`~/.trainml` by default). Once the active project is set, all subsequent operations will use the selected project.
This setting can also be overridden at runtime using the environment variable `TRAINML_PROJECT`:
```
TRAINML_PROJECT= python create_job.py
```
or by instantiating the trainml client with the `project` keyword argument:
```
import trainml
trainml = trainml.TrainML(project="PROJECT ID")
await trainml.jobs.create(...)
```
> You must specify the project ID (not name) when using the runtime options. The project ID can be found by running `trainml project list`.
## Usage
### Python SDK
The trainML SDK utilizes the [asyncio library](https://docs.python.org/3/library/asyncio.html) to ease the concurrent execution of long running tasks. An example of how to create a dataset from an S3 bucket and immediately run a training job on that dataset is the following:
```
from trainml.trainml import TrainML
import asyncio
trainml_client = TrainML()
# Create the dataset
dataset = asyncio.run(
trainml_client.datasets.create(
name="Example Dataset",
source_type="aws",
source_uri="s3://trainml-examples/data/cifar10",
)
)
print(dataset)
# Watch the log output, attach will return when data transfer is complete
asyncio.run(dataset.attach())
# Create the job
job = asyncio.run(
trainml_client.jobs.create(
name="Example Training Job",
type="training",
gpu_type="GTX 1060",
gpu_count=1,
disk_size=10,
workers=[
"PYTHONPATH=$PYTHONPATH:$TRAINML_MODEL_PATH python -m official.vision.image_classification.resnet_cifar_main --num_gpus=1 --data_dir=$TRAINML_DATA_PATH --model_dir=$TRAINML_OUTPUT_PATH --enable_checkpoint_and_export=True --train_epochs=10 --batch_size=1024",
],
data=dict(
datasets=[dict(id=dataset.id, type="existing")],
output_uri="s3://trainml-examples/output/resnet_cifar10",
output_type="aws",
),
model=dict(git_uri="git@github.com:trainML/test-private.git"),
)
)
print(job)
# Watch the log output, attach will return when the training job stops
asyncio.run(job.attach())
# Cleanup job and dataset
asyncio.run(job.remove())
asyncio.run(dataset.remove())
```
See more examples in the [examples folder](examples)
### Command Line Interface
The command line interface is rooted in the `trainml` command. To see the available options, run:
```
trainml --help
```
To list all jobs:
```
trainml job list
```
To list all datasets:
```
trainml dataset list
```
To connect to a job that requires the [connection capability](https://docs.trainml.ai/reference/connection-capability):
```
trainml job connect
```
To watch the realtime job logs:
```
trainml job attach
```
To create and open a notebook job:
```
trainml job create notebook "My Notebook Job"
```
To create a multi-GPU notebook job on a specific GPU type with larger scratch directory space:
```
trainml job create notebook --gpu-type "RTX 3090" --gpu-count 4 --disk-size 50 "My Notebook Job"
```
To run the model training code in the `train.py` file in your local `~/model-code` directory on the training data in your local `~/data` directory:
```
trainml job create training --model-dir ~/model-code --data-dir ~/data "My Training Job" "python train.py"
```
Stop a job by job ID:
```
trainml job stop fe52527c-1f4b-468f-b57d-86db864cc089
```
Stop a job by name:
```
trainml job stop "My Notebook Job"
```
Restart a notebook job:
```
trainml job start "My Notebook Job"
```
Remove a job by job ID:
```
trainml job remove fe52527c-1f4b-468f-b57d-86db864cc089
```
%package help
Summary: Development documents and examples for trainml
Provides: python3-trainml-doc
%description help
# trainML Python SDK and Command Line Tools
Provides programmatic access to [trainML platform](https://app.trainml.ai).
## Installation
Python 3.8 or above is required.
```
pip install trainml
```
## Authentication
### Prerequisites
You must have a valid [trainML account](https://app.trainml.ai). On the [account settings page](https://app.trainml.ai/account/settings) click the `Create` button in the `API Keys` section. This will automatically download a `credentials.json` file. This file can only be generated once per API key. Treat this file as a password, as anyone with access to your API key will have the ability to create and control resources in your trainML account. You can deactivate any API key by clicking the `Remove` button.
> Creating resources on the trainML platform requires a non-zero credit balance. To purchase credits or sign-up for automatic credit top-ups, visit the [billing page](https://app.trainml.ai/account/billing).
### Methods
#### Credentials File
The easiest way to authenticate is to place the credentials file downloaded into the `.trainml` folder of your home directory and ensure only you have access to it. From the directory that the `credentials.json` file was downloaded, run the following command:
```
mkdir -p ~/.trainml
mv credentials.json ~/.trainml/credentials.json
chmod 600 ~/.trainml/credentials.json
```
#### Environment Variables
You can also use environment variables `TRAINML_USER` and `TRAINML_KEY` and set them to their respective values from the `credentials.json` file.
```
export TRAINML_USER=<'user' field from credentials.json>
export TRAINML_KEY=<'key' field from credentials.json>
python create_job.py
```
Environment variables will override any credentials stored in `~/.trainml/credentials.json`
#### Runtime Variables
API credentials can also be passed directly to the TrainML object constructor at runtime.
```
import trainml
trainml = trainml.TrainML(user="user field from credentials.json",key="key field from credentials.json>")
await trainml.jobs.create(...)
```
Passing credentials to the TrainML constructor will override all other methods for setting credentials.
## Configuration
By default, all operations using the trainML SDK/CLI will use the Personal [project](https://docs.trainml.ai/reference/projects) for trainML account the API keys were generated from. To change the active project, run the configure command:
```
trainml configure
```
This command will output the currently configured active project (`UNSET` defaults to `Personal`) and allows you to specify any project you have access to as the new active project.
```
Current Active Project: Personal
Select Active Project: (My Other Project, Personal, Project Shared With Me) [Personal]:
```
Once you select a project, it will store the results of your selection in the `config.json` file in the `TRAINML_CONFIG_DIR` folder (`~/.trainml` by default). Once the active project is set, all subsequent operations will use the selected project.
This setting can also be overridden at runtime using the environment variable `TRAINML_PROJECT`:
```
TRAINML_PROJECT= python create_job.py
```
or by instantiating the trainml client with the `project` keyword argument:
```
import trainml
trainml = trainml.TrainML(project="PROJECT ID")
await trainml.jobs.create(...)
```
> You must specify the project ID (not name) when using the runtime options. The project ID can be found by running `trainml project list`.
## Usage
### Python SDK
The trainML SDK utilizes the [asyncio library](https://docs.python.org/3/library/asyncio.html) to ease the concurrent execution of long running tasks. An example of how to create a dataset from an S3 bucket and immediately run a training job on that dataset is the following:
```
from trainml.trainml import TrainML
import asyncio
trainml_client = TrainML()
# Create the dataset
dataset = asyncio.run(
trainml_client.datasets.create(
name="Example Dataset",
source_type="aws",
source_uri="s3://trainml-examples/data/cifar10",
)
)
print(dataset)
# Watch the log output, attach will return when data transfer is complete
asyncio.run(dataset.attach())
# Create the job
job = asyncio.run(
trainml_client.jobs.create(
name="Example Training Job",
type="training",
gpu_type="GTX 1060",
gpu_count=1,
disk_size=10,
workers=[
"PYTHONPATH=$PYTHONPATH:$TRAINML_MODEL_PATH python -m official.vision.image_classification.resnet_cifar_main --num_gpus=1 --data_dir=$TRAINML_DATA_PATH --model_dir=$TRAINML_OUTPUT_PATH --enable_checkpoint_and_export=True --train_epochs=10 --batch_size=1024",
],
data=dict(
datasets=[dict(id=dataset.id, type="existing")],
output_uri="s3://trainml-examples/output/resnet_cifar10",
output_type="aws",
),
model=dict(git_uri="git@github.com:trainML/test-private.git"),
)
)
print(job)
# Watch the log output, attach will return when the training job stops
asyncio.run(job.attach())
# Cleanup job and dataset
asyncio.run(job.remove())
asyncio.run(dataset.remove())
```
See more examples in the [examples folder](examples)
### Command Line Interface
The command line interface is rooted in the `trainml` command. To see the available options, run:
```
trainml --help
```
To list all jobs:
```
trainml job list
```
To list all datasets:
```
trainml dataset list
```
To connect to a job that requires the [connection capability](https://docs.trainml.ai/reference/connection-capability):
```
trainml job connect
```
To watch the realtime job logs:
```
trainml job attach
```
To create and open a notebook job:
```
trainml job create notebook "My Notebook Job"
```
To create a multi-GPU notebook job on a specific GPU type with larger scratch directory space:
```
trainml job create notebook --gpu-type "RTX 3090" --gpu-count 4 --disk-size 50 "My Notebook Job"
```
To run the model training code in the `train.py` file in your local `~/model-code` directory on the training data in your local `~/data` directory:
```
trainml job create training --model-dir ~/model-code --data-dir ~/data "My Training Job" "python train.py"
```
Stop a job by job ID:
```
trainml job stop fe52527c-1f4b-468f-b57d-86db864cc089
```
Stop a job by name:
```
trainml job stop "My Notebook Job"
```
Restart a notebook job:
```
trainml job start "My Notebook Job"
```
Remove a job by job ID:
```
trainml job remove fe52527c-1f4b-468f-b57d-86db864cc089
```
%prep
%autosetup -n trainml-0.4.16
%build
%py3_build
%install
%py3_install
install -d -m755 %{buildroot}/%{_pkgdocdir}
if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi
if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi
if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi
if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi
pushd %{buildroot}
if [ -d usr/lib ]; then
find usr/lib -type f -printf "\"/%h/%f\"\n" >> filelist.lst
fi
if [ -d usr/lib64 ]; then
find usr/lib64 -type f -printf "\"/%h/%f\"\n" >> filelist.lst
fi
if [ -d usr/bin ]; then
find usr/bin -type f -printf "\"/%h/%f\"\n" >> filelist.lst
fi
if [ -d usr/sbin ]; then
find usr/sbin -type f -printf "\"/%h/%f\"\n" >> filelist.lst
fi
touch doclist.lst
if [ -d usr/share/man ]; then
find usr/share/man -type f -printf "\"/%h/%f.gz\"\n" >> doclist.lst
fi
popd
mv %{buildroot}/filelist.lst .
mv %{buildroot}/doclist.lst .
%files -n python3-trainml -f filelist.lst
%dir %{python3_sitelib}/*
%files help -f doclist.lst
%{_docdir}/*
%changelog
* Tue Jun 20 2023 Python_Bot - 0.4.16-1
- Package Spec generated