summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorCoprDistGit <infra@openeuler.org>2023-05-18 07:17:40 +0000
committerCoprDistGit <infra@openeuler.org>2023-05-18 07:17:40 +0000
commit31efaedcc15e500c3c20d410b3c40246e4d45ba9 (patch)
treecb46028e4be8e0b2d0d5f541941722a70415199c
parent92aa76640ec4c913d3036f60283ee2614e10b6ba (diff)
automatic import of python-ai-traineree
-rw-r--r--.gitignore1
-rw-r--r--python-ai-traineree.spec548
-rw-r--r--sources1
3 files changed, 550 insertions, 0 deletions
diff --git a/.gitignore b/.gitignore
index e69de29..b6de7b9 100644
--- a/.gitignore
+++ b/.gitignore
@@ -0,0 +1 @@
+/ai-traineree-0.5.2.tar.gz
diff --git a/python-ai-traineree.spec b/python-ai-traineree.spec
new file mode 100644
index 0000000..179cc57
--- /dev/null
+++ b/python-ai-traineree.spec
@@ -0,0 +1,548 @@
+%global _empty_manifest_terminate_build 0
+Name: python-ai-traineree
+Version: 0.5.2
+Release: 1
+Summary: Yet another zoo of (Deep) Reinforcment Learning methods in Python using PyTorch
+License: Apache Software License
+URL: https://github.com/laszukdawid/ai-traineree
+Source0: https://mirrors.nju.edu.cn/pypi/web/packages/22/7c/19594210c0374278e9b1b5e5f0628cffe80505bdde448f3d19f957daac59/ai-traineree-0.5.2.tar.gz
+BuildArch: noarch
+
+Requires: python3-numpy
+Requires: python3-torch
+Requires: python3-jsons
+Requires: python3-gym
+Requires: python3-black
+Requires: python3-isort
+Requires: python3-pycodestyle
+Requires: python3-sphinx
+Requires: python3-sphinx-rtd-theme
+Requires: python3-matplotlib
+Requires: python3-tensorboard
+Requires: python3-gym
+Requires: python3-gym[all]
+Requires: python3-mlagents
+Requires: python3-neptune-client
+Requires: python3-psutil
+Requires: python3-pettingzoo[other]
+Requires: python3-matplotlib
+Requires: python3-tensorboard
+Requires: python3-mock
+Requires: python3-pytest
+Requires: python3-pytest-cov
+Requires: python3-pytest-mock
+Requires: python3-flake8
+Requires: python3-gym[box2d]
+Requires: python3-pettingzoo[other,sisl]
+
+%description
+# ai-traineree
+[![Build Status](https://app.travis-ci.com/laszukdawid/ai-traineree.svg?branch=master)](https://app.travis-ci.com/laszukdawid/ai-traineree)
+[![DocStatus](https://readthedocs.org/projects/ai-traineree/badge/?version=latest)](https://ai-traineree.readthedocs.io/)
+[![codecov](https://codecov.io/gh/laszukdawid/ai-traineree/branch/master/graph/badge.svg?token=S62DK7HPYA)](https://codecov.io/gh/laszukdawid/ai-traineree)
+[![Codacy Badge](https://app.codacy.com/project/badge/Grade/00563b8422454e10bb4ffab64068aa62)](https://www.codacy.com/gh/laszukdawid/ai-traineree/dashboard?utm_source=github.com&amp;utm_medium=referral&amp;utm_content=laszukdawid/ai-traineree&amp;utm_campaign=Badge_Grade)
+[![Discord chat](https://img.shields.io/discord/909530201837432873?logo=discord)](https://discord.gg/n4bN7ZHn)
+[![DOI](https://zenodo.org/badge/267484000.svg)](https://zenodo.org/badge/latestdoi/267484000)
+
+The intention is to have a zoo of Deep Reinforcment Learning methods and showcasing their application on some environments.
+
+Read more in the doc: [ReadTheDocs AI-Traineree](https://ai-traineree.readthedocs.io/).
+
+![CartPole-v1](./static/CartPole-v1.gif)
+![Snek](./static/hungrysnek.gif)
+
+## Why another?
+
+The main reason is the implemention philosophy.
+We strongly believe that agents should be emerged in the environment and not the other way round.
+Majority of the popular implementations pass environment instance to the agent as if the agent was the focus point.
+This might ease implementation of some algorithms but it isn't representative of the world;
+agents want to control the environment but that doesn't mean they can/should.
+
+That, and using PyTorch instead of Tensorflow or JAX.
+
+## Quick start
+
+To get started with training your RL agent you need three things: an agent, an environment and a runner. Let's say you want to train a DQN agent on OpenAI CartPole-v1:
+```python
+from ai_traineree.agents.dqn import DQNAgent
+from ai_traineree.runners.env_runner import EnvRunner
+from ai_traineree.tasks import GymTask
+
+task = GymTask('CartPole-v1')
+agent = DQNAgent(task.obs_space, task.action_space)
+env_runner = EnvRunner(task, agent)
+
+scores = env_runner.run()
+```
+
+or execute one of provided examples
+> \$ python -m examples.cart_dqn
+
+That's it.
+
+## Installation
+
+### PyPi (recommended)
+
+The quickest way to install package is through `pip`.
+
+> \$ pip install ai-traineree
+
+### Git repository clone
+
+As usual with Python, the expectation is to have own virtual environment and then pip install requirements. For example,
+```bash
+> python -m venv .venv
+> git clone git@github.com:laszukdawid/ai-traineree.git
+> source .venv/bin/activate
+> python setup.py install
+```
+
+## Current state
+
+### Playing gym
+One way to improve learning speed is to simply show them how to play or, more researchy/creepy, provide a proper seed.
+This isn't a general rule, since some algorithms train better without any human interaction, but since you're on GitHub... that's unlikely your case.
+Currently there's a script [`interact.py`](scripts/interact.py) which uses OpenAI Gym's play API to record moves and AI Traineree to store them
+in a buffer. Such buffers can be loaded by agents on initiation.
+
+This is just a beginning and there will be more work on these interactions.
+
+*Requirement*: Install `pygame`.
+
+### Agents
+
+| Short | Progress | Link | Full name | Doc |
+| ------- | --------------------------------------------- | -------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------- | ------------------------------------------------------------------------ |
+| DQN | [Implemented](ai_traineree/agents/dqn.py) | [DeepMind](https://deepmind.com/research/publications/human-level-control-through-deep-reinforcement-learning) | Deep Q-learning Network | [Doc](https://ai-traineree.readthedocs.io/en/latest/agents.html#dqn) |
+| DDPG | [Implemented](ai_traineree/agents/ddpg.py) | [arXiv](https://arxiv.org/abs/1509.02971) | Deep Deterministic Policy Gradient | [Doc](https://ai-traineree.readthedocs.io/en/latest/agents.html#ddpg) |
+| D4PG | [Implemented](ai_traineree/agents/d4pg.py) | [arXiv](https://arxiv.org/abs/1804.08617) | Distributed Distributional Deterministic Policy Gradients | [Doc](https://ai-traineree.readthedocs.io/en/latest/agents.html#d4pg) |
+| TD3 | [Implemented](ai_traineree/agents/td3.py) | [arXiv](https://arxiv.org/abs/1802.09477) | Twine Delayed Deep Deterministic policy gradient | [Doc](https://ai-traineree.readthedocs.io/en/latest/agents.html#td3) |
+| PPO | [Implemented](ai_traineree/agents/ppo.py) | [arXiv](https://arxiv.org/abs/1707.06347) | Proximal Policy Optimization | [Doc](https://ai-traineree.readthedocs.io/en/latest/agents.html#ppo) |
+| SAC | [Implemented](ai_traineree/agents/sac.py) | [arXiv](https://arxiv.org/abs/1801.01290) | Soft Actor Critic | [Doc](https://ai-traineree.readthedocs.io/en/latest/agents.html#sac) |
+| TRPO | | [arXiv](https://arxiv.org/abs/1502.05477) | Trust Region Policy Optimization |
+| RAINBOW | [Implemented](ai_traineree/agents/rainbow.py) | [arXiv](https://arxiv.org/abs/1710.02298) | DQN with a few improvements | [Doc](https://ai-traineree.readthedocs.io/en/latest/agents.html#rainbow) |
+
+### Multi agents
+
+We provide both Multi Agents agents entities and means to execute them against supported (below) environements.
+However, that doesn't mean one can be used without the other.
+
+| Short | Progress | Link | Full name | Doc |
+| ------ | ------------------------------------------------- | ----------------------------------------- | ---------------------- | ---------------------------------------------------------------------------- |
+| IQL | [Implemented](ai_traineree/multi_agent/iql.py) | | Independent Q-Learners | [Doc](https://ai-traineree.readthedocs.io/en/latest/multi_agent.html#iql) |
+| MADDPG | [Implemented](ai_traineree/multi_agent/maddpg.py) | [arXiv](https://arxiv.org/abs/1706.02275) | Multi agent DDPG | [Doc](https://ai-traineree.readthedocs.io/en/latest/multi_agent.html#maddpg) |
+
+### Loggers
+
+Supports using Tensorboard (via PyTorch's [SummaryWriter](https://pytorch.org/docs/stable/tensorboard.html)) and [Neptune](https://neptune.ai) to display metrics. Wrappers are provided as `TensorboardLogger` and `NeptuneLogger`.
+
+*Note*: In order to use Neptune one needs to install [neptune-client](https://pypi.org/project/neptune-client/) (`pip install neptune-client`).
+
+### Environments
+
+| Name | Progress | Link |
+| -------------------- | ------------------ | -------------------------------------------------------------------------------------------- |
+| OpenAI Gym - Classic | Done |
+| OpenAI Gym - Atari | Done |
+| OpenAI Gym - MuJoCo | Not interested. |
+| PettingZoo | Initial support | [Page](https://www.pettingzoo.ml/) / [GitHub](https://github.com/PettingZoo-Team/PettingZoo) |
+| Unity ML | Somehow supported. | [Page](https://unity3d.com/machine-learning) |
+| MAME Linux emulator | Interested. | [Official page](https://www.mamedev.org/) |
+
+### Development
+
+We are open to any contributions. If you want to contribute but don't know what then feel free to reach out (see Contact below).
+The best way to start is through updating documentation and adding tutorials.
+In addition there are many other things that we know of which need improvement but also plenty that we don't know of.
+
+Setting up development environment requires installing `dev` and `test` extra packages.
+The `dev` extras are for mainly for linting and formatting, and the `test` is for running tests.
+We recommend using `pip` so to install everything requires for development run
+
+```bash
+$ pip install -e .[dev,test]
+```
+
+Once installed, please configure your IDE to use `black` as formatter, `pycodestyle` as linter,
+and `isort` for sorting imports. All these are included in the `dev` extra packages.
+
+### Contact
+
+Should we focus on something specificallly? Let us know by opening a feature request [GitHub issue](https://github.com/laszukdawid/ai-traineree/issues) or contacting through [ai-traineree@dawid.lasz.uk](mailto:ai-traineree@dawid.lasz.uk).
+
+## Citing project
+
+```latex
+@misc{ai-traineree,
+ author = {Laszuk, Dawid},
+ title = {AI Traineree: Reinforcement learning toolset},
+ year = {2020},
+ publisher = {GitHub},
+ journal = {GitHub repository},
+ howpublished = {\url{https://github.com/laszukdawid/ai-traineree}},
+}
+```
+
+
+
+
+%package -n python3-ai-traineree
+Summary: Yet another zoo of (Deep) Reinforcment Learning methods in Python using PyTorch
+Provides: python-ai-traineree
+BuildRequires: python3-devel
+BuildRequires: python3-setuptools
+BuildRequires: python3-pip
+%description -n python3-ai-traineree
+# ai-traineree
+[![Build Status](https://app.travis-ci.com/laszukdawid/ai-traineree.svg?branch=master)](https://app.travis-ci.com/laszukdawid/ai-traineree)
+[![DocStatus](https://readthedocs.org/projects/ai-traineree/badge/?version=latest)](https://ai-traineree.readthedocs.io/)
+[![codecov](https://codecov.io/gh/laszukdawid/ai-traineree/branch/master/graph/badge.svg?token=S62DK7HPYA)](https://codecov.io/gh/laszukdawid/ai-traineree)
+[![Codacy Badge](https://app.codacy.com/project/badge/Grade/00563b8422454e10bb4ffab64068aa62)](https://www.codacy.com/gh/laszukdawid/ai-traineree/dashboard?utm_source=github.com&amp;utm_medium=referral&amp;utm_content=laszukdawid/ai-traineree&amp;utm_campaign=Badge_Grade)
+[![Discord chat](https://img.shields.io/discord/909530201837432873?logo=discord)](https://discord.gg/n4bN7ZHn)
+[![DOI](https://zenodo.org/badge/267484000.svg)](https://zenodo.org/badge/latestdoi/267484000)
+
+The intention is to have a zoo of Deep Reinforcment Learning methods and showcasing their application on some environments.
+
+Read more in the doc: [ReadTheDocs AI-Traineree](https://ai-traineree.readthedocs.io/).
+
+![CartPole-v1](./static/CartPole-v1.gif)
+![Snek](./static/hungrysnek.gif)
+
+## Why another?
+
+The main reason is the implemention philosophy.
+We strongly believe that agents should be emerged in the environment and not the other way round.
+Majority of the popular implementations pass environment instance to the agent as if the agent was the focus point.
+This might ease implementation of some algorithms but it isn't representative of the world;
+agents want to control the environment but that doesn't mean they can/should.
+
+That, and using PyTorch instead of Tensorflow or JAX.
+
+## Quick start
+
+To get started with training your RL agent you need three things: an agent, an environment and a runner. Let's say you want to train a DQN agent on OpenAI CartPole-v1:
+```python
+from ai_traineree.agents.dqn import DQNAgent
+from ai_traineree.runners.env_runner import EnvRunner
+from ai_traineree.tasks import GymTask
+
+task = GymTask('CartPole-v1')
+agent = DQNAgent(task.obs_space, task.action_space)
+env_runner = EnvRunner(task, agent)
+
+scores = env_runner.run()
+```
+
+or execute one of provided examples
+> \$ python -m examples.cart_dqn
+
+That's it.
+
+## Installation
+
+### PyPi (recommended)
+
+The quickest way to install package is through `pip`.
+
+> \$ pip install ai-traineree
+
+### Git repository clone
+
+As usual with Python, the expectation is to have own virtual environment and then pip install requirements. For example,
+```bash
+> python -m venv .venv
+> git clone git@github.com:laszukdawid/ai-traineree.git
+> source .venv/bin/activate
+> python setup.py install
+```
+
+## Current state
+
+### Playing gym
+One way to improve learning speed is to simply show them how to play or, more researchy/creepy, provide a proper seed.
+This isn't a general rule, since some algorithms train better without any human interaction, but since you're on GitHub... that's unlikely your case.
+Currently there's a script [`interact.py`](scripts/interact.py) which uses OpenAI Gym's play API to record moves and AI Traineree to store them
+in a buffer. Such buffers can be loaded by agents on initiation.
+
+This is just a beginning and there will be more work on these interactions.
+
+*Requirement*: Install `pygame`.
+
+### Agents
+
+| Short | Progress | Link | Full name | Doc |
+| ------- | --------------------------------------------- | -------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------- | ------------------------------------------------------------------------ |
+| DQN | [Implemented](ai_traineree/agents/dqn.py) | [DeepMind](https://deepmind.com/research/publications/human-level-control-through-deep-reinforcement-learning) | Deep Q-learning Network | [Doc](https://ai-traineree.readthedocs.io/en/latest/agents.html#dqn) |
+| DDPG | [Implemented](ai_traineree/agents/ddpg.py) | [arXiv](https://arxiv.org/abs/1509.02971) | Deep Deterministic Policy Gradient | [Doc](https://ai-traineree.readthedocs.io/en/latest/agents.html#ddpg) |
+| D4PG | [Implemented](ai_traineree/agents/d4pg.py) | [arXiv](https://arxiv.org/abs/1804.08617) | Distributed Distributional Deterministic Policy Gradients | [Doc](https://ai-traineree.readthedocs.io/en/latest/agents.html#d4pg) |
+| TD3 | [Implemented](ai_traineree/agents/td3.py) | [arXiv](https://arxiv.org/abs/1802.09477) | Twine Delayed Deep Deterministic policy gradient | [Doc](https://ai-traineree.readthedocs.io/en/latest/agents.html#td3) |
+| PPO | [Implemented](ai_traineree/agents/ppo.py) | [arXiv](https://arxiv.org/abs/1707.06347) | Proximal Policy Optimization | [Doc](https://ai-traineree.readthedocs.io/en/latest/agents.html#ppo) |
+| SAC | [Implemented](ai_traineree/agents/sac.py) | [arXiv](https://arxiv.org/abs/1801.01290) | Soft Actor Critic | [Doc](https://ai-traineree.readthedocs.io/en/latest/agents.html#sac) |
+| TRPO | | [arXiv](https://arxiv.org/abs/1502.05477) | Trust Region Policy Optimization |
+| RAINBOW | [Implemented](ai_traineree/agents/rainbow.py) | [arXiv](https://arxiv.org/abs/1710.02298) | DQN with a few improvements | [Doc](https://ai-traineree.readthedocs.io/en/latest/agents.html#rainbow) |
+
+### Multi agents
+
+We provide both Multi Agents agents entities and means to execute them against supported (below) environements.
+However, that doesn't mean one can be used without the other.
+
+| Short | Progress | Link | Full name | Doc |
+| ------ | ------------------------------------------------- | ----------------------------------------- | ---------------------- | ---------------------------------------------------------------------------- |
+| IQL | [Implemented](ai_traineree/multi_agent/iql.py) | | Independent Q-Learners | [Doc](https://ai-traineree.readthedocs.io/en/latest/multi_agent.html#iql) |
+| MADDPG | [Implemented](ai_traineree/multi_agent/maddpg.py) | [arXiv](https://arxiv.org/abs/1706.02275) | Multi agent DDPG | [Doc](https://ai-traineree.readthedocs.io/en/latest/multi_agent.html#maddpg) |
+
+### Loggers
+
+Supports using Tensorboard (via PyTorch's [SummaryWriter](https://pytorch.org/docs/stable/tensorboard.html)) and [Neptune](https://neptune.ai) to display metrics. Wrappers are provided as `TensorboardLogger` and `NeptuneLogger`.
+
+*Note*: In order to use Neptune one needs to install [neptune-client](https://pypi.org/project/neptune-client/) (`pip install neptune-client`).
+
+### Environments
+
+| Name | Progress | Link |
+| -------------------- | ------------------ | -------------------------------------------------------------------------------------------- |
+| OpenAI Gym - Classic | Done |
+| OpenAI Gym - Atari | Done |
+| OpenAI Gym - MuJoCo | Not interested. |
+| PettingZoo | Initial support | [Page](https://www.pettingzoo.ml/) / [GitHub](https://github.com/PettingZoo-Team/PettingZoo) |
+| Unity ML | Somehow supported. | [Page](https://unity3d.com/machine-learning) |
+| MAME Linux emulator | Interested. | [Official page](https://www.mamedev.org/) |
+
+### Development
+
+We are open to any contributions. If you want to contribute but don't know what then feel free to reach out (see Contact below).
+The best way to start is through updating documentation and adding tutorials.
+In addition there are many other things that we know of which need improvement but also plenty that we don't know of.
+
+Setting up development environment requires installing `dev` and `test` extra packages.
+The `dev` extras are for mainly for linting and formatting, and the `test` is for running tests.
+We recommend using `pip` so to install everything requires for development run
+
+```bash
+$ pip install -e .[dev,test]
+```
+
+Once installed, please configure your IDE to use `black` as formatter, `pycodestyle` as linter,
+and `isort` for sorting imports. All these are included in the `dev` extra packages.
+
+### Contact
+
+Should we focus on something specificallly? Let us know by opening a feature request [GitHub issue](https://github.com/laszukdawid/ai-traineree/issues) or contacting through [ai-traineree@dawid.lasz.uk](mailto:ai-traineree@dawid.lasz.uk).
+
+## Citing project
+
+```latex
+@misc{ai-traineree,
+ author = {Laszuk, Dawid},
+ title = {AI Traineree: Reinforcement learning toolset},
+ year = {2020},
+ publisher = {GitHub},
+ journal = {GitHub repository},
+ howpublished = {\url{https://github.com/laszukdawid/ai-traineree}},
+}
+```
+
+
+
+
+%package help
+Summary: Development documents and examples for ai-traineree
+Provides: python3-ai-traineree-doc
+%description help
+# ai-traineree
+[![Build Status](https://app.travis-ci.com/laszukdawid/ai-traineree.svg?branch=master)](https://app.travis-ci.com/laszukdawid/ai-traineree)
+[![DocStatus](https://readthedocs.org/projects/ai-traineree/badge/?version=latest)](https://ai-traineree.readthedocs.io/)
+[![codecov](https://codecov.io/gh/laszukdawid/ai-traineree/branch/master/graph/badge.svg?token=S62DK7HPYA)](https://codecov.io/gh/laszukdawid/ai-traineree)
+[![Codacy Badge](https://app.codacy.com/project/badge/Grade/00563b8422454e10bb4ffab64068aa62)](https://www.codacy.com/gh/laszukdawid/ai-traineree/dashboard?utm_source=github.com&amp;utm_medium=referral&amp;utm_content=laszukdawid/ai-traineree&amp;utm_campaign=Badge_Grade)
+[![Discord chat](https://img.shields.io/discord/909530201837432873?logo=discord)](https://discord.gg/n4bN7ZHn)
+[![DOI](https://zenodo.org/badge/267484000.svg)](https://zenodo.org/badge/latestdoi/267484000)
+
+The intention is to have a zoo of Deep Reinforcment Learning methods and showcasing their application on some environments.
+
+Read more in the doc: [ReadTheDocs AI-Traineree](https://ai-traineree.readthedocs.io/).
+
+![CartPole-v1](./static/CartPole-v1.gif)
+![Snek](./static/hungrysnek.gif)
+
+## Why another?
+
+The main reason is the implemention philosophy.
+We strongly believe that agents should be emerged in the environment and not the other way round.
+Majority of the popular implementations pass environment instance to the agent as if the agent was the focus point.
+This might ease implementation of some algorithms but it isn't representative of the world;
+agents want to control the environment but that doesn't mean they can/should.
+
+That, and using PyTorch instead of Tensorflow or JAX.
+
+## Quick start
+
+To get started with training your RL agent you need three things: an agent, an environment and a runner. Let's say you want to train a DQN agent on OpenAI CartPole-v1:
+```python
+from ai_traineree.agents.dqn import DQNAgent
+from ai_traineree.runners.env_runner import EnvRunner
+from ai_traineree.tasks import GymTask
+
+task = GymTask('CartPole-v1')
+agent = DQNAgent(task.obs_space, task.action_space)
+env_runner = EnvRunner(task, agent)
+
+scores = env_runner.run()
+```
+
+or execute one of provided examples
+> \$ python -m examples.cart_dqn
+
+That's it.
+
+## Installation
+
+### PyPi (recommended)
+
+The quickest way to install package is through `pip`.
+
+> \$ pip install ai-traineree
+
+### Git repository clone
+
+As usual with Python, the expectation is to have own virtual environment and then pip install requirements. For example,
+```bash
+> python -m venv .venv
+> git clone git@github.com:laszukdawid/ai-traineree.git
+> source .venv/bin/activate
+> python setup.py install
+```
+
+## Current state
+
+### Playing gym
+One way to improve learning speed is to simply show them how to play or, more researchy/creepy, provide a proper seed.
+This isn't a general rule, since some algorithms train better without any human interaction, but since you're on GitHub... that's unlikely your case.
+Currently there's a script [`interact.py`](scripts/interact.py) which uses OpenAI Gym's play API to record moves and AI Traineree to store them
+in a buffer. Such buffers can be loaded by agents on initiation.
+
+This is just a beginning and there will be more work on these interactions.
+
+*Requirement*: Install `pygame`.
+
+### Agents
+
+| Short | Progress | Link | Full name | Doc |
+| ------- | --------------------------------------------- | -------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------- | ------------------------------------------------------------------------ |
+| DQN | [Implemented](ai_traineree/agents/dqn.py) | [DeepMind](https://deepmind.com/research/publications/human-level-control-through-deep-reinforcement-learning) | Deep Q-learning Network | [Doc](https://ai-traineree.readthedocs.io/en/latest/agents.html#dqn) |
+| DDPG | [Implemented](ai_traineree/agents/ddpg.py) | [arXiv](https://arxiv.org/abs/1509.02971) | Deep Deterministic Policy Gradient | [Doc](https://ai-traineree.readthedocs.io/en/latest/agents.html#ddpg) |
+| D4PG | [Implemented](ai_traineree/agents/d4pg.py) | [arXiv](https://arxiv.org/abs/1804.08617) | Distributed Distributional Deterministic Policy Gradients | [Doc](https://ai-traineree.readthedocs.io/en/latest/agents.html#d4pg) |
+| TD3 | [Implemented](ai_traineree/agents/td3.py) | [arXiv](https://arxiv.org/abs/1802.09477) | Twine Delayed Deep Deterministic policy gradient | [Doc](https://ai-traineree.readthedocs.io/en/latest/agents.html#td3) |
+| PPO | [Implemented](ai_traineree/agents/ppo.py) | [arXiv](https://arxiv.org/abs/1707.06347) | Proximal Policy Optimization | [Doc](https://ai-traineree.readthedocs.io/en/latest/agents.html#ppo) |
+| SAC | [Implemented](ai_traineree/agents/sac.py) | [arXiv](https://arxiv.org/abs/1801.01290) | Soft Actor Critic | [Doc](https://ai-traineree.readthedocs.io/en/latest/agents.html#sac) |
+| TRPO | | [arXiv](https://arxiv.org/abs/1502.05477) | Trust Region Policy Optimization |
+| RAINBOW | [Implemented](ai_traineree/agents/rainbow.py) | [arXiv](https://arxiv.org/abs/1710.02298) | DQN with a few improvements | [Doc](https://ai-traineree.readthedocs.io/en/latest/agents.html#rainbow) |
+
+### Multi agents
+
+We provide both Multi Agents agents entities and means to execute them against supported (below) environements.
+However, that doesn't mean one can be used without the other.
+
+| Short | Progress | Link | Full name | Doc |
+| ------ | ------------------------------------------------- | ----------------------------------------- | ---------------------- | ---------------------------------------------------------------------------- |
+| IQL | [Implemented](ai_traineree/multi_agent/iql.py) | | Independent Q-Learners | [Doc](https://ai-traineree.readthedocs.io/en/latest/multi_agent.html#iql) |
+| MADDPG | [Implemented](ai_traineree/multi_agent/maddpg.py) | [arXiv](https://arxiv.org/abs/1706.02275) | Multi agent DDPG | [Doc](https://ai-traineree.readthedocs.io/en/latest/multi_agent.html#maddpg) |
+
+### Loggers
+
+Supports using Tensorboard (via PyTorch's [SummaryWriter](https://pytorch.org/docs/stable/tensorboard.html)) and [Neptune](https://neptune.ai) to display metrics. Wrappers are provided as `TensorboardLogger` and `NeptuneLogger`.
+
+*Note*: In order to use Neptune one needs to install [neptune-client](https://pypi.org/project/neptune-client/) (`pip install neptune-client`).
+
+### Environments
+
+| Name | Progress | Link |
+| -------------------- | ------------------ | -------------------------------------------------------------------------------------------- |
+| OpenAI Gym - Classic | Done |
+| OpenAI Gym - Atari | Done |
+| OpenAI Gym - MuJoCo | Not interested. |
+| PettingZoo | Initial support | [Page](https://www.pettingzoo.ml/) / [GitHub](https://github.com/PettingZoo-Team/PettingZoo) |
+| Unity ML | Somehow supported. | [Page](https://unity3d.com/machine-learning) |
+| MAME Linux emulator | Interested. | [Official page](https://www.mamedev.org/) |
+
+### Development
+
+We are open to any contributions. If you want to contribute but don't know what then feel free to reach out (see Contact below).
+The best way to start is through updating documentation and adding tutorials.
+In addition there are many other things that we know of which need improvement but also plenty that we don't know of.
+
+Setting up development environment requires installing `dev` and `test` extra packages.
+The `dev` extras are for mainly for linting and formatting, and the `test` is for running tests.
+We recommend using `pip` so to install everything requires for development run
+
+```bash
+$ pip install -e .[dev,test]
+```
+
+Once installed, please configure your IDE to use `black` as formatter, `pycodestyle` as linter,
+and `isort` for sorting imports. All these are included in the `dev` extra packages.
+
+### Contact
+
+Should we focus on something specificallly? Let us know by opening a feature request [GitHub issue](https://github.com/laszukdawid/ai-traineree/issues) or contacting through [ai-traineree@dawid.lasz.uk](mailto:ai-traineree@dawid.lasz.uk).
+
+## Citing project
+
+```latex
+@misc{ai-traineree,
+ author = {Laszuk, Dawid},
+ title = {AI Traineree: Reinforcement learning toolset},
+ year = {2020},
+ publisher = {GitHub},
+ journal = {GitHub repository},
+ howpublished = {\url{https://github.com/laszukdawid/ai-traineree}},
+}
+```
+
+
+
+
+%prep
+%autosetup -n ai-traineree-0.5.2
+
+%build
+%py3_build
+
+%install
+%py3_install
+install -d -m755 %{buildroot}/%{_pkgdocdir}
+if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi
+if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi
+if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi
+if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi
+pushd %{buildroot}
+if [ -d usr/lib ]; then
+ find usr/lib -type f -printf "/%h/%f\n" >> filelist.lst
+fi
+if [ -d usr/lib64 ]; then
+ find usr/lib64 -type f -printf "/%h/%f\n" >> filelist.lst
+fi
+if [ -d usr/bin ]; then
+ find usr/bin -type f -printf "/%h/%f\n" >> filelist.lst
+fi
+if [ -d usr/sbin ]; then
+ find usr/sbin -type f -printf "/%h/%f\n" >> filelist.lst
+fi
+touch doclist.lst
+if [ -d usr/share/man ]; then
+ find usr/share/man -type f -printf "/%h/%f.gz\n" >> doclist.lst
+fi
+popd
+mv %{buildroot}/filelist.lst .
+mv %{buildroot}/doclist.lst .
+
+%files -n python3-ai-traineree -f filelist.lst
+%dir %{python3_sitelib}/*
+
+%files help -f doclist.lst
+%{_docdir}/*
+
+%changelog
+* Thu May 18 2023 Python_Bot <Python_Bot@openeuler.org> - 0.5.2-1
+- Package Spec generated
diff --git a/sources b/sources
new file mode 100644
index 0000000..e1e6b1b
--- /dev/null
+++ b/sources
@@ -0,0 +1 @@
+94c07376a7a1e2dd37b6fd53f5967c68 ai-traineree-0.5.2.tar.gz