diff options
| -rw-r--r-- | .gitignore | 1 | ||||
| -rw-r--r-- | python-watiba.spec | 3318 | ||||
| -rw-r--r-- | sources | 1 |
3 files changed, 3320 insertions, 0 deletions
@@ -0,0 +1 @@ +/watiba-0.6.59.tar.gz diff --git a/python-watiba.spec b/python-watiba.spec new file mode 100644 index 0000000..d0caf12 --- /dev/null +++ b/python-watiba.spec @@ -0,0 +1,3318 @@ +%global _empty_manifest_terminate_build 0 +Name: python-watiba +Version: 0.6.59 +Release: 1 +Summary: Python syntactical sugar for embedded shell commands +License: MIT +URL: https://github.com/Raythonic/watiba +Source0: https://mirrors.nju.edu.cn/pypi/web/packages/74/42/84b3fa7e253b186eca318ed809307624cca7b095901dc5ea741263e83934/watiba-0.6.59.tar.gz +BuildArch: noarch + + +%description +# Watiba +#### Version: **0.6.59** +#### Date: 2021/12/04 + +Watiba, pronounced wah-TEE-bah, is a lightweight Python pre-compiler for embedding Linux shell +commands within Python applications. It is similar to other languages' syntactical enhancements where +XML or HTML is integrated into a language such as JavaScript. That is the concept applied here but integrating +BASH shell commands with Python. + +As you browse this document, you'll find Watiba is rich with features for shell command integration with Python. + +Features: +- Shell command integration with Python code +- In-line access to shell command results +- Current directory context maintained across commands throughout your Python code +- Async/promise support for integrated shell commands +- Remote shell command execution +- Remote shell command chaining and piping + +## Table of Contents +1. [Usage](#usage) +2. [Directory Context](#directory-context) +3. [Commands as Variables](#commands-as-variables) +4. [Command Results](#command-results) +5. [Asynchronous Spawning and Promises](#async-spawing-and-promises) + 1. [Useful Properties in Promise](#useful-properties-in-promise) + 2. [Spawn Controller](#spawn-controller) + 3. [Join, Wait or Watch](#join-wait-watch) + 4. [The Promise Tree](#promise-tree) + 5. [Threads](#threads) +6. [Remote Execution](#remote-execution) + 1. [Change SSH port for remote execution](#change-ssh-port) +7. [Command Hooks](#command-hooks) +8. [Command Chaining](#command-chaining) +9. [Command Chain Piping (Experimental)](#piping-output) +10. [Installation](#installation) +11. [Pre-compiling](#pre-compiling) +12. [Code Examples](#code-examples) + +<div id="usage"/> + +## Usage +Watiba files, suffixed with ".wt", are Python programs containing embedded shell commands. +Shell commands are expressed within backtick characters emulating BASH's original capture syntax. +They can be placed in any Python statement or expression. Watiba keeps track of the current working directory +after the execution of any shell command so that all subsequent shell commands keep context. For example: + +Basic example of embedded commands: +``` +#!/usr/bin/python3 + +# Typical Python program + +if __name__ == "__main__": + + # Change directory context + `cd /tmp` + + # Directory context maintained + for file in `ls -lrt`.stdout: # In-line access to command results + print(f"File in /tmp: {file}") +``` + +This loop will display the file list from /tmp. The `ls -lrt` is run in the +context of previous `cd /tmp`. + +<div id="commands-as-variables"/> + +#### Commands Expressed as Variables +Commands within backticks can _be_ a variable, but cannot contain snippets of Python code or Python variables. +The statement within the backticks _must_ be either a pure shell command or a Python variable containing a pure +shell command. To execute commands in a Python variable, prefix the variable name between backticks with a dollar sign. + +_A command variable is denoted by prepending a dollar sign on the variable name within backticks_: +``` +# Set the Python variable to the command +cmdA = 'echo "This is a line of output" > /tmp/blah.txt' +cmdB = 'cat /tmp/blah.txt' + +# Execute first command +`$cmdA` # Execute the command within Python variable cmdA + +# Execute second command +for line in `$cmdB`.stdout: + print(line) +``` + +_This example demonstrates keeping dir context and executing a command by variable_: +``` +#!/usr/bin/python3 + +if __name__ == "__main__": + # Change CWD to /tmp + `cd /tmp` + + # Set a command string + my_cmd = "tar -zxvf tmp.tar.gz" + + # Execute that command and save the command results in variable "w" + w = `$my_cmd` + if w.exit_code == 0: + for l in w.stderr: + print(l) +``` + +_These constructs are **not** supported_: + ``` +file_name = "blah.txt" + +# Python variable within backticks +`touch file_name` # NOT SUPPORTED! + +# Attempting to access Python variable with dollar sign +`touch $file_name` # NOT SUPPORTED! + +# Python within backticks is NOT SUPPORTED! +`if x not in l: ls -lrt x` +``` +<div id="directory-context"/> + +## Directory Context + +An important Watiba usage point is directory context is kept for dispersed shell commands. +Any command that changes the shell's CWD is discovered and kept by Watiba. Watiba achieves +this by tagging a `&& echo pwd` to the user's command, locating the result in the command's STDOUT, +and finally setting the Python environment to that CWD with `os.chdir(dir)`. This is automatic and +opaque to the user. The user will not see the results of the generated suffix. If the `echo` +suffix presents a problem for the user, it can be eliminated by prefixing the leading backtick with a +dash. The dash turns off the context tracking by not suffixing the command and so causes Watiba to +lose its context. However, the context is maintained _within_ the set of commands in the backticks just not +when it returns. For example, **out = -\`cd /tmp && ls -lrt\`** honors the ```cd``` within the scope +of that execution line, but not for any backticked commands that follow later in your code. + +**_Warning!_** The dash will cause Watiba to lose its directory context should the command +cause a CWD change either explicitly or implicitly. + +_Example_: +``` +`cd /tmp` # Context will be kept + +# This will print from /home/user, but context is NOT kept +for line in -`cd /home/user && ls -lrt`.stdout: + print(line) + +# This will print from /tmp, not /home/user +for line in `ls -lrt`.stdout: + print(line) +``` + +<div id="command-results"/> + +## Command Results +The results of the command issued in backticks are available in the properties +of the object returned by Watiba. Following are those properties: + +<table> + <th>Property</th><th>Data Type</th><th>Description</th> + <tr></tr> + <td valign="top">stdout</td><td valign="top">List</td><td valign="top">STDOUT lines from the command normalized for display</td> + <tr></tr> + <td valign="top">stderr</td><td valign="top">List</td><td valign="top">STDERR lines from the command normalized for display</td> + <tr></tr> + <td valign="top">exit_code</td><td valign="top">Integer</td><td valign="top">Exit code value from command</td> + <tr></tr> + <td valign="top">cwd</td><td valign="top">String</td><td valign="top">Current working directory <i>after</i> command was executed</td> +</table> + +Technically, the returned object for any shell command is defined in the WTOutput class. + +<div id="async-spawing-and-promises"/> + +## Asynchronous Spawning and Promises +Shell commands can be executed asynchronously with a defined resolver callback block. Each _spawn_ expression creates +and runs a new OS thread. The resolver is a callback block that follows the Watiba _spawn_ expression. The spawn +feature is executed when a ```spawn `cmd` args: resolver block``` code block is encountered. The +resolver is passed the results in the promise object. (The promise structure contains the properties +defined in section ["Results from Spawned Commands"](#spawn-results) The _spawn_ expression also returns a _promise_ object +to the caller of _spawn_. The promise object is passed to the _resolver block_ in argument _promise_. The +outer code can check its state with a call to _resolved()_ on the *returned* promise object. Output from the command +is found in _promise.output_. The examples throughout this README and in the _examples.wt_ file make this clear. + +<div id="useful-properties-in-promise"/> + +##### Useful properties in promise structure +A promise is either returned in assignment from outermost spawn, or passed to child spawns in argument "promise". + + <table> + <th>Property</th> + <th>Data Type</th> + <th>Description</th> + <tr></tr> + <td valign="top">host</td><td valign="top">String</td><td valign="top">Host name on which spawned command ran</td> + <tr></tr> + <td valign="top">children</td><td valign="top">List</td><td valign="top">Children promises for this promise node</td> + <tr></tr> + <td valign="top">parent</td><td valign="top">Reference</td><td valign="top">Parent promise node of child promise. None if root promise.</td> + <tr></tr> + <td valign="top">command</td><td valign="top">String</td><td valign="top">Shell command issued for this promise</td> + <tr></tr> + <td valign="top">resolved()</td><td valign="top">Method</td><td valign="top">Call to find out if this promise is resolved</td> + <tr></tr> + <td valign="top">resolve_parent()</td><td valign="top">Method</td><td valign="top">Call inside resolver block to resolve parent promise</td> + <tr></tr> + <td valign="top">tree_dump()</td><td valign="top">Method</td><td valign="top">Call to show the promise tree. Takes subtree argument otherwise it defaults to the root promise</td> + <tr></tr> + <td valign="top">join()</td><td valign="top">Method</td><td valign="top">Call to wait on on promise and all its children</td> + <tr></tr> + <td valign="top">wait()</td><td valign="top">Method</td><td valign="top">Call to wait on just this promise</td> + <tr></tr> + <td valign="top">watch()</td><td valign="top">Method</td><td valign="top">Call to create watcher on this promise</td> + <tr></tr> + <td valign="top">start_time</td><td valign="top">Time</td><td valign="top">Time that spawned command started</td> + <tr></tr> + <td valign="top">end_time</td><td valign="top">Time</td><td valign="top">Time that promise resolved</td> + </table> + +_Example of simple spawn_: +```buildoutcfg +prom = spawn `tar -zcvf big_file.tar.gz some_dir/*`: + # Resolver block to which "promise" and "args" is passed... + print(f"{promise.command} completed.") + return True # Resolve promise + +# Do other things while tar is running +# Finally wait for tar promise to resolve +prom.join() +``` + +<div id="spawn-controller"/> + +#### Spawn Controller +All spawned threads are managed by Watiba's Spawn Controller. The controller watches for too many threads and +incrementally slows down each thread start when that threshold is exceeded until either all the promises in the tree +resolve, or an expiration count is reached, at which time an exception is thrown on the last spawned command. +This exception is raised by the default error method. This method as well as other spawn controlling parameters +can be overridden. The controller's purpose is to not allow run away threads and provide signaling of possible +hung threads. + +_spawn-ctl_ example: +```buildoutcfg +# Only allow 20 spawns max, +# and increase slowdown by 1/2 second each 3rd cycle +...python code... +spawn-ctl {"max":20, "sleep-increment":.500} +``` + +Spawn control parameters: + +<table> + <th>Key Name</th> + <th>Data Type</th> + <th>Description</th> + <th>Default</th> + <tr></tr> + <td valign="top">max</td><td valign="top">Integer</td><td valign="top">The maximum number of spawned commands allowed before the controller enters slowdown mode</td><td valign="top">10</td> + <tr></tr> + <td valign="top">sleep-floor</td><td valign="top">Integer</td><td valign="top">Seconds of <i>starting</i> +sleep value when the controller enters slowdown mode</td><td valign="top">.125 (start at 1/8th second pause)</td> + <tr></tr> + <td valign="top">sleep-increment</td><td valign="top">Integer</td><td valign="top">Seconds the <i>amount</i> of seconds sleep will increase every 3rd cycle when in slowdown + mode</td><td valign="top">.125 (Increase pause 1/8th second every 3rd cycle)</td> + <tr></tr> + <td valign="top">sleep-ceiling</td><td valign="top">Integer</td><td valign="top">Seconds the <i>highest</i> length sleep value allowed when in slowdown mode + (As slow as it will get)</td><td valign="top">3 (won't get slower than 3 second pauses)</td> + <tr></tr> + <td valign="top">expire</td><td valign="top">Integer</td><td valign="top">Total number of slowdown cycles allowed before the error method is called</td><td valign="top">No expiration</td> + <tr></tr> + <td valign="top">error</td><td valign="top">Method</td><td valign="top"> + Callback method invoked when slowdown mode expires. Use this to catch hung commands. + This method is passed 2 arguments: + +- **promise** - The promise attempting execution at the time of expiration +- **count** - The thread count (unresolved promises) at the time of expiration + </td><td valign="top">Generic error handler. Just throws <i>WTSpawnException</i> that hold properties <i>promise</i> and <i>message</i></td></td> +</table> + <hr> + +**_spawn-ctl_** only overrides the values it sets and does not affect values not specified. _spawn-ctl_ statements can +set whichever values it wants, can be dispersed throughout your code (i.e. multiple _spawn-ctl_ statements) and +only affects subsequent spawn expressions. + +_Notes:_ +1. Arguments can be passed to the resolver by specifying a trailing variable after the command. If the arguments +variable is omitted, an empty dictionary, i.e. {}, is passed to the resolver in _args_. +**_Warning!_** Python threading does not deep copy objects passed as arguments to threads. What you place in ```args``` +of the spawn expression will only be shallow copied so if there are references to other objects, it's not likely to + survive the copy. +2. The resolver must return _True_ to set the promise to resolved, or _False_ to leave it unresolved. +3. A resolver can also set the promise to resolved by calling ```promise.set_resolved()```. This is handy in cases where +a resolver has spawned another command and doesn't want the outer promise resolved until the inner resolvers are done. +To resolve an outer, i.e. parent, resolver issue _promise.resolve_parent()_. Then the parent resolver can return +_False_ at the end of its block so it leaves the resolved determination to the inner resolver block. +4. Each promise object holds its OS thread object in property _thread_ and its thread id in property _thread_id_. This +can be useful for controlling the thread directly. For example, to signal a kill. +5. _spawn-ctl_ has no affect on _join_, _wait_, or _watch_. This is because _spawn-ctl_ establishes an upper end +throttle on the overall spawning process. When the number of spawns hits the max value, throttling (i.e. slowdown + mode) takes affect and will expire if none of the promises resolve. Conversely, the arguments used by _join_, + _wait_ and _watch_ control the sleep cycle and expiration of just those calls, not the spawned threads as a whole. When + an expiration is set for, say, _join_, then that join will expire at that time. When an expiration is set in + _spawn-ctl_, then if all the spawned threads as a whole don't resolve in time then an expiration function is called. + + +**_Spawn Syntax:_** +``` +my_promise = spawn `cmd` [args]: + resolver block (promise, args) + args passed in args + return resolved or unresolved (True or False) + ``` + +_Spawn with resolver arguments omitted_: +``` +my_promise = spawn `cmd`: + resolver block (promise, args) + return resolved or unresolved (True or False) +``` + +_Simple spawn example_: +```buildoutcfg +p = spawn `tar -zcvf /tmp/file.tar.gz /home/user/dir`: + # Resolver block to which "promise" and "args" are passed + # Resolver block is called when spawned command has completed + for line in promise.output.stderr: + print(line) + + # This marks the promise resolved + return True + +# Wait for spawned command to resolve (not merely complete) +try: + p.join({"expire": 3}) + print("tar resolved") +except Exception as ex: + print(ex.args) +``` + +_Example of file that overrides spawn controller parameters_: +``` +#!/usr/bin/python3 +def spawn_expired(promise, count): + print("I do nothing just to demonstrate the error callback.") + print(f"This command failed {promise.command} at this threshold {count}") + + raise Exception("Too many threads.") + +if __name__ == "__main__": + # Example showing default values + parms = {"max": 10, # Max number of threads allowed before slowdown mode + "sleep-floor": .125, # Starting sleep value + "sleep-ceiling": 3, # Maximum sleep value + "sleep-increment": .125, # Incremental sleep value + "expire": -1, # Default: no expiration + "error": spawn_expired # Method called upon slowdown expiration + } + + # Set spawn controller parameter values + spawn-ctl parms +``` + +<div id="join-wait-watch"/> + +#### Join, Wait, or Watch + +Once commands are spawned, the caller can wait for _all_ promises, including inner or child promises, to complete, or +the caller can wait for just a specific promise to complete. To wait for all _child_ promises including +the promise on which you're calling this method, call _join()_. It will wait for that promise and all its children. To +wait for just one specific promise, call _wait()_ on the promise of interest. To wait for _all_ promises in +the promise tree, call _join()_ on the root promise. + +_join_ and _wait_ can be controlled through parameters. Each are iterators paused with a sleep method and will throw +an expiration exception should you set a limit for iterations. If an expiration value is not set, +no exception will be thrown and the cycle will run only until the promise(s) are resolved. _join_ and _wait_ are not +affected by _spawn-ctl_. + +_watch_ is called to establish a separate asynchronous thread that will call back a function of your choosing should +the command the promise is attached to time out. This is different than _join_ and _wait_ in that _watch_ is not synchronous +and does not pause. This is used to keep an eye on a spawned command and take action should it hang. Your watcher +function is passed the promise on which the watcher was attached, and the arguments, if any, from the spawn expression. +If your command does not time out (i.e. hangs and expires), the watcher thread will quietly go away when the promise +is resolved. _watch_ expiration is expressed in **seconds**, unlike _join_ and _wait_ which are expressed as total +_iterations_ paused at the sleep value. _watch_'s polling cycle pause is .250 seconds, so the expiration value is +multiplied by 4. The default expiration is 15 seconds. + +Examples: +``` +# Spawn a thread running this command +p = spawn `ls -lrt`: + ## resolver block ## + return True + +# Wait for promises, pause for 1/4 second each iteration, and throw an exception after 4 iterations +(1 second) +try: + p.join({"sleep": .250, "expire": 4}) +except Exception as ex: + print(ex.args) + +# Wait for this promise, pause for 1 second each iteration, and throw an exception after 5 iterations +(5 seconds) +try: + p.wait({"sleep": 1, "expire": 5}) +except Exception as ex: + print(ex.args) + +# My watcher function (called if spawned command never resolves by its experation period) +def watcher(promise, args): + print(f"This promise is likely hung: {promise.command}") + print(f"and I still have the spawn expression's args: {args}") + +p = spawn `echo "hello" && sleep 5` args: + print(f"Args passed to me: {args}") + return True + +# Attach a watcher to this thread. It will be called upon experation. +p.watch(watcher) +print("watch() does not pause like join or wait") + +# Attach a watcher that will expire in 5 seconds +p.watch(watcher, {"expire": 5}) +``` + +**_join_ syntax** +``` +promise.join({optional args}) +Where args is a Python dictionary with the following options: + "sleep" - seconds of sleep for each iteration (fractions such as .5 are honored) + default: .5 seconds + "expire" - number of sleep iterations until an excpetions is raised + default: no expiration +Note: "args" is optional and can be omitted +``` + +_Example of joining parent and children promises_: +``` +p = spawn `ls *.txt`: + for f in promise.output.stdout: + cmd = f"tar -zcvf {f}.tar.gz {f}" + spawn `$cmd` {"file":f}: + print(f"{f} completed") + promise.resolve_parent() + return True + return False + +# Wait for all commands to complete +try: + p.join({"sleep":1, "expire":20}) +except Exception as ex: + print(ex.args) +``` + +**_wait_ syntax** +``` +promise.wait({optional args}) +Where args is a Python dictionary with the following options: + "sleep" - seconds of sleep for each iteration (fractions such as .5 are honored) + default: .5 seconds + "expire" - number of sleep iterations until an excpetions is raised + default: no expiration +Note: "args" is optional and can be omitted +``` + +_Example of waiting on just the parent promise_: +``` +p = spawn `ls *.txt`: + for f in promise.output.stdout: + cmd = f"tar -zcvf {f}.tar.gz {}" + spawn `$cmd` {"file":f}: + print(f"{f} completed") + promise.resolve_parent() # Wait completes here + return True + return False + +# Wait for just the parent promise to complete +try: + p.wait({"sleep":1, "expire":20}) +except Exception as ex: + print(ex.args) +``` + +**_watch_ syntax** +``` +promise.watch(callback, {optional args}) +Where args is a Python dictionary with the following options: + "sleep" - seconds of sleep for each iteration (fractions such as .5 are honored) + default: .5 seconds + "expire" - number of sleep iterations until an excpetions is raised + default: no expiration +Note: "args" is optional and can be omitted +``` + +_Example of creating a watcher_: +```buildoutcfg +# Define watcher method. Called if command times out (i.e. expires) +def time_out(promise, args): + print(f"Command {promise.command} timed out.") + +# Spawn a thread running some command that hangs +p = spawn `long-running.sh`: + print("Finally completed. Watcher method won't be called.") + return True + +p.watch(time_out) # Does not wait. Calls method "time_out" if this promise expires (i.e. command hangs) + +# Do other things... + +``` + +<div id="promise-tree"/> + +#### The Promise Tree +Each _spawn_ issued inserts its promise object into the promise tree. The outermost _spawn_ will generate the root +promise and each inner _spawn_ will be among its children. There's no limit to how far it can nest. _wait_ only applies +to the promise on which it is called and is how it is different than _join_. _wait_ does not consider any other +promise state but the one it's called for, whereas _join_ considers the one it's called for **and** anything below it +in the tree. + +The promise tree can be printed with the ```dump_tree()``` method on the promise. This method is intended for +diagnostic purposes where it must be determined why spawned commands hung. ```dump_tree(subtree)``` accepts +a subtree promise as an argument. If no arguments are passed, ```dump_tree()``` dumps from the root promise on down. +``` +# Simple example with no child promises +p = spawn `date`: + return True + +p.tree_dump() # Dump tree from root +# or +p.tree_dump(subtree_promise) # Dump tree from node in argument +``` + +Example dumping tree from subtree node: +```buildoutcfg +# Complex example with child and grandchild promises +# Demonstrates how to dump the promise tree from various points within it +p = spawn `date`: + # Spawn child command (child promise) + spawn `pwd`: + # Spawn a grandchild to the parent promise + spawn `python --version`: + promise.tree_dump(promise) # Dump the subtree from this point down + return False + # Spawn another child + spawn `echo "blah"`: + # Resolve parent promise + promise.resolve_parent() + # Resolve child promise + return True + # Do NOT resolve parent promise, let child do that + return False + +p.join() +p.tree_dump(p.children[0]) # Dump subtree from first child on down +p.tree_dump(p.children[1]) # Dump subtree from the second child +p.tree_dump(p.children[0].children[0]) # Dump subtree from the grandchild + +# Dump all children +for c in p.children: + p.tree_dump(c) +``` + +_Parent and child joins shown in these two examples_: + +``` +root_promise = spawn `ls -lr`: + for file in promise.stdout: + t = f"touch {file}" + spawn `$t` {"file" file}: # This promise is a child of root + print(f"{file} updated".) + spawn `echo "done" > /tmp/done"`: # Another child promise (root's grandchild) + print("Complete") + promise.resolve_parent() + return True + promise.resolve_parent() + return False + return False + +root_promise.join() # Wait on the root promise and all its children. Thus, waiting for everything. +``` + +``` +root_promise = spawn `ls -lr`: + for file in promise.output.stdout: + t = f"touch {file}" + spawn `$t` {"file" file}: # This promise is a child of root + print(f"{promise.args['file'])} updated") + promise.join() # Wait for this promise and its children but not its parent (root) + spawn `echo "done" > /tmp/done"`: + print("Complete") +``` + + + +_Resolving a parent promise_: +``` +p = spawn `ls -lrt`: + for f in promise.output.stdout: + cmd = f"touch {f}" + # Spawn command from this resolver and pass our promise + spawn `$cmd`: + print("Resolving all promises") + promise.resolve_parent() # Resolve parent promise here + return True # Resolve child promise + return False # Do NOT resolve parent promise here +p.join() # Wait for ALL promises to be resolved +``` + +<div id="spawn-results"/> + +### Results from Spawned Commands +Spawned commands return their results in the _promise.output_ property of the _promise_ object passed to +the resolver block, and in the spawn expression if there is an assignment in that spawn expression. + +The result properties can then be accessed as followed: + +<table> + <th>Property</th><th>Data Type</th><th>Description</th> + <tr></tr> + <td valign="top">promise.output.stdout</td><td valign="top">List</td><td valign="top">STDOUT lines from the command normalized for display</td> + <tr></tr> + <td valign="top">promise.output.stderr</td><td valign="top">List</td><td valign="top">STDERR lines from the command normalized for display</td> + <tr></tr> + <td valign="top">promise.output.exit_code</td><td valign="top">Integer</td><td valign="top">Exit code value from command</td> + <tr></tr> + <td valign="top">promise.output.cwd</td><td valign="top">String</td><td valign="top">Current working directory <i>after</i> command was executed</td> +</table> + + +_Notes:_ +1. Watiba backticked commands can exist within the resolver +2. Other _spawn_ blocks can be embedded within a resolver (recursion allowed) +3. The command within the _spawn_ definition can be a variable + (The same rules apply as for all backticked shell commands. This means the variable must contain + pure shell commands.) +4. The leading dash to ignore CWD _cannot_ be used in the _spawn_ expression +5. The _promise.output_ object is not available until _promise.resolved()_ returns True + +_Simple example with the shell command as a Python variable_: +``` +#!/usr/bin/python3 + +# run "date" command asynchronously +d = 'date "+%Y/%m/%d"' +spawn `$d`: + print(promise.output.stdout[0]) + return True + +``` + +_Example with shell commands executed within resolver block_: +``` +#!/usr/bin/python3 + +print("Running Watiba spawn with wait") +`rm /tmp/done` + +# run "ls -lrt" command asynchronously +p = spawn `ls -lrt`: + print(f"Exit code: {promise.output.exit_code}") + print(f"CWD: {promise.output.cwd}") + print(f"STDERR: {promise.output.stderr}") + + # Loop through STDOUT from command + for l in promise.output.stdout: + print(l) + `echo "Done" > /tmp/done` + + # Resolve promise + return True + +# Pause until spawn command is complete +p.wait() +print("complete") + +``` + +<div id="threads"/> + +### Threads +Each promise produced from a _spawn_ expression results in one OS thread. To access the +number of threads your code has spawned collectively, you can do the following: +``` +num_of_spawns = promise.spawn_count() # Returns number of nodes in the promise tree +num_of_resolved_promises = promise.resolved_count() # Returns the number of promises resolved in tree +``` +<div id="remote-execution"/> + +## Remote Execution +Shell commands can be executed remotely. This is achieved though the SSH command, issued by Watiba, and has the +following requirements: +- OpenSSH is installed on the local and remote hosts +- The local SSH key is in the remote's _authorized_keys_ file. _The details of this + process is beyond the scope of this README. For those instructions, consult www.ssh.com_ + +- Make sure that SSH'ing to the target host does not cause any prompts. + +Test that your SSH environment is setup first by manually entering: +``` +ssh {user}@{host} "ls -lrt" + +# For example +ssh rwalk@walkubu "ls -lrt" + +# If SSH prompts you, then Watiba remote execution cannot function. +``` + +To execute a command remotely, a _@host_ parameter is suffixed to the backticked command. The host name can be a +literal or a variable. To employ a variable, prepend a _$_ to the name following _@_ such as _@$var_. + +<div id="change-ssh-port"/> + +#### Change SSH port for remote execution +To change the default SSH port 22 to a custom value, add to your Watiba code: ```watiba-ctl {"ssh-port": custom port}``` +Example: +```buildoutcfg +watiba-ctl {"ssh-port": 2233} +``` +Examples: +```buildoutcfg +p = spawn `ls -lrt`@remoteserver {parms}: + for line in promise.output.stdout: + print(line) + return True + +``` +```buildoutcfg +remotename = "serverB" +p = spawn `ls -lrt`@$remotename {parms}: + for line in p.output.stdout: + print(line) + return True +``` +```buildoutcfg +out = `ls -lrt`@remoteserver +for line in out.stdout: + print(line) +``` +```buildoutcfg +remotename = "serverB" +out = `ls -lrt`@$remotename +for line in out.stdout: + print(line) +``` + + +<div id="command-hooks"/> + +## Command Hooks +Hooks are pre- or -post functions that are attached to a _command_ _pattern_, which is a regular expression (regex). Anytime Watiba encounters a command +that matches the pattern for the hook, the hook function is called. + +All commands, spawned, remote, or local, can have Python functions executed **before** exection, by default, or **post hooks** that are run **after** the command. (Note: Post hooks are not run for spwaned commands because the resolver function is a post hook itself.) These functions can be passed arguments, too. + +### Command Hook Expressions +``` +# Run before commands that match that pattern +hook-cmd "pattern" hook-function parms + +# Run before commands that match that pattern, but is non-recursive +hook-cmd-nr "pattern" hook-function parms + +# Run after commands that match that pattern +post-hook-cmd "pattern" hook-function parms + +# Run after commands that match that pattern, but is non-recursive +post-hook-cmd-nr "pattern" hook-function parms +``` + +### Hook Recursion +Hooks, which are nothing more than Python functions called before or after a command is run, can issue their own commands and, thus, cause the hook +to be recursively called. However, if the command in the hook block of code matches a command pattern that causes that same hook function to be run again, +an infinte loop can occur. To prevent that, use the **-nr** suffix on the Watiba hook expression. (-nr stands for non-recursive.) This will ensure that +the hook cannot be re-invoked for any commands that are within it. + +<br> +To attach a hook: +1. Code one or more Python functions that will be the hooks. At the end of each hook, you must return True if the hook was successful, or False +if something wrong. +2. Use the _hook-cmd_ expression to attach those hooks to a command +pattern, which is a regular expression +3. To remove the hooks, use the _remove-hooks "pattern"_ expression. If a pattern, i.e. command regex pattern, is omitted, then all command hooks are removed. + +**hook-cmd "command pattern" function parms** + +The first parameter always passed to the hook function is the Python _match_ object from the command match. This is provided so the hook has access +to the tokens on the command should it need them. + +Example: +``` +def my_hook(match, parms): + print(match.groups()) + print(f'Tar file name is {match.group(1)}') + print(parms["parmA"]) + print(parms["parmB"]) + return True # Successful execution + +def your_hook(match, parms): + # This hook doesn't need the match object, so ignores it + print(parms["something"]) + if parms["something-else"] != "blah": + return False # Failed execution + return True # Successful excution + + +# Add first hook to my tar command +hook-cmd "tar -zcvf (\S.*)" my_hook: {"parmA":"A", "parmB":"B"} + +# Add another hook to my tar command +hook-cmd "tar -zcvf (\S.*)" your_hook: {"parmD":1, "parmE":"something"} + +# Spawn command, but hooks will be invoked first... +spawn `tar -zcvf files.tar.gz /tmp/files/* `: + # Resolver code block + return True # Resolve promise +``` + +Your parameters are whatever is valid for Python. These are simply passed to their attached functions, essentially each one's key is the function name, as specified. + + +_Where are the hooks run for spawned commands?_ All hooks run under the thread of the issuer on the local host, not the target thread. + +_Where are the hooks run for remote commands?_ As with spawned commands, all hooks are issued on the local host, not the remote. Note that you +can have remote backticked commands in your hook and that will run those remotely. If your remote command matches a hook(s) pattern, then those hooks will be run. This means if your command pattern for the first remote call runs a hook that contains another remote command that matches that same command pattern, then the hook is run again. Since this can lead to infinte hook loops, Watiba offers a non-recursive definition for the command pattern. Note that this non-recursive setting +only applies to the command pattern and not the hook function itself. So if _hookA_ is run for two different command patterns, say, "ls -lrt" and "ls -laF" you can +make one non-recusrive and still run the same hook for both commands. For the recursive command pattern, the hook has no limit to its recursion. For non-recursive, +it will only be called once during the recursion process. + +To set a command pattern as non-recursive, use _hook-cmd-nr_. + +Example using a variation on a previous example: + +``` +def my_hook(match, parms) + `tar -zcvf /tmp/files` # my_hook will NOT because for this command even though it matches + print("Will be called only once!") + return True + +# Note the "-nr" on the expression. That's for non-recursive +hook-cmd-nr "tar -zcvf (\S.*)" my_hook: {"parmA":"A", "parmB":"B"} + +# my_hook will be called before this command runs +` tar -zcvf tarball.tar.gz /home/user/files.*` +``` + +<div id="command-chaining"/> + +## Command Chaining +Watiba extends its remote command execution to chaining commands across multiple remote hosts. This is achieved +by the _chain_ expression. This expression will execute the backticked command across a list of hosts, passed by +the user, sequentially, synchronously until the hosts list is exhausted, or the command fails. _chain_ returns a +Python dictionary where the keys are the host names and the values the WTOutput from the command run on that host. + +#### Chain Exception +The _chain_ expression raises a WTChainException on the first failed command. The exception raised +has the following properties: + +_WTChainException_: +<table> +<th>Property</th><th>Data Type</th><th>Description</th> +<tr></tr> +<td valign="top">command</td><td valign="top">String</td><td valign="top">Command that failed</td> +<tr></tr> +<td valign="top">host</td><td valign="top">String</td><td valign="top">Host where command failed</td> +<tr></tr> +<td valign="top">message</td><td valign="top">String</td><td valign="top">Error message</td> +<tr></tr> +<td valign="top">output</td><td valign="top">WTOutput structure: + +- stdout +- stderr +- exit_code +- cwd</td><td valign="top">Output from command</td> +</table> + +Import this exception to catch it: +```buildoutcfg +from watiba import WTChainException +``` + + +Examples: +``` +from watiba import WTChainException + +try: + out = chain `tar -zcvf backup/file.tar.gz dir/*` {"hosts", ["serverA", "serverB"]} + for host,output in out.items(): + print(f'{host} exit code: {output.exit_code}') + for line in output.stderr: + print(line) + except WTChainException(ex): + print(f"Error: {ex.message}") + print(f" host: {ex.host} exit code: {ex.output.exit_code} command: {ex.command}) + +``` + +<div id="piping-output"/> + +## Command Chain Piping (Experimental) +The _chain_ expression supports piping STDOUT and/or STDERR to other commands executed on remote servers. Complex +arrangements can be constructed through the Python dictionary passed to the _chain_ expression. The dictionary +contents function as follows: +- "hosts": [server, server, ...] This entry instructions _chain_ on which hosts the backticked command will run. + This is a required entry. + +- "stdout": {server:command, server:command, ...} + This is an optional entry. + +- "stderr": {server:command, server:command, ...} + This is an optional entry. + +Just like a _chain_ expression that does not pipe output, the return object is a dictionary of WTOutput object keyed +by the host name from the _hosts_ list and *not* from the commands recieving the piped output. + +If any command fails, a WTChainException is raised. Import this exception to catch it: +```buildoutcfg +from watiba import WTChainException +``` + +_Note_: _The piping feature is experimental as of this release, and a better design will eventually +supercede it._ + +Examples: +``` +from watiba import WTChainException + +# This is a simple chain with no piping +try: + args = {"hosts": ["serverA", "serverB", "serverC"]} + out = chain `ls -lrt dir/` args + for host, output in out.items(): + print(f'{host} exit code: {output.exit_code}') +except WTChainException as ex: + print(f'ERROR: {ex.message}, {ex.host}, {ex.command}, {ex.output.stderr}') +``` +``` +# This is a more complex chain that runs the "ls -lrt" command on each server listed in "hosts" +# and pipes the STDOUT output from serverC to serverV and serverD, to those commands, and serverB's STDERR +# to serverX and its command +try: + args = {"hosts": ["serverA", "serverB", "serverC"], + "stdout": {"serverC":{"serverV": "grep something", "serverD":"grep somethingelse"}}, + "stderr": {"serverB":{"serverX": "cat >> /tmp/serverC.err"}} + } + out = chain `ls -lrt dir/` args + for host, output in out.items(): + print(f'{host} exit code: {output.exit_code}') +except WTChainException as ex: + print(f'ERROR: {ex.message}, {ex.host}, {ex.command}, {ex.output.stderr}') +``` + +####How does this work? +Watiba will run the backticked command in the expression on each host listed in _hosts_, in sequence and synchronously. +If there is a "stdout" found in the arguments, then it will name the source host as the key, i.e. the host from which +STDOUT will be read, and fed to each host and command listed under that host. This is true for STDERR as well. + +The method in which Watiba feeds the piped output is through a an _echo_ command shell piped to the command to be run +on that host. So, "stdout": {"serverC":{"serverV": "grep something"}} causes Watiba to read each line of STDOUT from +serverC and issue ```echo "$line" | grep something``` on serverV. It is piping from serverC to serverV. + +<div id="installation"/> + +## Installation +### PIP +If you installed this as a Python package, e.g. pip, then the pre-compiler, _watiba-c_, +will be placed in your system's PATH by PIP. + +### GITHUB +If you cloned this from github, you'll still need to install the package with pip, first, for the +watbia module. Follow these steps to install Watiba locally. +``` +# Watiba package required +python3 -m pip install watiba +``` + + +<div id="pre-compiling"/> + +## Pre-compiling +Test that the pre-compiler functions in your environment: +``` +watiba-c version +``` +For example: +```buildoutcfg +rwalk@walkubu:~$ watiba-c version +Watiba 0.3.26 +``` + +To pre-compile a .wt file: +``` +watiba-c my_file.wt > my_file.py +chmod +x my_file.py +./my_file.py +``` + +Where _my_file.wt_ is your Watiba code. + +<div id="code-examples"/> + +## Code Examples + +**my_file.wt** + +``` +#!/usr/bin/python3 + +# Stand alone commands. One with directory context, one without + +# This CWD will be active until a subsequent command changes it +`cd /tmp` + +# Simple statement utilizing command and results in one statement +print(`cd /tmp`.cwd) + +# This will not change the Watiba CWD context, because of the dash prefix, but within +# the command itself the cd is honored. file.txt is created in /home/user/blah but +# this does not impact the CWD of any subsequent commands. They +# are still operating from the previous cd command to /tmp +-`cd /home/user/blah && touch file.txt` + +# This will print "/tmp" _not_ /home because of the leading dash on the command +print(f"CWD is not /home: {-`cd /home`.cwd)}" + +# This will find text files in /tmp/, not /home/user/blah (CWD context!) +w=`find . -name '*.txt'` +for l in w.stdout: + print(f"File: {l}") + + +# Embedding commands in print expressions that will print the stderr output, which tar writes to +print(`echo "Some textual comment" > /tmp/blah.txt && tar -zcvf /tmp/blah.tar.gz /tmp`).stdout) + +# This will print the first line of stdout from the echo +print(`echo "hello!"`.stdout[0]) + +# Example of more than one command in a statement line +if len(`ls -lrt`.stdout) > 0 or len(-`cd /tmp`.stdout) > 0: + print("You have stdout or stderr messages") + + +# Example of a command as a Python varible and +# receiving a Watiba object +cmd = "tar -zcvf /tmp/watiba_test.tar.gz /mnt/data/git/watiba/src" +cmd_results = `$cmd` +if cmd_results.exit_code == 0: + for l in cmd_results.stderr: + print(l) + +# Simple reading of command output +# Iterate on the stdout property +for l in `cat blah.txt`.stdout: + print(l) + +# Example of a failed command to see its exit code +xc = `lsvv -lrt`.exit_code +print(f"Return code: {xc}") + +# Example of running a command asynchronously and resolving promise +spawn `cd /tmp && tar -zxvf tarball.tar.gz`: + for l in promise.output.stderr: + print(l) + return True # Mark promise resolved + + +# List dirs from CWD, iterate through them, spawn a tar command +# then within the resolver, spawn a move command +# Demonstrates spawns within resolvers +for dir in `ls -d *`.stdout: + tar = "tar -zcvf {}.tar.gz {}" + prom = spawn `$tar` {"dir": dir}: + print(f"{}args['dir'] tar complete") + mv = f"mv -r {args['dir']}/* /tmp/." + spawn `$mv`: + print("Move done") + # Resolve outer promise + promise.resolve_parent() + return True + # Do not resolve this promise yet. Let the inner resolver do it + return False + prom.join() +``` + + + + +%package -n python3-watiba +Summary: Python syntactical sugar for embedded shell commands +Provides: python-watiba +BuildRequires: python3-devel +BuildRequires: python3-setuptools +BuildRequires: python3-pip +%description -n python3-watiba +# Watiba +#### Version: **0.6.59** +#### Date: 2021/12/04 + +Watiba, pronounced wah-TEE-bah, is a lightweight Python pre-compiler for embedding Linux shell +commands within Python applications. It is similar to other languages' syntactical enhancements where +XML or HTML is integrated into a language such as JavaScript. That is the concept applied here but integrating +BASH shell commands with Python. + +As you browse this document, you'll find Watiba is rich with features for shell command integration with Python. + +Features: +- Shell command integration with Python code +- In-line access to shell command results +- Current directory context maintained across commands throughout your Python code +- Async/promise support for integrated shell commands +- Remote shell command execution +- Remote shell command chaining and piping + +## Table of Contents +1. [Usage](#usage) +2. [Directory Context](#directory-context) +3. [Commands as Variables](#commands-as-variables) +4. [Command Results](#command-results) +5. [Asynchronous Spawning and Promises](#async-spawing-and-promises) + 1. [Useful Properties in Promise](#useful-properties-in-promise) + 2. [Spawn Controller](#spawn-controller) + 3. [Join, Wait or Watch](#join-wait-watch) + 4. [The Promise Tree](#promise-tree) + 5. [Threads](#threads) +6. [Remote Execution](#remote-execution) + 1. [Change SSH port for remote execution](#change-ssh-port) +7. [Command Hooks](#command-hooks) +8. [Command Chaining](#command-chaining) +9. [Command Chain Piping (Experimental)](#piping-output) +10. [Installation](#installation) +11. [Pre-compiling](#pre-compiling) +12. [Code Examples](#code-examples) + +<div id="usage"/> + +## Usage +Watiba files, suffixed with ".wt", are Python programs containing embedded shell commands. +Shell commands are expressed within backtick characters emulating BASH's original capture syntax. +They can be placed in any Python statement or expression. Watiba keeps track of the current working directory +after the execution of any shell command so that all subsequent shell commands keep context. For example: + +Basic example of embedded commands: +``` +#!/usr/bin/python3 + +# Typical Python program + +if __name__ == "__main__": + + # Change directory context + `cd /tmp` + + # Directory context maintained + for file in `ls -lrt`.stdout: # In-line access to command results + print(f"File in /tmp: {file}") +``` + +This loop will display the file list from /tmp. The `ls -lrt` is run in the +context of previous `cd /tmp`. + +<div id="commands-as-variables"/> + +#### Commands Expressed as Variables +Commands within backticks can _be_ a variable, but cannot contain snippets of Python code or Python variables. +The statement within the backticks _must_ be either a pure shell command or a Python variable containing a pure +shell command. To execute commands in a Python variable, prefix the variable name between backticks with a dollar sign. + +_A command variable is denoted by prepending a dollar sign on the variable name within backticks_: +``` +# Set the Python variable to the command +cmdA = 'echo "This is a line of output" > /tmp/blah.txt' +cmdB = 'cat /tmp/blah.txt' + +# Execute first command +`$cmdA` # Execute the command within Python variable cmdA + +# Execute second command +for line in `$cmdB`.stdout: + print(line) +``` + +_This example demonstrates keeping dir context and executing a command by variable_: +``` +#!/usr/bin/python3 + +if __name__ == "__main__": + # Change CWD to /tmp + `cd /tmp` + + # Set a command string + my_cmd = "tar -zxvf tmp.tar.gz" + + # Execute that command and save the command results in variable "w" + w = `$my_cmd` + if w.exit_code == 0: + for l in w.stderr: + print(l) +``` + +_These constructs are **not** supported_: + ``` +file_name = "blah.txt" + +# Python variable within backticks +`touch file_name` # NOT SUPPORTED! + +# Attempting to access Python variable with dollar sign +`touch $file_name` # NOT SUPPORTED! + +# Python within backticks is NOT SUPPORTED! +`if x not in l: ls -lrt x` +``` +<div id="directory-context"/> + +## Directory Context + +An important Watiba usage point is directory context is kept for dispersed shell commands. +Any command that changes the shell's CWD is discovered and kept by Watiba. Watiba achieves +this by tagging a `&& echo pwd` to the user's command, locating the result in the command's STDOUT, +and finally setting the Python environment to that CWD with `os.chdir(dir)`. This is automatic and +opaque to the user. The user will not see the results of the generated suffix. If the `echo` +suffix presents a problem for the user, it can be eliminated by prefixing the leading backtick with a +dash. The dash turns off the context tracking by not suffixing the command and so causes Watiba to +lose its context. However, the context is maintained _within_ the set of commands in the backticks just not +when it returns. For example, **out = -\`cd /tmp && ls -lrt\`** honors the ```cd``` within the scope +of that execution line, but not for any backticked commands that follow later in your code. + +**_Warning!_** The dash will cause Watiba to lose its directory context should the command +cause a CWD change either explicitly or implicitly. + +_Example_: +``` +`cd /tmp` # Context will be kept + +# This will print from /home/user, but context is NOT kept +for line in -`cd /home/user && ls -lrt`.stdout: + print(line) + +# This will print from /tmp, not /home/user +for line in `ls -lrt`.stdout: + print(line) +``` + +<div id="command-results"/> + +## Command Results +The results of the command issued in backticks are available in the properties +of the object returned by Watiba. Following are those properties: + +<table> + <th>Property</th><th>Data Type</th><th>Description</th> + <tr></tr> + <td valign="top">stdout</td><td valign="top">List</td><td valign="top">STDOUT lines from the command normalized for display</td> + <tr></tr> + <td valign="top">stderr</td><td valign="top">List</td><td valign="top">STDERR lines from the command normalized for display</td> + <tr></tr> + <td valign="top">exit_code</td><td valign="top">Integer</td><td valign="top">Exit code value from command</td> + <tr></tr> + <td valign="top">cwd</td><td valign="top">String</td><td valign="top">Current working directory <i>after</i> command was executed</td> +</table> + +Technically, the returned object for any shell command is defined in the WTOutput class. + +<div id="async-spawing-and-promises"/> + +## Asynchronous Spawning and Promises +Shell commands can be executed asynchronously with a defined resolver callback block. Each _spawn_ expression creates +and runs a new OS thread. The resolver is a callback block that follows the Watiba _spawn_ expression. The spawn +feature is executed when a ```spawn `cmd` args: resolver block``` code block is encountered. The +resolver is passed the results in the promise object. (The promise structure contains the properties +defined in section ["Results from Spawned Commands"](#spawn-results) The _spawn_ expression also returns a _promise_ object +to the caller of _spawn_. The promise object is passed to the _resolver block_ in argument _promise_. The +outer code can check its state with a call to _resolved()_ on the *returned* promise object. Output from the command +is found in _promise.output_. The examples throughout this README and in the _examples.wt_ file make this clear. + +<div id="useful-properties-in-promise"/> + +##### Useful properties in promise structure +A promise is either returned in assignment from outermost spawn, or passed to child spawns in argument "promise". + + <table> + <th>Property</th> + <th>Data Type</th> + <th>Description</th> + <tr></tr> + <td valign="top">host</td><td valign="top">String</td><td valign="top">Host name on which spawned command ran</td> + <tr></tr> + <td valign="top">children</td><td valign="top">List</td><td valign="top">Children promises for this promise node</td> + <tr></tr> + <td valign="top">parent</td><td valign="top">Reference</td><td valign="top">Parent promise node of child promise. None if root promise.</td> + <tr></tr> + <td valign="top">command</td><td valign="top">String</td><td valign="top">Shell command issued for this promise</td> + <tr></tr> + <td valign="top">resolved()</td><td valign="top">Method</td><td valign="top">Call to find out if this promise is resolved</td> + <tr></tr> + <td valign="top">resolve_parent()</td><td valign="top">Method</td><td valign="top">Call inside resolver block to resolve parent promise</td> + <tr></tr> + <td valign="top">tree_dump()</td><td valign="top">Method</td><td valign="top">Call to show the promise tree. Takes subtree argument otherwise it defaults to the root promise</td> + <tr></tr> + <td valign="top">join()</td><td valign="top">Method</td><td valign="top">Call to wait on on promise and all its children</td> + <tr></tr> + <td valign="top">wait()</td><td valign="top">Method</td><td valign="top">Call to wait on just this promise</td> + <tr></tr> + <td valign="top">watch()</td><td valign="top">Method</td><td valign="top">Call to create watcher on this promise</td> + <tr></tr> + <td valign="top">start_time</td><td valign="top">Time</td><td valign="top">Time that spawned command started</td> + <tr></tr> + <td valign="top">end_time</td><td valign="top">Time</td><td valign="top">Time that promise resolved</td> + </table> + +_Example of simple spawn_: +```buildoutcfg +prom = spawn `tar -zcvf big_file.tar.gz some_dir/*`: + # Resolver block to which "promise" and "args" is passed... + print(f"{promise.command} completed.") + return True # Resolve promise + +# Do other things while tar is running +# Finally wait for tar promise to resolve +prom.join() +``` + +<div id="spawn-controller"/> + +#### Spawn Controller +All spawned threads are managed by Watiba's Spawn Controller. The controller watches for too many threads and +incrementally slows down each thread start when that threshold is exceeded until either all the promises in the tree +resolve, or an expiration count is reached, at which time an exception is thrown on the last spawned command. +This exception is raised by the default error method. This method as well as other spawn controlling parameters +can be overridden. The controller's purpose is to not allow run away threads and provide signaling of possible +hung threads. + +_spawn-ctl_ example: +```buildoutcfg +# Only allow 20 spawns max, +# and increase slowdown by 1/2 second each 3rd cycle +...python code... +spawn-ctl {"max":20, "sleep-increment":.500} +``` + +Spawn control parameters: + +<table> + <th>Key Name</th> + <th>Data Type</th> + <th>Description</th> + <th>Default</th> + <tr></tr> + <td valign="top">max</td><td valign="top">Integer</td><td valign="top">The maximum number of spawned commands allowed before the controller enters slowdown mode</td><td valign="top">10</td> + <tr></tr> + <td valign="top">sleep-floor</td><td valign="top">Integer</td><td valign="top">Seconds of <i>starting</i> +sleep value when the controller enters slowdown mode</td><td valign="top">.125 (start at 1/8th second pause)</td> + <tr></tr> + <td valign="top">sleep-increment</td><td valign="top">Integer</td><td valign="top">Seconds the <i>amount</i> of seconds sleep will increase every 3rd cycle when in slowdown + mode</td><td valign="top">.125 (Increase pause 1/8th second every 3rd cycle)</td> + <tr></tr> + <td valign="top">sleep-ceiling</td><td valign="top">Integer</td><td valign="top">Seconds the <i>highest</i> length sleep value allowed when in slowdown mode + (As slow as it will get)</td><td valign="top">3 (won't get slower than 3 second pauses)</td> + <tr></tr> + <td valign="top">expire</td><td valign="top">Integer</td><td valign="top">Total number of slowdown cycles allowed before the error method is called</td><td valign="top">No expiration</td> + <tr></tr> + <td valign="top">error</td><td valign="top">Method</td><td valign="top"> + Callback method invoked when slowdown mode expires. Use this to catch hung commands. + This method is passed 2 arguments: + +- **promise** - The promise attempting execution at the time of expiration +- **count** - The thread count (unresolved promises) at the time of expiration + </td><td valign="top">Generic error handler. Just throws <i>WTSpawnException</i> that hold properties <i>promise</i> and <i>message</i></td></td> +</table> + <hr> + +**_spawn-ctl_** only overrides the values it sets and does not affect values not specified. _spawn-ctl_ statements can +set whichever values it wants, can be dispersed throughout your code (i.e. multiple _spawn-ctl_ statements) and +only affects subsequent spawn expressions. + +_Notes:_ +1. Arguments can be passed to the resolver by specifying a trailing variable after the command. If the arguments +variable is omitted, an empty dictionary, i.e. {}, is passed to the resolver in _args_. +**_Warning!_** Python threading does not deep copy objects passed as arguments to threads. What you place in ```args``` +of the spawn expression will only be shallow copied so if there are references to other objects, it's not likely to + survive the copy. +2. The resolver must return _True_ to set the promise to resolved, or _False_ to leave it unresolved. +3. A resolver can also set the promise to resolved by calling ```promise.set_resolved()```. This is handy in cases where +a resolver has spawned another command and doesn't want the outer promise resolved until the inner resolvers are done. +To resolve an outer, i.e. parent, resolver issue _promise.resolve_parent()_. Then the parent resolver can return +_False_ at the end of its block so it leaves the resolved determination to the inner resolver block. +4. Each promise object holds its OS thread object in property _thread_ and its thread id in property _thread_id_. This +can be useful for controlling the thread directly. For example, to signal a kill. +5. _spawn-ctl_ has no affect on _join_, _wait_, or _watch_. This is because _spawn-ctl_ establishes an upper end +throttle on the overall spawning process. When the number of spawns hits the max value, throttling (i.e. slowdown + mode) takes affect and will expire if none of the promises resolve. Conversely, the arguments used by _join_, + _wait_ and _watch_ control the sleep cycle and expiration of just those calls, not the spawned threads as a whole. When + an expiration is set for, say, _join_, then that join will expire at that time. When an expiration is set in + _spawn-ctl_, then if all the spawned threads as a whole don't resolve in time then an expiration function is called. + + +**_Spawn Syntax:_** +``` +my_promise = spawn `cmd` [args]: + resolver block (promise, args) + args passed in args + return resolved or unresolved (True or False) + ``` + +_Spawn with resolver arguments omitted_: +``` +my_promise = spawn `cmd`: + resolver block (promise, args) + return resolved or unresolved (True or False) +``` + +_Simple spawn example_: +```buildoutcfg +p = spawn `tar -zcvf /tmp/file.tar.gz /home/user/dir`: + # Resolver block to which "promise" and "args" are passed + # Resolver block is called when spawned command has completed + for line in promise.output.stderr: + print(line) + + # This marks the promise resolved + return True + +# Wait for spawned command to resolve (not merely complete) +try: + p.join({"expire": 3}) + print("tar resolved") +except Exception as ex: + print(ex.args) +``` + +_Example of file that overrides spawn controller parameters_: +``` +#!/usr/bin/python3 +def spawn_expired(promise, count): + print("I do nothing just to demonstrate the error callback.") + print(f"This command failed {promise.command} at this threshold {count}") + + raise Exception("Too many threads.") + +if __name__ == "__main__": + # Example showing default values + parms = {"max": 10, # Max number of threads allowed before slowdown mode + "sleep-floor": .125, # Starting sleep value + "sleep-ceiling": 3, # Maximum sleep value + "sleep-increment": .125, # Incremental sleep value + "expire": -1, # Default: no expiration + "error": spawn_expired # Method called upon slowdown expiration + } + + # Set spawn controller parameter values + spawn-ctl parms +``` + +<div id="join-wait-watch"/> + +#### Join, Wait, or Watch + +Once commands are spawned, the caller can wait for _all_ promises, including inner or child promises, to complete, or +the caller can wait for just a specific promise to complete. To wait for all _child_ promises including +the promise on which you're calling this method, call _join()_. It will wait for that promise and all its children. To +wait for just one specific promise, call _wait()_ on the promise of interest. To wait for _all_ promises in +the promise tree, call _join()_ on the root promise. + +_join_ and _wait_ can be controlled through parameters. Each are iterators paused with a sleep method and will throw +an expiration exception should you set a limit for iterations. If an expiration value is not set, +no exception will be thrown and the cycle will run only until the promise(s) are resolved. _join_ and _wait_ are not +affected by _spawn-ctl_. + +_watch_ is called to establish a separate asynchronous thread that will call back a function of your choosing should +the command the promise is attached to time out. This is different than _join_ and _wait_ in that _watch_ is not synchronous +and does not pause. This is used to keep an eye on a spawned command and take action should it hang. Your watcher +function is passed the promise on which the watcher was attached, and the arguments, if any, from the spawn expression. +If your command does not time out (i.e. hangs and expires), the watcher thread will quietly go away when the promise +is resolved. _watch_ expiration is expressed in **seconds**, unlike _join_ and _wait_ which are expressed as total +_iterations_ paused at the sleep value. _watch_'s polling cycle pause is .250 seconds, so the expiration value is +multiplied by 4. The default expiration is 15 seconds. + +Examples: +``` +# Spawn a thread running this command +p = spawn `ls -lrt`: + ## resolver block ## + return True + +# Wait for promises, pause for 1/4 second each iteration, and throw an exception after 4 iterations +(1 second) +try: + p.join({"sleep": .250, "expire": 4}) +except Exception as ex: + print(ex.args) + +# Wait for this promise, pause for 1 second each iteration, and throw an exception after 5 iterations +(5 seconds) +try: + p.wait({"sleep": 1, "expire": 5}) +except Exception as ex: + print(ex.args) + +# My watcher function (called if spawned command never resolves by its experation period) +def watcher(promise, args): + print(f"This promise is likely hung: {promise.command}") + print(f"and I still have the spawn expression's args: {args}") + +p = spawn `echo "hello" && sleep 5` args: + print(f"Args passed to me: {args}") + return True + +# Attach a watcher to this thread. It will be called upon experation. +p.watch(watcher) +print("watch() does not pause like join or wait") + +# Attach a watcher that will expire in 5 seconds +p.watch(watcher, {"expire": 5}) +``` + +**_join_ syntax** +``` +promise.join({optional args}) +Where args is a Python dictionary with the following options: + "sleep" - seconds of sleep for each iteration (fractions such as .5 are honored) + default: .5 seconds + "expire" - number of sleep iterations until an excpetions is raised + default: no expiration +Note: "args" is optional and can be omitted +``` + +_Example of joining parent and children promises_: +``` +p = spawn `ls *.txt`: + for f in promise.output.stdout: + cmd = f"tar -zcvf {f}.tar.gz {f}" + spawn `$cmd` {"file":f}: + print(f"{f} completed") + promise.resolve_parent() + return True + return False + +# Wait for all commands to complete +try: + p.join({"sleep":1, "expire":20}) +except Exception as ex: + print(ex.args) +``` + +**_wait_ syntax** +``` +promise.wait({optional args}) +Where args is a Python dictionary with the following options: + "sleep" - seconds of sleep for each iteration (fractions such as .5 are honored) + default: .5 seconds + "expire" - number of sleep iterations until an excpetions is raised + default: no expiration +Note: "args" is optional and can be omitted +``` + +_Example of waiting on just the parent promise_: +``` +p = spawn `ls *.txt`: + for f in promise.output.stdout: + cmd = f"tar -zcvf {f}.tar.gz {}" + spawn `$cmd` {"file":f}: + print(f"{f} completed") + promise.resolve_parent() # Wait completes here + return True + return False + +# Wait for just the parent promise to complete +try: + p.wait({"sleep":1, "expire":20}) +except Exception as ex: + print(ex.args) +``` + +**_watch_ syntax** +``` +promise.watch(callback, {optional args}) +Where args is a Python dictionary with the following options: + "sleep" - seconds of sleep for each iteration (fractions such as .5 are honored) + default: .5 seconds + "expire" - number of sleep iterations until an excpetions is raised + default: no expiration +Note: "args" is optional and can be omitted +``` + +_Example of creating a watcher_: +```buildoutcfg +# Define watcher method. Called if command times out (i.e. expires) +def time_out(promise, args): + print(f"Command {promise.command} timed out.") + +# Spawn a thread running some command that hangs +p = spawn `long-running.sh`: + print("Finally completed. Watcher method won't be called.") + return True + +p.watch(time_out) # Does not wait. Calls method "time_out" if this promise expires (i.e. command hangs) + +# Do other things... + +``` + +<div id="promise-tree"/> + +#### The Promise Tree +Each _spawn_ issued inserts its promise object into the promise tree. The outermost _spawn_ will generate the root +promise and each inner _spawn_ will be among its children. There's no limit to how far it can nest. _wait_ only applies +to the promise on which it is called and is how it is different than _join_. _wait_ does not consider any other +promise state but the one it's called for, whereas _join_ considers the one it's called for **and** anything below it +in the tree. + +The promise tree can be printed with the ```dump_tree()``` method on the promise. This method is intended for +diagnostic purposes where it must be determined why spawned commands hung. ```dump_tree(subtree)``` accepts +a subtree promise as an argument. If no arguments are passed, ```dump_tree()``` dumps from the root promise on down. +``` +# Simple example with no child promises +p = spawn `date`: + return True + +p.tree_dump() # Dump tree from root +# or +p.tree_dump(subtree_promise) # Dump tree from node in argument +``` + +Example dumping tree from subtree node: +```buildoutcfg +# Complex example with child and grandchild promises +# Demonstrates how to dump the promise tree from various points within it +p = spawn `date`: + # Spawn child command (child promise) + spawn `pwd`: + # Spawn a grandchild to the parent promise + spawn `python --version`: + promise.tree_dump(promise) # Dump the subtree from this point down + return False + # Spawn another child + spawn `echo "blah"`: + # Resolve parent promise + promise.resolve_parent() + # Resolve child promise + return True + # Do NOT resolve parent promise, let child do that + return False + +p.join() +p.tree_dump(p.children[0]) # Dump subtree from first child on down +p.tree_dump(p.children[1]) # Dump subtree from the second child +p.tree_dump(p.children[0].children[0]) # Dump subtree from the grandchild + +# Dump all children +for c in p.children: + p.tree_dump(c) +``` + +_Parent and child joins shown in these two examples_: + +``` +root_promise = spawn `ls -lr`: + for file in promise.stdout: + t = f"touch {file}" + spawn `$t` {"file" file}: # This promise is a child of root + print(f"{file} updated".) + spawn `echo "done" > /tmp/done"`: # Another child promise (root's grandchild) + print("Complete") + promise.resolve_parent() + return True + promise.resolve_parent() + return False + return False + +root_promise.join() # Wait on the root promise and all its children. Thus, waiting for everything. +``` + +``` +root_promise = spawn `ls -lr`: + for file in promise.output.stdout: + t = f"touch {file}" + spawn `$t` {"file" file}: # This promise is a child of root + print(f"{promise.args['file'])} updated") + promise.join() # Wait for this promise and its children but not its parent (root) + spawn `echo "done" > /tmp/done"`: + print("Complete") +``` + + + +_Resolving a parent promise_: +``` +p = spawn `ls -lrt`: + for f in promise.output.stdout: + cmd = f"touch {f}" + # Spawn command from this resolver and pass our promise + spawn `$cmd`: + print("Resolving all promises") + promise.resolve_parent() # Resolve parent promise here + return True # Resolve child promise + return False # Do NOT resolve parent promise here +p.join() # Wait for ALL promises to be resolved +``` + +<div id="spawn-results"/> + +### Results from Spawned Commands +Spawned commands return their results in the _promise.output_ property of the _promise_ object passed to +the resolver block, and in the spawn expression if there is an assignment in that spawn expression. + +The result properties can then be accessed as followed: + +<table> + <th>Property</th><th>Data Type</th><th>Description</th> + <tr></tr> + <td valign="top">promise.output.stdout</td><td valign="top">List</td><td valign="top">STDOUT lines from the command normalized for display</td> + <tr></tr> + <td valign="top">promise.output.stderr</td><td valign="top">List</td><td valign="top">STDERR lines from the command normalized for display</td> + <tr></tr> + <td valign="top">promise.output.exit_code</td><td valign="top">Integer</td><td valign="top">Exit code value from command</td> + <tr></tr> + <td valign="top">promise.output.cwd</td><td valign="top">String</td><td valign="top">Current working directory <i>after</i> command was executed</td> +</table> + + +_Notes:_ +1. Watiba backticked commands can exist within the resolver +2. Other _spawn_ blocks can be embedded within a resolver (recursion allowed) +3. The command within the _spawn_ definition can be a variable + (The same rules apply as for all backticked shell commands. This means the variable must contain + pure shell commands.) +4. The leading dash to ignore CWD _cannot_ be used in the _spawn_ expression +5. The _promise.output_ object is not available until _promise.resolved()_ returns True + +_Simple example with the shell command as a Python variable_: +``` +#!/usr/bin/python3 + +# run "date" command asynchronously +d = 'date "+%Y/%m/%d"' +spawn `$d`: + print(promise.output.stdout[0]) + return True + +``` + +_Example with shell commands executed within resolver block_: +``` +#!/usr/bin/python3 + +print("Running Watiba spawn with wait") +`rm /tmp/done` + +# run "ls -lrt" command asynchronously +p = spawn `ls -lrt`: + print(f"Exit code: {promise.output.exit_code}") + print(f"CWD: {promise.output.cwd}") + print(f"STDERR: {promise.output.stderr}") + + # Loop through STDOUT from command + for l in promise.output.stdout: + print(l) + `echo "Done" > /tmp/done` + + # Resolve promise + return True + +# Pause until spawn command is complete +p.wait() +print("complete") + +``` + +<div id="threads"/> + +### Threads +Each promise produced from a _spawn_ expression results in one OS thread. To access the +number of threads your code has spawned collectively, you can do the following: +``` +num_of_spawns = promise.spawn_count() # Returns number of nodes in the promise tree +num_of_resolved_promises = promise.resolved_count() # Returns the number of promises resolved in tree +``` +<div id="remote-execution"/> + +## Remote Execution +Shell commands can be executed remotely. This is achieved though the SSH command, issued by Watiba, and has the +following requirements: +- OpenSSH is installed on the local and remote hosts +- The local SSH key is in the remote's _authorized_keys_ file. _The details of this + process is beyond the scope of this README. For those instructions, consult www.ssh.com_ + +- Make sure that SSH'ing to the target host does not cause any prompts. + +Test that your SSH environment is setup first by manually entering: +``` +ssh {user}@{host} "ls -lrt" + +# For example +ssh rwalk@walkubu "ls -lrt" + +# If SSH prompts you, then Watiba remote execution cannot function. +``` + +To execute a command remotely, a _@host_ parameter is suffixed to the backticked command. The host name can be a +literal or a variable. To employ a variable, prepend a _$_ to the name following _@_ such as _@$var_. + +<div id="change-ssh-port"/> + +#### Change SSH port for remote execution +To change the default SSH port 22 to a custom value, add to your Watiba code: ```watiba-ctl {"ssh-port": custom port}``` +Example: +```buildoutcfg +watiba-ctl {"ssh-port": 2233} +``` +Examples: +```buildoutcfg +p = spawn `ls -lrt`@remoteserver {parms}: + for line in promise.output.stdout: + print(line) + return True + +``` +```buildoutcfg +remotename = "serverB" +p = spawn `ls -lrt`@$remotename {parms}: + for line in p.output.stdout: + print(line) + return True +``` +```buildoutcfg +out = `ls -lrt`@remoteserver +for line in out.stdout: + print(line) +``` +```buildoutcfg +remotename = "serverB" +out = `ls -lrt`@$remotename +for line in out.stdout: + print(line) +``` + + +<div id="command-hooks"/> + +## Command Hooks +Hooks are pre- or -post functions that are attached to a _command_ _pattern_, which is a regular expression (regex). Anytime Watiba encounters a command +that matches the pattern for the hook, the hook function is called. + +All commands, spawned, remote, or local, can have Python functions executed **before** exection, by default, or **post hooks** that are run **after** the command. (Note: Post hooks are not run for spwaned commands because the resolver function is a post hook itself.) These functions can be passed arguments, too. + +### Command Hook Expressions +``` +# Run before commands that match that pattern +hook-cmd "pattern" hook-function parms + +# Run before commands that match that pattern, but is non-recursive +hook-cmd-nr "pattern" hook-function parms + +# Run after commands that match that pattern +post-hook-cmd "pattern" hook-function parms + +# Run after commands that match that pattern, but is non-recursive +post-hook-cmd-nr "pattern" hook-function parms +``` + +### Hook Recursion +Hooks, which are nothing more than Python functions called before or after a command is run, can issue their own commands and, thus, cause the hook +to be recursively called. However, if the command in the hook block of code matches a command pattern that causes that same hook function to be run again, +an infinte loop can occur. To prevent that, use the **-nr** suffix on the Watiba hook expression. (-nr stands for non-recursive.) This will ensure that +the hook cannot be re-invoked for any commands that are within it. + +<br> +To attach a hook: +1. Code one or more Python functions that will be the hooks. At the end of each hook, you must return True if the hook was successful, or False +if something wrong. +2. Use the _hook-cmd_ expression to attach those hooks to a command +pattern, which is a regular expression +3. To remove the hooks, use the _remove-hooks "pattern"_ expression. If a pattern, i.e. command regex pattern, is omitted, then all command hooks are removed. + +**hook-cmd "command pattern" function parms** + +The first parameter always passed to the hook function is the Python _match_ object from the command match. This is provided so the hook has access +to the tokens on the command should it need them. + +Example: +``` +def my_hook(match, parms): + print(match.groups()) + print(f'Tar file name is {match.group(1)}') + print(parms["parmA"]) + print(parms["parmB"]) + return True # Successful execution + +def your_hook(match, parms): + # This hook doesn't need the match object, so ignores it + print(parms["something"]) + if parms["something-else"] != "blah": + return False # Failed execution + return True # Successful excution + + +# Add first hook to my tar command +hook-cmd "tar -zcvf (\S.*)" my_hook: {"parmA":"A", "parmB":"B"} + +# Add another hook to my tar command +hook-cmd "tar -zcvf (\S.*)" your_hook: {"parmD":1, "parmE":"something"} + +# Spawn command, but hooks will be invoked first... +spawn `tar -zcvf files.tar.gz /tmp/files/* `: + # Resolver code block + return True # Resolve promise +``` + +Your parameters are whatever is valid for Python. These are simply passed to their attached functions, essentially each one's key is the function name, as specified. + + +_Where are the hooks run for spawned commands?_ All hooks run under the thread of the issuer on the local host, not the target thread. + +_Where are the hooks run for remote commands?_ As with spawned commands, all hooks are issued on the local host, not the remote. Note that you +can have remote backticked commands in your hook and that will run those remotely. If your remote command matches a hook(s) pattern, then those hooks will be run. This means if your command pattern for the first remote call runs a hook that contains another remote command that matches that same command pattern, then the hook is run again. Since this can lead to infinte hook loops, Watiba offers a non-recursive definition for the command pattern. Note that this non-recursive setting +only applies to the command pattern and not the hook function itself. So if _hookA_ is run for two different command patterns, say, "ls -lrt" and "ls -laF" you can +make one non-recusrive and still run the same hook for both commands. For the recursive command pattern, the hook has no limit to its recursion. For non-recursive, +it will only be called once during the recursion process. + +To set a command pattern as non-recursive, use _hook-cmd-nr_. + +Example using a variation on a previous example: + +``` +def my_hook(match, parms) + `tar -zcvf /tmp/files` # my_hook will NOT because for this command even though it matches + print("Will be called only once!") + return True + +# Note the "-nr" on the expression. That's for non-recursive +hook-cmd-nr "tar -zcvf (\S.*)" my_hook: {"parmA":"A", "parmB":"B"} + +# my_hook will be called before this command runs +` tar -zcvf tarball.tar.gz /home/user/files.*` +``` + +<div id="command-chaining"/> + +## Command Chaining +Watiba extends its remote command execution to chaining commands across multiple remote hosts. This is achieved +by the _chain_ expression. This expression will execute the backticked command across a list of hosts, passed by +the user, sequentially, synchronously until the hosts list is exhausted, or the command fails. _chain_ returns a +Python dictionary where the keys are the host names and the values the WTOutput from the command run on that host. + +#### Chain Exception +The _chain_ expression raises a WTChainException on the first failed command. The exception raised +has the following properties: + +_WTChainException_: +<table> +<th>Property</th><th>Data Type</th><th>Description</th> +<tr></tr> +<td valign="top">command</td><td valign="top">String</td><td valign="top">Command that failed</td> +<tr></tr> +<td valign="top">host</td><td valign="top">String</td><td valign="top">Host where command failed</td> +<tr></tr> +<td valign="top">message</td><td valign="top">String</td><td valign="top">Error message</td> +<tr></tr> +<td valign="top">output</td><td valign="top">WTOutput structure: + +- stdout +- stderr +- exit_code +- cwd</td><td valign="top">Output from command</td> +</table> + +Import this exception to catch it: +```buildoutcfg +from watiba import WTChainException +``` + + +Examples: +``` +from watiba import WTChainException + +try: + out = chain `tar -zcvf backup/file.tar.gz dir/*` {"hosts", ["serverA", "serverB"]} + for host,output in out.items(): + print(f'{host} exit code: {output.exit_code}') + for line in output.stderr: + print(line) + except WTChainException(ex): + print(f"Error: {ex.message}") + print(f" host: {ex.host} exit code: {ex.output.exit_code} command: {ex.command}) + +``` + +<div id="piping-output"/> + +## Command Chain Piping (Experimental) +The _chain_ expression supports piping STDOUT and/or STDERR to other commands executed on remote servers. Complex +arrangements can be constructed through the Python dictionary passed to the _chain_ expression. The dictionary +contents function as follows: +- "hosts": [server, server, ...] This entry instructions _chain_ on which hosts the backticked command will run. + This is a required entry. + +- "stdout": {server:command, server:command, ...} + This is an optional entry. + +- "stderr": {server:command, server:command, ...} + This is an optional entry. + +Just like a _chain_ expression that does not pipe output, the return object is a dictionary of WTOutput object keyed +by the host name from the _hosts_ list and *not* from the commands recieving the piped output. + +If any command fails, a WTChainException is raised. Import this exception to catch it: +```buildoutcfg +from watiba import WTChainException +``` + +_Note_: _The piping feature is experimental as of this release, and a better design will eventually +supercede it._ + +Examples: +``` +from watiba import WTChainException + +# This is a simple chain with no piping +try: + args = {"hosts": ["serverA", "serverB", "serverC"]} + out = chain `ls -lrt dir/` args + for host, output in out.items(): + print(f'{host} exit code: {output.exit_code}') +except WTChainException as ex: + print(f'ERROR: {ex.message}, {ex.host}, {ex.command}, {ex.output.stderr}') +``` +``` +# This is a more complex chain that runs the "ls -lrt" command on each server listed in "hosts" +# and pipes the STDOUT output from serverC to serverV and serverD, to those commands, and serverB's STDERR +# to serverX and its command +try: + args = {"hosts": ["serverA", "serverB", "serverC"], + "stdout": {"serverC":{"serverV": "grep something", "serverD":"grep somethingelse"}}, + "stderr": {"serverB":{"serverX": "cat >> /tmp/serverC.err"}} + } + out = chain `ls -lrt dir/` args + for host, output in out.items(): + print(f'{host} exit code: {output.exit_code}') +except WTChainException as ex: + print(f'ERROR: {ex.message}, {ex.host}, {ex.command}, {ex.output.stderr}') +``` + +####How does this work? +Watiba will run the backticked command in the expression on each host listed in _hosts_, in sequence and synchronously. +If there is a "stdout" found in the arguments, then it will name the source host as the key, i.e. the host from which +STDOUT will be read, and fed to each host and command listed under that host. This is true for STDERR as well. + +The method in which Watiba feeds the piped output is through a an _echo_ command shell piped to the command to be run +on that host. So, "stdout": {"serverC":{"serverV": "grep something"}} causes Watiba to read each line of STDOUT from +serverC and issue ```echo "$line" | grep something``` on serverV. It is piping from serverC to serverV. + +<div id="installation"/> + +## Installation +### PIP +If you installed this as a Python package, e.g. pip, then the pre-compiler, _watiba-c_, +will be placed in your system's PATH by PIP. + +### GITHUB +If you cloned this from github, you'll still need to install the package with pip, first, for the +watbia module. Follow these steps to install Watiba locally. +``` +# Watiba package required +python3 -m pip install watiba +``` + + +<div id="pre-compiling"/> + +## Pre-compiling +Test that the pre-compiler functions in your environment: +``` +watiba-c version +``` +For example: +```buildoutcfg +rwalk@walkubu:~$ watiba-c version +Watiba 0.3.26 +``` + +To pre-compile a .wt file: +``` +watiba-c my_file.wt > my_file.py +chmod +x my_file.py +./my_file.py +``` + +Where _my_file.wt_ is your Watiba code. + +<div id="code-examples"/> + +## Code Examples + +**my_file.wt** + +``` +#!/usr/bin/python3 + +# Stand alone commands. One with directory context, one without + +# This CWD will be active until a subsequent command changes it +`cd /tmp` + +# Simple statement utilizing command and results in one statement +print(`cd /tmp`.cwd) + +# This will not change the Watiba CWD context, because of the dash prefix, but within +# the command itself the cd is honored. file.txt is created in /home/user/blah but +# this does not impact the CWD of any subsequent commands. They +# are still operating from the previous cd command to /tmp +-`cd /home/user/blah && touch file.txt` + +# This will print "/tmp" _not_ /home because of the leading dash on the command +print(f"CWD is not /home: {-`cd /home`.cwd)}" + +# This will find text files in /tmp/, not /home/user/blah (CWD context!) +w=`find . -name '*.txt'` +for l in w.stdout: + print(f"File: {l}") + + +# Embedding commands in print expressions that will print the stderr output, which tar writes to +print(`echo "Some textual comment" > /tmp/blah.txt && tar -zcvf /tmp/blah.tar.gz /tmp`).stdout) + +# This will print the first line of stdout from the echo +print(`echo "hello!"`.stdout[0]) + +# Example of more than one command in a statement line +if len(`ls -lrt`.stdout) > 0 or len(-`cd /tmp`.stdout) > 0: + print("You have stdout or stderr messages") + + +# Example of a command as a Python varible and +# receiving a Watiba object +cmd = "tar -zcvf /tmp/watiba_test.tar.gz /mnt/data/git/watiba/src" +cmd_results = `$cmd` +if cmd_results.exit_code == 0: + for l in cmd_results.stderr: + print(l) + +# Simple reading of command output +# Iterate on the stdout property +for l in `cat blah.txt`.stdout: + print(l) + +# Example of a failed command to see its exit code +xc = `lsvv -lrt`.exit_code +print(f"Return code: {xc}") + +# Example of running a command asynchronously and resolving promise +spawn `cd /tmp && tar -zxvf tarball.tar.gz`: + for l in promise.output.stderr: + print(l) + return True # Mark promise resolved + + +# List dirs from CWD, iterate through them, spawn a tar command +# then within the resolver, spawn a move command +# Demonstrates spawns within resolvers +for dir in `ls -d *`.stdout: + tar = "tar -zcvf {}.tar.gz {}" + prom = spawn `$tar` {"dir": dir}: + print(f"{}args['dir'] tar complete") + mv = f"mv -r {args['dir']}/* /tmp/." + spawn `$mv`: + print("Move done") + # Resolve outer promise + promise.resolve_parent() + return True + # Do not resolve this promise yet. Let the inner resolver do it + return False + prom.join() +``` + + + + +%package help +Summary: Development documents and examples for watiba +Provides: python3-watiba-doc +%description help +# Watiba +#### Version: **0.6.59** +#### Date: 2021/12/04 + +Watiba, pronounced wah-TEE-bah, is a lightweight Python pre-compiler for embedding Linux shell +commands within Python applications. It is similar to other languages' syntactical enhancements where +XML or HTML is integrated into a language such as JavaScript. That is the concept applied here but integrating +BASH shell commands with Python. + +As you browse this document, you'll find Watiba is rich with features for shell command integration with Python. + +Features: +- Shell command integration with Python code +- In-line access to shell command results +- Current directory context maintained across commands throughout your Python code +- Async/promise support for integrated shell commands +- Remote shell command execution +- Remote shell command chaining and piping + +## Table of Contents +1. [Usage](#usage) +2. [Directory Context](#directory-context) +3. [Commands as Variables](#commands-as-variables) +4. [Command Results](#command-results) +5. [Asynchronous Spawning and Promises](#async-spawing-and-promises) + 1. [Useful Properties in Promise](#useful-properties-in-promise) + 2. [Spawn Controller](#spawn-controller) + 3. [Join, Wait or Watch](#join-wait-watch) + 4. [The Promise Tree](#promise-tree) + 5. [Threads](#threads) +6. [Remote Execution](#remote-execution) + 1. [Change SSH port for remote execution](#change-ssh-port) +7. [Command Hooks](#command-hooks) +8. [Command Chaining](#command-chaining) +9. [Command Chain Piping (Experimental)](#piping-output) +10. [Installation](#installation) +11. [Pre-compiling](#pre-compiling) +12. [Code Examples](#code-examples) + +<div id="usage"/> + +## Usage +Watiba files, suffixed with ".wt", are Python programs containing embedded shell commands. +Shell commands are expressed within backtick characters emulating BASH's original capture syntax. +They can be placed in any Python statement or expression. Watiba keeps track of the current working directory +after the execution of any shell command so that all subsequent shell commands keep context. For example: + +Basic example of embedded commands: +``` +#!/usr/bin/python3 + +# Typical Python program + +if __name__ == "__main__": + + # Change directory context + `cd /tmp` + + # Directory context maintained + for file in `ls -lrt`.stdout: # In-line access to command results + print(f"File in /tmp: {file}") +``` + +This loop will display the file list from /tmp. The `ls -lrt` is run in the +context of previous `cd /tmp`. + +<div id="commands-as-variables"/> + +#### Commands Expressed as Variables +Commands within backticks can _be_ a variable, but cannot contain snippets of Python code or Python variables. +The statement within the backticks _must_ be either a pure shell command or a Python variable containing a pure +shell command. To execute commands in a Python variable, prefix the variable name between backticks with a dollar sign. + +_A command variable is denoted by prepending a dollar sign on the variable name within backticks_: +``` +# Set the Python variable to the command +cmdA = 'echo "This is a line of output" > /tmp/blah.txt' +cmdB = 'cat /tmp/blah.txt' + +# Execute first command +`$cmdA` # Execute the command within Python variable cmdA + +# Execute second command +for line in `$cmdB`.stdout: + print(line) +``` + +_This example demonstrates keeping dir context and executing a command by variable_: +``` +#!/usr/bin/python3 + +if __name__ == "__main__": + # Change CWD to /tmp + `cd /tmp` + + # Set a command string + my_cmd = "tar -zxvf tmp.tar.gz" + + # Execute that command and save the command results in variable "w" + w = `$my_cmd` + if w.exit_code == 0: + for l in w.stderr: + print(l) +``` + +_These constructs are **not** supported_: + ``` +file_name = "blah.txt" + +# Python variable within backticks +`touch file_name` # NOT SUPPORTED! + +# Attempting to access Python variable with dollar sign +`touch $file_name` # NOT SUPPORTED! + +# Python within backticks is NOT SUPPORTED! +`if x not in l: ls -lrt x` +``` +<div id="directory-context"/> + +## Directory Context + +An important Watiba usage point is directory context is kept for dispersed shell commands. +Any command that changes the shell's CWD is discovered and kept by Watiba. Watiba achieves +this by tagging a `&& echo pwd` to the user's command, locating the result in the command's STDOUT, +and finally setting the Python environment to that CWD with `os.chdir(dir)`. This is automatic and +opaque to the user. The user will not see the results of the generated suffix. If the `echo` +suffix presents a problem for the user, it can be eliminated by prefixing the leading backtick with a +dash. The dash turns off the context tracking by not suffixing the command and so causes Watiba to +lose its context. However, the context is maintained _within_ the set of commands in the backticks just not +when it returns. For example, **out = -\`cd /tmp && ls -lrt\`** honors the ```cd``` within the scope +of that execution line, but not for any backticked commands that follow later in your code. + +**_Warning!_** The dash will cause Watiba to lose its directory context should the command +cause a CWD change either explicitly or implicitly. + +_Example_: +``` +`cd /tmp` # Context will be kept + +# This will print from /home/user, but context is NOT kept +for line in -`cd /home/user && ls -lrt`.stdout: + print(line) + +# This will print from /tmp, not /home/user +for line in `ls -lrt`.stdout: + print(line) +``` + +<div id="command-results"/> + +## Command Results +The results of the command issued in backticks are available in the properties +of the object returned by Watiba. Following are those properties: + +<table> + <th>Property</th><th>Data Type</th><th>Description</th> + <tr></tr> + <td valign="top">stdout</td><td valign="top">List</td><td valign="top">STDOUT lines from the command normalized for display</td> + <tr></tr> + <td valign="top">stderr</td><td valign="top">List</td><td valign="top">STDERR lines from the command normalized for display</td> + <tr></tr> + <td valign="top">exit_code</td><td valign="top">Integer</td><td valign="top">Exit code value from command</td> + <tr></tr> + <td valign="top">cwd</td><td valign="top">String</td><td valign="top">Current working directory <i>after</i> command was executed</td> +</table> + +Technically, the returned object for any shell command is defined in the WTOutput class. + +<div id="async-spawing-and-promises"/> + +## Asynchronous Spawning and Promises +Shell commands can be executed asynchronously with a defined resolver callback block. Each _spawn_ expression creates +and runs a new OS thread. The resolver is a callback block that follows the Watiba _spawn_ expression. The spawn +feature is executed when a ```spawn `cmd` args: resolver block``` code block is encountered. The +resolver is passed the results in the promise object. (The promise structure contains the properties +defined in section ["Results from Spawned Commands"](#spawn-results) The _spawn_ expression also returns a _promise_ object +to the caller of _spawn_. The promise object is passed to the _resolver block_ in argument _promise_. The +outer code can check its state with a call to _resolved()_ on the *returned* promise object. Output from the command +is found in _promise.output_. The examples throughout this README and in the _examples.wt_ file make this clear. + +<div id="useful-properties-in-promise"/> + +##### Useful properties in promise structure +A promise is either returned in assignment from outermost spawn, or passed to child spawns in argument "promise". + + <table> + <th>Property</th> + <th>Data Type</th> + <th>Description</th> + <tr></tr> + <td valign="top">host</td><td valign="top">String</td><td valign="top">Host name on which spawned command ran</td> + <tr></tr> + <td valign="top">children</td><td valign="top">List</td><td valign="top">Children promises for this promise node</td> + <tr></tr> + <td valign="top">parent</td><td valign="top">Reference</td><td valign="top">Parent promise node of child promise. None if root promise.</td> + <tr></tr> + <td valign="top">command</td><td valign="top">String</td><td valign="top">Shell command issued for this promise</td> + <tr></tr> + <td valign="top">resolved()</td><td valign="top">Method</td><td valign="top">Call to find out if this promise is resolved</td> + <tr></tr> + <td valign="top">resolve_parent()</td><td valign="top">Method</td><td valign="top">Call inside resolver block to resolve parent promise</td> + <tr></tr> + <td valign="top">tree_dump()</td><td valign="top">Method</td><td valign="top">Call to show the promise tree. Takes subtree argument otherwise it defaults to the root promise</td> + <tr></tr> + <td valign="top">join()</td><td valign="top">Method</td><td valign="top">Call to wait on on promise and all its children</td> + <tr></tr> + <td valign="top">wait()</td><td valign="top">Method</td><td valign="top">Call to wait on just this promise</td> + <tr></tr> + <td valign="top">watch()</td><td valign="top">Method</td><td valign="top">Call to create watcher on this promise</td> + <tr></tr> + <td valign="top">start_time</td><td valign="top">Time</td><td valign="top">Time that spawned command started</td> + <tr></tr> + <td valign="top">end_time</td><td valign="top">Time</td><td valign="top">Time that promise resolved</td> + </table> + +_Example of simple spawn_: +```buildoutcfg +prom = spawn `tar -zcvf big_file.tar.gz some_dir/*`: + # Resolver block to which "promise" and "args" is passed... + print(f"{promise.command} completed.") + return True # Resolve promise + +# Do other things while tar is running +# Finally wait for tar promise to resolve +prom.join() +``` + +<div id="spawn-controller"/> + +#### Spawn Controller +All spawned threads are managed by Watiba's Spawn Controller. The controller watches for too many threads and +incrementally slows down each thread start when that threshold is exceeded until either all the promises in the tree +resolve, or an expiration count is reached, at which time an exception is thrown on the last spawned command. +This exception is raised by the default error method. This method as well as other spawn controlling parameters +can be overridden. The controller's purpose is to not allow run away threads and provide signaling of possible +hung threads. + +_spawn-ctl_ example: +```buildoutcfg +# Only allow 20 spawns max, +# and increase slowdown by 1/2 second each 3rd cycle +...python code... +spawn-ctl {"max":20, "sleep-increment":.500} +``` + +Spawn control parameters: + +<table> + <th>Key Name</th> + <th>Data Type</th> + <th>Description</th> + <th>Default</th> + <tr></tr> + <td valign="top">max</td><td valign="top">Integer</td><td valign="top">The maximum number of spawned commands allowed before the controller enters slowdown mode</td><td valign="top">10</td> + <tr></tr> + <td valign="top">sleep-floor</td><td valign="top">Integer</td><td valign="top">Seconds of <i>starting</i> +sleep value when the controller enters slowdown mode</td><td valign="top">.125 (start at 1/8th second pause)</td> + <tr></tr> + <td valign="top">sleep-increment</td><td valign="top">Integer</td><td valign="top">Seconds the <i>amount</i> of seconds sleep will increase every 3rd cycle when in slowdown + mode</td><td valign="top">.125 (Increase pause 1/8th second every 3rd cycle)</td> + <tr></tr> + <td valign="top">sleep-ceiling</td><td valign="top">Integer</td><td valign="top">Seconds the <i>highest</i> length sleep value allowed when in slowdown mode + (As slow as it will get)</td><td valign="top">3 (won't get slower than 3 second pauses)</td> + <tr></tr> + <td valign="top">expire</td><td valign="top">Integer</td><td valign="top">Total number of slowdown cycles allowed before the error method is called</td><td valign="top">No expiration</td> + <tr></tr> + <td valign="top">error</td><td valign="top">Method</td><td valign="top"> + Callback method invoked when slowdown mode expires. Use this to catch hung commands. + This method is passed 2 arguments: + +- **promise** - The promise attempting execution at the time of expiration +- **count** - The thread count (unresolved promises) at the time of expiration + </td><td valign="top">Generic error handler. Just throws <i>WTSpawnException</i> that hold properties <i>promise</i> and <i>message</i></td></td> +</table> + <hr> + +**_spawn-ctl_** only overrides the values it sets and does not affect values not specified. _spawn-ctl_ statements can +set whichever values it wants, can be dispersed throughout your code (i.e. multiple _spawn-ctl_ statements) and +only affects subsequent spawn expressions. + +_Notes:_ +1. Arguments can be passed to the resolver by specifying a trailing variable after the command. If the arguments +variable is omitted, an empty dictionary, i.e. {}, is passed to the resolver in _args_. +**_Warning!_** Python threading does not deep copy objects passed as arguments to threads. What you place in ```args``` +of the spawn expression will only be shallow copied so if there are references to other objects, it's not likely to + survive the copy. +2. The resolver must return _True_ to set the promise to resolved, or _False_ to leave it unresolved. +3. A resolver can also set the promise to resolved by calling ```promise.set_resolved()```. This is handy in cases where +a resolver has spawned another command and doesn't want the outer promise resolved until the inner resolvers are done. +To resolve an outer, i.e. parent, resolver issue _promise.resolve_parent()_. Then the parent resolver can return +_False_ at the end of its block so it leaves the resolved determination to the inner resolver block. +4. Each promise object holds its OS thread object in property _thread_ and its thread id in property _thread_id_. This +can be useful for controlling the thread directly. For example, to signal a kill. +5. _spawn-ctl_ has no affect on _join_, _wait_, or _watch_. This is because _spawn-ctl_ establishes an upper end +throttle on the overall spawning process. When the number of spawns hits the max value, throttling (i.e. slowdown + mode) takes affect and will expire if none of the promises resolve. Conversely, the arguments used by _join_, + _wait_ and _watch_ control the sleep cycle and expiration of just those calls, not the spawned threads as a whole. When + an expiration is set for, say, _join_, then that join will expire at that time. When an expiration is set in + _spawn-ctl_, then if all the spawned threads as a whole don't resolve in time then an expiration function is called. + + +**_Spawn Syntax:_** +``` +my_promise = spawn `cmd` [args]: + resolver block (promise, args) + args passed in args + return resolved or unresolved (True or False) + ``` + +_Spawn with resolver arguments omitted_: +``` +my_promise = spawn `cmd`: + resolver block (promise, args) + return resolved or unresolved (True or False) +``` + +_Simple spawn example_: +```buildoutcfg +p = spawn `tar -zcvf /tmp/file.tar.gz /home/user/dir`: + # Resolver block to which "promise" and "args" are passed + # Resolver block is called when spawned command has completed + for line in promise.output.stderr: + print(line) + + # This marks the promise resolved + return True + +# Wait for spawned command to resolve (not merely complete) +try: + p.join({"expire": 3}) + print("tar resolved") +except Exception as ex: + print(ex.args) +``` + +_Example of file that overrides spawn controller parameters_: +``` +#!/usr/bin/python3 +def spawn_expired(promise, count): + print("I do nothing just to demonstrate the error callback.") + print(f"This command failed {promise.command} at this threshold {count}") + + raise Exception("Too many threads.") + +if __name__ == "__main__": + # Example showing default values + parms = {"max": 10, # Max number of threads allowed before slowdown mode + "sleep-floor": .125, # Starting sleep value + "sleep-ceiling": 3, # Maximum sleep value + "sleep-increment": .125, # Incremental sleep value + "expire": -1, # Default: no expiration + "error": spawn_expired # Method called upon slowdown expiration + } + + # Set spawn controller parameter values + spawn-ctl parms +``` + +<div id="join-wait-watch"/> + +#### Join, Wait, or Watch + +Once commands are spawned, the caller can wait for _all_ promises, including inner or child promises, to complete, or +the caller can wait for just a specific promise to complete. To wait for all _child_ promises including +the promise on which you're calling this method, call _join()_. It will wait for that promise and all its children. To +wait for just one specific promise, call _wait()_ on the promise of interest. To wait for _all_ promises in +the promise tree, call _join()_ on the root promise. + +_join_ and _wait_ can be controlled through parameters. Each are iterators paused with a sleep method and will throw +an expiration exception should you set a limit for iterations. If an expiration value is not set, +no exception will be thrown and the cycle will run only until the promise(s) are resolved. _join_ and _wait_ are not +affected by _spawn-ctl_. + +_watch_ is called to establish a separate asynchronous thread that will call back a function of your choosing should +the command the promise is attached to time out. This is different than _join_ and _wait_ in that _watch_ is not synchronous +and does not pause. This is used to keep an eye on a spawned command and take action should it hang. Your watcher +function is passed the promise on which the watcher was attached, and the arguments, if any, from the spawn expression. +If your command does not time out (i.e. hangs and expires), the watcher thread will quietly go away when the promise +is resolved. _watch_ expiration is expressed in **seconds**, unlike _join_ and _wait_ which are expressed as total +_iterations_ paused at the sleep value. _watch_'s polling cycle pause is .250 seconds, so the expiration value is +multiplied by 4. The default expiration is 15 seconds. + +Examples: +``` +# Spawn a thread running this command +p = spawn `ls -lrt`: + ## resolver block ## + return True + +# Wait for promises, pause for 1/4 second each iteration, and throw an exception after 4 iterations +(1 second) +try: + p.join({"sleep": .250, "expire": 4}) +except Exception as ex: + print(ex.args) + +# Wait for this promise, pause for 1 second each iteration, and throw an exception after 5 iterations +(5 seconds) +try: + p.wait({"sleep": 1, "expire": 5}) +except Exception as ex: + print(ex.args) + +# My watcher function (called if spawned command never resolves by its experation period) +def watcher(promise, args): + print(f"This promise is likely hung: {promise.command}") + print(f"and I still have the spawn expression's args: {args}") + +p = spawn `echo "hello" && sleep 5` args: + print(f"Args passed to me: {args}") + return True + +# Attach a watcher to this thread. It will be called upon experation. +p.watch(watcher) +print("watch() does not pause like join or wait") + +# Attach a watcher that will expire in 5 seconds +p.watch(watcher, {"expire": 5}) +``` + +**_join_ syntax** +``` +promise.join({optional args}) +Where args is a Python dictionary with the following options: + "sleep" - seconds of sleep for each iteration (fractions such as .5 are honored) + default: .5 seconds + "expire" - number of sleep iterations until an excpetions is raised + default: no expiration +Note: "args" is optional and can be omitted +``` + +_Example of joining parent and children promises_: +``` +p = spawn `ls *.txt`: + for f in promise.output.stdout: + cmd = f"tar -zcvf {f}.tar.gz {f}" + spawn `$cmd` {"file":f}: + print(f"{f} completed") + promise.resolve_parent() + return True + return False + +# Wait for all commands to complete +try: + p.join({"sleep":1, "expire":20}) +except Exception as ex: + print(ex.args) +``` + +**_wait_ syntax** +``` +promise.wait({optional args}) +Where args is a Python dictionary with the following options: + "sleep" - seconds of sleep for each iteration (fractions such as .5 are honored) + default: .5 seconds + "expire" - number of sleep iterations until an excpetions is raised + default: no expiration +Note: "args" is optional and can be omitted +``` + +_Example of waiting on just the parent promise_: +``` +p = spawn `ls *.txt`: + for f in promise.output.stdout: + cmd = f"tar -zcvf {f}.tar.gz {}" + spawn `$cmd` {"file":f}: + print(f"{f} completed") + promise.resolve_parent() # Wait completes here + return True + return False + +# Wait for just the parent promise to complete +try: + p.wait({"sleep":1, "expire":20}) +except Exception as ex: + print(ex.args) +``` + +**_watch_ syntax** +``` +promise.watch(callback, {optional args}) +Where args is a Python dictionary with the following options: + "sleep" - seconds of sleep for each iteration (fractions such as .5 are honored) + default: .5 seconds + "expire" - number of sleep iterations until an excpetions is raised + default: no expiration +Note: "args" is optional and can be omitted +``` + +_Example of creating a watcher_: +```buildoutcfg +# Define watcher method. Called if command times out (i.e. expires) +def time_out(promise, args): + print(f"Command {promise.command} timed out.") + +# Spawn a thread running some command that hangs +p = spawn `long-running.sh`: + print("Finally completed. Watcher method won't be called.") + return True + +p.watch(time_out) # Does not wait. Calls method "time_out" if this promise expires (i.e. command hangs) + +# Do other things... + +``` + +<div id="promise-tree"/> + +#### The Promise Tree +Each _spawn_ issued inserts its promise object into the promise tree. The outermost _spawn_ will generate the root +promise and each inner _spawn_ will be among its children. There's no limit to how far it can nest. _wait_ only applies +to the promise on which it is called and is how it is different than _join_. _wait_ does not consider any other +promise state but the one it's called for, whereas _join_ considers the one it's called for **and** anything below it +in the tree. + +The promise tree can be printed with the ```dump_tree()``` method on the promise. This method is intended for +diagnostic purposes where it must be determined why spawned commands hung. ```dump_tree(subtree)``` accepts +a subtree promise as an argument. If no arguments are passed, ```dump_tree()``` dumps from the root promise on down. +``` +# Simple example with no child promises +p = spawn `date`: + return True + +p.tree_dump() # Dump tree from root +# or +p.tree_dump(subtree_promise) # Dump tree from node in argument +``` + +Example dumping tree from subtree node: +```buildoutcfg +# Complex example with child and grandchild promises +# Demonstrates how to dump the promise tree from various points within it +p = spawn `date`: + # Spawn child command (child promise) + spawn `pwd`: + # Spawn a grandchild to the parent promise + spawn `python --version`: + promise.tree_dump(promise) # Dump the subtree from this point down + return False + # Spawn another child + spawn `echo "blah"`: + # Resolve parent promise + promise.resolve_parent() + # Resolve child promise + return True + # Do NOT resolve parent promise, let child do that + return False + +p.join() +p.tree_dump(p.children[0]) # Dump subtree from first child on down +p.tree_dump(p.children[1]) # Dump subtree from the second child +p.tree_dump(p.children[0].children[0]) # Dump subtree from the grandchild + +# Dump all children +for c in p.children: + p.tree_dump(c) +``` + +_Parent and child joins shown in these two examples_: + +``` +root_promise = spawn `ls -lr`: + for file in promise.stdout: + t = f"touch {file}" + spawn `$t` {"file" file}: # This promise is a child of root + print(f"{file} updated".) + spawn `echo "done" > /tmp/done"`: # Another child promise (root's grandchild) + print("Complete") + promise.resolve_parent() + return True + promise.resolve_parent() + return False + return False + +root_promise.join() # Wait on the root promise and all its children. Thus, waiting for everything. +``` + +``` +root_promise = spawn `ls -lr`: + for file in promise.output.stdout: + t = f"touch {file}" + spawn `$t` {"file" file}: # This promise is a child of root + print(f"{promise.args['file'])} updated") + promise.join() # Wait for this promise and its children but not its parent (root) + spawn `echo "done" > /tmp/done"`: + print("Complete") +``` + + + +_Resolving a parent promise_: +``` +p = spawn `ls -lrt`: + for f in promise.output.stdout: + cmd = f"touch {f}" + # Spawn command from this resolver and pass our promise + spawn `$cmd`: + print("Resolving all promises") + promise.resolve_parent() # Resolve parent promise here + return True # Resolve child promise + return False # Do NOT resolve parent promise here +p.join() # Wait for ALL promises to be resolved +``` + +<div id="spawn-results"/> + +### Results from Spawned Commands +Spawned commands return their results in the _promise.output_ property of the _promise_ object passed to +the resolver block, and in the spawn expression if there is an assignment in that spawn expression. + +The result properties can then be accessed as followed: + +<table> + <th>Property</th><th>Data Type</th><th>Description</th> + <tr></tr> + <td valign="top">promise.output.stdout</td><td valign="top">List</td><td valign="top">STDOUT lines from the command normalized for display</td> + <tr></tr> + <td valign="top">promise.output.stderr</td><td valign="top">List</td><td valign="top">STDERR lines from the command normalized for display</td> + <tr></tr> + <td valign="top">promise.output.exit_code</td><td valign="top">Integer</td><td valign="top">Exit code value from command</td> + <tr></tr> + <td valign="top">promise.output.cwd</td><td valign="top">String</td><td valign="top">Current working directory <i>after</i> command was executed</td> +</table> + + +_Notes:_ +1. Watiba backticked commands can exist within the resolver +2. Other _spawn_ blocks can be embedded within a resolver (recursion allowed) +3. The command within the _spawn_ definition can be a variable + (The same rules apply as for all backticked shell commands. This means the variable must contain + pure shell commands.) +4. The leading dash to ignore CWD _cannot_ be used in the _spawn_ expression +5. The _promise.output_ object is not available until _promise.resolved()_ returns True + +_Simple example with the shell command as a Python variable_: +``` +#!/usr/bin/python3 + +# run "date" command asynchronously +d = 'date "+%Y/%m/%d"' +spawn `$d`: + print(promise.output.stdout[0]) + return True + +``` + +_Example with shell commands executed within resolver block_: +``` +#!/usr/bin/python3 + +print("Running Watiba spawn with wait") +`rm /tmp/done` + +# run "ls -lrt" command asynchronously +p = spawn `ls -lrt`: + print(f"Exit code: {promise.output.exit_code}") + print(f"CWD: {promise.output.cwd}") + print(f"STDERR: {promise.output.stderr}") + + # Loop through STDOUT from command + for l in promise.output.stdout: + print(l) + `echo "Done" > /tmp/done` + + # Resolve promise + return True + +# Pause until spawn command is complete +p.wait() +print("complete") + +``` + +<div id="threads"/> + +### Threads +Each promise produced from a _spawn_ expression results in one OS thread. To access the +number of threads your code has spawned collectively, you can do the following: +``` +num_of_spawns = promise.spawn_count() # Returns number of nodes in the promise tree +num_of_resolved_promises = promise.resolved_count() # Returns the number of promises resolved in tree +``` +<div id="remote-execution"/> + +## Remote Execution +Shell commands can be executed remotely. This is achieved though the SSH command, issued by Watiba, and has the +following requirements: +- OpenSSH is installed on the local and remote hosts +- The local SSH key is in the remote's _authorized_keys_ file. _The details of this + process is beyond the scope of this README. For those instructions, consult www.ssh.com_ + +- Make sure that SSH'ing to the target host does not cause any prompts. + +Test that your SSH environment is setup first by manually entering: +``` +ssh {user}@{host} "ls -lrt" + +# For example +ssh rwalk@walkubu "ls -lrt" + +# If SSH prompts you, then Watiba remote execution cannot function. +``` + +To execute a command remotely, a _@host_ parameter is suffixed to the backticked command. The host name can be a +literal or a variable. To employ a variable, prepend a _$_ to the name following _@_ such as _@$var_. + +<div id="change-ssh-port"/> + +#### Change SSH port for remote execution +To change the default SSH port 22 to a custom value, add to your Watiba code: ```watiba-ctl {"ssh-port": custom port}``` +Example: +```buildoutcfg +watiba-ctl {"ssh-port": 2233} +``` +Examples: +```buildoutcfg +p = spawn `ls -lrt`@remoteserver {parms}: + for line in promise.output.stdout: + print(line) + return True + +``` +```buildoutcfg +remotename = "serverB" +p = spawn `ls -lrt`@$remotename {parms}: + for line in p.output.stdout: + print(line) + return True +``` +```buildoutcfg +out = `ls -lrt`@remoteserver +for line in out.stdout: + print(line) +``` +```buildoutcfg +remotename = "serverB" +out = `ls -lrt`@$remotename +for line in out.stdout: + print(line) +``` + + +<div id="command-hooks"/> + +## Command Hooks +Hooks are pre- or -post functions that are attached to a _command_ _pattern_, which is a regular expression (regex). Anytime Watiba encounters a command +that matches the pattern for the hook, the hook function is called. + +All commands, spawned, remote, or local, can have Python functions executed **before** exection, by default, or **post hooks** that are run **after** the command. (Note: Post hooks are not run for spwaned commands because the resolver function is a post hook itself.) These functions can be passed arguments, too. + +### Command Hook Expressions +``` +# Run before commands that match that pattern +hook-cmd "pattern" hook-function parms + +# Run before commands that match that pattern, but is non-recursive +hook-cmd-nr "pattern" hook-function parms + +# Run after commands that match that pattern +post-hook-cmd "pattern" hook-function parms + +# Run after commands that match that pattern, but is non-recursive +post-hook-cmd-nr "pattern" hook-function parms +``` + +### Hook Recursion +Hooks, which are nothing more than Python functions called before or after a command is run, can issue their own commands and, thus, cause the hook +to be recursively called. However, if the command in the hook block of code matches a command pattern that causes that same hook function to be run again, +an infinte loop can occur. To prevent that, use the **-nr** suffix on the Watiba hook expression. (-nr stands for non-recursive.) This will ensure that +the hook cannot be re-invoked for any commands that are within it. + +<br> +To attach a hook: +1. Code one or more Python functions that will be the hooks. At the end of each hook, you must return True if the hook was successful, or False +if something wrong. +2. Use the _hook-cmd_ expression to attach those hooks to a command +pattern, which is a regular expression +3. To remove the hooks, use the _remove-hooks "pattern"_ expression. If a pattern, i.e. command regex pattern, is omitted, then all command hooks are removed. + +**hook-cmd "command pattern" function parms** + +The first parameter always passed to the hook function is the Python _match_ object from the command match. This is provided so the hook has access +to the tokens on the command should it need them. + +Example: +``` +def my_hook(match, parms): + print(match.groups()) + print(f'Tar file name is {match.group(1)}') + print(parms["parmA"]) + print(parms["parmB"]) + return True # Successful execution + +def your_hook(match, parms): + # This hook doesn't need the match object, so ignores it + print(parms["something"]) + if parms["something-else"] != "blah": + return False # Failed execution + return True # Successful excution + + +# Add first hook to my tar command +hook-cmd "tar -zcvf (\S.*)" my_hook: {"parmA":"A", "parmB":"B"} + +# Add another hook to my tar command +hook-cmd "tar -zcvf (\S.*)" your_hook: {"parmD":1, "parmE":"something"} + +# Spawn command, but hooks will be invoked first... +spawn `tar -zcvf files.tar.gz /tmp/files/* `: + # Resolver code block + return True # Resolve promise +``` + +Your parameters are whatever is valid for Python. These are simply passed to their attached functions, essentially each one's key is the function name, as specified. + + +_Where are the hooks run for spawned commands?_ All hooks run under the thread of the issuer on the local host, not the target thread. + +_Where are the hooks run for remote commands?_ As with spawned commands, all hooks are issued on the local host, not the remote. Note that you +can have remote backticked commands in your hook and that will run those remotely. If your remote command matches a hook(s) pattern, then those hooks will be run. This means if your command pattern for the first remote call runs a hook that contains another remote command that matches that same command pattern, then the hook is run again. Since this can lead to infinte hook loops, Watiba offers a non-recursive definition for the command pattern. Note that this non-recursive setting +only applies to the command pattern and not the hook function itself. So if _hookA_ is run for two different command patterns, say, "ls -lrt" and "ls -laF" you can +make one non-recusrive and still run the same hook for both commands. For the recursive command pattern, the hook has no limit to its recursion. For non-recursive, +it will only be called once during the recursion process. + +To set a command pattern as non-recursive, use _hook-cmd-nr_. + +Example using a variation on a previous example: + +``` +def my_hook(match, parms) + `tar -zcvf /tmp/files` # my_hook will NOT because for this command even though it matches + print("Will be called only once!") + return True + +# Note the "-nr" on the expression. That's for non-recursive +hook-cmd-nr "tar -zcvf (\S.*)" my_hook: {"parmA":"A", "parmB":"B"} + +# my_hook will be called before this command runs +` tar -zcvf tarball.tar.gz /home/user/files.*` +``` + +<div id="command-chaining"/> + +## Command Chaining +Watiba extends its remote command execution to chaining commands across multiple remote hosts. This is achieved +by the _chain_ expression. This expression will execute the backticked command across a list of hosts, passed by +the user, sequentially, synchronously until the hosts list is exhausted, or the command fails. _chain_ returns a +Python dictionary where the keys are the host names and the values the WTOutput from the command run on that host. + +#### Chain Exception +The _chain_ expression raises a WTChainException on the first failed command. The exception raised +has the following properties: + +_WTChainException_: +<table> +<th>Property</th><th>Data Type</th><th>Description</th> +<tr></tr> +<td valign="top">command</td><td valign="top">String</td><td valign="top">Command that failed</td> +<tr></tr> +<td valign="top">host</td><td valign="top">String</td><td valign="top">Host where command failed</td> +<tr></tr> +<td valign="top">message</td><td valign="top">String</td><td valign="top">Error message</td> +<tr></tr> +<td valign="top">output</td><td valign="top">WTOutput structure: + +- stdout +- stderr +- exit_code +- cwd</td><td valign="top">Output from command</td> +</table> + +Import this exception to catch it: +```buildoutcfg +from watiba import WTChainException +``` + + +Examples: +``` +from watiba import WTChainException + +try: + out = chain `tar -zcvf backup/file.tar.gz dir/*` {"hosts", ["serverA", "serverB"]} + for host,output in out.items(): + print(f'{host} exit code: {output.exit_code}') + for line in output.stderr: + print(line) + except WTChainException(ex): + print(f"Error: {ex.message}") + print(f" host: {ex.host} exit code: {ex.output.exit_code} command: {ex.command}) + +``` + +<div id="piping-output"/> + +## Command Chain Piping (Experimental) +The _chain_ expression supports piping STDOUT and/or STDERR to other commands executed on remote servers. Complex +arrangements can be constructed through the Python dictionary passed to the _chain_ expression. The dictionary +contents function as follows: +- "hosts": [server, server, ...] This entry instructions _chain_ on which hosts the backticked command will run. + This is a required entry. + +- "stdout": {server:command, server:command, ...} + This is an optional entry. + +- "stderr": {server:command, server:command, ...} + This is an optional entry. + +Just like a _chain_ expression that does not pipe output, the return object is a dictionary of WTOutput object keyed +by the host name from the _hosts_ list and *not* from the commands recieving the piped output. + +If any command fails, a WTChainException is raised. Import this exception to catch it: +```buildoutcfg +from watiba import WTChainException +``` + +_Note_: _The piping feature is experimental as of this release, and a better design will eventually +supercede it._ + +Examples: +``` +from watiba import WTChainException + +# This is a simple chain with no piping +try: + args = {"hosts": ["serverA", "serverB", "serverC"]} + out = chain `ls -lrt dir/` args + for host, output in out.items(): + print(f'{host} exit code: {output.exit_code}') +except WTChainException as ex: + print(f'ERROR: {ex.message}, {ex.host}, {ex.command}, {ex.output.stderr}') +``` +``` +# This is a more complex chain that runs the "ls -lrt" command on each server listed in "hosts" +# and pipes the STDOUT output from serverC to serverV and serverD, to those commands, and serverB's STDERR +# to serverX and its command +try: + args = {"hosts": ["serverA", "serverB", "serverC"], + "stdout": {"serverC":{"serverV": "grep something", "serverD":"grep somethingelse"}}, + "stderr": {"serverB":{"serverX": "cat >> /tmp/serverC.err"}} + } + out = chain `ls -lrt dir/` args + for host, output in out.items(): + print(f'{host} exit code: {output.exit_code}') +except WTChainException as ex: + print(f'ERROR: {ex.message}, {ex.host}, {ex.command}, {ex.output.stderr}') +``` + +####How does this work? +Watiba will run the backticked command in the expression on each host listed in _hosts_, in sequence and synchronously. +If there is a "stdout" found in the arguments, then it will name the source host as the key, i.e. the host from which +STDOUT will be read, and fed to each host and command listed under that host. This is true for STDERR as well. + +The method in which Watiba feeds the piped output is through a an _echo_ command shell piped to the command to be run +on that host. So, "stdout": {"serverC":{"serverV": "grep something"}} causes Watiba to read each line of STDOUT from +serverC and issue ```echo "$line" | grep something``` on serverV. It is piping from serverC to serverV. + +<div id="installation"/> + +## Installation +### PIP +If you installed this as a Python package, e.g. pip, then the pre-compiler, _watiba-c_, +will be placed in your system's PATH by PIP. + +### GITHUB +If you cloned this from github, you'll still need to install the package with pip, first, for the +watbia module. Follow these steps to install Watiba locally. +``` +# Watiba package required +python3 -m pip install watiba +``` + + +<div id="pre-compiling"/> + +## Pre-compiling +Test that the pre-compiler functions in your environment: +``` +watiba-c version +``` +For example: +```buildoutcfg +rwalk@walkubu:~$ watiba-c version +Watiba 0.3.26 +``` + +To pre-compile a .wt file: +``` +watiba-c my_file.wt > my_file.py +chmod +x my_file.py +./my_file.py +``` + +Where _my_file.wt_ is your Watiba code. + +<div id="code-examples"/> + +## Code Examples + +**my_file.wt** + +``` +#!/usr/bin/python3 + +# Stand alone commands. One with directory context, one without + +# This CWD will be active until a subsequent command changes it +`cd /tmp` + +# Simple statement utilizing command and results in one statement +print(`cd /tmp`.cwd) + +# This will not change the Watiba CWD context, because of the dash prefix, but within +# the command itself the cd is honored. file.txt is created in /home/user/blah but +# this does not impact the CWD of any subsequent commands. They +# are still operating from the previous cd command to /tmp +-`cd /home/user/blah && touch file.txt` + +# This will print "/tmp" _not_ /home because of the leading dash on the command +print(f"CWD is not /home: {-`cd /home`.cwd)}" + +# This will find text files in /tmp/, not /home/user/blah (CWD context!) +w=`find . -name '*.txt'` +for l in w.stdout: + print(f"File: {l}") + + +# Embedding commands in print expressions that will print the stderr output, which tar writes to +print(`echo "Some textual comment" > /tmp/blah.txt && tar -zcvf /tmp/blah.tar.gz /tmp`).stdout) + +# This will print the first line of stdout from the echo +print(`echo "hello!"`.stdout[0]) + +# Example of more than one command in a statement line +if len(`ls -lrt`.stdout) > 0 or len(-`cd /tmp`.stdout) > 0: + print("You have stdout or stderr messages") + + +# Example of a command as a Python varible and +# receiving a Watiba object +cmd = "tar -zcvf /tmp/watiba_test.tar.gz /mnt/data/git/watiba/src" +cmd_results = `$cmd` +if cmd_results.exit_code == 0: + for l in cmd_results.stderr: + print(l) + +# Simple reading of command output +# Iterate on the stdout property +for l in `cat blah.txt`.stdout: + print(l) + +# Example of a failed command to see its exit code +xc = `lsvv -lrt`.exit_code +print(f"Return code: {xc}") + +# Example of running a command asynchronously and resolving promise +spawn `cd /tmp && tar -zxvf tarball.tar.gz`: + for l in promise.output.stderr: + print(l) + return True # Mark promise resolved + + +# List dirs from CWD, iterate through them, spawn a tar command +# then within the resolver, spawn a move command +# Demonstrates spawns within resolvers +for dir in `ls -d *`.stdout: + tar = "tar -zcvf {}.tar.gz {}" + prom = spawn `$tar` {"dir": dir}: + print(f"{}args['dir'] tar complete") + mv = f"mv -r {args['dir']}/* /tmp/." + spawn `$mv`: + print("Move done") + # Resolve outer promise + promise.resolve_parent() + return True + # Do not resolve this promise yet. Let the inner resolver do it + return False + prom.join() +``` + + + + +%prep +%autosetup -n watiba-0.6.59 + +%build +%py3_build + +%install +%py3_install +install -d -m755 %{buildroot}/%{_pkgdocdir} +if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi +if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi +if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi +if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi +pushd %{buildroot} +if [ -d usr/lib ]; then + find usr/lib -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/lib64 ]; then + find usr/lib64 -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/bin ]; then + find usr/bin -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/sbin ]; then + find usr/sbin -type f -printf "/%h/%f\n" >> filelist.lst +fi +touch doclist.lst +if [ -d usr/share/man ]; then + find usr/share/man -type f -printf "/%h/%f.gz\n" >> doclist.lst +fi +popd +mv %{buildroot}/filelist.lst . +mv %{buildroot}/doclist.lst . + +%files -n python3-watiba -f filelist.lst +%dir %{python3_sitelib}/* + +%files help -f doclist.lst +%{_docdir}/* + +%changelog +* Wed May 10 2023 Python_Bot <Python_Bot@openeuler.org> - 0.6.59-1 +- Package Spec generated @@ -0,0 +1 @@ +878dd0c0b3d50c0c2ea2c3a2ad053b68 watiba-0.6.59.tar.gz |