Marcel

Tab completion wasn't working inside a nested pipeline, e.g. ls -dr ... | args [d: ls ...

Overhauled parser's tracking of context for use by tab completion.

Marcel now has an operator, `args`, that works like Linux's `xargs`.
For example, suppose you have a directory containing log files, and you want to
delete logs that are more than 100 days old. Locating
such files is easy:

shell script
ls -f | select (f: now() - f.mtime > days(100))


What you would like to do is to take each `File` in the resulting stream, and
delete it. The `args` operator takes items arriving on the input
stream, and makes them available to operators. So to do the file removal:

shell script
ls -f | select (f: now() - f.mtime > days(100)) | args [f: rm (f)]


- `ls ... | select ...` produces a stream of `File`s.
- These `File`s flow into the `args` input stream.
- `args` has a pipeline with parameter `f`. Each arriving `File` is bound to `f`, and
the pipeline is executed.
- `rm (f)` removes `File` `f`.

You can also pass all of the qualifying `File`s at once:

shell script
ls -f | select (f: now() - f.mtime > days(100)) | args --all [files: rm (quote_files(files))]


Now, all of the qualifying `File`s are bound to the pipelines `files` parameter.
Because the args pipeline relies on the host OSs `rm` command, carefully quoting
filenames is necessary. `quote_files` takes `files`, a `list` of `File`s, and returns
a string containing all the `File`s names, separated by spaces, and each quoted, (e.g.
to properly handle a filename such as `this filename has four spaces`.)

Actually, you can do the same cleanup by relying on the fact that `File`s implement
the `pathlib.Path` interface, which has an `unlink` method. So this works too:

shell script
ls -f | select (f: now() - f.mtime > days(100)) | map (f: f.unlink())

0.10.6 is a bugfix release, fixing issues raised by issue 1. See the discussion there for more details.

Reworked the grammar again for the > and >> symbols. Added testing through every path of Parser.pipeline().

Syntactic sugar has been sprinkled on top of the `store` and `load`
operators.

A stream carries tuples from one operator to another. `Store`
accumulates these tuples into a `list`, and `load` turns them back
into a stream. So, for example, you can store recently updated
files as follows:

shell script
ls -fr | select (f: now() - f.mtime < days(1)) | store recent


And then do further processing as follows:

shell script
load recent | select (f: f.suffix == '.py')


You can still use this syntax. Using the newly added syntax, these commands
can be written as:

shell script
ls -fr | select (f: now() - f.mtime < days(1)) > recent
recent > select (f: f.suffix == '.py')

Marcel has two new operators: `ifthen` and `ifelse`. Easiest to explain with an
example: Suppose you want to find all the `.py`
files in a directory, recursively.
You could do this:

shell script
ls -fr | select (f: f.suffix == '.py') | store py_files


After this runs, the variable `py_files` stores a `File` object for each qualifying file.
Similarly, to find all the `.txt` files:

shell script
ls -fr | select (f: f.suffix == '.txt') | store txt_files


You can combine these into a single command,
traversing the directory structure once instead of twice:

shell script
ls -fr \
| ifelse (f: f.suffix == '.py') [store py_files] \
| ifelse (f: f.suffix == '.txt') [store txt_files]


`ifelse` tests the current tuple from the input stream, using the given
predicate, and if it evalautes to `True`, then the
tuple is passed to the bracketed pipeline. If the predicate evaluates to `False`,
then the tuple is passed downstream instead.

`ifthen` is used to send qualifying tuples to the bracketed pipeline, while _all_
tuples are passed downstream. For example, you might want to create lists of files
that are not disjoint. The following command places `.py` files in `py_files` again,
and places files modified in the past day in `recent_updates`. Some files may appear in
both.

shell script
ls -fr \
| ifthen (f: f.suffix == '.py') [store py_files] \
| ifthen (f: now() - f.mtime < days(1)) [store recent_updates]


(You can examine the contents of the resulting variables using the `load` command,
e.g. `load py_files` will print the contained `File`s to stdout.)