- DocumentID: foreach
  Title: >+
    `foreach`
  CategoryID: commands
  Summary: >-
    Iterate through an array
  Description: |-
    `foreach` reads an array or map from STDIN and iterates through it, running
    a code block for each iteration with the value of the iterated element passed
    to it.

    By default `foreach`'s output data type is inherieted from its input data type.
    For example is STDIN is `yaml` then so will STDOUT. The only exception to this
    is if STDIN is `json` in which case STDOUT will be jsonlines (`jsonl`), or when
    additional flags are used such as `--jmap`.
  Usage: |-
    `{ code-block }` reads from a variable and writes to an array / unbuffered STDOUT:

    ```
    <stdin> -> foreach variable { code-block } -> <stdout>
    ```

    `{ code-block }` reads from STDIN and writes to an array / unbuffered STDOUT:

    ```
    <stdin> -> foreach { -> code-block } -> <stdout>
    ```

    `foreach` writes to a buffered JSON map:

    ```
    <stdin> -> foreach --jmap variable { code-block (map key) } { code-block (map value) } -> <stdout>
    ```
  Examples: |-
    There are two basic ways you can write a `foreach` loop depending on how you
    want the iterated element passed to the code block.
    
    The first option is to specify a temporary variable which can be read by the
    code block:

    ```
    » a [1..3] -> foreach i { out $i }
    1
    2
    3
    ```

    > Please note that the variable is specified **without** the dollar prefix,
    > then used in the code block **with** the dollar prefix.

    The second option is for the code block's STDIN to read the element:

    ```
    » a [1..3] -> foreach { -> cat }
    1
    2
    3
    ```

    > STDIN can only be read as the first command. If you cannot process the
    > element on the first command then it is recommended you use the first
    > option (passing a variable) instead.

    ### Writing JSON maps

    ```
    » ja [Monday..Friday] -> foreach --jmap day { out $day -> left 3 } { $day }
    {
        "Fri": "Friday",
        "Mon": "Monday",
        "Thu": "Thursday",
        "Tue": "Tuesday",
        "Wed": "Wednesday"
    } 
    ```

    ### Using steps to jump iterations by more than 1 (one)

    You can step through an array, list or table in jumps of user definable
    quantities. The value passed in STDIN and $VAR will be an array of all
    the records within that step range. For example:

    ```
    » %[1..10] -> foreach --step 3 value { out "Iteration $.i: $value" }
    Iteration 1: [
        1,
        2,
        3
    ]
    Iteration 2: [
        4,
        5,
        6
    ]
    Iteration 3: [
        7,
        8,
        9
    ]
    Iteration 4: [
        10
    ]
    ```
  Flags:
    --jmap: >-
      Write a `json` map to STDOUT instead of an array
    --step: >-
      <int> Iterates in steps. Value passed to block is an array of items in the step range. Not (yet) supported with `--jmap`

  Detail: |-
    ### Meta values

    Meta values are a JSON object stored as the variable `$.`. The meta variable
    will get overwritten by any other block which invokes meta values. So if you
    wish to persist meta values across blocks you will need to reassign `$.`, eg

    ```
    %[1..3] -> foreach {
        meta_parent = $.
        %[7..9] -> foreach {
            out "$(meta_parent.i): $.i"
        }
    }
    ```

    The following meta values are defined:

    * `i`: iteration number

    ### Preserving the data type (when no flags used)

    `foreach` will preserve the data type read from STDIN in all instances where
    data is being passed along the pipeline and push that data type out at the
    other end:

    * The temporary variable will be created with the same data-type as
      `foreach`'s STDIN, or the data type of the array element (eg if it is a
      string or number)
    * The code block's STDIN will have the same data-type as `foreach`'s STDIN
    * `foreeach`'s STDOUT will also be the same data-type as it's STDIN (or `jsonl`
      (jsonlines) where STDIN was `json` because `jsonl` better supports streaming)

    This last point means you may need to `cast` your data if you're writing
    data in a different format. For example the following is creating a YAML list
    however the data-type is defined as `json`:

    ```
    » ja [1..3] -> foreach i { out "- $i" }
    - 1
    - 2
    - 3

    » ja [1..3] -> foreach i { out "- $i" } -> debug -> [[ /Data-Type/Murex ]]
    json
    ```

    Thus any marshalling or other data-type-aware API's would fail because they
    are expecting `json` and receiving an incompatible data format.

    This can be resolved via `cast`:

    ```
    » ja [1..3] -> foreach i { out "- $i" } -> cast yaml
    - 1
    - 2
    - 3

    » ja [1..3] -> foreach i { out "- $i" } -> cast yaml -> debug -> [[ /Data-Type/Murex ]]
    yaml
    ```

    The output is the same but now it's defined as `yaml` so any further pipelined
    processes will now automatically use YAML marshallers when reading that data.
    
    {{ include "gen/includes/for-loop-json-tips.inc.md" }}
  Synonyms:
  Related:
  - formap
  - for
  - while
  - if
  - format
  - cast
  - a
  - ja
  - types/json
  - types/jsonl
  - types/yaml
  - out
  - left
  - debug
  - element
  - apis/ReadArrayWithType
  - break
