Metadata-Version: 2.1
Name: papermill
Version: 2.5.0
Summary: Parameterize and run Jupyter and nteract Notebooks
Home-page: https://github.com/nteract/papermill
Author: nteract contributors
Author-email: nteract@googlegroups.com
License: BSD
Project-URL: Documentation, https://papermill.readthedocs.io
Project-URL: Funding, https://nteract.io
Project-URL: Source, https://github.com/nteract/papermill/
Project-URL: Tracker, https://github.com/nteract/papermill/issues
Keywords: jupyter mapreduce nteract pipeline notebook
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: System Administrators
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: BSD License
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: click
Requires-Dist: pyyaml
Requires-Dist: nbformat >=5.1.2
Requires-Dist: nbclient >=0.2.0
Requires-Dist: tqdm >=4.32.2
Requires-Dist: requests
Requires-Dist: entrypoints
Requires-Dist: tenacity >=5.0.2
Requires-Dist: aiohttp ==3.9.0b0 ; python_version == "3.12"
Provides-Extra: all
Requires-Dist: boto3 ; extra == 'all'
Requires-Dist: azure-datalake-store >=0.0.30 ; extra == 'all'
Requires-Dist: azure-storage-blob >=12.1.0 ; extra == 'all'
Requires-Dist: requests >=2.21.0 ; extra == 'all'
Requires-Dist: azure-identity >=1.3.1 ; extra == 'all'
Requires-Dist: gcsfs >=0.2.0 ; extra == 'all'
Requires-Dist: pyarrow >=2.0 ; extra == 'all'
Requires-Dist: PyGithub >=1.55 ; extra == 'all'
Requires-Dist: black >=19.3b0 ; extra == 'all'
Provides-Extra: azure
Requires-Dist: azure-datalake-store >=0.0.30 ; extra == 'azure'
Requires-Dist: azure-storage-blob >=12.1.0 ; extra == 'azure'
Requires-Dist: requests >=2.21.0 ; extra == 'azure'
Requires-Dist: azure-identity >=1.3.1 ; extra == 'azure'
Provides-Extra: black
Requires-Dist: black >=19.3b0 ; extra == 'black'
Provides-Extra: dev
Requires-Dist: boto3 ; extra == 'dev'
Requires-Dist: botocore ; extra == 'dev'
Requires-Dist: codecov ; extra == 'dev'
Requires-Dist: coverage ; extra == 'dev'
Requires-Dist: google-compute-engine ; extra == 'dev'
Requires-Dist: ipython >=5.0 ; extra == 'dev'
Requires-Dist: ipywidgets ; extra == 'dev'
Requires-Dist: notebook ; extra == 'dev'
Requires-Dist: moto ; extra == 'dev'
Requires-Dist: pytest >=4.1 ; extra == 'dev'
Requires-Dist: pytest-cov >=2.6.1 ; extra == 'dev'
Requires-Dist: pytest-mock >=1.10 ; extra == 'dev'
Requires-Dist: pytest-env >=0.6.2 ; extra == 'dev'
Requires-Dist: requests >=2.21.0 ; extra == 'dev'
Requires-Dist: check-manifest ; extra == 'dev'
Requires-Dist: attrs >=17.4.0 ; extra == 'dev'
Requires-Dist: pre-commit ; extra == 'dev'
Requires-Dist: flake8 ; extra == 'dev'
Requires-Dist: tox ; extra == 'dev'
Requires-Dist: bumpversion ; extra == 'dev'
Requires-Dist: recommonmark ; extra == 'dev'
Requires-Dist: pip >=18.1 ; extra == 'dev'
Requires-Dist: wheel >=0.31.0 ; extra == 'dev'
Requires-Dist: setuptools >=38.6.0 ; extra == 'dev'
Requires-Dist: twine >=1.11.0 ; extra == 'dev'
Requires-Dist: azure-datalake-store >=0.0.30 ; extra == 'dev'
Requires-Dist: azure-storage-blob >=12.1.0 ; extra == 'dev'
Requires-Dist: azure-identity >=1.3.1 ; extra == 'dev'
Requires-Dist: gcsfs >=0.2.0 ; extra == 'dev'
Requires-Dist: black >=19.3b0 ; extra == 'dev'
Provides-Extra: docs
Requires-Dist: boto3 ; extra == 'docs'
Requires-Dist: azure-datalake-store >=0.0.30 ; extra == 'docs'
Requires-Dist: azure-storage-blob >=12.1.0 ; extra == 'docs'
Requires-Dist: requests >=2.21.0 ; extra == 'docs'
Requires-Dist: azure-identity >=1.3.1 ; extra == 'docs'
Requires-Dist: gcsfs >=0.2.0 ; extra == 'docs'
Requires-Dist: pyarrow >=2.0 ; extra == 'docs'
Requires-Dist: PyGithub >=1.55 ; extra == 'docs'
Requires-Dist: black >=19.3b0 ; extra == 'docs'
Requires-Dist: Sphinx >=3.5.4 ; extra == 'docs'
Requires-Dist: furo >=2021.4.11b34 ; extra == 'docs'
Requires-Dist: myst-parser >=0.13.7 ; extra == 'docs'
Requires-Dist: moto >=2.0.5 ; extra == 'docs'
Requires-Dist: sphinx-copybutton >=0.3.1 ; extra == 'docs'
Provides-Extra: gcs
Requires-Dist: gcsfs >=0.2.0 ; extra == 'gcs'
Provides-Extra: github
Requires-Dist: PyGithub >=1.55 ; extra == 'github'
Provides-Extra: hdfs
Requires-Dist: pyarrow >=2.0 ; extra == 'hdfs'
Provides-Extra: s3
Requires-Dist: boto3 ; extra == 's3'
Provides-Extra: test
Requires-Dist: boto3 ; extra == 'test'
Requires-Dist: botocore ; extra == 'test'
Requires-Dist: codecov ; extra == 'test'
Requires-Dist: coverage ; extra == 'test'
Requires-Dist: google-compute-engine ; extra == 'test'
Requires-Dist: ipython >=5.0 ; extra == 'test'
Requires-Dist: ipywidgets ; extra == 'test'
Requires-Dist: notebook ; extra == 'test'
Requires-Dist: moto ; extra == 'test'
Requires-Dist: pytest >=4.1 ; extra == 'test'
Requires-Dist: pytest-cov >=2.6.1 ; extra == 'test'
Requires-Dist: pytest-mock >=1.10 ; extra == 'test'
Requires-Dist: pytest-env >=0.6.2 ; extra == 'test'
Requires-Dist: requests >=2.21.0 ; extra == 'test'
Requires-Dist: check-manifest ; extra == 'test'
Requires-Dist: attrs >=17.4.0 ; extra == 'test'
Requires-Dist: pre-commit ; extra == 'test'
Requires-Dist: flake8 ; extra == 'test'
Requires-Dist: tox ; extra == 'test'
Requires-Dist: bumpversion ; extra == 'test'
Requires-Dist: recommonmark ; extra == 'test'
Requires-Dist: pip >=18.1 ; extra == 'test'
Requires-Dist: wheel >=0.31.0 ; extra == 'test'
Requires-Dist: setuptools >=38.6.0 ; extra == 'test'
Requires-Dist: twine >=1.11.0 ; extra == 'test'
Requires-Dist: azure-datalake-store >=0.0.30 ; extra == 'test'
Requires-Dist: azure-storage-blob >=12.1.0 ; extra == 'test'
Requires-Dist: azure-identity >=1.3.1 ; extra == 'test'
Requires-Dist: gcsfs >=0.2.0 ; extra == 'test'
Requires-Dist: black >=19.3b0 ; extra == 'test'

# <a href="https://github.com/nteract/papermill"><img src="https://media.githubusercontent.com/media/nteract/logos/master/nteract_papermill/exports/images/png/papermill_logo_wide.png" height="48px" /></a>

<!---(binder links generated at https://mybinder.readthedocs.io/en/latest/howto/badges.html and compressed at https://tinyurl.com) -->

[![CI](https://github.com/nteract/papermill/actions/workflows/ci.yml/badge.svg)](https://github.com/nteract/papermill/actions/workflows/ci.yml)
[![CI](https://github.com/nteract/papermill/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/nteract/papermill/actions/workflows/ci.yml)
[![image](https://codecov.io/github/nteract/papermill/coverage.svg?branch=main)](https://codecov.io/github/nteract/papermill?branch=main)
[![Documentation Status](https://readthedocs.org/projects/papermill/badge/?version=latest)](http://papermill.readthedocs.io/en/latest/?badge=latest)
[![badge](https://tinyurl.com/ybwovtw2)](https://mybinder.org/v2/gh/nteract/papermill/main?filepath=binder%2Fprocess_highlight_dates.ipynb)
[![badge](https://tinyurl.com/y7uz2eh9)](https://mybinder.org/v2/gh/nteract/papermill/main?)
[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/papermill)](https://pypi.org/project/papermill/)
[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/ambv/black)
[![papermill](https://snyk.io/advisor/python/papermill/badge.svg)](https://snyk.io/advisor/python/papermill)
[![Anaconda-Server Badge](https://anaconda.org/conda-forge/papermill/badges/downloads.svg)](https://anaconda.org/conda-forge/papermill)

**papermill** is a tool for parameterizing, executing, and analyzing
Jupyter Notebooks.

Papermill lets you:

- **parameterize** notebooks
- **execute** notebooks

This opens up new opportunities for how notebooks can be used. For
example:

- Perhaps you have a financial report that you wish to run with
  different values on the first or last day of a month or at the
  beginning or end of the year, **using parameters** makes this task
  easier.
- Do you want to run a notebook and depending on its results, choose a
  particular notebook to run next? You can now programmatically
  **execute a workflow** without having to copy and paste from
  notebook to notebook manually.

Papermill takes an *opinionated* approach to notebook parameterization and
execution based on our experiences using notebooks at scale in data
pipelines.

## Installation

From the command line:

```{.sourceCode .bash}
pip install papermill
```

For all optional io dependencies, you can specify individual bundles
like `s3`, or `azure` -- or use `all`. To use Black to format parameters you can add as an extra requires \['black'\].

```{.sourceCode .bash}
pip install papermill[all]
```

## Python Version Support

This library currently supports Python 3.8+ versions. As minor Python
versions are officially sunset by the Python org papermill will similarly
drop support in the future.

## Usage

### Parameterizing a Notebook

To parameterize your notebook designate a cell with the tag `parameters`.

![enable parameters in Jupyter](docs/img/enable_parameters.gif)

Papermill looks for the `parameters` cell and treats this cell as defaults for the parameters passed in at execution time. Papermill will add a new cell tagged with `injected-parameters` with input parameters in order to overwrite the values in `parameters`. If no cell is tagged with `parameters` the injected cell will be inserted at the top of the notebook.

Additionally, if you rerun notebooks through papermill and it will reuse the `injected-parameters` cell from the prior run. In this case Papermill will replace the old `injected-parameters` cell with the new run's inputs.

![image](docs/img/parameters.png)

### Executing a Notebook

The two ways to execute the notebook with parameters are: (1) through
the Python API and (2) through the command line interface.

#### Execute via the Python API

```{.sourceCode .python}
import papermill as pm

pm.execute_notebook(
   'path/to/input.ipynb',
   'path/to/output.ipynb',
   parameters = dict(alpha=0.6, ratio=0.1)
)
```

#### Execute via CLI

Here's an example of a local notebook being executed and output to an
Amazon S3 account:

```{.sourceCode .bash}
$ papermill local/input.ipynb s3://bkt/output.ipynb -p alpha 0.6 -p l1_ratio 0.1
```

**NOTE:**
If you use multiple AWS accounts, and you have [properly configured your AWS  credentials](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html), then you can specify which account to use by setting the `AWS_PROFILE` environment variable at the command-line. For example:

```{.sourceCode .bash}
$ AWS_PROFILE=dev_account papermill local/input.ipynb s3://bkt/output.ipynb -p alpha 0.6 -p l1_ratio 0.1
```

In the above example, two parameters are set: `alpha` and `l1_ratio` using `-p` (`--parameters` also works). Parameter values that look like booleans or numbers will be interpreted as such. Here are the different ways users may set parameters:

```{.sourceCode .bash}
$ papermill local/input.ipynb s3://bkt/output.ipynb -r version 1.0
```

Using `-r` or `--parameters_raw`, users can set parameters one by one. However, unlike `-p`, the parameter will remain a string, even if it may be interpreted as a number or boolean.

```{.sourceCode .bash}
$ papermill local/input.ipynb s3://bkt/output.ipynb -f parameters.yaml
```

Using `-f` or `--parameters_file`, users can provide a YAML file from which parameter values should be read.

```{.sourceCode .bash}
$ papermill local/input.ipynb s3://bkt/output.ipynb -y "
alpha: 0.6
l1_ratio: 0.1"
```

Using `-y` or `--parameters_yaml`, users can directly provide a YAML string containing parameter values.

```{.sourceCode .bash}
$ papermill local/input.ipynb s3://bkt/output.ipynb -b YWxwaGE6IDAuNgpsMV9yYXRpbzogMC4xCg==
```

Using `-b` or `--parameters_base64`, users can provide a YAML string, base64-encoded, containing parameter values.

When using YAML to pass arguments, through `-y`, `-b` or `-f`, parameter values can be arrays or dictionaries:

```{.sourceCode .bash}
$ papermill local/input.ipynb s3://bkt/output.ipynb -y "
x:
    - 0.0
    - 1.0
    - 2.0
    - 3.0
linear_function:
    slope: 3.0
    intercept: 1.0"
```

#### Supported Name Handlers

Papermill supports the following name handlers for input and output paths during execution:

- Local file system: `local`

- HTTP, HTTPS protocol:  `http://, https://`

- Amazon Web Services: [AWS S3](https://aws.amazon.com/s3/) `s3://`

- Azure: [Azure DataLake Store](https://docs.microsoft.com/en-us/azure/data-lake-store/data-lake-store-overview), [Azure Blob Store](https://docs.microsoft.com/en-us/azure/storage/blobs/storage-blobs-overview) `adl://, abs://`

- Google Cloud: [Google Cloud Storage](https://cloud.google.com/storage/) `gs://`

## Development Guide

Read [CONTRIBUTING.md](./CONTRIBUTING.md) for guidelines on how to setup a local development environment and make code changes back to Papermill.

For development guidelines look in the [DEVELOPMENT_GUIDE.md](./DEVELOPMENT_GUIDE.md) file. This should inform you on how to make particular additions to the code base.

## Documentation

We host the [Papermill documentation](http://papermill.readthedocs.io)
on ReadTheDocs.
