Jupyter Notebooks

class: left, middle, inverse, title-slide

.title[
# Jupyter Notebooks
]
.author[
### Sohail Nizam<br> <span style="font-size: 50%;"> Emory University<br> Department of Biostatistics and Bioinformatics </span>
]
.date[
### INFO550<br><br><svg viewBox="0 0 512 512" style="height:1em;position:relative;display:inline-block;top:.1em;fill:#ffffff;" xmlns="http://www.w3.org/2000/svg"> <path d="M326.612 185.391c59.747 59.809 58.927 155.698.36 214.59-.11.12-.24.25-.36.37l-67.2 67.2c-59.27 59.27-155.699 59.262-214.96 0-59.27-59.26-59.27-155.7 0-214.96l37.106-37.106c9.84-9.84 26.786-3.3 27.294 10.606.648 17.722 3.826 35.527 9.69 52.721 1.986 5.822.567 12.262-3.783 16.612l-13.087 13.087c-28.026 28.026-28.905 73.66-1.155 101.96 28.024 28.579 74.086 28.749 102.325.51l67.2-67.19c28.191-28.191 28.073-73.757 0-101.83-3.701-3.694-7.429-6.564-10.341-8.569a16.037 16.037 0 0 1-6.947-12.606c-.396-10.567 3.348-21.456 11.698-29.806l21.054-21.055c5.521-5.521 14.182-6.199 20.584-1.731a152.482 152.482 0 0 1 20.522 17.197zM467.547 44.449c-59.261-59.262-155.69-59.27-214.96 0l-67.2 67.2c-.12.12-.25.25-.36.37-58.566 58.892-59.387 154.781.36 214.59a152.454 152.454 0 0 0 20.521 17.196c6.402 4.468 15.064 3.789 20.584-1.731l21.054-21.055c8.35-8.35 12.094-19.239 11.698-29.806a16.037 16.037 0 0 0-6.947-12.606c-2.912-2.005-6.64-4.875-10.341-8.569-28.073-28.073-28.191-73.639 0-101.83l67.2-67.19c28.239-28.239 74.3-28.069 102.325.51 27.75 28.3 26.872 73.934-1.155 101.96l-13.087 13.087c-4.35 4.35-5.769 10.79-3.783 16.612 5.864 17.194 9.042 34.999 9.69 52.721.509 13.906 17.454 20.446 27.294 10.606l37.106-37.106c59.271-59.259 59.271-155.699.001-214.959z"></path></svg> <a href="https://bit.ly/info550">.white[bit.ly/info550]</a> <br> <svg viewBox="0 0 448 512" style="height:1em;position:relative;display:inline-block;top:.1em;fill:#ffffff;" xmlns="http://www.w3.org/2000/svg"> <path d="M448 360V24c0-13.3-10.7-24-24-24H96C43 0 0 43 0 96v320c0 53 43 96 96 96h328c13.3 0 24-10.7 24-24v-16c0-7.5-3.5-14.3-8.9-18.7-4.2-15.4-4.2-59.3 0-74.7 5.4-4.3 8.9-11.1 8.9-18.6zM128 134c0-3.3 2.7-6 6-6h212c3.3 0 6 2.7 6 6v20c0 3.3-2.7 6-6 6H134c-3.3 0-6-2.7-6-6v-20zm0 64c0-3.3 2.7-6 6-6h212c3.3 0 6 2.7 6 6v20c0 3.3-2.7 6-6 6H134c-3.3 0-6-2.7-6-6v-20zm253.4 250H96c-17.7 0-32-14.3-32-32 0-17.6 14.4-32 32-32h285.4c-1.9 17.1-1.9 46.9 0 64z"></path></svg> <a href="https://benkeser.github.io/info550/readings#jupyter-notebooks">.white[Additional reading]</a>
]

---

## An interactive way to create reports

Imagine taking a compiled Rmarkdown report and typing directly into each code or markdown chunk!
 
* Run code chunks one at a time
* Make changes to text, figures, and other output without recompiling
* Write bash code within the notebook to install packages
* Easily embed HTML into text chunks 
* Share your report as HTML, PDF, and more
* Share your notebook so others build off your work

---

## Jupyter Notebooks vs. Rmarkdown

Rmarkdown is best for:

* Polished reports
* Detailed formatting
* Seamlessly weaving output into documents without showing code
* R

Jupyter Notebooks is best for:

* Interactivity
* Collaboration
* Exploratory analysis 
* Python

---

## JSON files

* Jupyter Notebooks are saved as .ipynb files
* .ipynb files are [JSON](https://www.json.org/json-en.html) (Java Script Object Notation) files
* JSON files are just plain text and thus can be read/created in any programming language
* For python users: syntax looks just like a [python dictionary](https://docs.python.org/3/tutorial/datastructures.html#dictionaries)

---

## Opening Jupyter Notebooks

* Navigate to your project directory and run this command in terminal

```bash
jupyter notebook
```

* A browser window will open with address `http://localhost:8888/tree`
* You should see the files that live in the current directory

---

## Kernels

From the Jupyter Notebook docs:

"Kernels are programming language specific processes that run independently and interact with the Jupyter Applications and their user interfaces."

Find a list of all available kernels [here](https://github.com/jupyter/jupyter/wiki/Jupyter-kernels). We are most interested in:

* [Ipython](https://ipython.org/)
* [IRKernel](https://irkernel.github.io/)

---

## Closing Jupyter Notebooks

.pull-left[
* Closing the browser tab .red[will not stop the server] or shutdown notebooks 
* __Ctrl + c__ stops the server and shuts down all running notebooks
* Shutting down a notebook .red[kills the R/Python session]
* To keep the server running, the terminal window we ran `jupyter notebook` from must be kept open
]

.pull-right[

]

---

## Breakout Exercise 1

* Open a jupyter notebook
* Close the browser tab that opened automatically
* Figure out how to reopen the JN session in your browser

---

## Keyboard Shortcuts

There are tons of shortcuts! Don't need to learn them all.

<br>

---

## Magics (only for Ipython)

Magics are special commands that add functionality not readily available by just writing Python code and using the JN interface.

* Line magics start with `%`
* Cell magics stare with `%%`
* Run `%lsmagic` in a cell to list all available magics
* Many many magics!
* `%%bash` turns the whole cell into a bash cell
* `%%HTML` turns the whole cell into an HTML cell
* `%who_ls` lists all objects saved in the notebook
* `%timeit` provides a convenient way to time your code

---

## IRkernel magics?

.center[*We don’t and won’t support %%cell magic like %%HTML.*]

.center[\- The IRkernel dev team]

<br>

* Use the `system()` function to write bash code
* Use the `IRdisplay` library to embed images, html, etc

---

## Widgets (Ipython only)

Build interactive GUIs in your notebooks! From terminal, run:

```bash
pip install ipywidgets
```

Then, to test out basic widget functionality, within a code cell in an Ipython notebook, run:

```bash
from ipywidgets import interact, interactive, fixed, interact_manual
import ipywidgets as widgets
```

---

## Widgets (Ipython only)

__Example__: The `interact` widget generates UI controls for function arguments then calls the function with those arguments when you manipulate the controls interactively

```bash
from ipywidgets import *
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

x = np.linspace(0, 2 * np.pi)

def update(w = 1.0):
    fig = plt.figure()
    ax = fig.add_subplot(1, 1, 1)
    ax.plot(x, np.sin(w * x))

fig.canvas.draw()

interact(update);
```

---

## Interactive plots with plotly

[`plotly`](https://plotly.com/) is a great library for making interactive plots.
* [Example plots for R](https://plotly.com/r/)
* [Example plots for python](https://plotly.com/python/)

```r
library(plotly)

set.seed(123)

x <- rnorm(1000)
y <- rchisq(1000, df = 1, ncp = 0)
group <- sample(LETTERS[1:5], size = 1000, replace = T)
size <- sample(1:5, size = 1000, replace = T)

ds <- data.frame(x, y, group, size)

plot_ly(ds, x = x, y = y, mode = "markers", split = group, size = size, type = 'scatter') %>%
  layout(title = "Scatter Plot")
```

* Works with both Ipython and IRkernel!

---

## R and Python in the same notebook

R and python can be used in the same notebook via [`rpy2`](https://rpy2.github.io/).

```bash
pip install rpy2
```

```bash
%load_ext rpy2.ipython
import pandas as pd
import numpy as np
df = pd.DataFrame({
    'cups_of_coffee': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
    'productivity': [2, 5, 6, 8, 9, 8, 0, 1, 0, -1]
})
df.transpose()
```

```bash
%%R -i df 
library(ggplot2)
ggplot(df, aes(x=reorder(cups_of_coffee,productivity), y=productivity)) + geom_col() + coord_flip()
```

---

## Sharing notebooks

* We can share notebooks as .ipynb files
* Remember, this is just a plaintext JSON file!
* Save the notebook and it will show up in the directory you opened JN from
* Share as you would any text file
* Make sure the person recieving has the necessary packages and versions (Docker where are you??)

---

## Exporting notebooks

* We can export to many different file formats for static viewing

<br>

---

## Breakout Exercise 2

Here we'll work with the youtube video at this link: https://www.youtube.com/embed/dQw4w9WgXcQ

* Start a jupyter notebook session
* Start one Ipython kernel and one IRkernel
* Use a line magic to embed the video in the Ipython notebook (Hint: `%%HTML`)
* Use IRDisplay to embed the video in the IRkernel notebook (Hint: `IRdisplay::display_html()`)
* Save both notebooks as HTML files (save as something besides Untitled)
* Send both HTML files to someone in your breakout room
* Recieve two HTML files from someone in your breakout room
* Open the HTML files and watch the video

---

## Notebooks Online

* [mybinder](https://jupyter.org/try) 
  * Just for playing around
* [Google Colab](https://colab.research.google.com/) 
  * Not exactly JN but similar functionality 
  * Free access to CPUs/GPUs, and pre-built enviroments