Jupyter Notebook Basics
Welcome! This is the first “notebook” post for the ART Book website. This is also an introduction to how notebooks flow for the whole site:
- On the right-hand side of this page, you can see the table of contents for each of the headings of the notebook.
- These posts insert a Jupyter/IPython notebook into the body, providing a way to just see all of the material on the website
- If you want to run the notebook yourself, there is a button in the header of this page to download the notebook itself!
The notebook begins…now!
Example Jupyter Notebook
This is an example notebook that demonstrates some of the things that you can do with notebooks!
You will need a working Python environment and jupyterlab installed to run this example.
Basics
Cells
There are mainly two types of cells in a Jupyter notebook:
- Code cells: where code actually runs and produces an output immediately after.
- Markdown cells: where markdown text is displayed, such as right here!
Jupyter notebooks are actually super powerful, allowing for multiple languages and types of cells beyond these two, but this is what you will see 99% of the time here and in other notebooks.
Running Code
Because Python is so ubiquitous, the code cells in these notebooks mainly run Python code. To run code in any notebook once you have it in your hands, however, you need a kernel!
What is a kernel, you might ask? It is pretty much a code interpreter for the cells of a notebook. The Jupyter project, which is the format for these notebooks, uses the IPython kernel for interpreting Python code. That’s why they used to be called IPython notebook!
Now, you can simply get a Python session set up on your computer (I highly recommend mamba as a virtual environment manager), and pip install jupyterlab in the command line to get all of the machinery to run notebooks.
If you’re opening a notebook in the browser or an IDE, either way you need to make sure that the kernel being used is the one where you installed jupyterlab and your other dependencies (hence the recommendation for a virtual environment manager!).
Dependencies
To get your dependencies going for these notebooks, there is a relatively simple requirements.txt file that you can get you set up with
pip install -r requirements.txt
Virtual Environments
Why use a virtual environment? Please indulge the following dialectical detour, and consider the following scenario:
In whatever language you’re coding in, you start a new project.
You find module A from some library changed its API for function A.cool_func(a) to A.cool_func(a, b) in version v0.1.25.
They did this to make it work better, faster, etc. for some reason or another, and it does!
You start using that module and function, and it solves all of your coding problems.
Why reinvent the wheel, right?
Chances are that first project was not your last; after having great success with that project, you start another to do something else neat.
Now you find another module B that does other cool stuff that would be great to add to your new project.
Under the hood, it has it’s own dependency on module A (probably because it is popular because it does lots of cool stuff really well).
HOWEVER!
Writer of module B (translation: some programmer, a.k.a. another human) wrote it using module A at version v0.1.24 when it was was still using the old function signature A.cool_func(a).
If you use the new version of module A, then module B breaks.
What to do!?
Do you give up on your second project?
Do you give up on using module B?
Do you downgrade module A and break the old project on your computer so that the first project doesn’t work anymore?
In colloquial terms, you have found yourself in dependency hell.
Each option kinda stinks, and you’d think that there were some kind of workaround for this sort of thing, right? Well, as contrived as this example is, this sort of conundrum is suprisingly ubiquitous in software:
- If you have ever writen any software that uses external libraries and dependencies, you will quickly learn how much of a headache conflicting versions of things can be when that list of dependencies gets bigger.
- You’re likely to work on more than one single project on you computer, each with its own dependencies, increasing the chances of a conflicting versions of dependencies.
- If you plan on distributing your code to someone else (which is how everything on the internet works), you’re more likely to have the project run on your computer but break on someone else’s computer. If you don’t control what versions of which dependencies your project needs, then you can’t control what version of packages other people have on their systems, and this scenario becomes increasingly likely to happen. And then those people have their own set of other projects, making this possibility practically an inevitability!
There are lots of flavours of this problem, and there are lots of solutions to each of these different flavours, each with their own pros and cons.
Virtual environments are simple, even if a bit inelegant, solution!
Pretend that each project gets its own “sandbox” where they can install all of the dependencies that they need.
This way, they can resolve version conflicts on their own, and they won’t touch the packages in each other project’s sandbox.
Unless your virtual environment manager has some solution for maximizing the use of shared code and packages, this is often implemented by literally just having a new directory populated with a download of the programming language’s internals.
This is how venv and mamba/conda work!
The main downside is that you might end up with many downloads of the same package across multiple virtual environments (which will eat up space when you start talking multi-gigabyte packages like PyTorch with GPU support!), but if you have the space to spare on your system and on whatever other system you deploy to, then virtual environments work great!
Venv
If you want to just use the Python standard library, then venv is the tool for you!
If you have Python installed, just run
python3 -m venv ~/.venv/<your-venv-name>
where you replace <your-venv-name> with whatever you want to put in there for your project.
By convention, venv virtual environments are place in the home directory under ~/.venv/, but you can technically put them anywhere you want!
You then “activate” the environment before working (every time you start up the shell to work on your project) with:
source ~/.venv/<your-venv-name>/bin/activate
Then feel free to install your dependencies inside that environment with the usual
pip install <some-cool-package>
Mamba
Like I mentioned before, I highly recommend Mamba!
It is like conda, if you are familiar with that ecosystem, except it is rewritten in C++ rather than Python itself, making it do essentially the same things (dependency management, virtual environments, etc.) but faster!
The instructions to install it are found on the Miniforge GitHub repo. It is potentially convoluted to follow the instructions through those websites, so you can install it (on a Mac/Linux system) with:
curl -L -O "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$(uname -m).sh"
This command grabs the correct download script for your system architecture from their releases page. You then run that installation script with:
bash Miniforge3-$(uname)-$(uname -m).sh
Poetry
There is also a popular Python package management ecosystem called Poetry that has been gaining traction over the years. Feel free to check it out and see if it suits you!
Example Usage
Now, let’s try running some code blocks!
Input/Output
First, lets just see what a code block and its output look like:
# Let's check out where this script is running
import os
os.getcwd()
'c:\\Users\\Sasha\\Repos\\github\\art-book-online.github.io\\_notebooks\\notebooks'
That was pretty neat! We can see the code block that is run along with the output result of the cell immediately afterward. If you see this on the ART Book Online website, you probably see the location of this notebook temporary directory of some machine that built the website! If you run it for yourself, then you’ll see your own local directory.
Plots
But what if we wanted to do something fancier, such as generate and display a plot?
Sine Wave
Consider we want to calculate the sine function from $0$ to $2\pi$ radians.
To do that, we need to load a plotting library, such as matplotlib and plot some numbers that we generate with numpy.
To see the plot itself, we plt.show() it at the end of the cell, et voila!
# Import numpy for handling numbers, math, etc., and matplotlib for plotting
import numpy as np
import matplotlib.pyplot as plt
# Generate x values (e.g., from 0 to 2*pi with 100 evenly spaced points)
x = np.linspace(0, 2 * np.pi, 100)
# Calculate the sine of the x values to get the y values
y = np.sin(x)
# Plot the data
plt.plot(x, y)
# Add labels, a title, and a grid for better readability
plt.xlabel('Angle [radians]')
plt.ylabel('Amplitude')
plt.title('Sine Wave Plot')
plt.grid(True)
# Display the plot
plt.show()
Scatters
That’s super cool! But what about scatter plots? And plots with color?
To try that out, let’s plot some randomly-generated points and tinker with their colors.
# We'll use scikit-learn's make_blobs dataset generator as an example
from sklearn.datasets import make_blobs
# Generate the data
X, y = make_blobs(
n_samples=100, # The number of samples total
centers=3, # The number of blobs
cluster_std=0.5, # The standard deviation of the gaussian blobs
random_state=0, # Somerandom seed for the random generator
)
# Create the scatter plot
ax = plt.figure()
# Get the handle of the scatter plot for later
s = plt.scatter(
X[:, 0], # The x-values
X[:, 1], # The y-values
c=y, # The blob "labels"
s=50, # The marker size for the dots
alpha=0.7, # The opacity of the center of the dots
)
# Plot the legend in a hacky way, getting the "labels" in a colored legend
plt.legend(
*s.legend_elements(),
title="Classes",
)
# Add labels and a title
plt.xlabel("x")
plt.ylabel("y")
plt.title("Blob Scatters")
# Display the plot
plt.show()
Loading Data
Sweet! But what if we want to display some data that we loaded from disk?
Iris Dataset
Well then, that requires some useful tools, such as scikit-learn and pandas for handling and loading data!
# Load the dependencies for the iris dataset and the DataFrame type
from sklearn.datasets import load_iris
import pandas as pd
# Load the dataset as a DataFrame
iris = load_iris(as_frame=True)
# Extract the frame itself
data = iris['frame']
# Inspect the top few rows
data.head()
| sepal length (cm) | sepal width (cm) | petal length (cm) | petal width (cm) | target | |
|---|---|---|---|---|---|
| 0 | 5.1 | 3.5 | 1.4 | 0.2 | 0 |
| 1 | 4.9 | 3.0 | 1.4 | 0.2 | 0 |
| 2 | 4.7 | 3.2 | 1.3 | 0.2 | 0 |
| 3 | 4.6 | 3.1 | 1.5 | 0.2 | 0 |
| 4 | 5.0 | 3.6 | 1.4 | 0.2 | 0 |
Super cool! We generated a bunch of data earlier (a sine wave and some random Gaussians), but here we got some real-world data! Except there’s very quickly a hitch, even with such a small and relativley simple “real-world” dataset: it’s 4-D! How do we inspect 4-dimensional data? Even if we added a z-axis, that is still only three dimensions. We need to use some even fancier visualization techniques!
Pair Plots
One such technique is to do so-called “pair plots”, where we plot two dimension at a time, scattering each combination of dimensions of the dataset on their own.
Here, we’ll use the high-level seaborn package to do the heavy lifting!
# Import seaborn, aliased as sns by convention
import seaborn as sns
# Pairplot takes pd.DataFrames, which the data already is, and we color with the target values
sns.pairplot(data, hue='target')
# Display the plot here
plt.show()
TSNE
Another technique is to project the data into two dimensions just for visualization. What we plot isn’t the true shape of the data, but it helps to get an idea of the separability of the points in the higher space by projecting them down into the same relative distances in two dimensions.
# Grab TSNE from scikit-learn
from sklearn.manifold import TSNE
# For copying the original dataset
from copy import deepcopy
# For manipulating axis tick locations
from matplotlib import ticker
# Scatters points TSNE points and colors according to label
def add_2d_scatter(ax, points, colors, title=None):
x, y = points
# sa = ax.scatter(x, y, s=50, c=colors, alpha=0.8)
plt.scatter(x, y, s=50, c=colors, alpha=0.8)
ax.set_title(title)
ax.xaxis.set_major_formatter(ticker.NullFormatter())
ax.yaxis.set_major_formatter(ticker.NullFormatter())
return
# Generates the plot itself
def plot_2d(points, y, title):
cmap = plt.get_cmap('tab10')
colors = [cmap(i) for i in y]
fig, ax = plt.subplots(
facecolor="white",
constrained_layout=True,
)
fig.suptitle(title, size=16)
add_2d_scatter(ax, points, colors)
plt.show()
# Now, we initialize a TSNE module with its own set of hyperparameters for how
# it will be "trained" to create a mapping between the 4d data and its 2d projection.
t_sne = TSNE(
n_components=2,
perplexity=10,
init="random",
max_iter=250,
random_state=0,
)
# Extract the labels by copying the data and popping the label column
tsne_data = deepcopy(data)
y = tsne_data.pop('target')
# Fit the TSNE to the data, and return those transformed points
S_t_sne = t_sne.fit_transform(tsne_data)
Now that we have TSNE fitted to the data and the plotting functions defined, we can create a scatter plot of the projected data!
# Generate the plot
plot_2d(S_t_sne.T, y, "TSNE (FuzzyART Labels)")
So cool!
Conclusion
Imagine all of the things that you can do with this kind of a notebook workflow. I personally use notebooks for development of my experiment code because I can quickly inspect data, inspect my training results, and visualize everything with plots directly in the same workspace. There are even things you can do with widgets to make the code interactive if you want to change hyperparameters, etc. on the fly!
For now, however, hopefully you have a taste of what you should expect when reading through and working with the notebooks in the site. Enjoy!
Page built at: 2026-07-31 22:51:53 +0000