In this series of notebooks we discuss the basics of a vitally important concept: the mathematical function. So completely do the idea of mathematical functions permeate science and mathematics - from their use as modeling tools to being objects of study themselves - that most readers have certainly encountered them in some fashion at some point in their lives.
In machine learning / deep learning we are always dealing with mathematical functions - from the framing of a learning problem, to the derivation of a cost function, to the development and use of mathematical optimization techniques, to the design of features - they are ever-present in our work. And while from an outsider's perspective their usage in the field may seem to astound, arising in un-ending variation, insiders learn to appreciate how these many variations arise as small (albeit sometimes quite clever) twists on a handful of fundamental ideas. Indeed we will see that such a statement applies equally well to other major technical tools and frameworks used in machine learning / deep learning as well, particularly those which use mathematical functions as a basic building block. But lets not get ahead of ourselves - first things first.
In this short series of notebooks we review some fundamental ideas regarding mathematical functions, ideas that we will see used over and over again throughout our study of machine learning / deep learning. We begin here by introducing the notion of a mathematical function in terms of real data and then in terms of equations / formulae.
# run this cell to import all necessary libraries for the notebook experiments
import sys
sys.path.append('../../')
from mlrefined_libraries import basics_library as baslib
import pandas as pd
import numpy as np
What is a mathematical function? In this short Section we describe the fundamental idea of a mathematical function. We also introduce common function notation and detail the form in which functions are most commonly seen in today's modern world: as a table of values (or collection of data). Much of the discussion in these examples is obvious, it is largely the vocabulary (e.g., 'data is a mathematical function', 'a mathematical function is a rule that relates inputs to outputs') that we need to get straight in our minds.
To get an intuitive sense for what a mathematical function is, lets look a number of examples.
The Python cell below prints out the first few values dataset [1] - a listing of the annual total revenue of the fastfood resturant chain McDonald's over a period of five years.
# read in a dataset
mcdonalds_revenue = pd.read_csv('../../mlrefined_datasets/basics_datasets/mcdonalds_revenue.csv')
# display table of values
baslib.basics_plotter.table_plot(table = mcdonalds_revenue[:5])
This data consists of a table of values with two columns - the 'Year' revenue is tracked in the first column, and the total revenue of the company is in the second. When we examine a table like this we naturally scan across each row to process its information - in the Year 2005 the Revenue was 19.12 billion dollars, in 2006 the revenue was 22.90 billion dollars, etc., - and in processing we start to understand the relationship between the input value 'Year' and output value 'Revenue'. We start to see each row as a datapoint, consisting of one input value and one output value, or taken together an input/output pair. For example the first row contains a datapoint or input/output pair $(\text{Year},\text{Revenue}) = (2005,19.12)$, the second row $(\text{Year},\text{Revenue}) = (2006,20.90)$, and so on.
This might not be what you expect a 'mathematical function' to look like - but it is. This dataset is a typical example of a mathematical function - which is just a rule defining how input values and output values of a dataset or system relate to one another. Here the rule is explicit in the data itself: when the input 'Year' is 2005, the output 'Revenue' is 19.12 (billion dollars), when the input is 2006, the output is 20.90, and so forth. In other words, here the phrase 'mathematical function' just means 'a dataset consisting of input/output pairs'.
We often plot such a mathematical function to make its relationship more visually explicit - as we do in the next Python cell (note here that we plot the entire dataset which consists of 12 input/output pairs).
# create function
mcd_table = np.asarray(mcdonalds_revenue)
# use custom plotter to plot function table of values
baslib.basics_plotter.single_plot(table = mcd_table,xlabel = mcdonalds_revenue.columns[0],ylabel = mcdonalds_revenue.columns[1],plot_type = 'scatter',rotate_ylabel = 90,label_fontsize = 12)
Here each input/output pair from the data is drawn as a unique red circle - because the input 'Year' is naturally ordered from smallest to largest - the plot can easily be related to the raw table of values: points from the table - starting at the top and moving down - are here plotted from left to right.
Lets examine another dataset of a similar vein [2] - the total revenue of the Chiquita banana company. The next Python cell plots the first few points from this dataset.
# read in a dataset
chiquita_rev = pd.read_csv('../../mlrefined_datasets/basics_datasets/chiquita_revenue.csv')
# display table of values
baslib.basics_plotter.table_plot(table = chiquita_rev[:5])
Similar to the McDonalds data, this dataset also describes the revenue of a large company and is again a mathematical function. There are only slight differences here from the first dataset - first, we can see that the measurements are more frequent here than in first dataset, as the input 'Quarter' (whose values are listed as decimals) gives us four datapoints of revene per year. Secondly the output - here still 'Revenue' - is of a diffeerent scale than the first dataset, here we are dealing with hundreds of millions instead of billions as was the case previously. Nonetheless, we still have a mathematical function relating input values ('Quarter') to output values ('Revenue').
Again we can visualize the entire dataset to better appreciate the input/output relationship, as is done in the next Python cell.
# create function
chiq_table = np.asarray(chiquita_rev)
# use custom plotter to plot function table of values
baslib.basics_plotter.single_plot(table = chiq_table,xlabel = chiquita_rev.columns[0],ylabel = chiquita_rev.columns[1],plot_type = 'scatter',guides = 'on',rotate_ylabel = 90,label_fontsize = 12)
One again each input/output pair from the data is drawn here as a unique red circle - with added light red lines connecting consecutive datapoints for visualization purposes only. Again because the input 'Quarter' is naturally ordered from smallest to largest - the plot can easily be related to the raw table of values: points from the table - starting at the top and moving down - are here plotted from left to right on the graph.
A restaurant menu [3] - like the one from McDonalds printed out by the next Python cell - provides a cornucopia of mathematical functions.
# read in a dataset
mcd_menu = pd.read_csv('../../mlrefined_datasets/basics_datasets/mcdonalds_menu.csv')
# display the first few values from the table
baslib.basics_plotter.table_plot(table = mcd_menu[:5])
Here we have a dataset of food items, along with a large number of characteristics for each. Where is the mathematical function here? Where is its input and output? Here - unlike the previous examples - there are many choices. We decide what goes in, and what comes out. What rule about the quantities listed in this dataset might we like to better understand?
For example, we could decide to look at the relationship between the food item and its allotment of calories. Segmenting out just these two columns from the dataset we print out the first few data points using the next Python cell.
# display the first few values from the table - using only the 'Item' and 'Calories' columns
baslib.basics_plotter.table_plot(table = mcd_menu[['Item','Calories']][:5])
Notice here - unlike the previous examples - our input is not numerical. Thats fine - we still have a mathematical function relating inputs ('Item') to outputs ('Calories'). Because we have so much information on each food item, we could have just as well chosen a different output - say 'Total Fat'. In the next Python cell we print out our input 'Item' and this output together - just the first few data points.
# display the first few values from the table - using only the 'Item' and 'Total Fat' columns
baslib.basics_plotter.table_plot(table = mcd_menu[['Item','Total Fat']][:5])
Again we have a perfectly useful mathematical function relating inputs ('Item') to outputs ('Total Fat').
It is not always the case that a dataset / mathematical function comes in the form of a labeled table - as with the previous examples - where input and output are placed neatly in separate columns. Take a standard grayscale image - like handwritten digit '0' plotted by the next Python cell.
# load in image path
img_path = '../../mlrefined_images/basics_images/digit.png'
# plot image
baslib.image_function_plotter.reveal_imgpatch(img_path = img_path)
The left panel displays the raw image itself - so small that we can actually see all of the individual pixels that make it up. Although we may not always think of them as one, a grayscale image like this is in fact a mathematical function. Grayscale images are in fact 2-dimensional arrays of numbers with height and width equal to the number of pixels in the image itself. This view of our digit image is plotted in the middle panel in the Figure above. Here we see that this image is in fact an $8\times8$ array of numbers, with each number (printed red on top of its respective pixel) representing the intensity value of each pixel - ranging from 0 (completely back) to $255$ (completely white).
As a mathematical function the grayscale image relates a pair of indices - indicating a specific row and column of the array - (the inputs) to a given pixel intensity value (the output). For example, in the middle panel in the Figure above we can see that the index pair $(0,0)$ has pixel value $255$, the index pair $(0,2)$ has the pixel value $170$, the pair $(1,3)$ has value $0$, and so forth.
Note that we can if we desire write out this function as a table, as with the previous examples. Writing out the first four values in the first row we have
input: $(\text{row #},\text{column #})$ | output: $\text{pixel value}$ |
---|---|
$(0,0)$ | $255$ |
$(0,1)$ | $255$ |
$(0,2)$ | $170$ |
$(0,3)$ | $34$ |
$\vdots$ | $\vdots$ |
where the $\vdots$ notation indicate that the list keeps on going (in this case to list 60 further input/output pairs).
Regardless of how we record them, each input/output pair in a grayscale image is a 3-dimensional point - two values (indices) for each input, one value per output - and so if we want we can plot any grayscale image as a surface in 3-dimensional space. We do this for the digit image in the right panel of the Figure, and can now physically see how each input (a pair of indices) relates to its pixel value - that is the height of each output (we also color each output its corresponding grayscale value for visualization purposes).
Viewing a more complicated image in this way - as a surface in 3d - can be quite interesting visually speaking. Take for example the image [4] printed by the next Python cell. This cell will printout both the color and grayscale versions of an input image.
# Provide a path to a color image
img_path = '../../mlrefined_images/basics_images/nurg.jpg'
# show the color and grayscale version of this image
baslib.image_function_plotter.show_color_gray(img_path = img_path)
In the next Python cell we show the grayscale form of this image in 3-dimensions. We also animate a range of viewing angles on this surface just for fun - you can move the slider from left to right to transition smoothly between viewing angles. The pre-set viewing route takes you from 90 degrees above the image to 90 degrees below it in a spiral.
# Provide a path to a color image, this will be converted to grayscale for the animation
img_path = '../../mlrefined_images/basics_images/nurg.jpg'
# animate a range of viewing angles of this image in 3-dimensions
num_frames = 200
baslib.image_function_plotter.grayimg_as_function(img_path = img_path, num_frames = num_frames,shrink_factor = 0.1,plot_type = 'proto')