Unconventional Pandas: Colormaps#
Hello everyone! We have some exciting events coming up, including a NEW seminar series and a code review workshop series. In our brand new seminar series, we will share with you some of the hardest problems we have had to solve in pandas and NumPy (and, in our bonus session on September 16th, hard problems that we have had to solve in Matplotlib!). Then, next month starting October 12th, we will be holding our first ever “No Tears Code Review,” where we’ll take attendees througha a code review that will actually help them gain insight into their code and cause meaningful improvements to their approach.
Let’s get to the exciting content!
A response to “Try This: An Unconventional Pandas Tutorial”#
A little while ago, I stumbled across a Quantsight blog post by Eric Kelly & Tony Fast titled, “Try This: An Unconventional Pandas Tutorial”. (Unfortunately, at the time of writing this, the post was either taken down or moved, which is why I didn’t link directly to it.) The authors achieved what they set out to do, namely to demonstrate a novel use of pandas, outside of common data analysis. I loved the post, but thought to myself, “have they really demonstrated the power of pandas?”
After conferring with the authors of the original post, we agreed that it would be good to write a response of my own to build on top of what they created and further highlight the power and flexibility of the most popular data analysis Python has to offer.
Our goal here is to create a colormap primarily using pandas and some html rendering. We want to replicate the following from the aforementioned blog post.
The true superpower of pandas lies in its Index object. This object is considered in nearly all pandas methods, and it governs how data is aligned. For example, we can use the index of a Series to map the data from that Series to another datastructure. Subsequently, we can think of a DataFrame as a container of Series objects that all share the same Index.
Before we get to the code, let’s lay out some steps to create this colormap:
Make base data structures
Colors x RGB values (N x 3)
Levels of shading
Combine these data structures
Render the resultant dataframe as colormap
Let’s go ahead and load in the tools we’ll need to use:
from IPython.display import display, Markdown
from webcolors import rgb_to_name
from pandas import DataFrame, Series, MultiIndex, concat
from numpy import linspace, arange, where, asarray
from itertools import product as iter_product
Make base data structures#
Let’s tackle the core pieces of our colormap. As I mentioned earlier, we want to make 2 data structures:
Colors x RGB values (N x 3)
Levels of shading
Our Colors x RGB values will be a DataFrame where each row is a color, and each column corresponds to the rgb values that constitute that color.
We also have to consider a shading array that will be used to scale the rgb values. This will be represented by a Series.
values = [0, 255]
shading = Series(linspace(0, 1, num=10).round(2), name="shading")
rgb_df = (
DataFrame(
data=iter_product(values, repeat=3),
columns=["r", "g", "b"]
)
.pipe(lambda df: df.set_axis(df.apply(rgb_to_name, axis="columns")))
.rename_axis(index="name", columns="component")
)
display(
Markdown("**Levels of Shading**"),
shading,
Markdown("<br>**RGB DataFame**"),
rgb_df
)
Levels of Shading
0 0.00
1 0.11
2 0.22
3 0.33
4 0.44
5 0.56
6 0.67
7 0.78
8 0.89
9 1.00
Name: shading, dtype: float64
RGB DataFame
| component | r | g | b |
|---|---|---|---|
| name | |||
| black | 0 | 0 | 0 |
| blue | 0 | 0 | 255 |
| lime | 0 | 255 | 0 |
| cyan | 0 | 255 | 255 |
| red | 255 | 0 | 0 |
| magenta | 255 | 0 | 255 |
| yellow | 255 | 255 | 0 |
| white | 255 | 255 | 255 |
Combining Base Data Structures#
Now that we have our base data structures sorted out, we have to think of how we want to combine these to replicate the colormap at the top of this post. We then want to multiply the rgb values by the degree of shading, such that the rgb values composing red (255, 0, 0) become:
Shading example table: red only (255, 0, 0)
from numpy import multiply
DataFrame(
multiply.outer(shading.to_numpy(), rgb_df.loc['red', :].to_numpy()),
index=shading,
columns=rgb_df.columns
)
| component | r | g | b |
|---|---|---|---|
| shading | |||
| 0.00 | 0.00 | 0.0 | 0.0 |
| 0.11 | 28.05 | 0.0 | 0.0 |
| 0.22 | 56.10 | 0.0 | 0.0 |
| 0.33 | 84.15 | 0.0 | 0.0 |
| 0.44 | 112.20 | 0.0 | 0.0 |
| 0.56 | 142.80 | 0.0 | 0.0 |
| 0.67 | 170.85 | 0.0 | 0.0 |
| 0.78 | 198.90 | 0.0 | 0.0 |
| 0.89 | 226.95 | 0.0 | 0.0 |
| 1.00 | 255.00 | 0.0 | 0.0 |
To achieve this, we can use a MultiIndex to generate all combinations of rgb values at each level of shading. Then, we can use .reindex to achieve label-aligned titling of our data. Lastly, we can multiply the tiled rgb values by their corresponding levels of shading that we’ve aligned them to. We end up with a structured DataFrame of “color by (shading x rgb)”.
For you NumPy fans out there, this result is also attainable in a single step using np.multiply.outer to return a 3d array (or 2d array if you ravel’d the input) that you put into a hierarchical DataFrame, like the one outputted below.
shade_rgb_index = MultiIndex.from_product([shading, rgb_df.columns])
shade_rgb_df = (
rgb_df.reindex(columns=shade_rgb_index, level=1)
.mul(shade_rgb_index.get_level_values("shading"))
.astype(int)
)
display(Markdown("**Combined Colormap**"), shade_rgb_df)
Combined Colormap
| shading | 0.00 | 0.11 | 0.22 | 0.33 | ... | 0.67 | 0.78 | 0.89 | 1.00 | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| component | r | g | b | r | g | b | r | g | b | r | ... | b | r | g | b | r | g | b | r | g | b |
| name | |||||||||||||||||||||
| black | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ... | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| blue | 0 | 0 | 0 | 0 | 0 | 28 | 0 | 0 | 56 | 0 | ... | 170 | 0 | 0 | 198 | 0 | 0 | 226 | 0 | 0 | 255 |
| lime | 0 | 0 | 0 | 0 | 28 | 0 | 0 | 56 | 0 | 0 | ... | 0 | 0 | 198 | 0 | 0 | 226 | 0 | 0 | 255 | 0 |
| cyan | 0 | 0 | 0 | 0 | 28 | 28 | 0 | 56 | 56 | 0 | ... | 170 | 0 | 198 | 198 | 0 | 226 | 226 | 0 | 255 | 255 |
| red | 0 | 0 | 0 | 28 | 0 | 0 | 56 | 0 | 0 | 84 | ... | 0 | 198 | 0 | 0 | 226 | 0 | 0 | 255 | 0 | 0 |
| magenta | 0 | 0 | 0 | 28 | 0 | 28 | 56 | 0 | 56 | 84 | ... | 170 | 198 | 0 | 198 | 226 | 0 | 226 | 255 | 0 | 255 |
| yellow | 0 | 0 | 0 | 28 | 28 | 0 | 56 | 56 | 0 | 84 | ... | 0 | 198 | 198 | 0 | 226 | 226 | 0 | 255 | 255 | 0 |
| white | 0 | 0 | 0 | 28 | 28 | 28 | 56 | 56 | 56 | 84 | ... | 170 | 198 | 198 | 198 | 226 | 226 | 226 | 255 | 255 | 255 |
8 rows × 30 columns
Display Our Colormap#
What enables us to use pandas in this unconventional way is its powerful styling interface. This interface lets us control how DataFrames are rendered into HTML, which is useful if you’re working within a Jupyter notebook or planning to push output from a DataFrame into any web front-end.
I know the display_colormap function is a bit long, but I wanted to highlight the utility of DataFrame styling by adding many options to consider when reporting your DataFrame. DataFrame.style is extremely flexible, and it can require a little bit of digging to use effectively.
def display_colormap(
df, header_font_size=15, rgb_font_size=0, cell_width=80, cell_height=60
):
"""Displays RGB colors in jupyter
"""
def _rgb_style(rgb_values):
"""rgb tuple -> inline html styling
"""
# Thank you @seaborn.utils/relative_luminence
rgb = asarray(rgb_values) / 255
rgb = where(rgb <= .03928, rgb / 12.92, ((rgb + .055) / 1.055) ** 2.4)
lum = rgb.dot([.2126, .7152, .0722]).item()
color = "black" if lum > .408 else "white"
rgb_css = ", ".join(map(str, rgb_values))
return f"""
background-color: rgb({rgb_css});
color: {color};
"""
### Make outputted table look nice
# - 'selector' determines the element that 'props' will be applied to
# - `.index_name` refers to the name of the column and row index
# specifically refers to "shading" and "name" in our example
header_css = {
'selector': 'th:not(.index_name)', # all headers that are NOT "shading" or "name"
'props': f'font-size: {header_font_size}px;'
}
name_css = {
'selector': '.index_name',
'props': f'font-size: 20px;'
}
tablecell_css = {
"width": f"{cell_width}px",
"height": f"{cell_height}px",
"font-size": f"{rgb_font_size}px",
}
return (
# combines all rgb values into tuples as prep for _rgb_style function
df.groupby("shading", axis="columns").apply(DataFrame.agg, tuple, axis="columns")
.pipe(lambda d: d.set_axis(d.columns.map('{:.2f}'.format), axis="columns"))
.style
.set_table_styles(
[header_css, name_css]
) # add table-level css
.set_properties(**tablecell_css) # set td css properties
.applymap(_rgb_style) # convert each rgb tuple to inline html style
)
display(
Markdown("**DataFrame Colormap**"),
shade_rgb_df.pipe(display_colormap, rgb_font_size=15, cell_width=100, cell_height=60)
)
/tmp/ipykernel_902997/2331890231.py:42: FutureWarning: DataFrame.groupby with axis=1 is deprecated. Do `frame.T.groupby(...)` without axis instead.
df.groupby("shading", axis="columns").apply(DataFrame.agg, tuple, axis="columns")
/tmp/ipykernel_902997/2331890231.py:49: FutureWarning: Styler.applymap has been deprecated. Use Styler.map instead.
.applymap(_rgb_style) # convert each rgb tuple to inline html style
DataFrame Colormap
| shading | 0.00 | 0.11 | 0.22 | 0.33 | 0.44 | 0.56 | 0.67 | 0.78 | 0.89 | 1.00 |
|---|---|---|---|---|---|---|---|---|---|---|
| name | ||||||||||
| black | (0, 0, 0) | (0, 0, 0) | (0, 0, 0) | (0, 0, 0) | (0, 0, 0) | (0, 0, 0) | (0, 0, 0) | (0, 0, 0) | (0, 0, 0) | (0, 0, 0) |
| blue | (0, 0, 0) | (0, 0, 28) | (0, 0, 56) | (0, 0, 84) | (0, 0, 112) | (0, 0, 142) | (0, 0, 170) | (0, 0, 198) | (0, 0, 226) | (0, 0, 255) |
| lime | (0, 0, 0) | (0, 28, 0) | (0, 56, 0) | (0, 84, 0) | (0, 112, 0) | (0, 142, 0) | (0, 170, 0) | (0, 198, 0) | (0, 226, 0) | (0, 255, 0) |
| cyan | (0, 0, 0) | (0, 28, 28) | (0, 56, 56) | (0, 84, 84) | (0, 112, 112) | (0, 142, 142) | (0, 170, 170) | (0, 198, 198) | (0, 226, 226) | (0, 255, 255) |
| red | (0, 0, 0) | (28, 0, 0) | (56, 0, 0) | (84, 0, 0) | (112, 0, 0) | (142, 0, 0) | (170, 0, 0) | (198, 0, 0) | (226, 0, 0) | (255, 0, 0) |
| magenta | (0, 0, 0) | (28, 0, 28) | (56, 0, 56) | (84, 0, 84) | (112, 0, 112) | (142, 0, 142) | (170, 0, 170) | (198, 0, 198) | (226, 0, 226) | (255, 0, 255) |
| yellow | (0, 0, 0) | (28, 28, 0) | (56, 56, 0) | (84, 84, 0) | (112, 112, 0) | (142, 142, 0) | (170, 170, 0) | (198, 198, 0) | (226, 226, 0) | (255, 255, 0) |
| white | (0, 0, 0) | (28, 28, 28) | (56, 56, 56) | (84, 84, 84) | (112, 112, 112) | (142, 142, 142) | (170, 170, 170) | (198, 198, 198) | (226, 226, 226) | (255, 255, 255) |
Thanks to pandas.loc’s label-based slicing syntax, we can subset this color map in a very human-readable way:
# Rows: ['lime', 'cyan', 'yellow']
# Columns: all columns
shade_rgb_df.loc[["lime", "cyan", "yellow"], :].pipe(display_colormap)
/tmp/ipykernel_902997/2331890231.py:42: FutureWarning: DataFrame.groupby with axis=1 is deprecated. Do `frame.T.groupby(...)` without axis instead.
df.groupby("shading", axis="columns").apply(DataFrame.agg, tuple, axis="columns")
/tmp/ipykernel_902997/2331890231.py:49: FutureWarning: Styler.applymap has been deprecated. Use Styler.map instead.
.applymap(_rgb_style) # convert each rgb tuple to inline html style
| shading | 0.00 | 0.11 | 0.22 | 0.33 | 0.44 | 0.56 | 0.67 | 0.78 | 0.89 | 1.00 |
|---|---|---|---|---|---|---|---|---|---|---|
| name | ||||||||||
| lime | (0, 0, 0) | (0, 28, 0) | (0, 56, 0) | (0, 84, 0) | (0, 112, 0) | (0, 142, 0) | (0, 170, 0) | (0, 198, 0) | (0, 226, 0) | (0, 255, 0) |
| cyan | (0, 0, 0) | (0, 28, 28) | (0, 56, 56) | (0, 84, 84) | (0, 112, 112) | (0, 142, 142) | (0, 170, 170) | (0, 198, 198) | (0, 226, 226) | (0, 255, 255) |
| yellow | (0, 0, 0) | (28, 28, 0) | (56, 56, 0) | (84, 84, 0) | (112, 112, 0) | (142, 142, 0) | (170, 170, 0) | (198, 198, 0) | (226, 226, 0) | (255, 255, 0) |
# Rows: Slice rows from blue to red"
# Columns: Columns with shading >= 0.5
shade_rgb_df.loc["blue":"red", 0.5:].pipe(display_colormap)
/tmp/ipykernel_902997/2331890231.py:42: FutureWarning: DataFrame.groupby with axis=1 is deprecated. Do `frame.T.groupby(...)` without axis instead.
df.groupby("shading", axis="columns").apply(DataFrame.agg, tuple, axis="columns")
/tmp/ipykernel_902997/2331890231.py:49: FutureWarning: Styler.applymap has been deprecated. Use Styler.map instead.
.applymap(_rgb_style) # convert each rgb tuple to inline html style
| shading | 0.56 | 0.67 | 0.78 | 0.89 | 1.00 |
|---|---|---|---|---|---|
| name | |||||
| blue | (0, 0, 142) | (0, 0, 170) | (0, 0, 198) | (0, 0, 226) | (0, 0, 255) |
| lime | (0, 142, 0) | (0, 170, 0) | (0, 198, 0) | (0, 226, 0) | (0, 255, 0) |
| cyan | (0, 142, 142) | (0, 170, 170) | (0, 198, 198) | (0, 226, 226) | (0, 255, 255) |
| red | (142, 0, 0) | (170, 0, 0) | (198, 0, 0) | (226, 0, 0) | (255, 0, 0) |
We can even mix colors! Simply taking the .mean() of 2 or more colors will average their RGB values.
# Color Mixing
(
shade_rgb_df.loc[["blue", "red"], :].mean() # average together rgb values of "blue" and "red"
.to_frame("purple").T # reshape our DataFrame to be used by `display_colormap`
.pipe(display_colormap) # display colors instead of rgb values
)
/tmp/ipykernel_902997/2331890231.py:42: FutureWarning: DataFrame.groupby with axis=1 is deprecated. Do `frame.T.groupby(...)` without axis instead.
df.groupby("shading", axis="columns").apply(DataFrame.agg, tuple, axis="columns")
/tmp/ipykernel_902997/2331890231.py:49: FutureWarning: Styler.applymap has been deprecated. Use Styler.map instead.
.applymap(_rgb_style) # convert each rgb tuple to inline html style
| shading | 0.00 | 0.11 | 0.22 | 0.33 | 0.44 | 0.56 | 0.67 | 0.78 | 0.89 | 1.00 |
|---|---|---|---|---|---|---|---|---|---|---|
| purple | (0.0, 0.0, 0.0) | (14.0, 0.0, 14.0) | (28.0, 0.0, 28.0) | (42.0, 0.0, 42.0) | (56.0, 0.0, 56.0) | (71.0, 0.0, 71.0) | (85.0, 0.0, 85.0) | (99.0, 0.0, 99.0) | (113.0, 0.0, 113.0) | (127.5, 0.0, 127.5) |
Now that we have a way to easily visualize our colormap DataFrame, we can begin to envision what we can do with it. As per the example above, we know that mixing red and blue should give us a shade of purple. Mathematically, we can represent mixing two colors as taking the mean across their corresponding RGB values. The fun part is that we can mix as many of these base colors together as we want!
Let’s define a custom mix_colors function so that we don’t need to repeat the long chain of commands above.
Mixing Colors from our colormap#
def mix_colors(df, colors, weights=None, name=None):
"""Mixes 2 colors from `df` into a Series
"""
if weights is None:
weights = [1] * len(colors)
return (
shade_rgb_df
.loc[colors, :] # Select rows from supplied color names
.T.dot(weights).div(sum(weights)) # calculate weighted average
.rename(name)
)
# we can use `mix_colors` to calculate the average rgb values among 2 or more colors
display(
# our display_colormap function expects input in the following format
shade_rgb_df.pipe(mix_colors, ["red", "blue"], name="purple").to_frame().T,
shade_rgb_df.pipe(mix_colors, ["blue", "lime"], name="turquoise").to_frame().T
)
| shading | 0.00 | 0.11 | 0.22 | 0.33 | ... | 0.67 | 0.78 | 0.89 | 1.00 | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| component | r | g | b | r | g | b | r | g | b | r | ... | b | r | g | b | r | g | b | r | g | b |
| purple | 0.0 | 0.0 | 0.0 | 14.0 | 0.0 | 14.0 | 28.0 | 0.0 | 28.0 | 42.0 | ... | 85.0 | 99.0 | 0.0 | 99.0 | 113.0 | 0.0 | 113.0 | 127.5 | 0.0 | 127.5 |
1 rows × 30 columns
| shading | 0.00 | 0.11 | 0.22 | 0.33 | ... | 0.67 | 0.78 | 0.89 | 1.00 | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| component | r | g | b | r | g | b | r | g | b | r | ... | b | r | g | b | r | g | b | r | g | b |
| turquoise | 0.0 | 0.0 | 0.0 | 0.0 | 14.0 | 14.0 | 0.0 | 28.0 | 28.0 | 0.0 | ... | 85.0 | 0.0 | 99.0 | 99.0 | 0.0 | 113.0 | 113.0 | 0.0 | 127.5 | 127.5 |
1 rows × 30 columns
mix_colors takes rows from our inputted DataFrame to calculate their averaged rgb values and outputs a single Series. If we make a number of these Series and concatenate them, we can derive a new color map based on combinations of the original.
mixed_color_df = concat([
shade_rgb_df.pipe(mix_colors, ["red", "blue"], name="purple"),
shade_rgb_df.pipe(mix_colors, ["blue", "lime"], name="turquoise"),
shade_rgb_df.pipe(mix_colors, ["red", "white"], name="pink"),
shade_rgb_df.pipe(mix_colors, ["red", "yellow"], name="orange"),
# complex color mixed from 3 inputs!
shade_rgb_df.pipe(mix_colors, ["blue", "lime", "yellow"], name="green"),
], axis=1).T.astype(int)
display(
Markdown("**Mixed DataFrame Colormap**"),
mixed_color_df.pipe(display_colormap, rgb_font_size=14, cell_width=100)
)
/tmp/ipykernel_902997/2331890231.py:42: FutureWarning: DataFrame.groupby with axis=1 is deprecated. Do `frame.T.groupby(...)` without axis instead.
df.groupby("shading", axis="columns").apply(DataFrame.agg, tuple, axis="columns")
/tmp/ipykernel_902997/2331890231.py:49: FutureWarning: Styler.applymap has been deprecated. Use Styler.map instead.
.applymap(_rgb_style) # convert each rgb tuple to inline html style
Mixed DataFrame Colormap
| shading | 0.00 | 0.11 | 0.22 | 0.33 | 0.44 | 0.56 | 0.67 | 0.78 | 0.89 | 1.00 |
|---|---|---|---|---|---|---|---|---|---|---|
| purple | (0, 0, 0) | (14, 0, 14) | (28, 0, 28) | (42, 0, 42) | (56, 0, 56) | (71, 0, 71) | (85, 0, 85) | (99, 0, 99) | (113, 0, 113) | (127, 0, 127) |
| turquoise | (0, 0, 0) | (0, 14, 14) | (0, 28, 28) | (0, 42, 42) | (0, 56, 56) | (0, 71, 71) | (0, 85, 85) | (0, 99, 99) | (0, 113, 113) | (0, 127, 127) |
| pink | (0, 0, 0) | (28, 14, 14) | (56, 28, 28) | (84, 42, 42) | (112, 56, 56) | (142, 71, 71) | (170, 85, 85) | (198, 99, 99) | (226, 113, 113) | (255, 127, 127) |
| orange | (0, 0, 0) | (28, 14, 0) | (56, 28, 0) | (84, 42, 0) | (112, 56, 0) | (142, 71, 0) | (170, 85, 0) | (198, 99, 0) | (226, 113, 0) | (255, 127, 0) |
| green | (0, 0, 0) | (9, 18, 9) | (18, 37, 18) | (28, 56, 28) | (37, 74, 37) | (47, 94, 47) | (56, 113, 56) | (66, 132, 66) | (75, 150, 75) | (85, 170, 85) |
Weighted Color Mixing#
What’s even more interesting than simply mixing two or more colors? Using weights!
If you looked closely at the mix_colors function, you’ll see that there is another argument we have yet to use! The weights parameter lets us control how much a single color should be represented in the final mixed color.
For example, if we wanted a pink that was more red than white, we could use something like:
shade_rgb_df.pipe(mix_colors, ["red", "white"], weights=[2, 1], name="dark pink")
shading component
0.00 r 0.000000
g 0.000000
b 0.000000
0.11 r 28.000000
g 9.333333
b 9.333333
0.22 r 56.000000
g 18.666667
b 18.666667
0.33 r 84.000000
g 28.000000
b 28.000000
0.44 r 112.000000
g 37.333333
b 37.333333
0.56 r 142.000000
g 47.333333
b 47.333333
0.67 r 170.000000
g 56.666667
b 56.666667
0.78 r 198.000000
g 66.000000
b 66.000000
0.89 r 226.000000
g 75.333333
b 75.333333
1.00 r 255.000000
g 85.000000
b 85.000000
Name: dark pink, dtype: float64
This will create a shade of pink who has twice as much red than white. Think of it a little like mixing paint: in this case we have a 2:1 ratio of red paint to white paint. We can use these weights in this fashion to create some very complex color spaces.
mixed_color_df = concat([
shade_rgb_df.pipe(mix_colors, ["red", "white"], weights=[0, 1], name="white"),
shade_rgb_df.pipe(mix_colors, ["red", "white"], weights=[1, 2], name="light pink"),
shade_rgb_df.pipe(mix_colors, ["red", "white"], weights=[1, 1], name="med pink"),
shade_rgb_df.pipe(mix_colors, ["red", "white"], weights=[2, 1], name="dark pink"),
shade_rgb_df.pipe(mix_colors, ["red", "white"], weights=[1, 0], name="red"),
], axis=1).T.astype(int)
display(
Markdown("**Sequential White-Red Color Space**"),
# rgb_font_size=0 will make the text disappear
mixed_color_df.pipe(display_colormap, rgb_font_size=0, cell_width=100)
)
/tmp/ipykernel_902997/2331890231.py:42: FutureWarning: DataFrame.groupby with axis=1 is deprecated. Do `frame.T.groupby(...)` without axis instead.
df.groupby("shading", axis="columns").apply(DataFrame.agg, tuple, axis="columns")
/tmp/ipykernel_902997/2331890231.py:49: FutureWarning: Styler.applymap has been deprecated. Use Styler.map instead.
.applymap(_rgb_style) # convert each rgb tuple to inline html style
Sequential White-Red Color Space
| shading | 0.00 | 0.11 | 0.22 | 0.33 | 0.44 | 0.56 | 0.67 | 0.78 | 0.89 | 1.00 |
|---|---|---|---|---|---|---|---|---|---|---|
| white | (0, 0, 0) | (28, 28, 28) | (56, 56, 56) | (84, 84, 84) | (112, 112, 112) | (142, 142, 142) | (170, 170, 170) | (198, 198, 198) | (226, 226, 226) | (255, 255, 255) |
| light pink | (0, 0, 0) | (28, 18, 18) | (56, 37, 37) | (84, 56, 56) | (112, 74, 74) | (142, 94, 94) | (170, 113, 113) | (198, 132, 132) | (226, 150, 150) | (255, 170, 170) |
| med pink | (0, 0, 0) | (28, 14, 14) | (56, 28, 28) | (84, 42, 42) | (112, 56, 56) | (142, 71, 71) | (170, 85, 85) | (198, 99, 99) | (226, 113, 113) | (255, 127, 127) |
| dark pink | (0, 0, 0) | (28, 9, 9) | (56, 18, 18) | (84, 28, 28) | (112, 37, 37) | (142, 47, 47) | (170, 56, 56) | (198, 66, 66) | (226, 75, 75) | (255, 85, 85) |
| red | (0, 0, 0) | (28, 0, 0) | (56, 0, 0) | (84, 0, 0) | (112, 0, 0) | (142, 0, 0) | (170, 0, 0) | (198, 0, 0) | (226, 0, 0) | (255, 0, 0) |
We can use weights in this fashion to fade one color into another. Let’s encapsulate this behavior in a function and see what we can do with it!
Instead of creating white to red, let’s map out all of the colors from red to blue at various levels of mixing. We can use this form of “mixing” to interpolate values between our basic rgb colors and fill out an entire color space!
def color_fade(df, l_color, r_color, n_steps=11):
fade_weights = arange(n_steps)
color_data = []
color_index = []
# calculate a weighted mixture for each color pair
# [(10, 0), (9, 1), (8, 2), ... (1, 9), (0, 10)]
for weights in zip(fade_weights[::-1], fade_weights):
# Get the Series for the current weighted mix
sequential_cmap = mix_colors(
df, [l_color, r_color], weights=weights, name=weights
)
# store our output for later concatenation
color_data.append(sequential_cmap)
color_index.append((l_color, r_color) + weights)
# Construct a meaningful index for the new color space
new_index = MultiIndex.from_tuples(
color_index, names=["color1", "color2", "weight1", "weight2"]
)
return (
concat(color_data, axis=1).T
.set_axis(new_index)
)
red_blue_fade_df = color_fade(shade_rgb_df, "red", "blue", n_steps=11)
display(
Markdown("**Sequential Red-Blue Color Space**"),
red_blue_fade_df.pipe(display_colormap, cell_height=50, cell_width=80),
)
/tmp/ipykernel_902997/2331890231.py:42: FutureWarning: DataFrame.groupby with axis=1 is deprecated. Do `frame.T.groupby(...)` without axis instead.
df.groupby("shading", axis="columns").apply(DataFrame.agg, tuple, axis="columns")
/tmp/ipykernel_902997/2331890231.py:49: FutureWarning: Styler.applymap has been deprecated. Use Styler.map instead.
.applymap(_rgb_style) # convert each rgb tuple to inline html style
Sequential Red-Blue Color Space
| shading | 0.00 | 0.11 | 0.22 | 0.33 | 0.44 | 0.56 | 0.67 | 0.78 | 0.89 | 1.00 | |||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| color1 | color2 | weight1 | weight2 | ||||||||||
| red | blue | 10 | 0 | (0.0, 0.0, 0.0) | (28.0, 0.0, 0.0) | (56.0, 0.0, 0.0) | (84.0, 0.0, 0.0) | (112.0, 0.0, 0.0) | (142.0, 0.0, 0.0) | (170.0, 0.0, 0.0) | (198.0, 0.0, 0.0) | (226.0, 0.0, 0.0) | (255.0, 0.0, 0.0) |
| 9 | 1 | (0.0, 0.0, 0.0) | (25.2, 0.0, 2.8) | (50.4, 0.0, 5.6) | (75.6, 0.0, 8.4) | (100.8, 0.0, 11.2) | (127.8, 0.0, 14.2) | (153.0, 0.0, 17.0) | (178.2, 0.0, 19.8) | (203.4, 0.0, 22.6) | (229.5, 0.0, 25.5) | ||
| 8 | 2 | (0.0, 0.0, 0.0) | (22.4, 0.0, 5.6) | (44.8, 0.0, 11.2) | (67.2, 0.0, 16.8) | (89.6, 0.0, 22.4) | (113.6, 0.0, 28.4) | (136.0, 0.0, 34.0) | (158.4, 0.0, 39.6) | (180.8, 0.0, 45.2) | (204.0, 0.0, 51.0) | ||
| 7 | 3 | (0.0, 0.0, 0.0) | (19.6, 0.0, 8.4) | (39.2, 0.0, 16.8) | (58.8, 0.0, 25.2) | (78.4, 0.0, 33.6) | (99.4, 0.0, 42.6) | (119.0, 0.0, 51.0) | (138.6, 0.0, 59.4) | (158.2, 0.0, 67.8) | (178.5, 0.0, 76.5) | ||
| 6 | 4 | (0.0, 0.0, 0.0) | (16.8, 0.0, 11.2) | (33.6, 0.0, 22.4) | (50.4, 0.0, 33.6) | (67.2, 0.0, 44.8) | (85.2, 0.0, 56.8) | (102.0, 0.0, 68.0) | (118.8, 0.0, 79.2) | (135.6, 0.0, 90.4) | (153.0, 0.0, 102.0) | ||
| 5 | 5 | (0.0, 0.0, 0.0) | (14.0, 0.0, 14.0) | (28.0, 0.0, 28.0) | (42.0, 0.0, 42.0) | (56.0, 0.0, 56.0) | (71.0, 0.0, 71.0) | (85.0, 0.0, 85.0) | (99.0, 0.0, 99.0) | (113.0, 0.0, 113.0) | (127.5, 0.0, 127.5) | ||
| 4 | 6 | (0.0, 0.0, 0.0) | (11.2, 0.0, 16.8) | (22.4, 0.0, 33.6) | (33.6, 0.0, 50.4) | (44.8, 0.0, 67.2) | (56.8, 0.0, 85.2) | (68.0, 0.0, 102.0) | (79.2, 0.0, 118.8) | (90.4, 0.0, 135.6) | (102.0, 0.0, 153.0) | ||
| 3 | 7 | (0.0, 0.0, 0.0) | (8.4, 0.0, 19.6) | (16.8, 0.0, 39.2) | (25.2, 0.0, 58.8) | (33.6, 0.0, 78.4) | (42.6, 0.0, 99.4) | (51.0, 0.0, 119.0) | (59.4, 0.0, 138.6) | (67.8, 0.0, 158.2) | (76.5, 0.0, 178.5) | ||
| 2 | 8 | (0.0, 0.0, 0.0) | (5.6, 0.0, 22.4) | (11.2, 0.0, 44.8) | (16.8, 0.0, 67.2) | (22.4, 0.0, 89.6) | (28.4, 0.0, 113.6) | (34.0, 0.0, 136.0) | (39.6, 0.0, 158.4) | (45.2, 0.0, 180.8) | (51.0, 0.0, 204.0) | ||
| 1 | 9 | (0.0, 0.0, 0.0) | (2.8, 0.0, 25.2) | (5.6, 0.0, 50.4) | (8.4, 0.0, 75.6) | (11.2, 0.0, 100.8) | (14.2, 0.0, 127.8) | (17.0, 0.0, 153.0) | (19.8, 0.0, 178.2) | (22.6, 0.0, 203.4) | (25.5, 0.0, 229.5) | ||
| 0 | 10 | (0.0, 0.0, 0.0) | (0.0, 0.0, 28.0) | (0.0, 0.0, 56.0) | (0.0, 0.0, 84.0) | (0.0, 0.0, 112.0) | (0.0, 0.0, 142.0) | (0.0, 0.0, 170.0) | (0.0, 0.0, 198.0) | (0.0, 0.0, 226.0) | (0.0, 0.0, 255.0) |
Well that’s neat! We now have a programmatic way to fade one color into another and visualize the interpolation of the color space!
But, why stop with two colors? With a little additional code, the fade function can chain together multiple pairs of colors to show a longer sequential_fade. Let’s try using the fade function to create a rainbow by linking together multiple fades.
from itertools import pairwise
def sequential_fade(df, colors, n_steps=11):
linear_color_space = []
# iterate through all sequential pairs of colors
# (1, 2, 3, 4) -> [(1, 2), (2, 3), (3, 4)]
for l_color, r_color in pairwise(colors):
bi_color_space = color_fade(df, l_color, r_color, n_steps=n_steps)
linear_color_space.append(bi_color_space)
return concat(linear_color_space)
# Construct a rainbow from mixing the following colors in pairwise order
rainbow_space = sequential_fade(
shade_rgb_df, ["red", "yellow", "lime", "blue", "magenta"], n_steps=7
)
display(
Markdown("**Sequential Red-Blue Color Space**"),
rainbow_space.pipe(display_colormap, cell_height=20, cell_width=70)
)
/tmp/ipykernel_902997/2331890231.py:42: FutureWarning: DataFrame.groupby with axis=1 is deprecated. Do `frame.T.groupby(...)` without axis instead.
df.groupby("shading", axis="columns").apply(DataFrame.agg, tuple, axis="columns")
/tmp/ipykernel_902997/2331890231.py:49: FutureWarning: Styler.applymap has been deprecated. Use Styler.map instead.
.applymap(_rgb_style) # convert each rgb tuple to inline html style
Sequential Red-Blue Color Space
| shading | 0.00 | 0.11 | 0.22 | 0.33 | 0.44 | 0.56 | 0.67 | 0.78 | 0.89 | 1.00 | |||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| color1 | color2 | weight1 | weight2 | ||||||||||
| red | yellow | 6 | 0 | (0.0, 0.0, 0.0) | (28.0, 0.0, 0.0) | (56.0, 0.0, 0.0) | (84.0, 0.0, 0.0) | (112.0, 0.0, 0.0) | (142.0, 0.0, 0.0) | (170.0, 0.0, 0.0) | (198.0, 0.0, 0.0) | (226.0, 0.0, 0.0) | (255.0, 0.0, 0.0) |
| 5 | 1 | (0.0, 0.0, 0.0) | (28.0, 4.666666666666667, 0.0) | (56.0, 9.333333333333334, 0.0) | (84.0, 14.0, 0.0) | (112.0, 18.666666666666668, 0.0) | (142.0, 23.666666666666668, 0.0) | (170.0, 28.333333333333332, 0.0) | (198.0, 33.0, 0.0) | (226.0, 37.666666666666664, 0.0) | (255.0, 42.5, 0.0) | ||
| 4 | 2 | (0.0, 0.0, 0.0) | (28.0, 9.333333333333334, 0.0) | (56.0, 18.666666666666668, 0.0) | (84.0, 28.0, 0.0) | (112.0, 37.333333333333336, 0.0) | (142.0, 47.333333333333336, 0.0) | (170.0, 56.666666666666664, 0.0) | (198.0, 66.0, 0.0) | (226.0, 75.33333333333333, 0.0) | (255.0, 85.0, 0.0) | ||
| 3 | 3 | (0.0, 0.0, 0.0) | (28.0, 14.0, 0.0) | (56.0, 28.0, 0.0) | (84.0, 42.0, 0.0) | (112.0, 56.0, 0.0) | (142.0, 71.0, 0.0) | (170.0, 85.0, 0.0) | (198.0, 99.0, 0.0) | (226.0, 113.0, 0.0) | (255.0, 127.5, 0.0) | ||
| 2 | 4 | (0.0, 0.0, 0.0) | (28.0, 18.666666666666668, 0.0) | (56.0, 37.333333333333336, 0.0) | (84.0, 56.0, 0.0) | (112.0, 74.66666666666667, 0.0) | (142.0, 94.66666666666667, 0.0) | (170.0, 113.33333333333333, 0.0) | (198.0, 132.0, 0.0) | (226.0, 150.66666666666666, 0.0) | (255.0, 170.0, 0.0) | ||
| 1 | 5 | (0.0, 0.0, 0.0) | (28.0, 23.333333333333332, 0.0) | (56.0, 46.666666666666664, 0.0) | (84.0, 70.0, 0.0) | (112.0, 93.33333333333333, 0.0) | (142.0, 118.33333333333333, 0.0) | (170.0, 141.66666666666666, 0.0) | (198.0, 165.0, 0.0) | (226.0, 188.33333333333334, 0.0) | (255.0, 212.5, 0.0) | ||
| 0 | 6 | (0.0, 0.0, 0.0) | (28.0, 28.0, 0.0) | (56.0, 56.0, 0.0) | (84.0, 84.0, 0.0) | (112.0, 112.0, 0.0) | (142.0, 142.0, 0.0) | (170.0, 170.0, 0.0) | (198.0, 198.0, 0.0) | (226.0, 226.0, 0.0) | (255.0, 255.0, 0.0) | ||
| yellow | lime | 6 | 0 | (0.0, 0.0, 0.0) | (28.0, 28.0, 0.0) | (56.0, 56.0, 0.0) | (84.0, 84.0, 0.0) | (112.0, 112.0, 0.0) | (142.0, 142.0, 0.0) | (170.0, 170.0, 0.0) | (198.0, 198.0, 0.0) | (226.0, 226.0, 0.0) | (255.0, 255.0, 0.0) |
| 5 | 1 | (0.0, 0.0, 0.0) | (23.333333333333332, 28.0, 0.0) | (46.666666666666664, 56.0, 0.0) | (70.0, 84.0, 0.0) | (93.33333333333333, 112.0, 0.0) | (118.33333333333333, 142.0, 0.0) | (141.66666666666666, 170.0, 0.0) | (165.0, 198.0, 0.0) | (188.33333333333334, 226.0, 0.0) | (212.5, 255.0, 0.0) | ||
| 4 | 2 | (0.0, 0.0, 0.0) | (18.666666666666668, 28.0, 0.0) | (37.333333333333336, 56.0, 0.0) | (56.0, 84.0, 0.0) | (74.66666666666667, 112.0, 0.0) | (94.66666666666667, 142.0, 0.0) | (113.33333333333333, 170.0, 0.0) | (132.0, 198.0, 0.0) | (150.66666666666666, 226.0, 0.0) | (170.0, 255.0, 0.0) | ||
| 3 | 3 | (0.0, 0.0, 0.0) | (14.0, 28.0, 0.0) | (28.0, 56.0, 0.0) | (42.0, 84.0, 0.0) | (56.0, 112.0, 0.0) | (71.0, 142.0, 0.0) | (85.0, 170.0, 0.0) | (99.0, 198.0, 0.0) | (113.0, 226.0, 0.0) | (127.5, 255.0, 0.0) | ||
| 2 | 4 | (0.0, 0.0, 0.0) | (9.333333333333334, 28.0, 0.0) | (18.666666666666668, 56.0, 0.0) | (28.0, 84.0, 0.0) | (37.333333333333336, 112.0, 0.0) | (47.333333333333336, 142.0, 0.0) | (56.666666666666664, 170.0, 0.0) | (66.0, 198.0, 0.0) | (75.33333333333333, 226.0, 0.0) | (85.0, 255.0, 0.0) | ||
| 1 | 5 | (0.0, 0.0, 0.0) | (4.666666666666667, 28.0, 0.0) | (9.333333333333334, 56.0, 0.0) | (14.0, 84.0, 0.0) | (18.666666666666668, 112.0, 0.0) | (23.666666666666668, 142.0, 0.0) | (28.333333333333332, 170.0, 0.0) | (33.0, 198.0, 0.0) | (37.666666666666664, 226.0, 0.0) | (42.5, 255.0, 0.0) | ||
| 0 | 6 | (0.0, 0.0, 0.0) | (0.0, 28.0, 0.0) | (0.0, 56.0, 0.0) | (0.0, 84.0, 0.0) | (0.0, 112.0, 0.0) | (0.0, 142.0, 0.0) | (0.0, 170.0, 0.0) | (0.0, 198.0, 0.0) | (0.0, 226.0, 0.0) | (0.0, 255.0, 0.0) | ||
| lime | blue | 6 | 0 | (0.0, 0.0, 0.0) | (0.0, 28.0, 0.0) | (0.0, 56.0, 0.0) | (0.0, 84.0, 0.0) | (0.0, 112.0, 0.0) | (0.0, 142.0, 0.0) | (0.0, 170.0, 0.0) | (0.0, 198.0, 0.0) | (0.0, 226.0, 0.0) | (0.0, 255.0, 0.0) |
| 5 | 1 | (0.0, 0.0, 0.0) | (0.0, 23.333333333333332, 4.666666666666667) | (0.0, 46.666666666666664, 9.333333333333334) | (0.0, 70.0, 14.0) | (0.0, 93.33333333333333, 18.666666666666668) | (0.0, 118.33333333333333, 23.666666666666668) | (0.0, 141.66666666666666, 28.333333333333332) | (0.0, 165.0, 33.0) | (0.0, 188.33333333333334, 37.666666666666664) | (0.0, 212.5, 42.5) | ||
| 4 | 2 | (0.0, 0.0, 0.0) | (0.0, 18.666666666666668, 9.333333333333334) | (0.0, 37.333333333333336, 18.666666666666668) | (0.0, 56.0, 28.0) | (0.0, 74.66666666666667, 37.333333333333336) | (0.0, 94.66666666666667, 47.333333333333336) | (0.0, 113.33333333333333, 56.666666666666664) | (0.0, 132.0, 66.0) | (0.0, 150.66666666666666, 75.33333333333333) | (0.0, 170.0, 85.0) | ||
| 3 | 3 | (0.0, 0.0, 0.0) | (0.0, 14.0, 14.0) | (0.0, 28.0, 28.0) | (0.0, 42.0, 42.0) | (0.0, 56.0, 56.0) | (0.0, 71.0, 71.0) | (0.0, 85.0, 85.0) | (0.0, 99.0, 99.0) | (0.0, 113.0, 113.0) | (0.0, 127.5, 127.5) | ||
| 2 | 4 | (0.0, 0.0, 0.0) | (0.0, 9.333333333333334, 18.666666666666668) | (0.0, 18.666666666666668, 37.333333333333336) | (0.0, 28.0, 56.0) | (0.0, 37.333333333333336, 74.66666666666667) | (0.0, 47.333333333333336, 94.66666666666667) | (0.0, 56.666666666666664, 113.33333333333333) | (0.0, 66.0, 132.0) | (0.0, 75.33333333333333, 150.66666666666666) | (0.0, 85.0, 170.0) | ||
| 1 | 5 | (0.0, 0.0, 0.0) | (0.0, 4.666666666666667, 23.333333333333332) | (0.0, 9.333333333333334, 46.666666666666664) | (0.0, 14.0, 70.0) | (0.0, 18.666666666666668, 93.33333333333333) | (0.0, 23.666666666666668, 118.33333333333333) | (0.0, 28.333333333333332, 141.66666666666666) | (0.0, 33.0, 165.0) | (0.0, 37.666666666666664, 188.33333333333334) | (0.0, 42.5, 212.5) | ||
| 0 | 6 | (0.0, 0.0, 0.0) | (0.0, 0.0, 28.0) | (0.0, 0.0, 56.0) | (0.0, 0.0, 84.0) | (0.0, 0.0, 112.0) | (0.0, 0.0, 142.0) | (0.0, 0.0, 170.0) | (0.0, 0.0, 198.0) | (0.0, 0.0, 226.0) | (0.0, 0.0, 255.0) | ||
| blue | magenta | 6 | 0 | (0.0, 0.0, 0.0) | (0.0, 0.0, 28.0) | (0.0, 0.0, 56.0) | (0.0, 0.0, 84.0) | (0.0, 0.0, 112.0) | (0.0, 0.0, 142.0) | (0.0, 0.0, 170.0) | (0.0, 0.0, 198.0) | (0.0, 0.0, 226.0) | (0.0, 0.0, 255.0) |
| 5 | 1 | (0.0, 0.0, 0.0) | (4.666666666666667, 0.0, 28.0) | (9.333333333333334, 0.0, 56.0) | (14.0, 0.0, 84.0) | (18.666666666666668, 0.0, 112.0) | (23.666666666666668, 0.0, 142.0) | (28.333333333333332, 0.0, 170.0) | (33.0, 0.0, 198.0) | (37.666666666666664, 0.0, 226.0) | (42.5, 0.0, 255.0) | ||
| 4 | 2 | (0.0, 0.0, 0.0) | (9.333333333333334, 0.0, 28.0) | (18.666666666666668, 0.0, 56.0) | (28.0, 0.0, 84.0) | (37.333333333333336, 0.0, 112.0) | (47.333333333333336, 0.0, 142.0) | (56.666666666666664, 0.0, 170.0) | (66.0, 0.0, 198.0) | (75.33333333333333, 0.0, 226.0) | (85.0, 0.0, 255.0) | ||
| 3 | 3 | (0.0, 0.0, 0.0) | (14.0, 0.0, 28.0) | (28.0, 0.0, 56.0) | (42.0, 0.0, 84.0) | (56.0, 0.0, 112.0) | (71.0, 0.0, 142.0) | (85.0, 0.0, 170.0) | (99.0, 0.0, 198.0) | (113.0, 0.0, 226.0) | (127.5, 0.0, 255.0) | ||
| 2 | 4 | (0.0, 0.0, 0.0) | (18.666666666666668, 0.0, 28.0) | (37.333333333333336, 0.0, 56.0) | (56.0, 0.0, 84.0) | (74.66666666666667, 0.0, 112.0) | (94.66666666666667, 0.0, 142.0) | (113.33333333333333, 0.0, 170.0) | (132.0, 0.0, 198.0) | (150.66666666666666, 0.0, 226.0) | (170.0, 0.0, 255.0) | ||
| 1 | 5 | (0.0, 0.0, 0.0) | (23.333333333333332, 0.0, 28.0) | (46.666666666666664, 0.0, 56.0) | (70.0, 0.0, 84.0) | (93.33333333333333, 0.0, 112.0) | (118.33333333333333, 0.0, 142.0) | (141.66666666666666, 0.0, 170.0) | (165.0, 0.0, 198.0) | (188.33333333333334, 0.0, 226.0) | (212.5, 0.0, 255.0) | ||
| 0 | 6 | (0.0, 0.0, 0.0) | (28.0, 0.0, 28.0) | (56.0, 0.0, 56.0) | (84.0, 0.0, 84.0) | (112.0, 0.0, 112.0) | (142.0, 0.0, 142.0) | (170.0, 0.0, 170.0) | (198.0, 0.0, 198.0) | (226.0, 0.0, 226.0) | (255.0, 0.0, 255.0) |
Wrap-Up#
And there you have it: creating, managing, and combining colormaps, all done in pandas.
If you have any insights or questions, feel free to tag me in a post on Twitter.
Until next time!