Unconventional Pandas: Colormaps#

Hello everyone! We have some exciting events coming up, including a NEW seminar series and a code review workshop series. In our brand new seminar series, we will share with you some of the hardest problems we have had to solve in pandas and NumPy (and, in our bonus session on September 16th, hard problems that we have had to solve in Matplotlib!). Then, next month starting October 12th, we will be holding our first ever “No Tears Code Review,” where we’ll take attendees througha a code review that will actually help them gain insight into their code and cause meaningful improvements to their approach.

Let’s get to the exciting content!

A response to “Try This: An Unconventional Pandas Tutorial”#

A little while ago, I stumbled across a Quantsight blog post by Eric Kelly & Tony Fast titled, “Try This: An Unconventional Pandas Tutorial”. (Unfortunately, at the time of writing this, the post was either taken down or moved, which is why I didn’t link directly to it.) The authors achieved what they set out to do, namely to demonstrate a novel use of pandas, outside of common data analysis. I loved the post, but thought to myself, “have they really demonstrated the power of pandas?”

After conferring with the authors of the original post, we agreed that it would be good to write a response of my own to build on top of what they created and further highlight the power and flexibility of the most popular data analysis Python has to offer.

Our goal here is to create a colormap primarily using pandas and some html rendering. We want to replicate the following from the aforementioned blog post.

The true superpower of pandas lies in its Index object. This object is considered in nearly all pandas methods, and it governs how data is aligned. For example, we can use the index of a Series to map the data from that Series to another datastructure. Subsequently, we can think of a DataFrame as a container of Series objects that all share the same Index.

Before we get to the code, let’s lay out some steps to create this colormap:

  1. Make base data structures

    1. Colors x RGB values (N x 3)

    2. Levels of shading

  2. Combine these data structures

  3. Render the resultant dataframe as colormap

Let’s go ahead and load in the tools we’ll need to use:

from IPython.display import display, Markdown
from webcolors import rgb_to_name
from pandas import DataFrame, Series, MultiIndex, concat
from numpy import linspace, arange, where, asarray

from itertools import product as iter_product

Make base data structures#

Let’s tackle the core pieces of our colormap. As I mentioned earlier, we want to make 2 data structures:

  1. Colors x RGB values (N x 3)

  2. Levels of shading

Our Colors x RGB values will be a DataFrame where each row is a color, and each column corresponds to the rgb values that constitute that color.

We also have to consider a shading array that will be used to scale the rgb values. This will be represented by a Series.

values = [0, 255]
shading = Series(linspace(0, 1, num=10).round(2), name="shading")

rgb_df = (
    DataFrame(
        data=iter_product(values, repeat=3),
        columns=["r", "g", "b"]
    )
    .pipe(lambda df: df.set_axis(df.apply(rgb_to_name, axis="columns")))
    .rename_axis(index="name", columns="component")
)

display(
    Markdown("**Levels of Shading**"),
    shading,
    Markdown("<br>**RGB DataFame**"),
    rgb_df
)

Levels of Shading

0    0.00
1    0.11
2    0.22
3    0.33
4    0.44
5    0.56
6    0.67
7    0.78
8    0.89
9    1.00
Name: shading, dtype: float64


RGB DataFame

component r g b
name
black 0 0 0
blue 0 0 255
lime 0 255 0
cyan 0 255 255
red 255 0 0
magenta 255 0 255
yellow 255 255 0
white 255 255 255

Combining Base Data Structures#

Now that we have our base data structures sorted out, we have to think of how we want to combine these to replicate the colormap at the top of this post. We then want to multiply the rgb values by the degree of shading, such that the rgb values composing red (255, 0, 0) become:

Shading example table: red only (255, 0, 0)

from numpy import multiply

DataFrame(
    multiply.outer(shading.to_numpy(), rgb_df.loc['red', :].to_numpy()),
    index=shading,
    columns=rgb_df.columns
)
component r g b
shading
0.00 0.00 0.0 0.0
0.11 28.05 0.0 0.0
0.22 56.10 0.0 0.0
0.33 84.15 0.0 0.0
0.44 112.20 0.0 0.0
0.56 142.80 0.0 0.0
0.67 170.85 0.0 0.0
0.78 198.90 0.0 0.0
0.89 226.95 0.0 0.0
1.00 255.00 0.0 0.0

To achieve this, we can use a MultiIndex to generate all combinations of rgb values at each level of shading. Then, we can use .reindex to achieve label-aligned titling of our data. Lastly, we can multiply the tiled rgb values by their corresponding levels of shading that we’ve aligned them to. We end up with a structured DataFrame of “color by (shading x rgb)”.

For you NumPy fans out there, this result is also attainable in a single step using np.multiply.outer to return a 3d array (or 2d array if you ravel’d the input) that you put into a hierarchical DataFrame, like the one outputted below.

shade_rgb_index = MultiIndex.from_product([shading, rgb_df.columns])

shade_rgb_df = (
    rgb_df.reindex(columns=shade_rgb_index, level=1)
    .mul(shade_rgb_index.get_level_values("shading"))
    .astype(int)
)

display(Markdown("**Combined Colormap**"), shade_rgb_df)

Combined Colormap

shading 0.00 0.11 0.22 0.33 ... 0.67 0.78 0.89 1.00
component r g b r g b r g b r ... b r g b r g b r g b
name
black 0 0 0 0 0 0 0 0 0 0 ... 0 0 0 0 0 0 0 0 0 0
blue 0 0 0 0 0 28 0 0 56 0 ... 170 0 0 198 0 0 226 0 0 255
lime 0 0 0 0 28 0 0 56 0 0 ... 0 0 198 0 0 226 0 0 255 0
cyan 0 0 0 0 28 28 0 56 56 0 ... 170 0 198 198 0 226 226 0 255 255
red 0 0 0 28 0 0 56 0 0 84 ... 0 198 0 0 226 0 0 255 0 0
magenta 0 0 0 28 0 28 56 0 56 84 ... 170 198 0 198 226 0 226 255 0 255
yellow 0 0 0 28 28 0 56 56 0 84 ... 0 198 198 0 226 226 0 255 255 0
white 0 0 0 28 28 28 56 56 56 84 ... 170 198 198 198 226 226 226 255 255 255

8 rows × 30 columns

Display Our Colormap#

What enables us to use pandas in this unconventional way is its powerful styling interface. This interface lets us control how DataFrames are rendered into HTML, which is useful if you’re working within a Jupyter notebook or planning to push output from a DataFrame into any web front-end.

I know the display_colormap function is a bit long, but I wanted to highlight the utility of DataFrame styling by adding many options to consider when reporting your DataFrame. DataFrame.style is extremely flexible, and it can require a little bit of digging to use effectively.

def display_colormap(
    df, header_font_size=15, rgb_font_size=0, cell_width=80, cell_height=60
):
    """Displays RGB colors in jupyter
    """
    
    def _rgb_style(rgb_values):
        """rgb tuple -> inline html styling
        """
        # Thank you @seaborn.utils/relative_luminence
        rgb = asarray(rgb_values) / 255
        rgb = where(rgb <= .03928, rgb / 12.92, ((rgb + .055) / 1.055) ** 2.4)
        lum = rgb.dot([.2126, .7152, .0722]).item()
        color = "black" if lum > .408 else "white"
        
        rgb_css = ", ".join(map(str, rgb_values))
        return f"""
            background-color: rgb({rgb_css});
            color: {color};
            """

    ### Make outputted table look nice
    # - 'selector' determines the element that 'props' will be applied to
    # - `.index_name` refers to the name of the column and row index
    #      specifically refers to "shading" and "name" in our example
    header_css = {
        'selector': 'th:not(.index_name)', # all headers that are NOT "shading" or "name"
        'props': f'font-size: {header_font_size}px;'
    }
    name_css = {
        'selector': '.index_name',
        'props': f'font-size: 20px;'
    }
    tablecell_css = {
        "width": f"{cell_width}px",
        "height": f"{cell_height}px", 
        "font-size": f"{rgb_font_size}px",
    }
    
    return (
        # combines all rgb values into tuples as prep for _rgb_style function
        df.groupby("shading", axis="columns").apply(DataFrame.agg, tuple, axis="columns")
        .pipe(lambda d: d.set_axis(d.columns.map('{:.2f}'.format), axis="columns"))
        .style
            .set_table_styles(
                [header_css, name_css]
            )                                 # add table-level css
            .set_properties(**tablecell_css)  # set td css properties
            .applymap(_rgb_style)             # convert each rgb tuple to inline html style
    )

display(
    Markdown("**DataFrame Colormap**"),
    shade_rgb_df.pipe(display_colormap, rgb_font_size=15, cell_width=100, cell_height=60)
)
/tmp/ipykernel_902997/2331890231.py:42: FutureWarning: DataFrame.groupby with axis=1 is deprecated. Do `frame.T.groupby(...)` without axis instead.
  df.groupby("shading", axis="columns").apply(DataFrame.agg, tuple, axis="columns")
/tmp/ipykernel_902997/2331890231.py:49: FutureWarning: Styler.applymap has been deprecated. Use Styler.map instead.
  .applymap(_rgb_style)             # convert each rgb tuple to inline html style

DataFrame Colormap

shading 0.00 0.11 0.22 0.33 0.44 0.56 0.67 0.78 0.89 1.00
name                    
black (0, 0, 0) (0, 0, 0) (0, 0, 0) (0, 0, 0) (0, 0, 0) (0, 0, 0) (0, 0, 0) (0, 0, 0) (0, 0, 0) (0, 0, 0)
blue (0, 0, 0) (0, 0, 28) (0, 0, 56) (0, 0, 84) (0, 0, 112) (0, 0, 142) (0, 0, 170) (0, 0, 198) (0, 0, 226) (0, 0, 255)
lime (0, 0, 0) (0, 28, 0) (0, 56, 0) (0, 84, 0) (0, 112, 0) (0, 142, 0) (0, 170, 0) (0, 198, 0) (0, 226, 0) (0, 255, 0)
cyan (0, 0, 0) (0, 28, 28) (0, 56, 56) (0, 84, 84) (0, 112, 112) (0, 142, 142) (0, 170, 170) (0, 198, 198) (0, 226, 226) (0, 255, 255)
red (0, 0, 0) (28, 0, 0) (56, 0, 0) (84, 0, 0) (112, 0, 0) (142, 0, 0) (170, 0, 0) (198, 0, 0) (226, 0, 0) (255, 0, 0)
magenta (0, 0, 0) (28, 0, 28) (56, 0, 56) (84, 0, 84) (112, 0, 112) (142, 0, 142) (170, 0, 170) (198, 0, 198) (226, 0, 226) (255, 0, 255)
yellow (0, 0, 0) (28, 28, 0) (56, 56, 0) (84, 84, 0) (112, 112, 0) (142, 142, 0) (170, 170, 0) (198, 198, 0) (226, 226, 0) (255, 255, 0)
white (0, 0, 0) (28, 28, 28) (56, 56, 56) (84, 84, 84) (112, 112, 112) (142, 142, 142) (170, 170, 170) (198, 198, 198) (226, 226, 226) (255, 255, 255)

Thanks to pandas.loc’s label-based slicing syntax, we can subset this color map in a very human-readable way:

# Rows:    ['lime', 'cyan', 'yellow']
# Columns: all columns
shade_rgb_df.loc[["lime", "cyan", "yellow"], :].pipe(display_colormap)
/tmp/ipykernel_902997/2331890231.py:42: FutureWarning: DataFrame.groupby with axis=1 is deprecated. Do `frame.T.groupby(...)` without axis instead.
  df.groupby("shading", axis="columns").apply(DataFrame.agg, tuple, axis="columns")
/tmp/ipykernel_902997/2331890231.py:49: FutureWarning: Styler.applymap has been deprecated. Use Styler.map instead.
  .applymap(_rgb_style)             # convert each rgb tuple to inline html style
shading 0.00 0.11 0.22 0.33 0.44 0.56 0.67 0.78 0.89 1.00
name                    
lime (0, 0, 0) (0, 28, 0) (0, 56, 0) (0, 84, 0) (0, 112, 0) (0, 142, 0) (0, 170, 0) (0, 198, 0) (0, 226, 0) (0, 255, 0)
cyan (0, 0, 0) (0, 28, 28) (0, 56, 56) (0, 84, 84) (0, 112, 112) (0, 142, 142) (0, 170, 170) (0, 198, 198) (0, 226, 226) (0, 255, 255)
yellow (0, 0, 0) (28, 28, 0) (56, 56, 0) (84, 84, 0) (112, 112, 0) (142, 142, 0) (170, 170, 0) (198, 198, 0) (226, 226, 0) (255, 255, 0)
# Rows:    Slice rows from blue to red"
# Columns: Columns with shading >= 0.5
shade_rgb_df.loc["blue":"red", 0.5:].pipe(display_colormap)
/tmp/ipykernel_902997/2331890231.py:42: FutureWarning: DataFrame.groupby with axis=1 is deprecated. Do `frame.T.groupby(...)` without axis instead.
  df.groupby("shading", axis="columns").apply(DataFrame.agg, tuple, axis="columns")
/tmp/ipykernel_902997/2331890231.py:49: FutureWarning: Styler.applymap has been deprecated. Use Styler.map instead.
  .applymap(_rgb_style)             # convert each rgb tuple to inline html style
shading 0.56 0.67 0.78 0.89 1.00
name          
blue (0, 0, 142) (0, 0, 170) (0, 0, 198) (0, 0, 226) (0, 0, 255)
lime (0, 142, 0) (0, 170, 0) (0, 198, 0) (0, 226, 0) (0, 255, 0)
cyan (0, 142, 142) (0, 170, 170) (0, 198, 198) (0, 226, 226) (0, 255, 255)
red (142, 0, 0) (170, 0, 0) (198, 0, 0) (226, 0, 0) (255, 0, 0)

We can even mix colors! Simply taking the .mean() of 2 or more colors will average their RGB values.

# Color Mixing
(
    shade_rgb_df.loc[["blue", "red"], :].mean()  # average together rgb values of "blue" and "red"
    .to_frame("purple").T                        # reshape our DataFrame to be used by `display_colormap`
    .pipe(display_colormap)                      # display colors instead of rgb values
)
/tmp/ipykernel_902997/2331890231.py:42: FutureWarning: DataFrame.groupby with axis=1 is deprecated. Do `frame.T.groupby(...)` without axis instead.
  df.groupby("shading", axis="columns").apply(DataFrame.agg, tuple, axis="columns")
/tmp/ipykernel_902997/2331890231.py:49: FutureWarning: Styler.applymap has been deprecated. Use Styler.map instead.
  .applymap(_rgb_style)             # convert each rgb tuple to inline html style
shading 0.00 0.11 0.22 0.33 0.44 0.56 0.67 0.78 0.89 1.00
purple (0.0, 0.0, 0.0) (14.0, 0.0, 14.0) (28.0, 0.0, 28.0) (42.0, 0.0, 42.0) (56.0, 0.0, 56.0) (71.0, 0.0, 71.0) (85.0, 0.0, 85.0) (99.0, 0.0, 99.0) (113.0, 0.0, 113.0) (127.5, 0.0, 127.5)

Now that we have a way to easily visualize our colormap DataFrame, we can begin to envision what we can do with it. As per the example above, we know that mixing red and blue should give us a shade of purple. Mathematically, we can represent mixing two colors as taking the mean across their corresponding RGB values. The fun part is that we can mix as many of these base colors together as we want!

Let’s define a custom mix_colors function so that we don’t need to repeat the long chain of commands above.

Mixing Colors from our colormap#

def mix_colors(df, colors, weights=None, name=None):
    """Mixes 2 colors from `df` into a Series
    """
    if weights is None:
        weights = [1] * len(colors)
        
    return (
        shade_rgb_df
        .loc[colors, :]        # Select rows from supplied color names
        .T.dot(weights).div(sum(weights)) # calculate weighted average
        .rename(name)
   )

# we can use `mix_colors` to calculate the average rgb values among 2 or more colors
display(
    # our display_colormap function expects input in the following format
    shade_rgb_df.pipe(mix_colors, ["red",  "blue"], name="purple").to_frame().T,
    shade_rgb_df.pipe(mix_colors, ["blue",  "lime"], name="turquoise").to_frame().T
)
shading 0.00 0.11 0.22 0.33 ... 0.67 0.78 0.89 1.00
component r g b r g b r g b r ... b r g b r g b r g b
purple 0.0 0.0 0.0 14.0 0.0 14.0 28.0 0.0 28.0 42.0 ... 85.0 99.0 0.0 99.0 113.0 0.0 113.0 127.5 0.0 127.5

1 rows × 30 columns

shading 0.00 0.11 0.22 0.33 ... 0.67 0.78 0.89 1.00
component r g b r g b r g b r ... b r g b r g b r g b
turquoise 0.0 0.0 0.0 0.0 14.0 14.0 0.0 28.0 28.0 0.0 ... 85.0 0.0 99.0 99.0 0.0 113.0 113.0 0.0 127.5 127.5

1 rows × 30 columns

mix_colors takes rows from our inputted DataFrame to calculate their averaged rgb values and outputs a single Series. If we make a number of these Series and concatenate them, we can derive a new color map based on combinations of the original.

mixed_color_df = concat([
    shade_rgb_df.pipe(mix_colors, ["red",  "blue"], name="purple"),
    shade_rgb_df.pipe(mix_colors, ["blue", "lime"],  name="turquoise"),
    shade_rgb_df.pipe(mix_colors, ["red",  "white"], name="pink"),
    shade_rgb_df.pipe(mix_colors, ["red",  "yellow"], name="orange"),
    
    # complex color mixed from 3 inputs!
    shade_rgb_df.pipe(mix_colors, ["blue",  "lime", "yellow"], name="green"),
], axis=1).T.astype(int)

display(
    Markdown("**Mixed DataFrame Colormap**"), 
    mixed_color_df.pipe(display_colormap, rgb_font_size=14, cell_width=100)
)
/tmp/ipykernel_902997/2331890231.py:42: FutureWarning: DataFrame.groupby with axis=1 is deprecated. Do `frame.T.groupby(...)` without axis instead.
  df.groupby("shading", axis="columns").apply(DataFrame.agg, tuple, axis="columns")
/tmp/ipykernel_902997/2331890231.py:49: FutureWarning: Styler.applymap has been deprecated. Use Styler.map instead.
  .applymap(_rgb_style)             # convert each rgb tuple to inline html style

Mixed DataFrame Colormap

shading 0.00 0.11 0.22 0.33 0.44 0.56 0.67 0.78 0.89 1.00
purple (0, 0, 0) (14, 0, 14) (28, 0, 28) (42, 0, 42) (56, 0, 56) (71, 0, 71) (85, 0, 85) (99, 0, 99) (113, 0, 113) (127, 0, 127)
turquoise (0, 0, 0) (0, 14, 14) (0, 28, 28) (0, 42, 42) (0, 56, 56) (0, 71, 71) (0, 85, 85) (0, 99, 99) (0, 113, 113) (0, 127, 127)
pink (0, 0, 0) (28, 14, 14) (56, 28, 28) (84, 42, 42) (112, 56, 56) (142, 71, 71) (170, 85, 85) (198, 99, 99) (226, 113, 113) (255, 127, 127)
orange (0, 0, 0) (28, 14, 0) (56, 28, 0) (84, 42, 0) (112, 56, 0) (142, 71, 0) (170, 85, 0) (198, 99, 0) (226, 113, 0) (255, 127, 0)
green (0, 0, 0) (9, 18, 9) (18, 37, 18) (28, 56, 28) (37, 74, 37) (47, 94, 47) (56, 113, 56) (66, 132, 66) (75, 150, 75) (85, 170, 85)

Weighted Color Mixing#

What’s even more interesting than simply mixing two or more colors? Using weights!

If you looked closely at the mix_colors function, you’ll see that there is another argument we have yet to use! The weights parameter lets us control how much a single color should be represented in the final mixed color.

For example, if we wanted a pink that was more red than white, we could use something like:

shade_rgb_df.pipe(mix_colors, ["red",  "white"], weights=[2, 1], name="dark pink")
shading  component
0.00     r              0.000000
         g              0.000000
         b              0.000000
0.11     r             28.000000
         g              9.333333
         b              9.333333
0.22     r             56.000000
         g             18.666667
         b             18.666667
0.33     r             84.000000
         g             28.000000
         b             28.000000
0.44     r            112.000000
         g             37.333333
         b             37.333333
0.56     r            142.000000
         g             47.333333
         b             47.333333
0.67     r            170.000000
         g             56.666667
         b             56.666667
0.78     r            198.000000
         g             66.000000
         b             66.000000
0.89     r            226.000000
         g             75.333333
         b             75.333333
1.00     r            255.000000
         g             85.000000
         b             85.000000
Name: dark pink, dtype: float64

This will create a shade of pink who has twice as much red than white. Think of it a little like mixing paint: in this case we have a 2:1 ratio of red paint to white paint. We can use these weights in this fashion to create some very complex color spaces.

mixed_color_df = concat([
    shade_rgb_df.pipe(mix_colors, ["red",  "white"], weights=[0, 1], name="white"),
    shade_rgb_df.pipe(mix_colors, ["red",  "white"], weights=[1, 2], name="light pink"),
    shade_rgb_df.pipe(mix_colors, ["red",  "white"], weights=[1, 1], name="med pink"),
    shade_rgb_df.pipe(mix_colors, ["red",  "white"], weights=[2, 1], name="dark pink"),
    shade_rgb_df.pipe(mix_colors, ["red",  "white"], weights=[1, 0], name="red"),
], axis=1).T.astype(int)

display(
    Markdown("**Sequential White-Red Color Space**"), 
    # rgb_font_size=0 will make the text disappear
    mixed_color_df.pipe(display_colormap, rgb_font_size=0, cell_width=100)
)
/tmp/ipykernel_902997/2331890231.py:42: FutureWarning: DataFrame.groupby with axis=1 is deprecated. Do `frame.T.groupby(...)` without axis instead.
  df.groupby("shading", axis="columns").apply(DataFrame.agg, tuple, axis="columns")
/tmp/ipykernel_902997/2331890231.py:49: FutureWarning: Styler.applymap has been deprecated. Use Styler.map instead.
  .applymap(_rgb_style)             # convert each rgb tuple to inline html style

Sequential White-Red Color Space

shading 0.00 0.11 0.22 0.33 0.44 0.56 0.67 0.78 0.89 1.00
white (0, 0, 0) (28, 28, 28) (56, 56, 56) (84, 84, 84) (112, 112, 112) (142, 142, 142) (170, 170, 170) (198, 198, 198) (226, 226, 226) (255, 255, 255)
light pink (0, 0, 0) (28, 18, 18) (56, 37, 37) (84, 56, 56) (112, 74, 74) (142, 94, 94) (170, 113, 113) (198, 132, 132) (226, 150, 150) (255, 170, 170)
med pink (0, 0, 0) (28, 14, 14) (56, 28, 28) (84, 42, 42) (112, 56, 56) (142, 71, 71) (170, 85, 85) (198, 99, 99) (226, 113, 113) (255, 127, 127)
dark pink (0, 0, 0) (28, 9, 9) (56, 18, 18) (84, 28, 28) (112, 37, 37) (142, 47, 47) (170, 56, 56) (198, 66, 66) (226, 75, 75) (255, 85, 85)
red (0, 0, 0) (28, 0, 0) (56, 0, 0) (84, 0, 0) (112, 0, 0) (142, 0, 0) (170, 0, 0) (198, 0, 0) (226, 0, 0) (255, 0, 0)

We can use weights in this fashion to fade one color into another. Let’s encapsulate this behavior in a function and see what we can do with it!

Instead of creating white to red, let’s map out all of the colors from red to blue at various levels of mixing. We can use this form of “mixing” to interpolate values between our basic rgb colors and fill out an entire color space!

def color_fade(df, l_color, r_color, n_steps=11):
    fade_weights = arange(n_steps)
    color_data = []
    color_index = []
    
    # calculate a weighted mixture for each color pair
    # [(10, 0), (9, 1), (8, 2), ... (1, 9), (0, 10)]
    for weights in zip(fade_weights[::-1], fade_weights):
        # Get the Series for the current weighted mix
        sequential_cmap = mix_colors(
            df, [l_color, r_color], weights=weights, name=weights
        )
        # store our output for later concatenation
        color_data.append(sequential_cmap)
        color_index.append((l_color, r_color) + weights)
            
    # Construct a meaningful index for the new color space
    new_index = MultiIndex.from_tuples(
        color_index, names=["color1", "color2", "weight1", "weight2"]
    )
    return (
        concat(color_data, axis=1).T
        .set_axis(new_index)
    )

red_blue_fade_df = color_fade(shade_rgb_df, "red", "blue", n_steps=11)
display(
    Markdown("**Sequential Red-Blue Color Space**"),
    red_blue_fade_df.pipe(display_colormap, cell_height=50, cell_width=80),
)
/tmp/ipykernel_902997/2331890231.py:42: FutureWarning: DataFrame.groupby with axis=1 is deprecated. Do `frame.T.groupby(...)` without axis instead.
  df.groupby("shading", axis="columns").apply(DataFrame.agg, tuple, axis="columns")
/tmp/ipykernel_902997/2331890231.py:49: FutureWarning: Styler.applymap has been deprecated. Use Styler.map instead.
  .applymap(_rgb_style)             # convert each rgb tuple to inline html style

Sequential Red-Blue Color Space

      shading 0.00 0.11 0.22 0.33 0.44 0.56 0.67 0.78 0.89 1.00
color1 color2 weight1 weight2                    
red blue 10 0 (0.0, 0.0, 0.0) (28.0, 0.0, 0.0) (56.0, 0.0, 0.0) (84.0, 0.0, 0.0) (112.0, 0.0, 0.0) (142.0, 0.0, 0.0) (170.0, 0.0, 0.0) (198.0, 0.0, 0.0) (226.0, 0.0, 0.0) (255.0, 0.0, 0.0)
9 1 (0.0, 0.0, 0.0) (25.2, 0.0, 2.8) (50.4, 0.0, 5.6) (75.6, 0.0, 8.4) (100.8, 0.0, 11.2) (127.8, 0.0, 14.2) (153.0, 0.0, 17.0) (178.2, 0.0, 19.8) (203.4, 0.0, 22.6) (229.5, 0.0, 25.5)
8 2 (0.0, 0.0, 0.0) (22.4, 0.0, 5.6) (44.8, 0.0, 11.2) (67.2, 0.0, 16.8) (89.6, 0.0, 22.4) (113.6, 0.0, 28.4) (136.0, 0.0, 34.0) (158.4, 0.0, 39.6) (180.8, 0.0, 45.2) (204.0, 0.0, 51.0)
7 3 (0.0, 0.0, 0.0) (19.6, 0.0, 8.4) (39.2, 0.0, 16.8) (58.8, 0.0, 25.2) (78.4, 0.0, 33.6) (99.4, 0.0, 42.6) (119.0, 0.0, 51.0) (138.6, 0.0, 59.4) (158.2, 0.0, 67.8) (178.5, 0.0, 76.5)
6 4 (0.0, 0.0, 0.0) (16.8, 0.0, 11.2) (33.6, 0.0, 22.4) (50.4, 0.0, 33.6) (67.2, 0.0, 44.8) (85.2, 0.0, 56.8) (102.0, 0.0, 68.0) (118.8, 0.0, 79.2) (135.6, 0.0, 90.4) (153.0, 0.0, 102.0)
5 5 (0.0, 0.0, 0.0) (14.0, 0.0, 14.0) (28.0, 0.0, 28.0) (42.0, 0.0, 42.0) (56.0, 0.0, 56.0) (71.0, 0.0, 71.0) (85.0, 0.0, 85.0) (99.0, 0.0, 99.0) (113.0, 0.0, 113.0) (127.5, 0.0, 127.5)
4 6 (0.0, 0.0, 0.0) (11.2, 0.0, 16.8) (22.4, 0.0, 33.6) (33.6, 0.0, 50.4) (44.8, 0.0, 67.2) (56.8, 0.0, 85.2) (68.0, 0.0, 102.0) (79.2, 0.0, 118.8) (90.4, 0.0, 135.6) (102.0, 0.0, 153.0)
3 7 (0.0, 0.0, 0.0) (8.4, 0.0, 19.6) (16.8, 0.0, 39.2) (25.2, 0.0, 58.8) (33.6, 0.0, 78.4) (42.6, 0.0, 99.4) (51.0, 0.0, 119.0) (59.4, 0.0, 138.6) (67.8, 0.0, 158.2) (76.5, 0.0, 178.5)
2 8 (0.0, 0.0, 0.0) (5.6, 0.0, 22.4) (11.2, 0.0, 44.8) (16.8, 0.0, 67.2) (22.4, 0.0, 89.6) (28.4, 0.0, 113.6) (34.0, 0.0, 136.0) (39.6, 0.0, 158.4) (45.2, 0.0, 180.8) (51.0, 0.0, 204.0)
1 9 (0.0, 0.0, 0.0) (2.8, 0.0, 25.2) (5.6, 0.0, 50.4) (8.4, 0.0, 75.6) (11.2, 0.0, 100.8) (14.2, 0.0, 127.8) (17.0, 0.0, 153.0) (19.8, 0.0, 178.2) (22.6, 0.0, 203.4) (25.5, 0.0, 229.5)
0 10 (0.0, 0.0, 0.0) (0.0, 0.0, 28.0) (0.0, 0.0, 56.0) (0.0, 0.0, 84.0) (0.0, 0.0, 112.0) (0.0, 0.0, 142.0) (0.0, 0.0, 170.0) (0.0, 0.0, 198.0) (0.0, 0.0, 226.0) (0.0, 0.0, 255.0)

Well that’s neat! We now have a programmatic way to fade one color into another and visualize the interpolation of the color space!

But, why stop with two colors? With a little additional code, the fade function can chain together multiple pairs of colors to show a longer sequential_fade. Let’s try using the fade function to create a rainbow by linking together multiple fades.

from itertools import pairwise

def sequential_fade(df, colors, n_steps=11):
    linear_color_space = []
    
    # iterate through all sequential pairs of colors
    # (1, 2, 3, 4) -> [(1, 2), (2, 3), (3, 4)]
    for l_color, r_color in pairwise(colors):
        bi_color_space = color_fade(df, l_color, r_color, n_steps=n_steps)
        linear_color_space.append(bi_color_space)
    return concat(linear_color_space)

# Construct a rainbow from mixing the following colors in pairwise order
rainbow_space = sequential_fade(
    shade_rgb_df, ["red", "yellow", "lime", "blue", "magenta"], n_steps=7
)

display(
    Markdown("**Sequential Red-Blue Color Space**"),
    rainbow_space.pipe(display_colormap, cell_height=20, cell_width=70)
)
/tmp/ipykernel_902997/2331890231.py:42: FutureWarning: DataFrame.groupby with axis=1 is deprecated. Do `frame.T.groupby(...)` without axis instead.
  df.groupby("shading", axis="columns").apply(DataFrame.agg, tuple, axis="columns")
/tmp/ipykernel_902997/2331890231.py:49: FutureWarning: Styler.applymap has been deprecated. Use Styler.map instead.
  .applymap(_rgb_style)             # convert each rgb tuple to inline html style

Sequential Red-Blue Color Space

      shading 0.00 0.11 0.22 0.33 0.44 0.56 0.67 0.78 0.89 1.00
color1 color2 weight1 weight2                    
red yellow 6 0 (0.0, 0.0, 0.0) (28.0, 0.0, 0.0) (56.0, 0.0, 0.0) (84.0, 0.0, 0.0) (112.0, 0.0, 0.0) (142.0, 0.0, 0.0) (170.0, 0.0, 0.0) (198.0, 0.0, 0.0) (226.0, 0.0, 0.0) (255.0, 0.0, 0.0)
5 1 (0.0, 0.0, 0.0) (28.0, 4.666666666666667, 0.0) (56.0, 9.333333333333334, 0.0) (84.0, 14.0, 0.0) (112.0, 18.666666666666668, 0.0) (142.0, 23.666666666666668, 0.0) (170.0, 28.333333333333332, 0.0) (198.0, 33.0, 0.0) (226.0, 37.666666666666664, 0.0) (255.0, 42.5, 0.0)
4 2 (0.0, 0.0, 0.0) (28.0, 9.333333333333334, 0.0) (56.0, 18.666666666666668, 0.0) (84.0, 28.0, 0.0) (112.0, 37.333333333333336, 0.0) (142.0, 47.333333333333336, 0.0) (170.0, 56.666666666666664, 0.0) (198.0, 66.0, 0.0) (226.0, 75.33333333333333, 0.0) (255.0, 85.0, 0.0)
3 3 (0.0, 0.0, 0.0) (28.0, 14.0, 0.0) (56.0, 28.0, 0.0) (84.0, 42.0, 0.0) (112.0, 56.0, 0.0) (142.0, 71.0, 0.0) (170.0, 85.0, 0.0) (198.0, 99.0, 0.0) (226.0, 113.0, 0.0) (255.0, 127.5, 0.0)
2 4 (0.0, 0.0, 0.0) (28.0, 18.666666666666668, 0.0) (56.0, 37.333333333333336, 0.0) (84.0, 56.0, 0.0) (112.0, 74.66666666666667, 0.0) (142.0, 94.66666666666667, 0.0) (170.0, 113.33333333333333, 0.0) (198.0, 132.0, 0.0) (226.0, 150.66666666666666, 0.0) (255.0, 170.0, 0.0)
1 5 (0.0, 0.0, 0.0) (28.0, 23.333333333333332, 0.0) (56.0, 46.666666666666664, 0.0) (84.0, 70.0, 0.0) (112.0, 93.33333333333333, 0.0) (142.0, 118.33333333333333, 0.0) (170.0, 141.66666666666666, 0.0) (198.0, 165.0, 0.0) (226.0, 188.33333333333334, 0.0) (255.0, 212.5, 0.0)
0 6 (0.0, 0.0, 0.0) (28.0, 28.0, 0.0) (56.0, 56.0, 0.0) (84.0, 84.0, 0.0) (112.0, 112.0, 0.0) (142.0, 142.0, 0.0) (170.0, 170.0, 0.0) (198.0, 198.0, 0.0) (226.0, 226.0, 0.0) (255.0, 255.0, 0.0)
yellow lime 6 0 (0.0, 0.0, 0.0) (28.0, 28.0, 0.0) (56.0, 56.0, 0.0) (84.0, 84.0, 0.0) (112.0, 112.0, 0.0) (142.0, 142.0, 0.0) (170.0, 170.0, 0.0) (198.0, 198.0, 0.0) (226.0, 226.0, 0.0) (255.0, 255.0, 0.0)
5 1 (0.0, 0.0, 0.0) (23.333333333333332, 28.0, 0.0) (46.666666666666664, 56.0, 0.0) (70.0, 84.0, 0.0) (93.33333333333333, 112.0, 0.0) (118.33333333333333, 142.0, 0.0) (141.66666666666666, 170.0, 0.0) (165.0, 198.0, 0.0) (188.33333333333334, 226.0, 0.0) (212.5, 255.0, 0.0)
4 2 (0.0, 0.0, 0.0) (18.666666666666668, 28.0, 0.0) (37.333333333333336, 56.0, 0.0) (56.0, 84.0, 0.0) (74.66666666666667, 112.0, 0.0) (94.66666666666667, 142.0, 0.0) (113.33333333333333, 170.0, 0.0) (132.0, 198.0, 0.0) (150.66666666666666, 226.0, 0.0) (170.0, 255.0, 0.0)
3 3 (0.0, 0.0, 0.0) (14.0, 28.0, 0.0) (28.0, 56.0, 0.0) (42.0, 84.0, 0.0) (56.0, 112.0, 0.0) (71.0, 142.0, 0.0) (85.0, 170.0, 0.0) (99.0, 198.0, 0.0) (113.0, 226.0, 0.0) (127.5, 255.0, 0.0)
2 4 (0.0, 0.0, 0.0) (9.333333333333334, 28.0, 0.0) (18.666666666666668, 56.0, 0.0) (28.0, 84.0, 0.0) (37.333333333333336, 112.0, 0.0) (47.333333333333336, 142.0, 0.0) (56.666666666666664, 170.0, 0.0) (66.0, 198.0, 0.0) (75.33333333333333, 226.0, 0.0) (85.0, 255.0, 0.0)
1 5 (0.0, 0.0, 0.0) (4.666666666666667, 28.0, 0.0) (9.333333333333334, 56.0, 0.0) (14.0, 84.0, 0.0) (18.666666666666668, 112.0, 0.0) (23.666666666666668, 142.0, 0.0) (28.333333333333332, 170.0, 0.0) (33.0, 198.0, 0.0) (37.666666666666664, 226.0, 0.0) (42.5, 255.0, 0.0)
0 6 (0.0, 0.0, 0.0) (0.0, 28.0, 0.0) (0.0, 56.0, 0.0) (0.0, 84.0, 0.0) (0.0, 112.0, 0.0) (0.0, 142.0, 0.0) (0.0, 170.0, 0.0) (0.0, 198.0, 0.0) (0.0, 226.0, 0.0) (0.0, 255.0, 0.0)
lime blue 6 0 (0.0, 0.0, 0.0) (0.0, 28.0, 0.0) (0.0, 56.0, 0.0) (0.0, 84.0, 0.0) (0.0, 112.0, 0.0) (0.0, 142.0, 0.0) (0.0, 170.0, 0.0) (0.0, 198.0, 0.0) (0.0, 226.0, 0.0) (0.0, 255.0, 0.0)
5 1 (0.0, 0.0, 0.0) (0.0, 23.333333333333332, 4.666666666666667) (0.0, 46.666666666666664, 9.333333333333334) (0.0, 70.0, 14.0) (0.0, 93.33333333333333, 18.666666666666668) (0.0, 118.33333333333333, 23.666666666666668) (0.0, 141.66666666666666, 28.333333333333332) (0.0, 165.0, 33.0) (0.0, 188.33333333333334, 37.666666666666664) (0.0, 212.5, 42.5)
4 2 (0.0, 0.0, 0.0) (0.0, 18.666666666666668, 9.333333333333334) (0.0, 37.333333333333336, 18.666666666666668) (0.0, 56.0, 28.0) (0.0, 74.66666666666667, 37.333333333333336) (0.0, 94.66666666666667, 47.333333333333336) (0.0, 113.33333333333333, 56.666666666666664) (0.0, 132.0, 66.0) (0.0, 150.66666666666666, 75.33333333333333) (0.0, 170.0, 85.0)
3 3 (0.0, 0.0, 0.0) (0.0, 14.0, 14.0) (0.0, 28.0, 28.0) (0.0, 42.0, 42.0) (0.0, 56.0, 56.0) (0.0, 71.0, 71.0) (0.0, 85.0, 85.0) (0.0, 99.0, 99.0) (0.0, 113.0, 113.0) (0.0, 127.5, 127.5)
2 4 (0.0, 0.0, 0.0) (0.0, 9.333333333333334, 18.666666666666668) (0.0, 18.666666666666668, 37.333333333333336) (0.0, 28.0, 56.0) (0.0, 37.333333333333336, 74.66666666666667) (0.0, 47.333333333333336, 94.66666666666667) (0.0, 56.666666666666664, 113.33333333333333) (0.0, 66.0, 132.0) (0.0, 75.33333333333333, 150.66666666666666) (0.0, 85.0, 170.0)
1 5 (0.0, 0.0, 0.0) (0.0, 4.666666666666667, 23.333333333333332) (0.0, 9.333333333333334, 46.666666666666664) (0.0, 14.0, 70.0) (0.0, 18.666666666666668, 93.33333333333333) (0.0, 23.666666666666668, 118.33333333333333) (0.0, 28.333333333333332, 141.66666666666666) (0.0, 33.0, 165.0) (0.0, 37.666666666666664, 188.33333333333334) (0.0, 42.5, 212.5)
0 6 (0.0, 0.0, 0.0) (0.0, 0.0, 28.0) (0.0, 0.0, 56.0) (0.0, 0.0, 84.0) (0.0, 0.0, 112.0) (0.0, 0.0, 142.0) (0.0, 0.0, 170.0) (0.0, 0.0, 198.0) (0.0, 0.0, 226.0) (0.0, 0.0, 255.0)
blue magenta 6 0 (0.0, 0.0, 0.0) (0.0, 0.0, 28.0) (0.0, 0.0, 56.0) (0.0, 0.0, 84.0) (0.0, 0.0, 112.0) (0.0, 0.0, 142.0) (0.0, 0.0, 170.0) (0.0, 0.0, 198.0) (0.0, 0.0, 226.0) (0.0, 0.0, 255.0)
5 1 (0.0, 0.0, 0.0) (4.666666666666667, 0.0, 28.0) (9.333333333333334, 0.0, 56.0) (14.0, 0.0, 84.0) (18.666666666666668, 0.0, 112.0) (23.666666666666668, 0.0, 142.0) (28.333333333333332, 0.0, 170.0) (33.0, 0.0, 198.0) (37.666666666666664, 0.0, 226.0) (42.5, 0.0, 255.0)
4 2 (0.0, 0.0, 0.0) (9.333333333333334, 0.0, 28.0) (18.666666666666668, 0.0, 56.0) (28.0, 0.0, 84.0) (37.333333333333336, 0.0, 112.0) (47.333333333333336, 0.0, 142.0) (56.666666666666664, 0.0, 170.0) (66.0, 0.0, 198.0) (75.33333333333333, 0.0, 226.0) (85.0, 0.0, 255.0)
3 3 (0.0, 0.0, 0.0) (14.0, 0.0, 28.0) (28.0, 0.0, 56.0) (42.0, 0.0, 84.0) (56.0, 0.0, 112.0) (71.0, 0.0, 142.0) (85.0, 0.0, 170.0) (99.0, 0.0, 198.0) (113.0, 0.0, 226.0) (127.5, 0.0, 255.0)
2 4 (0.0, 0.0, 0.0) (18.666666666666668, 0.0, 28.0) (37.333333333333336, 0.0, 56.0) (56.0, 0.0, 84.0) (74.66666666666667, 0.0, 112.0) (94.66666666666667, 0.0, 142.0) (113.33333333333333, 0.0, 170.0) (132.0, 0.0, 198.0) (150.66666666666666, 0.0, 226.0) (170.0, 0.0, 255.0)
1 5 (0.0, 0.0, 0.0) (23.333333333333332, 0.0, 28.0) (46.666666666666664, 0.0, 56.0) (70.0, 0.0, 84.0) (93.33333333333333, 0.0, 112.0) (118.33333333333333, 0.0, 142.0) (141.66666666666666, 0.0, 170.0) (165.0, 0.0, 198.0) (188.33333333333334, 0.0, 226.0) (212.5, 0.0, 255.0)
0 6 (0.0, 0.0, 0.0) (28.0, 0.0, 28.0) (56.0, 0.0, 56.0) (84.0, 0.0, 84.0) (112.0, 0.0, 112.0) (142.0, 0.0, 142.0) (170.0, 0.0, 170.0) (198.0, 0.0, 198.0) (226.0, 0.0, 226.0) (255.0, 0.0, 255.0)

Wrap-Up#

And there you have it: creating, managing, and combining colormaps, all done in pandas.

If you have any insights or questions, feel free to tag me in a post on Twitter.

Until next time!