The right way to create fast heatmaps?

Community Support
1

I need to draw a heatmap in realtime, using the method described in unemployment.py — Bokeh 2.4.2 Documentation, I have achieved my goal.
The problem is that my heatmap is quite large - about 20-50 rows and about a thousand columns, and here the method of constructing hetmap with the help of a rect glyph is no longer suitable - everything becomes terribly slow.
There is a way to build a very fast map using image glyph, But! The problem is that the image glyph builds the image in rows, and in my case I need to build in columns - to easily add new data through source.stream.

Those

  • data = [[1], [2], [3]] will show a single-column heatmap
  • data = [[1,4], [2,5], [3,6]] - 2 columns
  • data = [[1,4,7], [2,5,8], [3,6,9]] - 3 columns

and so on

This method is unsuitable for adding data through source.stream - you have to change the source data entirely.

Is there a way that image glyph with data = [[1,2,3]] would draw 1 column, with data = [[1,2,3], [4,5,6]] 2 columns?

from bokeh.plotting import curdoc, figure, ColumnDataSource, show

data = [[1,2,3]] # I want a column, not a row

img_source = ColumnDataSource(data=dict(vol=[]))

p = figure(tooltips=[("value", "@vol")])
p.x_range.range_padding = p.y_range.range_padding = 0

p.image (image="vol", x=0, y=0, dw=1, dh=1, palette ="Spectral11", 
         source=img_source)

img_source.stream(dict(vol=[data]))

curdoc().add_root(p)
2

It might be possible to use CustomJSTransform to transpose the data on the JS side. Then you could store columns as the elements of .data (which would allow you to stream new columns) and the transform on the JS side would transpose everything before rendering (since the renderer expects the items of .data to be rows as you have noted). I haven’t ever tried this explicitly but happy to answer questions or test out code if you make a stab at it.

3

Thanks to Bryan for the hint, but it looks like it will not work here, if briefly - during the transforming, we inevitably violate the dimensions of the input and output data.

Here is the result of my attempt:
So, we have the data set [[1,2,3], [4,5,6], [7,8,9]], which needs to be transformed so that each new row becomes a column, i.e. [[1,4,7], [2,5,8], [3,6,9]]
Check it with this code:

from bokeh.plotting import curdoc, figure
from bokeh.models import ColumnDataSource, CustomJSTransform
from bokeh.transform import transform
from bokeh.driving import count, repeat

img_source = ColumnDataSource(data=dict(vol=[]))

v_func = """
return [[[1,4,7], [2,5,8], [3,6,9]]]
"""

rows_to_col = CustomJSTransform(v_func=v_func)
p = figure(tooltips=[("value", "@vol")])
p.x_range.range_padding = p.y_range.range_padding = 0
p.image(image=transform("vol",rows_to_col), x=0, y=0, dw=1, dh=1, palette="Spectral11", source=img_source)

img_source.stream(dict(vol=[1]))
curdoc().add_root(p)

22

And now let’s try adding new columns with stream:

from bokeh.plotting import curdoc, figure
from bokeh.models import ColumnDataSource, CustomJSTransform
from bokeh.transform import transform
from bokeh.driving import count, repeat

data_feed = [[1,2,3],[4,5,6],[7,8,9]]
img_source = ColumnDataSource(data=dict(vol=[]))

v_func = """
console.log(xs)
arr = xs
if(arr.length !=0){
    twisted_arr = arr[0].map(function(e) {return [e];})
    for (let row = 1; row < arr.length; row++){ 
        twisted_arr.map(function(e,i) {return [e.push(arr[row][i])];}) 
        }

    console.log([twisted_arr] )
    console.log("-----------------------")
    return [twisted_arr]
}
return xs
"""
rows_to_col = CustomJSTransform(v_func=v_func)
p = figure(tooltips=[("value", "@vol")])
p.x_range.range_padding = p.y_range.range_padding = 0
p.image(image=transform("vol",rows_to_col), x=0, y=0, dw=1, dh=1, palette="Spectral11", source=img_source)

@repeat([0, 1, 2])
def update(i):
    img_source.stream(dict(vol=[data_feed[i]]))

curdoc().add_periodic_callback(update, 1000)
curdoc().add_root(p)

And our problem is an error due to the fact that the output should have a one-dimensional array of arrays, and the input has a two-dimensional array…

111
1111360×768 31.2 KB

4

@Andromeda I am not sure your twisted_arr code is correct. I would expect JS code to transpose a “2d array” to look more like this:

function transpose(arr) {
  const rows = arr.length, cols = arr[0].length;
  const result = []
  for (let j = 0; j < cols; j++) {
    result[j] = Array(rows)
  }
  for (let i = 0; i < rows; i++) {
    for (let j = 0; j < cols; j++) {
      result[j][i] = arr[i][j]
    }
  }
  return result
}

which will always return a array containing subarrays (just like the input).