Way to update only changed pixels of image_rgba

I’m trying to use bokeh to play a sequence of video frames in a jupyter notebook. My goal is to be able to do image processing on the frames in python, and then view the results at 30fps in the notebook. The image processing will use multiprocessing, so the main bottleneck will be sending updates to the browser, and updating the image canvas.

I’m using image_rgba to show processed frames. However, when I update the plot, and use push_notebook to show the update, the update is very slow (a couple seconds), even with a 640x360x4 (RGBA) image.

I’m wondering if there is a way for bokeh to only update “changed pixels” of the plot (only 1-10% of the pixels have changed, since these are frames from a movie). One idea would be to push updated pixel coordinates/values to javascript, then have a javascript function update the canvas pixels directly. I’m wondering if bokeh can already handle this “update changed pixels only” task, or if anyone has advice on how to implement the “push changes to javascript, then update canvas directly” solution (e.g., how do I access/update a target canvas?). Or, if bokeh seems like the wrong tool for this task, if anyone has advice for an alternative approach, I’d be happy to hear that too!

Here’s an example of draw/update using image_rgba and push_notebook (too slow!)


from bokeh.plotting import figure

from bokeh.io import output_notebook, show, push_notebook

from PIL import Image

from skvideo.datasets import bigbuckbunny

from pims import Video

import numpy as np

def frame2img(frame):

img = Image.fromarray(frame)

img = np.asarray(img.convert(“RGBA”))

img = np.flipud(img)

return img


initial draw

frames = Video(bigbuckbunny())

img = frame2img(frames[0])

plot = figure(plot_width=img.shape[1], plot_height=img.shape[0],

x_range=[0, img.shape[0]], y_range=[0, img.shape[1]])

plot.image_rgba(image=[img], x=[0], y=[0],

dw=[img.shape[0]], dh=[img.shape[1]])

target = show(plot, notebook_handle=True)

in a serpate cell…


img = frame2img(frames[1])

plot.image_rgba(image=[img], x=[0], y=[0], dw=[img.shape[0]], dh=[img.shape[1]])


any ideas for a faster way to update (e.g., update only the changed pixels?)