I’m trying to use bokeh to play a sequence of video frames in a jupyter notebook. My goal is to be able to do image processing on the frames in python, and then view the results at 30fps in the notebook. The image processing will use multiprocessing, so the main bottleneck will be sending updates to the browser, and updating the image canvas.
I’m using image_rgba to show processed frames. However, when I update the plot, and use push_notebook to show the update, the update is very slow (a couple seconds), even with a 640x360x4 (RGBA) image.
I’m wondering if there is a way for bokeh to only update “changed pixels” of the plot (only 1-10% of the pixels have changed, since these are frames from a movie). One idea would be to push updated pixel coordinates/values to javascript, then have a javascript function update the canvas pixels directly. I’m wondering if bokeh can already handle this “update changed pixels only” task, or if anyone has advice on how to implement the “push changes to javascript, then update canvas directly” solution (e.g., how do I access/update a target canvas?). Or, if bokeh seems like the wrong tool for this task, if anyone has advice for an alternative approach, I’d be happy to hear that too!
Here’s an example of draw/update using image_rgba and push_notebook (too slow!)
impots
from bokeh.plotting import figure
from bokeh.io import output_notebook, show, push_notebook
from PIL import Image
from skvideo.datasets import bigbuckbunny
from pims import Video
import numpy as np
def frame2img(frame):
img = Image.fromarray(frame)
img = np.asarray(img.convert(“RGBA”))
img = np.flipud(img)
return img
output_notebook()
initial draw
frames = Video(bigbuckbunny())
img = frame2img(frames[0])
plot = figure(plot_width=img.shape[1], plot_height=img.shape[0],
x_range=[0, img.shape[0]], y_range=[0, img.shape[1]])
plot.image_rgba(image=[img], x=[0], y=[0],
dw=[img.shape[0]], dh=[img.shape[1]])
target = show(plot, notebook_handle=True)
in a serpate cell…
update
img = frame2img(frames[1])
plot.image_rgba(image=[img], x=[0], y=[0], dw=[img.shape[0]], dh=[img.shape[1]])
push_notebook(handle=target)