I’m developing a component to display images in a grid. All the images are displayed as one bokeh model which is rendered in a panel Bokeh pane. Panel is used for callbacks and widgets.
I had an original implementation of this which rendered every image using holoviews in its own panel pane, but it was horrendously slow to update. It literally took about a second per image to update, and you could see it updating the images one by one. I then switched to using bokeh but still rendering each image individually and it was still very slow. After some research I discovered a post on panel’s discourse describing a similar problem where one of the panel developers commented it was likely due to Bokeh’s layout engine. So I set out to create something using bokeh only and taking care not to trigger a relayout or update something unnecessary.
The logic I’m using is that I have a single ColumnDataSource, and a list of figures that can only grow. When a caller sets the list of images, the following happens:
- If there are more images than figures, I create additional figures. If there are fewer, I set the visible property of the extra figures to False. All figures have a CDSView with an IndexFilter pointing to a single image. This index never changes, as the index is simply the index of the figure in my list of figures.
- Update ColumnDataSource.data
- Update the frame_width, frame_height, x_range and y_range of each figure
- If new figures were created, merge their toolbars with the main toolbar (like
gridplotwithmerge_toolbars=True, but dynamic) and add them to theGridBox.
What doesn’t work is the following:
If a caller provides a list of images that’s smaller than any previously provided list, any updates to the list of images doesn’t render afterwards. If you run below code, try this sequence:
- change start index → OK
- increase num images → OK
- change Ncols → relayout OK
- decrease num images → OK
- change start index → this stops working after step 4 (also if you skip steps 1-3)
I’ve really been struggling with why this doesn’t work, and I’m not sure whether I’m doing something wrong or whether I’ve stumbled upon a bug. If I set a breakpoint at the end of all my updates and run show(self.view.object) there, it shows the current state of the plot in a new browser window. If I do the same on step 4 above, I get an empty page (actually there was a point while I was messing with this where this showed the correct plot, but the main plot connected to the server didn’t update). I’ve also been checking the javascript console of my browser and I can see the client receive all the updates as I would expect them.
Since I’m posting a pretty lengthy example below, I have a few extra questions I would appreciate some help with too:
- In
_update_toolbar(), I first tried updating thetoolbarsandtoolsattributes of theProxyToolbarobject. This didn’t activate the tools on any new figures that were added. However if I pass multiple images into the constructor, the tools do work for all those initial figures. I don’t really understand what’s going on there since both are using the same code. As you can see, I ended up creating a whole newToolbarBoxto make it work. - I have a callback for my
max_widthandmax_heightparameters, which is supposed to change the size of all the images in the grid. In its current state this doesn’t work, but admittedly I haven’t spent a lot of time looking into why. If anyone has any pointers that’d be helpful. Do I need to update something about theGridBox, or create a newgridplot, or …?
I guess the overarching theme here is I don’t understand which attribute changes will actually trigger updates. From the documentation it seems like any attribute update on a bokeh object should do that, but that hasn’t been my experience. Any clarifications would be helpful, as well as suggestions on how to debug problems like this.
If you want to run this, pass a folder containing png images to the TestImageGrid constructor.
Code:
import itertools
import math
import cv2
import numpy as np
import param as pm
import panel as pn
from pathlib import Path
from bokeh.plotting import figure, Figure, gridplot, show
from bokeh.models import ColumnDataSource, LinearColorMapper, CDSView, IndexFilter, BoxZoomTool, \
WheelZoomTool, ProxyToolbar, ToolbarBox
class ImageGrid(pm.Parameterized):
images = pm.List(class_=np.ndarray, doc="List of images to display", precedence=-1)
ncols = pm.Integer(default=3, bounds=(1, None), doc="Number of columns in grid")
max_width = pm.Integer(default=300, bounds=(1, None), doc="Max width of a single image")
max_height = pm.Integer(default=300, bounds=(1, None), doc="Max height of a single image")
view = pm.ClassSelector(class_=pn.pane.Bokeh, constant=True, precedence=-1)
_source = pm.ClassSelector(class_=ColumnDataSource, constant=True, precedence=-1)
_figures = pm.List(class_=Figure, constant=True, precedence=-1)
def __init__(self, **kwargs):
super().__init__(view=pn.pane.Bokeh(gridplot(None)),
_source=ColumnDataSource(),
**kwargs)
if self.images:
self._update_figures()
def _add_figure(self):
index = len(self._figures)
fig = figure(match_aspect=True, margin=10)
color_mapper = LinearColorMapper(palette="Greys256")
cds_view = CDSView(source=self._source, filters=[IndexFilter([index])])
fig.image(source=self._source, view=cds_view,
image='image', x='x', y='y', dw='dw', dh='dh',
color_mapper=color_mapper)
box_zoom = fig.select(type=BoxZoomTool)
scroll_zoom = fig.select(type=WheelZoomTool)
if box_zoom:
box_zoom.match_aspect = True
if scroll_zoom:
scroll_zoom.zoom_on_axis = False
fig.toolbar_location = None # will be added to merged toolbar
self._figures.append(fig)
def _update_num_figures(self):
num_figures = len(self._figures)
num_images = len(self.images)
if num_figures < num_images:
for _ in range(num_images - num_figures):
self._add_figure()
for index, fig in enumerate(self._figures):
fig.visible = index < num_images
def _update_source(self):
heights = [image.shape[0] for image in self.images]
widths = [image.shape[1] for image in self.images]
self._source.data = dict(
x=[0] * len(self.images),
y=heights,
dw=widths,
dh=heights,
image=[image[::-1] for image in self.images]
)
@pm.depends('max_width', 'max_height', watch=True)
def _set_image_dimensions(self):
for index, (fig, image) in enumerate(zip(self._active_figures, self.images)):
h, w = image.shape
fig.x_range.update(start=0, end=w, bounds=(0, w))
fig.y_range.update(start=h, end=0, bounds=(0, h))
if h > w:
fig.frame_height = self.max_height
fig.frame_width = int(fig.frame_height * w / h)
else:
fig.frame_width = self.max_width
fig.frame_height = int(fig.frame_width * h / w)
def _update_toolbar(self):
# Add to merged toolbar (see gridplot implementation)
toolbars = [fig.toolbar for fig in self._figures]
tools = list(itertools.chain.from_iterable([fig.tools for fig in self._figures]))
# This doesn't work
# proxy = self.view.object.children[0].toolbar
# proxy.update(toolbars=toolbars, tools=tools)
# This does work
proxy = ProxyToolbar(toolbars=toolbars, tools=tools)
self.view.object.children = [ToolbarBox(toolbar=proxy, toolbar_location='above')] + \
self.view.object.children[1:]
@pm.depends('images', watch=True)
def _update_figures(self):
self._update_num_figures()
self._update_source()
self._set_image_dimensions()
if len(self.view.object.children[1].children) < len(self.images):
self._update_toolbar()
self._update_grid()
pass # set a breakpoint here and run `show(self.view.object)`
@pm.depends('ncols', watch=True)
def _update_grid(self):
r, c = np.unravel_index(np.arange(len(self.images)), (self.nrows, self.ncols))
gridbox = self.view.object.children[1]
gridbox.children = list(zip(self._active_figures, r, c))
@property
def nrows(self):
return math.ceil(len(self.images) / self.ncols)
@property
def _active_figures(self):
return self._figures[:len(self.images)]
class TestImageGrid(pm.Parameterized):
start_index = pm.Integer(0)
num_images = pm.Integer(1, bounds=(0, None))
load_func = pm.Callable(lambda filename: cv2.imread(str(filename), cv2.IMREAD_UNCHANGED),
precedence=-1)
folder = pm.Foldername(constant=True, precedence=-1)
files = pm.List(class_=Path, constant=True, precedence=-1)
imagegrid = pm.ClassSelector(class_=ImageGrid, constant=True, precedence=-1)
def __init__(self, **kwargs):
super().__init__(imagegrid=ImageGrid(), **kwargs)
with pm.edit_constant(self):
self.files = list(Path(self.folder).glob('*.png'))
self.param.start_index.bounds = 0, len(self.files)
self.view = pn.Row(
pn.WidgetBox(pn.Param(self.param), pn.Param(self.imagegrid.param)),
self.imagegrid.view
)
self._set_images()
self._stepsize()
@pm.depends('start_index', 'num_images', watch=True)
def _set_images(self):
last_index = self.start_index + self.num_images
images = [self.load_func(file) for file in self.files[self.start_index:last_index]]
self.imagegrid.images = images
@pm.depends('num_images', watch=True)
def _stepsize(self):
self.param.start_index.step = self.num_images
def view():
tig = TestImageGrid(folder="images/")
return tig.view
if __name__.startswith("bokeh"):
view().servable()
elif __name__ == '__main__':
pn.serve(view, port=8920)