Z order control

Community Support
,
1

The only issue I have found on this so far is: There should be a way to control order that renderers are drawn. · Issue #696 · bokeh/bokeh · GitHub
At the time it was marked as solved by: Add glyphs/sprint example by mattpap · Pull Request #1484 · bokeh/bokeh · GitHub
However, I think the solution was only regarding several levels.

My situation is as follows:
I have many glyphs (polygons) which I add via stream iteratively. Sometimes I need to push new polygon to the background, sometimes I need to place it on top.

Finally, I need to rearrange their z order.

Performance is fairly critical, re-plotting all is not an option as it takes ~6 seconds to plot, while I need a close-to-live performance.

Could not find much information about it.

Appreciate any help.

2

The issue above only concerned the relative order of entire glyphs, e.g. how to relatively order entire sets of elements each generated by separate, different calls to calls to the circle glyph method. The issue was not about the order the individual circle elements drawn for one call to the circle method.

In fact, there is no mechanism to specify this. In general the order of individual items for a single glyph method can be affected by many things: presence or absence of active selections, CDS views and filters, interactive level-of-detail decimation, hover highlighting activations, webgl vs canvas vs svg rendering backend, muted status, and point culling from spatial indexing, among others. Because of this complexity, there is no guarantee regarding the order that a glyph will draw its individual elements, and no way to exert control over this order.

If you need to control the placement of, say, individual circles on the screen, you will have to have separate calls to circle(...) for each, and order the glyph renderers for those circles in the renderers list (as described in the issue). I don’t have any other practical recommendations to offer [1]. Bluntly, if this is not workable then you will probably need to look at other tools besides Bokeh.

  1. At the extreme you could in principle create a custom extension ti draw whatever you want in whatever order you want, but for you what you are describing you’d have to jettison a huge amount of existing BokehJS machinery. It would be a highly nontrivial undertaking, and in addition, the BokehJS API is not yet entirely stabilized so it might require future upkeep. ↩︎

3

" If you need to control the placement of, say, individual circles on the screen, you will have to have separate calls to circle(...) for each, and order the glyph renderers for those circles in the renderers list (as described in the issue)."

I don’t entirely understand what you mean by this. Could you explain?

4

Perhaps an example will be more instructive:

r1 = p.circle([1,2,3], [1,2,3])
r2 = p.circle([4,5,6], [4,5,6])

You can make all the “r1” circles draw before all the “r2” circles (or vice-versa, by switching the order of the calls). That is the finest-grained control that Bokeh offers. [1]

You cannot control the order of the individual items that “r1” renders or the individual items that “r2” renders.

  1. This applies to any glyph method, I’m just using circles as a concrete example. ↩︎

5

Ah ok, as simple as that. I just got confused by the emphasis on individual calls. I think multi_line / multi_polygon drawing obey the order in the same manner.

Thank you for your help.

One last question, is the functionality that I am after makes sense on js side? I.e. is there a minimal computational cost method to tell js to change the order of display?

6

I.e. is there a minimal computational cost method to tell js to change the order of display?

I don’t really understand the question. As I mentioned there are several interactive features that intersect all at once that can all influence the order. As an example: when there is an active selection, all the selected points are drawn as one batch, and all the unselected points in another batch, i.e. “out of order”. It’s not about computation cost, it’s about complexity that is incompatible with a simple order notion.

Since you mentioned streaming, I had one idea that might be useful in case it would suffice to split the glyphs up into just a few “sub-levels” within the glyph rendering level that Bokeh implicitly cares about:

  • have one CDS, that you stream data, including a column for what “sub-level”
  • make multiple calls glyph calls, with a GroupFilter keyed to one “sub-level”
  • these glyphs should be added by you in the order you want the “sub-levels” to appear

That might work. I am not aware of anyone every trying, or for that matter, trying to use CDS filters together with streaming. As with any large project, the less well-traveled areas of the project are always more likely to have not-yet-know issues. You’d just have to try and see.

7

I see the answer to my question. I meant if js theoretically supports z-order display without needing to re-render the plot. I.e. if there are no hardcoded dead-ends that would prevent that. But as it can bring to front say on hover, it should theoretically be able to do what I am after.

Thanks for idea. However, I need to re-order up to 10000 glyphs individually. Streaming them alongside the app is not a bottleneck, so was hoping that ordering them at the same time is possible too.

8

There might also be a misunderstanding here. Bokeh is a raster drawing tool that draws on a raster canvas. The entire plot (i.e. the canvas) is always completely re-drawn on any change, including streaming. The savings from stream come from reduced network traffic (i.e. sending only new data points), not from any “partial” drawing updates.

9

Ok, thanks for this. Then this just confirms that, if done properly, performance is not going to be an issue.

10

Just in case anyone else needs this. Simply reordering on js side does the trick. Here is 4k glyphs being reordered at 20hz.

11

I don’t actually know what this means, I am sure future readers would appreciate an MRE demonstration or at least a sketch of your approach.

12

I stored 2 sets of data, but it is possible to work out a solution without needing to do so, just need to store current order separately instead.

    import time
    import numpy as np
    from bokeh.plotting import show, figure
    from bokeh.models import ColumnDataSource, CustomJS
    from bokeh.models.widgets import Button
    from bokeh.layouts import column
    from bokeh.io import output_notebook, push_notebook
    from bokeh import events
    output_notebook()

    # MAIN
    N = 4000
    p = figure()
    x = np.random.uniform(size=(N, 2)).tolist()
    y = np.random.uniform(size=(N, 2)).tolist()
    c = np.random.choice(['red', 'green'], N).tolist()
    o = np.arange(N).tolist()
    cds = ColumnDataSource(data=dict(
        x=x, y=y, c=c,
        x_use=x, y_use=y, c_use=c,
        order=o))
    p.multi_line(xs='x_use', ys='y_use', color='c_use', source=cds, line_width=12)

    callback = CustomJS(args=dict(cds=cds), code="""
        const x_new = cds.data["order"].map(i => cds.data.x[i]);
        const y_new = cds.data["order"].map(i => cds.data.y[i]);
        const c_new = cds.data["order"].map(i => cds.data.c[i]);
        cds.data['x_use'] = x_new;
        cds.data['y_use'] = y_new;
        cds.data['c_use'] = c_new;
    """)

    cds.js_on_change('data', callback)
    h = show(p, notebook_handle=True)

    # NEW CELL
    for i in range(100):
        time.sleep(0.05)
        cds.data['order'] = np.random.choice(o, N, replace=False).tolist()
        push_notebook(handle=h)
1 Like
13

I’m honestly not sure why that would work. I’d expect a full assignment e.g. cds.data = new_data would be necessary to automatically trigger re-render, but then also I’d expect that to trigger and infinite callback cascade since the callback is on data. But if you say it is working for you I’ll take your word.

14

I agree that there is some ambiguity here. At least from my side.

I found that some use source.selected.indices = [] to trigger callback on CDS.selected.js_on_change, which then does the required job.

I would appreciate some insights or alternative better ways to do this. If you are surprised that this works then I think there is a risk of this breaking in the future.

Could this be related to the fact that I am running in notebook? And this will cause issues you mentioned if deployed as an app?

15

The data change on the python side triggers both a redraw, and the CustomJS callback execution. I suppose what might be the case is that the CustomJS callback just happens to be the one executing first. That’s just speculation, but it would explain things. However, there is no guarantee of that order so as you note it’s probably not reliable to rely on.

Setting some other property that can trigger a redraw is a way avoid the callback cylce, it’s slightly hacky but will get the job done. There is in principle a request_redraw that can be called on plot views, but I have no direct experience with it, see this comment for some info:

1 Like
16

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.