Getting selected glyph data properties via CustomJS

Community Support
1

I’m trying to get the ColumnDataSource of a selected item (e.g. via Tap tool) into a Python variable. I’m sure this is simple but I don’t understand the CustomJS mechanism well enough to work out how to do it. It doesn’t help that my JS knowledge is virtually nonexistent!

I have a chart - it’s actually a process tree - I want people to be able to select one or more items and be able to retrieve the corresponding items from the ColumnDataSource (just getting the indices of the rect glyphs would be enough). All I want to do have the values persisted to some kind of array. I can then use this to pull more detailed information about the selected process(es). This would be some other python code that does queries from other sources - probably triggered by some other widget rather than from a bokeh event.

    from bokeh.io import output_notebook, show, output_file, reset_output
    from bokeh.plotting import figure
    from bokeh.transform import dodge, factor_cmap
    from bokeh.models import HoverTool, ColumnDataSource, CustomJS
    from bokeh.models.callbacks import CustomJS

    reset_output()
    output_notebook()
    n_levels = p_tree_srt["PlotLevel"].unique()

    source = ColumnDataSource(data=p_tree_srt)
    TOOLTIPS = [
        ("Name", "@NewProcessName"),
        ("Cmd", "@CommandLine"),
        ("User", "@SubjectUserName"),
        ("Time", "@TimeGenerated{%F %T}")
    ]
    plot_height = 35 * len(p_tree_srt)
    p = figure(title="ProcessTree", plot_width=900, plot_height=3500,
               x_range=(-1, max(n_levels) + 1),
               tools=["hover", "xwheel_zoom", "box_zoom", "reset", "save", "xpan", "tap"])
    hover = HoverTool(tooltips=TOOLTIPS, formatters={'TimeGenerated': 'datetime'})
    p.add_tools(hover)

    r = p.rect("PlotLevel", "Row", 3.5, 0.95, source=source, fill_alpha=0.6)
    text_props = {"source": source, "text_align": "left", "text_baseline": "middle"}
    x = dodge("PlotLevel", -1.7, range=p.x_range)
    p.text(x=x, y="Row", text="CommandLine", text_font_size="8pt", **text_props)
    p.text(x=x, y=dodge("Row", 0.3, range=p.y_range), text="NewProcessId",
           text_font_size="6pt", **text_props)
    p.text(x=dodge("PlotLevel", -1.4, range=p.x_range), y=dodge("Row", 0.3, range=p.y_range), text="Exe",
           text_font_size="6pt", **text_props)

    p.outline_line_color = None
    p.grid.grid_line_color = None
    p.axis.axis_line_color = None
    p.axis.major_tick_line_color = None
    p.axis.major_label_standoff = 0
    p.hover.renderers = [r] # only hover element boxes

    target = ColumnDataSource()
    callback = CustomJS(args=dict(source=source, target=target), code="""
            // get data source from Callback args
            var inds = source.selected.indices;
            
            target.data = inds;
            cb_data.source.change.emit();  // I've tried multiple things here
        """)
    p.js_on_event('tap', callback)
    show(p)
2

I’m not a JS guy myself, but my first guess would be to just keep adding selected indices to the target list:

target = ColumnDataSource(data=dict(ids=[]))
callback = CustomJS(args=dict(source=source, target=target), code="""
            // get data source from Callback args
            var inds = source.selected.indices;
            
            // concatenate old selection with new selection
            target.data["ids"] = target.data["ids"].concat(inds);
        """)

Then you could use python unique again to extract only unique elements if some where selected multiple times.

1 Like
3

Thanks Matthias,
I’ve made a bit of progress.
First bit of illumination is using a ColumnDataSource object as the way to return data from the CustomJS code.
I can see that target.data is getting updated in the JS (from the console.log output) but this value doesn’t seem to make it to the target variable defined in the Python code.

data = {'proc_key': []}
target = ColumnDataSource(data)
callback = CustomJS(args=dict(source=source, target=target), code="""
        // get data source from Callback args
        var inds = source.selected.indices;
        for (var i = 0; i < inds.length; i++) {
            console.log(source.data['proc_key'][inds[i]]);
            target.data['proc_key'].push(source.data['proc_key'][inds[i]])
        }
        console.log(target.data);
        target.change.emit();
    """)

JS console output (first item is the “proc_key”, second is the target.data value with the key updated.

c:\program files (x86)\google\update\googleupdate.exe0x1fa40x3e72019-02-10 05:28:02.243000
Target data
{…}
​proc_key: Array [ "c:\\program files (x86)\\google\\update\\googleupdate.exe0x1fa40x3e72019-02-10 05:28:02.243000" ]
4

After searching a bit I found this issue: Data Source not synchronising client initiated changes to the server side. · Issue #7106 · bokeh/bokeh · GitHub
It seems you have to re-assign or copy the data.

Taking your code that means

data = {'proc_key': []}
target = ColumnDataSource(data)
callback = CustomJS(args=dict(source=source, target=target), code="""
        // get data source from Callback args
        var inds = source.selected.indices;

        // assign data to new data object
        var data = target.data;

        // update new data
        for (var i = 0; i < inds.length; i++) {
            console.log(source.data['proc_key'][inds[i]]);
            data['proc_key'].push(source.data['proc_key'][inds[i]])
        }

        // exchange old with new data
        target.data = data;
        
        console.log(target.data);
        target.change.emit();
    """)

I tested it using bokeh serve and it worked for me.

@Bryan 's statement was

The use case for updating JS data to sync python is by far less common (usually it is some computation in python using numpy or pandas, etc to update the data) so I don’t think I would do anything about this except possibly try to document it better.

1 Like
5

Thanks again Matthias.
I’m not using Bokeh server - just the local JS client.
I can see that the output data source (“target.data”) is updated before the JS callback returns but this never makes it in the corresponding Python object. I’m concluding - probably obvious when you think about it - that when the Bokeh JS object is instantiated it uses the Python objects as sources for data but then is essentially disconnected from them - so updates to the JS model are not propagated back to their Python parents.

However, I’ve found a bit of a hacky workaround using IPython.notebook.kernel.execute to create or overwrite a variable in the Python user namespace. This seems to work but it feels like I’m tampering with fundamental laws of physics and it may possibly mess me up at some point.

Here’s an example:

on_color_change = CustomJS(args=dict(plot=circ, selector=dropdown, range_tool=range_tool,
                                    out=out_source),
                    code="""
    var out_max = `'max': ${range_tool.x_range.end}`;
    var out_min = `'min': ${range_tool.x_range.start}`;
    var out_col = `'color': ${selector.value}`;
    var out_data = `foo={ ${out_max}, ${out_min}, ${out_col} }`;
    
    console.log(out_data);
    IPython.notebook.kernel.execute(out_data);
""")
6

when the Bokeh JS object is instantiated it uses the Python objects as sources for data but then is essentially disconnected from them - so updates to the JS model are not propagated back to their Python parents.

That is exactly the case for everything outside a Bokeh server application [1] (the Bokeh Server is the thing the keeps both sides in sync).

However, I’ve found a bit of a hacky workaround using IPython.notebook.kernel.execute to create or overwrite a variable in the Python user namespace.

Certainly tilts towards “hacky” but also certainly something people do, and evidently with some success. It might lead to some surprising behavior in some situations, but the potential for out-of-order cell execution in notebooks means that possibility always lurks, with or without Bokeh.

The “non-hacky” way would be to embed a Bokeh server app in a notebook. FWIW I think this is fairly straightforward to do, but it does carry its own assumptions and requirements. Basically, all the “app” code needs to be collected in a single function:

https://github.com/bokeh/bokeh/blob/master/examples/howto/server_embed/notebook_embed.ipynb

There is some interest and exploratory work ongoing around just letting arbitrary Python Bokeh objects be automatically synced to JS objects. Frankly I am not very optimistic about it, though. The above-mentioned potential for out-of-order cell execution (the notebook’s worst mis-feature IMO) just makes this impossible to make workable in a reasonable way.

  1. Half-exception: push_notebook can update the JS from Python, in notebooks. But there is no mechanism to go the other direction, to update Python from JS, which is what you are after. ↩︎

1 Like
7

Thanks Bryan,
Yeah the whole idea of eval’ing a python statement in a JS callback makes feel a bit dirty but I think if I’m careful I can use this to return Tap and box select selections. The main danger seems to be over-writing some existing var but I can pass the var name to the JS callback.
I like the idea of embedding the Bokeh server in the notebook - I’m going to experiment with this. In the meantime I think I’ve ended up with a pretty cool-looking Process tree viewer (the good looks are all Bokeh). Awesome stuff!

8

Another thing you can consider: define something mutable like a dict and have the eval’ed Python code update something inside the dict, rather than overwriting things. Maybe nicer in a small way?

In the meantime I think I’ve ended up with a pretty cool-looking Process tree viewer (the good looks are all Bokeh). Awesome stuff!

Thanks for the kind words! If it’s public/shareable, we love to tweet or retweet flashy screenshots!

9

The dict is a good idea, I’d thought of wrapping the whole thing in a class an using a class or function and requiring the user to specify a return variable.

If it’s public/shareable, we love to tweet or retweet flashy screenshots!

It needs a bit of tidying up and am going to put it in our msticpy package but happy to share it. Here’s what it currently looks like.

ProcessTree
ProcessTree1020×689 46.3 KB

10

Actually yes, having kernel.execute call funtions or methods is even cleaner and better. That looks awesome thanks for sharing! Please let us know if you tweet a release out :smiley:

11

Can you give more details of using IPython.notebook.kernel.execute(out_data);?
It seems only work in Jupyter notebook but raise an error ReferenceError: Can’t find variable: IPython when run in Pycharm.

12

@1113 kernel.execute is a Jupyter API specifically for executing Python code in a Jupyter kernel, from a notebook front end. It’'s not going to be work/be available outside the notebook.