Synchronizing ColumnDataSource with bokeh server with when editing data in JS callbacks

Hi, I experience the following problem, with example code in this gist: custom-draw-lines · GitHub

The first part is a plot with built-in PolyDrawTool shown with bokeh server. After drawing with PolyDrawTool and clicking on ‘copy’ button, I can show the coordinates of drawn lines in python by running the callback_data cell. Clicking the button copies the data source data into a dict which is referenced in python code, using python callback.

Here is the code for that part:

callback_data = {}
def plots_server(doc):
    source = ColumnDataSource(data=dict(xs=[], ys=[]))
    plot = figure(x_range=(0, 10), y_range=(0, 10))
    plot.title.text = "Draw with built-in tool"
    renderer = plot.multi_line('xs', 'ys', line_width=4, line_color='#000000', source=source, line_cap='round')
    plot.add_tools(PolyDrawTool(renderers=[renderer]))
    def button_callback():
        callback_data['data'] = source.data
    button = Button(label="copy data")
    button.on_click(button_callback)
    doc.add_root(column(plot, widgetbox([button,])))
show(plots_server)

The second part is a similar plot where lines are drawn using custom javascript callbacks (tap to start drawing, tap to finish drawing). But data source is not updated on the python side. When I run callback_data cell, it always show {'data': {'xs': [], 'ys': []}}.

I wonder if I missed calling some additional sync function on javascript or python side to make it work similar to the built-in line editor. Or maybe there is a better approach on making a custom drawing tool that would synchronize data back to python side when running in bokeh server.

Here is the code for the second part:

ADD_LINE_JS = """
            var tool_name = 'test-line-plugin';
            // Init globals.
            if (window[tool_name] === undefined) {
              window[tool_name] = {'state': 'none'};
            }
            var tool = window[tool_name];
            // Shorthands.
            var cx = cb_obj.x; // current mouse x
            var cy = cb_obj.y; // current mouse y
            var xs = data.data.xs; // x of lines
            var ys = data.data.ys; // y of lines
            // React to mouse events.
              if (cb_obj.event_name == 'tap') {
                if (tool.state == 'none') {
                  // First tap: add start and end point of line.
                  tool.state = 'adding';
                  // Draw line.
                  xs.push([cx, cx]);
                  ys.push([cy, cy]);
                  // Update the plot.
                  data.change.emit();
                } else if (tool.state == 'adding') {
                  // Second tap: end point of line.
                  tool.state = 'none';
                  xs[xs.length-1][1] = cx;
                  ys[ys.length-1][1] = cy;
                  data.change.emit();
                }
              } else if (cb_obj.event_name == 'mousemove') {
                if (tool.state == 'adding') {
                  // Between first and second tap: intermediate end point.
                  xs[xs.length-1][1] = cx;
                  ys[ys.length-1][1] = cy;
                  data.change.emit();
                }
              }
        """

callback_data = {}
def plots_server(doc):
    source = ColumnDataSource(data=dict(xs=[], ys=[]))
    plot = figure(x_range=(0, 10), y_range=(0, 10))
    plot.title.text = "Draw with custom js callbacks"
    renderer = plot.multi_line('xs', 'ys', line_width=4, line_color='#000000', source=source, line_cap='round')
    plot.js_on_event(events.Tap, CustomJS(args=dict(data=source), code=ADD_LINE_JS))
    plot.js_on_event(events.MouseMove, CustomJS(args=dict(data=source), code=ADD_LINE_JS))
    def button_callback():
        callback_data['data'] = source.data
    button = Button(label="copy data")
    button.on_click(button_callback)
    doc.add_root(column(plot, widgetbox([button,])))
show(plots_server)

This example uses has custom line editor solely to show the problem with data source sync. My original goal is to draw Bezier curves, and since there is no built-in editor, I made a custom one using similar callbacks.

I used bokeh 1.4.0 for testing.

I’ve looked into bokeh source code and spotted these two lines, which actually solve the problem:

I’ve added them to the javascript callback and it started to work properly. Here’s the updated version of javascript code:

ADD_LINE_JS = """
            var tool_name = 'test-line-plugin';
            // Init globals.
            if (window[tool_name] === undefined) {
              window[tool_name] = {'state': 'none'};
            }
            var tool = window[tool_name];
            // Shorthands.
            var cx = cb_obj.x; // current mouse x
            var cy = cb_obj.y; // current mouse y
            var xs = data.data.xs; // x of lines
            var ys = data.data.ys; // y of lines
            // React to mouse events.
              if (cb_obj.event_name == 'tap') {
                if (tool.state == 'none') {
                  // First tap: add start and end point of line.
                  tool.state = 'adding';
                  // Draw line.
                  xs.push([cx, cx]);
                  ys.push([cy, cy]);
                  // Update the plot.
                  data.change.emit();
                } else if (tool.state == 'adding') {
                  // Second tap: end point of line.
                  tool.state = 'none';
                  xs[xs.length-1][1] = cx;
                  ys[ys.length-1][1] = cy;
                  data.change.emit();
                  data.data = data.data // this line is added
                  data.properties.data.change.emit() // this line is added
                }
              } else if (cb_obj.event_name == 'mousemove') {
                if (tool.state == 'adding') {
                  // Between first and second tap: intermediate end point.
                  xs[xs.length-1][1] = cx;
                  ys[ys.length-1][1] = cy;
                  data.change.emit();
                }
              }
        """

You should actually be getting a huge, multi-line console warning about trying to use .on_click with show. Did it now appear for you?

When you use show, you are generating standalone HTML output, i.e. Bokeh creates a pile of HTML and Javascript that can be displayed in a browser, or a notebook output cell, but it is a one-way trip. There is no further connection or communication to any Python process (so as a corollary, your on_click callback function has no place to execute). If you want to use real Python callbacks, that always means you are looking at creating at Bokeh server application to run on a Bokeh server.

You can find an example of embedding a Bokeh server app in a notebook here:

https://github.com/bokeh/bokeh/blob/master/examples/howto/server_embed/notebook_embed.ipynb

You an also see a discussion of the possibility of using kernel.execute in a CustomJS callback in the notebook, in order to update Python variables in the notebook, here:

Hi @Bryan, thanks for replying. You misunderstood my question. I’m using bokeh server, not generating standalone output, and there was communication between python and javascript. The problem was that this communication was going on properly behind the scenes when using PolyDrawTool to edit data source. Unlike in the case where I manually updated data source, where calling cds.change.emit(); was not enough. As it appears, I needed to add few more lines for things to work:

cds.data = cds.data;
cds.properties.data.change.emit();

Yes, it seems this related to the topic Getting selected glyph data properties via CustomJS - #4 by Matthias and Issue #7106.

Yes, there are lots of questions, and I usually manage to be quick and right, but every once in a while, only manage quick. The thread @Matthias linked has a lot of discussion around this topic.