Question about session.loop_until_closed

Community Support
1

Hello,

I am a new user of Bokeh, and am attempting to build out a very simple application that uses bokeh serve to plot data streamed from the client (currently I’m using canned data but eventually you could imagine this connected to an embedded hardware target like a thermometer that is providing time-series data to be plotted). However, it seems like the loop_until_closed function recommended by the various server examples is at odds with interactive usage, since (as far as I’ve been able to tell) there is no way to get it to halt and return short of restarting the client python interpreter. So I have a couple of questions:

  • Is there any way to get session.loop_until_closed() to return? I would have thought that closing the browser window rendering the plot might have done it but that doesn’t seem to be the case…

  • If there is not, that’s fine but is there any other way to launch the server session with periodic callbacks running, that would (ideally) kick the event loop into a background process so that the python interpreter can be used to run other commands even as the plot is streamed to the browser, OR (less ideal but still much better than what I have) is there a way I can do this type of plot such that when I close the plot in the browser, this is detected and code after loop_until_closed() can be run?

I apologize if this is not the right venue for this type of question but I’m hoping that someone can help - I really like the idea of bokeh but I’m having trouble getting from canned examples into writing my own code at this point. A couple other details are below. Thanks in advance!

Brian

Implementation details:

Bokeh version: 0.11

Python: Anaconda 2.4.1 (Python 2.7.x), 64-bit

Operating System: Windows 7 x64

Default browser: Chrome

The code I’m attempting to run

import time

from random import shuffle

import numpy as np

from bokeh.client import push_session

from bokeh.plotting import figure, show, curdoc

from bokeh.models import HoverTool

p = figure(plot_width=600, plot_height=600, tools=‘pan,box_zoom,box_select,crosshair,resize,reset,hover’)

p = figure(x_range=(-1, 20), y_range=(-2,2), plot_width=400, plot_height=400)

r1 = p.line([0], [0], name=‘ex_line’)

session = push_session(curdoc())

ds = r1.data_source

i = 1

def update():

global i

ds.data['x'].append(i)

ds.data['y'].append(np.random.random())

ds.trigger('data',ds.data,ds.data)

i = i + 1

curdoc().add_periodic_callback(update,100)

session.show()

session.loop_until_closed()

``

2

Hi,

···

First, some conceptual explanation of what’s happening here.

This is a client script; it opens a websocket to the server and pushes the document (in push_session()), then while the session is open it sends any changes to the document over that websocket to the server.

loop_until_closed() runs a Tornado IOLoop (IOLoop.start() in the Tornado docs) until the session is closed, either because the server exits (kill the server) or because you call session.close(). The reason we run the IOLoop is to wait for events from the server, and to run the periodic callback (or any other callback you might add).

session.show() opens a browser, which creates its own separate websocket connection to the server, using the same session ID as the session you called show() on. The browser’s connection is independent of the client’s connection, but because of the shared session ID, both connections are looking at the same document. Changes to the document made in any of the three places (browser client, python client, or server) will be synced to the other two places.

Anyway, so some answers with that background -

On Tue, Jan 12, 2016 at 1:55 PM, [email protected] wrote:

  • Is there any way to get session.loop_until_closed() to return? I would have thought that closing the browser window rendering the plot might have done it but that doesn’t seem to be the case…

session.close() ends the connection to the server, or if the server exits the session will close. Closing the browser window closes the other connection (from the browser), which doesn’t affect your connection.

You can if you wish call session.close() from a callback - either a periodic callback, or a callback in response to a button click or any change to the document that you like.

You could also create the Tornado IOLoop yourself (by importing Tornado) and pass it in to push_session(io_loop=myloop). Then you can myloop.start() and myloop.stop() as you see fit, and not use session.loop_until_closed().

However, really you probably do not want to stop the IOLoop, because you need it in order to get events and run callbacks.

  • If there is not, that’s fine but is there any other way to launch the server session with periodic callbacks running, that would (ideally) kick the event loop into a background process so that the python interpreter can be used to run other commands even as the plot is streamed to the browser, OR (less ideal but still much better than what I have) is there a way I can do this type of plot such that when I close the plot in the browser, this is detected and code after loop_until_closed() can be run?

These are really general questions about threads and Tornado, so you could probably find a lot of good resources out there by searching for Tornado and Python threads, rather than looking for Bokeh stuff specifically.

I think your easiest approach is probably to make your other commands run from the IOLoop, rather than trying to avoid running the IOLoop. You can schedule callbacks with add_timeout_callback, add_next_tick_callback, add_periodic_callback, and also with on_change on the Document or any Model. You can also make Tornado call a callback if there’s input on a file descriptor, though Bokeh doesn’t wrap that API, you could use it directly from Tornado to detect say someone typing a command. I don’t know offhand how to do this but the Tornado docs should have it.

A possible approach I don’t know much about would be to start a separate thread; I do not know how well this will really work, or how threads interact with Tornado’s IOLoop. In any case it’s just going to make the whole situation much more complicated… if the IOLoop seems complicated, adding threads to it makes it complicated-squared.

Another angle you could take is to move your app to be a server script, rather than a client (that is, make it a thing you run under bokeh serve myapp.py like examples/app/sliders.py etc.). You could make the server script do the periodic updates, and make your client do the interaction, or something like that. Or… maybe you could have two different clients, one that drives the updates, and the other that does user interaction. In any case, think about options where you have two processes (which means two scripts probably, either two clients or a client and a server one).

It’s possible two processes is easier than trying to set up IO callbacks on the IOLoop - it sort of depends on which one is more natural for your app and more familiar to you.

Havoc

3

Thanks for the very detailed response! I will definitely look more closely at the Tornado API to build my understanding of how the server event loop works, and I’ll pursue the specific suggestions you’ve made about adding a button or other technique within the event loop to stop execution.

···

On Tuesday, January 12, 2016 at 9:25:14 PM UTC-5, Havoc Pennington wrote:

Hi,

First, some conceptual explanation of what’s happening here.

This is a client script; it opens a websocket to the server and pushes the document (in push_session()), then while the session is open it sends any changes to the document over that websocket to the server.

loop_until_closed() runs a Tornado IOLoop (IOLoop.start() in the Tornado docs) until the session is closed, either because the server exits (kill the server) or because you call session.close(). The reason we run the IOLoop is to wait for events from the server, and to run the periodic callback (or any other callback you might add).

session.show() opens a browser, which creates its own separate websocket connection to the server, using the same session ID as the session you called show() on. The browser’s connection is independent of the client’s connection, but because of the shared session ID, both connections are looking at the same document. Changes to the document made in any of the three places (browser client, python client, or server) will be synced to the other two places.

Anyway, so some answers with that background -

On Tue, Jan 12, 2016 at 1:55 PM, [email protected] wrote:

  • Is there any way to get session.loop_until_closed() to return? I would have thought that closing the browser window rendering the plot might have done it but that doesn’t seem to be the case…

session.close() ends the connection to the server, or if the server exits the session will close. Closing the browser window closes the other connection (from the browser), which doesn’t affect your connection.

You can if you wish call session.close() from a callback - either a periodic callback, or a callback in response to a button click or any change to the document that you like.

You could also create the Tornado IOLoop yourself (by importing Tornado) and pass it in to push_session(io_loop=myloop). Then you can myloop.start() and myloop.stop() as you see fit, and not use session.loop_until_closed().

However, really you probably do not want to stop the IOLoop, because you need it in order to get events and run callbacks.

  • If there is not, that’s fine but is there any other way to launch the server session with periodic callbacks running, that would (ideally) kick the event loop into a background process so that the python interpreter can be used to run other commands even as the plot is streamed to the browser, OR (less ideal but still much better than what I have) is there a way I can do this type of plot such that when I close the plot in the browser, this is detected and code after loop_until_closed() can be run?

These are really general questions about threads and Tornado, so you could probably find a lot of good resources out there by searching for Tornado and Python threads, rather than looking for Bokeh stuff specifically.

I think your easiest approach is probably to make your other commands run from the IOLoop, rather than trying to avoid running the IOLoop. You can schedule callbacks with add_timeout_callback, add_next_tick_callback, add_periodic_callback, and also with on_change on the Document or any Model. You can also make Tornado call a callback if there’s input on a file descriptor, though Bokeh doesn’t wrap that API, you could use it directly from Tornado to detect say someone typing a command. I don’t know offhand how to do this but the Tornado docs should have it.

A possible approach I don’t know much about would be to start a separate thread; I do not know how well this will really work, or how threads interact with Tornado’s IOLoop. In any case it’s just going to make the whole situation much more complicated… if the IOLoop seems complicated, adding threads to it makes it complicated-squared.

Another angle you could take is to move your app to be a server script, rather than a client (that is, make it a thing you run under bokeh serve myapp.py like examples/app/sliders.py etc.). You could make the server script do the periodic updates, and make your client do the interaction, or something like that. Or… maybe you could have two different clients, one that drives the updates, and the other that does user interaction. In any case, think about options where you have two processes (which means two scripts probably, either two clients or a client and a server one).

It’s possible two processes is easier than trying to set up IO callbacks on the IOLoop - it sort of depends on which one is more natural for your app and more familiar to you.

Havoc