I have two separate bokeh server applications running on separate hosts in our network.
Measurement.py, which
provides control of a data acquisition system
displays the acquired raw data
save the raw data to a database and appends new data, when available
can start-up Analysis.py with a database link as request-argument in new browser window
Analysis.py which:
can open a database link given as request-argument to access raw data
display the raw data (again)
make an in depth analysis (fitting, statistics etc.) of the data
reload updated raw data sets from the same data base link
Now, it would be nice to have the capability to send an (asynchronous inter-process communication (IPC)) message from Measurement.py to Analysis.py, whenever the raw-data has been updated.
Analysis.py could then reload the data and progress with the analysis.
The rabbit hole I went down into was the idea to use a websocket between the two processes.
However, the combination of the usual asyncio examples for general websockets with the challenge of an already running “tornado.IOloop” in the bokeh servers is beyond my comprehension.
The more I read the more confused I get.
Hence my question is:
Is there some example code on how to setup an additional websocket (server, client) in a bokeh server application and “await” a message on that websocket from an external application?
Note 1: If there is a simpler solution to accomplish this IPC task, I would be happy about suggestions, too.
Note 2: Of course Analysis.py could poll the data base for changes, but that’s only the last resort, here.
Note 3: I think this here probably points in a good direction, but raises more questions to me than answers.
Hi @ThomasB this is more or less uncharted territory, so I don’t have any examples that I can point to. Adding a websocket handler seem plausible, but as you have observed, would involve a fair amount of coding with low-level Bokeh components. Another idea might be to leverage some other existing tool, e.g maybe Redis Pub/Sub would be useful for something like this (I have very little actual experience with Redis, but it seems applicable from my basic knowledge. There are probably other tools to consider as well.)
Hi @Bryan , thanks for looking into this.
Probably the focus of my question was not very clear, so I’d like to make it clearer:
It is not so much about what kind of IPC-tool to use, but rather on how to get my own asyncio/await functionality hooked up in the IOLoop of bokeh/tornado, that is started by bokeh server.
In relation to my sample case, Measurement.py could use a blocking call to send out the message “hey I have new raw data for you”, but (no matter what IPC I use), the Analysis.py should not block the bokeh GUI while it is waiting for such a message. So I need to use asyncio to await that “event”.
Hence it would be nice to have some rudimentary sample code on how to link up user-methods to the running IOLoop.
Hi @ThomasB unfortunately I don’t personally know of anything immediately at hand that I can point to. Broadly, I can just point to this doc section about embedding the Bokeh server as a library:
Doing that, you could e.g. add Tornado request handlers for websockets directly in the standard way Tornado allows.
I’d really very much for the the “Bokeh server protocol” to be liberated from Tornado, i.e. be something that could be used easily from a plain-python asyncio program, with the tornado bokeh serve just becoming something more like a “reference implementation”. Unfortunately that’s a heavy lift that will no longer fit in my (part) time available to work on Bokeh. Perhaps some day we can propose and obtain funding for that kind of work.