Streaming data without bokeh-server

Bokeh is very promising, but the main thing that has kept me back from using it so far is that I can’t figure out how to stream data to update plots without using bokeh-server. My particular use case is that I have a Tornado web app which logs data from some remote hardware and browsers can subscribe to a WebSocket or EventSource to receive new data. Ideally, the web app would generate and send the initial plot down the wire, but then only send new data from there on out, leaving the client-side Javascript to append to the plot. I could in principle generate a new plot each time, but this is quite inefficient once the dataset grows large enough.

The plot generation part is easy: I can just use bokeh.embed.components to generate the initial HTML and ensure that the required Javascript is already loaded in the browser (an alternative which might be even nicer would be to JSON serialize the plot and render from that). But then how do I use BokehJS to update data? The documentation seems to be a little lacking in this area.

Thanks for your help,
Mike

hi Mike

I am not sure if AjaxDataSource depends on the bokeh-server

If not , it might solve your problem

···

On Sunday, October 25, 2015 at 7:47:00 PM UTC+8, [email protected] wrote:

Bokeh is very promising, but the main thing that has kept me back from using it so far is that I can’t figure out how to stream data to update plots without using bokeh-server. My particular use case is that I have a Tornado web app which logs data from some remote hardware and browsers can subscribe to a WebSocket or EventSource to receive new data. Ideally, the web app would generate and send the initial plot down the wire, but then only send new data from there on out, leaving the client-side Javascript to append to the plot. I could in principle generate a new plot each time, but this is quite inefficient once the dataset grows large enough.

The plot generation part is easy: I can just use bokeh.embed.components to generate the initial HTML and ensure that the required Javascript is already loaded in the browser (an alternative which might be even nicer would be to JSON serialize the plot and render from that). But then how do I use BokehJS to update data? The documentation seems to be a little lacking in this area.

Thanks for your help,
Mike

Hi,

···

On Monday, October 26, 2015 at 11:35:38 AM UTC+1, Naich An wrote:

I am not sure if AjaxDataSource depends on the bokeh-server

If not , it might solve your problem

I was not aware of this, so thanks for suggesting it. I now see that while this is not particularly well-documented, there is at least an example: https://github.com/bokeh/bokeh/blob/master/examples/plotting/file/ajax_source.py

However, this is more of a workaround than a true solution. The problem here is that to use this properly, the server needs to keep all data in memory and send it all down the wire every time a client wants an update. The model I have in mind is instead to only send new data points and have them appended to existing data. I’ll look through bokeh.models.sources and see if I can create a custom DataSource that does what I want.

Mike

Hi Mike,

A few random comments:

in general, because Bokeh is actually a cross-langage project, adding a new model (like a new DataSource) means also adding a corresponding JavaScript implementation. So this would currently mean deploying your own custom build of BokehJS. There is currently work in flight to make the model system extensible, so that anyone can easily add their own new custom models aith JS implementations, without requiring to build BokehJS. This will land in 0.11 release.

The AjaxDataSource does have an append mode:

  http://bokeh.pydata.org/en/latest/docs/reference/models/sources.html#bokeh.models.sources.AjaxDataSource.mode

although do not know that it has been exercised very often by anyone yet.

If the REST api data needs to be adapted from a different format, it's also certainly possible to write a small bit of JS to make the AJAX call, and to push updates into your data source. (To be honest I thought AjaxDataSource accepted a callback that would enable this, but apparently it has not been implemented yet).

Finally note the the completely new server is being written from the ground up with data patching as a primary operation in the protocol in order to improve streaming support.

···

On Oct 26, 2015, at 6:10 AM, [email protected] wrote:

Hi,

On Monday, October 26, 2015 at 11:35:38 AM UTC+1, Naich An wrote:

I am not sure if AjaxDataSource depends on the bokeh-server

If not , it might solve your problem

I was not aware of this, so thanks for suggesting it. I now see that while this is not particularly well-documented, there is at least an example: https://github.com/bokeh/bokeh/blob/master/examples/plotting/file/ajax_source.py

However, this is more of a workaround than a true solution. The problem here is that to use this properly, the server needs to keep all data in memory and send it all down the wire every time a client wants an update. The model I have in mind is instead to only send new data points and have them appended to existing data. I'll look through `bokeh.models.sources` and see if I can create a custom `DataSource` that does what I want.

Mike

--
You received this message because you are subscribed to the Google Groups "Bokeh Discussion - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/4579c0ec-d5f8-420c-a1b6-95e186896b93%40continuum.io.
For more options, visit https://groups.google.com/a/continuum.io/d/optout.

Hi Bryan,

in general, because Bokeh is actually a cross-langage project, adding a new model (like a new DataSource) means also adding a corresponding JavaScript implementation. So this would currently mean deploying your own custom build of BokehJS. There is currently work in flight to make the model system extensible, so that anyone can easily add their own new custom models aith JS implementations, without requiring to build BokehJS. This will land in 0.11 release.

This is good news. I had read somewhere that work on improving BokehJS was underway, although I haven’t found much in the way of a definitive description on what is in the works.

The AjaxDataSource does have an append mode:

    [http://bokeh.pydata.org/en/latest/docs/reference/models/sources.html#bokeh.models.sources.AjaxDataSource.mode](http://bokeh.pydata.org/en/latest/docs/reference/models/sources.html#bokeh.models.sources.AjaxDataSource.mode)

although do not know that it has been exercised very often by anyone yet.

I don’t know how I missed that option on the first reading. A quick test shows that it more or less does what I want, so thanks for pointing this out to me.

Finally note the the completely new server is being written from the ground up with data patching as a primary operation in the protocol in order to improve streaming support.

Interesting. I’ll look forward to this.

Thanks for all your hard work on this project. It’s already very nice, and it seems like it’s only getting better!

Mike

···

On Monday, October 26, 2015 at 2:50:59 PM UTC+1, Bryan Van de ven wrote:

Hi all,
What is the status of this feature?

I mean, is there now a way to periodically update a plot (i.e. to stream data) without starting a Bokeh server?

Thanks.

N.

···

Le dimanche 25 octobre 2015 12:47:00 UTC+1, [email protected] a écrit :

Bokeh is very promising, but the main thing that has kept me back from using it so far is that I can’t figure out how to stream data to update plots without using bokeh-server. My particular use case is that I have a Tornado web app which logs data from some remote hardware and browsers can subscribe to a WebSocket or EventSource to receive new data. Ideally, the web app would generate and send the initial plot down the wire, but then only send new data from there on out, leaving the client-side Javascript to append to the plot. I could in principle generate a new plot each time, but this is quite inefficient once the dataset grows large enough.

The plot generation part is easy: I can just use bokeh.embed.components to generate the initial HTML and ensure that the required Javascript is already loaded in the browser (an alternative which might be even nicer would be to JSON serialize the plot and render from that). But then how do I use BokehJS to update data? The documentation seems to be a little lacking in this area.

Thanks for your help,
Mike

For something already built-in to Bokeh, the only option is really the AjaxDataSource:

  http://bokeh.pydata.org/en/latest/docs/reference/models/sources.html#bokeh.models.sources.AjaxDataSource

There's an example here:

  https://github.com/bokeh/bokeh/blob/master/examples/howto/ajax_source.py

Otherwise, if your needs are more customized or sophisticated, you are probably looking at creating a custom extension:

  http://bokeh.pydata.org/en/latest/docs/user_guide/extensions.html

that suits your specific needs.

Thanks,

Bryan

···

On Jan 5, 2017, at 4:42 PM, Nicolas Leclercq <[email protected]> wrote:

Hi all,
What is the status of this feature?
I mean, is there now a way to periodically update a plot (i.e. to stream data) without starting a Bokeh server?
Thanks.
N.

Le dimanche 25 octobre 2015 12:47:00 UTC+1, [email protected] a écrit :
Bokeh is very promising, but the main thing that has kept me back from using it so far is that I can't figure out how to stream data to update plots without using bokeh-server. My particular use case is that I have a Tornado web app which logs data from some remote hardware and browsers can subscribe to a WebSocket or EventSource to receive new data. Ideally, the web app would generate and send the initial plot down the wire, but then only send new data from there on out, leaving the client-side Javascript to append to the plot. I could in principle generate a new plot each time, but this is quite inefficient once the dataset grows large enough.

The plot generation part is easy: I can just use `bokeh.embed.components` to generate the initial HTML and ensure that the required Javascript is already loaded in the browser (an alternative which might be even nicer would be to JSON serialize the plot and render from that). But then how do I use BokehJS to update data? The documentation seems to be a little lacking in this area.

Thanks for your help,
Mike

--
You received this message because you are subscribed to the Google Groups "Bokeh Discussion - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/485f9c98-2f7b-4c5b-aaea-e9ac48c8be50%40continuum.io.
For more options, visit https://groups.google.com/a/continuum.io/d/optout.