Differences in plot update mechanisms

Community Support
1

Hello there,

I’m new to bokeh and already very found of it, so before anything else: Thanks a lot for this great tool! Now back to business: It took me quite a while to find ways to successfully live update plot data, and now that I managed I have at least two option, where I don’t really know which one to prefer and what are the technical differences. In both ways I am using a thread to read out the data (which changes depending on user input through a slider), but before I say anything wrong, here is the code (methods in a class), method 1:

def __update1(self, update_interval):

Benchmarking average on 200 runs:

only self.session.store_document(self.document): 0.134s, with few values for y: 0.087s

while True:
# getting the new data
d2d.simu()
y = d2d.get_x()

   for i in range(len(y)):
     self.source.data['p'+str(i)] = y[i]

 self.session.store_document(self.document)

 time.sleep(update_interval)

``

And method 2:

def __update2(self,update_interval):

self.p is the figure

my_renderer = self.p.select(dict(type=GlyphRenderer))
data_src = my_renderer[0].data_source

Benchmark average on 200 runs: only cursession.store_objects: 0.091s, with few values for y: 0.046s

while True:
# getting the new data
d2d.simu()
y = d2d.get_x()

 for i in range(len(y)):
   data_src.data["p"+str(i)] = y[i]

 cursession().store_objects(data_src)

 time.sleep(update_interval)

``

I’ve included the short benchmark test, because it might be interesting to see that the second method is noticeably faster. Although both methods work, there a some differences in user experience:

Method 1 doesn’t allow to zoom/move the graph - yes, theoretically it works, but with every store_document call, the plot resets to match all the data - therefor the user can’t zoom in etc. This is of course not a wanted behavior, are there ways to stop it from resetting?
Method 2 doesn’t have this problem, but when I open the website (embedded in Flask) on a second computer, the data will be updated for both if one of the users changes the data. Well, that’s not too bad considering that the app crashes with two users on method 1, but still not a wanted behavior. But this is probably a topic for itself :wink:

I just found a third option to update the plot, with the AjaxDataSource from bokeh.models.sources, but I didn’t look to much into it yet - it seems a bit risky doing so much on the client side, but if you tell me that’s the only true way to do it, I’m happy to try it out.

Thank you very much and I hope you can enlighten me about the relevant differences in the methods and what would be the proper way to live update plot data.

Best regards
Clemens

2

Hi Clemens,

Thank you very much for you kind words.

Regarding your question. With store_document you store the entire document object on the server (and all it’s models/objects). This mean everything is updated and re-created. This is why you are facing that weird behaviour when zooming and panning. From your code I think it’s not what you want to do.

In your case you just want to update your DataSource because it has changed. Calling store_objects you are just updating the objects you need. This means that all the objects related to your document are not being updated entirely and this is why both users see the same thing. Because the document is the same. If you need a document “per user” you then need to not use the same document but honestly I don’t understand because you say you want to show the same data (updated by __updated2) so I’d think you want all the plots visualized by your users to be updated synchronously.

AjaxDataSource is a slightly different object and let’s you update your plots by pointing the source to an API that will generate JSON data that “feeds” the source itself, without having to use a bokeh-server at all. Depending on what you need it may be the best solution for you but make sure you read the documentation and try it first to understand how it works and its limitations. Also, what do you mean by “it seems a bit risky doing so much on the client side”?

Also, what is your stack design? Are you using bokeh-server to serve your app or are you integrating it inside your own flask app?

Cheers

Fabio

···

On Monday, May 4, 2015 at 1:33:57 PM UTC+2, Clemens Blank wrote:

Hello there,

I’m new to bokeh and already very found of it, so before anything else: Thanks a lot for this great tool! Now back to business: It took me quite a while to find ways to successfully live update plot data, and now that I managed I have at least two option, where I don’t really know which one to prefer and what are the technical differences. In both ways I am using a thread to read out the data (which changes depending on user input through a slider), but before I say anything wrong, here is the code (methods in a class), method 1:

def __update1(self, update_interval):

Benchmarking average on 200 runs:

only self.session.store_document(self.document): 0.134s, with few values for y: 0.087s

while True:
# getting the new data
d2d.simu()
y = d2d.get_x()

   for i in range(len(y)):
     self.source.data['p'+str(i)] = y[i]

 self.session.store_document(self.document)

 time.sleep(update_interval)

``

And method 2:

def __update2(self,update_interval):

self.p is the figure

my_renderer = self.p.select(dict(type=GlyphRenderer))
data_src = my_renderer[0].data_source

Benchmark average on 200 runs: only cursession.store_objects: 0.091s, with few values for y: 0.046s

while True:
# getting the new data
d2d.simu()
y = d2d.get_x()

 for i in range(len(y)):
   data_src.data["p"+str(i)] = y[i]

 cursession().store_objects(data_src)

 time.sleep(update_interval)

``

I’ve included the short benchmark test, because it might be interesting to see that the second method is noticeably faster. Although both methods work, there a some differences in user experience:

Method 1 doesn’t allow to zoom/move the graph - yes, theoretically it works, but with every store_document call, the plot resets to match all the data - therefor the user can’t zoom in etc. This is of course not a wanted behavior, are there ways to stop it from resetting?
Method 2 doesn’t have this problem, but when I open the website (embedded in Flask) on a second computer, the data will be updated for both if one of the users changes the data. Well, that’s not too bad considering that the app crashes with two users on method 1, but still not a wanted behavior. But this is probably a topic for itself :wink:

I just found a third option to update the plot, with the AjaxDataSource from bokeh.models.sources, but I didn’t look to much into it yet - it seems a bit risky doing so much on the client side, but if you tell me that’s the only true way to do it, I’m happy to try it out.

Thank you very much and I hope you can enlighten me about the relevant differences in the methods and what would be the proper way to live update plot data.

Best regards
Clemens

3

Hi Fabio,

thanks for the explanation, this helped me a lot to understand, updating data_source seems to be the right thing for me. Stupid me, I had exactly the same data source, so of course it had to be the same for every user :wink:

Concerning AjaxDataSource: First of all, as you might have already guessed, I’m absolutely new to everything of programming and web etc. - so I just might not know what I’m talking about, but I thought with bokeh-server the plotting/computing is done on the server, and therefore it might work better on low-end/tablets devices? Plus, it might be less traffic needed when the data communication python - bokeh-server is on the server, instead of always sending the full data to the client?

I’m talking about ±10 lines with each ±1000 points, updating as much as possible (30 times a second?) for a smooth look.

My stack design? Everything is new to me and I’ve just started, I’m writing my masters thesis and would like to have an interface to present and update the data in real time (e.g. using sliders to set parameters etc.).
The data is computed by matlab and accessed through python, so it’s basically like this:

User changes slider → set new parameter in matlab → compute new data in matlab → send data back to python/bokeh to rerender.

This all should be visible in the browser, so right now my approach is to use flask (to make some interface around the graphs) + bokeh-server (or without, when using AjaxDataSource).

Does this sound like a realistic approach?

Cheers

Clemens

···

2015-05-05 13:22 GMT+02:00 Fabio Pliger [email protected]:

Hi Clemens,

Thank you very much for you kind words.

Regarding your question. With store_document you store the entire document object on the server (and all it’s models/objects). This mean everything is updated and re-created. This is why you are facing that weird behaviour when zooming and panning. From your code I think it’s not what you want to do.

In your case you just want to update your DataSource because it has changed. Calling store_objects you are just updating the objects you need. This means that all the objects related to your document are not being updated entirely and this is why both users see the same thing. Because the document is the same. If you need a document “per user” you then need to not use the same document but honestly I don’t understand because you say you want to show the same data (updated by __updated2) so I’d think you want all the plots visualized by your users to be updated synchronously.

AjaxDataSource is a slightly different object and let’s you update your plots by pointing the source to an API that will generate JSON data that “feeds” the source itself, without having to use a bokeh-server at all. Depending on what you need it may be the best solution for you but make sure you read the documentation and try it first to understand how it works and its limitations. Also, what do you mean by “it seems a bit risky doing so much on the client side”?

Also, what is your stack design? Are you using bokeh-server to serve your app or are you integrating it inside your own flask app?

Cheers

Fabio

On Monday, May 4, 2015 at 1:33:57 PM UTC+2, Clemens Blank wrote:

Hello there,

I’m new to bokeh and already very found of it, so before anything else: Thanks a lot for this great tool! Now back to business: It took me quite a while to find ways to successfully live update plot data, and now that I managed I have at least two option, where I don’t really know which one to prefer and what are the technical differences. In both ways I am using a thread to read out the data (which changes depending on user input through a slider), but before I say anything wrong, here is the code (methods in a class), method 1:

def __update1(self, update_interval):

Benchmarking average on 200 runs:

only self.session.store_document(self.document): 0.134s, with few values for y: 0.087s

while True:
# getting the new data
d2d.simu()
y = d2d.get_x()

   for i in range(len(y)):
     self.source.data['p'+str(i)] = y[i]

 self.session.store_document(self.document)

 time.sleep(update_interval)

``

And method 2:

def __update2(self,update_interval):

self.p is the figure

my_renderer = self.p.select(dict(type=GlyphRenderer))
data_src = my_renderer[0].data_source

Benchmark average on 200 runs: only cursession.store_objects: 0.091s, with few values for y: 0.046s

while True:
# getting the new data
d2d.simu()
y = d2d.get_x()

 for i in range(len(y)):
   data_src.data["p"+str(i)] = y[i]

 cursession().store_objects(data_src)

 time.sleep(update_interval)

``

I’ve included the short benchmark test, because it might be interesting to see that the second method is noticeably faster. Although both methods work, there a some differences in user experience:

Method 1 doesn’t allow to zoom/move the graph - yes, theoretically it works, but with every store_document call, the plot resets to match all the data - therefor the user can’t zoom in etc. This is of course not a wanted behavior, are there ways to stop it from resetting?
Method 2 doesn’t have this problem, but when I open the website (embedded in Flask) on a second computer, the data will be updated for both if one of the users changes the data. Well, that’s not too bad considering that the app crashes with two users on method 1, but still not a wanted behavior. But this is probably a topic for itself :wink:

I just found a third option to update the plot, with the AjaxDataSource from bokeh.models.sources, but I didn’t look to much into it yet - it seems a bit risky doing so much on the client side, but if you tell me that’s the only true way to do it, I’m happy to try it out.

Thank you very much and I hope you can enlighten me about the relevant differences in the methods and what would be the proper way to live update plot data.

Best regards
Clemens

–

You received this message because you are subscribed to the Google Groups “Bokeh Discussion - Public” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To post to this group, send email to [email protected].

To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/8df3f41b-3c7d-4816-a5b5-9e9214e41e25%40continuum.io.

For more options, visit https://groups.google.com/a/continuum.io/d/optout.

4

Hi Fabio,

thanks for the explanation, this helped me a lot to understand, updating
data_source seems to be the right thing for me. Stupid me, I had exactly
the same data source, so of course it had to be the same for every user :wink:

Concerning AjaxDataSource: First of all, as you might have already
guessed, I'm absolutely new to everything of programming and web etc. -

We all have been new to something, so no worries :wink:

so I just might not know what I'm talking about, but I thought with
bokeh-server the plotting/computing is done on the server, and therefore it
might work better on low-end/tablets devices? Plus, it might be less
traffic needed when the data communication python - bokeh-server is on the
server, instead of always sending the full data to the client?

Yes, the AjaxDataSource should let you save some bandwith but IMO using
bokeh-server is good fit too. It's also important to consider the clients
support you want for you app.

I'm talking about ±10 lines with each ±1000 points, updating as much as
possible (30 times a second?) for a smooth look.

You should be fine.

My stack design? Everything is new to me and I've just started, I'm
writing my masters thesis and would like to have an interface to present
and update the data in real time (e.g. using sliders to set parameters
etc.).
The data is computed by matlab and accessed through python, so it's
basically like this:

User changes slider -> set new parameter in matlab -> compute new data in
matlab -> send data back to python/bokeh to rerender.

This all should be visible in the browser, so right now my approach is to
use flask (to make some interface around the graphs) + bokeh-server (or
without, when using AjaxDataSource).

Does this sound like a realistic approach?

Yes. I think so. I'd suggest you to do some preliminary exploration with
bokeh-server and AjaxDataSource and measure simple raw results regards
performance and support on the devices/browsers you'd like to support.

Cheers

Fabio

···

On Tue, May 5, 2015 at 3:45 PM, Clemens Blank <[email protected]> wrote: