control on callback flow

hello,

Attempting to grasp some control on the callbacks, I become really confused and i believe at the origin of some of the mess in my program.

The simple example below illustrates the problem: Callbacks are called sometimes and at unexpected times.

I have no idea what could be done to circumvent this problem, and if this should be qualified as a bug

import time

from bokeh.io import curdoc

from bokeh.layouts import column

from bokeh.models import ColumnDataSource

from bokeh.models.widgets import DataTable, TableColumn, Button

def cb(attr,old,new):

print(‘source’,attr,old[‘x’][0],’ --> ',new[‘x’][0])

source = ColumnDataSource(dict(index=[0], x=[False]))

source.on_change(‘data’,cb)

b = Button(width=100)

def button_cb():

new = not source.data[‘x’][0]

print(‘patch’,new)

source.patch({ ‘x’: [(0,new)] })

time.sleep(1)

print()

b.on_click(lambda: button_cb())

data_table = DataTable(source=source, width=100, columns=[TableColumn(field=“x”, title=“x”)])

curdoc().add_root(column(b,data_table))

Quickly hitting two and then three times the button, one gets (notice the 2 delayed callbacks after hitting the button three times):

patch True

source data True --> True

patch False

source data False --> False

patch True

source data True --> True

patch False

source data False --> False

patch True

source data True --> True

source data True --> False

source data False --> True

Hi,

Tornado (which the Boke server is built on) is an asynchronous web frame work. You should do everything possible to minimize any blocking in the callbacks. In particular, it's almost certainly always a mistake to call "sleep" inside one for any reason. The reason being that other events (e.g the other data change event) can come in at any point in time. If you block Tornado in one callback, that affects when others can get processed. If I remove the sleep from your code, the print statements are always paired, which is presumably what you expect/desire.

If you do need to perform long blocking tasks in the server, the best option is probably to update from a thread using "add_next_tick_callback" as described here:

  https://bokeh.pydata.org/en/latest/docs/user_guide/server.html#updating-from-threads

Thanks,

Bryan

···

On Dec 3, 2017, at 19:48, chupach <[email protected]> wrote:

hello,

Attempting to grasp some control on the callbacks, I become really confused and i believe at the origin of some of the mess in my program.
The simple example below illustrates the problem: Callbacks are called sometimes and at unexpected times.
I have no idea what could be done to circumvent this problem, and if this should be qualified as a bug

import time
from bokeh.io import curdoc
from bokeh.layouts import column
from bokeh.models import ColumnDataSource
from bokeh.models.widgets import DataTable, TableColumn, Button

def cb(attr,old,new):
  print('source',attr,old['x'][0],' --> ',new['x'][0])

source = ColumnDataSource(dict(index=[0], x=[False]))
source.on_change('data',cb)

b = Button(width=100)
def button_cb():
  new = not source.data['x'][0]
  print('patch',new)
  source.patch({ 'x': [(0,new)] })
  time.sleep(1)
  print()
b.on_click(lambda: button_cb())

data_table = DataTable(source=source, width=100, columns=[TableColumn(field="x", title="x")])
curdoc().add_root(column(b,data_table))

Quickly hitting two and then three times the button, one gets (notice the 2 delayed callbacks after hitting the button three times):

patch True
source data True --> True

patch False
source data False --> False

patch True
source data True --> True

patch False
source data False --> False

patch True
source data True --> True

source data True --> False
source data False --> True

--
You received this message because you are subscribed to the Google Groups "Bokeh Discussion - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/b23789bb-9faf-47c9-aa3a-23f42ecdd7ac%40continuum.io.
For more options, visit https://groups.google.com/a/continuum.io/d/optout.

thank you Bryan,
i believe i partly understand what you say, thinking of blocking thread.

the sleep is just simulating some computation times.

**My main point here was that **I do not understand is the logic of what we see when running the example i provided:

  • hit button twice, execute cb twice (cb is the source callback),

  • hit button three times, exexute cb 5 times (2 delayed with old and new which are different)

Just trying to get a grasp on the logic of the calls.

Bryan, the document from the link you provided states:

"Updating from Unlocked Callbacks

Normally Bokeh session callbacks recursively lock the document until all future work they initiate is completed. "

I would then expect i hit button, button_cb patch the source and then cb (source callback) is called.

And full stop.

The sleep time is introduced or not should not matter.

And then the stack of execution of callback proceeds with further button hits (in turn calling cb).

Why should it be more complicated than that ?

Hi,

Callbacks are initiated by events received from the client, so the data source callback actually requires a network round trip. An outgoing message from the serve to update the data source in the browser, and a message from the browser saying "the data source has changed" (which is what triggers the python callback).In the presence of callbacks that block tornado, these events can pile up.

Thanks,

Bryan

···

On Dec 4, 2017, at 13:11, chupach <[email protected]> wrote:

Bryan, the document from the link you provided states:

"Updating from Unlocked Callbacks¶

Normally Bokeh session callbacks recursively lock the document until all future work they initiate is completed. "

I would then expect i hit button, button_cb patch the source and then cb (source callback) is called.
And full stop.
The sleep time is introduced or not should not matter.

And then the stack of execution of callback proceeds with further button hits (in turn calling cb).

Why should it be more complicated than that ?

--
You received this message because you are subscribed to the Google Groups "Bokeh Discussion - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/10abebbd-d6d8-4860-889e-c0911b8631e0%40continuum.io.
For more options, visit https://groups.google.com/a/continuum.io/d/optout.

Bryan,

these async stuffs are complicated for non specialists…

so you suggest that the behaviour of calling the source callback 5 times when patching the source 3 times and the non locked document reversing the task order as normal

ok well, so no other choice than to follow your blocking recommendation as my only way out.

thanks.

···

On Monday, December 4, 2017 at 9:19:39 PM UTC+1, Bryan Van de ven wrote:

Hi,

Callbacks are initiated by events received from the client, so the data source callback actually requires a network round trip. An outgoing message from the serve to update the data source in the browser, and a message from the browser saying “the data source has changed” (which is what triggers the python callback).In the presence of callbacks that block tornado, these events can pile up.

Thanks,

Bryan

On Dec 4, 2017, at 13:11, chupach [email protected] wrote:

Bryan, the document from the link you provided states:

"Updating from Unlocked Callbacks¶

Normally Bokeh session callbacks recursively lock the document until all future work they initiate is completed. "

I would then expect i hit button, button_cb patch the source and then cb (source callback) is called.

And full stop.

The sleep time is introduced or not should not matter.

And then the stack of execution of callback proceeds with further button hits (in turn calling cb).

Why should it be more complicated than that ?


You received this message because you are subscribed to the Google Groups “Bokeh Discussion - Public” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To post to this group, send email to [email protected].

To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/10abebbd-d6d8-4860-889e-c0911b8631e0%40continuum.io.

For more options, visit https://groups.google.com/a/continuum.io/d/optout.

Bokeh definitely tries to make things simple and hide complicated details when possible, but some things are just inherently complex and Bokeh cannot always mask everything. However, the advice in this case is fairly simple to express: avoid long blocking work in call backs. And in case there is long blocking work to perform, we have provided examples in the docs that can be used as a starting point.

Thanks,

Bryan

···

On Dec 4, 2017, at 16:44, chupach <[email protected]> wrote:

Bryan,
these async stuffs are complicated for non specialists...
so you suggest that the behaviour of calling the source callback 5 times when patching the source 3 times and the non locked document reversing the task order as normal
ok well, so no other choice than to follow your blocking recommendation as my only way out.
thanks.

On Monday, December 4, 2017 at 9:19:39 PM UTC+1, Bryan Van de ven wrote:
Hi,

Callbacks are initiated by events received from the client, so the data source callback actually requires a network round trip. An outgoing message from the serve to update the data source in the browser, and a message from the browser saying "the data source has changed" (which is what triggers the python callback).In the presence of callbacks that block tornado, these events can pile up.

Thanks,

Bryan

> On Dec 4, 2017, at 13:11, chupach <[email protected]> wrote:
>
> Bryan, the document from the link you provided states:
>
> "Updating from Unlocked Callbacks¶
>
> Normally Bokeh session callbacks recursively lock the document until all future work they initiate is completed. "
>
>
>
> I would then expect i hit button, button_cb patch the source and then cb (source callback) is called.
> And full stop.
> The sleep time is introduced or not should not matter.
>
> And then the stack of execution of callback proceeds with further button hits (in turn calling cb).
>
> Why should it be more complicated than that ?
>
>
>
> --
> You received this message because you are subscribed to the Google Groups "Bokeh Discussion - Public" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
> To post to this group, send email to [email protected].
> To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/10abebbd-d6d8-4860-889e-c0911b8631e0%40continuum.io.
> For more options, visit https://groups.google.com/a/continuum.io/d/optout.

--
You received this message because you are subscribed to the Google Groups "Bokeh Discussion - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/ca8b2cc8-c992-44d3-a2d8-4e9a34f2b068%40continuum.io.
For more options, visit https://groups.google.com/a/continuum.io/d/optout.