Bokeh non-blocking task

I’m trying to update plots using asyncio. The structure of my callback is:

def periodic_plot_update():
    update_plot("", "", "")
    set_title("", "", "")

async def apc_cb():

    _ = await asyncio.wrap_future(
                executor.submit(periodic_plot_update), loop=None
            )
    doc.add_next_tick_callback(partial(self.dummy_update))

apc_cb.nolock = True

self.apc = doc.add_periodic_callback(
        apc_cb,
        5000,
)

In the update_plot function, I do data processing and update the column data source by cds.update(data=df). dummy_update is a print function to let me know the update was run. Now I’m having a RuntimeError: RuntimeError: _pending_writes should be non-None when we have a document lock, and we should have the lock when the document changes. The error is at the cds.update step and I’m not sure why. Does anyone have any clue why? I did some small test, running print(“something”) inside the periodic_plot_update function, and it ran, but when involving the cds, it doesn’t.

Thank you,
H

See:

Hi Bryan. Wasn’t this not advised anymore as the flag without_document_lock was not smart enough? I have seen in another example where async def was used.

I don’t have any specific recollection of that, but I may just be forgetting. In any case, I don’t actually know, and haven’t ever tried to do what you are doing, so I think you will just have to experiment with without_document_lock and/or wrapping changes to Bokeh objects inside another add_next_tick_callback that your async function submits.

I was referring to this github issue: [BUG] Docs still mention @gen.coroutine · Issue #11716 · bokeh/bokeh · GitHub

It appears that was resolved here (for upcoming 3.0 release):

In the mean time the issue you link shows a workaround near the bottom.

Is there anyway to get it to work with Bokeh 2.4.2? I have issues with running Bokeh 3.0.0 as a drop-in replacement for my app.

As I said there is a workaround to try at the bottom:

[BUG] Docs still mention @gen.coroutine · Issue #11716 · bokeh/bokeh · GitHub

Are you saying you tried manually adding nolock and that does not work? That is the only suggestion I have for 2.4.2 at this time.

Yes. That was what I wrote in the original code. Blocking tasks such as sleep(longtime) work, but tasks regarding document updates (data update, plot title text setting,…) don’t work with this workaround.

OK, well you still need to add another layer of callbacks. You need to put all code that changes Bokeh models in a callback of their own. Then use add_next_tick_callback on that from your async function. This is described in the docs I linked:

As with the thread example above, all actions that update document state must go through a next tick callback .

I should add: there has been proposals to remove the document locking Bokeh does and in general make it play nicer with async. But this is a big task, I mostly only have enough enough async expertise to be a little dangerous, and definitely don’t (presently) have time/bandwidth. I would be happy to provide guidance or collaborate on this with a new contributor who has deep async expertise, and wants to dive into a non-trivial problem, though.

So after tinkering for a bit, I got it to work by wrapping the update_plot function and set_title function inside the doc.add_next_tick_callback(fn).

def periodic_plot_update():
    doc.add_next_tick_callback(partial(update_plot,"", "", ""))
    doc.add_next_tick_callback(partial(set_title, "", "", ""))

async def apc_cb():
    _ = await asyncio.wrap_future(
                executor.submit(periodic_plot_update), loop=None
    )

apc_cb.nolock = True

self.apc = doc.add_periodic_callback(
        apc_cb,
        5000,
)

Now background tasks involving change of the document works without blocking.

Thank you Bryan for the suggestion.

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.