How to properly remove an object from a document to avoid the "_pending_writes..." error?

Hi all,

My application removes existing models - i.e. layouts or figures - from its associated Bokeh document each time a new acquisition sequence is launch.
The data attached to each model is periodically streamed. Each data update generates seems to generate a 'pending write' on the (embedded) server queue in order to update the frontend.
Despite the fact that the periodic update mechanism is disabled when I remove the models from the document, I systematically get the following error:

Traceback (most recent call last):
File “DataStreamingModel.py”, line 693, in __clear_models self._doc.remove_root(m, setter=session)
File “/home/informatique/ica/leclercq/miniconda2/lib/python2.7/site-packages/bokeh/document.py”, line 579, in remove_root self._trigger_on_change(RootRemovedEvent(self, model, setter))
File “/home/informatique/ica/leclercq/miniconda2/lib/python2.7/site-packages/bokeh/document.py”, line 721, in _trigger_on_change self._with_self_as_curdoc(invoke_callbacks)
File “/home/informatique/ica/leclercq/miniconda2/lib/python2.7/site-packages/bokeh/document.py”, line 703, in _with_self_as_curdoc return f()
File “/home/informatique/ica/leclercq/miniconda2/lib/python2.7/site-packages/bokeh/document.py”, line 720, in invoke_callbacks cb(event)
File “/home/informatique/ica/leclercq/miniconda2/lib/python2.7/site-packages/bokeh/document.py”, line 683, in self._callbacks[receiver] = lambda event: event.dispatch(receiver)
File “/home/informatique/ica/leclercq/miniconda2/lib/python2.7/site-packages/bokeh/document.py”, line 111, in dispatch receiver._document_patched(self)
File “/home/informatique/ica/leclercq/miniconda2/lib/python2.7/site-packages/bokeh/server/session.py”, line 170, in _document_patched raise RuntimeError…
RuntimeError: _pending_writes should be non-None when we have a document lock, and we should have the lock when the document changes

``


So the questions is: do I have to 'lock' the document - in a way or another - before trying to remove any model?

Otherwise, is there a way to clear the 'pending writes' on the server?

Thanks for your help.
Nicolas.

Can you provide a minimal example that reproduce the problem? (i.e. with fake or synthetic data if need be) there might be a bug or issue but it’s impossible to say anything specific without code or being able to reproduce. In general “pending writes” is not something users should ever see or worry about but maybe your use pattern is one we did not foresee (again code needed, to be able to say anything for sure).

Thanks,

Bryan

···

On Feb 27, 2017, at 05:58, nicolas.fr [email protected] wrote:

Hi all,


My application removes existing models - i.e. layouts or figures - from its associated Bokeh document each time a new acquisition sequence is launch.
The data attached to each model is periodically streamed. Each data update generates seems to generate a 'pending write' on the (embedded) server queue in order to update the frontend.
Despite the fact that the periodic update mechanism is disabled when I remove the models from the document, I systematically get the following error:


Traceback (most recent call last):
File “DataStreamingModel.py”, line 693, in __clear_models self._doc.remove_root(m, setter=session)
File “/home/informatique/ica/leclercq/miniconda2/lib/python2.7/site-packages/bokeh/document.py”, line 579, in remove_root self._trigger_on_change(RootRemovedEvent(self, model, setter))
File “/home/informatique/ica/leclercq/miniconda2/lib/python2.7/site-packages/bokeh/document.py”, line 721, in _trigger_on_change self._with_self_as_curdoc(invoke_callbacks)
File “/home/informatique/ica/leclercq/miniconda2/lib/python2.7/site-packages/bokeh/document.py”, line 703, in _with_self_as_curdoc return f()
File “/home/informatique/ica/leclercq/miniconda2/lib/python2.7/site-packages/bokeh/document.py”, line 720, in invoke_callbacks cb(event)
File “/home/informatique/ica/leclercq/miniconda2/lib/python2.7/site-packages/bokeh/document.py”, line 683, in self._callbacks[receiver] = lambda event: event.dispatch(receiver)
File “/home/informatique/ica/leclercq/miniconda2/lib/python2.7/site-packages/bokeh/document.py”, line 111, in dispatch receiver._document_patched(self)
File “/home/informatique/ica/leclercq/miniconda2/lib/python2.7/site-packages/bokeh/server/session.py”, line 170, in _document_patched raise RuntimeError…
RuntimeError: _pending_writes should be non-None when we have a document lock, and we should have the lock when the document changes

``



So the questions is: do I have to 'lock' the document - in a way or another - before trying to remove any model?


Otherwise, is there a way to clear the 'pending writes' on the server?


Thanks for your help.
Nicolas.


You received this message because you are subscribed to the Google Groups “Bokeh Discussion - Public” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To post to this group, send email to [email protected].

To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/e570b66f-3ba9-4b4c-aa86-1a75a0d67dce%40continuum.io.

For more options, visit https://groups.google.com/a/continuum.io/d/optout.

Bryan,

Attached is some code allowing to reproduce the problem. Note that the problem could be my code itself :slight_smile:

Requirements and and dependencies:

  1. numpy
  2. six
  3. ipywidgets
  4. bokeh

Running the example (sorry for the obvious steps):

  1. make sure that the directory in which PendingWriteRuntimeError.py is located is in your PYTHONPATH

  2. open the PendingWriteRuntimeError.ipynb notebook

  3. optionally, adapt the Bokeh server address in celle 4 (ipa=‘127.0.0.1’)

  4. execute all cells

  5. click on the ‘start’ button of the last cell

  6. wait a few seconds, then click on ‘stop’ (not exit ‘x’)

  7. restart the simulation by clicking on ‘start’ again

  8. you should see the exception in the log
    Notes:

  9. The problem is normally hidden by the ‘clear_output’ call @ PendingWriteRuntimeError.py:626 (commented for the purpose of this test)

  10. The call to ‘remove_root’ generating the problem is triggered by the following sequence: start@512 → __clear_all_for_restart@575 → __clear_models@625 → remove_root@689

N.

PendingWriteRuntimeError.ipynb (3.34 KB)

PendingWriteRuntimeError.py (39.3 KB)

Hi,

Sorry for the delay. I can't actually reproduce any problems with this code using current master. Can you try this with master (or perhaps the last dev build) and report back if you still see an issue?

Thanks,

Bryan

···

On Feb 27, 2017, at 11:34, nicolas.fr <[email protected]> wrote:

Bryan,

Attached is some code allowing to reproduce the problem. Note that the problem could be my code itself :slight_smile:

Requirements and and dependencies:
  • numpy
  • six
  • ipywidgets
  • bokeh

Running the example (sorry for the obvious steps):
  • make sure that the directory in which PendingWriteRuntimeError.py is located is in your PYTHONPATH
  • open the PendingWriteRuntimeError.ipynb notebook
  • optionally, adapt the Bokeh server address in celle 4 (ipa='127.0.0.1')
  • execute all cells
  • click on the 'start' button of the last cell
  • wait a few seconds, then click on 'stop' (not exit 'x')
  • restart the simulation by clicking on 'start' again
  • you should see the exception in the log
Notes:
  • The problem is normally hidden by the 'clear_output' call @ PendingWriteRuntimeError.py:626 (commented for the purpose of this test)
  • The call to 'remove_root' generating the problem is triggered by the following sequence: start@512 -> __clear_all_for_restart@575 -> __clear_models@625 -> remove_root@689
N.

--
You received this message because you are subscribed to the Google Groups "Bokeh Discussion - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/26e0397f-e8cd-4c04-b724-1b1c73a329c8%40continuum.io\.
For more options, visit https://groups.google.com/a/continuum.io/d/optout\.
<PendingWriteRuntimeError.ipynb><PendingWriteRuntimeError.py>

You’re right Brian, the current master seems to solve the problem.
Thanks.

···

Le samedi 11 mars 2017 05:18:04 UTC+1, Bryan Van de ven a écrit :

Hi,

Sorry for the delay. I can’t actually reproduce any problems with this code using current master. Can you try this with master (or perhaps the last dev build) and report back if you still see an issue?

Thanks,

Bryan

On Feb 27, 2017, at 11:34, nicolas.fr [email protected] wrote:

Bryan,

Attached is some code allowing to reproduce the problem. Note that the problem could be my code itself :slight_smile:

Requirements and and dependencies:

    • numpy
    • six
    • ipywidgets
    • bokeh

Running the example (sorry for the obvious steps):

    • make sure that the directory in which PendingWriteRuntimeError.py is located is in your PYTHONPATH
    • open the PendingWriteRuntimeError.ipynb  notebook
    • optionally, adapt the Bokeh server address in celle 4 (ipa='127.0.0.1')
    • execute all cells
    • click on the 'start' button of the last cell
    • wait a few seconds, then click on 'stop' (not exit 'x')
    • restart the simulation by clicking on 'start' again
    • you should see the exception in the log

Notes:

    • The problem is normally hidden by the 'clear_output' call @  PendingWriteRuntimeError.py:626 (commented for the purpose of this test)
    • The call to 'remove_root' generating the problem is triggered by the following sequence: start@512 -> __clear_all_for_restart@575 -> __clear_models@625 -> remove_root@689

N.


You received this message because you are subscribed to the Google Groups “Bokeh Discussion - Public” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To post to this group, send email to [email protected].

To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/26e0397f-e8cd-4c04-b724-1b1c73a329c8%40continuum.io.

For more options, visit https://groups.google.com/a/continuum.io/d/optout.

<PendingWriteRuntimeError.ipynb><PendingWriteRuntimeError.py>