Jitter in bokeh serve (on mac). Errors on linux.

I’ve run into bokeh server problems when running on linux, and I wrote a simpler program to see if the error persists. The simpler program is as follows:

from numpy import linspace, sin, cos, pi

import bokeh.plotting as plt
from bokeh.models import Range1d

fig_size = 750
ind = 0

def draw_map():
    tools = "pan,wheel_zoom,reset,previewsave"
    fig = plt.figure(title='', logo=None, tools=tools, webgl=True, x_range=Range1d(0, 15, bounds='auto'),
                     y_range=Range1d(-2, 2, bounds='auto'),
                     toolbar_location='above', plot_width=fig_size, plot_height=fig_size, min_border=10)
    return fig

def read_tracks():
    n = 50
    tracks = [list(linspace(0, 4 * pi, n)), list(sin(linspace(0, 4 * pi, n)))]
    return tracks

tracks = read_tracks()
fig = draw_map()
route = fig.line([0, 0], [0, 0], color='red', line_width=3, alpha=0.5, name='track')

def update():
    global ind
    print(ind)
    route.data_source.data['x'] = tracks[0][ind:]
    route.data_source.data['y'] = tracks[1][ind:]
    ind = ind + 1 if ind < len(tracks[0]) - 2 else 0

plt.curdoc().add_periodic_callback(update, 500)

plt.curdoc().add_root(fig)…

``

On mac, I see the display jitter. The sin curve being updated jumps for a very short time. I output ‘ind’ within the callback, and though on mac it is sequential, I find that on linux it is not sequential. There seems to be two threads going on.

Moreover, on mac if I use plt.curdoc().add_periodic_callback(update, 100), the plot errors when ind reaches 98.

On mac, I see the display The program is run using “bokeh serve --show program.py”

I’m using the latest version of bokeh.

Can anyone else replicate this problem?

File attached.

test.py (1 KB)

Hi,

First, as mentioned often, you need to update the .data dictionary "in one go", not update individual columns one after the other. So:

  route.data_source.data = dict(x=tracks[0][ind:], y=tracks[1][ind:])

The reason is because every update triggers a redraw, if you do thing separately, there is a moment where x and y are out of sync. This might be your issue, I am not certain (no linux VM to test with).

Also things work with webgl? I was just trying things out, and was seeing problems on OSX unless I turned webgl of

Bryan

···

On Oct 5, 2016, at 6:35 PM, waqy <[email protected]> wrote:

File attached.

--
You received this message because you are subscribed to the Google Groups "Bokeh Discussion - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/53c83a0c-5548-4fec-89e3-49037da61b3b%40continuum.io\.
For more options, visit https://groups.google.com/a/continuum.io/d/optout\.
<test.py>

Yes that helped. The jitter was caused by out of sync redraw.

webgl works with update rate of 500ms but causes errors at 100ms. Messages like “2016-10-05 21:52:48,410 404 POST /jsnlog.logger (::1) 0.46ms” appear on failure.

This happens on osx. Haven’t been able to check on linux – will do so tomorrow at work.

···

On Wednesday, October 5, 2016 at 5:31:22 PM UTC-7, Bryan Van de ven wrote:

Hi,

First, as mentioned often, you need to update the .data dictionary “in one go”, not update individual columns one after the other. So:

    route.data_source.data = dict(x=tracks[0][ind:], y=tracks[1][ind:])

The reason is because every update triggers a redraw, if you do thing separately, there is a moment where x and y are out of sync. This might be your issue, I am not certain (no linux VM to test with).

Also things work with webgl? I was just trying things out, and was seeing problems on OSX unless I turned webgl of

Bryan

On Oct 5, 2016, at 6:35 PM, waqy [email protected] wrote:

File attached.


You received this message because you are subscribed to the Google Groups “Bokeh Discussion - Public” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To post to this group, send email to [email protected].

To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/53c83a0c-5548-4fec-89e3-49037da61b3b%40continuum.io.

For more options, visit https://groups.google.com/a/continuum.io/d/optout.

<test.py>

Well, I discovered an interesting behavior in bokeh server. Once we start an app, and the program execution starts from the top once the webpage is opened. In another browser tab is opened, then the program starts from the top. The two pages will not be in sync. Does this happen in app mode only? Is it possible to open it in shared mode, where both page are mirror images? Or would I have to move to client mode?

I inserted the following code, and it makes both the first browser tab stop for 10 seconds while the seconds one starts up, and after that both the pages work (nut out of sync)

from time import sleep
print('Sleeping...')
sleep(10)
print('Active...')

``

test.py (1.04 KB)

Waqy,

This is exactly the intended behavior. When hitting the plain app URL, every tab gets its own new session, and a completely independent copy of the app. That is 100% by design.

If you want to have two people viewing the exact same session (i.e. "google apps shared editing" kind of mode) then yes, you'd have to have to use bokeh.client and push_session, and then you would have to have both users go to the exact URL to request that same exact session, with a URL that has the "?session-id=" HTTP argument (something like that, would have double check specifics).

Finally it is important to note that in general, Bokeh apps should not perform long blocking operations. All execution happens inside a Tornado IOLoop, if one client blocks, it blocks all clients, as you can see. While you can run more servers behind load balancers, or start with --thread=N, those are not really the best solutions to performing long blocking operations (they are solutions for scaling, which is different) If you need to do blocking work you should push that work off to a thread you create yourself. There is an example of this in the User's Guide.

Thanks,

Bryan

···

On Oct 6, 2016, at 12:41 PM, waqy <[email protected]> wrote:

Well, I discovered an interesting behavior in bokeh server. Once we start an app, and the program execution starts from the top once the webpage is opened. In another browser tab is opened, then the program starts from the top. The two pages will not be in sync. Does this happen in app mode only? Is it possible to open it in shared mode, where both page are mirror images? Or would I have to move to client mode?

I inserted the following code, and it makes both the first browser tab stop for 10 seconds while the seconds one starts up, and after that both the pages work (nut out of sync)
from time import sleep
print('Sleeping...')
sleep(10)
print('Active...')

--
You received this message because you are subscribed to the Google Groups "Bokeh Discussion - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/16292fd1-de17-45f8-a21b-aae106c8ccc3%40continuum.io\.
For more options, visit https://groups.google.com/a/continuum.io/d/optout\.
<test.py>

Thanks Bryan.

···

On Thursday, October 6, 2016 at 11:37:44 AM UTC-7, Bryan Van de ven wrote:

Waqy,

This is exactly the intended behavior. When hitting the plain app URL, every tab gets its own new session, and a completely independent copy of the app. That is 100% by design.

If you want to have two people viewing the exact same session (i.e. “google apps shared editing” kind of mode) then yes, you’d have to have to use bokeh.client and push_session, and then you would have to have both users go to the exact URL to request that same exact session, with a URL that has the “?session-id=” HTTP argument (something like that, would have double check specifics).

Finally it is important to note that in general, Bokeh apps should not perform long blocking operations. All execution happens inside a Tornado IOLoop, if one client blocks, it blocks all clients, as you can see. While you can run more servers behind load balancers, or start with --thread=N, those are not really the best solutions to performing long blocking operations (they are solutions for scaling, which is different) If you need to do blocking work you should push that work off to a thread you create yourself. There is an example of this in the User’s Guide.

Thanks,

Bryan

On Oct 6, 2016, at 12:41 PM, waqy [email protected] wrote:

Well, I discovered an interesting behavior in bokeh server. Once we start an app, and the program execution starts from the top once the webpage is opened. In another browser tab is opened, then the program starts from the top. The two pages will not be in sync. Does this happen in app mode only? Is it possible to open it in shared mode, where both page are mirror images? Or would I have to move to client mode?

I inserted the following code, and it makes both the first browser tab stop for 10 seconds while the seconds one starts up, and after that both the pages work (nut out of sync)

from time import sleep

print(‘Sleeping…’)

sleep(10)

print(‘Active…’)


You received this message because you are subscribed to the Google Groups “Bokeh Discussion - Public” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To post to this group, send email to [email protected].

To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/16292fd1-de17-45f8-a21b-aae106c8ccc3%40continuum.io.

For more options, visit https://groups.google.com/a/continuum.io/d/optout.

<test.py>

It might help to explicitly define some terms, too. I'd like to write this up with some diagrams for the docs, but I will dump some comments here for future reference:

A Bokeh model is an object like a Plot, Range1d, HoverTool, etc.

A Bokeh Document is a collection of Bokeh Models.

The Bokeh server has (in-memory) Python versions of Documents. It can serialize these documents, so that a browser can create a JS version of the Document. The job of the Bokeh server is to keep the Python and JS versions of the document in sync, at all times. Additionally, if there are callbacks defined, it runs any callback code whenever the Python version of the document receives an update.

The Python "Bokeh app" code is really a "factory for Documents" -- whenever the server needs a document, it executes the python app code. That's why at the end you typically do "curdoc().add_root(...)" That line is actually the what specifies how the a blank document should be modified.

The desired default behaviour is that any new connection to a plain Bokeh app URL generates an independent session, with a new document. From this it follows, every connection results in the python "app code" being run, to create a new document for the new session that was just requested.

It's possible for multiple browsers to connect to the same existing session on a Bokeh server (so think: shared google doc editing. Or: one user moves a slider and another user sees the slider move too). That might be useful in case, but is definitely not the common case. So it's not the default. To connect to a pre-existing session on the server you have to be explicit, and put the session id in the URL.

···

On Oct 6, 2016, at 1:37 PM, Bryan Van de Ven <[email protected]> wrote:

Waqy,

This is exactly the intended behavior. When hitting the plain app URL, every tab gets its own new session, and a completely independent copy of the app. That is 100% by design.

If you want to have two people viewing the exact same session (i.e. "google apps shared editing" kind of mode) then yes, you'd have to have to use bokeh.client and push_session, and then you would have to have both users go to the exact URL to request that same exact session, with a URL that has the "?session-id=" HTTP argument (something like that, would have double check specifics).

Finally it is important to note that in general, Bokeh apps should not perform long blocking operations. All execution happens inside a Tornado IOLoop, if one client blocks, it blocks all clients, as you can see. While you can run more servers behind load balancers, or start with --thread=N, those are not really the best solutions to performing long blocking operations (they are solutions for scaling, which is different) If you need to do blocking work you should push that work off to a thread you create yourself. There is an example of this in the User's Guide.

Thanks,

Bryan

On Oct 6, 2016, at 12:41 PM, waqy <[email protected]> wrote:

Well, I discovered an interesting behavior in bokeh server. Once we start an app, and the program execution starts from the top once the webpage is opened. In another browser tab is opened, then the program starts from the top. The two pages will not be in sync. Does this happen in app mode only? Is it possible to open it in shared mode, where both page are mirror images? Or would I have to move to client mode?

I inserted the following code, and it makes both the first browser tab stop for 10 seconds while the seconds one starts up, and after that both the pages work (nut out of sync)
from time import sleep
print('Sleeping...')
sleep(10)
print('Active...')

--
You received this message because you are subscribed to the Google Groups "Bokeh Discussion - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/16292fd1-de17-45f8-a21b-aae106c8ccc3%40continuum.io\.
For more options, visit https://groups.google.com/a/continuum.io/d/optout\.
<test.py>