Display bokeh application in a remote jupyter notebook

I am running a jupyter notebook remotely using ssh tunnel. I am using

bokeh.io.notebook.show_app(modify_doc, None, “http://localhost:9999”, port = 9998)

to display an application in the notebook. I have been able to use this command to display a simple application, but it does not work when running on my original modify_doc function. The error:

ERROR:bokeh.server.protocol_handler:error handling message Message ‘PULL-DOC-REQ’ (revision 1) content: {}: TypeError(‘keys must be a string’,)

My modify_doc function also works when the notebook is locally run. Both are running on Bokeh 1.2.0.

There’s not really enough information to speculate. The only thing that comes to mind is that in the past there were users who had issues connecting to apps running on a JupyterHub instance. As a result, the notebook_url parameter was expanded to be able to accept a callable to handle proxies:

notebook_url can also be a function that takes one int for the bound server port. If the port is provided, the function needs to generate the full public URL to the bokeh server. If None is passed, the function is to generate the origin URL.

How do I determine the full public url to the server? I am running the notebook remotely, and I have tunneled ports 9998-9999. A little more output on some code I have tried:

import bokeh
from bokeh.plotting import figure

def modify_doc(doc):   
    network_plot = bokeh.plotting.figure(title='test')
    doc.add_root(network_plot)

bokeh.io.notebook.show_app(modify_doc, None, "http://localhost:9999", port = 9998)

This will output an empty figure with tools and all. However, when I attempt to add a renderer to the figure no output appears:

import bokeh
from bokeh.plotting import figure
from bokeh.models import GraphRenderer, Oval

def modify_doc(doc):   
    network_plot = bokeh.plotting.figure(title='test')

    graph = GraphRenderer()
    N = 8
    node_indices = list(range(N))
    graph.node_renderer.glyph = Oval(height=0.1, width=0.2)
    graph.node_renderer.data_source.data = dict(
    index=node_indices)
    graph.edge_renderer.data_source.data = dict(
        start=[0]*N,
        end=node_indices)

    network_plot.renderers.append(graph)

    doc.add_root(network_plot)

bokeh.io.notebook.show_app(modify_doc, None, "http://localhost:9999", port = 9998)

Any plot, even an empty one, means the app machinery is working, and that this is not a network issue. But this seems to contradict your earlier statement that there were protocol errors on PULL-DOC-REQ since if there was a protocol error at that stage, you would not see any plot at all. So have things changed?

Yes I should’ve clarified that PULL-DOC-REQ must have been a different issue. My original code displays a much larger graph than the small example above, but I can’t seem to display that one.

@Bryan Any suggestions to try on this? I’m following along on this git issue to see if I can come up with a solution but wanted to get some advice here.

Edit: I’ve gotten this basic application showing with the code below, but still cannot display a basic graph using graph renderer. When using a graph renderer for the figure, the only output that appears is that BokehJS 1.2.0 successfully loaded.

import bokeh
from bokeh.plotting import figure
from bokeh.io import output_notebook
from bokeh.io.state import curstate
from bokeh.io.notebook import show_app

output_notebook()

def modify_doc(doc):   
    p = figure(title='test')

    p.circle([1, 2, 3, 4, 5], [6, 7, 2, 4, 5], size=20, color="navy", alpha=0.5)

    doc.add_root(p)

bokeh.io.notebook.show_app(modify_doc, curstate(), "http://localhost:9999", port = 9998)

Ah good I was actually going to suggest trying that next. Are there any messages in the browser’s JavaScript console in the non-working case? You can can set the environment variable

BOKEH_LOG_LEVEL=debug

to get more chatty output in the JS console. (Or even “trace” but that is quite voluminous). There is also a Python log level that can be adjusted but I am not really sure how to see the Bokeh server log output inline in a notebook. Maybe a Jupyter person can help with how to access logging output in a notebook. In a any case, looking for any error messages in either log.

If it works with everything except GraphRenderer, that may unfortunately point to a bug specific to it. GraphRenderer is the first and so-far only example of a “compound” renderer so it’s definitely possible it may suffer from issues other renderers do not.

How exactly do you set the environment variable? I also had to change around to using JupyterLab. I cannot get the code working on the instructions on Bokeh for JupyterHub using
remote_jupyter_proxy_url inside show.

It varies by platform and shell. On a bash-like shell on Linux or OSX it would be just what I put above. There is different syntax on Windows and in some other *nix shells, but I am not familiar with them.

I have zero experience with JupyterHub and no running instance to play around with, so unfortunately I may not be a person who is able to offer much more help. I’ll reiterate the ask to report about any output or messages in the browser JS console, even without setting the debug level, since there may be relevant clues there.

@jareducherek it might also be productive to ping the person who actually added the features and wrote those docs for JupyterHub directly, since it’s not an area of expertise for me. GH user cbanek added those in this PR:

You could trying @-pinging them in that PR

UPDATE: I went and ahead and posted a comment there with a link back here, hopefully they will see it and be able to chime in

Okay thanks, I believe I do need to implement what that PR uses to be able to view the application in JupyterHub, I will mess around with that. With a plain show right now, I get net::ERR_CONNECTION_REFUSED in the JS console.

Hi there, I worked on the PR for bokeh + JupyterHub. Just to make sure I understand the problem, it sounds like you’re using jupyterlab (and not hub) and ssh tunneling the connection (which I guess is port 8888?). Are you running it remotely in a container? The network topology is key to the problem.

The reason behind the PR is that bokeh injects javascript to connect back to the server, and so it puts the URL to connect to in there. When you’re inside a container and/or behind a proxy, the addresses that the bokeh python part knows about might not be what the browser needs to connect to.

I agree completely with what Bryan is suggesting to look at the console logs. If you see connection refused, look closely at the URL you are trying to connect to in the browser. I also like to do wgets and other things to make sure where the server is running just to be sure. If jupyterlab is running in a container, that port might not be exported either. This is why I mention the nbserverproxy extension. This allows for you to use the same tunneled port for the bokeh traffic as well.

Once you figure out what URL you need to connect to, and what bokeh is putting in there, you can write a function for notebook_url to put in the right URL (which is the address the browser connects to).

I hope this helps, although reading this, it does seem very complicated.

Thanks for the reply. I am running JupyterLab within a container. I will go check the console logs again, but I cannot figure out how to set the BOKEH_LOG_LEVEL=debug when I am running in a JupyterLab browser. I have enabled nbserverproxy extension, but because I am running JupyterLab, the os.environ['JUPYTERHUB_SERVICE_PREFIX'] portion of your code obviously doesn’t work, and I’m not sure how to replace that.

Lastly, could you expand on how exactly I figure out what URL I need to connect to? Is that the URL that nbserverproxy opens up? Any information you have is extremely helpful, as I’m not familiar with this area at all.

UPDATE: I am following the docs for /jupyter-server-proxy. I believe the problem arises from the fact that I don’t know how to open up a proxy server. I will continue messing with the docs but once I determine that I can specify what port is being used for the proxy for bokeh to use.

I don’t know much about the BOKEH_LOG_LEVEL parts, so I can’t help you much there.

So, what JUPYTERHUB_SERVICE_PREFIX is is the prefix for hub, and looks something like ‘/nb/user/cbanek/’ for us. This just allows us to make a URL like jupyterhubservice.lsst.io/nb/user/cbanek/’. Since you are using lab instead of hub, you can probably just omit this part.

Now that you’ve installed nbserverproxy, you should be able to use URLs to connect to various ports in your jupyterlab container. So if you’re running bokeh on port 9999, it’s probably running on localhost:9999 in the container. But since 9999 might not be exported by your container, you can use the nbserverproxy url to access it. So if you connect to your lab, and take off the /lab on the url, and add /proxy/9999 you should be able to connect to your bokeh service inside the container.

Now let’s tie it all together. Since you are ssh tunneling (I’m not sure exactly where), you’re connecting to something like localhost:8888/lab (or whatever the lab url is) in your browser, you might end up instead needing to connect to localhost:8888/proxy/9999/ (9999 being the port bokeh server is running on) to get your bokeh graph. You can load this in a browser standalone and I’ve had the bokeh graph work and load fine. To get it to load in your notebook, you need to inject that URL into the notebook_url parameter, or make a function that returns that URL. This will get your proper URL injected into the javascript to show the graph correctly using the bokeh jupyterlab extensions.

@jareducherek Just checking in if you’ve had any luck on this. Let me know if you need anything else.

@cbanek I am getting some help from some other people who have more experience with this kind of stuff. Running the notebook within a Docker container is also complicating things more for sure.

They’re probably trying to figure out exactly what jupyter-server-proxy does in the container.