Bokeh server error 500

Community Support
1

I have a mini-dashboard that I want to display using bokeh serve through command line. I launch it using “nohup bokeh serve”, however, after like 2-3 hours I can’t reach it due to 500: Internal Server Error. I am not the only one using the dashboard (it is launched on a remote server), that may be the reason to the error, but I’m not sure.

Using strace, I can find out that the bokeh process is sleeping due to epoll_wait.

Is this the expected behaviour or not? If yes, should I switch to some other method of deploying the dashboard, with which I won’t encounter this problem (e.g. using Flask)?

2

@steppenwolf This is not only not expected, but also not really heard of—I don’t recall a single mention of it in ~10 years. Unfortunately we would need quite a bit more information to speculate with any detail, starting with a complete Minimal Reproducible Example and detailed instructions for running and reproducing. Information (e.g. stack traces or error messages) from logs are also advised. You also did not mention any versions above.

Otherwise I can only note that the only way to get a 500 response from the Bokeh server itself is if the websocket connection has a problem. But to be honest looking through connection.py I don’t offhand see how any actual codepaths could ever land there, without the connection being closed on the server but somehow not closed on the browser. And I don’t have any idea how that could possibly occur.

3

HI,
Never done anything with mini-dashboard, so not sure if referring to same issue, but IMHO it is rather easy to reproduce “500”-error by raising ValueError etc.

"""
Example code for producing 500: Internal Server Error
Python 3.8
Bokeh 2.2.3
Spyder 4.1.5
Firefox 67.0.4
"""
from tornado.ioloop import IOLoop
from bokeh.application.handlers import FunctionHandler
from bokeh.application import Application
from bokeh.server.server import Server

def app_init(doc):
    a = 1 / 0       # Produces: 500: Internal Server Error
    
io_loop = IOLoop.current()
bokeh_app = Application(FunctionHandler(app_init))
server = Server({"/": bokeh_app}, io_loop=io_loop)    
server.start()
io_loop.add_callback(server.show, "/")

Noticed this when a function returned 3 parameters, but caller consumed only two, yielding unpacking exception.

1 Like
4

I had thought that was a different 50x error. In that case @heke the most likely explanation is that you app code is raising an exception somewhere in a callback.

5

To add some firsthand experience here … I have seen 500 errors on occasion during early stage development or when adding complex new features.

The problem – when manifested – invariably traced back to a bug in my code and not core bokeh functionality.

If it shows up immediately when a session is created, then it usually points to a problem in syntax building up the UI and data and analysis methods that support it.

If it shows up sometime later as the OP mentions, that is typically due to a user interaction via callback (as @Bryan surmises) or some other deferred code path that exposes the bug.

6

Yeah, as a newbie, I assumed that the error 500, that @steppenwolf encounters, originates from Bokeh server. I see that too, “sometimes”, and it always traces back to a bug in my Python app (as anticipated by @_jm in other comment). I guess the best way for @steppenwolf to debug the problem is to use try-except constructs to capture the failure, as most likely the problem is in the code, not in Bokeh server.

I humbly emit a wish that in future releases the Bokeh server would be a bit more vocal on runtime exceptions, as that would make debugging easier.

…and thanks for the great work. Bokeh is awesome!

7

I humbly emit a wish that in future releases the Bokeh server would be a bit more vocal on runtime exceptions, as that would make debugging easier.

We absolutely, already do provide as much information as possible, wherever it is possible. E.g. if you jsut run bokeh serve --show app.py on a script that contains 1/0 and nothing else you will see the full stack trace in the server console log:

2021-06-08 17:30:50,287 Error running application handler <bokeh.application.handlers.script.ScriptHandler object at 0x7fd31837c4c0>: division by zero
File "app.py", line 1, in <module>:
1/0 Traceback (most recent call last):
  File "/Users/bryan/anaconda/envs/bk-232/lib/python3.9/site-packages/bokeh/application/handlers/code_runner.py", line 197, in run
    exec(self._code, module.__dict__)
  File "/Users/bryan/tmp/app.py", line 1, in <module>
    1/0
ZeroDivisionError: division by zero

And the same is true of any exception in a callback (including in scripts that use the Server API programmatically):

ERROR:bokeh.server.protocol_handler:error handling message
 message: Message 'PATCH-DOC' content: {'events': [{'kind': 'ModelChanged', 'model': {'id': '1041'}, 'attr': 'value', 'new': 1}], 'references': []} 
 error: ZeroDivisionError('division by zero')
Traceback (most recent call last):
  File "/Users/bryan/anaconda/envs/bk-232/lib/python3.9/site-packages/bokeh/server/protocol_handler.py", line 90, in handle
    work = await handler(message, connection)
..........
  File "/Users/bryan/work/bokeh/examples/howto/server_embed/standalone_embed.py", line 18, in callback
    1/0

In general the Bokeh server console log will have this information. But not every situation is equal, e.g. things may not be easily viewable or accessible in notebook settings. I would always advise dev/testing outside the notebook since the notebook always adds complexity and levels of distance/abstraction.

If you saying you have seen an exception in app code that did not result in a stack trace being logged in the console, then we would need a Minimal Reproducible Example to investigate.

1 Like
8

Yes. In my case, the error is always immediately apparent from a full-stack trace in the server console log. Then again, I only run the server from a terminal or an online platform-as-a-service which has terminal like logs … never use notebooks.

9

Thanks everyone!

As it turned out, my error was because of gitlab runner regularly deleting some files that were needed for the dashboard and not because of bokeh itself. Sorry for wasting your time.

1 Like
10

Hi Bryan,

Thanks a lot!
When I run a code that launches the server inside the code, won’t get much feedback. An almost minimum code is shown above in comment 3. Running that from Spyder gives error 500. Strangely, running from command line won’t even open the browser. Perhaps setting up a logger in the code is the way to go in this case.

Cheers,
Heke

11

@heke If you are using Bokeh programmatically in contexts that hide stdout from you, that’s not something we could possibly detect or do anything about. Bokeh logs to stdout by default but it’s using standard Python logging calls. You could configure the logging system to output to a file instead but that is up to you to do.

Edit:

Strangely, running from command line won’t even open the browser.

It doesn’t open the browser because you exit without starting the ioloop That entire script is essentially a no-op.

12

@Bryan, Wow! Thank you! Truly appreciate! Completely overlooked the ioloop start, as for some strange reason could run the nearly minimal code, as well as a pretty widget callback intensive application under work without ioloop start, on Spyder, so did not suspect that they’ll fail on launch from a command prompt. Wondering why it is happening. Is Spyder starting ioloop automagically? (Maybe not actually a Bokeh issue, but…)

Thanks for the stdout info. I originally wondered if the browser could show some traceback info along with 500, but guessing that there are some security issues related to it.

Cheers,
Heke

1 Like