Tornado Keepalive in Embedded Server

[andrew.waites]

Bokeh 2.3.1

I have a flask app running in a docker container calling a bokeh server (pull_session) in another container - both on Azure.

The app is working but I have just noticed the following mesage on the bokeh server log file repeated every 35 or so seconds…

2021-09-08T09:14:11.705447616Z 2021-09-08 09:14:11,704 Exception in callback <bound method BokehTornado._keep_alive of <bokeh.server.tornado.BokehTornado object at 0x7f45b31c0e80>>
2021-09-08T09:14:11.705496919Z Traceback (most recent call last):
2021-09-08T09:14:11.705508420Z   File "/opt/conda/lib/python3.9/site-packages/tornado/ioloop.py", line 905, in _run
2021-09-08T09:14:11.705518320Z     return self.callback()
2021-09-08T09:14:11.705526721Z   File "/opt/conda/lib/python3.9/site-packages/bokeh/server/tornado.py", line 704, in _keep_alive
2021-09-08T09:14:11.705535622Z     c.send_ping()
2021-09-08T09:14:11.705543922Z   File "/opt/conda/lib/python3.9/site-packages/bokeh/server/connection.py", line 75, in send_ping
2021-09-08T09:14:11.705552323Z     self._socket.ping(codecs.encode(str(self._ping_count), "utf-8"))
2021-09-08T09:14:11.705560623Z   File "/opt/conda/lib/python3.9/site-packages/tornado/websocket.py", line 445, in ping
2021-09-08T09:14:11.705569624Z     raise WebSocketClosedError()
2021-09-08T09:14:11.705597626Z tornado.websocket.WebSocketClosedError

The app is continuing to work but clearly there remains some form of unhappiness between the flask and bokeh servers. I did look at the tickets regarding tornado max message size but that appears to have been fixed prior to 2.3.1.

The calling code looks like…

@home_bp.route('/emaths_app', methods=['GET'])
def emaths_app():
    """Electoral Maths Embed"""
    if session["amalfiblueUser"].EMAccess:
        if session["amalfiblueUser"].EMDivisions != "All":
            div=session["amalfiblueUser"].EMDivisions
        else:
            div="Kennedy"
            
        with pull_session(url="https://{uri}/electoralmaths".format(uri=app.config["BOKEH_SERVER_URI"]), arguments=dict(div=div) , max_message_size=41943040) as bokeh_session:
            script = server_session(session_id=bokeh_session.id, url="https://{uri}/electoralmaths".format(uri=app.config["BOKEH_SERVER_URI"]))
            return render_template("embed.jinja2", 
                user=session["amalfiblueUser"],
                script=script, 
                title='Electoral Maths'
            )
    else:
        return redirect(url_for("home_bp.index"))

The bokeh server code is pretty big but runs happily enough when not embedded.

[Bryan]

HI @andrew.waites It looks like the keepalive job is running past the session close. I’m not sure why that would be, offhand. It would take proper investigation of a Minimal Reproducible Example to try and determine more.

Otherwise, I can only mention that:

• the messages are innocuous, if annoying
• the keep-alive interval, you could increase it or even set to zero to disable. Though be aware that some browsers may close WS connections without them, so YMMV

If you want to look more into it, the relevant callback is here:

bokeh/bokeh/blob/b9650d5f9576c4a6a22ad5a9c72b6b1caff9a600/bokeh/server/tornado.py#L757-L760

One possibility might be to catch a WS closed exception right there, and remove the client in that case. But it would be better to try and discover why the client is not being removed in your situation. it could either be be a bug, or it could be a usage issue, it’s not possible to speculate without actual debugging.