Subprotocol Websocket issues when embedding a Bokeh session and reverse proxying using Apache/Nginx

[nicholas-miklaucic]

I’m trying to embed an interactive visualization in another site. It works fine when both the Bokeh server and the site are being run from the same computer, but I’m trying to offload the Bokeh server to a different IP address and use SSL. When I do this, I get a rather strange issue.

https://constellate.dev/GDrzvqv3 is an example visualization, that works fine for me when I navigate to the site directly. But when I then try to embed it using an autoload script, I get the following error from the server:

2022-06-25 21:02:11,677 WebSocket connection opened
2022-06-25 21:02:11,678 ServerConnection created
2022-06-25 21:02:52,405 WebSocket connection opened
2022-06-25 21:02:52,406 Uncaught exception GET /GDrzvqv3/ws (107.137.157.121)
HTTPServerRequest(protocol='http', host='constellate.dev', method='GET', uri='/GDrzvqv3/ws', version='HTTP/1.1', remote_ip='107.137.157.121')
Traceback (most recent call last):
  File "/home/nicholas/.cache/pypoetry/virtualenvs/constellate-Kl69nrwf-py3.9/lib/python3.9/site-packages/tornado/websocket.py", line 954, in _accept_connection
    open_result = handler.open(*handler.open_args, **handler.open_kwargs)
  File "/home/nicholas/.cache/pypoetry/virtualenvs/constellate-Kl69nrwf-py3.9/lib/python3.9/site-packages/tornado/web.py", line 3173, in wrapper
    return method(self, *args, **kwargs)
  File "/home/nicholas/.cache/pypoetry/virtualenvs/constellate-Kl69nrwf-py3.9/lib/python3.9/site-packages/bokeh/server/views/ws.py", line 139, in open
    raise ProtocolError("Subprotocol header is not 'bokeh'")
bokeh.protocol.exceptions.ProtocolError: Subprotocol header is not 'bokeh'

The same thing happened when I tried using Nginx. I’m using Apache now, so here’s the configuration I’m using to do that. (Because the URLs are auto-generated, this doesn’t quite match the config given in the docs, but given that it works when you navigate to the site directly I don’t think I made any really simple goofs setting that up.)

Any advice? Here’s my Apache configuration.

Listen 443
<VirtualHost *:443>
    ServerName constellate.dev

    SSLEngine On
    SSLCertificateFile "/etc/letsencrypt/live/constellate.dev/fullchain.pem"
    SSLCertificateKeyFile "/etc/letsencrypt/live/constellate.dev/privkey.pem"

    RewriteEngine on
    RewriteCond %{HTTP:Upgrade} websocket [NC]
    RewriteCond %{HTTP:Connection} upgrade [NC]
    RewriteRule ^/?(.*) "ws://127.0.0.1:5006/$1" [P,L]

    ProxyPreserveHost On
    ProxyPass / http://127.0.0.1:5006/
    ProxyPassReverse / http://127.0.0.1:5006/

    <Directory />
        Require all granted
        Options -Indexes
    </Directory>

    # Alias /static /path/to/bokeh/server/static                                                         
    # <Directory /path/to/bokeh/server/static>                                                           
        # # directives to effect the static directory                                                    
        # Options +Indexes                                                                               
    # </Directory>                                                                                       
</VirtualHost>

[Bryan]

This will have to be investigated directly. It’s possible there is a bug, or it’s possible there is some usage/configuration problem. It’s not possible to speculate without a complete Minimal Reproducible Example, including all relevant version/environment information. Given the complexity with the proxy, ideally it would be supplied as a GitHub repo that can be cloned.

[nicholas-miklaucic]

Thanks for the help. (And thanks for working on Bokeh more generally!)

I haven’t managed to reproduce this on a single machine using localhost: there’s some problem with HTTPS specifically, or at least it’s appeared that way to me in trying to reproduce it that way. That means I unfortunately can’t make your life as easy as I’d like. What I do have is this repo, which contains a minimal embed example, the setup I’ve done from the command line on the server, and every file I’ve edited that affects how Apache operates.

Given that everything works as far as running Bokeh directly from the URL https://constellate.dev/sliders, I don’t think there are any issues in the SSL setup. The server is an IBM Cloud machine, but again I’m not sure how that could break embedding but not directly using the site.

Hopefully this is helpful. Let me know if there’s anything I forgot to add or any discrepancies you’re getting. Sorry the example isn’t as clean as I’d like, but it does appear to be some issue that only pops up with all of the different pieces together.

[Bryan]

Thanks for the MRE @nicholas-miklaucic As it is a bit more involved than usual, I will have so find some dedicated time to sit down with it some night this week.

[Bryan]

@nicholas-miklaucic I am not sure I will be able to try to run the repo, it’s just seems considerably involved, more than I have bandwidth for at present. Scanning over it, and comparing to the docs at

https://docs.bokeh.org/en/dev-3.0/docs/user_guide/server.html#apache

It does seem that your config doesn’t have all the options listed in the Bokeh docs, e.g. ones like

    ProxyPreserveHost On
    ProxyPass /myapp/ws ws://127.0.0.1:5100/myapp/ws
    ProxyPassReverse /myapp/ws ws://127.0.0.1:5100/myapp/ws

    ProxyPass /myapp http://127.0.0.1:5100/myapp/
    ProxyPassReverse /myapp http://127.0.0.1:5100/myapp/

Can you confirm whether these options help? If not, you might also consider asking in an Apache help forum, specifically about what it takes to ensure subprotocols are properly included when proxying webscockets. Unfortunately I personally have next to no practical experience with Apache.

[nicholas-miklaucic]

I did try those options and I didn’t get anything to work. Thanks for the support: it’s probably not a Bokeh issue per se, so hopefully I can track down the issue on a different forum. If I do, I’ll come back and suggest some documentation additions if necessary.