For now I’ve ended up doing something more like the Flask integration, except sharing the Tornado IOLoop. Both servers are running on different ports, and then server_document just connects to the Bokeh server. It works quite nicely once you set allow_websocket_origin.
Jupyter app extensions looks interesting but AFAIK no one has ever looked into it. It’s definitely not something that will fit into my own bandwidth envelope any time soon, so if you already have some expertise, I’d encourage you to try to look into it (I’m happy to answer any technical questions). As you have noted a Bokeh server is basically just a collection of handlers that need to be installed on an IOLoop somehow.