If you need/want to do this without a script tag for now I’d suggest you look at that autoload.js script and copy/take the parts you need. I’ve made an issue about BokehJS embedding API for server app:
It’s never really come up previously but the question has been raised a few times this month so it’s time to make some improvements.
I got a bokeh session open with:
Just to clarify terminology: just starting a bokeh server does not open any sessions, in and of itself. Sessions are only made whenever connections to the server actually come in.