Pull_session doesn't work if i don't include 'from bokeh.server.server import Server' statement which i am not using anywhere

Gopi_M · August 22, 2020, 9:50am

Hello experts,

I was getting this error message even though a sample example code shown in an online course was working fine for the instructor:
OSError: Cannot pull session document because we failed to connect to the server (to start the server, try the 'bokeh serve' command)

I tried a lot and accidentally found that my code works if i include below additional import statement but interestingly i am not using it anywhere in my code.
from bokeh.server.server import Server

Here is my code:

from flask import Flask, render_template
from bokeh.embed import server_document,server_session
from bokeh.server.server import Server  # If i comment this, I  get above 'Cannot pull session document...' error. 
from bokeh.client import pull_session

app = Flask(__name__)

#create index page function
@app.route("/")

def index():
    myurl = "http://localhost:5006/mybokeh"
    session = pull_session(url=myurl)
    bokeh_script = server_session(None, url=myurl, session_id=session.id)
    return render_template("index.html",bokeh_script=bokeh_script)

#run the app
if __name__ == "__main__":
    app.run(debug=True)

Any ideas why this is happening so?

Bryan · August 22, 2020, 3:23pm

I’m extremely skeptical that there is any real connection between these two events. The only place in the entire codebase that has that error text is here:

github.com

bokeh/bokeh/blob/branch-2.2/bokeh/client/session.py#L364-L369

    
      
          if not self.connected:
              if self.error_reason is ErrorReason.HTTP_ERROR:
                  if self.error_code == 404:
                      raise OSError(f"Check your application path! The given Path is not valid: {self.url}")
                  raise OSError(f"We received an HTTP-Error. Disconnected with error code: {self.error_id}, given message: {self.error_message}")
              raise OSError("We failed to connect to the server (to start the server, try the 'bokeh serve' command)")

Which can only happen when there really is a bona-fide HTTP ERROR as a result of a connection request that failed. So what I actually think is that there was just some coincidence, and that the Bokeh server could not actually be reached, just as the error states (e.g. it wasn’t running, or hadn’t finished starting, had crashed, was on a different port, etc).

Gopi_M · August 23, 2020, 5:35am

Thanks for your response, Bryan. I left the bokeh server untouched (i.e. left continuously running) and tested by only changing the flask code about 8-12 times by commenting and uncommenting that line. Each time the code ran with that line commented it throws pull_session failed error msg and if i uncomment it works perfectly fine with the same running session.

In fact, before we tried that import statement my colleague and I had spent over a day in figuring out why it’s throwing a pull session error (when bokeh serve is running) until he accidentally found out it worked when he inserted that line. I also tested and found this to be happening on my machine too.

As such the code works with that import statement but i am just curious as to why this is happening and how it was working in an online tutorial without that import.

Bryan · August 23, 2020, 4:35pm

@Gopi_M I don’t have an explanation for you, because it makes no sense to me. However, I can reproduce he problem, on Windows only. The example/embed/server_session example works just fine on OSX and Linux, which is probably why this bug has not been noticed before. In any case, please file a bug report on GitHub

Bryan · August 24, 2020, 10:26pm

FWIW My speculation is that this necessary codepath is executed from a transitive import of bokeh.server.torndao

github.com

bokeh/bokeh/blob/branch-2.3/bokeh/server/tornado.py#L714-L720

    
      
          #-----------------------------------------------------------------------------
          # Code
          #-----------------------------------------------------------------------------
          
          
BokehTornado.__doc__ = format_docstring(
              BokehTornado.__doc__,
              DEFAULT_CHECK_UNUSED_MS=DEFAULT_CHECK_UNUSED_MS,

This is necessary because Python 3.8 made a breaking change for Tornado on Winwdows. We added that code to make things work by default in the bokeh.server case but evidently did not make sure it was also present for bokeh.client.

@Gopi_M it would be helpful if you can confirm that the issue (only) affects Windows + Python 3.8, and if so then also filing a bug report.

Gopi_M · August 26, 2020, 10:14am

Hi Bryan, I am using Win10 and Python 3.8.3 and it certainly has this behaviour on my machine (Don’t have other OS for me to try, unfortunately). I have raised a bug report as you suggested. Here is the link

Bryan · August 26, 2020, 11:40pm

@Gopi_M can you confirm if adding this to your script fixes the issue (i.e. that things work without the server import)?

import asyncio
asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())

Gopi_M · August 28, 2020, 1:18pm

Hi Bryan, Thanks for looking in to this. It works fine if i include the 2 lines you provided. (and with the earlier import statement commented out). Hope this helps.

Gopi_M · October 8, 2020, 2:14pm

Hi Bryan,
The above solution worked fine on my Win10 laptop. However, when i am importing the code to Ubuntu 20.04, I am getting below error msg:
AttributeError: module 'asyncio' has no attribute 'WindowsSelectorEventLoopPolicy'

I tried a bit of search, even stackoverflow didn’t have much on this. Any chance there is a workaround for this? or at least some idea on what’s causing this issue in Ubuntu?

Here is the full Flask code:

from bokeh.resources import CDN
from flask import Flask, render_template
from bokeh.embed import server_document,server_session
from bokeh.client import pull_session
import asyncio
asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())

app = Flask(__name__)

CDN.legacy = True
@app.route("/")

def index():
    myurl = "http://localhost:5006/kw_Clustering_Comparison"
    session = pull_session(url=myurl)
    bokeh_script = server_session(None, url=myurl, session_id=session.id)
    return render_template("index.html",bokeh_script=bokeh_script)

if __name__ == "__main__":
    app.run(debug=True)

The Bokeh code is working fine and the above myurl shows the app correctly. Only the hosting through Flask is having the issue in Ubuntu.

FWIW, I have Python 3.8.5 (in Ubuntu and 3.8.3 in Win10)

When I tried to remove asyncio statements i get these messages:
* Serving Flask app “app” (lazy loading)
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: on
Traceback (most recent call last):
File “app.py”, line 25, in
app.run(debug=True)
File “/home/ubuntu/.local/lib/python3.8/site-packages/flask/app.py”, line 990, in run
run_simple(host, port, self, **options)
File “/usr/local/lib/python3.8/dist-packages/werkzeug/serving.py”, line 990, in run_simple
s.bind(server_address)
OSError: [Errno 98] Address already in use

Bryan · October 8, 2020, 3:37pm

@Gopi_M WindowsSelectorEventLoopPolicy only exists on Windows. If you need to run the same code on multiple platforms you will have to put this code behind a conditional that explicitly checks against sys.platform to make sure things are running on Windows.

OSError: [Errno 98] Address already in use

This is an error from the operating system stating that two (processes) are trying to listen on the same IP and port. I don’t think this has anything to do with any imports. In my experience this always indicates that a previous script or program did not shut down completely or properly.

Gopi_M · October 8, 2020, 4:35pm

Thanks for the super fast response, Bryan. Appreciate it!
So,do I simply remove those 2 asyncio lines for ubuntu? or should I replace with any other code?
(I will check the Errno 98 separately, thanks for your pointer.)

Bryan · October 8, 2020, 5:03pm

@Gopi_M You could do that I suppose, but no, what I meant is to use an if test to run different code on different platforms. This is what Bokeh does:

github.com

bokeh/bokeh/blob/dcaeba25b720528e32460a988a364733389fcd5f/bokeh/util/tornado.py#L48-L54

    
      
          def fixup_windows_event_loop_policy() -> None:
              if sys.platform == 'win32' and sys.version_info[:3] >= (3, 8, 0):
                  import asyncio
                  if type(asyncio.get_event_loop_policy()) is asyncio.WindowsProactorEventLoopPolicy:
                      # WindowsProactorEventLoopPolicy is not compatible with tornado 6
                      # fallback to the pre-3.8 default of WindowsSelectorEventLoopPolicy
                      asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())