Bokeh Server Embed in Flask Application "Port in Use" OSError

Community Support
1

I am trying to embed a Bokeh server in a Flask application, but I keep getting the error “OSError: [Errno 98] Address already in use”

Now I know for a fact that before I enter “flask run” on the command line, there are no other processes running on the default Bokeh server port (5006), because I kill any hanging processes using lsof -i and then kill -9 PID.

I suspect that it has something to do with flask attempting to execute the bk_worker function multiple times, each with the same port.

When I execute my plot python script using “bokeh serve plot.py” from the command line, it runs fine, and I can access the plot at http://localhost:5006/plot. However, when I try to execute from within the Flask app, I get the OSError.

This is the code I have executing in init.py. I have tried using both the Bokeh-recommended method of using Thread, as well as just calling bk_worker directly since Flask does some multiprocessing stuff as well. Either way, I receive the same OSError for the port already being in use.

init.py

app = Flask(__name__)
app.config.from_object(Config)
db = SQLAlchemy(app)

def bk_worker():
    server = Server({'/plot': plot}, io_loop=IOLoop(), allow_websocket_origin=["localhost:{}".format(5000)])
    server.start()
    server.io_loop.start()

#from threading import Thread
#Thread(target=bk_worker).start()
bk_worker()

This is the code I am using from within my application’s routes.py to get the bokeh plot, which I then pass to the html render.

routes.py

@app.route('/bokeh_plot', methods=['GET', 'POST'])
def bokeh_plot():
    script = server_document('http://localhost:5006/plot')
    return render_template('bokeh_plot.html', script=script)

The plot itself that I am linking the server to is in a file called plot.py with the following format. I know this code works, because I can serve it using the bokeh serve command

plot.py

def plot(doc):
    ...code to make plot...
    return plot

My first thought is that I am not running the bk_worker function in the correct Flask file? Or I am not understanding something about the way my ports are configured? Most of the examples I see have the entire application running in a single file, but my application is a little more complex, so I have the bk_worker function executing in my init.py. I have also tried putting it in routes.py under the ‘/plot’ route that calls it. My file layout structure is generally the same as in the Flask Mega Tutorial

Environment:
I am developing in Windows Subsystem for Linux with Ubuntu 20.04 LTS.
Python Version: 3.6.10
Bokeh Version: 2.3.0
Flask Version: 1.1.2

2

Update

I was able to get this error to go away by following the instructions for running Bokeh with Flask on gunicorn. This handles multi-threaded situations and creates multiple ports, which is how Flask defaults to now even without gunicorn. I am not using gunicorn.

However, my plots still are not showing up on the webpage even though this error is now gone. I am passing the port from my init.py to my routes.py through an os environment variable, which I’m not sure is the correct way to implement this. I’ll keep troubleshooting but I’d appreciate anyone’s thoughts on how to best handle this.

4

Are there any JS console errors or messages in in the browser?

5

Yes I am getting an error that it cannot connect to the websocket.

image
image778×195 10 KB

So I am wondering if I am maybe not accessing the right port for the thread this is running on? in my init.py file, I have the following:

sockets, port = bind_sockets("localhost", 0)
os.environ['BOKEH_PORT'] = str(port)

And then in my routes.py, I have

port = os.environ.get('BOKEH_PORT')
plot = server_document('http://localhost:%d/plot' % int(port))

I have also tried implementing it using a global variable instead of an OS environment variable.

I have a file called settings.py with the following:

def init():
    global port
    port = 0

In my init.py, I have the following:

from app import settings

settings.init()
sockets, settings.port = bind_sockets("localhost", 0)

And in routes.py, I have

from app import settings
plot = server_document('http://localhost:%d/plot' % int(settings.port))

I have print statements and both work correctly for getting a port through to the server_document call, but both have the error that the socket connection is lost and I can’t determine why

6

For a complete update, I have updated my init.py code for the bk_worker to the following:

plot = Application(FunctionHandler(plot))

from app import settings

settings.init()
sockets, settings.port = bind_sockets("localhost", 0)

def bk_worker():
      asyncio.set_event_loop(asyncio.new_event_loop())
      bokeh_tornado = BokehTornado({'/ais_plot': ais_plot}, extra_websocket_origins=["localhost:5000"])
      bokeh_http = HTTPServer(bokeh_tornado)
      bokeh_http.add_sockets(sockets)
      server = BaseServer(IOLoop.current(), bokeh_tornado, bokeh_http)
      server.start()
      server.io_loop.start()

t = Thread(target=bk_worker)
t.daemon = True
t.start()
7

Were you not adding extra_websocket_origins before? That’s typically the cause of a failed WS connection like in your screenshot. Bokeh has to be conservative and require explicit whitelisting of origins for security considerations (to prevent any random site from embedding your app without your knowledge or consent).

8

Yes so I actually ended up getting it working! I had to update the extra websocket origins to have the dynamically generated port, and I am using the port as a global variable within the thread.

extra_websocket_origins=["127.0.0.1:5000", "127.0.0.1:"+str(settings.port)])
1 Like