Deploy Bokeh Server into production on IP address

Sorry in advance if any of what I say sounds dumb, I don’t have any experience working with servers in any capacity.

I’m having trouble deploying a bokeh server on my local IP address that allows multiple users to use it (who are on the same network) at the same time. I’m using Flask and based it off the flask example on github. I’m trying to allow multiple users to use the application off the link “IP:5001,” where I’m referring to my IP address as IP here for security purposes. It allows a user to use it who is not me but doesn’t allow multiple users to use at the same time. I would’ve thought to increase the number of processes beyond 1 but I’m on a windows machine so was looking for some help on how to do this because I’m such a newbie with this stuff. I tried to use the gunicorn example but I’m on anaconda and there were no anaconda repositories for asyncio that work in Windows 64 bit. Thanks so much in advance for any help.

The basic code is as follows:

app = Flask(__name__, static_url_path='/static')
def bkapp(doc):
    doc.add_root(tabs)

@app.route('/', methods=['GET'])
def bkapp_page():
    script = server_document('http://IP:5006/bkapp')
    return render_template("embed.html", script=script, template="Flask")


def bk_worker():

    server = Server({'/bkapp': bkapp}, io_loop=IOLoop(), allow_websocket_origin=["IP:5001"])
    server.start()
    server.io_loop.start()


Thread(target=bk_worker).start()

if __name__ == '__main__':

    app.run(port=5001, host='IP')

@alexgrossman

The bokeh server example flask_embed.py,which uses the Flask development server, from which your app is derived works well with several users on an internal network using the server’s IP address instead of localhost just as you’ve done.

As a starting point, I would confirm that works as expected for multiple concurrent users in your network environment.

If so, you should probably next inspect the differences between your application and the illustrative one in flask_embed.py.

As the bokeh server is single-threaded, is your tabs model doing anything in a blocking mode? If so, you’ll want to go through the bokeh server documentation on how to have a thread for any necessary blocking opeations, etc.

Not sure what you mean by blocking mode. I’m now trying to use the waitress package to do it, on a modification of the “gunicorn” example. Should it work to use waitress with the traditional flask example to allow for multiple users? I do think the issue is the single thread because I need multiple users to be able to use it at once. I’m really lacking some understanding with respect to what is supposed to be happening here.

@alexgrossman

Can you run the flask_embed.py example as is, with the small exception to change the localhost to your IP? And then try to have multiple concurrent users? That works fine for several users under the Flask development server with only one process.

By blocking mode, I mean that your code is running an operation or function that causes the server to wait on input from a user such that it cannot do anything else like service requests coming in from other users to update their session(s).

No functions should be blocking. And the flask example doesn’t work as is for multiple users on my company’s network. That’s what I was using originally before trying to use waitress and asyncio to address the fact that it didn’t work.

@alexgrossman

I see. It works for me; I tested with three concurrent users via laptop and two cellphones to confirm.

The server was running on Mac OSX in my case with Python 3.8.3, bokeh 2.1.1, and Flask 1.1.2 all under the most recent Anaconda distribution.

I’m using an earlier version of Bokeh because of the bug regarding the changing of dropdowns visually when switching the options of the dropdown.

I have gotten it to work for multiple users using waitress and asyncio, but cannot get render_template to work with it currently.

I’m using the same code as the Flask example, but in main i’m using:
t = Thread(target=bk_worker)
t.daemon = True
t.start()

t.join()
print('Served')
waitress.serve(app, listen='url:5006')

And adding to the top of bk_worker:

**    asyncio.set_event_loop_policy(asyncio.WindowsProactorEventLoopPolicy())
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)**

@alexgrossman

What is the nature of it not working? Are there any printouts in the server terminal window when different users attempt to use the app?

Can you run from one computer with multiple browsers or browser tabs? If so that might rule out the software and point to something in your network topology.

Other users just get blocked and yes terminal outputs:

WARNING:bokeh.server.views.ws:Failed sending message as connection was closed

And it just wouldn’t open for those other users.

Works now with the edits above but render_template isn’t.

Note: with the code additions, it forced me to use computer_name.network.com:port instead of IP:port

Okay. The devs might have some input on why the websockets are being closed perhaps related to the implementation of things in Windows, which I do not use.

I do not have any experience with waitress, but the first thing I would check is that embed.html used by render_template is in the place that waitress expects. Does it look for this in the templates directory?

You can also use the JavaScript console for your client’s browser to see if there are any error logs there to see what is going on. Each browser is different, but for Chrome, right-click, select Inspect, and then activate the Console tab to see the JavaScript console.

Yes, embed is in templates folder. It worked fine when running on my IP. Now when using computer_name.network.com:port instead, it doesn’t seem to be recognizing it. Before, it would also load the html from my embed before loading the bokeh script. Now all of it loads when the url is put in the browser before showing any html

Using in Firefox rather than chrome because the data requirements for this script made chrome ungodly slow.

That’s fine. You can access the JavaScript console from Firefox too. CTRL+SHIFT+J or COMMAND+SHIFT+J (on Mac).

That might give some insight into why the rendering is not happening in the client’s browser assuming there are no errors related to it in the server console.

@alexgrossman

One additional thought… Given the claim that waitress is essentially a Windows equivalent to gunicorn for Unix operating systems, you might want to model your application on the bokeh flask_gunicorn_embed.py example on GitHub. The details of how this sets up/coordinates the server, IOLoop, and Tornado server might be relevant.

That’s what I’ve been trying to do. I’ve had success having multiple users use it at the same time, just not with also using an HTML template. The HTML template loads when I remove t.join() from above but in that case it seems to interpret the html file as a javascript file because I’m getting a syntax error of “<”

@alexgrossman

Here’s a version that works with waitress. I essentially modified the flask_gunicorn_embed.py example from bokeh’s GitHub by adding the APP_SERVER_ADDR and APP_SERVER_PORT variables.

I run it using the waitress-serve command line option as follows.

waitress-serve --listen=192.168.0.135:8080 --threads=4 flask_waitress_embed:app

flask_waitress_embed.py

try:
    import asyncio
except ImportError:
    raise RuntimeError("This example requries Python3 / asyncio")

from threading import Thread

from flask import Flask, render_template
from tornado.httpserver import HTTPServer
from tornado.ioloop import IOLoop

from bokeh.application import Application
from bokeh.application.handlers import FunctionHandler
from bokeh.embed import server_document
from bokeh.layouts import column
from bokeh.models import ColumnDataSource, Slider
from bokeh.plotting import figure
from bokeh.sampledata.sea_surface_temperature import sea_surface_temperature
from bokeh.server.server import BaseServer
from bokeh.server.tornado import BokehTornado
from bokeh.server.util import bind_sockets
from bokeh.themes import Theme

APP_SERVER_ADDR = "192.168.0.135"
APP_SERVER_PORT = 8080

if __name__ == '__main__':
    import sys
    sys.exit()


app = Flask(__name__)

def bkapp(doc):
    df = sea_surface_temperature.copy()
    source = ColumnDataSource(data=df)

    plot = figure(x_axis_type='datetime', y_range=(0, 25), y_axis_label='Temperature (Celsius)',
                  title="Sea Surface Temperature at 43.18, -70.43")
    plot.line('time', 'temperature', source=source)

    def callback(attr, old, new):
        if new == 0:
            data = df
        else:
            data = df.rolling('{0}D'.format(new)).mean()
        source.data = ColumnDataSource.from_df(data)

    slider = Slider(start=0, end=30, value=0, step=1, title="Smoothing by N Days")
    slider.on_change('value', callback)

    doc.add_root(column(slider, plot))

    doc.theme = Theme(filename="theme.yaml")

# can't use shortcuts here, since we are passing to low level BokehTornado
bkapp = Application(FunctionHandler(bkapp))

sockets, port = bind_sockets(APP_SERVER_ADDR, 0)

@app.route('/', methods=['GET'])
def bkapp_page():
    script = server_document('http://%s:%d/bkapp' % (APP_SERVER_ADDR,port))
    return render_template("embed.html", script=script, template="Flask")

def bk_worker():
    asyncio.set_event_loop(asyncio.new_event_loop())

    bokeh_tornado = BokehTornado({'/bkapp': bkapp}, extra_websocket_origins=["%s:%d" % (APP_SERVER_ADDR,APP_SERVER_PORT)])
    bokeh_http = HTTPServer(bokeh_tornado)
    bokeh_http.add_sockets(sockets)

    server = BaseServer(IOLoop.current(), bokeh_tornado, bokeh_http)
    server.start()
    server.io_loop.start()

t = Thread(target=bk_worker)
t.daemon = True
t.start()

Every time I try that on my computer I get:

RuntimeError: There is no current event loop in thread ‘Thread-1’.

What version of bokeh, tornado, etc are you using?

This issue on GitHub notes a similar problem and mentions issues related to asyncio and tornado changes related to Tornado 5.0.

https://github.com/tornadoweb/tornado/issues/2308

I can’t figure out my tornado version, definitely using Bokeh 2.0.2

It works for multiple users, but when loading for another user (which takes about 4-5 minutes in this case), callbacks no longer work. Plus the template doesn’t load.

@alexgrossman

To clarify, the flask_waitress_embed.py example I posted takes 4-5 minutes to load or your app derived from that does? The example I posted loads fairly quickly even on a cellphone (a few seconds) and does respond to callbacks over WiFi with a smallish lag on the phone.

I can rollback to bokeh 2.0.2 in another environment to check, but I cannot (easily) run a server under Windows.

My app derived from that. It’s the exact same outside of the bkapp function

Does the reference waitress example now work on your setup or are there issues with the template rendering there too? That’s all I can really run for comparison / testing purposes.

Once that works, the next step is to pare down your code or build up incrementally from that example within the bk_app function to see what causes breakages or very unresponsive behavior.

With the waitress example you posted I get the same error of:

RuntimeError: There is no current event loop in thread ‘Thread-1’.

I’m using Tornado 6.0.3 by the way with Bokeh 2.0.2

Could downgrading Tornado fix the issue or could that mess up what’s being output by Bokeh?

I wouldn’t go that route. First, I’m not sure that’s a problem with that per se and second, Bokeh’s release notes have specific requirements for Tornado > 5.0/5.1 for releases 2.0/2.1, respectively.

Do you have any other suggestions? Sorry for being annoying, I just spent a month building a bokeh app that doesn’t work for multiple users at once so pretty stressed about this not working no matter what I try

Also, if the application was smaller, I believe the Flask app would work. it doesn’t seem to work because since it takes 5 mins to load. The one thread can only load one user at once. We then see one user taking 10 mins to load and by the time that even happens, the user gets bounced out because the bokeh Token expires. (Plus I need the multiple threads so that it doesn’t take 30 minutes when it is 6 users, which is realistically the most that would likely ever be on this tool)

It is certainly not annoying, so don’t worry about that.

Stressful, I get it. Deep breaths.

We know it works in principle with waitress, we just need to figure out if its due to the OS where you’re running the server, package versions, etc. And, for what it’s worth, I have a fairly complex self-hosted bokeh app of my own so there is no fundamental problem jumping from the illustrative examples to real scientific or visualization problems.

If you have the ability to set up a separate Python virtual environment with the latest versions of everything, you could try that to see if the waitress example works there. If not, it seems like the only difference from your setup and mine is the OS.

I’d also look seriously into profiling where things are taking so long in the startup. If it’s computations on the server is there anything that can be done to streamline? If it is pushing complex layouts or models or whatever to the client that’s another thing to investigate.

The problem is that, in production, I can’t use a newer version of bokeh because I ran into issues as in Select options update - #19 by _jm

I also don’t have the ability to set up a new virtual environment that easily because of corporate restrictions. And I can’t really speed it up that much, it’s pulling in and manipulating a couple of Gigs of data which is what makes it slow - I already have different runs for twice a day that do a lot of pre-computations to make these runs as fast as possible.

I’m not too worried about converting the simple example into my own app, I’m just worried about getting this simple example working.

Understood. I was just suggesting this as a temporary step to isolate if the problem is due to package versions or something more fundamental like running on Windows versus Linux (or OS X).

I understand, I just would have to upgrade and then downgrade again and if it was to work, I would just be in a terrible spot anyways because I’d be stuck between picking between two versions, each of which has issues for what I need to do.

That’s why I was looking at this:

But if I was to downgrade then bokeh would no longer work so really unsure about what to do here.

@Bryan I was wondering if you have any suggestions? I saw this thread (Tornado 5.0 compatibility · Issue #7308 · bokeh/bokeh · GitHub), unsure if this still applies here or is relevant. To summarize, the flask_gunicorn_embed.py example doesn’t work on Windows, even when using waitress instead of gunicorn as I get the issue:

RuntimeError: There is no current event loop in thread ‘Thread-1’.

This seems to have been addressed in bokeh in early 2018. See https://github.com/bokeh/bokeh/pull/7329.

And bokeh 2.0.2 that you’re using is circa Q1/Q2 2020.

For what it is worth, I have been working heavily in 2.0.2 for most of my development, which involves a lot of server deployment for multiple users. But, unfortunately, I’m doing everything in Linux and sometimes OS X.

Got it. I would of course just use multiple processes on the simple flask app but since I’m on windows I can’t. I can’t really upgrade or downgrade Bokeh as I said so I’m kind of stuck. I tried using asyncio.set_event_loop_policy(asyncio.WindowsProactorEventLoopPolicy())
as well with the example but it didn’t work either.

@alexgrossman

A poor-man’s short-term solution might be to run your flask app six times, for example, with each set up for a separate port. Then you’ll have to do the bookkeeping/management of having your users only log in to their assigned port to avoid collisions. Would that work, albeit inelegantly, until you can investigate further?

So unfortunately I have zero experience with waitress (this is the first I’ve heard of it) and not much experience with Windows either. I’ve been variously asking the community for years if anyone would like to join the project to help anchor support on Windows but to date, no one has wanted to do that.

It’s hard to make specific suggestions without knowing what your actual constraints. E.g. can you run multiple instances of the server app on different ports, and then put them behind an Nginx load balancer so that all users see one URL?

I also don’t really have context for the statements about 5, 10 minutes to load. Are you doing 10 minutes of blocking work on every session creation? Can anything be done once up front e.g in a server lifecycle hook for server start?

I think so, definitely not permanently of course, but I do get the following error when trying to do so (when using same IP but different port):

OSError: [WinError 10048] Only one usage of each socket address (protocol/network address/port) is normally permitted

Me neither. I would be happy to run multiple instances of the app on different ports. I just personally have zero experience with this stuff (I studied AI so hadn’t even heard of a port until using Bokeh)

I would ignore the 5/10 minutes to load, as that is just a function of how much data needs to be opened. I’m not sure what a server lifecycle hook does or how I would use it but if it would reduce the load time to instantly then I guess that would potentially solve the issue with the flask app.

Sorry again for my lack of knowledge regarding these things. I really do enjoy using Bokeh, the only reason I’m so stressed about this right now is because I thought multiprocess deployment would be easy and I already presented this to my bosses and said I would get a link to the group soon but I can’t of course because of these issues.

Not sure why this is happening if you’re indeed using unique addresses/port combinations in the multiple instances being run. Perhaps another process is already using the ports you’ve chosen.

Look here for information on the error for Windows systems, and one of the responses shows how to check for processes using the ports that are causing the errors.

https://stackoverflow.com/questions/12362542/python-server-only-one-usage-of-each-socket-address-is-normally-permitted

See the Directory Format section of the bokeh server user’s guide.

If you include an app_hooks.py file with the functions named therein, bokeh will run those at the appropriate times of the server lifecycle.

on_server_loaded() gets run when you start the server on your Windows machine.

on_session_created() gets run when each client session starts.

So if you need to do any setup work that is common to all users you could do that upfront when the server is spooled up via on_server_loaded() and have it ready to go when users connect.

I believe it’s overlapping because it won’t run unless the port referenced in server_document is 5006 so it has to be that one for both right now

Would I be able to save everything to a dictionary that is then referenced when different users log on? I already precompute everything in a single dictionary as is. And no user is ever changing data within my app. Are there any examples of this?

Oh no. That definitely won’t work and it doesn’t need to be that way.

See the example I posted for the waittress embed problem. Not for the server implementation, but just for how the ports and addresses were handled. I used variables defined at the topmost scope of the file and then referenced them everywhere the IP addresses and ports are prescribed.

My python code doesn’t seem to run when I do what you did though. Not sure what the issue is here.

Where does the server_context input to this function come from? Or is that not something I need to add? With respect to the on_server_loaded as in the documentation. And if I run some stuff in the app_hooks.py file, how do I then reference that dictionary in my main py file? I’m also a little confused with respect to running this as a flask app as I do now. It seems that lifecycle hooks need to be used with bokeh-serve which i don’t use.

You don’t need to provide it. Bokeh provides it for the server when it calls that method; you don’t explicitly call that method, bokeh will do it automatically if the app_hooks.py file exists in the directory structure.

What I think you need to do is assign our data to that server context. And then when the session is created for a given user, make a copy of it. I believe the server_context is accessible via the session context when the session created in on_session_created().

You have access to the session context as session_context property of the bokeh document for each client session.

How do i refer to session_context in my main.py file? Sorry for all of the questions, just struggling to find any examples.