I am running a bokeh server instance which is managed directly by systemd1. Occasionally, an exception is thrown which leaves the server in a broken state until until it is restarted. This is an undesirable failure mode as it requires external monitoring of either websocket responses or logs in order to detect a fault. Ideally, unhandled app exceptions would cause the server instance to die, which in my case would be immediately restarted by systemd. A definitive failure mode would also be preferable when load-balancing between multiple instances.
I am clueless about the workings of tornado but after a quick skim of
bokeh/server/server.py, I suspect that exceptions are being caught by default in the tornado event loop? Is there a reasonably straight forward path to add support for app generation exceptions to be a fatal error? Any pointers would be greatly appreciated.