How to bootstrap a Bokeh server from a module within an "egg" package?

Hello there.

I’m working on a project named pyklatt, and under a submodule called tools there is an application that makes use of Bokeh server. The current structure looks like this, and the standalone tool is in the file kldiff.py.

pyklatt/
├── __init__.py
├── __main__.py
├── klatt.py
└── tools/
    ├── __init__.py
    └── kldiff.py     <--

The code works fine as a standalone script when called via main() function from bokeh bootstrap module (you will see the whole script later on this issue):

from bokeh.command.bootstrap import main
main(['bokeh', 'serve', '--show', 'kldiff.py'])

The problem is that Python modules installed via setuptools are stored into an .egg package file, and when I try to execute the same code as a module ( e.g., python -m pyklatt.tools.kldiff ), it yields the following error:

ERROR: Path for Bokeh server application does not exist: /home/cassio/anaconda3/envs/k88-prod/lib/python3.8/site-packages/pyklatt-0.0.1-py3.8.egg/pyklatt/tools/kldiff.py

Full working code example:

# standalone module to be executed from CLI as '$ python3 kldiff.py'
import sys

import numpy as np

from bokeh.layouts import column
from bokeh.models import ColumnDataSource
from bokeh.plotting import curdoc, figure

# dummy var for example purposes
x = np.arange(0, 100, 0.1)
IINT16 = np.iinfo(np.int16)

wg_plots = []
for i in range(3):
    raw_1 = np.sin(x) * IINT16.max // 3
    raw_2 = np.cos(x) * IINT16.max // 3

    src = ColumnDataSource(data=dict(xmax=range(len(x)),
                                     raw_1=raw_1.copy(),
                                     raw_2=raw_2.copy()))

    p = figure(plot_width=1200, plot_height=300)
    p.line('xmax', 'raw_1', source=src,
           color=np.random.choice(['red', 'green', 'blue', 'yellow', 'cyan']))
    p.line('xmax', 'raw_2', source=src,
           color=np.random.choice(['red', 'green', 'blue', 'yellow', 'cyan']))

    wg_plots.append(p)

curdoc().add_root(column(wg_plots))
curdoc().title = "kldiff"

if __name__ == '__main__':
    from bokeh.command.bootstrap import main
    base_cmd = 'bokeh serve --show %s' % sys.argv[0]
    main(base_cmd.split())

@cassota As it is now, the bokeh serve command line application looks for actual files to load and parse, so I don’t think it is going to work with modules inside eggs since AFAIK eggs store all the modules in some zipped format. I think you will need to embed the Bokeh server as a library, so that you can configure the Bokeh app programmatically once the module code is already loaded by Python.

1 Like

Thank you, @Bryan. That worked pretty well.

Here’s the modified example for further reference:

# standalone module to be executed from CLI as '$ python3 kldiff.py'
import sys

import numpy as np

from bokeh.layouts import column
from bokeh.models import ColumnDataSource
from bokeh.plotting import figure
from bokeh.server.server import Server

# dummy var for example purposes
x = np.arange(0, 100, 0.1)
IINT16 = np.iinfo(np.int16)


def bkapp(doc):
    wg_plots = []
    for i in range(3):
        raw_1 = np.sin(x) * IINT16.max // 3
        raw_2 = np.cos(x) * IINT16.max // 3

        src = ColumnDataSource(data=dict(xmax=range(len(x)),
                                         raw_1=raw_1.copy(),
                                         raw_2=raw_2.copy()))

        p = figure(plot_width=1200, plot_height=300)
        p.line('xmax', 'raw_1', source=src,
               color=np.random.choice(['red', 'green', 'blue', 'yellow']))
        p.line('xmax', 'raw_2', source=src,
               color=np.random.choice(['red', 'green', 'blue', 'yellow']))

        wg_plots.append(p)

    doc.add_root(column(wg_plots))
    doc.title = "kldiff"


server = Server({'/': bkapp}, num_procs=1)
server.start()

if __name__ == '__main__':
    print('Hi I am beling called from "%s"' % sys.argv[0])
    server.io_loop.add_callback(server.show, "/")
    server.io_loop.start()

In my real app module, the following is printed on stdout. It is indeed being called from inside the .egg file:

$ python -m pyklatt.tools.kldiffi ~cassio/fb-gitlab/fb-audio-corpora/dectalk11k-harvard2/data/paul/paul_hldata[0-2].k88.raw

Hi I am being called from “/home/cassio/anaconda3/envs/k88-prod/lib/python3.8/site-packages/pyklatt-0.0.3-py3.8.egg/pyklatt/tools/kldiffi.py”
Opening in existing browser session.

1 Like