I am working on a multi-class classification app in bokeh server (version 2.0.2), which can have 30-40 classes in a typical use scenario. At present, the updating of graphics is responsive and quick.
However, the initial session startup can take approx. 2-3 seconds for the server side work to generate models and plots. A lot of the graphics are common among the plots and can be carried out in an iterator over the number of classes.
Conceptually, it is straightforward to leverage the multi-core CPU and multiprocessing capabilities (either through python’s multiprocessing or pathos’ multiprocess) to lessen the startup time.
I am running into similar problems as cited in this stackoverflow topic related to multiprocessing as it pertains to bokeh. The failure mode depends on whether using core python multiprocessing (e.g. often fails due to pickling limitations of lambda functions used in bokeh) or pathos’ multiprocess, but both methods are unsuccessful.
Are there any success cases in distributing bokeh server-side computations to take advantage of multi-core CPU? My use case/problem is different than that in the cited stackoverflow topic, in that I am narrowly focused on startup time of sessions.
An example of the error when trying to concurrently generate multiple figures via multiprocessing pools follows.
MaybeEncodingError: Error sending result: '[Figure(id='1005', ...), Figure(id='1057', ...)]'. Reason: 'PicklingError("Can't pickle <function ColumnDataSource.<lambda> at 0x128fc2440>: attribute lookup ColumnDataSource.<lambda> on bokeh.models.sources failed")'