Flask + Bokeh + CustomJS Callback Issues

I’ve got a rather complex Flask + Bokeh project that retrieves a .tsv, visualizes it, and lets the user mess with the visualization.

Because of the complexity of the project, the script defining the ‘route’ to the server, the script defining the bokeh logic, and the .html script are in three different files. I implemented all the bokeh stuff (including figure construction, population of the source variable, and CustomJS logic) as methods/instance variables of a user-defined class called BokehObject, to pay the ‘technical burden’ up front and make the ‘routes’ script as neat as possible. I’ve attempted to replicate that in my minimum reproducible example shown below. The only difference is that the ‘routes’ file and the BokehObject class implementation are in the same script.

I’ve already confirmed the method returning the list containing the CustomJS object is correctly returning that CustomJS object and the widgets/controls - but the JS code itself doesn’t appear to be functioning properly. The widgets appear in the application, but the sliders do not appear to be functional. They don’t filter the data or update the number right above the widget.

I have no idea how to debug/step-through Javascript code that is embedded in a Python script, but executed by the server! As far as I can tell, the JavaScript syntax is right. But I must be missing something somewhere, either in the control_list logic or the actual callback code. If I had to guess though, I think the issue may be the widget logic in control_array, so a few pointers would be much appreciated.

I’ve attached the two scripts needed for a minimum reproducible example. The first one defines the BokehObject class I made and a basic ‘route’ to localhost:5000. To reproduce the problem, this file should be named app.py.

Preface: I apologize for many of the unneeded extra steps included in this example implementation of my BokehObject class. It will seem totally unneeded here. I made some lazy adjustments from the more complicated project, such that the class implementation would be involved for the example. Additionally, I included a lot unneeded imports and whitespace. I’ve included them just in case some weird masking issue I am unaware of is going on, and in the real project the weird whitespace is helping keep my script organized. Here is app.py:

from flask import Flask, render_template
import pandas as pd
import numpy as np
import sys
import os
import json
from math import pi
from operator import itemgetter
import re
import hashlib
from os.path import dirname, join, isfile
from bokeh.io import curdoc
from bokeh.plotting import figure, output_file, show
from bokeh.layouts import row, layout, widgetbox, column
from bokeh.models import ColumnDataSource, HoverTool, Div, FactorRange, Range1d, TapTool, CustomJS, ColorBar, CDSView, GroupFilter, MultiSelect, MultiChoice
from bokeh.models.widgets import Button, Slider, Select, TextInput, RadioButtonGroup, DataTable, TableColumn, NumberFormatter, Panel, Tabs, HTMLTemplateFormatter
from bokeh.models.callbacks import CustomJS
from bokeh.palettes import RdYlBu11, Spectral4, Spectral11, Set1, Viridis256
from bokeh.util import logconfig
from bokeh.resources import INLINE
from bokeh.embed import components, autoload_static
from bokeh.transform import factor_cmap, linear_cmap
from pathlib import Path
import math

app = Flask(__name__)

def index():

    class BokehObject:

        def __init__(self):
            self.source = self.source_make()
            self.df = self.just_df()
            self.dotplot = self.make_figures()
            self.controls_and_logic = self.control_jslogic()

        def just_df(self):

            d = {'LEVEL': ['superkingdom', 'phylum', 'class', 'order', 'family', 'genus', 'species', 'strain'], 
                'NAME': ['Viruses', 'dsDNA viruses, no RNA stage', 'Poxviridae - no_o_rank - no_c_rank', 'Poxviridae - no_o_rank', 'Poxviridae', 'Orthopoxvirus', 'Vaccinia virus', 'Vaccinia virus strain WR (Western Reserve'],
                'READ_COUNT' : [10239.0, 35237.0, 0.0, 0.0, 10240.0, 10242.0, 10245.0, 10245.1], 'LINEAR_COV' : [6723.01, 6723.01, 6723.01, 6723.01, 6723.01, 2008.6, 2008.6, 2008.6], 
                'SCORE' : [5.1547, 5.1547, 5.1547, 5.1547, 5.1547, 5.1547, 1.5651, 1.5651]}

            d = pd.DataFrame(data=d)

            return d

        def source_make(self):

            d = {'LEVEL': ['superkingdom', 'phylum', 'class', 'order', 'family', 'genus', 'species', 'strain'], 
                'NAME': ['Viruses', 'dsDNA viruses, no RNA stage', 'Poxviridae - no_o_rank - no_c_rank', 'Poxviridae - no_o_rank', 'Poxviridae', 'Orthopoxvirus', 'Vaccinia virus', 'Vaccinia virus strain WR (Western Reserve'],
                'READ_COUNT' : [10239.0, 35237.0, 0.0, 0.0, 10240.0, 10242.0, 10245.0, 10245.1], 'LINEAR_COV' : [6723.01, 6723.01, 6723.01, 6723.01, 6723.01, 2008.6, 2008.6, 2008.6], 
                'SCORE' : [5.1547, 5.1547, 5.1547, 5.1547, 5.1547, 5.1547, 1.5651, 1.5651]}

            df = pd.DataFrame(data=d)

            source = ColumnDataSource()
            source.data = dict(
                    rank = df['LEVEL'].tolist(),
                    taxa = df['NAME'].tolist(),
                    reads = df['READ_COUNT'].tolist(),
                    score = df['SCORE'].tolist())

            return source

        def control_jslogic(self):

            control_list = []

            controls = {
            "reads" : Slider(             title = "Minimum Raw Read Count",   value = 0,   start = 0,   end = 500000,   step = 1000),
            "score" : Slider(             title = "Minimum Score",            value = 0.5, start = 0.5, end = 10,     step = 0.1)

            controls_array = controls.values()

            callback = CustomJS(args=dict(source=self.source, controls=controls), code="""
                if (!window.full_data_save) {
                    window.full_data_save = JSON.parse(JSON.stringify(source.data));
                var full_data = window.full_data_save;
                var full_data_length = full_data.x.length;
                var new_data = { rank: [], taxa: [], reads: [], score: []}
                for (var i = 0; i < full_data_length; i++) {
                    if (full_data.rank[i] === null || full_data.taxa[i] === null || full_data.reads[i] === null || full_data.score === null)
                    if (
                    full_data.reads[i] > controls.reads.value &&
                    full_data.score[i] > controls.score.value
                    )   { 
                    Object.keys(new_data).forEach(key => new_data[key].push(full_data[key][i]));
                source.data = new_data;

            control_list = [controls_array, callback]
            return control_list

        def make_figures(self):

            fig = figure(plot_height=600, plot_width=720, x_range = FactorRange(*self.df["NAME"].unique()))
            fig.circle(x="taxa", y="reads", source=self.source, size=5)            
            fig.xaxis.axis_label              = "Score"
            fig.yaxis.axis_label              = "Number of Reads"
            fig.xaxis.major_label_orientation = 3.1415926/4
            fig.yaxis.axis_label              = "Read Count"

            return fig

    bo = BokehObject()

    fig = bo.dotplot

    control_list = bo.controls_and_logic
    control_array = control_list[0]
    callback = control_list[1]

    for single_control in control_array:
        single_control.js_on_change('value', callback)

    inputs_column = column(*control_array, width=320, height=1000)
    layout_row = row([inputs_column, fig])

    script, div = components(layout_row)
    return render_template(

if __name__ == "__main__":

This next one is the html template (named index.html):

<!DOCTYPE html>
<html lang="en">
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
{{ js_resources|indent(4)|safe }}
{{ css_resources|indent(4)|safe }}
{{ plot_script|indent(4)|safe }}
<h1 style="text-align: center; margin-bottom: 40px;">Flask + Bokeh</h1>
<div style="display: flex; justify-content: center;">
    {{ plot_div|indent(4)|safe }}

To run this example, you’d need to create a new python project, and arrange the files like so:

Project Name:
→ app.py
→ templates
-------> index.html

I’ve abstracted away a lot of the detail from the real project that might come into play here; I honestly don’t know. These are the main potential factors I can think of:

  1. The file index.html has a different name and more complicated route (it is a page that is part of a larger GUI), whose path is defined by inserting an ‘ID’ number related to the .tsv file the dataframe is constructed from. As far as I can tell though, the dataframe and the source variable is populating correctly, and is being passed to the CustomJS widget controls properly. In this example I created a mini dataframe that looks sort of like the real data instead.

  2. There are a lot more columns in the source variable and a lot more widgets in the visualizer. I’m not sure if they are interacting in some strange way.

  3. I made a separate forum post related to this same project, which you can find here: Issue with components() return value
    In that post, I mention an example I followed to get the script and div variables that components() requires to pass to the .html and construct the webpage. That example can be found here:
    While comparing my script with the example, you’ll notice that I follow the syntax for the CustomJS() part very closely, and that the widgets on the subsequent webpage appear to be functional. Maybe there is a problem with the call to components() after all?

I can post the JS logs here if needed. I don’t have much experience with javascript and have had difficulty making heads or tails of what is going on.

@loremaster.cerberus If you run with BOKEH_MINIFIED=no and then open the browser debugger (and set to stop on errors) you can see that the error is you are trying to access an x field on the data that does not exist:

Which makes sense, the only fields you appear to be putting in the data source are these:

source.data = dict(
    rank = df['LEVEL'].tolist(),
    taxa = df['NAME'].tolist(),
    reads = df['READ_COUNT'].tolist(),
    score = df['SCORE'].tolist()

Bryan, this simple fix worked for my minimal example! However, I’m still struggling to get the real version working - I made sure that var full_data_length is referencing ‘taxa’ instead of ‘x’.

What happens if the dict() in source.data is actually way more complex than just the four {key:value} pairs I included in this minimal example? Does every single pair need to be represented in var new data and the if (full_data.var[i] === null || ...) statement? For testing purposes, I was trying to keep things in the callback as simple as possible. I can’t just comment out the additional lines making up the construction of source.data because it would break everything else.

Also, there are a few columns in the dataframe I used that have a NULL value. As far as I understand though, that shouldn’t matter for a given row unless every value is NULL, due to the || statement connecting all of the if (full_data.var[i] === null || ...) statements.

It’s not too bad in my case; it’s only like 40-something-odd columns I’d need to add (and eventually I would need to add them anyway for callback logic purposes). But if this were true, it would make callbacks impractical for folks with hundreds of columns in their dataframe.

@loremaster.cerberus I’m sorry I can’t really answer the questions above because I don’t really have any context at all about what you are actually trying to accomplish. i.e. I don’t know the reason you are doing if (full_data.var[i] === null || ...) in the first place so I can’t really advise you about making it better. I have to be up front and state that the most efficient way I am able to leverage my expertise is on small, focused technical questions with well-defined answers, that I can digest in a glance (or very quickly run and experiment with).

It seems like your actual question is about CustomJS, and has nothing to do with Flask. I would suggest trying to strip away all the things that are unrelated to the CustomJS interaction you want to develop, e.g. make a very small standalone script to experiment on in isolation, the does contain anything relevant to your real issue, but nothing extra. That will help me, but probably also help you clarify your actual need.

This looks like something that would be good to know how to do - but it’s just a bit over my head…
Can you explain step by step so I can try it?

How do you run with “BOKEH_MINIFIED=no” is that an environment variable? In the shell that is running the browser? Or set in the javascript somewhere?

And then “stop on errors” is set in the browser debugger?

@jcarson It is a shell environment variable. Setting environment variables differs by platform and by shell, so you will have to look up how to do it on your specific setup. In this case, I set it before running the flask app but you could set it before running a Bokeh server app, or a script that generates Bokeh standalone output.

Using the browser dev tools is also something that differs by browser, so again I’ll have to defer to other information online specific to whatever browser you are using. AFAIK most of them offer settings e.g. to pause on all exceptions, or only on uncaught exceptions, or only explicit manual breakpoints. (FYI turning the minification off with the env var is necessary to be able to step through the code effectively)

@Bryan, the series of full_data.var[i] === null statements are meant to ensure the callback reaches to every row of data in source even when null values are encountered in certain columns.

I will mention that I’ve gotten the simplest of the widgets in my application running. All of the Slider() objects appear to be functional. However, many of the more difficult JSCallback interactions I’d still like to include to the app seem totally out of reach. Bryan, I searched through the forum for examples/support for more complex use cases of widgets, and happened upon this page where you responded to another user, @CONSUMERTEC. You directed them to the examples directory of the repo, but even here I wasn’t able to find concrete examples of several widget implementations.

Take the Multichoice class, for example. There are 16 results in the repo; two of the files there, inputs.py and interaction_multichoice.py encompass what is already posted in the JSCallbacks docs. There is one other file called test_multi_choice.py in which a Multichoice widget input_box is defined and added to a layout, but the test didn’t appear to include any example of use of that widget alongside a CustomJS() callback!

I suspect that my ‘real issue’ as you allude to above in the implementation of widgets is not being held up on the Bokeh/Python side, but rather on the Javascript logic side. I’ve no idea where I should go to see examples of Bokeh widgets being deployed on a server via legitimate JS callback. As you said, it’s not Flask that is the problem. It’s the JS. Do you know of any other resources that might be able to help?

Aside from that the other option is to post smaller examples of single controls here and work them out with the forum community one-at-a-time. But that is arduous work. I’d rather gain the skills to solve this JS problems on my own.

TL;DR: Seeing examples on the examples repo that go something like: CustomJS(code="console.log('button: click ', this.toString())")) aren’t helpful. Or if they are, I’m not sure how they are helpful

Hello loremaster,

I would like to recommend this example:

[Meetups/Meaningful insights with Bokeh Dashboard at master · parulnith/Meetups · GitHub](https://github.com/parulnith/Meetups/tree/master/Meaningful insights with Bokeh Dashboard)

Hope this help.



@CONSUMERTEC I took a look at Bokeh_DashboardCode.py from the meetup example.
The way they implement their custom features feel fundamentally different from what I’m trying to do here. In my case, I have a neat little dictionary “control_list” that contains all the widgets I want to add, and then the single callback accesses that dictionary and considers some user-specified logic dictating how that widget should be used. So what I’m really looking for is insight into that Javascript logic. The folks who wrote this script use more of a “Frankenstein” and almost pure-python approach. I couldn’t find any reference to the JSCallback() function.

My assumption is that I’ll need to access @Bryan’s expertise in the way he described earlier: by one at a time, going through a series of small scale, technical widget examples, then working to scale them up to my real project. By the time I’m done I think it’d make for a great resource for the community. I’ll start a dedicated thread. @Bryan, I’m hoping to use the exact same version of ‘index.html’ found in this thread in all cases as well as slight adjustments to app.py. Should I repost them in the new thread and reference this one to explain why I’m posting them again? I want to make sure not to clutter the forum with a slew of my own posts that mostly talk about the same stuff.

@loremaster.cerberus I am afraid I am occupied by some family concerns right now. I am not going to be able to provide any immediate detailed help at the level you seem to be expecting. What I will say say is this: What you should probably search for in the examples and docs is simply CustomJS, not any specific widget. The reason is that most CustomJS callbacks look very similar in overall struture, regardless of what the widget is. That is, a typical CustomJS code looks like:

  1. get the widget’s value (or maybe multiple widgets’ values)
  2. do some computation, maybe involving a data source
  3. set some properties or update a data source with the result

Step 1) is looking up slider.value or input.text or souce.data["foo"] or whatever

Step 3) is often like source.data = new_data or div.text = new_text

Obviously there are minor differences, but the general idea and structure is the same for almost every example, regardless of the the widgets involved. You need to figure out step 2) but to be honest, I am not going to really be able to help much with that in any case, since you are the one who knows what the computation to carry out is. Once you have the inputs to your computation, it’s really just a pure JS programming question, and I am a pretty terrible JS programmer.

1 Like