Slider object not updating plot, callback function not working

Hi,

I am having trouble connecting my slider object through a callback. I am trying to change the data on the plot based on the slider value. I am using bokeh 1.3.4 but can use the latest version (2.2 I believe) if needed, but I found it easier to use 1.3.4.

I am using the output of a tsne model to plot data through a scatter plot on Bokeh. Everything works fine except the callback for the slider.

Slider Code:

y_labels = y_pred

#data sources
#desc contains the cluster value for each record

source = ColumnDataSource(data=dict(
x=vis_x, y=vis_y, x_backup=vis_x, y_backup=vis_y, desc=y_labels, message=df['message'], date=df['date'], forms=df['forms'], status=df['status'], substatus=df['substatus'], response=df['response'], labels=["C-" + str(x) for x in y_labels] ))

# map colors
mapper = linear_cmap(field_name='desc',
palette=Category20[10],
low=min(y_labels), high=max(y_labels))

# prepare the figure
plot = figure(plot_width=1200, plot_height=850,
tools=['lasso_select', 'box_select', 'pan', 'box_zoom', 'reset', 'save', 'tap'],
title="Clustering and LDA",
toolbar_location="above")

# plot settings
plot.scatter('x', 'y', size=5,
source=source,
fill_color=mapper,
line_alpha=0.3,
line_color="black",
legend='labels')
plot.legend.background_fill_alpha = 0.6

# Keywords
text_banner = Paragraph(text='Keywords: Slide to specific cluster to see the keywords.', height=45)
input_callback_1 = input_callback(plot, source, text_banner, topics)
# currently selected
div_curr = Div(text="""Click on a plot to see more information.""", height=150)
callback_selected = CustomJS(args=dict(source=source, current_selection=div_curr), code=selected_code())
taptool = plot.select(type=TapTool)
taptool.callback = callback_selected

# WIDGETS - HERE IS WHERE THE PROBLEM BEGINS
slider = Slider(start=0, end=10, value=10, step=1, title="Cluster #", callback=input_callback_1)

# pass call back arguments input_callback_1.args["slider"] = slider

ā€˜ā€™ā€™

Callback Code:

def input_callback(plot, source, out_text, topics):
# slider call back for cluster selection
callback = CustomJS(args=dict(p=plot, source=source, out_text=out_text, topics=topics), code="""

            var data = source.data; 
	var cluster = slider.value;
	        		        
            x = data['x'];
            y = data['y'];
            x_backup = data['x_backup'];
            y_backup = data['y_backup'];
            labels = list(set(data['desc']));
            desc = data['desc']
            message = data['message'];
            date = data['date'];
            forms = data['forms'];
            status = data['status'];
            substatus = data['substatus'];
            hac_code = data['hac_code'];
            
            if (cluster == '10') {
                    out_text.text = 'Keywords: Slide to specific cluster to see the keywords.';
                    for (i = 0; i < x.length; i++) {
                        if(desc[i] == cluster) {
                                x[i] = x_backup[i];
							    y[i] = y_backup[i];
					}   else {
							    x[i] = undefined;
							    y[i] = undefined;
						}
                    }
                }
                else {
                    out_text.text = 'Keywords: ' + console.log(topics[Number(cluster)].toString());
                    for (i = 0; i < x.length; i++) {
                        if(desc[i] == cluster) {
								    x[i] = x_backup[i];
								    y[i] = y_backup[i];
							} else {
								x[i] = undefined;
								y[i] = undefined;
							}
                        } 
                }
    source.change.emit();
""")
return callback

ā€˜ā€™ā€™

I have been trying to understand callbacks and CustomJS for a week and Iā€™m not getting anywhere. any help is appreciated!

Hi @Arjun_Saravanan please edit your post to use code formatting so that the code is intelligible (either with the </> icon on the editing toolbar, or triple backtick ``` fences around the code blocks).

1 Like

one problem you have is that slider is returning an int, so your variable cluster is an int, but in your line if (cluster == '10') { you are trying to compare an into to a string, which doesnā€™t work.

1 Like

Have made the following edits requested by Bryan and ben_hastings. Unfortunately the actual callback function is still not running. Please advise.

@Arjun_Saravanan your code is still not formatted. You need to enclose the entire code block in triple backtick fences, not individual lines. you can seen an example here under Block Code Formating. I will take a look at the code but only as soon as I can copy and paste it and run it.

Slider Code:

Copy to clipboard


y_labels = y_pred


# data sources

source = ColumnDataSource(data=dict(
x=vis_x,     y=vis_y,     x_backup=vis_x,     y_backup=vis_y,     desc=y_labels,     message=df['message'],     date=df['date'],     forms=df['forms'],     status=df['status'],     substatus=df['substatus'],     response=df['response'],     labels=["C-" + str(x) for x in y_labels]     ))

# map colors
mapper = linear_cmap(field_name='desc',
palette=Category20[10],
low=min(y_labels), high=max(y_labels))

# prepare the figure
plot = figure(plot_width=1200, plot_height=850,
tools=['lasso_select', 'box_select', 'pan', 'box_zoom', 'reset', 'save', 'tap'],
title="Clustering and LDA",
toolbar_location="above")

# plot settings
plot.scatter('x', 'y', size=5,
source=source,
fill_color=mapper,
line_alpha=0.3,
line_color="black",
legend='labels')
plot.legend.background_fill_alpha = 0.6

# keywords
text_banner = Paragraph(text='Keywords: Slide to specific cluster to see the keywords.', height=45)
input_callback_1 = input_callback(plot, source, text_banner, topics)

# currently selected
div_curr = Div(text="""Click on a plot to see more information.""", height=150)
callback_selected = CustomJS(args=dict(source=source, current_selection=div_curr), code=selected_code())
taptool = plot.select(type=TapTool)
taptool.callback = callback_selected

# widgets
slider = Slider(start=0, end=10, value=10, step=1, title="Cluster #", callback=input_callback_1)

# pass call back arguments 
input_callback_1.args["slider"] = slider

Callback code:

Copy to clipboard

def input_callback(plot, source, out_text, topics):

# slider call back for cluster selection
callback = CustomJS(args=dict(p=plot, source=source, out_text=out_text, topics=topics), code="""

        var data = source.data; 
	var cluster = slider.value;
	        		        
            x = data['x'];
            y = data['y'];
            x_backup = data['x_backup'];
            y_backup = data['y_backup'];
            labels = list(set(data['desc']));
            desc = data['desc']
            message = data['message'];
            date = data['date'];
            forms = data['forms'];
            status = data['status'];
            substatus = data['substatus'];
            hac_code = data['hac_code'];
            
            if (cluster == 10) {
                    out_text.text = 'Keywords: Slide to specific cluster to see the keywords.';
                    for (i = 0; i < x.length; i++) {
                        if(desc[i] == cluster) {

@Arjun_Saravanan sorry I did not realize earlier that this was only partial code. In order to try and help you, I want to actually run real code. Usually if I can run real code I can spot an issue in very short order. What I am asking for is a Complete Minimal Reproducer that I can copy into and editor, save, and run with no modifications (e.g. is is not missing any imports)

Otherwise, all I can offer you is general advice: start smaller with something that works that you understand, and build up incrementally from there in discrete steps (so that you can know exactly what change ā€œwent wrongā€) and also to make sure to check the browser JS console for any useful errors or messages.

And thatā€™s exactly whatā€™s needed. In fact, the more different it is compared to the original code, the better it can be for the people trying to help you. Simply because a real production code is usually very different from the code that constitutes a Minimal Reproducible Example, which Bryan has asked you about in his previous post with a link to a guide on how to create such an example.

Please come up with such an example that can be run on our side without any modifications, and then we will be able to help you without spending hours upon hours trying to modify and run the incomplete code while also trying to preserve the undesired behavior based only on a description of it.

Iā€™m afraid not as it would go out of scope of a mere support for an open-source library and would be more at the level of commercial consulting.

I donā€™t think that anything can be achieved here by belaboring the point. There has to be an MRE, just as described by the link posted above.

I have constructed a MRE. Please let me know if there are any questions.

Output: A plot that contains 5 scatter plot points. The goal is for the slider object to only show the cluster that is selected, not all clusters (filtering). The slider object is defined under widgets of the ā€˜Bokeh Plotā€™ section.

import pandas as pd
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.cluster import KMeans
from sklearn.manifold import TSNE

df = pd.DataFrame(
    {'message': ['gray dog jumped over fence while other dog ate gray rabbit',
                 'Some animals have no idea what doing life yes talking about cats',
                 'roses red violets blue jasmine  white it smells tulips nice flowers not smell spring up bunches Spring',
                 'animal poem awaits elephants gray horses eat hay neigh ok dear friends poem for',
                 'flowers nice come all kinds colors must appreciate all flowers regardless status income race gender']})

def vectorize(text):
    vectorizer = TfidfVectorizer()
    X = vectorizer.fit_transform(text)
    return X

vector = vectorize(df['message'].values)
arr = vector.toarray()

k = 3
kmeans = KMeans(n_clusters=k)
y_pred = kmeans.fit_predict(arr)
df['kmean_fit'] = y_pred

tsne = TSNE()
xe = tsne.fit_transform(arr)
vis_x = xe[:, 0]
vis_y = xe[:, 1]

topics = ['first topic', 'second topic', 'third topic', 'fourth topic', 'fifth topic']

#------------------CALLBACK CODE-------------------

# handle the keywords and search
def input_callback(plot, source, out_text, topics):
    # slider call back for cluster selection
    callback = CustomJS(args=dict(p=plot, source=source, out_text=out_text, topics=topics), code="""

                var data = source.data; 
		        var cluster = slider.value;

                source = ColumnDataSource(data=dict(
                x=vis_x,
                y=vis_y,
                desc=y_labels,
                message=df['message'],
                labels=["C-" + str(x) for x in y_labels]
                
                        out_text.text = 'Keywords: Slide to specific cluster to see the keywords.';
                        for (i = 0; i < x.length; i++) {
                            if(desc[i] == cluster) {
                                    x[i] = x[i];
    							    y[i] = y[i];
    					}   else {
    							    x[i] = undefined;
    							    y[i] = undefined;
    						}
                        }
            }
        source.change.emit();
    """)
    return callback

# --------------------------BOKEH PLOT----------------------------

from call_backs import input_callback, selected_code
from bokeh.models import ColumnDataSource, HoverTool, LinearColorMapper, CustomJS, Slider, TapTool, TextInput
from bokeh.palettes import Category20
from bokeh.transform import linear_cmap, transform
from bokeh.io import output_file, show, output_notebook
from bokeh.plotting import figure
from bokeh.models import RadioButtonGroup, TextInput, Div, Paragraph
from bokeh.layouts import column, widgetbox, row, layout

# target labels
y_labels = y_pred

# data sources
source = ColumnDataSource(data=dict(
    x=vis_x,
    y=vis_y,
    desc=y_labels,
    message=df['message'],
    labels=["C-" + str(x) for x in y_labels]
))

# map colors
mapper = linear_cmap(field_name='desc',
                     palette=Category20[3],
                     low=min(y_labels), high=max(y_labels))

# prepare the figure
plot = figure(plot_width=1200, plot_height=850,
              tools=['lasso_select', 'box_select', 'pan', 'box_zoom', 'reset', 'save', 'tap'],
              title="LDA and Clustering",
              toolbar_location="above")

# plot settings
plot.scatter('x', 'y', size=50,
             source=source,
             fill_color=mapper,
             line_alpha=0.3,
             line_color="black",
             legend='labels')
plot.legend.background_fill_alpha = 0.6

# Keywords
text_banner = Paragraph(text='Keywords: Slide to specific cluster to see the keywords.', height=45)
input_callback_1 = input_callback(plot, source, text_banner, topics)

# WIDGETS
slider = Slider(start=0, end=3, value=1, step=1, title="Cluster #")

# pass call back arguments
input_callback_1.args["slider"] = slider

# STYLE
slider.sizing_mode = "stretch_width"
slider.margin = 15

text_banner.style = {'color': '#0269A4', 'font-family': 'Helvetica Neue, Helvetica, Arial, sans-serif;',
                     'font-size': '1.1em'}
text_banner.sizing_mode = "scale_both"
text_banner.margin = 20

plot.sizing_mode = "scale_both"
plot.margin = 5

r = row(text_banner)
r.sizing_mode = "stretch_width"

# LAYOUT OF THE PAGE
l = layout([
    [slider],
    [text_banner],
    [plot],
])
l.sizing_mode = "scale_both"

# show
show(l)

@Arjun_Saravanan

I donā€™t see where youā€™re actually registering the callback. Try adding this to the file

slider.js_on_change('value', input_callback_1)

Also, when I add that, it goes into the callback but fails b/c of a syntax error. The callback does work if I change it to a simple console.log(). So, youā€™ll need to debug the JS code too.

1 Like

@Arjun_Saravanan you have created a CustomJS callback but then you never actually hook it up to anything with js_on_change or js_on_event. See:

For many examples of how to use those methods to attach callbacks to properties and events.

But your callback code has bigger issues. Starting with:

source = ColumnDataSource(data=dict(...))

That is Python code, not Javascript code. They only thing a CustomJS is capable of executing is JavaScript code (because it runs in the browser). Again, the docs linked above are a good source of examples you can emulate and study.

Specifically for callbacks to do filtering, you might also search here for CDSView since updating a view with filter on the source will be simpler than all the work you are trying to do by hand.

(@p-himik it would be good to add a section to that chapter about callbacks for filtering if you know a good example to point to offhand)

1 Like

From a quick search of my answers here:

2 Likes

Appreciate the feedback as I have looked through the follow comments. This is what I have so far, although Iā€™m sure my BooleanFilter is not set up correctly. Any thoughts?

options = sorted(set(source.data['desc']))
initial_value = [options[0]]
bool = BooleanFilter([True if x in initial_value else False for x in source.data['desc']])

view = CDSView(source=source, filters=[bool])

# WIDGETS
slider = Slider(start=0, end=3, value=1, step=1, title="Cluster #")
slider.js_on_change('value', CustomJS(args=dict(b=bool, ds=source),
                                      code="""
                                      const val = cb_obj.value;
                                      f.booleans = Array.from(source.data['desc']).map(d =>  val.includes(d));
                                      source.change.emit();
                                      """))

You copy-pasted the CustomJS code without changing it in a way that matters.

  • You pass b=bool, but in the JS code youā€™re using f instead of b
  • You pass ds=source, but in the JS code youā€™re using source instead of ds
  • cb_obj.value in this case is a number since slider.value is a number, and it doesnā€™t have an includes method

Sorry about that, I have fixed the arguments, however for the callback object value, what would be the best way to use the boolean values and set up the booleanfilter accordingly?

I donā€™t know - it depends on your application logic, itā€™s up to you. true - the datum ends up being displayed, false - it ends up hidden. How you derive those boolean values is up to you.

Thank you everyone for your help. The following code runs with the expected results.

#!/usr/bin/env python
# coding: utf-8

# In[17]:


import pandas as pd
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.cluster import KMeans
from sklearn.manifold import TSNE

# dataframe with data
df = pd.DataFrame(
    {'message': ['gray dog jumped over fence while other dog ate gray rabbit',
                 'Some animals have no idea what doing life yes talking about cats',
                 'roses red violets blue jasmine  white it smells tulips nice flowers not smell spring up bunches Spring',
                 'animal poem awaits elephants gray horses eat hay neigh ok dear friends poem for',
                 'flowers nice come all kinds colors must appreciate all flowers regardless status income race gender']})


# creates a vectorization of data. We will use these vector values for plotting later.
def vectorize(text):
    vectorizer = TfidfVectorizer()
    X = vectorizer.fit_transform(text)
    return X


vector = vectorize(df['message'].values)
arr = vector.toarray()

# kmeans algorithm running on data
k = 3
kmeans = KMeans(n_clusters=k)
y_pred = kmeans.fit_predict(arr)
df['kmean_fit'] = y_pred

# tsne will plot the data on a plot
tsne = TSNE()
xe = tsne.fit_transform(arr)
vis_x = xe[:, 0]
vis_y = xe[:, 1]


# --------------------------BOKEH PLOT----------------------------

# Bokeh plot is the plotting interface that I am using for this code.
# It is necessary to install bokeh for using the plotting features

from bokeh.models.callbacks import CustomJS
from bokeh.models import ColumnDataSource, HoverTool, LinearColorMapper, CustomJS, Slider, BooleanFilter, CDSView, Legend, LegendItem
from bokeh.palettes import Category20
from bokeh.transform import linear_cmap, transform
from bokeh.io import output_file, show, output_notebook
from bokeh.plotting import figure
from bokeh.models import TextInput, Div, Paragraph
from bokeh.layouts import column, widgetbox, row, layout

# target labels
y_labels = y_pred

# data source
source = ColumnDataSource(data=dict(
    x=vis_x,
    y=vis_y,
    desc=y_labels,
    message=df['message'],
    labels=["C-" + str(x) for x in y_labels]
))

# Keywords
text_banner = Paragraph(text='Keywords: Slide to specific cluster to see the keywords.', height=45)

# options = sorted(set(source.data['desc']))

bool = BooleanFilter([True if x else False for x in source.data['desc']])


view = CDSView(source=source, filters=[bool])

# WIDGETS
slider = Slider(start=0, end=3, value=0, step=1, title="Cluster #")
slider.js_on_change('value', CustomJS(args=dict(b=bool, source=source),
                                      code="""
                                      const val = cb_obj.value; 
                                      if (val == 3) {
                                            b.booleans = Array.from(source.data['desc'], x=> x = x)
                                      }
                                      else {
                                            b.booleans = Array.from(source.data['desc'], x=> x == val);
                                      }
                                      source.change.emit()
                                      """))

# map colors
mapper = linear_cmap(field_name='desc',
                     palette=Category20[3],
                     low=min(y_labels), high=max(y_labels))

# prepare the figure
plot = figure(plot_width=1200, plot_height=850,
              tools=['lasso_select', 'box_select', 'pan', 'box_zoom', 'reset', 'save', 'tap'],
              title="LDA and Clustering",
              toolbar_location="above")

# plot settings
plot.scatter('x', 'y', size=30,
             source=source, view=view,
             fill_color=mapper,
             line_alpha=0.3,
             line_color="black",
             legend='labels',
             muted_alpha=0.2)

plot.legend.background_fill_alpha = 0.8
plot.legend.location = "top_right"
plot.legend.click_policy = "mute"

# STYLE
slider.sizing_mode = "stretch_width"
slider.margin = 15

text_banner.style = {'color': '#0269A4', 'font-family': 'Helvetica Neue, Helvetica, Arial, sans-serif;',
                     'font-size': '1.1em'}
text_banner.sizing_mode = "scale_both"
text_banner.margin = 20

plot.sizing_mode = "scale_both"
plot.margin = 5

r = row(text_banner)
r.sizing_mode = "stretch_width"

# LAYOUT OF THE PAGE
l = layout([plot],
           [slider],
           [text_banner])
l.sizing_mode = "scale_both"

# show
show(l)
1 Like

I guess now a concern would be how to come back to the original, unfiltered view of the data. Navigating to the last cluster gives this view (not the best solution).