Slider not updating plots (without Custom JS) Help with implementing Bokeh App or Server to make this code work

I need help with setting up bokeh server to update my plots with sliders interaction. I understand from https://stackoverflow.com/questions/37534959/bokeh-sliders-associated-with-a-plot-without-bokeh-server/56882151#56882151 this post that when we use Attribute.on_change(‘value’,function) which is a realPython code, it does work with html to have interactive data visualization.

I need to make the plot interactive and want it to update by moving of sliders. “Hoping that I have defined the updating function callback correctly, it should work if the bokeh server is setup the right way.
Please find Function call back under sliders where I have attempted to use slider values to update my data frame using rangeslider values” All i want to do here is use the slider values and filter the data frames and use these data frames to update the colum data source and thus update the plots

There is an option of custom JavaScript but I don’t know how to write a custom JavaScript, nor do I know how to set up a bokeh server. As I have really large scale data, I feel it is better to use bokeh server (webgl) and I gave it my attempt and it is too advanced for me to understand how to implement it. I need help with slider update, drop down menu and color update. Here is my code using Bokeh server.

I have written the code on latest Python 3.7 and latest Bokeh 1.2.0 and latest Pandas on Spyder IDE. I tried doing custom JS since I don’t really know programming on JavaScript, I would rather avoid it completely.

import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
######
import tkinter as tk
from tkinter import filedialog
import pandas as pd
from pandas import DataFrame
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
from bokeh.models import HoverTool
from bokeh.themes import built_in_themes
##
# Bokeh libraries
from bokeh.io import curdoc 
from bokeh.layouts import row, column
from bokeh.models import CustomJS, Slider
from bokeh.plotting import figure, output_file, show, ColumnDataSource
from bokeh.plotting import figure, show
from bokeh.layouts import row, column, gridplot
from bokeh.models.widgets import Tabs, Panel
from bokeh.core.properties import Float, Instance, Tuple, Bool, Enum
from bokeh.models import InputWidget
from bokeh.models.callbacks import Callback
from bokeh.core.enums import SliderCallbackPolicy
from bokeh.layouts import column
from bokeh.models import Slider, CustomJS, ColumnDataSource , Select
from bokeh.io import show
from bokeh.plotting import Figure
from bokeh.io import output_file, show
from bokeh.models.widgets import RangeSlider
from tornado.ioloop import IOLoop
from bokeh.application.handlers import FunctionHandler
from bokeh.application import Application
from bokeh.models import ColumnDataSource
from bokeh.plotting import figure
from bokeh.server.server import Server
doc = curdoc()
curdoc().theme = 'caliber'
#Import excel file
Data = pd.read_excel (r'C:\Users\vsrigiri\Desktop\app\try.xlsx') 
## Define data frames
# use the below code for extracting imp parameters from large excel file
df = pd.DataFrame (Data, columns= ['No.','a','z_1', 'z_2','u'])
#df = pd.DataFrame(Data) to import all data

########later you may give legends to the required lables for now iterate over the df Dataframe
legends= {'parameter_names':  ['XX','YY (in mm)', 'ZZ','MM', 'NN']}
symbols = {'symbols' :        ['No.','a','z_1', 'z_2','u']}
df_leg = pd.DataFrame(legends, columns= ['parameter_names'])
df_sym = pd.DataFrame(symbols,columns=['symbols'])
#convert Dataframe to columnDatasource
source = ColumnDataSource(df)

#Slider Definition

a_Slider= RangeSlider(start=min(df['a']), end=max(df['a']), value=(min(df['a']),max(df['a'])), step=1, title='YY in mm')
z_1_Slider = RangeSlider(start=min(df['z_1']), end=max(df['z_1']), value=(min(df['z_1']),max(df['z_1'])), step=1, title='ZZ')
z_2_Slider = RangeSlider(start=min(df['z_2']), end=max(df['z_2']), value=(min(df['z_2']),max(df['z_2'])), step=1, title='MM')
u_Slider = RangeSlider(start=min(df['u']), end=max(df['u']), value=(min(df['u']),max(df['u'])), step=.01, title='NN')
#function to call back
#to update graphs as slider updates
def callback(attr, old, new): 
    # Value for the range slider is a tuple (start, end)
    # Read the current value of the slider: scale    """
    a_s = a_Slider.value[0]
    a_e = a_Slider.value[1]
    z_1_s= z_1_Slider.value[0]
    z_1_e= z_1_Slider.value[1]
    z_2_s= z_2_Slider.value[0]
    z_2_e= z_2_Slider.value[1]
    u_s= u_Slider.value[0]
    u_e= u_Slider.value[1]
    #problematic part has been updating the data frame first sort this and next move on to linking this to the plot and updating the plot
    ##Create New Datasource from which dolumn datasource can be developed
    #update the slider range
    #finding the indicies which do not fall under selected range in slider
    dff= pd.DataFrame(df[(df.a >= a_s) & (df.a <= a_e) & (df.z_1 >= z_1_s) & (df.z_1 <= z_1_e) & (df.z_2 >= z_2_s) & (df.z_2 <= z_2_e)& (df.u >= u_s)& (df.u <= u_e)])
    source.data = ColumnDataSource.from_df(dff)
#function ends here

# Updating graphs from slider values
#updating all the slider values 
#a_slider is an object and i have created it before and i have to update this object    
a_Slider.on_change("value",callback)
z_1_Slider.on_change("value",callback)
z_2_Slider.on_change("value",callback)
u_Slider.on_change("value",callback)
#lEARN TO ITERATE OVER THE DATA FRAMES AND ASSIGN VALUES FOR TOOLTIPS ATTRIBUTES
tooltips = [("Index", "$index"),
            ("XX", "@No."),
            ('YY in mm' ,'@a'),
            ('ZZ','@z_1'),
            ('MM','@z_2'),
            ('NN','@u')]
hover = HoverTool(tooltips=tooltips)
#Plotting
#Learn to make drop down with a loop for graphs
#Define the tools Needed
toollist = ['lasso_select', 'tap', 'reset', 'save','crosshair','wheel_zoom','pan','tap','hover','box_select']
plot = figure(title="FOS ag Pvs FOS ag RBG", plot_height=400, plot_width=400,
              tools=toollist,x_range=[min(df['a']),max(df['a'])], y_range=[min(df['z_1']),max(df['z_1'])])
sctr = figure(title="FOS ag P vs FOS ag RBP", plot_height=400, plot_width=400,
              tools=toollist,x_range=[min(df['a']),max(df['a'])], y_range=[min(df['z_2']),max(df['z_2'])])
comp = figure(title="Load Distribution comparision", plot_height=400, plot_width=400,
              tools=toollist,x_range=[min(df['a']),max(df['a'])], y_range=[min(df['u']),max(df['u'])])

plot.scatter(x='a', y='z_1',source=source.data)
sctr.scatter(x='a', y='z_2',source=source.data)
comp.scatter(x='a', y='u',source=source.data)
##LEARN TO GIVE THE LABLES iterating over DATAFRAMES dfleg and dfsym
plot.xaxis.axis_label = 'FOS ag PG'
plot.yaxis.axis_label = 'FOS ag RBG'

sctr.xaxis.axis_label = 'FOS ag PP'
sctr.yaxis.axis_label = 'FOS ag RBP'

comp.xaxis.axis_label = 'FOS ag RpP'
comp.yaxis.axis_label = 'FOS ag RBg'

# Add the HoverTool to the figure
plot.add_tools(HoverTool(tooltips=tooltips))
sctr.add_tools(HoverTool(tooltips=tooltips))
comp.add_tools(HoverTool(tooltips=tooltips))
#
Graphs = row([plot,sctr,comp])
Controls1= column([a_Slider,z_1_Slider])
Controls2= column([z_2_Slider,u_Slider])
#layout= row(Controls1, Controls2,Graphs)
grid = gridplot([[Graphs], 
                [Controls1,Controls2]])

    #This is important! Save curdoc() to make sure all threads
    curdoc().add_root(grid)
    show(grid)
    curdoc().title = "codeinprocess"

Basically I have about 60 such parameters and about 150 thousand combinations of them. I am trying to fix up some 15 to 20 sliders and trying to play with them to see which of these combinations fall into my criteria based on some engineering filters.

Initially, I am trying to filter data based on some engineering conventions and later I am trying to visualize the filtered data and select the best possible combinations which will give me an idea of what range each of 15 parameters should fall in for optimum design

I am expecting the sliders to work with bokeh server first So a step by step guide would help.

on a second priority if you could guide me with template to create and update plots with drop down select options and color coding based on the same filtering criteria to understand which side of the graph has favorable points.

Are you running this using the bokeh command? I.e.

bokeh serve myapp.py

Bokeh server apps are executed by the bokeh serve command, not the “regular” way with python myapp.py. This is analogous to the way Jupyter notebooks are executed by the juptyer notebook command.

Also, I am not really sure what you expect, but I cannot imagine any circumstance in which starting a Tk event loop inside a Bokeh app would ever make any sense. Bokeh apps are web apps that display in web browsers (and only in web browsers). Tk widgets cannot display in browsers at all.

1 Like

Thanks a lot Bryan for your reponse
i have been reading a lot of stuff on git hub community support to come to this stage from absolute zero in the span of 2 weeks thanks to you guys
Well i now understand that i should not use tk to initiate the bokeh app,

  1. what do you suggest me to do to import the excel file with similar functionality??
  2. have i written the Update function right?? updating the columndatasource from the filtered dataframe values using slider values?? and how do i update the plots?
  3. i tried ‘bokeh serve myapp.py’ : and it says “Path for Bokeh server application does not exist: C:\myapp.py” i get this error
    bokeh serve --show myapp.py : and it says " ‘serve’ is not recognized as an internal or external command, operable program or batch file."
    i replaced the word myapp with my file name and i still get the same error
    How exactly do i set up the server? tutorials arent there to really understand what i am doing with the bokeh server part
    all i understand is bokeh is a web app and needs data exchange thru a document created and modified by the server as explained on the web page, i really dont understand how to execute it
    I now understand that i should not run it as a normal python script in my python console with run button, instead i have to open command prompt and use bokeh serve or bokeh show commands both wont work for me

This is what is happening when i set up the python bokeh server using python - m bokeh serve which did show server details
andwhen i tried this i got an error

(base) C:>bokeh serve --codeinprocess.py
usage: C:\ProgramData\Anaconda3\Scripts\bokeh [-h] [-v]
{html,info,json,png,sampledata,secret,serve,static,svg}

C:\ProgramData\Anaconda3\Scripts\bokeh: error: unrecognized arguments: --codeinprocess.py

“”” :bokeh.embed.util:

You are generating standalone HTML/JS output, but trying to use real Python

callbacks (i.e. with on_change or on_event). This combination cannot work.

Only JavaScript callbacks may be used with standalone output. For more

information on JavaScript callbacks with Bokeh, see:

http://bokeh.pydata.org/en/latest/docs/user_guide/interaction/callbacks.html

Alternatively, to use real Python callbacks, a Bokeh server application may

be used. For more information on building and running Bokeh applications, see:

http://bokeh.pydata.org/en/latest/docs/user_guide/server.html “””

I am getting this error

I don’t have a suggestion for that, that is beyond the scope of this forum. There are, I believe, a few different Python libraries for dealing with excel files, but I am not personally familiar with any of them.

  1. have i written the Update function right?? updating the columndatasource from the filtered dataframe values using slider values?? and how do i update the plots?

I can only make general comments based on a quick glance. At a minimum, you should change this:

source.data = newsoource.data  # BAD

The .data attribute of one CDS should only be set from a plain Python dict. The CDS .data is a complicated structure that handles all the automatic synchronization from Python to the browser. You should not “move” one from one CDS to another. Attempting to do this will raise an error in Bokeh 2.0 later this year. More typically, with dataframes you could do:

source.data = ColumnDataSource.from_df(new_df)  # GOOD

" ‘serve’ is not recognized as an internal or external command, operable program or batch file."

I think this is a windows packaging issue, I guess. You can also alwayts invoke the server on any platform using Python’s -m option instead:

python -m bokeh serve --show <your app module here>

error: unrecognized arguments: --codeinprocess.py

The Bokeh server program has no command line argument --codeinprocess.py. Perhaps you meant

bokeh serve --show codeinprocess.py

You are generating standalone HTML/JS output, but trying to use real Python

This means you are trying to run e.g. python codeinprocess.py instead of using bokeh serve. Real python callbacks, i.e. with on_change can only function in a Bokeh server (the Bokeh server is the persistent Python process that actually runs the callback code you want to run)

@Bryan thankd a lot for your response i successfully ranu this code as a bokeh app n ran some example files which run perfectly fine but after making suggested changes to the code, like rempving the thinker import of excel file by directly importing excel file using panndas and alsochanging, to source.data = ColumnDataSource.from_df(dff) i still wasnt able to get the slider work
any comments ?? need help with the code now and i did set uå the bokeh server and its orking fine i was able run the code as bokeh app

Not without more information: descriptions of expected vs actual behavior, screenshots, python or JS error messagss, updated actual code samples… just stating “it doesn’t work” does not provide anything to work with to try to help.

@ bryan, I have updated the code in the main question, However i am writing them again on the updates here
I have removed the module of importing excel sheet from thinker and i am importing them directly from the path.
Every thing is working fine and sliders update the graph now but if we go with higher No of data points
like 150 thousand rows and 150 collumns it is taking about 20 mins to plot the initial graph and if i pull a slider then, HTML page shows the following error
Uncaught Error: not connected so cannot send [object Object]
at t.send (connection.js:108)
at t._document_changed (session.js:67)
at _document_listener (session.js:11)
at P._trigger_on_change (document.js:252)
at P._notify_change (document.js:264)
at t.c._tell_document_about_change (has_props.js:439)
at t.c.setv (has_props.js:243)
at t.set [as value] (has_props.js:91)
at e.t._slide (abstract_slider.js:146)
at Object. (abstract_slider.js:69)
connection.js:108 Uncaught Error: not connected so cannot send [object Object]
at t.send (connection.js:108)
at t._document_changed (session.js:67)
at _document_listener (session.js:11)
at P._trigger_on_change (document.js:252)
at P._notify_change (document.js:264)
at t.c._tell_document_about_change (has_props.js:439)
at t.c.setv (has_props.js:243)
at t.set [as value] (has_props.js:91)
at e.t._change (abstract_slider.js:157)
at Object. (abstract_slider.js:70)
connection.js:108 Uncaught Error: not connected so cannot send [object Object]
at t.send (connection.js:108)
at t._document_changed (session.js:67)
at _document_listener (session.js:11)
at P._trigger_on_change (document.js:252)
at P._notify_change (document.js:264)
at t.c._tell_document_about_change (has_props.js:439)
at t.c.setv (has_props.js:243)
at t._layout (plot_canvas.js:616)
at e.t._layout (layout_dom.js:190)
at e.t._layout (layout_dom.js:190)

and console shows the following :slight_smile:(base) C:>CD C:\Users\vsrigiri\Desktop

(base) C:\Users\vsrigiri\Desktop>bokeh serve app
2019-07-08 12:48:01,972 Starting Bokeh server version 1.0.4 (running on Tornado 6.0.2)
2019-07-08 12:48:01,988 Bokeh app running at: http://localhost:5006/app
2019-07-08 12:48:01,988 Starting Bokeh server with process id: 8424
2019-07-08 12:56:19,322 200 GET /app (::1) 482827.34ms
2019-07-08 13:04:41,533 200 GET /app (::1) 502210.08ms
2019-07-08 13:04:42,708 101 GET /app/ws?bokeh-protocol-version=1.0&bokeh-session-id=SG4jh70bvGgeTRbi0bh28davFyCfqXffJqSEezEdJV2K (::1) 0.00ms
2019-07-08 13:04:42,723 WebSocket connection opened
2019-07-08 13:04:42,724 ServerConnection created
2019-07-08 13:05:54,907 WebSocket connection closed: code=None, reason=None
the process had surprisingly terminated

In the cases that aren’t woking like you would like, how big is the data generated here (how many rows and columns), and how long does it take to run on its own (i,e. outside of a Bokeh server app):

pd.DataFrame(df[(df.a >= a_s) & (df.a <= a_e) & (df.z_1 >= z_1_s) & (df.z_1 <= z_1_e) & (df.z_2 >= z_2_s) & (df.z_2 <= z_2_e)& (df.u >= u_s)& (df.u <= u_e)])

For that matter, how big is the original df DataFrame? Every clause up there, e.g. (df.a >= a_s) will generate a temporary data structure, and you have alot of them.

If that line take a long time to run on its own, Bokeh cannot make it somehow take less time than it takes. And if that line generates a gigabyte of data, then the next line would be attempting to send a gigabyte over the network to the browser, which is probably not going to work well (time to look at things like Datashader)

Similar comments apply to the startup computations. My best advice at this stage is to do some simple “print statement debugging” to try and determine where your code is getting hung up. I can’t do that for you since I don’t have the data needed to run your code and reproduce what you are seeing.

@ Bryan Thanks for your response
The excel sheet i am talking about is of the order of 100 thousand rows and 150 collumns i read and i am making a data frame of same 100000 rows and 15 columns the line may depend upon the range selected in the slider
fully open range may have same number as mentioned above.
data shader needs holoviews but i dont have any intensions of using holoviews here
Yeah i did try that and debug the code and it did work but when i tried to pull the slider every thing froze and stopped working

Thanks a lot for your help

data shader needs holoviews

This is not technically true. Holoviews makes it easier to use Datashader and Bokeh together, but Datashader is a standalone library that can be used on its own

I would suggest profiling the pandas operations you are doing in a Jupyter Notebook, or IPython, etc. (or more precisely, not inside the Bokeh app) to get some idea of how expensive they are on their own. And also try to characterize how much data you are trying to send on a typical update.

the process had surprisingly terminated

Do you mean the Bokeh server process terminated? Or? if so it looks like something in your callback is crashing Python. As I mentioned AFAIK the code above potentially created several large temporary objects. Maybe you are running into a memory issue? You are reading the entire spreadsheet in to memory in one go with read_csv that alone is quite large.

i did try it on the spyder it took me about 6 to 8 minutes to plot the whole graph
If you have any suggestions on improving the times then they are welcome.
I shall check on how to implement holoviews
yeah i am reading entire spreadsheet using read excel and picking the columns i need

I ll try to implement data shader without holoviews

I’m sorry if I am not communicating this clearly. I am suggesting to time the pandas operations without any Bokeh usage all. Investigate the pandas times, in a notebook or ipython, withtout ever importing anything from Bokeh. I am trying to get you to investigate how long the pandas operations take, completely indepedently of Bokeh. I.e., if just loading the file takes 6 minutes by itself, in a Jupyter notebook, there is nothing Bokeh can do to make that any faster. Or maybe the slowness is the network transfer. Or maybe it is the render. The only way to find out is to break the problem apart and investigate pieces independently, to try and ascertain where the issue actually lies.

yeah I get your point
you are talking about how long it takes to populate the dataframe df in the program, i understand you want to attack particular part problem
I will give you exact numbers running the program in parts

1 Like

@Bryan, I dont know If it is relevant to this question but
i am not able to visualize this data interactively however i thought it would be easier to see visualize the data with Boxplots
but bokeh.plotting.boxplot dosent seem to work
do you have any suggestions of making box plots interactive with pandas dataframes
i dont have any code that can describe this coz there is no library i found but i want interactive box plots like the scatter plots shown

@VITTAL_S_S please post separate individual topics for unrelated questions