How to handle querying multiple databases that sometimes return nonetypes -- Updating From Threads

Community Support
1

Hi Bokeh Users,

I’ve implemented a great bokeh chart (thanks Bryan VDV for the quick help on the recent fill scatter plot question). I’m using the “updating from threads” bokeh example. The problem I’m running into is querying multiple databases and sometimes returning a nonetype-- the script fails when I receive nonetype. The reason I would receive a Nonetype is because I check every 10 seconds and sometimes there are updates during this 10 second interval.

Here’s an example of my plot output to get a better understanding of how I’m visualizing the data:

I’m going to write out some pseudo code and hope that someone wouldn’t mind informing me of python/bokeh best practices when querying multiple databases that sometimes return no results. I’m modifying the script from Running a Bokeh server — Bokeh 2.4.2 Documentation

… more applicable code above, see link if you want to know …

link: Running a Bokeh server — Bokeh 2.4.2 Documentation

def qry_from_db1():

sql query to return data from db1

return d1_a, d1_b

def qry_from_db2():

sql query to return data from db2

pass

def qry_from_db3():

sql query to return data from db3

pass

source = ColumnDataSource(data=dict(data_db1_a=data_db1_a, data_db1_b=data_db1_b,data_db2=data_db2, data_db3=data_db3))

This is important! Save curdoc() to make sure all threads see then same document.

doc = curdoc()

@gen.coroutine
def update(data_db1, data_db2, data_db3):
source.stream(dict(data_db1_a=[data_db1_a], data_db1_b=[data_db1_b], data_db2=[data_db2], data_db3=[data_db3]))

def blocking_task():
while True:

do some blocking computation

time.sleep(10)

HERE IS WHERE I NEED HELP WITH BEST PRACTICES AROUND

QUERYING MULTIPLE DATABASES AND OCCASSIONALLY 1 OF

THE 3 RETURNS NONE AND I NEED TO HANDLE IT BETTER

data_db1_a, data_db1_b = qry_from_db1()
data_db2 = qry_from_db2()
data_db3 = qry_from_db3()

but update the document from callback

doc.add_next_tick_callback(partial(update, data_db1_a=data_db1_a, data_db1_b=data_db1_b, data_db2=data_db2, data_db3=data_db3))

p = figure()
l = p.circle(x=‘data_db1_a’, y=‘data_db1_b’, source=source)

more data plotted…

doc.add_root(p)

thread = Thread(target=blocking_task)

``

``

Thanks for your time,


Pete
2

After some more digging, it seems like someone else has asked a similar question but with no final solution:
https://groups.google.com/a/continuum.io/forum/#!searchin/bokeh/query/bokeh/PT-HEL-ad1k/QmZltD1kAAAJ

This post and the other two posts referenced deal with streaming data to multiple documents simultaneously. I believe my request is slightly different in that I’m streaming multiple data sources to one document.

···

On Wednesday, July 5, 2017 at 8:04:16 AM UTC-5, Peter West wrote:

Hi Bokeh Users,

I’ve implemented a great bokeh chart (thanks Bryan VDV for the quick help on the recent fill scatter plot question). I’m using the “updating from threads” bokeh example. The problem I’m running into is querying multiple databases and sometimes returning a nonetype-- the script fails when I receive nonetype. The reason I would receive a Nonetype is because I check every 10 seconds and sometimes there are updates during this 10 second interval.

Here’s an example of my plot output to get a better understanding of how I’m visualizing the data:

I’m going to write out some pseudo code and hope that someone wouldn’t mind informing me of python/bokeh best practices when querying multiple databases that sometimes return no results. I’m modifying the script from http://bokeh.pydata.org/en/latest/docs/user_guide/server.html#updating-from-threads:

… more applicable code above, see link if you want to know …

link: http://bokeh.pydata.org/en/latest/docs/user_guide/server.html#updating-from-threads

def qry_from_db1():

sql query to return data from db1

return d1_a, d1_b

def qry_from_db2():

sql query to return data from db2

pass

def qry_from_db3():

sql query to return data from db3

pass

source = ColumnDataSource(data=dict(data_db1_a=data_db1_a, data_db1_b=data_db1_b,data_db2=data_db2, data_db3=data_db3))

This is important! Save curdoc() to make sure all threads see then same document.

doc = curdoc()

@gen.coroutine
def update(data_db1, data_db2, data_db3):
source.stream(dict(data_db1_a=[data_db1_a], data_db1_b=[data_db1_b], data_db2=[data_db2], data_db3=[data_db3]))

def blocking_task():
while True:

do some blocking computation

time.sleep(10)

HERE IS WHERE I NEED HELP WITH BEST PRACTICES AROUND

QUERYING MULTIPLE DATABASES AND OCCASSIONALLY 1 OF

THE 3 RETURNS NONE AND I NEED TO HANDLE IT BETTER

data_db1_a, data_db1_b = qry_from_db1()
data_db2 = qry_from_db2()
data_db3 = qry_from_db3()

but update the document from callback

doc.add_next_tick_callback(partial(update, data_db1_a=data_db1_a, data_db1_b=data_db1_b, data_db2=data_db2, data_db3=data_db3))

p = figure()
l = p.circle(x=‘data_db1_a’, y=‘data_db1_b’, source=source)

more data plotted…

doc.add_root(p)

thread = Thread(target=blocking_task)

``

``

Thanks for your time,


Pete
3

Here is a similar question from 2015 with old code that I believe no longer applies… “Multiple Live Streaming Data in a Single Figure”:
https://groups.google.com/a/continuum.io/forum/#!searchin/bokeh/query/bokeh/cjdrKNi0_OA/5_7aVjenEgAJ

···

On Wednesday, July 5, 2017 at 8:38:44 AM UTC-5, Peter West wrote:

After some more digging, it seems like someone else has asked a similar question but with no final solution:
https://groups.google.com/a/continuum.io/forum/#!searchin/bokeh/query/bokeh/PT-HEL-ad1k/QmZltD1kAAAJ

This post and the other two posts referenced deal with streaming data to multiple documents simultaneously. I believe my request is slightly different in that I’m streaming multiple data sources to one document.

On Wednesday, July 5, 2017 at 8:04:16 AM UTC-5, Peter West wrote:

Hi Bokeh Users,

I’ve implemented a great bokeh chart (thanks Bryan VDV for the quick help on the recent fill scatter plot question). I’m using the “updating from threads” bokeh example. The problem I’m running into is querying multiple databases and sometimes returning a nonetype-- the script fails when I receive nonetype. The reason I would receive a Nonetype is because I check every 10 seconds and sometimes there are updates during this 10 second interval.

Here’s an example of my plot output to get a better understanding of how I’m visualizing the data:

I’m going to write out some pseudo code and hope that someone wouldn’t mind informing me of python/bokeh best practices when querying multiple databases that sometimes return no results. I’m modifying the script from http://bokeh.pydata.org/en/latest/docs/user_guide/server.html#updating-from-threads:

… more applicable code above, see link if you want to know …

link: http://bokeh.pydata.org/en/latest/docs/user_guide/server.html#updating-from-threads

def qry_from_db1():

sql query to return data from db1

return d1_a, d1_b

def qry_from_db2():

sql query to return data from db2

pass

def qry_from_db3():

sql query to return data from db3

pass

source = ColumnDataSource(data=dict(data_db1_a=data_db1_a, data_db1_b=data_db1_b,data_db2=data_db2, data_db3=data_db3))

This is important! Save curdoc() to make sure all threads see then same document.

doc = curdoc()

@gen.coroutine
def update(data_db1, data_db2, data_db3):
source.stream(dict(data_db1_a=[data_db1_a], data_db1_b=[data_db1_b], data_db2=[data_db2], data_db3=[data_db3]))

def blocking_task():
while True:

do some blocking computation

time.sleep(10)

HERE IS WHERE I NEED HELP WITH BEST PRACTICES AROUND

QUERYING MULTIPLE DATABASES AND OCCASSIONALLY 1 OF

THE 3 RETURNS NONE AND I NEED TO HANDLE IT BETTER

data_db1_a, data_db1_b = qry_from_db1()
data_db2 = qry_from_db2()
data_db3 = qry_from_db3()

but update the document from callback

doc.add_next_tick_callback(partial(update, data_db1_a=data_db1_a, data_db1_b=data_db1_b, data_db2=data_db2, data_db3=data_db3))

p = figure()
l = p.circle(x=‘data_db1_a’, y=‘data_db1_b’, source=source)

more data plotted…

doc.add_root(p)

thread = Thread(target=blocking_task)

``

``

Thanks for your time,


Pete