Bokeh Stream Timeseries Data with Flask - AjaxDataSource

Hello,

I have tried my best to get time series data streaming onto a Bokeh chart rendered through Flask.
I have used http://stackoverflow.com/questions/37083998/flask-bokeh-ajaxdatasource as a base, and it works great except whenever a datetime is passed as the X axis through an AJAX request.

Like the post, I am trying to use AJAX to fetch some JSON data. My data is retrieved from an SQLlite database query. The flask jsonify does not handle datetime stamps very well so I have implemented the Flask Custom JSON Encoder from here http://flask.pocoo.org/snippets/119/

I tried manually passing in a datetime stamp and it plots fine(manual meaning not a stream). However when i tried passing a datetime as a string into the figure, it dosent work. So this to me says that Bokeh expects a datetime to be an actual datetime and not a string representation.

So as I am using JSON, I need to make sure the response is parsed to convert the X axis data from a JSON string to an actual datetime object.

I have tried going through the Bokeh code, i have read the documentation but i’ve got nowhere. I just cant figure out how to process the data before it is passed to the chart.
I assume that streaming timeseries data with flask is a pretty common thing but ive not found much online.
Here is my code minus the database stuff. I know the imports need tidying i just wanted to get it working for now.

from flask import Flask, jsonify
from flask.json import JSONEncoder
from datetime import datetime
from jinja2 import Template
from bokeh.plotting import figure
from bokeh.models import AjaxDataSource, Range1d, RemoteSource
from bokeh.embed import components
from bokeh.resources import INLINE
from bokeh.util.string import encode_utf8

class CustomJSONEncoder(JSONEncoder):

def default(self, obj):
    try:
        if isinstance(obj, datetime):
            if obj.utcoffset() is not None:
                obj = obj - obj.utcoffset()
            millis = int(
                calendar.timegm(obj.timetuple()) * 1000 +
                obj.microsecond / 1000
            )
            return millis
        iterable = iter(obj)
    except TypeError:
        pass
    else:
        return list(iterable)
    return JSONEncoder.default(self, obj)

app = Flask(name)
app.json_encoder = CustomJSONEncoder

template = Template(’’’

Streaming Example {{ js_resources }} {{ css_resources }} {{ plot_div }} {{ plot_script }} ''')

# my code

@app.route("/data", methods=[‘POST’, ‘GET’])
def get_latest_entry():
con = engine.connect()
data = con.execute(“SELECT id, data, time FROM ashley ORDER BY time DESC limit 1”)
id, data, time = data.first()
data = float(data)
# reformatted_time = datetime.datetime.strptime(time, ‘%Y-%m-%d %H:%M:%S.%f’)
json_data = jsonify(x=[time], y=[data])
# print json_data.data
con.close()
return json_data

@app.route("/")
def simple():
source = AjaxDataSource(data_url=“http://localhost:5000/data”, polling_interval=1000, mode=‘append’)
#only ever triggered at the start
source.data = dict(x=, y=)

fig = figure(title="Streaming Example", x_axis_type="datetime")
fig.line('x', 'y', source=source)


# works
# time = datetime.datetime.now()
# fig = figure(title="Streaming Example", x_axis_type="datetime")
# data =  {"x": [time, time],"y": [0.6, 1.4]}
# fig.line('x', 'y', source=data)


# dosent work
# time = str(datetime.datetime.now())
# fig = figure(title="Streaming Example", x_axis_type="datetime")
# data =  {"x": [time, time],"y": [0.6, 1.4]}
# fig.line('x', 'y', source=data)


js_resources = INLINE.render_js()
css_resources = INLINE.render_css()

script, div = components(fig, INLINE)

html = template.render(
    plot_script=script,
    plot_div=div,
    js_resources=js_resources,
    css_resources=css_resources
)
return encode_utf8(html)

app.run(debug=True)

The database is updated regularly by a seperate program. Any pointers on getting the datetime data streaming would be greatly appreciated.

Thanks,

Ashley

Hi Ashley,

Do you have any updates on this?

Thanks

···

On Friday, May 12, 2017 at 6:28:45 PM UTC+3, Ashley wrote:

Hello,

I have tried my best to get time series data streaming onto a Bokeh chart rendered through Flask.
I have used http://stackoverflow.com/questions/37083998/flask-bokeh-ajaxdatasource as a base, and it works great except whenever a datetime is passed as the X axis through an AJAX request.

Like the post, I am trying to use AJAX to fetch some JSON data. My data is retrieved from an SQLlite database query. The flask jsonify does not handle datetime stamps very well so I have implemented the Flask Custom JSON Encoder from here http://flask.pocoo.org/snippets/119/

I tried manually passing in a datetime stamp and it plots fine(manual meaning not a stream). However when i tried passing a datetime as a string into the figure, it dosent work. So this to me says that Bokeh expects a datetime to be an actual datetime and not a string representation.

So as I am using JSON, I need to make sure the response is parsed to convert the X axis data from a JSON string to an actual datetime object.

I have tried going through the Bokeh code, i have read the documentation but i’ve got nowhere. I just cant figure out how to process the data before it is passed to the chart.
I assume that streaming timeseries data with flask is a pretty common thing but ive not found much online.
Here is my code minus the database stuff. I know the imports need tidying i just wanted to get it working for now.

from flask import Flask, jsonify
from flask.json import JSONEncoder
from datetime import datetime
from jinja2 import Template
from bokeh.plotting import figure
from bokeh.models import AjaxDataSource, Range1d, RemoteSource
from bokeh.embed import components
from bokeh.resources import INLINE
from bokeh.util.string import encode_utf8

class CustomJSONEncoder(JSONEncoder):

def default(self, obj):
    try:
        if isinstance(obj, datetime):
            if obj.utcoffset() is not None:
                obj = obj - obj.utcoffset()
            millis = int(
                calendar.timegm(obj.timetuple()) * 1000 +
                obj.microsecond / 1000
            )
            return millis
        iterable = iter(obj)
    except TypeError:
        pass
    else:
        return list(iterable)
    return JSONEncoder.default(self, obj)

app = Flask(name)
app.json_encoder = CustomJSONEncoder

template = Template(’’’

Streaming Example {{ js_resources }} {{ css_resources }} {{ plot_div }} {{ plot_script }} ''')

# my code

@app.route("/data", methods=[‘POST’, ‘GET’])
def get_latest_entry():
con = engine.connect()
data = con.execute(“SELECT id, data, time FROM ashley ORDER BY time DESC limit 1”)
id, data, time = data.first()
data = float(data)
# reformatted_time = datetime.datetime.strptime(time, ‘%Y-%m-%d %H:%M:%S.%f’)
json_data = jsonify(x=[time], y=[data])
# print json_data.data
con.close()
return json_data

@app.route("/")
def simple():
source = AjaxDataSource(data_url=“http://localhost:5000/data”, polling_interval=1000, mode=‘append’)
#only ever triggered at the start
source.data = dict(x=, y=)

fig = figure(title="Streaming Example", x_axis_type="datetime")
fig.line('x', 'y', source=source)


# works
# time = datetime.datetime.now()
# fig = figure(title="Streaming Example", x_axis_type="datetime")
# data =  {"x": [time, time],"y": [0.6, 1.4]}
# fig.line('x', 'y', source=data)


# dosent work
# time = str(datetime.datetime.now())
# fig = figure(title="Streaming Example", x_axis_type="datetime")
# data =  {"x": [time, time],"y": [0.6, 1.4]}
# fig.line('x', 'y', source=data)


js_resources = INLINE.render_js()
css_resources = INLINE.render_css()

script, div = components(fig, INLINE)

html = template.render(
    plot_script=script,
    plot_div=div,
    js_resources=js_resources,
    css_resources=css_resources
)
return encode_utf8(html)

app.run(debug=True)

The database is updated regularly by a seperate program. Any pointers on getting the datetime data streaming would be greatly appreciated.

Thanks,

Ashley

Hi Ashley,

If I’m understanding correctly you have a graph with a datetime x axis and just want the plot to stream data from the ajax calls? If so, I ran into a similar problem trying to get that to work as well. Since bokeh datetime graphs actually store the ‘datetime’ values just a microseconds (float) I just converted my datetime objects to micro-seconds from the epoch and then fed in those floats into the bokeh graph and it worked.

See:

https://github.com/hhprogram/PyramidSite/blob/master/webgraphing/views/ajaxView.py

data_route() method is the method polled by the AjaxDataSource that ‘streams’ the data to the bokeh plot. Hopefully, this addresses your issue.

  • show quoted text -
···

On Friday, May 12, 2017 at 8:28:45 AM UTC-7, Ashley wrote:

Hello,

I have tried my best to get time series data streaming onto a Bokeh chart rendered through Flask.
I have used http://stackoverflow.com/questions/37083998/flask-bokeh-ajaxdatasource as a base, and it works great except whenever a datetime is passed as the X axis through an AJAX request.

Like the post, I am trying to use AJAX to fetch some JSON data. My data is retrieved from an SQLlite database query. The flask jsonify does not handle datetime stamps very well so I have implemented the Flask Custom JSON Encoder from here http://flask.pocoo.org/snippets/119/

I tried manually passing in a datetime stamp and it plots fine(manual meaning not a stream). However when i tried passing a datetime as a string into the figure, it dosent work. So this to me says that Bokeh expects a datetime to be an actual datetime and not a string representation.

So as I am using JSON, I need to make sure the response is parsed to convert the X axis data from a JSON string to an actual datetime object.

I have tried going through the Bokeh code, i have read the documentation but i’ve got nowhere. I just cant figure out how to process the data before it is passed to the chart.
I assume that streaming timeseries data with flask is a pretty common thing but ive not found much online.
Here is my code minus the database stuff. I know the imports need tidying i just wanted to get it working for now.

from flask import Flask, jsonify
from flask.json import JSONEncoder
from datetime import datetime
from jinja2 import Template
from bokeh.plotting import figure
from bokeh.models import AjaxDataSource, Range1d, RemoteSource
from bokeh.embed import components
from bokeh.resources import INLINE
from bokeh.util.string import encode_utf8

class CustomJSONEncoder(JSONEncoder):

def default(self, obj):
    try:
        if isinstance(obj, datetime):
            if obj.utcoffset() is not None:
                obj = obj - obj.utcoffset()
            millis = int(
                calendar.timegm(obj.timetuple()) * 1000 +
                obj.microsecond / 1000
            )
            return millis
        iterable = iter(obj)
    except TypeError:
        pass
    else:
        return list(iterable)
    return JSONEncoder.default(self, obj)

app = Flask(name)
app.json_encoder = CustomJSONEncoder

template = Template(’’’

Streaming Example {{ js_resources }} {{ css_resources }} {{ plot_div }} {{ plot_script }} ''')

# my code

@app.route("/data", methods=[‘POST’, ‘GET’])
def get_latest_entry():
con = engine.connect()
data = con.execute(“SELECT id, data, time FROM ashley ORDER BY time DESC limit 1”)
id, data, time = data.first()
data = float(data)
# reformatted_time = datetime.datetime.strptime(time, ‘%Y-%m-%d %H:%M:%S.%f’)
json_data = jsonify(x=[time], y=[data])
# print json_data.data
con.close()
return json_data

@app.route("/")
def simple():
source = AjaxDataSource(data_url=“http://localhost:5000/data”, polling_interval=1000, mode=‘append’)
#only ever triggered at the start
source.data = dict(x=, y=)

fig = figure(title="Streaming Example", x_axis_type="datetime")
fig.line('x', 'y', source=source)


# works
# time = datetime.datetime.now()
# fig = figure(title="Streaming Example", x_axis_type="datetime")
# data =  {"x": [time, time],"y": [0.6, 1.4]}
# fig.line('x', 'y', source=data)


# dosent work
# time = str(datetime.datetime.now())
# fig = figure(title="Streaming Example", x_axis_type="datetime")
# data =  {"x": [time, time],"y": [0.6, 1.4]}
# fig.line('x', 'y', source=data)


js_resources = INLINE.render_js()
css_resources = INLINE.render_css()

script, div = components(fig, INLINE)

html = template.render(
    plot_script=script,
    plot_div=div,
    js_resources=js_resources,
    css_resources=css_resources
)
return encode_utf8(html)

app.run(debug=True)

The database is updated regularly by a seperate program. Any pointers on getting the datetime data streaming would be greatly appreciated.

Thanks,

Ashley