Hi Bokeh-Community,
I have a question regarding streaming of data in bokeh with python. It’s complicated so I will first just write down some important notes:
- I want to display real-time sensor data with python and bokeh.
- There will be a maximum of 8 new points per second.
- Every time I add a new point I want to adapt the start and the end of the x_range of the figure.
- It would be nice if I could commit every new point (x- and y-value) in a single step (self._bokeh_datasource.stream(new_data) → new_data should be just one point in the graph).
- I want to illustrate the data in a line graph.
- The graph should be able to illustrate up to 50000 points.
I have already realised a working solution but there are some problems. I have started a bokeh server which runs in the background (with the implementation of the figure and line). In another program I have pulled the session of the server and streamed new data into the ColumnDataSource of the bokeh server application. After streaming new data into the ColumnDataSource I also adjust the x_range (start and end). Now there are two problems:
- If I add one new point per second my method works fine. It shows always the newest data from the sensor. The only problem is that this realisation needs a lot of RAM. If I add 6000 points it uses something like 200 MB of the RAM. I don’t think that the application really needs so much memory for just 6000 Points. Do you know where the problem is? This application has to run on a single-board-computer so it shouldn’t need that much memory.
- How should I realise the livelog with up to 8 points per second? It would be very nice if I could add every single point within this second. One thing to think of is that I also want to adjust the x_range of the figure. This means that there will be 3 changes for the webserver per new point.
Here is an example application. First start bokeh_plot.py with
bokeh serve --show bokeh_plot.py --allow-websocket-origin=localhost --allow-websocket-origin=localhost:5006
# bokeh_plot.py:
from bokeh.plotting import figure, curdoc
from bokeh.models import ColumnDataSource, Range1d
datasource = ColumnDataSource(data=dict(x_achse=[ ], y_achse=[ ]))
plot = figure(plot_height=288, plot_width=577, tools="xpan", toolbar_location=None, x_range=Range1d(bounds=(0, None)))
plot.line(x='x_axis', y='y_axis', source=datasource, line_color=None, line_width=2, name="sam_line")
curdoc().add_root(plot)
After that you can start bokeh_program.py
# bokeh_program.py:
from bokeh.client import pull_session
import time
import random
bokeh_session = pull_session(url="http://localhost:5006/bokeh_plot", session_id="foo")
bokeh_session_document = bokeh_session.document
bokeh_graph = bokeh_session_document.get_model_by_name("sam_line")
bokeh_graph_datasource = bokeh_graph.data_source
bokeh_plot_figure = bokeh_session_document.roots[0]
x_value = 0
while True is True:
y_value = random.random()
new_data = {
'x_axis': [x_value],
'y_axis': [y_value],
}
bokeh_graph_datasource.stream(new_data)
if x_value < 100:
bokeh_plot_figure.x_range.start = 0
else:
bokeh_plot_figure.x_range.start = x_value - 100
bokeh_plot_figure.x_range.end = x_value
x_value += 1
time.sleep(0.03)
(The sleep time is low so that it starts to lag faster)
You can see the plot with a webbrowser on http://localhost:5006/bokeh_plot?bokeh-session-id=foo
This example simulates my application. The used RAM of bokeh_program.py increases and the application starts to lag after some minutes.
Summary:
Question 1: Why is the used RAM increasing so fast?
Question 2: Is this the right way to implement my use case?
Thanks for your help in advance.
With best regards
Manuel