Hi,
I have a large amount of data (4GB) that need to be plotted with Bokeh. I don’t need to plot all data at once. Rather, I can plot a small portion of data and can use a scroll bar to adjust the viewing window. Is this possible with Bokeh? I am really new with Bokeh so please bear with me if the question has been mentioned before.
Another related question is do I need to load all the data to memory before plotting? Can Bokeh load only the part being plotted? I think this might help reduce the memory consumption and provide some speedup overall. Thanks a lot!
I would recommend aggregating or summarizing the data before trying to
visualize it. That sounds like too much data to visually just throw
into a chart or figure and have it be meaningful. Pandas and Numpy are
very useful for doing this sort of thing with Bokeh visualizations.
Also, oerhaps check out datashader also:
"""
Datashader is a graphics pipeline system for creating meaningful
representations of large amounts of data.
"""
···
On Wed, Jan 11, 2017 at 3:05 PM, <[email protected]> wrote:
Hi,
I have a large amount of data (4GB) that need to be plotted with Bokeh. I
don't need to plot all data at once. Rather, I can plot a small portion of
data and can use a scroll bar to adjust the viewing window. Is this possible
with Bokeh? I am really new with Bokeh so please bear with me if the
question has been mentioned before.
Another related question is do I need to load all the data to memory before
plotting? Can Bokeh load only the part being plotted? I think this might
help reduce the memory consumption and provide some speedup overall. Thanks
a lot!
--
Alex G Rice
[email protected]
[email protected]
+1 505-603-5639
Datashader would be very useful for plotting 4GB of data, particularly if there are some meaningful ways to aggregate it. Datashader also works well with dask to support out-of-core datasets; see e.g. the OSM datashader example. But note that currently datashader does not include any special support for partitioning your data, and thus it needs to make a complete pass through the dataset each time it needs to display something. Adding support for some form of partitioning (quadtrees, etc.) is on the roadmap, but does not currently have any funding, and so it will depend on getting some form of external support (or code contributions). We’re looking into options now for obtaining such funding, but don’t have anything to announce yet.
···
On Sun, Jan 15, 2017 at 4:41 PM, Alex Rice [email protected] wrote:
On Wed, Jan 11, 2017 at 3:05 PM, [email protected] wrote:
Hi,
I have a large amount of data (4GB) that need to be plotted with Bokeh. I
don’t need to plot all data at once. Rather, I can plot a small portion of
data and can use a scroll bar to adjust the viewing window. Is this possible
with Bokeh? I am really new with Bokeh so please bear with me if the
question has been mentioned before.
Another related question is do I need to load all the data to memory before
plotting? Can Bokeh load only the part being plotted? I think this might
help reduce the memory consumption and provide some speedup overall. Thanks
a lot!
I would recommend aggregating or summarizing the data before trying to
visualize it. That sounds like too much data to visually just throw
into a chart or figure and have it be meaningful. Pandas and Numpy are
very useful for doing this sort of thing with Bokeh visualizations.
Also, oerhaps check out datashader also:
“”"
Datashader is a graphics pipeline system for creating meaningful
representations of large amounts of data.
“”"
https://github.com/bokeh/datashader
–
Alex G Rice
[email protected]
[email protected]
+1 505-603-5639
–
You received this message because you are subscribed to the Google Groups “Bokeh Discussion - Public” group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/CACmK6BuRUR626BWj%2BZL_cU_xCKnvonDWYEAk7kJR3JQ%3DNQjFkQ%40mail.gmail.com.
For more options, visit https://groups.google.com/a/continuum.io/d/optout.
Jim