I have 10 Tbytes of uint8 of radio telescope data that I would like to browse. I can load 10GBytes at a time.
bokeh.plotting.line fails if the number of points I indicate is more than 64K (2^16). Is this a problem for the server? Here is my code. 10GBytes is already loaded
···
import numpy as np
from bokeh.plotting import figure, output_file, show, output_notebook
bDataLength = 2**15
bDataStart = 0
x = np.arange(bDataLength)
y = telescopekData[bDataStart : bDataStart + bDataLength]
output_notebook()
p = figure(title=“summed aligned”, x_axis_label=‘x’, y_axis_label=‘y’)
The server uses all the same plotting code. The only thing I can think of it might add is that you might be able to write code to dynamically sub sample the data based on the current axis range or something along those lines.
The server would probably break if you try to send 10tb to the client. But if you arrange to subset the data and only send what’s needed on demand to the client, it could work.
I have 10 Tbytes of uint8 of radio telescope data that I would like to browse. I can load 10GBytes at a time.
bokeh.plotting.line fails if the number of points I indicate is more than 64K (2^16). Is this a problem for the server? Here is my code. 10GBytes is already loaded
import numpy as np
from bokeh.plotting import figure, output_file, show, output_notebook
bDataLength = 2**15
bDataStart = 0
x = np.arange(bDataLength)
y = telescopekData[bDataStart : bDataStart + bDataLength]
output_notebook()
p = figure(title=“summed aligned”, x_axis_label=‘x’, y_axis_label=‘y’)
p.line(x, y, line_width=2)
show(p)
–
You received this message because you are subscribed to the Google Groups “Bokeh Discussion - Public” group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
You'd never want to to send anything like 10gb into the browser. Pretty sure any browser would fall over. I'll mention a few things. For directly sending larger than 64k points, there is partial WebGL support in Bokeh:
With this you can reasonably send up to a few 100s of thousands of points. But please note that bowser support for WebGL is inconsistent.
For data your size, you absolutely have to consider downsampling techniques. And this is not even just a tech problem, it is an "efficacy of visualization" problem. If you naively send 100 million points and only have one million pixels, it's questionable that you can get a useful visualization, even with something like alpha compositing. There's two thinks to mention here:
You can certainly use the new Bokeh server to perform downsampling, and send the result to the browser. I would like to have a version of a previous ocean data downsampling data demo on the new server very soon (there's just about an infinite number of things on the todo list, though... working as fast as I can).
We are also working on a project called DataShader (it's in the Bokeh GitHub org as a public repo) that intends to do perceptually accurate downsampling automatically. We demonstrated an earlier version of this at Strata and other conferences to interactively browse the full 350 million point across all scales with a perceptually accurate downsampling with categorical blends (on a laptop!) and it was received very well. As part of the work to make it a public repo it's undergoing a large refactor. Expect more announcements about DataShader in the next month or two.
Bryan
···
On Jan 6, 2016, at 9:04 PM, Havoc Pennington <[email protected]> wrote:
The server uses all the same plotting code. The only thing I can think of it might add is that you might be able to write code to dynamically sub sample the data based on the current axis range or something along those lines.
The server would probably break if you try to send 10tb to the client. But if you arrange to subset the data and only send what's needed on demand to the client, it could work.
I have 10 Tbytes of uint8 of radio telescope data that I would like to browse. I can load 10GBytes at a time.
bokeh.plotting.line fails if the number of points I indicate is more than 64K (2^16). Is this a problem for the server? Here is my code. 10GBytes is already loaded
**********************************************
import numpy as np
from bokeh.plotting import figure, output_file, show, output_notebook
bDataLength = 2**15
bDataStart = 0
x = np.arange(bDataLength)
y = telescopekData[bDataStart : bDataStart + bDataLength]
output_notebook()
p = figure(title="summed aligned", x_axis_label='x', y_axis_label='y')