ColumnDataSource object variables when dealing with dynamic number of data sources

On Jul 13, 2015, at 8:36 PM, Andrew Joros <andrew.joros@gmail.com> wrote:

Hi all,

Great to meet some of the Bokeh team at SciPy (Christine and Bryan). Great people! And thanks for being so understanding to us Bokeh app newbies.

I am currently stuck on understanding ColumnDataSource. If I am to make a multi-line time series plot app with check boxes that enable/disable respective lines, am I required to create a ColumnDataSource object variable for each line in the multi-line time series plot. If so, this could be problematic for me as each time series data line will be retrieved from a CSV file. If I am to have a changing amount of CSV files in a folder, how can one dynamically create these ColumnDataSource object variables?

Thanks!

Andy

--
You received this message because you are subscribed to the Google Groups "Bokeh Discussion - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bokeh+unsubscribe@continuum.io.
To post to this group, send email to bokeh@continuum.io.
To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/54fc2e86-0925-42cf-a022-531129c05fcc%40continuum.io\.
For more options, visit https://groups.google.com/a/continuum.io/d/optout\.

On Monday, July 13, 2015 at 7:14:19 PM UTC-7, Bryan Van de ven wrote:

You will probably have to construct new glyphs and glyph renderers (each with their own data source) on demand for each file.

Bryan

On Jul 13, 2015, at 8:36 PM, Andrew Joros andrew...@gmail.com wrote:

Hi all,

Great to meet some of the Bokeh team at SciPy (Christine and Bryan). Great people! And thanks for being so understanding to us Bokeh app newbies.

I am currently stuck on understanding ColumnDataSource. If I am to make a multi-line time series plot app with check boxes that enable/disable respective lines, am I required to create a ColumnDataSource object variable for each line in the multi-line time series plot. If so, this could be problematic for me as each time series data line will be retrieved from a CSV file. If I am to have a changing amount of CSV files in a folder, how can one dynamically create these ColumnDataSource object variables?

Thanks!

Andy

–
You received this message because you are subscribed to the Google Groups “Bokeh Discussion - Public” group.

To unsubscribe from this group and stop receiving emails from it, send an email to bokeh+un...@continuum.io.

To post to this group, send email to bo...@continuum.io.

To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/54fc2e86-0925-42cf-a022-531129c05fcc%40continuum.io.

For more options, visit https://groups.google.com/a/continuum.io/d/optout.

On Jul 13, 2015, at 10:42 PM, Andrew Joros <andrew.joros@gmail.com> wrote:

Ok, so correct me if I'm wrong, but there is no way to create a X-amount ColumnDataSource's given X-amount of CSV files. It's strictly a static process? I thought Fabio's example was quasi-dynamic, but I could be wrong?

On Monday, July 13, 2015 at 7:14:19 PM UTC-7, Bryan Van de ven wrote:
You will probably have to construct new glyphs and glyph renderers (each with their own data source) on demand for each file.

Bryan

> On Jul 13, 2015, at 8:36 PM, Andrew Joros <andrew...@gmail.com> wrote:
>
> Hi all,
>
> Great to meet some of the Bokeh team at SciPy (Christine and Bryan). Great people! And thanks for being so understanding to us Bokeh app newbies.
>
> I am currently stuck on understanding ColumnDataSource. If I am to make a multi-line time series plot app with check boxes that enable/disable respective lines, am I required to create a ColumnDataSource object variable for each line in the multi-line time series plot. If so, this could be problematic for me as each time series data line will be retrieved from a CSV file. If I am to have a changing amount of CSV files in a folder, how can one dynamically create these ColumnDataSource object variables?
>
> Thanks!
>
> Andy
>
> --
> You received this message because you are subscribed to the Google Groups "Bokeh Discussion - Public" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to bokeh+un...@continuum.io.
> To post to this group, send email to bo...@continuum.io.
> To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/54fc2e86-0925-42cf-a022-531129c05fcc%40continuum.io\.
> For more options, visit https://groups.google.com/a/continuum.io/d/optout\.

--
You received this message because you are subscribed to the Google Groups "Bokeh Discussion - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bokeh+unsubscribe@continuum.io.
To post to this group, send email to bokeh@continuum.io.
To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/855dac52-e0ac-41ce-a8dc-7d871f4bb746%40continuum.io\.
For more options, visit https://groups.google.com/a/continuum.io/d/optout\.

On Tuesday, July 14, 2015 at 5:59:17 AM UTC+2, Bryan Van de ven wrote:

First, as mentioned, it’s not just column data sources. You will almost certainly have to create Glyph+GlyphRenderer+ColumnDataSources for each CSV. Basically, all the work that p.line(…) does every time you want to draw a new line for a new set of data. But no, you can certainly create new objects as-needed.

There are two potential approaches:

• without a bokeh server, you can query some REST API on some other server (that you would have to write, presumably) to get information about the file system and/or serve up file contents. Then you can add/delete/modify models directly on the client, in JavaScript. This will require custom JS to do some of the work

• with a bokeh server, you can run a python script in the server using --script, there you have access to the filesystem, and can create the new objects you need (in python) and store them, which will result in the page/visualization being updated.

There’s possibly other avenues as well, but it’s hard to say anything specific without details of the desired UX. The choices are really more to do with general web-app dev and not anything specific to Bokeh, per se. I think we’d have to start discussing a commercial services engagement to get into that level of support, however.

Bryan

On Jul 13, 2015, at 10:42 PM, Andrew Joros andrew.joros@gmail.com wrote:

Ok, so correct me if I’m wrong, but there is no way to create a X-amount ColumnDataSource’s given X-amount of CSV files. It’s strictly a static process? I thought Fabio’s example was quasi-dynamic, but I could be wrong?

On Monday, July 13, 2015 at 7:14:19 PM UTC-7, Bryan Van de ven wrote:

You will probably have to construct new glyphs and glyph renderers (each with their own data source) on demand for each file.

Bryan

On Jul 13, 2015, at 8:36 PM, Andrew Joros andrew...@gmail.com wrote:

Hi all,

Great to meet some of the Bokeh team at SciPy (Christine and Bryan). Great people! And thanks for being so understanding to us Bokeh app newbies.

I am currently stuck on understanding ColumnDataSource. If I am to make a multi-line time series plot app with check boxes that enable/disable respective lines, am I required to create a ColumnDataSource object variable for each line in the multi-line time series plot. If so, this could be problematic for me as each time series data line will be retrieved from a CSV file. If I am to have a changing amount of CSV files in a folder, how can one dynamically create these ColumnDataSource object variables?

Thanks!

Andy

–
You received this message because you are subscribed to the Google Groups “Bokeh Discussion - Public” group.
To unsubscribe from this group and stop receiving emails from it, send an email to bokeh+un...@continuum.io.
To post to this group, send email to bo...@continuum.io.
To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/54fc2e86-0925-42cf-a022-531129c05fcc%40continuum.io.
For more options, visit https://groups.google.com/a/continuum.io/d/optout.

–
You received this message because you are subscribed to the Google Groups “Bokeh Discussion - Public” group.

To unsubscribe from this group and stop receiving emails from it, send an email to bokeh+unsubscribe@continuum.io.

To post to this group, send email to bokeh@continuum.io.

To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/855dac52-e0ac-41ce-a8dc-7d871f4bb746%40continuum.io.

For more options, visit https://groups.google.com/a/continuum.io/d/optout.

On Tuesday, July 14, 2015 at 2:06:37 AM UTC-7, Fabio Pliger wrote:

Hi Andrew,

Bryan first suggestion is probably what I’d suggest in first place too… but there are a few blurry things I’m not sure I understand in your use case.

You want be able to walk over a folder (with N csv files in it) and create a plot that has N checkboxes (one for each csv file in the folder) and N timeseries lines on your plot (one for each csv file in your folder). When the user clicks on one of those boxes you want to toggle the corresponding line on the plot. Is this correct?

If it is correct, I’m also assuming that when you say:

If I am to have a changing amount of CSV files in a folder, how can one dynamically create these ColumnDataSource object variables?

you mean that you want the interface to be updated with the new csv file when user reloads the page and not live…

So. Back to your problem, as I said, you could use Bryan suggestion and create an AjaxDataSource that controls the data for all your csv files. For that you’ll need to:

• create your REST API interface that returns the json object with the data you’ll currently see on the interface. So if, for instance, you want to just show the first 4 csv files and have the other N-4 not showing your API endpoint (that you’ll pass to the AjaxDataSource) will return a json object with full lists of data for the 4 csv files and empty lists for the other N-4 csv files…

• When the clicks on a checkbox you can send a POST/PUT request to your service to control that data to enable/disable the sources you are returning with the endpoint you’ve connected to your AjaxDataSource

There are other things that influence the best solution… How big is your data? How many files you think you’ll have to handle?

Fabio

On Tuesday, July 14, 2015 at 5:59:17 AM UTC+2, Bryan Van de ven wrote:

First, as mentioned, it’s not just column data sources. You will almost certainly have to create Glyph+GlyphRenderer+ColumnDataSources for each CSV. Basically, all the work that p.line(…) does every time you want to draw a new line for a new set of data. But no, you can certainly create new objects as-needed.

There are two potential approaches:

• without a bokeh server, you can query some REST API on some other server (that you would have to write, presumably) to get information about the file system and/or serve up file contents. Then you can add/delete/modify models directly on the client, in JavaScript. This will require custom JS to do some of the work

• with a bokeh server, you can run a python script in the server using --script, there you have access to the filesystem, and can create the new objects you need (in python) and store them, which will result in the page/visualization being updated.

There’s possibly other avenues as well, but it’s hard to say anything specific without details of the desired UX. The choices are really more to do with general web-app dev and not anything specific to Bokeh, per se. I think we’d have to start discussing a commercial services engagement to get into that level of support, however.

Bryan

On Jul 13, 2015, at 10:42 PM, Andrew Joros andrew...@gmail.com wrote:

Ok, so correct me if I’m wrong, but there is no way to create a X-amount ColumnDataSource’s given X-amount of CSV files. It’s strictly a static process? I thought Fabio’s example was quasi-dynamic, but I could be wrong?

On Monday, July 13, 2015 at 7:14:19 PM UTC-7, Bryan Van de ven wrote:

You will probably have to construct new glyphs and glyph renderers (each with their own data source) on demand for each file.

Bryan

On Jul 13, 2015, at 8:36 PM, Andrew Joros andrew...@gmail.com wrote:

Hi all,

Great to meet some of the Bokeh team at SciPy (Christine and Bryan). Great people! And thanks for being so understanding to us Bokeh app newbies.

I am currently stuck on understanding ColumnDataSource. If I am to make a multi-line time series plot app with check boxes that enable/disable respective lines, am I required to create a ColumnDataSource object variable for each line in the multi-line time series plot. If so, this could be problematic for me as each time series data line will be retrieved from a CSV file. If I am to have a changing amount of CSV files in a folder, how can one dynamically create these ColumnDataSource object variables?

Thanks!

Andy

–
You received this message because you are subscribed to the Google Groups “Bokeh Discussion - Public” group.
To unsubscribe from this group and stop receiving emails from it, send an email to bokeh+un...@continuum.io.
To post to this group, send email to bo...@continuum.io.
To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/54fc2e86-0925-42cf-a022-531129c05fcc%40continuum.io.
For more options, visit https://groups.google.com/a/continuum.io/d/optout.

–
You received this message because you are subscribed to the Google Groups “Bokeh Discussion - Public” group.

To unsubscribe from this group and stop receiving emails from it, send an email to bokeh+un...@continuum.io.

To post to this group, send email to bo...@continuum.io.

To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/855dac52-e0ac-41ce-a8dc-7d871f4bb746%40continuum.io.

For more options, visit https://groups.google.com/a/continuum.io/d/optout.

On Tuesday, July 14, 2015 at 11:34:30 AM UTC+2, Andrew Joros wrote:

On Tuesday, July 14, 2015 at 2:00:33 PM UTC-7, Fabio Pliger wrote:

On Tuesday, July 14, 2015 at 11:34:30 AM UTC+2, Andrew Joros wrote:
[cut]

• create your REST API interface that returns the json object with the data you’ll currently see on the interface. So if, for instance, you want to just show the first 4 csv files and have the other N-4 not showing your API endpoint (that you’ll pass to the AjaxDataSource) will return a json object with full lists of data for the 4 csv files and empty lists for the other N-4 csv files…
• When the clicks on a checkbox you can send a POST/PUT request to your service to control that data to enable/disable the sources you are returning with the endpoint you’ve connected to your AjaxDataSource

**I wont lie, this portion seems a little daunting for my end and I feel would be easier with an example. Bryan previosly sent some code my way that I saw used AjaxDataSource (https://gist.github.com/bryevdv/aeec7a86827c13bf8e1f). Is that link a useful example for my case, or would you happen to have a better one? **

I think it may sound way mode complex that what it actually is… Bryan example is valid, yes. If shows the sort of flow you need but with a completely different dataset and logic. You’ll need to create a flask view that will handle all your logic and that you can point AjaxDataSource to.

That said, I may be able to include some useful code for your use case on a demo I’m preparing for next week. In that case I’ll ping you (no promises though

There are other things that influence the best solution… How big is your data? How many files you think you’ll have to handle?
**Each CSV file is approx. 23KB and about 11-15 CSV files. Fairly small right now for this small project I’m working on. **

In the future I would like to use a meteorological PostGreSQL DB which will be about 6GB for the prototype. The final product will be working with ~70GB I think.

Hope that answers some questions

Thanks again for your time, much appreciated,

Well… that makes your design choice even more important. IMHO you should think about a design that you can scale at this early stage.

Cheers

Fabio

On Thu, Jul 16, 2015 at 12:04 AM, Andrew Joros andrew.joros@gmail.com wrote:

Fabio, thanks for that input. Would you happen to have a go-to link on learning how to create this flask view and make it work with AjaxDataSource.

There are also a couple of examples in out repo that you can check:

https://github.com/bokeh/bokeh/blob/master/examples/plotting/file/ajax_source.py

https://github.com/bokeh/bokeh/blob/master/examples/plotting/file/ajax_source_realtime.py

I’m also planning to release a few more examples next week after my EuroPython talk.

In regards to the design choice, right now I am just going to worry about this small use-case. Better to understand the fundamentals IMO.

Sure. I didn’t meant to say anything different. My intention was to say: learn about the options server, AjaxDataSource, etc…, understand the differences, pick the one that is the best compromise for your use case (usability, easy to use, scalability, etc…)

Cheers

Thanks again

On Tuesday, July 14, 2015 at 2:00:33 PM UTC-7, Fabio Pliger wrote:

On Tuesday, July 14, 2015 at 11:34:30 AM UTC+2, Andrew Joros wrote:
[cut]

• create your REST API interface that returns the json object with the data you’ll currently see on the interface. So if, for instance, you want to just show the first 4 csv files and have the other N-4 not showing your API endpoint (that you’ll pass to the AjaxDataSource) will return a json object with full lists of data for the 4 csv files and empty lists for the other N-4 csv files…
• When the clicks on a checkbox you can send a POST/PUT request to your service to control that data to enable/disable the sources you are returning with the endpoint you’ve connected to your AjaxDataSource

**I wont lie, this portion seems a little daunting for my end and I feel would be easier with an example. Bryan previosly sent some code my way that I saw used AjaxDataSource (https://gist.github.com/bryevdv/aeec7a86827c13bf8e1f). Is that link a useful example for my case, or would you happen to have a better one? **

I think it may sound way mode complex that what it actually is… Bryan example is valid, yes. If shows the sort of flow you need but with a completely different dataset and logic. You’ll need to create a flask view that will handle all your logic and that you can point AjaxDataSource to.

That said, I may be able to include some useful code for your use case on a demo I’m preparing for next week. In that case I’ll ping you (no promises though

There are other things that influence the best solution… How big is your data? How many files you think you’ll have to handle?
**Each CSV file is approx. 23KB and about 11-15 CSV files. Fairly small right now for this small project I’m working on. **

In the future I would like to use a meteorological PostGreSQL DB which will be about 6GB for the prototype. The final product will be working with ~70GB I think.

Hope that answers some questions

Thanks again for your time, much appreciated,

Well… that makes your design choice even more important. IMHO you should think about a design that you can scale at this early stage.

Cheers

Fabio

–

You received this message because you are subscribed to the Google Groups “Bokeh Discussion - Public” group.

To unsubscribe from this group and stop receiving emails from it, send an email to bokeh+unsubscribe@continuum.io.

To post to this group, send email to bokeh@continuum.io.

To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/817199fa-54e7-4e2c-9668-3feb4cb052a9%40continuum.io.

For more options, visit https://groups.google.com/a/continuum.io/d/optout.

–

Fabio Pliger