Pass dataframe from server_lifecycle to main

I have a large set of data that I read into a pandas dataframe in the on_server_loaded function in my server_lifecycle.py. I now need to access that dataframe from my main app. How would I do this?

This is a use-case that has been discussed. We have not had time to formalize any new API or recommended guidance. But for the time being, something like this might works:

* on_server_loaded gets passed a ServerContext, attach your big DataFrame as an attr to that
* on_session_created gets passed the ServerContext and the document, copy a reference to the data from the ServerContext to the document
* then it should be available as curdoc().foo

You might need to use underscored attr names, e.g. "._foo".

This is all untested, we are in the middle of a release frenzy so I can't dive in any deeper right now. I can possibly try to work up an example next week. Alternatively if you get something along these lines working, please share so that it can inform discussions about making a "supported" way of doing this.

Thanks,

Bryan

···

On Jun 22, 2016, at 10:16 AM, [email protected] wrote:

I have a large set of data that I read into a pandas dataframe in the on_server_loaded function in my server_lifecycle.py. I now need to access that dataframe from my main app. How would I do this?

--
You received this message because you are subscribed to the Google Groups "Bokeh Discussion - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/2a31040f-09e6-4d6d-88a6-24510897af03%40continuum.io.
For more options, visit https://groups.google.com/a/continuum.io/d/optout.

The way I originally intended it to work is that you would store globals in your server_lifecycle.py, and then “push” them as needed to sessions when sessions are created. So on_server_loaded saves a global, then in on_session_created modify your session’s document however you want (possibly based on the global data).

For more see this post: https://groups.google.com/a/continuum.io/d/msg/bokeh/qVVfCUu_eow/cDZjQcVbBQAJ

Bokeh should really have a simple API like the suggested add_session_updater there, would be a great project for someone to take on.

I don’t think you need to stick anything on the ServerContext; as far as I know, a global variable in server_lifecycle.py would be just as good.

Havoc

···

On Wed, Jun 22, 2016 at 11:35 AM, Bryan Van de Ven [email protected] wrote:

This is a use-case that has been discussed. We have not had time to formalize any new API or recommended guidance. But for the time being, something like this might works:

  • on_server_loaded gets passed a ServerContext, attach your big DataFrame as an attr to that

  • on_session_created gets passed the ServerContext and the document, copy a reference to the data from the ServerContext to the document

  • then it should be available as curdoc().foo

You might need to use underscored attr names, e.g. “._foo”.

This is all untested, we are in the middle of a release frenzy so I can’t dive in any deeper right now. I can possibly try to work up an example next week. Alternatively if you get something along these lines working, please share so that it can inform discussions about making a “supported” way of doing this.

Thanks,

Bryan

On Jun 22, 2016, at 10:16 AM, [email protected] wrote:

I have a large set of data that I read into a pandas dataframe in the on_server_loaded function in my server_lifecycle.py. I now need to access that dataframe from my main app. How would I do this?

You received this message because you are subscribed to the Google Groups “Bokeh Discussion - Public” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To post to this group, send email to [email protected].

To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/2a31040f-09e6-4d6d-88a6-24510897af03%40continuum.io.

For more options, visit https://groups.google.com/a/continuum.io/d/optout.

You received this message because you are subscribed to the Google Groups “Bokeh Discussion - Public” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To post to this group, send email to [email protected].

To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/4C51A715-89B1-4F6F-B7ED-021823D1398E%40continuum.io.
For more options, visit https://groups.google.com/a/continuum.io/d/optout.

Havoc Pennington

Senior Software Architect

Yes that seems better. As it turns out I am going to try and re-do the spectrogram example as a server app, to demonstrate using threads with the server app, as well as lifecycle hooks, so I may have more to say about this sooner.

Bryan

···

On Jun 22, 2016, at 2:40 PM, Havoc Pennington <[email protected]> wrote:

The way I originally intended it to work is that you would store globals in your server_lifecycle.py, and then "push" them as needed to sessions when sessions are created. So on_server_loaded saves a global, then in on_session_created modify your session's document however you want (possibly based on the global data).

For more see this post: https://groups.google.com/a/continuum.io/d/msg/bokeh/qVVfCUu_eow/cDZjQcVbBQAJ

Bokeh should really have a simple API like the suggested add_session_updater there, would be a great project for someone to take on.

I don't think you need to stick anything on the ServerContext; as far as I know, a global variable in server_lifecycle.py would be just as good.

Havoc

On Wed, Jun 22, 2016 at 11:35 AM, Bryan Van de Ven <[email protected]> wrote:
This is a use-case that has been discussed. We have not had time to formalize any new API or recommended guidance. But for the time being, something like this might works:

* on_server_loaded gets passed a ServerContext, attach your big DataFrame as an attr to that
* on_session_created gets passed the ServerContext and the document, copy a reference to the data from the ServerContext to the document
* then it should be available as curdoc().foo

You might need to use underscored attr names, e.g. "._foo".

This is all untested, we are in the middle of a release frenzy so I can't dive in any deeper right now. I can possibly try to work up an example next week. Alternatively if you get something along these lines working, please share so that it can inform discussions about making a "supported" way of doing this.

Thanks,

Bryan

> On Jun 22, 2016, at 10:16 AM, [email protected] wrote:
>
> I have a large set of data that I read into a pandas dataframe in the on_server_loaded function in my server_lifecycle.py. I now need to access that dataframe from my main app. How would I do this?
>
> --
> You received this message because you are subscribed to the Google Groups "Bokeh Discussion - Public" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
> To post to this group, send email to [email protected].
> To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/2a31040f-09e6-4d6d-88a6-24510897af03%40continuum.io.
> For more options, visit https://groups.google.com/a/continuum.io/d/optout.

--
You received this message because you are subscribed to the Google Groups "Bokeh Discussion - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/4C51A715-89B1-4F6F-B7ED-021823D1398E%40continuum.io.
For more options, visit https://groups.google.com/a/continuum.io/d/optout.

--
Havoc Pennington
Senior Software Architect

--
You received this message because you are subscribed to the Google Groups "Bokeh Discussion - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/CAC%2B_nE2vsA2XpsaXu%3DOsJLjDDdSSu7eoRm0kYTgR5ugp3Sc2xw%40mail.gmail.com.
For more options, visit https://groups.google.com/a/continuum.io/d/optout.

I ended up using

on_server_load:

setattr(server_context,‘df’,dataframe)

on_session_created:
setattr(session_context._document,‘df’,session_context.server_context.df)

then in main…

main:

curdoc().df

and that’s worked out nicely so far. great product btw

···

On Wednesday, June 22, 2016 at 10:19:43 AM UTC-5, [email protected] wrote:

I have a large set of data that I read into a pandas dataframe in the on_server_loaded function in my server_lifecycle.py. I now need to access that dataframe from my main app. How would I do this?

Adam that's great, I've actually just had success with a slightly different approach that I will share soon in an example. This is great tho we can figure out a nice convenience function to codify how to accomplish this use-case.

Bryan

···

On Jun 22, 2016, at 4:30 PM, [email protected] wrote:

I ended up using

on_server_load:
     <read data to dataframe>
     setattr(server_context,'df',dataframe)
on_session_created:
     setattr(session_context._document,'df',session_context.server_context.df)

then in main...
main:
     curdoc().df

and that's worked out nicely so far. great product btw

On Wednesday, June 22, 2016 at 10:19:43 AM UTC-5, [email protected] wrote:
I have a large set of data that I read into a pandas dataframe in the on_server_loaded function in my server_lifecycle.py. I now need to access that dataframe from my main app. How would I do this?

--
You received this message because you are subscribed to the Google Groups "Bokeh Discussion - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/8aa29952-d268-446b-b853-8494feef55e8%40continuum.io.
For more options, visit https://groups.google.com/a/continuum.io/d/optout.

FYI you can see my approach in this PR:

  https://github.com/bokeh/bokeh/pull/4650

Thanks,

Bryan

···

On Jun 22, 2016, at 5:00 PM, Bryan Van de Ven <[email protected]> wrote:

Adam that's great, I've actually just had success with a slightly different approach that I will share soon in an example. This is great tho we can figure out a nice convenience function to codify how to accomplish this use-case.

Bryan

On Jun 22, 2016, at 4:30 PM, [email protected] wrote:

I ended up using

on_server_load:
    <read data to dataframe>
    setattr(server_context,'df',dataframe)
on_session_created:
    setattr(session_context._document,'df',session_context.server_context.df)

then in main...
main:
    curdoc().df

and that's worked out nicely so far. great product btw

On Wednesday, June 22, 2016 at 10:19:43 AM UTC-5, [email protected] wrote:
I have a large set of data that I read into a pandas dataframe in the on_server_loaded function in my server_lifecycle.py. I now need to access that dataframe from my main app. How would I do this?

--
You received this message because you are subscribed to the Google Groups "Bokeh Discussion - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/8aa29952-d268-446b-b853-8494feef55e8%40continuum.io.
For more options, visit https://groups.google.com/a/continuum.io/d/optout.