☰Read-only archive⌕
Looking for help with Bokeh?

This is a read-only archive. Ask questions and join the conversation on GitHub.

Visit Community Support on GitHub →
Archived discussions · Captured October 28, 2026

Streaming with NaN values

Community Support
Meir_Tseitlin
Meir_Tseitlin

I upgraded Bokeh with 0.12.4 and I think the workaround with streaming NaN values no longer works.

I am patching my DataFrame with ‘NaN’ string before turning it to dict and feeding to ColumnDataSource.stream :
stream(df.fillna(‘NaN’).to_dict(orient=“list”))

``

I have several line charts in gridplot and the chart with NaNs is not displaying any line. (No errors produced as well). Stream update contains a single number with NaN, I am willing to connect a line between available numbers.

Any ideas?

Thanks

BTW: Is there anything I should do to enable binary protocol instead of JSON?

⌁   Link to post
Bryan
Bryan

Hi,

NumPy arrays (and thus also Pandas Series that are Float32, Float64, or Int32 dtype are automatically serialized using the binary protocol. This also no includes nans and infs. There is no need for any workaround when starting form those array types.

I can't quite tell from your message exactly what you are doing, so I can't elaborate any further. Maybe you can provide more code for more context.

Bryan

···

On Feb 13, 2017, at 14:04, Meir Tseitlin <meir@imubit.com> wrote:

I upgraded Bokeh with 0.12.4 and I think the workaround with streaming NaN values no longer works.

I am patching my DataFrame with 'NaN' string before turning it to dict and feeding to ColumnDataSource.stream :
stream(df.fillna('NaN').to_dict(orient="list"))

I have several line charts in gridplot and the chart with NaNs is not displaying any line. (No errors produced as well). Stream update contains a single number with NaN, I am willing to connect a line between available numbers.

Any ideas?

Thanks

BTW: Is there anything I should do to enable binary protocol instead of JSON?

--
You received this message because you are subscribed to the Google Groups "Bokeh Discussion - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bokeh+unsubscribe@continuum.io.
To post to this group, send email to bokeh@continuum.io.
To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/772d38de-e3cb-461b-93bc-2a28861b03aa%40continuum.io\.
For more options, visit https://groups.google.com/a/continuum.io/d/optout\.

⌁   Link to post
Meir_Tseitlin
Meir_Tseitlin

Thank you Bryan,

  • I am creating several figures with time-series data (each figure has a single line), I provide a single ColumnDataSource with the data to all figures (each ‘y’ has it’s own name and ‘x’=‘time’).
  • I am running Bokeh server with add_periodic_callback, streaming additional data to ColumnDataSource
  • The figure containing NaNs does not draw any line.
  • My types are float64, but if I am not doing fillna(‘NaN’), I receive: “Out of range float values are not JSON compliant: nan”

Example of dict I am feeding to stream:
{

u’var9’: [

122.876,

122.876,

122.88,

122.889,

122.885,

122.884,

122.885,

122.887,

122.875

],

u’var40’: [

‘NaN’,

‘NaN’,

53.7135,

‘NaN’,

‘NaN’,

‘NaN’,

‘NaN’,

‘NaN’,

‘NaN’

],

u’var17’: [

22.9009,

23.0699,

22.6906,

22.8626,

22.8737,

22.6161,

23.0456,

22.9609,

22.9154

],

u’var7’: [

2799.43,

2799.77,

2799.13,

2799.05,

2799.37,

2799.4,

2798.81,

2799.76,

2799.08

],

‘time’: [

Timestamp(‘2017-02-13 20:32:35.263716’),

Timestamp(‘2017-02-13 20:32:35.671465’),

Timestamp(‘2017-02-13 20:32:36.079146’),

Timestamp(‘2017-02-13 20:32:36.489926’),

Timestamp(‘2017-02-13 20:32:36.896583’),

Timestamp(‘2017-02-13 20:32:37.302591’),

Timestamp(‘2017-02-13 20:32:37.708200’),

Timestamp(‘2017-02-13 20:32:38.112995’),

Timestamp(‘2017-02-13 20:32:38.520433’)

]

}

···

On Monday, February 13, 2017 at 10:18:54 PM UTC+2, Bryan Van de ven wrote:

Hi,

NumPy arrays (and thus also Pandas Series that are Float32, Float64, or Int32 dtype are automatically serialized using the binary protocol. This also no includes nans and infs. There is no need for any workaround when starting form those array types.

I can’t quite tell from your message exactly what you are doing, so I can’t elaborate any further. Maybe you can provide more code for more context.

Bryan

On Feb 13, 2017, at 14:04, Meir Tseitlin me...@imubit.com wrote:

I upgraded Bokeh with 0.12.4 and I think the workaround with streaming NaN values no longer works.

I am patching my DataFrame with ‘NaN’ string before turning it to dict and feeding to ColumnDataSource.stream :

stream(df.fillna(‘NaN’).to_dict(orient=“list”))

I have several line charts in gridplot and the chart with NaNs is not displaying any line. (No errors produced as well). Stream update contains a single number with NaN, I am willing to connect a line between available numbers.

Any ideas?

Thanks

BTW: Is there anything I should do to enable binary protocol instead of JSON?

–
You received this message because you are subscribed to the Google Groups “Bokeh Discussion - Public” group.

To unsubscribe from this group and stop receiving emails from it, send an email to bokeh+un...@continuum.io.

To post to this group, send email to bo...@continuum.io.

To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/772d38de-e3cb-461b-93bc-2a28861b03aa%40continuum.io.

For more options, visit https://groups.google.com/a/continuum.io/d/optout.

⌁   Link to post
Bryan
Bryan

The columns in your dict are python lists, not arrays. You need to construct actual NumPy arrays for the columns in order for Bokeh to be able to use the binary array protocol.

Thanks,

Bryan

···

On Feb 13, 2017, at 14:36, Meir Tseitlin <meir@imubit.com> wrote:

Thank you Bryan,

  • I am creating several figures with time-series data (each figure has a single line), I provide a single ColumnDataSource with the data to all figures (each 'y' has it's own name and 'x'='time').
  • I am running Bokeh server with add_periodic_callback, streaming additional data to ColumnDataSource
  • The figure containing NaNs does not draw any line.
  • My types are float64, but if I am not doing fillna('NaN'), I receive: "Out of range float values are not JSON compliant: nan"

Example of dict I am feeding to stream:
{
  u'var9': [
    122.876,
    122.876,
    122.88,
    122.889,
    122.885,
    122.884,
    122.885,
    122.887,
    122.875
  ],
  u'var40': [
    'NaN',
    'NaN',
    53.7135,
    'NaN',
    'NaN',
    'NaN',
    'NaN',
    'NaN',
    'NaN'
  ],
  u'var17': [
    22.9009,
    23.0699,
    22.6906,
    22.8626,
    22.8737,
    22.6161,
    23.0456,
    22.9609,
    22.9154
  ],
  u'var7': [
    2799.43,
    2799.77,
    2799.13,
    2799.05,
    2799.37,
    2799.4,
    2798.81,
    2799.76,
    2799.08
  ],
  'time': [
    Timestamp('2017-02-13 20:32:35.263716'),
    Timestamp('2017-02-13 20:32:35.671465'),
    Timestamp('2017-02-13 20:32:36.079146'),
    Timestamp('2017-02-13 20:32:36.489926'),
    Timestamp('2017-02-13 20:32:36.896583'),
    Timestamp('2017-02-13 20:32:37.302591'),
    Timestamp('2017-02-13 20:32:37.708200'),
    Timestamp('2017-02-13 20:32:38.112995'),
    Timestamp('2017-02-13 20:32:38.520433')
  ]
}

On Monday, February 13, 2017 at 10:18:54 PM UTC+2, Bryan Van de ven wrote:
Hi,

NumPy arrays (and thus also Pandas Series that are Float32, Float64, or Int32 dtype are automatically serialized using the binary protocol. This also no includes nans and infs. There is no need for any workaround when starting form those array types.

I can't quite tell from your message exactly what you are doing, so I can't elaborate any further. Maybe you can provide more code for more context.

Bryan

> On Feb 13, 2017, at 14:04, Meir Tseitlin <me...@imubit.com> wrote:
>
> I upgraded Bokeh with 0.12.4 and I think the workaround with streaming NaN values no longer works.
>
> I am patching my DataFrame with 'NaN' string before turning it to dict and feeding to ColumnDataSource.stream :
> stream(df.fillna('NaN').to_dict(orient="list"))
>
> I have several line charts in gridplot and the chart with NaNs is not displaying any line. (No errors produced as well). Stream update contains a single number with NaN, I am willing to connect a line between available numbers.
>
> Any ideas?
>
> Thanks
>
> BTW: Is there anything I should do to enable binary protocol instead of JSON?
>
> --
> You received this message because you are subscribed to the Google Groups "Bokeh Discussion - Public" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to bokeh+un...@continuum.io.
> To post to this group, send email to bo...@continuum.io.
> To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/772d38de-e3cb-461b-93bc-2a28861b03aa%40continuum.io\.
> For more options, visit https://groups.google.com/a/continuum.io/d/optout\.

--
You received this message because you are subscribed to the Google Groups "Bokeh Discussion - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bokeh+unsubscribe@continuum.io.
To post to this group, send email to bokeh@continuum.io.
To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/6ed629f8-38de-4258-a91c-ccd7d101ffc5%40continuum.io\.
For more options, visit https://groups.google.com/a/continuum.io/d/optout\.

⌁   Link to post
Meir_Tseitlin
Meir_Tseitlin

Hi Bryan,

I am still struggling with this issue.

Can you provide an example of how to stream data (ColumnDataSource.stream) with numpy arrays?

Thanks

···

On Tuesday, February 14, 2017 at 12:49:33 AM UTC+2, Bryan Van de ven wrote:

The columns in your dict are python lists, not arrays. You need to construct actual NumPy arrays for the columns in order for Bokeh to be able to use the binary array protocol.

Thanks,

Bryan

On Feb 13, 2017, at 14:36, Meir Tseitlin me...@imubit.com wrote:

Thank you Bryan,

    • I am creating several figures with time-series data (each figure has a single line), I provide a single ColumnDataSource with the data to all figures (each 'y' has it's own name and 'x'='time').
    • I am running Bokeh server with add_periodic_callback, streaming additional data to  ColumnDataSource
    • The figure containing NaNs does not draw any line.
    • My types are float64, but if I am not doing fillna('NaN'), I receive: "Out of range float values are not JSON compliant: nan"

Example of dict I am feeding to stream:

{

u’var9’: [

122.876,
122.876,
122.88,
122.889,
122.885,
122.884,
122.885,
122.887,
122.875

],

u’var40’: [

'NaN',
'NaN',
53.7135,
'NaN',
'NaN',
'NaN',
'NaN',
'NaN',
'NaN'

],

u’var17’: [

22.9009,
23.0699,
22.6906,
22.8626,
22.8737,
22.6161,
23.0456,
22.9609,
22.9154

],

u’var7’: [

2799.43,
2799.77,
2799.13,
2799.05,
2799.37,
2799.4,
2798.81,
2799.76,
2799.08

],

‘time’: [

Timestamp('2017-02-13 20:32:35.263716'),
Timestamp('2017-02-13 20:32:35.671465'),
Timestamp('2017-02-13 20:32:36.079146'),
Timestamp('2017-02-13 20:32:36.489926'),
Timestamp('2017-02-13 20:32:36.896583'),
Timestamp('2017-02-13 20:32:37.302591'),
Timestamp('2017-02-13 20:32:37.708200'),
Timestamp('2017-02-13 20:32:38.112995'),
Timestamp('2017-02-13 20:32:38.520433')

]

}

On Monday, February 13, 2017 at 10:18:54 PM UTC+2, Bryan Van de ven wrote:

Hi,

NumPy arrays (and thus also Pandas Series that are Float32, Float64, or Int32 dtype are automatically serialized using the binary protocol. This also no includes nans and infs. There is no need for any workaround when starting form those array types.

I can’t quite tell from your message exactly what you are doing, so I can’t elaborate any further. Maybe you can provide more code for more context.

Bryan

On Feb 13, 2017, at 14:04, Meir Tseitlin me...@imubit.com wrote:

I upgraded Bokeh with 0.12.4 and I think the workaround with streaming NaN values no longer works.

I am patching my DataFrame with ‘NaN’ string before turning it to dict and feeding to ColumnDataSource.stream :
stream(df.fillna(‘NaN’).to_dict(orient=“list”))

I have several line charts in gridplot and the chart with NaNs is not displaying any line. (No errors produced as well). Stream update contains a single number with NaN, I am willing to connect a line between available numbers.

Any ideas?

Thanks

BTW: Is there anything I should do to enable binary protocol instead of JSON?

–
You received this message because you are subscribed to the Google Groups “Bokeh Discussion - Public” group.
To unsubscribe from this group and stop receiving emails from it, send an email to bokeh+un...@continuum.io.
To post to this group, send email to bo...@continuum.io.
To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/772d38de-e3cb-461b-93bc-2a28861b03aa%40continuum.io.
For more options, visit https://groups.google.com/a/continuum.io/d/optout.

–
You received this message because you are subscribed to the Google Groups “Bokeh Discussion - Public” group.

To unsubscribe from this group and stop receiving emails from it, send an email to bokeh+un...@continuum.io.

To post to this group, send email to bo...@continuum.io.

To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/6ed629f8-38de-4258-a91c-ccd7d101ffc5%40continuum.io.

For more options, visit https://groups.google.com/a/continuum.io/d/optout.

⌁   Link to post
Meir_Tseitlin
Meir_Tseitlin

I think I figured out how to send it as numpy arrays (just converted value arrays to numpy) within the dictionary (leaving time values as standard array).

Yet the effect is very similar - when one of numpy arrays contain nan - the whole update is not displayed.

···

On Wednesday, April 12, 2017 at 1:13:30 PM UTC+3, Meir Tseitlin wrote:

Hi Bryan,

I am still struggling with this issue.

Can you provide an example of how to stream data (ColumnDataSource.stream) with numpy arrays?

Thanks

On Tuesday, February 14, 2017 at 12:49:33 AM UTC+2, Bryan Van de ven wrote:

The columns in your dict are python lists, not arrays. You need to construct actual NumPy arrays for the columns in order for Bokeh to be able to use the binary array protocol.

Thanks,

Bryan

On Feb 13, 2017, at 14:36, Meir Tseitlin me...@imubit.com wrote:

Thank you Bryan,

    • I am creating several figures with time-series data (each figure has a single line), I provide a single ColumnDataSource with the data to all figures (each 'y' has it's own name and 'x'='time').
    • I am running Bokeh server with add_periodic_callback, streaming additional data to  ColumnDataSource
    • The figure containing NaNs does not draw any line.
    • My types are float64, but if I am not doing fillna('NaN'), I receive: "Out of range float values are not JSON compliant: nan"

Example of dict I am feeding to stream:

{

u’var9’: [

122.876,
122.876,
122.88,
122.889,
122.885,
122.884,
122.885,
122.887,
122.875

],

u’var40’: [

'NaN',
'NaN',
53.7135,
'NaN',
'NaN',
'NaN',
'NaN',
'NaN',
'NaN'

],

u’var17’: [

22.9009,
23.0699,
22.6906,
22.8626,
22.8737,
22.6161,
23.0456,
22.9609,
22.9154

],

u’var7’: [

2799.43,
2799.77,
2799.13,
2799.05,
2799.37,
2799.4,
2798.81,
2799.76,
2799.08

],

‘time’: [

Timestamp('2017-02-13 20:32:35.263716'),
Timestamp('2017-02-13 20:32:35.671465'),
Timestamp('2017-02-13 20:32:36.079146'),
Timestamp('2017-02-13 20:32:36.489926'),
Timestamp('2017-02-13 20:32:36.896583'),
Timestamp('2017-02-13 20:32:37.302591'),
Timestamp('2017-02-13 20:32:37.708200'),
Timestamp('2017-02-13 20:32:38.112995'),
Timestamp('2017-02-13 20:32:38.520433')

]

}

On Monday, February 13, 2017 at 10:18:54 PM UTC+2, Bryan Van de ven wrote:

Hi,

NumPy arrays (and thus also Pandas Series that are Float32, Float64, or Int32 dtype are automatically serialized using the binary protocol. This also no includes nans and infs. There is no need for any workaround when starting form those array types.

I can’t quite tell from your message exactly what you are doing, so I can’t elaborate any further. Maybe you can provide more code for more context.

Bryan

On Feb 13, 2017, at 14:04, Meir Tseitlin me...@imubit.com wrote:

I upgraded Bokeh with 0.12.4 and I think the workaround with streaming NaN values no longer works.

I am patching my DataFrame with ‘NaN’ string before turning it to dict and feeding to ColumnDataSource.stream :
stream(df.fillna(‘NaN’).to_dict(orient=“list”))

I have several line charts in gridplot and the chart with NaNs is not displaying any line. (No errors produced as well). Stream update contains a single number with NaN, I am willing to connect a line between available numbers.

Any ideas?

Thanks

BTW: Is there anything I should do to enable binary protocol instead of JSON?

–
You received this message because you are subscribed to the Google Groups “Bokeh Discussion - Public” group.
To unsubscribe from this group and stop receiving emails from it, send an email to bokeh+un...@continuum.io.
To post to this group, send email to bo...@continuum.io.
To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/772d38de-e3cb-461b-93bc-2a28861b03aa%40continuum.io.
For more options, visit https://groups.google.com/a/continuum.io/d/optout.

–
You received this message because you are subscribed to the Google Groups “Bokeh Discussion - Public” group.

To unsubscribe from this group and stop receiving emails from it, send an email to bokeh+un...@continuum.io.

To post to this group, send email to bo...@continuum.io.

To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/6ed629f8-38de-4258-a91c-ccd7d101ffc5%40continuum.io.

For more options, visit https://groups.google.com/a/continuum.io/d/optout.

⌁   Link to post
Bryan
Bryan

Yes, this is expected and also beyond our control. There is no support in the JSON specification for "nan" values. So if you use lists (which get converted to JSON) then things will not work. However if you use numpy arrays then they get encoded in a different way that allows for nan's to be communicated properly. If you need to stream nan's then you need to put the data in a numpy array.

Thanks,

Bryan

···

On Apr 12, 2017, at 06:50, Meir Tseitlin <meir@imubit.com> wrote:

I think I figured out how to send it as numpy arrays (just converted value arrays to numpy) within the dictionary (leaving time values as standard array).

Yet the effect is very similar - when one of numpy arrays contain nan - the whole update is not displayed.

On Wednesday, April 12, 2017 at 1:13:30 PM UTC+3, Meir Tseitlin wrote:
Hi Bryan,

I am still struggling with this issue.

Can you provide an example of how to stream data (ColumnDataSource.stream) with numpy arrays?

Thanks

On Tuesday, February 14, 2017 at 12:49:33 AM UTC+2, Bryan Van de ven wrote:
The columns in your dict are python lists, not arrays. You need to construct actual NumPy arrays for the columns in order for Bokeh to be able to use the binary array protocol.

Thanks,

Bryan

> On Feb 13, 2017, at 14:36, Meir Tseitlin <me...@imubit.com> wrote:
>
> Thank you Bryan,
>
> • I am creating several figures with time-series data (each figure has a single line), I provide a single ColumnDataSource with the data to all figures (each 'y' has it's own name and 'x'='time').
> • I am running Bokeh server with add_periodic_callback, streaming additional data to ColumnDataSource
> • The figure containing NaNs does not draw any line.
> • My types are float64, but if I am not doing fillna('NaN'), I receive: "Out of range float values are not JSON compliant: nan"
>
> Example of dict I am feeding to stream:
> {
> u'var9': [
> 122.876,
> 122.876,
> 122.88,
> 122.889,
> 122.885,
> 122.884,
> 122.885,
> 122.887,
> 122.875
> ],
> u'var40': [
> 'NaN',
> 'NaN',
> 53.7135,
> 'NaN',
> 'NaN',
> 'NaN',
> 'NaN',
> 'NaN',
> 'NaN'
> ],
> u'var17': [
> 22.9009,
> 23.0699,
> 22.6906,
> 22.8626,
> 22.8737,
> 22.6161,
> 23.0456,
> 22.9609,
> 22.9154
> ],
> u'var7': [
> 2799.43,
> 2799.77,
> 2799.13,
> 2799.05,
> 2799.37,
> 2799.4,
> 2798.81,
> 2799.76,
> 2799.08
> ],
> 'time': [
> Timestamp('2017-02-13 20:32:35.263716'),
> Timestamp('2017-02-13 20:32:35.671465'),
> Timestamp('2017-02-13 20:32:36.079146'),
> Timestamp('2017-02-13 20:32:36.489926'),
> Timestamp('2017-02-13 20:32:36.896583'),
> Timestamp('2017-02-13 20:32:37.302591'),
> Timestamp('2017-02-13 20:32:37.708200'),
> Timestamp('2017-02-13 20:32:38.112995'),
> Timestamp('2017-02-13 20:32:38.520433')
> ]
> }
>
>
> On Monday, February 13, 2017 at 10:18:54 PM UTC+2, Bryan Van de ven wrote:
> Hi,
>
> NumPy arrays (and thus also Pandas Series that are Float32, Float64, or Int32 dtype are automatically serialized using the binary protocol. This also no includes nans and infs. There is no need for any workaround when starting form those array types.
>
> I can't quite tell from your message exactly what you are doing, so I can't elaborate any further. Maybe you can provide more code for more context.
>
> Bryan
>
> > On Feb 13, 2017, at 14:04, Meir Tseitlin <me...@imubit.com> wrote:
> >
> > I upgraded Bokeh with 0.12.4 and I think the workaround with streaming NaN values no longer works.
> >
> > I am patching my DataFrame with 'NaN' string before turning it to dict and feeding to ColumnDataSource.stream :
> > stream(df.fillna('NaN').to_dict(orient="list"))
> >
> > I have several line charts in gridplot and the chart with NaNs is not displaying any line. (No errors produced as well). Stream update contains a single number with NaN, I am willing to connect a line between available numbers.
> >
> > Any ideas?
> >
> > Thanks
> >
> > BTW: Is there anything I should do to enable binary protocol instead of JSON?
> >
> > --
> > You received this message because you are subscribed to the Google Groups "Bokeh Discussion - Public" group.
> > To unsubscribe from this group and stop receiving emails from it, send an email to bokeh+un...@continuum.io.
> > To post to this group, send email to bo...@continuum.io.
> > To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/772d38de-e3cb-461b-93bc-2a28861b03aa%40continuum.io\.
> > For more options, visit https://groups.google.com/a/continuum.io/d/optout\.
>
>
> --
> You received this message because you are subscribed to the Google Groups "Bokeh Discussion - Public" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to bokeh+un...@continuum.io.
> To post to this group, send email to bo...@continuum.io.
> To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/6ed629f8-38de-4258-a91c-ccd7d101ffc5%40continuum.io\.
> For more options, visit https://groups.google.com/a/continuum.io/d/optout\.

--
You received this message because you are subscribed to the Google Groups "Bokeh Discussion - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bokeh+unsubscribe@continuum.io.
To post to this group, send email to bokeh@continuum.io.
To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/0a2008db-0046-46d1-943b-3644576fd36f%40continuum.io\.
For more options, visit https://groups.google.com/a/continuum.io/d/optout\.

⌁   Link to post

End of archived discussion · Replies are disabled