# Streaming with NaN values

**URL:** <https://discourse.bokeh.org/t/streaming-with-nan-values/1970>\
**Category:** Community Support\
**Created:** [February 13, 2017, 8:04pm UTC](https://discourse.bokeh.org/t/streaming-with-nan-values/1970 "2017-02-13T20:04:44Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Meir\_Tseitlin](https://discourse.bokeh.org/letter_avatar_proxy/v4/letter/m/838e76/32.png) [@Meir\_Tseitlin](https://discourse.bokeh.org/u/Meir_Tseitlin)\
**Post date:** [February 13, 2017, 8:04pm UTC](https://discourse.bokeh.org/t/streaming-with-nan-values/1970/1 "2017-02-13T20:04:44Z")

</div>

I upgraded Bokeh with 0.12.4 and I think the workaround with streaming NaN values no longer works.

I am patching my DataFrame with ‘NaN’ string before turning it to dict and feeding to ColumnDataSource.stream :  
stream(df.fillna(‘NaN’).to\_dict(orient=“list”))

``

I have several line charts in gridplot and the chart with NaNs is not displaying any line. (No errors produced as well). Stream update contains a single number with NaN, I am willing to connect a line between available numbers.

Any ideas?

Thanks

BTW: Is there anything I should do to enable binary protocol instead of JSON?

---

<div class="post-metadata">

**Author:** ![Bryan](https://discourse.bokeh.org/user_avatar/discourse.bokeh.org/bryan/32/3625_2.png) [@Bryan](https://discourse.bokeh.org/u/Bryan)\
**Post date:** [February 13, 2017, 8:18pm UTC](https://discourse.bokeh.org/t/streaming-with-nan-values/1970/2 "2017-02-13T20:18:51Z")

</div>

Hi,

NumPy arrays (and thus also Pandas Series that are Float32, Float64, or Int32 dtype are automatically serialized using the binary protocol. This also no includes nans and infs. There is no need for any workaround when starting form those array types.

I can't quite tell from your message exactly what you are doing, so I can't elaborate any further. Maybe you can provide more code for more context.

Bryan

> **···**
>
> > On Feb 13, 2017, at 14:04, Meir Tseitlin \<meir@imubit.com\> wrote:
> > 
> > I upgraded Bokeh with 0.12.4 and I think the workaround with streaming NaN values no longer works.
> > 
> > I am patching my DataFrame with 'NaN' string before turning it to dict and feeding to ColumnDataSource.stream :  
> > stream(df.fillna('NaN').to\_dict(orient="list"))
> > 
> > I have several line charts in gridplot and the chart with NaNs is not displaying any line. (No errors produced as well). Stream update contains a single number with NaN, I am willing to connect a line between available numbers.
> > 
> > Any ideas?
> > 
> > Thanks
> > 
> > BTW: Is there anything I should do to enable binary protocol instead of JSON?
> > 
> > --  
> > You received this message because you are subscribed to the Google Groups "Bokeh Discussion - Public" group.  
> > To unsubscribe from this group and stop receiving emails from it, send an email to bokeh+unsubscribe@continuum.io.  
> > To post to this group, send email to bokeh@continuum.io.  
> > To view this discussion on the web visit [https://groups.google.com/a/continuum.io/d/msgid/bokeh/772d38de-e3cb-461b-93bc-2a28861b03aa%40continuum.io\](https://groups.google.com/a/continuum.io/d/msgid/bokeh/772d38de-e3cb-461b-93bc-2a28861b03aa%40continuum.io%5C).  
> > For more options, visit [https://groups.google.com/a/continuum.io/d/optout\](https://groups.google.com/a/continuum.io/d/optout%5C).

---

<div class="post-metadata">

**Author:** ![Meir\_Tseitlin](https://discourse.bokeh.org/letter_avatar_proxy/v4/letter/m/838e76/32.png) [@Meir\_Tseitlin](https://discourse.bokeh.org/u/Meir_Tseitlin)\
**Post date:** [February 13, 2017, 8:36pm UTC](https://discourse.bokeh.org/t/streaming-with-nan-values/1970/3 "2017-02-13T20:36:05Z")

</div>

Thank you Bryan,

- I am creating several figures with time-series data (each figure has a single line), I provide a single ColumnDataSource with the data to all figures (each ‘y’ has it’s own name and ‘x’=‘time’).
- I am running Bokeh server with add\_periodic\_callback, streaming additional data to ColumnDataSource
- The figure containing NaNs does not draw any line.
- My types are float64, but if I am not doing fillna(‘NaN’), I receive: “Out of range float values are not JSON compliant: nan”

Example of dict I am feeding to stream:  
{

u’var9’: [

122.876,

122.876,

122.88,

122.889,

122.885,

122.884,

122.885,

122.887,

122.875

],

u’var40’: [

‘NaN’,

‘NaN’,

53.7135,

‘NaN’,

‘NaN’,

‘NaN’,

‘NaN’,

‘NaN’,

‘NaN’

],

u’var17’: [

22.9009,

23.0699,

22.6906,

22.8626,

22.8737,

22.6161,

23.0456,

22.9609,

22.9154

],

u’var7’: [

2799.43,

2799.77,

2799.13,

2799.05,

2799.37,

2799.4,

2798.81,

2799.76,

2799.08

],

‘time’: [

Timestamp(‘2017-02-13 20:32:35.263716’),

Timestamp(‘2017-02-13 20:32:35.671465’),

Timestamp(‘2017-02-13 20:32:36.079146’),

Timestamp(‘2017-02-13 20:32:36.489926’),

Timestamp(‘2017-02-13 20:32:36.896583’),

Timestamp(‘2017-02-13 20:32:37.302591’),

Timestamp(‘2017-02-13 20:32:37.708200’),

Timestamp(‘2017-02-13 20:32:38.112995’),

Timestamp(‘2017-02-13 20:32:38.520433’)

]

}

> **···**
>
> On Monday, February 13, 2017 at 10:18:54 PM UTC+2, Bryan Van de ven wrote:
> 
> > Hi,
> > 
> > NumPy arrays (and thus also Pandas Series that are Float32, Float64, or Int32 dtype are automatically serialized using the binary protocol. This also no includes nans and infs. There is no need for any workaround when starting form those array types.
> > 
> > I can’t quite tell from your message exactly what you are doing, so I can’t elaborate any further. Maybe you can provide more code for more context.
> > 
> > Bryan
> > 
> > > On Feb 13, 2017, at 14:04, Meir Tseitlin [me...@imubit.com](mailto:me...@imubit.com) wrote:
> > 
> > > I upgraded Bokeh with 0.12.4 and I think the workaround with streaming NaN values no longer works.
> > 
> > > I am patching my DataFrame with ‘NaN’ string before turning it to dict and feeding to ColumnDataSource.stream :
> > 
> > > stream(df.fillna(‘NaN’).to\_dict(orient=“list”))
> > 
> > > I have several line charts in gridplot and the chart with NaNs is not displaying any line. (No errors produced as well). Stream update contains a single number with NaN, I am willing to connect a line between available numbers.
> > 
> > > Any ideas?
> > 
> > > Thanks
> > 
> > > BTW: Is there anything I should do to enable binary protocol instead of JSON?
> > > 
> > > –  
> > > You received this message because you are subscribed to the Google Groups “Bokeh Discussion - Public” group.
> > 
> > > To unsubscribe from this group and stop receiving emails from it, send an email to [bokeh+un...@continuum.io](mailto:bokeh+un...@continuum.io).
> > 
> > > To post to this group, send email to [bo...@continuum.io](mailto:bo...@continuum.io).
> > 
> > > To view this discussion on the web visit [https://groups.google.com/a/continuum.io/d/msgid/bokeh/772d38de-e3cb-461b-93bc-2a28861b03aa%40continuum.io](https://groups.google.com/a/continuum.io/d/msgid/bokeh/772d38de-e3cb-461b-93bc-2a28861b03aa%40continuum.io).
> > 
> > > For more options, visit [https://groups.google.com/a/continuum.io/d/optout](https://groups.google.com/a/continuum.io/d/optout).

---

<div class="post-metadata">

**Author:** ![Bryan](https://discourse.bokeh.org/user_avatar/discourse.bokeh.org/bryan/32/3625_2.png) [@Bryan](https://discourse.bokeh.org/u/Bryan)\
**Post date:** [February 13, 2017, 10:49pm UTC](https://discourse.bokeh.org/t/streaming-with-nan-values/1970/4 "2017-02-13T22:49:30Z")

</div>

The columns in your dict are python lists, not arrays. You need to construct actual NumPy arrays for the columns in order for Bokeh to be able to use the binary array protocol.

Thanks,

Bryan

> **···**
>
> > On Feb 13, 2017, at 14:36, Meir Tseitlin \<meir@imubit.com\> wrote:
> > 
> > Thank you Bryan,
> > 
> > &nbsp;&nbsp;• I am creating several figures with time-series data (each figure has a single line), I provide a single ColumnDataSource with the data to all figures (each 'y' has it's own name and 'x'='time').  
> > &nbsp;&nbsp;• I am running Bokeh server with add\_periodic\_callback, streaming additional data to ColumnDataSource  
> > &nbsp;&nbsp;• The figure containing NaNs does not draw any line.  
> > &nbsp;&nbsp;• My types are float64, but if I am not doing fillna('NaN'), I receive: "Out of range float values are not JSON compliant: nan"
> > 
> > Example of dict I am feeding to stream:  
> > {  
> > &nbsp;&nbsp;u'var9': [  
> > &nbsp;&nbsp;&nbsp;&nbsp;122.876,  
> > &nbsp;&nbsp;&nbsp;&nbsp;122.876,  
> > &nbsp;&nbsp;&nbsp;&nbsp;122.88,  
> > &nbsp;&nbsp;&nbsp;&nbsp;122.889,  
> > &nbsp;&nbsp;&nbsp;&nbsp;122.885,  
> > &nbsp;&nbsp;&nbsp;&nbsp;122.884,  
> > &nbsp;&nbsp;&nbsp;&nbsp;122.885,  
> > &nbsp;&nbsp;&nbsp;&nbsp;122.887,  
> > &nbsp;&nbsp;&nbsp;&nbsp;122.875  
> > &nbsp;&nbsp;],  
> > &nbsp;&nbsp;u'var40': [  
> > &nbsp;&nbsp;&nbsp;&nbsp;'NaN',  
> > &nbsp;&nbsp;&nbsp;&nbsp;'NaN',  
> > &nbsp;&nbsp;&nbsp;&nbsp;53.7135,  
> > &nbsp;&nbsp;&nbsp;&nbsp;'NaN',  
> > &nbsp;&nbsp;&nbsp;&nbsp;'NaN',  
> > &nbsp;&nbsp;&nbsp;&nbsp;'NaN',  
> > &nbsp;&nbsp;&nbsp;&nbsp;'NaN',  
> > &nbsp;&nbsp;&nbsp;&nbsp;'NaN',  
> > &nbsp;&nbsp;&nbsp;&nbsp;'NaN'  
> > &nbsp;&nbsp;],  
> > &nbsp;&nbsp;u'var17': [  
> > &nbsp;&nbsp;&nbsp;&nbsp;22.9009,  
> > &nbsp;&nbsp;&nbsp;&nbsp;23.0699,  
> > &nbsp;&nbsp;&nbsp;&nbsp;22.6906,  
> > &nbsp;&nbsp;&nbsp;&nbsp;22.8626,  
> > &nbsp;&nbsp;&nbsp;&nbsp;22.8737,  
> > &nbsp;&nbsp;&nbsp;&nbsp;22.6161,  
> > &nbsp;&nbsp;&nbsp;&nbsp;23.0456,  
> > &nbsp;&nbsp;&nbsp;&nbsp;22.9609,  
> > &nbsp;&nbsp;&nbsp;&nbsp;22.9154  
> > &nbsp;&nbsp;],  
> > &nbsp;&nbsp;u'var7': [  
> > &nbsp;&nbsp;&nbsp;&nbsp;2799.43,  
> > &nbsp;&nbsp;&nbsp;&nbsp;2799.77,  
> > &nbsp;&nbsp;&nbsp;&nbsp;2799.13,  
> > &nbsp;&nbsp;&nbsp;&nbsp;2799.05,  
> > &nbsp;&nbsp;&nbsp;&nbsp;2799.37,  
> > &nbsp;&nbsp;&nbsp;&nbsp;2799.4,  
> > &nbsp;&nbsp;&nbsp;&nbsp;2798.81,  
> > &nbsp;&nbsp;&nbsp;&nbsp;2799.76,  
> > &nbsp;&nbsp;&nbsp;&nbsp;2799.08  
> > &nbsp;&nbsp;],  
> > &nbsp;&nbsp;'time': [  
> > &nbsp;&nbsp;&nbsp;&nbsp;Timestamp('2017-02-13 20:32:35.263716'),  
> > &nbsp;&nbsp;&nbsp;&nbsp;Timestamp('2017-02-13 20:32:35.671465'),  
> > &nbsp;&nbsp;&nbsp;&nbsp;Timestamp('2017-02-13 20:32:36.079146'),  
> > &nbsp;&nbsp;&nbsp;&nbsp;Timestamp('2017-02-13 20:32:36.489926'),  
> > &nbsp;&nbsp;&nbsp;&nbsp;Timestamp('2017-02-13 20:32:36.896583'),  
> > &nbsp;&nbsp;&nbsp;&nbsp;Timestamp('2017-02-13 20:32:37.302591'),  
> > &nbsp;&nbsp;&nbsp;&nbsp;Timestamp('2017-02-13 20:32:37.708200'),  
> > &nbsp;&nbsp;&nbsp;&nbsp;Timestamp('2017-02-13 20:32:38.112995'),  
> > &nbsp;&nbsp;&nbsp;&nbsp;Timestamp('2017-02-13 20:32:38.520433')  
> > &nbsp;&nbsp;]  
> > }
> > 
> > On Monday, February 13, 2017 at 10:18:54 PM UTC+2, Bryan Van de ven wrote:  
> > Hi,
> > 
> > NumPy arrays (and thus also Pandas Series that are Float32, Float64, or Int32 dtype are automatically serialized using the binary protocol. This also no includes nans and infs. There is no need for any workaround when starting form those array types.
> > 
> > I can't quite tell from your message exactly what you are doing, so I can't elaborate any further. Maybe you can provide more code for more context.
> > 
> > Bryan
> > 
> > \> On Feb 13, 2017, at 14:04, Meir Tseitlin \<me...@imubit.com\> wrote:  
> > \>  
> > \> I upgraded Bokeh with 0.12.4 and I think the workaround with streaming NaN values no longer works.  
> > \>  
> > \> I am patching my DataFrame with 'NaN' string before turning it to dict and feeding to ColumnDataSource.stream :  
> > \> stream(df.fillna('NaN').to\_dict(orient="list"))  
> > \>  
> > \> I have several line charts in gridplot and the chart with NaNs is not displaying any line. (No errors produced as well). Stream update contains a single number with NaN, I am willing to connect a line between available numbers.  
> > \>  
> > \> Any ideas?  
> > \>  
> > \> Thanks  
> > \>  
> > \> BTW: Is there anything I should do to enable binary protocol instead of JSON?  
> > \>  
> > \> --  
> > \> You received this message because you are subscribed to the Google Groups "Bokeh Discussion - Public" group.  
> > \> To unsubscribe from this group and stop receiving emails from it, send an email to bokeh+un...@continuum.io.  
> > \> To post to this group, send email to bo...@continuum.io.  
> > \> To view this discussion on the web visit [https://groups.google.com/a/continuum.io/d/msgid/bokeh/772d38de-e3cb-461b-93bc-2a28861b03aa%40continuum.io\](https://groups.google.com/a/continuum.io/d/msgid/bokeh/772d38de-e3cb-461b-93bc-2a28861b03aa%40continuum.io%5C).  
> > \> For more options, visit [https://groups.google.com/a/continuum.io/d/optout\](https://groups.google.com/a/continuum.io/d/optout%5C).
> > 
> > --  
> > You received this message because you are subscribed to the Google Groups "Bokeh Discussion - Public" group.  
> > To unsubscribe from this group and stop receiving emails from it, send an email to bokeh+unsubscribe@continuum.io.  
> > To post to this group, send email to bokeh@continuum.io.  
> > To view this discussion on the web visit [https://groups.google.com/a/continuum.io/d/msgid/bokeh/6ed629f8-38de-4258-a91c-ccd7d101ffc5%40continuum.io\](https://groups.google.com/a/continuum.io/d/msgid/bokeh/6ed629f8-38de-4258-a91c-ccd7d101ffc5%40continuum.io%5C).  
> > For more options, visit [https://groups.google.com/a/continuum.io/d/optout\](https://groups.google.com/a/continuum.io/d/optout%5C).

---

<div class="post-metadata">

**Author:** ![Meir\_Tseitlin](https://discourse.bokeh.org/letter_avatar_proxy/v4/letter/m/838e76/32.png) [@Meir\_Tseitlin](https://discourse.bokeh.org/u/Meir_Tseitlin)\
**Post date:** [April 12, 2017, 10:13am UTC](https://discourse.bokeh.org/t/streaming-with-nan-values/1970/5 "2017-04-12T10:13:29Z")

</div>

Hi Bryan,

I am still struggling with this issue.

Can you provide an example of how to stream data (ColumnDataSource.stream) with numpy arrays?

Thanks

> **···**
>
> On Tuesday, February 14, 2017 at 12:49:33 AM UTC+2, Bryan Van de ven wrote:
> 
> > The columns in your dict are python lists, not arrays. You need to construct actual NumPy arrays for the columns in order for Bokeh to be able to use the binary array protocol.
> > 
> > Thanks,
> > 
> > Bryan
> > 
> > > On Feb 13, 2017, at 14:36, Meir Tseitlin [me...@imubit.com](mailto:me...@imubit.com) wrote:
> > 
> > > Thank you Bryan,
> > 
> > > ```
> > > • I am creating several figures with time-series data (each figure has a single line), I provide a single ColumnDataSource with the data to all figures (each 'y' has it's own name and 'x'='time').
> > > 
> > > ```
> > 
> > > ```
> > > • I am running Bokeh server with add_periodic_callback, streaming additional data to ColumnDataSource
> > > 
> > > ```
> > 
> > > ```
> > > • The figure containing NaNs does not draw any line.
> > > 
> > > ```
> > 
> > > ```
> > > • My types are float64, but if I am not doing fillna('NaN'), I receive: "Out of range float values are not JSON compliant: nan"
> > > 
> > > ```
> > 
> > > Example of dict I am feeding to stream:
> > 
> > > {
> > 
> > > u’var9’: [
> > 
> > > ```
> > > 122.876,
> > > 
> > > ```
> > 
> > > ```
> > > 122.876,
> > > 
> > > ```
> > 
> > > ```
> > > 122.88,
> > > 
> > > ```
> > 
> > > ```
> > > 122.889,
> > > 
> > > ```
> > 
> > > ```
> > > 122.885,
> > > 
> > > ```
> > 
> > > ```
> > > 122.884,
> > > 
> > > ```
> > 
> > > ```
> > > 122.885,
> > > 
> > > ```
> > 
> > > ```
> > > 122.887,
> > > 
> > > ```
> > 
> > > ```
> > > 122.875
> > > 
> > > ```
> > 
> > > ],
> > 
> > > u’var40’: [
> > 
> > > ```
> > > 'NaN',
> > > 
> > > ```
> > 
> > > ```
> > > 'NaN',
> > > 
> > > ```
> > 
> > > ```
> > > 53.7135,
> > > 
> > > ```
> > 
> > > ```
> > > 'NaN',
> > > 
> > > ```
> > 
> > > ```
> > > 'NaN',
> > > 
> > > ```
> > 
> > > ```
> > > 'NaN',
> > > 
> > > ```
> > 
> > > ```
> > > 'NaN',
> > > 
> > > ```
> > 
> > > ```
> > > 'NaN',
> > > 
> > > ```
> > 
> > > ```
> > > 'NaN'
> > > 
> > > ```
> > 
> > > ],
> > 
> > > u’var17’: [
> > 
> > > ```
> > > 22.9009,
> > > 
> > > ```
> > 
> > > ```
> > > 23.0699,
> > > 
> > > ```
> > 
> > > ```
> > > 22.6906,
> > > 
> > > ```
> > 
> > > ```
> > > 22.8626,
> > > 
> > > ```
> > 
> > > ```
> > > 22.8737,
> > > 
> > > ```
> > 
> > > ```
> > > 22.6161,
> > > 
> > > ```
> > 
> > > ```
> > > 23.0456,
> > > 
> > > ```
> > 
> > > ```
> > > 22.9609,
> > > 
> > > ```
> > 
> > > ```
> > > 22.9154
> > > 
> > > ```
> > 
> > > ],
> > 
> > > u’var7’: [
> > 
> > > ```
> > > 2799.43,
> > > 
> > > ```
> > 
> > > ```
> > > 2799.77,
> > > 
> > > ```
> > 
> > > ```
> > > 2799.13,
> > > 
> > > ```
> > 
> > > ```
> > > 2799.05,
> > > 
> > > ```
> > 
> > > ```
> > > 2799.37,
> > > 
> > > ```
> > 
> > > ```
> > > 2799.4,
> > > 
> > > ```
> > 
> > > ```
> > > 2798.81,
> > > 
> > > ```
> > 
> > > ```
> > > 2799.76,
> > > 
> > > ```
> > 
> > > ```
> > > 2799.08
> > > 
> > > ```
> > 
> > > ],
> > 
> > > ‘time’: [
> > 
> > > ```
> > > Timestamp('2017-02-13 20:32:35.263716'),
> > > 
> > > ```
> > 
> > > ```
> > > Timestamp('2017-02-13 20:32:35.671465'),
> > > 
> > > ```
> > 
> > > ```
> > > Timestamp('2017-02-13 20:32:36.079146'),
> > > 
> > > ```
> > 
> > > ```
> > > Timestamp('2017-02-13 20:32:36.489926'),
> > > 
> > > ```
> > 
> > > ```
> > > Timestamp('2017-02-13 20:32:36.896583'),
> > > 
> > > ```
> > 
> > > ```
> > > Timestamp('2017-02-13 20:32:37.302591'),
> > > 
> > > ```
> > 
> > > ```
> > > Timestamp('2017-02-13 20:32:37.708200'),
> > > 
> > > ```
> > 
> > > ```
> > > Timestamp('2017-02-13 20:32:38.112995'),
> > > 
> > > ```
> > 
> > > ```
> > > Timestamp('2017-02-13 20:32:38.520433')
> > > 
> > > ```
> > 
> > > ]
> > 
> > > }
> > 
> > > On Monday, February 13, 2017 at 10:18:54 PM UTC+2, Bryan Van de ven wrote:
> > 
> > > Hi,
> > > 
> > > NumPy arrays (and thus also Pandas Series that are Float32, Float64, or Int32 dtype are automatically serialized using the binary protocol. This also no includes nans and infs. There is no need for any workaround when starting form those array types.
> > > 
> > > I can’t quite tell from your message exactly what you are doing, so I can’t elaborate any further. Maybe you can provide more code for more context.
> > > 
> > > Bryan
> > > 
> > > > On Feb 13, 2017, at 14:04, Meir Tseitlin [me...@imubit.com](mailto:me...@imubit.com) wrote:
> > > > 
> > > > I upgraded Bokeh with 0.12.4 and I think the workaround with streaming NaN values no longer works.
> > > > 
> > > > I am patching my DataFrame with ‘NaN’ string before turning it to dict and feeding to ColumnDataSource.stream :  
> > > > stream(df.fillna(‘NaN’).to\_dict(orient=“list”))
> > > > 
> > > > I have several line charts in gridplot and the chart with NaNs is not displaying any line. (No errors produced as well). Stream update contains a single number with NaN, I am willing to connect a line between available numbers.
> > > > 
> > > > Any ideas?
> > > > 
> > > > Thanks
> > > > 
> > > > BTW: Is there anything I should do to enable binary protocol instead of JSON?
> > > > 
> > > > –  
> > > > You received this message because you are subscribed to the Google Groups “Bokeh Discussion - Public” group.  
> > > > To unsubscribe from this group and stop receiving emails from it, send an email to [bokeh+un...@continuum.io](mailto:bokeh+un...@continuum.io).  
> > > > To post to this group, send email to [bo...@continuum.io](mailto:bo...@continuum.io).  
> > > > To view this discussion on the web visit [https://groups.google.com/a/continuum.io/d/msgid/bokeh/772d38de-e3cb-461b-93bc-2a28861b03aa%40continuum.io](https://groups.google.com/a/continuum.io/d/msgid/bokeh/772d38de-e3cb-461b-93bc-2a28861b03aa%40continuum.io).  
> > > > For more options, visit [https://groups.google.com/a/continuum.io/d/optout](https://groups.google.com/a/continuum.io/d/optout).
> > > 
> > > –  
> > > You received this message because you are subscribed to the Google Groups “Bokeh Discussion - Public” group.
> > 
> > > To unsubscribe from this group and stop receiving emails from it, send an email to [bokeh+un...@continuum.io](mailto:bokeh+un...@continuum.io).
> > 
> > > To post to this group, send email to [bo...@continuum.io](mailto:bo...@continuum.io).
> > 
> > > To view this discussion on the web visit [https://groups.google.com/a/continuum.io/d/msgid/bokeh/6ed629f8-38de-4258-a91c-ccd7d101ffc5%40continuum.io](https://groups.google.com/a/continuum.io/d/msgid/bokeh/6ed629f8-38de-4258-a91c-ccd7d101ffc5%40continuum.io).
> > 
> > > For more options, visit [https://groups.google.com/a/continuum.io/d/optout](https://groups.google.com/a/continuum.io/d/optout).

---

<div class="post-metadata">

**Author:** ![Meir\_Tseitlin](https://discourse.bokeh.org/letter_avatar_proxy/v4/letter/m/838e76/32.png) [@Meir\_Tseitlin](https://discourse.bokeh.org/u/Meir_Tseitlin)\
**Post date:** [April 12, 2017, 11:50am UTC](https://discourse.bokeh.org/t/streaming-with-nan-values/1970/6 "2017-04-12T11:50:25Z")

</div>

I think I figured out how to send it as numpy arrays (just converted value arrays to numpy) within the dictionary (leaving time values as standard array).

Yet the effect is very similar - when one of numpy arrays contain nan - the whole update is not displayed.

> **···**
>
> On Wednesday, April 12, 2017 at 1:13:30 PM UTC+3, Meir Tseitlin wrote:
> 
> > Hi Bryan,
> 
> > I am still struggling with this issue.
> 
> > 
> 
> > Can you provide an example of how to stream data (ColumnDataSource.stream) with numpy arrays?
> 
> > 
> 
> > Thanks
> > 
> > On Tuesday, February 14, 2017 at 12:49:33 AM UTC+2, Bryan Van de ven wrote:
> > 
> > > The columns in your dict are python lists, not arrays. You need to construct actual NumPy arrays for the columns in order for Bokeh to be able to use the binary array protocol.
> > > 
> > > Thanks,
> > > 
> > > Bryan
> > > 
> > > > On Feb 13, 2017, at 14:36, Meir Tseitlin [me...@imubit.com](mailto:me...@imubit.com) wrote:
> > > 
> > > > Thank you Bryan,
> > > 
> > > > ```
> > > > • I am creating several figures with time-series data (each figure has a single line), I provide a single ColumnDataSource with the data to all figures (each 'y' has it's own name and 'x'='time').
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > • I am running Bokeh server with add_periodic_callback, streaming additional data to ColumnDataSource
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > • The figure containing NaNs does not draw any line.
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > • My types are float64, but if I am not doing fillna('NaN'), I receive: "Out of range float values are not JSON compliant: nan"
> > > > 
> > > > ```
> > > 
> > > > Example of dict I am feeding to stream:
> > > 
> > > > {
> > > 
> > > > u’var9’: [
> > > 
> > > > ```
> > > > 122.876,
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > 122.876,
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > 122.88,
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > 122.889,
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > 122.885,
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > 122.884,
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > 122.885,
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > 122.887,
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > 122.875
> > > > 
> > > > ```
> > > 
> > > > ],
> > > 
> > > > u’var40’: [
> > > 
> > > > ```
> > > > 'NaN',
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > 'NaN',
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > 53.7135,
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > 'NaN',
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > 'NaN',
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > 'NaN',
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > 'NaN',
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > 'NaN',
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > 'NaN'
> > > > 
> > > > ```
> > > 
> > > > ],
> > > 
> > > > u’var17’: [
> > > 
> > > > ```
> > > > 22.9009,
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > 23.0699,
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > 22.6906,
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > 22.8626,
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > 22.8737,
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > 22.6161,
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > 23.0456,
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > 22.9609,
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > 22.9154
> > > > 
> > > > ```
> > > 
> > > > ],
> > > 
> > > > u’var7’: [
> > > 
> > > > ```
> > > > 2799.43,
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > 2799.77,
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > 2799.13,
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > 2799.05,
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > 2799.37,
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > 2799.4,
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > 2798.81,
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > 2799.76,
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > 2799.08
> > > > 
> > > > ```
> > > 
> > > > ],
> > > 
> > > > ‘time’: [
> > > 
> > > > ```
> > > > Timestamp('2017-02-13 20:32:35.263716'),
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > Timestamp('2017-02-13 20:32:35.671465'),
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > Timestamp('2017-02-13 20:32:36.079146'),
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > Timestamp('2017-02-13 20:32:36.489926'),
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > Timestamp('2017-02-13 20:32:36.896583'),
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > Timestamp('2017-02-13 20:32:37.302591'),
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > Timestamp('2017-02-13 20:32:37.708200'),
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > Timestamp('2017-02-13 20:32:38.112995'),
> > > > 
> > > > ```
> > > 
> > > > ```
> > > > Timestamp('2017-02-13 20:32:38.520433')
> > > > 
> > > > ```
> > > 
> > > > ]
> > > 
> > > > }
> > > 
> > > > On Monday, February 13, 2017 at 10:18:54 PM UTC+2, Bryan Van de ven wrote:
> > > 
> > > > Hi,
> > > > 
> > > > NumPy arrays (and thus also Pandas Series that are Float32, Float64, or Int32 dtype are automatically serialized using the binary protocol. This also no includes nans and infs. There is no need for any workaround when starting form those array types.
> > > > 
> > > > I can’t quite tell from your message exactly what you are doing, so I can’t elaborate any further. Maybe you can provide more code for more context.
> > > > 
> > > > Bryan
> > > > 
> > > > > On Feb 13, 2017, at 14:04, Meir Tseitlin [me...@imubit.com](mailto:me...@imubit.com) wrote:
> > > > > 
> > > > > I upgraded Bokeh with 0.12.4 and I think the workaround with streaming NaN values no longer works.
> > > > > 
> > > > > I am patching my DataFrame with ‘NaN’ string before turning it to dict and feeding to ColumnDataSource.stream :  
> > > > > stream(df.fillna(‘NaN’).to\_dict(orient=“list”))
> > > > > 
> > > > > I have several line charts in gridplot and the chart with NaNs is not displaying any line. (No errors produced as well). Stream update contains a single number with NaN, I am willing to connect a line between available numbers.
> > > > > 
> > > > > Any ideas?
> > > > > 
> > > > > Thanks
> > > > > 
> > > > > BTW: Is there anything I should do to enable binary protocol instead of JSON?
> > > > > 
> > > > > –  
> > > > > You received this message because you are subscribed to the Google Groups “Bokeh Discussion - Public” group.  
> > > > > To unsubscribe from this group and stop receiving emails from it, send an email to [bokeh+un...@continuum.io](mailto:bokeh+un...@continuum.io).  
> > > > > To post to this group, send email to [bo...@continuum.io](mailto:bo...@continuum.io).  
> > > > > To view this discussion on the web visit [https://groups.google.com/a/continuum.io/d/msgid/bokeh/772d38de-e3cb-461b-93bc-2a28861b03aa%40continuum.io](https://groups.google.com/a/continuum.io/d/msgid/bokeh/772d38de-e3cb-461b-93bc-2a28861b03aa%40continuum.io).  
> > > > > For more options, visit [https://groups.google.com/a/continuum.io/d/optout](https://groups.google.com/a/continuum.io/d/optout).
> > > > 
> > > > –  
> > > > You received this message because you are subscribed to the Google Groups “Bokeh Discussion - Public” group.
> > > 
> > > > To unsubscribe from this group and stop receiving emails from it, send an email to [bokeh+un...@continuum.io](mailto:bokeh+un...@continuum.io).
> > > 
> > > > To post to this group, send email to [bo...@continuum.io](mailto:bo...@continuum.io).
> > > 
> > > > To view this discussion on the web visit [https://groups.google.com/a/continuum.io/d/msgid/bokeh/6ed629f8-38de-4258-a91c-ccd7d101ffc5%40continuum.io](https://groups.google.com/a/continuum.io/d/msgid/bokeh/6ed629f8-38de-4258-a91c-ccd7d101ffc5%40continuum.io).
> > > 
> > > > For more options, visit [https://groups.google.com/a/continuum.io/d/optout](https://groups.google.com/a/continuum.io/d/optout).

---

<div class="post-metadata">

**Author:** ![Bryan](https://discourse.bokeh.org/user_avatar/discourse.bokeh.org/bryan/32/3625_2.png) [@Bryan](https://discourse.bokeh.org/u/Bryan)\
**Post date:** [April 22, 2017, 10:02pm UTC](https://discourse.bokeh.org/t/streaming-with-nan-values/1970/7 "2017-04-22T22:02:38Z")

</div>

Yes, this is expected and also beyond our control. There is no support in the JSON specification for "nan" values. So if you use lists (which get converted to JSON) then things will not work. However if you use numpy arrays then they get encoded in a different way that allows for nan's to be communicated properly. If you need to stream nan's then you need to put the data in a numpy array.

Thanks,

Bryan

> **···**
>
> > On Apr 12, 2017, at 06:50, Meir Tseitlin \<meir@imubit.com\> wrote:
> > 
> > I think I figured out how to send it as numpy arrays (just converted value arrays to numpy) within the dictionary (leaving time values as standard array).
> > 
> > Yet the effect is very similar - when one of numpy arrays contain nan - the whole update is not displayed.
> > 
> > On Wednesday, April 12, 2017 at 1:13:30 PM UTC+3, Meir Tseitlin wrote:  
> > Hi Bryan,
> > 
> > I am still struggling with this issue.
> > 
> > Can you provide an example of how to stream data (ColumnDataSource.stream) with numpy arrays?
> > 
> > Thanks
> > 
> > On Tuesday, February 14, 2017 at 12:49:33 AM UTC+2, Bryan Van de ven wrote:  
> > The columns in your dict are python lists, not arrays. You need to construct actual NumPy arrays for the columns in order for Bokeh to be able to use the binary array protocol.
> > 
> > Thanks,
> > 
> > Bryan
> > 
> > \> On Feb 13, 2017, at 14:36, Meir Tseitlin \<me...@imubit.com\> wrote:  
> > \>  
> > \> Thank you Bryan,  
> > \>  
> > \> • I am creating several figures with time-series data (each figure has a single line), I provide a single ColumnDataSource with the data to all figures (each 'y' has it's own name and 'x'='time').  
> > \> • I am running Bokeh server with add\_periodic\_callback, streaming additional data to ColumnDataSource  
> > \> • The figure containing NaNs does not draw any line.  
> > \> • My types are float64, but if I am not doing fillna('NaN'), I receive: "Out of range float values are not JSON compliant: nan"  
> > \>  
> > \> Example of dict I am feeding to stream:  
> > \> {  
> > \> u'var9': [  
> > \> 122.876,  
> > \> 122.876,  
> > \> 122.88,  
> > \> 122.889,  
> > \> 122.885,  
> > \> 122.884,  
> > \> 122.885,  
> > \> 122.887,  
> > \> 122.875  
> > \> ],  
> > \> u'var40': [  
> > \> 'NaN',  
> > \> 'NaN',  
> > \> 53.7135,  
> > \> 'NaN',  
> > \> 'NaN',  
> > \> 'NaN',  
> > \> 'NaN',  
> > \> 'NaN',  
> > \> 'NaN'  
> > \> ],  
> > \> u'var17': [  
> > \> 22.9009,  
> > \> 23.0699,  
> > \> 22.6906,  
> > \> 22.8626,  
> > \> 22.8737,  
> > \> 22.6161,  
> > \> 23.0456,  
> > \> 22.9609,  
> > \> 22.9154  
> > \> ],  
> > \> u'var7': [  
> > \> 2799.43,  
> > \> 2799.77,  
> > \> 2799.13,  
> > \> 2799.05,  
> > \> 2799.37,  
> > \> 2799.4,  
> > \> 2798.81,  
> > \> 2799.76,  
> > \> 2799.08  
> > \> ],  
> > \> 'time': [  
> > \> Timestamp('2017-02-13 20:32:35.263716'),  
> > \> Timestamp('2017-02-13 20:32:35.671465'),  
> > \> Timestamp('2017-02-13 20:32:36.079146'),  
> > \> Timestamp('2017-02-13 20:32:36.489926'),  
> > \> Timestamp('2017-02-13 20:32:36.896583'),  
> > \> Timestamp('2017-02-13 20:32:37.302591'),  
> > \> Timestamp('2017-02-13 20:32:37.708200'),  
> > \> Timestamp('2017-02-13 20:32:38.112995'),  
> > \> Timestamp('2017-02-13 20:32:38.520433')  
> > \> ]  
> > \> }  
> > \>  
> > \>  
> > \> On Monday, February 13, 2017 at 10:18:54 PM UTC+2, Bryan Van de ven wrote:  
> > \> Hi,  
> > \>  
> > \> NumPy arrays (and thus also Pandas Series that are Float32, Float64, or Int32 dtype are automatically serialized using the binary protocol. This also no includes nans and infs. There is no need for any workaround when starting form those array types.  
> > \>  
> > \> I can't quite tell from your message exactly what you are doing, so I can't elaborate any further. Maybe you can provide more code for more context.  
> > \>  
> > \> Bryan  
> > \>  
> > \> \> On Feb 13, 2017, at 14:04, Meir Tseitlin \<me...@imubit.com\> wrote:  
> > \> \>  
> > \> \> I upgraded Bokeh with 0.12.4 and I think the workaround with streaming NaN values no longer works.  
> > \> \>  
> > \> \> I am patching my DataFrame with 'NaN' string before turning it to dict and feeding to ColumnDataSource.stream :  
> > \> \> stream(df.fillna('NaN').to\_dict(orient="list"))  
> > \> \>  
> > \> \> I have several line charts in gridplot and the chart with NaNs is not displaying any line. (No errors produced as well). Stream update contains a single number with NaN, I am willing to connect a line between available numbers.  
> > \> \>  
> > \> \> Any ideas?  
> > \> \>  
> > \> \> Thanks  
> > \> \>  
> > \> \> BTW: Is there anything I should do to enable binary protocol instead of JSON?  
> > \> \>  
> > \> \> --  
> > \> \> You received this message because you are subscribed to the Google Groups "Bokeh Discussion - Public" group.  
> > \> \> To unsubscribe from this group and stop receiving emails from it, send an email to bokeh+un...@continuum.io.  
> > \> \> To post to this group, send email to bo...@continuum.io.  
> > \> \> To view this discussion on the web visit [https://groups.google.com/a/continuum.io/d/msgid/bokeh/772d38de-e3cb-461b-93bc-2a28861b03aa%40continuum.io\](https://groups.google.com/a/continuum.io/d/msgid/bokeh/772d38de-e3cb-461b-93bc-2a28861b03aa%40continuum.io%5C).  
> > \> \> For more options, visit [https://groups.google.com/a/continuum.io/d/optout\](https://groups.google.com/a/continuum.io/d/optout%5C).  
> > \>  
> > \>  
> > \> --  
> > \> You received this message because you are subscribed to the Google Groups "Bokeh Discussion - Public" group.  
> > \> To unsubscribe from this group and stop receiving emails from it, send an email to bokeh+un...@continuum.io.  
> > \> To post to this group, send email to bo...@continuum.io.  
> > \> To view this discussion on the web visit [https://groups.google.com/a/continuum.io/d/msgid/bokeh/6ed629f8-38de-4258-a91c-ccd7d101ffc5%40continuum.io\](https://groups.google.com/a/continuum.io/d/msgid/bokeh/6ed629f8-38de-4258-a91c-ccd7d101ffc5%40continuum.io%5C).  
> > \> For more options, visit [https://groups.google.com/a/continuum.io/d/optout\](https://groups.google.com/a/continuum.io/d/optout%5C).
> > 
> > --  
> > You received this message because you are subscribed to the Google Groups "Bokeh Discussion - Public" group.  
> > To unsubscribe from this group and stop receiving emails from it, send an email to bokeh+unsubscribe@continuum.io.  
> > To post to this group, send email to bokeh@continuum.io.  
> > To view this discussion on the web visit [https://groups.google.com/a/continuum.io/d/msgid/bokeh/0a2008db-0046-46d1-943b-3644576fd36f%40continuum.io\](https://groups.google.com/a/continuum.io/d/msgid/bokeh/0a2008db-0046-46d1-943b-3644576fd36f%40continuum.io%5C).  
> > For more options, visit [https://groups.google.com/a/continuum.io/d/optout\](https://groups.google.com/a/continuum.io/d/optout%5C).

---

<div class="post-metadata">

**Author:** ![system](https://discourse-uploads.bokeh.org/original/2X/1/14eb24f9e10e4e9b78e3f10f66c04ffa96446e5a.svg) [@system](https://discourse.bokeh.org/u/system)\
**Post date:** [July 21, 2021, 9:11pm UTC](https://discourse.bokeh.org/t/streaming-with-nan-values/1970/8 "2021-07-21T21:11:10Z")

</div>