HoverTool displaying image

Juan_Nunez-Iglesias · May 3, 2016, 6:58am

Hi,

I'm making a 2D scatter plot in which every point corresponds to an image in a dataset. Is it possible to make a HoverTool display the image (preferably colormapped)?

Thanks!

Juan.

Bryan · May 3, 2016, 9:38am

Hi Juan,

Can you elaborate, are the images available via a URL (e.g png's or jpg's)? If so, there is an example already in the user's guide:

Bokeh Docs

Or are you asking about rendering image *plots* inside tooltips? If so, that might be possible, although I am not aware of it ever being demonstrated.

Bryan

···

On May 3, 2016, at 1:58 AM, Juan Nunez-Iglesias <[email protected]> wrote:

Hi,

I'm making a 2D scatter plot in which every point corresponds to an image in a dataset. Is it possible to make a HoverTool display the image (preferably colormapped)?

Thanks!

Juan.

--
You received this message because you are subscribed to the Google Groups "Bokeh Discussion - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/etPan.57284c0c.3e483177.6231%40jni-lscc-mbp.local\.
For more options, visit https://groups.google.com/a/continuum.io/d/optout\.

Juan_Nunez-Iglesias · May 4, 2016, 1:00am

Hi Bryan!

Thanks for the (as always) prompt response! The images are numpy arrays in-memory, but it would not be too terrible to colormap them, save them to disk, and then use file:/// URLs.

A general solution with image plots (or even any plot!) would be welcome, but for now I will use the file-save workaround. Thank you!

Juan.

···

On Tuesday, May 3, 2016 at 7:38:21 PM UTC+10, Bryan Van de ven wrote:

Hi Juan,

Can you elaborate, are the images available via a URL (e.g png’s or jpg’s)? If so, there is an example already in the user’s guide:
    [http://bokeh.pydata.org/en/0.11.1/docs/user_guide/tools.html#custom-tooltip](http://bokeh.pydata.org/en/0.11.1/docs/user_guide/tools.html#custom-tooltip)
Or are you asking about rendering image plots inside tooltips? If so, that might be possible, although I am not aware of it ever being demonstrated.

Bryan

On May 3, 2016, at 1:58 AM, Juan Nunez-Iglesias [email protected] wrote:

Hi,

I’m making a 2D scatter plot in which every point corresponds to an image in a dataset. Is it possible to make a HoverTool display the image (preferably colormapped)?

Thanks!

Juan.

–
You received this message because you are subscribed to the Google Groups “Bokeh Discussion - Public” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To post to this group, send email to [email protected].

To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/etPan.57284c0c.3e483177.6231%40jni-lscc-mbp.local.

For more options, visit https://groups.google.com/a/continuum.io/d/optout.

Mateusz_Paprocki · May 4, 2016, 9:02am

Hi,

···

On Wed, May 4, 2016 at 3:00 AM, Juan Nunez-Iglesias [email protected] wrote:

Hi Bryan!

Thanks for the (as always) prompt response! The images are numpy arrays in-memory, but it would not be too terrible to colormap them, save them to disk, and then use file:/// URLs.

you can skip saving to disk part by base64 encoding images. See [1] as an example, just instead of creating ImageURL glyph, you will be using appropriate data source’s field name (urls in this example) in tooltips. Still you have to do color mapping on your own. Alternatively, you could display images on a separate plot and use HoverTools’ callback to change image(s) when inspected data points change. Then you wouldn’t have to do color mapping, but I think you don’t want to have to plots on-screen. Technically it should be possible to embed plot in a tooltip, but I think we didn’t attempt to do this yet.

Mateusz

A general solution with image plots (or even any plot!) would be welcome, but for now I will use the file-save workaround. Thank you!

Juan.

On Tuesday, May 3, 2016 at 7:38:21 PM UTC+10, Bryan Van de ven wrote:
Hi Juan,

Can you elaborate, are the images available via a URL (e.g png’s or jpg’s)? If so, there is an example already in the user’s guide:
    [http://bokeh.pydata.org/en/0.11.1/docs/user_guide/tools.html#custom-tooltip](http://bokeh.pydata.org/en/0.11.1/docs/user_guide/tools.html#custom-tooltip)
Or are you asking about rendering image plots inside tooltips? If so, that might be possible, although I am not aware of it ever being demonstrated.

Bryan

On May 3, 2016, at 1:58 AM, Juan Nunez-Iglesias [email protected] wrote:

Hi,

I’m making a 2D scatter plot in which every point corresponds to an image in a dataset. Is it possible to make a HoverTool display the image (preferably colormapped)?

Thanks!

Juan.

–
You received this message because you are subscribed to the Google Groups “Bokeh Discussion - Public” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To post to this group, send email to [email protected].

To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/etPan.57284c0c.3e483177.6231%40jni-lscc-mbp.local.

For more options, visit https://groups.google.com/a/continuum.io/d/optout.

–

You received this message because you are subscribed to the Google Groups “Bokeh Discussion - Public” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To post to this group, send email to [email protected].

To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/64f3d270-990d-498e-9935-e9706c8295da%40continuum.io.

For more options, visit https://groups.google.com/a/continuum.io/d/optout.

Mateusz_Paprocki · May 4, 2016, 11:43am

Hi,

Hi Bryan!

Thanks for the (as always) prompt response! The images are numpy arrays
in-memory, but it would not be too terrible to colormap them, save them to
disk, and then use file:/// URLs.

you can skip saving to disk part by base64 encoding images. See [1] as an
example, just instead of creating ImageURL glyph, you will be using
appropriate data source's field name (`urls` in this example) in tooltips.
Still you have to do color mapping on your own. Alternatively, you could
display images on a separate plot and use HoverTools' callback to change
image(s) when inspected data points change. Then you wouldn't have to do
color mapping, but I think you don't want to have to plots on-screen.
Technically it should be possible to embed plot in a tooltip, but I think
we didn't attempt to do this yet.

[1] https://github.com/bokeh/bokeh/blob/master/examples/models/donut.py#L95

···

On Wed, May 4, 2016 at 11:02 AM, Mateusz Paprocki < [email protected]> wrote:

On Wed, May 4, 2016 at 3:00 AM, Juan Nunez-Iglesias <[email protected]> > wrote:

Mateusz

A general solution with image plots (or even *any* plot!) would be
welcome, but for now I will use the file-save workaround. Thank you!

Juan.

On Tuesday, May 3, 2016 at 7:38:21 PM UTC+10, Bryan Van de ven wrote:

Hi Juan,

Can you elaborate, are the images available via a URL (e.g png's or
jpg's)? If so, there is an example already in the user's guide:

Bokeh Docs

Or are you asking about rendering image *plots* inside tooltips? If so,
that might be possible, although I am not aware of it ever being
demonstrated.

Bryan

> On May 3, 2016, at 1:58 AM, Juan Nunez-Iglesias <[email protected]> >>> wrote:
>
> Hi,
>
> I'm making a 2D scatter plot in which every point corresponds to an
image in a dataset. Is it possible to make a HoverTool display the image
(preferably colormapped)?
>
> Thanks!
>
> Juan.
>
>
> --
> You received this message because you are subscribed to the Google
Groups "Bokeh Discussion - Public" group.
> To unsubscribe from this group and stop receiving emails from it, send
an email to bokeh+un...@continuum.io.
> To post to this group, send email to bo...@continuum.io.
> To view this discussion on the web visit
https://groups.google.com/a/continuum.io/d/msgid/bokeh/etPan.57284c0c.3e483177.6231%40jni-lscc-mbp.local\.

> For more options, visit
https://groups.google.com/a/continuum.io/d/optout\.

--

You received this message because you are subscribed to the Google Groups
"Bokeh Discussion - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/a/continuum.io/d/msgid/bokeh/64f3d270-990d-498e-9935-e9706c8295da%40continuum.io
<https://groups.google.com/a/continuum.io/d/msgid/bokeh/64f3d270-990d-498e-9935-e9706c8295da%40continuum.io?utm_medium=email&utm_source=footer>
.

For more options, visit https://groups.google.com/a/continuum.io/d/optout
.

Mateusz_Paprocki · May 4, 2016, 12:14pm

Hi,

Hi Bryan!

Thanks for the (as always) prompt response! The images are numpy arrays
in-memory, but it would not be too terrible to colormap them, save them to
disk, and then use file:/// URLs.

you can skip saving to disk part by base64 encoding images. See [1] as an
example, just instead of creating ImageURL glyph, you will be using
appropriate data source's field name (`urls` in this example) in tooltips.
Still you have to do color mapping on your own. Alternatively, you could
display images on a separate plot and use HoverTools' callback to change
image(s) when inspected data points change.

Show how to connect two bokeh plots with hover tool · GitHub shows how
to link two plots with hover tool.

Then you wouldn't have to do color mapping, but I think you don't want to
have to plots on-screen. Technically it should be possible to embed plot in
a tooltip, but I think we didn't attempt to do this yet.

After some investigation, this is not trivial to implement currently.

Mateusz

···

On Wed, May 4, 2016 at 11:02 AM, Mateusz Paprocki < [email protected]> wrote:

On Wed, May 4, 2016 at 3:00 AM, Juan Nunez-Iglesias <[email protected]> > wrote:

Mateusz

A general solution with image plots (or even *any* plot!) would be
welcome, but for now I will use the file-save workaround. Thank you!

Juan.

On Tuesday, May 3, 2016 at 7:38:21 PM UTC+10, Bryan Van de ven wrote:

Hi Juan,

Can you elaborate, are the images available via a URL (e.g png's or
jpg's)? If so, there is an example already in the user's guide:

Bokeh Docs

Or are you asking about rendering image *plots* inside tooltips? If so,
that might be possible, although I am not aware of it ever being
demonstrated.

Bryan

> On May 3, 2016, at 1:58 AM, Juan Nunez-Iglesias <[email protected]> >>> wrote:
>
> Hi,
>
> I'm making a 2D scatter plot in which every point corresponds to an
image in a dataset. Is it possible to make a HoverTool display the image
(preferably colormapped)?
>
> Thanks!
>
> Juan.
>
>
> --
> You received this message because you are subscribed to the Google
Groups "Bokeh Discussion - Public" group.
> To unsubscribe from this group and stop receiving emails from it, send
an email to bokeh+un...@continuum.io.
> To post to this group, send email to bo...@continuum.io.
> To view this discussion on the web visit
https://groups.google.com/a/continuum.io/d/msgid/bokeh/etPan.57284c0c.3e483177.6231%40jni-lscc-mbp.local\.

> For more options, visit
https://groups.google.com/a/continuum.io/d/optout\.

--

You received this message because you are subscribed to the Google Groups
"Bokeh Discussion - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/a/continuum.io/d/msgid/bokeh/64f3d270-990d-498e-9935-e9706c8295da%40continuum.io
<https://groups.google.com/a/continuum.io/d/msgid/bokeh/64f3d270-990d-498e-9935-e9706c8295da%40continuum.io?utm_medium=email&utm_source=footer>
.

For more options, visit https://groups.google.com/a/continuum.io/d/optout
.

Juan_Nunez-Iglesias · May 10, 2016, 8:16am

Thanks, Mateusz! The b64 encoding works great and is definitely cleaner than the temporary directory approach! I have the code working here:

github.com

jni/blob-explorer/blob/master/picker.py#L119


      
          def b64_image_files(images, colormap='magma'):
              cmap = cm.get_cmap(colormap)
              urls = []
              for im in images:
                  png = to_png(img_as_ubyte(cmap(im)))
                  url = 'data:image/png;base64,' + base64.b64encode(png).decode('utf-8')
                  urls.append(url)
              return urls
          
          
          def bokeh_plot(df):
              tooltip = """
                  <div>
                      <div>
                          <img
                          src="@image_files" height="60" alt="image"
                          style="float: left; margin: 0px 15px 15px 0px; image-rendering: pixelated;"
                          border="2"
                          ></img>
                      </div>
                      <div>

It’s a bit messy, but after some cleanup and with some sample data it could make a good example plot! Here’s a teaser screenshot:

Thank you both for your help and for all the work that’s led to this!

Juan.

···

On Wednesday, May 4, 2016 at 10:14:27 PM UTC+10, mateusz.paprocki wrote:

On Wed, May 4, 2016 at 11:02 AM, Mateusz Paprocki [email protected] wrote:

Hi,

On Wed, May 4, 2016 at 3:00 AM, Juan Nunez-Iglesias [email protected] wrote:

Hi Bryan!

Thanks for the (as always) prompt response! The images are numpy arrays in-memory, but it would not be too terrible to colormap them, save them to disk, and then use file:/// URLs.

you can skip saving to disk part by base64 encoding images. See [1] as an example, just instead of creating ImageURL glyph, you will be using appropriate data source’s field name (urls in this example) in tooltips. Still you have to do color mapping on your own. Alternatively, you could display images on a separate plot and use HoverTools’ callback to change image(s) when inspected data points change.

https://gist.github.com/mattpap/eb15e406be17de249a8b7b6e30cf93ab shows how to link two plots with hover tool.

Then you wouldn’t have to do color mapping, but I think you don’t want to have to plots on-screen. Technically it should be possible to embed plot in a tooltip, but I think we didn’t attempt to do this yet.

After some investigation, this is not trivial to implement currently.

Mateusz

A general solution with image plots (or even any plot!) would be welcome, but for now I will use the file-save workaround. Thank you!

Juan.

On Tuesday, May 3, 2016 at 7:38:21 PM UTC+10, Bryan Van de ven wrote:
Hi Juan,

Can you elaborate, are the images available via a URL (e.g png’s or jpg’s)? If so, there is an example already in the user’s guide:
    [http://bokeh.pydata.org/en/0.11.1/docs/user_guide/tools.html#custom-tooltip](http://bokeh.pydata.org/en/0.11.1/docs/user_guide/tools.html#custom-tooltip)
Or are you asking about rendering image plots inside tooltips? If so, that might be possible, although I am not aware of it ever being demonstrated.

Bryan

On May 3, 2016, at 1:58 AM, Juan Nunez-Iglesias [email protected] wrote:

Hi,

I’m making a 2D scatter plot in which every point corresponds to an image in a dataset. Is it possible to make a HoverTool display the image (preferably colormapped)?

Thanks!

Juan.

–
You received this message because you are subscribed to the Google Groups “Bokeh Discussion - Public” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To post to this group, send email to [email protected].

To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/etPan.57284c0c.3e483177.6231%40jni-lscc-mbp.local.

For more options, visit https://groups.google.com/a/continuum.io/d/optout.

–

You received this message because you are subscribed to the Google Groups “Bokeh Discussion - Public” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To post to this group, send email to [email protected].

To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/64f3d270-990d-498e-9935-e9706c8295da%40continuum.io.

For more options, visit https://groups.google.com/a/continuum.io/d/optout.

hectormz · April 17, 2020, 2:48pm

@Juan_Nunez-Iglesias I happened upon this yesterday and it was exactly what I needed. Funny coincidence. Thanks for the github repo too!

Coddy · July 29, 2020, 7:41pm

Hey what’s the purpose of
blob-explorer/picker.py at bd9fa676a2a23317e2ea84bdf48b19e71b9e75d4 · jni/blob-explorer · GitHub

Why can’t you achieve that with .tobytes() method in PIL package.

Juan_Nunez-Iglesias · July 30, 2020, 5:22am

The answer is generally “I didn’t know any better”, but in the tobytes() documentation, it says:

This method returns the raw image data from the internal storage. For compressed image data (e.g. PNG, JPEG) use save() , with a BytesIO parameter for in-memory data.

Which is exactly what the linked line does?

Coddy · July 31, 2020, 3:07pm

def to_png(arr):
    out = BytesIO()
    im = Image.fromarray(arr, 'RGB')
    im.save(out, format='png')
    return out.getvalue()

def b64_image_files(chosen_indices):
    urls = []
    for i in range(len(np.take(train_set.data, chosen_indices, axis=0))):
        png = to_png(np.take(train_set.data, chosen_indices, axis=0)[i])
        url = 'data:image/png;base64,' + base64.b64encode(png).decode('utf-8')
        urls.append(url)
    return urls

Coddy · January 15, 2021, 3:51pm

@Bryan This answer needs to be in the documentation of image_url API. It will save a lot of human hours worth of work.

carolyn · January 15, 2021, 4:21pm

Hi @Coddy,

PRs with documentation updates + examples are always welcome! If you’re not comfortable submitting a PR yourself, please create a GH issue to track the documentation update request.

Bryan · January 15, 2021, 5:33pm

@Coddy OSS projects usually have limited resources relative to potential work, and so have to prioritize out of necessity. As best I can tell after nearly nine years of supporting Bokeh, very few people ever actually want to do what is described in this thread. As a result, it has not risen as a priority.

The best way to “move the needle” on things like this is just as @carolyn suggests: to add your contributions and priorities to the collaborative effort. Besides adding content to the narrative docs, another possibility with perhaps the lowest barrier to entry would be to contribute a simple, focused example to add to the examples/plotting/file directory of the repo. That could server as an immediate and easily linkable reference any time someone has a similar question.