# Access box\_select attributes set in figure

**URL:** <https://discourse.bokeh.org/t/access-box-select-attributes-set-in-figure/8696>\
**Category:** Community Support\
**Created:** [December 1, 2021, 11:54am UTC](https://discourse.bokeh.org/t/access-box-select-attributes-set-in-figure/8696 "2021-12-01T11:54:31Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![Wall.E](https://discourse.bokeh.org/user_avatar/discourse.bokeh.org/wall.e/32/3759_2.png) [@Wall.E](https://discourse.bokeh.org/u/Wall.E)\
**Post date:** [December 1, 2021, 11:54am UTC](https://discourse.bokeh.org/t/access-box-select-attributes-set-in-figure/8696/1 "2021-12-01T11:54:31Z")

</div>

I’m trying to get the image `(x, y)` coordinates hovered by hand-drawn rectangle over an image displayed with `figure.image()`. The app runs in Jupyter Lab, with synchronization.  
To do that I’m trying to use the `box_select` tool set in the `tools` input of a `figure` object.

```auto
    TOOLTIPS = [
        ("x", "$x"),
        ("y", "$y"),
        ("value", "@image"),
    ]
    tools = ['box_select', 'hover', 'reset']
    # Create the figure axis
    fig = figure(
               tooltips=TOOLTIPS,
               tools=tools)

```

At first, I tried using `data_source.selected.on_change('indices', callback)` but that does not seem to trigger anything with an image. It only works when selecting glyphs in e.g. scatter plots.  
The [topic closest to this question](https://discourse.bokeh.org/t/bokeh-how-to-access-the-coordinates-of-the-box-select-tool-rect/2012) does not seem to apply anymore, as `fig.tool_events` issues the error that `tool_events` is an unexpected attribute.

Any idea how I can get the image coordinates of the selection rectangle?

---

<div class="post-metadata">

**Author:** ![gmerritt123](https://discourse.bokeh.org/user_avatar/discourse.bokeh.org/gmerritt123/32/4101_2.png) [@gmerritt123](https://discourse.bokeh.org/u/gmerritt123)\
**Post date:** [December 1, 2021, 1:52pm UTC](https://discourse.bokeh.org/t/access-box-select-attributes-set-in-figure/8696/2 "2021-12-01T13:52:50Z")

</div>

Relevant: [Correct way to use Selection (Lasso, Box) on an Image - #7 by Bryan](https://discourse.bokeh.org/t/correct-way-to-use-selection-lasso-box-on-an-image/7552/7)

Additional question regarding your use case: 1) Do you just want the xy coords of an arbitrarily drawn box, or do you need the image “pixel” values contained in that selection?

Former would be pretty trivial with the draw tools, latter more complicated as you’d need to slice and dice a 2d array (as that’s how the image pixels are stored).

---

<div class="post-metadata">

**Author:** ![Wall.E](https://discourse.bokeh.org/user_avatar/discourse.bokeh.org/wall.e/32/3759_2.png) [@Wall.E](https://discourse.bokeh.org/u/Wall.E)\
**Post date:** [December 1, 2021, 2:22pm UTC](https://discourse.bokeh.org/t/access-box-select-attributes-set-in-figure/8696/3 "2021-12-01T14:22:07Z")

</div>

The former. xy coords of the drawn rectangle in the plotted image axes coordinates. Can be a top left vertex and width,height or two opposite vertices. I plot the image this way:

```auto
source = ColumnDataSource({'image': [samples[0]]})
fig = figure(x_axis_label='X [px]',
               y_axis_label='Y [px]',
               tooltips=TOOLTIPS,
               tools=tools)
im = fig.image(source=source, x=0, y=0, dw=600, dh=659)

```

where “samples” is just a list of 2D numpy arrays and my image arrays have dimensions of `dw` and `dh` pixels (here, 659 x 600 px).

 ![2021-12-01 09_09_45-STEREO_Bokeh_tr (7) - JupyterLab](https://discourse-uploads.bokeh.org/original/2X/a/a2d0054365c13dfbe5f8efb74236d2eee4f4b0c9.jpeg)

At your [linked topic](https://discourse.bokeh.org/t/correct-way-to-use-selection-lasso-box-on-an-image/7552/3), I see a `SelectionGeometry` mentionned but I don’t see how this can be used in combination with the `BoxSelectTool`.

As a simple workaround, I have used the `Tap` event which directly gives the clicked point in the x,y axes coordinates. Doing it twice can give me a rectangle but that’s not great. Then I round them up to integers and I can use them to work with my image array.

```auto
    def callback_tap(event):
        coords=(event.x,event.y)
        print(coords)
    fig.on_event(Tap, callback_tap)

```

---

<div class="post-metadata">

**Author:** ![gmerritt123](https://discourse.bokeh.org/user_avatar/discourse.bokeh.org/gmerritt123/32/4101_2.png) [@gmerritt123](https://discourse.bokeh.org/u/gmerritt123)\
**Post date:** [December 1, 2021, 2:27pm UTC](https://discourse.bokeh.org/t/access-box-select-attributes-set-in-figure/8696/4 "2021-12-01T14:27:02Z")

</div>

Right, I was linking to that just for reading up on the complexity of selecting image pixels (it can be done, but it’s more complicated than people seem to think).

Check out the BoxEditTool, this is what you’re after: [Configuring plot tools — Bokeh 2.4.2 Documentation](https://docs.bokeh.org/en/latest/docs/user_guide/tools.html#edit-tools)

Basically you create an empty CDS, drive a rect glyph with it (so you can control the alpha/formatting of it etc), and assign the box edit tool to be able to add to it. You can also specify the max number, so for you just limit it to one (so user draws one box at a time). Then with a callback retrieve the values that the user just added to the CDS and bam you have the coords of the box they drew.

---

<div class="post-metadata">

**Author:** ![Wall.E](https://discourse.bokeh.org/user_avatar/discourse.bokeh.org/wall.e/32/3759_2.png) [@Wall.E](https://discourse.bokeh.org/u/Wall.E)\
**Post date:** [December 1, 2021, 4:25pm UTC](https://discourse.bokeh.org/t/access-box-select-attributes-set-in-figure/8696/5 "2021-12-01T16:25:12Z")

</div>

Ok, I tried the `BoxEditTool` but the documentation would really need an example.

I tried this minimalistic example, but the rectangle won’t show dragging the mouse with the tool.

```auto
from bokeh.plotting import figure
from bokeh.layouts import column
from bokeh.models import ColumnDataSource, BoxEditTool
from bokeh.io import show
output_notebook()

def bkapp(doc):

    source0 = ColumnDataSource(data=dict(
    x0=[1, 2, 3, 4, 5],
    y0=[2, 5, 8, 2, 7],
    ))

    tools = ['hover', 'reset']

    plot = figure(x_axis_label='x',
                  y_axis_label='y)',
                  title="Test BoxEditTool",
                  tools=tools)
    
    source1 = ColumnDataSource({'x':[], 'y':[], 'width':[], 'height':[]})
    source2 = ColumnDataSource({'x':[], 'y':[], 'width':[], 'height':[]})

    r1 = plot.rect('x', 'y', 'width', 'height', source=source1)
    r2 = plot.rect('x', 'y', 'width', 'height', source=source2)
    tool = BoxEditTool(renderers=[r1, r2], num_objects=2)
    plot.add_tools(tool)
    
    # Add the data renderer (here a scatter plot)
    scatter = plot.scatter(x='x0', y='y0', source=source0)

    doc.add_root(column(plot))
    
show(bkapp)

```

---

<div class="post-metadata">

**Author:** ![gmerritt123](https://discourse.bokeh.org/user_avatar/discourse.bokeh.org/gmerritt123/32/4101_2.png) [@gmerritt123](https://discourse.bokeh.org/u/gmerritt123)\
**Post date:** [December 1, 2021, 4:31pm UTC](https://discourse.bokeh.org/t/access-box-select-attributes-set-in-figure/8696/6 "2021-12-01T16:31:00Z")

</div>

hold shift my friend 😃

---

<div class="post-metadata">

**Author:** ![gmerritt123](https://discourse.bokeh.org/user_avatar/discourse.bokeh.org/gmerritt123/32/4101_2.png) [@gmerritt123](https://discourse.bokeh.org/u/gmerritt123)\
**Post date:** [December 1, 2021, 4:40pm UTC](https://discourse.bokeh.org/t/access-box-select-attributes-set-in-figure/8696/7 "2021-12-01T16:40:02Z")

</div>

I altered your example a tiny bit just to make it a quick and dirty standalone, but here’s what I mean by the callback part:

```auto
from bokeh.plotting import figure
from bokeh.layouts import column
from bokeh.models import ColumnDataSource, BoxEditTool, CustomJS
from bokeh.plotting import show

def bkapp():

    source0 = ColumnDataSource(data=dict(
    x0=[1, 2, 3, 4, 5],
    y0=[2, 5, 8, 2, 7],
    ))

    tools = ['hover', 'reset']

    plot = figure(x_axis_label='x',
                  y_axis_label='y)',
                  title="Test BoxEditTool",
                  tools=tools)
    
    source = ColumnDataSource({'x':[], 'y':[], 'width':[], 'height':[]})

    r1 = plot.rect('x', 'y', 'width', 'height', source=source)
    tool = BoxEditTool(renderers=[r1], num_objects=1)
    plot.add_tools(tool)
    
    cb = CustomJS(args=dict(source=source)
                  ,code='''
                  console.log(source.data)
                  const xl = source.data['x'][0]-source.data['width'][0]/2
                  const xr = source.data['x'][0]+source.data['width'][0]/2
                  const yl = source.data['y'][0]-source.data['height'][0]/2
                  const yu = source.data['y'][0]+source.data['height'][0]/2
                  console.log('xleft = '+xl.toString())
                  console.log('xright = '+xr.toString())
                  console.log('yupper = '+yu.toString())
                  console.log('ylower = '+yl.toString())
                  ''')
    source.js_on_change('data',cb)
    
    # Add the data renderer (here a scatter plot)
    scatter = plot.scatter(x='x0', y='y0', source=source0)

    return plot

p = bkapp()
show(p)

```

![box](https://discourse-uploads.bokeh.org/original/2X/6/6c11d413ecb64777145715a1e72b4073681ff131.gif)

---

<div class="post-metadata">

**Author:** ![Wall.E](https://discourse.bokeh.org/user_avatar/discourse.bokeh.org/wall.e/32/3759_2.png) [@Wall.E](https://discourse.bokeh.org/u/Wall.E)\
**Post date:** [December 1, 2021, 4:50pm UTC](https://discourse.bokeh.org/t/access-box-select-attributes-set-in-figure/8696/8 "2021-12-01T16:50:25Z")

</div>

> [@gmerritt123](#):
>
> hold shift my friend

I did, doesn’t work in my Jupyter Lab. I saw it working in a browser when exporting an html file with `output_file()` but that is not our workflow… and that’s frustrating not to have it work in Jupyter Lab.

Regarding your way with customJS, we are JS-averse, but i’ll take a look.

---

<div class="post-metadata">

**Author:** ![gmerritt123](https://discourse.bokeh.org/user_avatar/discourse.bokeh.org/gmerritt123/32/4101_2.png) [@gmerritt123](https://discourse.bokeh.org/u/gmerritt123)\
**Post date:** [December 1, 2021, 4:57pm UTC](https://discourse.bokeh.org/t/access-box-select-attributes-set-in-figure/8696/9 "2021-12-01T16:57:31Z")

</div>

> I did, doesn’t work in my Jupyter Lab.

That is a different problem then, way out of my domain.

> Regarding your way with customJS, we are JS-averse, but i’ll take a look.

There 100% is a python callback style way to do this too, just retrieve source.data in the python callback function and do what you want with it then.

---

<div class="post-metadata">

**Author:** ![Wall.E](https://discourse.bokeh.org/user_avatar/discourse.bokeh.org/wall.e/32/3759_2.png) [@Wall.E](https://discourse.bokeh.org/u/Wall.E)\
**Post date:** [December 1, 2021, 5:04pm UTC](https://discourse.bokeh.org/t/access-box-select-attributes-set-in-figure/8696/10 "2021-12-01T17:04:01Z")

</div>

Ok, we’re nearly there then. I just need to fix that Jupyter Lab not showing the rectangles. I’ll create a new topic for this specific problem. Thanks.

---

<div class="post-metadata">

**Author:** ![Wall.E](https://discourse.bokeh.org/user_avatar/discourse.bokeh.org/wall.e/32/3759_2.png) [@Wall.E](https://discourse.bokeh.org/u/Wall.E)\
**Post date:** [December 1, 2021, 7:34pm UTC](https://discourse.bokeh.org/t/access-box-select-attributes-set-in-figure/8696/11 "2021-12-01T19:34:04Z")

</div>

We may have spoken too fast. For a python callback to be triggered with `.on_change()`, we still need an attribute to be changed. However, `BoxEditTool` seems to only have scatter glyph attritubes for handling that, due to the [inheritence](http://docs.bokeh.org/en/latest/docs/reference/models/tools.html?highlight=boxedittool#boxedittool) from `EditTool`:

> [`renderers`](http://docs.bokeh.org/en/latest/docs/reference/models/tools.html?highlight=boxedittool#bokeh.models.BoxEditTool.renderers) An explicit list of renderers corresponding to scatter glyphs that may be edited.

The customJS callback does show, but that JS side (browser side). Getting things synchronized with my underlying Python environment running the server remains unaddressed by `BoxEditTool` in combination with the `image()`

---

<div class="post-metadata">

**Author:** ![gmerritt123](https://discourse.bokeh.org/user_avatar/discourse.bokeh.org/gmerritt123/32/4101_2.png) [@gmerritt123](https://discourse.bokeh.org/u/gmerritt123)\
**Post date:** [December 1, 2021, 10:06pm UTC](https://discourse.bokeh.org/t/access-box-select-attributes-set-in-figure/8696/12 "2021-12-01T22:06:09Z")

</div>

I think you’ve read documentation that has been copy pasted from the point edit tool. Literally replace the word scatter in that sentence with rect… it handles rect glyphs (and probably quads but I dunno I’d have to test it) just fine. If it didn’t then my CustomJS approach wouldn’t work either. Anyway, here’s the python server side equivalent, adapted from your example. I replaced the console.logs with a div on the side that basically shows you how to get at the box dims in the same manner as my CustomJS effort above.

```auto
from bokeh.plotting import figure, curdoc
from bokeh.layouts import row
from bokeh.models import ColumnDataSource, BoxEditTool, Div
from bokeh.plotting import show

source0 = ColumnDataSource(data=dict(
x0=[1, 2, 3, 4, 5],
y0=[2, 5, 8, 2, 7],
))

tools = ['hover', 'reset']

plot = figure(x_axis_label='x',
              y_axis_label='y)',
              title="Test BoxEditTool",
              tools=tools)

source = ColumnDataSource({'x':[], 'y':[], 'width':[], 'height':[]})

r1 = plot.rect('x', 'y', 'width', 'height', source=source)
tool = BoxEditTool(renderers=[r1], num_objects=1)
plot.add_tools(tool)

div = Div(text='Box Info:')

def getBoxDims(attrname,old,new):
    xl = source.data['x'][0]-source.data['width'][0]/2
    xr = source.data['x'][0]+source.data['width'][0]/2
    yl = source.data['y'][0]-source.data['height'][0]/2
    yu = source.data['y'][0]+source.data['height'][0]/2
    div.text='Box Info: Xdims: '+str(xl)+','+str(xr)+', Ydims: '+str(yl)+','+str(yu) 
    
source.on_change('data',getBoxDims)    

# Add the data renderer (here a scatter plot)
scatter = plot.scatter(x='x0', y='y0', source=source0)

curdoc().add_root(row([plot,div]))

```

… I also don’t really see how the image glyph has anything to do with the implementation of simply drawing a box on a figure and getting the dimensions of said box, other than of course (for your specific use case) the box is gonna be drawn over top of an image.

---

<div class="post-metadata">

**Author:** ![Bryan](https://discourse.bokeh.org/user_avatar/discourse.bokeh.org/bryan/32/3625_2.png) [@Bryan](https://discourse.bokeh.org/u/Bryan)\
**Post date:** [December 1, 2021, 10:58pm UTC](https://discourse.bokeh.org/t/access-box-select-attributes-set-in-figure/8696/13 "2021-12-01T22:58:07Z")

</div>

> [@gmerritt123](#):
>
> I think you’ve read documentation that has been copy pasted from the point edit tool.

Looks like. Bokeh’s docs comprise about 35000+ lines of text and code and have mostly been written by 2-3 people. For sure there’s copy-pasta here and there. Please create a [GitHub Issue](https://github.com/bokeh/bokeh/issues) so it can be fixed before the next release.

---

<div class="post-metadata">

**Author:** ![Wall.E](https://discourse.bokeh.org/user_avatar/discourse.bokeh.org/wall.e/32/3759_2.png) [@Wall.E](https://discourse.bokeh.org/u/Wall.E)\
**Post date:** [December 2, 2021, 7:09pm UTC](https://discourse.bokeh.org/t/access-box-select-attributes-set-in-figure/8696/14 "2021-12-02T19:09:57Z")

</div>

> [@gmerritt123](#):
>
> Anyway, here’s the python server side equivalent, adapted from your example.

This is working pretty great if I dismiss my need for Jupyter Lab. I can live with that for now. That is very helpful, thanks a lot.  
@Bryan [suggested](https://discourse.bokeh.org/t/jupyter-lab-server-app-not-showing-rectangle-drawn-with-boxedittool/8698/10) also to try the `SelectionGeometry` event with box\_select. That would address the original question and is worth a look, as `BoxSelectTool` has no issue with Jupyter Lab.

---

<div class="post-metadata">

**Author:** ![system](https://discourse-uploads.bokeh.org/original/2X/1/14eb24f9e10e4e9b78e3f10f66c04ffa96446e5a.svg) [@system](https://discourse.bokeh.org/u/system)\
**Post date:** [March 2, 2022, 7:10pm UTC](https://discourse.bokeh.org/t/access-box-select-attributes-set-in-figure/8696/15 "2022-03-02T19:10:52Z")

</div>

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.