Custom tool plotting a vertical line profile of an image

A new example is suggested for the docs. It illustrates:

  • the usage of image.source for dynamically updating image contents and
  • demonstrates how to change data of one subplot from a tool in another subplot.
    image
import numpy as np
import bokeh.plotting as bp
from bokeh.models import CustomJS
from bokeh.layouts import layout, column, row

from bokeh.io import reset_output
from PIL import Image

from bokeh.core.properties import Instance, Float, Array
from bokeh.io import output_file, show, output_notebook
from bokeh.models import ColumnDataSource, Tool
from bokeh.plotting import figure
from bokeh.util.compiler import TypeScript
from bokeh.layouts import layout, column, row

# image vertical profile tool
TS_CODE = """
import {GestureTool, GestureToolView} from "models/tools/gestures/gesture_tool"
import {ColumnDataSource} from "models/sources/column_data_source"
import {GestureEvent} from "core/ui_events"
import {View} from "core/view"
import * as p from "core/properties"

export class DrawToolView extends GestureToolView {
  model: DrawTool
  inv: number

  constructor(options: View.Options) {
    super(options)
    this.inv = NaN
  }

  invert_vline(x: number) {
    var h = this.model.im_src.data["dh"][0],
        w = this.model.im_src.data["dw"][0],
        im = this.model.im_src.data["image"][0];
    
    for(var y=0; y<h; y++) im[y*w+x] = 255-im[y*w+x];
  }

  //this is executed when the pan/drag event starts
  _pan_start(_ev: GestureEvent): void {
    this.model.line_src.data = {x: [], y: []}
  }

  //this is executed on subsequent mouse/touch moves
  _pan(ev: GestureEvent): void {
    const {frame} = this.plot_view
    const {sx, sy} = ev
    if (!frame.bbox.contains(sx, sy))
      return

    const x = frame.xscales.default.invert(sx)

    var rx = Math.round(x);
    
    if(this.inv != rx){
      var res = Array(w),
          w = this.model.im_src.data["dw"][0],
          im = this.model.im_src.data["image"][0]
          
      if(!isNaN(this.inv)){
        this.invert_vline(this.inv)
      }
  
      for(var i=0; i<w; i++) res[i] = im[i*w+rx];

      this.model.line_src.data = {
        x: Array(w).fill(0).map(Number.call, Number), 
        y: res
      };
      this.model.line_src.change.emit()
    
      this.invert_vline(rx)
      this.inv = rx
      this.model.im_src.change.emit()
    }
  }

  // this is executed then the pan/drag ends
  _pan_end(_ev: GestureEvent): void {}
}

export namespace DrawTool {
  export type Attrs = p.AttrsOf<Props>

  export type Props = GestureTool.Props & {
    im_src: p.Property<ColumnDataSource>,
    line_src: p.Property<ColumnDataSource>
  }
}

export interface DrawTool extends DrawTool.Attrs {}

export class DrawTool extends GestureTool {
  properties: DrawTool.Props

  constructor(attrs?: Partial<DrawTool.Attrs>) {
    super(attrs)
  }

  tool_name = "Drag Span"
  icon = "bk-tool-icon-lasso-select"
  event_type = "pan" as "pan"
  default_order = 12

  static initClass(): void {
    this.prototype.type = "DrawTool"
    this.prototype.default_view = DrawToolView

    this.define<DrawTool.Props>({
      im_src: [ p.Instance ],
      line_src: [ p.Instance ]
    })
  }
}
DrawTool.initClass()
"""

class DrawTool(Tool):
    __implementation__ = TypeScript(TS_CODE)
    im_src = Instance(ColumnDataSource)
    line_src = Instance(ColumnDataSource)
output_notebook()

im = Image.open(r'D:\image.jpg')
im_arr = np.array(im)[:,:,0]
h, w = im_arr.shape

im_src = ColumnDataSource(data=dict(image=[np.flipud(im_arr)], x=[0], y=[0], dw=[w], dh=[h]))
line_src = ColumnDataSource(data=dict(x=[], y=[]))

p1 = figure(plot_width=600, plot_height=200, x_range=(0, w), y_range=(0, h), 
            tools=[DrawTool(im_src=im_src, line_src=line_src)])
p2 = figure(plot_width=600, plot_height=200)

im = p1.image(image='image', x='x', y='y', dw='dw', dh='dh', source=im_src, palette='Greys256')
p2.line('x', 'y', source=line_src)

bp.show(column(p1, p2))

Two "todo"s immediately observable are:

  • tool icon needs to be updated
  • click event doesn’t get processed, only drag event

Anything else?

2 Likes

bugfix of occasional plotting inverted profiles

So, just some suggestions, feel free take or leave :slight_smile:

  • A real custom icon for the toolbar can be provided as a base64 dataurl
  • Might suggest SliceTool as the tool name
  • For an example/demo it might be nice to synthesize some data rather than load an image

It is slightly more generic in that it handles both grayscale and color images plus it is works not only along vertical line but along arbitrary path set by consecutive mouse clicks.

Here in this example those features can be safely omitted for the sake of simplicity of realization plus for certain uses only horizontal/vertical profiles are necessary and arbitrary lines only hinder user experience.

The corresponding function in opencv is called cvSampleLine OpenCV: C API

So I’d stick to something like ProfileTool, VProfileTool.

  • image - This tool is supposed to be used for image processing. Typical workflow there is grab a raw image from camera and try to extract something useful out of it. Reading an existing image looks more natural for me in this context.

I took the image from this blog post Antialiasing: To Splat Or Not – Nathan Reed’s coding blog and as the author generated it himself I don’t think he would mind if we use it here. This image fits the tool well because the profile plot change upon mouse move is significant, predictable and readily verifiable: one can draw the profile by hand precisely just by looking at the image.

I’ve generated a better quality image using the script from that blog:
aa1

Also I’ve have drawn a logo draft:

logo

Haven’t found any clues in the docs on how to include it in base64 form.

Hrm, it seems I was thinking of CustomAction.icon. I thought that facility was more concretely available on base classes. However, I think if you add a string property named icon to your extension, and set the value to a base64 encoded data-url of the icon image, then the icon should be picked up and display.

An online data-url generator I have used is: data: URI Generator

If you want to make a GH issue to suggest clearing up/documenting custom icons better, I think that would be helpful.

This tool is supposed to be used for image processing.

Just FYI I have seen (and created) very similar tools for slightly different contexts. E.g. a “function explorer” where the cross-section interrogates the values of an implicit function surface. There’s still a color-mapped image to represent the surface, but it’s not really image processing, per se. In these contexts I’ve heard it called “slicing”. This was also why I suggested synthesizing data. But I’m happy to defer to your preferences for the example!