advice for making a new type of Plot

Community Support
1

Hi all, we are attempting to reimplement an old Perl web app called
'CMAP' into modern Javascript/HTML.

Here is a screenshot of the old 'comparative map' aka CMAP. Sometimes
a picture is worth 1024 words, right?

620×831 70.8 KB

Bokeh seemed like it would be useful to add a lot of tools and utility
out of the box, and we are biased because we love Python and Conda,
but I am having quite a bit trouble getting started.

As you can see, there are correspondences drawn and features hilited
between each map (each vertical spine thing is a map). So this needs
to be a single plot and not a dom-layouted collection of different
plots, so the correspondences can be drawn. Also note each map has
it's own coordinate system, so the default axes which Figure provides
would not make sense. However we would like to utilize the other
prebuilt tools which are available with Figure class (pan, lasso,
save, undo, etc).

After several days of studying the Bokeh docs, source code, examples,
I can think of several approaches to produce something like this in
Bokeh, but I am not sure which of these are are possible, recommend,
or most bokeh-ish way:

1) Use the existing Figure class, but turn off the default axes, and
then use the Plotting API to draw all the tick marks, rectangles,
labels, correspondences, etc, basically everything you see in that
jpg. i.e. Use the existing Glyph types in the plotting api.

2) Create CustomModels for the entire datastructure, all wrapped in a
single LayoutDOM. From what I have learned, this would involve pretty
much re-implementing most of the Figure class, therefore probably an
inordinate amount of work.

3) Make a subclass of the Figure class and add new types of
GlyphRenderers to make the layouting and rendering more modular and
less procedural.

4) Make a subclass of the Plot class and add new types of
GlyphRenderers, and re-implement some stuff we need from Figure, to
me the rendering more modular and less procedural.

5) Copy what is going on in the Charts API to make a new hi-level
chart, kinda like bokeh.charts.BoxPlot for example.

#1 seems like the simplest way to proceed, but also could rapidly
become a procedural mess- if you look at the jpg there is a lot going
on. Much more complicated than the examples of 'plotting with basic
glyphs' as shown in the users's guide.

#5 is worth mentioning, because this CMAP tool is intended for the
bioinformatics community to re-use, and so the result should be fairly
hi-level and easy to use and configure.

Any advice would be most appreciated! Sorry for the long post.

···

--
Alex Rice <[email protected]>
Software Engineer
National Center for Genome Resources (NCGR)
http://ncgr.org
+1 505-995-4457

2

Hi Alex,

This seems like an interesting project. I hope to leave a more detailed response soon, but I am a bit pressed for time today, so hopefully a few general notes might be helpful.

* I think you'll end up turning off default axes, etc in any case. Nothing that is built in looks like this.

* Although it's not been demonstrated, I can imagine a custom extension that draws correspondences between different Bokeh plots on different canvases (by computing the absolute coordinates and using CSS or something). I don't think it would be utterly trivial, but the benefit of just having three separate plots might justify looking at that.

* You don't have to subclass Figure to add new glyph renderers. You'd only have to subclass to add new glyph methods. i.e. you can create a new usable FooGlyph without also subclassing to add a foo_glyph method (but maybe you want to, just pointing out that its a separable concern)

Another question: Is this a visualization that needs to be updatable, streaming, etc? Once rendered in a browser, will one of these plots ever need to be updated "in place" or is each individual render more static?

Thanks,

Bryan

···

On Dec 13, 2016, at 4:06 PM, Alex Rice <rice.guido > @gmail.com> wrote:

Hi all, we are attempting to reimplement an old Perl web app called
'CMAP' into modern Javascript/HTML.

Here is a screenshot of the old 'comparative map' aka CMAP. Sometimes
a picture is worth 1024 words, right?

http://gmod.org/mediawiki/images/4/40/Cmap_sample1.jpg

Bokeh seemed like it would be useful to add a lot of tools and utility
out of the box, and we are biased because we love Python and Conda,
but I am having quite a bit trouble getting started.

As you can see, there are correspondences drawn and features hilited
between each map (each vertical spine thing is a map). So this needs
to be a single plot and not a dom-layouted collection of different
plots, so the correspondences can be drawn. Also note each map has
it's own coordinate system, so the default axes which Figure provides
would not make sense. However we would like to utilize the other
prebuilt tools which are available with Figure class (pan, lasso,
save, undo, etc).

After several days of studying the Bokeh docs, source code, examples,
I can think of several approaches to produce something like this in
Bokeh, but I am not sure which of these are are possible, recommend,
or most bokeh-ish way:

1) Use the existing Figure class, but turn off the default axes, and
then use the Plotting API to draw all the tick marks, rectangles,
labels, correspondences, etc, basically everything you see in that
jpg. i.e. Use the existing Glyph types in the plotting api.

2) Create CustomModels for the entire datastructure, all wrapped in a
single LayoutDOM. From what I have learned, this would involve pretty
much re-implementing most of the Figure class, therefore probably an
inordinate amount of work.

3) Make a subclass of the Figure class and add new types of
GlyphRenderers to make the layouting and rendering more modular and
less procedural.

4) Make a subclass of the Plot class and add new types of
GlyphRenderers, and re-implement some stuff we need from Figure, to
me the rendering more modular and less procedural.

5) Copy what is going on in the Charts API to make a new hi-level
chart, kinda like bokeh.charts.BoxPlot for example.

#1 seems like the simplest way to proceed, but also could rapidly
become a procedural mess- if you look at the jpg there is a lot going
on. Much more complicated than the examples of 'plotting with basic
glyphs' as shown in the users's guide.

#5 is worth mentioning, because this CMAP tool is intended for the
bioinformatics community to re-use, and so the result should be fairly
hi-level and easy to use and configure.

Any advice would be most appreciated! Sorry for the long post.

--
Alex Rice <[email protected]>
Software Engineer
National Center for Genome Resources (NCGR)
http://ncgr.org
+1 505-995-4457

--
You received this message because you are subscribed to the Google Groups "Bokeh Discussion - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/CACmK6Bu6-QE1JUjN_CDUPRNEuG0SKOyYck0vRJJQ5TQMkjMhZw%40mail.gmail.com\.
For more options, visit https://groups.google.com/a/continuum.io/d/optout\.

3

Hi Alex,

This seems like an interesting project. I hope to leave a more detailed response soon, but I am a bit pressed for time today, so hopefully a few general notes might be helpful.

* I think you'll end up turning off default axes, etc in any case. Nothing that is built in looks like this.

OK sounds good

* Although it's not been demonstrated, I can imagine a custom extension that draws correspondences between different Bokeh plots on different canvases (by computing the absolute coordinates and using CSS or something). I don't think it would be utterly trivial, but the benefit of just having three separate plots might justify looking at that.

Good to know the tradeoffs. There will need to be some interactivity
too, like if we mouseover a feature with a range (different start and
end position), then all the other features falling within that range
would be hilited. I hadn't thought of a separate plot/canvas to draw
the correspondences, but that sounds like a nice design actually.

* You don't have to subclass Figure to add new glyph renderers. You'd only have to subclass to add new glyph methods. i.e. you can create a new usable FooGlyph without also subclassing to add a foo_glyph method (but maybe you want to, just pointing out that its a separable concern)

Aha, so you are saying if we implemented a FooGlyph, we could just use
like Plot.add_glyph() or something to display it? And subclassing
Figure would not be necessary?

Another question: Is this a visualization that needs to be updatable, streaming, etc? Once rendered in a browser, will one of these plots ever need to be updated "in place" or is each individual render more static?

In general these are static maps which have been computed ahead of
time. For simplicity of use/deployment we are leaning toward these
being static client side visualizations, and maybe implement in more
bokeh server type of apps after we get the basics covered.

However they come in collections (Map Sets) and the user should to be
able to select what maps to display in the center, left, and right
panels. In a horizontal layout from 1 to 3 maps will be allowed. This
map-selection idea might in itself necessitate the use of bokeh server
app, so as not bog down the client with too much data at load time.

Also for a radial or hive-plot layout, there could be more than 3
maps. But we are not going to attempt anything except the horizontal
layout at first!

There is also a progressive enhancement effect that is going on. In
the screenshot, you are only seeing the features for which there are
correspondences (red text) plus whatever else could be labelled in the
available space (black text). If you zoom in on an individual map, it
allocates more space for labels, and more labels appear. I thought of
trying to use your DataShaders library for this aspect. But could not
quite grasp how that would work or even if it is a good fit compared
with the current fill-up-the-vertical-space-with-labels algorithm. I
am thinking DataShader is more suitable for Raster image types of
problems.

Thanks in advance for any other suggestions you can think of!

···

On Tue, Dec 13, 2016 at 3:25 PM, Bryan Van de Ven <[email protected]> wrote:

4

I can't see any straightforward way that datashader would be useful for
this problem. Drawing lines between items in different coordinate systems
seems like it's always going to be difficult in anything based on Bokeh or
Matplotlib, as well. If you were ok with a somewhat different approach
compared to the existing tool, you could use HoloViews (see holoviews.org),
which in its upcoming 1.7 release makes it simpler to provide custom sorts
of interactivity that could let you e.g. click on an item in one coordinate
system and have corresponding items in other plots be highlighted. Pure
Bokeh could also do that without too much effort. But something like
NetworkX might be more suited to this task, because it handles connections
between items more directly than a plotting program would.

···

On Tue, Dec 13, 2016 at 4:50 PM, Alex Rice <[email protected]> wrote:

On Tue, Dec 13, 2016 at 3:25 PM, Bryan Van de Ven <[email protected]> > wrote:

There is also a progressive enhancement effect that is going on. In
the screenshot, you are only seeing the features for which there are
correspondences (red text) plus whatever else could be labelled in the
available space (black text). If you zoom in on an individual map, it
allocates more space for labels, and more labels appear. I thought of
trying to use your DataShaders library for this aspect. But could not
quite grasp how that would work or even if it is a good fit compared
with the current fill-up-the-vertical-space-with-labels algorithm. I
am thinking DataShader is more suitable for Raster image types of
problems.

5

Thanks for the tips, James. Those look like useful libraries.

···

On Tue, Dec 13, 2016 at 4:36 PM, James Bednar <[email protected]> wrote:

I can't see any straightforward way that datashader would be useful for this
problem. Drawing lines between items in different coordinate systems seems
like it's always going to be difficult in anything based on Bokeh or
Matplotlib, as well. If you were ok with a somewhat different approach
compared to the existing tool, you could use HoloViews (see holoviews.org),
which in its upcoming 1.7 release makes it simpler to provide custom sorts
of interactivity that could let you e.g. click on an item in one coordinate
system and have corresponding items in other plots be highlighted. Pure
Bokeh could also do that without too much effort. But something like
NetworkX might be more suited to this task, because it handles connections
between items more directly than a plotting program would.