Client-side Molecular Structure Generation on Hover

[rwilson]

Hi Community,
I’m attempting to use Bokeh to create plots annotated with tooltip chemical structures in Jupyter notebooks. Because the number of plot elements can be extremely high, and molecular structure generation from a chemical description (SMILES) is expensive, I would like to implement this on-demand, in a JavaScript function that is called on hover.
As an example, this is the desired functionality, but it requires all SVGs to be pre-computed, here by calling the function make_svg in Python.

from bokeh.plotting import ColumnDataSource, figure, show
from rdkit.Chem import MolFromSmiles
from rdkit.Chem.Draw import rdMolDraw2D

def make_svg(smiles: str): # return SVG str from SMILES str
    x = rdMolDraw2D.MolDraw2DSVG(100, 100)
    x.DrawMolecule(MolFromSmiles(smiles))
    x.FinishDrawing()
    return x.GetDrawingText()

source = ColumnDataSource({'X':[1], 'Y':[1],
                           'smiles':['c1ccccc1'],
                           'svg':[make_svg('c1ccccc1')]})

p = figure(tooltips="@svg")
p.circle('X', 'Y', size=10, source=source)
show(p)

What I really want is for the browser to compute that structure instead of Python. To that end, I tried to put together a call with an external tool instead, shown below.

<div>
    <img data-smiles="c1ccccc1" 
        data-smiles-options="{'width':100,'height':100 }"/>
    <script> type="text/javascript" src="https://unpkg.com/[email protected]/dist/smiles-drawer.min.js"></script>
    <script>SmiDrawer.apply()</script>
</div>

And this produces the expected benzene ring in Jupyter. However, when I plug it into the tooltip string (either directly, or substituting @smiles for the dummy “c1cccccc1”) I only get an empty box on hover.

Unfortunately JavaScript and HTML are foreign to me, so I’m not sure if I need to debug or if I’m doing something fundamentally incorrect. Because I don’t get the expected benzene ring on hover even without @smiles substitution, I suspect that I can’t src scripts in this context, or I’m misunderstanding something else. Before I get too far into learning JS fundamentals, I’d appreciate any sanity or feasibility checks from the community. Thanks!

[Bryan]

Tooltip strings are purely templates, they are not executed as code at any point, so putting code inside them will have no effect.

It is possible to have code execute on hover using a Bokeh CustomJSHover hover formatter. But those were really only created with custom formatting text output for the tooltip in mind. I expect there might be a way to to bang things into working, but I don’t know how to do it offhand. If the CustomJSHover generated the <svg> tag text and the hover field was marked {safe} it might work.

I can’t offer much more than that level of speculation without a complete Minimal Reproducible Example to actually run directly and experiment on.

cc @mateusz @gmerritt123 @James_A_Bednar1 who might be interested int this use-case.

[rwilson]

Thanks very much Bryan; your speculation will keep me from chasing any dead ends. I’m including an attempt at a CustomJSHover MRE here. This example uses an ethane SMILES (‘CC’) that should render as a single line, so that the resulting SVG is minimal. I’ve checked that the JS function produces correct SVGs, but I’m not sure how to combine CustomJSHover with {safe}. Right now, this example leaves the text of the SVG in the tooltip instead of rendering it.

from bokeh.plotting import ColumnDataSource, figure, show
from bokeh.models import CustomJSHover, HoverTool

make_svg = CustomJSHover(code="""
import("https://unpkg.com/[email protected]/dist/smiles-drawer.min.js")

let drawer = new SmiDrawer()
let svg = document.createElementNS("http://www.w3.org/2000/svg", "svg")
svg.setAttribute('width', '50px')
svg.setAttribute('height', '50px')

drawer.draw(value, svg)
return svg.outerHTML""")

source = ColumnDataSource({'X':[1],'Y':[1], 'smiles':['CC']})

hover = HoverTool(
    tooltips = [('structure', '@smiles{custom}')],
    formatters = {'@smiles':make_svg})

fig = figure()
fig.add_tools(hover)
fig.circle('X', 'Y', size=10, source=source)
show(fig)

[Bryan]

Hi @rwilson thanks very much for the MRE, that’s a good starting point we can actually experiment from over the next few days.

FYI the ability to use import directly in a CusomJS is still a work in progress Add support for `CustomESM` callbacks by mattpap · Pull Request #12812 · bokeh/bokeh · GitHub expected to be part of Bokeh 3.2 in a few months. @gmerritt123 can hopefully point to some examples of how to load external JS libraries with current latest releases.

[gmerritt123]

I think I got this working (not actually sure what it’s supposed to look like)… but it seems to draw the svg on hover now. Dang this is cool if so. My strategy was to use the CustomJSHover to set the hover.tooltips property TO THE HTML TEXT we want instead of using it to “return” a string of the html:

from bokeh.plotting import ColumnDataSource, figure, show
from bokeh.models import CustomJSHover, HoverTool
from bokeh.embed import components
from bokeh.resources import Resources

source = ColumnDataSource({'X':[1],'Y':[1], 'smiles':['CC']})

hover = HoverTool(
    tooltips = [('structure', '@smiles{custom}')]
    )

make_svg = CustomJSHover(args=dict(hover=hover),code='''
let drawer = new SmiDrawer()
console.log(drawer)
let svg = document.createElementNS("http://www.w3.org/2000/svg", "svg")
svg.setAttribute('width', '50px')
svg.setAttribute('height', '50px')
drawer.draw(value, svg)
hover.tooltips = "<svg>"+svg.outerHTML+"</svg>"
''')

hover.formatters = {'@smiles':make_svg}

fig = figure()
fig.add_tools(hover)
fig.circle('X', 'Y', size=10, source=source)


def save_html_wJSResources(bk_obj,fname,resources_list,html_title='Bokeh Plot'):
    '''function to save a bokeh figure/layout/widget but with additional JS resources imported at the top of the html
    resources_list is a list input of where to import additional JS libs so they can be utilized into CustomJS etc in bokeh work
    e.g. ['http://d3js.org/d3.v6.js']
    '''
    script, div = components(bk_obj)
    resources = Resources()
    # print(resources)
    
    tpl = '''<!DOCTYPE html>
    <html lang="en">
        <head>
            <meta charset="utf-8">
            <title>'''+html_title+'''</title>
            '''
    tpl = tpl+resources.render_js()            
    for r in resources_list:
        tpl = tpl +'''\n<script src="'''+r+'''"></script>'''
      
    tpl = tpl+script+\
        '''
            </head>
            <body>'''\
                +div+\
            '''</body>
        </html>'''
    
    with open(fname,'w') as f:
        f.write(tpl)


save_html_wJSResources(bk_obj=fig,fname='C:\Repo\Proofing\Smle.html'
                       ,resources_list=['https://unpkg.com/[email protected]/dist/smiles-drawer.min.js']
                       ,html_title='Smile')

A few awkward things I noticed:

1. You can’t really mark a tooltip {safe} and {custom} at the same time?
2. The required order of instantiation is weird. You have to
   a) make the Hovertool with a tooltip pointing to the value field and it needs the {custom} tag to trigger the CustomJSHover. Then:
   b) you need to write out that CustomJSHover AFTER that so you can pass the hover in as an arg, and therefore have access to changing its tooltips property. Finally,
   c) you need to set the formatters property on the hover AFTER you’ve done a and b so you can point the initial tooltip to trigger the CustomJSHover

The little convenience save function at the bottom should help you automate the embedding of <script> type="text/javascript" src="https://unpkg.com/[email protected]/dist/smiles-drawer.min.js"></script> in your html (i.e. what @Bryan was talking about re loading of external JS libraries)

EDIT → I just tried it with ‘c1ccccc1’ as the value and it does indeed draw that structure! Awesome

[Bryan]

My strategy was to use the CustomJSHover to set the hover.tooltips property TO THE HTML TEXT we want instead of using it to “return” a string of the html:

Definitely not the intended usage, but I’ve seen CustomJSHover used similarly in other contexts. Certainly a reasonable workaround in the current situation!

You can’t really mark a tooltip {safe} and {custom} at the same time?

This seems like a simple oversight and hopefully simple to fix. Can you file a GH issue about it?

[rwilson]

Wow, this is really impressive. Thank you both for your contribution. I can take care of cleaning up some minor problems, but more seriously, however, I’m getting the same structure rendered for every SMILES input. I think the assignment to hover.tooltips might prevent the CustomJSHover from being called again? Or possibly there’s some weird caching behavior on my machine. If you get a chance, I’m providing a more useful ColumnDataSource that can test a few molecules instead.

source = ColumnDataSource(dict(
    X=[0,0,1,1,0.5],
    Y=[0,1,0,1,0.5],
    smiles=['C1CC1', 'C1CCC1', 'C1CCCC1', 'C1CCCCC1', 'c1ccccc1']))

These SMILES should render as ▽, ◇, ⬠, ⬡ , ⏣. I also have a script for plotting a larger public drug discovery set (~330k chemicals) that I can share if needed, and I’ll post that for a demo if this is nailed down.
I’ll continue to play with this to see if there’s an easy solution, although it does seem like the {custom}{safe} flags might be the more fundamentally correct solution and there’s probably no point in me trying too hard to circumvent that.