Crossplatform Tool for 1st Quality Control on Hydrographic Cruise Data

We have released a Crossplatform Tool for 1st Quality Control on Hydrographic Cruise Data, developed at the IIM (CSIC) and funded by AtlantOS.

We have embedded and packaged Bokeh with Electron. The communication between both works through JS signaling.

Also, we’ve integrated octave/matlab routine calculations in the application. The motivation of doing this (because you can do all this with python) is the availability of many tested code in that language and easiness for potential users that come from that language.

You can check all the details and give feedback in the GitHub Project

3 Likes

This is really great! Can you give a high level overview of what the Electron integration looks like ? What would it take to make a toy demo example? Are there things that could go in the core library to make things better or simpler?

Yes, I think the possibility to make a simple RPC (calling Python methods and get the answer directly from Electron) will help a lot. Besides, the code to communicate both would be less cumbersome.

Actually the connection I made is kind of hacky, using dummy invisible objects and sending signals from JavaScript. I have also applied the solution you gave me time ago, and I built something similar in the opposite direction.

I am going to be on vacation for a couple of weeks. Let’s see if I can make some example when I go back. Or maybe I can point to the important sections in the source code of the project

1 Like

Well, I write here some general explanation about the Electron integration I made with the Bokeh Server.

Firstly I run bokeh with the library python-shell this way:

var user_options = [
    '-m', 'bokeh', 'serve',
    '--port', self.bokeh_port,
    '--log-format', '"%(asctime)s %(levelname)s %(message)s"',
    '--log-file', loc.log_python
]
self.python_options = {
    mode: 'text',               // actually I do not need to return anything,
    pythonPath: self.python_path,
    pythonOptions: aux_options,
    scriptPath: self.ocean_data_qc_path
};
self.shell = python_shell.run(
    '', self.python_options, (err, results) => {
        if (err || typeof(results) == 'undefined') {
            // ... the bokeh logger is used instead of getting the messages here

Then, the Electron process should be bound to the python process where the Bokeh Server is running.

I built a mechanism to call python methods “on the fly” as it was a web service or a simple rpc call. I make a call and I get an answer asynchronously like this:

var expr_name = self.comp_param_name.val();
var params = {
    'object': 'computed.parameter',
    'method': 'compute_equation',
    'args': {
        'computed_param_name': expr_name,
        'eq': self.equation_text.val(),
    }
}
tools.call_promise(params).then((result) => {
    lg.info('>> VALIDATE EQUATION RESULT: ' + JSON.stringify(result, null, 4));
    if ('success' in result && result['success'] == true) {

The call_promise function is responsible for sending a signal (it waits for the answer as well) to the iframe where bokeh is running. I built a dummy bokeh form where I fill a text input with the information I want to send to python. Then I trigger the click event of a button and I already have the information in the python side. This is kind of hacky as you can see:

window.onmessage = function(e){
    if (e.data.signal == 'call-python-promise') {
        // this updates dummy text field value and triggers the click event of the bridge_button

        var models = window.Bokeh.index[Object.keys(window.Bokeh.index)[0]].model.document._all_models;
        var model_id = null;
        $.each(models, function (m_id) {
            if (models[m_id].constructor.name == 'TextInput') {
                model_id = m_id;
                // console.log('>> TextInput Name: ' + models[m_id].attributes.name)
            }
        });
        var input_bridge_text = models[model_id];
        input_bridge_text.value = JSON.stringify(e.data.message_data);
        var button = $('.bridge_button>div>button')
        button.click();

If I remember well I needed to add the environment variable BOKEH_SIMPLE_IDS=true to make this code work because I work with the ids that bokeh generates for the objects.

Finally I receive the data in the python side and I run the suitable method:

elif obj == 'computed.parameter':
    method = getattr(self.env.cruise_data.cp_param, method_str)

result = False
try:
    if args is not False and method is not False:
        result = method(args)
    else:
        result = method()

If some error is thrown in the execution of the method, a popup meesage is shown to the user. To understand what is env in the previous code you can read the answer of my own question in here (env = BokehSharedData in the answer)

Sometimes I needed to call JS code from python, so what I did is to send a signal in the other direction. I also use this to send the answer to the front-end and continue with the execution in the tools.call_promise function

def call_js(self, params={}):
    if params != {}:
        params = json.dumps(params, sort_keys=True)
    signal = 'js-call'
    js_code = """
        window.top.postMessage({{
            'signal': '{}',
            'params': {},
        }}, '*');                        // to main_renderer.js
    """.format(signal, params)
    self.bridge_plot_callback.code = js_code
    self.bridge_trigger.glyph.size += 1  # triggers the callback

Note: I know there is another way to embed the bokeh server (with server_document instead of using an iframe). But time ago I got poorer performance results. Though I think it is worthwhile to try it again.

Note2: This works well if the arguments to send from both sides are small. If the amount of data is bigger, sending signals is not a good approach.

I would build a minimal example with all of this or a simpler version, but I think this is still a cumbersome integration. If I find a better solution or if there are some new functionalities in Bokeh to make all of this easier I will update the post.

What do you think?

Hey @jcacabelos!

Thanks a lot for such a great job of connecting electron with Bokeh. I’ve started working on an application that follows in your footsteps, so again, thanks for laying out the groundwork!

In relation to your Note1, did you consider loading bokeh server localhost instead of adding it as an iframe? It seems to be a lot faster (in my case at least).
e.g.

ipcRenderer.on("load-bokeh-in-window", (event, args) => {
    logger.info("-- LOAD BOKEH IN NEW WINDOW -- ");

    // get web_contents
    web_contents = require('electron').remote.getCurrentWebContents()
    let html_path = "http://localhost:" + bokeh_port + "/app?backend=webgl"
    web_contents.loadURL(html_path);
});
1 Like

Well, the decision of using an iframe was time ago, so I don’t have the reasons fresh haha.

Anyway I think I did it in that way to keep the interaction between bokeh and electron. I believe that if I loaded bokeh as you suggest I could not trigger operations from electron to bokeh and viceversa and keep all the electron interacion (ipcMain + ipcRender) intact. And I reckon that’s why I used the signaling approach. Let me know if I am wrong.

  • Are you using pure JS and calling the bokeh api directly after loading the application like that? If not, how do you trigger any action in bokeh?
  • Can you send signals from ipcMain to ipcRenderer and viceversa without any problem?

This is the diagram of my approach:

Thanks for the reply.
I am still trying to wrap my head around the ipcMain/ipcRenderer architecture, but I managed to maintain the cooperation between the main and renderer processes. Actions such as menu, context menu and similar, are sitting under the ipcMain process whereas bokeh servers localhost is a ipcRenderer. My app is still somewhat less complex than yours, but I can send data from python → javascript and viceversa.

To trigger event in python from JS, I use your approach by having a couple of invisible widgets in the bokeh document (TextInput for exchanging data and Button to trigger exchange which are called using jQuery. In order to trigger events from python in electron I simply use bokeh’s CustomJS functionality. The code is currently pure JS on the electron side and python on bokeh side.

1 Like