Crossplatform Tool for 1st Quality Control on Hydrographic Cruise Data

We have released a Crossplatform Tool for 1st Quality Control on Hydrographic Cruise Data, developed at the IIM (CSIC) and funded by AtlantOS.

We have embedded and packaged Bokeh with Electron. The communication between both works through JS signaling.

Also, we’ve integrated octave/matlab routine calculations in the application. The motivation of doing this (because you can do all this with python) is the availability of many tested code in that language and easiness for potential users that come from that language.

You can check all the details and give feedback in the GitHub Project

3 Likes

This is really great! Can you give a high level overview of what the Electron integration looks like ? What would it take to make a toy demo example? Are there things that could go in the core library to make things better or simpler?

Yes, I think the possibility to make a simple RPC (calling Python methods and get the answer directly from Electron) will help a lot. Besides, the code to communicate both would be less cumbersome.

Actually the connection I made is kind of hacky, using dummy invisible objects and sending signals from JavaScript. I have also applied the solution you gave me time ago, and I built something similar in the opposite direction.

I am going to be on vacation for a couple of weeks. Let’s see if I can make some example when I go back. Or maybe I can point to the important sections in the source code of the project

1 Like

Well, I write here some general explanation about the Electron integration I made with the Bokeh Server.

Firstly I run bokeh with the library python-shell this way:

var user_options = [
    '-m', 'bokeh', 'serve',
    '--port', self.bokeh_port,
    '--log-format', '"%(asctime)s %(levelname)s %(message)s"',
    '--log-file', loc.log_python
]
self.python_options = {
    mode: 'text',               // actually I do not need to return anything,
    pythonPath: self.python_path,
    pythonOptions: aux_options,
    scriptPath: self.ocean_data_qc_path
};
self.shell = python_shell.run(
    '', self.python_options, (err, results) => {
        if (err || typeof(results) == 'undefined') {
            // ... the bokeh logger is used instead of getting the messages here

Then, the Electron process should be bound to the python process where the Bokeh Server is running.

I built a mechanism to call python methods “on the fly” as it was a web service or a simple rpc call. I make a call and I get an answer asynchronously like this:

var expr_name = self.comp_param_name.val();
var params = {
    'object': 'computed.parameter',
    'method': 'compute_equation',
    'args': {
        'computed_param_name': expr_name,
        'eq': self.equation_text.val(),
    }
}
tools.call_promise(params).then((result) => {
    lg.info('>> VALIDATE EQUATION RESULT: ' + JSON.stringify(result, null, 4));
    if ('success' in result && result['success'] == true) {

The call_promise function is responsible for sending a signal (it waits for the answer as well) to the iframe where bokeh is running. I built a dummy bokeh form where I fill a text input with the information I want to send to python. Then I trigger the click event of a button and I already have the information in the python side. This is kind of hacky as you can see:

window.onmessage = function(e){
    if (e.data.signal == 'call-python-promise') {
        // this updates dummy text field value and triggers the click event of the bridge_button

        var models = window.Bokeh.index[Object.keys(window.Bokeh.index)[0]].model.document._all_models;
        var model_id = null;
        $.each(models, function (m_id) {
            if (models[m_id].constructor.name == 'TextInput') {
                model_id = m_id;
                // console.log('>> TextInput Name: ' + models[m_id].attributes.name)
            }
        });
        var input_bridge_text = models[model_id];
        input_bridge_text.value = JSON.stringify(e.data.message_data);
        var button = $('.bridge_button>div>button')
        button.click();

If I remember well I needed to add the environment variable BOKEH_SIMPLE_IDS=true to make this code work because I work with the ids that bokeh generates for the objects.

Finally I receive the data in the python side and I run the suitable method:

elif obj == 'computed.parameter':
    method = getattr(self.env.cruise_data.cp_param, method_str)

result = False
try:
    if args is not False and method is not False:
        result = method(args)
    else:
        result = method()

If some error is thrown in the execution of the method, a popup meesage is shown to the user. To understand what is env in the previous code you can read the answer of my own question in here (env = BokehSharedData in the answer)

Sometimes I needed to call JS code from python, so what I did is to send a signal in the other direction. I also use this to send the answer to the front-end and continue with the execution in the tools.call_promise function

def call_js(self, params={}):
    if params != {}:
        params = json.dumps(params, sort_keys=True)
    signal = 'js-call'
    js_code = """
        window.top.postMessage({{
            'signal': '{}',
            'params': {},
        }}, '*');                        // to main_renderer.js
    """.format(signal, params)
    self.bridge_plot_callback.code = js_code
    self.bridge_trigger.glyph.size += 1  # triggers the callback

Note: I know there is another way to embed the bokeh server (with server_document instead of using an iframe). But time ago I got poorer performance results. Though I think it is worthwhile to try it again.

Note2: This works well if the arguments to send from both sides are small. If the amount of data is bigger, sending signals is not a good approach.

I would build a minimal example with all of this or a simpler version, but I think this is still a cumbersome integration. If I find a better solution or if there are some new functionalities in Bokeh to make all of this easier I will update the post.

What do you think?