Loading issues in Angular SPA due to circular dependencies in bokehjs lib

We are using Bokeh 2.2.3 in a setup where the @bokeh/bokehjs lib is npm installed into our Angular 9 SPA, and we are “linking” against the @bokeh/bokehjs/build/js/lib/* files. We just switched from Bokeh 2.0.2 and in 2.2.3 we started to get uncaught ReferenceErrors. Quick research told us these are typically manifestation of circular import dependencies. The attached image shows an example, with one such loop

Screen Shot 2020-11-18 at 11.00.23 PM2730×842 406 KB

I ran a more complete analysis on the entire lib directory of 2.2.3, using “madge” (GitHub - pahen/madge: Create graphs from your CommonJS, AMD or ES6 module dependencies), and it has found 13 circular dependency loops in the code base:

Zharaszti-mlt lib $ madge --circular .
Processed 429 files (4.1s) (28 warnings)

Found 13 circular dependencies!

1. core/util/array.js > core/util/arrayable.js > core/util/types.js
2. core/has_props.js > document/index.js > document/document.js > base.js
3. models/index.js > models/annotations/index.js > models/annotations/annotation.js > models/renderers/renderer.js > core/visuals.js > models/canvas/canvas.js > core/has_props.js > document/index.js > document/document.js > base.js
4. core/has_props.js > document/index.js > document/document.js
5. core/has_props.js > document/index.js > document/document.js > document/events.js
6. core/has_props.js > document/index.js > document/document.js > model.js
7. models/canvas/canvas.js > core/has_props.js > document/index.js > document/document.js > models/layouts/layout_dom.js
8. core/has_props.js > document/index.js > document/document.js > models/sources/column_data_source.js > models/sources/columnar_data_source.js > core/selection_manager.js
9. core/visuals.js > models/canvas/canvas.js > core/has_props.js > document/index.js > document/document.js > models/sources/column_data_source.js > models/sources/columnar_data_source.js > core/selection_manager.js > models/renderers/glyph_renderer.js > models/glyphs/harea.js > models/glyphs/area.js > models/glyphs/glyph.js
10. models/renderers/renderer.js > core/visuals.js > models/canvas/canvas.js > core/has_props.js > document/index.js > document/document.js > models/sources/column_data_source.js > models/sources/columnar_data_source.js > core/selection_manager.js > models/renderers/glyph_renderer.js > models/renderers/data_renderer.js
11. models/sources/columnar_data_source.js > core/selection_manager.js > models/renderers/glyph_renderer.js > models/sources/cds_view.js
12. models/glyphs/circle.js > models/glyphs/webgl/markers.js
13. models/widgets/tables/cell_editors.js > models/widgets/tables/data_table.js > models/widgets/tables/table_column.js

Any advice?

PS In case anyone tries to run madge on the lib dir under node_modules, it will produce an empty graph, because it detects that the dependencies are under node_modules, hence ignores them. One has to copy the lib directory out to a tmp dir and run madge there. In case this helps others…

Interesting tidbit: in Bokeh 2.0.2, the previous version with which we integrated, madge also detects circular dependencies, but only two instances:

Zharaszti-mlt lib $ madge . --circular
Processed 395 files (3s) (1 warning)

Found 2 circular dependencies!

1. models/glyphs/glyph.js > models/glyphs/line.js > models/glyphs/xy_glyph.js
2. models/sources/columnar_data_source.js > core/selection_manager.js > models/renderers/glyph_renderer.js > models/sources/cds_view.js

I was wondering why these did not result in the same issue, but I think not all circular dependencies (at the import level) are necessarily bad, only those where references to exported objects are used before the code that defines them actually initializes them. So it may be that a few surgical rearrangements in 2.2.3 could fix the issue.

cc @mateusz can you comment on this?

Having cycling module dependencies isn’t ideal, but also isn’t uncommon. Despite those cycles, our bundles work fine, so I’m wondering why webpack fails. In any case, most of those either trivial to fix and/or look pretty bad from software engineering perspective, so we will get those resolved. I started PR Detect dependency cycles in build and fix existing cycles by mattpap · Pull Request #10707 · bokeh/bokeh · GitHub, which adds cycle detection to our build system and starts to fix those issues.

@mateusz do you have any suggestions for what could be tried / investigated in the mean time?

@zsolt do you have a toy reproducer handy that you could share?

@Bryan @mateusz
I can try to put together a simple Angular hello world app that installs and preloads Bokeh in the way we do and which recreates the issue. If I manage to do that, I will publish it on github so folks can fork/clone it. Would that work?

@zsolt that would be very helpful!

@mateusz @Bryan

Here is the “repro redux,” a simple Angular 9 app with just a few line changes that manifests the issue. The initial commit is made by the angular CLI and is a working boilerplate. The 2nd contains essentially two things:

1. The result of npm i -s @bokeh/bokehjs
2. A single import of one bokeh model (LayoutDOM) and a reference to it to pass tree shaking.

To reproduce the issue:

git clone git@github.com:zsolt-haraszti/bokeh-angular-app-to-recreate-bokeh-import-issue.git
cd bokeh-angular-app-to-recreate-bokeh-import-issue
npm i
ng serve --port 3333 -o

On my Mac the last step opens a new Chrome tab, which stays blank. The developer console shows the import error:

Screen Shot 2020-11-19 at 10.23.57 PM2202×1988 281 KB

@mateusz @Bryan
To illustrate how this error is a hit or miss, I added a 2nd branch to the above github repo which inserts just one additional bokeh import before the previous one, and all of a sudden the import runtime error is gone. So the issue is very much dependent on the order by which bokeh import statements are inserted in the host code and encountered by the webpack loader.

This also gives me a hope that I can find a quick workaround to our issue to bridge us in the meantime.

I am happy to report that with a one-line change in our production app we can avoid the import issue, so we are unblocked.

I strongly suggest to follow through with PR https://github.com/bokeh/bokeh/pull/10707 since this issue can hit others (or even us again), and I think @mateusz you think so too.

Best,
Zsolt

@zsolt

Can you elaborate on the one-line workaround that unblocked your issue?

There is another user @MarkM experiencing a similar issue here …

I checked it the branch of the github repo that @zsolt mentioned and i can see a fix to get bokeh to load was

import {add_document_standalone} from ‘@bokeh/embed’;

I will try to investigate further. But the one line fix that zsolt mentioned was after this so i would still be keen to hear what that was as i suspect it was something different.

I still cannot get this to work at all. I have tried different combinations of imports with no success at all. Too many circular dependencies.

Based on the repo by @zsolt I added this line
import {add_document_standalone} from ‘@bokeh/bokehjs/build/js/lib/embed/standalone’;
and I get a “ReferenceError: Cannot access ‘ColumnarDataSource’ before initialization” error on that line.

Is anyone able to help? This is a complete blocker.

FYI