Bokeh server with geopandas, pyproj

Community Support
,
1

I have a server app that I am unable to run due to this error when a request to the server is made:

2022-11-10 10:48:30,259 Error running application handler <bokeh.application.handlers.script.ScriptHandler object at 0x7fee615112b0>: module 'importlib' has no attribute 'metadata'
File '__init__.py', line 70, in <module>:
__version__ = importlib.metadata.version(__package__) Traceback (most recent call last):
  File "/home/UA/tcarman2/miniconda3/envs/py38/lib/python3.8/site-packages/bokeh/application/handlers/code_runner.py", line 231, in run
    exec(self._code, module.__dict__)
  File "/home/UA/tcarman2/dvm-dos-tem/scripts/io_view.py", line 23, in <module>
    import geopandas
  File "/home/UA/tcarman2/miniconda3/envs/py38/lib/python3.8/site-packages/geopandas/__init__.py", line 1, in <module>
    from geopandas._config import options  # noqa
  File "/home/UA/tcarman2/miniconda3/envs/py38/lib/python3.8/site-packages/geopandas/_config.py", line 109, in <module>
    default_value=_default_use_pygeos(),
  File "/home/UA/tcarman2/miniconda3/envs/py38/lib/python3.8/site-packages/geopandas/_config.py", line 95, in _default_use_pygeos
    import geopandas._compat as compat
  File "/home/UA/tcarman2/miniconda3/envs/py38/lib/python3.8/site-packages/geopandas/_compat.py", line 9, in <module>
    import pyproj
  File "/home/UA/tcarman2/miniconda3/envs/py38/lib/python3.8/site-packages/pyproj/__init__.py", line 70, in <module>
    __version__ = importlib.metadata.version(__package__)
AttributeError: module 'importlib' has no attribute 'metadata'

The error seems to come from the import geopandas, however I am able to import geopandas just fine if I am not using bokeh, i.e.:

$ python --version
Python 3.8.13
$ python -c "import geopandas as gpd; print(gpd.__version__)"
0.12.1

I also have the importlib.metadata package installed and all other programs seem to be using it just fine, but for some reason when I have imported bokeh

2

What version of Bokeh? What platform? Can you provide a complete Minimal Reproducible Example?

Otherwise all I can note offhand is that the import in your code with the exception is different from the “test import” at the bottom of the post, so you should verify that the actual same import from the app works.

Edit: also, it looks like the issue is actually with pyproj, not geopandas itself:

  File "/home/UA/tcarman2/miniconda3/envs/py38/lib/python3.8/site-packages/pyproj/__init__.py", line 70, in <module>
    __version__ = importlib.metadata.version(__package__)
3

Hi, sorry, I am hopelessly tangled up in this stuff so will do my best:

  1. bokeh==3.0.1, installed with pip inside a conda environment. I setup the Conda environment as follows:
conda create -n py38 python=3.8
conda install gdal    # for libgdal
pip install gdal      # for python bindings
python -c "from osgeo import gdal" # works
pip install ipython pandas geopandas fiona rasterio
pip install netCDF4 configobj
  1. These imports work fine in my same Conda environment (cribbed from my Traceback above):
>>> import geopandas._compat as compat
>>> import pyproj
>>> from geopandas._config import options

Minimum Reproducible Example:

Platform: On a compute note of an HPC cluster running CentOS v7.x I think. I have been using Conda to manage packages in my home directory.

Run with:

$ bokeh serve --port 8686 MRE_io_view.py 
2022-11-10 11:47:43,312 Starting Bokeh server version 3.0.1 (running on Tornado 6.2)
2022-11-10 11:47:44,091 User authentication hooks NOT provided (default user enabled)
2022-11-10 11:47:44,102 Bokeh app running at: http://localhost:8686/MRE_io_view
2022-11-10 11:47:44,102 Starting Bokeh server with process id: 10551
BokehDeprecationWarning: tile_providers module was deprecated in Bokeh 3.0.0 and will be removed, use add_tile directly instead.
2022-11-10 11:47:48,508 Error running application handler <bokeh.application.handlers.script.ScriptHandler object at 0x7f47b76e3070>: module 'importlib' has no attribute 'metadata'
File '__init__.py', line 70, in <module>:
__version__ = importlib.metadata.version(__package__) Traceback (most recent call last):
  File "/home/UA/tcarman2/miniconda3/envs/py38/lib/python3.8/site-packages/bokeh/application/handlers/code_runner.py", line 231, in run
    exec(self._code, module.__dict__)
  File "/home/UA/tcarman2/dvm-dos-tem/scripts/MRE_io_view.py", line 28, in <module>
    import geopandas
  File "/home/UA/tcarman2/miniconda3/envs/py38/lib/python3.8/site-packages/geopandas/__init__.py", line 1, in <module>
    from geopandas._config import options  # noqa
  File "/home/UA/tcarman2/miniconda3/envs/py38/lib/python3.8/site-packages/geopandas/_config.py", line 109, in <module>
    default_value=_default_use_pygeos(),
  File "/home/UA/tcarman2/miniconda3/envs/py38/lib/python3.8/site-packages/geopandas/_config.py", line 95, in _default_use_pygeos
    import geopandas._compat as compat
  File "/home/UA/tcarman2/miniconda3/envs/py38/lib/python3.8/site-packages/geopandas/_compat.py", line 9, in <module>
    import pyproj
  File "/home/UA/tcarman2/miniconda3/envs/py38/lib/python3.8/site-packages/pyproj/__init__.py", line 70, in <module>
    __version__ = importlib.metadata.version(__package__)
AttributeError: module 'importlib' has no attribute 'metadata'
 
2022-11-10 11:47:48,837 WebSocket connection opened
2022-11-10 11:47:48,838 ServerConnection created

Code:

#!/usr/bin/env python

# Minimum Reproducible Example for 
# Interactive viewer for mapping collections of dvm-dos-tem input datasets.

import os

from osgeo import gdal
from osgeo import osr

import shapely

import geojson
import json
import glob

from bokeh.io import output_file, show, curdoc, save
from bokeh.models.sources import GeoJSONDataSource
from bokeh.models import HoverTool, TapTool, Patches
from bokeh.models.callbacks import CustomJS
from bokeh.plotting import figure

from bokeh.tile_providers import get_provider, Vendors

import geopandas

print("I am here.")

p = figure()
curdoc().add_root(p)

Requirements.txt:

affine==2.3.1
anyio==3.6.2
argon2-cffi==21.3.0
argon2-cffi-bindings==21.2.0
asttokens==2.1.0
attrs==22.1.0
backcall==0.2.0
beautifulsoup4==4.11.1
bleach==5.0.1
bokeh==3.0.1
certifi==2022.9.24
cffi==1.15.1
cftime==1.6.2
click==8.1.3
click-plugins==1.1.1
cligj==0.7.2
configobj==5.0.6
contourpy==1.0.6
coverage==5.5
debugpy==1.6.3
decorator==5.1.1
defusedxml==0.7.1
deprecation==2.1.0
entrypoints==0.4
executing==1.2.0
eyed3==0.9.7
fastjsonschema==2.16.2
filetype==1.2.0
Fiona==1.8.22
GDAL==3.5.3
geojson==2.5.0
geopandas==0.12.1
idna==3.4
importlib-resources==5.10.0
ipykernel==6.17.1
ipython==8.6.0
ipython-genutils==0.2.0
ipywidgets==8.0.2
jedi==0.18.1
Jinja2==3.1.2
jsonschema==4.17.0
jupyter==1.0.0
jupyter-console==6.4.4
jupyter-server==1.23.1
jupyter_client==7.4.5
jupyter_core==5.0.0
jupyterlab-pygments==0.2.2
jupyterlab-widgets==3.0.3
MarkupSafe==2.1.1
matplotlib-inline==0.1.6
mistune==2.0.4
munch==2.5.0
nbclassic==0.4.8
nbclient==0.7.0
nbconvert==7.2.4
nbformat==5.7.0
nest-asyncio==1.5.6
netCDF4==1.6.1
notebook==6.5.2
notebook_shim==0.2.2
numpy @ file:///home/conda/feedstock_root/build_artifacts/numpy_1666788162538/work
packaging==21.3
pandas==1.5.1
pandocfilters==1.5.0
parso==0.8.3
pexpect==4.8.0
pickleshare==0.7.5
Pillow==9.3.0
pkgutil_resolve_name==1.3.10
platformdirs==2.5.3
prometheus-client==0.15.0
prompt-toolkit==3.0.32
psutil==5.9.4
ptyprocess==0.7.0
pure-eval==0.2.2
pycparser==2.21
pyglet==1.1.4
Pygments==2.13.0
pyparsing==3.0.9
pyproj==3.4.0
pyrsistent==0.19.2
python-dateutil==2.8.2
pytz==2022.6
PyYAML==6.0
pyzmq==24.0.1
qtconsole==5.4.0
QtPy==2.3.0
rasterio==1.3.3
Send2Trash==1.8.0
Shapely==1.8.5.post1
six==1.16.0
sniffio==1.3.0
snuggs==1.4.7
soupsieve==2.3.2.post1
stack-data==0.6.0
terminado==0.17.0
tinycss2==1.2.1
toml==0.10.2
tornado==6.2
traitlets==5.5.0
wcwidth==0.2.5
webencodings==0.5.1
websocket-client==1.4.2
widgetsnbextension==4.0.3
xyzservices==2022.9.0
zipp @ file:///home/conda/feedstock_root/build_artifacts/zipp_1666647772197/work
4

Does manually (re-)installing importlib_metadata as suggested here:

make any difference? If not I will have to try to reproduce locally with the information above.

5

@Bryan I found that same thread and tried installing and upgrading importlib_metadata several times and several ways, to no avail… I am really at a loss here and this is exposing how little I understand about python import system and packaging in general…sorry :face_with_spiral_eyes:

6

I’m fairly new to Conda (used pyenv mostly in the past) so I am not sure if I provided enough info for replicating. Let me know if I missed anything.

7

Well, this is really strange, and I don’t really understand the details. However, I think pyproj needs to implement the same fix as mypy did here:

Specifically, I think they need to do this:

import importlib.metadata as importlib_metadata

including the as rename. I don’t know why that should be necessary, there is something very odd about importlib.metadata in my experience (have seen similar issues with it elsewhere). But if I make that change by hand in pyproj/__init__.py then things work. I’d encourage you to open an issue on the pyproj issue tracker about this.

10

I’ve also made an inquiry on the main Python forum

11

Interestingly on my end, if I manually edit pyproj/__init__.py as you did, it does not fix the problem for me, but I do get a different Traceback:

2022-11-10 13:05:49,462 Error running application handler <bokeh.application.handlers.script.ScriptHandler object at 0x7f5dfb918160>: name 'importlib' is not defined
File '__init__.py', line 70, in <module>:
__version__ = importlib.metadata.version(__package__) Traceback (most recent call last):
  File "/home/UA/tcarman2/miniconda3/envs/py38/lib/python3.8/site-packages/bokeh/application/handlers/code_runner.py", line 231, in run
    exec(self._code, module.__dict__)
  File "/home/UA/tcarman2/dvm-dos-tem/scripts/io_view.py", line 38, in <module>
    import geopandas
  File "/home/UA/tcarman2/miniconda3/envs/py38/lib/python3.8/site-packages/geopandas/__init__.py", line 1, in <module>
    from geopandas._config import options  # noqa
  File "/home/UA/tcarman2/miniconda3/envs/py38/lib/python3.8/site-packages/geopandas/_config.py", line 109, in <module>
    default_value=_default_use_pygeos(),
  File "/home/UA/tcarman2/miniconda3/envs/py38/lib/python3.8/site-packages/geopandas/_config.py", line 95, in _default_use_pygeos
    import geopandas._compat as compat
  File "/home/UA/tcarman2/miniconda3/envs/py38/lib/python3.8/site-packages/geopandas/_compat.py", line 9, in <module>
    import pyproj
  File "/home/UA/tcarman2/miniconda3/envs/py38/lib/python3.8/site-packages/pyproj/__init__.py", line 70, in <module>
    __version__ = importlib.metadata.version(__package__)
NameError: name 'importlib' is not defined
12

You also have to make this corresponding edit:

    __version__ = importlib_metadata.version(__package__)

edit: specifically, use importlib_metadata there, not importlib.metadata

13

Ahh, of course!
Phew that fixed it! I will try to write this up on pyproj issue tracker. Thanks!!! I am still fairly confused, but at least got things running and learned something on the way…

14

FWIW Bokeh server is in the unusual position of being a Python application that runs other Python applications, which sometimes introduces weirdness. I can believe there is something we might need to do on our end to make things more “normal” for the app code, but I don’t know what that might be. Hopefully wider discussions can shed some light,

15

Posted to proj where I was quickly informed that my issue was duplicate. I was just too thick to understand the connection! :stuck_out_tongue_winking_eye:

My write up:

Original issue:

16

@tcarman I think we also found a solution for a change on the Bokeh end: Module 'importlib' has no attribute 'metadata' - #12 by bryevdv - Packaging - Discussions on Python.org

17

And a a Bokeh issue for completeness:

This should be in a 3.1 or 3.0.2 release around new years, unless we get a bunch of reports then we might cut a 3.0.2 sooner.

18

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.