Have you tried Geoviews? Supposed to be able to use it in different projections: Projections — GeoViews 1.9.2+gad6e3f0-dirty documentation
I’d like to use it in 4326 as well, or even better, a Cartesian projection like a UTM. It would be nice to have the tiles work as a basemap but not have to be in Web Mercator. I’d be curious if you get 4326 to work with Geoviews and the tiles working too.