DataCube Content Refresh Issue

[Anthony_Wang]

Hello,

I’m trying use DataCube to represent the bit statuses that are decoded from different integer codes. Ultimately I want to create something like this:

• Tier 1 Code
  • Bit 1 Position | Bit 1 Definition | Bit 1 status
  • Bit 2 Position | Bit 2 Definition | Bit 2 status
    …
• Tier 2 Code
  • Bit 1 Position | Bit 1 Definition | Bit 1 status
  • Bit 2 Position | Bit 2 Definition | Bit 2 status
    …
• Tier 3 Code
  • Bit 1 Position | Bit 1 Definition | Bit 1 status
  • Bit 2 Position | Bit 2 Definition | Bit 2 status
    …

I have a text box for user to enter the codes, and when submit is pressed, the table should be updated with the newly decoded values. I chose DataCube because I like the idea of making my table collapsible so only the statuses from the expanded system is shown rather than displayed all at once.

I have a callback for the DataCube that updates the data content in the ColumnDataSource when new values are submitted by the user, and the values are being updated correctly. However, the issue I have is the updated table values don’t reflect on the DataCube until I either collapse/expand a row, or when I scroll up and down on the table. It’s almost as if the DataCube has a refresh functionality that is called only when there is a display change on the table, but that functionality is not triggered when data source is updated. Am I doing anything wrong, or is this a known issue/feature of the DataCube. If so, are there any workarounds?

I won’t provide the entire code to avoid overcrowding this question, but CreateStackStatusList() basically retrieves the current bit status based on what the user inputs and decodes them. source data is udpated with a pandas dataframe. I’ve test the user input and data conversion pipeline and it works as expected. Plus, when the data is refreshed on the DataCube, the values are correct. I’m just struggling to figure out if there is a way to manually refresh the DataCube in my callback function. Thanks!

Here is a snippet of my current data cube:

Here is the code:

def submit_callback():
    """
    call back function when submit button is pressed. 
    """
    table_source.data = pd.DataFrame(CreateStackStatusList())
    cube.columns = cube.columns
    cube.source = table_source

table_source = ColumnDataSource(data=pd.DataFrame(CreateStackStatusList()))


target = ColumnDataSource(data=dict(row_indices=[], labels=[]))
formatter = StringFormatter(font_style='bold')
columns = [
    TableColumn(field='Bit', title='Bit', width=40),
    TableColumn(field='Name', title='Bit Name', width=100, formatter=formatter),
    TableColumn(field='Status', title='Bit Status', width=50, formatter=HTMLTemplateFormatter(
        template='<div style="background:<%= (value == 1) ? "lightgreen" : "lightpink" %>;"><%= value %></div>')),
]


grouping = [ GroupingInfo(getter='Tier', aggregators=[]), ]
cube = DataCube(source=table_source, columns=columns, grouping=grouping, target=target, height=1000)

button = Button(label="Submit")
button.on_event('button_click', submit_callback)

[Anthony_Wang]

I was able to make a minimum reproducible code that should be runnable. To run, just need to use bokeh serve

from bokeh.models import ColumnDataSource, DataTable
from bokeh.models import Label, NumericInput, Button
from bokeh.layouts import column, row
from bokeh.io import curdoc
from bokeh.layouts import layout
from bokeh.models import (ColumnDataSource, DataCube, GroupingInfo,
                          StringFormatter, SumAggregator, TableColumn)
import pandas as pd

# Define the data for each tier and status

def ReadUserInputAndReturnList():
    """
    Create a list of bit status to display in tabulated format.
    Returned list is in the form of:
    [
        {'Tier': tier_name, 'Bit': bit_position, 'Name': bit_name, 'Status': 'bit_status'},
        {'Tier': tier_name, 'Bit': bit_position, 'Name': bit_name, 'Status': 'bit_status'},
        ...
        ...
        {'Tier': tier_name, 'Bit': bit_position, 'Name': bit_name, 'Status': 'bit_status'},
    ]
    """
    data_list = [
        {'Tier': 'Tier1', 'Bit': 1, 'Status': 'Status A', 'Value': tier1_bit1_input.value if tier1_bit1_input.value else 0},
        {'Tier': 'Tier1', 'Bit': 2, 'Status': 'Status B', 'Value': tier1_bit2_input.value if tier1_bit2_input.value else 0},
        {'Tier': 'Tier1', 'Bit': 3, 'Status': 'Status C', 'Value': tier1_bit3_input.value if tier1_bit3_input.value else 0},
        {'Tier': 'Tier2', 'Bit': 1, 'Status': 'Status D', 'Value': tier2_bit1_input.value if tier2_bit1_input.value else 0},
        {'Tier': 'Tier2', 'Bit': 2, 'Status': 'Status E', 'Value': tier2_bit2_input.value if tier2_bit2_input.value else 0},
        {'Tier': 'Tier2', 'Bit': 3, 'Status': 'Status F', 'Value': tier2_bit3_input.value if tier2_bit3_input.value else 0},
        {'Tier': 'Tier3', 'Bit': 1, 'Status': 'Status G', 'Value': tier3_bit1_input.value if tier3_bit1_input.value else 0},
        {'Tier': 'Tier3', 'Bit': 2, 'Status': 'Status H', 'Value': tier3_bit2_input.value if tier3_bit2_input.value else 0},
        {'Tier': 'Tier3', 'Bit': 3, 'Status': 'Status I', 'Value': tier3_bit3_input.value if tier3_bit3_input.value else 0},
    ]
    return data_list

def submit_callback():
    """
    call back function when submit button is pressed. 
    """
    udpated_data = ReadUserInputAndReturnList()
    source.data = pd.DataFrame(udpated_data)
    # source.columns = cube.columns
    # source.source = source
    print('done')

# Setup UI Widgets
tier1_bit1_input  = NumericInput(value=0, low=0, high=100, title="Enter Tier 1 Bit 1 Stat:", width=150)
tier1_bit2_input  = NumericInput(value=0, low=0, high=100, title="Enter Tier 1 Bit 2 Stat:", width=150)
tier1_bit3_input  = NumericInput(value=0, low=0, high=100, title="Enter Tier 1 Bit 3 Stat:", width=150)
tier2_bit1_input  = NumericInput(value=0, low=0, high=100, title="Enter Tier 2 Bit 1 Stat:", width=150)
tier2_bit2_input  = NumericInput(value=0, low=0, high=100, title="Enter Tier 2 Bit 2 Stat:", width=150)
tier2_bit3_input  = NumericInput(value=0, low=0, high=100, title="Enter Tier 2 Bit 3 Stat:", width=150)
tier3_bit1_input  = NumericInput(value=0, low=0, high=100, title="Enter Tier 3 Bit 1 Stat:", width=150)
tier3_bit2_input  = NumericInput(value=0, low=0, high=100, title="Enter Tier 3 Bit 2 Stat:", width=150)
tier3_bit3_input  = NumericInput(value=0, low=0, high=100, title="Enter Tier 3 Bit 3 Stat:", width=150)
button = Button(label="Submit")
button.on_event('button_click', submit_callback)

# Create a ColumnDataSource from the data
source = ColumnDataSource(data=pd.DataFrame(ReadUserInputAndReturnList()))

# Create the target ColumnDataSource
target = ColumnDataSource(data=dict(row_indices=[], labels=[]))

# Create the formatter for bold text
formatter = StringFormatter(font_style='bold')

# Define the columns of the DataCube
columns = [
    TableColumn(field='Bit', title='Bit', width=50),
    TableColumn(field='Status', title='Bit Definition', width=100),
    TableColumn(field='Value', title='Bit Value', width=100),
]

# Define the grouping based on 'Tier'
grouping = [
    GroupingInfo(getter='Tier', aggregators=[]),
]

# Create the DataCube widget
cube = DataCube(source=source, columns=columns, grouping=grouping, target=target)



input_box = row(tier1_bit1_input,tier1_bit2_input,tier1_bit3_input,tier2_bit1_input,tier2_bit2_input,tier2_bit3_input,tier3_bit1_input,tier3_bit2_input,tier3_bit3_input)
inputs = row(input_box, button, width=1200, height=100)
layout = column(inputs, cube, sizing_mode="stretch_width", height=800)
# Add the layout to the current document
curdoc().add_root(layout)

[Anthony_Wang]

To reproduce the issue:

1. Start with initial condition where tier 1 is expanded in the datacube, and numeric input is set to default values:
   
   

[Anthony_Wang]

2. Enter values into Tier 1 numeric input boxes, and click submit. The DataCube is not updated.
   
   

[Anthony_Wang]

3. Expand Tier 2…now Tier 1 values in the DataCube is updated.
   
   

[Anthony_Wang]

However, once the data cube is expanded, the values on the datacube updates as soon as user input is submitted.

[Bryan]

@Anthony_Wang The DataCube was added a number of years ago by a new contributor, who has not since stayed around to continue work and development on it. To be completely honest, I would describe DataCube as very little-used and not as well-supported as most of the library. Besides some unit tests, there’s a single simple example that typically gets checked at releases. Otherwise I don’t think most of the active core devs have much experience with it. With hindsight, it’s arguable we should not have accepted into the library at the time, without a clearer plan and commitment from someone regarding ongoing maintenance.

TLDR; if you think something about DataCube is not working as you would expect, there’s a very high chance its just a bug. All I can advise at this point is to submit a GitHub Issue with full details, but I can’t make any promises about when it might be looked at or resolved. (Or even that a resolution might not just be to deprecate it entirely.)

[Anthony_Wang]

@Bryan Thanks for taking the time to explain this.