[FEATURE] Generate graphs in canvas from query data by PhilippMatthes · Pull Request #891 · perses/plugins

PhilippMatthes · 2026-10-23T14:27:22Z

Description

Add a data-driven mode to the canvas panel that derives nodes and edges live from query results instead of a hand-drawn graph, addressing the need to visualize dynamic topologies (e.g. frequently changing service or host relationships) where manual authoring is not feasible.

Closes perses/perses#4634

Infer graph structure from series labels: one node per distinct node-id label value, edges from configurable source/target labels (endpoints not among the generated nodes are dropped).
Lay out generated graphs with a deterministic, dependency-free force simulation so a given topology always settles to the same positions; nodes only move when nodes/edges are added or removed, not when metric values change.
Drive visuals from each series' last value: node color via panel thresholds, optional node-size scaling via per-threshold multipliers, edge color via dedicated edge thresholds (falling back to node thresholds), edge thickness via edge-thickness thresholds, and label interpolation for node/edge captions.
Add a Manual / From data mode toggle to the settings editor with the shared styling controls plus the new graph-generation and node-size panels.
Switch the read-only panel to view-mode zoom/pan (plain wheel zoom, left/middle drag pan) and clip edges to node perimeters so lines and arrowheads touch the shape boundary.
Keep every representation coherent: TypeScript runtime/editor, CUE schema, Go SDK builders, docs, and tests.

Screenshots

Checklist

Pull request has a descriptive title and context useful to a reviewer.
Pull request title follows the [<catalog_entry>] <commit message> naming convention using one of the
following catalog_entry values: FEATURE, ENHANCEMENT, BUGFIX, BREAKINGCHANGE, DOC,IGNORE.
All commits have DCO signoffs.

UI Changes

Changes that impact the UI include screenshots and/or screencasts of the relevant changes.
Code follows the UI guidelines.

Add a data-driven mode to the canvas panel that derives nodes and edges live from query results instead of a hand-drawn graph, addressing the need to visualize dynamic topologies (e.g. frequently changing service or host relationships) where manual authoring is not feasible. Closes perses/perses#4634 - Infer graph structure from series labels: one node per distinct node-id label value, edges from configurable source/target labels (endpoints not among the generated nodes are dropped). - Lay out generated graphs with a deterministic, dependency-free force simulation so a given topology always settles to the same positions; nodes only move when nodes/edges are added or removed, not when metric values change. - Drive visuals from each series' last value: node color via panel thresholds, optional node-size scaling via per-threshold multipliers, edge color via dedicated edge thresholds (falling back to node thresholds), edge thickness via edge-thickness thresholds, and label interpolation for node/edge captions. - Add a Manual / From data mode toggle to the settings editor with the shared styling controls plus the new graph-generation and node-size panels. - Switch the read-only panel to view-mode zoom/pan (plain wheel zoom, left/middle drag pan) and clip edges to node perimeters so lines and arrowheads touch the shape boundary. - Keep every representation coherent: TypeScript runtime/editor, CUE schema, Go SDK builders, docs, and tests. Signed-off-by: Philipp Matthes <p.matthes@sap.com>

PhilippMatthes · 2026-10-23T15:29:22Z

@SoWieMarkus and I will polish this PR a little more until we put it into ready for review

Signed-off-by: Markus Wieland <markus.wieland@sap.com>

SoWieMarkus · 2026-10-28T10:21:22Z

To the reviewers: since this feature needs metrics in a specific format (source/target labels for edges), which the Perses dev stack for metrics doesn't provide, we added a small demo exporter for it. To test the plugin you can check out the README.

It's basically an nginx serving a static file of metrics, which can be added as a scrape target to the dev Prometheus. We couldn't figure out whether there's a "correct" way to provide demo data in a special format. If there is one, please let us know and we'll switch to it. :)

The demo dashboard is then available under http://localhost:8080/projects/perses/dashboards/canvasgraphdemo?var-job=tempo&start=5m&refresh=5s

Signed-off-by: Markus Wieland <markus.wieland@sap.com>

Copilot

🟡 Changes recommended

The uncapped layout integration can diverge on high-degree hub graphs, producing extreme coordinates that make the graph effectively unviewable.

9 open issues

What changed in this PR

Adds a query-driven mode to the Canvas plugin for visualizing dynamic service and host topologies without manually drawing nodes and edges.

Changes:

Generates graphs from series labels, with automatic force layout and value-driven styling.
Adds graph-generation settings, node-size controls, independent edge colors, view-mode navigation, and perimeter-clipped edges.
Extends TypeScript, CUE, and Go contracts, with utility tests and a runnable Prometheus demo.

File	Description
docs/canvas/examples/README.md	Documents demo setup and usage.
docs/canvas/examples/metrics/topology.prom	Supplies sample topology metrics.
docs/canvas/examples/metrics/nginx.conf	Serves metrics with a Prometheus-compatible content type.
docs/canvas/examples/docker-compose.yml	Runs the demo metrics server.
docs/canvas/examples/dashboard.json	Provides a configured graph dashboard.
canvas/src/utils/panelUtils.ts	Resolves threshold-based node scaling.
canvas/src/utils/panelUtils.test.ts	Tests node-size threshold selection.
canvas/src/utils/graphLayout.ts	Implements automatic force layout.
canvas/src/utils/graphLayout.test.ts	Tests layout positioning and determinism.
canvas/src/utils/graphGeneration.ts	Builds nodes and edges from query labels.
canvas/src/utils/graphGeneration.test.ts	Tests graph generation and data bindings.
canvas/src/utils/edgeUtils.ts	Clips edges to node boundaries.
canvas/src/utils/edgeUtils.test.ts	Tests edge clipping geometry.
canvas/src/model.ts	Defines generation and styling options.
canvas/src/hooks/useZoom.ts	Adds read-only zoom and pan gestures.
canvas/src/components/shared/IconNode.tsx	Adds contrasting outlines to circular nodes.
canvas/src/components/settings/NodeSizeSettings.tsx	Adds base-size and multiplier controls.
canvas/src/components/settings/GraphGenerationSettings.tsx	Configures query and label mappings.
canvas/src/components/settings/GlobalSettingsEditor.tsx	Integrates mode selection and styling controls.
canvas/src/components/panel/PanelNodeLayer.tsx	Binds generated nodes to their series.
canvas/src/components/panel/PanelEdgeLayer.tsx	Applies generated bindings, edge colors, and clipping.
canvas/src/components/panel/CanvasPanel.tsx	Integrates generation, layout, scaling, and navigation.
canvas/sdk/go/canvas.go	Exposes new configuration through Go types and builders.
canvas/schemas/panels/canvas/canvas.cue	Adds schema definitions for the new options.
canvas/README.md	Documents data-driven mode and styling behavior.

_{Review effort: Balanced}

- Bound the force simulation with a cooling schedule so high-degree hub graphs stay viewable instead of diverging to extreme coordinates. - Cache the layout by topology key so the simulation reruns only when the node/edge set changes, not on every value refresh or viewport resize. - Show the node-size controls only in From data mode, where generated nodes actually opt into threshold-based sizing. - Base edge-thickness breakpoints on the edge thresholds (falling back to node thresholds), matching edge color resolution. - Give per-threshold size multiplier inputs accessible names. - Prune node-size multipliers whose threshold no longer exists so they cannot linger as active-but-hidden scales. - Collapse clipped edges to a point when nodes overlap instead of reversing the line, and skip rendering the collapsed edge. - Encode generated edge ids unambiguously so endpoint pairs sharing a separator-joined form no longer collide. - Apply the edge label template to both directions of a bidirectional edge. Signed-off-by: Philipp Matthes <p.matthes@sap.com>

PhilippMatthes · 2026-10-29T12:19:37Z

@AntoineThebaud if you like, you can retrigger the review bot, so we verify that all feedback has been adressed properly.

Copilot

🟡 Changes recommended

Topology-key collisions and stale cached node specs can reuse incorrect layouts and ignore appearance edits, while threshold changes leave hidden edge-width settings active.

3 open issues

9 resolved since last review

_{Review effort: Balanced}

- Cache only node positions by id (not whole node specs) so appearance edits like default size and label template take effect on refresh without rerunning the force simulation. - Reconcile edgeThresholdWidths with the effective edge thresholds (edge thresholds, or node thresholds as fallback) so a width cannot linger as an active-but-hidden step after its breakpoint changes. - Build the topology key from JSON-serialized sorted id arrays so it is unambiguous (no separator collision) and independent of series order. Signed-off-by: Philipp Matthes <p.matthes@sap.com>

PhilippMatthes · 2026-10-29T15:34:19Z

We're converging ;) @ibakshay if you like you can hit the button one more time.

Copilot

🔵 Needs a closer look

Unresolved issues allow query ordering to shift node layouts, reciprocal edges and labels to overlap, and arrowheads on short segments to intrude into target nodes.

No open issues 🎉

3 previously missed issues

In code that hasn't changed since last review

Prevent arrowheads from exceeding clipped edge segments

canvas/src/components/panel/PanelEdgeLayer.tsx:92

Clipping can leave a segment shorter than its arrowhead: two 48px nodes 60px apart leave a 12px gap, but an 8px stroke produces a 20px arrowhead at 1× zoom. EdgeLines.shortenEnd leaves such short segments unchanged, so the marker extends into the target node. After clipping, shrink the marker or suppress edges whose directional segments cannot fit their arrowheads, accounting for zoom.

Prevent overlap for reciprocal edge paths

canvas/src/utils/graphGeneration.ts:114

If the edge query returns both A→B and B→A, the generated edges share the same path, so their colors and widths overpaint each other and their value labels overlap near the midpoint. This obscures directional metrics in reciprocal connections. Merge these rows into one bidirectional edge with separate series for each direction, or render them along separate paths.

Canonicalize simulation inputs before layout

canvas/src/utils/graphLayout.ts:58

The force simulation still depends on incoming node/edge order, so reopening the same topology with reordered query results can visibly move nodes; sorting only the final centroid does not prevent this. Canonicalize both inputs before the simulation while preserving the returned node order, and add a larger reordered-topology regression test.

3 resolved since last review

_{Review effort: Balanced}

PhilippMatthes · 2026-10-29T15:53:08Z

Taking care of the missed comments and also the failing docs check.

Signed-off-by: Philipp Matthes <p.matthes@sap.com>

- Canonicalize the force-simulation node and edge order (id-sorted) so a topology laid out again with reordered query results settles to identical positions; map back to input order on return. Add a larger shuffled-input regression test. - Merge reciprocal A->B / B->A edge rows into a single bidirectional edge carrying a per-direction series, so the two directions no longer overpaint one path or overlap their labels. Plumb a reverse-series map through to the edge layer. - Shrink an edge's arrowhead to fit when a node-clipped directional segment is shorter than the marker, so arrowheads no longer intrude into the target node; accounts for zoom via the already-scaled stroke width. - Fix checkdocs: write the localhost dev-stack URLs in the example README as inline code instead of autolinks so mdox does not try to reach them. Signed-off-by: Philipp Matthes <p.matthes@sap.com>

PhilippMatthes · 2026-10-29T16:22:46Z

Updated screenshot: two edges A<-->B and B<-->A are now resolved into one bidirectional edge.

Gladorme · 2026-10-29T16:24:04Z

+small microservice topology served as Prometheus metrics, plus a ready-to-apply dashboard. Use it to
+see nodes and edges generated live from query data.
+
+The stock Perses dev metrics do not carry paired `source`/`target` labels, so they cannot exercise


To me this PR looks ultra-specific to SAP metrics? Or node_load / edge_traffic are something used elsewhere?

What about creating the topology based on OTel span count connector or service graph connector? The implem is "generic" enough to support it too?

The example is inspired by a generic networking monitoring use case. Take a look at the Grafana docs, they have it similar, albeit with a more bogus example.

The scope of this feature is to allow users to generate the graph from metrics. However I don't see anything that would block us from supporting other datasources.

PhilippMatthes · 2026-10-29T16:24:20Z

@ibakshay thanks for bearing with me, let's try again if all feedback was successfully addressed.

Copilot

🟡 Changes recommended

Reciprocal query-row reordering changes the edge ID and topologyKey despite unchanged topology, causing CanvasPanel to unnecessarily rerun all 300 force iterations.

1 open issue

Use canonical pair identity to prevent order-only topology reruns · New

_{Review effort: Balanced}

Derive a generated edge's id from the canonical unordered pair key rather than the first-seen orientation. Reordering reciprocal A->B / B->A query rows no longer flips the id or the topology key, so CanvasPanel does not rerun the force simulation on an order-only refresh. Orientation is still carried by the entry's source/target for the directional series. Signed-off-by: Philipp Matthes <p.matthes@sap.com>

PhilippMatthes · 2026-10-30T08:47:37Z

@ibakshay thanks for triggering the ai again. Ready for the human ping pong now ;)

adrianSepiol · 2026-10-30T10:27:45Z

Hi @PhilippMatthes, thanks for the PR! I've started reviewing it, and I have a few thoughts on its scope:

Zoom control changes: I noticed the PR also changes the zoom control. I'm not sure that fits the scope of this PR. Could we discuss whether we need this change in a separate issue?
New edge settings: The new edge settings might be easier to add in their own PR. Right now the legend doesn't show these settings yet. Moving them to a follow-up PR would keep this one smaller and let us handle the legend there.

Splitting things up this way should make the review faster and keep this PR focused. What do you think?

PhilippMatthes · 2026-10-30T11:04:49Z

@adrianSepiol you are right, it'll make it much easier to review when we split this into multiple parts and keep the scope of this PR minimal. @SoWieMarkus will work on this and get back to you.

Keep this branch focused on automatic graph generation. Remove the edge color scheme (edgeThresholds), node-size scaling (nodeThresholdSizes, sizeMode scaling), and the view-mode zoom interaction change, which move to follow-up PRs stacked on top of this one. Generation plumbing is retained, including the no-op sizeMode:'threshold' on generated nodes and nodeDefaultSize. Signed-off-by: Markus Wieland <markus.wieland@sap.com>

SoWieMarkus · 2026-10-30T12:12:24Z

@adrianSepiol I remvoed the changes from the branch. I will open the follow up PRs after this PR.

adrianSepiol

I've tested it locally and it works well. Left some questions/suggestions in comments.

adrianSepiol · 2026-10-30T14:05:37Z

+    // Apply the cached position onto the freshly generated node (keeping its current label/size), so
+    // appearance edits take effect without rerunning the simulation.
+    const offsetX = (props.contentDimensions?.width ?? 600) / 2;
+    const offsetY = (props.contentDimensions?.height ?? 400) / 2;


We could extract this 600 and 400 to constants together with their previous usage.

adrianSepiol · 2026-11-02T10:34:26Z

+
+  return (
+    <Stack spacing={2}>
+      <Typography variant="body2" color="text.secondary">


What do you think of pulling this up under the mode switch so it is first thing that catches users eye when mode is changed?

adrianSepiol · 2026-11-02T10:35:55Z

-      </OptionsEditorGrid>
+      <ToggleButtonGroup exclusive size="small" value={mode} onChange={onModeChange} sx={MODE_TOGGLE_SX}>
+        <ToggleButton value="manual">Manual</ToggleButton>
+        <ToggleButton value="auto">From data</ToggleButton>


Not sure if it is just me but "From data" is not intuitive name for what it does

[ENHANCEMENT] Enable instant queries in example dashboard

01fa4c1

Signed-off-by: Markus Wieland <markus.wieland@sap.com>

[FIX] remove dead code

1873118

Signed-off-by: Markus Wieland <markus.wieland@sap.com>

SoWieMarkus force-pushed the feat/canvas-graph-generation branch from a59150b to 1873118 Compare October 28, 2026 10:21

PhilippMatthes changed the title ~~[FEATURE] Generate graphs from query data~~ [FEATURE] Generate graphs in canvas from query data Oct 28, 2026

PhilippMatthes marked this pull request as ready for review October 28, 2026 13:50

PhilippMatthes requested review from a team, AntoineThebaud and Nexucis as code owners October 28, 2026 13:50

PhilippMatthes requested review from shahrokni and removed request for a team October 28, 2026 13:50

AntoineThebaud requested a balanced review from Copilot October 29, 2026 10:27

Copilot started reviewing on behalf of AntoineThebaud October 29, 2026 10:29

View session

Copilot AI reviewed Oct 29, 2026

View reviewed changes

This comment was marked as resolved.

Sign in to view

ibakshay requested a balanced review from Copilot October 29, 2026 14:36

Copilot started reviewing on behalf of ibakshay October 29, 2026 14:36

View session

Copilot AI reviewed Oct 29, 2026

View reviewed changes

Comment thread canvas/src/components/panel/CanvasPanel.tsx Resolved

Comment thread canvas/src/components/settings/GlobalSettingsEditor.tsx Resolved

Comment thread canvas/src/utils/graphGeneration.ts Resolved

ibakshay requested a balanced review from Copilot October 29, 2026 15:36

Copilot started reviewing on behalf of ibakshay October 29, 2026 15:36

View session

Copilot AI reviewed Oct 29, 2026

View reviewed changes

PhilippMatthes added 2 commits October 29, 2026 17:10

[FIX] Fix misaligned whitespace in topology.prom

439dc32

Signed-off-by: Philipp Matthes <p.matthes@sap.com>

This comment was marked as resolved.

Sign in to view

Gladorme reviewed Oct 29, 2026

View reviewed changes

ibakshay requested a balanced review from Copilot October 29, 2026 21:27

Copilot started reviewing on behalf of ibakshay October 29, 2026 21:27

View session

Copilot AI reviewed Oct 29, 2026

View reviewed changes

Comment thread canvas/src/utils/graphGeneration.ts Resolved

SoWieMarkus force-pushed the feat/canvas-graph-generation branch from 04914c3 to 81fd3fb Compare October 30, 2026 11:52

SoWieMarkus mentioned this pull request Oct 30, 2026

[FEATURE] Canvas: edge color scheme and node-size scaling PhilippMatthes/perses-plugins#1

Draft

5 tasks done

adrianSepiol reviewed Nov 2, 2026

View reviewed changes

Conversation

PhilippMatthes commented Oct 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Screenshots

Checklist

UI Changes

Uh oh!

PhilippMatthes commented Oct 23, 2026

Uh oh!

SoWieMarkus commented Oct 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment was marked as resolved.

PhilippMatthes commented Oct 29, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Uh oh!

PhilippMatthes commented Oct 29, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

PhilippMatthes commented Oct 29, 2026

Uh oh!

This comment was marked as resolved.

PhilippMatthes commented Oct 29, 2026

Uh oh!

Gladorme Oct 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

PhilippMatthes Oct 29, 2026

Choose a reason for hiding this comment

Uh oh!

PhilippMatthes commented Oct 29, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

PhilippMatthes commented Oct 30, 2026

Uh oh!

adrianSepiol commented Oct 30, 2026

Uh oh!

PhilippMatthes commented Oct 30, 2026

Uh oh!

SoWieMarkus commented Oct 30, 2026

Uh oh!

adrianSepiol left a comment

Choose a reason for hiding this comment

Uh oh!

adrianSepiol Oct 30, 2026

Choose a reason for hiding this comment

Uh oh!

adrianSepiol Nov 2, 2026

Choose a reason for hiding this comment

Uh oh!

adrianSepiol Nov 2, 2026

Choose a reason for hiding this comment

PhilippMatthes commented Oct 23, 2026 •

edited

Loading

SoWieMarkus commented Oct 28, 2026 •

edited

Loading

Gladorme Oct 29, 2026 •

edited

Loading