Skip to content

feat(workflow-operator): volcano plot significance cut-offs - #9041

Open
eugenegujing wants to merge 1 commit into
apache:mainfrom
eugenegujing:feat/volcano-plot-significance-thresholds
Open

eugenegujing wants to merge 1 commit into
apache:mainfrom
eugenegujing:feat/volcano-plot-significance-thresholds

Conversation

@eugenegujing

Copy link
Copy Markdown
Contributor

What changes were proposed in this PR?

The Volcano Plot draws a plain scatter of effect size vs -log10(p), colored by effect size, which is already the x axis. Nothing shows where the significance bar is or which points clear it. This PR adds three optional properties, all off by default, so saved workflows render as before.

Property Type Effect when set
Significance threshold number, 0 < x <= 1 dashed horizontal line at -log10(threshold)
Effect-size threshold number, > 0 dashed vertical lines at +threshold and -threshold
Color by significance boolean with both thresholds set, points are colored Up / Down / Not significant with a legend instead of the continuous RdBu scale
Before:  px.scatter(color=<effect column>, color_continuous_scale="RdBu")
After:   same by default
         + fig.add_hline(...)                        when a significance threshold is set
         + fig.add_vline(+eff) / fig.add_vline(-eff) when an effect-size threshold is set
         + color="Significance", 3-color map         when the toggle is on and both thresholds are set

Categories use strict comparisons on the selected p-value column: Up = p < sig and effect > eff, Down = p < sig and effect < -eff, everything else Not significant. The category is written to a column named Significance (also the legend title); an input column of that name would be shadowed inside the operator only. Colors are ColorBrewer RdBu end points #B2182B / #2166AC plus grey #9E9E9E.

Validation: the JSON schema bounds above stop bad values in the property panel. For hand-edited JSON the operator also ignores a threshold it cannot draw (NaN, infinite, <= 0, or a p cut-off > 1) and falls back to the continuous coloring if the toggle is on but a threshold is missing. A cut-off of exactly 1 is allowed and draws its line at y = 0. If the cut-off's -log10 is beyond the data range the line may fall outside the auto-ranged axis; extending the range is left for a follow-up.

Independent of the label-fix (#9037) and hover-column (#9038) PRs; all three touch neighbouring lines of VolcanoPlotOpDesc.scala, so whichever lands second needs a small rebase.

Figures below are the operator's generated Python from main and from this branch, run on the 430-gene example workflow from the issue, with thresholds 0.05 and 1.

Before After (lines only) After (lines + Color by significance)
before-main thresholds-after-lines-only thresholds-after-colored

Any related issues, documentation, discussions?

Closes #9039

PR #8504 adds a generateStandaloneCode() path to this operator that copies the current template. Whichever lands first, the other needs a follow-up.

How was this PR tested?

New tests in VolcanoPlotOpDescSpec, written before the source change:

  • defaults: no lines, continuous coloring; each threshold alone emits only its own line(s); both with the toggle off keep the continuous coloring
  • toggle on with both thresholds: strict comparisons, np.select column, exact color map and category order, no color_continuous_scale
  • toggle on with one or no threshold: continuous fallback
  • zero / negative thresholds and a p cut-off above 1 emit no line of their own; Infinity, -Infinity, NaN from JSON never reach the Python
  • a cut-off of exactly 1 draws its line and still colors by category
  • thresholds are emitted as valid Python floats, checked for 1e-300, Double.MIN_VALUE and 1e21 in every site they appear
  • the generated JSON schema carries exclusiveMinimum / maximum on the two thresholds
  • JSON deserialization: numbers, numeric strings, null, blank, absent, and the boolean toggle
  • a workflow JSON saved before this change generates byte-for-byte the same code as a default-constructed operator
sbt -Dsbt.server.forcestart=true "WorkflowOperator/testOnly *VolcanoPlotOpDescSpec"
# Tests: succeeded 24, failed 0, canceled 0, ignored 0, pending 1   (the pending one is pre-existing)

sbt -Dsbt.server.forcestart=true "WorkflowOperator/scalafmtAll" "WorkflowOperator/Test/scalafmtAll"
sbt -Dsbt.server.forcestart=true "WorkflowOperator/scalafixAll --check"
# success

Smoke run outside Texera (pytexera stubbed, pandas + plotly 5.24.1) on the 430-gene mock table, 420 rows after the operator drops zero and missing p-values:

Configuration Lines Traces
defaults none 1 continuous trace, 420 points
0.05 / 1, toggle off hline y = 1.301, vlines x = ±1 1 continuous trace
0.05 / 1, toggle on same Up 35, Down 35, Not significant 350
0.05 only, toggle on hline only continuous (fallback)
0.05 / -1, toggle on hline only continuous (fallback)
2 / 1, toggle on vlines only continuous (fallback)
"Infinity" / "NaN" from JSON none continuous, no such token in the code
1e-300 / 1e21, toggle on hline y = 300, vlines x = ±1e21 Not significant 420
1 / 1, toggle on hline y = 0, vlines x = ±1 Up 41, Down 38, Not significant 341

The exponent-form dumps pass python -m py_compile.

Was this PR authored or co-authored using generative AI tooling?

Yes. Generated-by: Claude Code (claude-fable-5-1). Co-authored with Claude; the author reviewed the change before submission.

Add optional Significance threshold and Effect-size threshold properties that draw dashed cut-off lines at -log10(threshold) and at plus and minus the effect-size threshold.
Add a Color by significance toggle that, with both thresholds set, colors points Up, Down or Not significant instead of by effect size.
Bound both thresholds in the JSON schema and ignore a non-positive, non-finite or above-one value instead of drawing it.
@github-actions

Copy link
Copy Markdown
Contributor

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

  • Contributors with relevant context: @kz930, @Yicong-Huang
    You can notify them by mentioning @kz930, @Yicong-Huang in a comment.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Volcano Plot: optional significance / effect-size thresholds with cut-off lines and up / down / not-significant coloring

1 participant