Skip to content

Issue 525 FE Basis refactor implementation#561

Draft
zasexton wants to merge 37 commits into
SimVascular:mainfrom
zasexton:issue-525
Draft

Issue 525 FE Basis refactor implementation#561
zasexton wants to merge 37 commits into
SimVascular:mainfrom
zasexton:issue-525

Conversation

@zasexton

@zasexton zasexton commented Jun 2, 2026

Copy link
Copy Markdown
Collaborator

Current situation

Tracks #525.

The solver currently relies on legacy table-driven shape-function paths in nn.cpp, which makes basis evaluation, Hessian support, node-ordering validation, and parity testing difficult to extend. This PR introduces a self-contained FE Basis layer while preserving the existing solver-facing storage contracts.

Release Notes

  • Added a new FE Basis module with Lagrange and Serendipity basis support for mapped solver elements, including values, gradients, Hessians, node-ordering conventions, factory creation, and cache support.
  • Added FE math and quadrature utilities used by the new basis implementation.
  • Routed supported nn::get_gnn and nn::get_gn_nxx volume/face evaluations through the new FE Basis adapter.
  • Added SVMP_BASIS_MODE=auto|legacy|fe to compare legacy and FE Basis evaluation paths at runtime.
  • Added typed FE/basis exception paths for unsupported elements, invalid configurations, and backend failures.
  • Added a basis comparison harness for running pytest integration cases under legacy and FE modes and comparing outputs, timing, fields, and iteration data.
  • Build migration: solver code now requires a C++20-capable compiler.

Documentation

  • Added Doxygen-style documentation to the new FE Basis, FE Math, and Quadrature APIs.
  • No solver XML migration is expected for users; basis mode selection is controlled through SVMP_BASIS_MODE.

Testing

  • Added GoogleTest coverage under tests/unitTests/FE/Basis for Lagrange/Serendipity basis evaluation, Hessians, cache/factory behavior, error paths, node ordering, solver adapter parity, and supported mapped element coverage.
  • Added GoogleTest coverage under tests/unitTests/FE/Math for matrix/vector operations, expression helpers, math constants, and dense linear algebra.

Code of Conduct & Contributing Guidelines

@zasexton zasexton self-assigned this Jun 2, 2026
@zasexton

zasexton commented Jun 3, 2026

Copy link
Copy Markdown
Collaborator Author

From the CI test cases we got the following:

  • macOS: 12 failed, 227 passed
  • Ubuntu: 21 failed, 218 passed
  • Failures are all FSI / FSI-ustruct pipe cases.

The affected meshes are HEX8. Current code allocated Nxx but did not populate HEX8 second derivatives; this branch now computes nonzero HEX8 Hessians via FE Basis. FSI uses those in Code/Source/solver/fsi.cpp:191, so the stored legacy reference VTUs no longer match.

@zasexton

zasexton commented Jun 3, 2026

Copy link
Copy Markdown
Collaborator Author

In the FSI fluid assembly, Nxx is passed into the element kernel and used in terms that contribute to the assembled equations, especially VMS/stabilization terms.

  • Code/Source/solver/fluid.cpp:1841: Nwxx is used to build uxx at lines 1841-
    1860.

  • Code/Source/solver/fluid.cpp:1880: uxx feeds d2u2.

  • Code/Source/solver/fluid.cpp:1933: uxx also feeds es_x, then mu_x at lines
    1948-1963.

  • Code/Source/solver/fluid.cpp:2027: d2u2 and mu_x enter rS.

  • Code/Source/solver/fluid.cpp:2037: rS enters up.

  • Code/Source/solver/fluid.cpp:2103: up-dependent quantities are used when
    assembling lR, the local residual, at lines 2103-2106.

That means:

  • in the current code, it solves a discrete problem where HEX8 second-derivative contributions are effectively zero or absent when using the old basis functions

  • when using the new FE Basis functions, it solves a different discrete problem where HEX8 mapped Hessian contributions are present

Therefore the Hessian is not just participating in the tangent assembly and may not necessarily converge on the same solution as the current reference vtu. This is also why we do not see such CI failures for test cases that use Tet elements because TET4 have zero reference second derivatives or for TET10 Hessian values are provided. For such elements, introduction of the new Basis function infrastructure did not cause a change in the residual being solved for these cases.

@zasexton

zasexton commented Jun 3, 2026

Copy link
Copy Markdown
Collaborator Author

Overall, we see that providing the non-zero Hessian values slightly increases the number of linear iterations required but does not impact the number of nonlinear iterations for the FSI test cases considered. More importantly, this difference in how the Hessian is used in the formulated residual suggested that the problem being solved is slightly different depending on if a user is providing brick elements versus TET10 (which shouldn't be the case)

convergence-iteration-behavior

@zasexton

zasexton commented Jun 4, 2026

Copy link
Copy Markdown
Collaborator Author

Confirmed. The difference in accuracy for the FSI test cases when employing the new FE Basis functions is due to the non-zero Hessian values for the reference elements. We can see that when we zero-out the values for the Hessian on the HEX8 cases that we recover the same behavior as the current solver. @ktbolt do you think it would make sense to update the reference solutions for these test cases or is there an alternative strategy that we should look at?

hex8-hessian-confirmation-by-case

@aabrown100-git

Copy link
Copy Markdown
Collaborator

@zasexton Were Hessians also zeroed out in the Fortran svFSI?

I think we should update reference solutions unless there is a good reason to use zero Hessians. @sujaldave, @hanzhao2020 Is there any reason (in VMS or something) to force $u_{xx}$ to be zero in HEX8 or higher order elements?

@zasexton

zasexton commented Jun 4, 2026

Copy link
Copy Markdown
Collaborator Author

@aabrown100-git yes, the Hessian was also effectively zero for the HEX8 in the original fortran svFSI. Only TET10, TRI6, QUD8, QUD9, and LIN2 had cases in the GETGNNxx subroutine. All other cases return the zero-initialized Hessian.

Regenerate affected FSI and FSI-ustruct HEX8 result_005.vtu references for the FE Basis path with nonzero HEX8 Hessian contributions.

Update the pipe_3d PETSc and Trilinos references to match the base pipe_3d reference, preserving the existing shared-reference pattern across linear algebra variants.
@codecov

codecov Bot commented Jun 8, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 93.01418% with 197 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.73%. Comparing base (5118d56) to head (3e03138).

Files with missing lines Patch % Lines
Code/Source/solver/nn.cpp 60.79% 89 Missing ⚠️
tests/unitTests/FE/Basis/test_BasisErrorPaths.cpp 84.21% 27 Missing ⚠️
tests/unitTests/FE/Basis/test_LagrangeBasis.cpp 95.62% 14 Missing ⚠️
Code/Source/solver/FE/Basis/BasisTraits.h 77.27% 10 Missing ⚠️
Code/Source/solver/fs.cpp 47.36% 10 Missing ⚠️
Code/Source/solver/FE/Basis/SerendipityBasis.cpp 97.80% 9 Missing ⚠️
tests/unitTests/FE/Basis/test_BasisHessians.cpp 95.47% 9 Missing ⚠️
...Source/solver/FE/Basis/NodeOrderingConventions.cpp 97.13% 7 Missing ⚠️
Code/Source/solver/FE/Basis/LagrangeBasis.cpp 98.35% 6 Missing ⚠️
...ests/unitTests/FE/Math/test_DenseLinearAlgebra.cpp 96.96% 6 Missing ⚠️
... and 6 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #561      +/-   ##
==========================================
+ Coverage   69.08%   71.73%   +2.64%     
==========================================
  Files         181      237      +56     
  Lines       34256    37019    +2763     
  Branches     5931     6471     +540     
==========================================
+ Hits        23665    26554    +2889     
+ Misses      10454    10247     -207     
- Partials      137      218      +81     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@aabrown100-git aabrown100-git self-requested a review June 8, 2026 23:05

@aabrown100-git aabrown100-git left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some comments!

Comment thread Code/Source/solver/FE/Math/DenseLinearAlgebra.h
Comment thread Code/Source/solver/nn.cpp Outdated
Comment thread Documentation/Doxyfile
@aabrown100-git

Copy link
Copy Markdown
Collaborator

@zasexton Also, are you planning on adding an independent design document? Something I found helpful was asking Codex to explain the flow from mesh input to basis function generation to mapping to existing N, Nx, and Nxx arrays for TET4 (as an example). Perhaps something like that? It would also help to understand the existing basis function code surrounding this PR. For example, that element types/order are determined from the provided mesh, and that N, Nx, and Nxx are cached and correspond to the parametric element, but basis functions and gradients w.r.t to the physical element in the reference configuration are computed during assembly (I think).

@zasexton

zasexton commented Jun 9, 2026

Copy link
Copy Markdown
Collaborator Author

In general, I only assume that we derive the topology from the mesh information. Although often mesh structure can imply interpolation order, this does not always have to be true; I believe the solution basis order may be a separate discretization choice. The geometry mapping basis should always be interpolated from the mesh nodes provided, but the field solution basis functions need not be assumed the same. One practical example could be the Taylor-Hood elements where mixed fields expect different solution field orders (e.g. velocity P2, pressure P1). If we only allow for isoparametric order then these types of mixed order formulations require much more delicate flagging and checking (which I think might be the case currently). So for flow diagram type of illustration, any mesh information would be directly linked to the geometric basis while the field solution basis/orders rely on information from practical formulation choices.

@aabrown100-git

Copy link
Copy Markdown
Collaborator

@zasexton Thanks for explaining. I think a DESIGN.md in the Basis folder (or somewhere similar) explaining that point, other relevant details, and the design of the Basis module itself would be helpful.

@zasexton

Copy link
Copy Markdown
Collaborator Author

I think I'll plan to add these notes to the Basis topic in the doxygen documentation which will overview the layout and purpose of this submodule. This way the design overview for this (and eventually other topics) will live in the github pages.

@aabrown100-git

Copy link
Copy Markdown
Collaborator

@zasexton good idea! Will that be written in the comments of the code, or a separate file?

namespace {
using Vec3 = math::Vector<Real, 3>;

void store_gradient(const Gradient& gradient, Real* dst) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zasexton What is all of this Real type about ? Let's get rid of it unless there is a very very very good reason to use it.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right now Real is an alias for the scalar type and defaults to double. My thoughts were that as we refactored the FE machinery we would use Real which would serve as the solver-wide scalar alias policy. Specifically I implemented this in case there was ever a need to switch to float for a type of GPU support in the future. I can go ahead and replace these if this seems to be more confusion than it is worth.

Real t,
Real* values,
Real* gradients,
Real* hessians) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zasexton Raw pointers ?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know, I know....my thoughts here: these calls allow for genuine zero-allocation evaluation paths for solver production runs and compiler restriction optimizations. The evaluate_values along with their gradient and hessian analogues offer the safe, sized, allocated API while the *_to methods offer these allocation-free methods. I could replace the raw pointers with std::span and remove the restrict to avoid directly using raw pointers while maintaining the flat, non-owning, no-allocation buffer model. We just wouldn't be able to use the compiler restrict anymore which is probably fine if we are always caching these values anyway.

dst[2] = gradient[2];
}

void evaluate_hex8_reference(Real r,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zasexton What exactly is this function doing and what are the parameters ?

Isn't the Hessian of a hex8 element 0 ?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pure second derivative terms are 0 but the mixed terms are not.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zasexton What I meant was why did the original code set the Hessian to 0 and why is that being changed ?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure why the original code set the Hessian to zero...in the svFSI the Hessian is zero initialized and never populated. So the svFSI and svMultiPhysics codes are doing the same thing I'm just not sure why

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as for why it's being changed: I don't think it makes sense that a change in the element type should affect the formulated residual that is being solved. Either we chose to zero out the Hessian completely (effectively eliminating these terms in the formulation) or we consider reference solutions that appropriately include these terms (as seems to be the case in the fsi formulation intent). For linear tet elements this doesn't matter since the Hessian is completely zero, but this is a special case.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should include non-zero Hessians unless there is a real numerical reason not to, but I don't think that's the case (although I haven't looked closely into this)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I remember correctly, the convergence of residual-based stabilization (SUPG, VMS-LES) rests on the assumption that the residual appearing in the stabilization terms is zero if evaluated on the exact solution (that is, the method must be consistent, or weakly consistent at least). If that residual does not include the laplacian term, then there is an inconsistency, and the method is not convergent. Optimistically, the error (numerical vs. exact solution) will stagnate as the mesh is refined. In light of this, I think that neglecting the Hessian is wrong, and including it is correct.

I do not know how this will affect the convergence properties of the non-linear and linear solvers and the system's conditioning - they might be better or they might be worse. Most likely they will vary between scenarios. But I think this is a secondary issue, and should not affect the decision of whether to include the Hessian or not.

values[i] = Real(0.125) * ar * bs * ct;
}
if (gradients) {
Real* g = gradients + i * 3u;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zasexton Offsetting from a raw pointer ? Yuck, like C !

@zasexton zasexton Jun 11, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mainly I'm trying to avoid having to create temporary Gradient or Hessian objects within functions that are internal. since the output buffer shape is pretty simple for gradients and hessians I opted for the offsets. I can change these if to have an accessor if you think this would be better long-term.

@zasexton

Copy link
Copy Markdown
Collaborator Author

@aabrown100-git I'm intending for it to live in the BasisFunction.h file so that doxygen can build it. I believe this will follow a similar style to how deal.ii creates topic sections for different components of the code. An example from the compiled html pages is attached here along with a crude input/output data flow diagram to conceptually communicate where the Basis function objects responsibilities sit in the chain of calls. Let me know what you think.
basis_dox_inputs-responsibilities

@michelebucelli michelebucelli left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @zasexton, I left a few comments mostly related to housekeeping - I haven't yet dug into the more logic-heavy part of the PR.

# NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS

set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD 20)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What features from C++ 20 are required now?

I don't know about recent updates, but until last year or so compiler support for C++ 20 was still a bit sketchy, and requiring developers to update to the latest compiler versions might be a big ask, especially in HPC environments.

Comment on lines -89 to +92
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++20")

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to the above, but also: would it make sense to take this version number from the CMake variable CMAKE_CXX_STANDARD?

Comment thread Documentation/Doxyfile
Comment on lines +194 to +197
MATHJAX_VERSION = MathJax_2
MATHJAX_FORMAT = HTML-CSS
MATHJAX_RELPATH = https://cdn.jsdelivr.net/npm/mathjax@2
MATHJAX_EXTENSIONS = TeX/AMSmath TeX/AMSsymbols

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just out of curiosity, what prompted this downgrading to version 2? Are there new features that are currently unsupported?

#include <atomic>
#ifdef EIGEN_USE_GPU
#include <chrono>
#endif

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do I understand correctly that these lines are changing code inside the Eigen code? If so, this seems potentially dangerous to me, as we risk branching into a customized version of Eigen that might be hard to update with new Eigen versions if necessary.

What issue prompted introducing this change? Maybe we can figure out a workaround.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants