Skip to content

Fix progress bar iteration speed accuracy and speed#901

Open
PlagueKind wants to merge 1 commit into
ostris:mainfrom
PlagueKind:ostris/fix/progressbar
Open

Fix progress bar iteration speed accuracy and speed#901
PlagueKind wants to merge 1 commit into
ostris:mainfrom
PlagueKind:ostris/fix/progressbar

Conversation

@PlagueKind

Copy link
Copy Markdown
Contributor

This change replaces tqdm’s internal rate estimation with a deterministic, time-based EMA smoothing approach.

Problem:

  • tqdm’s built-in rate estimator becomes unstable under non-uniform step timing
  • save/sample/log pauses cause delayed convergence and “catch-up” behavior in reported it/s

Solution:

  • Introduce external time-based rate calculation using perf_counter()
  • Apply lightweight EMA smoothing to stabilize display without lag
  • Preserve existing postfix metrics system (losses, lr, etc.)
  • Avoid modifying training loop behavior, only affects progress display

Result:

  • Stable iteration speed reporting
  • No catch-up artifacts after pauses
  • No dependency on tqdm internal smoothing model

@PlagueKind PlagueKind force-pushed the ostris/fix/progressbar branch 3 times, most recently from c7d10bd to 2d81839 Compare June 24, 2026 14:42
@PlagueKind PlagueKind closed this Jun 28, 2026
@PlagueKind PlagueKind force-pushed the ostris/fix/progressbar branch from 2d81839 to f743ccf Compare June 28, 2026 19:54
@PlagueKind

Copy link
Copy Markdown
Contributor Author

Switched to simplified code by @fatalis

@PlagueKind PlagueKind reopened this Jun 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant