Skip to content

Use a unix socket between httpd and Foreman's Puma#593

Draft
ekohl wants to merge 2 commits into
theforeman:masterfrom
ekohl:foreman-unix-socket
Draft

Use a unix socket between httpd and Foreman's Puma#593
ekohl wants to merge 2 commits into
theforeman:masterfrom
ekohl:foreman-unix-socket

Conversation

@ekohl

@ekohl ekohl commented Jun 23, 2026

Copy link
Copy Markdown
Member

Why are you introducing these changes? (Problem description, related links)

#118 included this previously, but splitting the focus.

In foreman-installer we saw using a unix socket had reliability benefits.

What are the changes introduced in this pull request?

How to test this pull request

Steps to reproduce:

  • The Foreman UI loads
  • Test suite passes

Checklist

  • Tests added/updated (if applicable)
  • Documentation updated (if applicable)

@ekohl ekohl mentioned this pull request Jun 23, 2026
Open
@ekohl

ekohl commented Jun 23, 2026

Copy link
Copy Markdown
Member Author

Looking at the logs I see in the DB migration:

Jun 23 18:30:34 quadlet.example.com foreman-db-migrate[68208]: 2026-06-23T18:30:34 [W|app|] Failed to register foreman_remote_execution plugin (PG::UndefinedTable: ERROR:  relation "job_invocations" does not exist
Jun 23 18:30:34 quadlet.example.com foreman-db-migrate[68208]:  | LINE 9:  WHERE a.attrelid = '"job_invocations"'::regclass
Jun 23 18:30:34 quadlet.example.com foreman-db-migrate[68208]:  |                             ^
Jun 23 18:30:34 quadlet.example.com foreman-db-migrate[68208]:  | )
Jun 23 18:30:34 quadlet.example.com foreman-db-migrate[68196]: 2026-06-23T18:30:34 [W|app|] Failed to register foreman_remote_execution plugin (PG::UndefinedTable: ERROR:  relation "job_invocations" does not exist
Jun 23 18:30:34 quadlet.example.com foreman-db-migrate[68196]:  | LINE 9:  WHERE a.attrelid = '"job_invocations"'::regclass
Jun 23 18:30:34 quadlet.example.com foreman-db-migrate[68196]:  |                             ^
Jun 23 18:30:34 quadlet.example.com foreman-db-migrate[68196]:  | )

And then later on service startup:

Jun 23 18:31:46 quadlet.example.com foreman[68464]: 2026-06-23T18:31:46 [W|dyn|] Error on on_execution_plan_save event
Jun 23 18:31:46 quadlet.example.com foreman[68464]: 2026-06-23T18:31:46 [I|dyn|] Backtrace for 'Error on on_execution_plan_save event' error (ActiveRecord::SerializationFailure): PG::TRSerializationFailure: ERROR:  could not serialize access due to read/write dependencies among transactions
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | DETAIL:  Reason code: Canceled on identification as a pivot, during conflict in checking.
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | HINT:  The transaction might succeed if retried.
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | 
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/connection_adapters/postgresql_adapter.rb:768:in `exec_params'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/connection_adapters/postgresql_adapter.rb:768:in `block (2 levels) in exec_no_cache'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activesupport-7.0.10/lib/active_support/concurrency/share_lock.rb:187:in `yield_shares'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activesupport-7.0.10/lib/active_support/dependencies/interlock.rb:41:in `permit_concurrent_loads'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/connection_adapters/postgresql_adapter.rb:767:in `block in exec_no_cache'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activesupport-7.0.10/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activesupport-7.0.10/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activesupport-7.0.10/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activesupport-7.0.10/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/connection_adapters/abstract_adapter.rb:752:in `block in log'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activesupport-7.0.10/lib/active_support/notifications/instrumenter.rb:24:in `instrument'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/connection_adapters/abstract_adapter.rb:743:in `log'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/connection_adapters/postgresql_adapter.rb:766:in `exec_no_cache'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/connection_adapters/postgresql_adapter.rb:745:in `execute_and_clear'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/connection_adapters/postgresql/database_statements.rb:54:in `exec_query'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/connection_adapters/abstract/database_statements.rb:132:in `exec_insert'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/connection_adapters/postgresql/database_statements.rb:88:in `exec_insert'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/connection_adapters/abstract/database_statements.rb:167:in `insert'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/connection_adapters/abstract/query_cache.rb:22:in `insert'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/persistence.rb:496:in `_insert_record'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/persistence.rb:1098:in `_create_record'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/counter_cache.rb:166:in `_create_record'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/locking/optimistic.rb:84:in `_create_record'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/attribute_methods/dirty.rb:226:in `_create_record'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/callbacks.rb:459:in `block in _create_record'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activesupport-7.0.10/lib/active_support/callbacks.rb:107:in `run_callbacks'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activesupport-7.0.10/lib/active_support/callbacks.rb:929:in `_run_create_callbacks'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/callbacks.rb:459:in `_create_record'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/timestamp.rb:108:in `_create_record'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/persistence.rb:1069:in `create_or_update'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/callbacks.rb:455:in `block in create_or_update'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activesupport-7.0.10/lib/active_support/callbacks.rb:118:in `block in run_callbacks'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/autosave_association.rb:370:in `around_save_collection_association'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activesupport-7.0.10/lib/active_support/callbacks.rb:127:in `block in run_callbacks'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activesupport-7.0.10/lib/active_support/callbacks.rb:138:in `run_callbacks'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activesupport-7.0.10/lib/active_support/callbacks.rb:929:in `_run_save_callbacks'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/callbacks.rb:455:in `create_or_update'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/timestamp.rb:126:in `create_or_update'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/persistence.rb:648:in `save!'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/validations.rb:53:in `save!'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/transactions.rb:302:in `block in save!'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/transactions.rb:354:in `block in with_transaction_returning_status'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/connection_adapters/abstract/database_statements.rb:314:in `transaction'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/transactions.rb:350:in `with_transaction_returning_status'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/transactions.rb:302:in `save!'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/suppressor.rb:54:in `save!'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/foreman-tasks-12.2.4/app/lib/actions/task_synchronization.rb:47:in `on_execution_plan_save'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/foreman-tasks-12.2.4/app/lib/actions/task_synchronization.rb:32:in `sync_execution_plan_to_task'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/execution_plan/hooks.rb:69:in `block (2 levels) in run'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/execution_plan/hooks.rb:67:in `each'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/execution_plan/hooks.rb:67:in `block in run'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/executors.rb:18:in `run_user_code'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/execution_plan/hooks.rb:66:in `run'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/execution_plan.rb:163:in `block (2 levels) in run_hooks'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware/stack.rb:28:in `pass'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware.rb:20:in `pass'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware.rb:57:in `hook'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware/stack.rb:24:in `call'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware/stack.rb:28:in `pass'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware.rb:20:in `pass'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware.rb:57:in `hook'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware/stack.rb:24:in `call'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware/stack.rb:28:in `pass'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware.rb:20:in `pass'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware.rb:57:in `hook'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware/stack.rb:24:in `call'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware/stack.rb:28:in `pass'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware.rb:20:in `pass'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/foreman-tasks-12.2.4/app/lib/actions/middleware/load_setting_values.rb:30:in `hook'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware/stack.rb:24:in `call'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware/stack.rb:28:in `pass'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware.rb:20:in `pass'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/foreman-tasks-12.2.4/app/lib/actions/middleware/keep_current_request_id.rb:26:in `block in hook'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/foreman-tasks-12.2.4/app/lib/actions/middleware/keep_current_request_id.rb:52:in `restore_current_request_id'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/foreman-tasks-12.2.4/app/lib/actions/middleware/keep_current_request_id.rb:25:in `hook'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware/stack.rb:24:in `call'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware/stack.rb:28:in `pass'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware.rb:20:in `pass'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/foreman-tasks-12.2.4/app/lib/actions/middleware/keep_current_timezone.rb:24:in `block in hook'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/foreman-tasks-12.2.4/app/lib/actions/middleware/keep_current_timezone.rb:44:in `restore_curent_timezone'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/foreman-tasks-12.2.4/app/lib/actions/middleware/keep_current_timezone.rb:24:in `hook'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware/stack.rb:24:in `call'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware/stack.rb:28:in `pass'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware.rb:20:in `pass'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/foreman-tasks-12.2.4/app/lib/actions/middleware/keep_current_taxonomies.rb:23:in `block in hook'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/foreman-tasks-12.2.4/app/lib/actions/middleware/keep_current_taxonomies.rb:45:in `restore_current_taxonomies'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/foreman-tasks-12.2.4/app/lib/actions/middleware/keep_current_taxonomies.rb:23:in `hook'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware/stack.rb:24:in `call'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware/stack.rb:28:in `pass'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware.rb:20:in `pass'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware.rb:57:in `hook'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware/stack.rb:24:in `call'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware/stack.rb:28:in `pass'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware.rb:20:in `pass'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/foreman-tasks-12.2.4/app/lib/actions/middleware/keep_current_user.rb:28:in `block in hook'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/foreman-tasks-12.2.4/app/lib/actions/middleware/keep_current_user.rb:48:in `restore_current_user'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/foreman-tasks-12.2.4/app/lib/actions/middleware/keep_current_user.rb:28:in `hook'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware/stack.rb:24:in `call'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware/world.rb:31:in `execute'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/execution_plan.rb:162:in `block in run_hooks'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/execution_plan.rb:161:in `each'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/execution_plan.rb:161:in `run_hooks'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/execution_plan.rb:259:in `delay'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/world.rb:201:in `delay_with_options'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/world.rb:195:in `delay'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/foreman-tasks-12.2.4/lib/foreman_tasks.rb:61:in `delay'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/foreman-tasks-12.2.4/app/models/foreman_tasks/recurring_logic.rb:78:in `trigger_repeat_after'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/foreman-tasks-12.2.4/app/models/foreman_tasks/recurring_logic.rb:70:in `start_after'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/foreman-tasks-12.2.4/app/models/foreman_tasks/recurring_logic.rb:64:in `start'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/foreman-tasks-12.2.4/lib/foreman_tasks.rb:99:in `block (2 levels) in register_scheduled_task'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/foreman/app/models/concerns/foreman/thread_session.rb:108:in `as'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/foreman/app/models/concerns/foreman/thread_session.rb:114:in `as_anonymous_admin'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/foreman-tasks-12.2.4/lib/foreman_tasks.rb:96:in `block in register_scheduled_task'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/connection_adapters/abstract/transaction.rb:319:in `block in within_new_transaction'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activesupport-7.0.10/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activesupport-7.0.10/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activesupport-7.0.10/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activesupport-7.0.10/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/connection_adapters/abstract/transaction.rb:317:in `within_new_transaction'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/connection_adapters/abstract/database_statements.rb:316:in `transaction'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/transactions.rb:209:in `transaction'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/foreman-tasks-12.2.4/lib/foreman_tasks.rb:90:in `register_scheduled_task'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/foreman-tasks-12.2.4/lib/foreman_tasks/engine.rb:174:in `block (2 levels) in <class:Engine>'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/rails/configuration.rb:68:in `block in run_on_init_hooks'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/rails/configuration.rb:68:in `each'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/rails/configuration.rb:68:in `run_on_init_hooks'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/rails.rb:41:in `block in initialize!'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | <internal:kernel>:90:in `tap'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/rails.rb:39:in `initialize!'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | config/puma/production.rb:58:in `block in _load_from'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/puma-6.6.1/lib/puma/configuration.rb:329:in `block in run_hooks'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/puma-6.6.1/lib/puma/configuration.rb:323:in `each'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/puma-6.6.1/lib/puma/configuration.rb:323:in `run_hooks'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/puma-6.6.1/lib/puma/cluster/worker.rb:57:in `run'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/puma-6.6.1/lib/puma/cluster.rb:230:in `worker'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/puma-6.6.1/lib/puma/cluster.rb:106:in `block in spawn_worker'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activesupport-7.0.10/lib/active_support/fork_tracker.rb:20:in `block in fork'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activesupport-7.0.10/lib/active_support/fork_tracker.rb:18:in `fork'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activesupport-7.0.10/lib/active_support/fork_tracker.rb:18:in `fork'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/puma-6.6.1/lib/puma/cluster.rb:106:in `spawn_worker'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/puma-6.6.1/lib/puma/cluster.rb:86:in `block in spawn_workers'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/puma-6.6.1/lib/puma/cluster.rb:79:in `times'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/puma-6.6.1/lib/puma/cluster.rb:79:in `spawn_workers'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/puma-6.6.1/lib/puma/cluster.rb:442:in `run'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/puma-6.6.1/lib/puma/launcher.rb:203:in `run'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/puma-6.6.1/lib/rack/handler/puma.rb:79:in `run'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/rack-2.2.23/lib/rack/server.rb:327:in `start'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/railties-7.0.10/lib/rails/commands/server/server_command.rb:38:in `start'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/railties-7.0.10/lib/rails/commands/server/server_command.rb:143:in `block in perform'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | <internal:kernel>:90:in `tap'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/railties-7.0.10/lib/rails/commands/server/server_command.rb:134:in `perform'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/thor-1.3.0/lib/thor/command.rb:28:in `run'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/thor-1.3.0/lib/thor/invocation.rb:127:in `invoke_command'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/thor-1.3.0/lib/thor.rb:527:in `dispatch'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/railties-7.0.10/lib/rails/command/base.rb:87:in `perform'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/railties-7.0.10/lib/rails/command.rb:48:in `invoke'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/railties-7.0.10/lib/rails/commands.rb:18:in `<top (required)>'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:160:in `require'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:160:in `rescue in require'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:149:in `require'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/foreman/bin/rails:9:in `<main>'
Jun 23 18:31:46 quadlet.example.com foreman[68464]: 2026-06-23T18:31:46 [E|bac|] PG::InFailedSqlTransaction: ERROR:  current transaction is aborted, commands ignored until end of transaction block
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  |  (ActiveRecord::StatementInvalid)
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/connection_adapters/postgresql_adapter.rb:768:in `exec_params'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/connection_adapters/postgresql_adapter.rb:768:in `block (2 levels) in exec_no_cache'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activesupport-7.0.10/lib/active_support/concurrency/share_lock.rb:187:in `yield_shares'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activesupport-7.0.10/lib/active_support/dependencies/interlock.rb:41:in `permit_concurrent_loads'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/connection_adapters/postgresql_adapter.rb:767:in `block in exec_no_cache'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activesupport-7.0.10/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activesupport-7.0.10/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activesupport-7.0.10/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activesupport-7.0.10/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/connection_adapters/abstract_adapter.rb:752:in `block in log'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activesupport-7.0.10/lib/active_support/notifications/instrumenter.rb:24:in `instrument'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/connection_adapters/abstract_adapter.rb:743:in `log'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/connection_adapters/postgresql_adapter.rb:766:in `exec_no_cache'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/connection_adapters/postgresql_adapter.rb:745:in `execute_and_clear'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/connection_adapters/postgresql/database_statements.rb:54:in `exec_query'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/connection_adapters/abstract/database_statements.rb:560:in `select'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/connection_adapters/abstract/database_statements.rb:66:in `select_all'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/connection_adapters/abstract/query_cache.rb:110:in `select_all'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/querying.rb:54:in `_query_by_sql'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/relation.rb:942:in `block in exec_main_query'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/relation.rb:962:in `skip_query_cache_if_necessary'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/relation.rb:928:in `exec_main_query'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/relation.rb:914:in `block in exec_queries'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/relation.rb:962:in `skip_query_cache_if_necessary'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/relation.rb:908:in `exec_queries'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/relation.rb:695:in `load'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/relation.rb:250:in `records'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/relation/delegation.rb:88:in `each'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/foreman/app/registries/setting_registry.rb:183:in `load_values'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/foreman-tasks-12.2.4/app/lib/actions/middleware/load_setting_values.rb:36:in `reload_setting_values'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/foreman-tasks-12.2.4/app/lib/actions/middleware/load_setting_values.rb:9:in `delay'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware/stack.rb:24:in `call'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware/stack.rb:28:in `pass'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware.rb:20:in `pass'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/foreman-tasks-12.2.4/app/lib/actions/middleware/keep_current_request_id.rb:5:in `delay'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware/stack.rb:24:in `call'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware/stack.rb:28:in `pass'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware.rb:20:in `pass'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/foreman-tasks-12.2.4/app/lib/actions/middleware/keep_current_timezone.rb:5:in `delay'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware/stack.rb:24:in `call'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware/stack.rb:28:in `pass'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware.rb:20:in `pass'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/foreman-tasks-12.2.4/app/lib/actions/middleware/keep_current_taxonomies.rb:5:in `delay'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware/stack.rb:24:in `call'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware/stack.rb:28:in `pass'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware.rb:20:in `pass'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware.rb:29:in `delay'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware/stack.rb:24:in `call'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware/stack.rb:28:in `pass'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware.rb:20:in `pass'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/foreman-tasks-12.2.4/app/lib/actions/middleware/keep_current_user.rb:5:in `delay'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware/stack.rb:24:in `call'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/middleware/world.rb:31:in `execute'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/action.rb:329:in `block in execute_delay'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/action.rb:491:in `block in with_error_handling'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/action.rb:491:in `catch'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/action.rb:491:in `with_error_handling'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/action.rb:328:in `execute_delay'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/execution_plan/steps/plan_step.rb:41:in `delay'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/execution_plan.rb:260:in `delay'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/world.rb:201:in `delay_with_options'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/world.rb:195:in `delay'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/foreman-tasks-12.2.4/lib/foreman_tasks.rb:61:in `delay'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/foreman-tasks-12.2.4/app/models/foreman_tasks/recurring_logic.rb:78:in `trigger_repeat_after'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/foreman-tasks-12.2.4/app/models/foreman_tasks/recurring_logic.rb:70:in `start_after'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/foreman-tasks-12.2.4/app/models/foreman_tasks/recurring_logic.rb:64:in `start'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/foreman-tasks-12.2.4/lib/foreman_tasks.rb:99:in `block (2 levels) in register_scheduled_task'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/foreman/app/models/concerns/foreman/thread_session.rb:108:in `as'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/foreman/app/models/concerns/foreman/thread_session.rb:114:in `as_anonymous_admin'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/foreman-tasks-12.2.4/lib/foreman_tasks.rb:96:in `block in register_scheduled_task'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/connection_adapters/abstract/transaction.rb:319:in `block in within_new_transaction'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activesupport-7.0.10/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activesupport-7.0.10/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activesupport-7.0.10/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activesupport-7.0.10/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/connection_adapters/abstract/transaction.rb:317:in `within_new_transaction'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/connection_adapters/abstract/database_statements.rb:316:in `transaction'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activerecord-7.0.10/lib/active_record/transactions.rb:209:in `transaction'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/foreman-tasks-12.2.4/lib/foreman_tasks.rb:90:in `register_scheduled_task'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/foreman-tasks-12.2.4/lib/foreman_tasks/engine.rb:174:in `block (2 levels) in <class:Engine>'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/rails/configuration.rb:68:in `block in run_on_init_hooks'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/rails/configuration.rb:68:in `each'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/rails/configuration.rb:68:in `run_on_init_hooks'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/rails.rb:41:in `block in initialize!'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | <internal:kernel>:90:in `tap'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/dynflow-2.0.1/lib/dynflow/rails.rb:39:in `initialize!'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | config/puma/production.rb:58:in `block in _load_from'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/puma-6.6.1/lib/puma/configuration.rb:329:in `block in run_hooks'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/puma-6.6.1/lib/puma/configuration.rb:323:in `each'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/puma-6.6.1/lib/puma/configuration.rb:323:in `run_hooks'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/puma-6.6.1/lib/puma/cluster/worker.rb:57:in `run'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/puma-6.6.1/lib/puma/cluster.rb:230:in `worker'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/puma-6.6.1/lib/puma/cluster.rb:106:in `block in spawn_worker'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activesupport-7.0.10/lib/active_support/fork_tracker.rb:20:in `block in fork'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/activesupport-7.0.10/lib/active_support/fork_tracker.rb:18:in `fork'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/puma-6.6.1/lib/puma/cluster.rb:106:in `spawn_worker'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/puma-6.6.1/lib/puma/cluster.rb:86:in `block in spawn_workers'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/puma-6.6.1/lib/puma/cluster.rb:79:in `times'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/puma-6.6.1/lib/puma/cluster.rb:79:in `spawn_workers'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/puma-6.6.1/lib/puma/cluster.rb:442:in `run'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/puma-6.6.1/lib/puma/launcher.rb:203:in `run'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/puma-6.6.1/lib/rack/handler/puma.rb:79:in `run'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/rack-2.2.23/lib/rack/server.rb:327:in `start'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/railties-7.0.10/lib/rails/commands/server/server_command.rb:38:in `start'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/railties-7.0.10/lib/rails/commands/server/server_command.rb:143:in `block in perform'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | <internal:kernel>:90:in `tap'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/railties-7.0.10/lib/rails/commands/server/server_command.rb:134:in `perform'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/thor-1.3.0/lib/thor/command.rb:28:in `run'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/thor-1.3.0/lib/thor/invocation.rb:127:in `invoke_command'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/thor-1.3.0/lib/thor.rb:527:in `dispatch'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/railties-7.0.10/lib/rails/command/base.rb:87:in `perform'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/railties-7.0.10/lib/rails/command.rb:48:in `invoke'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/gems/gems/railties-7.0.10/lib/rails/commands.rb:18:in `<top (required)>'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:160:in `require'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:160:in `rescue in require'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:149:in `require'
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | /usr/share/foreman/bin/rails:9:in `<main>'
Jun 23 18:31:46 quadlet.example.com foreman[68464]: [21] WARNING hook before_worker_boot failed with exception (ActiveRecord::StatementInvalid) PG::InFailedSqlTransaction: ERROR:  current transaction is aborted, commands ignored until end of transaction block
Jun 23 18:31:46 quadlet.example.com foreman[68464]: 
Jun 23 18:31:46 quadlet.example.com foreman[68464]: [1] - Worker 3 (PID: 21) booted in 0.39s, phase: 0
Jun 23 18:31:46 quadlet.example.com foreman[68464]: [25] WARNING hook before_worker_boot failed with exception (ActiveRecord::StatementInvalid) PG::InFailedSqlTransaction: ERROR:  current transaction is aborted, commands ignored until end of transaction block
Jun 23 18:31:46 quadlet.example.com foreman[68464]: 
Jun 23 18:31:46 quadlet.example.com foreman[68464]: [1] - Worker 4 (PID: 25) booted in 0.39s, phase: 0
Jun 23 18:31:46 quadlet.example.com foreman[68464]: [14] WARNING hook before_worker_boot failed with exception (ActiveRecord::SerializationFailure) PG::TRSerializationFailure: ERROR:  could not serialize access due to read/write dependencies among transactions
Jun 23 18:31:46 quadlet.example.com foreman[68464]: DETAIL:  Reason code: Canceled on identification as a pivot, during conflict out checking.
Jun 23 18:31:46 quadlet.example.com foreman[68464]: HINT:  The transaction might succeed if retried.
Jun 23 18:31:46 quadlet.example.com foreman[68464]: 
Jun 23 18:31:46 quadlet.example.com foreman[68464]: [1] - Worker 1 (PID: 14) booted in 0.41s, phase: 0
Jun 23 18:31:46 quadlet.example.com foreman[68464]: [33] WARNING hook before_worker_boot failed with exception (ActiveRecord::SerializationFailure) PG::TRSerializationFailure: ERROR:  could not serialize access due to read/write dependencies among transactions
Jun 23 18:31:46 quadlet.example.com foreman[68464]: DETAIL:  Reason code: Canceled on identification as a pivot, during conflict out checking.
Jun 23 18:31:46 quadlet.example.com foreman[68464]: HINT:  The transaction might succeed if retried.
Jun 23 18:31:46 quadlet.example.com foreman[68464]: 
Jun 23 18:31:46 quadlet.example.com foreman[68464]: [1] - Worker 5 (PID: 33) booted in 0.39s, phase: 0
Jun 23 18:31:46 quadlet.example.com foreman[68464]: 2026-06-23T18:31:46 [W|dyn|] Error on on_execution_plan_save event
Jun 23 18:31:46 quadlet.example.com foreman[68464]: 2026-06-23T18:31:46 [I|dyn|] Backtrace for 'Error on on_execution_plan_save event' error (ActiveRecord::SerializationFailure): PG::TRSerializationFailure: ERROR:  could not serialize access due to read/write dependencies among transactions
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | DETAIL:  Reason code: Canceled on identification as a pivot, during conflict in checking.
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  | HINT:  The transaction might succeed if retried.
Jun 23 18:31:46 quadlet.example.com foreman[68464]:  |

So I wonder what's going on here and if it's related to my change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

Not yet reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@ekohl