Use systemd socket activation for Foreman by ekohl · Pull Request #118 · theforeman/foremanctl

ekohl · 2025-03-11T20:07:47Z

@ehelms as you requested, an example of using systemd socket activation an unix sockets.

It changes the connections to go through Apache, which may or may not be right.

ehelms · 2025-03-11T21:31:55Z

Instead of enforcing Apache to be present and an ordering, could you use the unix_socket parameter of the uri module to do the service up test?

ekohl · 2025-03-11T22:48:07Z

Instead of enforcing Apache to be present and an ordering, could you use the unix_socket parameter of the uri module to do the service up test?

I had that for a moment, but the type of the result changes to a dict and didn't want to deal with it. Also didn't know if the smart_proxy module from FAM supports unix sockets. Then I took the easy route and relied on Apache though I kind of agree with you that it'd be nicer if it was more standalone.

On the other hand, perhaps we should rely more on health checks for ready detection. Then the Smart Proxy registration also feels like it doesn't belong in the Foreman role. Perhaps we need to swap the order of Foreman and Foreman Proxy. That stuff is easier in Puppet IMHO.

ehelms · 2025-03-12T01:33:58Z

Then the Smart Proxy registration also feels like it doesn't belong in the Foreman role.

Not that you say that, I am surprised it's in that role as well due to what I would have thought are order of operations. I suppose it's there as the last action with the way the playbook is organized. I think in past iterations where we played with Ansible we made registration it's own role to control it better. Ahh yes, back in the day, we had a spark of an idea.

Then you have #81 which changes things and negates this being here all together I think.

evgeni · 2025-03-13T14:32:02Z

  ansible.builtin.uri:
-    url: 'http://{{ ansible_hostname }}:3000/api/v2/ping'
+    url: '{{ foreman_url }}/api/v2/ping'
+    validate_certs: false # TODO


Suggested change

validate_certs: false # TODO

ca_path: '{{ ca_certificate }}'

When I try this I get:

The conditional check 'foreman_status.status == 200' failed. The error was: error while evaluating conditional (foreman_status.status == 200): 'dict object' has no attribute 'status'. 'dict object' has no attribute 'status'

Weird. In Line 126 you mean?

evgeni · 2025-03-13T14:32:28Z

    url: "https://{{ ansible_fqdn }}:9090"
-    server_url: "http://{{ ansible_fqdn }}:3000"
+    server_url: "{{ foreman_url }}"
+    validate_certs: false # TODO


We don't support ca_path today in FAM. Should we?

I was looking for it and now I'm wondering if validate_certs: '{{ ca_certificate }}' works. https://docs.python-requests.org/en/latest/user/advanced/#ssl-cert-verification says verify='/path/to/certfile' is supported and looking at https://github.com/theforeman/foreman-ansible-modules/blob/2343e66adf65aac42f2d26a076d555102320bcc6/plugins/inventory/foreman.py#L266 I'd think that works. Not sure about https://github.com/theforeman/foreman-ansible-modules/blob/2343e66adf65aac42f2d26a076d555102320bcc6/plugins/module_utils/foreman_helper.py#L617 but if we go that route, it'd make sense to me to copy requests' API because it saves a parameter.

It saves one, yes, but diverges from ansible.builtin.uri API, which is the one Ansible users are more familiar with.

The problem (today) is https://github.com/theforeman/foreman-ansible-modules/blob/2343e66adf65aac42f2d26a076d555102320bcc6/plugins/module_utils/foreman_helper.py#L373 which makes it a boolean, not a string.

evgeni · 2025-03-13T14:37:18Z

+        [Unit]
+        Requires=foreman.socket


This is the only thing needed for it to do socket activation? 🤯

Mhh, but the tests are failing with error 500, so maybe it isn't?

Does the socket "just" need to be started too?

This is the only thing needed for it to do socket activation? 🤯

Pretty much yes. It ends up setting the right env vars and Puma picks that up because we tell it to (https://github.com/theforeman/foreman/blob/75dc40095546966477a927911e715263c42d1bea/config/puma/production.rb#L35-L38).

Mhh, but the tests are failing with error 500, so maybe it isn't?

I think that's because of async. Looking at the Pulp failures I get the impression the service is started before migrations ran. That's why I'll split this off into 2 PRs.

Does the socket "just" need to be started too?

Starting foreman.service (as we do) will activate the socket so it shouldn't be needed to enable it, but we could. In theory we could only start the socket and let foreman.service start on demand but I'm not sure we want to go that route.

Sorry, async was a different patch. Apache fails with:

[Thu Mar 13 18:02:39.887738 2025] [proxy:warn] [pid 707:tid 891] [client 192.168.122.14:52218] AH01144: No protocol handler was valid for the URL / (scheme 'unix'). If you are using a DSO version of mod_proxy, make sure the proxy submodules are included in the configuration using LoadModule.

Turns out the template had a trailing slash and then it couldn't find the right socket anymore.

Starting foreman.service (as we do) will activate the socket so it shouldn't be needed to enable it, but we could. In theory we could only start the socket and let foreman.service start on demand but I'm not sure we want to go that route.

There is a good reason to start from Ansible: we should restart it if a change is made.

evgeni · 2025-03-13T14:42:43Z

Also didn't know if the smart_proxy module from FAM supports unix sockets.

It does not, because requests can't. There is https://github.com/msabramo/requests-unixsocket but I never tried it.

ekohl · 2025-03-13T18:14:47Z

Next issue:

type=AVC msg=audit(1741889614.482:20207): avc:  denied  { write } for  pid=58362 comm="httpd" name="foreman.sock" dev="tmpfs" ino=2284 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:var_run_t:s0 tclass=sock_file permissive=0

We handled this in our foreman-selinux package but that's no longer installed on the host.

evgeni · 2025-03-14T11:01:29Z

Next issue:

type=AVC msg=audit(1741889614.482:20207): avc:  denied  { write } for  pid=58362 comm="httpd" name="foreman.sock" dev="tmpfs" ino=2284 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:var_run_t:s0 tclass=sock_file permissive=0

We handled this in our foreman-selinux package but that's no longer installed on the host.

Nothing prevents us from installing it again.

Or we extend/adjust
https://github.com/theforeman/foreman-quadlet/blob/3c0ec9ca701dc196c693f812a265f558c1e3d511/roles/httpd/tasks/main.yml#L9-L13

audit2allow says daemons_enable_cluster_mode could help? -- Edit: tested. nope it does not

ekohl · 2025-03-14T22:05:50Z

audit2allow says daemons_enable_cluster_mode could help? -- Edit: tested. nope it does not

If you use /run/httpd.* it gets the httpd_var_run_t type and then the boolean does work. Still doesn't feel right, but if it helps move this forward it can be a decent hack.

ehelms · 2025-05-14T14:37:46Z

@ekohl Would you mind rebasing when you get a moment?

ehelms · 2025-05-15T12:00:12Z

    foreman_oauth_consumer_secret: abcdefghijklmnopqrstuvwxyz123456
+    foreman_listen_stream: /run/httpd.foreman.sock
+    foreman_url: "https://{{ ansible_fqdn }}"
+    httpd_foreman_backend: "unix://{{ foreman_listen_stream }}|http://%{HTTP_HOST}/"


I'm surprised all three of these are not the role defaults.

Because they're connected: my thought was that the roles should be dependent and only on the playbook level you can relate them.

ehelms · 2025-05-15T14:11:22Z

 ---
 foreman_container_image: "quay.io/evgeni/foreman-rpm"
 foreman_container_tag: "nightly"
+foreman_listen_stream: localhost:3000


I think what I don't know enough about is why we wouldn't default this to /run/httpd.foreman.sock if our intent is to use systemd sock activation.

ekohl · 2026-03-19T08:56:03Z

Rebased to resolve conflicts, but I haven't taken this for a spin myself yet.

ekohl · 2026-04-23T11:19:56Z

Will rebase once #463 is merged.

pablomh · 2026-04-23T12:34:13Z

Hi @ekohl. Will this be used for inter-container communication or only for activation?

ekohl · 2026-04-23T15:31:30Z

Just for Apache -> Foreman. Technically that's crossing the host -> container boundary. Other services talk to Foreman via Apache. No services other than Apache will talk to the unix socket.

ekohl · 2026-05-14T13:04:16Z

Rebased to resolve conflicts now that #463 is merged.

ehelms · 2026-05-14T20:07:04Z

I think this needs some test fixes, the tests at tests/foreman_test.py:5-6 hardcode:

FOREMAN_HOST = 'localhost'
FOREMAN_PORT = 3000

ekohl · 2026-06-23T18:18:25Z

I've reduced the scope to just socket activation now, still using the TCP socket. This may already solve the issue IPv6-only has. Then in #593 we can deal with the unix socket.

evgeni reviewed Mar 13, 2025

View reviewed changes

Comment thread roles/foreman/tasks/main.yaml Outdated

evgeni reviewed Mar 13, 2025

View reviewed changes

ekohl force-pushed the socket-activation branch from 3c95a0d to 19a28df Compare March 13, 2025 18:15

evgeni reviewed Mar 14, 2025

View reviewed changes

Comment thread playbooks/deploy.yaml Outdated

evgeni reviewed Mar 14, 2025

View reviewed changes

Comment thread roles/foreman/tasks/main.yaml Outdated

evgeni reviewed Mar 14, 2025

View reviewed changes

Comment thread roles/foreman/tasks/main.yaml Outdated

evgeni reviewed Mar 14, 2025

View reviewed changes

Comment thread roles/foreman/tasks/main.yaml Outdated

ekohl force-pushed the socket-activation branch from 19a28df to 995fb64 Compare March 14, 2025 22:05

ekohl mentioned this pull request May 14, 2025

Add foreman service handler for certificate changes #147

Closed

ekohl force-pushed the socket-activation branch from 995fb64 to 0885411 Compare May 15, 2025 07:51

ehelms reviewed May 15, 2025

View reviewed changes

ekohl mentioned this pull request Sep 5, 2025

only emit password once #228

Merged

ekohl mentioned this pull request Mar 17, 2026

Switch container networking from host to named bridge networks #403

Open

ekohl force-pushed the socket-activation branch from 0885411 to dd7e7fb Compare March 19, 2026 08:55

ekohl force-pushed the socket-activation branch from dd7e7fb to a11bd1d Compare May 14, 2026 13:03

ekohl force-pushed the socket-activation branch from a11bd1d to e735d7f Compare May 14, 2026 20:37

ekohl force-pushed the socket-activation branch from e735d7f to c8ba5bb Compare June 23, 2026 18:14

ekohl mentioned this pull request Jun 23, 2026

Use a unix socket between httpd and Foreman's Puma #593

Draft

2 tasks

Use systemd socket activation for Foreman

cd61710

ekohl force-pushed the socket-activation branch from c8ba5bb to cd61710 Compare June 23, 2026 19:55

Uh oh!

Conversation

ekohl commented Mar 11, 2025

Uh oh!

ehelms commented Mar 11, 2025

Uh oh!

ekohl commented Mar 11, 2025

Uh oh!

ehelms commented Mar 12, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

evgeni commented Mar 13, 2025

Uh oh!

ekohl commented Mar 13, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

evgeni commented Mar 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ekohl commented Mar 14, 2025

Uh oh!

ehelms commented May 14, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ekohl commented Mar 19, 2026

Uh oh!

ekohl commented Apr 23, 2026

Uh oh!

pablomh commented Apr 23, 2026

Uh oh!

ekohl commented Apr 23, 2026

Uh oh!

ekohl commented May 14, 2026

Uh oh!

ehelms commented May 14, 2026

Uh oh!

ekohl commented Jun 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

evgeni commented Mar 14, 2025 •

edited

Loading