feat: make amount of items sent on email configurable via occ

[cristianscheid]

• Resolves: #

Summary

Context

• activity emails are sent to users via the OCA\Activity\BackgroundJob\EmailNotification background job
• the background job calls MailQueueHandler::sendEmails()
• MailQueueHandler::ENTRY_LIMIT defines how many activities are fully shown in the email body, anything beyond that is truncated, as shown below:

Changes introduced by this PR

• adds an admin-configurable setting to control the max number of activities fully shown in notification emails:

occ config:app:set activity mail_max_items --value=500

• falls back to 200 (original value) when no configuration is set
• adds MailQueueHandler::MAIL_MAX_ITEMS_CAP as a safeguard against excessively large values

Checklist

• Code is properly formatted
• Sign-off message is added to all commits
• Tests (unit, integration, api and/or acceptance) are included
• Screenshots before/after for front-end changes
• Documentation (manuals or wiki) has been updated or is not required
• Backports requested where applicable (ex: critical bugfixes)
• Labels added where applicable (ex: bug/enhancement, 3. to review, feature component)
• Milestone added for target branch/version (ex: 32.x for stable32)

AI (if applicable)

• The content of this PR was partly or fully generated using AI

[cristianscheid]

Please note that psalm CI error is unrelated.

[miaulalala]

Can you please add some tests?

[miaulalala]

Did you test the performance when having that many items?

stale

[cristianscheid]

Did you test the performance when having that many items?

I tested with the following env (nextcloud/nextcloud-docker-dev):

• created 30 users (user1 to user30)

for i in $(seq 1 30); do
  OC_PASS=admin docker exec -e OC_PASS=admin master-nextcloud-1 occ user:add --password-from-env --email user$i@mail.com user$i
done

• all of them with the following settings on "personal settings > notifications"
  • mail notification enabled for "a file or folder has been shared"
  • send activity emails set as "as soon as possible"
• set mail_max_items to max value (1000)

docker exec master-nextcloud-1 occ config:app:set activity mail_max_items --value=1000

• simulated 4 different scenarios
  • send mail to 1 user (1005 activities)
  • send mail to 10 users (1005 activities for each user)
  • send mail to 20 users (1005 activities for each user)
  • send mail to 30 users (1005 activities for each user)
• 1005 was used to test if the emails display correctly the remaining 5 summarized as "and 5 more"
• to create the entries on DB (oc_activity_mq table):

-- 1 user
SET @now = UNIX_TIMESTAMP();
INSERT INTO `oc_activity_mq` (`amq_timestamp`, `amq_latest_send`, `amq_type`, `amq_affecteduser`, `amq_appid`, `amq_subject`, `amq_subjectparams`, `object_type`, `object_id`)
SELECT @now, @now, 'shared', u.user, 'files_sharing', 'shared_with_by', '[{"462":"\/mock.json"},"admin"]', 'files', 462
FROM (SELECT 1 FROM information_schema.columns LIMIT 1005) t
CROSS JOIN (SELECT 'user1' AS user) u;

-- 10 users
SET @now = UNIX_TIMESTAMP();
INSERT INTO `oc_activity_mq` (`amq_timestamp`, `amq_latest_send`, `amq_type`, `amq_affecteduser`, `amq_appid`, `amq_subject`, `amq_subjectparams`, `object_type`, `object_id`)
SELECT @now, @now, 'shared', u.user, 'files_sharing', 'shared_with_by', '[{"462":"\/mock.json"},"admin"]', 'files', 462
FROM (SELECT 1 FROM information_schema.columns LIMIT 1005) t
CROSS JOIN (SELECT 'user1' AS user UNION ALL SELECT 'user2' UNION ALL SELECT 'user3' UNION ALL SELECT 'user4' UNION ALL SELECT 'user5' UNION ALL SELECT 'user6' UNION ALL SELECT 'user7' UNION ALL SELECT 'user8' UNION ALL SELECT 'user9' UNION ALL SELECT 'user10') u;

-- 20 users
SET @now = UNIX_TIMESTAMP();
INSERT INTO `oc_activity_mq` (`amq_timestamp`, `amq_latest_send`, `amq_type`, `amq_affecteduser`, `amq_appid`, `amq_subject`, `amq_subjectparams`, `object_type`, `object_id`)
SELECT @now, @now, 'shared', u.user, 'files_sharing', 'shared_with_by', '[{"462":"\/mock.json"},"admin"]', 'files', 462
FROM (SELECT 1 FROM information_schema.columns LIMIT 1005) t
CROSS JOIN (SELECT 'user1' AS user UNION ALL SELECT 'user2' UNION ALL SELECT 'user3' UNION ALL SELECT 'user4' UNION ALL SELECT 'user5' UNION ALL SELECT 'user6' UNION ALL SELECT 'user7' UNION ALL SELECT 'user8' UNION ALL SELECT 'user9' UNION ALL SELECT 'user10' UNION ALL SELECT 'user11' UNION ALL SELECT 'user12' UNION ALL SELECT 'user13' UNION ALL SELECT 'user14' UNION ALL SELECT 'user15' UNION ALL SELECT 'user16' UNION ALL SELECT 'user17' UNION ALL SELECT 'user18' UNION ALL SELECT 'user19' UNION ALL SELECT 'user20') u;

-- 30 users
SET @now = UNIX_TIMESTAMP();
INSERT INTO `oc_activity_mq` (`amq_timestamp`, `amq_latest_send`, `amq_type`, `amq_affecteduser`, `amq_appid`, `amq_subject`, `amq_subjectparams`, `object_type`, `object_id`)
SELECT @now, @now, 'shared', u.user, 'files_sharing', 'shared_with_by', '[{"462":"\/mock.json"},"admin"]', 'files', 462
FROM (SELECT 1 FROM information_schema.columns LIMIT 1005) t
CROSS JOIN (SELECT 'user1' AS user UNION ALL SELECT 'user2' UNION ALL SELECT 'user3' UNION ALL SELECT 'user4' UNION ALL SELECT 'user5' UNION ALL SELECT 'user6' UNION ALL SELECT 'user7' UNION ALL SELECT 'user8' UNION ALL SELECT 'user9' UNION ALL SELECT 'user10' UNION ALL SELECT 'user11' UNION ALL SELECT 'user12' UNION ALL SELECT 'user13' UNION ALL SELECT 'user14' UNION ALL SELECT 'user15' UNION ALL SELECT 'user16' UNION ALL SELECT 'user17' UNION ALL SELECT 'user18' UNION ALL SELECT 'user19' UNION ALL SELECT 'user20' UNION ALL SELECT 'user21' UNION ALL SELECT 'user22' UNION ALL SELECT 'user23' UNION ALL SELECT 'user24' UNION ALL SELECT 'user25' UNION ALL SELECT 'user26' UNION ALL SELECT 'user27' UNION ALL SELECT 'user28' UNION ALL SELECT 'user29' UNION ALL SELECT 'user30') u;

• before inserting activities on DB for each scenario I emptied the table to start fresh
• after populated the table, identified the job id that has to be run (OCA\Activity\BackgroundJob\EmailNotification):

docker exec master-nextcloud-1 occ background-job:list | grep EmailNotification

• executed the job with time command, to see how long it takes:

time docker exec master-nextcloud-1 occ background-job:execute <job-id>

• results are as follows:

1 user   | 1005 activities               | real 0m5.473s
10 users | 1005 activities for each user | real 0m39.466s
20 users | 1005 activities for each user | real 0m47.544s
30 users | 1005 activities for each user | real 1m0.324s

• the emails displayed correctly 1000 emails, and 5 left as "and 5 more":

• each email with size aprox. 1.99MB:

@miaulalala not sure if this is the kind of testing you had in mind. If you think a different approach would be more appropriate, feel free to let me know.

[miaulalala]

Did you test the performance when having that many items?

I tested with the following env (nextcloud/nextcloud-docker-dev):

* created 30 users (user1 to user30)

for i in $(seq 1 30); do
  OC_PASS=admin docker exec -e OC_PASS=admin master-nextcloud-1 occ user:add --password-from-env --email user$i@mail.com user$i
done

* all of them with the following settings on "personal settings > notifications"
  
  * mail notification enabled for "a file or folder has been shared"
  * send activity emails set as "as soon as possible"

* set `mail_max_items` to max value (1000)

docker exec master-nextcloud-1 occ config:app:set activity mail_max_items --value=1000

* simulated 4 different scenarios
  
  * send mail to 1 user (1005 activities)
  * send mail to 10 users (1005 activities for each user)
  * send mail to 20 users (1005 activities for each user)
  * send mail to 30 users (1005 activities for each user)

* 1005 was used to test if the emails display correctly the remaining 5 summarized as "and 5 more"

* to create the entries on DB (`oc_activity_mq` table):

-- 1 user
SET @now = UNIX_TIMESTAMP();
INSERT INTO `oc_activity_mq` (`amq_timestamp`, `amq_latest_send`, `amq_type`, `amq_affecteduser`, `amq_appid`, `amq_subject`, `amq_subjectparams`, `object_type`, `object_id`)
SELECT @now, @now, 'shared', u.user, 'files_sharing', 'shared_with_by', '[{"462":"\/mock.json"},"admin"]', 'files', 462
FROM (SELECT 1 FROM information_schema.columns LIMIT 1005) t
CROSS JOIN (SELECT 'user1' AS user) u;

-- 10 users
SET @now = UNIX_TIMESTAMP();
INSERT INTO `oc_activity_mq` (`amq_timestamp`, `amq_latest_send`, `amq_type`, `amq_affecteduser`, `amq_appid`, `amq_subject`, `amq_subjectparams`, `object_type`, `object_id`)
SELECT @now, @now, 'shared', u.user, 'files_sharing', 'shared_with_by', '[{"462":"\/mock.json"},"admin"]', 'files', 462
FROM (SELECT 1 FROM information_schema.columns LIMIT 1005) t
CROSS JOIN (SELECT 'user1' AS user UNION ALL SELECT 'user2' UNION ALL SELECT 'user3' UNION ALL SELECT 'user4' UNION ALL SELECT 'user5' UNION ALL SELECT 'user6' UNION ALL SELECT 'user7' UNION ALL SELECT 'user8' UNION ALL SELECT 'user9' UNION ALL SELECT 'user10') u;

-- 20 users
SET @now = UNIX_TIMESTAMP();
INSERT INTO `oc_activity_mq` (`amq_timestamp`, `amq_latest_send`, `amq_type`, `amq_affecteduser`, `amq_appid`, `amq_subject`, `amq_subjectparams`, `object_type`, `object_id`)
SELECT @now, @now, 'shared', u.user, 'files_sharing', 'shared_with_by', '[{"462":"\/mock.json"},"admin"]', 'files', 462
FROM (SELECT 1 FROM information_schema.columns LIMIT 1005) t
CROSS JOIN (SELECT 'user1' AS user UNION ALL SELECT 'user2' UNION ALL SELECT 'user3' UNION ALL SELECT 'user4' UNION ALL SELECT 'user5' UNION ALL SELECT 'user6' UNION ALL SELECT 'user7' UNION ALL SELECT 'user8' UNION ALL SELECT 'user9' UNION ALL SELECT 'user10' UNION ALL SELECT 'user11' UNION ALL SELECT 'user12' UNION ALL SELECT 'user13' UNION ALL SELECT 'user14' UNION ALL SELECT 'user15' UNION ALL SELECT 'user16' UNION ALL SELECT 'user17' UNION ALL SELECT 'user18' UNION ALL SELECT 'user19' UNION ALL SELECT 'user20') u;

-- 30 users
SET @now = UNIX_TIMESTAMP();
INSERT INTO `oc_activity_mq` (`amq_timestamp`, `amq_latest_send`, `amq_type`, `amq_affecteduser`, `amq_appid`, `amq_subject`, `amq_subjectparams`, `object_type`, `object_id`)
SELECT @now, @now, 'shared', u.user, 'files_sharing', 'shared_with_by', '[{"462":"\/mock.json"},"admin"]', 'files', 462
FROM (SELECT 1 FROM information_schema.columns LIMIT 1005) t
CROSS JOIN (SELECT 'user1' AS user UNION ALL SELECT 'user2' UNION ALL SELECT 'user3' UNION ALL SELECT 'user4' UNION ALL SELECT 'user5' UNION ALL SELECT 'user6' UNION ALL SELECT 'user7' UNION ALL SELECT 'user8' UNION ALL SELECT 'user9' UNION ALL SELECT 'user10' UNION ALL SELECT 'user11' UNION ALL SELECT 'user12' UNION ALL SELECT 'user13' UNION ALL SELECT 'user14' UNION ALL SELECT 'user15' UNION ALL SELECT 'user16' UNION ALL SELECT 'user17' UNION ALL SELECT 'user18' UNION ALL SELECT 'user19' UNION ALL SELECT 'user20' UNION ALL SELECT 'user21' UNION ALL SELECT 'user22' UNION ALL SELECT 'user23' UNION ALL SELECT 'user24' UNION ALL SELECT 'user25' UNION ALL SELECT 'user26' UNION ALL SELECT 'user27' UNION ALL SELECT 'user28' UNION ALL SELECT 'user29' UNION ALL SELECT 'user30') u;

* before inserting activities on DB for each scenario I emptied the table to start fresh

* after populated the table, identified the job id that has to be run (`OCA\Activity\BackgroundJob\EmailNotification`):

docker exec master-nextcloud-1 occ background-job:list | grep EmailNotification

* executed the job with `time` command, to see how long it takes:

time docker exec master-nextcloud-1 occ background-job:execute <job-id>

* results are as follows:

1 user   | 1005 activities               | real 0m5.473s
10 users | 1005 activities for each user | real 0m39.466s
20 users | 1005 activities for each user | real 0m47.544s
30 users | 1005 activities for each user | real 1m0.324s

* the emails displayed correctly 1000 emails, and 5 left as "and 5 more":

* each email with size aprox. 1.99MB:

@miaulalala not sure if this is the kind of testing you had in mind. If you think a different approach would be more appropriate, feel free to let me know.

Nice, that's exactly what I had in mind.

[AndyScherzinger]

/backport to stable34

[AndyScherzinger]

/backport to stable33

[AndyScherzinger]

@cristianscheid triggered backports down to 33