CoolCMS API

CronFailureManager
in package

FinalYes

Applies a retry back-off to failing items of cron queues.

Without it a permanently failing item at the head of a queue (order expiry, catch-up invoices, invoice schedules, expiry reminders) is retried every run in the same order, eats the whole batch and the healthy items behind it are never processed.

One shared mechanism: the table cms_system_cron_failures, keyed by (queue, item id), holds the failure count, the truncated last error and next_attempt_at. A queue:

  • takes findParkedIds() before selecting a batch and excludes them from the query (the batch is filled with other rows, selection stays deterministic by the queue's order),
  • calls recordFailure() after a failure (back-off 1 min, 5 min, 30 min, 2 h, then 24 h),
  • calls recordSuccess() after a success (the row disappears),
  • calls sendExhaustedNotification() at the end of a run; items that just exhausted MAX_ATTEMPTS produce one notification per queue run (not per item). An exhausted item stays parked and is retried once every 24 h.

Writes use raw DBAL without Api reads, so there is no UnitOfWork::clear() in the middle of the caller's batch. The only Api read (the notification type) runs in sendExhaustedNotification() after the batch. Best-effort: a bookkeeping error is logged and the queue continues without back-off. Time is computed by PHP (write and comparison), so the DB time zone does not apply.

Table of Contents

Constants

FAILED_NOTIFICATION_TYPE  : string = 'CRON_ITEM_FAILED'
Notification type in the admin bell.
MAX_ATTEMPTS  : int = 5
After this many consecutive failures a notification is sent and the item stays on the 24 h back-off.
RETRY_BACKOFF_MINUTES  : array<string|int, mixed> = [1, 5, 30, 120, 1440]
Back-off in minutes by consecutive failures; the 5th and later mean 24 h.
ERROR_MAX_LENGTH  : int = 255
PARKED_LIMIT  : int = 1000
Cap of the parked id list used in the queue's `NOT IN` selection.
STALE_DAYS  : int = 7
A row not refreshed for this many days belongs to an item that left the queue another way (canceled order, deleted advert); a parked item is retried at least once every 24 h.
TABLE  : string = 'cms_system_cron_failures'

Properties

$connection  : Connection
$exhausted  : array<string, array{count: int, failureId: int, itemId: int, error: string}>
$notificationManager  : NotificationManager
$notificationTypeManager  : NotificationTypeManager

Methods

__construct()  : mixed
findExistingFailureIds()  : array<int, int>
Returns which bookkeeping rows still exist; the probe of the orphan notification cleanup.
findParkedIds()  : array<int, int>
Returns the ids of queue items that are not due yet; batch selection skips them. Also cleans up rows of items that have left the queue meanwhile ({@see STALE_DAYS}).
recordFailure()  : void
Records an item failure: one more attempt, the last error and the back-off before the next attempt.
recordSuccess()  : void
Records a success (or that the item left the queue another way); the failure record is deleted.
retryDelayMinutes()  : int
Back-off in minutes before the next attempt after `$attempts` consecutive failures.
sendExhaustedNotification()  : void
Sends one notification per queue run when some items exhausted their attempts. The link (`itemId`) points to the bookkeeping row of the last of them; when the item recovers, the row disappears and the orphan notification cleanup deletes the notification ({@see \App\UI\Api\System\Models\Helpers\CronFailureNotificationTargetProbe}).
now()  : DateTimeImmutable
shortenError()  : string
Stores only the first line, truncated: DB exceptions carry the full SQL with values and mail send errors carry the recipient address, so e-mails are masked (the same mask as on the Api output) before the text is truncated.
toInt()  : int

Constants

FAILED_NOTIFICATION_TYPE

Notification type in the admin bell.

public string FAILED_NOTIFICATION_TYPE = 'CRON_ITEM_FAILED'

MAX_ATTEMPTS

After this many consecutive failures a notification is sent and the item stays on the 24 h back-off.

public int MAX_ATTEMPTS = 5

RETRY_BACKOFF_MINUTES

Back-off in minutes by consecutive failures; the 5th and later mean 24 h.

public array<string|int, mixed> RETRY_BACKOFF_MINUTES = [1, 5, 30, 120, 1440]

PARKED_LIMIT

Cap of the parked id list used in the queue's `NOT IN` selection.

private int PARKED_LIMIT = 1000

STALE_DAYS

A row not refreshed for this many days belongs to an item that left the queue another way (canceled order, deleted advert); a parked item is retried at least once every 24 h.

private int STALE_DAYS = 7

Properties

$exhausted

private array<string, array{count: int, failureId: int, itemId: int, error: string}> $exhausted = []

queue => items that exhausted their attempts in this run

Methods

findExistingFailureIds()

Returns which bookkeeping rows still exist; the probe of the orphan notification cleanup.

public findExistingFailureIds(array<int, int> $ids) : array<int, int>
Parameters
$ids : array<int, int>
Return values
array<int, int>

findParkedIds()

Returns the ids of queue items that are not due yet; batch selection skips them. Also cleans up rows of items that have left the queue meanwhile ({@see STALE_DAYS}).

public findParkedIds(string $queue) : array<int, int>
Parameters
$queue : string
Return values
array<int, int>

recordFailure()

Records an item failure: one more attempt, the last error and the back-off before the next attempt.

public recordFailure(string $queue, int $itemId, string $error) : void
Parameters
$queue : string
$itemId : int
$error : string

recordSuccess()

Records a success (or that the item left the queue another way); the failure record is deleted.

public recordSuccess(string $queue, int $itemId) : void
Parameters
$queue : string
$itemId : int

retryDelayMinutes()

Back-off in minutes before the next attempt after `$attempts` consecutive failures.

public static retryDelayMinutes(int $attempts) : int
Parameters
$attempts : int
Return values
int

sendExhaustedNotification()

Sends one notification per queue run when some items exhausted their attempts. The link (`itemId`) points to the bookkeeping row of the last of them; when the item recovers, the row disappears and the orphan notification cleanup deletes the notification ({@see \App\UI\Api\System\Models\Helpers\CronFailureNotificationTargetProbe}).

public sendExhaustedNotification(string $queue) : void

Best-effort: a missing type is skipped silently, an error is only logged.

Parameters
$queue : string

shortenError()

Stores only the first line, truncated: DB exceptions carry the full SQL with values and mail send errors carry the recipient address, so e-mails are masked (the same mask as on the Api output) before the text is truncated.

private shortenError(string $error) : string
Parameters
$error : string
Return values
string

toInt()

private static toInt(mixed $value) : int
Parameters
$value : mixed
Return values
int
On this page

Search results