On November 4, 2025, between 20:10 UTC and 21:54 UTC, some customers with our US data center may have experienced delays in message sending. This issue did not affect data ingestion and no data was lost.
A configuration change caused a failure in processing attribute updates from campaign attribute & webhook actions resulting in campaign journeys stalling. The configuration change was reverted, restoring the attribute update processing and message sending.
A configuration change to the internal authentication system used for inter-service communication unexpectedly caused our services responsible for attribute processing to fail, leading to a backlog in message sending.
Engineers reverted the configuration change at 21:07 UTC, allowing attribute processing to resume successfully. The team monitored the system until confirming recovery at 21:54 UTC.
To improve resilience, the team is enhancing monitoring to detect early signs of attribute processing faults and updating alerting to provide clearer failure context and faster remediation. These measures are being tracked and prioritized within our internal development process.