Incident - 2025-07-05

Summary

On July 5th, from 1:08pm to 4:33pm (CEST, UTC+02:00), Forest Admin experience an outage lasting 3 hours and 25 minutes, during which the service was unavailable.

Origin

As part of our ongoing infrastructure migration to AWS, we moved scheduled jobs from Heroku Scheduler to our new Forest Admin Scheduler. Unfortunately, the migration of one specific job failed and caused an issue in our production database.

Resolution

Upon identifying the root cause, we immediately restored a backup of the production database, paused then corrected the failed job. The service was fully restored by 4:33pm (CEST, UTC+02:00).

Context

This migration to AWS was initiated following two significant outages on Heroku (in November and June), which highlighted limitations in its reliability for our needs. We are committed to providing a more robust, reliable and resilient platform for our customers, and Heroku is no longer our best partner in doing so. Moving to AWS is a key part of this strategy.
We expect the full migration to be completed by the end of Q3 2025.
To minimize disruption, we will consult our customers to define an appropriate maintenance window for the final steps of the migration.