We are continuing to work on a fix to this issue, and greatly appreciate your patience as we do so. After much testing and many failures with other methods, we have decided to segment the data in order to shrink the data set that needs the index re-added, and therefore should speed up the recovery process. When trying to work with the table as a whole, the sheer size of it caused issues with timeouts, out-of-memory errors, and other failures that have prevented us from restoring full functionality. We have also identified ways to add more resiliency to the Reportage application to avoid any future issues like this.
We would like to note that this issue appears to only affect some interaction reports. Our testing shows that smaller interaction reports can still be pulled and displayed successfully, but they time out as the size of the report increases. We haven't noticed any similar slowdowns for other types of reports.
Again, we thank you for your patience, and we will continue to update this incident as we have more information.
Posted Dec 06, 2024 - 04:08 UTC
Identified
While getting the Reportage database prepared for an update, an index was removed that causes interactions reports to either take a long time to load or not load altogether. We are attempting to re-add the index but unfortunately, due to the size of the table, this is taking longer than expected.
Posted Nov 08, 2024 - 16:23 UTC
Investigating
We're experiencing performance issues with Reportage that are resulting in slow response times and timeouts and are actively looking into the issue.