Issue opened on monitor failure.

Issue resolved on monitor recovery.

Issue opened on monitor failure.

Issue resolved on monitor recovery.

Issue opened on monitor failure.

Issue resolved on monitor recovery.

Issue opened on monitor failure.

Issue resolved on monitor recovery.

Post-maintenance Redis issues have caused intermittent Wave availability for some users.

The related Redis issue has been identified and resolved. This issue has been confirmed to be corrected.

Issue opened on monitor failure.

Issue resolved on monitor recovery.

Issue opened on monitor failure.

Issue resolved on monitor recovery.

We are investigating an issue affecting the workflows running on Google Batch compute environments: executions are not affected but log fetching is returning an error. Our team is working on a fix

Team is working on a fix that may have been caused by a dependency mismatch.

Fix has been deployed to Production environment and it is currently under test

Fix has been tested in production and confirmed pipelines' logs are now available again on Google based Compute environments

Issue opened on monitor failure.

During a production upgrade the database wasn't able to perform a migration: we are working to scale the database instance vertically

Issue resolved on monitor recovery.

Issue opened on monitor failure.

Issue resolved on monitor recovery.

Issue re-opened on monitor failure.

Issue resolved on monitor recovery.

Issue opened on monitor failure.

Issue resolved on monitor recovery.

Issue opened on monitor failure.

Issue resolved on monitor recovery.

Issue opened on monitor failure.

Issue resolved on monitor recovery.

Issue opened on monitor failure.

Issue resolved on monitor recovery.

Issue opened on monitor failure.

Issue resolved on monitor recovery.

Issue opened on monitor failure.

Issue resolved on monitor recovery.

Issue opened on monitor failure.

Issue resolved on monitor recovery.

Issue opened on monitor failure.

Issue resolved on monitor recovery.

Issue opened on monitor failure.

Issue resolved on monitor recovery.

Issue re-opened on monitor failure.

Issue resolved on monitor recovery.

Issue opened on monitor failure.

Issue resolved on monitor recovery.

Issue opened on monitor failure.

Issue resolved on monitor recovery.

Issue re-opened on monitor failure.

Issue resolved on monitor recovery.

Issue opened on monitor failure.

Issue resolved on monitor recovery.

Issue re-opened on monitor failure.

Issue resolved on monitor recovery.

Issue opened on monitor failure.

Issue resolved on monitor recovery.

Issue re-opened on monitor failure.

Issue resolved on monitor recovery.

Issue re-opened on monitor failure.

Issue resolved on monitor recovery.

Issue re-opened on monitor failure.

Issue resolved on monitor recovery.

Seqera AI (https://seqera.io/ask-ai/) has been reporting intermittent failures Engineering identified the issue with an upstream vendor

Our engineering team has not reported any new failure since 15:47 UTC

The upstream vendor reported pushing a fix to their system Engineering continues monitoring the status of Seqera AI

Engineering reports the incident has been resolved

Issue opened on monitor failure.

Issue resolved on monitor recovery.

Issue opened on monitor failure.

Issue resolved on monitor recovery.

We are currently investigating reports of performance issues related to data-studios in Seqera Cloud Platform & hosting environment

We are aware of an issue impacting the wave-build service , our engineers are actively investigating this issue and will update with time to resolution shortly.

We have identified the root cause of the outage and have remediated the issue.

Wave build is operating normally - utilization has returned to expected levels.

Issue opened on monitor failure.

Issue resolved on monitor recovery.

It has been detected that the Seqera Platform Cloud is unavailable. The cause has been immediately determined and a fix is underway.

The issue has been identified and resolved.

We have identified an issue with Seqera Cloud that was preventing it from loading. We have potentially resolved the issue and are confirming the fix before marking this issue as closed.

The issue was identified to be an infrastructure problem that has been rectified. No further issues should remain.

Issue opened on monitor failure.

Issue resolved on monitor recovery.

An issue has been reporting in which some users are getting errors reporting "missing required parameters" on pipelines that were previously operating correctly. We are currently investigating the issue.

This issue has been identified and a workaround resolution has been discovered. Multiple customers reported 'Missing required parameter:' errors like: * Missing required parameter: --input * Missing required parameter: --outdir while executing pipelines that previously ran without incident. The error was replicated on Seqera Platform by leaving the 'pipeline parameters' field empty in the Launchpad pipeline configuration empty and supplying parameters via the schema-defined form only. The error is bypassed if any parameter is supplied via 'pipeline parameters' in the Launchpad configuration. Affected customers can use this as a workaround, adding to pipeline Launchpad configuration any parameter common to workflow executions.

We have identified a degraded service event affecting certain pipeline conditions for cloud users and those on enterprise version 24.1.0. The issue impacts users of nf-core pipelines or pipelines based on the nf-core template, specifically those relying on shared configs from https://nf-co.re/configs/ or those custom configs where an external includeConfig is utilized. Under these circumstances, new pipelines within these parameters cannot be created or launched. Our team is currently investigating the issue, and a fix will be released shortly.

A patch for Seqera Platform Cloud has been released and this should no longer be impacting Cloud users. A patch for enterprise is imminent and this incident will be closed for now as no existing enterprise users are affected by this.

An issue has been identified in which the execution of pipelines may have been affected due to a bug introduced by the upgrade of a library used by the platform to serialize/deserialize some data stored in our caching layer.

The problem has been solved and pipelines executions suffering from this issue should be working properly now.

Wave is reporting errors causing it to be unavailable for a subset of nextflow/platform users.

The issue was determined to be caused by a Redis cache problem, affecting only customers using the old platform endpoint https://api.tower.nf .

No further issues should exist.

A recent update to Wave caused inconsistency in the composition of container names which may have resulted in some users being unable to run pipelines relying on Wave.

This change has been rolled back and should no longer cause issues.

Issue opened on monitor failure.

Issue resolved on monitor recovery.

Issue opened on monitor failure.

Issue resolved on monitor recovery.

Start of Wave errors Retroactively, it was discovered that transient spikes in Wave container errors had occurred causing temporary degradation of service and impacting a limited number of customers.

A potential issue with the Wave service has been discovered impacting a limited number of clients.

Wave issues have been confirmed and a potential cause has been pinpointed.

A change has been pushed to the Seqera platform that has remedied the problem. The initial issue was caused by a misconfiguration that was exhausting the available connection pool.

An issue has been identified that is causing jobs in Platform Cloud to stall in the "Submitted" state.

The problem was caused by a stuck thread in our job scheduler component that was preventing job submissions from being performed correctly. We restarted the service and implemented a code fix to prevent the scenario from happening again. All stalled jobs should now be proceeding as normal.

A timeout issue has been identified that is impacting users of Wave containers [HTTP status=408].

The issue has been identified and resolved.

An incident with quay.io container registry is impacting the launch of data pipelines on Seqera Enterprise on-prem installations. We are monitoring the problem. Please contact Seqera support for more details.

The incident was solved by quay.io team.

An "Expired License Error" prevented a small number of users running pipelines on AWS Batch from launching pipelines between midnight and 09:00 CET on November 1st. We have now fixed the issue and we will be implementing solutions to prevent this from happening in the future. We apologize for any inconvenience this may have caused.

A temporary connection failure has been reported by some users. We are investigating the possible problem.

A failure during the rollout of a system upgrade caused some backend replicas not to reach the ready status, resulting in a temporary system unreliability.

An issue has been identified in the Wave service that caused a partial system outage from Fri, September 30 15:30 until Mon, October 2 12:20. The problem is related to a bug in Java HTTP client used by Wave backend to access remote repositories, which caused the system to consume the Java VM heap memory in an uncontrolled manner and exhaust the underlying threads pool handling the incoming client requests. https://bugs.openjdk.org/browse/JDK-8308144 A solution has been delivered to prevent this problem.

Issue opened on monitor failure.

Issue resolved on monitor recovery.

Issue opened on monitor failure.

Issue resolved on monitor recovery.

Issue opened on monitor failure.

Issue resolved on monitor recovery.

Issue opened on monitor failure.

Issue resolved on monitor recovery.

Issue opened on monitor failure.

Issue resolved on monitor recovery.