Copr on Packit Service Status

[Resolved] Copr outage - stuck hypervisor

Tue, 13 Jan 2026 02:00:00 +0100

There was a Copr outage and as a result Copr builds were stuck in a pending state.

Now the outage is over and any pending jobs should be processed automatically. If they aren’t trigger them manually.

However, there is a long queue of builds in Packit and it could take hours for it to decrease.

[Resolved] Copr outage

Wed, 26 Nov 2025 08:00:00 +0100

Copr services are temporarily unavailable due to service outage.

In Packit, this results in Copr builds stuck in a pending state.

[Resolved] Fedora Infra - Datacenter Move outage

Mon, 30 Jun 2025 01:00:00 +0000

Fedora Infrastructure is currently ongoing migration to different datacenter, during this migration majority of the contributor-facing services will be down.

Therefore it is expected that sync-release (i.e., propose-downstream and pull-from-upstream), koji_build, and bodhi_update Packit workflows will be affected.

Copr build jobs may be disrupted, full outage is not expected.

For more details watch:

Update - 4th July

Most services in Fedora Infrastructure should be up and running again. Packit has resumed processing copr_build jobs, but there are still delays due to jobs being queued during the outage in previous days. We are continuing to investigate issues related to pushing and pulling from dist-git, which are currently impacting downstream jobs. We’ll share further updates as soon as we have more information.

[Resolved] Copr SRPM builds failing

Wed, 05 Mar 2025 08:00:00 +0100

SRPM builds in Copr have been failing, the reason is still unknown, please be patient. You can monitor #buildsys:fedoraproject.org on Matrix for updates.

Update: Copr SRPM builds are working again. You can retrigger failed jobs with /packit build or /packit rebuild-failed.

[Resolved] Maintenance outage

Mon, 03 Feb 2025 15:15:00 +0100

Today’s weekly production deployment requires a smaller outage to ensure proper redeployment without any inconsistencies.

We expect an outage for up to 15 minutes. There will be no jobs processed from 2025-02-03 15:15 UTC till the successful redeployment.

In case you are affected, please retrigger the jobs accordingly (via comments, or Re-Run button on GitHub Checks) once the outage is over.

Thanks for the understanding.

[Resolved] Copr outage

Mon, 23 Dec 2024 06:00:00 +0100

Copr services are temporarily unavailable due to a service disruption.

In Packit, this results in Copr builds stuck in a pending state.

[Resolved] Copr outage - system update

Wed, 04 Dec 2024 10:00:00 +0100

There is a Copr outage and as a result Copr builds can be stuck in a pending state.

Update: the outage is over, any pending jobs should be processed automatically. In case some jobs failed to be triggered, you can trigger them manually.

[Resolved] Maintenance outage

Wed, 09 Oct 2024 20:00:00 +0200

Database volume needs to be enlarged to accomodate the load on the Packit Service. To minimize the amount of jobs affected by this maintenance there will be no jobs processed, but they should be enqueued until the outage is over.

In case you are affected, please retrigger the jobs accordingly (via comment on the affected pull requests, or Re-Run button on GitHub Checks).

Thanks for understanding.

[Resolved] Fedora infrastructure outage

Sun, 15 Sep 2024 08:00:00 +0000

Fedora infrastructure is having an outage resulting in some Copr build jobs failing with error Failed to download metadata... Librepo error: Cannot download repomd.xml... You can watch the status of the issue here: https://pagure.io/fedora-infrastructure/issue/12183.

[Resolved] Many Copr builds pending

Tue, 23 Apr 2024 14:00:00 +0100

The Copr backend is under a huge load and many builds are still in a pending state. The Copr builds will take more time than usual to finish. You can monitor the pending queue here: https://copr.fedorainfracloud.org/status/stats/.

[Resolved] Copr builds stuck in pending or running state

Wed, 06 Dec 2023 12:00:00 +0100

Copr builds could have been stuck in a pending or running state. The issue has been resolved, it may take a while for the affected Copr builds to finish, but there is no need to retrigger them.

[Resolved] Copr Upgrade

Tue, 28 Nov 2023 13:30:00 +0000

Due to an upgrade of the Fedora Copr Servers, Copr builds might have been delayed or failing.

[Resolved] GitLab.com outage

Tue, 10 Oct 2023 04:00:00 +0200

GitLab.com workflows are disrupted due to the issue with detecting the upstream/downstream repository in the current working directory.

Self-hosted GitLab forges are not affected.

[Resolved] COPR build failures

Sat, 09 Sep 2023 12:00:00 +0000

Copr is experiencing issues with builds. We are sorry about that. You can follow the progress in the Copr issue.

The failure in the COPR logs looks like the following: Machine 5b8689e424334255ab9ace8a58343076 still running. Killing...

In case you are building in your own COPR project you can go to your project settings and change Mock bootstrap from default to Enable for a quick workaround.

[Resolved] Copr frontend instability

Tue, 05 Sep 2023 15:15:00 +0200

Copr is experiencing issues with its frontend which is currently mitigated while Copr team investigates.

In case some of your builds were not submitted, please retrigger.

[Resolved] Copr Upgrade

Wed, 16 Aug 2023 14:30:00 +0200

There will be a Copr outage starting at

date --date "2023-08-16 12:30 UTC"

which will last approximately 2 hours. The build queue processing will be stopped. During the outage time, we expect a very short downtime for the DNF package storage (less than 10 minutes).

Reason for outage:

We will update the infrastructure machines to the latest packages that are currently being developed.

Affected Services:

https://copr.fedorainfracloud.org/ https://download.copr.fedorainfracloud.org/results/

Upstream ticket:

https://github.com/fedora-copr/copr/issues/2824

Please join #fedora-buildsys, #fedora-admin or #fedora-noc on irc.libera.chat or add comments to this ticket.

[Resolved] Upgrade of Copr servers

Thu, 25 May 2023 10:00:00 +0000

Due to an upgrade of the Fedora Copr Servers, Copr builds might be delayed or failing. The upgrade should last approximately 3 hours.

[Resolved] Upgrade of Copr servers

Wed, 05 Apr 2023 12:00:00 +0000

Due to an upgrade of the Fedora Copr Servers, Copr builds might be delayed or failing. The upgrade should last approximately 3 hours.

[Resolved] Running grouping migrations

Tue, 14 Feb 2023 02:00:00 +0100

We are updating our database schema to facilitate further development. As a part of this process, multiple database migrations will have to be run. The first step is grouping together related builds and test runs. The migrations will happen during the next deployment to production, on Tuesday February 14 at 2:00AM CET. This set of migrations is expected to take roughly 40 minutes, after which the service should come back up.

Since significant refactoring was paired with these changes, there may be some bugs/regressions that we did not spot on staging. Please, contact us via the usual means if you spot anything suspicious.

[Resolved] Upgrade of Copr Servers

Mon, 28 Nov 2022 16:00:00 +0100

Due to an upgrade of the Fedora Copr Servers, Copr builds might have been failing.

[Resolved] Copr storage move

Fri, 28 Oct 2022 21:00:00 +0200

Copr team is performing a backend storage move over the weekend of Oct 28-30th. Building in Copr will be disrupted or unavailable during this period.

For more info, please follow the Copr outage announcement.

Update: the upgrade was successful and Copr is fully operational.

[Resolved] Issues with the task execution

Tue, 11 Oct 2022 13:57:00 +0200

We are experiencing issues with the execution of the tasks. (The statuses are waiting in the The task was accepted. state.) We are investigating the cause. Sorry about the possible troubles. We will try to fix this as soon as possible.

Updates:

Tue Oct 11 12:24:02 UTC: We are slowly going through the queue. Please, be patient.
Tue Oct 11 13:12:36 UTC: We have gone through the queue and you should experience regular response times. The original cause is still not known but the problem probably occured during an SRPM upload to Copr. (It can be a fault on our or Copr side.)
Wed Oct 12 05:27:46 UTC: Service operated normally since the last update. To reduce the chance of this, we plan to move the building of SRPMs to Copr. (Currently default for new installations and opt in for other projects.)

[Resolved] Upgrade of Copr Servers

Wed, 22 Jun 2022 14:00:00 +0200

Due to an ongoing upgrade of the Fedora Copr Servers, building might be delayed or failing. The upgrade should last ~3 hours.