Experiencing disruptions

Outage →

all

There has been a Packit outage. We are investigating the reason of it. The jobs in the queue are now being picked up again and should be processed gradually. We are very sorry for the inconvenience!


Core   (?) Systems we have complete control over
API   (?) prod.packit.dev/api - If API is down, we're not able to receive your requests. Operational
Workers   (?) If workers are down, received requests are waiting in a queue to be proceeded. Operational
Dashboard   (?) dashboard.packit.dev Operational
Others   (?) Systems we use, but have no direct control over
Copr   (?) copr.fedorainfracloud.org - If it's down, we can't build your packages. Operational
Testing Farm   (?) api.dev.testing-farm.io - If it's down, we can't run tests. Operational
Koji   (?) koji.fedoraproject.org Operational
Bodhi   (?) bodhi.fedoraproject.org Operational
Incidents UptimeRobot Testing Farm Status Packit docs Packit dashboard

2024 (9)

April 25, 2024 at 2:30 PM UTC

Outage

▲ This issue is not resolved yet
April 23, 2024 at 2:00 PM UTC

Many Copr builds pending

Resolved after 7h 0m of downtime
April 5, 2024 at 10:15 AM UTC

GitHub outage

Resolved after 60m of downtime
February 26, 2024 at 1:00 PM UTC

Fedora infrastructure authentication problem

Resolved after 28m of downtime
February 21, 2024 at 9:00 PM UTC

Koji Outage

Resolved after 3h 0m of downtime
February 21, 2024 at 12:00 AM UTC

Pull From Upstream Outage

Resolved after 38h 40m of downtime
January 22, 2024 at 10:00 PM UTC

Fedora infrastructure authentication problem

Resolved after 23h 0m of downtime
January 11, 2024 at 9:30 AM UTC

Outage of automatic triggering of downstream Koji builds

Resolved after 101h 50m of downtime
January 9, 2024 at 2:45 PM UTC

GitHub outage

Resolved after 7320h 55m of downtime

2023 (38)

December 22, 2023 at 7:00 PM UTC

Christmas is here 🎄

Resolved after 254h 0m of downtime
December 19, 2023 at 5:30 AM UTC

GitLab outage

Resolved after 30h 30m of downtime
December 10, 2023 at 8:45 PM UTC

Production instance firewall

Resolved after 1596h 15m of downtime
December 10, 2023 at 8:40 PM UTC

Testing Farm outage

Resolved after 15h 20m of downtime
December 10, 2023 at 8:15 PM UTC

Production move

Resolved after 21h 45m of downtime
December 6, 2023 at 12:00 PM UTC

Copr builds stuck in pending or running state

Resolved after 4h 0m of downtime
November 28, 2023 at 1:30 PM UTC

Copr Upgrade

Resolved after 3h 15m of downtime
November 14, 2023 at 10:00 AM UTC

COPR builds failing for epel8

Resolved after 7h 15m of downtime
November 12, 2023 at 3:00 PM UTC

Production move

Resolved after 2h 15m of downtime
November 7, 2023 at 2:45 PM UTC

GitHub outage

Resolved after 45m of downtime
October 10, 2023 at 4:00 AM UTC

GitLab.com outage

Resolved after 34h 0m of downtime
October 2, 2023 at 8:00 AM UTC

Cluster upgrade

Resolved after 2h 0m of downtime
September 9, 2023 at 12:00 PM UTC

COPR build failures

Resolved after 51h 0m of downtime
September 5, 2023 at 3:15 PM UTC

Copr frontend instability

Resolved in under a minute
August 28, 2023 at 10:00 AM UTC

System upgrade scheduled

Resolved after 1h 45m of downtime
August 24, 2023 at 3:40 PM UTC

Testing Farm outage

Resolved after 20m of downtime
August 21, 2023 at 2:45 PM UTC

Packit outage now resolved

Resolved after 30m of downtime
August 16, 2023 at 2:30 PM UTC

Copr Upgrade

Resolved after 2h 0m of downtime
August 15, 2023 at 4:00 AM UTC

Outage

Resolved after 7h 3m of downtime
July 24, 2023 at 2:30 PM UTC

Staging instance firewall

Resolved after 3342h 30m of downtime
July 19, 2023 at 8:00 AM UTC

Testing Farm testing blocked

Resolved after 3h 0m of downtime
July 3, 2023 at 8:00 AM UTC

Guest couldn't be provisioned: Artemis resource ended in 'error' state

Resolved after 4h 0m of downtime
June 22, 2023 at 9:30 AM UTC

Fedora Rawhide Testing Blocked on dnf5

Resolved after 635h 0m of downtime
May 25, 2023 at 10:00 AM UTC

Upgrade of Copr servers

Resolved after 3h 0m of downtime
May 11, 2023 at 3:30 PM UTC

GitHub outage

Resolved after 5h 30m of downtime
May 10, 2023 at 2:00 PM UTC

Testing Farm errors

Resolved after 25h 0m of downtime
May 9, 2023 at 1:50 PM UTC

GitHub outage

Resolved after 9h 25m of downtime
May 4, 2023 at 11:30 AM UTC

Bodhi Updates not created

Resolved after 148h 30m of downtime
May 3, 2023 at 10:00 AM UTC

Cluster Update

Resolved after 1h 15m of downtime
April 26, 2023 at 4:40 AM UTC

Scheduled DB Cleanup

Resolved after 37m of downtime
April 11, 2023 at 6:28 PM UTC

Issue with pushing to dist-git

Resolved after 22h 2m of downtime
April 5, 2023 at 12:00 PM UTC

Upgrade of Copr servers

Resolved after 3h 0m of downtime
March 9, 2023 at 10:00 AM UTC

SLO1 Error Budget issues

Resolved after 2184h 0m of downtime
February 14, 2023 at 2:00 AM UTC

Running grouping migrations

Resolved after 2h 0m of downtime
January 31, 2023 at 10:00 AM UTC

SLO2 Error Budget Depleted in a Day

Resolved after 24h 0m of downtime
January 24, 2023 at 3:00 PM UTC

Testing Farm outage

Resolved after 19h 0m of downtime
January 20, 2023 at 9:19 PM UTC

SRPM builds stuck in Submitted state

Resolved after 12h 32m of downtime
January 6, 2023 at 10:00 AM UTC

System upgrade scheduled

Resolved after 12h 0m of downtime

2022 (19)

December 23, 2022 at 6:00 PM UTC

Christmas is here 🎄

Resolved after 230h 0m of downtime
November 28, 2022 at 4:00 PM UTC

Upgrade of Copr Servers

Resolved after 5h 0m of downtime
November 25, 2022 at 8:35 AM UTC

Staging is unstable

Resolved after 125h 31m of downtime
November 24, 2022 at 11:24 PM UTC

Outage

Resolved after 9h 11m of downtime
November 24, 2022 at 7:00 PM UTC

System upgrade scheduled

Resolved after 4h 0m of downtime
November 1, 2022 at 3:00 AM UTC

Copr builds and tests on commits and releases ignored

Resolved after 31h 30m of downtime
October 28, 2022 at 9:00 PM UTC

Copr storage move

Resolved after 35h 0m of downtime
October 13, 2022 at 7:00 PM UTC

System upgrade scheduled

Resolved after 1h 37m of downtime
October 11, 2022 at 1:57 PM UTC

Issues with the task execution

Resolved after 1h 15m of downtime
October 10, 2022 at 3:14 PM UTC

Internal Testing Farm infrastructure is having an outage

Resolved after 30h 20m of downtime
October 5, 2022 at 4:00 PM UTC

GitHub Webhooks not working

Resolved after 2h 0m of downtime
September 22, 2022 at 8:00 AM UTC

September 22th outage

Resolved after 24h 0m of downtime
September 1, 2022 at 1:00 AM UTC

3/4 of SLO1 Error Budget consumed in 6 weeks

Resolved after 984h 0m of downtime
August 23, 2022 at 4:00 AM UTC

Failing SRPM builds done in Copr

Resolved after 5h 0m of downtime
July 4, 2022 at 8:00 AM UTC

Summer is here ☀️

Resolved after 106h 0m of downtime
June 22, 2022 at 2:00 PM UTC

Upgrade of Copr Servers

Resolved after 2h 10m of downtime
June 21, 2022 at 10:18 AM UTC

SRPM builds done in Copr are failing

Resolved after 52m of downtime
February 3, 2022 at 8:37 AM UTC

Networking Issue

Resolved after 2h 40m of downtime
February 1, 2022 at 2:00 AM UTC

February 1st outage

Resolved after 2h 15m of downtime

2021 (10)

December 8, 2021 at 2:00 AM UTC

New production deployment

Resolved after 574h 0m of downtime
December 7, 2021 at 9:06 AM UTC

Longer response times

Resolved after 2h 43m of downtime
November 18, 2021 at 2:33 AM UTC

Problems with production database

Resolved after 21h 37m of downtime
November 18, 2021 at 1:00 AM UTC

Moving production Packit Service to a new cluster

Resolved after 1h 33m of downtime
September 7, 2021 at 8:30 PM UTC

Slow SRPM builds

Resolved after 211h 24m of downtime
August 27, 2021 at 3:00 PM UTC

August 27th outage

Resolved after 19h 0m of downtime
July 1, 2021 at 10:55 AM UTC

Summer is here ☀️ #2

Resolved after 130h 15m of downtime
June 25, 2021 at 10:55 AM UTC

Summer is here ☀️

Resolved after 82h 15m of downtime
June 4, 2021 at 7:00 AM UTC

PostgreSQL db upgrade

Resolved after 2h 10m of downtime
June 2, 2021 at 11:16 AM UTC

Dashboard - unable to connect

Resolved after 24m of downtime