[2023-10-25 01:33:41,737][  INFO][PID:18146] Marking build as starting
[2023-10-25 01:33:41,767][  INFO][PID:18146] Trying to allocate VM
[2023-10-25 01:33:44,862][  INFO][PID:18146] Allocated host ResallocHost, ticket_id=204806, hostname=10.0.1.12
[2023-10-25 01:33:44,863][  INFO][PID:18146] Allocating ssh connection to builder
[2023-10-25 01:33:44,863][  INFO][PID:18146] Checking that builder machine is OK
[2023-10-25 01:33:45,045][  INFO][PID:18146] Filling build.info file with builder info
[2023-10-25 01:33:45,047][  INFO][PID:18146] Marking build as running on frontend
[2023-10-25 01:33:45,065][  INFO][PID:18146] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --build-id 91946 --chroot openeuler-23.09-x86_64 --detached
[2023-10-25 01:33:45,599][  INFO][PID:18146] Downloading the builder-live.log file, attempt 1
[2023-10-25 02:19:33,859][WARNING][PID:18146] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2023-10-25 02:19:43,864][  INFO][PID:18146] Downloading the builder-live.log file, attempt 2
[2023-10-25 02:19:56,969][WARNING][PID:18146] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2023-10-25 02:20:06,979][  INFO][PID:18146] Downloading the builder-live.log file, attempt 3
[2023-10-25 02:20:20,077][WARNING][PID:18146] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2023-10-25 02:20:30,088][  INFO][PID:18146] Downloading the builder-live.log file, attempt 4
[2023-10-25 02:20:43,181][ ERROR][PID:18146] Giving up for unstable SSH, failures: 1698200373.8593533, 1698200396.9690177, 1698200420.0779068, 1698200443.1811576
[2023-10-25 02:20:43,182][ ERROR][PID:18146] Re-try request for task on 'ResallocHost, ticket_id=204806, hostname=10.0.1.12': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2023-10-25 02:20:43,182][  INFO][PID:18146] Releasing VM back to pool
[2023-10-25 02:20:43,194][  INFO][PID:18146] Retry #1 (on other host)
[2023-10-25 02:20:43,196][  INFO][PID:18146] Trying to allocate VM
[2023-10-25 02:20:46,227][  INFO][PID:18146] Allocated host ResallocHost, ticket_id=204813, hostname=10.0.1.127
[2023-10-25 02:20:46,228][  INFO][PID:18146] Allocating ssh connection to builder
[2023-10-25 02:20:46,228][  INFO][PID:18146] Checking that builder machine is OK
[2023-10-25 02:20:46,396][  INFO][PID:18146] Filling build.info file with builder info
[2023-10-25 02:20:46,397][  INFO][PID:18146] Not re-notifying FE and msg buses for the new host.
[2023-10-25 02:20:46,398][  INFO][PID:18146] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --build-id 91946 --chroot openeuler-23.09-x86_64 --detached
[2023-10-25 02:20:46,824][  INFO][PID:18146] Downloading the builder-live.log file, attempt 1
[2023-10-25 03:07:42,876][WARNING][PID:18146] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2023-10-25 03:07:52,886][  INFO][PID:18146] Downloading the builder-live.log file, attempt 2
[2023-10-25 03:08:05,998][WARNING][PID:18146] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2023-10-25 03:08:16,009][  INFO][PID:18146] Downloading the builder-live.log file, attempt 3
[2023-10-25 03:08:29,097][WARNING][PID:18146] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2023-10-25 03:08:39,107][  INFO][PID:18146] Downloading the builder-live.log file, attempt 4
[2023-10-25 03:08:52,198][ ERROR][PID:18146] Giving up for unstable SSH, failures: 1698203262.8760564, 1698203285.9987028, 1698203309.0973322, 1698203332.1983056
[2023-10-25 03:08:52,199][ ERROR][PID:18146] Re-try request for task on 'ResallocHost, ticket_id=204813, hostname=10.0.1.127': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2023-10-25 03:08:52,199][  INFO][PID:18146] Releasing VM back to pool
[2023-10-25 03:08:52,211][  INFO][PID:18146] Retry #2 (on other host)
[2023-10-25 03:08:52,212][  INFO][PID:18146] Trying to allocate VM
[2023-10-25 03:08:55,243][  INFO][PID:18146] Allocated host ResallocHost, ticket_id=204846, hostname=10.0.1.120
[2023-10-25 03:08:55,243][  INFO][PID:18146] Allocating ssh connection to builder
[2023-10-25 03:08:55,244][  INFO][PID:18146] Checking that builder machine is OK
[2023-10-25 03:08:55,407][  INFO][PID:18146] Filling build.info file with builder info
[2023-10-25 03:08:55,409][  INFO][PID:18146] Not re-notifying FE and msg buses for the new host.
[2023-10-25 03:08:55,409][  INFO][PID:18146] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --build-id 91946 --chroot openeuler-23.09-x86_64 --detached
[2023-10-25 03:08:55,814][  INFO][PID:18146] Downloading the builder-live.log file, attempt 1
[2023-10-25 03:52:12,270][WARNING][PID:18146] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2023-10-25 03:52:22,280][  INFO][PID:18146] Downloading the builder-live.log file, attempt 2
[2023-10-25 03:52:50,606][WARNING][PID:18146] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2023-10-25 03:53:00,617][  INFO][PID:18146] Downloading the builder-live.log file, attempt 3
[2023-10-25 03:53:13,709][WARNING][PID:18146] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2023-10-25 03:53:23,712][  INFO][PID:18146] Downloading the builder-live.log file, attempt 4
[2023-10-25 03:53:36,814][ ERROR][PID:18146] Giving up for unstable SSH, failures: 1698205932.2701254, 1698205970.606636, 1698205993.7095878, 1698206016.8146312
[2023-10-25 03:53:36,815][ ERROR][PID:18146] Re-try request for task on 'ResallocHost, ticket_id=204846, hostname=10.0.1.120': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2023-10-25 03:53:36,816][  INFO][PID:18146] Releasing VM back to pool
[2023-10-25 03:53:36,829][ ERROR][PID:18146] Backend process error: Three host tried without success: {'10.0.1.120', '10.0.1.127', '10.0.1.12'}
[2023-10-25 03:53:36,829][WARNING][PID:18146] Switching not-finished job state to 'failed'
[2023-10-25 03:53:36,830][  INFO][PID:18146] Worker failed build, took 2681.4201426506042
[2023-10-25 03:53:36,852][  INFO][PID:18146] Compressing /var/lib/copr/public_html/results/bigclouds/busybox/openeuler-23.09-x86_64/00091946-pytorch/builder-live.log by gzip
[2023-10-25 03:53:36,853][  INFO][PID:18146] Running command 'gzip /var/lib/copr/public_html/results/bigclouds/busybox/openeuler-23.09-x86_64/00091946-pytorch/builder-live.log' as PID 19469
[2023-10-25 03:53:36,854][  INFO][PID:18146] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/bigclouds/busybox/openeuler-23.09-x86_64/00091946-pytorch/builder-live.log)
[2023-10-25 03:53:36,854][  INFO][PID:18146] Compressing /var/lib/copr/public_html/results/bigclouds/busybox/openeuler-23.09-x86_64/00091946-pytorch/backend.log by gzip