[2023-10-24 06:59:21,625][  INFO][PID:17605] Marking build as starting
[2023-10-24 06:59:21,655][  INFO][PID:17605] Trying to allocate VM
[2023-10-24 06:59:24,713][  INFO][PID:17605] Allocated host ResallocHost, ticket_id=204779, hostname=10.0.1.14
[2023-10-24 06:59:24,713][  INFO][PID:17605] Allocating ssh connection to builder
[2023-10-24 06:59:24,714][  INFO][PID:17605] Checking that builder machine is OK
[2023-10-24 06:59:24,885][  INFO][PID:17605] Filling build.info file with builder info
[2023-10-24 06:59:24,887][  INFO][PID:17605] Marking build as running on frontend
[2023-10-24 06:59:24,920][  INFO][PID:17605] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --build-id 91933 --chroot openeuler-23.09-x86_64 --detached
[2023-10-24 06:59:25,372][  INFO][PID:17605] Downloading the builder-live.log file, attempt 1
[2023-10-24 07:49:27,292][WARNING][PID:17605] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2023-10-24 07:49:37,299][  INFO][PID:17605] Downloading the builder-live.log file, attempt 2
[2023-10-24 07:49:50,378][WARNING][PID:17605] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2023-10-24 07:50:00,389][  INFO][PID:17605] Downloading the builder-live.log file, attempt 3
[2023-10-24 07:50:13,483][WARNING][PID:17605] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2023-10-24 07:50:23,484][  INFO][PID:17605] Downloading the builder-live.log file, attempt 4
[2023-10-24 07:50:36,590][ ERROR][PID:17605] Giving up for unstable SSH, failures: 1698133767.2924414, 1698133790.3789232, 1698133813.4838874, 1698133836.5906231
[2023-10-24 07:50:36,591][ ERROR][PID:17605] Re-try request for task on 'ResallocHost, ticket_id=204779, hostname=10.0.1.14': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2023-10-24 07:50:36,592][  INFO][PID:17605] Releasing VM back to pool
[2023-10-24 07:50:36,604][  INFO][PID:17605] Retry #1 (on other host)
[2023-10-24 07:50:36,606][  INFO][PID:17605] Trying to allocate VM
[2023-10-24 07:50:39,638][  INFO][PID:17605] Allocated host ResallocHost, ticket_id=204794, hostname=10.0.1.115
[2023-10-24 07:50:39,639][  INFO][PID:17605] Allocating ssh connection to builder
[2023-10-24 07:50:39,639][  INFO][PID:17605] Checking that builder machine is OK
[2023-10-24 07:50:39,811][  INFO][PID:17605] Filling build.info file with builder info
[2023-10-24 07:50:39,813][  INFO][PID:17605] Not re-notifying FE and msg buses for the new host.
[2023-10-24 07:50:39,814][  INFO][PID:17605] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --build-id 91933 --chroot openeuler-23.09-x86_64 --detached
[2023-10-24 07:50:40,291][  INFO][PID:17605] Downloading the builder-live.log file, attempt 1
[2023-10-24 08:37:39,425][WARNING][PID:17605] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2023-10-24 08:37:49,434][  INFO][PID:17605] Downloading the builder-live.log file, attempt 2
[2023-10-24 08:38:02,541][WARNING][PID:17605] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2023-10-24 08:38:12,544][  INFO][PID:17605] Downloading the builder-live.log file, attempt 3
[2023-10-24 08:38:25,639][WARNING][PID:17605] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2023-10-24 08:38:35,650][  INFO][PID:17605] Downloading the builder-live.log file, attempt 4
[2023-10-24 08:38:48,744][ ERROR][PID:17605] Giving up for unstable SSH, failures: 1698136659.42571, 1698136682.5418158, 1698136705.6399355, 1698136728.744151
[2023-10-24 08:38:48,745][ ERROR][PID:17605] Re-try request for task on 'ResallocHost, ticket_id=204794, hostname=10.0.1.115': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2023-10-24 08:38:48,745][  INFO][PID:17605] Releasing VM back to pool
[2023-10-24 08:38:48,759][  INFO][PID:17605] Retry #2 (on other host)
[2023-10-24 08:38:48,760][  INFO][PID:17605] Trying to allocate VM
[2023-10-24 08:38:51,792][  INFO][PID:17605] Allocated host ResallocHost, ticket_id=204795, hostname=10.0.1.7
[2023-10-24 08:38:51,793][  INFO][PID:17605] Allocating ssh connection to builder
[2023-10-24 08:38:51,794][  INFO][PID:17605] Checking that builder machine is OK
[2023-10-24 08:38:51,978][  INFO][PID:17605] Filling build.info file with builder info
[2023-10-24 08:38:51,980][  INFO][PID:17605] Not re-notifying FE and msg buses for the new host.
[2023-10-24 08:38:51,980][  INFO][PID:17605] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --build-id 91933 --chroot openeuler-23.09-x86_64 --detached
[2023-10-24 08:38:52,514][  INFO][PID:17605] Downloading the builder-live.log file, attempt 1
[2023-10-24 09:23:22,786][WARNING][PID:17605] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2023-10-24 09:23:32,788][  INFO][PID:17605] Downloading the builder-live.log file, attempt 2
[2023-10-24 09:24:01,326][WARNING][PID:17605] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2023-10-24 09:24:11,330][  INFO][PID:17605] Downloading the builder-live.log file, attempt 3
[2023-10-24 09:24:24,430][WARNING][PID:17605] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2023-10-24 09:24:34,441][  INFO][PID:17605] Downloading the builder-live.log file, attempt 4
[2023-10-24 09:24:47,533][ ERROR][PID:17605] Giving up for unstable SSH, failures: 1698139402.7868762, 1698139441.3266451, 1698139464.4306602, 1698139487.533809
[2023-10-24 09:24:47,534][ ERROR][PID:17605] Re-try request for task on 'ResallocHost, ticket_id=204795, hostname=10.0.1.7': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2023-10-24 09:24:47,535][  INFO][PID:17605] Releasing VM back to pool
[2023-10-24 09:24:47,548][ ERROR][PID:17605] Backend process error: Three host tried without success: {'10.0.1.14', '10.0.1.7', '10.0.1.115'}
[2023-10-24 09:24:47,549][WARNING][PID:17605] Switching not-finished job state to 'failed'
[2023-10-24 09:24:47,549][  INFO][PID:17605] Worker failed build, took 2755.5690422058105
[2023-10-24 09:24:47,571][  INFO][PID:17605] Compressing /var/lib/copr/public_html/results/bigclouds/busybox/openeuler-23.09-x86_64/00091933-pytorch/builder-live.log by gzip
[2023-10-24 09:24:47,572][  INFO][PID:17605] Running command 'gzip /var/lib/copr/public_html/results/bigclouds/busybox/openeuler-23.09-x86_64/00091933-pytorch/builder-live.log' as PID 17925
[2023-10-24 09:24:47,573][  INFO][PID:17605] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/bigclouds/busybox/openeuler-23.09-x86_64/00091933-pytorch/builder-live.log)
[2023-10-24 09:24:47,573][  INFO][PID:17605] Compressing /var/lib/copr/public_html/results/bigclouds/busybox/openeuler-23.09-x86_64/00091933-pytorch/backend.log by gzip
[2023-10-24 09:24:47,574][  INFO][PID:17605] Running command 'gzip /var/lib/copr/public_html/results/bigclouds/busybox/openeuler-23.09-x86_64/00091933-pytorch/backend.log' as PID 17926