Harvest for wikipedia NL Created 28 May 16:17

Harvesting Log

(735 lines)

# Logfile created on 2020-05-28 15:34:58 -0400 by logger.rb/v1.4.2
[INFO] [2020-05-28 15:34:58] ## HARVEST: type = re_download_opendata_-harvest
[INFO] [2020-05-28 15:35:01] ## remove_type: ScientificName
[INFO] [2020-05-28 15:35:02] ++ Batch removal of 950717 instances...
[INFO] [2020-05-28 15:35:02] [15:35:02.177] Batch 0...
[INFO] [2020-05-28 15:35:04] [15:35:04.042] Batch 1...
[INFO] [2020-05-28 15:35:06] [15:35:06.051] Batch 2...
[INFO] [2020-05-28 15:35:07] [15:35:07.813] Batch 3...
[INFO] [2020-05-28 15:35:11] [15:35:11.260] Batch 4...
[INFO] [2020-05-28 15:35:15] [15:35:15.140] Batch 5...
[INFO] [2020-05-28 15:35:18] [15:35:18.957] Batch 6...
[INFO] [2020-05-28 15:35:22] [15:35:22.566] Batch 7...
[INFO] [2020-05-28 15:35:27] [15:35:27.440] Batch 8...
[INFO] [2020-05-28 15:35:31] [15:35:31.163] Batch 9...
[INFO] [2020-05-28 15:35:34] [15:35:34.176] Batch 10...
[INFO] [2020-05-28 15:35:36] [15:35:36.894] Batch 11...
[INFO] [2020-05-28 15:35:39] [15:35:39.510] Batch 12...
[INFO] [2020-05-28 15:35:42] [15:35:42.136] Batch 13...
[INFO] [2020-05-28 15:35:45] [15:35:45.003] Batch 14...
[INFO] [2020-05-28 15:35:47] [15:35:47.094] Batch 15...
[INFO] [2020-05-28 15:35:50] [15:35:50.090] Batch 16...
[INFO] [2020-05-28 15:35:52] [15:35:52.541] Batch 17...
[INFO] [2020-05-28 15:35:54] [15:35:54.916] Batch 18...
[INFO] [2020-05-28 15:35:56] [15:35:56.723] Batch 19...
[INFO] [2020-05-28 15:35:59] [15:35:59.533] Batch 20...
[INFO] [2020-05-28 15:36:02] [15:36:02.431] Batch 21...
[INFO] [2020-05-28 15:36:04] [15:36:04.620] Batch 22...
[INFO] [2020-05-28 15:36:06] [15:36:06.783] Batch 23...
[INFO] [2020-05-28 15:36:09] [15:36:09.684] Batch 24...
[INFO] [2020-05-28 15:36:11] [15:36:11.950] Batch 25...
[INFO] [2020-05-28 15:36:15] [15:36:15.194] Batch 26...
[INFO] [2020-05-28 15:36:18] [15:36:18.222] Batch 27...
[INFO] [2020-05-28 15:36:21] [15:36:21.203] Batch 28...
[INFO] [2020-05-28 15:36:24] [15:36:24.440] Batch 29...
[INFO] [2020-05-28 15:36:26] [15:36:26.643] Batch 30...
[INFO] [2020-05-28 15:36:29] [15:36:29.888] Batch 31...
[INFO] [2020-05-28 15:36:32] [15:36:32.077] Batch 32...
[INFO] [2020-05-28 15:36:34] [15:36:34.962] Batch 33...
[INFO] [2020-05-28 15:36:37] [15:36:37.248] Batch 34...
[INFO] [2020-05-28 15:36:39] [15:36:39.488] Batch 35...
[INFO] [2020-05-28 15:36:42] [15:36:42.579] Batch 36...
[INFO] [2020-05-28 15:36:45] [15:36:45.083] Batch 37...
[INFO] [2020-05-28 15:36:47] [15:36:47.963] Batch 38...
[INFO] [2020-05-28 15:36:50] [15:36:50.724] Batch 39...
[INFO] [2020-05-28 15:36:54] [15:36:54.558] Batch 40...
[INFO] [2020-05-28 15:36:58] [15:36:58.358] Batch 41...
[INFO] [2020-05-28 15:37:01] [15:37:01.645] Batch 42...
[INFO] [2020-05-28 15:37:03] [15:37:03.987] Batch 43...
[INFO] [2020-05-28 15:37:07] [15:37:07.135] Batch 44...
[INFO] [2020-05-28 15:37:12] [15:37:12.274] Batch 45...
[INFO] [2020-05-28 15:37:18] [15:37:18.580] Batch 46...
[INFO] [2020-05-28 15:37:26] [15:37:26.274] Batch 47...
[INFO] [2020-05-28 15:37:32] [15:37:32.179] Batch 48...
[INFO] [2020-05-28 15:37:40] [15:37:40.005] Batch 49...
[INFO] [2020-05-28 15:37:45] [15:37:45.818] Batch 50...
[INFO] [2020-05-28 15:37:50] [15:37:50.430] Batch 51...
[INFO] [2020-05-28 15:37:55] [15:37:55.408] Batch 52...
[INFO] [2020-05-28 15:38:01] [15:38:01.134] Batch 53...
[INFO] [2020-05-28 15:38:04] [15:38:04.889] Batch 54...
[INFO] [2020-05-28 15:38:10] [15:38:10.522] Batch 55...
[INFO] [2020-05-28 15:38:15] [15:38:15.788] Batch 56...
[INFO] [2020-05-28 15:38:21] [15:38:21.119] Batch 57...
[INFO] [2020-05-28 15:38:26] [15:38:26.150] Batch 58...
[INFO] [2020-05-28 15:38:30] [15:38:30.740] Batch 59...
[INFO] [2020-05-28 15:38:35] [15:38:35.803] Batch 60...
[INFO] [2020-05-28 15:38:40] [15:38:40.736] Batch 61...
[INFO] [2020-05-28 15:38:44] [15:38:44.953] Batch 62...
[INFO] [2020-05-28 15:38:49] [15:38:49.097] Batch 63...
[INFO] [2020-05-28 15:38:54] [15:38:54.589] Batch 64...
[INFO] [2020-05-28 15:38:59] [15:38:59.288] Batch 65...
[INFO] [2020-05-28 15:39:04] [15:39:04.324] Batch 66...
[INFO] [2020-05-28 15:39:08] [15:39:08.781] Batch 67...
[INFO] [2020-05-28 15:39:13] [15:39:13.625] Batch 68...
[INFO] [2020-05-28 15:39:18] [15:39:18.663] Batch 69...
[INFO] [2020-05-28 15:39:23] [15:39:23.253] Batch 70...
[INFO] [2020-05-28 15:39:27] [15:39:27.785] Batch 71...
[INFO] [2020-05-28 15:39:32] [15:39:32.613] Batch 72...
[INFO] [2020-05-28 15:39:37] [15:39:37.469] Batch 73...
[INFO] [2020-05-28 15:39:42] [15:39:42.043] Batch 74...
[INFO] [2020-05-28 15:39:46] [15:39:46.879] Batch 75...
[INFO] [2020-05-28 15:39:52] [15:39:52.139] Batch 76...
[INFO] [2020-05-28 15:39:58] [15:39:58.022] Batch 77...
[INFO] [2020-05-28 15:40:03] [15:40:03.349] Batch 78...
[INFO] [2020-05-28 15:40:08] [15:40:08.430] Batch 79...
[INFO] [2020-05-28 15:40:13] [15:40:13.324] Batch 80...
[INFO] [2020-05-28 15:40:17] [15:40:17.515] Batch 81...
[INFO] [2020-05-28 15:40:22] [15:40:22.404] Batch 82...
[INFO] [2020-05-28 15:40:25] [15:40:25.900] Batch 83...
[INFO] [2020-05-28 15:40:30] [15:40:30.156] Batch 84...
[INFO] [2020-05-28 15:40:34] [15:40:34.366] Batch 85...
[INFO] [2020-05-28 15:40:38] [15:40:38.022] Batch 86...
[INFO] [2020-05-28 15:40:42] [15:40:42.702] Batch 87...
[INFO] [2020-05-28 15:40:46] [15:40:46.747] Batch 88...
[INFO] [2020-05-28 15:40:51] [15:40:51.175] Batch 89...
[INFO] [2020-05-28 15:40:55] [15:40:55.737] Batch 90...
[INFO] [2020-05-28 15:40:59] [15:40:59.932] Batch 91...
[INFO] [2020-05-28 15:41:02] [15:41:02.970] Batch 92...
[INFO] [2020-05-28 15:41:07] [15:41:07.343] Batch 93...
[INFO] [2020-05-28 15:41:12] [15:41:12.187] Batch 94...
[INFO] [2020-05-28 15:41:17] [15:41:17.220] Batch 95...
[INFO] [2020-05-28 15:41:18] [15:41:18.608] Removed 950717 Scientificnames
[INFO] [2020-05-28 15:41:18] ## remove_type: Vernacular
[INFO] [2020-05-28 15:41:18] ++ Calling delete_all on 0 instances...
[INFO] [2020-05-28 15:41:18] [15:41:18.611] Removed 0 Vernaculars
[INFO] [2020-05-28 15:41:18] ## remove_type: Article
[INFO] [2020-05-28 15:41:20] ++ Batch removal of 975151 instances...
[INFO] [2020-05-28 15:41:20] [15:41:20.566] Batch 0...
[INFO] [2020-05-28 15:41:25] [15:41:25.554] Batch 1...
[INFO] [2020-05-28 15:41:29] [15:41:29.713] Batch 2...
[INFO] [2020-05-28 15:41:32] [15:41:32.256] Batch 3...
[INFO] [2020-05-28 15:41:34] [15:41:34.003] Batch 4...
[INFO] [2020-05-28 15:41:35] [15:41:35.744] Batch 5...
[INFO] [2020-05-28 15:41:37] [15:41:37.743] Batch 6...
[INFO] [2020-05-28 15:41:39] [15:41:39.862] Batch 7...
[INFO] [2020-05-28 15:41:41] [15:41:41.350] Batch 8...
[INFO] [2020-05-28 15:41:43] [15:41:43.175] Batch 9...
[INFO] [2020-05-28 15:41:45] [15:41:45.516] Batch 10...
[INFO] [2020-05-28 15:41:47] [15:41:47.244] Batch 11...
[INFO] [2020-05-28 15:41:49] [15:41:49.049] Batch 12...
[INFO] [2020-05-28 15:41:51] [15:41:51.204] Batch 13...
[INFO] [2020-05-28 15:41:53] [15:41:53.132] Batch 14...
[INFO] [2020-05-28 15:41:54] [15:41:54.579] Batch 15...
[INFO] [2020-05-28 15:41:56] [15:41:56.376] Batch 16...
[INFO] [2020-05-28 15:41:58] [15:41:58.550] Batch 17...
[INFO] [2020-05-28 15:42:00] [15:42:00.560] Batch 18...
[INFO] [2020-05-28 15:42:02] [15:42:02.086] Batch 19...
[INFO] [2020-05-28 15:42:04] [15:42:04.298] Batch 20...
[INFO] [2020-05-28 15:42:06] [15:42:06.124] Batch 21...
[INFO] [2020-05-28 15:42:07] [15:42:07.816] Batch 22...
[INFO] [2020-05-28 15:42:09] [15:42:09.600] Batch 23...
[INFO] [2020-05-28 15:42:11] [15:42:11.745] Batch 24...
[INFO] [2020-05-28 15:42:13] [15:42:13.537] Batch 25...
[INFO] [2020-05-28 15:42:15] [15:42:15.066] Batch 26...
[INFO] [2020-05-28 15:42:16] [15:42:16.797] Batch 27...
[INFO] [2020-05-28 15:42:18] [15:42:18.893] Batch 28...
[INFO] [2020-05-28 15:42:20] [15:42:20.836] Batch 29...
[INFO] [2020-05-28 15:42:22] [15:42:22.277] Batch 30...
[INFO] [2020-05-28 15:42:24] [15:42:24.225] Batch 31...
[INFO] [2020-05-28 15:42:26] [15:42:26.273] Batch 32...
[INFO] [2020-05-28 15:42:28] [15:42:28.154] Batch 33...
[INFO] [2020-05-28 15:42:29] [15:42:29.574] Batch 34...
[INFO] [2020-05-28 15:42:31] [15:42:31.807] Batch 35...
[INFO] [2020-05-28 15:42:33] [15:42:33.695] Batch 36...
[INFO] [2020-05-28 15:42:35] [15:42:35.203] Batch 37...
[INFO] [2020-05-28 15:42:36] [15:42:36.738] Batch 38...
[INFO] [2020-05-28 15:42:38] [15:42:38.873] Batch 39...
[INFO] [2020-05-28 15:42:40] [15:42:40.786] Batch 40...
[INFO] [2020-05-28 15:42:42] [15:42:42.287] Batch 41...
[INFO] [2020-05-28 15:42:44] [15:42:44.612] Batch 42...
[INFO] [2020-05-28 15:42:46] [15:42:46.279] Batch 43...
[INFO] [2020-05-28 15:42:48] [15:42:48.010] Batch 44...
[INFO] [2020-05-28 15:42:49] [15:42:49.800] Batch 45...
[INFO] [2020-05-28 15:42:51] [15:42:51.778] Batch 46...
[INFO] [2020-05-28 15:42:53] [15:42:53.214] Batch 47...
[INFO] [2020-05-28 15:42:55] [15:42:55.004] Batch 48...
[INFO] [2020-05-28 15:42:56] [15:42:56.898] Batch 49...
[INFO] [2020-05-28 15:42:59] [15:42:59.068] Batch 50...
[INFO] [2020-05-28 15:43:00] [15:43:00.600] Batch 51...
[INFO] [2020-05-28 15:43:02] [15:43:02.281] Batch 52...
[INFO] [2020-05-28 15:43:04] [15:43:04.474] Batch 53...
[INFO] [2020-05-28 15:43:06] [15:43:06.451] Batch 54...
[INFO] [2020-05-28 15:43:08] [15:43:08.070] Batch 55...
[INFO] [2020-05-28 15:43:09] [15:43:09.600] Batch 56...
[INFO] [2020-05-28 15:43:11] [15:43:11.613] Batch 57...
[INFO] [2020-05-28 15:43:13] [15:43:13.593] Batch 58...
[INFO] [2020-05-28 15:43:15] [15:43:15.244] Batch 59...
[INFO] [2020-05-28 15:43:16] [15:43:16.787] Batch 60...
[INFO] [2020-05-28 15:43:18] [15:43:18.797] Batch 61...
[INFO] [2020-05-28 15:43:20] [15:43:20.484] Batch 62...
[INFO] [2020-05-28 15:43:21] [15:43:21.895] Batch 63...
[INFO] [2020-05-28 15:43:23] [15:43:23.820] Batch 64...
[INFO] [2020-05-28 15:43:25] [15:43:25.703] Batch 65...
[INFO] [2020-05-28 15:43:27] [15:43:27.628] Batch 66...
[INFO] [2020-05-28 15:43:29] [15:43:29.026] Batch 67...
[INFO] [2020-05-28 15:43:31] [15:43:31.132] Batch 68...
[INFO] [2020-05-28 15:43:32] [15:43:32.883] Batch 69...
[INFO] [2020-05-28 15:43:34] [15:43:34.831] Batch 70...
[INFO] [2020-05-28 15:43:36] [15:43:36.317] Batch 71...
[INFO] [2020-05-28 15:43:38] [15:43:38.478] Batch 72...
[INFO] [2020-05-28 15:43:40] [15:43:40.182] Batch 73...
[INFO] [2020-05-28 15:43:41] [15:43:41.865] Batch 74...
[INFO] [2020-05-28 15:43:43] [15:43:43.767] Batch 75...
[INFO] [2020-05-28 15:43:45] [15:43:45.791] Batch 76...
[INFO] [2020-05-28 15:43:47] [15:43:47.500] Batch 77...
[INFO] [2020-05-28 15:43:48] [15:43:48.845] Batch 78...
[INFO] [2020-05-28 15:43:50] [15:43:50.834] Batch 79...
[INFO] [2020-05-28 15:43:52] [15:43:52.642] Batch 80...
[INFO] [2020-05-28 15:43:54] [15:43:54.509] Batch 81...
[INFO] [2020-05-28 15:43:55] [15:43:55.978] Batch 82...
[INFO] [2020-05-28 15:43:58] [15:43:58.059] Batch 83...
[INFO] [2020-05-28 15:43:59] [15:43:59.732] Batch 84...
[INFO] [2020-05-28 15:44:01] [15:44:01.272] Batch 85...
[INFO] [2020-05-28 15:44:03] [15:44:03.138] Batch 86...
[INFO] [2020-05-28 15:44:05] [15:44:05.130] Batch 87...
[INFO] [2020-05-28 15:44:07] [15:44:07.002] Batch 88...
[INFO] [2020-05-28 15:44:08] [15:44:08.423] Batch 89...
[INFO] [2020-05-28 15:44:10] [15:44:10.541] Batch 90...
[INFO] [2020-05-28 15:44:12] [15:44:12.027] Batch 91...
[INFO] [2020-05-28 15:44:14] [15:44:14.070] Batch 92...
[INFO] [2020-05-28 15:44:15] [15:44:15.481] Batch 93...
[INFO] [2020-05-28 15:44:17] [15:44:17.529] Batch 94...
[INFO] [2020-05-28 15:44:19] [15:44:19.199] Batch 95...
[INFO] [2020-05-28 15:44:20] [15:44:20.715] Batch 96...
[INFO] [2020-05-28 15:44:22] [15:44:22.385] Batch 97...
[INFO] [2020-05-28 15:44:24] [15:44:24.216] Removed 975151 Articles
[INFO] [2020-05-28 15:44:24] ## remove_type: Medium
[INFO] [2020-05-28 15:44:24] ++ Calling delete_all on 0 instances...
[INFO] [2020-05-28 15:44:24] [15:44:24.220] Removed 0 Media
[INFO] [2020-05-28 15:44:24] ## remove_type: Trait
[INFO] [2020-05-28 15:44:24] ++ Calling delete_all on 0 instances...
[INFO] [2020-05-28 15:44:24] [15:44:24.223] Removed 0 Traits
[INFO] [2020-05-28 15:44:24] ## remove_type: MetaTrait
[INFO] [2020-05-28 15:44:24] ++ Calling delete_all on 0 instances...
[INFO] [2020-05-28 15:44:24] [15:44:24.274] Removed 0 Metatraits
[INFO] [2020-05-28 15:44:24] ## remove_type: OccurrenceMetadatum
[INFO] [2020-05-28 15:44:24] ++ Calling delete_all on 0 instances...
[INFO] [2020-05-28 15:44:24] [15:44:24.300] Removed 0 Occurrencemetadata
[INFO] [2020-05-28 15:44:24] ## remove_type: Assoc
[INFO] [2020-05-28 15:44:24] ++ Calling delete_all on 0 instances...
[INFO] [2020-05-28 15:44:24] [15:44:24.303] Removed 0 Assocs
[INFO] [2020-05-28 15:44:24] ## remove_type: MetaAssoc
[INFO] [2020-05-28 15:44:24] ++ Calling delete_all on 0 instances...
[INFO] [2020-05-28 15:44:24] [15:44:24.306] Removed 0 Metaassocs
[INFO] [2020-05-28 15:44:24] ## remove_type: Identifier
[INFO] [2020-05-28 15:44:24] ++ Calling delete_all on 0 instances...
[INFO] [2020-05-28 15:44:24] [15:44:24.308] Removed 0 Identifiers
[INFO] [2020-05-28 15:44:24] ## remove_type: Reference
[INFO] [2020-05-28 15:44:24] ++ Calling delete_all on 0 instances...
[INFO] [2020-05-28 15:44:24] [15:44:24.311] Removed 0 References
[INFO] [2020-05-28 15:44:26] Starting batch with ID 38226891...
[INFO] [2020-05-28 15:44:27] Starting batch with ID 38226891...
[INFO] [2020-05-28 15:44:29] Starting batch with ID 38323847...
[INFO] [2020-05-28 15:44:30] Starting batch with ID 38246796...
[INFO] [2020-05-28 15:44:32] Starting batch with ID 38280778...
[INFO] [2020-05-28 15:44:33] Starting batch with ID 38220137...
[INFO] [2020-05-28 15:44:35] Starting batch with ID 38285795...
[INFO] [2020-05-28 15:44:36] Starting batch with ID 38238931...
[INFO] [2020-05-28 15:44:38] Starting batch with ID 38340755...
[INFO] [2020-05-28 15:44:39] Starting batch with ID 38315468...
[INFO] [2020-05-28 15:44:41] Starting batch with ID 38400721...
[INFO] [2020-05-28 15:44:42] Starting batch with ID 38426009...
[INFO] [2020-05-28 15:44:44] Starting batch with ID 38494813...
[INFO] [2020-05-28 15:44:45] Starting batch with ID 38366899...
[INFO] [2020-05-28 15:44:47] Starting batch with ID 38435400...
[INFO] [2020-05-28 15:44:48] Starting batch with ID 38356220...
[INFO] [2020-05-28 15:44:50] Starting batch with ID 38396368...
[INFO] [2020-05-28 15:44:52] Starting batch with ID 38469592...
[INFO] [2020-05-28 15:44:54] Starting batch with ID 38404508...
[INFO] [2020-05-28 15:44:55] Starting batch with ID 38428960...
[INFO] [2020-05-28 15:44:57] Starting batch with ID 38497124...
[INFO] [2020-05-28 15:44:59] Starting batch with ID 38510563...
[INFO] [2020-05-28 15:45:01] Starting batch with ID 38585890...
[INFO] [2020-05-28 15:45:03] Starting batch with ID 38587300...
[INFO] [2020-05-28 15:45:04] Starting batch with ID 38534087...
[INFO] [2020-05-28 15:45:06] Starting batch with ID 38607564...
[INFO] [2020-05-28 15:45:08] Starting batch with ID 38504708...
[INFO] [2020-05-28 15:45:10] Starting batch with ID 38558067...
[INFO] [2020-05-28 15:45:12] Starting batch with ID 38590467...
[INFO] [2020-05-28 15:45:14] Starting batch with ID 38815502...
[INFO] [2020-05-28 15:45:17] Starting batch with ID 38776540...
[INFO] [2020-05-28 15:45:19] Starting batch with ID 38776540...
[INFO] [2020-05-28 15:45:21] Starting batch with ID 38701209...
[INFO] [2020-05-28 15:45:24] Starting batch with ID 38796424...
[INFO] [2020-05-28 15:45:26] Starting batch with ID 38921660...
[INFO] [2020-05-28 15:45:29] Starting batch with ID 38920871...
[INFO] [2020-05-28 15:45:31] Starting batch with ID 38689913...
[INFO] [2020-05-28 15:45:34] Starting batch with ID 38887427...
[INFO] [2020-05-28 15:45:36] Starting batch with ID 38789492...
[INFO] [2020-05-28 15:45:39] Starting batch with ID 38727135...
[INFO] [2020-05-28 15:45:43] Starting batch with ID 38733135...
[INFO] [2020-05-28 15:45:45] Starting batch with ID 38733148...
[INFO] [2020-05-28 15:45:47] Starting batch with ID 38714939...
[INFO] [2020-05-28 15:45:50] Starting batch with ID 38756901...
[INFO] [2020-05-28 15:45:52] Starting batch with ID 38747337...
[INFO] [2020-05-28 15:45:54] Starting batch with ID 38806027...
[INFO] [2020-05-28 15:45:57] Starting batch with ID 38990081...
[INFO] [2020-05-28 15:45:59] Starting batch with ID 39054137...
[INFO] [2020-05-28 15:46:02] Starting batch with ID 39068811...
[INFO] [2020-05-28 15:46:04] Starting batch with ID 39067268...
[INFO] [2020-05-28 15:46:06] Starting batch with ID 39084869...
[INFO] [2020-05-28 15:46:08] Starting batch with ID 39133468...
[INFO] [2020-05-28 15:46:11] Starting batch with ID 39106698...
[INFO] [2020-05-28 15:46:13] Starting batch with ID 38968124...
[INFO] [2020-05-28 15:46:15] Starting batch with ID 39100486...
[INFO] [2020-05-28 15:46:18] Starting batch with ID 39014047...
[INFO] [2020-05-28 15:46:20] Starting batch with ID 38973382...
[INFO] [2020-05-28 15:46:22] Starting batch with ID 39048645...
[INFO] [2020-05-28 15:46:24] Starting batch with ID 38965509...
[INFO] [2020-05-28 15:46:26] Starting batch with ID 39097441...
[INFO] [2020-05-28 15:46:28] Starting batch with ID 38631126...
[INFO] [2020-05-28 15:46:31] Starting batch with ID 38650546...
[INFO] [2020-05-28 15:46:33] Starting batch with ID 38616930...
[INFO] [2020-05-28 15:46:35] Starting batch with ID 38653163...
[INFO] [2020-05-28 15:46:37] Starting batch with ID 38667519...
[INFO] [2020-05-28 15:46:39] Starting batch with ID 38271032...
[INFO] [2020-05-28 15:46:42] Starting batch with ID 38204002...
[INFO] [2020-05-28 15:46:44] Starting batch with ID 38291932...
[INFO] [2020-05-28 15:46:47] Starting batch with ID 38256640...
[INFO] [2020-05-28 15:46:50] Starting batch with ID 38340026...
[INFO] [2020-05-28 15:46:53] Starting batch with ID 38285594...
[INFO] [2020-05-28 15:46:56] Starting batch with ID 38325244...
[INFO] [2020-05-28 15:47:00] Starting batch with ID 38277129...
[INFO] [2020-05-28 15:47:03] Starting batch with ID 38386555...
[INFO] [2020-05-28 15:47:08] Starting batch with ID 38501512...
[INFO] [2020-05-28 15:47:12] Starting batch with ID 38422932...
[INFO] [2020-05-28 15:47:17] Starting batch with ID 38489660...
[INFO] [2020-05-28 15:47:24] Starting batch with ID 38437853...
[INFO] [2020-05-28 15:47:33] Starting batch with ID 38406886...
[INFO] [2020-05-28 15:47:45] Starting batch with ID 38508371...
[INFO] [2020-05-28 15:48:01] Starting batch with ID 38592730...
[INFO] [2020-05-28 15:48:05] Starting batch with ID 38550142...
[INFO] [2020-05-28 15:48:11] Starting batch with ID 38603111...
[INFO] [2020-05-28 15:48:16] Starting batch with ID 38578270...
[INFO] [2020-05-28 15:48:21] Starting batch with ID 38646334...
[INFO] [2020-05-28 15:48:28] Starting batch with ID 38634000...
[INFO] [2020-05-28 15:48:33] Starting batch with ID 38685206...
[INFO] [2020-05-28 15:48:39] Starting batch with ID 38707400...
[INFO] [2020-05-28 15:48:46] Starting batch with ID 38813198...
[INFO] [2020-05-28 15:48:52] Starting batch with ID 38999538...
[INFO] [2020-05-28 15:48:59] Starting batch with ID 39147383...
[INFO] [2020-05-28 15:49:08] Starting batch with ID 38917387...
[INFO] [2020-05-28 15:49:18] Starting batch with ID 38873886...
[INFO] [2020-05-28 15:49:24] Starting batch with ID 38889569...
[INFO] [2020-05-28 15:49:31] Starting batch with ID 38834480...
[INFO] [2020-05-28 15:49:36] Starting batch with ID 38945421...
[INFO] [2020-05-28 15:49:42] Starting batch with ID 38993377...
[INFO] [2020-05-28 15:49:47] Starting batch with ID 38958729...
[INFO] [2020-05-28 15:49:52] Starting batch with ID 38938194...
[INFO] [2020-05-28 15:49:58] Starting batch with ID 38904129...
[INFO] [2020-05-28 15:50:04] Starting batch with ID 39124339...
[INFO] [2020-05-28 15:50:13] Starting batch with ID 39080103...
[INFO] [2020-05-28 15:50:20] Starting batch with ID 39111348...
[INFO] [2020-05-28 15:50:30] Starting batch with ID 39006452...
[INFO] [2020-05-28 15:50:37] Starting batch with ID 39047400...
[INFO] [2020-05-28 15:50:44] Starting batch with ID 38734822...
[INFO] [2020-05-28 15:50:50] Starting batch with ID 38754891...
[INFO] [2020-05-28 15:50:55] Starting batch with ID 38759970...
[INFO] [2020-05-28 15:51:02] Starting batch with ID 38598270...
[INFO] [2020-05-28 15:51:11] Starting batch with ID 38820644...
[INFO] [2020-05-28 15:51:41] Failed! ...Sleeping for a moment...
[INFO] [2020-05-28 15:52:41] Re-trying...
[INFO] [2020-05-28 15:53:06] Starting batch with ID 38839577...
[INFO] [2020-05-28 15:53:08] Starting batch with ID 38726920...
[INFO] [2020-05-28 15:53:09] Starting batch with ID 38829611...
[INFO] [2020-05-28 15:53:10] Starting batch with ID 38761441...
[INFO] [2020-05-28 15:53:11] Starting batch with ID 38761441...
[INFO] [2020-05-28 15:53:12] Starting batch with ID 38638440...
[INFO] [2020-05-28 15:53:13] Starting batch with ID 38638440...
[INFO] [2020-05-28 15:53:14] Starting batch with ID 38590361...
[INFO] [2020-05-28 15:53:15] Starting batch with ID 38590361...
[INFO] [2020-05-28 15:53:17] Starting batch with ID 38615752...
[INFO] [2020-05-28 15:53:18] Starting batch with ID 38615752...
[INFO] [2020-05-28 15:53:18] Starting batch with ID 38795346...
[INFO] [2020-05-28 15:53:20] Starting batch with ID 38795346...
[INFO] [2020-05-28 15:53:21] Starting batch with ID 38787871...
[INFO] [2020-05-28 15:53:22] Starting batch with ID 38787871...
[INFO] [2020-05-28 15:53:23] Starting batch with ID 38776030...
[INFO] [2020-05-28 15:53:24] Starting batch with ID 38776030...
[INFO] [2020-05-28 15:53:25] Starting batch with ID 39077084...
[INFO] [2020-05-28 15:53:26] Starting batch with ID 38917925...
[INFO] [2020-05-28 15:53:27] Starting batch with ID 38997144...
[INFO] [2020-05-28 15:53:29] Starting batch with ID 39097524...
[INFO] [2020-05-28 15:53:31] Starting batch with ID 38965358...
[INFO] [2020-05-28 15:53:32] Starting batch with ID 39084898...
[INFO] [2020-05-28 15:53:34] Starting batch with ID 38979797...
[INFO] [2020-05-28 15:53:35] Starting batch with ID 39069033...
[INFO] [2020-05-28 15:53:36] Starting batch with ID 38942518...
[INFO] [2020-05-28 15:53:37] Starting batch with ID 39028125...
[INFO] [2020-05-28 15:53:39] Starting batch with ID 38896067...
[INFO] [2020-05-28 15:53:42] Starting batch with ID 38936372...
[INFO] [2020-05-28 15:53:43] Starting batch with ID 39048848...
[INFO] [2020-05-28 15:53:45] Starting batch with ID 38910790...
[INFO] [2020-05-28 15:53:46] Starting batch with ID 39041313...
[INFO] [2020-05-28 15:53:48] Starting batch with ID 38540176...
[INFO] [2020-05-28 15:53:49] Starting batch with ID 38560692...
[INFO] [2020-05-28 15:53:51] Starting batch with ID 38553381...
[INFO] [2020-05-28 15:53:53] Starting batch with ID 38579693...
[INFO] [2020-05-28 15:53:54] Starting batch with ID 39145159...
[INFO] [2020-05-28 15:53:56] Starting batch with ID 39117130...
[INFO] [2020-05-28 15:53:58] Starting batch with ID 39117130...
[INFO] [2020-05-28 15:53:59] Starting batch with ID 39014720...
[INFO] [2020-05-28 15:54:01] Starting batch with ID 38230895...
[INFO] [2020-05-28 15:54:02] Starting batch with ID 38298720...
[INFO] [2020-05-28 15:54:04] Starting batch with ID 38296744...
[INFO] [2020-05-28 15:54:05] Starting batch with ID 38269669...
[INFO] [2020-05-28 15:54:07] Starting batch with ID 38266780...
[INFO] [2020-05-28 15:54:09] Starting batch with ID 38314515...
[INFO] [2020-05-28 15:54:11] Starting batch with ID 38224420...
[INFO] [2020-05-28 15:54:12] Starting batch with ID 38256677...
[INFO] [2020-05-28 15:54:14] Starting batch with ID 38310856...
[INFO] [2020-05-28 15:54:15] Starting batch with ID 38329687...
[INFO] [2020-05-28 15:54:17] Starting batch with ID 38364421...
[INFO] [2020-05-28 15:54:19] Starting batch with ID 38391316...
[INFO] [2020-05-28 15:54:20] Starting batch with ID 38438156...
[INFO] [2020-05-28 15:54:22] Starting batch with ID 38405763...
[INFO] [2020-05-28 15:54:24] Starting batch with ID 38409186...
[INFO] [2020-05-28 15:54:25] Starting batch with ID 38338607...
[INFO] [2020-05-28 15:54:27] Starting batch with ID 38398730...
[INFO] [2020-05-28 15:54:28] Starting batch with ID 38452734...
[INFO] [2020-05-28 15:54:30] Starting batch with ID 38468751...
[INFO] [2020-05-28 15:54:31] Starting batch with ID 38524727...
[INFO] [2020-05-28 15:54:33] Starting batch with ID 38521936...
[INFO] [2020-05-28 15:54:34] Starting batch with ID 38472881...
[INFO] [2020-05-28 15:54:36] Starting batch with ID 38473401...
[INFO] [2020-05-28 15:54:38] Starting batch with ID 38526235...
[INFO] [2020-05-28 15:54:39] Starting batch with ID 38216143...
[INFO] [2020-05-28 15:54:41] Starting batch with ID 38233428...
[INFO] [2020-05-28 15:54:43] Starting batch with ID 38612502...
[INFO] [2020-05-28 15:54:44] Starting batch with ID 38612502...
[INFO] [2020-05-28 15:54:46] Starting batch with ID 38608762...
[INFO] [2020-05-28 15:54:48] Starting batch with ID 38610374...
[INFO] [2020-05-28 15:54:49] Starting batch with ID 38596053...
[INFO] [2020-05-28 15:54:51] Starting batch with ID 38600412...
[INFO] [2020-05-28 15:54:52] Starting batch with ID 38752633...
[INFO] [2020-05-28 15:54:55] Starting batch with ID 38548529...
[INFO] [2020-05-28 15:54:56] Starting batch with ID 38721232...
[INFO] [2020-05-28 15:54:58] Starting batch with ID 38570448...
[INFO] [2020-05-28 15:54:59] Starting batch with ID 38730994...
[INFO] [2020-05-28 15:55:02] Starting batch with ID 38662160...
[INFO] [2020-05-28 15:55:03] Starting batch with ID 38663332...
[INFO] [2020-05-28 15:55:05] Starting batch with ID 38591087...
[INFO] [2020-05-28 15:55:07] Starting batch with ID 38640533...
[INFO] [2020-05-28 15:55:09] Starting batch with ID 38547012...
[INFO] [2020-05-28 15:55:11] Starting batch with ID 38272387...
[INFO] [2020-05-28 15:55:13] Starting batch with ID 38272387...
[INFO] [2020-05-28 15:55:14] Starting batch with ID 38807102...
[INFO] [2020-05-28 15:55:16] Starting batch with ID 38986849...
[INFO] [2020-05-28 15:55:19] Starting batch with ID 38847209...
[INFO] [2020-05-28 15:55:21] Starting batch with ID 38902119...
[INFO] [2020-05-28 15:55:23] Starting batch with ID 38973128...
[INFO] [2020-05-28 15:55:25] Starting batch with ID 38955359...
[INFO] [2020-05-28 15:55:28] Starting batch with ID 38840157...
[INFO] [2020-05-28 15:55:30] Starting batch with ID 38897624...
[INFO] [2020-05-28 15:55:32] Starting batch with ID 38781657...
[INFO] [2020-05-28 15:55:35] Starting batch with ID 38882271...
[INFO] [2020-05-28 15:55:38] Starting batch with ID 38938509...
[INFO] [2020-05-28 15:55:41] Starting batch with ID 38875492...
[INFO] [2020-05-28 15:55:44] Starting batch with ID 38929353...
[INFO] [2020-05-28 15:55:46] Starting batch with ID 38867661...
[INFO] [2020-05-28 15:55:50] Starting batch with ID 38267572...
[INFO] [2020-05-28 15:55:53] Starting batch with ID 38531694...
[INFO] [2020-05-28 15:55:56] Starting batch with ID 38456980...
[INFO] [2020-05-28 15:55:59] Starting batch with ID 38320286...
[INFO] [2020-05-28 15:56:01] Starting batch with ID 38414683...
[INFO] [2020-05-28 15:56:04] Starting batch with ID 38282055...
[INFO] [2020-05-28 15:56:07] Starting batch with ID 38328239...
[INFO] [2020-05-28 15:56:10] Starting batch with ID 38487433...
[INFO] [2020-05-28 15:56:13] Starting batch with ID 38366179...
[INFO] [2020-05-28 15:56:15] Starting batch with ID 38468502...
[INFO] [2020-05-28 15:56:18] Starting batch with ID 38290021...
[INFO] [2020-05-28 15:56:20] Starting batch with ID 38517709...
[INFO] [2020-05-28 15:56:23] Starting batch with ID 38387247...
[INFO] [2020-05-28 15:56:25] Starting batch with ID 38307404...
[INFO] [2020-05-28 15:56:29] Starting batch with ID 38431774...
[INFO] [2020-05-28 15:56:31] Starting batch with ID 39065561...
[INFO] [2020-05-28 15:56:34] Starting batch with ID 39036799...
[INFO] [2020-05-28 15:56:36] Starting batch with ID 39019661...
[INFO] [2020-05-28 15:56:39] Starting batch with ID 39007889...
[INFO] [2020-05-28 15:56:41] Starting batch with ID 39079146...
[INFO] [2020-05-28 15:56:44] Starting batch with ID 39003065...
[INFO] [2020-05-28 15:56:46] Starting batch with ID 39090081...
[INFO] [2020-05-28 15:56:48] Starting batch with ID 39103902...
[INFO] [2020-05-28 15:56:51] Starting batch with ID 39059547...
[INFO] [2020-05-28 15:56:54] Starting batch with ID 39117384...
[INFO] [2020-05-28 15:56:57] Starting batch with ID 38578155...
[INFO] [2020-05-28 15:57:00] Starting batch with ID 38663237...
[INFO] [2020-05-28 15:57:03] Starting batch with ID 38568797...
[INFO] [2020-05-28 15:57:07] Starting batch with ID 38648862...
[INFO] [2020-05-28 15:57:10] Starting batch with ID 38595934...
[INFO] [2020-05-28 15:57:13] Starting batch with ID 38716356...
[INFO] [2020-05-28 15:57:17] Starting batch with ID 38723409...
[INFO] [2020-05-28 15:57:20] Starting batch with ID 38560224...
[INFO] [2020-05-28 15:57:25] Starting batch with ID 38529427...
[INFO] [2020-05-28 15:57:30] Starting batch with ID 38713750...
[INFO] [2020-05-28 15:57:35] Starting batch with ID 38680882...
[INFO] [2020-05-28 15:57:42] Starting batch with ID 38689357...
[INFO] [2020-05-28 15:57:49] Starting batch with ID 38903782...
[INFO] [2020-05-28 15:57:52] Starting batch with ID 38784131...
[INFO] [2020-05-28 15:57:54] Starting batch with ID 38960391...
[INFO] [2020-05-28 15:57:55] Starting batch with ID 38785688...
[INFO] [2020-05-28 15:57:56] Starting batch with ID 38826418...
[INFO] [2020-05-28 15:57:57] Starting batch with ID 38826418...
[INFO] [2020-05-28 15:57:58] Starting batch with ID 38808311...
[INFO] [2020-05-28 15:57:59] Starting batch with ID 38808311...
[INFO] [2020-05-28 15:58:00] Starting batch with ID 38969700...
[INFO] [2020-05-28 15:58:01] Starting batch with ID 38791395...
[INFO] [2020-05-28 15:58:02] Starting batch with ID 38938839...
[INFO] [2020-05-28 15:58:03] Starting batch with ID 38856102...
[INFO] [2020-05-28 15:58:04] Starting batch with ID 38768403...
[INFO] [2020-05-28 15:58:05] Starting batch with ID 38926729...
[INFO] [2020-05-28 15:58:07] Starting batch with ID 39115048...
[INFO] [2020-05-28 15:58:08] Starting batch with ID 39018340...
[INFO] [2020-05-28 15:58:09] Starting batch with ID 39052323...
[INFO] [2020-05-28 15:58:10] Starting batch with ID 39095102...
[INFO] [2020-05-28 15:58:11] Starting batch with ID 38998782...
[INFO] [2020-05-28 15:58:12] Starting batch with ID 38989250...
[INFO] [2020-05-28 15:58:13] Starting batch with ID 39123352...
[INFO] [2020-05-28 15:58:14] Starting batch with ID 39109156...
[INFO] [2020-05-28 15:58:15] Starting batch with ID 39080688...
[INFO] [2020-05-28 15:58:17] Starting batch with ID 38223462...
[INFO] [2020-05-28 15:58:18] Starting batch with ID 38240272...
[INFO] [2020-05-28 15:58:19] Starting batch with ID 38267422...
[INFO] [2020-05-28 15:58:20] Starting batch with ID 38203163...
[INFO] [2020-05-28 15:58:21] Starting batch with ID 38280933...
[INFO] [2020-05-28 15:58:22] Starting batch with ID 38414664...
[INFO] [2020-05-28 15:58:23] Starting batch with ID 38426624...
[INFO] [2020-05-28 15:58:24] Starting batch with ID 38428145...
[INFO] [2020-05-28 15:58:25] Starting batch with ID 38450430...
[INFO] [2020-05-28 15:58:27] Starting batch with ID 38364209...
[INFO] [2020-05-28 15:58:28] Starting batch with ID 38384227...
[INFO] [2020-05-28 15:58:29] Starting batch with ID 38399100...
[INFO] [2020-05-28 15:58:30] Starting batch with ID 38462374...
[INFO] [2020-05-28 15:58:31] Starting batch with ID 38298960...
[INFO] [2020-05-28 15:58:32] Starting batch with ID 38309088...
[INFO] [2020-05-28 15:58:33] Starting batch with ID 38331658...
[INFO] [2020-05-28 15:58:34] Starting batch with ID 38289675...
[INFO] [2020-05-28 15:58:36] Starting batch with ID 38307494...
[INFO] [2020-05-28 15:58:37] Starting batch with ID 38348670...
[INFO] [2020-05-28 15:58:38] Starting batch with ID 38358532...
[INFO] [2020-05-28 15:58:39] Starting batch with ID 38467744...
[INFO] [2020-05-28 15:58:40] Starting batch with ID 38475940...
[INFO] [2020-05-28 15:58:41] Starting batch with ID 38485046...
[INFO] [2020-05-28 15:58:42] Starting batch with ID 38490791...
[INFO] [2020-05-28 15:58:42] ## remove_type: Node
[INFO] [2020-05-28 15:58:42] ++ Batch removal of 950717 instances...
[INFO] [2020-05-28 15:58:42] [15:58:42.518] Batch 0...
[INFO] [2020-05-28 15:58:44] [15:58:44.182] Batch 1...
[INFO] [2020-05-28 15:58:45] [15:58:45.638] Batch 2...
[INFO] [2020-05-28 15:58:46] [15:58:46.974] Batch 3...
[INFO] [2020-05-28 15:58:48] [15:58:48.927] Batch 4...
[INFO] [2020-05-28 15:58:50] [15:58:50.937] Batch 5...
[INFO] [2020-05-28 15:58:53] [15:58:53.106] Batch 6...
[INFO] [2020-05-28 15:58:56] [15:58:56.281] Batch 7...
[INFO] [2020-05-28 15:58:59] [15:58:59.885] Batch 8...
[INFO] [2020-05-28 15:59:01] [15:59:01.972] Batch 9...
[INFO] [2020-05-28 15:59:03] [15:59:03.996] Batch 10...
[INFO] [2020-05-28 15:59:05] [15:59:05.833] Batch 11...
[INFO] [2020-05-28 15:59:07] [15:59:07.203] Batch 12...
[INFO] [2020-05-28 15:59:09] [15:59:09.335] Batch 13...
[INFO] [2020-05-28 15:59:11] [15:59:11.207] Batch 14...
[INFO] [2020-05-28 15:59:12] [15:59:12.965] Batch 15...
[INFO] [2020-05-28 15:59:14] [15:59:14.459] Batch 16...
[INFO] [2020-05-28 15:59:16] [15:59:16.444] Batch 17...
[INFO] [2020-05-28 15:59:18] [15:59:18.284] Batch 18...
[INFO] [2020-05-28 15:59:19] [15:59:19.649] Batch 19...
[INFO] [2020-05-28 15:59:21] [15:59:21.452] Batch 20...
[INFO] [2020-05-28 15:59:22] [15:59:22.960] Batch 21...
[INFO] [2020-05-28 15:59:24] [15:59:24.674] Batch 22...
[INFO] [2020-05-28 15:59:26] [15:59:26.285] Batch 23...
[INFO] [2020-05-28 15:59:27] [15:59:27.635] Batch 24...
[INFO] [2020-05-28 15:59:29] [15:59:29.588] Batch 25...
[INFO] [2020-05-28 15:59:31] [15:59:31.349] Batch 26...
[INFO] [2020-05-28 15:59:33] [15:59:33.837] Batch 27...
[INFO] [2020-05-28 15:59:35] [15:59:35.185] Batch 28...
[INFO] [2020-05-28 15:59:37] [15:59:37.223] Batch 29...
[INFO] [2020-05-28 15:59:39] [15:59:39.070] Batch 30...
[INFO] [2020-05-28 15:59:40] [15:59:40.975] Batch 31...
[INFO] [2020-05-28 15:59:42] [15:59:42.540] Batch 32...
[INFO] [2020-05-28 15:59:44] [15:59:44.253] Batch 33...
[INFO] [2020-05-28 15:59:45] [15:59:45.732] Batch 34...
[INFO] [2020-05-28 15:59:47] [15:59:47.786] Batch 35...
[INFO] [2020-05-28 15:59:49] [15:59:49.785] Batch 36...
[INFO] [2020-05-28 15:59:51] [15:59:51.752] Batch 37...
[INFO] [2020-05-28 15:59:53] [15:59:53.610] Batch 38...
[INFO] [2020-05-28 15:59:55] [15:59:55.082] Batch 39...
[INFO] [2020-05-28 15:59:57] [15:59:57.132] Batch 40...
[INFO] [2020-05-28 15:59:59] [15:59:59.296] Batch 41...
[INFO] [2020-05-28 16:00:00] [16:00:00.901] Batch 42...
[INFO] [2020-05-28 16:00:02] [16:00:02.935] Batch 43...
[INFO] [2020-05-28 16:00:04] [16:00:04.686] Batch 44...
[INFO] [2020-05-28 16:00:07] [16:00:07.913] Batch 45...
[INFO] [2020-05-28 16:00:11] [16:00:11.199] Batch 46...
[INFO] [2020-05-28 16:00:14] [16:00:14.390] Batch 47...
[INFO] [2020-05-28 16:00:17] [16:00:17.031] Batch 48...
[INFO] [2020-05-28 16:00:19] [16:00:19.822] Batch 49...
[INFO] [2020-05-28 16:00:22] [16:00:22.296] Batch 50...
[INFO] [2020-05-28 16:00:24] [16:00:24.757] Batch 51...
[INFO] [2020-05-28 16:00:27] [16:00:27.503] Batch 52...
[INFO] [2020-05-28 16:00:29] [16:00:29.907] Batch 53...
[INFO] [2020-05-28 16:00:32] [16:00:32.506] Batch 54...
[INFO] [2020-05-28 16:00:35] [16:00:35.172] Batch 55...
[INFO] [2020-05-28 16:00:37] [16:00:37.911] Batch 56...
[INFO] [2020-05-28 16:00:40] [16:00:40.661] Batch 57...
[INFO] [2020-05-28 16:00:43] [16:00:43.499] Batch 58...
[INFO] [2020-05-28 16:00:45] [16:00:45.897] Batch 59...
[INFO] [2020-05-28 16:00:48] [16:00:48.193] Batch 60...
[INFO] [2020-05-28 16:00:50] [16:00:50.471] Batch 61...
[INFO] [2020-05-28 16:00:52] [16:00:52.738] Batch 62...
[INFO] [2020-05-28 16:00:54] [16:00:54.847] Batch 63...
[INFO] [2020-05-28 16:00:56] [16:00:56.994] Batch 64...
[INFO] [2020-05-28 16:00:59] [16:00:59.518] Batch 65...
[INFO] [2020-05-28 16:01:01] [16:01:01.871] Batch 66...
[INFO] [2020-05-28 16:01:04] [16:01:04.453] Batch 67...
[INFO] [2020-05-28 16:01:06] [16:01:06.503] Batch 68...
[INFO] [2020-05-28 16:01:08] [16:01:08.896] Batch 69...
[INFO] [2020-05-28 16:01:11] [16:01:11.684] Batch 70...
[INFO] [2020-05-28 16:01:13] [16:01:13.786] Batch 71...
[INFO] [2020-05-28 16:01:16] [16:01:16.126] Batch 72...
[INFO] [2020-05-28 16:01:18] [16:01:18.540] Batch 73...
[INFO] [2020-05-28 16:01:21] [16:01:21.125] Batch 74...
[INFO] [2020-05-28 16:01:23] [16:01:23.871] Batch 75...
[INFO] [2020-05-28 16:01:26] [16:01:26.499] Batch 76...
[INFO] [2020-05-28 16:01:29] [16:01:29.495] Batch 77...
[INFO] [2020-05-28 16:01:31] [16:01:31.965] Batch 78...
[INFO] [2020-05-28 16:01:34] [16:01:34.733] Batch 79...
[INFO] [2020-05-28 16:01:37] [16:01:37.074] Batch 80...
[INFO] [2020-05-28 16:01:39] [16:01:39.438] Batch 81...
[INFO] [2020-05-28 16:01:41] [16:01:41.752] Batch 82...
[INFO] [2020-05-28 16:01:43] [16:01:43.519] Batch 83...
[INFO] [2020-05-28 16:01:45] [16:01:45.725] Batch 84...
[INFO] [2020-05-28 16:01:47] [16:01:47.898] Batch 85...
[INFO] [2020-05-28 16:01:50] [16:01:50.121] Batch 86...
[INFO] [2020-05-28 16:01:52] [16:01:52.384] Batch 87...
[INFO] [2020-05-28 16:01:54] [16:01:54.704] Batch 88...
[INFO] [2020-05-28 16:01:57] [16:01:57.020] Batch 89...
[INFO] [2020-05-28 16:02:00] [16:02:00.250] Batch 90...
[INFO] [2020-05-28 16:02:02] [16:02:02.322] Batch 91...
[INFO] [2020-05-28 16:02:04] [16:02:04.430] Batch 92...
[INFO] [2020-05-28 16:02:06] [16:02:06.573] Batch 93...
[INFO] [2020-05-28 16:02:08] [16:02:08.941] Batch 94...
[INFO] [2020-05-28 16:02:11] [16:02:11.253] Batch 95...
[INFO] [2020-05-28 16:02:12] [16:02:12.183] Removed 950717 Nodes
[START] [2020-05-28 16:16:38] logged process
[START] [2020-05-28 16:16:38] Creating resource from OpenData
[START] [2020-05-28 16:17:05] logged process
[START] [2020-05-28 16:17:05] Parse meta.xml file and create formats with fields
[STOP] [2020-05-28 16:17:07] Parse meta.xml file and create formats with fields
[STOP] [2020-05-28 16:17:07] Creating resource from OpenData
[START] [2020-05-28 16:17:07] logged process
[START] [2020-05-28 16:17:07] create_harvest_instance
[STOP] [2020-05-28 16:17:09] create_harvest_instance
[START] [2020-05-28 16:17:09] fetch_files
[STOP] [2020-05-28 16:17:09] fetch_files
[START] [2020-05-28 16:17:09] validate_each_file
[STOP] [2020-05-28 16:19:08] validate_each_file
[START] [2020-05-28 16:19:08] convert_to_csv
[CMD] [2020-05-28 16:19:08] /usr/bin/sort /app/public/converted_csv/wntgd_nodes_21024.csv > /app/public/converted_csv/wntgd_nodes_21024.csv_sorted
[CMD] [2020-05-28 16:19:09] /usr/bin/sort /app/public/converted_csv/wntgd_media_21025.csv > /app/public/converted_csv/wntgd_media_21025.csv_sorted
[STOP] [2020-05-28 16:19:12] convert_to_csv
[START] [2020-05-28 16:19:12] calculate_delta
[CMD] [2020-05-28 16:19:12] echo "0a" > /app/public/diff/wntgd_nodes_21024.diff
[CMD] [2020-05-28 16:19:12] tail -n +1 /app/public/converted_csv/wntgd_nodes_21024.csv >> /app/public/diff/wntgd_nodes_21024.diff
[CMD] [2020-05-28 16:19:12] echo "." >> /app/public/diff/wntgd_nodes_21024.diff
[CMD] [2020-05-28 16:19:12] echo "0a" > /app/public/diff/wntgd_media_21025.diff
[CMD] [2020-05-28 16:19:12] tail -n +1 /app/public/converted_csv/wntgd_media_21025.csv >> /app/public/diff/wntgd_media_21025.diff
[CMD] [2020-05-28 16:19:14] echo "." >> /app/public/diff/wntgd_media_21025.diff
[STOP] [2020-05-28 16:19:14] calculate_delta
[START] [2020-05-28 16:19:14] parse_diff_and_store
[INFO] [2020-05-28 16:19:14] Loading nodes diff file into memory (true lines)...
[INFO] [2020-05-28 16:30:01] Loading media diff file into memory (true lines)...
[INFO] [2020-05-28 19:23:00] Storing 953243 ScientificNames
[INFO] [2020-05-28 19:23:00] Processing group of 953243 in 954 groups of 1000
[INFO] [2020-05-28 19:35:27] Average Time: 0.777
[INFO] [2020-05-28 19:35:27] Total Time: 12m28s
[INFO] [2020-05-28 19:35:27] last 3 / first 3: 0.03
[INFO] [2020-05-28 19:35:27] Std.Dev: 3.619254066793322; Max: 33.88
[INFO] [2020-05-28 19:35:27] Storing 953254 Identifiers
[INFO] [2020-05-28 19:35:27] Processing group of 953254 in 954 groups of 1000
[INFO] [2020-05-28 19:39:39] Average Time: 0.259
[INFO] [2020-05-28 19:39:39] Total Time: 4m12s
[INFO] [2020-05-28 19:39:39] last 3 / first 3: 0.59
[INFO] [2020-05-28 19:39:39] Std.Dev: 2.203860249652868; Max: 34.62
[INFO] [2020-05-28 19:39:39] Storing 953243 Nodes
[INFO] [2020-05-28 19:39:39] Processing group of 953243 in 954 groups of 1000
[INFO] [2020-05-28 19:59:31] Average Time: 1.244
[INFO] [2020-05-28 19:59:31] Total Time: 19m52s
[INFO] [2020-05-28 19:59:31] last 3 / first 3: 0.68
[INFO] [2020-05-28 19:59:31] Std.Dev: 5.6558818940992746; Max: 37.79
[INFO] [2020-05-28 19:59:31] Storing 1609587 ArticlesSections
[INFO] [2020-05-28 19:59:31] Processing group of 1609587 in 1610 groups of 1000
[INFO] [2020-05-28 20:05:08] Average Time: 0.204
[INFO] [2020-05-28 20:05:08] Total Time: 5m38s
[INFO] [2020-05-28 20:05:08] last 3 / first 3: 0.87
[INFO] [2020-05-28 20:05:08] Std.Dev: 2.389351376419969; Max: 42.46
[INFO] [2020-05-28 20:05:08] Storing 1609587 Articles
[INFO] [2020-05-28 20:05:08] Processing group of 1609587 in 1610 groups of 1000
[INFO] [2020-05-28 20:41:05] Average Time: 1.333
[INFO] [2020-05-28 20:41:05] Total Time: 35m57s
[INFO] [2020-05-28 20:41:05] last 3 / first 3: 0.65
[INFO] [2020-05-28 20:41:05] Std.Dev: 6.035312750802563; Max: 43.85
[STOP] [2020-05-28 20:41:05] parse_diff_and_store
[START] [2020-05-28 20:41:05] resolve_keys
[STOP] [2020-05-28 20:53:15] resolve_keys
[ERR] [2020-05-28 20:53:15] ActiveRecord::StatementInvalid
[ERR] [2020-05-28 20:53:15] Mysql2::Error: The total number of locks exceeds the lock table size: UPDATE `articles` t JOIN `nodes` o ON (t.`node_resource_pk` = o.`resource_pk` AND t.harvest_id = 2848 AND o.harvest_id = t.harvest_id ) SET t.`node_id` = o.`id` WHERE t.id >= 13588992 AND t.id <= 13652991
[ERR] [2020-05-28 20:53:15] ./config/initializers/core_extensions.rb:50:in `clean_execute'
[ERR] [2020-05-28 20:53:15] ./config/initializers/core_extensions.rb:40:in `propagate_id'
[ERR] [2020-05-28 20:53:15] ../models/resource_harvester.rb:577:in `propagate_id'
[ERR] [2020-05-28 20:53:15] ../models/resource_harvester.rb:469:in `resolve_article_keys'
[ERR] [2020-05-28 20:53:15] ../models/resource_harvester.rb:429:in `resolve_keys'
[ERR] [2020-05-28 20:53:15] ../models/resource_harvester.rb:86:in `block (3 levels) in start'
[ERR] [2020-05-28 20:53:15] ../models/logged_process.rb:19:in `run_step'
[ERR] [2020-05-28 20:53:15] ../models/resource_harvester.rb:86:in `block (2 levels) in start'
[ERR] [2020-05-28 20:53:15] ../models/resource_harvester.rb:75:in `each_key'
[ERR] [2020-05-28 20:53:15] ../models/resource_harvester.rb:75:in `block in start'
[ERR] [2020-05-28 20:53:15] ../models/resource.rb:151:in `lock'
[ERR] [2020-05-28 20:53:15] ../models/resource_harvester.rb:72:in `start'
[ERR] [2020-05-28 20:53:15] ../models/resource.rb:232:in `harvest'
[ERR] [2020-05-28 20:53:15] ../models/resource.rb:208:in `re_download_opendata_and_harvest'
[ERR] [2020-05-28 20:53:15] bin/rails:4:in `require'
[ERR] [2020-05-28 20:53:15] bin/rails:4:in `<main>'
[STOP] [2020-05-28 20:53:15] logged process, took 16567.28
[INFO] [2020-05-29 10:22:01] ## HARVEST: type = resume_-harvest
[START] [2020-05-29 10:22:02] logged process
[ERR] [2020-05-29 10:22:02][hdls] *****
[ERR] [2020-05-29 10:22:02][hdls] ***** HARVEST ATTEMPT FAILED: This resource is locked; assuming it is already running. Remove lock if not.
[ERR] [2020-05-29 10:22:02][hdls] *****
[INFO] [2020-05-29 10:24:02] ## HARVEST: type = resume_-harvest
[START] [2020-05-29 10:24:02] logged process
[INFO] [2020-05-29 10:24:02] Already completed stage create_harvest_instance, skipping...
[INFO] [2020-05-29 10:24:02] Already completed stage fetch_files, skipping...
[INFO] [2020-05-29 10:24:02] Already completed stage validate_each_file, skipping...
[INFO] [2020-05-29 10:24:02] Already completed stage convert_to_csv, skipping...
[INFO] [2020-05-29 10:24:02] Already completed stage calculate_delta, skipping...
[INFO] [2020-05-29 10:24:02] Already completed stage parse_diff_and_store, skipping...
[START] [2020-05-29 10:24:02] resolve_keys
[STOP] [2020-05-29 10:28:04] resolve_keys
[ERR] [2020-05-29 10:28:04] ActiveRecord::StatementInvalid
[ERR] [2020-05-29 10:28:04] Mysql2::Error: The total number of locks exceeds the lock table size: UPDATE `articles` t JOIN `nodes` o ON (t.`node_resource_pk` = o.`resource_pk` AND t.harvest_id = 2848 AND o.harvest_id = t.harvest_id ) SET t.`node_id` = o.`id` WHERE t.id >= 13588992 AND t.id <= 13652991
[ERR] [2020-05-29 10:28:04] ./config/initializers/core_extensions.rb:50:in `clean_execute'
[ERR] [2020-05-29 10:28:04] ./config/initializers/core_extensions.rb:40:in `propagate_id'
[ERR] [2020-05-29 10:28:04] ../models/resource_harvester.rb:577:in `propagate_id'
[ERR] [2020-05-29 10:28:04] ../models/resource_harvester.rb:469:in `resolve_article_keys'
[ERR] [2020-05-29 10:28:04] ../models/resource_harvester.rb:429:in `resolve_keys'
[ERR] [2020-05-29 10:28:04] ../models/resource_harvester.rb:86:in `block (3 levels) in start'
[ERR] [2020-05-29 10:28:04] ../models/logged_process.rb:19:in `run_step'
[ERR] [2020-05-29 10:28:04] ../models/resource_harvester.rb:86:in `block (2 levels) in start'
[ERR] [2020-05-29 10:28:04] ../models/resource_harvester.rb:75:in `each_key'
[ERR] [2020-05-29 10:28:04] ../models/resource_harvester.rb:75:in `block in start'
[ERR] [2020-05-29 10:28:04] ../models/resource.rb:151:in `lock'
[ERR] [2020-05-29 10:28:04] ../models/resource_harvester.rb:72:in `start'
[ERR] [2020-05-29 10:28:04] ../models/resource.rb:259:in `resume'
[ERR] [2020-05-29 10:28:04] bin/rails:4:in `require'
[ERR] [2020-05-29 10:28:04] bin/rails:4:in `<main>'
[STOP] [2020-05-29 10:28:04] logged process, took 241.55

Latest Process

29 May 10:24 EXITED