Stage:
completed
Fetched:
06 Mar 10:49
Validated:
06 Mar 10:50
Deltas Created
06 Mar 10:50
Units Normalized:
06 Mar 20:58
Ancestry Built:
06 Mar 13:26
Nodes Matched:
06 Mar 20:27
Names Parsed:
06 Mar 13:44
New Models Stored:
06 Mar 11:30
Indexed:
06 Mar 20:58
Completed:
10 Mar 19:45
Time to Harvest:
5 minutes
Harvesting Log
(853 lines)
[INFO] [2025-03-06 10:49:28] Created harvest instance #4567
[STOP] [2025-03-06 10:49:28] create_harvest_instance
[START] [2025-03-06 10:49:28] fetch_files
[STOP] [2025-03-06 10:49:28] fetch_files
[START] [2025-03-06 10:49:28] validate_each_file
[INFO] [2025-03-06 10:49:28] Created new folder: /app/public/converted_csv
[INFO] [2025-03-06 10:49:28] Looping over 2 formats...
[INFO] [2025-03-06 10:49:28] ...nodes (/app/public/data/i_nat_taxonomy_v/taxa.tsv)
[INFO] [2025-03-06 10:50:08] Valid: /app/public/data/i_nat_taxonomy_v/converted_csv/i_nat_taxonomy_v_nodes_31227.csv (1334824 lines)
[INFO] [2025-03-06 10:50:08] ...vernaculars (/app/public/data/i_nat_taxonomy_v/vernacular.tsv)
[INFO] [2025-03-06 10:50:28] Valid: /app/public/data/i_nat_taxonomy_v/converted_csv/i_nat_taxonomy_v_vernaculars_31228.csv (1324058 lines)
[STOP] [2025-03-06 10:50:28] validate_each_file
[START] [2025-03-06 10:50:28] convert_to_csv
[INFO] [2025-03-06 10:50:28] Looping over 2 formats...
[INFO] [2025-03-06 10:50:28] ...nodes (/app/public/data/i_nat_taxonomy_v/taxa.tsv)
[CMD] [2025-03-06 10:50:28] /usr/bin/sort /app/public/data/i_nat_taxonomy_v/converted_csv/i_nat_taxonomy_v_nodes_31227.csv > /app/public/data/i_nat_taxonomy_v/converted_csv/i_nat_taxonomy_v_nodes_31227.csv_sorted
[INFO] [2025-03-06 10:50:32] Converted: /app/public/data/i_nat_taxonomy_v/converted_csv/i_nat_taxonomy_v_nodes_31227.csv (1334824 lines)
[INFO] [2025-03-06 10:50:32] ...vernaculars (/app/public/data/i_nat_taxonomy_v/vernacular.tsv)
[CMD] [2025-03-06 10:50:32] /usr/bin/sort /app/public/data/i_nat_taxonomy_v/converted_csv/i_nat_taxonomy_v_vernaculars_31228.csv > /app/public/data/i_nat_taxonomy_v/converted_csv/i_nat_taxonomy_v_vernaculars_31228.csv_sorted
[INFO] [2025-03-06 10:50:34] Converted: /app/public/data/i_nat_taxonomy_v/converted_csv/i_nat_taxonomy_v_vernaculars_31228.csv (1324058 lines)
[STOP] [2025-03-06 10:50:34] convert_to_csv
[START] [2025-03-06 10:50:34] calculate_delta
[INFO] [2025-03-06 10:50:34] Created diff dir: /app/public/diff
[INFO] [2025-03-06 10:50:34] Looping over 2 formats...
[INFO] [2025-03-06 10:50:34] ...nodes (/app/public/data/i_nat_taxonomy_v/taxa.tsv)
[CMD] [2025-03-06 10:50:34] echo "0a" > /app/public/data/i_nat_taxonomy_v/diff/i_nat_taxonomy_v_nodes_31227.diff
[CMD] [2025-03-06 10:50:34] tail -n +1 /app/public/data/i_nat_taxonomy_v/converted_csv/i_nat_taxonomy_v_nodes_31227.csv >> /app/public/data/i_nat_taxonomy_v/diff/i_nat_taxonomy_v_nodes_31227.diff
[CMD] [2025-03-06 10:50:37] echo "." >> /app/public/data/i_nat_taxonomy_v/diff/i_nat_taxonomy_v_nodes_31227.diff
[INFO] [2025-03-06 10:50:37] Created diff: /app/public/data/i_nat_taxonomy_v/diff/i_nat_taxonomy_v_nodes_31227.diff (1334826 lines)
[INFO] [2025-03-06 10:50:37] ...vernaculars (/app/public/data/i_nat_taxonomy_v/vernacular.tsv)
[CMD] [2025-03-06 10:50:37] echo "0a" > /app/public/data/i_nat_taxonomy_v/diff/i_nat_taxonomy_v_vernaculars_31228.diff
[CMD] [2025-03-06 10:50:37] tail -n +1 /app/public/data/i_nat_taxonomy_v/converted_csv/i_nat_taxonomy_v_vernaculars_31228.csv >> /app/public/data/i_nat_taxonomy_v/diff/i_nat_taxonomy_v_vernaculars_31228.diff
[CMD] [2025-03-06 10:50:38] echo "." >> /app/public/data/i_nat_taxonomy_v/diff/i_nat_taxonomy_v_vernaculars_31228.diff
[INFO] [2025-03-06 10:50:39] Created diff: /app/public/data/i_nat_taxonomy_v/diff/i_nat_taxonomy_v_vernaculars_31228.diff (1324060 lines)
[STOP] [2025-03-06 10:50:39] calculate_delta
[START] [2025-03-06 10:50:39] parse_diff_and_store
[INFO] [2025-03-06 10:50:40] Handling diff: /app/public/data/i_nat_taxonomy_v/diff/i_nat_taxonomy_v_nodes_31227.diff (1334826 lines)
[INFO] [2025-03-06 10:50:40] Loading nodes diff file into memory (1334826 lines)...
[INFO] [2025-03-06 10:50:43] Storing 9999 ScientificNames (19998/10000/1334826)
[INFO] [2025-03-06 10:50:46] Storing 9999 Nodes (19998/10000/1334826)
[INFO] [2025-03-06 10:50:52] Storing 10000 ScientificNames (39998/20000/1334826)
[INFO] [2025-03-06 10:50:55] Storing 10000 Nodes (39998/20000/1334826)
[INFO] [2025-03-06 10:51:01] Storing 10000 ScientificNames (59998/30000/1334826)
[INFO] [2025-03-06 10:51:04] Storing 10000 Nodes (59998/30000/1334826)
[INFO] [2025-03-06 10:51:10] Storing 10000 ScientificNames (79998/40000/1334826)
[INFO] [2025-03-06 10:51:13] Storing 10000 Nodes (79998/40000/1334826)
[INFO] [2025-03-06 10:51:19] Storing 10000 ScientificNames (99998/50000/1334826)
[INFO] [2025-03-06 10:51:22] Storing 10000 Nodes (99998/50000/1334826)
[INFO] [2025-03-06 10:51:29] Storing 10000 ScientificNames (119998/60000/1334826)
[INFO] [2025-03-06 10:51:32] Storing 10000 Nodes (119998/60000/1334826)
[INFO] [2025-03-06 10:51:38] Storing 10000 ScientificNames (139998/70000/1334826)
[INFO] [2025-03-06 10:51:41] Storing 10000 Nodes (139998/70000/1334826)
[INFO] [2025-03-06 10:51:47] Storing 10000 ScientificNames (159998/80000/1334826)
[INFO] [2025-03-06 10:51:50] Storing 10000 Nodes (159998/80000/1334826)
[INFO] [2025-03-06 10:51:57] Storing 10000 ScientificNames (179998/90000/1334826)
[INFO] [2025-03-06 10:52:00] Storing 10000 Nodes (179998/90000/1334826)
[INFO] [2025-03-06 10:52:06] Storing 10000 ScientificNames (199998/100000/1334826)
[INFO] [2025-03-06 10:52:09] Storing 10000 Nodes (199998/100000/1334826)
[INFO] [2025-03-06 10:52:16] Storing 10000 ScientificNames (219998/110000/1334826)
[INFO] [2025-03-06 10:52:19] Storing 10000 Nodes (219998/110000/1334826)
[INFO] [2025-03-06 10:52:26] Storing 10000 ScientificNames (239998/120000/1334826)
[INFO] [2025-03-06 10:52:29] Storing 10000 Nodes (239998/120000/1334826)
[INFO] [2025-03-06 10:52:36] Storing 10000 ScientificNames (259998/130000/1334826)
[INFO] [2025-03-06 10:52:39] Storing 10000 Nodes (259998/130000/1334826)
[INFO] [2025-03-06 10:52:46] Storing 10000 ScientificNames (279998/140000/1334826)
[INFO] [2025-03-06 10:52:49] Storing 10000 Nodes (279998/140000/1334826)
[INFO] [2025-03-06 10:52:56] Storing 10000 ScientificNames (299998/150000/1334826)
[INFO] [2025-03-06 10:52:59] Storing 10000 Nodes (299998/150000/1334826)
[INFO] [2025-03-06 10:53:06] Storing 10000 ScientificNames (319998/160000/1334826)
[INFO] [2025-03-06 10:53:09] Storing 10000 Nodes (319998/160000/1334826)
[INFO] [2025-03-06 10:53:16] Storing 10000 ScientificNames (339998/170000/1334826)
[INFO] [2025-03-06 10:53:19] Storing 10000 Nodes (339998/170000/1334826)
[INFO] [2025-03-06 10:53:26] Storing 10000 ScientificNames (359998/180000/1334826)
[INFO] [2025-03-06 10:53:30] Storing 10000 Nodes (359998/180000/1334826)
[INFO] [2025-03-06 10:53:37] Storing 10000 ScientificNames (379998/190000/1334826)
[INFO] [2025-03-06 10:53:40] Storing 10000 Nodes (379998/190000/1334826)
[INFO] [2025-03-06 10:53:47] Storing 10000 ScientificNames (399998/200000/1334826)
[INFO] [2025-03-06 10:53:51] Storing 10000 Nodes (399998/200000/1334826)
[INFO] [2025-03-06 10:53:58] Storing 10000 ScientificNames (419998/210000/1334826)
[INFO] [2025-03-06 10:54:01] Storing 10000 Nodes (419998/210000/1334826)
[INFO] [2025-03-06 10:54:09] Storing 10000 ScientificNames (439998/220000/1334826)
[INFO] [2025-03-06 10:54:12] Storing 10000 Nodes (439998/220000/1334826)
[INFO] [2025-03-06 10:54:20] Storing 10000 ScientificNames (459998/230000/1334826)
[INFO] [2025-03-06 10:54:23] Storing 10000 Nodes (459998/230000/1334826)
[INFO] [2025-03-06 10:54:31] Storing 10000 ScientificNames (479998/240000/1334826)
[INFO] [2025-03-06 10:54:34] Storing 10000 Nodes (479998/240000/1334826)
[INFO] [2025-03-06 10:54:42] Storing 10000 ScientificNames (499998/250000/1334826)
[INFO] [2025-03-06 10:54:45] Storing 10000 Nodes (499998/250000/1334826)
[INFO] [2025-03-06 10:54:53] Storing 10000 ScientificNames (519998/260000/1334826)
[INFO] [2025-03-06 10:54:56] Storing 10000 Nodes (519998/260000/1334826)
[INFO] [2025-03-06 10:55:04] Storing 10000 ScientificNames (539998/270000/1334826)
[INFO] [2025-03-06 10:55:07] Storing 10000 Nodes (539998/270000/1334826)
[INFO] [2025-03-06 10:55:15] Storing 10000 ScientificNames (559998/280000/1334826)
[INFO] [2025-03-06 10:55:18] Storing 10000 Nodes (559998/280000/1334826)
[INFO] [2025-03-06 10:55:27] Storing 10000 ScientificNames (579998/290000/1334826)
[INFO] [2025-03-06 10:55:30] Storing 10000 Nodes (579998/290000/1334826)
[INFO] [2025-03-06 10:55:38] Storing 10000 ScientificNames (599998/300000/1334826)
[INFO] [2025-03-06 10:55:41] Storing 10000 Nodes (599998/300000/1334826)
[INFO] [2025-03-06 10:55:49] Storing 10000 ScientificNames (619998/310000/1334826)
[INFO] [2025-03-06 10:55:53] Storing 10000 Nodes (619998/310000/1334826)
[INFO] [2025-03-06 10:56:01] Storing 10000 ScientificNames (639998/320000/1334826)
[INFO] [2025-03-06 10:56:04] Storing 10000 Nodes (639998/320000/1334826)
[INFO] [2025-03-06 10:56:12] Storing 10000 ScientificNames (659998/330000/1334826)
[INFO] [2025-03-06 10:56:16] Storing 10000 Nodes (659998/330000/1334826)
[INFO] [2025-03-06 10:56:24] Storing 10000 ScientificNames (679998/340000/1334826)
[INFO] [2025-03-06 10:56:27] Storing 10000 Nodes (679998/340000/1334826)
[INFO] [2025-03-06 10:56:36] Storing 10000 ScientificNames (699998/350000/1334826)
[INFO] [2025-03-06 10:56:39] Storing 10000 Nodes (699998/350000/1334826)
[INFO] [2025-03-06 10:56:48] Storing 10000 ScientificNames (719998/360000/1334826)
[INFO] [2025-03-06 10:56:51] Storing 10000 Nodes (719998/360000/1334826)
[INFO] [2025-03-06 10:57:00] Storing 10000 ScientificNames (739998/370000/1334826)
[INFO] [2025-03-06 10:57:03] Storing 10000 Nodes (739998/370000/1334826)
[INFO] [2025-03-06 10:57:11] Storing 10000 ScientificNames (759998/380000/1334826)
[INFO] [2025-03-06 10:57:15] Storing 10000 Nodes (759998/380000/1334826)
[INFO] [2025-03-06 10:57:23] Storing 10000 ScientificNames (779998/390000/1334826)
[INFO] [2025-03-06 10:57:27] Storing 10000 Nodes (779998/390000/1334826)
[INFO] [2025-03-06 10:57:36] Storing 10000 ScientificNames (799998/400000/1334826)
[INFO] [2025-03-06 10:57:39] Storing 10000 Nodes (799998/400000/1334826)
[INFO] [2025-03-06 10:57:48] Storing 10000 ScientificNames (819998/410000/1334826)
[INFO] [2025-03-06 10:57:51] Storing 10000 Nodes (819998/410000/1334826)
[INFO] [2025-03-06 10:58:00] Storing 10000 ScientificNames (839998/420000/1334826)
[INFO] [2025-03-06 10:58:03] Storing 10000 Nodes (839998/420000/1334826)
[INFO] [2025-03-06 10:58:12] Storing 10000 ScientificNames (859998/430000/1334826)
[INFO] [2025-03-06 10:58:15] Storing 10000 Nodes (859998/430000/1334826)
[INFO] [2025-03-06 10:58:24] Storing 10000 ScientificNames (879998/440000/1334826)
[INFO] [2025-03-06 10:58:27] Storing 10000 Nodes (879998/440000/1334826)
[INFO] [2025-03-06 10:58:37] Storing 10000 ScientificNames (899998/450000/1334826)
[INFO] [2025-03-06 10:58:40] Storing 10000 Nodes (899998/450000/1334826)
[INFO] [2025-03-06 10:58:50] Storing 10000 ScientificNames (919998/460000/1334826)
[INFO] [2025-03-06 10:58:53] Storing 10000 Nodes (919998/460000/1334826)
[INFO] [2025-03-06 10:59:03] Storing 10000 ScientificNames (939998/470000/1334826)
[INFO] [2025-03-06 10:59:06] Storing 10000 Nodes (939998/470000/1334826)
[INFO] [2025-03-06 10:59:16] Storing 10000 ScientificNames (959998/480000/1334826)
[INFO] [2025-03-06 10:59:19] Storing 10000 Nodes (959998/480000/1334826)
[INFO] [2025-03-06 10:59:28] Storing 10000 ScientificNames (979998/490000/1334826)
[INFO] [2025-03-06 10:59:32] Storing 10000 Nodes (979998/490000/1334826)
[INFO] [2025-03-06 10:59:42] Storing 10000 ScientificNames (999998/500000/1334826)
[INFO] [2025-03-06 10:59:45] Storing 10000 Nodes (999998/500000/1334826)
[INFO] [2025-03-06 10:59:55] Storing 10000 ScientificNames (1019998/510000/1334826)
[INFO] [2025-03-06 10:59:58] Storing 10000 Nodes (1019998/510000/1334826)
[INFO] [2025-03-06 11:00:09] Storing 10000 ScientificNames (1039998/520000/1334826)
[INFO] [2025-03-06 11:00:12] Storing 10000 Nodes (1039998/520000/1334826)
[INFO] [2025-03-06 11:00:23] Storing 10000 ScientificNames (1059998/530000/1334826)
[INFO] [2025-03-06 11:00:26] Storing 10000 Nodes (1059998/530000/1334826)
[INFO] [2025-03-06 11:00:36] Storing 10000 ScientificNames (1079998/540000/1334826)
[INFO] [2025-03-06 11:00:39] Storing 10000 Nodes (1079998/540000/1334826)
[INFO] [2025-03-06 11:00:50] Storing 10000 ScientificNames (1099998/550000/1334826)
[INFO] [2025-03-06 11:00:53] Storing 10000 Nodes (1099998/550000/1334826)
[INFO] [2025-03-06 11:01:04] Storing 10000 ScientificNames (1119998/560000/1334826)
[INFO] [2025-03-06 11:01:07] Storing 10000 Nodes (1119998/560000/1334826)
[INFO] [2025-03-06 11:01:17] Storing 10000 ScientificNames (1139998/570000/1334826)
[INFO] [2025-03-06 11:01:21] Storing 10000 Nodes (1139998/570000/1334826)
[INFO] [2025-03-06 11:01:32] Storing 10000 ScientificNames (1159998/580000/1334826)
[INFO] [2025-03-06 11:01:35] Storing 10000 Nodes (1159998/580000/1334826)
[INFO] [2025-03-06 11:01:45] Storing 10000 ScientificNames (1179998/590000/1334826)
[INFO] [2025-03-06 11:01:48] Storing 10000 Nodes (1179998/590000/1334826)
[INFO] [2025-03-06 11:02:00] Storing 10000 ScientificNames (1199998/600000/1334826)
[INFO] [2025-03-06 11:02:03] Storing 10000 Nodes (1199998/600000/1334826)
[INFO] [2025-03-06 11:02:13] Storing 10000 ScientificNames (1219998/610000/1334826)
[INFO] [2025-03-06 11:02:16] Storing 10000 Nodes (1219998/610000/1334826)
[INFO] [2025-03-06 11:02:27] Storing 10000 ScientificNames (1239998/620000/1334826)
[INFO] [2025-03-06 11:02:31] Storing 10000 Nodes (1239998/620000/1334826)
[INFO] [2025-03-06 11:02:42] Storing 10000 ScientificNames (1259998/630000/1334826)
[INFO] [2025-03-06 11:02:46] Storing 10000 Nodes (1259998/630000/1334826)
[INFO] [2025-03-06 11:02:57] Storing 10000 ScientificNames (1279998/640000/1334826)
[INFO] [2025-03-06 11:03:00] Storing 10000 Nodes (1279998/640000/1334826)
[INFO] [2025-03-06 11:03:12] Storing 10000 ScientificNames (1299998/650000/1334826)
[INFO] [2025-03-06 11:03:15] Storing 10000 Nodes (1299998/650000/1334826)
[INFO] [2025-03-06 11:03:26] Storing 10000 ScientificNames (1319998/660000/1334826)
[INFO] [2025-03-06 11:03:29] Storing 10000 Nodes (1319998/660000/1334826)
[INFO] [2025-03-06 11:03:41] Storing 10000 ScientificNames (1339998/670000/1334826)
[INFO] [2025-03-06 11:03:44] Storing 10000 Nodes (1339998/670000/1334826)
[INFO] [2025-03-06 11:03:56] Storing 10000 ScientificNames (1359998/680000/1334826)
[INFO] [2025-03-06 11:04:00] Storing 10000 Nodes (1359998/680000/1334826)
[INFO] [2025-03-06 11:04:11] Storing 10000 ScientificNames (1379998/690000/1334826)
[INFO] [2025-03-06 11:04:14] Storing 10000 Nodes (1379998/690000/1334826)
[INFO] [2025-03-06 11:04:26] Storing 10000 ScientificNames (1399998/700000/1334826)
[INFO] [2025-03-06 11:04:29] Storing 10000 Nodes (1399998/700000/1334826)
[INFO] [2025-03-06 11:04:41] Storing 10000 ScientificNames (1419998/710000/1334826)
[INFO] [2025-03-06 11:04:44] Storing 10000 Nodes (1419998/710000/1334826)
[INFO] [2025-03-06 11:04:56] Storing 10000 ScientificNames (1439998/720000/1334826)
[INFO] [2025-03-06 11:04:59] Storing 10000 Nodes (1439998/720000/1334826)
[INFO] [2025-03-06 11:05:12] Storing 10000 ScientificNames (1459998/730000/1334826)
[INFO] [2025-03-06 11:05:15] Storing 10000 Nodes (1459998/730000/1334826)
[WARN] [2025-03-06 11:05:18] Filtered Scientific Name `Morpho epistrophus catenaria/catenarius` to `Morpho epistrophus catenariacatenarius`
[INFO] [2025-03-06 11:05:28] Storing 10000 ScientificNames (1479998/740000/1334826)
[INFO] [2025-03-06 11:05:31] Storing 10000 Nodes (1479998/740000/1334826)
[INFO] [2025-03-06 11:05:43] Storing 10000 ScientificNames (1499998/750000/1334826)
[INFO] [2025-03-06 11:05:46] Storing 10000 Nodes (1499998/750000/1334826)
[INFO] [2025-03-06 11:05:59] Storing 10000 ScientificNames (1519998/760000/1334826)
[INFO] [2025-03-06 11:06:02] Storing 10000 Nodes (1519998/760000/1334826)
[INFO] [2025-03-06 11:06:15] Storing 10000 ScientificNames (1539998/770000/1334826)
[INFO] [2025-03-06 11:06:18] Storing 10000 Nodes (1539998/770000/1334826)
[INFO] [2025-03-06 11:06:30] Storing 10000 ScientificNames (1559998/780000/1334826)
[INFO] [2025-03-06 11:06:34] Storing 10000 Nodes (1559998/780000/1334826)
[INFO] [2025-03-06 11:06:46] Storing 10000 ScientificNames (1579998/790000/1334826)
[INFO] [2025-03-06 11:06:49] Storing 10000 Nodes (1579998/790000/1334826)
[INFO] [2025-03-06 11:07:02] Storing 10000 ScientificNames (1599998/800000/1334826)
[INFO] [2025-03-06 11:07:06] Storing 10000 Nodes (1599998/800000/1334826)
[WARN] [2025-03-06 11:07:09] Filtered Scientific Name `Triboniophorus nov “kaputar""` to `Triboniophorus nov “kaputar`
[INFO] [2025-03-06 11:07:19] Storing 10000 ScientificNames (1619998/810000/1334826)
[INFO] [2025-03-06 11:07:22] Storing 10000 Nodes (1619998/810000/1334826)
[INFO] [2025-03-06 11:07:35] Storing 10000 ScientificNames (1639998/820000/1334826)
[INFO] [2025-03-06 11:07:38] Storing 10000 Nodes (1639998/820000/1334826)
[INFO] [2025-03-06 11:07:52] Storing 10000 ScientificNames (1659998/830000/1334826)
[INFO] [2025-03-06 11:07:55] Storing 10000 Nodes (1659998/830000/1334826)
[INFO] [2025-03-06 11:08:09] Storing 10000 ScientificNames (1679998/840000/1334826)
[INFO] [2025-03-06 11:08:12] Storing 10000 Nodes (1679998/840000/1334826)
[INFO] [2025-03-06 11:08:25] Storing 10000 ScientificNames (1699998/850000/1334826)
[INFO] [2025-03-06 11:08:29] Storing 10000 Nodes (1699998/850000/1334826)
[INFO] [2025-03-06 11:08:42] Storing 10000 ScientificNames (1719998/860000/1334826)
[INFO] [2025-03-06 11:08:45] Storing 10000 Nodes (1719998/860000/1334826)
[INFO] [2025-03-06 11:09:00] Storing 10000 ScientificNames (1739998/870000/1334826)
[INFO] [2025-03-06 11:09:03] Storing 10000 Nodes (1739998/870000/1334826)
[INFO] [2025-03-06 11:09:17] Storing 10000 ScientificNames (1759998/880000/1334826)
[INFO] [2025-03-06 11:09:20] Storing 10000 Nodes (1759998/880000/1334826)
[INFO] [2025-03-06 11:09:35] Storing 10000 ScientificNames (1779998/890000/1334826)
[INFO] [2025-03-06 11:09:38] Storing 10000 Nodes (1779998/890000/1334826)
[INFO] [2025-03-06 11:09:51] Storing 10000 ScientificNames (1799998/900000/1334826)
[INFO] [2025-03-06 11:09:54] Storing 10000 Nodes (1799998/900000/1334826)
[INFO] [2025-03-06 11:10:08] Storing 10000 ScientificNames (1819998/910000/1334826)
[INFO] [2025-03-06 11:10:11] Storing 10000 Nodes (1819998/910000/1334826)
[INFO] [2025-03-06 11:10:26] Storing 10000 ScientificNames (1839998/920000/1334826)
[INFO] [2025-03-06 11:10:29] Storing 10000 Nodes (1839998/920000/1334826)
[INFO] [2025-03-06 11:10:43] Storing 10000 ScientificNames (1859998/930000/1334826)
[INFO] [2025-03-06 11:10:47] Storing 10000 Nodes (1859998/930000/1334826)
[INFO] [2025-03-06 11:11:01] Storing 10000 ScientificNames (1879998/940000/1334826)
[INFO] [2025-03-06 11:11:05] Storing 10000 Nodes (1879998/940000/1334826)
[INFO] [2025-03-06 11:11:19] Storing 10000 ScientificNames (1899998/950000/1334826)
[INFO] [2025-03-06 11:11:22] Storing 10000 Nodes (1899998/950000/1334826)
[INFO] [2025-03-06 11:11:37] Storing 10000 ScientificNames (1919998/960000/1334826)
[INFO] [2025-03-06 11:11:41] Storing 10000 Nodes (1919998/960000/1334826)
[INFO] [2025-03-06 11:11:56] Storing 10000 ScientificNames (1939998/970000/1334826)
[INFO] [2025-03-06 11:11:59] Storing 10000 Nodes (1939998/970000/1334826)
[INFO] [2025-03-06 11:12:14] Storing 10000 ScientificNames (1959998/980000/1334826)
[INFO] [2025-03-06 11:12:17] Storing 10000 Nodes (1959998/980000/1334826)
[INFO] [2025-03-06 11:12:33] Storing 10000 ScientificNames (1979998/990000/1334826)
[INFO] [2025-03-06 11:12:36] Storing 10000 Nodes (1979998/990000/1334826)
[INFO] [2025-03-06 11:12:51] Storing 10000 ScientificNames (1999998/1000000/1334826)
[INFO] [2025-03-06 11:12:54] Storing 10000 Nodes (1999998/1000000/1334826)
[INFO] [2025-03-06 11:13:09] Storing 10000 ScientificNames (2019998/1010000/1334826)
[INFO] [2025-03-06 11:13:12] Storing 10000 Nodes (2019998/1010000/1334826)
[INFO] [2025-03-06 11:13:28] Storing 10000 ScientificNames (2039998/1020000/1334826)
[INFO] [2025-03-06 11:13:31] Storing 10000 Nodes (2039998/1020000/1334826)
[INFO] [2025-03-06 11:13:48] Storing 10000 ScientificNames (2059998/1030000/1334826)
[INFO] [2025-03-06 11:13:51] Storing 10000 Nodes (2059998/1030000/1334826)
[INFO] [2025-03-06 11:14:06] Storing 10000 ScientificNames (2079998/1040000/1334826)
[INFO] [2025-03-06 11:14:10] Storing 10000 Nodes (2079998/1040000/1334826)
[INFO] [2025-03-06 11:14:24] Storing 10000 ScientificNames (2099998/1050000/1334826)
[INFO] [2025-03-06 11:14:27] Storing 10000 Nodes (2099998/1050000/1334826)
[INFO] [2025-03-06 11:14:44] Storing 10000 ScientificNames (2119998/1060000/1334826)
[INFO] [2025-03-06 11:14:47] Storing 10000 Nodes (2119998/1060000/1334826)
[INFO] [2025-03-06 11:15:03] Storing 10000 ScientificNames (2139998/1070000/1334826)
[INFO] [2025-03-06 11:15:06] Storing 10000 Nodes (2139998/1070000/1334826)
[INFO] [2025-03-06 11:15:23] Storing 10000 ScientificNames (2159998/1080000/1334826)
[INFO] [2025-03-06 11:15:26] Storing 10000 Nodes (2159998/1080000/1334826)
[INFO] [2025-03-06 11:15:42] Storing 10000 ScientificNames (2179998/1090000/1334826)
[INFO] [2025-03-06 11:15:45] Storing 10000 Nodes (2179998/1090000/1334826)
[INFO] [2025-03-06 11:16:02] Storing 10000 ScientificNames (2199998/1100000/1334826)
[INFO] [2025-03-06 11:16:05] Storing 10000 Nodes (2199998/1100000/1334826)
[INFO] [2025-03-06 11:16:21] Storing 10000 ScientificNames (2219998/1110000/1334826)
[INFO] [2025-03-06 11:16:24] Storing 10000 Nodes (2219998/1110000/1334826)
[INFO] [2025-03-06 11:16:41] Storing 10000 ScientificNames (2239998/1120000/1334826)
[INFO] [2025-03-06 11:16:44] Storing 10000 Nodes (2239998/1120000/1334826)
[INFO] [2025-03-06 11:17:01] Storing 10000 ScientificNames (2259998/1130000/1334826)
[INFO] [2025-03-06 11:17:04] Storing 10000 Nodes (2259998/1130000/1334826)
[INFO] [2025-03-06 11:17:21] Storing 10000 ScientificNames (2279998/1140000/1334826)
[INFO] [2025-03-06 11:17:24] Storing 10000 Nodes (2279998/1140000/1334826)
[INFO] [2025-03-06 11:17:41] Storing 10000 ScientificNames (2299998/1150000/1334826)
[INFO] [2025-03-06 11:17:44] Storing 10000 Nodes (2299998/1150000/1334826)
[INFO] [2025-03-06 11:18:01] Storing 10000 ScientificNames (2319998/1160000/1334826)
[INFO] [2025-03-06 11:18:04] Storing 10000 Nodes (2319998/1160000/1334826)
[WARN] [2025-03-06 11:18:07] Filtered Scientific Name `Acanthanura ""henderickxi""` to `Acanthanura henderickxi`
[INFO] [2025-03-06 11:18:22] Storing 10000 ScientificNames (2339998/1170000/1334826)
[INFO] [2025-03-06 11:18:25] Storing 10000 Nodes (2339998/1170000/1334826)
[INFO] [2025-03-06 11:18:43] Storing 10000 ScientificNames (2359998/1180000/1334826)
[INFO] [2025-03-06 11:18:46] Storing 10000 Nodes (2359998/1180000/1334826)
[INFO] [2025-03-06 11:19:04] Storing 10000 ScientificNames (2379998/1190000/1334826)
[INFO] [2025-03-06 11:19:07] Storing 10000 Nodes (2379998/1190000/1334826)
[INFO] [2025-03-06 11:19:24] Storing 10000 ScientificNames (2399998/1200000/1334826)
[INFO] [2025-03-06 11:19:27] Storing 10000 Nodes (2399998/1200000/1334826)
[INFO] [2025-03-06 11:19:45] Storing 10000 ScientificNames (2419998/1210000/1334826)
[INFO] [2025-03-06 11:19:48] Storing 10000 Nodes (2419998/1210000/1334826)
[INFO] [2025-03-06 11:20:06] Storing 10000 ScientificNames (2439998/1220000/1334826)
[INFO] [2025-03-06 11:20:09] Storing 10000 Nodes (2439998/1220000/1334826)
[INFO] [2025-03-06 11:20:27] Storing 10000 ScientificNames (2459998/1230000/1334826)
[INFO] [2025-03-06 11:20:30] Storing 10000 Nodes (2459998/1230000/1334826)
[INFO] [2025-03-06 11:20:48] Storing 10000 ScientificNames (2479998/1240000/1334826)
[INFO] [2025-03-06 11:20:51] Storing 10000 Nodes (2479998/1240000/1334826)
[INFO] [2025-03-06 11:21:09] Storing 10000 ScientificNames (2499998/1250000/1334826)
[INFO] [2025-03-06 11:21:12] Storing 10000 Nodes (2499998/1250000/1334826)
[INFO] [2025-03-06 11:21:30] Storing 10000 ScientificNames (2519998/1260000/1334826)
[INFO] [2025-03-06 11:21:33] Storing 10000 Nodes (2519998/1260000/1334826)
[INFO] [2025-03-06 11:21:51] Storing 10000 ScientificNames (2539998/1270000/1334826)
[INFO] [2025-03-06 11:21:54] Storing 10000 Nodes (2539998/1270000/1334826)
[INFO] [2025-03-06 11:22:13] Storing 10000 ScientificNames (2559998/1280000/1334826)
[INFO] [2025-03-06 11:22:16] Storing 10000 Nodes (2559998/1280000/1334826)
[INFO] [2025-03-06 11:22:34] Storing 10000 ScientificNames (2579998/1290000/1334826)
[INFO] [2025-03-06 11:22:37] Storing 10000 Nodes (2579998/1290000/1334826)
[INFO] [2025-03-06 11:22:55] Storing 10000 ScientificNames (2599998/1300000/1334826)
[INFO] [2025-03-06 11:22:59] Storing 10000 Nodes (2599998/1300000/1334826)
[INFO] [2025-03-06 11:23:17] Storing 10000 ScientificNames (2619998/1310000/1334826)
[INFO] [2025-03-06 11:23:20] Storing 10000 Nodes (2619998/1310000/1334826)
[INFO] [2025-03-06 11:23:39] Storing 10000 ScientificNames (2639998/1320000/1334826)
[INFO] [2025-03-06 11:23:42] Storing 10000 Nodes (2639998/1320000/1334826)
[INFO] [2025-03-06 11:24:00] Storing 10000 ScientificNames (2659998/1330000/1334826)
[INFO] [2025-03-06 11:24:03] Storing 10000 Nodes (2659998/1330000/1334826)
[INFO] [2025-03-06 11:24:20] Storing 4825 ScientificNames (2669648/1334824/1334826)
[INFO] [2025-03-06 11:24:22] Storing 4825 Nodes (2669648/1334824/1334826)
[INFO] [2025-03-06 11:24:24] Handling diff: /app/public/data/i_nat_taxonomy_v/diff/i_nat_taxonomy_v_vernaculars_31228.diff (1324060 lines)
[INFO] [2025-03-06 11:24:24] Loading vernaculars diff file into memory (1324060 lines)...
[INFO] [2025-03-06 11:24:25] Storing 9999 Vernaculars (9999/10000/1324060)
[INFO] [2025-03-06 11:24:28] Storing 10000 Vernaculars (19999/20000/1324060)
[INFO] [2025-03-06 11:24:31] Storing 10000 Vernaculars (29999/30000/1324060)
[INFO] [2025-03-06 11:24:35] Storing 10000 Vernaculars (39999/40000/1324060)
[INFO] [2025-03-06 11:24:37] Storing 10000 Vernaculars (49999/50000/1324060)
[INFO] [2025-03-06 11:24:40] Storing 10000 Vernaculars (59999/60000/1324060)
[INFO] [2025-03-06 11:24:43] Storing 10000 Vernaculars (69999/70000/1324060)
[INFO] [2025-03-06 11:24:46] Storing 10000 Vernaculars (79999/80000/1324060)
[INFO] [2025-03-06 11:24:48] Storing 10000 Vernaculars (89999/90000/1324060)
[INFO] [2025-03-06 11:24:51] Storing 10000 Vernaculars (99999/100000/1324060)
[INFO] [2025-03-06 11:24:54] Storing 10000 Vernaculars (109999/110000/1324060)
[INFO] [2025-03-06 11:24:56] Storing 10000 Vernaculars (119999/120000/1324060)
[INFO] [2025-03-06 11:24:59] Storing 10000 Vernaculars (129999/130000/1324060)
[INFO] [2025-03-06 11:25:02] Storing 10000 Vernaculars (139999/140000/1324060)
[INFO] [2025-03-06 11:25:05] Storing 10000 Vernaculars (149999/150000/1324060)
[INFO] [2025-03-06 11:25:08] Storing 10000 Vernaculars (159999/160000/1324060)
[INFO] [2025-03-06 11:25:10] Storing 10000 Vernaculars (169999/170000/1324060)
[INFO] [2025-03-06 11:25:13] Storing 10000 Vernaculars (179999/180000/1324060)
[INFO] [2025-03-06 11:25:16] Storing 10000 Vernaculars (189999/190000/1324060)
[INFO] [2025-03-06 11:25:19] Storing 10000 Vernaculars (199999/200000/1324060)
[INFO] [2025-03-06 11:25:22] Storing 10000 Vernaculars (209999/210000/1324060)
[INFO] [2025-03-06 11:25:24] Storing 10000 Vernaculars (219999/220000/1324060)
[INFO] [2025-03-06 11:25:27] Storing 10000 Vernaculars (229999/230000/1324060)
[INFO] [2025-03-06 11:25:30] Storing 10000 Vernaculars (239999/240000/1324060)
[INFO] [2025-03-06 11:25:33] Storing 10000 Vernaculars (249999/250000/1324060)
[INFO] [2025-03-06 11:25:36] Storing 10000 Vernaculars (259999/260000/1324060)
[INFO] [2025-03-06 11:25:38] Storing 10000 Vernaculars (269999/270000/1324060)
[INFO] [2025-03-06 11:25:41] Storing 10000 Vernaculars (279999/280000/1324060)
[INFO] [2025-03-06 11:25:44] Storing 10000 Vernaculars (289999/290000/1324060)
[INFO] [2025-03-06 11:25:47] Storing 10000 Vernaculars (299999/300000/1324060)
[INFO] [2025-03-06 11:25:49] Storing 10000 Vernaculars (309999/310000/1324060)
[INFO] [2025-03-06 11:25:52] Storing 10000 Vernaculars (319999/320000/1324060)
[INFO] [2025-03-06 11:25:55] Storing 10000 Vernaculars (329999/330000/1324060)
[INFO] [2025-03-06 11:25:58] Storing 10000 Vernaculars (339999/340000/1324060)
[INFO] [2025-03-06 11:26:01] Storing 10000 Vernaculars (349999/350000/1324060)
[INFO] [2025-03-06 11:26:03] Storing 10000 Vernaculars (359999/360000/1324060)
[INFO] [2025-03-06 11:26:06] Storing 10000 Vernaculars (369999/370000/1324060)
[INFO] [2025-03-06 11:26:09] Storing 10000 Vernaculars (379999/380000/1324060)
[INFO] [2025-03-06 11:26:12] Storing 10000 Vernaculars (389999/390000/1324060)
[INFO] [2025-03-06 11:26:15] Storing 10000 Vernaculars (399999/400000/1324060)
[INFO] [2025-03-06 11:26:18] Storing 10000 Vernaculars (409999/410000/1324060)
[INFO] [2025-03-06 11:26:20] Storing 10000 Vernaculars (419999/420000/1324060)
[INFO] [2025-03-06 11:26:23] Storing 10000 Vernaculars (429999/430000/1324060)
[INFO] [2025-03-06 11:26:26] Storing 10000 Vernaculars (439999/440000/1324060)
[INFO] [2025-03-06 11:26:29] Storing 10000 Vernaculars (449999/450000/1324060)
[INFO] [2025-03-06 11:26:32] Storing 10000 Vernaculars (459999/460000/1324060)
[INFO] [2025-03-06 11:26:34] Storing 10000 Vernaculars (469999/470000/1324060)
[INFO] [2025-03-06 11:26:37] Storing 10000 Vernaculars (479999/480000/1324060)
[INFO] [2025-03-06 11:26:40] Storing 10000 Vernaculars (489999/490000/1324060)
[INFO] [2025-03-06 11:26:43] Storing 10000 Vernaculars (499999/500000/1324060)
[INFO] [2025-03-06 11:26:46] Storing 10000 Vernaculars (509999/510000/1324060)
[INFO] [2025-03-06 11:26:48] Storing 10000 Vernaculars (519999/520000/1324060)
[INFO] [2025-03-06 11:26:51] Storing 10000 Vernaculars (529999/530000/1324060)
[INFO] [2025-03-06 11:26:54] Storing 10000 Vernaculars (539999/540000/1324060)
[INFO] [2025-03-06 11:26:57] Storing 10000 Vernaculars (549999/550000/1324060)
[INFO] [2025-03-06 11:27:00] Storing 10000 Vernaculars (559999/560000/1324060)
[INFO] [2025-03-06 11:27:03] Storing 10000 Vernaculars (569999/570000/1324060)
[INFO] [2025-03-06 11:27:05] Storing 10000 Vernaculars (579999/580000/1324060)
[INFO] [2025-03-06 11:27:08] Storing 10000 Vernaculars (589999/590000/1324060)
[INFO] [2025-03-06 11:27:11] Storing 10000 Vernaculars (599999/600000/1324060)
[INFO] [2025-03-06 11:27:14] Storing 10000 Vernaculars (609999/610000/1324060)
[INFO] [2025-03-06 11:27:17] Storing 10000 Vernaculars (619999/620000/1324060)
[INFO] [2025-03-06 11:27:20] Storing 10000 Vernaculars (629999/630000/1324060)
[INFO] [2025-03-06 11:27:23] Storing 10000 Vernaculars (639999/640000/1324060)
[INFO] [2025-03-06 11:27:25] Storing 10000 Vernaculars (649999/650000/1324060)
[INFO] [2025-03-06 11:27:28] Storing 10000 Vernaculars (659999/660000/1324060)
[INFO] [2025-03-06 11:27:31] Storing 10000 Vernaculars (669999/670000/1324060)
[INFO] [2025-03-06 11:27:34] Storing 10000 Vernaculars (679999/680000/1324060)
[INFO] [2025-03-06 11:27:37] Storing 10000 Vernaculars (689999/690000/1324060)
[INFO] [2025-03-06 11:27:40] Storing 10000 Vernaculars (699999/700000/1324060)
[INFO] [2025-03-06 11:27:42] Storing 10000 Vernaculars (709999/710000/1324060)
[INFO] [2025-03-06 11:27:45] Storing 10000 Vernaculars (719999/720000/1324060)
[INFO] [2025-03-06 11:27:48] Storing 10000 Vernaculars (729999/730000/1324060)
[INFO] [2025-03-06 11:27:51] Storing 10000 Vernaculars (739999/740000/1324060)
[INFO] [2025-03-06 11:27:54] Storing 10000 Vernaculars (749999/750000/1324060)
[INFO] [2025-03-06 11:27:56] Storing 10000 Vernaculars (759999/760000/1324060)
[INFO] [2025-03-06 11:28:00] Storing 10000 Vernaculars (769999/770000/1324060)
[INFO] [2025-03-06 11:28:02] Storing 10000 Vernaculars (779999/780000/1324060)
[INFO] [2025-03-06 11:28:05] Storing 10000 Vernaculars (789999/790000/1324060)
[INFO] [2025-03-06 11:28:08] Storing 10000 Vernaculars (799999/800000/1324060)
[INFO] [2025-03-06 11:28:11] Storing 10000 Vernaculars (809999/810000/1324060)
[INFO] [2025-03-06 11:28:13] Storing 10000 Vernaculars (819999/820000/1324060)
[INFO] [2025-03-06 11:28:16] Storing 10000 Vernaculars (829999/830000/1324060)
[INFO] [2025-03-06 11:28:19] Storing 10000 Vernaculars (839999/840000/1324060)
[INFO] [2025-03-06 11:28:22] Storing 10000 Vernaculars (849999/850000/1324060)
[INFO] [2025-03-06 11:28:25] Storing 10000 Vernaculars (859999/860000/1324060)
[INFO] [2025-03-06 11:28:28] Storing 10000 Vernaculars (869999/870000/1324060)
[INFO] [2025-03-06 11:28:31] Storing 10000 Vernaculars (879999/880000/1324060)
[INFO] [2025-03-06 11:28:34] Storing 10000 Vernaculars (889999/890000/1324060)
[INFO] [2025-03-06 11:28:36] Storing 10000 Vernaculars (899999/900000/1324060)
[INFO] [2025-03-06 11:28:39] Storing 10000 Vernaculars (909999/910000/1324060)
[INFO] [2025-03-06 11:28:42] Storing 10000 Vernaculars (919999/920000/1324060)
[INFO] [2025-03-06 11:28:45] Storing 10000 Vernaculars (929999/930000/1324060)
[INFO] [2025-03-06 11:28:48] Storing 10000 Vernaculars (939999/940000/1324060)
[INFO] [2025-03-06 11:28:50] Storing 10000 Vernaculars (949999/950000/1324060)
[INFO] [2025-03-06 11:28:53] Storing 10000 Vernaculars (959999/960000/1324060)
[INFO] [2025-03-06 11:28:56] Storing 10000 Vernaculars (969999/970000/1324060)
[INFO] [2025-03-06 11:28:59] Storing 10000 Vernaculars (979999/980000/1324060)
[INFO] [2025-03-06 11:29:02] Storing 10000 Vernaculars (989999/990000/1324060)
[INFO] [2025-03-06 11:29:05] Storing 10000 Vernaculars (999999/1000000/1324060)
[INFO] [2025-03-06 11:29:08] Storing 10000 Vernaculars (1009999/1010000/1324060)
[INFO] [2025-03-06 11:29:10] Storing 10000 Vernaculars (1019999/1020000/1324060)
[INFO] [2025-03-06 11:29:13] Storing 10000 Vernaculars (1029999/1030000/1324060)
[INFO] [2025-03-06 11:29:16] Storing 10000 Vernaculars (1039999/1040000/1324060)
[INFO] [2025-03-06 11:29:19] Storing 10000 Vernaculars (1049999/1050000/1324060)
[INFO] [2025-03-06 11:29:22] Storing 10000 Vernaculars (1059999/1060000/1324060)
[INFO] [2025-03-06 11:29:24] Storing 10000 Vernaculars (1069999/1070000/1324060)
[INFO] [2025-03-06 11:29:27] Storing 10000 Vernaculars (1079999/1080000/1324060)
[INFO] [2025-03-06 11:29:30] Storing 10000 Vernaculars (1089999/1090000/1324060)
[INFO] [2025-03-06 11:29:33] Storing 10000 Vernaculars (1099999/1100000/1324060)
[INFO] [2025-03-06 11:29:36] Storing 10000 Vernaculars (1109999/1110000/1324060)
[INFO] [2025-03-06 11:29:39] Storing 10000 Vernaculars (1119999/1120000/1324060)
[INFO] [2025-03-06 11:29:41] Storing 10000 Vernaculars (1129999/1130000/1324060)
[INFO] [2025-03-06 11:29:44] Storing 10000 Vernaculars (1139999/1140000/1324060)
[INFO] [2025-03-06 11:29:47] Storing 10000 Vernaculars (1149999/1150000/1324060)
[INFO] [2025-03-06 11:29:50] Storing 10000 Vernaculars (1159999/1160000/1324060)
[INFO] [2025-03-06 11:29:53] Storing 10000 Vernaculars (1169999/1170000/1324060)
[INFO] [2025-03-06 11:29:56] Storing 10000 Vernaculars (1179999/1180000/1324060)
[INFO] [2025-03-06 11:29:59] Storing 10000 Vernaculars (1189999/1190000/1324060)
[INFO] [2025-03-06 11:30:01] Storing 10000 Vernaculars (1199999/1200000/1324060)
[INFO] [2025-03-06 11:30:05] Storing 10000 Vernaculars (1209999/1210000/1324060)
[INFO] [2025-03-06 11:30:07] Storing 10000 Vernaculars (1219999/1220000/1324060)
[INFO] [2025-03-06 11:30:10] Storing 10000 Vernaculars (1229999/1230000/1324060)
[INFO] [2025-03-06 11:30:13] Storing 10000 Vernaculars (1239999/1240000/1324060)
[INFO] [2025-03-06 11:30:16] Storing 10000 Vernaculars (1249999/1250000/1324060)
[INFO] [2025-03-06 11:30:19] Storing 10000 Vernaculars (1259999/1260000/1324060)
[INFO] [2025-03-06 11:30:22] Storing 10000 Vernaculars (1269999/1270000/1324060)
[INFO] [2025-03-06 11:30:24] Storing 10000 Vernaculars (1279999/1280000/1324060)
[INFO] [2025-03-06 11:30:27] Storing 10000 Vernaculars (1289999/1290000/1324060)
[INFO] [2025-03-06 11:30:30] Storing 10000 Vernaculars (1299999/1300000/1324060)
[INFO] [2025-03-06 11:30:33] Storing 10000 Vernaculars (1309999/1310000/1324060)
[INFO] [2025-03-06 11:30:36] Storing 10000 Vernaculars (1319999/1320000/1324060)
[INFO] [2025-03-06 11:30:38] Storing 4059 Vernaculars (1324058/1324058/1324060)
[STOP] [2025-03-06 11:30:39] parse_diff_and_store
[START] [2025-03-06 11:30:39] resolve_keys
[2025-03-06 11:38:22] Resolving downloaded urls (this is not actually downloading them yet)
[INFO] [2025-03-06 11:38:32] Occurrences to nodes (through scientific_names)...
[INFO] [2025-03-06 11:38:32] traits to occurrences...
[INFO] [2025-03-06 11:38:32] traits to nodes (through occurrences)...
[INFO] [2025-03-06 11:38:32] Traits to sex term...
[INFO] [2025-03-06 11:38:32] Traits to lifestage term...
[INFO] [2025-03-06 11:38:32] MetaTraits to traits...
[INFO] [2025-03-06 11:38:32] MetaTraits (simple, measurement row refers to parent) to traits...
[INFO] [2025-03-06 11:38:32] Assocs to occurrences...
[INFO] [2025-03-06 11:38:32] Assocs to nodes...
[INFO] [2025-03-06 11:38:32] Assoc to sex term...
[INFO] [2025-03-06 11:38:32] Assoc to lifestage term...
[INFO] [2025-03-06 11:38:32] MetaAssoc to assocs...
[STOP] [2025-03-06 11:38:32] resolve_keys
[START] [2025-03-06 11:38:32] hold_for_later_1
[STOP] [2025-03-06 11:38:32] hold_for_later_1
[START] [2025-03-06 11:38:32] hold_for_later_2
[STOP] [2025-03-06 11:38:32] hold_for_later_2
[START] [2025-03-06 11:38:32] resolve_missing_parents
[STOP] [2025-03-06 11:38:46] resolve_missing_parents
[START] [2025-03-06 11:38:46] rebuild_nodes
[START] [2025-03-06 11:38:46] Flattener#flatten
[START] [2025-03-06 11:38:46] Flattener#study_resource
[START] [2025-03-06 11:40:12] Flattener#build_ancestry
[STOP] [2025-03-06 12:47:08] Flattener#build_ancestry
[INFO] [2025-03-06 12:47:08] 1334824 ancestry keys
[START] [2025-03-06 12:47:08] build_node_ancestors
[INFO] [2025-03-06 12:47:08] old ancestors deleted.
[STOP] [2025-03-06 13:14:11] build_node_ancestors
[START] [2025-03-06 13:14:15] Flattener#propagate_ancestor_ids
[STOP] [2025-03-06 13:26:35] Flattener#propagate_ancestor_ids
[STOP] [2025-03-06 13:26:35] Flattener#flatten
[STOP] [2025-03-06 13:26:35] rebuild_nodes
[START] [2025-03-06 13:26:35] resolve_missing_media_owners
[STOP] [2025-03-06 13:26:35] resolve_missing_media_owners
[START] [2025-03-06 13:26:35] sanitize_media_verbatims
[STOP] [2025-03-06 13:26:35] sanitize_media_verbatims
[START] [2025-03-06 13:26:35] queue_downloads
[STOP] [2025-03-06 13:26:35] queue_downloads
[START] [2025-03-06 13:26:35] parse_names
[WARN] [2025-03-06 13:26:38] I see 1334824 names which still need to be parsed.
[WARN] [2025-03-06 13:26:39] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2025-03-06 13:26:49] Names to parse: 10000 formatted: 10000 learned: 9997 parsed: 10000
[WARN] [2025-03-06 13:26:58] Names to parse: 10000 formatted: 10000 learned: 9998 parsed: 10000
[WARN] [2025-03-06 13:27:08] Names to parse: 10000 formatted: 10000 learned: 9998 parsed: 10000
[WARN] [2025-03-06 13:27:17] Names to parse: 10000 formatted: 10000 learned: 9991 parsed: 10000
[WARN] [2025-03-06 13:27:26] Names to parse: 10000 formatted: 10000 learned: 9997 parsed: 10000
[WARN] [2025-03-06 13:27:36] Names to parse: 10000 formatted: 10000 learned: 9994 parsed: 10000
[WARN] [2025-03-06 13:27:44] Names to parse: 10000 formatted: 10000 learned: 9998 parsed: 10000
[WARN] [2025-03-06 13:27:53] Names to parse: 10000 formatted: 10000 learned: 9996 parsed: 10000
[WARN] [2025-03-06 13:28:02] Names to parse: 10000 formatted: 10000 learned: 9994 parsed: 10000
[WARN] [2025-03-06 13:28:11] Names to parse: 10000 formatted: 10000 learned: 9990 parsed: 10000
[WARN] [2025-03-06 13:28:19] Names to parse: 10000 formatted: 10000 learned: 9989 parsed: 10000
[WARN] [2025-03-06 13:28:28] Names to parse: 10000 formatted: 10000 learned: 9994 parsed: 10000
[WARN] [2025-03-06 13:28:36] Names to parse: 10000 formatted: 10000 learned: 9985 parsed: 10000
[WARN] [2025-03-06 13:28:45] Names to parse: 10000 formatted: 10000 learned: 9972 parsed: 10000
[WARN] [2025-03-06 13:28:54] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2025-03-06 13:29:03] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2025-03-06 13:29:11] Names to parse: 10000 formatted: 10000 learned: 9995 parsed: 10000
[WARN] [2025-03-06 13:29:19] Names to parse: 10000 formatted: 10000 learned: 9997 parsed: 10000
[WARN] [2025-03-06 13:29:28] Names to parse: 10000 formatted: 10000 learned: 9998 parsed: 10000
[WARN] [2025-03-06 13:29:36] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2025-03-06 13:29:45] Names to parse: 10000 formatted: 10000 learned: 9996 parsed: 10000
[WARN] [2025-03-06 13:29:53] Names to parse: 10000 formatted: 10000 learned: 9974 parsed: 10000
[WARN] [2025-03-06 13:30:01] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2025-03-06 13:30:09] Names to parse: 10000 formatted: 10000 learned: 9995 parsed: 10000
[WARN] [2025-03-06 13:30:17] Names to parse: 10000 formatted: 10000 learned: 9998 parsed: 10000
[WARN] [2025-03-06 13:30:26] Names to parse: 10000 formatted: 10000 learned: 9998 parsed: 10000
[WARN] [2025-03-06 13:30:34] Names to parse: 10000 formatted: 10000 learned: 9975 parsed: 10000
[WARN] [2025-03-06 13:30:42] Names to parse: 10000 formatted: 10000 learned: 9994 parsed: 10000
[WARN] [2025-03-06 13:30:49] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2025-03-06 13:30:58] Names to parse: 10000 formatted: 10000 learned: 9997 parsed: 10000
[WARN] [2025-03-06 13:31:06] Names to parse: 10000 formatted: 10000 learned: 9997 parsed: 10000
[WARN] [2025-03-06 13:31:14] Names to parse: 10000 formatted: 10000 learned: 9988 parsed: 10000
[WARN] [2025-03-06 13:31:22] Names to parse: 10000 formatted: 10000 learned: 9989 parsed: 10000
[WARN] [2025-03-06 13:31:30] Names to parse: 10000 formatted: 10000 learned: 9984 parsed: 10000
[WARN] [2025-03-06 13:31:38] Names to parse: 10000 formatted: 10000 learned: 9993 parsed: 10000
[WARN] [2025-03-06 13:31:46] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2025-03-06 13:31:53] Names to parse: 10000 formatted: 10000 learned: 9984 parsed: 10000
[WARN] [2025-03-06 13:32:01] Names to parse: 10000 formatted: 10000 learned: 9989 parsed: 10000
[WARN] [2025-03-06 13:32:09] Names to parse: 10000 formatted: 10000 learned: 9973 parsed: 10000
[WARN] [2025-03-06 13:32:17] Names to parse: 10000 formatted: 10000 learned: 9976 parsed: 10000
[WARN] [2025-03-06 13:32:25] Names to parse: 10000 formatted: 10000 learned: 9960 parsed: 10000
[WARN] [2025-03-06 13:32:33] Names to parse: 10000 formatted: 10000 learned: 9971 parsed: 10000
[WARN] [2025-03-06 13:32:41] Names to parse: 10000 formatted: 10000 learned: 9976 parsed: 10000
[WARN] [2025-03-06 13:32:49] Names to parse: 10000 formatted: 10000 learned: 9981 parsed: 10000
[WARN] [2025-03-06 13:32:57] Names to parse: 10000 formatted: 10000 learned: 9983 parsed: 10000
[WARN] [2025-03-06 13:33:05] Names to parse: 10000 formatted: 10000 learned: 9993 parsed: 10000
[WARN] [2025-03-06 13:33:13] Names to parse: 10000 formatted: 10000 learned: 9983 parsed: 10000
[WARN] [2025-03-06 13:33:21] Names to parse: 10000 formatted: 10000 learned: 9995 parsed: 10000
[WARN] [2025-03-06 13:33:30] Names to parse: 10000 formatted: 10000 learned: 9988 parsed: 10000
[WARN] [2025-03-06 13:33:37] Names to parse: 10000 formatted: 10000 learned: 9989 parsed: 10000
[WARN] [2025-03-06 13:33:45] Names to parse: 10000 formatted: 10000 learned: 9976 parsed: 10000
[WARN] [2025-03-06 13:33:53] Names to parse: 10000 formatted: 10000 learned: 9982 parsed: 10000
[WARN] [2025-03-06 13:34:00] Names to parse: 10000 formatted: 10000 learned: 9991 parsed: 10000
[WARN] [2025-03-06 13:34:08] Names to parse: 10000 formatted: 10000 learned: 9979 parsed: 10000
[WARN] [2025-03-06 13:34:16] Names to parse: 10000 formatted: 10000 learned: 9927 parsed: 10000
[WARN] [2025-03-06 13:34:24] Names to parse: 10000 formatted: 10000 learned: 9956 parsed: 10000
[WARN] [2025-03-06 13:34:32] Names to parse: 10000 formatted: 10000 learned: 9987 parsed: 10000
[WARN] [2025-03-06 13:34:40] Names to parse: 10000 formatted: 10000 learned: 9988 parsed: 10000
[WARN] [2025-03-06 13:34:48] Names to parse: 10000 formatted: 10000 learned: 9941 parsed: 10000
[WARN] [2025-03-06 13:34:56] Names to parse: 10000 formatted: 10000 learned: 9997 parsed: 10000
[WARN] [2025-03-06 13:35:04] Names to parse: 10000 formatted: 10000 learned: 9994 parsed: 10000
[WARN] [2025-03-06 13:35:11] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2025-03-06 13:35:19] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2025-03-06 13:35:26] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2025-03-06 13:35:34] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2025-03-06 13:35:41] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2025-03-06 13:35:50] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2025-03-06 13:35:57] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2025-03-06 13:36:05] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2025-03-06 13:36:12] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2025-03-06 13:36:20] Names to parse: 10000 formatted: 10000 learned: 9996 parsed: 10000
[WARN] [2025-03-06 13:36:27] Names to parse: 10000 formatted: 10000 learned: 9997 parsed: 10000
[WARN] [2025-03-06 13:36:34] Names to parse: 10000 formatted: 10000 learned: 9998 parsed: 10000
[WARN] [2025-03-06 13:36:42] Names to parse: 10000 formatted: 10000 learned: 9996 parsed: 10000
[WARN] [2025-03-06 13:36:50] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2025-03-06 13:36:58] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2025-03-06 13:37:05] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2025-03-06 13:37:12] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2025-03-06 13:37:20] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2025-03-06 13:37:27] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2025-03-06 13:37:35] Names to parse: 10000 formatted: 10000 learned: 9997 parsed: 10000
[WARN] [2025-03-06 13:37:42] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2025-03-06 13:37:50] Names to parse: 10000 formatted: 10000 learned: 9991 parsed: 10000
[WARN] [2025-03-06 13:37:57] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2025-03-06 13:38:05] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2025-03-06 13:38:12] Names to parse: 10000 formatted: 10000 learned: 9997 parsed: 10000
[WARN] [2025-03-06 13:38:19] Names to parse: 10000 formatted: 10000 learned: 9994 parsed: 10000
[WARN] [2025-03-06 13:38:27] Names to parse: 10000 formatted: 10000 learned: 9995 parsed: 10000
[WARN] [2025-03-06 13:38:34] Names to parse: 10000 formatted: 10000 learned: 9997 parsed: 10000
[WARN] [2025-03-06 13:38:42] Names to parse: 10000 formatted: 10000 learned: 9995 parsed: 10000
[WARN] [2025-03-06 13:38:49] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2025-03-06 13:38:57] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2025-03-06 13:39:04] Names to parse: 10000 formatted: 10000 learned: 9997 parsed: 10000
[WARN] [2025-03-06 13:39:11] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2025-03-06 13:39:19] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2025-03-06 13:39:26] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2025-03-06 13:39:34] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2025-03-06 13:39:41] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2025-03-06 13:39:49] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2025-03-06 13:39:56] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2025-03-06 13:40:04] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2025-03-06 13:40:12] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2025-03-06 13:40:19] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2025-03-06 13:40:27] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2025-03-06 13:40:34] Names to parse: 10000 formatted: 10000 learned: 9998 parsed: 10000
[WARN] [2025-03-06 13:40:41] Names to parse: 10000 formatted: 10000 learned: 9994 parsed: 10000
[WARN] [2025-03-06 13:40:49] Names to parse: 10000 formatted: 10000 learned: 9989 parsed: 10000
[WARN] [2025-03-06 13:40:56] Names to parse: 10000 formatted: 10000 learned: 9995 parsed: 10000
[WARN] [2025-03-06 13:41:04] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2025-03-06 13:41:11] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2025-03-06 13:41:18] Names to parse: 10000 formatted: 10000 learned: 9950 parsed: 10000
[WARN] [2025-03-06 13:41:25] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2025-03-06 13:41:33] Names to parse: 10000 formatted: 10000 learned: 9996 parsed: 10000
[WARN] [2025-03-06 13:41:40] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2025-03-06 13:41:48] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2025-03-06 13:41:55] Names to parse: 10000 formatted: 10000 learned: 9998 parsed: 10000
[WARN] [2025-03-06 13:42:02] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2025-03-06 13:42:10] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2025-03-06 13:42:17] Names to parse: 10000 formatted: 10000 learned: 9998 parsed: 10000
[WARN] [2025-03-06 13:42:24] Names to parse: 10000 formatted: 10000 learned: 9996 parsed: 10000
[WARN] [2025-03-06 13:42:32] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2025-03-06 13:42:39] Names to parse: 10000 formatted: 10000 learned: 9996 parsed: 10000
[WARN] [2025-03-06 13:42:46] Names to parse: 10000 formatted: 10000 learned: 9998 parsed: 10000
[WARN] [2025-03-06 13:42:54] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2025-03-06 13:43:01] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2025-03-06 13:43:08] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2025-03-06 13:43:16] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2025-03-06 13:43:23] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2025-03-06 13:43:30] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2025-03-06 13:43:38] Names to parse: 10000 formatted: 10000 learned: 9989 parsed: 10000
[WARN] [2025-03-06 13:43:45] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2025-03-06 13:43:53] Names to parse: 10000 formatted: 10000 learned: 9998 parsed: 10000
[WARN] [2025-03-06 13:43:59] Names to parse: 4824 formatted: 4824 learned: 4814 parsed: 4824
[STOP] [2025-03-06 13:44:05] parse_names
[START] [2025-03-06 13:44:05] denormalize_canonical_names_to_nodes
[STOP] [2025-03-06 13:45:09] denormalize_canonical_names_to_nodes
[START] [2025-03-06 13:45:09] match_nodes
[START] [2025-03-06 13:45:12] map_all_nodes_to_pages
[STOP] [2025-03-06 20:26:55] map_all_nodes_to_pages
[INFO] [2025-03-06 20:26:55] 108359 Unmatched nodes (of 1334824)! That's too many to output. Full list in /app/public/data/i_nat_taxonomy_v/unmatched_nodes.txt ; First 10: Canonical: Pellenattus peninsularis; Node#170068971; ResourceID: https://www.inaturalist.org/taxa/1562418; Canonical: Kochiproductus afghanus; Node#170403189; ResourceID: https://www.inaturalist.org/taxa/527878; Canonical: Paucispinifera magnispina; Node#170410890; ResourceID: https://www.inaturalist.org/taxa/536092; Canonical: Aseptella satunensis; Node#170814148; ResourceID: https://www.inaturalist.org/taxa/961250; Canonical: Magadania; Node#170725986; ResourceID: https://www.inaturalist.org/taxa/869034; Canonical: Acrothele; Node#170244028; ResourceID: https://www.inaturalist.org/taxa/342415; Canonical: Acrothele olenekensis; Node#170244029; ResourceID: https://www.inaturalist.org/taxa/342416; Canonical: Acrothele aculeata; Node#170244030; ResourceID: https://www.inaturalist.org/taxa/342417; Canonical: Acrothele pileolus; Node#170244033; ResourceID: https://www.inaturalist.org/taxa/342420; Canonical: Lingulella; Node#170726120; ResourceID: https://www.inaturalist.org/taxa/869176
[START] [2025-03-06 20:26:55] update_nodes
[STOP] [2025-03-06 20:27:22] update_nodes
[STOP] [2025-03-06 20:27:22] match_nodes
[START] [2025-03-06 20:27:22] reindex_search
[STOP] [2025-03-06 20:58:32] reindex_search
[START] [2025-03-06 20:58:32] normalize_units
[STOP] [2025-03-06 20:58:32] normalize_units
[START] [2025-03-06 20:58:32] calculate_statistics
[INFO] [2025-03-06 21:08:32] Duplicate page_id count: 2
[STOP] [2025-03-06 21:08:32] calculate_statistics
[START] [2025-03-06 21:08:32] complete_harvest_instance
[START] [2025-03-06 21:08:32] overall_tsv_creation
[INFO] [2025-03-06 21:08:33] Exporting 1334824 nodes as TSV in batches of 10000...
[INFO] [2025-03-06 21:08:33] Processing group of 1334824 in 134 batches of 10000
[INFO] [2025-03-06 21:11:06] Processed 10000/1334824 nodes
[INFO] [2025-03-06 21:13:36] Processed 20000/1334824 nodes
[INFO] [2025-03-06 21:16:02] Processed 30000/1334824 nodes
[2025-03-06 21:18:19] Encountered new language, please assign it to a Locale and give it a name: myn
[INFO] [2025-03-06 21:18:19] Average Time: 18.487
[INFO] [2025-03-06 21:18:19] Total Time: 9m47s
[STOP] [2025-03-06 21:18:19] overall_tsv_creation
[STOP] [2025-03-06 21:18:19] complete_harvest_instance
[ERR] [2025-03-06 21:18:19] RuntimeError
[ERR] [2025-03-06 21:18:19] Exception of class ActiveRecord::StatementInvalid, you should rescue those here.
[ERR] [2025-03-06 21:18:19] ../models/web_db.rb:122:in `rescue in raw_create'
[ERR] [2025-03-06 21:18:19] ../models/web_db.rb:119:in `raw_create'
[ERR] [2025-03-06 21:18:19] ../models/web_db.rb:86:in `language'
[ERR] [2025-03-06 21:18:19] ../models/publisher.rb:733:in `build_vernacular'
[ERR] [2025-03-06 21:18:19] ../models/publisher.rb:432:in `block in build_vernaculars'
[ERR] [2025-03-06 21:18:19] ../models/publisher.rb:430:in `build_vernaculars'
[ERR] [2025-03-06 21:18:19] ../models/publisher.rb:166:in `block in nodes_to_hashes'
[ERR] [2025-03-06 21:18:19] ../models/publisher.rb:156:in `each'
[ERR] [2025-03-06 21:18:19] ../models/publisher.rb:156:in `nodes_to_hashes'
[ERR] [2025-03-06 21:18:19] ../models/publisher.rb:127:in `block in create_tsv'
[ERR] [2025-03-06 21:18:19] ../models/logged_process.rb:76:in `block in in_batches'
[ERR] [2025-03-06 21:18:19] ../models/logged_process.rb:74:in `in_batches'
[ERR] [2025-03-06 21:18:19] ../models/publisher.rb:125:in `create_tsv'
[ERR] [2025-03-06 21:18:19] ../models/publisher.rb:82:in `block in by_resource'
[ERR] [2025-03-06 21:18:19] ../models/logged_process.rb:43:in `run_step'
[ERR] [2025-03-06 21:18:19] ../models/publisher.rb:79:in `by_resource'
[ERR] [2025-03-06 21:18:19] ../models/publisher.rb:23:in `by_resource'
[ERR] [2025-03-06 21:18:19] ../models/resource_harvester.rb:723:in `complete_harvest_instance'
[ERR] [2025-03-06 21:18:19] ../models/resource_harvester.rb:86:in `block (2 levels) in start'
[ERR] [2025-03-06 21:18:19] ../models/logged_process.rb:43:in `run_step'
[ERR] [2025-03-06 21:18:19] ../models/resource_harvester.rb:86:in `block in start'
[ERR] [2025-03-06 21:18:19] ../models/resource_harvester.rb:75:in `each_key'
[ERR] [2025-03-06 21:18:19] ../models/resource_harvester.rb:75:in `start'
[ERR] [2025-03-06 21:18:19] ../models/resource.rb:305:in `harvest'
[ERR] [2025-03-06 21:18:19] ../models/resource.rb:281:in `re_download_opendata_and_harvest'
[ERR] [2025-03-06 21:18:19] bin/rails:4:in `require'
[ERR] [2025-03-06 21:18:19] bin/rails:4:in `<main>'
[STOP] [2025-03-06 21:18:19] logged process, took 37731.1
[INFO] [2025-03-10 14:35:50] ## HARVEST: type = resume_harvest
[START] [2025-03-10 14:35:53] logged process: 3ed880c974c94c96ddaa2af0d10dce98429c2bea Undo a "recommended" change that doesn't work
[INFO] [2025-03-10 14:35:53] Already completed stage create_harvest_instance, skipping...
[INFO] [2025-03-10 14:35:53] Already completed stage fetch_files, skipping...
[INFO] [2025-03-10 14:35:53] Already completed stage validate_each_file, skipping...
[INFO] [2025-03-10 14:35:53] Already completed stage convert_to_csv, skipping...
[INFO] [2025-03-10 14:35:53] Already completed stage calculate_delta, skipping...
[INFO] [2025-03-10 14:35:53] Already completed stage parse_diff_and_store, skipping...
[INFO] [2025-03-10 14:35:53] Already completed stage resolve_keys, skipping...
[INFO] [2025-03-10 14:35:53] Already completed stage hold_for_later_1, skipping...
[INFO] [2025-03-10 14:35:53] Already completed stage hold_for_later_2, skipping...
[INFO] [2025-03-10 14:35:53] Already completed stage resolve_missing_parents, skipping...
[INFO] [2025-03-10 14:35:53] Already completed stage rebuild_nodes, skipping...
[INFO] [2025-03-10 14:35:53] Already completed stage resolve_missing_media_owners, skipping...
[INFO] [2025-03-10 14:35:53] Already completed stage sanitize_media_verbatims, skipping...
[INFO] [2025-03-10 14:35:53] Already completed stage queue_downloads, skipping...
[INFO] [2025-03-10 14:35:53] Already completed stage parse_names, skipping...
[INFO] [2025-03-10 14:35:53] Already completed stage denormalize_canonical_names_to_nodes, skipping...
[INFO] [2025-03-10 14:35:53] Already completed stage match_nodes, skipping...
[INFO] [2025-03-10 14:35:53] Already completed stage reindex_search, skipping...
[INFO] [2025-03-10 14:35:53] Already completed stage normalize_units, skipping...
[INFO] [2025-03-10 14:35:53] Already completed stage calculate_statistics, skipping...
[START] [2025-03-10 14:35:53] complete_harvest_instance
[START] [2025-03-10 14:35:53] overall_tsv_creation
[INFO] [2025-03-10 14:35:54] It looks like we have already exported up to node 169545839, resuming.
[INFO] [2025-03-10 14:35:54] Exporting another 1304824 nodes as TSV in batches of 10000...
[INFO] [2025-03-10 14:35:54] Processing group of 1304824 in 131 batches of 10000
[INFO] [2025-03-10 14:38:33] Processed 10000/1304824 nodes
[2025-03-10 14:40:46] Encountered new language, please assign it to a Locale and give it a name: fil
[INFO] [2025-03-10 14:41:01] Processed 20000/1304824 nodes
[INFO] [2025-03-10 14:43:30] Processed 30000/1304824 nodes
[INFO] [2025-03-10 14:45:59] Processed 40000/1304824 nodes
[INFO] [2025-03-10 14:48:26] Processed 50000/1304824 nodes
[INFO] [2025-03-10 14:50:50] Processed 60000/1304824 nodes
[INFO] [2025-03-10 14:53:17] Processed 70000/1304824 nodes
[INFO] [2025-03-10 14:55:42] Processed 80000/1304824 nodes
[INFO] [2025-03-10 14:58:06] Processed 90000/1304824 nodes
[INFO] [2025-03-10 15:00:31] Processed 100000/1304824 nodes
[INFO] [2025-03-10 15:02:55] Processed 110000/1304824 nodes
[INFO] [2025-03-10 15:05:20] Processed 120000/1304824 nodes
[INFO] [2025-03-10 15:07:43] Processed 130000/1304824 nodes
[INFO] [2025-03-10 15:10:05] Processed 140000/1304824 nodes
[INFO] [2025-03-10 15:12:29] Processed 150000/1304824 nodes
[INFO] [2025-03-10 15:14:53] Processed 160000/1304824 nodes
[INFO] [2025-03-10 15:17:18] Processed 170000/1304824 nodes
[INFO] [2025-03-10 15:19:38] Processed 180000/1304824 nodes
[INFO] [2025-03-10 15:22:02] Processed 190000/1304824 nodes
[INFO] [2025-03-10 15:24:27] Processed 200000/1304824 nodes
[INFO] [2025-03-10 15:26:51] Processed 210000/1304824 nodes
[INFO] [2025-03-10 15:29:16] Processed 220000/1304824 nodes
[INFO] [2025-03-10 15:31:41] Processed 230000/1304824 nodes
[INFO] [2025-03-10 15:34:04] Processed 240000/1304824 nodes
[INFO] [2025-03-10 15:36:29] Processed 250000/1304824 nodes
[INFO] [2025-03-10 15:38:51] Processed 260000/1304824 nodes
[INFO] [2025-03-10 15:41:15] Processed 270000/1304824 nodes
[INFO] [2025-03-10 15:43:38] Processed 280000/1304824 nodes
[INFO] [2025-03-10 15:45:59] Processed 290000/1304824 nodes
[INFO] [2025-03-10 15:48:23] Processed 300000/1304824 nodes
[INFO] [2025-03-10 15:50:46] Processed 310000/1304824 nodes
[INFO] [2025-03-10 15:53:10] Processed 320000/1304824 nodes
[INFO] [2025-03-10 15:55:33] Processed 330000/1304824 nodes
[INFO] [2025-03-10 15:57:55] Processed 340000/1304824 nodes
[INFO] [2025-03-10 16:00:17] Processed 350000/1304824 nodes
[INFO] [2025-03-10 16:02:40] Processed 360000/1304824 nodes
[INFO] [2025-03-10 16:05:02] Processed 370000/1304824 nodes
[INFO] [2025-03-10 16:07:25] Processed 380000/1304824 nodes
[INFO] [2025-03-10 16:09:49] Processed 390000/1304824 nodes
[INFO] [2025-03-10 16:12:11] Processed 400000/1304824 nodes
[INFO] [2025-03-10 16:14:34] Processed 410000/1304824 nodes
[INFO] [2025-03-10 16:16:54] Processed 420000/1304824 nodes
[INFO] [2025-03-10 16:19:15] Processed 430000/1304824 nodes
[INFO] [2025-03-10 16:21:36] Processed 440000/1304824 nodes
[INFO] [2025-03-10 16:23:56] Processed 450000/1304824 nodes
[INFO] [2025-03-10 16:26:18] Processed 460000/1304824 nodes
[INFO] [2025-03-10 16:28:40] Processed 470000/1304824 nodes
[INFO] [2025-03-10 16:31:00] Processed 480000/1304824 nodes
[INFO] [2025-03-10 16:33:17] Processed 490000/1304824 nodes
[INFO] [2025-03-10 16:35:39] Processed 500000/1304824 nodes
[INFO] [2025-03-10 16:38:00] Processed 510000/1304824 nodes
[INFO] [2025-03-10 16:40:20] Processed 520000/1304824 nodes
[INFO] [2025-03-10 16:42:42] Processed 530000/1304824 nodes
[INFO] [2025-03-10 16:45:01] Processed 540000/1304824 nodes
[INFO] [2025-03-10 16:47:25] Processed 550000/1304824 nodes
[INFO] [2025-03-10 16:49:52] Processed 560000/1304824 nodes
[INFO] [2025-03-10 16:52:16] Processed 570000/1304824 nodes
[INFO] [2025-03-10 16:54:42] Processed 580000/1304824 nodes
[INFO] [2025-03-10 16:57:11] Processed 590000/1304824 nodes
[INFO] [2025-03-10 16:59:33] Processed 600000/1304824 nodes
[INFO] [2025-03-10 17:01:57] Processed 610000/1304824 nodes
[INFO] [2025-03-10 17:04:20] Processed 620000/1304824 nodes
[INFO] [2025-03-10 17:06:41] Processed 630000/1304824 nodes
[INFO] [2025-03-10 17:09:01] Processed 640000/1304824 nodes
[INFO] [2025-03-10 17:11:21] Processed 650000/1304824 nodes
[INFO] [2025-03-10 17:13:42] Processed 660000/1304824 nodes
[INFO] [2025-03-10 17:16:01] Processed 670000/1304824 nodes
[INFO] [2025-03-10 17:18:23] Processed 680000/1304824 nodes
[INFO] [2025-03-10 17:20:46] Processed 690000/1304824 nodes
[INFO] [2025-03-10 17:23:12] Processed 700000/1304824 nodes
[INFO] [2025-03-10 17:25:31] Processed 710000/1304824 nodes
[INFO] [2025-03-10 17:27:54] Processed 720000/1304824 nodes
[INFO] [2025-03-10 17:30:15] Processed 730000/1304824 nodes
[INFO] [2025-03-10 17:32:33] Processed 740000/1304824 nodes
[INFO] [2025-03-10 17:34:54] Processed 750000/1304824 nodes
[INFO] [2025-03-10 17:37:18] Processed 760000/1304824 nodes
[INFO] [2025-03-10 17:39:43] Processed 770000/1304824 nodes
[INFO] [2025-03-10 17:42:06] Processed 780000/1304824 nodes
[INFO] [2025-03-10 17:44:30] Processed 790000/1304824 nodes
[INFO] [2025-03-10 17:46:51] Processed 800000/1304824 nodes
[INFO] [2025-03-10 17:49:20] Processed 810000/1304824 nodes
[INFO] [2025-03-10 17:51:47] Processed 820000/1304824 nodes
[INFO] [2025-03-10 17:54:12] Processed 830000/1304824 nodes
[INFO] [2025-03-10 17:56:36] Processed 840000/1304824 nodes
[INFO] [2025-03-10 17:58:59] Processed 850000/1304824 nodes
[INFO] [2025-03-10 18:01:25] Processed 860000/1304824 nodes
[INFO] [2025-03-10 18:03:49] Processed 870000/1304824 nodes
[INFO] [2025-03-10 18:06:12] Processed 880000/1304824 nodes
[INFO] [2025-03-10 18:08:38] Processed 890000/1304824 nodes
[INFO] [2025-03-10 18:11:03] Processed 900000/1304824 nodes
[INFO] [2025-03-10 18:13:24] Processed 910000/1304824 nodes
[INFO] [2025-03-10 18:15:46] Processed 920000/1304824 nodes
[INFO] [2025-03-10 18:18:08] Processed 930000/1304824 nodes
[INFO] [2025-03-10 18:20:30] Processed 940000/1304824 nodes
[INFO] [2025-03-10 18:22:55] Processed 950000/1304824 nodes
[INFO] [2025-03-10 18:25:15] Processed 960000/1304824 nodes
[INFO] [2025-03-10 18:27:37] Processed 970000/1304824 nodes
[INFO] [2025-03-10 18:29:58] Processed 980000/1304824 nodes
[INFO] [2025-03-10 18:32:20] Processed 990000/1304824 nodes
[INFO] [2025-03-10 18:34:40] Processed 1000000/1304824 nodes
[INFO] [2025-03-10 18:37:03] Processed 1010000/1304824 nodes
[INFO] [2025-03-10 18:39:23] Processed 1020000/1304824 nodes
[INFO] [2025-03-10 18:41:47] Processed 1030000/1304824 nodes
[INFO] [2025-03-10 18:44:09] Processed 1040000/1304824 nodes
[INFO] [2025-03-10 18:46:33] Processed 1050000/1304824 nodes
[INFO] [2025-03-10 18:48:58] Processed 1060000/1304824 nodes
[INFO] [2025-03-10 18:51:21] Processed 1070000/1304824 nodes
[INFO] [2025-03-10 18:53:45] Processed 1080000/1304824 nodes
[INFO] [2025-03-10 18:56:07] Processed 1090000/1304824 nodes
[INFO] [2025-03-10 18:58:31] Processed 1100000/1304824 nodes
[INFO] [2025-03-10 19:00:53] Processed 1110000/1304824 nodes
[INFO] [2025-03-10 19:03:15] Processed 1120000/1304824 nodes
[INFO] [2025-03-10 19:05:32] Processed 1130000/1304824 nodes
[INFO] [2025-03-10 19:07:53] Processed 1140000/1304824 nodes
[INFO] [2025-03-10 19:10:14] Processed 1150000/1304824 nodes
[INFO] [2025-03-10 19:12:36] Processed 1160000/1304824 nodes
[INFO] [2025-03-10 19:14:55] Processed 1170000/1304824 nodes
[INFO] [2025-03-10 19:17:16] Processed 1180000/1304824 nodes
[INFO] [2025-03-10 19:19:37] Processed 1190000/1304824 nodes
[INFO] [2025-03-10 19:21:59] Processed 1200000/1304824 nodes
[INFO] [2025-03-10 19:24:18] Processed 1210000/1304824 nodes
[INFO] [2025-03-10 19:26:39] Processed 1220000/1304824 nodes
[INFO] [2025-03-10 19:28:58] Processed 1230000/1304824 nodes
[INFO] [2025-03-10 19:31:20] Processed 1240000/1304824 nodes
[INFO] [2025-03-10 19:33:41] Processed 1250000/1304824 nodes
[INFO] [2025-03-10 19:36:03] Processed 1260000/1304824 nodes
[INFO] [2025-03-10 19:38:22] Processed 1270000/1304824 nodes
[INFO] [2025-03-10 19:40:44] Processed 1280000/1304824 nodes
[INFO] [2025-03-10 19:43:07] Processed 1290000/1304824 nodes
[INFO] [2025-03-10 19:45:27] Processed 1300000/1304824 nodes
[INFO] [2025-03-10 19:45:45] Processed 1304824/1304824 nodes
[INFO] [2025-03-10 19:45:45] Average Time: 23.472
[INFO] [2025-03-10 19:45:45] Total Time: 5h9m52s
[INFO] [2025-03-10 19:45:45] last 3 / first 3: 1.14
[INFO] [2025-03-10 19:45:45] Std.Dev: 2.813; Max: 29.3
[STOP] [2025-03-10 19:45:45] overall_tsv_creation
[INFO] [2025-03-10 19:45:45] Done. Check your files:
[INFO] [2025-03-10 19:45:45] (1334824 lines) /app/public/data/i_nat_taxonomy_v/publish_nodes.tsv
[INFO] [2025-03-10 19:45:46] (15363931 lines) /app/public/data/i_nat_taxonomy_v/publish_node_ancestors.tsv
[INFO] [2025-03-10 19:45:46] (1334824 lines) /app/public/data/i_nat_taxonomy_v/publish_scientific_names.tsv
[INFO] [2025-03-10 19:45:47] (1324057 lines) /app/public/data/i_nat_taxonomy_v/publish_vernaculars.tsv
[STOP] [2025-03-10 19:45:49] complete_harvest_instance
[START] [2025-03-10 19:45:49] completed
[STOP] [2025-03-10 19:45:49] completed
[STOP] [2025-03-10 19:45:49] logged process, took 18596.3
Latest Process