Harvest for
wikipedia EN
Created
07 Jul 19:19
Stage:
completed
Fetched:
07 Jul 19:19
Validated:
07 Jul 19:21
Deltas Created
07 Jul 19:23
Units Normalized:
07 Jul 23:55
Ancestry Built:
07 Jul 21:37
Nodes Matched:
07 Jul 23:39
Names Parsed:
07 Jul 21:42
New Models Stored:
07 Jul 20:36
Indexed:
07 Jul 23:55
Completed:
08 Jul 00:50
Time to Harvest:
6 minutes
Harvesting Log
(530 lines)
[INFO] [2023-07-07 19:19:51] Created harvest instance #4367
[STOP] [2023-07-07 19:19:51] create_harvest_instance
[START] [2023-07-07 19:19:51] fetch_files
[STOP] [2023-07-07 19:19:51] fetch_files
[START] [2023-07-07 19:19:51] validate_each_file
[INFO] [2023-07-07 19:19:51] Looping over 2 formats...
[INFO] [2023-07-07 19:19:51] ...nodes (/app/public/data/wiki_english/taxon.tab)
[INFO] [2023-07-07 19:20:06] Valid: /app/public/data/wiki_english/converted_csv/wiki_english_nodes_30418.csv (452624 lines)
[INFO] [2023-07-07 19:20:06] ...media (/app/public/data/wiki_english/media_resource.tab)
[INFO] [2023-07-07 19:21:55] Valid: /app/public/data/wiki_english/converted_csv/wiki_english_media_30417.csv (866565 lines)
[STOP] [2023-07-07 19:21:55] validate_each_file
[START] [2023-07-07 19:21:55] convert_to_csv
[INFO] [2023-07-07 19:21:55] Looping over 2 formats...
[INFO] [2023-07-07 19:21:55] ...nodes (/app/public/data/wiki_english/taxon.tab)
[CMD] [2023-07-07 19:21:55] /usr/bin/sort /app/public/data/wiki_english/converted_csv/wiki_english_nodes_30418.csv > /app/public/data/wiki_english/converted_csv/wiki_english_nodes_30418.csv_sorted
[INFO] [2023-07-07 19:21:56] Converted: /app/public/data/wiki_english/converted_csv/wiki_english_nodes_30418.csv (452624 lines)
[INFO] [2023-07-07 19:21:56] ...media (/app/public/data/wiki_english/media_resource.tab)
[CMD] [2023-07-07 19:21:56] /usr/bin/sort /app/public/data/wiki_english/converted_csv/wiki_english_media_30417.csv > /app/public/data/wiki_english/converted_csv/wiki_english_media_30417.csv_sorted
[INFO] [2023-07-07 19:22:56] Converted: /app/public/data/wiki_english/converted_csv/wiki_english_media_30417.csv (866565 lines)
[STOP] [2023-07-07 19:22:56] convert_to_csv
[START] [2023-07-07 19:22:56] calculate_delta
[INFO] [2023-07-07 19:22:56] Looping over 2 formats...
[INFO] [2023-07-07 19:22:56] ...nodes (/app/public/data/wiki_english/taxon.tab)
[CMD] [2023-07-07 19:22:56] echo "0a" > /app/public/data/wiki_english/diff/wiki_english_nodes_30418.diff
[CMD] [2023-07-07 19:22:57] tail -n +1 /app/public/data/wiki_english/converted_csv/wiki_english_nodes_30418.csv >> /app/public/data/wiki_english/diff/wiki_english_nodes_30418.diff
[CMD] [2023-07-07 19:22:57] echo "." >> /app/public/data/wiki_english/diff/wiki_english_nodes_30418.diff
[INFO] [2023-07-07 19:22:58] Created diff: /app/public/data/wiki_english/diff/wiki_english_nodes_30418.diff (452626 lines)
[INFO] [2023-07-07 19:22:58] ...media (/app/public/data/wiki_english/media_resource.tab)
[CMD] [2023-07-07 19:22:58] echo "0a" > /app/public/data/wiki_english/diff/wiki_english_media_30417.diff
[CMD] [2023-07-07 19:22:58] tail -n +1 /app/public/data/wiki_english/converted_csv/wiki_english_media_30417.csv >> /app/public/data/wiki_english/diff/wiki_english_media_30417.diff
[CMD] [2023-07-07 19:23:29] echo "." >> /app/public/data/wiki_english/diff/wiki_english_media_30417.diff
[INFO] [2023-07-07 19:23:35] Created diff: /app/public/data/wiki_english/diff/wiki_english_media_30417.diff (866567 lines)
[STOP] [2023-07-07 19:23:35] calculate_delta
[START] [2023-07-07 19:23:35] parse_diff_and_store
[INFO] [2023-07-07 19:23:35] Handling diff: /app/public/data/wiki_english/diff/wiki_english_nodes_30418.diff (452626 lines)
[INFO] [2023-07-07 19:23:35] Loading nodes diff file into memory (452626 lines)...
[INFO] [2023-07-07 19:23:39] Storing 9999 ScientificNames (29997/10000/452626)
[INFO] [2023-07-07 19:23:42] Storing 9999 Identifiers (29997/10000/452626)
[INFO] [2023-07-07 19:23:43] Storing 9999 Nodes (29997/10000/452626)
[INFO] [2023-07-07 19:23:49] Storing 10000 ScientificNames (59997/20000/452626)
[INFO] [2023-07-07 19:23:52] Storing 10000 Identifiers (59997/20000/452626)
[INFO] [2023-07-07 19:23:53] Storing 10000 Nodes (59997/20000/452626)
[WARN] [2023-07-07 19:23:57] Filtered Scientific Name `Trachycephalus "vermiculatus"` to `Trachycephalus vermiculatus`
[WARN] [2023-07-07 19:23:57] Filtered Scientific Name `Neotrombicula fujigmo` to `Neotrombicula fujigmo`
[INFO] [2023-07-07 19:23:59] Storing 10000 ScientificNames (89997/30000/452626)
[INFO] [2023-07-07 19:24:02] Storing 10000 Identifiers (89997/30000/452626)
[INFO] [2023-07-07 19:24:03] Storing 10000 Nodes (89997/30000/452626)
[INFO] [2023-07-07 19:24:10] Storing 10000 ScientificNames (119997/40000/452626)
[INFO] [2023-07-07 19:24:12] Storing 10000 Identifiers (119997/40000/452626)
[INFO] [2023-07-07 19:24:13] Storing 10000 Nodes (119997/40000/452626)
[INFO] [2023-07-07 19:24:20] Storing 10000 ScientificNames (149997/50000/452626)
[INFO] [2023-07-07 19:24:23] Storing 10000 Identifiers (149997/50000/452626)
[INFO] [2023-07-07 19:24:24] Storing 10000 Nodes (149997/50000/452626)
[INFO] [2023-07-07 19:24:30] Storing 10000 ScientificNames (179997/60000/452626)
[INFO] [2023-07-07 19:24:33] Storing 10000 Identifiers (179997/60000/452626)
[INFO] [2023-07-07 19:24:34] Storing 10000 Nodes (179997/60000/452626)
[INFO] [2023-07-07 19:24:41] Storing 10000 ScientificNames (209997/70000/452626)
[INFO] [2023-07-07 19:24:44] Storing 10000 Identifiers (209997/70000/452626)
[INFO] [2023-07-07 19:24:45] Storing 10000 Nodes (209997/70000/452626)
[INFO] [2023-07-07 19:24:51] Storing 10000 ScientificNames (239997/80000/452626)
[INFO] [2023-07-07 19:24:54] Storing 10000 Identifiers (239997/80000/452626)
[INFO] [2023-07-07 19:24:55] Storing 10000 Nodes (239997/80000/452626)
[INFO] [2023-07-07 19:25:02] Storing 10000 ScientificNames (269997/90000/452626)
[INFO] [2023-07-07 19:25:05] Storing 10000 Identifiers (269997/90000/452626)
[INFO] [2023-07-07 19:25:06] Storing 10000 Nodes (269997/90000/452626)
[INFO] [2023-07-07 19:25:13] Storing 10000 ScientificNames (299997/100000/452626)
[INFO] [2023-07-07 19:25:15] Storing 10000 Identifiers (299997/100000/452626)
[INFO] [2023-07-07 19:25:16] Storing 10000 Nodes (299997/100000/452626)
[INFO] [2023-07-07 19:25:23] Storing 10000 ScientificNames (329997/110000/452626)
[INFO] [2023-07-07 19:25:26] Storing 10000 Identifiers (329997/110000/452626)
[INFO] [2023-07-07 19:25:27] Storing 10000 Nodes (329997/110000/452626)
[INFO] [2023-07-07 19:25:34] Storing 10000 ScientificNames (359997/120000/452626)
[INFO] [2023-07-07 19:25:37] Storing 10000 Identifiers (359997/120000/452626)
[INFO] [2023-07-07 19:25:38] Storing 10000 Nodes (359997/120000/452626)
[INFO] [2023-07-07 19:25:45] Storing 10000 ScientificNames (389997/130000/452626)
[INFO] [2023-07-07 19:25:48] Storing 10000 Identifiers (389997/130000/452626)
[INFO] [2023-07-07 19:25:49] Storing 10000 Nodes (389997/130000/452626)
[INFO] [2023-07-07 19:25:56] Storing 10000 ScientificNames (419997/140000/452626)
[INFO] [2023-07-07 19:25:59] Storing 10000 Identifiers (419997/140000/452626)
[INFO] [2023-07-07 19:26:00] Storing 10000 Nodes (419997/140000/452626)
[INFO] [2023-07-07 19:26:07] Storing 10000 ScientificNames (449997/150000/452626)
[INFO] [2023-07-07 19:26:10] Storing 10000 Identifiers (449997/150000/452626)
[INFO] [2023-07-07 19:26:11] Storing 10000 Nodes (449997/150000/452626)
[INFO] [2023-07-07 19:26:18] Storing 10000 ScientificNames (479997/160000/452626)
[INFO] [2023-07-07 19:26:21] Storing 10000 Identifiers (479997/160000/452626)
[INFO] [2023-07-07 19:26:22] Storing 10000 Nodes (479997/160000/452626)
[INFO] [2023-07-07 19:26:29] Storing 10000 ScientificNames (509997/170000/452626)
[INFO] [2023-07-07 19:26:32] Storing 10000 Identifiers (509997/170000/452626)
[INFO] [2023-07-07 19:26:33] Storing 10000 Nodes (509997/170000/452626)
[INFO] [2023-07-07 19:26:40] Storing 10000 ScientificNames (539997/180000/452626)
[INFO] [2023-07-07 19:26:43] Storing 10000 Identifiers (539997/180000/452626)
[INFO] [2023-07-07 19:26:44] Storing 10000 Nodes (539997/180000/452626)
[INFO] [2023-07-07 19:26:51] Storing 10000 ScientificNames (569998/190000/452626)
[INFO] [2023-07-07 19:26:54] Storing 10001 Identifiers (569998/190000/452626)
[INFO] [2023-07-07 19:26:55] Storing 10000 Nodes (569998/190000/452626)
[INFO] [2023-07-07 19:27:03] Storing 10000 ScientificNames (599999/200000/452626)
[INFO] [2023-07-07 19:27:06] Storing 10001 Identifiers (599999/200000/452626)
[INFO] [2023-07-07 19:27:07] Storing 10000 Nodes (599999/200000/452626)
[INFO] [2023-07-07 19:27:14] Storing 10000 ScientificNames (629999/210000/452626)
[INFO] [2023-07-07 19:27:17] Storing 10000 Identifiers (629999/210000/452626)
[INFO] [2023-07-07 19:27:18] Storing 10000 Nodes (629999/210000/452626)
[INFO] [2023-07-07 19:27:25] Storing 10000 ScientificNames (659999/220000/452626)
[INFO] [2023-07-07 19:27:28] Storing 10000 Identifiers (659999/220000/452626)
[INFO] [2023-07-07 19:27:29] Storing 10000 Nodes (659999/220000/452626)
[INFO] [2023-07-07 19:27:37] Storing 10000 ScientificNames (689999/230000/452626)
[INFO] [2023-07-07 19:27:40] Storing 10000 Identifiers (689999/230000/452626)
[INFO] [2023-07-07 19:27:41] Storing 10000 Nodes (689999/230000/452626)
[INFO] [2023-07-07 19:27:49] Storing 10000 ScientificNames (719999/240000/452626)
[INFO] [2023-07-07 19:27:52] Storing 10000 Identifiers (719999/240000/452626)
[INFO] [2023-07-07 19:27:53] Storing 10000 Nodes (719999/240000/452626)
[INFO] [2023-07-07 19:28:01] Storing 10000 ScientificNames (749999/250000/452626)
[INFO] [2023-07-07 19:28:04] Storing 10000 Identifiers (749999/250000/452626)
[INFO] [2023-07-07 19:28:05] Storing 10000 Nodes (749999/250000/452626)
[WARN] [2023-07-07 19:28:10] Filtered Scientific Name `/Gunneridae` to `Gunneridae`
[INFO] [2023-07-07 19:28:13] Storing 10000 ScientificNames (779999/260000/452626)
[INFO] [2023-07-07 19:28:15] Storing 10000 Identifiers (779999/260000/452626)
[INFO] [2023-07-07 19:28:17] Storing 10000 Nodes (779999/260000/452626)
[INFO] [2023-07-07 19:28:25] Storing 10000 ScientificNames (809999/270000/452626)
[INFO] [2023-07-07 19:28:28] Storing 10000 Identifiers (809999/270000/452626)
[INFO] [2023-07-07 19:28:29] Storing 10000 Nodes (809999/270000/452626)
[INFO] [2023-07-07 19:28:37] Storing 10000 ScientificNames (839999/280000/452626)
[INFO] [2023-07-07 19:28:39] Storing 10000 Identifiers (839999/280000/452626)
[INFO] [2023-07-07 19:28:41] Storing 10000 Nodes (839999/280000/452626)
[INFO] [2023-07-07 19:28:49] Storing 10000 ScientificNames (869999/290000/452626)
[INFO] [2023-07-07 19:28:52] Storing 10000 Identifiers (869999/290000/452626)
[INFO] [2023-07-07 19:28:52] Storing 10000 Nodes (869999/290000/452626)
[INFO] [2023-07-07 19:29:01] Storing 10000 ScientificNames (899999/300000/452626)
[INFO] [2023-07-07 19:29:03] Storing 10000 Identifiers (899999/300000/452626)
[INFO] [2023-07-07 19:29:04] Storing 10000 Nodes (899999/300000/452626)
[WARN] [2023-07-07 19:29:10] Filtered Scientific Name `/Eudicotyledoneae` to `Eudicotyledoneae`
[WARN] [2023-07-07 19:29:10] Filtered Scientific Name `/Mesangiospermae` to `Mesangiospermae`
[WARN] [2023-07-07 19:29:10] Filtered Scientific Name `/Pan-Angiospermae` to `Pan-Angiospermae`
[INFO] [2023-07-07 19:29:13] Storing 10000 ScientificNames (930000/310000/452626)
[INFO] [2023-07-07 19:29:16] Storing 10001 Identifiers (930000/310000/452626)
[INFO] [2023-07-07 19:29:17] Storing 10000 Nodes (930000/310000/452626)
[INFO] [2023-07-07 19:29:25] Storing 10000 ScientificNames (960000/320000/452626)
[INFO] [2023-07-07 19:29:28] Storing 10000 Identifiers (960000/320000/452626)
[INFO] [2023-07-07 19:29:29] Storing 10000 Nodes (960000/320000/452626)
[INFO] [2023-07-07 19:29:37] Storing 10000 ScientificNames (990000/330000/452626)
[INFO] [2023-07-07 19:29:40] Storing 10000 Identifiers (990000/330000/452626)
[INFO] [2023-07-07 19:29:41] Storing 10000 Nodes (990000/330000/452626)
[INFO] [2023-07-07 19:29:49] Storing 10000 ScientificNames (1020000/340000/452626)
[INFO] [2023-07-07 19:29:52] Storing 10000 Identifiers (1020000/340000/452626)
[INFO] [2023-07-07 19:29:53] Storing 10000 Nodes (1020000/340000/452626)
[INFO] [2023-07-07 19:30:01] Storing 10000 ScientificNames (1050000/350000/452626)
[INFO] [2023-07-07 19:30:04] Storing 10000 Identifiers (1050000/350000/452626)
[INFO] [2023-07-07 19:30:05] Storing 10000 Nodes (1050000/350000/452626)
[INFO] [2023-07-07 19:30:14] Storing 10000 ScientificNames (1080000/360000/452626)
[INFO] [2023-07-07 19:30:16] Storing 10000 Identifiers (1080000/360000/452626)
[INFO] [2023-07-07 19:30:17] Storing 10000 Nodes (1080000/360000/452626)
[INFO] [2023-07-07 19:30:26] Storing 10000 ScientificNames (1110000/370000/452626)
[INFO] [2023-07-07 19:30:29] Storing 10000 Identifiers (1110000/370000/452626)
[INFO] [2023-07-07 19:30:30] Storing 10000 Nodes (1110000/370000/452626)
[INFO] [2023-07-07 19:30:38] Storing 10000 ScientificNames (1140000/380000/452626)
[INFO] [2023-07-07 19:30:41] Storing 10000 Identifiers (1140000/380000/452626)
[INFO] [2023-07-07 19:30:42] Storing 10000 Nodes (1140000/380000/452626)
[WARN] [2023-07-07 19:30:45] Filtered Scientific Name `/Pentapetalae` to `Pentapetalae`
[WARN] [2023-07-07 19:30:46] Filtered Scientific Name `Homalocephala polycephala` to `Homalocephala polycephala`
[INFO] [2023-07-07 19:30:50] Storing 10000 ScientificNames (1170000/390000/452626)
[INFO] [2023-07-07 19:30:53] Storing 10000 Identifiers (1170000/390000/452626)
[INFO] [2023-07-07 19:30:54] Storing 10000 Nodes (1170000/390000/452626)
[INFO] [2023-07-07 19:31:03] Storing 10000 ScientificNames (1200000/400000/452626)
[INFO] [2023-07-07 19:31:06] Storing 10000 Identifiers (1200000/400000/452626)
[INFO] [2023-07-07 19:31:07] Storing 10000 Nodes (1200000/400000/452626)
[INFO] [2023-07-07 19:31:16] Storing 10000 ScientificNames (1230000/410000/452626)
[INFO] [2023-07-07 19:31:19] Storing 10000 Identifiers (1230000/410000/452626)
[INFO] [2023-07-07 19:31:20] Storing 10000 Nodes (1230000/410000/452626)
[INFO] [2023-07-07 19:31:28] Storing 10000 ScientificNames (1260000/420000/452626)
[INFO] [2023-07-07 19:31:31] Storing 10000 Identifiers (1260000/420000/452626)
[INFO] [2023-07-07 19:31:32] Storing 10000 Nodes (1260000/420000/452626)
[INFO] [2023-07-07 19:31:41] Storing 10000 ScientificNames (1290002/430000/452626)
[INFO] [2023-07-07 19:31:44] Storing 10002 Identifiers (1290002/430000/452626)
[INFO] [2023-07-07 19:31:45] Storing 10000 Nodes (1290002/430000/452626)
[INFO] [2023-07-07 19:31:53] Storing 10000 ScientificNames (1320002/440000/452626)
[INFO] [2023-07-07 19:31:56] Storing 10000 Identifiers (1320002/440000/452626)
[INFO] [2023-07-07 19:31:57] Storing 10000 Nodes (1320002/440000/452626)
[INFO] [2023-07-07 19:32:06] Storing 10000 ScientificNames (1350002/450000/452626)
[INFO] [2023-07-07 19:32:09] Storing 10000 Identifiers (1350002/450000/452626)
[INFO] [2023-07-07 19:32:10] Storing 10000 Nodes (1350002/450000/452626)
[INFO] [2023-07-07 19:32:17] Storing 2625 ScientificNames (1357877/452624/452626)
[INFO] [2023-07-07 19:32:18] Storing 2625 Identifiers (1357877/452624/452626)
[INFO] [2023-07-07 19:32:18] Storing 2625 Nodes (1357877/452624/452626)
[INFO] [2023-07-07 19:32:25] Handling diff: /app/public/data/wiki_english/diff/wiki_english_media_30417.diff (866567 lines)
[INFO] [2023-07-07 19:32:25] Loading media diff file into memory (866567 lines)...
[INFO] [2023-07-07 19:33:03] Storing 9999 ArticlesSections (19998/10000/866567)
[INFO] [2023-07-07 19:33:04] Storing 9999 Articles (19998/10000/866567)
[INFO] [2023-07-07 19:33:46] Storing 10000 ArticlesSections (39998/20000/866567)
[INFO] [2023-07-07 19:33:47] Storing 10000 Articles (39998/20000/866567)
[INFO] [2023-07-07 19:34:29] Storing 10000 ArticlesSections (59998/30000/866567)
[INFO] [2023-07-07 19:34:30] Storing 10000 Articles (59998/30000/866567)
[INFO] [2023-07-07 19:35:12] Storing 10000 ArticlesSections (79998/40000/866567)
[INFO] [2023-07-07 19:35:13] Storing 10000 Articles (79998/40000/866567)
[INFO] [2023-07-07 19:35:53] Storing 10000 ArticlesSections (99998/50000/866567)
[INFO] [2023-07-07 19:35:53] Storing 10000 Articles (99998/50000/866567)
[INFO] [2023-07-07 19:36:35] Storing 10000 ArticlesSections (119998/60000/866567)
[INFO] [2023-07-07 19:36:36] Storing 10000 Articles (119998/60000/866567)
[INFO] [2023-07-07 19:37:18] Storing 10000 ArticlesSections (139998/70000/866567)
[INFO] [2023-07-07 19:37:19] Storing 10000 Articles (139998/70000/866567)
[INFO] [2023-07-07 19:38:01] Storing 10000 ArticlesSections (159998/80000/866567)
[INFO] [2023-07-07 19:38:02] Storing 10000 Articles (159998/80000/866567)
[INFO] [2023-07-07 19:38:44] Storing 10000 ArticlesSections (179998/90000/866567)
[INFO] [2023-07-07 19:38:44] Storing 10000 Articles (179998/90000/866567)
[INFO] [2023-07-07 19:39:28] Storing 10000 ArticlesSections (199998/100000/866567)
[INFO] [2023-07-07 19:39:29] Storing 10000 Articles (199998/100000/866567)
[INFO] [2023-07-07 19:40:12] Storing 10000 ArticlesSections (219998/110000/866567)
[INFO] [2023-07-07 19:40:12] Storing 10000 Articles (219998/110000/866567)
[INFO] [2023-07-07 19:40:55] Storing 10000 ArticlesSections (239998/120000/866567)
[INFO] [2023-07-07 19:40:56] Storing 10000 Articles (239998/120000/866567)
[INFO] [2023-07-07 19:41:39] Storing 10000 ArticlesSections (259998/130000/866567)
[INFO] [2023-07-07 19:41:40] Storing 10000 Articles (259998/130000/866567)
[INFO] [2023-07-07 19:42:23] Storing 10000 ArticlesSections (279998/140000/866567)
[INFO] [2023-07-07 19:42:24] Storing 10000 Articles (279998/140000/866567)
[INFO] [2023-07-07 19:43:04] Storing 10000 ArticlesSections (299998/150000/866567)
[INFO] [2023-07-07 19:43:05] Storing 10000 Articles (299998/150000/866567)
[INFO] [2023-07-07 19:43:46] Storing 10000 ArticlesSections (319998/160000/866567)
[INFO] [2023-07-07 19:43:47] Storing 10000 Articles (319998/160000/866567)
[INFO] [2023-07-07 19:44:31] Storing 10000 ArticlesSections (339998/170000/866567)
[INFO] [2023-07-07 19:44:31] Storing 10000 Articles (339998/170000/866567)
[INFO] [2023-07-07 19:45:15] Storing 10000 ArticlesSections (359998/180000/866567)
[INFO] [2023-07-07 19:45:15] Storing 10000 Articles (359998/180000/866567)
[INFO] [2023-07-07 19:45:57] Storing 10000 ArticlesSections (379998/190000/866567)
[INFO] [2023-07-07 19:45:58] Storing 10000 Articles (379998/190000/866567)
[INFO] [2023-07-07 19:46:40] Storing 10000 ArticlesSections (399998/200000/866567)
[INFO] [2023-07-07 19:46:41] Storing 10000 Articles (399998/200000/866567)
[INFO] [2023-07-07 19:47:23] Storing 10000 ArticlesSections (419998/210000/866567)
[INFO] [2023-07-07 19:47:24] Storing 10000 Articles (419998/210000/866567)
[INFO] [2023-07-07 19:48:07] Storing 10000 ArticlesSections (439998/220000/866567)
[INFO] [2023-07-07 19:48:07] Storing 10000 Articles (439998/220000/866567)
[INFO] [2023-07-07 19:48:51] Storing 10000 ArticlesSections (459998/230000/866567)
[INFO] [2023-07-07 19:48:52] Storing 10000 Articles (459998/230000/866567)
[INFO] [2023-07-07 19:49:35] Storing 10000 ArticlesSections (479998/240000/866567)
[INFO] [2023-07-07 19:49:35] Storing 10000 Articles (479998/240000/866567)
[INFO] [2023-07-07 19:50:18] Storing 10000 ArticlesSections (499998/250000/866567)
[INFO] [2023-07-07 19:50:18] Storing 10000 Articles (499998/250000/866567)
[INFO] [2023-07-07 19:51:00] Storing 10000 ArticlesSections (519998/260000/866567)
[INFO] [2023-07-07 19:51:00] Storing 10000 Articles (519998/260000/866567)
[INFO] [2023-07-07 19:51:41] Storing 10000 ArticlesSections (539998/270000/866567)
[INFO] [2023-07-07 19:51:42] Storing 10000 Articles (539998/270000/866567)
[INFO] [2023-07-07 19:52:26] Storing 10000 ArticlesSections (559998/280000/866567)
[INFO] [2023-07-07 19:52:27] Storing 10000 Articles (559998/280000/866567)
[INFO] [2023-07-07 19:53:09] Storing 10000 ArticlesSections (579998/290000/866567)
[INFO] [2023-07-07 19:53:10] Storing 10000 Articles (579998/290000/866567)
[INFO] [2023-07-07 19:53:53] Storing 10000 ArticlesSections (599998/300000/866567)
[INFO] [2023-07-07 19:53:54] Storing 10000 Articles (599998/300000/866567)
[INFO] [2023-07-07 19:54:37] Storing 10000 ArticlesSections (619998/310000/866567)
[INFO] [2023-07-07 19:54:37] Storing 10000 Articles (619998/310000/866567)
[INFO] [2023-07-07 19:55:22] Storing 10000 ArticlesSections (639998/320000/866567)
[INFO] [2023-07-07 19:55:23] Storing 10000 Articles (639998/320000/866567)
[INFO] [2023-07-07 19:56:06] Storing 10000 ArticlesSections (659998/330000/866567)
[INFO] [2023-07-07 19:56:07] Storing 10000 Articles (659998/330000/866567)
[INFO] [2023-07-07 19:56:50] Storing 10000 ArticlesSections (679998/340000/866567)
[INFO] [2023-07-07 19:56:51] Storing 10000 Articles (679998/340000/866567)
[INFO] [2023-07-07 19:57:35] Storing 10000 ArticlesSections (699998/350000/866567)
[INFO] [2023-07-07 19:57:36] Storing 10000 Articles (699998/350000/866567)
[INFO] [2023-07-07 19:58:21] Storing 10000 ArticlesSections (719998/360000/866567)
[INFO] [2023-07-07 19:58:21] Storing 10000 Articles (719998/360000/866567)
[INFO] [2023-07-07 19:59:05] Storing 10000 ArticlesSections (739998/370000/866567)
[INFO] [2023-07-07 19:59:06] Storing 10000 Articles (739998/370000/866567)
[INFO] [2023-07-07 19:59:50] Storing 10000 ArticlesSections (759998/380000/866567)
[INFO] [2023-07-07 19:59:50] Storing 10000 Articles (759998/380000/866567)
[INFO] [2023-07-07 20:00:35] Storing 10000 ArticlesSections (779998/390000/866567)
[INFO] [2023-07-07 20:00:35] Storing 10000 Articles (779998/390000/866567)
[INFO] [2023-07-07 20:01:19] Storing 10000 ArticlesSections (799998/400000/866567)
[INFO] [2023-07-07 20:01:19] Storing 10000 Articles (799998/400000/866567)
[INFO] [2023-07-07 20:02:01] Storing 10000 ArticlesSections (819998/410000/866567)
[INFO] [2023-07-07 20:02:02] Storing 10000 Articles (819998/410000/866567)
[INFO] [2023-07-07 20:02:47] Storing 10000 ArticlesSections (839998/420000/866567)
[INFO] [2023-07-07 20:02:47] Storing 10000 Articles (839998/420000/866567)
[INFO] [2023-07-07 20:03:31] Storing 10000 ArticlesSections (859998/430000/866567)
[INFO] [2023-07-07 20:03:31] Storing 10000 Articles (859998/430000/866567)
[INFO] [2023-07-07 20:04:15] Storing 10000 ArticlesSections (879998/440000/866567)
[INFO] [2023-07-07 20:04:15] Storing 10000 Articles (879998/440000/866567)
[INFO] [2023-07-07 20:05:01] Storing 10000 ArticlesSections (899998/450000/866567)
[INFO] [2023-07-07 20:05:01] Storing 10000 Articles (899998/450000/866567)
[INFO] [2023-07-07 20:05:47] Storing 10000 ArticlesSections (919998/460000/866567)
[INFO] [2023-07-07 20:05:48] Storing 10000 Articles (919998/460000/866567)
[INFO] [2023-07-07 20:06:32] Storing 10000 ArticlesSections (939998/470000/866567)
[INFO] [2023-07-07 20:06:32] Storing 10000 Articles (939998/470000/866567)
[INFO] [2023-07-07 20:07:20] Storing 10000 ArticlesSections (959998/480000/866567)
[INFO] [2023-07-07 20:07:20] Storing 10000 Articles (959998/480000/866567)
[INFO] [2023-07-07 20:08:06] Storing 10000 ArticlesSections (979998/490000/866567)
[INFO] [2023-07-07 20:08:07] Storing 10000 Articles (979998/490000/866567)
[INFO] [2023-07-07 20:08:52] Storing 10000 ArticlesSections (999998/500000/866567)
[INFO] [2023-07-07 20:08:53] Storing 10000 Articles (999998/500000/866567)
[INFO] [2023-07-07 20:09:37] Storing 10000 ArticlesSections (1019998/510000/866567)
[INFO] [2023-07-07 20:09:38] Storing 10000 Articles (1019998/510000/866567)
[INFO] [2023-07-07 20:10:26] Storing 10000 ArticlesSections (1039998/520000/866567)
[INFO] [2023-07-07 20:10:26] Storing 10000 Articles (1039998/520000/866567)
[INFO] [2023-07-07 20:11:10] Storing 10000 ArticlesSections (1059998/530000/866567)
[INFO] [2023-07-07 20:11:11] Storing 10000 Articles (1059998/530000/866567)
[INFO] [2023-07-07 20:11:55] Storing 10000 ArticlesSections (1079998/540000/866567)
[INFO] [2023-07-07 20:11:55] Storing 10000 Articles (1079998/540000/866567)
[INFO] [2023-07-07 20:12:40] Storing 10000 ArticlesSections (1099998/550000/866567)
[INFO] [2023-07-07 20:12:40] Storing 10000 Articles (1099998/550000/866567)
[INFO] [2023-07-07 20:13:26] Storing 10000 ArticlesSections (1119998/560000/866567)
[INFO] [2023-07-07 20:13:26] Storing 10000 Articles (1119998/560000/866567)
[INFO] [2023-07-07 20:14:11] Storing 10000 ArticlesSections (1139998/570000/866567)
[INFO] [2023-07-07 20:14:12] Storing 10000 Articles (1139998/570000/866567)
[INFO] [2023-07-07 20:14:56] Storing 10000 ArticlesSections (1159998/580000/866567)
[INFO] [2023-07-07 20:14:57] Storing 10000 Articles (1159998/580000/866567)
[INFO] [2023-07-07 20:15:43] Storing 10000 ArticlesSections (1179998/590000/866567)
[INFO] [2023-07-07 20:15:43] Storing 10000 Articles (1179998/590000/866567)
[INFO] [2023-07-07 20:16:29] Storing 10000 ArticlesSections (1199998/600000/866567)
[INFO] [2023-07-07 20:16:29] Storing 10000 Articles (1199998/600000/866567)
[INFO] [2023-07-07 20:17:14] Storing 10000 ArticlesSections (1219998/610000/866567)
[INFO] [2023-07-07 20:17:14] Storing 10000 Articles (1219998/610000/866567)
[INFO] [2023-07-07 20:17:58] Storing 10000 ArticlesSections (1239998/620000/866567)
[INFO] [2023-07-07 20:17:59] Storing 10000 Articles (1239998/620000/866567)
[INFO] [2023-07-07 20:18:45] Storing 10000 ArticlesSections (1259998/630000/866567)
[INFO] [2023-07-07 20:18:45] Storing 10000 Articles (1259998/630000/866567)
[INFO] [2023-07-07 20:19:30] Storing 10000 ArticlesSections (1279998/640000/866567)
[INFO] [2023-07-07 20:19:31] Storing 10000 Articles (1279998/640000/866567)
[INFO] [2023-07-07 20:20:15] Storing 10000 ArticlesSections (1299998/650000/866567)
[INFO] [2023-07-07 20:20:15] Storing 10000 Articles (1299998/650000/866567)
[INFO] [2023-07-07 20:20:59] Storing 10000 ArticlesSections (1319998/660000/866567)
[INFO] [2023-07-07 20:21:00] Storing 10000 Articles (1319998/660000/866567)
[INFO] [2023-07-07 20:21:44] Storing 10000 ArticlesSections (1339998/670000/866567)
[INFO] [2023-07-07 20:21:44] Storing 10000 Articles (1339998/670000/866567)
[INFO] [2023-07-07 20:22:29] Storing 10000 ArticlesSections (1359998/680000/866567)
[INFO] [2023-07-07 20:22:29] Storing 10000 Articles (1359998/680000/866567)
[INFO] [2023-07-07 20:23:14] Storing 10000 ArticlesSections (1379998/690000/866567)
[INFO] [2023-07-07 20:23:14] Storing 10000 Articles (1379998/690000/866567)
[INFO] [2023-07-07 20:24:00] Storing 10000 ArticlesSections (1399998/700000/866567)
[INFO] [2023-07-07 20:24:01] Storing 10000 Articles (1399998/700000/866567)
[INFO] [2023-07-07 20:24:45] Storing 10000 ArticlesSections (1419998/710000/866567)
[INFO] [2023-07-07 20:24:46] Storing 10000 Articles (1419998/710000/866567)
[INFO] [2023-07-07 20:25:31] Storing 10000 ArticlesSections (1439998/720000/866567)
[INFO] [2023-07-07 20:25:31] Storing 10000 Articles (1439998/720000/866567)
[INFO] [2023-07-07 20:26:16] Storing 10000 ArticlesSections (1459998/730000/866567)
[INFO] [2023-07-07 20:26:17] Storing 10000 Articles (1459998/730000/866567)
[INFO] [2023-07-07 20:27:02] Storing 10000 ArticlesSections (1479998/740000/866567)
[INFO] [2023-07-07 20:27:03] Storing 10000 Articles (1479998/740000/866567)
[INFO] [2023-07-07 20:27:49] Storing 10000 ArticlesSections (1499998/750000/866567)
[INFO] [2023-07-07 20:27:49] Storing 10000 Articles (1499998/750000/866567)
[INFO] [2023-07-07 20:28:35] Storing 10000 ArticlesSections (1519998/760000/866567)
[INFO] [2023-07-07 20:28:35] Storing 10000 Articles (1519998/760000/866567)
[INFO] [2023-07-07 20:29:23] Storing 10000 ArticlesSections (1539998/770000/866567)
[INFO] [2023-07-07 20:29:23] Storing 10000 Articles (1539998/770000/866567)
[INFO] [2023-07-07 20:30:08] Storing 10000 ArticlesSections (1559998/780000/866567)
[INFO] [2023-07-07 20:30:09] Storing 10000 Articles (1559998/780000/866567)
[INFO] [2023-07-07 20:30:54] Storing 10000 ArticlesSections (1579998/790000/866567)
[INFO] [2023-07-07 20:30:55] Storing 10000 Articles (1579998/790000/866567)
[INFO] [2023-07-07 20:31:40] Storing 10000 ArticlesSections (1599998/800000/866567)
[INFO] [2023-07-07 20:31:41] Storing 10000 Articles (1599998/800000/866567)
[INFO] [2023-07-07 20:32:25] Storing 10000 ArticlesSections (1619998/810000/866567)
[INFO] [2023-07-07 20:32:26] Storing 10000 Articles (1619998/810000/866567)
[INFO] [2023-07-07 20:33:11] Storing 10000 ArticlesSections (1639998/820000/866567)
[INFO] [2023-07-07 20:33:12] Storing 10000 Articles (1639998/820000/866567)
[INFO] [2023-07-07 20:33:57] Storing 10000 ArticlesSections (1659998/830000/866567)
[INFO] [2023-07-07 20:33:58] Storing 10000 Articles (1659998/830000/866567)
[INFO] [2023-07-07 20:34:43] Storing 10000 ArticlesSections (1679998/840000/866567)
[INFO] [2023-07-07 20:34:43] Storing 10000 Articles (1679998/840000/866567)
[INFO] [2023-07-07 20:35:31] Storing 10000 ArticlesSections (1699998/850000/866567)
[INFO] [2023-07-07 20:35:32] Storing 10000 Articles (1699998/850000/866567)
[INFO] [2023-07-07 20:36:18] Storing 10000 ArticlesSections (1719998/860000/866567)
[INFO] [2023-07-07 20:36:19] Storing 10000 Articles (1719998/860000/866567)
[INFO] [2023-07-07 20:36:51] Storing 6566 ArticlesSections (1733130/866565/866567)
[INFO] [2023-07-07 20:36:51] Storing 6566 Articles (1733130/866565/866567)
[STOP] [2023-07-07 20:36:55] parse_diff_and_store
[START] [2023-07-07 20:36:55] resolve_keys
[2023-07-07 20:39:17] Resolving downloaded urls (this is not actually downloading them yet)
[INFO] [2023-07-07 21:04:00] Occurrences to nodes (through scientific_names)...
[INFO] [2023-07-07 21:04:00] traits to occurrences...
[INFO] [2023-07-07 21:04:00] traits to nodes (through occurrences)...
[INFO] [2023-07-07 21:04:00] Traits to sex term...
[INFO] [2023-07-07 21:04:00] Traits to lifestage term...
[INFO] [2023-07-07 21:04:00] MetaTraits to traits...
[INFO] [2023-07-07 21:04:00] MetaTraits (simple, measurement row refers to parent) to traits...
[INFO] [2023-07-07 21:04:00] Assocs to occurrences...
[INFO] [2023-07-07 21:04:00] Assocs to nodes...
[INFO] [2023-07-07 21:04:00] Assoc to sex term...
[INFO] [2023-07-07 21:04:00] Assoc to lifestage term...
[INFO] [2023-07-07 21:04:00] MetaAssoc to assocs...
[STOP] [2023-07-07 21:04:00] resolve_keys
[START] [2023-07-07 21:04:00] hold_for_later_1
[STOP] [2023-07-07 21:04:00] hold_for_later_1
[START] [2023-07-07 21:04:00] hold_for_later_2
[STOP] [2023-07-07 21:04:00] hold_for_later_2
[START] [2023-07-07 21:04:00] resolve_missing_parents
[STOP] [2023-07-07 21:04:54] resolve_missing_parents
[START] [2023-07-07 21:04:54] rebuild_nodes
[START] [2023-07-07 21:04:54] Flattener#flatten
[START] [2023-07-07 21:04:54] Flattener#study_resource
[START] [2023-07-07 21:06:25] Flattener#build_ancestry
[STOP] [2023-07-07 21:13:58] Flattener#build_ancestry
[INFO] [2023-07-07 21:13:58] 452624 ancestry keys
[START] [2023-07-07 21:13:58] build_node_ancestors
[INFO] [2023-07-07 21:13:58] old ancestors deleted.
[STOP] [2023-07-07 21:32:01] build_node_ancestors
[START] [2023-07-07 21:32:04] Flattener#propagate_ancestor_ids
[STOP] [2023-07-07 21:37:04] Flattener#propagate_ancestor_ids
[STOP] [2023-07-07 21:37:04] Flattener#flatten
[STOP] [2023-07-07 21:37:04] rebuild_nodes
[START] [2023-07-07 21:37:04] resolve_missing_media_owners
[STOP] [2023-07-07 21:37:04] resolve_missing_media_owners
[START] [2023-07-07 21:37:04] sanitize_media_verbatims
[STOP] [2023-07-07 21:37:04] sanitize_media_verbatims
[START] [2023-07-07 21:37:04] queue_downloads
[STOP] [2023-07-07 21:37:04] queue_downloads
[START] [2023-07-07 21:37:04] parse_names
[WARN] [2023-07-07 21:37:05] I see 452624 names which still need to be parsed.
[WARN] [2023-07-07 21:37:06] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2023-07-07 21:37:13] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2023-07-07 21:37:20] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2023-07-07 21:37:27] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2023-07-07 21:37:35] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2023-07-07 21:37:42] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2023-07-07 21:37:49] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2023-07-07 21:37:56] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2023-07-07 21:38:04] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2023-07-07 21:38:11] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2023-07-07 21:38:18] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2023-07-07 21:38:25] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2023-07-07 21:38:32] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2023-07-07 21:38:39] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2023-07-07 21:38:46] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2023-07-07 21:38:53] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2023-07-07 21:39:00] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2023-07-07 21:39:07] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2023-07-07 21:39:14] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2023-07-07 21:39:21] Names to parse: 10000 formatted: 10000 learned: 9998 parsed: 10000
[WARN] [2023-07-07 21:39:28] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2023-07-07 21:39:35] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2023-07-07 21:39:42] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2023-07-07 21:39:49] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2023-07-07 21:39:56] Names to parse: 10000 formatted: 10000 learned: 9998 parsed: 10000
[WARN] [2023-07-07 21:40:03] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2023-07-07 21:40:11] Names to parse: 10000 formatted: 10000 learned: 9997 parsed: 10000
[WARN] [2023-07-07 21:40:18] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2023-07-07 21:40:25] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2023-07-07 21:40:32] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2023-07-07 21:40:40] Names to parse: 10000 formatted: 10000 learned: 9994 parsed: 10000
[WARN] [2023-07-07 21:40:47] Names to parse: 10000 formatted: 10000 learned: 9993 parsed: 10000
[WARN] [2023-07-07 21:40:54] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2023-07-07 21:41:01] Names to parse: 10000 formatted: 10000 learned: 9998 parsed: 10000
[WARN] [2023-07-07 21:41:08] Names to parse: 10000 formatted: 10000 learned: 9996 parsed: 10000
[WARN] [2023-07-07 21:41:15] Names to parse: 10000 formatted: 10000 learned: 9998 parsed: 10000
[WARN] [2023-07-07 21:41:23] Names to parse: 10000 formatted: 10000 learned: 9995 parsed: 10000
[WARN] [2023-07-07 21:41:30] Names to parse: 10000 formatted: 10000 learned: 9997 parsed: 10000
[WARN] [2023-07-07 21:41:37] Names to parse: 10000 formatted: 10000 learned: 9997 parsed: 10000
[WARN] [2023-07-07 21:41:44] Names to parse: 10000 formatted: 10000 learned: 9997 parsed: 10000
[WARN] [2023-07-07 21:41:51] Names to parse: 10000 formatted: 10000 learned: 9993 parsed: 10000
[WARN] [2023-07-07 21:41:58] Names to parse: 10000 formatted: 10000 learned: 9997 parsed: 10000
[WARN] [2023-07-07 21:42:05] Names to parse: 10000 formatted: 10000 learned: 9993 parsed: 10000
[WARN] [2023-07-07 21:42:12] Names to parse: 10000 formatted: 10000 learned: 9996 parsed: 10000
[WARN] [2023-07-07 21:42:19] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2023-07-07 21:42:25] Names to parse: 2624 formatted: 2624 learned: 2623 parsed: 2624
[STOP] [2023-07-07 21:42:28] parse_names
[START] [2023-07-07 21:42:28] denormalize_canonical_names_to_nodes
[STOP] [2023-07-07 21:42:38] denormalize_canonical_names_to_nodes
[START] [2023-07-07 21:42:38] match_nodes
[START] [2023-07-07 21:42:39] map_all_nodes_to_pages
[STOP] [2023-07-07 23:39:21] map_all_nodes_to_pages
[INFO] [2023-07-07 23:39:21] 27085 Unmatched nodes (of 452624)! That's too many to output. Full list in /app/public/data/wiki_english/unmatched_nodes.txt ; First 10: Canonical: Namibiocesa barbata; Node#135160363; ResourceID: Q105972194; Canonical: Hayasakaia; Node#135161149; ResourceID: Q106169081; Canonical: Tepidamorphaceae; Node#135164874; ResourceID: Q107026917; Canonical: Kaistiaceae; Node#135164882; ResourceID: Q107027039; Canonical: Anisephyra ocularia; Node#135166675; ResourceID: Q107500881; Canonical: Desulfocapsaceae; Node#135173745; ResourceID: Q111031988; Canonical: Atribacter; Node#135174269; ResourceID: Q111621287; Canonical: Polarisedimenticola svalbardensis; Node#135174674; ResourceID: Q111936904; Canonical: Teinolophidae; Node#135175328; ResourceID: Q113499438; Canonical: Gibbavasis; Node#135176084; ResourceID: Q115526192
[START] [2023-07-07 23:39:21] update_nodes
[STOP] [2023-07-07 23:39:49] update_nodes
[STOP] [2023-07-07 23:39:49] match_nodes
[START] [2023-07-07 23:39:49] reindex_search
[STOP] [2023-07-07 23:55:54] reindex_search
[START] [2023-07-07 23:55:54] normalize_units
[STOP] [2023-07-07 23:55:54] normalize_units
[START] [2023-07-07 23:55:54] calculate_statistics
[INFO] [2023-07-07 23:56:16] Duplicate page_id count: 0
[STOP] [2023-07-07 23:56:16] calculate_statistics
[START] [2023-07-07 23:56:16] complete_harvest_instance
[START] [2023-07-07 23:56:16] overall_tsv_creation
[INFO] [2023-07-07 23:56:17] Exporting 452624 nodes as TSV in batches of 10000...
[INFO] [2023-07-07 23:56:17] Processing group of 452624 in 46 batches of 10000
[INFO] [2023-07-07 23:57:09] Processed 10000/452624 nodes
[INFO] [2023-07-07 23:58:09] Processed 20000/452624 nodes
[INFO] [2023-07-07 23:59:17] Processed 30000/452624 nodes
[INFO] [2023-07-08 00:00:33] Processed 40000/452624 nodes
[INFO] [2023-07-08 00:01:49] Processed 50000/452624 nodes
[INFO] [2023-07-08 00:03:06] Processed 60000/452624 nodes
[INFO] [2023-07-08 00:04:26] Processed 70000/452624 nodes
[INFO] [2023-07-08 00:05:41] Processed 80000/452624 nodes
[INFO] [2023-07-08 00:06:58] Processed 90000/452624 nodes
[INFO] [2023-07-08 00:08:09] Processed 100000/452624 nodes
[INFO] [2023-07-08 00:09:23] Processed 110000/452624 nodes
[INFO] [2023-07-08 00:10:33] Processed 120000/452624 nodes
[INFO] [2023-07-08 00:11:43] Processed 130000/452624 nodes
[INFO] [2023-07-08 00:12:49] Processed 140000/452624 nodes
[INFO] [2023-07-08 00:14:00] Processed 150000/452624 nodes
[INFO] [2023-07-08 00:15:09] Processed 160000/452624 nodes
[INFO] [2023-07-08 00:16:22] Processed 170000/452624 nodes
[INFO] [2023-07-08 00:17:32] Processed 180000/452624 nodes
[INFO] [2023-07-08 00:18:41] Processed 190000/452624 nodes
[INFO] [2023-07-08 00:19:57] Processed 200000/452624 nodes
[INFO] [2023-07-08 00:21:05] Processed 210000/452624 nodes
[INFO] [2023-07-08 00:22:17] Processed 220000/452624 nodes
[INFO] [2023-07-08 00:23:33] Processed 230000/452624 nodes
[INFO] [2023-07-08 00:24:45] Processed 240000/452624 nodes
[INFO] [2023-07-08 00:26:00] Processed 250000/452624 nodes
[INFO] [2023-07-08 00:27:11] Processed 260000/452624 nodes
[INFO] [2023-07-08 00:28:29] Processed 270000/452624 nodes
[INFO] [2023-07-08 00:29:41] Processed 280000/452624 nodes
[INFO] [2023-07-08 00:30:51] Processed 290000/452624 nodes
[INFO] [2023-07-08 00:32:04] Processed 300000/452624 nodes
[INFO] [2023-07-08 00:33:18] Processed 310000/452624 nodes
[INFO] [2023-07-08 00:34:31] Processed 320000/452624 nodes
[INFO] [2023-07-08 00:35:41] Processed 330000/452624 nodes
[INFO] [2023-07-08 00:37:00] Processed 340000/452624 nodes
[INFO] [2023-07-08 00:38:14] Processed 350000/452624 nodes
[INFO] [2023-07-08 00:39:26] Processed 360000/452624 nodes
[INFO] [2023-07-08 00:40:35] Processed 370000/452624 nodes
[INFO] [2023-07-08 00:41:47] Processed 380000/452624 nodes
[INFO] [2023-07-08 00:42:59] Processed 390000/452624 nodes
[INFO] [2023-07-08 00:44:17] Processed 400000/452624 nodes
[INFO] [2023-07-08 00:45:30] Processed 410000/452624 nodes
[INFO] [2023-07-08 00:46:45] Processed 420000/452624 nodes
[INFO] [2023-07-08 00:48:05] Processed 430000/452624 nodes
[INFO] [2023-07-08 00:49:18] Processed 440000/452624 nodes
[INFO] [2023-07-08 00:50:33] Processed 450000/452624 nodes
[INFO] [2023-07-08 00:50:57] Processed 452624/452624 nodes
[INFO] [2023-07-08 00:50:57] Average Time: 39.562
[INFO] [2023-07-08 00:50:57] Total Time: 54m40s
[INFO] [2023-07-08 00:50:57] last 3 / first 3: 1.02
[INFO] [2023-07-08 00:50:57] Std.Dev: 5.297; Max: 46.7
[STOP] [2023-07-08 00:50:57] overall_tsv_creation
[INFO] [2023-07-08 00:50:57] Done. Check your files:
[INFO] [2023-07-08 00:50:57] (452624 lines) /app/public/data/wiki_english/publish_nodes.tsv
[INFO] [2023-07-08 00:50:57] (452629 lines) /app/public/data/wiki_english/publish_identifiers.tsv
[INFO] [2023-07-08 00:50:57] (10949063 lines) /app/public/data/wiki_english/publish_node_ancestors.tsv
[INFO] [2023-07-08 00:50:57] (452624 lines) /app/public/data/wiki_english/publish_scientific_names.tsv
[INFO] [2023-07-08 00:50:58] (6218720 lines) /app/public/data/wiki_english/publish_articles.tsv
[INFO] [2023-07-08 00:50:58] (866565 lines) /app/public/data/wiki_english/publish_content_sections.tsv
[STOP] [2023-07-08 00:50:59] complete_harvest_instance
[START] [2023-07-08 00:50:59] completed
[STOP] [2023-07-08 00:50:59] completed
[STOP] [2023-07-08 00:50:59] logged process, took 19867.72
Latest Process