Harvest for Fishbase Created 10 May 09:30

Harvesting Log

[INFO] [2023-05-10 09:30:27] Created harvest instance #4348
[STOP] [2023-05-10 09:30:27] create_harvest_instance
[START] [2023-05-10 09:30:27] fetch_files
[STOP] [2023-05-10 09:30:27] fetch_files
[START] [2023-05-10 09:30:27] validate_each_file
[INFO] [2023-05-10 09:30:27] Looping over 7 formats...
[INFO] [2023-05-10 09:30:27] ...agents (/app/public/data/fishbase/agent.tab)
[INFO] [2023-05-10 09:30:27] Valid: /app/public/data/fishbase/converted_csv/fishbase_agents_30352.csv (160 lines)
[INFO] [2023-05-10 09:30:27] ...refs (/app/public/data/fishbase/reference.tab)
[INFO] [2023-05-10 09:30:28] Valid: /app/public/data/fishbase/converted_csv/fishbase_refs_30349.csv (36512 lines)
[INFO] [2023-05-10 09:30:28] ...nodes (/app/public/data/fishbase/taxon.tab)
[INFO] [2023-05-10 09:30:34] Valid: /app/public/data/fishbase/converted_csv/fishbase_nodes_30350.csv (98579 lines)
[INFO] [2023-05-10 09:30:34] ...media (/app/public/data/fishbase/media_resource.tab)
[INFO] [2023-05-10 09:30:42] Valid: /app/public/data/fishbase/converted_csv/fishbase_media_30353.csv (106403 lines)
[INFO] [2023-05-10 09:30:42] ...vernaculars (/app/public/data/fishbase/vernacular_name.tab)
[INFO] [2023-05-10 09:30:45] Valid: /app/public/data/fishbase/converted_csv/fishbase_vernaculars_30351.csv (187049 lines)
[INFO] [2023-05-10 09:30:45] ...occurrences (/app/public/data/fishbase/occurrence_specific.tab)
[INFO] [2023-05-10 09:30:48] Valid: /app/public/data/fishbase/converted_csv/fishbase_occurrences_30354.csv (156986 lines)
[INFO] [2023-05-10 09:30:48] ...measurements (/app/public/data/fishbase/measurement_or_fact_specific.tab)
[INFO] [2023-05-10 09:30:59] Valid: /app/public/data/fishbase/converted_csv/fishbase_measurements_30355.csv (171420 lines)
[STOP] [2023-05-10 09:30:59] validate_each_file
[START] [2023-05-10 09:30:59] convert_to_csv
[INFO] [2023-05-10 09:30:59] Looping over 7 formats...
[INFO] [2023-05-10 09:30:59] ...agents (/app/public/data/fishbase/agent.tab)
[CMD] [2023-05-10 09:30:59] /usr/bin/sort /app/public/data/fishbase/converted_csv/fishbase_agents_30352.csv > /app/public/data/fishbase/converted_csv/fishbase_agents_30352.csv_sorted
[INFO] [2023-05-10 09:30:59] Converted: /app/public/data/fishbase/converted_csv/fishbase_agents_30352.csv (160 lines)
[INFO] [2023-05-10 09:30:59] ...refs (/app/public/data/fishbase/reference.tab)
[CMD] [2023-05-10 09:30:59] /usr/bin/sort /app/public/data/fishbase/converted_csv/fishbase_refs_30349.csv > /app/public/data/fishbase/converted_csv/fishbase_refs_30349.csv_sorted
[INFO] [2023-05-10 09:30:59] Converted: /app/public/data/fishbase/converted_csv/fishbase_refs_30349.csv (36512 lines)
[INFO] [2023-05-10 09:30:59] ...nodes (/app/public/data/fishbase/taxon.tab)
[CMD] [2023-05-10 09:30:59] /usr/bin/sort /app/public/data/fishbase/converted_csv/fishbase_nodes_30350.csv > /app/public/data/fishbase/converted_csv/fishbase_nodes_30350.csv_sorted
[INFO] [2023-05-10 09:31:00] Converted: /app/public/data/fishbase/converted_csv/fishbase_nodes_30350.csv (98579 lines)
[INFO] [2023-05-10 09:31:00] ...media (/app/public/data/fishbase/media_resource.tab)
[CMD] [2023-05-10 09:31:00] /usr/bin/sort /app/public/data/fishbase/converted_csv/fishbase_media_30353.csv > /app/public/data/fishbase/converted_csv/fishbase_media_30353.csv_sorted
[INFO] [2023-05-10 09:31:01] Converted: /app/public/data/fishbase/converted_csv/fishbase_media_30353.csv (106403 lines)
[INFO] [2023-05-10 09:31:01] ...vernaculars (/app/public/data/fishbase/vernacular_name.tab)
[CMD] [2023-05-10 09:31:01] /usr/bin/sort /app/public/data/fishbase/converted_csv/fishbase_vernaculars_30351.csv > /app/public/data/fishbase/converted_csv/fishbase_vernaculars_30351.csv_sorted
[INFO] [2023-05-10 09:31:02] Converted: /app/public/data/fishbase/converted_csv/fishbase_vernaculars_30351.csv (187049 lines)
[INFO] [2023-05-10 09:31:02] ...occurrences (/app/public/data/fishbase/occurrence_specific.tab)
[CMD] [2023-05-10 09:31:02] /usr/bin/sort /app/public/data/fishbase/converted_csv/fishbase_occurrences_30354.csv > /app/public/data/fishbase/converted_csv/fishbase_occurrences_30354.csv_sorted
[INFO] [2023-05-10 09:31:02] Converted: /app/public/data/fishbase/converted_csv/fishbase_occurrences_30354.csv (156986 lines)
[INFO] [2023-05-10 09:31:02] ...measurements (/app/public/data/fishbase/measurement_or_fact_specific.tab)
[CMD] [2023-05-10 09:31:02] /usr/bin/sort /app/public/data/fishbase/converted_csv/fishbase_measurements_30355.csv > /app/public/data/fishbase/converted_csv/fishbase_measurements_30355.csv_sorted
[INFO] [2023-05-10 09:31:04] Converted: /app/public/data/fishbase/converted_csv/fishbase_measurements_30355.csv (171420 lines)
[STOP] [2023-05-10 09:31:04] convert_to_csv
[START] [2023-05-10 09:31:04] calculate_delta
[INFO] [2023-05-10 09:31:04] Looping over 7 formats...
[INFO] [2023-05-10 09:31:04] ...agents (/app/public/data/fishbase/agent.tab)
[CMD] [2023-05-10 09:31:04] echo "0a" > /app/public/data/fishbase/diff/fishbase_agents_30352.diff
[CMD] [2023-05-10 09:31:04] tail -n +1 /app/public/data/fishbase/converted_csv/fishbase_agents_30352.csv >> /app/public/data/fishbase/diff/fishbase_agents_30352.diff
[CMD] [2023-05-10 09:31:04] echo "." >> /app/public/data/fishbase/diff/fishbase_agents_30352.diff
[INFO] [2023-05-10 09:31:04] Created diff: /app/public/data/fishbase/diff/fishbase_agents_30352.diff (162 lines)
[INFO] [2023-05-10 09:31:04] ...refs (/app/public/data/fishbase/reference.tab)
[CMD] [2023-05-10 09:31:04] echo "0a" > /app/public/data/fishbase/diff/fishbase_refs_30349.diff
[CMD] [2023-05-10 09:31:04] tail -n +1 /app/public/data/fishbase/converted_csv/fishbase_refs_30349.csv >> /app/public/data/fishbase/diff/fishbase_refs_30349.diff
[CMD] [2023-05-10 09:31:05] echo "." >> /app/public/data/fishbase/diff/fishbase_refs_30349.diff
[INFO] [2023-05-10 09:31:05] Created diff: /app/public/data/fishbase/diff/fishbase_refs_30349.diff (36514 lines)
[INFO] [2023-05-10 09:31:05] ...nodes (/app/public/data/fishbase/taxon.tab)
[CMD] [2023-05-10 09:31:05] echo "0a" > /app/public/data/fishbase/diff/fishbase_nodes_30350.diff
[CMD] [2023-05-10 09:31:05] tail -n +1 /app/public/data/fishbase/converted_csv/fishbase_nodes_30350.csv >> /app/public/data/fishbase/diff/fishbase_nodes_30350.diff
[CMD] [2023-05-10 09:31:05] echo "." >> /app/public/data/fishbase/diff/fishbase_nodes_30350.diff
[INFO] [2023-05-10 09:31:05] Created diff: /app/public/data/fishbase/diff/fishbase_nodes_30350.diff (98581 lines)
[INFO] [2023-05-10 09:31:05] ...media (/app/public/data/fishbase/media_resource.tab)
[CMD] [2023-05-10 09:31:05] echo "0a" > /app/public/data/fishbase/diff/fishbase_media_30353.diff
[CMD] [2023-05-10 09:31:06] tail -n +1 /app/public/data/fishbase/converted_csv/fishbase_media_30353.csv >> /app/public/data/fishbase/diff/fishbase_media_30353.diff
[CMD] [2023-05-10 09:31:06] echo "." >> /app/public/data/fishbase/diff/fishbase_media_30353.diff
[INFO] [2023-05-10 09:31:06] Created diff: /app/public/data/fishbase/diff/fishbase_media_30353.diff (106405 lines)
[INFO] [2023-05-10 09:31:06] ...vernaculars (/app/public/data/fishbase/vernacular_name.tab)
[CMD] [2023-05-10 09:31:06] echo "0a" > /app/public/data/fishbase/diff/fishbase_vernaculars_30351.diff
[CMD] [2023-05-10 09:31:07] tail -n +1 /app/public/data/fishbase/converted_csv/fishbase_vernaculars_30351.csv >> /app/public/data/fishbase/diff/fishbase_vernaculars_30351.diff
[CMD] [2023-05-10 09:31:07] echo "." >> /app/public/data/fishbase/diff/fishbase_vernaculars_30351.diff
[INFO] [2023-05-10 09:31:07] Created diff: /app/public/data/fishbase/diff/fishbase_vernaculars_30351.diff (187051 lines)
[INFO] [2023-05-10 09:31:07] ...occurrences (/app/public/data/fishbase/occurrence_specific.tab)
[CMD] [2023-05-10 09:31:07] echo "0a" > /app/public/data/fishbase/diff/fishbase_occurrences_30354.diff
[CMD] [2023-05-10 09:31:07] tail -n +1 /app/public/data/fishbase/converted_csv/fishbase_occurrences_30354.csv >> /app/public/data/fishbase/diff/fishbase_occurrences_30354.diff
[CMD] [2023-05-10 09:31:07] echo "." >> /app/public/data/fishbase/diff/fishbase_occurrences_30354.diff
[INFO] [2023-05-10 09:31:07] Created diff: /app/public/data/fishbase/diff/fishbase_occurrences_30354.diff (156988 lines)
[INFO] [2023-05-10 09:31:07] ...measurements (/app/public/data/fishbase/measurement_or_fact_specific.tab)
[CMD] [2023-05-10 09:31:07] echo "0a" > /app/public/data/fishbase/diff/fishbase_measurements_30355.diff
[CMD] [2023-05-10 09:31:07] tail -n +1 /app/public/data/fishbase/converted_csv/fishbase_measurements_30355.csv >> /app/public/data/fishbase/diff/fishbase_measurements_30355.diff
[CMD] [2023-05-10 09:31:08] echo "." >> /app/public/data/fishbase/diff/fishbase_measurements_30355.diff
[INFO] [2023-05-10 09:31:09] Created diff: /app/public/data/fishbase/diff/fishbase_measurements_30355.diff (171422 lines)
[STOP] [2023-05-10 09:31:09] calculate_delta
[START] [2023-05-10 09:31:09] parse_diff_and_store
[INFO] [2023-05-10 09:31:09] Handling diff: /app/public/data/fishbase/diff/fishbase_agents_30352.diff (162 lines)
[INFO] [2023-05-10 09:31:09] Loading agents diff file into memory (162 lines)...
[INFO] [2023-05-10 09:31:09] Storing 160 Attributions (160/160/162)
[INFO] [2023-05-10 09:31:09] Handling diff: /app/public/data/fishbase/diff/fishbase_refs_30349.diff (36514 lines)
[INFO] [2023-05-10 09:31:09] Loading refs diff file into memory (36514 lines)...
[INFO] [2023-05-10 09:31:10] Storing 9999 References (9999/10000/36514)
[INFO] [2023-05-10 09:31:13] Storing 10000 References (19999/20000/36514)
[INFO] [2023-05-10 09:31:17] Storing 10000 References (29999/30000/36514)
[INFO] [2023-05-10 09:31:19] Storing 6513 References (36512/36512/36514)
[INFO] [2023-05-10 09:31:21] Handling diff: /app/public/data/fishbase/diff/fishbase_nodes_30350.diff (98581 lines)
[INFO] [2023-05-10 09:31:21] Loading nodes diff file into memory (98581 lines)...
[WARN] [2023-05-10 09:31:21] Filtered Scientific Name `Metriaclima sp. "aurora blue" Not applicable` to `Metriaclima sp. aurora blue Not applicable`
[WARN] [2023-05-10 09:31:22] Filtered Scientific Name `Metriaclima sp. "aurora chinuni" Not applicable` to `Metriaclima sp. aurora chinuni Not applicable`
[INFO] [2023-05-10 09:31:23] Storing 9999 ScientificNames (9999/10000/98581)
[WARN] [2023-05-10 09:31:27] Filtered Scientific Name `Nothobranchius melanospilus "Ifakara" Not applicable` to `Nothobranchius melanospilus Ifakara Not applicable`
[WARN] [2023-05-10 09:31:27] Filtered Scientific Name `Metriaclima sp. "kingsizei north" Not applicable` to `Metriaclima sp. kingsizei north Not applicable`
[WARN] [2023-05-10 09:31:28] Filtered Scientific Name `Metriaclima sp. "blue reef" Not applicable` to `Metriaclima sp. blue reef Not applicable`
[INFO] [2023-05-10 09:31:28] Storing 10000 ScientificNames (19999/20000/98581)
[WARN] [2023-05-10 09:31:32] Filtered Scientific Name `Metriaclima sp. "aurora yellow" Not applicable` to `Metriaclima sp. aurora yellow Not applicable`
[WARN] [2023-05-10 09:31:33] Filtered Scientific Name `Metriaclima sp. "aurora north" Not applicable` to `Metriaclima sp. aurora north Not applicable`
[INFO] [2023-05-10 09:31:33] Storing 10000 ScientificNames (29999/30000/98581)
[WARN] [2023-05-10 09:31:36] Filtered Scientific Name `Metriaclima sp. "aurora lumbaulo" Not applicable` to `Metriaclima sp. aurora lumbaulo Not applicable`
[WARN] [2023-05-10 09:31:37] Filtered Scientific Name `Metriaclima sp. "aurora black tail" Not applicable` to `Metriaclima sp. aurora black tail Not applicable`
[INFO] [2023-05-10 09:31:39] Storing 10219 ScientificNames (47668/40000/98581)
[INFO] [2023-05-10 09:31:42] Storing 421 Nodes (47668/40000/98581)
[INFO] [2023-05-10 09:31:42] Storing 7029 NodesReferences (47668/40000/98581)
[WARN] [2023-05-10 09:32:00] SKIPPED 186 Scientific names (348499/50000/98581) with resource_pks already be in the database!
[WARN] [2023-05-10 09:32:00] SKIPPED 186 Nodes (348499/50000/98581) with resource_pks already be in the database!
[INFO] [2023-05-10 09:32:00] Storing 13278 ScientificNames (348499/50000/98581)
[INFO] [2023-05-10 09:32:04] Storing 13278 Nodes (348499/50000/98581)
[INFO] [2023-05-10 09:32:08] Storing 273903 NodesReferences (348499/50000/98581)
[WARN] [2023-05-10 09:32:42] SKIPPED 1876 Scientific names (588388/60000/98581) with resource_pks already be in the database!
[WARN] [2023-05-10 09:32:42] SKIPPED 1876 Nodes (588388/60000/98581) with resource_pks already be in the database!
[INFO] [2023-05-10 09:32:42] Storing 11472 ScientificNames (588388/60000/98581)
[INFO] [2023-05-10 09:32:46] Storing 11472 Nodes (588388/60000/98581)
[INFO] [2023-05-10 09:32:50] Storing 213193 NodesReferences (588388/60000/98581)
[WARN] [2023-05-10 09:33:14] SKIPPED 2356 Scientific names (696582/70000/98581) with resource_pks already be in the database!
[WARN] [2023-05-10 09:33:14] SKIPPED 2356 Nodes (696582/70000/98581) with resource_pks already be in the database!
[INFO] [2023-05-10 09:33:14] Storing 10569 ScientificNames (696582/70000/98581)
[INFO] [2023-05-10 09:33:17] Storing 10569 Nodes (696582/70000/98581)
[INFO] [2023-05-10 09:33:20] Storing 82344 NodesReferences (696582/70000/98581)
[WARN] [2023-05-10 09:33:35] SKIPPED 1742 Scientific names (822930/80000/98581) with resource_pks already be in the database!
[WARN] [2023-05-10 09:33:35] SKIPPED 1742 Nodes (822930/80000/98581) with resource_pks already be in the database!
[INFO] [2023-05-10 09:33:35] Storing 10379 ScientificNames (822930/80000/98581)
[INFO] [2023-05-10 09:33:38] Storing 5310 Nodes (822930/80000/98581)
[INFO] [2023-05-10 09:33:39] Storing 107175 NodesReferences (822930/80000/98581)
[INFO] [2023-05-10 09:33:48] Storing 10000 ScientificNames (832930/90000/98581)
[INFO] [2023-05-10 09:33:54] Storing 8580 ScientificNames (841510/98579/98581)
[INFO] [2023-05-10 09:33:56] Handling diff: /app/public/data/fishbase/diff/fishbase_media_30353.diff (106405 lines)
[INFO] [2023-05-10 09:33:56] Loading media diff file into memory (106405 lines)...
[INFO] [2023-05-10 09:34:05] Storing 9968 ArticlesReferences (39965/10000/106405)
[INFO] [2023-05-10 09:34:05] Storing 9999 ContentAttributions (39965/10000/106405)
[INFO] [2023-05-10 09:34:06] Storing 9999 ArticlesSections (39965/10000/106405)
[INFO] [2023-05-10 09:34:07] Storing 9999 Articles (39965/10000/106405)
[INFO] [2023-05-10 09:34:19] Storing 9964 ArticlesReferences (79929/20000/106405)
[INFO] [2023-05-10 09:34:19] Storing 10000 ContentAttributions (79929/20000/106405)
[INFO] [2023-05-10 09:34:20] Storing 10000 ArticlesSections (79929/20000/106405)
[INFO] [2023-05-10 09:34:21] Storing 10000 Articles (79929/20000/106405)
[INFO] [2023-05-10 09:34:33] Storing 10000 ArticlesReferences (119929/30000/106405)
[INFO] [2023-05-10 09:34:34] Storing 10000 ContentAttributions (119929/30000/106405)
[INFO] [2023-05-10 09:34:35] Storing 10000 ArticlesSections (119929/30000/106405)
[INFO] [2023-05-10 09:34:35] Storing 10000 Articles (119929/30000/106405)
[INFO] [2023-05-10 09:34:48] Storing 10000 ArticlesReferences (159929/40000/106405)
[INFO] [2023-05-10 09:34:49] Storing 10000 ContentAttributions (159929/40000/106405)
[INFO] [2023-05-10 09:34:50] Storing 10000 ArticlesSections (159929/40000/106405)
[INFO] [2023-05-10 09:34:50] Storing 10000 Articles (159929/40000/106405)
[INFO] [2023-05-10 09:35:02] Storing 9917 ArticlesReferences (199846/50000/106405)
[INFO] [2023-05-10 09:35:03] Storing 10000 ContentAttributions (199846/50000/106405)
[INFO] [2023-05-10 09:35:04] Storing 10000 ArticlesSections (199846/50000/106405)
[INFO] [2023-05-10 09:35:04] Storing 10000 Articles (199846/50000/106405)
[INFO] [2023-05-10 09:35:15] Storing 9865 ArticlesReferences (239711/60000/106405)
[INFO] [2023-05-10 09:35:16] Storing 10000 ContentAttributions (239711/60000/106405)
[INFO] [2023-05-10 09:35:16] Storing 10000 ArticlesSections (239711/60000/106405)
[INFO] [2023-05-10 09:35:17] Storing 10000 Articles (239711/60000/106405)
[INFO] [2023-05-10 09:35:28] Storing 9963 ArticlesReferences (279674/70000/106405)
[INFO] [2023-05-10 09:35:28] Storing 10000 ContentAttributions (279674/70000/106405)
[INFO] [2023-05-10 09:35:29] Storing 10000 ArticlesSections (279674/70000/106405)
[INFO] [2023-05-10 09:35:30] Storing 10000 Articles (279674/70000/106405)
[INFO] [2023-05-10 09:35:40] Storing 9788 ArticlesReferences (319462/80000/106405)
[INFO] [2023-05-10 09:35:41] Storing 10000 ContentAttributions (319462/80000/106405)
[INFO] [2023-05-10 09:35:42] Storing 10000 ArticlesSections (319462/80000/106405)
[INFO] [2023-05-10 09:35:42] Storing 10000 Articles (319462/80000/106405)
[INFO] [2023-05-10 09:35:55] Storing 9804 ArticlesReferences (358924/90000/106405)
[INFO] [2023-05-10 09:35:56] Storing 10000 ContentAttributions (358924/90000/106405)
[INFO] [2023-05-10 09:35:57] Storing 9658 ArticlesSections (358924/90000/106405)
[INFO] [2023-05-10 09:35:57] Storing 9658 Articles (358924/90000/106405)
[INFO] [2023-05-10 09:36:00] Storing 342 Media (358924/90000/106405)
[INFO] [2023-05-10 09:36:04] Storing 10000 ContentAttributions (378924/100000/106405)
[INFO] [2023-05-10 09:36:05] Storing 10000 Media (378924/100000/106405)
[INFO] [2023-05-10 09:36:12] Storing 6404 ContentAttributions (391732/106403/106405)
[INFO] [2023-05-10 09:36:13] Storing 6404 Media (391732/106403/106405)
[INFO] [2023-05-10 09:36:15] Handling diff: /app/public/data/fishbase/diff/fishbase_vernaculars_30351.diff (187051 lines)
[INFO] [2023-05-10 09:36:15] Loading vernaculars diff file into memory (187051 lines)...
[INFO] [2023-05-10 09:36:16] Storing 9999 Vernaculars (9999/10000/187051)
[INFO] [2023-05-10 09:36:19] Storing 10000 Vernaculars (19999/20000/187051)
[INFO] [2023-05-10 09:36:22] Storing 10000 Vernaculars (29999/30000/187051)
[INFO] [2023-05-10 09:36:25] Storing 10000 Vernaculars (39999/40000/187051)
[INFO] [2023-05-10 09:36:27] Storing 10000 Vernaculars (49999/50000/187051)
[INFO] [2023-05-10 09:36:30] Storing 10000 Vernaculars (59999/60000/187051)
[INFO] [2023-05-10 09:36:33] Storing 10000 Vernaculars (69999/70000/187051)
[INFO] [2023-05-10 09:36:36] Storing 10000 Vernaculars (79999/80000/187051)
[INFO] [2023-05-10 09:36:38] Storing 10000 Vernaculars (89999/90000/187051)
[INFO] [2023-05-10 09:36:41] Storing 10000 Vernaculars (99999/100000/187051)
[INFO] [2023-05-10 09:36:44] Storing 10000 Vernaculars (109999/110000/187051)
[INFO] [2023-05-10 09:36:47] Storing 10000 Vernaculars (119999/120000/187051)
[INFO] [2023-05-10 09:36:49] Storing 10000 Vernaculars (129999/130000/187051)
[INFO] [2023-05-10 09:36:52] Storing 10000 Vernaculars (139999/140000/187051)
[INFO] [2023-05-10 09:36:54] Storing 10000 Vernaculars (149999/150000/187051)
[INFO] [2023-05-10 09:36:57] Storing 10000 Vernaculars (159999/160000/187051)
[INFO] [2023-05-10 09:37:00] Storing 10000 Vernaculars (169999/170000/187051)
[INFO] [2023-05-10 09:37:03] Storing 10000 Vernaculars (179999/180000/187051)
[INFO] [2023-05-10 09:37:05] Storing 7050 Vernaculars (187049/187049/187051)
[INFO] [2023-05-10 09:37:06] Handling diff: /app/public/data/fishbase/diff/fishbase_occurrences_30354.diff (156988 lines)
[INFO] [2023-05-10 09:37:06] Loading occurrences diff file into memory (156988 lines)...
[INFO] [2023-05-10 09:37:08] Storing 9999 Occurrences (14795/10000/156988)
[INFO] [2023-05-10 09:37:09] Storing 4796 OccurrenceMetadata (14795/10000/156988)
[WARN] [2023-05-10 09:37:11] SKIPPED 4751 Occurrence metadata (29546/20000/156988) with resource_pks already be in the database!
[INFO] [2023-05-10 09:37:11] Storing 10000 Occurrences (29546/20000/156988)
[INFO] [2023-05-10 09:37:13] Storing 0 OccurrenceMetadata (29546/20000/156988)
[WARN] [2023-05-10 09:37:13] No models to import, skipping!
[WARN] [2023-05-10 09:37:14] SKIPPED 4876 Occurrence metadata (44422/30000/156988) with resource_pks already be in the database!
[INFO] [2023-05-10 09:37:14] Storing 10000 Occurrences (44422/30000/156988)
[INFO] [2023-05-10 09:37:15] Storing 0 OccurrenceMetadata (44422/30000/156988)
[WARN] [2023-05-10 09:37:15] No models to import, skipping!
[WARN] [2023-05-10 09:37:17] SKIPPED 4858 Occurrence metadata (59280/40000/156988) with resource_pks already be in the database!
[INFO] [2023-05-10 09:37:17] Storing 10000 Occurrences (59280/40000/156988)
[INFO] [2023-05-10 09:37:19] Storing 0 OccurrenceMetadata (59280/40000/156988)
[WARN] [2023-05-10 09:37:19] No models to import, skipping!
[WARN] [2023-05-10 09:37:20] SKIPPED 4733 Occurrence metadata (74013/50000/156988) with resource_pks already be in the database!
[INFO] [2023-05-10 09:37:20] Storing 10000 Occurrences (74013/50000/156988)
[INFO] [2023-05-10 09:37:21] Storing 0 OccurrenceMetadata (74013/50000/156988)
[WARN] [2023-05-10 09:37:21] No models to import, skipping!
[WARN] [2023-05-10 09:37:23] SKIPPED 4732 Occurrence metadata (88745/60000/156988) with resource_pks already be in the database!
[INFO] [2023-05-10 09:37:23] Storing 10000 Occurrences (88745/60000/156988)
[INFO] [2023-05-10 09:37:24] Storing 0 OccurrenceMetadata (88745/60000/156988)
[WARN] [2023-05-10 09:37:24] No models to import, skipping!
[WARN] [2023-05-10 09:37:26] SKIPPED 4841 Occurrence metadata (103586/70000/156988) with resource_pks already be in the database!
[INFO] [2023-05-10 09:37:26] Storing 10000 Occurrences (103586/70000/156988)
[INFO] [2023-05-10 09:37:28] Storing 0 OccurrenceMetadata (103586/70000/156988)
[WARN] [2023-05-10 09:37:28] No models to import, skipping!
[WARN] [2023-05-10 09:37:29] SKIPPED 4566 Occurrence metadata (118152/80000/156988) with resource_pks already be in the database!
[INFO] [2023-05-10 09:37:29] Storing 10000 Occurrences (118152/80000/156988)
[INFO] [2023-05-10 09:37:31] Storing 0 OccurrenceMetadata (118152/80000/156988)
[WARN] [2023-05-10 09:37:31] No models to import, skipping!
[WARN] [2023-05-10 09:37:32] SKIPPED 4716 Occurrence metadata (132868/90000/156988) with resource_pks already be in the database!
[INFO] [2023-05-10 09:37:32] Storing 10000 Occurrences (132868/90000/156988)
[INFO] [2023-05-10 09:37:34] Storing 0 OccurrenceMetadata (132868/90000/156988)
[WARN] [2023-05-10 09:37:34] No models to import, skipping!
[WARN] [2023-05-10 09:37:36] SKIPPED 4697 Occurrence metadata (147565/100000/156988) with resource_pks already be in the database!
[INFO] [2023-05-10 09:37:36] Storing 10000 Occurrences (147565/100000/156988)
[INFO] [2023-05-10 09:37:37] Storing 0 OccurrenceMetadata (147565/100000/156988)
[WARN] [2023-05-10 09:37:37] No models to import, skipping!
[WARN] [2023-05-10 09:37:39] SKIPPED 4716 Occurrence metadata (162281/110000/156988) with resource_pks already be in the database!
[INFO] [2023-05-10 09:37:39] Storing 10000 Occurrences (162281/110000/156988)
[INFO] [2023-05-10 09:37:40] Storing 0 OccurrenceMetadata (162281/110000/156988)
[WARN] [2023-05-10 09:37:40] No models to import, skipping!
[WARN] [2023-05-10 09:37:42] SKIPPED 4739 Occurrence metadata (177020/120000/156988) with resource_pks already be in the database!
[INFO] [2023-05-10 09:37:42] Storing 10000 Occurrences (177020/120000/156988)
[INFO] [2023-05-10 09:37:43] Storing 0 OccurrenceMetadata (177020/120000/156988)
[WARN] [2023-05-10 09:37:43] No models to import, skipping!
[WARN] [2023-05-10 09:37:45] SKIPPED 4816 Occurrence metadata (191836/130000/156988) with resource_pks already be in the database!
[INFO] [2023-05-10 09:37:45] Storing 10000 Occurrences (191836/130000/156988)
[INFO] [2023-05-10 09:37:47] Storing 0 OccurrenceMetadata (191836/130000/156988)
[WARN] [2023-05-10 09:37:47] No models to import, skipping!
[WARN] [2023-05-10 09:37:49] SKIPPED 4779 Occurrence metadata (206615/140000/156988) with resource_pks already be in the database!
[INFO] [2023-05-10 09:37:49] Storing 10000 Occurrences (206615/140000/156988)
[INFO] [2023-05-10 09:37:50] Storing 0 OccurrenceMetadata (206615/140000/156988)
[WARN] [2023-05-10 09:37:50] No models to import, skipping!
[WARN] [2023-05-10 09:37:52] SKIPPED 4702 Occurrence metadata (221317/150000/156988) with resource_pks already be in the database!
[INFO] [2023-05-10 09:37:52] Storing 10000 Occurrences (221317/150000/156988)
[INFO] [2023-05-10 09:37:53] Storing 0 OccurrenceMetadata (221317/150000/156988)
[WARN] [2023-05-10 09:37:53] No models to import, skipping!
[WARN] [2023-05-10 09:37:55] SKIPPED 3492 Occurrence metadata (231796/156986/156988) with resource_pks already be in the database!
[INFO] [2023-05-10 09:37:55] Storing 6987 Occurrences (231796/156986/156988)
[INFO] [2023-05-10 09:37:56] Storing 0 OccurrenceMetadata (231796/156986/156988)
[WARN] [2023-05-10 09:37:56] No models to import, skipping!
[INFO] [2023-05-10 09:37:56] Handling diff: /app/public/data/fishbase/diff/fishbase_measurements_30355.diff (171422 lines)
[INFO] [2023-05-10 09:37:56] Loading measurements diff file into memory (171422 lines)...
[INFO] [2023-05-10 09:38:01] Storing 9999 Traits (28707/10000/171422)
[INFO] [2023-05-10 09:38:04] Storing 14441 MetaTraits (28707/10000/171422)
[INFO] [2023-05-10 09:38:06] Storing 4267 TraitsReferences (28707/10000/171422)
[INFO] [2023-05-10 09:38:11] Storing 4219 TraitsReferences (57386/20000/171422)
[INFO] [2023-05-10 09:38:11] Storing 10000 Traits (57386/20000/171422)
[INFO] [2023-05-10 09:38:14] Storing 14460 MetaTraits (57386/20000/171422)
[INFO] [2023-05-10 09:38:21] Storing 4203 TraitsReferences (86011/30000/171422)
[INFO] [2023-05-10 09:38:21] Storing 10000 Traits (86011/30000/171422)
[INFO] [2023-05-10 09:38:24] Storing 14422 MetaTraits (86011/30000/171422)
[INFO] [2023-05-10 09:38:31] Storing 4260 TraitsReferences (114754/40000/171422)
[INFO] [2023-05-10 09:38:31] Storing 10000 Traits (114754/40000/171422)
[INFO] [2023-05-10 09:38:35] Storing 14483 MetaTraits (114754/40000/171422)
[INFO] [2023-05-10 09:38:41] Storing 10000 Traits (143528/50000/171422)
[INFO] [2023-05-10 09:38:45] Storing 14527 MetaTraits (143528/50000/171422)
[INFO] [2023-05-10 09:38:47] Storing 4247 TraitsReferences (143528/50000/171422)
[INFO] [2023-05-10 09:38:52] Storing 10000 Traits (172161/60000/171422)
[INFO] [2023-05-10 09:38:57] Storing 14430 MetaTraits (172161/60000/171422)
[INFO] [2023-05-10 09:38:59] Storing 4203 TraitsReferences (172161/60000/171422)
[INFO] [2023-05-10 09:39:04] Storing 10000 Traits (200845/70000/171422)
[INFO] [2023-05-10 09:39:08] Storing 14492 MetaTraits (200845/70000/171422)
[INFO] [2023-05-10 09:39:10] Storing 4192 TraitsReferences (200845/70000/171422)
[INFO] [2023-05-10 09:39:15] Storing 4169 TraitsReferences (229462/80000/171422)
[INFO] [2023-05-10 09:39:16] Storing 10000 Traits (229462/80000/171422)
[INFO] [2023-05-10 09:39:19] Storing 14448 MetaTraits (229462/80000/171422)
[INFO] [2023-05-10 09:39:26] Storing 10000 Traits (258143/90000/171422)
[INFO] [2023-05-10 09:39:29] Storing 14506 MetaTraits (258143/90000/171422)
[INFO] [2023-05-10 09:39:31] Storing 4175 TraitsReferences (258143/90000/171422)
[INFO] [2023-05-10 09:39:37] Storing 10000 Traits (287040/100000/171422)
[INFO] [2023-05-10 09:39:40] Storing 14529 MetaTraits (287040/100000/171422)
[INFO] [2023-05-10 09:39:42] Storing 4368 TraitsReferences (287040/100000/171422)
[INFO] [2023-05-10 09:39:48] Storing 10000 Traits (315783/110000/171422)
[INFO] [2023-05-10 09:39:51] Storing 14523 MetaTraits (315783/110000/171422)
[INFO] [2023-05-10 09:39:53] Storing 4220 TraitsReferences (315783/110000/171422)
[INFO] [2023-05-10 09:39:59] Storing 10000 Traits (344490/120000/171422)
[INFO] [2023-05-10 09:40:03] Storing 14553 MetaTraits (344490/120000/171422)
[INFO] [2023-05-10 09:40:05] Storing 4154 TraitsReferences (344490/120000/171422)
[INFO] [2023-05-10 09:40:11] Storing 4128 TraitsReferences (373086/130000/171422)
[INFO] [2023-05-10 09:40:11] Storing 10000 Traits (373086/130000/171422)
[INFO] [2023-05-10 09:40:14] Storing 14468 MetaTraits (373086/130000/171422)
[INFO] [2023-05-10 09:40:22] Storing 4277 TraitsReferences (401776/140000/171422)
[INFO] [2023-05-10 09:40:22] Storing 10000 Traits (401776/140000/171422)
[INFO] [2023-05-10 09:40:25] Storing 14413 MetaTraits (401776/140000/171422)
[INFO] [2023-05-10 09:40:33] Storing 10000 Traits (430411/150000/171422)
[INFO] [2023-05-10 09:40:36] Storing 14436 MetaTraits (430411/150000/171422)
[INFO] [2023-05-10 09:40:38] Storing 4199 TraitsReferences (430411/150000/171422)
[INFO] [2023-05-10 09:40:43] Storing 10000 Traits (459091/160000/171422)
[INFO] [2023-05-10 09:40:47] Storing 14424 MetaTraits (459091/160000/171422)
[INFO] [2023-05-10 09:40:48] Storing 4256 TraitsReferences (459091/160000/171422)
[INFO] [2023-05-10 09:40:54] Storing 4195 TraitsReferences (487830/170000/171422)
[INFO] [2023-05-10 09:40:55] Storing 10000 Traits (487830/170000/171422)
[INFO] [2023-05-10 09:40:58] Storing 14544 MetaTraits (487830/170000/171422)
[INFO] [2023-05-10 09:41:01] Storing 1421 Traits (491946/171420/171422)
[INFO] [2023-05-10 09:41:02] Storing 2072 MetaTraits (491946/171420/171422)
[INFO] [2023-05-10 09:41:02] Storing 623 TraitsReferences (491946/171420/171422)
[STOP] [2023-05-10 09:41:02] parse_diff_and_store
[START] [2023-05-10 09:41:02] resolve_keys
[2023-05-10 09:41:56] Resolving downloaded urls (this is not actually downloading them yet)
[INFO] [2023-05-10 09:42:22] Occurrences to nodes (through scientific_names)...
[INFO] [2023-05-10 09:42:26] traits to occurrences...
[INFO] [2023-05-10 09:42:33] traits to nodes (through occurrences)...
[INFO] [2023-05-10 09:42:38] Traits to sex term...
[INFO] [2023-05-10 09:42:41] Traits to lifestage term...
[INFO] [2023-05-10 09:42:44] MetaTraits to traits...
[INFO] [2023-05-10 09:42:50] MetaTraits (simple, measurement row refers to parent) to traits...
[INFO] [2023-05-10 09:42:55] Assocs to occurrences...
[INFO] [2023-05-10 09:42:55] Assocs to nodes...
[INFO] [2023-05-10 09:42:55] Assoc to sex term...
[INFO] [2023-05-10 09:42:55] Assoc to lifestage term...
[INFO] [2023-05-10 09:42:55] MetaAssoc to assocs...
[STOP] [2023-05-10 09:42:58] resolve_keys
[START] [2023-05-10 09:42:58] hold_for_later_1
[STOP] [2023-05-10 09:42:58] hold_for_later_1
[START] [2023-05-10 09:42:58] hold_for_later_2
[STOP] [2023-05-10 09:42:58] hold_for_later_2
[START] [2023-05-10 09:42:58] resolve_missing_parents
[STOP] [2023-05-10 09:42:59] resolve_missing_parents
[START] [2023-05-10 09:42:59] rebuild_nodes
[START] [2023-05-10 09:42:59] Flattener#flatten
[START] [2023-05-10 09:42:59] Flattener#study_resource
[START] [2023-05-10 09:42:59] Flattener#build_ancestry
[STOP] [2023-05-10 09:43:04] Flattener#build_ancestry
[INFO] [2023-05-10 09:43:04] 41050 ancestry keys
[START] [2023-05-10 09:43:04] build_node_ancestors
[INFO] [2023-05-10 09:43:04] old ancestors deleted.
[STOP] [2023-05-10 09:43:25] build_node_ancestors
[START] [2023-05-10 09:43:27] Flattener#propagate_ancestor_ids
[STOP] [2023-05-10 09:43:33] Flattener#propagate_ancestor_ids
[STOP] [2023-05-10 09:43:33] Flattener#flatten
[STOP] [2023-05-10 09:43:33] rebuild_nodes
[START] [2023-05-10 09:43:33] resolve_missing_media_owners
[STOP] [2023-05-10 09:43:33] resolve_missing_media_owners
[START] [2023-05-10 09:43:33] sanitize_media_verbatims
[STOP] [2023-05-10 09:43:33] sanitize_media_verbatims
[START] [2023-05-10 09:43:33] queue_downloads
[STOP] [2023-05-10 09:43:33] queue_downloads
[START] [2023-05-10 09:43:33] parse_names
[WARN] [2023-05-10 09:43:34] I see 104496 names which still need to be parsed.
[INFO] [2023-05-10 09:43:34] 0% of media downloaded
[INFO] [2023-05-10 09:43:34] 0% of media downloaded
[INFO] [2023-05-10 09:43:34] 0% of media downloaded
[WARN] [2023-05-10 09:43:36] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[INFO] [2023-05-10 09:43:36] 0% of media downloaded
[WARN] [2023-05-10 09:43:44] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2023-05-10 09:43:53] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2023-05-10 09:44:02] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2023-05-10 09:44:11] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2023-05-10 09:44:20] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2023-05-10 09:44:30] Names to parse: 10000 formatted: 10000 learned: 9998 parsed: 10000
[WARN] [2023-05-10 09:44:40] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2023-05-10 09:44:49] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2023-05-10 09:44:59] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2023-05-10 09:45:08] Names to parse: 4496 formatted: 4496 learned: 4496 parsed: 4496
[STOP] [2023-05-10 09:45:12] parse_names
[START] [2023-05-10 09:45:12] denormalize_canonical_names_to_nodes
[STOP] [2023-05-10 09:45:13] denormalize_canonical_names_to_nodes
[START] [2023-05-10 09:45:13] match_nodes
[START] [2023-05-10 09:45:13] map_all_nodes_to_pages
[INFO] [2023-05-10 09:53:02] 100% of media downloaded
[INFO] [2023-05-10 09:53:02] 100% of media downloaded
[INFO] [2023-05-10 09:53:02] 100% of media downloaded
[INFO] [2023-05-10 09:53:02] 100% of media downloaded
[INFO] [2023-05-10 09:53:02] 100% of media downloaded
[ERR] [2023-05-10 09:53:02][hdls] NO additional images were found to download
[ERR] [2023-05-10 09:53:02][hdls] NO additional images were found to download
[ERR] [2023-05-10 09:53:02][hdls] NO additional images were found to download
[ERR] [2023-05-10 09:53:02][hdls] NO additional images were found to download
[ERR] [2023-05-10 09:53:02][hdls] NO additional images were found to download
[ERR] [2023-05-10 09:53:02][hdls] NO additional images were found to download
[ERR] [2023-05-10 09:53:02][hdls] NO additional images were found to download
[ERR] [2023-05-10 09:53:02][hdls] NO additional images were found to download
[ERR] [2023-05-10 09:53:02][hdls] NO additional images were found to download
[ERR] [2023-05-10 09:53:02][hdls] NO additional images were found to download
[ERR] [2023-05-10 09:53:02][hdls] NO additional images were found to download
[ERR] [2023-05-10 09:53:02][hdls] NO additional images were found to download
[STOP] [2023-05-10 10:07:02] map_all_nodes_to_pages
[INFO] [2023-05-10 10:07:02] 689 Unmatched nodes (of 41050)! That's too many to output. Full list in /app/public/data/fishbase/unmatched_nodes.txt ; First 10: Canonical: Pampus cinereus; Node#134777056; ResourceID: FB-69458; Canonical: Trichiurus japonicus; Node#134777336; ResourceID: FB-69710; Canonical: Pseudogobiopsis lumbantobing; Node#134776839; ResourceID: FB-69261; Canonical: Pseudogobiopsis rubrimaculosa; Node#134776923; ResourceID: FB-69335; Canonical: Schismatogobius limmoni; Node#134777388; ResourceID: FB-69756; Canonical: Eviota maculosa; Node#134776530; ResourceID: FB-68987; Canonical: Eviota gunawanae; Node#134776532; ResourceID: FB-68989; Canonical: Amblyeleotris cephalotaenia; Node#134767499; ResourceID: FB-60453; Canonical: Gobius xoriguer; Node#134776972; ResourceID: FB-69383; Canonical: Trimma unisquame; Node#134765377; ResourceID: FB-58501
[START] [2023-05-10 10:07:02] update_nodes
[STOP] [2023-05-10 10:07:20] update_nodes
[STOP] [2023-05-10 10:07:20] match_nodes
[START] [2023-05-10 10:07:20] reindex_search
[STOP] [2023-05-10 10:08:06] reindex_search
[START] [2023-05-10 10:08:06] normalize_units
[STOP] [2023-05-10 10:11:50] normalize_units
[START] [2023-05-10 10:11:50] calculate_statistics
[INFO] [2023-05-10 10:12:00] Duplicate page_id count: 0
[STOP] [2023-05-10 10:12:00] calculate_statistics
[START] [2023-05-10 10:12:00] complete_harvest_instance
[START] [2023-05-10 10:12:00] overall_tsv_creation
[INFO] [2023-05-10 10:12:00] Exporting 41050 nodes as TSV in batches of 10000...
[INFO] [2023-05-10 10:12:00] Processing group of 41050 in 5 batches of 10000
[INFO] [2023-05-10 10:13:43] 37539 Traits (unfiltered) and 0 associations...
[INFO] [2023-05-10 10:13:43] Building Traits map for 10000 nodes (this can take a while)...
[INFO] [2023-05-10 10:14:30] Mapped 37539 traits (54308 meta) for 10000 nodes.
[INFO] [2023-05-10 10:14:30] Building Associations map (this can take a while)...
[INFO] [2023-05-10 10:14:33] Done. 0 assocs mapped (0 meta).
[INFO] [2023-05-10 10:14:33] Adding 37539 traits...
[INFO] [2023-05-10 10:14:42] 17547 metadata added.
[INFO] [2023-05-10 10:14:42] Adding 0 assocs...
[INFO] [2023-05-10 10:14:42] 0 metadata added.
[INFO] [2023-05-10 10:15:32] Processed 10000/41050 nodes
[INFO] [2023-05-10 10:17:58] 42104 Traits (unfiltered) and 0 associations...
[INFO] [2023-05-10 10:17:58] Building Traits map for 10000 nodes (this can take a while)...
[INFO] [2023-05-10 10:19:01] Mapped 42104 traits (61553 meta) for 10000 nodes.
[INFO] [2023-05-10 10:19:01] Building Associations map (this can take a while)...
[INFO] [2023-05-10 10:19:04] Done. 0 assocs mapped (0 meta).
[INFO] [2023-05-10 10:19:04] Adding 42104 traits...
[INFO] [2023-05-10 10:19:14] 18188 metadata added.
[INFO] [2023-05-10 10:19:14] Adding 0 assocs...
[INFO] [2023-05-10 10:19:14] 0 metadata added.
[INFO] [2023-05-10 10:20:12] Processed 20000/41050 nodes
[INFO] [2023-05-10 10:22:11] 41365 Traits (unfiltered) and 0 associations...
[INFO] [2023-05-10 10:22:11] Building Traits map for 10000 nodes (this can take a while)...
[INFO] [2023-05-10 10:23:12] Mapped 41365 traits (59447 meta) for 10000 nodes.
[INFO] [2023-05-10 10:23:12] Building Associations map (this can take a while)...
[INFO] [2023-05-10 10:23:14] Done. 0 assocs mapped (0 meta).
[INFO] [2023-05-10 10:23:14] Adding 41365 traits...
[INFO] [2023-05-10 10:23:25] 15797 metadata added.
[INFO] [2023-05-10 10:23:25] Adding 0 assocs...
[INFO] [2023-05-10 10:23:25] 0 metadata added.
[INFO] [2023-05-10 10:24:21] Processed 30000/41050 nodes
[INFO] [2023-05-10 10:26:43] 45639 Traits (unfiltered) and 0 associations...
[INFO] [2023-05-10 10:26:43] Building Traits map for 10000 nodes (this can take a while)...
[INFO] [2023-05-10 10:27:52] Mapped 45639 traits (66072 meta) for 10000 nodes.
[INFO] [2023-05-10 10:27:52] Building Associations map (this can take a while)...
[INFO] [2023-05-10 10:27:54] Done. 0 assocs mapped (0 meta).
[INFO] [2023-05-10 10:27:54] Adding 45639 traits...
[INFO] [2023-05-10 10:28:11] 18867 metadata added.
[INFO] [2023-05-10 10:28:11] Adding 0 assocs...
[INFO] [2023-05-10 10:28:11] 0 metadata added.
[INFO] [2023-05-10 10:29:04] Processed 40000/41050 nodes
[INFO] [2023-05-10 10:29:32] 4773 Traits (unfiltered) and 0 associations...
[INFO] [2023-05-10 10:29:32] Building Traits map for 1050 nodes (this can take a while)...
[INFO] [2023-05-10 10:29:35] Mapped 4773 traits (6791 meta) for 1050 nodes.
[INFO] [2023-05-10 10:29:35] Building Associations map (this can take a while)...
[INFO] [2023-05-10 10:29:35] Done. 0 assocs mapped (0 meta).
[INFO] [2023-05-10 10:29:35] Adding 4773 traits...
[INFO] [2023-05-10 10:29:35] 1941 metadata added.
[INFO] [2023-05-10 10:29:35] Adding 0 assocs...
[INFO] [2023-05-10 10:29:35] 0 metadata added.
[INFO] [2023-05-10 10:30:20] Processed 41050/41050 nodes
[INFO] [2023-05-10 10:30:20] Average Time: 176.712
[INFO] [2023-05-10 10:30:20] Total Time: 18m20s
[STOP] [2023-05-10 10:30:20] overall_tsv_creation
[INFO] [2023-05-10 10:30:20] Done. Check your files:
[INFO] [2023-05-10 10:30:20] (41050 lines) /app/public/data/fishbase/publish_nodes.tsv
[INFO] [2023-05-10 10:30:20] (239535 lines) /app/public/data/fishbase/publish_node_ancestors.tsv
[INFO] [2023-05-10 10:30:20] (104496 lines) /app/public/data/fishbase/publish_scientific_names.tsv
[INFO] [2023-05-10 10:30:21] (16746 lines) /app/public/data/fishbase/publish_media.tsv
[INFO] [2023-05-10 10:30:21] (89657 lines) /app/public/data/fishbase/publish_articles.tsv
[INFO] [2023-05-10 10:30:21] (16746 lines) /app/public/data/fishbase/publish_image_info.tsv
[INFO] [2023-05-10 10:30:21] (187049 lines) /app/public/data/fishbase/publish_vernaculars.tsv
[INFO] [2023-05-10 10:30:21] (36443 lines) /app/public/data/fishbase/publish_references.tsv
[INFO] [2023-05-10 10:30:21] (106403 lines) /app/public/data/fishbase/publish_attributions.tsv
[INFO] [2023-05-10 10:30:22] (89657 lines) /app/public/data/fishbase/publish_content_sections.tsv
[INFO] [2023-05-10 10:30:22] (36443 lines) /app/public/data/fishbase/publish_referents.tsv
[INFO] [2023-05-10 10:30:22] (171421 lines) /app/public/data/fishbase/publish_traits.tsv
[INFO] [2023-05-10 10:30:22] (72341 lines) /app/public/data/fishbase/publish_metadata.tsv
[STOP] [2023-05-10 10:30:22] complete_harvest_instance
[START] [2023-05-10 10:30:22] completed
[STOP] [2023-05-10 10:30:22] completed
[STOP] [2023-05-10 10:30:22] logged process, took 3595.3

Latest Process

10 May 09:30 EXITED currently processing 0