Harvest for Sal et al Created 10 Jun 12:48

Harvesting Log

[INFO] [2021-06-10 12:48:48] Created harvest instance #4027
[STOP] [2021-06-10 12:48:48] create_harvest_instance
[START] [2021-06-10 12:48:48] fetch_files
[STOP] [2021-06-10 12:48:48] fetch_files
[START] [2021-06-10 12:48:48] validate_each_file
[INFO] [2021-06-10 12:48:48] Looping over 8 formats...
[INFO] [2021-06-10 12:48:48] ...agents (/app/public/data/Sal_et_al/agents.txt)
[INFO] [2021-06-10 12:48:48] Valid: /app/public/converted_csv/Sal_et_al_agents_4027.csv (0 lines)
[INFO] [2021-06-10 12:48:48] ...refs (/app/public/data/Sal_et_al/references.txt)
[INFO] [2021-06-10 12:48:48] Valid: /app/public/converted_csv/Sal_et_al_refs_4027.csv (0 lines)
[INFO] [2021-06-10 12:48:48] ...nodes (/app/public/data/Sal_et_al/taxa.txt)
[INFO] [2021-06-10 12:48:48] Valid: /app/public/converted_csv/Sal_et_al_nodes_4027.csv (513 lines)
[INFO] [2021-06-10 12:48:48] ...media (/app/public/data/Sal_et_al/media.txt)
[INFO] [2021-06-10 12:48:48] Valid: /app/public/converted_csv/Sal_et_al_media_4027.csv (0 lines)
[INFO] [2021-06-10 12:48:48] ...vernaculars (/app/public/data/Sal_et_al/common names.txt)
[INFO] [2021-06-10 12:48:48] Valid: /app/public/converted_csv/Sal_et_al_vernaculars_4027.csv (0 lines)
[INFO] [2021-06-10 12:48:48] ...occurrences (/app/public/data/Sal_et_al/occurrences.txt)
[INFO] [2021-06-10 12:48:49] Valid: /app/public/converted_csv/Sal_et_al_occurrences_4027.csv (1234 lines)
[INFO] [2021-06-10 12:48:49] ...assocs (/app/public/data/Sal_et_al/associations.txt)
[INFO] [2021-06-10 12:48:49] Valid: /app/public/converted_csv/Sal_et_al_assocs_4027.csv (0 lines)
[INFO] [2021-06-10 12:48:49] ...measurements (/app/public/data/Sal_et_al/measurements or facts.txt)
[INFO] [2021-06-10 12:48:49] Valid: /app/public/converted_csv/Sal_et_al_measurements_4027.csv (2468 lines)
[STOP] [2021-06-10 12:48:49] validate_each_file
[START] [2021-06-10 12:48:49] convert_to_csv
[INFO] [2021-06-10 12:48:49] Looping over 8 formats...
[INFO] [2021-06-10 12:48:49] ...agents (/app/public/data/Sal_et_al/agents.txt)
[CMD] [2021-06-10 12:48:49] /usr/bin/sort /app/public/converted_csv/Sal_et_al_agents_4027.csv > /app/public/converted_csv/Sal_et_al_agents_4027.csv_sorted
[INFO] [2021-06-10 12:48:50] Converted: /app/public/converted_csv/Sal_et_al_agents_4027.csv (0 lines)
[INFO] [2021-06-10 12:48:50] ...refs (/app/public/data/Sal_et_al/references.txt)
[CMD] [2021-06-10 12:48:50] /usr/bin/sort /app/public/converted_csv/Sal_et_al_refs_4027.csv > /app/public/converted_csv/Sal_et_al_refs_4027.csv_sorted
[INFO] [2021-06-10 12:48:50] Converted: /app/public/converted_csv/Sal_et_al_refs_4027.csv (0 lines)
[INFO] [2021-06-10 12:48:50] ...nodes (/app/public/data/Sal_et_al/taxa.txt)
[CMD] [2021-06-10 12:48:50] /usr/bin/sort /app/public/converted_csv/Sal_et_al_nodes_4027.csv > /app/public/converted_csv/Sal_et_al_nodes_4027.csv_sorted
[INFO] [2021-06-10 12:48:51] Converted: /app/public/converted_csv/Sal_et_al_nodes_4027.csv (513 lines)
[INFO] [2021-06-10 12:48:51] ...media (/app/public/data/Sal_et_al/media.txt)
[CMD] [2021-06-10 12:48:51] /usr/bin/sort /app/public/converted_csv/Sal_et_al_media_4027.csv > /app/public/converted_csv/Sal_et_al_media_4027.csv_sorted
[INFO] [2021-06-10 12:48:51] Converted: /app/public/converted_csv/Sal_et_al_media_4027.csv (0 lines)
[INFO] [2021-06-10 12:48:51] ...vernaculars (/app/public/data/Sal_et_al/common names.txt)
[CMD] [2021-06-10 12:48:51] /usr/bin/sort /app/public/converted_csv/Sal_et_al_vernaculars_4027.csv > /app/public/converted_csv/Sal_et_al_vernaculars_4027.csv_sorted
[INFO] [2021-06-10 12:48:52] Converted: /app/public/converted_csv/Sal_et_al_vernaculars_4027.csv (0 lines)
[INFO] [2021-06-10 12:48:52] ...occurrences (/app/public/data/Sal_et_al/occurrences.txt)
[CMD] [2021-06-10 12:48:52] /usr/bin/sort /app/public/converted_csv/Sal_et_al_occurrences_4027.csv > /app/public/converted_csv/Sal_et_al_occurrences_4027.csv_sorted
[INFO] [2021-06-10 12:48:52] Converted: /app/public/converted_csv/Sal_et_al_occurrences_4027.csv (1234 lines)
[INFO] [2021-06-10 12:48:52] ...assocs (/app/public/data/Sal_et_al/associations.txt)
[CMD] [2021-06-10 12:48:52] /usr/bin/sort /app/public/converted_csv/Sal_et_al_assocs_4027.csv > /app/public/converted_csv/Sal_et_al_assocs_4027.csv_sorted
[INFO] [2021-06-10 12:48:53] Converted: /app/public/converted_csv/Sal_et_al_assocs_4027.csv (0 lines)
[INFO] [2021-06-10 12:48:53] ...measurements (/app/public/data/Sal_et_al/measurements or facts.txt)
[CMD] [2021-06-10 12:48:53] /usr/bin/sort /app/public/converted_csv/Sal_et_al_measurements_4027.csv > /app/public/converted_csv/Sal_et_al_measurements_4027.csv_sorted
[INFO] [2021-06-10 12:48:53] Converted: /app/public/converted_csv/Sal_et_al_measurements_4027.csv (2468 lines)
[STOP] [2021-06-10 12:48:53] convert_to_csv
[START] [2021-06-10 12:48:53] calculate_delta
[INFO] [2021-06-10 12:48:53] Looping over 8 formats...
[INFO] [2021-06-10 12:48:53] ...agents (/app/public/data/Sal_et_al/agents.txt)
[CMD] [2021-06-10 12:48:53] echo "0a" > /app/public/diff/Sal_et_al_agents_4027.diff
[CMD] [2021-06-10 12:48:54] tail -n +1 /app/public/converted_csv/Sal_et_al_agents_4027.csv >> /app/public/diff/Sal_et_al_agents_4027.diff
[CMD] [2021-06-10 12:48:54] echo "." >> /app/public/diff/Sal_et_al_agents_4027.diff
[INFO] [2021-06-10 12:48:55] Created diff: /app/public/diff/Sal_et_al_agents_4027.diff (2 lines)
[INFO] [2021-06-10 12:48:55] ...refs (/app/public/data/Sal_et_al/references.txt)
[CMD] [2021-06-10 12:48:55] echo "0a" > /app/public/diff/Sal_et_al_refs_4027.diff
[CMD] [2021-06-10 12:48:55] tail -n +1 /app/public/converted_csv/Sal_et_al_refs_4027.csv >> /app/public/diff/Sal_et_al_refs_4027.diff
[CMD] [2021-06-10 12:48:55] echo "." >> /app/public/diff/Sal_et_al_refs_4027.diff
[INFO] [2021-06-10 12:48:56] Created diff: /app/public/diff/Sal_et_al_refs_4027.diff (2 lines)
[INFO] [2021-06-10 12:48:56] ...nodes (/app/public/data/Sal_et_al/taxa.txt)
[CMD] [2021-06-10 12:48:56] echo "0a" > /app/public/diff/Sal_et_al_nodes_4027.diff
[CMD] [2021-06-10 12:48:56] tail -n +1 /app/public/converted_csv/Sal_et_al_nodes_4027.csv >> /app/public/diff/Sal_et_al_nodes_4027.diff
[CMD] [2021-06-10 12:48:57] echo "." >> /app/public/diff/Sal_et_al_nodes_4027.diff
[INFO] [2021-06-10 12:48:57] Created diff: /app/public/diff/Sal_et_al_nodes_4027.diff (515 lines)
[INFO] [2021-06-10 12:48:57] ...media (/app/public/data/Sal_et_al/media.txt)
[CMD] [2021-06-10 12:48:57] echo "0a" > /app/public/diff/Sal_et_al_media_4027.diff
[CMD] [2021-06-10 12:48:58] tail -n +1 /app/public/converted_csv/Sal_et_al_media_4027.csv >> /app/public/diff/Sal_et_al_media_4027.diff
[CMD] [2021-06-10 12:48:58] echo "." >> /app/public/diff/Sal_et_al_media_4027.diff
[INFO] [2021-06-10 12:48:59] Created diff: /app/public/diff/Sal_et_al_media_4027.diff (2 lines)
[INFO] [2021-06-10 12:48:59] ...vernaculars (/app/public/data/Sal_et_al/common names.txt)
[CMD] [2021-06-10 12:48:59] echo "0a" > /app/public/diff/Sal_et_al_vernaculars_4027.diff
[CMD] [2021-06-10 12:48:59] tail -n +1 /app/public/converted_csv/Sal_et_al_vernaculars_4027.csv >> /app/public/diff/Sal_et_al_vernaculars_4027.diff
[CMD] [2021-06-10 12:49:00] echo "." >> /app/public/diff/Sal_et_al_vernaculars_4027.diff
[INFO] [2021-06-10 12:49:00] Created diff: /app/public/diff/Sal_et_al_vernaculars_4027.diff (2 lines)
[INFO] [2021-06-10 12:49:00] ...occurrences (/app/public/data/Sal_et_al/occurrences.txt)
[CMD] [2021-06-10 12:49:00] echo "0a" > /app/public/diff/Sal_et_al_occurrences_4027.diff
[CMD] [2021-06-10 12:49:01] tail -n +1 /app/public/converted_csv/Sal_et_al_occurrences_4027.csv >> /app/public/diff/Sal_et_al_occurrences_4027.diff
[CMD] [2021-06-10 12:49:01] echo "." >> /app/public/diff/Sal_et_al_occurrences_4027.diff
[INFO] [2021-06-10 12:49:02] Created diff: /app/public/diff/Sal_et_al_occurrences_4027.diff (1236 lines)
[INFO] [2021-06-10 12:49:02] ...assocs (/app/public/data/Sal_et_al/associations.txt)
[CMD] [2021-06-10 12:49:02] echo "0a" > /app/public/diff/Sal_et_al_assocs_4027.diff
[CMD] [2021-06-10 12:49:02] tail -n +1 /app/public/converted_csv/Sal_et_al_assocs_4027.csv >> /app/public/diff/Sal_et_al_assocs_4027.diff
[CMD] [2021-06-10 12:49:03] echo "." >> /app/public/diff/Sal_et_al_assocs_4027.diff
[INFO] [2021-06-10 12:49:03] Created diff: /app/public/diff/Sal_et_al_assocs_4027.diff (2 lines)
[INFO] [2021-06-10 12:49:03] ...measurements (/app/public/data/Sal_et_al/measurements or facts.txt)
[CMD] [2021-06-10 12:49:03] echo "0a" > /app/public/diff/Sal_et_al_measurements_4027.diff
[CMD] [2021-06-10 12:49:04] tail -n +1 /app/public/converted_csv/Sal_et_al_measurements_4027.csv >> /app/public/diff/Sal_et_al_measurements_4027.diff
[CMD] [2021-06-10 12:49:04] echo "." >> /app/public/diff/Sal_et_al_measurements_4027.diff
[INFO] [2021-06-10 12:49:05] Created diff: /app/public/diff/Sal_et_al_measurements_4027.diff (2470 lines)
[STOP] [2021-06-10 12:49:05] calculate_delta
[START] [2021-06-10 12:49:05] parse_diff_and_store
[INFO] [2021-06-10 12:49:05] Handling diff: /app/public/diff/Sal_et_al_agents_4027.diff (2 lines)
[INFO] [2021-06-10 12:49:05] Loading agents diff file into memory (2 /app/public/diff/Sal_et_al_agents_4027.diff lines)...
[INFO] [2021-06-10 12:49:05] Handling diff: /app/public/diff/Sal_et_al_refs_4027.diff (2 lines)
[INFO] [2021-06-10 12:49:06] Loading refs diff file into memory (2 /app/public/diff/Sal_et_al_refs_4027.diff lines)...
[INFO] [2021-06-10 12:49:06] Handling diff: /app/public/diff/Sal_et_al_nodes_4027.diff (515 lines)
[INFO] [2021-06-10 12:49:07] Loading nodes diff file into memory (515 /app/public/diff/Sal_et_al_nodes_4027.diff lines)...
[INFO] [2021-06-10 12:49:08] Handling diff: /app/public/diff/Sal_et_al_media_4027.diff (2 lines)
[INFO] [2021-06-10 12:49:08] Loading media diff file into memory (2 /app/public/diff/Sal_et_al_media_4027.diff lines)...
[INFO] [2021-06-10 12:49:09] Handling diff: /app/public/diff/Sal_et_al_vernaculars_4027.diff (2 lines)
[INFO] [2021-06-10 12:49:09] Loading vernaculars diff file into memory (2 /app/public/diff/Sal_et_al_vernaculars_4027.diff lines)...
[INFO] [2021-06-10 12:49:09] Handling diff: /app/public/diff/Sal_et_al_occurrences_4027.diff (1236 lines)
[INFO] [2021-06-10 12:49:10] Loading occurrences diff file into memory (1236 /app/public/diff/Sal_et_al_occurrences_4027.diff lines)...
[INFO] [2021-06-10 12:49:10] Handling diff: /app/public/diff/Sal_et_al_assocs_4027.diff (2 lines)
[INFO] [2021-06-10 12:49:11] Loading assocs diff file into memory (2 /app/public/diff/Sal_et_al_assocs_4027.diff lines)...
[INFO] [2021-06-10 12:49:11] Handling diff: /app/public/diff/Sal_et_al_measurements_4027.diff (2470 lines)
[INFO] [2021-06-10 12:49:12] Loading measurements diff file into memory (2470 /app/public/diff/Sal_et_al_measurements_4027.diff lines)...
[INFO] [2021-06-10 12:49:14] Storing 874 ScientificNames
[INFO] [2021-06-10 12:49:14] Processing group of 874 in 1 groups of 1000
[INFO] [2021-06-10 12:49:14] Average Time: 0.52
[INFO] [2021-06-10 12:49:14] Total Time: 1s
[INFO] [2021-06-10 12:49:14] Storing 874 Nodes
[INFO] [2021-06-10 12:49:14] Processing group of 874 in 1 groups of 1000
[INFO] [2021-06-10 12:49:15] Average Time: 0.28
[INFO] [2021-06-10 12:49:15] Total Time: 1s
[INFO] [2021-06-10 12:49:15] Storing 1234 Occurrences
[INFO] [2021-06-10 12:49:15] Processing group of 1234 in 2 groups of 1000
[INFO] [2021-06-10 12:49:15] Average Time: 0.075
[INFO] [2021-06-10 12:49:15] Total Time: 1s
[INFO] [2021-06-10 12:49:15] Storing 2468 Traits
[INFO] [2021-06-10 12:49:15] Processing group of 2468 in 3 groups of 1000
[INFO] [2021-06-10 12:49:16] Average Time: 0.26
[INFO] [2021-06-10 12:49:16] Total Time: 1s
[INFO] [2021-06-10 12:49:16] Storing 8128 MetaTraits
[INFO] [2021-06-10 12:49:16] Processing group of 8128 in 9 groups of 1000
[INFO] [2021-06-10 12:49:17] Average Time: 0.113
[INFO] [2021-06-10 12:49:17] Total Time: 2s
[INFO] [2021-06-10 12:49:17] last 3 / first 3: 0.5
[INFO] [2021-06-10 12:49:17] Std.Dev: 0.05477225575051661; Max: 0.24
[STOP] [2021-06-10 12:49:17] parse_diff_and_store
[START] [2021-06-10 12:49:17] resolve_keys
[INFO] [2021-06-10 12:49:24] Occurrences to nodes (through scientific_names)...
[INFO] [2021-06-10 12:49:24] traits to occurrences...
[INFO] [2021-06-10 12:49:24] traits to nodes (through occurrences)...
[INFO] [2021-06-10 12:49:24] Traits to sex term...
[INFO] [2021-06-10 12:49:24] Traits to lifestage term...
[INFO] [2021-06-10 12:49:24] MetaTraits to traits...
[INFO] [2021-06-10 12:49:24] MetaTraits (simple, measurement row refers to parent) to traits...
[INFO] [2021-06-10 12:49:24] Assocs to occurrences...
[INFO] [2021-06-10 12:49:24] Assocs to nodes...
[INFO] [2021-06-10 12:49:24] Assoc to sex term...
[INFO] [2021-06-10 12:49:24] Assoc to lifestage term...
[INFO] [2021-06-10 12:49:24] MetaAssoc to assocs...
[STOP] [2021-06-10 12:49:24] resolve_keys
[START] [2021-06-10 12:49:24] hold_for_later_1
[STOP] [2021-06-10 12:49:24] hold_for_later_1
[START] [2021-06-10 12:49:24] hold_for_later_2
[STOP] [2021-06-10 12:49:24] hold_for_later_2
[START] [2021-06-10 12:49:24] resolve_missing_parents
[STOP] [2021-06-10 12:49:24] resolve_missing_parents
[START] [2021-06-10 12:49:24] rebuild_nodes
[START] [2021-06-10 12:49:24] Flattener#flatten
[START] [2021-06-10 12:49:24] Flattener#study_resource
[START] [2021-06-10 12:49:24] Flattener#build_ancestry
[STOP] [2021-06-10 12:49:24] Flattener#build_ancestry
[INFO] [2021-06-10 12:49:24] 874 ancestry keys
[START] [2021-06-10 12:49:24] build_node_ancestors
[INFO] [2021-06-10 12:49:24] old ancestors deleted.
[STOP] [2021-06-10 12:49:25] build_node_ancestors
[START] [2021-06-10 12:49:25] Flattener#propagate_ancestor_ids
[STOP] [2021-06-10 12:49:25] Flattener#propagate_ancestor_ids
[STOP] [2021-06-10 12:49:25] Flattener#flatten
[STOP] [2021-06-10 12:49:25] rebuild_nodes
[START] [2021-06-10 12:49:25] resolve_missing_media_owners
[STOP] [2021-06-10 12:49:25] resolve_missing_media_owners
[START] [2021-06-10 12:49:25] sanitize_media_verbatims
[STOP] [2021-06-10 12:49:25] sanitize_media_verbatims
[START] [2021-06-10 12:49:25] queue_downloads
[STOP] [2021-06-10 12:49:25] queue_downloads
[START] [2021-06-10 12:49:25] parse_names
[WARN] [2021-06-10 12:49:25] I see 874 names which still need to be parsed.
[WARN] [2021-06-10 12:49:27] I see 101 names which still need to be parsed.
[WARN] [2021-06-10 12:49:28] I see 2 names which still need to be parsed.
[WARN] [2021-06-10 12:49:29] I see 1 names which still need to be parsed.
[STOP] [2021-06-10 12:49:30] parse_names
[START] [2021-06-10 12:49:30] denormalize_canonical_names_to_nodes
[STOP] [2021-06-10 12:49:30] denormalize_canonical_names_to_nodes
[START] [2021-06-10 12:49:30] match_nodes
[START] [2021-06-10 12:49:30] map_all_nodes_to_pages
[STOP] [2021-06-10 12:49:40] map_all_nodes_to_pages
[INFO] [2021-06-10 12:49:40] 80 Unmatched nodes (of 874)! That's too many to output. Full list in /app/public/data/Sal_et_al/unmatched_nodes.txt ; First 10: Canonical: Acanthoica quattrospina; Node#95973513; ResourceID: Acanthoica quattrospina; Canonical: Discosphaera tubifera; Node#95973865; ResourceID: Discosphaera tubifera; Canonical: Rhabdosphaera; Node#95974258; ResourceID: Rhabdosphaera; Canonical: Rhabdosphaera clavigera; Node#95974259; ResourceID: Rhabdosphaera clavigera; Canonical: Calciopappus rigidus; Node#95973636; ResourceID: Calciopappus rigidus; Canonical: Syracosphaera pirus; Node#95974317; ResourceID: Syracosphaera pirus; Canonical: Syracosphaera prolongata; Node#95974318; ResourceID: Syracosphaera prolongata; Canonical: Calciosolenia brasiliensis; Node#95973640; ResourceID: Calciosolenia brasiliensis; Canonical: Calciosolenia murrayi; Node#95973641; ResourceID: Calciosolenia murrayi; Canonical: Emiliana huxleyi; Node#95973880; ResourceID: Emiliana huxleyi
[START] [2021-06-10 12:49:40] update_nodes
[STOP] [2021-06-10 12:49:40] update_nodes
[STOP] [2021-06-10 12:49:40] match_nodes
[START] [2021-06-10 12:49:40] reindex_search
[STOP] [2021-06-10 12:49:41] reindex_search
[START] [2021-06-10 12:49:41] normalize_units
[STOP] [2021-06-10 12:49:49] normalize_units
[START] [2021-06-10 12:49:49] calculate_statistics
[2021-06-10 12:49:49] (NEAR) DUPLICATE TRAITS FOUND! There are only 2350 (of 2468 total) unique traits.
[2021-06-10 12:49:53] (Near) duplicate trait pairs (up to 100):
[2021-06-10 12:49:53] (resource_pk: 1005, id: 219532172), (resource_pk: 1006, id: 219532173)
[2021-06-10 12:49:53] (resource_pk: 1035, id: 219532205), (resource_pk: 866, id: 219534484)
[2021-06-10 12:49:53] (resource_pk: 1036, id: 219532206), (resource_pk: 865, id: 219534483)
[2021-06-10 12:49:53] (resource_pk: 1044, id: 219532215), (resource_pk: 1045, id: 219532216)
[2021-06-10 12:49:53] (resource_pk: 1046, id: 219532217), (resource_pk: 1047, id: 219532218)
[2021-06-10 12:49:53] (resource_pk: 105, id: 219532221), (resource_pk: 123, id: 219532421)
[2021-06-10 12:49:53] (resource_pk: 105, id: 219532221), (resource_pk: 63, id: 219534222)
[2021-06-10 12:49:53] (resource_pk: 1053, id: 219532225), (resource_pk: 1055, id: 219532227)
[2021-06-10 12:49:53] (resource_pk: 1053, id: 219532225), (resource_pk: 1063, id: 219532236)
[2021-06-10 12:49:53] (resource_pk: 1054, id: 219532226), (resource_pk: 1060, id: 219532233)
[2021-06-10 12:49:53] (resource_pk: 1055, id: 219532227), (resource_pk: 1063, id: 219532236)
[2021-06-10 12:49:53] (resource_pk: 106, id: 219532232), (resource_pk: 95, id: 219534577)
[2021-06-10 12:49:53] (resource_pk: 1066, id: 219532239), (resource_pk: 1071, id: 219532245)
[2021-06-10 12:49:53] (resource_pk: 1067, id: 219532240), (resource_pk: 1072, id: 219532246)
[2021-06-10 12:49:53] (resource_pk: 1207, id: 219532396), (resource_pk: 1208, id: 219532397)
[2021-06-10 12:49:53] (resource_pk: 1217, id: 219532407), (resource_pk: 383, id: 219533948)
[2021-06-10 12:49:53] (resource_pk: 1217, id: 219532407), (resource_pk: 796, id: 219534406)
[2021-06-10 12:49:53] (resource_pk: 123, id: 219532421), (resource_pk: 63, id: 219534222)
[2021-06-10 12:49:53] (resource_pk: 124, id: 219532432), (resource_pk: 125, id: 219532443)
[2021-06-10 12:49:53] (resource_pk: 1297, id: 219532495), (resource_pk: 1339, id: 219532542)
[2021-06-10 12:49:53] (resource_pk: 1297, id: 219532495), (resource_pk: 1357, id: 219532562)
[2021-06-10 12:49:53] (resource_pk: 1315, id: 219532516), (resource_pk: 1396, id: 219532605)
[2021-06-10 12:49:53] (resource_pk: 1329, id: 219532531), (resource_pk: 1340, id: 219532544)
[2021-06-10 12:49:53] (resource_pk: 1339, id: 219532542), (resource_pk: 1357, id: 219532562)
[2021-06-10 12:49:53] (resource_pk: 1358, id: 219532563), (resource_pk: 1359, id: 219532564)
[2021-06-10 12:49:53] (resource_pk: 1397, id: 219532606), (resource_pk: 1398, id: 219532607)
[2021-06-10 12:49:53] (resource_pk: 1469, id: 219532686), (resource_pk: 1472, id: 219532690)
[2021-06-10 12:49:53] (resource_pk: 1470, id: 219532688), (resource_pk: 1480, id: 219532699)
[2021-06-10 12:49:53] (resource_pk: 1471, id: 219532689), (resource_pk: 1487, id: 219532706)
[2021-06-10 12:49:53] (resource_pk: 1477, id: 219532695), (resource_pk: 1483, id: 219532702)
[2021-06-10 12:49:53] (resource_pk: 1478, id: 219532696), (resource_pk: 1482, id: 219532701)
[2021-06-10 12:49:53] (resource_pk: 1499, id: 219532719), (resource_pk: 1500, id: 219532722)
[2021-06-10 12:49:53] (resource_pk: 1519, id: 219532742), (resource_pk: 1522, id: 219532746)
[2021-06-10 12:49:53] (resource_pk: 1520, id: 219532744), (resource_pk: 1523, id: 219532747)
[2021-06-10 12:49:53] (resource_pk: 1539, id: 219532764), (resource_pk: 1547, id: 219532773)
[2021-06-10 12:49:53] (resource_pk: 1540, id: 219532766), (resource_pk: 1550, id: 219532777)
[2021-06-10 12:49:53] (resource_pk: 1541, id: 219532767), (resource_pk: 1548, id: 219532774)
[2021-06-10 12:49:53] (resource_pk: 1610, id: 219532844), (resource_pk: 1611, id: 219532845)
[2021-06-10 12:49:53] (resource_pk: 1617, id: 219532851), (resource_pk: 2030, id: 219533311)
[2021-06-10 12:49:53] (resource_pk: 1617, id: 219532851), (resource_pk: 2451, id: 219533778)
[2021-06-10 12:49:53] (resource_pk: 162, id: 219532854), (resource_pk: 81, id: 219534422)
[2021-06-10 12:49:53] (resource_pk: 1627, id: 219532862), (resource_pk: 1629, id: 219532864)
[2021-06-10 12:49:53] (resource_pk: 163, id: 219532865), (resource_pk: 164, id: 219532876)
[2021-06-10 12:49:53] (resource_pk: 1630, id: 219532866), (resource_pk: 1631, id: 219532867)
[2021-06-10 12:49:53] (resource_pk: 1753, id: 219533002), (resource_pk: 1768, id: 219533018)
[2021-06-10 12:49:53] (resource_pk: 1772, id: 219533023), (resource_pk: 1773, id: 219533024)
[2021-06-10 12:49:53] (resource_pk: 1799, id: 219533052), (resource_pk: 1803, id: 219533058)
[2021-06-10 12:49:53] (resource_pk: 1801, id: 219533056), (resource_pk: 1802, id: 219533057)
[2021-06-10 12:49:53] (resource_pk: 1801, id: 219533056), (resource_pk: 1806, id: 219533061)
[2021-06-10 12:49:53] (resource_pk: 1802, id: 219533057), (resource_pk: 1806, id: 219533061)
[2021-06-10 12:49:53] (resource_pk: 1811, id: 219533067), (resource_pk: 1818, id: 219533074)
[2021-06-10 12:49:53] (resource_pk: 1817, id: 219533073), (resource_pk: 1820, id: 219533077)
[2021-06-10 12:49:53] (resource_pk: 1880, id: 219533143), (resource_pk: 1908, id: 219533174)
[2021-06-10 12:49:53] (resource_pk: 1924, id: 219533192), (resource_pk: 1928, id: 219533196)
[2021-06-10 12:49:53] (resource_pk: 1935, id: 219533204), (resource_pk: 1954, id: 219533225)
[2021-06-10 12:49:53] (resource_pk: 1938, id: 219533207), (resource_pk: 1959, id: 219533230)
[2021-06-10 12:49:53] (resource_pk: 1939, id: 219533208), (resource_pk: 1960, id: 219533232)
[2021-06-10 12:49:53] (resource_pk: 1941, id: 219533211), (resource_pk: 1962, id: 219533234)
[2021-06-10 12:49:53] (resource_pk: 1943, id: 219533213), (resource_pk: 1967, id: 219533239)
[2021-06-10 12:49:53] (resource_pk: 1964, id: 219533236), (resource_pk: 1984, id: 219533258)
[2021-06-10 12:49:53] (resource_pk: 1964, id: 219533236), (resource_pk: 1999, id: 219533274)
[2021-06-10 12:49:53] (resource_pk: 1984, id: 219533258), (resource_pk: 1999, id: 219533274)
[2021-06-10 12:49:53] (resource_pk: 2030, id: 219533311), (resource_pk: 2451, id: 219533778)
[2021-06-10 12:49:53] (resource_pk: 2043, id: 219533325), (resource_pk: 2046, id: 219533328)
[2021-06-10 12:49:53] (resource_pk: 2088, id: 219533374), (resource_pk: 2095, id: 219533382)
[2021-06-10 12:49:53] (resource_pk: 2091, id: 219533378), (resource_pk: 2098, id: 219533385)
[2021-06-10 12:49:53] (resource_pk: 2091, id: 219533378), (resource_pk: 2102, id: 219533391)
[2021-06-10 12:49:53] (resource_pk: 2094, id: 219533381), (resource_pk: 2206, id: 219533506)
[2021-06-10 12:49:53] (resource_pk: 2098, id: 219533385), (resource_pk: 2102, id: 219533391)
[2021-06-10 12:49:53] (resource_pk: 2099, id: 219533386), (resource_pk: 2270, id: 219533577)
[2021-06-10 12:49:53] (resource_pk: 2100, id: 219533389), (resource_pk: 2269, id: 219533575)
[2021-06-10 12:49:53] (resource_pk: 2107, id: 219533396), (resource_pk: 2122, id: 219533413)
[2021-06-10 12:49:53] (resource_pk: 2114, id: 219533404), (resource_pk: 2115, id: 219533405)
[2021-06-10 12:49:53] (resource_pk: 2118, id: 219533408), (resource_pk: 2120, id: 219533411)
[2021-06-10 12:49:53] (resource_pk: 2188, id: 219533485), (resource_pk: 2189, id: 219533486)
[2021-06-10 12:49:53] (resource_pk: 2218, id: 219533519), (resource_pk: 2222, id: 219533524)
[2021-06-10 12:49:53] (resource_pk: 2239, id: 219533542), (resource_pk: 2240, id: 219533544)
[2021-06-10 12:49:53] (resource_pk: 2278, id: 219533585), (resource_pk: 2279, id: 219533586)
[2021-06-10 12:49:53] (resource_pk: 2280, id: 219533588), (resource_pk: 2281, id: 219533589)
[2021-06-10 12:49:53] (resource_pk: 2287, id: 219533595), (resource_pk: 2289, id: 219533597)
[2021-06-10 12:49:53] (resource_pk: 2287, id: 219533595), (resource_pk: 2297, id: 219533606)
[2021-06-10 12:49:53] (resource_pk: 2288, id: 219533596), (resource_pk: 2294, id: 219533603)
[2021-06-10 12:49:53] (resource_pk: 2289, id: 219533597), (resource_pk: 2297, id: 219533606)
[2021-06-10 12:49:53] (resource_pk: 2300, id: 219533611), (resource_pk: 2305, id: 219533616)
[2021-06-10 12:49:53] (resource_pk: 2301, id: 219533612), (resource_pk: 2306, id: 219533617)
[2021-06-10 12:49:53] (resource_pk: 235, id: 219533665), (resource_pk: 238, id: 219533698)
[2021-06-10 12:49:53] (resource_pk: 236, id: 219533676), (resource_pk: 246, id: 219533787)
[2021-06-10 12:49:53] (resource_pk: 237, id: 219533687), (resource_pk: 253, id: 219533804)
[2021-06-10 12:49:53] (resource_pk: 243, id: 219533754), (resource_pk: 249, id: 219533799)
[2021-06-10 12:49:53] (resource_pk: 244, id: 219533765), (resource_pk: 248, id: 219533798)
[2021-06-10 12:49:53] (resource_pk: 2441, id: 219533767), (resource_pk: 2442, id: 219533768)
[2021-06-10 12:49:53] (resource_pk: 265, id: 219533817), (resource_pk: 266, id: 219533818)
[2021-06-10 12:49:53] (resource_pk: 285, id: 219533839), (resource_pk: 288, id: 219533842)
[2021-06-10 12:49:53] (resource_pk: 286, id: 219533840), (resource_pk: 289, id: 219533843)
[2021-06-10 12:49:53] (resource_pk: 305, id: 219533862), (resource_pk: 313, id: 219533871)
[2021-06-10 12:49:53] (resource_pk: 306, id: 219533863), (resource_pk: 316, id: 219533874)
[2021-06-10 12:49:53] (resource_pk: 307, id: 219533864), (resource_pk: 314, id: 219533872)
[2021-06-10 12:49:53] (resource_pk: 376, id: 219533940), (resource_pk: 377, id: 219533941)
[2021-06-10 12:49:53] (resource_pk: 383, id: 219533948), (resource_pk: 796, id: 219534406)
[2021-06-10 12:49:53] (resource_pk: 393, id: 219533959), (resource_pk: 395, id: 219533961)
[STOP] [2021-06-10 12:49:53] calculate_statistics
[START] [2021-06-10 12:49:53] complete_harvest_instance
[START] [2021-06-10 12:49:53] overall_tsv_creation
[INFO] [2021-06-10 12:49:53] Processing group of 874 in 1 batches of 10000
[INFO] [2021-06-10 12:50:31] 2464 Traits (unfiltered)...
[INFO] [2021-06-10 12:51:19] 2464 Traits (filtered)...
[INFO] [2021-06-10 12:51:19] 0 Associations (filtered)...
[INFO] [2021-06-10 12:51:20] 2464 metadata added.
[INFO] [2021-06-10 12:51:20] 0 metadata added.
[INFO] [2021-06-10 12:51:48] Average Time: 91.62
[INFO] [2021-06-10 12:51:48] Total Time: 1m55s
[STOP] [2021-06-10 12:51:48] overall_tsv_creation
[INFO] [2021-06-10 12:51:48] Done. Check your files:
[INFO] [2021-06-10 12:51:48] (774 lines) /app/public/data/Sal_et_al/publish_nodes.tsv
[INFO] [2021-06-10 12:51:49] (3968 lines) /app/public/data/Sal_et_al/publish_node_ancestors.tsv
[INFO] [2021-06-10 12:51:49] (874 lines) /app/public/data/Sal_et_al/publish_scientific_names.tsv
[INFO] [2021-06-10 12:51:50] (2465 lines) /app/public/data/Sal_et_al/publish_traits.tsv
[INFO] [2021-06-10 12:51:50] (2465 lines) /app/public/data/Sal_et_al/publish_metadata.tsv
[STOP] [2021-06-10 12:51:50] complete_harvest_instance
[START] [2021-06-10 12:51:50] completed
[STOP] [2021-06-10 12:51:50] completed
[STOP] [2021-06-10 12:51:50] logged process, took 182.68

Latest Process

10 Jun 12:48 EXITED currently processing 0