Harvest for Bergtold 1917 Created 31 May 13:58

Stage: completed
Fetched: 31 May 13:58
Validated: 31 May 13:58
Deltas Created 31 May 13:58
Units Normalized: 31 May 14:00
Ancestry Built: 31 May 13:58
Nodes Matched: 31 May 14:00
Names Parsed: 31 May 13:58
New Models Stored: 31 May 13:58
Indexed: 31 May 14:00
Completed: 31 May 14:01
Time to Harvest: less than a minute

Harvesting Log

(325 lines)
[INFO] [2021-05-31 13:58:16] Created harvest instance #3923
[STOP] [2021-05-31 13:58:16] create_harvest_instance
[START] [2021-05-31 13:58:16] fetch_files
[STOP] [2021-05-31 13:58:16] fetch_files
[START] [2021-05-31 13:58:16] validate_each_file
[INFO] [2021-05-31 13:58:16] Looping over 8 formats...
[INFO] [2021-05-31 13:58:16] ...agents (/app/public/data/bergtold_bergtol/agents.txt)
[INFO] [2021-05-31 13:58:16] Valid: /app/public/converted_csv/bergtold_bergtol_agents_3923.csv (0 lines)
[INFO] [2021-05-31 13:58:16] ...refs (/app/public/data/bergtold_bergtol/references.txt)
[INFO] [2021-05-31 13:58:16] Valid: /app/public/converted_csv/bergtold_bergtol_refs_3923.csv (155 lines)
[INFO] [2021-05-31 13:58:16] ...nodes (/app/public/data/bergtold_bergtol/taxa.txt)
[INFO] [2021-05-31 13:58:16] Valid: /app/public/converted_csv/bergtold_bergtol_nodes_3923.csv (778 lines)
[INFO] [2021-05-31 13:58:16] ...media (/app/public/data/bergtold_bergtol/media.txt)
[INFO] [2021-05-31 13:58:16] Valid: /app/public/converted_csv/bergtold_bergtol_media_3923.csv (0 lines)
[INFO] [2021-05-31 13:58:16] ...vernaculars (/app/public/data/bergtold_bergtol/common names.txt)
[INFO] [2021-05-31 13:58:16] Valid: /app/public/converted_csv/bergtold_bergtol_vernaculars_3923.csv (1 lines)
[INFO] [2021-05-31 13:58:16] ...occurrences (/app/public/data/bergtold_bergtol/occurrences.txt)
[INFO] [2021-05-31 13:58:16] Valid: /app/public/converted_csv/bergtold_bergtol_occurrences_3923.csv (1330 lines)
[INFO] [2021-05-31 13:58:16] ...assocs (/app/public/data/bergtold_bergtol/associations.txt)
[INFO] [2021-05-31 13:58:16] Valid: /app/public/converted_csv/bergtold_bergtol_assocs_3923.csv (0 lines)
[INFO] [2021-05-31 13:58:16] ...measurements (/app/public/data/bergtold_bergtol/measurements or facts.txt)
[INFO] [2021-05-31 13:58:16] Valid: /app/public/converted_csv/bergtold_bergtol_measurements_3923.csv (1719 lines)
[STOP] [2021-05-31 13:58:16] validate_each_file
[START] [2021-05-31 13:58:16] convert_to_csv
[INFO] [2021-05-31 13:58:16] Looping over 8 formats...
[INFO] [2021-05-31 13:58:16] ...agents (/app/public/data/bergtold_bergtol/agents.txt)
[CMD] [2021-05-31 13:58:16] /usr/bin/sort /app/public/converted_csv/bergtold_bergtol_agents_3923.csv > /app/public/converted_csv/bergtold_bergtol_agents_3923.csv_sorted
[INFO] [2021-05-31 13:58:16] Converted: /app/public/converted_csv/bergtold_bergtol_agents_3923.csv (0 lines)
[INFO] [2021-05-31 13:58:16] ...refs (/app/public/data/bergtold_bergtol/references.txt)
[CMD] [2021-05-31 13:58:16] /usr/bin/sort /app/public/converted_csv/bergtold_bergtol_refs_3923.csv > /app/public/converted_csv/bergtold_bergtol_refs_3923.csv_sorted
[INFO] [2021-05-31 13:58:17] Converted: /app/public/converted_csv/bergtold_bergtol_refs_3923.csv (155 lines)
[INFO] [2021-05-31 13:58:17] ...nodes (/app/public/data/bergtold_bergtol/taxa.txt)
[CMD] [2021-05-31 13:58:17] /usr/bin/sort /app/public/converted_csv/bergtold_bergtol_nodes_3923.csv > /app/public/converted_csv/bergtold_bergtol_nodes_3923.csv_sorted
[INFO] [2021-05-31 13:58:17] Converted: /app/public/converted_csv/bergtold_bergtol_nodes_3923.csv (778 lines)
[INFO] [2021-05-31 13:58:17] ...media (/app/public/data/bergtold_bergtol/media.txt)
[CMD] [2021-05-31 13:58:17] /usr/bin/sort /app/public/converted_csv/bergtold_bergtol_media_3923.csv > /app/public/converted_csv/bergtold_bergtol_media_3923.csv_sorted
[INFO] [2021-05-31 13:58:18] Converted: /app/public/converted_csv/bergtold_bergtol_media_3923.csv (0 lines)
[INFO] [2021-05-31 13:58:18] ...vernaculars (/app/public/data/bergtold_bergtol/common names.txt)
[CMD] [2021-05-31 13:58:18] /usr/bin/sort /app/public/converted_csv/bergtold_bergtol_vernaculars_3923.csv > /app/public/converted_csv/bergtold_bergtol_vernaculars_3923.csv_sorted
[INFO] [2021-05-31 13:58:18] Converted: /app/public/converted_csv/bergtold_bergtol_vernaculars_3923.csv (1 lines)
[INFO] [2021-05-31 13:58:18] ...occurrences (/app/public/data/bergtold_bergtol/occurrences.txt)
[CMD] [2021-05-31 13:58:18] /usr/bin/sort /app/public/converted_csv/bergtold_bergtol_occurrences_3923.csv > /app/public/converted_csv/bergtold_bergtol_occurrences_3923.csv_sorted
[INFO] [2021-05-31 13:58:18] Converted: /app/public/converted_csv/bergtold_bergtol_occurrences_3923.csv (1330 lines)
[INFO] [2021-05-31 13:58:18] ...assocs (/app/public/data/bergtold_bergtol/associations.txt)
[CMD] [2021-05-31 13:58:18] /usr/bin/sort /app/public/converted_csv/bergtold_bergtol_assocs_3923.csv > /app/public/converted_csv/bergtold_bergtol_assocs_3923.csv_sorted
[INFO] [2021-05-31 13:58:19] Converted: /app/public/converted_csv/bergtold_bergtol_assocs_3923.csv (0 lines)
[INFO] [2021-05-31 13:58:19] ...measurements (/app/public/data/bergtold_bergtol/measurements or facts.txt)
[CMD] [2021-05-31 13:58:19] /usr/bin/sort /app/public/converted_csv/bergtold_bergtol_measurements_3923.csv > /app/public/converted_csv/bergtold_bergtol_measurements_3923.csv_sorted
[INFO] [2021-05-31 13:58:19] Converted: /app/public/converted_csv/bergtold_bergtol_measurements_3923.csv (1719 lines)
[STOP] [2021-05-31 13:58:19] convert_to_csv
[START] [2021-05-31 13:58:19] calculate_delta
[INFO] [2021-05-31 13:58:19] Looping over 8 formats...
[INFO] [2021-05-31 13:58:19] ...agents (/app/public/data/bergtold_bergtol/agents.txt)
[CMD] [2021-05-31 13:58:19] echo "0a" > /app/public/diff/bergtold_bergtol_agents_3923.diff
[CMD] [2021-05-31 13:58:19] tail -n +1 /app/public/converted_csv/bergtold_bergtol_agents_3923.csv >> /app/public/diff/bergtold_bergtol_agents_3923.diff
[CMD] [2021-05-31 13:58:20] echo "." >> /app/public/diff/bergtold_bergtol_agents_3923.diff
[INFO] [2021-05-31 13:58:20] Created diff: /app/public/diff/bergtold_bergtol_agents_3923.diff (2 lines)
[INFO] [2021-05-31 13:58:20] ...refs (/app/public/data/bergtold_bergtol/references.txt)
[CMD] [2021-05-31 13:58:20] echo "0a" > /app/public/diff/bergtold_bergtol_refs_3923.diff
[CMD] [2021-05-31 13:58:21] tail -n +1 /app/public/converted_csv/bergtold_bergtol_refs_3923.csv >> /app/public/diff/bergtold_bergtol_refs_3923.diff
[CMD] [2021-05-31 13:58:21] echo "." >> /app/public/diff/bergtold_bergtol_refs_3923.diff
[INFO] [2021-05-31 13:58:21] Created diff: /app/public/diff/bergtold_bergtol_refs_3923.diff (157 lines)
[INFO] [2021-05-31 13:58:21] ...nodes (/app/public/data/bergtold_bergtol/taxa.txt)
[CMD] [2021-05-31 13:58:21] echo "0a" > /app/public/diff/bergtold_bergtol_nodes_3923.diff
[CMD] [2021-05-31 13:58:22] tail -n +1 /app/public/converted_csv/bergtold_bergtol_nodes_3923.csv >> /app/public/diff/bergtold_bergtol_nodes_3923.diff
[CMD] [2021-05-31 13:58:22] echo "." >> /app/public/diff/bergtold_bergtol_nodes_3923.diff
[INFO] [2021-05-31 13:58:23] Created diff: /app/public/diff/bergtold_bergtol_nodes_3923.diff (780 lines)
[INFO] [2021-05-31 13:58:23] ...media (/app/public/data/bergtold_bergtol/media.txt)
[CMD] [2021-05-31 13:58:23] echo "0a" > /app/public/diff/bergtold_bergtol_media_3923.diff
[CMD] [2021-05-31 13:58:23] tail -n +1 /app/public/converted_csv/bergtold_bergtol_media_3923.csv >> /app/public/diff/bergtold_bergtol_media_3923.diff
[CMD] [2021-05-31 13:58:23] echo "." >> /app/public/diff/bergtold_bergtol_media_3923.diff
[INFO] [2021-05-31 13:58:24] Created diff: /app/public/diff/bergtold_bergtol_media_3923.diff (2 lines)
[INFO] [2021-05-31 13:58:24] ...vernaculars (/app/public/data/bergtold_bergtol/common names.txt)
[CMD] [2021-05-31 13:58:24] echo "0a" > /app/public/diff/bergtold_bergtol_vernaculars_3923.diff
[CMD] [2021-05-31 13:58:24] tail -n +1 /app/public/converted_csv/bergtold_bergtol_vernaculars_3923.csv >> /app/public/diff/bergtold_bergtol_vernaculars_3923.diff
[CMD] [2021-05-31 13:58:24] echo "." >> /app/public/diff/bergtold_bergtol_vernaculars_3923.diff
[INFO] [2021-05-31 13:58:25] Created diff: /app/public/diff/bergtold_bergtol_vernaculars_3923.diff (3 lines)
[INFO] [2021-05-31 13:58:25] ...occurrences (/app/public/data/bergtold_bergtol/occurrences.txt)
[CMD] [2021-05-31 13:58:25] echo "0a" > /app/public/diff/bergtold_bergtol_occurrences_3923.diff
[CMD] [2021-05-31 13:58:25] tail -n +1 /app/public/converted_csv/bergtold_bergtol_occurrences_3923.csv >> /app/public/diff/bergtold_bergtol_occurrences_3923.diff
[CMD] [2021-05-31 13:58:26] echo "." >> /app/public/diff/bergtold_bergtol_occurrences_3923.diff
[INFO] [2021-05-31 13:58:26] Created diff: /app/public/diff/bergtold_bergtol_occurrences_3923.diff (1332 lines)
[INFO] [2021-05-31 13:58:26] ...assocs (/app/public/data/bergtold_bergtol/associations.txt)
[CMD] [2021-05-31 13:58:26] echo "0a" > /app/public/diff/bergtold_bergtol_assocs_3923.diff
[CMD] [2021-05-31 13:58:26] tail -n +1 /app/public/converted_csv/bergtold_bergtol_assocs_3923.csv >> /app/public/diff/bergtold_bergtol_assocs_3923.diff
[CMD] [2021-05-31 13:58:27] echo "." >> /app/public/diff/bergtold_bergtol_assocs_3923.diff
[INFO] [2021-05-31 13:58:27] Created diff: /app/public/diff/bergtold_bergtol_assocs_3923.diff (2 lines)
[INFO] [2021-05-31 13:58:27] ...measurements (/app/public/data/bergtold_bergtol/measurements or facts.txt)
[CMD] [2021-05-31 13:58:27] echo "0a" > /app/public/diff/bergtold_bergtol_measurements_3923.diff
[CMD] [2021-05-31 13:58:27] tail -n +1 /app/public/converted_csv/bergtold_bergtol_measurements_3923.csv >> /app/public/diff/bergtold_bergtol_measurements_3923.diff
[CMD] [2021-05-31 13:58:28] echo "." >> /app/public/diff/bergtold_bergtol_measurements_3923.diff
[INFO] [2021-05-31 13:58:28] Created diff: /app/public/diff/bergtold_bergtol_measurements_3923.diff (1721 lines)
[STOP] [2021-05-31 13:58:28] calculate_delta
[START] [2021-05-31 13:58:28] parse_diff_and_store
[INFO] [2021-05-31 13:58:28] Handling diff: /app/public/diff/bergtold_bergtol_agents_3923.diff (2 lines)
[INFO] [2021-05-31 13:58:29] Loading agents diff file into memory (2 /app/public/diff/bergtold_bergtol_agents_3923.diff lines)...
[INFO] [2021-05-31 13:58:29] Handling diff: /app/public/diff/bergtold_bergtol_refs_3923.diff (157 lines)
[INFO] [2021-05-31 13:58:29] Loading refs diff file into memory (157 /app/public/diff/bergtold_bergtol_refs_3923.diff lines)...
[INFO] [2021-05-31 13:58:30] Handling diff: /app/public/diff/bergtold_bergtol_nodes_3923.diff (780 lines)
[INFO] [2021-05-31 13:58:30] Loading nodes diff file into memory (780 /app/public/diff/bergtold_bergtol_nodes_3923.diff lines)...
[INFO] [2021-05-31 13:58:31] Handling diff: /app/public/diff/bergtold_bergtol_media_3923.diff (2 lines)
[INFO] [2021-05-31 13:58:31] Loading media diff file into memory (2 /app/public/diff/bergtold_bergtol_media_3923.diff lines)...
[INFO] [2021-05-31 13:58:32] Handling diff: /app/public/diff/bergtold_bergtol_vernaculars_3923.diff (3 lines)
[INFO] [2021-05-31 13:58:32] Loading vernaculars diff file into memory (3 /app/public/diff/bergtold_bergtol_vernaculars_3923.diff lines)...
[INFO] [2021-05-31 13:58:33] Handling diff: /app/public/diff/bergtold_bergtol_occurrences_3923.diff (1332 lines)
[INFO] [2021-05-31 13:58:33] Loading occurrences diff file into memory (1332 /app/public/diff/bergtold_bergtol_occurrences_3923.diff lines)...
[INFO] [2021-05-31 13:58:34] Handling diff: /app/public/diff/bergtold_bergtol_assocs_3923.diff (2 lines)
[INFO] [2021-05-31 13:58:34] Loading assocs diff file into memory (2 /app/public/diff/bergtold_bergtol_assocs_3923.diff lines)...
[INFO] [2021-05-31 13:58:34] Handling diff: /app/public/diff/bergtold_bergtol_measurements_3923.diff (1721 lines)
[INFO] [2021-05-31 13:58:35] Loading measurements diff file into memory (1721 /app/public/diff/bergtold_bergtol_measurements_3923.diff lines)...
[INFO] [2021-05-31 13:58:36] Storing 155 References
[INFO] [2021-05-31 13:58:36] Processing group of 155 in 1 groups of 1000
[INFO] [2021-05-31 13:58:36] Average Time: 0.03
[INFO] [2021-05-31 13:58:36] Total Time: 1s
[INFO] [2021-05-31 13:58:36] Storing 1215 ScientificNames
[INFO] [2021-05-31 13:58:36] Processing group of 1215 in 2 groups of 1000
[INFO] [2021-05-31 13:58:36] Average Time: 0.18
[INFO] [2021-05-31 13:58:36] Total Time: 1s
[INFO] [2021-05-31 13:58:36] Storing 1215 Nodes
[INFO] [2021-05-31 13:58:36] Processing group of 1215 in 2 groups of 1000
[INFO] [2021-05-31 13:58:37] Average Time: 0.15
[INFO] [2021-05-31 13:58:37] Total Time: 1s
[INFO] [2021-05-31 13:58:37] Storing 1 Vernaculars
[INFO] [2021-05-31 13:58:37] Processing group of 1 in 1 groups of 1000
[INFO] [2021-05-31 13:58:37] Average Time: 0.0
[INFO] [2021-05-31 13:58:37] Total Time: 1s
[INFO] [2021-05-31 13:58:37] Storing 1330 Occurrences
[INFO] [2021-05-31 13:58:37] Processing group of 1330 in 2 groups of 1000
[INFO] [2021-05-31 13:58:37] Average Time: 0.12
[INFO] [2021-05-31 13:58:37] Total Time: 1s
[INFO] [2021-05-31 13:58:37] Storing 1719 Traits
[INFO] [2021-05-31 13:58:37] Processing group of 1719 in 2 groups of 1000
[INFO] [2021-05-31 13:58:38] Average Time: 0.25
[INFO] [2021-05-31 13:58:38] Total Time: 1s
[INFO] [2021-05-31 13:58:38] Storing 2939 MetaTraits
[INFO] [2021-05-31 13:58:38] Processing group of 2939 in 3 groups of 1000
[INFO] [2021-05-31 13:58:38] Average Time: 0.11
[INFO] [2021-05-31 13:58:38] Total Time: 1s
[INFO] [2021-05-31 13:58:38] Storing 1664 TraitsReferences
[INFO] [2021-05-31 13:58:38] Processing group of 1664 in 2 groups of 1000
[INFO] [2021-05-31 13:58:38] Average Time: 0.05
[INFO] [2021-05-31 13:58:38] Total Time: 1s
[STOP] [2021-05-31 13:58:38] parse_diff_and_store
[START] [2021-05-31 13:58:38] resolve_keys
[INFO] [2021-05-31 13:58:44] Occurrences to nodes (through scientific_names)...
[INFO] [2021-05-31 13:58:45] traits to occurrences...
[INFO] [2021-05-31 13:58:45] traits to nodes (through occurrences)...
[INFO] [2021-05-31 13:58:45] Traits to sex term...
[INFO] [2021-05-31 13:58:45] Traits to lifestage term...
[INFO] [2021-05-31 13:58:45] MetaTraits to traits...
[INFO] [2021-05-31 13:58:45] MetaTraits (simple, measurement row refers to parent) to traits...
[INFO] [2021-05-31 13:58:45] Assocs to occurrences...
[INFO] [2021-05-31 13:58:45] Assocs to nodes...
[INFO] [2021-05-31 13:58:45] Assoc to sex term...
[INFO] [2021-05-31 13:58:45] Assoc to lifestage term...
[INFO] [2021-05-31 13:58:45] MetaAssoc to assocs...
[STOP] [2021-05-31 13:58:45] resolve_keys
[START] [2021-05-31 13:58:45] hold_for_later_1
[STOP] [2021-05-31 13:58:45] hold_for_later_1
[START] [2021-05-31 13:58:45] hold_for_later_2
[STOP] [2021-05-31 13:58:45] hold_for_later_2
[START] [2021-05-31 13:58:45] resolve_missing_parents
[STOP] [2021-05-31 13:58:45] resolve_missing_parents
[START] [2021-05-31 13:58:45] rebuild_nodes
[START] [2021-05-31 13:58:45] Flattener#flatten
[START] [2021-05-31 13:58:45] Flattener#study_resource
[START] [2021-05-31 13:58:45] Flattener#build_ancestry
[STOP] [2021-05-31 13:58:45] Flattener#build_ancestry
[INFO] [2021-05-31 13:58:45] 1215 ancestry keys
[START] [2021-05-31 13:58:45] build_node_ancestors
[INFO] [2021-05-31 13:58:45] old ancestors deleted.
[STOP] [2021-05-31 13:58:45] build_node_ancestors
[START] [2021-05-31 13:58:46] Flattener#propagate_ancestor_ids
[STOP] [2021-05-31 13:58:46] Flattener#propagate_ancestor_ids
[STOP] [2021-05-31 13:58:46] Flattener#flatten
[STOP] [2021-05-31 13:58:46] rebuild_nodes
[START] [2021-05-31 13:58:46] resolve_missing_media_owners
[STOP] [2021-05-31 13:58:46] resolve_missing_media_owners
[START] [2021-05-31 13:58:46] sanitize_media_verbatims
[STOP] [2021-05-31 13:58:46] sanitize_media_verbatims
[START] [2021-05-31 13:58:46] queue_downloads
[STOP] [2021-05-31 13:58:46] queue_downloads
[START] [2021-05-31 13:58:46] parse_names
[WARN] [2021-05-31 13:58:46] I see 1215 names which still need to be parsed.
[WARN] [2021-05-31 13:58:48] I see 19 names which still need to be parsed.
[STOP] [2021-05-31 13:58:49] parse_names
[START] [2021-05-31 13:58:49] denormalize_canonical_names_to_nodes
[STOP] [2021-05-31 13:58:49] denormalize_canonical_names_to_nodes
[START] [2021-05-31 13:58:49] match_nodes
[START] [2021-05-31 13:58:49] map_all_nodes_to_pages
[STOP] [2021-05-31 14:00:01] map_all_nodes_to_pages
[INFO] [2021-05-31 14:00:01] 559 Unmatched nodes (of 1215)! That's too many to output. Full list in /app/public/data/bergtold_bergtol/unmatched_nodes.txt ; First 10: Canonical: Accentor modularis; Node#95111525; ResourceID: Accentor_modularis; Canonical: Accipiter velox; Node#95111532; ResourceID: Accipiter_velox; Canonical: Acredula rosea; Node#95111534; ResourceID: Acredula_rosea; Canonical: Acrocephalus phragmitis; Node#95111537; ResourceID: Acrocephalus_phragmitis; Canonical: Acrocephalus turdoides; Node#95111538; ResourceID: Acrocephalus_turdoides; Canonical: Actitis mascularius; Node#95111540; ResourceID: Actitis_mascularius; Canonical: Aegialitis hiaticula; Node#95111544; ResourceID: Aegialitis_hiaticula; Canonical: Aegialitis vociferus; Node#95111545; ResourceID: Aegialitis_vociferus; Canonical: Aimophila cassinii; Node#95111549; ResourceID: Aimophila_cassinii; Canonical: Alanda; Node#95111553; ResourceID: Alanda
[START] [2021-05-31 14:00:01] update_nodes
[STOP] [2021-05-31 14:00:01] update_nodes
[STOP] [2021-05-31 14:00:01] match_nodes
[START] [2021-05-31 14:00:01] reindex_search
[STOP] [2021-05-31 14:00:02] reindex_search
[START] [2021-05-31 14:00:02] normalize_units
[STOP] [2021-05-31 14:00:08] normalize_units
[START] [2021-05-31 14:00:08] calculate_statistics
[2021-05-31 14:00:08] (NEAR) DUPLICATE TRAITS FOUND! There are only 1555 (of 1664 total) unique traits.
[2021-05-31 14:00:10] (Near) duplicate trait pairs (up to 100):
[2021-05-31 14:00:10] (resource_pk: M_00003, id: 214761489), (resource_pk: M_00004, id: 214761490)
[2021-05-31 14:00:10] (resource_pk: M_00006, id: 214761492), (resource_pk: M_00007, id: 214761493)
[2021-05-31 14:00:10] (resource_pk: M_00006, id: 214761492), (resource_pk: M_00008, id: 214761494)
[2021-05-31 14:00:10] (resource_pk: M_00007, id: 214761493), (resource_pk: M_00008, id: 214761494)
[2021-05-31 14:00:10] (resource_pk: M_00014, id: 214761500), (resource_pk: M_00016, id: 214761502)
[2021-05-31 14:00:10] (resource_pk: M_00015, id: 214761501), (resource_pk: M_00021, id: 214761507)
[2021-05-31 14:00:10] (resource_pk: M_00051, id: 214761535), (resource_pk: M_00052, id: 214761536)
[2021-05-31 14:00:10] (resource_pk: M_00061, id: 214761544), (resource_pk: M_00062, id: 214761545)
[2021-05-31 14:00:10] (resource_pk: M_00061, id: 214761544), (resource_pk: M_00063, id: 214761546)
[2021-05-31 14:00:10] (resource_pk: M_00062, id: 214761545), (resource_pk: M_00063, id: 214761546)
[2021-05-31 14:00:10] (resource_pk: M_00076, id: 214761559), (resource_pk: M_00077, id: 214761560)
[2021-05-31 14:00:10] (resource_pk: M_00076, id: 214761559), (resource_pk: M_00079, id: 214761562)
[2021-05-31 14:00:10] (resource_pk: M_00077, id: 214761560), (resource_pk: M_00079, id: 214761562)
[2021-05-31 14:00:10] (resource_pk: M_00088, id: 214761571), (resource_pk: M_00089, id: 214761572)
[2021-05-31 14:00:10] (resource_pk: M_00090, id: 214761573), (resource_pk: M_00091, id: 214761574)
[2021-05-31 14:00:10] (resource_pk: M_00093, id: 214761576), (resource_pk: M_00094, id: 214761577)
[2021-05-31 14:00:10] (resource_pk: M_00112, id: 214761595), (resource_pk: M_00113, id: 214761596)
[2021-05-31 14:00:10] (resource_pk: M_00115, id: 214761598), (resource_pk: M_00117, id: 214761600)
[2021-05-31 14:00:10] (resource_pk: M_00127, id: 214761610), (resource_pk: M_00129, id: 214761612)
[2021-05-31 14:00:10] (resource_pk: M_00127, id: 214761610), (resource_pk: M_00130, id: 214761613)
[2021-05-31 14:00:10] (resource_pk: M_00127, id: 214761610), (resource_pk: M_00275, id: 214761758)
[2021-05-31 14:00:10] (resource_pk: M_00129, id: 214761612), (resource_pk: M_00130, id: 214761613)
[2021-05-31 14:00:10] (resource_pk: M_00129, id: 214761612), (resource_pk: M_00275, id: 214761758)
[2021-05-31 14:00:10] (resource_pk: M_00130, id: 214761613), (resource_pk: M_00275, id: 214761758)
[2021-05-31 14:00:10] (resource_pk: M_00133, id: 214761616), (resource_pk: M_00134, id: 214761617)
[2021-05-31 14:00:10] (resource_pk: M_00171, id: 214761654), (resource_pk: M_00172, id: 214761655)
[2021-05-31 14:00:10] (resource_pk: M_00175, id: 214761658), (resource_pk: M_00176, id: 214761659)
[2021-05-31 14:00:10] (resource_pk: M_00175, id: 214761658), (resource_pk: M_00177, id: 214761660)
[2021-05-31 14:00:10] (resource_pk: M_00176, id: 214761659), (resource_pk: M_00177, id: 214761660)
[2021-05-31 14:00:10] (resource_pk: M_00183, id: 214761666), (resource_pk: M_00185, id: 214761668)
[2021-05-31 14:00:10] (resource_pk: M_00209, id: 214761692), (resource_pk: M_00210, id: 214761693)
[2021-05-31 14:00:10] (resource_pk: M_00223, id: 214761706), (resource_pk: M_00224, id: 214761707)
[2021-05-31 14:00:10] (resource_pk: M_00226, id: 214761709), (resource_pk: M_00227, id: 214761710)
[2021-05-31 14:00:10] (resource_pk: M_00229, id: 214761712), (resource_pk: M_00231, id: 214761714)
[2021-05-31 14:00:10] (resource_pk: M_00229, id: 214761712), (resource_pk: M_00233, id: 214761716)
[2021-05-31 14:00:10] (resource_pk: M_00230, id: 214761713), (resource_pk: M_00232, id: 214761715)
[2021-05-31 14:00:10] (resource_pk: M_00231, id: 214761714), (resource_pk: M_00233, id: 214761716)
[2021-05-31 14:00:10] (resource_pk: M_00242, id: 214761725), (resource_pk: M_00243, id: 214761726)
[2021-05-31 14:00:10] (resource_pk: M_00244, id: 214761727), (resource_pk: M_00245, id: 214761728)
[2021-05-31 14:00:10] (resource_pk: M_00249, id: 214761732), (resource_pk: M_00250, id: 214761733)
[2021-05-31 14:00:10] (resource_pk: M_00252, id: 214761735), (resource_pk: M_00253, id: 214761736)
[2021-05-31 14:00:10] (resource_pk: M_00255, id: 214761738), (resource_pk: M_00256, id: 214761739)
[2021-05-31 14:00:10] (resource_pk: M_00269, id: 214761752), (resource_pk: M_00270, id: 214761753)
[2021-05-31 14:00:10] (resource_pk: M_00277, id: 214761760), (resource_pk: M_00283, id: 214761766)
[2021-05-31 14:00:10] (resource_pk: M_00294, id: 214761777), (resource_pk: M_00295, id: 214761778)
[2021-05-31 14:00:10] (resource_pk: M_00304, id: 214761787), (resource_pk: M_00305, id: 214761788)
[2021-05-31 14:00:10] (resource_pk: M_00309, id: 214761792), (resource_pk: M_00310, id: 214761793)
[2021-05-31 14:00:10] (resource_pk: M_00309, id: 214761792), (resource_pk: M_00311, id: 214761794)
[2021-05-31 14:00:10] (resource_pk: M_00309, id: 214761792), (resource_pk: M_00312, id: 214761795)
[2021-05-31 14:00:10] (resource_pk: M_00310, id: 214761793), (resource_pk: M_00311, id: 214761794)
[2021-05-31 14:00:10] (resource_pk: M_00310, id: 214761793), (resource_pk: M_00312, id: 214761795)
[2021-05-31 14:00:10] (resource_pk: M_00311, id: 214761794), (resource_pk: M_00312, id: 214761795)
[2021-05-31 14:00:10] (resource_pk: M_00330, id: 214761813), (resource_pk: M_00331, id: 214761814)
[2021-05-31 14:00:10] (resource_pk: M_00349, id: 214761832), (resource_pk: M_00350, id: 214761833)
[2021-05-31 14:00:10] (resource_pk: M_00359, id: 214761842), (resource_pk: M_00360, id: 214761843)
[2021-05-31 14:00:10] (resource_pk: M_00361, id: 214761844), (resource_pk: M_00362, id: 214761845)
[2021-05-31 14:00:10] (resource_pk: M_00367, id: 214761850), (resource_pk: M_00369, id: 214761852)
[2021-05-31 14:00:10] (resource_pk: M_00371, id: 214761854), (resource_pk: M_00373, id: 214761856)
[2021-05-31 14:00:10] (resource_pk: M_00371, id: 214761854), (resource_pk: M_00374, id: 214761857)
[2021-05-31 14:00:10] (resource_pk: M_00373, id: 214761856), (resource_pk: M_00374, id: 214761857)
[2021-05-31 14:00:10] (resource_pk: M_00383, id: 214761866), (resource_pk: M_00384, id: 214761867)
[2021-05-31 14:00:10] (resource_pk: M_00414, id: 214761897), (resource_pk: M_00415, id: 214761898)
[2021-05-31 14:00:10] (resource_pk: M_00418, id: 214761901), (resource_pk: M_00419, id: 214761902)
[2021-05-31 14:00:10] (resource_pk: M_00418, id: 214761901), (resource_pk: M_00420, id: 214761903)
[2021-05-31 14:00:10] (resource_pk: M_00419, id: 214761902), (resource_pk: M_00420, id: 214761903)
[2021-05-31 14:00:10] (resource_pk: M_00440, id: 214761923), (resource_pk: M_00441, id: 214761924)
[2021-05-31 14:00:10] (resource_pk: M_00451, id: 214761934), (resource_pk: M_00452, id: 214761935)
[2021-05-31 14:00:10] (resource_pk: M_00473, id: 214761956), (resource_pk: M_00477, id: 214761960)
[2021-05-31 14:00:10] (resource_pk: M_00481, id: 214761964), (resource_pk: M_00482, id: 214761965)
[2021-05-31 14:00:10] (resource_pk: M_00497, id: 214761980), (resource_pk: M_00498, id: 214761981)
[2021-05-31 14:00:10] (resource_pk: M_00507, id: 214761990), (resource_pk: M_00508, id: 214761991)
[2021-05-31 14:00:10] (resource_pk: M_00519, id: 214762000), (resource_pk: M_00521, id: 214762002)
[2021-05-31 14:00:10] (resource_pk: M_00524, id: 214762005), (resource_pk: M_00525, id: 214762006)
[2021-05-31 14:00:10] (resource_pk: M_00526, id: 214762007), (resource_pk: M_00527, id: 214762008)
[2021-05-31 14:00:10] (resource_pk: M_00528, id: 214762009), (resource_pk: M_00529, id: 214762010)
[2021-05-31 14:00:10] (resource_pk: M_00534, id: 214762015), (resource_pk: M_00535, id: 214762016)
[2021-05-31 14:00:10] (resource_pk: M_00537, id: 214762018), (resource_pk: M_00538, id: 214762019)
[2021-05-31 14:00:10] (resource_pk: M_00540, id: 214762021), (resource_pk: M_00541, id: 214762022)
[2021-05-31 14:00:10] (resource_pk: M_00580, id: 214762058), (resource_pk: M_00583, id: 214762061)
[2021-05-31 14:00:10] (resource_pk: M_00582, id: 214762060), (resource_pk: M_00584, id: 214762062)
[2021-05-31 14:00:10] (resource_pk: M_00585, id: 214762063), (resource_pk: M_00586, id: 214762064)
[2021-05-31 14:00:10] (resource_pk: M_00590, id: 214762068), (resource_pk: M_00591, id: 214762069)
[2021-05-31 14:00:10] (resource_pk: M_00609, id: 214762087), (resource_pk: M_00610, id: 214762088)
[2021-05-31 14:00:10] (resource_pk: M_00616, id: 214762094), (resource_pk: M_00617, id: 214762095)
[2021-05-31 14:00:10] (resource_pk: M_00651, id: 214762128), (resource_pk: M_00652, id: 214762129)
[2021-05-31 14:00:10] (resource_pk: M_00683, id: 214762160), (resource_pk: M_00684, id: 214762161)
[2021-05-31 14:00:10] (resource_pk: M_00687, id: 214762164), (resource_pk: M_00688, id: 214762165)
[2021-05-31 14:00:10] (resource_pk: M_00707, id: 214762184), (resource_pk: M_00708, id: 214762185)
[2021-05-31 14:00:10] (resource_pk: M_00719, id: 214762196), (resource_pk: M_00720, id: 214762197)
[2021-05-31 14:00:10] (resource_pk: M_00723, id: 214762200), (resource_pk: M_00724, id: 214762201)
[2021-05-31 14:00:10] (resource_pk: M_00763, id: 214762240), (resource_pk: M_00764, id: 214762241)
[2021-05-31 14:00:10] (resource_pk: M_00763, id: 214762240), (resource_pk: M_00765, id: 214762242)
[2021-05-31 14:00:10] (resource_pk: M_00764, id: 214762241), (resource_pk: M_00765, id: 214762242)
[2021-05-31 14:00:10] (resource_pk: M_00811, id: 214762288), (resource_pk: M_00812, id: 214762289)
[2021-05-31 14:00:10] (resource_pk: M_00829, id: 214762306), (resource_pk: M_00830, id: 214762307)
[2021-05-31 14:00:10] (resource_pk: M_00838, id: 214762315), (resource_pk: M_00839, id: 214762316)
[2021-05-31 14:00:10] (resource_pk: M_00872, id: 214762349), (resource_pk: M_00873, id: 214762350)
[2021-05-31 14:00:10] (resource_pk: M_00890, id: 214762367), (resource_pk: M_00892, id: 214762369)
[2021-05-31 14:00:10] (resource_pk: M_00891, id: 214762368), (resource_pk: M_00893, id: 214762370)
[2021-05-31 14:00:10] (resource_pk: M_00891, id: 214762368), (resource_pk: M_00894, id: 214762371)
[STOP] [2021-05-31 14:00:10] calculate_statistics
[START] [2021-05-31 14:00:10] complete_harvest_instance
[START] [2021-05-31 14:00:10] overall_tsv_creation
[INFO] [2021-05-31 14:00:10] Processing group of 1215 in 1 batches of 10000
[INFO] [2021-05-31 14:00:46] 1649 Traits (unfiltered)...
[INFO] [2021-05-31 14:01:16] 1649 Traits (filtered)...
[INFO] [2021-05-31 14:01:16] 0 Associations (filtered)...
[INFO] [2021-05-31 14:01:16] 1852 metadata added.
[INFO] [2021-05-31 14:01:16] 0 metadata added.
[INFO] [2021-05-31 14:01:39] Average Time: 66.82
[INFO] [2021-05-31 14:01:39] Total Time: 1m30s
[STOP] [2021-05-31 14:01:39] overall_tsv_creation
[INFO] [2021-05-31 14:01:39] Done. Check your files:
[INFO] [2021-05-31 14:01:40] (1196 lines) /app/public/data/bergtold_bergtol/publish_nodes.tsv
[INFO] [2021-05-31 14:01:40] (3148 lines) /app/public/data/bergtold_bergtol/publish_node_ancestors.tsv
[INFO] [2021-05-31 14:01:40] (1215 lines) /app/public/data/bergtold_bergtol/publish_scientific_names.tsv
[INFO] [2021-05-31 14:01:41] (1 lines) /app/public/data/bergtold_bergtol/publish_vernaculars.tsv
[INFO] [2021-05-31 14:01:41] (1650 lines) /app/public/data/bergtold_bergtol/publish_traits.tsv
[INFO] [2021-05-31 14:01:42] (1853 lines) /app/public/data/bergtold_bergtol/publish_metadata.tsv
[STOP] [2021-05-31 14:01:42] complete_harvest_instance
[START] [2021-05-31 14:01:42] completed
[STOP] [2021-05-31 14:01:42] completed
[STOP] [2021-05-31 14:01:42] logged process, took 206.46

Latest Process