Harvest for Mackenzie 2003 Created 19 Apr 14:41

Harvesting Log

# Logfile created on 2020-04-03 15:04:26 -0400 by logger.rb/56815
[START] [2020-04-03 15:04:26] logged process
[START] [2020-04-03 15:04:26] create_harvest_instance
[STOP] [2020-04-03 15:04:26] create_harvest_instance
[START] [2020-04-03 15:04:26] fetch_files
[STOP] [2020-04-03 15:04:26] fetch_files
[START] [2020-04-03 15:04:26] validate_each_file
[STOP] [2020-04-03 15:04:26] validate_each_file
[START] [2020-04-03 15:04:26] convert_to_csv
[CMD] [2020-04-03 15:04:26] /usr/bin/sort /app/public/converted_csv/mackenzie_macken_nodes_20580.csv > /app/public/converted_csv/mackenzie_macken_nodes_20580.csv_sorted
[CMD] [2020-04-03 15:04:26] /usr/bin/sort /app/public/converted_csv/mackenzie_macken_occurrences_20581.csv > /app/public/converted_csv/mackenzie_macken_occurrences_20581.csv_sorted
[CMD] [2020-04-03 15:04:26] /usr/bin/sort /app/public/converted_csv/mackenzie_macken_measurements_20582.csv > /app/public/converted_csv/mackenzie_macken_measurements_20582.csv_sorted
[STOP] [2020-04-03 15:04:26] convert_to_csv
[START] [2020-04-03 15:04:26] calculate_delta
[CMD] [2020-04-03 15:04:26] echo "0a" > /app/public/diff/mackenzie_macken_nodes_20580.diff
[CMD] [2020-04-03 15:04:26] tail -n +1 /app/public/converted_csv/mackenzie_macken_nodes_20580.csv >> /app/public/diff/mackenzie_macken_nodes_20580.diff
[CMD] [2020-04-03 15:04:26] echo "." >> /app/public/diff/mackenzie_macken_nodes_20580.diff
[CMD] [2020-04-03 15:04:26] echo "0a" > /app/public/diff/mackenzie_macken_occurrences_20581.diff
[CMD] [2020-04-03 15:04:26] tail -n +1 /app/public/converted_csv/mackenzie_macken_occurrences_20581.csv >> /app/public/diff/mackenzie_macken_occurrences_20581.diff
[CMD] [2020-04-03 15:04:26] echo "." >> /app/public/diff/mackenzie_macken_occurrences_20581.diff
[CMD] [2020-04-03 15:04:26] echo "0a" > /app/public/diff/mackenzie_macken_measurements_20582.diff
[CMD] [2020-04-03 15:04:26] tail -n +1 /app/public/converted_csv/mackenzie_macken_measurements_20582.csv >> /app/public/diff/mackenzie_macken_measurements_20582.diff
[CMD] [2020-04-03 15:04:26] echo "." >> /app/public/diff/mackenzie_macken_measurements_20582.diff
[STOP] [2020-04-03 15:04:26] calculate_delta
[START] [2020-04-03 15:04:26] parse_diff_and_store
[INFO] [2020-04-03 15:04:26] Loading nodes diff file into memory (true lines)...
[INFO] [2020-04-03 15:04:26] Loading occurrences diff file into memory (true lines)...
[INFO] [2020-04-03 15:04:26] Loading measurements diff file into memory (true lines)...
[INFO] [2020-04-03 15:04:26] Storing 2 ScientificNames
[INFO] [2020-04-03 15:04:26] Processing group of 2 in 1 groups of 1000
[INFO] [2020-04-03 15:04:26] Average Time: 0.0
[INFO] [2020-04-03 15:04:26] Total Time: 1s
[INFO] [2020-04-03 15:04:26] Storing 2 Nodes
[INFO] [2020-04-03 15:04:26] Processing group of 2 in 1 groups of 1000
[INFO] [2020-04-03 15:04:26] Average Time: 0.0
[INFO] [2020-04-03 15:04:26] Total Time: 1s
[INFO] [2020-04-03 15:04:26] Storing 2 Occurrences
[INFO] [2020-04-03 15:04:26] Processing group of 2 in 1 groups of 1000
[INFO] [2020-04-03 15:04:26] Average Time: 0.0
[INFO] [2020-04-03 15:04:26] Total Time: 1s
[INFO] [2020-04-03 15:04:26] Storing 8 Traits
[INFO] [2020-04-03 15:04:26] Processing group of 8 in 1 groups of 1000
[INFO] [2020-04-03 15:04:27] Average Time: 0.0
[INFO] [2020-04-03 15:04:27] Total Time: 1s
[INFO] [2020-04-03 15:04:27] Storing 6 MetaTraits
[INFO] [2020-04-03 15:04:27] Processing group of 6 in 1 groups of 1000
[INFO] [2020-04-03 15:04:27] Average Time: 0.0
[INFO] [2020-04-03 15:04:27] Total Time: 1s
[STOP] [2020-04-03 15:04:27] parse_diff_and_store
[START] [2020-04-03 15:04:27] resolve_keys
[INFO] [2020-04-03 15:04:32] Occurrences to nodes (through scientific_names)...
[INFO] [2020-04-03 15:04:32] traits to occurrences...
[INFO] [2020-04-03 15:04:32] traits to nodes (through occurrences)...
[INFO] [2020-04-03 15:04:32] Traits to sex term...
[INFO] [2020-04-03 15:04:32] Traits to lifestage term...
[INFO] [2020-04-03 15:04:32] MetaTraits to traits...
[INFO] [2020-04-03 15:04:32] MetaTraits (simple, measurement row refers to parent) to traits...
[INFO] [2020-04-03 15:04:32] Assocs to occurrences...
[INFO] [2020-04-03 15:04:32] Assocs to nodes...
[INFO] [2020-04-03 15:04:32] Assoc to sex term...
[INFO] [2020-04-03 15:04:32] Assoc to lifestage term...
[STOP] [2020-04-03 15:04:32] resolve_keys
[START] [2020-04-03 15:04:32] hold_for_later_1
[STOP] [2020-04-03 15:04:32] hold_for_later_1
[START] [2020-04-03 15:04:32] hold_for_later_2
[STOP] [2020-04-03 15:04:32] hold_for_later_2
[START] [2020-04-03 15:04:32] resolve_missing_parents
[STOP] [2020-04-03 15:04:32] resolve_missing_parents
[START] [2020-04-03 15:04:32] rebuild_nodes
[START] [2020-04-03 15:04:32] Flattener#flatten
[START] [2020-04-03 15:04:32] Flattener#study_resource
[START] [2020-04-03 15:04:32] Flattener#build_ancestry
[STOP] [2020-04-03 15:04:32] Flattener#build_ancestry
[INFO] [2020-04-03 15:04:32] 2 ancestry keys
[START] [2020-04-03 15:04:32] build_node_ancestors
[INFO] [2020-04-03 15:04:32] old ancestors deleted.
[STOP] [2020-04-03 15:04:32] build_node_ancestors
[WARN] [2020-04-03 15:04:32] Flattener: nothing to flatten! (Completely flat resource?)
[STOP] [2020-04-03 15:04:32] Flattener#flatten
[STOP] [2020-04-03 15:04:32] rebuild_nodes
[START] [2020-04-03 15:04:32] resolve_missing_media_owners
[STOP] [2020-04-03 15:04:32] resolve_missing_media_owners
[START] [2020-04-03 15:04:32] sanitize_media_verbatims
[STOP] [2020-04-03 15:04:32] sanitize_media_verbatims
[START] [2020-04-03 15:04:32] queue_downloads
[STOP] [2020-04-03 15:04:32] queue_downloads
[START] [2020-04-03 15:04:32] parse_names
[WARN] [2020-04-03 15:04:32] I see 2 names which still need to be parsed.
[STOP] [2020-04-03 15:04:33] parse_names
[START] [2020-04-03 15:04:33] denormalize_canonical_names_to_nodes
[STOP] [2020-04-03 15:04:33] denormalize_canonical_names_to_nodes
[START] [2020-04-03 15:04:33] match_nodes
[START] [2020-04-03 15:04:33] map_all_nodes_to_pages
[STOP] [2020-04-03 15:04:33] map_all_nodes_to_pages
[INFO] [2020-04-03 15:04:33] ZERO unmatched nodes (of 2)! Nicely done.
[START] [2020-04-03 15:04:33] update_nodes
[STOP] [2020-04-03 15:04:33] update_nodes
[STOP] [2020-04-03 15:04:33] match_nodes
[START] [2020-04-03 15:04:33] reindex_search
[STOP] [2020-04-03 15:04:33] reindex_search
[START] [2020-04-03 15:04:33] normalize_units
[STOP] [2020-04-03 15:04:33] normalize_units
[START] [2020-04-03 15:04:33] calculate_statistics
[2020-04-03 15:04:33] ZERO NODE ANCESTORS. Is this actually a completely flat resource?
[STOP] [2020-04-03 15:04:33] calculate_statistics
[START] [2020-04-03 15:04:33] complete_harvest_instance
[START] [2020-04-03 15:04:33] overall_tsv_creation
[INFO] [2020-04-03 15:04:33] Processing group of 2 in 1 batches of 10000
[INFO] [2020-04-03 15:05:22] 2 Traits (unfiltered)...
[INFO] [2020-04-03 15:05:35] 2 Traits (filtered)...
[INFO] [2020-04-03 15:05:35] 0 Associations (filtered)...
[INFO] [2020-04-03 15:06:11] 12 metadata added.
[INFO] [2020-04-03 15:06:11] 0 metadata added.
[INFO] [2020-04-03 15:06:11] Average Time: 69.42
[INFO] [2020-04-03 15:06:11] Total Time: 1m38s
[STOP] [2020-04-03 15:06:11] overall_tsv_creation
[INFO] [2020-04-03 15:06:11] Done. Check your files:
[INFO] [2020-04-03 15:06:11] (2 lines) /app/public/data/mackenzie_macken/publish_nodes.tsv
[INFO] [2020-04-03 15:06:11] (2 lines) /app/public/data/mackenzie_macken/publish_scientific_names.tsv
[INFO] [2020-04-03 15:06:11] (3 lines) /app/public/data/mackenzie_macken/publish_traits.tsv
[INFO] [2020-04-03 15:06:11] (13 lines) /app/public/data/mackenzie_macken/publish_metadata.tsv
[STOP] [2020-04-03 15:06:11] complete_harvest_instance
[START] [2020-04-03 15:06:11] completed
[STOP] [2020-04-03 15:06:11] completed
[STOP] [2020-04-03 15:06:11] logged process, took 105.14
[INFO] [2020-06-16 13:55:17] ## HARVEST: type = re_download_opendata_-harvest
[INFO] [2020-06-16 13:55:18] ## remove_type: ScientificName
[INFO] [2020-06-16 13:55:18] ++ Calling delete_all on 2 instances...
[INFO] [2020-06-16 13:55:18] [13:55:18.107] Removed 2 Scientificnames
[INFO] [2020-06-16 13:55:18] ## remove_type: Vernacular
[INFO] [2020-06-16 13:55:18] ++ Calling delete_all on 0 instances...
[INFO] [2020-06-16 13:55:18] [13:55:18.109] Removed 0 Vernaculars
[INFO] [2020-06-16 13:55:18] ## remove_type: Article
[INFO] [2020-06-16 13:55:18] ++ Calling delete_all on 0 instances...
[INFO] [2020-06-16 13:55:18] [13:55:18.112] Removed 0 Articles
[INFO] [2020-06-16 13:55:18] ## remove_type: Medium
[INFO] [2020-06-16 13:55:18] ++ Calling delete_all on 0 instances...
[INFO] [2020-06-16 13:55:18] [13:55:18.115] Removed 0 Media
[INFO] [2020-06-16 13:55:18] ## remove_type: Trait
[INFO] [2020-06-16 13:55:18] ++ Calling delete_all on 8 instances...
[INFO] [2020-06-16 13:55:18] [13:55:18.119] Removed 8 Traits
[INFO] [2020-06-16 13:55:18] ## remove_type: MetaTrait
[INFO] [2020-06-16 13:55:18] ++ Calling delete_all on 6 instances...
[INFO] [2020-06-16 13:55:18] [13:55:18.122] Removed 6 Metatraits
[INFO] [2020-06-16 13:55:18] ## remove_type: OccurrenceMetadatum
[INFO] [2020-06-16 13:55:18] ++ Calling delete_all on 0 instances...
[INFO] [2020-06-16 13:55:18] [13:55:18.125] Removed 0 Occurrencemetadata
[INFO] [2020-06-16 13:55:18] ## remove_type: Assoc
[INFO] [2020-06-16 13:55:18] ++ Calling delete_all on 0 instances...
[INFO] [2020-06-16 13:55:18] [13:55:18.128] Removed 0 Assocs
[INFO] [2020-06-16 13:55:18] ## remove_type: MetaAssoc
[INFO] [2020-06-16 13:55:18] ++ Calling delete_all on 0 instances...
[INFO] [2020-06-16 13:55:18] [13:55:18.131] Removed 0 Metaassocs
[INFO] [2020-06-16 13:55:18] ## remove_type: Identifier
[INFO] [2020-06-16 13:55:18] ++ Calling delete_all on 0 instances...
[INFO] [2020-06-16 13:55:18] [13:55:18.134] Removed 0 Identifiers
[INFO] [2020-06-16 13:55:18] ## remove_type: Reference
[INFO] [2020-06-16 13:55:18] ++ Calling delete_all on 0 instances...
[INFO] [2020-06-16 13:55:18] [13:55:18.136] Removed 0 References
[INFO] [2020-06-16 13:55:18] Starting batch with ID 68185326...
[INFO] [2020-06-16 13:55:18] Starting batch with ID 68185326...
[INFO] [2020-06-16 13:55:18] Starting batch with ID 68185326...
[INFO] [2020-06-16 13:55:18] Starting batch with ID 68185326...
[INFO] [2020-06-16 13:55:18] Starting batch with ID 68185326...
[INFO] [2020-06-16 13:55:18] Starting batch with ID 68185326...
[INFO] [2020-06-16 13:55:18] Starting batch with ID 68185326...
[INFO] [2020-06-16 13:55:18] Starting batch with ID 68185326...
[INFO] [2020-06-16 13:55:18] Starting batch with ID 68185326...
[INFO] [2020-06-16 13:55:18] Starting batch with ID 68185326...
[INFO] [2020-06-16 13:55:18] Starting batch with ID 68185326...
[INFO] [2020-06-16 13:55:18] Starting batch with ID 68185326...
[INFO] [2020-06-16 13:55:18] Starting batch with ID 68185326...
[INFO] [2020-06-16 13:55:18] Starting batch with ID 68185326...
[INFO] [2020-06-16 13:55:18] Starting batch with ID 68185326...
[INFO] [2020-06-16 13:55:18] Starting batch with ID 68185326...
[INFO] [2020-06-16 13:55:18] Starting batch with ID 68185326...
[INFO] [2020-06-16 13:55:18] Starting batch with ID 68185326...
[INFO] [2020-06-16 13:55:18] Starting batch with ID 68185326...
[INFO] [2020-06-16 13:55:18] Starting batch with ID 68185326...
[INFO] [2020-06-16 13:55:19] Starting batch with ID 68185326...
[INFO] [2020-06-16 13:55:19] ## remove_type: Node
[INFO] [2020-06-16 13:55:19] ++ Calling delete_all on 2 instances...
[INFO] [2020-06-16 13:55:19] [13:55:19.068] Removed 2 Nodes
[START] [2020-06-16 13:55:19] logged process
[START] [2020-06-16 13:55:19] Creating resource from OpenData
[START] [2020-06-16 13:55:19] logged process
[START] [2020-06-16 13:55:19] Parse meta.xml file and create formats with fields
[STOP] [2020-06-16 13:55:19] Parse meta.xml file and create formats with fields
[STOP] [2020-06-16 13:55:19] Creating resource from OpenData
[START] [2020-06-16 13:55:19] logged process
[START] [2020-06-16 13:55:19] create_harvest_instance
[STOP] [2020-06-16 13:55:20] create_harvest_instance
[START] [2020-06-16 13:55:20] fetch_files
[STOP] [2020-06-16 13:55:20] fetch_files
[START] [2020-06-16 13:55:20] validate_each_file
[STOP] [2020-06-16 13:55:20] validate_each_file
[START] [2020-06-16 13:55:20] convert_to_csv
[CMD] [2020-06-16 13:55:20] /usr/bin/sort /app/public/converted_csv/mackenzie_macken_nodes_21376.csv > /app/public/converted_csv/mackenzie_macken_nodes_21376.csv_sorted
[CMD] [2020-06-16 13:55:20] /usr/bin/sort /app/public/converted_csv/mackenzie_macken_occurrences_21377.csv > /app/public/converted_csv/mackenzie_macken_occurrences_21377.csv_sorted
[CMD] [2020-06-16 13:55:20] /usr/bin/sort /app/public/converted_csv/mackenzie_macken_measurements_21378.csv > /app/public/converted_csv/mackenzie_macken_measurements_21378.csv_sorted
[STOP] [2020-06-16 13:55:20] convert_to_csv
[START] [2020-06-16 13:55:20] calculate_delta
[CMD] [2020-06-16 13:55:20] echo "0a" > /app/public/diff/mackenzie_macken_nodes_21376.diff
[CMD] [2020-06-16 13:55:20] tail -n +1 /app/public/converted_csv/mackenzie_macken_nodes_21376.csv >> /app/public/diff/mackenzie_macken_nodes_21376.diff
[CMD] [2020-06-16 13:55:20] echo "." >> /app/public/diff/mackenzie_macken_nodes_21376.diff
[CMD] [2020-06-16 13:55:20] echo "0a" > /app/public/diff/mackenzie_macken_occurrences_21377.diff
[CMD] [2020-06-16 13:55:20] tail -n +1 /app/public/converted_csv/mackenzie_macken_occurrences_21377.csv >> /app/public/diff/mackenzie_macken_occurrences_21377.diff
[CMD] [2020-06-16 13:55:20] echo "." >> /app/public/diff/mackenzie_macken_occurrences_21377.diff
[CMD] [2020-06-16 13:55:20] echo "0a" > /app/public/diff/mackenzie_macken_measurements_21378.diff
[CMD] [2020-06-16 13:55:20] tail -n +1 /app/public/converted_csv/mackenzie_macken_measurements_21378.csv >> /app/public/diff/mackenzie_macken_measurements_21378.diff
[CMD] [2020-06-16 13:55:20] echo "." >> /app/public/diff/mackenzie_macken_measurements_21378.diff
[STOP] [2020-06-16 13:55:20] calculate_delta
[START] [2020-06-16 13:55:20] parse_diff_and_store
[INFO] [2020-06-16 13:55:20] Loading nodes diff file into memory (true lines)...
[INFO] [2020-06-16 13:55:21] Loading occurrences diff file into memory (true lines)...
[INFO] [2020-06-16 13:55:21] Loading measurements diff file into memory (true lines)...
[INFO] [2020-06-16 13:55:21] Storing 2 ScientificNames
[INFO] [2020-06-16 13:55:21] Processing group of 2 in 1 groups of 1000
[INFO] [2020-06-16 13:55:21] Average Time: 0.0
[INFO] [2020-06-16 13:55:21] Total Time: 1s
[INFO] [2020-06-16 13:55:21] Storing 2 Nodes
[INFO] [2020-06-16 13:55:21] Processing group of 2 in 1 groups of 1000
[INFO] [2020-06-16 13:55:21] Average Time: 0.0
[INFO] [2020-06-16 13:55:21] Total Time: 1s
[INFO] [2020-06-16 13:55:21] Storing 2 Occurrences
[INFO] [2020-06-16 13:55:21] Processing group of 2 in 1 groups of 1000
[INFO] [2020-06-16 13:55:21] Average Time: 0.0
[INFO] [2020-06-16 13:55:21] Total Time: 1s
[INFO] [2020-06-16 13:55:21] Storing 8 Traits
[INFO] [2020-06-16 13:55:21] Processing group of 8 in 1 groups of 1000
[INFO] [2020-06-16 13:55:21] Average Time: 0.0
[INFO] [2020-06-16 13:55:21] Total Time: 1s
[INFO] [2020-06-16 13:55:21] Storing 6 MetaTraits
[INFO] [2020-06-16 13:55:21] Processing group of 6 in 1 groups of 1000
[INFO] [2020-06-16 13:55:21] Average Time: 0.0
[INFO] [2020-06-16 13:55:21] Total Time: 1s
[STOP] [2020-06-16 13:55:21] parse_diff_and_store
[START] [2020-06-16 13:55:21] resolve_keys
[INFO] [2020-06-16 13:55:28] Occurrences to nodes (through scientific_names)...
[INFO] [2020-06-16 13:55:28] traits to occurrences...
[INFO] [2020-06-16 13:55:28] traits to nodes (through occurrences)...
[INFO] [2020-06-16 13:55:28] Traits to sex term...
[INFO] [2020-06-16 13:55:28] Traits to lifestage term...
[INFO] [2020-06-16 13:55:28] MetaTraits to traits...
[INFO] [2020-06-16 13:55:28] MetaTraits (simple, measurement row refers to parent) to traits...
[INFO] [2020-06-16 13:55:28] Assocs to occurrences...
[INFO] [2020-06-16 13:55:28] Assocs to nodes...
[INFO] [2020-06-16 13:55:28] Assoc to sex term...
[INFO] [2020-06-16 13:55:28] Assoc to lifestage term...
[STOP] [2020-06-16 13:55:28] resolve_keys
[START] [2020-06-16 13:55:28] hold_for_later_1
[STOP] [2020-06-16 13:55:28] hold_for_later_1
[START] [2020-06-16 13:55:28] hold_for_later_2
[STOP] [2020-06-16 13:55:28] hold_for_later_2
[START] [2020-06-16 13:55:28] resolve_missing_parents
[STOP] [2020-06-16 13:55:28] resolve_missing_parents
[START] [2020-06-16 13:55:28] rebuild_nodes
[START] [2020-06-16 13:55:28] Flattener#flatten
[START] [2020-06-16 13:55:28] Flattener#study_resource
[START] [2020-06-16 13:55:28] Flattener#build_ancestry
[STOP] [2020-06-16 13:55:28] Flattener#build_ancestry
[INFO] [2020-06-16 13:55:28] 2 ancestry keys
[START] [2020-06-16 13:55:28] build_node_ancestors
[INFO] [2020-06-16 13:55:28] old ancestors deleted.
[STOP] [2020-06-16 13:55:28] build_node_ancestors
[WARN] [2020-06-16 13:55:28] Flattener: nothing to flatten! (Completely flat resource?)
[STOP] [2020-06-16 13:55:28] Flattener#flatten
[STOP] [2020-06-16 13:55:28] rebuild_nodes
[START] [2020-06-16 13:55:28] resolve_missing_media_owners
[STOP] [2020-06-16 13:55:28] resolve_missing_media_owners
[START] [2020-06-16 13:55:28] sanitize_media_verbatims
[STOP] [2020-06-16 13:55:28] sanitize_media_verbatims
[START] [2020-06-16 13:55:28] queue_downloads
[STOP] [2020-06-16 13:55:28] queue_downloads
[START] [2020-06-16 13:55:28] parse_names
[WARN] [2020-06-16 13:55:28] I see 2 names which still need to be parsed.
[STOP] [2020-06-16 13:55:29] parse_names
[START] [2020-06-16 13:55:29] denormalize_canonical_names_to_nodes
[STOP] [2020-06-16 13:55:29] denormalize_canonical_names_to_nodes
[START] [2020-06-16 13:55:29] match_nodes
[START] [2020-06-16 13:55:29] map_all_nodes_to_pages
[STOP] [2020-06-16 13:55:29] map_all_nodes_to_pages
[INFO] [2020-06-16 13:55:29] ZERO unmatched nodes (of 2)! Nicely done.
[START] [2020-06-16 13:55:29] update_nodes
[STOP] [2020-06-16 13:55:29] update_nodes
[STOP] [2020-06-16 13:55:29] match_nodes
[START] [2020-06-16 13:55:29] reindex_search
[STOP] [2020-06-16 13:55:29] reindex_search
[START] [2020-06-16 13:55:29] normalize_units
[STOP] [2020-06-16 13:55:29] normalize_units
[START] [2020-06-16 13:55:29] calculate_statistics
[2020-06-16 13:55:29] ZERO NODE ANCESTORS. Is this actually a completely flat resource?
[STOP] [2020-06-16 13:55:29] calculate_statistics
[START] [2020-06-16 13:55:29] complete_harvest_instance
[START] [2020-06-16 13:55:29] overall_tsv_creation
[INFO] [2020-06-16 13:55:29] Processing group of 2 in 1 batches of 10000
[INFO] [2020-06-16 13:56:19] 2 Traits (unfiltered)...
[INFO] [2020-06-16 13:56:33] 2 Traits (filtered)...
[INFO] [2020-06-16 13:56:33] 0 Associations (filtered)...
[INFO] [2020-06-16 13:57:10] 12 metadata added.
[INFO] [2020-06-16 13:57:10] 0 metadata added.
[INFO] [2020-06-16 13:57:10] Average Time: 73.02
[INFO] [2020-06-16 13:57:10] Total Time: 1m42s
[STOP] [2020-06-16 13:57:10] overall_tsv_creation
[INFO] [2020-06-16 13:57:10] Done. Check your files:
[INFO] [2020-06-16 13:57:10] (2 lines) /app/public/data/mackenzie_macken/publish_nodes.tsv
[INFO] [2020-06-16 13:57:10] (2 lines) /app/public/data/mackenzie_macken/publish_scientific_names.tsv
[INFO] [2020-06-16 13:57:10] (3 lines) /app/public/data/mackenzie_macken/publish_traits.tsv
[INFO] [2020-06-16 13:57:10] (13 lines) /app/public/data/mackenzie_macken/publish_metadata.tsv
[STOP] [2020-06-16 13:57:10] complete_harvest_instance
[START] [2020-06-16 13:57:10] completed
[STOP] [2020-06-16 13:57:10] completed
[STOP] [2020-06-16 13:57:10] logged process, took 111.04
[INFO] [2020-11-13 11:46:01] ## HARVEST: type = re_download_opendata_-harvest
[INFO] [2020-11-13 11:46:02] ## remove_type: ScientificName
[INFO] [2020-11-13 11:46:02] ++ Calling delete_all on 2 instances...
[INFO] [2020-11-13 11:46:02] [11:46:02.352] Removed 2 Scientificnames
[INFO] [2020-11-13 11:46:02] ## remove_type: Vernacular
[INFO] [2020-11-13 11:46:02] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-13 11:46:02] [11:46:02.355] Removed 0 Vernaculars
[INFO] [2020-11-13 11:46:02] ## remove_type: Article
[INFO] [2020-11-13 11:46:02] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-13 11:46:02] [11:46:02.358] Removed 0 Articles
[INFO] [2020-11-13 11:46:02] ## remove_type: Medium
[INFO] [2020-11-13 11:46:02] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-13 11:46:02] [11:46:02.361] Removed 0 Media
[INFO] [2020-11-13 11:46:02] ## remove_type: Trait
[INFO] [2020-11-13 11:46:02] ++ Calling delete_all on 8 instances...
[INFO] [2020-11-13 11:46:02] [11:46:02.365] Removed 8 Traits
[INFO] [2020-11-13 11:46:02] ## remove_type: MetaTrait
[INFO] [2020-11-13 11:46:02] ++ Calling delete_all on 6 instances...
[INFO] [2020-11-13 11:46:02] [11:46:02.372] Removed 6 Metatraits
[INFO] [2020-11-13 11:46:02] ## remove_type: OccurrenceMetadatum
[INFO] [2020-11-13 11:46:02] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-13 11:46:02] [11:46:02.374] Removed 0 Occurrencemetadata
[INFO] [2020-11-13 11:46:02] ## remove_type: Assoc
[INFO] [2020-11-13 11:46:02] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-13 11:46:02] [11:46:02.377] Removed 0 Assocs
[INFO] [2020-11-13 11:46:02] ## remove_type: MetaAssoc
[INFO] [2020-11-13 11:46:02] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-13 11:46:02] [11:46:02.380] Removed 0 Metaassocs
[INFO] [2020-11-13 11:46:02] ## remove_type: Identifier
[INFO] [2020-11-13 11:46:02] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-13 11:46:02] [11:46:02.383] Removed 0 Identifiers
[INFO] [2020-11-13 11:46:02] ## remove_type: Reference
[INFO] [2020-11-13 11:46:02] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-13 11:46:02] [11:46:02.386] Removed 0 References
[INFO] [2020-11-13 11:46:02] Starting batch with ID 80104454...
[INFO] [2020-11-13 11:46:02] Starting batch with ID 80104454...
[INFO] [2020-11-13 11:46:02] Starting batch with ID 80104454...
[INFO] [2020-11-13 11:46:02] Starting batch with ID 80104454...
[INFO] [2020-11-13 11:46:02] Starting batch with ID 80104454...
[INFO] [2020-11-13 11:46:02] ## remove_type: Node
[INFO] [2020-11-13 11:46:02] ++ Calling delete_all on 2 instances...
[INFO] [2020-11-13 11:46:02] [11:46:02.541] Removed 2 Nodes
[START] [2020-11-13 11:46:02] logged process: 58bbc42b01abb4c1b2698de049792ffb4b63b979

[START] [2020-11-13 11:46:02] Creating resource from OpenData
[START] [2020-11-13 11:46:03] logged process: 58bbc42b01abb4c1b2698de049792ffb4b63b979

[START] [2020-11-13 11:46:03] Parse meta.xml file and create formats with fields
[STOP] [2020-11-13 11:46:03] Parse meta.xml file and create formats with fields
[STOP] [2020-11-13 11:46:03] Creating resource from OpenData
[START] [2020-11-13 11:46:03] logged process: 58bbc42b01abb4c1b2698de049792ffb4b63b979

[START] [2020-11-13 11:46:03] create_harvest_instance
[STOP] [2020-11-13 11:46:04] create_harvest_instance
[START] [2020-11-13 11:46:04] fetch_files
[STOP] [2020-11-13 11:46:04] fetch_files
[START] [2020-11-13 11:46:04] validate_each_file
[STOP] [2020-11-13 11:46:04] validate_each_file
[START] [2020-11-13 11:46:04] convert_to_csv
[CMD] [2020-11-13 11:46:04] /usr/bin/sort /app/public/converted_csv/mackenzie_macken_nodes_24189.csv > /app/public/converted_csv/mackenzie_macken_nodes_24189.csv_sorted
[CMD] [2020-11-13 11:46:04] /usr/bin/sort /app/public/converted_csv/mackenzie_macken_occurrences_24190.csv > /app/public/converted_csv/mackenzie_macken_occurrences_24190.csv_sorted
[CMD] [2020-11-13 11:46:04] /usr/bin/sort /app/public/converted_csv/mackenzie_macken_measurements_24191.csv > /app/public/converted_csv/mackenzie_macken_measurements_24191.csv_sorted
[STOP] [2020-11-13 11:46:04] convert_to_csv
[START] [2020-11-13 11:46:04] calculate_delta
[CMD] [2020-11-13 11:46:04] echo "0a" > /app/public/diff/mackenzie_macken_nodes_24189.diff
[CMD] [2020-11-13 11:46:04] tail -n +1 /app/public/converted_csv/mackenzie_macken_nodes_24189.csv >> /app/public/diff/mackenzie_macken_nodes_24189.diff
[CMD] [2020-11-13 11:46:04] echo "." >> /app/public/diff/mackenzie_macken_nodes_24189.diff
[CMD] [2020-11-13 11:46:04] echo "0a" > /app/public/diff/mackenzie_macken_occurrences_24190.diff
[CMD] [2020-11-13 11:46:04] tail -n +1 /app/public/converted_csv/mackenzie_macken_occurrences_24190.csv >> /app/public/diff/mackenzie_macken_occurrences_24190.diff
[CMD] [2020-11-13 11:46:04] echo "." >> /app/public/diff/mackenzie_macken_occurrences_24190.diff
[CMD] [2020-11-13 11:46:04] echo "0a" > /app/public/diff/mackenzie_macken_measurements_24191.diff
[CMD] [2020-11-13 11:46:04] tail -n +1 /app/public/converted_csv/mackenzie_macken_measurements_24191.csv >> /app/public/diff/mackenzie_macken_measurements_24191.diff
[CMD] [2020-11-13 11:46:04] echo "." >> /app/public/diff/mackenzie_macken_measurements_24191.diff
[STOP] [2020-11-13 11:46:04] calculate_delta
[START] [2020-11-13 11:46:04] parse_diff_and_store
[INFO] [2020-11-13 11:46:04] Loading nodes diff file into memory (true lines)...
[INFO] [2020-11-13 11:46:04] Loading occurrences diff file into memory (true lines)...
[INFO] [2020-11-13 11:46:04] Loading measurements diff file into memory (true lines)...
[INFO] [2020-11-13 11:46:06] Storing 2 ScientificNames
[INFO] [2020-11-13 11:46:06] Processing group of 2 in 1 groups of 1000
[INFO] [2020-11-13 11:46:06] Average Time: 0.0
[INFO] [2020-11-13 11:46:06] Total Time: 1s
[INFO] [2020-11-13 11:46:06] Storing 2 Nodes
[INFO] [2020-11-13 11:46:06] Processing group of 2 in 1 groups of 1000
[INFO] [2020-11-13 11:46:06] Average Time: 0.0
[INFO] [2020-11-13 11:46:06] Total Time: 1s
[INFO] [2020-11-13 11:46:06] Storing 2 Occurrences
[INFO] [2020-11-13 11:46:06] Processing group of 2 in 1 groups of 1000
[INFO] [2020-11-13 11:46:06] Average Time: 0.0
[INFO] [2020-11-13 11:46:06] Total Time: 1s
[INFO] [2020-11-13 11:46:06] Storing 8 Traits
[INFO] [2020-11-13 11:46:06] Processing group of 8 in 1 groups of 1000
[INFO] [2020-11-13 11:46:06] Average Time: 0.0
[INFO] [2020-11-13 11:46:06] Total Time: 1s
[INFO] [2020-11-13 11:46:06] Storing 4 MetaTraits
[INFO] [2020-11-13 11:46:06] Processing group of 4 in 1 groups of 1000
[INFO] [2020-11-13 11:46:06] Average Time: 0.0
[INFO] [2020-11-13 11:46:06] Total Time: 1s
[STOP] [2020-11-13 11:46:06] parse_diff_and_store
[START] [2020-11-13 11:46:06] resolve_keys
[INFO] [2020-11-13 11:46:11] Occurrences to nodes (through scientific_names)...
[INFO] [2020-11-13 11:46:11] traits to occurrences...
[INFO] [2020-11-13 11:46:11] traits to nodes (through occurrences)...
[INFO] [2020-11-13 11:46:11] Traits to sex term...
[INFO] [2020-11-13 11:46:11] Traits to lifestage term...
[INFO] [2020-11-13 11:46:11] MetaTraits to traits...
[INFO] [2020-11-13 11:46:11] MetaTraits (simple, measurement row refers to parent) to traits...
[INFO] [2020-11-13 11:46:11] Assocs to occurrences...
[INFO] [2020-11-13 11:46:11] Assocs to nodes...
[INFO] [2020-11-13 11:46:11] Assoc to sex term...
[INFO] [2020-11-13 11:46:11] Assoc to lifestage term...
[INFO] [2020-11-13 11:46:11] MetaAssoc to assocs...
[STOP] [2020-11-13 11:46:11] resolve_keys
[START] [2020-11-13 11:46:11] hold_for_later_1
[STOP] [2020-11-13 11:46:11] hold_for_later_1
[START] [2020-11-13 11:46:11] hold_for_later_2
[STOP] [2020-11-13 11:46:11] hold_for_later_2
[START] [2020-11-13 11:46:11] resolve_missing_parents
[STOP] [2020-11-13 11:46:11] resolve_missing_parents
[START] [2020-11-13 11:46:11] rebuild_nodes
[START] [2020-11-13 11:46:11] Flattener#flatten
[START] [2020-11-13 11:46:11] Flattener#study_resource
[START] [2020-11-13 11:46:11] Flattener#build_ancestry
[STOP] [2020-11-13 11:46:11] Flattener#build_ancestry
[INFO] [2020-11-13 11:46:11] 2 ancestry keys
[START] [2020-11-13 11:46:11] build_node_ancestors
[INFO] [2020-11-13 11:46:11] old ancestors deleted.
[STOP] [2020-11-13 11:46:11] build_node_ancestors
[WARN] [2020-11-13 11:46:11] Flattener: nothing to flatten! (Completely flat resource?)
[STOP] [2020-11-13 11:46:11] Flattener#flatten
[STOP] [2020-11-13 11:46:11] rebuild_nodes
[START] [2020-11-13 11:46:11] resolve_missing_media_owners
[STOP] [2020-11-13 11:46:11] resolve_missing_media_owners
[START] [2020-11-13 11:46:11] sanitize_media_verbatims
[STOP] [2020-11-13 11:46:11] sanitize_media_verbatims
[START] [2020-11-13 11:46:11] queue_downloads
[STOP] [2020-11-13 11:46:11] queue_downloads
[START] [2020-11-13 11:46:11] parse_names
[WARN] [2020-11-13 11:46:11] I see 2 names which still need to be parsed.
[STOP] [2020-11-13 11:46:12] parse_names
[START] [2020-11-13 11:46:12] denormalize_canonical_names_to_nodes
[STOP] [2020-11-13 11:46:12] denormalize_canonical_names_to_nodes
[START] [2020-11-13 11:46:12] match_nodes
[START] [2020-11-13 11:46:12] map_all_nodes_to_pages
[STOP] [2020-11-13 11:46:12] map_all_nodes_to_pages
[INFO] [2020-11-13 11:46:12] ZERO unmatched nodes (of 2)! Nicely done.
[START] [2020-11-13 11:46:12] update_nodes
[STOP] [2020-11-13 11:46:12] update_nodes
[STOP] [2020-11-13 11:46:12] match_nodes
[START] [2020-11-13 11:46:12] reindex_search
[STOP] [2020-11-13 11:46:12] reindex_search
[START] [2020-11-13 11:46:12] normalize_units
[STOP] [2020-11-13 11:46:12] normalize_units
[START] [2020-11-13 11:46:12] calculate_statistics
[2020-11-13 11:46:12] ZERO NODE ANCESTORS. Is this actually a completely flat resource?
[STOP] [2020-11-13 11:46:12] calculate_statistics
[START] [2020-11-13 11:46:12] complete_harvest_instance
[START] [2020-11-13 11:46:12] overall_tsv_creation
[INFO] [2020-11-13 11:46:12] Processing group of 2 in 1 batches of 10000
[INFO] [2020-11-13 11:46:52] 2 Traits (unfiltered)...
[INFO] [2020-11-13 11:47:29] 2 Traits (filtered)...
[INFO] [2020-11-13 11:47:29] 0 Associations (filtered)...
[INFO] [2020-11-13 11:47:29] 10 metadata added.
[INFO] [2020-11-13 11:47:29] 0 metadata added.
[INFO] [2020-11-13 11:47:29] Average Time: 52.5
[INFO] [2020-11-13 11:47:29] Total Time: 1m18s
[STOP] [2020-11-13 11:47:29] overall_tsv_creation
[INFO] [2020-11-13 11:47:29] Done. Check your files:
[INFO] [2020-11-13 11:47:29] (2 lines) /app/public/data/mackenzie_macken/publish_nodes.tsv
[INFO] [2020-11-13 11:47:29] (2 lines) /app/public/data/mackenzie_macken/publish_scientific_names.tsv
[INFO] [2020-11-13 11:47:29] (3 lines) /app/public/data/mackenzie_macken/publish_traits.tsv
[INFO] [2020-11-13 11:47:29] (9 lines) /app/public/data/mackenzie_macken/publish_metadata.tsv
[STOP] [2020-11-13 11:47:29] complete_harvest_instance
[START] [2020-11-13 11:47:29] completed
[STOP] [2020-11-13 11:47:29] completed
[STOP] [2020-11-13 11:47:29] logged process, took 86.74
[INFO] [2020-11-13 13:27:21] ## HARVEST: type = re_download_opendata_-harvest
[INFO] [2020-11-13 13:27:22] ## remove_type: ScientificName
[INFO] [2020-11-13 13:27:22] ++ Calling delete_all on 2 instances...
[INFO] [2020-11-13 13:27:22] [13:27:22.510] Removed 2 Scientificnames
[INFO] [2020-11-13 13:27:22] ## remove_type: Vernacular
[INFO] [2020-11-13 13:27:22] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-13 13:27:22] [13:27:22.512] Removed 0 Vernaculars
[INFO] [2020-11-13 13:27:22] ## remove_type: Article
[INFO] [2020-11-13 13:27:22] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-13 13:27:22] [13:27:22.515] Removed 0 Articles
[INFO] [2020-11-13 13:27:22] ## remove_type: Medium
[INFO] [2020-11-13 13:27:22] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-13 13:27:22] [13:27:22.518] Removed 0 Media
[INFO] [2020-11-13 13:27:22] ## remove_type: Trait
[INFO] [2020-11-13 13:27:22] ++ Calling delete_all on 8 instances...
[INFO] [2020-11-13 13:27:22] [13:27:22.522] Removed 8 Traits
[INFO] [2020-11-13 13:27:22] ## remove_type: MetaTrait
[INFO] [2020-11-13 13:27:22] ++ Calling delete_all on 4 instances...
[INFO] [2020-11-13 13:27:22] [13:27:22.525] Removed 4 Metatraits
[INFO] [2020-11-13 13:27:22] ## remove_type: OccurrenceMetadatum
[INFO] [2020-11-13 13:27:22] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-13 13:27:22] [13:27:22.528] Removed 0 Occurrencemetadata
[INFO] [2020-11-13 13:27:22] ## remove_type: Assoc
[INFO] [2020-11-13 13:27:22] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-13 13:27:22] [13:27:22.531] Removed 0 Assocs
[INFO] [2020-11-13 13:27:22] ## remove_type: MetaAssoc
[INFO] [2020-11-13 13:27:22] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-13 13:27:22] [13:27:22.534] Removed 0 Metaassocs
[INFO] [2020-11-13 13:27:22] ## remove_type: Identifier
[INFO] [2020-11-13 13:27:22] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-13 13:27:22] [13:27:22.537] Removed 0 Identifiers
[INFO] [2020-11-13 13:27:22] ## remove_type: Reference
[INFO] [2020-11-13 13:27:22] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-13 13:27:22] [13:27:22.540] Removed 0 References
[INFO] [2020-11-13 13:27:22] Starting batch with ID 81659595...
[INFO] [2020-11-13 13:27:22] Starting batch with ID 81659595...
[INFO] [2020-11-13 13:27:22] Starting batch with ID 81659595...
[INFO] [2020-11-13 13:27:22] Starting batch with ID 81659595...
[INFO] [2020-11-13 13:27:22] Starting batch with ID 81659595...
[INFO] [2020-11-13 13:27:22] Starting batch with ID 81659595...
[INFO] [2020-11-13 13:27:22] Starting batch with ID 81659595...
[INFO] [2020-11-13 13:27:22] Starting batch with ID 81659595...
[INFO] [2020-11-13 13:27:22] Starting batch with ID 81659595...
[INFO] [2020-11-13 13:27:22] Starting batch with ID 81659595...
[INFO] [2020-11-13 13:27:22] Starting batch with ID 81659595...
[INFO] [2020-11-13 13:27:22] Starting batch with ID 81659595...
[INFO] [2020-11-13 13:27:22] Starting batch with ID 81659595...
[INFO] [2020-11-13 13:27:22] Starting batch with ID 81659595...
[INFO] [2020-11-13 13:27:22] Starting batch with ID 81659595...
[INFO] [2020-11-13 13:27:22] Starting batch with ID 81659595...
[INFO] [2020-11-13 13:27:22] Starting batch with ID 81659595...
[INFO] [2020-11-13 13:27:22] Starting batch with ID 81659595...
[INFO] [2020-11-13 13:27:22] Starting batch with ID 81659595...
[INFO] [2020-11-13 13:27:22] Starting batch with ID 81659595...
[INFO] [2020-11-13 13:27:23] Starting batch with ID 81659595...
[INFO] [2020-11-13 13:27:23] Starting batch with ID 81659595...
[INFO] [2020-11-13 13:27:23] ## remove_type: Node
[INFO] [2020-11-13 13:27:23] ++ Calling delete_all on 2 instances...
[INFO] [2020-11-13 13:27:23] [13:27:23.302] Removed 2 Nodes
[START] [2020-11-13 13:27:23] logged process: 58bbc42b01abb4c1b2698de049792ffb4b63b979

[START] [2020-11-13 13:27:23] Creating resource from OpenData
[START] [2020-11-13 13:27:23] logged process: 58bbc42b01abb4c1b2698de049792ffb4b63b979

[START] [2020-11-13 13:27:23] Parse meta.xml file and create formats with fields
[STOP] [2020-11-13 13:27:23] Parse meta.xml file and create formats with fields
[STOP] [2020-11-13 13:27:23] Creating resource from OpenData
[START] [2020-11-13 13:27:23] logged process: 58bbc42b01abb4c1b2698de049792ffb4b63b979

[START] [2020-11-13 13:27:23] create_harvest_instance
[STOP] [2020-11-13 13:27:25] create_harvest_instance
[START] [2020-11-13 13:27:25] fetch_files
[STOP] [2020-11-13 13:27:25] fetch_files
[START] [2020-11-13 13:27:25] validate_each_file
[STOP] [2020-11-13 13:27:25] validate_each_file
[START] [2020-11-13 13:27:25] convert_to_csv
[CMD] [2020-11-13 13:27:25] /usr/bin/sort /app/public/converted_csv/mackenzie_macken_nodes_24221.csv > /app/public/converted_csv/mackenzie_macken_nodes_24221.csv_sorted
[CMD] [2020-11-13 13:27:25] /usr/bin/sort /app/public/converted_csv/mackenzie_macken_occurrences_24222.csv > /app/public/converted_csv/mackenzie_macken_occurrences_24222.csv_sorted
[CMD] [2020-11-13 13:27:25] /usr/bin/sort /app/public/converted_csv/mackenzie_macken_measurements_24223.csv > /app/public/converted_csv/mackenzie_macken_measurements_24223.csv_sorted
[STOP] [2020-11-13 13:27:25] convert_to_csv
[START] [2020-11-13 13:27:25] calculate_delta
[CMD] [2020-11-13 13:27:25] echo "0a" > /app/public/diff/mackenzie_macken_nodes_24221.diff
[CMD] [2020-11-13 13:27:25] tail -n +1 /app/public/converted_csv/mackenzie_macken_nodes_24221.csv >> /app/public/diff/mackenzie_macken_nodes_24221.diff
[CMD] [2020-11-13 13:27:25] echo "." >> /app/public/diff/mackenzie_macken_nodes_24221.diff
[CMD] [2020-11-13 13:27:25] echo "0a" > /app/public/diff/mackenzie_macken_occurrences_24222.diff
[CMD] [2020-11-13 13:27:25] tail -n +1 /app/public/converted_csv/mackenzie_macken_occurrences_24222.csv >> /app/public/diff/mackenzie_macken_occurrences_24222.diff
[CMD] [2020-11-13 13:27:25] echo "." >> /app/public/diff/mackenzie_macken_occurrences_24222.diff
[CMD] [2020-11-13 13:27:25] echo "0a" > /app/public/diff/mackenzie_macken_measurements_24223.diff
[CMD] [2020-11-13 13:27:25] tail -n +1 /app/public/converted_csv/mackenzie_macken_measurements_24223.csv >> /app/public/diff/mackenzie_macken_measurements_24223.diff
[CMD] [2020-11-13 13:27:25] echo "." >> /app/public/diff/mackenzie_macken_measurements_24223.diff
[STOP] [2020-11-13 13:27:25] calculate_delta
[START] [2020-11-13 13:27:25] parse_diff_and_store
[INFO] [2020-11-13 13:27:25] Loading nodes diff file into memory (true lines)...
[INFO] [2020-11-13 13:27:25] Loading occurrences diff file into memory (true lines)...
[INFO] [2020-11-13 13:27:26] Loading measurements diff file into memory (true lines)...
[INFO] [2020-11-13 13:27:26] Storing 2 ScientificNames
[INFO] [2020-11-13 13:27:26] Processing group of 2 in 1 groups of 1000
[INFO] [2020-11-13 13:27:26] Average Time: 0.0
[INFO] [2020-11-13 13:27:26] Total Time: 1s
[INFO] [2020-11-13 13:27:26] Storing 2 Nodes
[INFO] [2020-11-13 13:27:26] Processing group of 2 in 1 groups of 1000
[INFO] [2020-11-13 13:27:26] Average Time: 0.0
[INFO] [2020-11-13 13:27:26] Total Time: 1s
[INFO] [2020-11-13 13:27:26] Storing 2 Occurrences
[INFO] [2020-11-13 13:27:26] Processing group of 2 in 1 groups of 1000
[INFO] [2020-11-13 13:27:26] Average Time: 0.0
[INFO] [2020-11-13 13:27:26] Total Time: 1s
[INFO] [2020-11-13 13:27:26] Storing 2 OccurrenceMetadata
[INFO] [2020-11-13 13:27:26] Processing group of 2 in 1 groups of 1000
[INFO] [2020-11-13 13:27:26] Average Time: 0.0
[INFO] [2020-11-13 13:27:26] Total Time: 1s
[INFO] [2020-11-13 13:27:26] Storing 8 Traits
[INFO] [2020-11-13 13:27:26] Processing group of 8 in 1 groups of 1000
[INFO] [2020-11-13 13:27:26] Average Time: 0.0
[INFO] [2020-11-13 13:27:26] Total Time: 1s
[INFO] [2020-11-13 13:27:26] Storing 2 MetaTraits
[INFO] [2020-11-13 13:27:26] Processing group of 2 in 1 groups of 1000
[INFO] [2020-11-13 13:27:26] Average Time: 0.0
[INFO] [2020-11-13 13:27:26] Total Time: 1s
[STOP] [2020-11-13 13:27:26] parse_diff_and_store
[START] [2020-11-13 13:27:26] resolve_keys
[INFO] [2020-11-13 13:27:32] Occurrences to nodes (through scientific_names)...
[INFO] [2020-11-13 13:27:32] traits to occurrences...
[INFO] [2020-11-13 13:27:32] traits to nodes (through occurrences)...
[INFO] [2020-11-13 13:27:32] Traits to sex term...
[INFO] [2020-11-13 13:27:32] Traits to lifestage term...
[INFO] [2020-11-13 13:27:32] MetaTraits to traits...
[INFO] [2020-11-13 13:27:32] MetaTraits (simple, measurement row refers to parent) to traits...
[INFO] [2020-11-13 13:27:32] Assocs to occurrences...
[INFO] [2020-11-13 13:27:32] Assocs to nodes...
[INFO] [2020-11-13 13:27:32] Assoc to sex term...
[INFO] [2020-11-13 13:27:32] Assoc to lifestage term...
[INFO] [2020-11-13 13:27:32] MetaAssoc to assocs...
[STOP] [2020-11-13 13:27:32] resolve_keys
[START] [2020-11-13 13:27:32] hold_for_later_1
[STOP] [2020-11-13 13:27:32] hold_for_later_1
[START] [2020-11-13 13:27:32] hold_for_later_2
[STOP] [2020-11-13 13:27:32] hold_for_later_2
[START] [2020-11-13 13:27:32] resolve_missing_parents
[STOP] [2020-11-13 13:27:32] resolve_missing_parents
[START] [2020-11-13 13:27:32] rebuild_nodes
[START] [2020-11-13 13:27:32] Flattener#flatten
[START] [2020-11-13 13:27:32] Flattener#study_resource
[START] [2020-11-13 13:27:32] Flattener#build_ancestry
[STOP] [2020-11-13 13:27:32] Flattener#build_ancestry
[INFO] [2020-11-13 13:27:32] 2 ancestry keys
[START] [2020-11-13 13:27:32] build_node_ancestors
[INFO] [2020-11-13 13:27:32] old ancestors deleted.
[STOP] [2020-11-13 13:27:32] build_node_ancestors
[WARN] [2020-11-13 13:27:32] Flattener: nothing to flatten! (Completely flat resource?)
[STOP] [2020-11-13 13:27:32] Flattener#flatten
[STOP] [2020-11-13 13:27:32] rebuild_nodes
[START] [2020-11-13 13:27:32] resolve_missing_media_owners
[STOP] [2020-11-13 13:27:32] resolve_missing_media_owners
[START] [2020-11-13 13:27:32] sanitize_media_verbatims
[STOP] [2020-11-13 13:27:32] sanitize_media_verbatims
[START] [2020-11-13 13:27:32] queue_downloads
[STOP] [2020-11-13 13:27:32] queue_downloads
[START] [2020-11-13 13:27:32] parse_names
[WARN] [2020-11-13 13:27:32] I see 2 names which still need to be parsed.
[STOP] [2020-11-13 13:27:33] parse_names
[START] [2020-11-13 13:27:33] denormalize_canonical_names_to_nodes
[STOP] [2020-11-13 13:27:33] denormalize_canonical_names_to_nodes
[START] [2020-11-13 13:27:33] match_nodes
[START] [2020-11-13 13:27:33] map_all_nodes_to_pages
[STOP] [2020-11-13 13:27:33] map_all_nodes_to_pages
[INFO] [2020-11-13 13:27:33] ZERO unmatched nodes (of 2)! Nicely done.
[START] [2020-11-13 13:27:33] update_nodes
[STOP] [2020-11-13 13:27:33] update_nodes
[STOP] [2020-11-13 13:27:33] match_nodes
[START] [2020-11-13 13:27:33] reindex_search
[STOP] [2020-11-13 13:27:33] reindex_search
[START] [2020-11-13 13:27:33] normalize_units
[STOP] [2020-11-13 13:27:33] normalize_units
[START] [2020-11-13 13:27:33] calculate_statistics
[2020-11-13 13:27:33] ZERO NODE ANCESTORS. Is this actually a completely flat resource?
[STOP] [2020-11-13 13:27:33] calculate_statistics
[START] [2020-11-13 13:27:33] complete_harvest_instance
[START] [2020-11-13 13:27:33] overall_tsv_creation
[INFO] [2020-11-13 13:27:33] Processing group of 2 in 1 batches of 10000
[INFO] [2020-11-13 13:28:12] 2 Traits (unfiltered)...
[INFO] [2020-11-13 13:28:50] 2 Traits (filtered)...
[INFO] [2020-11-13 13:28:50] 0 Associations (filtered)...
[INFO] [2020-11-13 13:28:50] 10 metadata added.
[INFO] [2020-11-13 13:28:50] 0 metadata added.
[INFO] [2020-11-13 13:28:50] Average Time: 52.32
[INFO] [2020-11-13 13:28:50] Total Time: 1m17s
[STOP] [2020-11-13 13:28:50] overall_tsv_creation
[INFO] [2020-11-13 13:28:50] Done. Check your files:
[INFO] [2020-11-13 13:28:50] (2 lines) /app/public/data/mackenzie_macken/publish_nodes.tsv
[INFO] [2020-11-13 13:28:50] (2 lines) /app/public/data/mackenzie_macken/publish_scientific_names.tsv
[INFO] [2020-11-13 13:28:50] (3 lines) /app/public/data/mackenzie_macken/publish_traits.tsv
[INFO] [2020-11-13 13:28:50] (9 lines) /app/public/data/mackenzie_macken/publish_metadata.tsv
[STOP] [2020-11-13 13:28:50] complete_harvest_instance
[START] [2020-11-13 13:28:50] completed
[STOP] [2020-11-13 13:28:50] completed
[STOP] [2020-11-13 13:28:50] logged process, took 86.64
[INFO] [2021-04-19 14:39:27] ## HARVEST: type = re_download_opendata_-harvest
[INFO] [2021-04-19 14:41:04] ## remove_type: ScientificName
[INFO] [2021-04-19 14:41:04] ++ Calling delete_all on 2 instances...
[INFO] [2021-04-19 14:41:04] [14:41:04.858] Removed 2 Scientificnames
[INFO] [2021-04-19 14:41:04] ## remove_type: Vernacular
[INFO] [2021-04-19 14:41:04] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-19 14:41:04] [14:41:04.859] Removed 0 Vernaculars
[INFO] [2021-04-19 14:41:04] ## remove_type: Article
[INFO] [2021-04-19 14:41:04] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-19 14:41:04] [14:41:04.860] Removed 0 Articles
[INFO] [2021-04-19 14:41:04] ## remove_type: Medium
[INFO] [2021-04-19 14:41:04] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-19 14:41:04] [14:41:04.862] Removed 0 Media
[INFO] [2021-04-19 14:41:04] ## remove_type: Trait
[INFO] [2021-04-19 14:41:04] ++ Calling delete_all on 8 instances...
[INFO] [2021-04-19 14:41:04] [14:41:04.864] Removed 8 Traits
[INFO] [2021-04-19 14:41:04] ## remove_type: MetaTrait
[INFO] [2021-04-19 14:41:04] ++ Calling delete_all on 2 instances...
[INFO] [2021-04-19 14:41:04] [14:41:04.865] Removed 2 Metatraits
[INFO] [2021-04-19 14:41:04] ## remove_type: OccurrenceMetadatum
[INFO] [2021-04-19 14:41:04] ++ Calling delete_all on 2 instances...
[INFO] [2021-04-19 14:41:04] [14:41:04.867] Removed 2 Occurrencemetadata
[INFO] [2021-04-19 14:41:04] ## remove_type: Assoc
[INFO] [2021-04-19 14:41:04] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-19 14:41:04] [14:41:04.868] Removed 0 Assocs
[INFO] [2021-04-19 14:41:04] ## remove_type: MetaAssoc
[INFO] [2021-04-19 14:41:04] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-19 14:41:04] [14:41:04.869] Removed 0 Metaassocs
[INFO] [2021-04-19 14:41:04] ## remove_type: Identifier
[INFO] [2021-04-19 14:41:04] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-19 14:41:04] [14:41:04.871] Removed 0 Identifiers
[INFO] [2021-04-19 14:41:04] ## remove_type: Reference
[INFO] [2021-04-19 14:41:04] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-19 14:41:04] [14:41:04.872] Removed 0 References
[INFO] [2021-04-19 14:41:04] Starting batch with ID 81659618...
[INFO] [2021-04-19 14:41:04] Starting batch with ID 81659618...
[INFO] [2021-04-19 14:41:04] Starting batch with ID 81659618...
[INFO] [2021-04-19 14:41:04] Starting batch with ID 81659618...
[INFO] [2021-04-19 14:41:04] Starting batch with ID 81659618...
[INFO] [2021-04-19 14:41:04] Starting batch with ID 81659618...
[INFO] [2021-04-19 14:41:04] Starting batch with ID 81659618...
[INFO] [2021-04-19 14:41:04] Starting batch with ID 81659618...
[INFO] [2021-04-19 14:41:04] Starting batch with ID 81659618...
[INFO] [2021-04-19 14:41:04] Starting batch with ID 81659618...
[INFO] [2021-04-19 14:41:04] Starting batch with ID 81659618...
[INFO] [2021-04-19 14:41:04] Starting batch with ID 81659618...
[INFO] [2021-04-19 14:41:04] Starting batch with ID 81659618...
[INFO] [2021-04-19 14:41:04] Starting batch with ID 81659618...
[INFO] [2021-04-19 14:41:04] Starting batch with ID 81659618...
[INFO] [2021-04-19 14:41:04] Starting batch with ID 81659618...
[INFO] [2021-04-19 14:41:04] Starting batch with ID 81659618...
[INFO] [2021-04-19 14:41:04] Starting batch with ID 81659618...
[INFO] [2021-04-19 14:41:05] Starting batch with ID 81659618...
[INFO] [2021-04-19 14:41:05] Starting batch with ID 81659618...
[INFO] [2021-04-19 14:41:05] Starting batch with ID 81659618...
[INFO] [2021-04-19 14:41:05] Starting batch with ID 81659618...
[INFO] [2021-04-19 14:41:05] Starting batch with ID 81659618...
[INFO] [2021-04-19 14:41:05] Starting batch with ID 81659618...
[INFO] [2021-04-19 14:41:05] Starting batch with ID 81659618...
[INFO] [2021-04-19 14:41:05] Starting batch with ID 81659618...
[INFO] [2021-04-19 14:41:05] Starting batch with ID 81659618...
[INFO] [2021-04-19 14:41:05] Starting batch with ID 81659618...
[INFO] [2021-04-19 14:41:05] Starting batch with ID 81659618...
[INFO] [2021-04-19 14:41:05] Starting batch with ID 81659618...
[INFO] [2021-04-19 14:41:05] Starting batch with ID 81659618...
[INFO] [2021-04-19 14:41:05] Starting batch with ID 81659618...
[INFO] [2021-04-19 14:41:05] ## remove_type: Node
[INFO] [2021-04-19 14:41:05] ++ Calling delete_all on 2 instances...
[INFO] [2021-04-19 14:41:05] [14:41:05.094] Removed 2 Nodes
[START] [2021-04-19 14:41:05] logged process: 5ecc716a6a5541910d0c854f5a0c8d1651b82ad0 Improved MetaXml.ignore and added publisher to media (ignored)
[START] [2021-04-19 14:41:05] Creating resource from OpenData
[START] [2021-04-19 14:41:05] logged process: 5ecc716a6a5541910d0c854f5a0c8d1651b82ad0 Improved MetaXml.ignore and added publisher to media (ignored)
[START] [2021-04-19 14:41:05] Parse meta.xml file and create formats with fields
[STOP] [2021-04-19 14:41:05] Parse meta.xml file and create formats with fields
[STOP] [2021-04-19 14:41:05] Creating resource from OpenData
[START] [2021-04-19 14:41:05] logged process: 5ecc716a6a5541910d0c854f5a0c8d1651b82ad0 Improved MetaXml.ignore and added publisher to media (ignored)
[START] [2021-04-19 14:41:05] create_harvest_instance
[INFO] [2021-04-19 14:41:05] Created harvest instance #3747
[STOP] [2021-04-19 14:41:05] create_harvest_instance
[START] [2021-04-19 14:41:05] fetch_files
[STOP] [2021-04-19 14:41:05] fetch_files
[START] [2021-04-19 14:41:05] validate_each_file
[INFO] [2021-04-19 14:41:05] Looping over 3 formats...
[INFO] [2021-04-19 14:41:05] ...nodes (/app/public/data/mackenzie_macken/taxa.txt)
[INFO] [2021-04-19 14:41:05] Valid: /app/public/converted_csv/mackenzie_macken_nodes_3747.csv (2 lines)
[INFO] [2021-04-19 14:41:05] ...occurrences (/app/public/data/mackenzie_macken/occurrences.txt)
[INFO] [2021-04-19 14:41:05] Valid: /app/public/converted_csv/mackenzie_macken_occurrences_3747.csv (2 lines)
[INFO] [2021-04-19 14:41:05] ...measurements (/app/public/data/mackenzie_macken/measurementsorfacts.txt)
[INFO] [2021-04-19 14:41:05] Valid: /app/public/converted_csv/mackenzie_macken_measurements_3747.csv (8 lines)
[STOP] [2021-04-19 14:41:05] validate_each_file
[START] [2021-04-19 14:41:05] convert_to_csv
[INFO] [2021-04-19 14:41:05] Looping over 3 formats...
[INFO] [2021-04-19 14:41:05] ...nodes (/app/public/data/mackenzie_macken/taxa.txt)
[CMD] [2021-04-19 14:41:05] /usr/bin/sort /app/public/converted_csv/mackenzie_macken_nodes_3747.csv > /app/public/converted_csv/mackenzie_macken_nodes_3747.csv_sorted
[INFO] [2021-04-19 14:41:05] Converted: /app/public/converted_csv/mackenzie_macken_nodes_3747.csv (2 lines)
[INFO] [2021-04-19 14:41:05] ...occurrences (/app/public/data/mackenzie_macken/occurrences.txt)
[CMD] [2021-04-19 14:41:05] /usr/bin/sort /app/public/converted_csv/mackenzie_macken_occurrences_3747.csv > /app/public/converted_csv/mackenzie_macken_occurrences_3747.csv_sorted
[INFO] [2021-04-19 14:41:05] Converted: /app/public/converted_csv/mackenzie_macken_occurrences_3747.csv (2 lines)
[INFO] [2021-04-19 14:41:05] ...measurements (/app/public/data/mackenzie_macken/measurementsorfacts.txt)
[CMD] [2021-04-19 14:41:05] /usr/bin/sort /app/public/converted_csv/mackenzie_macken_measurements_3747.csv > /app/public/converted_csv/mackenzie_macken_measurements_3747.csv_sorted
[INFO] [2021-04-19 14:41:05] Converted: /app/public/converted_csv/mackenzie_macken_measurements_3747.csv (8 lines)
[STOP] [2021-04-19 14:41:05] convert_to_csv
[START] [2021-04-19 14:41:05] calculate_delta
[INFO] [2021-04-19 14:41:05] Looping over 3 formats...
[INFO] [2021-04-19 14:41:05] ...nodes (/app/public/data/mackenzie_macken/taxa.txt)
[CMD] [2021-04-19 14:41:05] echo "0a" > /app/public/diff/mackenzie_macken_nodes_3747.diff
[CMD] [2021-04-19 14:41:05] tail -n +1 /app/public/converted_csv/mackenzie_macken_nodes_3747.csv >> /app/public/diff/mackenzie_macken_nodes_3747.diff
[CMD] [2021-04-19 14:41:05] echo "." >> /app/public/diff/mackenzie_macken_nodes_3747.diff
[INFO] [2021-04-19 14:41:05] Created diff: /app/public/diff/mackenzie_macken_nodes_3747.diff (4 lines)
[INFO] [2021-04-19 14:41:05] ...occurrences (/app/public/data/mackenzie_macken/occurrences.txt)
[CMD] [2021-04-19 14:41:05] echo "0a" > /app/public/diff/mackenzie_macken_occurrences_3747.diff
[CMD] [2021-04-19 14:41:05] tail -n +1 /app/public/converted_csv/mackenzie_macken_occurrences_3747.csv >> /app/public/diff/mackenzie_macken_occurrences_3747.diff
[CMD] [2021-04-19 14:41:05] echo "." >> /app/public/diff/mackenzie_macken_occurrences_3747.diff
[INFO] [2021-04-19 14:41:05] Created diff: /app/public/diff/mackenzie_macken_occurrences_3747.diff (4 lines)
[INFO] [2021-04-19 14:41:05] ...measurements (/app/public/data/mackenzie_macken/measurementsorfacts.txt)
[CMD] [2021-04-19 14:41:05] echo "0a" > /app/public/diff/mackenzie_macken_measurements_3747.diff
[CMD] [2021-04-19 14:41:05] tail -n +1 /app/public/converted_csv/mackenzie_macken_measurements_3747.csv >> /app/public/diff/mackenzie_macken_measurements_3747.diff
[CMD] [2021-04-19 14:41:05] echo "." >> /app/public/diff/mackenzie_macken_measurements_3747.diff
[INFO] [2021-04-19 14:41:05] Created diff: /app/public/diff/mackenzie_macken_measurements_3747.diff (10 lines)
[STOP] [2021-04-19 14:41:05] calculate_delta
[START] [2021-04-19 14:41:05] parse_diff_and_store
[INFO] [2021-04-19 14:41:05] Handling diff: /app/public/diff/mackenzie_macken_nodes_3747.diff (4 lines)
[INFO] [2021-04-19 14:41:05] Loading nodes diff file into memory (4 /app/public/diff/mackenzie_macken_nodes_3747.diff lines)...
[INFO] [2021-04-19 14:41:05] Handling diff: /app/public/diff/mackenzie_macken_occurrences_3747.diff (4 lines)
[INFO] [2021-04-19 14:41:05] Loading occurrences diff file into memory (4 /app/public/diff/mackenzie_macken_occurrences_3747.diff lines)...
[INFO] [2021-04-19 14:41:05] Handling diff: /app/public/diff/mackenzie_macken_measurements_3747.diff (10 lines)
[INFO] [2021-04-19 14:41:05] Loading measurements diff file into memory (10 /app/public/diff/mackenzie_macken_measurements_3747.diff lines)...
[INFO] [2021-04-19 14:41:05] Storing 2 ScientificNames
[INFO] [2021-04-19 14:41:05] Processing group of 2 in 1 groups of 1000
[INFO] [2021-04-19 14:41:05] Average Time: 0.0
[INFO] [2021-04-19 14:41:05] Total Time: 1s
[INFO] [2021-04-19 14:41:05] Storing 2 Nodes
[INFO] [2021-04-19 14:41:05] Processing group of 2 in 1 groups of 1000
[INFO] [2021-04-19 14:41:05] Average Time: 0.0
[INFO] [2021-04-19 14:41:05] Total Time: 1s
[INFO] [2021-04-19 14:41:05] Storing 2 Occurrences
[INFO] [2021-04-19 14:41:05] Processing group of 2 in 1 groups of 1000
[INFO] [2021-04-19 14:41:06] Average Time: 0.0
[INFO] [2021-04-19 14:41:06] Total Time: 1s
[INFO] [2021-04-19 14:41:06] Storing 2 OccurrenceMetadata
[INFO] [2021-04-19 14:41:06] Processing group of 2 in 1 groups of 1000
[INFO] [2021-04-19 14:41:06] Average Time: 0.0
[INFO] [2021-04-19 14:41:06] Total Time: 1s
[INFO] [2021-04-19 14:41:06] Storing 8 Traits
[INFO] [2021-04-19 14:41:06] Processing group of 8 in 1 groups of 1000
[INFO] [2021-04-19 14:41:06] Average Time: 0.0
[INFO] [2021-04-19 14:41:06] Total Time: 1s
[INFO] [2021-04-19 14:41:06] Storing 2 MetaTraits
[INFO] [2021-04-19 14:41:06] Processing group of 2 in 1 groups of 1000
[INFO] [2021-04-19 14:41:06] Average Time: 0.0
[INFO] [2021-04-19 14:41:06] Total Time: 1s
[STOP] [2021-04-19 14:41:06] parse_diff_and_store
[START] [2021-04-19 14:41:06] resolve_keys
[INFO] [2021-04-19 14:41:11] Occurrences to nodes (through scientific_names)...
[INFO] [2021-04-19 14:41:11] traits to occurrences...
[INFO] [2021-04-19 14:41:11] traits to nodes (through occurrences)...
[INFO] [2021-04-19 14:41:11] Traits to sex term...
[INFO] [2021-04-19 14:41:11] Traits to lifestage term...
[INFO] [2021-04-19 14:41:11] MetaTraits to traits...
[INFO] [2021-04-19 14:41:11] MetaTraits (simple, measurement row refers to parent) to traits...
[INFO] [2021-04-19 14:41:11] Assocs to occurrences...
[INFO] [2021-04-19 14:41:11] Assocs to nodes...
[INFO] [2021-04-19 14:41:11] Assoc to sex term...
[INFO] [2021-04-19 14:41:11] Assoc to lifestage term...
[INFO] [2021-04-19 14:41:11] MetaAssoc to assocs...
[STOP] [2021-04-19 14:41:11] resolve_keys
[START] [2021-04-19 14:41:11] hold_for_later_1
[STOP] [2021-04-19 14:41:11] hold_for_later_1
[START] [2021-04-19 14:41:12] hold_for_later_2
[STOP] [2021-04-19 14:41:12] hold_for_later_2
[START] [2021-04-19 14:41:12] resolve_missing_parents
[STOP] [2021-04-19 14:41:12] resolve_missing_parents
[START] [2021-04-19 14:41:12] rebuild_nodes
[START] [2021-04-19 14:41:12] Flattener#flatten
[START] [2021-04-19 14:41:12] Flattener#study_resource
[START] [2021-04-19 14:41:12] Flattener#build_ancestry
[STOP] [2021-04-19 14:41:12] Flattener#build_ancestry
[INFO] [2021-04-19 14:41:12] 2 ancestry keys
[START] [2021-04-19 14:41:12] build_node_ancestors
[INFO] [2021-04-19 14:41:12] old ancestors deleted.
[STOP] [2021-04-19 14:41:12] build_node_ancestors
[WARN] [2021-04-19 14:41:12] Flattener: nothing to flatten! (Completely flat resource?)
[STOP] [2021-04-19 14:41:12] Flattener#flatten
[STOP] [2021-04-19 14:41:12] rebuild_nodes
[START] [2021-04-19 14:41:12] resolve_missing_media_owners
[STOP] [2021-04-19 14:41:12] resolve_missing_media_owners
[START] [2021-04-19 14:41:12] sanitize_media_verbatims
[STOP] [2021-04-19 14:41:12] sanitize_media_verbatims
[START] [2021-04-19 14:41:12] queue_downloads
[STOP] [2021-04-19 14:41:12] queue_downloads
[START] [2021-04-19 14:41:12] parse_names
[WARN] [2021-04-19 14:41:12] I see 2 names which still need to be parsed.
[STOP] [2021-04-19 14:41:13] parse_names
[START] [2021-04-19 14:41:13] denormalize_canonical_names_to_nodes
[STOP] [2021-04-19 14:41:13] denormalize_canonical_names_to_nodes
[START] [2021-04-19 14:41:13] match_nodes
[START] [2021-04-19 14:41:13] map_all_nodes_to_pages
[STOP] [2021-04-19 14:41:13] map_all_nodes_to_pages
[INFO] [2021-04-19 14:41:13] ZERO unmatched nodes (of 2)! Nicely done.
[START] [2021-04-19 14:41:13] update_nodes
[STOP] [2021-04-19 14:41:13] update_nodes
[STOP] [2021-04-19 14:41:13] match_nodes
[START] [2021-04-19 14:41:13] reindex_search
[STOP] [2021-04-19 14:41:13] reindex_search
[START] [2021-04-19 14:41:13] normalize_units
[STOP] [2021-04-19 14:41:13] normalize_units
[START] [2021-04-19 14:41:13] calculate_statistics
[2021-04-19 14:41:13] ZERO NODE ANCESTORS. Is this actually a completely flat resource?
[STOP] [2021-04-19 14:41:13] calculate_statistics
[START] [2021-04-19 14:41:13] complete_harvest_instance
[START] [2021-04-19 14:41:13] overall_tsv_creation
[INFO] [2021-04-19 14:41:13] Processing group of 2 in 1 batches of 10000
[INFO] [2021-04-19 14:41:49] 2 Traits (unfiltered)...
[INFO] [2021-04-19 14:42:22] 2 Traits (filtered)...
[INFO] [2021-04-19 14:42:22] 0 Associations (filtered)...
[INFO] [2021-04-19 14:42:22] 6 metadata added.
[INFO] [2021-04-19 14:42:22] 0 metadata added.
[INFO] [2021-04-19 14:42:49] Average Time: 72.2
[INFO] [2021-04-19 14:42:49] Total Time: 1m37s
[STOP] [2021-04-19 14:42:49] overall_tsv_creation
[INFO] [2021-04-19 14:42:49] Done. Check your files:
[INFO] [2021-04-19 14:42:49] (2 lines) /app/public/data/mackenzie_macken/publish_nodes.tsv
[INFO] [2021-04-19 14:42:49] (2 lines) /app/public/data/mackenzie_macken/publish_scientific_names.tsv
[INFO] [2021-04-19 14:42:49] (3 lines) /app/public/data/mackenzie_macken/publish_traits.tsv
[INFO] [2021-04-19 14:42:49] (7 lines) /app/public/data/mackenzie_macken/publish_metadata.tsv
[STOP] [2021-04-19 14:42:49] complete_harvest_instance
[START] [2021-04-19 14:42:49] completed
[STOP] [2021-04-19 14:42:49] completed
[STOP] [2021-04-19 14:42:49] logged process, took 104.03

Latest Process

19 Apr 14:41 EXITED currently processing 0