Stage:
completed
Fetched:
19 Apr 17:01
Validated:
19 Apr 17:01
Deltas Created
19 Apr 17:01
Units Normalized:
19 Apr 17:01
Ancestry Built:
19 Apr 17:01
Nodes Matched:
19 Apr 17:01
Names Parsed:
19 Apr 17:01
New Models Stored:
19 Apr 17:01
Indexed:
19 Apr 17:01
Completed:
19 Apr 17:03
Time to Harvest:
less than a minute
Harvesting Log
(650 lines)
# Logfile created on 2020-12-08 15:37:01 -0500 by logger.rb/v1.4.2
[START] [2020-12-08 15:37:01] logged process: 58bbc42b01abb4c1b2698de049792ffb4b63b979
[START] [2020-12-08 15:37:01] Creating resource from OpenData
[START] [2020-12-08 15:37:01] logged process: 58bbc42b01abb4c1b2698de049792ffb4b63b979
[START] [2020-12-08 15:37:01] Parse meta.xml file and create formats with fields
[STOP] [2020-12-08 15:37:01] Parse meta.xml file and create formats with fields
[STOP] [2020-12-08 15:37:01] Creating resource from OpenData
[INFO] [2020-12-08 15:38:29] ## HARVEST: type = -harvest
[START] [2020-12-08 15:38:31] logged process: 58bbc42b01abb4c1b2698de049792ffb4b63b979
[START] [2020-12-08 15:38:31] create_harvest_instance
[STOP] [2020-12-08 15:38:32] create_harvest_instance
[START] [2020-12-08 15:38:32] fetch_files
[STOP] [2020-12-08 15:38:32] fetch_files
[START] [2020-12-08 15:38:32] validate_each_file
[STOP] [2020-12-08 15:38:32] validate_each_file
[START] [2020-12-08 15:38:32] convert_to_csv
[CMD] [2020-12-08 15:38:32] /usr/bin/sort /app/public/converted_csv/delwiche_cooper__nodes_25321.csv > /app/public/converted_csv/delwiche_cooper__nodes_25321.csv_sorted
[CMD] [2020-12-08 15:38:32] /usr/bin/sort /app/public/converted_csv/delwiche_cooper__occurrences_25322.csv > /app/public/converted_csv/delwiche_cooper__occurrences_25322.csv_sorted
[CMD] [2020-12-08 15:38:32] /usr/bin/sort /app/public/converted_csv/delwiche_cooper__measurements_25323.csv > /app/public/converted_csv/delwiche_cooper__measurements_25323.csv_sorted
[STOP] [2020-12-08 15:38:32] convert_to_csv
[START] [2020-12-08 15:38:32] calculate_delta
[CMD] [2020-12-08 15:38:32] echo "0a" > /app/public/diff/delwiche_cooper__nodes_25321.diff
[CMD] [2020-12-08 15:38:32] tail -n +1 /app/public/converted_csv/delwiche_cooper__nodes_25321.csv >> /app/public/diff/delwiche_cooper__nodes_25321.diff
[CMD] [2020-12-08 15:38:32] echo "." >> /app/public/diff/delwiche_cooper__nodes_25321.diff
[CMD] [2020-12-08 15:38:32] echo "0a" > /app/public/diff/delwiche_cooper__occurrences_25322.diff
[CMD] [2020-12-08 15:38:32] tail -n +1 /app/public/converted_csv/delwiche_cooper__occurrences_25322.csv >> /app/public/diff/delwiche_cooper__occurrences_25322.diff
[CMD] [2020-12-08 15:38:32] echo "." >> /app/public/diff/delwiche_cooper__occurrences_25322.diff
[CMD] [2020-12-08 15:38:32] echo "0a" > /app/public/diff/delwiche_cooper__measurements_25323.diff
[CMD] [2020-12-08 15:38:32] tail -n +1 /app/public/converted_csv/delwiche_cooper__measurements_25323.csv >> /app/public/diff/delwiche_cooper__measurements_25323.diff
[CMD] [2020-12-08 15:38:32] echo "." >> /app/public/diff/delwiche_cooper__measurements_25323.diff
[STOP] [2020-12-08 15:38:32] calculate_delta
[START] [2020-12-08 15:38:32] parse_diff_and_store
[INFO] [2020-12-08 15:38:32] Loading nodes diff file into memory (true lines)...
[INFO] [2020-12-08 15:38:32] Loading occurrences diff file into memory (true lines)...
[INFO] [2020-12-08 15:38:32] Loading measurements diff file into memory (true lines)...
[INFO] [2020-12-08 15:38:33] Storing 6 ScientificNames
[INFO] [2020-12-08 15:38:33] Processing group of 6 in 1 groups of 1000
[INFO] [2020-12-08 15:38:33] Average Time: 0.01
[INFO] [2020-12-08 15:38:33] Total Time: 1s
[INFO] [2020-12-08 15:38:33] Storing 6 Nodes
[INFO] [2020-12-08 15:38:33] Processing group of 6 in 1 groups of 1000
[INFO] [2020-12-08 15:38:33] Average Time: 0.01
[INFO] [2020-12-08 15:38:33] Total Time: 1s
[INFO] [2020-12-08 15:38:33] Storing 6 Occurrences
[INFO] [2020-12-08 15:38:33] Processing group of 6 in 1 groups of 1000
[INFO] [2020-12-08 15:38:33] Average Time: 0.0
[INFO] [2020-12-08 15:38:33] Total Time: 1s
[INFO] [2020-12-08 15:38:33] Storing 39 Traits
[INFO] [2020-12-08 15:38:33] Processing group of 39 in 1 groups of 1000
[INFO] [2020-12-08 15:38:33] Average Time: 0.01
[INFO] [2020-12-08 15:38:33] Total Time: 1s
[INFO] [2020-12-08 15:38:33] Storing 6 MetaTraits
[INFO] [2020-12-08 15:38:33] Processing group of 6 in 1 groups of 1000
[INFO] [2020-12-08 15:38:33] Average Time: 0.0
[INFO] [2020-12-08 15:38:33] Total Time: 1s
[STOP] [2020-12-08 15:38:33] parse_diff_and_store
[START] [2020-12-08 15:38:33] resolve_keys
[INFO] [2020-12-08 15:38:39] Occurrences to nodes (through scientific_names)...
[INFO] [2020-12-08 15:38:39] traits to occurrences...
[INFO] [2020-12-08 15:38:39] traits to nodes (through occurrences)...
[INFO] [2020-12-08 15:38:39] Traits to sex term...
[INFO] [2020-12-08 15:38:39] Traits to lifestage term...
[INFO] [2020-12-08 15:38:39] MetaTraits to traits...
[INFO] [2020-12-08 15:38:39] MetaTraits (simple, measurement row refers to parent) to traits...
[INFO] [2020-12-08 15:38:39] Assocs to occurrences...
[INFO] [2020-12-08 15:38:39] Assocs to nodes...
[INFO] [2020-12-08 15:38:39] Assoc to sex term...
[INFO] [2020-12-08 15:38:39] Assoc to lifestage term...
[INFO] [2020-12-08 15:38:39] MetaAssoc to assocs...
[STOP] [2020-12-08 15:38:39] resolve_keys
[START] [2020-12-08 15:38:39] hold_for_later_1
[STOP] [2020-12-08 15:38:39] hold_for_later_1
[START] [2020-12-08 15:38:39] hold_for_later_2
[STOP] [2020-12-08 15:38:39] hold_for_later_2
[START] [2020-12-08 15:38:39] resolve_missing_parents
[STOP] [2020-12-08 15:38:39] resolve_missing_parents
[START] [2020-12-08 15:38:39] rebuild_nodes
[START] [2020-12-08 15:38:39] Flattener#flatten
[START] [2020-12-08 15:38:39] Flattener#study_resource
[START] [2020-12-08 15:38:39] Flattener#build_ancestry
[STOP] [2020-12-08 15:38:39] Flattener#build_ancestry
[INFO] [2020-12-08 15:38:39] 6 ancestry keys
[START] [2020-12-08 15:38:39] build_node_ancestors
[INFO] [2020-12-08 15:38:39] old ancestors deleted.
[STOP] [2020-12-08 15:38:39] build_node_ancestors
[WARN] [2020-12-08 15:38:39] Flattener: nothing to flatten! (Completely flat resource?)
[STOP] [2020-12-08 15:38:39] Flattener#flatten
[STOP] [2020-12-08 15:38:39] rebuild_nodes
[START] [2020-12-08 15:38:39] resolve_missing_media_owners
[STOP] [2020-12-08 15:38:39] resolve_missing_media_owners
[START] [2020-12-08 15:38:39] sanitize_media_verbatims
[STOP] [2020-12-08 15:38:39] sanitize_media_verbatims
[START] [2020-12-08 15:38:39] queue_downloads
[STOP] [2020-12-08 15:38:39] queue_downloads
[START] [2020-12-08 15:38:39] parse_names
[WARN] [2020-12-08 15:38:39] I see 6 names which still need to be parsed.
[STOP] [2020-12-08 15:38:40] parse_names
[START] [2020-12-08 15:38:40] denormalize_canonical_names_to_nodes
[STOP] [2020-12-08 15:38:40] denormalize_canonical_names_to_nodes
[START] [2020-12-08 15:38:40] match_nodes
[START] [2020-12-08 15:38:40] map_all_nodes_to_pages
[STOP] [2020-12-08 15:38:40] map_all_nodes_to_pages
[INFO] [2020-12-08 15:38:40] ZERO unmatched nodes (of 6)! Nicely done.
[START] [2020-12-08 15:38:40] update_nodes
[STOP] [2020-12-08 15:38:40] update_nodes
[STOP] [2020-12-08 15:38:40] match_nodes
[START] [2020-12-08 15:38:40] reindex_search
[STOP] [2020-12-08 15:38:40] reindex_search
[START] [2020-12-08 15:38:40] normalize_units
[STOP] [2020-12-08 15:38:40] normalize_units
[START] [2020-12-08 15:38:40] calculate_statistics
[2020-12-08 15:38:40] ZERO NODE ANCESTORS. Is this actually a completely flat resource?
[STOP] [2020-12-08 15:38:40] calculate_statistics
[START] [2020-12-08 15:38:40] complete_harvest_instance
[START] [2020-12-08 15:38:40] overall_tsv_creation
[INFO] [2020-12-08 15:38:40] Processing group of 6 in 1 batches of 10000
[INFO] [2020-12-08 15:39:19] 6 Traits (unfiltered)...
[INFO] [2020-12-08 15:39:57] 6 Traits (filtered)...
[INFO] [2020-12-08 15:39:57] 0 Associations (filtered)...
[INFO] [2020-12-08 15:39:57] 39 metadata added.
[INFO] [2020-12-08 15:39:57] 0 metadata added.
[INFO] [2020-12-08 15:39:57] Average Time: 53.72
[INFO] [2020-12-08 15:39:57] Total Time: 1m17s
[STOP] [2020-12-08 15:39:57] overall_tsv_creation
[INFO] [2020-12-08 15:39:57] Done. Check your files:
[INFO] [2020-12-08 15:39:57] (6 lines) /app/public/data/delwiche_cooper_/publish_nodes.tsv
[INFO] [2020-12-08 15:39:57] (6 lines) /app/public/data/delwiche_cooper_/publish_scientific_names.tsv
[INFO] [2020-12-08 15:39:57] (7 lines) /app/public/data/delwiche_cooper_/publish_traits.tsv
[INFO] [2020-12-08 15:39:57] (40 lines) /app/public/data/delwiche_cooper_/publish_metadata.tsv
[STOP] [2020-12-08 15:39:57] complete_harvest_instance
[START] [2020-12-08 15:39:57] completed
[STOP] [2020-12-08 15:39:57] completed
[STOP] [2020-12-08 15:39:57] logged process, took 86.33
[INFO] [2020-12-08 15:46:12] ## HARVEST: type = re_download_opendata_-harvest
[INFO] [2020-12-08 15:46:14] ## remove_type: ScientificName
[INFO] [2020-12-08 15:46:14] ++ Calling delete_all on 6 instances...
[INFO] [2020-12-08 15:46:14] [15:46:14.473] Removed 6 Scientificnames
[INFO] [2020-12-08 15:46:14] ## remove_type: Vernacular
[INFO] [2020-12-08 15:46:14] ++ Calling delete_all on 0 instances...
[INFO] [2020-12-08 15:46:14] [15:46:14.476] Removed 0 Vernaculars
[INFO] [2020-12-08 15:46:14] ## remove_type: Article
[INFO] [2020-12-08 15:46:14] ++ Calling delete_all on 0 instances...
[INFO] [2020-12-08 15:46:14] [15:46:14.479] Removed 0 Articles
[INFO] [2020-12-08 15:46:14] ## remove_type: Medium
[INFO] [2020-12-08 15:46:14] ++ Calling delete_all on 0 instances...
[INFO] [2020-12-08 15:46:14] [15:46:14.482] Removed 0 Media
[INFO] [2020-12-08 15:46:14] ## remove_type: Trait
[INFO] [2020-12-08 15:46:14] ++ Calling delete_all on 39 instances...
[INFO] [2020-12-08 15:46:14] [15:46:14.486] Removed 39 Traits
[INFO] [2020-12-08 15:46:14] ## remove_type: MetaTrait
[INFO] [2020-12-08 15:46:14] ++ Calling delete_all on 6 instances...
[INFO] [2020-12-08 15:46:14] [15:46:14.489] Removed 6 Metatraits
[INFO] [2020-12-08 15:46:14] ## remove_type: OccurrenceMetadatum
[INFO] [2020-12-08 15:46:14] ++ Calling delete_all on 0 instances...
[INFO] [2020-12-08 15:46:14] [15:46:14.492] Removed 0 Occurrencemetadata
[INFO] [2020-12-08 15:46:14] ## remove_type: Assoc
[INFO] [2020-12-08 15:46:14] ++ Calling delete_all on 0 instances...
[INFO] [2020-12-08 15:46:14] [15:46:14.495] Removed 0 Assocs
[INFO] [2020-12-08 15:46:14] ## remove_type: MetaAssoc
[INFO] [2020-12-08 15:46:14] ++ Calling delete_all on 0 instances...
[INFO] [2020-12-08 15:46:14] [15:46:14.498] Removed 0 Metaassocs
[INFO] [2020-12-08 15:46:14] ## remove_type: Identifier
[INFO] [2020-12-08 15:46:14] ++ Calling delete_all on 0 instances...
[INFO] [2020-12-08 15:46:14] [15:46:14.500] Removed 0 Identifiers
[INFO] [2020-12-08 15:46:14] ## remove_type: Reference
[INFO] [2020-12-08 15:46:14] ++ Calling delete_all on 0 instances...
[INFO] [2020-12-08 15:46:14] [15:46:14.502] Removed 0 References
[INFO] [2020-12-08 15:46:14] Starting batch with ID 86121288...
[INFO] [2020-12-08 15:46:14] Starting batch with ID 86121288...
[INFO] [2020-12-08 15:46:14] Starting batch with ID 86121288...
[INFO] [2020-12-08 15:46:14] Starting batch with ID 86121288...
[INFO] [2020-12-08 15:46:14] Starting batch with ID 86121288...
[INFO] [2020-12-08 15:46:14] Starting batch with ID 86121288...
[INFO] [2020-12-08 15:46:14] Starting batch with ID 86121288...
[INFO] [2020-12-08 15:46:14] Starting batch with ID 86121288...
[INFO] [2020-12-08 15:46:14] Starting batch with ID 86121288...
[INFO] [2020-12-08 15:46:14] Starting batch with ID 86121288...
[INFO] [2020-12-08 15:46:14] Starting batch with ID 86121288...
[INFO] [2020-12-08 15:46:14] Starting batch with ID 86121288...
[INFO] [2020-12-08 15:46:14] Starting batch with ID 86121288...
[INFO] [2020-12-08 15:46:14] Starting batch with ID 86121288...
[INFO] [2020-12-08 15:46:14] Starting batch with ID 86121288...
[INFO] [2020-12-08 15:46:14] Starting batch with ID 86121288...
[INFO] [2020-12-08 15:46:14] Starting batch with ID 86121288...
[INFO] [2020-12-08 15:46:14] Starting batch with ID 86121288...
[INFO] [2020-12-08 15:46:14] Starting batch with ID 86121288...
[INFO] [2020-12-08 15:46:14] Starting batch with ID 86121288...
[INFO] [2020-12-08 15:46:14] Starting batch with ID 86121288...
[INFO] [2020-12-08 15:46:14] ## remove_type: Node
[INFO] [2020-12-08 15:46:14] ++ Calling delete_all on 6 instances...
[INFO] [2020-12-08 15:46:14] [15:46:14.967] Removed 6 Nodes
[START] [2020-12-08 15:46:15] logged process: 58bbc42b01abb4c1b2698de049792ffb4b63b979
[START] [2020-12-08 15:46:15] Creating resource from OpenData
[START] [2020-12-08 15:46:15] logged process: 58bbc42b01abb4c1b2698de049792ffb4b63b979
[START] [2020-12-08 15:46:15] Parse meta.xml file and create formats with fields
[STOP] [2020-12-08 15:46:15] Parse meta.xml file and create formats with fields
[STOP] [2020-12-08 15:46:15] Creating resource from OpenData
[START] [2020-12-08 15:46:15] logged process: 58bbc42b01abb4c1b2698de049792ffb4b63b979
[START] [2020-12-08 15:46:15] create_harvest_instance
[STOP] [2020-12-08 15:46:16] create_harvest_instance
[START] [2020-12-08 15:46:16] fetch_files
[STOP] [2020-12-08 15:46:16] fetch_files
[START] [2020-12-08 15:46:16] validate_each_file
[STOP] [2020-12-08 15:46:16] validate_each_file
[START] [2020-12-08 15:46:16] convert_to_csv
[CMD] [2020-12-08 15:46:16] /usr/bin/sort /app/public/converted_csv/delwiche_cooper__nodes_25327.csv > /app/public/converted_csv/delwiche_cooper__nodes_25327.csv_sorted
[CMD] [2020-12-08 15:46:16] /usr/bin/sort /app/public/converted_csv/delwiche_cooper__occurrences_25328.csv > /app/public/converted_csv/delwiche_cooper__occurrences_25328.csv_sorted
[CMD] [2020-12-08 15:46:16] /usr/bin/sort /app/public/converted_csv/delwiche_cooper__measurements_25329.csv > /app/public/converted_csv/delwiche_cooper__measurements_25329.csv_sorted
[STOP] [2020-12-08 15:46:16] convert_to_csv
[START] [2020-12-08 15:46:16] calculate_delta
[CMD] [2020-12-08 15:46:16] echo "0a" > /app/public/diff/delwiche_cooper__nodes_25327.diff
[CMD] [2020-12-08 15:46:16] tail -n +1 /app/public/converted_csv/delwiche_cooper__nodes_25327.csv >> /app/public/diff/delwiche_cooper__nodes_25327.diff
[CMD] [2020-12-08 15:46:16] echo "." >> /app/public/diff/delwiche_cooper__nodes_25327.diff
[CMD] [2020-12-08 15:46:16] echo "0a" > /app/public/diff/delwiche_cooper__occurrences_25328.diff
[CMD] [2020-12-08 15:46:16] tail -n +1 /app/public/converted_csv/delwiche_cooper__occurrences_25328.csv >> /app/public/diff/delwiche_cooper__occurrences_25328.diff
[CMD] [2020-12-08 15:46:16] echo "." >> /app/public/diff/delwiche_cooper__occurrences_25328.diff
[CMD] [2020-12-08 15:46:16] echo "0a" > /app/public/diff/delwiche_cooper__measurements_25329.diff
[CMD] [2020-12-08 15:46:16] tail -n +1 /app/public/converted_csv/delwiche_cooper__measurements_25329.csv >> /app/public/diff/delwiche_cooper__measurements_25329.diff
[CMD] [2020-12-08 15:46:16] echo "." >> /app/public/diff/delwiche_cooper__measurements_25329.diff
[STOP] [2020-12-08 15:46:16] calculate_delta
[START] [2020-12-08 15:46:16] parse_diff_and_store
[INFO] [2020-12-08 15:46:16] Loading nodes diff file into memory (true lines)...
[INFO] [2020-12-08 15:46:16] Loading occurrences diff file into memory (true lines)...
[INFO] [2020-12-08 15:46:16] Loading measurements diff file into memory (true lines)...
[INFO] [2020-12-08 15:46:17] Storing 6 ScientificNames
[INFO] [2020-12-08 15:46:17] Processing group of 6 in 1 groups of 1000
[INFO] [2020-12-08 15:46:17] Average Time: 0.0
[INFO] [2020-12-08 15:46:17] Total Time: 1s
[INFO] [2020-12-08 15:46:17] Storing 6 Nodes
[INFO] [2020-12-08 15:46:17] Processing group of 6 in 1 groups of 1000
[INFO] [2020-12-08 15:46:17] Average Time: 0.0
[INFO] [2020-12-08 15:46:17] Total Time: 1s
[INFO] [2020-12-08 15:46:17] Storing 6 Occurrences
[INFO] [2020-12-08 15:46:17] Processing group of 6 in 1 groups of 1000
[INFO] [2020-12-08 15:46:17] Average Time: 0.0
[INFO] [2020-12-08 15:46:17] Total Time: 1s
[INFO] [2020-12-08 15:46:17] Storing 39 Traits
[INFO] [2020-12-08 15:46:17] Processing group of 39 in 1 groups of 1000
[INFO] [2020-12-08 15:46:17] Average Time: 0.01
[INFO] [2020-12-08 15:46:17] Total Time: 1s
[INFO] [2020-12-08 15:46:17] Storing 6 MetaTraits
[INFO] [2020-12-08 15:46:17] Processing group of 6 in 1 groups of 1000
[INFO] [2020-12-08 15:46:17] Average Time: 0.0
[INFO] [2020-12-08 15:46:17] Total Time: 1s
[STOP] [2020-12-08 15:46:17] parse_diff_and_store
[START] [2020-12-08 15:46:17] resolve_keys
[INFO] [2020-12-08 15:46:23] Occurrences to nodes (through scientific_names)...
[INFO] [2020-12-08 15:46:23] traits to occurrences...
[INFO] [2020-12-08 15:46:23] traits to nodes (through occurrences)...
[INFO] [2020-12-08 15:46:23] Traits to sex term...
[INFO] [2020-12-08 15:46:23] Traits to lifestage term...
[INFO] [2020-12-08 15:46:23] MetaTraits to traits...
[INFO] [2020-12-08 15:46:23] MetaTraits (simple, measurement row refers to parent) to traits...
[INFO] [2020-12-08 15:46:23] Assocs to occurrences...
[INFO] [2020-12-08 15:46:23] Assocs to nodes...
[INFO] [2020-12-08 15:46:23] Assoc to sex term...
[INFO] [2020-12-08 15:46:23] Assoc to lifestage term...
[INFO] [2020-12-08 15:46:23] MetaAssoc to assocs...
[STOP] [2020-12-08 15:46:23] resolve_keys
[START] [2020-12-08 15:46:23] hold_for_later_1
[STOP] [2020-12-08 15:46:23] hold_for_later_1
[START] [2020-12-08 15:46:23] hold_for_later_2
[STOP] [2020-12-08 15:46:23] hold_for_later_2
[START] [2020-12-08 15:46:23] resolve_missing_parents
[STOP] [2020-12-08 15:46:23] resolve_missing_parents
[START] [2020-12-08 15:46:23] rebuild_nodes
[START] [2020-12-08 15:46:23] Flattener#flatten
[START] [2020-12-08 15:46:23] Flattener#study_resource
[START] [2020-12-08 15:46:23] Flattener#build_ancestry
[STOP] [2020-12-08 15:46:23] Flattener#build_ancestry
[INFO] [2020-12-08 15:46:23] 6 ancestry keys
[START] [2020-12-08 15:46:23] build_node_ancestors
[INFO] [2020-12-08 15:46:23] old ancestors deleted.
[STOP] [2020-12-08 15:46:23] build_node_ancestors
[WARN] [2020-12-08 15:46:23] Flattener: nothing to flatten! (Completely flat resource?)
[STOP] [2020-12-08 15:46:23] Flattener#flatten
[STOP] [2020-12-08 15:46:23] rebuild_nodes
[START] [2020-12-08 15:46:23] resolve_missing_media_owners
[STOP] [2020-12-08 15:46:23] resolve_missing_media_owners
[START] [2020-12-08 15:46:23] sanitize_media_verbatims
[STOP] [2020-12-08 15:46:23] sanitize_media_verbatims
[START] [2020-12-08 15:46:23] queue_downloads
[STOP] [2020-12-08 15:46:23] queue_downloads
[START] [2020-12-08 15:46:23] parse_names
[WARN] [2020-12-08 15:46:23] I see 6 names which still need to be parsed.
[STOP] [2020-12-08 15:46:24] parse_names
[START] [2020-12-08 15:46:24] denormalize_canonical_names_to_nodes
[STOP] [2020-12-08 15:46:24] denormalize_canonical_names_to_nodes
[START] [2020-12-08 15:46:24] match_nodes
[START] [2020-12-08 15:46:24] map_all_nodes_to_pages
[STOP] [2020-12-08 15:46:24] map_all_nodes_to_pages
[INFO] [2020-12-08 15:46:24] ZERO unmatched nodes (of 6)! Nicely done.
[START] [2020-12-08 15:46:24] update_nodes
[STOP] [2020-12-08 15:46:24] update_nodes
[STOP] [2020-12-08 15:46:24] match_nodes
[START] [2020-12-08 15:46:24] reindex_search
[STOP] [2020-12-08 15:46:24] reindex_search
[START] [2020-12-08 15:46:24] normalize_units
[STOP] [2020-12-08 15:46:24] normalize_units
[START] [2020-12-08 15:46:24] calculate_statistics
[2020-12-08 15:46:24] ZERO NODE ANCESTORS. Is this actually a completely flat resource?
[STOP] [2020-12-08 15:46:24] calculate_statistics
[START] [2020-12-08 15:46:24] complete_harvest_instance
[START] [2020-12-08 15:46:24] overall_tsv_creation
[INFO] [2020-12-08 15:46:24] Processing group of 6 in 1 batches of 10000
[INFO] [2020-12-08 15:47:03] 6 Traits (unfiltered)...
[INFO] [2020-12-08 15:47:41] 6 Traits (filtered)...
[INFO] [2020-12-08 15:47:41] 0 Associations (filtered)...
[INFO] [2020-12-08 15:47:41] 39 metadata added.
[INFO] [2020-12-08 15:47:41] 0 metadata added.
[INFO] [2020-12-08 15:47:41] Average Time: 53.85
[INFO] [2020-12-08 15:47:41] Total Time: 1m17s
[STOP] [2020-12-08 15:47:41] overall_tsv_creation
[INFO] [2020-12-08 15:47:41] Done. Check your files:
[INFO] [2020-12-08 15:47:41] (6 lines) /app/public/data/delwiche_cooper_/publish_nodes.tsv
[INFO] [2020-12-08 15:47:41] (6 lines) /app/public/data/delwiche_cooper_/publish_scientific_names.tsv
[INFO] [2020-12-08 15:47:41] (7 lines) /app/public/data/delwiche_cooper_/publish_traits.tsv
[INFO] [2020-12-08 15:47:41] (40 lines) /app/public/data/delwiche_cooper_/publish_metadata.tsv
[STOP] [2020-12-08 15:47:41] complete_harvest_instance
[START] [2020-12-08 15:47:41] completed
[STOP] [2020-12-08 15:47:41] completed
[STOP] [2020-12-08 15:47:41] logged process, took 86.08
[INFO] [2021-04-19 16:47:57] ## HARVEST: type = re_download_opendata_-harvest
[INFO] [2021-04-19 17:01:32] ## remove_type: ScientificName
[INFO] [2021-04-19 17:01:32] ++ Calling delete_all on 6 instances...
[INFO] [2021-04-19 17:01:32] [17:01:32.857] Removed 6 Scientificnames
[INFO] [2021-04-19 17:01:32] ## remove_type: Vernacular
[INFO] [2021-04-19 17:01:32] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-19 17:01:32] [17:01:32.858] Removed 0 Vernaculars
[INFO] [2021-04-19 17:01:32] ## remove_type: Article
[INFO] [2021-04-19 17:01:32] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-19 17:01:32] [17:01:32.859] Removed 0 Articles
[INFO] [2021-04-19 17:01:32] ## remove_type: Medium
[INFO] [2021-04-19 17:01:32] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-19 17:01:32] [17:01:32.861] Removed 0 Media
[INFO] [2021-04-19 17:01:32] ## remove_type: Trait
[INFO] [2021-04-19 17:01:32] ++ Calling delete_all on 39 instances...
[INFO] [2021-04-19 17:01:32] [17:01:32.864] Removed 39 Traits
[INFO] [2021-04-19 17:01:32] ## remove_type: MetaTrait
[INFO] [2021-04-19 17:01:32] ++ Calling delete_all on 6 instances...
[INFO] [2021-04-19 17:01:32] [17:01:32.867] Removed 6 Metatraits
[INFO] [2021-04-19 17:01:32] ## remove_type: OccurrenceMetadatum
[INFO] [2021-04-19 17:01:32] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-19 17:01:32] [17:01:32.868] Removed 0 Occurrencemetadata
[INFO] [2021-04-19 17:01:32] ## remove_type: Assoc
[INFO] [2021-04-19 17:01:32] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-19 17:01:32] [17:01:32.869] Removed 0 Assocs
[INFO] [2021-04-19 17:01:32] ## remove_type: MetaAssoc
[INFO] [2021-04-19 17:01:32] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-19 17:01:32] [17:01:32.871] Removed 0 Metaassocs
[INFO] [2021-04-19 17:01:32] ## remove_type: Identifier
[INFO] [2021-04-19 17:01:32] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-19 17:01:32] [17:01:32.872] Removed 0 Identifiers
[INFO] [2021-04-19 17:01:32] ## remove_type: Reference
[INFO] [2021-04-19 17:01:32] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-19 17:01:32] [17:01:32.873] Removed 0 References
[INFO] [2021-04-19 17:01:32] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:32] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:32] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:32] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:32] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:32] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:32] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:32] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:32] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:32] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:32] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:32] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:32] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:32] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:32] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:32] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121292...
[INFO] [2021-04-19 17:01:33] Starting batch with ID 86121293...
[INFO] [2021-04-19 17:01:33] ## remove_type: Node
[INFO] [2021-04-19 17:01:33] ++ Calling delete_all on 6 instances...
[INFO] [2021-04-19 17:01:33] [17:01:33.854] Removed 6 Nodes
[START] [2021-04-19 17:01:33] logged process: 5ecc716a6a5541910d0c854f5a0c8d1651b82ad0 Improved MetaXml.ignore and added publisher to media (ignored)
[START] [2021-04-19 17:01:33] Creating resource from OpenData
[START] [2021-04-19 17:01:34] logged process: 5ecc716a6a5541910d0c854f5a0c8d1651b82ad0 Improved MetaXml.ignore and added publisher to media (ignored)
[START] [2021-04-19 17:01:34] Parse meta.xml file and create formats with fields
[STOP] [2021-04-19 17:01:34] Parse meta.xml file and create formats with fields
[STOP] [2021-04-19 17:01:34] Creating resource from OpenData
[START] [2021-04-19 17:01:34] logged process: 5ecc716a6a5541910d0c854f5a0c8d1651b82ad0 Improved MetaXml.ignore and added publisher to media (ignored)
[START] [2021-04-19 17:01:34] create_harvest_instance
[INFO] [2021-04-19 17:01:34] Created harvest instance #3813
[STOP] [2021-04-19 17:01:34] create_harvest_instance
[START] [2021-04-19 17:01:34] fetch_files
[STOP] [2021-04-19 17:01:34] fetch_files
[START] [2021-04-19 17:01:34] validate_each_file
[INFO] [2021-04-19 17:01:34] Looping over 3 formats...
[INFO] [2021-04-19 17:01:34] ...nodes (/app/public/data/delwiche_cooper_/taxa.txt)
[INFO] [2021-04-19 17:01:34] Valid: /app/public/converted_csv/delwiche_cooper__nodes_3813.csv (6 lines)
[INFO] [2021-04-19 17:01:34] ...occurrences (/app/public/data/delwiche_cooper_/occurrences.txt)
[INFO] [2021-04-19 17:01:34] Valid: /app/public/converted_csv/delwiche_cooper__occurrences_3813.csv (6 lines)
[INFO] [2021-04-19 17:01:34] ...measurements (/app/public/data/delwiche_cooper_/measurementsorfacts.txt)
[INFO] [2021-04-19 17:01:34] Valid: /app/public/converted_csv/delwiche_cooper__measurements_3813.csv (39 lines)
[STOP] [2021-04-19 17:01:34] validate_each_file
[START] [2021-04-19 17:01:34] convert_to_csv
[INFO] [2021-04-19 17:01:34] Looping over 3 formats...
[INFO] [2021-04-19 17:01:34] ...nodes (/app/public/data/delwiche_cooper_/taxa.txt)
[CMD] [2021-04-19 17:01:34] /usr/bin/sort /app/public/converted_csv/delwiche_cooper__nodes_3813.csv > /app/public/converted_csv/delwiche_cooper__nodes_3813.csv_sorted
[INFO] [2021-04-19 17:01:34] Converted: /app/public/converted_csv/delwiche_cooper__nodes_3813.csv (6 lines)
[INFO] [2021-04-19 17:01:34] ...occurrences (/app/public/data/delwiche_cooper_/occurrences.txt)
[CMD] [2021-04-19 17:01:34] /usr/bin/sort /app/public/converted_csv/delwiche_cooper__occurrences_3813.csv > /app/public/converted_csv/delwiche_cooper__occurrences_3813.csv_sorted
[INFO] [2021-04-19 17:01:34] Converted: /app/public/converted_csv/delwiche_cooper__occurrences_3813.csv (6 lines)
[INFO] [2021-04-19 17:01:34] ...measurements (/app/public/data/delwiche_cooper_/measurementsorfacts.txt)
[CMD] [2021-04-19 17:01:34] /usr/bin/sort /app/public/converted_csv/delwiche_cooper__measurements_3813.csv > /app/public/converted_csv/delwiche_cooper__measurements_3813.csv_sorted
[INFO] [2021-04-19 17:01:34] Converted: /app/public/converted_csv/delwiche_cooper__measurements_3813.csv (39 lines)
[STOP] [2021-04-19 17:01:34] convert_to_csv
[START] [2021-04-19 17:01:34] calculate_delta
[INFO] [2021-04-19 17:01:34] Looping over 3 formats...
[INFO] [2021-04-19 17:01:34] ...nodes (/app/public/data/delwiche_cooper_/taxa.txt)
[CMD] [2021-04-19 17:01:34] echo "0a" > /app/public/diff/delwiche_cooper__nodes_3813.diff
[CMD] [2021-04-19 17:01:34] tail -n +1 /app/public/converted_csv/delwiche_cooper__nodes_3813.csv >> /app/public/diff/delwiche_cooper__nodes_3813.diff
[CMD] [2021-04-19 17:01:34] echo "." >> /app/public/diff/delwiche_cooper__nodes_3813.diff
[INFO] [2021-04-19 17:01:34] Created diff: /app/public/diff/delwiche_cooper__nodes_3813.diff (8 lines)
[INFO] [2021-04-19 17:01:34] ...occurrences (/app/public/data/delwiche_cooper_/occurrences.txt)
[CMD] [2021-04-19 17:01:34] echo "0a" > /app/public/diff/delwiche_cooper__occurrences_3813.diff
[CMD] [2021-04-19 17:01:34] tail -n +1 /app/public/converted_csv/delwiche_cooper__occurrences_3813.csv >> /app/public/diff/delwiche_cooper__occurrences_3813.diff
[CMD] [2021-04-19 17:01:34] echo "." >> /app/public/diff/delwiche_cooper__occurrences_3813.diff
[INFO] [2021-04-19 17:01:34] Created diff: /app/public/diff/delwiche_cooper__occurrences_3813.diff (8 lines)
[INFO] [2021-04-19 17:01:34] ...measurements (/app/public/data/delwiche_cooper_/measurementsorfacts.txt)
[CMD] [2021-04-19 17:01:34] echo "0a" > /app/public/diff/delwiche_cooper__measurements_3813.diff
[CMD] [2021-04-19 17:01:34] tail -n +1 /app/public/converted_csv/delwiche_cooper__measurements_3813.csv >> /app/public/diff/delwiche_cooper__measurements_3813.diff
[CMD] [2021-04-19 17:01:34] echo "." >> /app/public/diff/delwiche_cooper__measurements_3813.diff
[INFO] [2021-04-19 17:01:34] Created diff: /app/public/diff/delwiche_cooper__measurements_3813.diff (41 lines)
[STOP] [2021-04-19 17:01:34] calculate_delta
[START] [2021-04-19 17:01:34] parse_diff_and_store
[INFO] [2021-04-19 17:01:34] Handling diff: /app/public/diff/delwiche_cooper__nodes_3813.diff (8 lines)
[INFO] [2021-04-19 17:01:34] Loading nodes diff file into memory (8 /app/public/diff/delwiche_cooper__nodes_3813.diff lines)...
[INFO] [2021-04-19 17:01:34] Handling diff: /app/public/diff/delwiche_cooper__occurrences_3813.diff (8 lines)
[INFO] [2021-04-19 17:01:34] Loading occurrences diff file into memory (8 /app/public/diff/delwiche_cooper__occurrences_3813.diff lines)...
[INFO] [2021-04-19 17:01:34] Handling diff: /app/public/diff/delwiche_cooper__measurements_3813.diff (41 lines)
[INFO] [2021-04-19 17:01:34] Loading measurements diff file into memory (41 /app/public/diff/delwiche_cooper__measurements_3813.diff lines)...
[INFO] [2021-04-19 17:01:34] Storing 6 ScientificNames
[INFO] [2021-04-19 17:01:34] Processing group of 6 in 1 groups of 1000
[INFO] [2021-04-19 17:01:34] Average Time: 0.0
[INFO] [2021-04-19 17:01:34] Total Time: 1s
[INFO] [2021-04-19 17:01:34] Storing 6 Nodes
[INFO] [2021-04-19 17:01:34] Processing group of 6 in 1 groups of 1000
[INFO] [2021-04-19 17:01:34] Average Time: 0.0
[INFO] [2021-04-19 17:01:34] Total Time: 1s
[INFO] [2021-04-19 17:01:34] Storing 6 Occurrences
[INFO] [2021-04-19 17:01:34] Processing group of 6 in 1 groups of 1000
[INFO] [2021-04-19 17:01:34] Average Time: 0.0
[INFO] [2021-04-19 17:01:34] Total Time: 1s
[INFO] [2021-04-19 17:01:34] Storing 39 Traits
[INFO] [2021-04-19 17:01:34] Processing group of 39 in 1 groups of 1000
[INFO] [2021-04-19 17:01:34] Average Time: 0.01
[INFO] [2021-04-19 17:01:34] Total Time: 1s
[INFO] [2021-04-19 17:01:34] Storing 6 MetaTraits
[INFO] [2021-04-19 17:01:34] Processing group of 6 in 1 groups of 1000
[INFO] [2021-04-19 17:01:34] Average Time: 0.0
[INFO] [2021-04-19 17:01:34] Total Time: 1s
[STOP] [2021-04-19 17:01:34] parse_diff_and_store
[START] [2021-04-19 17:01:34] resolve_keys
[INFO] [2021-04-19 17:01:41] Occurrences to nodes (through scientific_names)...
[INFO] [2021-04-19 17:01:41] traits to occurrences...
[INFO] [2021-04-19 17:01:41] traits to nodes (through occurrences)...
[INFO] [2021-04-19 17:01:41] Traits to sex term...
[INFO] [2021-04-19 17:01:41] Traits to lifestage term...
[INFO] [2021-04-19 17:01:41] MetaTraits to traits...
[INFO] [2021-04-19 17:01:41] MetaTraits (simple, measurement row refers to parent) to traits...
[INFO] [2021-04-19 17:01:41] Assocs to occurrences...
[INFO] [2021-04-19 17:01:41] Assocs to nodes...
[INFO] [2021-04-19 17:01:41] Assoc to sex term...
[INFO] [2021-04-19 17:01:41] Assoc to lifestage term...
[INFO] [2021-04-19 17:01:41] MetaAssoc to assocs...
[STOP] [2021-04-19 17:01:41] resolve_keys
[START] [2021-04-19 17:01:41] hold_for_later_1
[STOP] [2021-04-19 17:01:41] hold_for_later_1
[START] [2021-04-19 17:01:41] hold_for_later_2
[STOP] [2021-04-19 17:01:41] hold_for_later_2
[START] [2021-04-19 17:01:41] resolve_missing_parents
[STOP] [2021-04-19 17:01:41] resolve_missing_parents
[START] [2021-04-19 17:01:41] rebuild_nodes
[START] [2021-04-19 17:01:41] Flattener#flatten
[START] [2021-04-19 17:01:41] Flattener#study_resource
[START] [2021-04-19 17:01:41] Flattener#build_ancestry
[STOP] [2021-04-19 17:01:41] Flattener#build_ancestry
[INFO] [2021-04-19 17:01:41] 6 ancestry keys
[START] [2021-04-19 17:01:41] build_node_ancestors
[INFO] [2021-04-19 17:01:41] old ancestors deleted.
[STOP] [2021-04-19 17:01:41] build_node_ancestors
[WARN] [2021-04-19 17:01:41] Flattener: nothing to flatten! (Completely flat resource?)
[STOP] [2021-04-19 17:01:41] Flattener#flatten
[STOP] [2021-04-19 17:01:41] rebuild_nodes
[START] [2021-04-19 17:01:41] resolve_missing_media_owners
[STOP] [2021-04-19 17:01:41] resolve_missing_media_owners
[START] [2021-04-19 17:01:41] sanitize_media_verbatims
[STOP] [2021-04-19 17:01:41] sanitize_media_verbatims
[START] [2021-04-19 17:01:41] queue_downloads
[STOP] [2021-04-19 17:01:41] queue_downloads
[START] [2021-04-19 17:01:41] parse_names
[WARN] [2021-04-19 17:01:41] I see 6 names which still need to be parsed.
[STOP] [2021-04-19 17:01:42] parse_names
[START] [2021-04-19 17:01:42] denormalize_canonical_names_to_nodes
[STOP] [2021-04-19 17:01:42] denormalize_canonical_names_to_nodes
[START] [2021-04-19 17:01:42] match_nodes
[START] [2021-04-19 17:01:42] map_all_nodes_to_pages
[STOP] [2021-04-19 17:01:42] map_all_nodes_to_pages
[INFO] [2021-04-19 17:01:42] ZERO unmatched nodes (of 6)! Nicely done.
[START] [2021-04-19 17:01:42] update_nodes
[STOP] [2021-04-19 17:01:42] update_nodes
[STOP] [2021-04-19 17:01:42] match_nodes
[START] [2021-04-19 17:01:42] reindex_search
[STOP] [2021-04-19 17:01:42] reindex_search
[START] [2021-04-19 17:01:42] normalize_units
[STOP] [2021-04-19 17:01:42] normalize_units
[START] [2021-04-19 17:01:42] calculate_statistics
[2021-04-19 17:01:42] ZERO NODE ANCESTORS. Is this actually a completely flat resource?
[STOP] [2021-04-19 17:01:42] calculate_statistics
[START] [2021-04-19 17:01:42] complete_harvest_instance
[START] [2021-04-19 17:01:42] overall_tsv_creation
[INFO] [2021-04-19 17:01:42] Processing group of 6 in 1 batches of 10000
[INFO] [2021-04-19 17:02:19] 6 Traits (unfiltered)...
[INFO] [2021-04-19 17:02:52] 6 Traits (filtered)...
[INFO] [2021-04-19 17:02:52] 0 Associations (filtered)...
[INFO] [2021-04-19 17:02:52] 33 metadata added.
[INFO] [2021-04-19 17:02:52] 0 metadata added.
[INFO] [2021-04-19 17:03:18] Average Time: 71.76
[INFO] [2021-04-19 17:03:18] Total Time: 1m36s
[STOP] [2021-04-19 17:03:18] overall_tsv_creation
[INFO] [2021-04-19 17:03:18] Done. Check your files:
[INFO] [2021-04-19 17:03:18] (6 lines) /app/public/data/delwiche_cooper_/publish_nodes.tsv
[INFO] [2021-04-19 17:03:18] (6 lines) /app/public/data/delwiche_cooper_/publish_scientific_names.tsv
[INFO] [2021-04-19 17:03:18] (7 lines) /app/public/data/delwiche_cooper_/publish_traits.tsv
[INFO] [2021-04-19 17:03:18] (34 lines) /app/public/data/delwiche_cooper_/publish_metadata.tsv
[STOP] [2021-04-19 17:03:18] complete_harvest_instance
[START] [2021-04-19 17:03:18] completed
[STOP] [2021-04-19 17:03:18] completed
[STOP] [2021-04-19 17:03:18] logged process, took 104.37
Latest Process