Stage:
completed
Fetched:
19 Apr 15:21
Validated:
19 Apr 15:21
Deltas Created
19 Apr 15:21
Units Normalized:
19 Apr 15:21
Ancestry Built:
19 Apr 15:21
Nodes Matched:
19 Apr 15:21
Names Parsed:
19 Apr 15:21
New Models Stored:
19 Apr 15:21
Indexed:
19 Apr 15:21
Completed:
19 Apr 15:23
Time to Harvest:
less than a minute
Harvesting Log
(604 lines)
# Logfile created on 2020-08-27 15:38:49 -0400 by logger.rb/v1.4.2
[START] [2020-08-27 15:38:49] logged process
[START] [2020-08-27 15:38:49] Creating resource from OpenData
[START] [2020-08-27 15:38:49] logged process
[START] [2020-08-27 15:38:49] Parse meta.xml file and create formats with fields
[STOP] [2020-08-27 15:38:50] Parse meta.xml file and create formats with fields
[STOP] [2020-08-27 15:38:50] Creating resource from OpenData
[INFO] [2020-11-19 14:10:56] ## HARVEST: type = re_download_opendata_-harvest
[INFO] [2020-11-19 14:10:58] ## remove_type: ScientificName
[INFO] [2020-11-19 14:10:58] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-19 14:10:58] [14:10:58.419] Removed 0 Scientificnames
[INFO] [2020-11-19 14:10:58] ## remove_type: Vernacular
[INFO] [2020-11-19 14:10:58] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-19 14:10:58] [14:10:58.422] Removed 0 Vernaculars
[INFO] [2020-11-19 14:10:58] ## remove_type: Article
[INFO] [2020-11-19 14:10:58] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-19 14:10:58] [14:10:58.425] Removed 0 Articles
[INFO] [2020-11-19 14:10:58] ## remove_type: Medium
[INFO] [2020-11-19 14:10:58] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-19 14:10:58] [14:10:58.429] Removed 0 Media
[INFO] [2020-11-19 14:10:58] ## remove_type: Trait
[INFO] [2020-11-19 14:10:58] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-19 14:10:58] [14:10:58.432] Removed 0 Traits
[INFO] [2020-11-19 14:10:58] ## remove_type: MetaTrait
[INFO] [2020-11-19 14:10:58] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-19 14:10:58] [14:10:58.434] Removed 0 Metatraits
[INFO] [2020-11-19 14:10:58] ## remove_type: OccurrenceMetadatum
[INFO] [2020-11-19 14:10:58] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-19 14:10:58] [14:10:58.437] Removed 0 Occurrencemetadata
[INFO] [2020-11-19 14:10:58] ## remove_type: Assoc
[INFO] [2020-11-19 14:10:58] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-19 14:10:58] [14:10:58.440] Removed 0 Assocs
[INFO] [2020-11-19 14:10:58] ## remove_type: MetaAssoc
[INFO] [2020-11-19 14:10:58] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-19 14:10:58] [14:10:58.442] Removed 0 Metaassocs
[INFO] [2020-11-19 14:10:58] ## remove_type: Identifier
[INFO] [2020-11-19 14:10:58] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-19 14:10:58] [14:10:58.445] Removed 0 Identifiers
[INFO] [2020-11-19 14:10:58] ## remove_type: Reference
[INFO] [2020-11-19 14:10:58] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-19 14:10:58] [14:10:58.447] Removed 0 References
[INFO] [2020-11-19 14:10:58] ## remove_type: Node
[INFO] [2020-11-19 14:10:58] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-19 14:10:58] [14:10:58.469] Removed 0 Nodes
[START] [2020-11-19 14:10:58] logged process: 58bbc42b01abb4c1b2698de049792ffb4b63b979
[START] [2020-11-19 14:10:58] Creating resource from OpenData
[START] [2020-11-19 14:10:58] logged process: 58bbc42b01abb4c1b2698de049792ffb4b63b979
[START] [2020-11-19 14:10:58] Parse meta.xml file and create formats with fields
[STOP] [2020-11-19 14:10:58] Parse meta.xml file and create formats with fields
[STOP] [2020-11-19 14:10:58] Creating resource from OpenData
[START] [2020-11-19 14:10:58] logged process: 58bbc42b01abb4c1b2698de049792ffb4b63b979
[START] [2020-11-19 14:10:58] create_harvest_instance
[STOP] [2020-11-19 14:11:00] create_harvest_instance
[START] [2020-11-19 14:11:00] fetch_files
[STOP] [2020-11-19 14:11:00] fetch_files
[START] [2020-11-19 14:11:00] validate_each_file
[STOP] [2020-11-19 14:11:00] validate_each_file
[START] [2020-11-19 14:11:00] convert_to_csv
[CMD] [2020-11-19 14:11:00] /usr/bin/sort /app/public/converted_csv/morehouse_et_al__refs_24542.csv > /app/public/converted_csv/morehouse_et_al__refs_24542.csv_sorted
[CMD] [2020-11-19 14:11:00] /usr/bin/sort /app/public/converted_csv/morehouse_et_al__nodes_24543.csv > /app/public/converted_csv/morehouse_et_al__nodes_24543.csv_sorted
[CMD] [2020-11-19 14:11:00] /usr/bin/sort /app/public/converted_csv/morehouse_et_al__occurrences_24544.csv > /app/public/converted_csv/morehouse_et_al__occurrences_24544.csv_sorted
[CMD] [2020-11-19 14:11:00] /usr/bin/sort /app/public/converted_csv/morehouse_et_al__measurements_24545.csv > /app/public/converted_csv/morehouse_et_al__measurements_24545.csv_sorted
[STOP] [2020-11-19 14:11:00] convert_to_csv
[START] [2020-11-19 14:11:00] calculate_delta
[CMD] [2020-11-19 14:11:00] echo "0a" > /app/public/diff/morehouse_et_al__refs_24542.diff
[CMD] [2020-11-19 14:11:00] tail -n +1 /app/public/converted_csv/morehouse_et_al__refs_24542.csv >> /app/public/diff/morehouse_et_al__refs_24542.diff
[CMD] [2020-11-19 14:11:00] echo "." >> /app/public/diff/morehouse_et_al__refs_24542.diff
[CMD] [2020-11-19 14:11:00] echo "0a" > /app/public/diff/morehouse_et_al__nodes_24543.diff
[CMD] [2020-11-19 14:11:00] tail -n +1 /app/public/converted_csv/morehouse_et_al__nodes_24543.csv >> /app/public/diff/morehouse_et_al__nodes_24543.diff
[CMD] [2020-11-19 14:11:00] echo "." >> /app/public/diff/morehouse_et_al__nodes_24543.diff
[CMD] [2020-11-19 14:11:00] echo "0a" > /app/public/diff/morehouse_et_al__occurrences_24544.diff
[CMD] [2020-11-19 14:11:00] tail -n +1 /app/public/converted_csv/morehouse_et_al__occurrences_24544.csv >> /app/public/diff/morehouse_et_al__occurrences_24544.diff
[CMD] [2020-11-19 14:11:00] echo "." >> /app/public/diff/morehouse_et_al__occurrences_24544.diff
[CMD] [2020-11-19 14:11:00] echo "0a" > /app/public/diff/morehouse_et_al__measurements_24545.diff
[CMD] [2020-11-19 14:11:00] tail -n +1 /app/public/converted_csv/morehouse_et_al__measurements_24545.csv >> /app/public/diff/morehouse_et_al__measurements_24545.diff
[CMD] [2020-11-19 14:11:00] echo "." >> /app/public/diff/morehouse_et_al__measurements_24545.diff
[STOP] [2020-11-19 14:11:00] calculate_delta
[START] [2020-11-19 14:11:00] parse_diff_and_store
[INFO] [2020-11-19 14:11:00] Loading refs diff file into memory (true lines)...
[INFO] [2020-11-19 14:11:00] Loading nodes diff file into memory (true lines)...
[INFO] [2020-11-19 14:11:00] Loading occurrences diff file into memory (true lines)...
[INFO] [2020-11-19 14:11:00] Loading measurements diff file into memory (true lines)...
[INFO] [2020-11-19 14:11:01] Storing 1 ScientificNames
[INFO] [2020-11-19 14:11:01] Processing group of 1 in 1 groups of 1000
[INFO] [2020-11-19 14:11:01] Average Time: 0.0
[INFO] [2020-11-19 14:11:01] Total Time: 1s
[INFO] [2020-11-19 14:11:01] Storing 1 Nodes
[INFO] [2020-11-19 14:11:01] Processing group of 1 in 1 groups of 1000
[INFO] [2020-11-19 14:11:01] Average Time: 0.0
[INFO] [2020-11-19 14:11:01] Total Time: 1s
[INFO] [2020-11-19 14:11:01] Storing 1 Occurrences
[INFO] [2020-11-19 14:11:01] Processing group of 1 in 1 groups of 1000
[INFO] [2020-11-19 14:11:01] Average Time: 0.0
[INFO] [2020-11-19 14:11:01] Total Time: 1s
[INFO] [2020-11-19 14:11:01] Storing 30 Traits
[INFO] [2020-11-19 14:11:01] Processing group of 30 in 1 groups of 1000
[INFO] [2020-11-19 14:11:01] Average Time: 0.01
[INFO] [2020-11-19 14:11:01] Total Time: 1s
[INFO] [2020-11-19 14:11:01] Storing 4 MetaTraits
[INFO] [2020-11-19 14:11:01] Processing group of 4 in 1 groups of 1000
[INFO] [2020-11-19 14:11:01] Average Time: 0.0
[INFO] [2020-11-19 14:11:01] Total Time: 1s
[STOP] [2020-11-19 14:11:01] parse_diff_and_store
[START] [2020-11-19 14:11:01] resolve_keys
[INFO] [2020-11-19 14:11:07] Occurrences to nodes (through scientific_names)...
[INFO] [2020-11-19 14:11:07] traits to occurrences...
[INFO] [2020-11-19 14:11:07] traits to nodes (through occurrences)...
[INFO] [2020-11-19 14:11:07] Traits to sex term...
[INFO] [2020-11-19 14:11:07] Traits to lifestage term...
[INFO] [2020-11-19 14:11:07] MetaTraits to traits...
[INFO] [2020-11-19 14:11:07] MetaTraits (simple, measurement row refers to parent) to traits...
[INFO] [2020-11-19 14:11:07] Assocs to occurrences...
[INFO] [2020-11-19 14:11:07] Assocs to nodes...
[INFO] [2020-11-19 14:11:07] Assoc to sex term...
[INFO] [2020-11-19 14:11:07] Assoc to lifestage term...
[INFO] [2020-11-19 14:11:07] MetaAssoc to assocs...
[STOP] [2020-11-19 14:11:07] resolve_keys
[START] [2020-11-19 14:11:07] hold_for_later_1
[STOP] [2020-11-19 14:11:07] hold_for_later_1
[START] [2020-11-19 14:11:07] hold_for_later_2
[STOP] [2020-11-19 14:11:07] hold_for_later_2
[START] [2020-11-19 14:11:07] resolve_missing_parents
[STOP] [2020-11-19 14:11:07] resolve_missing_parents
[START] [2020-11-19 14:11:07] rebuild_nodes
[START] [2020-11-19 14:11:07] Flattener#flatten
[START] [2020-11-19 14:11:07] Flattener#study_resource
[START] [2020-11-19 14:11:07] Flattener#build_ancestry
[STOP] [2020-11-19 14:11:07] Flattener#build_ancestry
[INFO] [2020-11-19 14:11:07] 1 ancestry keys
[START] [2020-11-19 14:11:07] build_node_ancestors
[INFO] [2020-11-19 14:11:07] old ancestors deleted.
[STOP] [2020-11-19 14:11:07] build_node_ancestors
[WARN] [2020-11-19 14:11:07] Flattener: nothing to flatten! (Completely flat resource?)
[STOP] [2020-11-19 14:11:07] Flattener#flatten
[STOP] [2020-11-19 14:11:07] rebuild_nodes
[START] [2020-11-19 14:11:07] resolve_missing_media_owners
[STOP] [2020-11-19 14:11:07] resolve_missing_media_owners
[START] [2020-11-19 14:11:07] sanitize_media_verbatims
[STOP] [2020-11-19 14:11:07] sanitize_media_verbatims
[START] [2020-11-19 14:11:07] queue_downloads
[STOP] [2020-11-19 14:11:07] queue_downloads
[START] [2020-11-19 14:11:07] parse_names
[WARN] [2020-11-19 14:11:07] I see 1 names which still need to be parsed.
[STOP] [2020-11-19 14:11:08] parse_names
[START] [2020-11-19 14:11:09] denormalize_canonical_names_to_nodes
[STOP] [2020-11-19 14:11:09] denormalize_canonical_names_to_nodes
[START] [2020-11-19 14:11:09] match_nodes
[START] [2020-11-19 14:11:09] map_all_nodes_to_pages
[STOP] [2020-11-19 14:11:09] map_all_nodes_to_pages
[INFO] [2020-11-19 14:11:09] ZERO unmatched nodes (of 1)! Nicely done.
[START] [2020-11-19 14:11:09] update_nodes
[STOP] [2020-11-19 14:11:09] update_nodes
[STOP] [2020-11-19 14:11:09] match_nodes
[START] [2020-11-19 14:11:09] reindex_search
[STOP] [2020-11-19 14:11:09] reindex_search
[START] [2020-11-19 14:11:09] normalize_units
[STOP] [2020-11-19 14:11:09] normalize_units
[START] [2020-11-19 14:11:09] calculate_statistics
[2020-11-19 14:11:09] ZERO NODE ANCESTORS. Is this actually a completely flat resource?
[STOP] [2020-11-19 14:11:09] calculate_statistics
[START] [2020-11-19 14:11:09] complete_harvest_instance
[START] [2020-11-19 14:11:09] overall_tsv_creation
[INFO] [2020-11-19 14:11:09] Processing group of 1 in 1 batches of 10000
[INFO] [2020-11-19 14:11:52] 2 Traits (unfiltered)...
[INFO] [2020-11-19 14:12:34] 2 Traits (filtered)...
[INFO] [2020-11-19 14:12:34] 0 Associations (filtered)...
[INFO] [2020-11-19 14:12:34] 32 metadata added.
[INFO] [2020-11-19 14:12:34] 0 metadata added.
[INFO] [2020-11-19 14:12:34] Average Time: 65.97
[INFO] [2020-11-19 14:12:34] Total Time: 1m26s
[STOP] [2020-11-19 14:12:34] overall_tsv_creation
[INFO] [2020-11-19 14:12:34] Done. Check your files:
[INFO] [2020-11-19 14:12:34] (1 lines) /app/public/data/morehouse_et_al_/publish_nodes.tsv
[INFO] [2020-11-19 14:12:34] (1 lines) /app/public/data/morehouse_et_al_/publish_scientific_names.tsv
[INFO] [2020-11-19 14:12:34] (3 lines) /app/public/data/morehouse_et_al_/publish_traits.tsv
[INFO] [2020-11-19 14:12:34] (33 lines) /app/public/data/morehouse_et_al_/publish_metadata.tsv
[STOP] [2020-11-19 14:12:34] complete_harvest_instance
[START] [2020-11-19 14:12:34] completed
[STOP] [2020-11-19 14:12:34] completed
[STOP] [2020-11-19 14:12:34] logged process, took 95.94
[INFO] [2020-11-19 14:24:16] ## HARVEST: type = re_download_opendata_-harvest
[INFO] [2020-11-19 14:24:20] ## remove_type: ScientificName
[INFO] [2020-11-19 14:24:20] ++ Calling delete_all on 1 instances...
[INFO] [2020-11-19 14:24:20] [14:24:20.806] Removed 1 Scientificnames
[INFO] [2020-11-19 14:24:20] ## remove_type: Vernacular
[INFO] [2020-11-19 14:24:20] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-19 14:24:20] [14:24:20.809] Removed 0 Vernaculars
[INFO] [2020-11-19 14:24:20] ## remove_type: Article
[INFO] [2020-11-19 14:24:20] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-19 14:24:20] [14:24:20.812] Removed 0 Articles
[INFO] [2020-11-19 14:24:20] ## remove_type: Medium
[INFO] [2020-11-19 14:24:20] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-19 14:24:20] [14:24:20.816] Removed 0 Media
[INFO] [2020-11-19 14:24:20] ## remove_type: Trait
[INFO] [2020-11-19 14:24:20] ++ Calling delete_all on 30 instances...
[INFO] [2020-11-19 14:24:20] [14:24:20.820] Removed 30 Traits
[INFO] [2020-11-19 14:24:20] ## remove_type: MetaTrait
[INFO] [2020-11-19 14:24:20] ++ Calling delete_all on 4 instances...
[INFO] [2020-11-19 14:24:20] [14:24:20.824] Removed 4 Metatraits
[INFO] [2020-11-19 14:24:20] ## remove_type: OccurrenceMetadatum
[INFO] [2020-11-19 14:24:20] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-19 14:24:20] [14:24:20.827] Removed 0 Occurrencemetadata
[INFO] [2020-11-19 14:24:20] ## remove_type: Assoc
[INFO] [2020-11-19 14:24:20] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-19 14:24:20] [14:24:20.829] Removed 0 Assocs
[INFO] [2020-11-19 14:24:20] ## remove_type: MetaAssoc
[INFO] [2020-11-19 14:24:20] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-19 14:24:20] [14:24:20.832] Removed 0 Metaassocs
[INFO] [2020-11-19 14:24:20] ## remove_type: Identifier
[INFO] [2020-11-19 14:24:20] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-19 14:24:20] [14:24:20.835] Removed 0 Identifiers
[INFO] [2020-11-19 14:24:20] ## remove_type: Reference
[INFO] [2020-11-19 14:24:20] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-19 14:24:20] [14:24:20.837] Removed 0 References
[INFO] [2020-11-19 14:24:20] Starting batch with ID 84620950...
[INFO] [2020-11-19 14:24:20] Starting batch with ID 84620950...
[INFO] [2020-11-19 14:24:20] Starting batch with ID 84620950...
[INFO] [2020-11-19 14:24:20] Starting batch with ID 84620950...
[INFO] [2020-11-19 14:24:20] Starting batch with ID 84620950...
[INFO] [2020-11-19 14:24:20] Starting batch with ID 84620950...
[INFO] [2020-11-19 14:24:20] Starting batch with ID 84620950...
[INFO] [2020-11-19 14:24:20] Starting batch with ID 84620950...
[INFO] [2020-11-19 14:24:20] Starting batch with ID 84620950...
[INFO] [2020-11-19 14:24:20] Starting batch with ID 84620950...
[INFO] [2020-11-19 14:24:20] Starting batch with ID 84620950...
[INFO] [2020-11-19 14:24:20] Starting batch with ID 84620950...
[INFO] [2020-11-19 14:24:20] Starting batch with ID 84620950...
[INFO] [2020-11-19 14:24:21] Starting batch with ID 84620950...
[INFO] [2020-11-19 14:24:21] Starting batch with ID 84620950...
[INFO] [2020-11-19 14:24:21] Starting batch with ID 84620950...
[INFO] [2020-11-19 14:24:21] Starting batch with ID 84620950...
[INFO] [2020-11-19 14:24:21] Starting batch with ID 84620950...
[INFO] [2020-11-19 14:24:21] ## remove_type: Node
[INFO] [2020-11-19 14:24:21] ++ Calling delete_all on 1 instances...
[INFO] [2020-11-19 14:24:21] [14:24:21.409] Removed 1 Nodes
[START] [2020-11-19 14:24:21] logged process: 58bbc42b01abb4c1b2698de049792ffb4b63b979
[START] [2020-11-19 14:24:21] Creating resource from OpenData
[START] [2020-11-19 14:24:21] logged process: 58bbc42b01abb4c1b2698de049792ffb4b63b979
[START] [2020-11-19 14:24:21] Parse meta.xml file and create formats with fields
[STOP] [2020-11-19 14:24:22] Parse meta.xml file and create formats with fields
[STOP] [2020-11-19 14:24:22] Creating resource from OpenData
[START] [2020-11-19 14:24:22] logged process: 58bbc42b01abb4c1b2698de049792ffb4b63b979
[START] [2020-11-19 14:24:22] create_harvest_instance
[STOP] [2020-11-19 14:24:23] create_harvest_instance
[START] [2020-11-19 14:24:23] fetch_files
[STOP] [2020-11-19 14:24:23] fetch_files
[START] [2020-11-19 14:24:23] validate_each_file
[STOP] [2020-11-19 14:24:23] validate_each_file
[START] [2020-11-19 14:24:23] convert_to_csv
[CMD] [2020-11-19 14:24:23] /usr/bin/sort /app/public/converted_csv/morehouse_et_al__refs_24566.csv > /app/public/converted_csv/morehouse_et_al__refs_24566.csv_sorted
[CMD] [2020-11-19 14:24:23] /usr/bin/sort /app/public/converted_csv/morehouse_et_al__nodes_24567.csv > /app/public/converted_csv/morehouse_et_al__nodes_24567.csv_sorted
[CMD] [2020-11-19 14:24:23] /usr/bin/sort /app/public/converted_csv/morehouse_et_al__occurrences_24568.csv > /app/public/converted_csv/morehouse_et_al__occurrences_24568.csv_sorted
[CMD] [2020-11-19 14:24:23] /usr/bin/sort /app/public/converted_csv/morehouse_et_al__measurements_24569.csv > /app/public/converted_csv/morehouse_et_al__measurements_24569.csv_sorted
[STOP] [2020-11-19 14:24:23] convert_to_csv
[START] [2020-11-19 14:24:23] calculate_delta
[CMD] [2020-11-19 14:24:23] echo "0a" > /app/public/diff/morehouse_et_al__refs_24566.diff
[CMD] [2020-11-19 14:24:23] tail -n +1 /app/public/converted_csv/morehouse_et_al__refs_24566.csv >> /app/public/diff/morehouse_et_al__refs_24566.diff
[CMD] [2020-11-19 14:24:23] echo "." >> /app/public/diff/morehouse_et_al__refs_24566.diff
[CMD] [2020-11-19 14:24:23] echo "0a" > /app/public/diff/morehouse_et_al__nodes_24567.diff
[CMD] [2020-11-19 14:24:23] tail -n +1 /app/public/converted_csv/morehouse_et_al__nodes_24567.csv >> /app/public/diff/morehouse_et_al__nodes_24567.diff
[CMD] [2020-11-19 14:24:23] echo "." >> /app/public/diff/morehouse_et_al__nodes_24567.diff
[CMD] [2020-11-19 14:24:23] echo "0a" > /app/public/diff/morehouse_et_al__occurrences_24568.diff
[CMD] [2020-11-19 14:24:23] tail -n +1 /app/public/converted_csv/morehouse_et_al__occurrences_24568.csv >> /app/public/diff/morehouse_et_al__occurrences_24568.diff
[CMD] [2020-11-19 14:24:23] echo "." >> /app/public/diff/morehouse_et_al__occurrences_24568.diff
[CMD] [2020-11-19 14:24:23] echo "0a" > /app/public/diff/morehouse_et_al__measurements_24569.diff
[CMD] [2020-11-19 14:24:23] tail -n +1 /app/public/converted_csv/morehouse_et_al__measurements_24569.csv >> /app/public/diff/morehouse_et_al__measurements_24569.diff
[CMD] [2020-11-19 14:24:23] echo "." >> /app/public/diff/morehouse_et_al__measurements_24569.diff
[STOP] [2020-11-19 14:24:23] calculate_delta
[START] [2020-11-19 14:24:23] parse_diff_and_store
[INFO] [2020-11-19 14:24:23] Loading refs diff file into memory (true lines)...
[INFO] [2020-11-19 14:24:23] Loading nodes diff file into memory (true lines)...
[INFO] [2020-11-19 14:24:23] Loading occurrences diff file into memory (true lines)...
[INFO] [2020-11-19 14:24:23] Loading measurements diff file into memory (true lines)...
[INFO] [2020-11-19 14:24:25] Storing 1 ScientificNames
[INFO] [2020-11-19 14:24:25] Processing group of 1 in 1 groups of 1000
[INFO] [2020-11-19 14:24:25] Average Time: 0.0
[INFO] [2020-11-19 14:24:25] Total Time: 1s
[INFO] [2020-11-19 14:24:25] Storing 1 Nodes
[INFO] [2020-11-19 14:24:25] Processing group of 1 in 1 groups of 1000
[INFO] [2020-11-19 14:24:25] Average Time: 0.0
[INFO] [2020-11-19 14:24:25] Total Time: 1s
[INFO] [2020-11-19 14:24:25] Storing 1 Occurrences
[INFO] [2020-11-19 14:24:25] Processing group of 1 in 1 groups of 1000
[INFO] [2020-11-19 14:24:25] Average Time: 0.0
[INFO] [2020-11-19 14:24:25] Total Time: 1s
[INFO] [2020-11-19 14:24:25] Storing 30 Traits
[INFO] [2020-11-19 14:24:25] Processing group of 30 in 1 groups of 1000
[INFO] [2020-11-19 14:24:25] Average Time: 0.01
[INFO] [2020-11-19 14:24:25] Total Time: 1s
[INFO] [2020-11-19 14:24:25] Storing 4 MetaTraits
[INFO] [2020-11-19 14:24:25] Processing group of 4 in 1 groups of 1000
[INFO] [2020-11-19 14:24:25] Average Time: 0.0
[INFO] [2020-11-19 14:24:25] Total Time: 1s
[STOP] [2020-11-19 14:24:25] parse_diff_and_store
[START] [2020-11-19 14:24:25] resolve_keys
[INFO] [2020-11-19 14:24:30] Occurrences to nodes (through scientific_names)...
[INFO] [2020-11-19 14:24:30] traits to occurrences...
[INFO] [2020-11-19 14:24:30] traits to nodes (through occurrences)...
[INFO] [2020-11-19 14:24:30] Traits to sex term...
[INFO] [2020-11-19 14:24:31] Traits to lifestage term...
[INFO] [2020-11-19 14:24:31] MetaTraits to traits...
[INFO] [2020-11-19 14:24:31] MetaTraits (simple, measurement row refers to parent) to traits...
[INFO] [2020-11-19 14:24:31] Assocs to occurrences...
[INFO] [2020-11-19 14:24:31] Assocs to nodes...
[INFO] [2020-11-19 14:24:31] Assoc to sex term...
[INFO] [2020-11-19 14:24:31] Assoc to lifestage term...
[INFO] [2020-11-19 14:24:31] MetaAssoc to assocs...
[STOP] [2020-11-19 14:24:31] resolve_keys
[START] [2020-11-19 14:24:31] hold_for_later_1
[STOP] [2020-11-19 14:24:31] hold_for_later_1
[START] [2020-11-19 14:24:31] hold_for_later_2
[STOP] [2020-11-19 14:24:31] hold_for_later_2
[START] [2020-11-19 14:24:31] resolve_missing_parents
[STOP] [2020-11-19 14:24:31] resolve_missing_parents
[START] [2020-11-19 14:24:31] rebuild_nodes
[START] [2020-11-19 14:24:31] Flattener#flatten
[START] [2020-11-19 14:24:31] Flattener#study_resource
[START] [2020-11-19 14:24:31] Flattener#build_ancestry
[STOP] [2020-11-19 14:24:31] Flattener#build_ancestry
[INFO] [2020-11-19 14:24:31] 1 ancestry keys
[START] [2020-11-19 14:24:31] build_node_ancestors
[INFO] [2020-11-19 14:24:31] old ancestors deleted.
[STOP] [2020-11-19 14:24:31] build_node_ancestors
[WARN] [2020-11-19 14:24:31] Flattener: nothing to flatten! (Completely flat resource?)
[STOP] [2020-11-19 14:24:31] Flattener#flatten
[STOP] [2020-11-19 14:24:31] rebuild_nodes
[START] [2020-11-19 14:24:31] resolve_missing_media_owners
[STOP] [2020-11-19 14:24:31] resolve_missing_media_owners
[START] [2020-11-19 14:24:31] sanitize_media_verbatims
[STOP] [2020-11-19 14:24:31] sanitize_media_verbatims
[START] [2020-11-19 14:24:31] queue_downloads
[STOP] [2020-11-19 14:24:31] queue_downloads
[START] [2020-11-19 14:24:31] parse_names
[WARN] [2020-11-19 14:24:31] I see 1 names which still need to be parsed.
[STOP] [2020-11-19 14:24:32] parse_names
[START] [2020-11-19 14:24:32] denormalize_canonical_names_to_nodes
[STOP] [2020-11-19 14:24:32] denormalize_canonical_names_to_nodes
[START] [2020-11-19 14:24:32] match_nodes
[START] [2020-11-19 14:24:32] map_all_nodes_to_pages
[STOP] [2020-11-19 14:24:32] map_all_nodes_to_pages
[INFO] [2020-11-19 14:24:32] ZERO unmatched nodes (of 1)! Nicely done.
[START] [2020-11-19 14:24:32] update_nodes
[STOP] [2020-11-19 14:24:32] update_nodes
[STOP] [2020-11-19 14:24:32] match_nodes
[START] [2020-11-19 14:24:32] reindex_search
[STOP] [2020-11-19 14:24:32] reindex_search
[START] [2020-11-19 14:24:32] normalize_units
[STOP] [2020-11-19 14:24:32] normalize_units
[START] [2020-11-19 14:24:32] calculate_statistics
[2020-11-19 14:24:32] ZERO NODE ANCESTORS. Is this actually a completely flat resource?
[STOP] [2020-11-19 14:24:32] calculate_statistics
[START] [2020-11-19 14:24:32] complete_harvest_instance
[START] [2020-11-19 14:24:32] overall_tsv_creation
[INFO] [2020-11-19 14:24:32] Processing group of 1 in 1 batches of 10000
[INFO] [2020-11-19 14:25:15] 2 Traits (unfiltered)...
[INFO] [2020-11-19 14:25:57] 2 Traits (filtered)...
[INFO] [2020-11-19 14:25:57] 0 Associations (filtered)...
[INFO] [2020-11-19 14:25:57] 32 metadata added.
[INFO] [2020-11-19 14:25:57] 0 metadata added.
[INFO] [2020-11-19 14:25:57] Average Time: 65.79
[INFO] [2020-11-19 14:25:57] Total Time: 1m26s
[STOP] [2020-11-19 14:25:57] overall_tsv_creation
[INFO] [2020-11-19 14:25:57] Done. Check your files:
[INFO] [2020-11-19 14:25:57] (1 lines) /app/public/data/morehouse_et_al_/publish_nodes.tsv
[INFO] [2020-11-19 14:25:57] (1 lines) /app/public/data/morehouse_et_al_/publish_scientific_names.tsv
[INFO] [2020-11-19 14:25:57] (3 lines) /app/public/data/morehouse_et_al_/publish_traits.tsv
[INFO] [2020-11-19 14:25:57] (33 lines) /app/public/data/morehouse_et_al_/publish_metadata.tsv
[STOP] [2020-11-19 14:25:57] complete_harvest_instance
[START] [2020-11-19 14:25:57] completed
[STOP] [2020-11-19 14:25:57] completed
[STOP] [2020-11-19 14:25:57] logged process, took 95.73
[INFO] [2021-04-19 15:16:02] ## HARVEST: type = re_download_opendata_-harvest
[INFO] [2021-04-19 15:21:34] ## remove_type: ScientificName
[INFO] [2021-04-19 15:21:34] ++ Calling delete_all on 1 instances...
[INFO] [2021-04-19 15:21:34] [15:21:34.434] Removed 1 Scientificnames
[INFO] [2021-04-19 15:21:34] ## remove_type: Vernacular
[INFO] [2021-04-19 15:21:34] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-19 15:21:34] [15:21:34.436] Removed 0 Vernaculars
[INFO] [2021-04-19 15:21:34] ## remove_type: Article
[INFO] [2021-04-19 15:21:34] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-19 15:21:34] [15:21:34.438] Removed 0 Articles
[INFO] [2021-04-19 15:21:34] ## remove_type: Medium
[INFO] [2021-04-19 15:21:34] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-19 15:21:34] [15:21:34.439] Removed 0 Media
[INFO] [2021-04-19 15:21:34] ## remove_type: Trait
[INFO] [2021-04-19 15:21:34] ++ Calling delete_all on 30 instances...
[INFO] [2021-04-19 15:21:34] [15:21:34.442] Removed 30 Traits
[INFO] [2021-04-19 15:21:34] ## remove_type: MetaTrait
[INFO] [2021-04-19 15:21:34] ++ Calling delete_all on 4 instances...
[INFO] [2021-04-19 15:21:34] [15:21:34.444] Removed 4 Metatraits
[INFO] [2021-04-19 15:21:34] ## remove_type: OccurrenceMetadatum
[INFO] [2021-04-19 15:21:34] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-19 15:21:34] [15:21:34.446] Removed 0 Occurrencemetadata
[INFO] [2021-04-19 15:21:34] ## remove_type: Assoc
[INFO] [2021-04-19 15:21:34] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-19 15:21:34] [15:21:34.447] Removed 0 Assocs
[INFO] [2021-04-19 15:21:34] ## remove_type: MetaAssoc
[INFO] [2021-04-19 15:21:34] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-19 15:21:34] [15:21:34.449] Removed 0 Metaassocs
[INFO] [2021-04-19 15:21:34] ## remove_type: Identifier
[INFO] [2021-04-19 15:21:34] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-19 15:21:34] [15:21:34.450] Removed 0 Identifiers
[INFO] [2021-04-19 15:21:34] ## remove_type: Reference
[INFO] [2021-04-19 15:21:34] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-19 15:21:34] [15:21:34.452] Removed 0 References
[INFO] [2021-04-19 15:21:34] Starting batch with ID 84620953...
[INFO] [2021-04-19 15:21:34] Starting batch with ID 84620953...
[INFO] [2021-04-19 15:21:34] Starting batch with ID 84620953...
[INFO] [2021-04-19 15:21:34] Starting batch with ID 84620953...
[INFO] [2021-04-19 15:21:34] Starting batch with ID 84620953...
[INFO] [2021-04-19 15:21:34] Starting batch with ID 84620953...
[INFO] [2021-04-19 15:21:34] Starting batch with ID 84620953...
[INFO] [2021-04-19 15:21:34] Starting batch with ID 84620953...
[INFO] [2021-04-19 15:21:34] Starting batch with ID 84620953...
[INFO] [2021-04-19 15:21:34] Starting batch with ID 84620953...
[INFO] [2021-04-19 15:21:34] Starting batch with ID 84620953...
[INFO] [2021-04-19 15:21:34] Starting batch with ID 84620953...
[INFO] [2021-04-19 15:21:34] Starting batch with ID 84620953...
[INFO] [2021-04-19 15:21:34] Starting batch with ID 84620953...
[INFO] [2021-04-19 15:21:34] Starting batch with ID 84620953...
[INFO] [2021-04-19 15:21:34] Starting batch with ID 84620953...
[INFO] [2021-04-19 15:21:34] Starting batch with ID 84620953...
[INFO] [2021-04-19 15:21:34] Starting batch with ID 84620953...
[INFO] [2021-04-19 15:21:34] Starting batch with ID 84620953...
[INFO] [2021-04-19 15:21:34] Starting batch with ID 84620953...
[INFO] [2021-04-19 15:21:34] Starting batch with ID 84620953...
[INFO] [2021-04-19 15:21:34] Starting batch with ID 84620953...
[INFO] [2021-04-19 15:21:34] ## remove_type: Node
[INFO] [2021-04-19 15:21:34] ++ Calling delete_all on 1 instances...
[INFO] [2021-04-19 15:21:34] [15:21:34.612] Removed 1 Nodes
[START] [2021-04-19 15:21:34] logged process: 5ecc716a6a5541910d0c854f5a0c8d1651b82ad0 Improved MetaXml.ignore and added publisher to media (ignored)
[START] [2021-04-19 15:21:34] Creating resource from OpenData
[START] [2021-04-19 15:21:34] logged process: 5ecc716a6a5541910d0c854f5a0c8d1651b82ad0 Improved MetaXml.ignore and added publisher to media (ignored)
[START] [2021-04-19 15:21:34] Parse meta.xml file and create formats with fields
[STOP] [2021-04-19 15:21:35] Parse meta.xml file and create formats with fields
[STOP] [2021-04-19 15:21:35] Creating resource from OpenData
[START] [2021-04-19 15:21:35] logged process: 5ecc716a6a5541910d0c854f5a0c8d1651b82ad0 Improved MetaXml.ignore and added publisher to media (ignored)
[START] [2021-04-19 15:21:35] create_harvest_instance
[INFO] [2021-04-19 15:21:35] Created harvest instance #3763
[STOP] [2021-04-19 15:21:35] create_harvest_instance
[START] [2021-04-19 15:21:35] fetch_files
[STOP] [2021-04-19 15:21:35] fetch_files
[START] [2021-04-19 15:21:35] validate_each_file
[INFO] [2021-04-19 15:21:35] Looping over 4 formats...
[INFO] [2021-04-19 15:21:35] ...refs (/app/public/data/morehouse_et_al_/references.tsv)
[INFO] [2021-04-19 15:21:35] Valid: /app/public/converted_csv/morehouse_et_al__refs_3763.csv (0 lines)
[INFO] [2021-04-19 15:21:35] ...nodes (/app/public/data/morehouse_et_al_/taxa.txt)
[INFO] [2021-04-19 15:21:35] Valid: /app/public/converted_csv/morehouse_et_al__nodes_3763.csv (1 lines)
[INFO] [2021-04-19 15:21:35] ...occurrences (/app/public/data/morehouse_et_al_/occurrences.txt)
[INFO] [2021-04-19 15:21:35] Valid: /app/public/converted_csv/morehouse_et_al__occurrences_3763.csv (1 lines)
[INFO] [2021-04-19 15:21:35] ...measurements (/app/public/data/morehouse_et_al_/measurementsorfacts.txt)
[INFO] [2021-04-19 15:21:35] Valid: /app/public/converted_csv/morehouse_et_al__measurements_3763.csv (30 lines)
[STOP] [2021-04-19 15:21:35] validate_each_file
[START] [2021-04-19 15:21:35] convert_to_csv
[INFO] [2021-04-19 15:21:35] Looping over 4 formats...
[INFO] [2021-04-19 15:21:35] ...refs (/app/public/data/morehouse_et_al_/references.tsv)
[CMD] [2021-04-19 15:21:35] /usr/bin/sort /app/public/converted_csv/morehouse_et_al__refs_3763.csv > /app/public/converted_csv/morehouse_et_al__refs_3763.csv_sorted
[INFO] [2021-04-19 15:21:35] Converted: /app/public/converted_csv/morehouse_et_al__refs_3763.csv (0 lines)
[INFO] [2021-04-19 15:21:35] ...nodes (/app/public/data/morehouse_et_al_/taxa.txt)
[CMD] [2021-04-19 15:21:35] /usr/bin/sort /app/public/converted_csv/morehouse_et_al__nodes_3763.csv > /app/public/converted_csv/morehouse_et_al__nodes_3763.csv_sorted
[INFO] [2021-04-19 15:21:35] Converted: /app/public/converted_csv/morehouse_et_al__nodes_3763.csv (1 lines)
[INFO] [2021-04-19 15:21:35] ...occurrences (/app/public/data/morehouse_et_al_/occurrences.txt)
[CMD] [2021-04-19 15:21:35] /usr/bin/sort /app/public/converted_csv/morehouse_et_al__occurrences_3763.csv > /app/public/converted_csv/morehouse_et_al__occurrences_3763.csv_sorted
[INFO] [2021-04-19 15:21:35] Converted: /app/public/converted_csv/morehouse_et_al__occurrences_3763.csv (1 lines)
[INFO] [2021-04-19 15:21:35] ...measurements (/app/public/data/morehouse_et_al_/measurementsorfacts.txt)
[CMD] [2021-04-19 15:21:35] /usr/bin/sort /app/public/converted_csv/morehouse_et_al__measurements_3763.csv > /app/public/converted_csv/morehouse_et_al__measurements_3763.csv_sorted
[INFO] [2021-04-19 15:21:35] Converted: /app/public/converted_csv/morehouse_et_al__measurements_3763.csv (30 lines)
[STOP] [2021-04-19 15:21:35] convert_to_csv
[START] [2021-04-19 15:21:35] calculate_delta
[INFO] [2021-04-19 15:21:35] Looping over 4 formats...
[INFO] [2021-04-19 15:21:35] ...refs (/app/public/data/morehouse_et_al_/references.tsv)
[CMD] [2021-04-19 15:21:35] echo "0a" > /app/public/diff/morehouse_et_al__refs_3763.diff
[CMD] [2021-04-19 15:21:35] tail -n +1 /app/public/converted_csv/morehouse_et_al__refs_3763.csv >> /app/public/diff/morehouse_et_al__refs_3763.diff
[CMD] [2021-04-19 15:21:35] echo "." >> /app/public/diff/morehouse_et_al__refs_3763.diff
[INFO] [2021-04-19 15:21:35] Created diff: /app/public/diff/morehouse_et_al__refs_3763.diff (2 lines)
[INFO] [2021-04-19 15:21:35] ...nodes (/app/public/data/morehouse_et_al_/taxa.txt)
[CMD] [2021-04-19 15:21:35] echo "0a" > /app/public/diff/morehouse_et_al__nodes_3763.diff
[CMD] [2021-04-19 15:21:35] tail -n +1 /app/public/converted_csv/morehouse_et_al__nodes_3763.csv >> /app/public/diff/morehouse_et_al__nodes_3763.diff
[CMD] [2021-04-19 15:21:35] echo "." >> /app/public/diff/morehouse_et_al__nodes_3763.diff
[INFO] [2021-04-19 15:21:35] Created diff: /app/public/diff/morehouse_et_al__nodes_3763.diff (3 lines)
[INFO] [2021-04-19 15:21:35] ...occurrences (/app/public/data/morehouse_et_al_/occurrences.txt)
[CMD] [2021-04-19 15:21:35] echo "0a" > /app/public/diff/morehouse_et_al__occurrences_3763.diff
[CMD] [2021-04-19 15:21:35] tail -n +1 /app/public/converted_csv/morehouse_et_al__occurrences_3763.csv >> /app/public/diff/morehouse_et_al__occurrences_3763.diff
[CMD] [2021-04-19 15:21:35] echo "." >> /app/public/diff/morehouse_et_al__occurrences_3763.diff
[INFO] [2021-04-19 15:21:35] Created diff: /app/public/diff/morehouse_et_al__occurrences_3763.diff (3 lines)
[INFO] [2021-04-19 15:21:35] ...measurements (/app/public/data/morehouse_et_al_/measurementsorfacts.txt)
[CMD] [2021-04-19 15:21:35] echo "0a" > /app/public/diff/morehouse_et_al__measurements_3763.diff
[CMD] [2021-04-19 15:21:35] tail -n +1 /app/public/converted_csv/morehouse_et_al__measurements_3763.csv >> /app/public/diff/morehouse_et_al__measurements_3763.diff
[CMD] [2021-04-19 15:21:35] echo "." >> /app/public/diff/morehouse_et_al__measurements_3763.diff
[INFO] [2021-04-19 15:21:35] Created diff: /app/public/diff/morehouse_et_al__measurements_3763.diff (32 lines)
[STOP] [2021-04-19 15:21:35] calculate_delta
[START] [2021-04-19 15:21:35] parse_diff_and_store
[INFO] [2021-04-19 15:21:35] Handling diff: /app/public/diff/morehouse_et_al__refs_3763.diff (2 lines)
[INFO] [2021-04-19 15:21:35] Loading refs diff file into memory (2 /app/public/diff/morehouse_et_al__refs_3763.diff lines)...
[INFO] [2021-04-19 15:21:35] Handling diff: /app/public/diff/morehouse_et_al__nodes_3763.diff (3 lines)
[INFO] [2021-04-19 15:21:35] Loading nodes diff file into memory (3 /app/public/diff/morehouse_et_al__nodes_3763.diff lines)...
[INFO] [2021-04-19 15:21:35] Handling diff: /app/public/diff/morehouse_et_al__occurrences_3763.diff (3 lines)
[INFO] [2021-04-19 15:21:35] Loading occurrences diff file into memory (3 /app/public/diff/morehouse_et_al__occurrences_3763.diff lines)...
[INFO] [2021-04-19 15:21:35] Handling diff: /app/public/diff/morehouse_et_al__measurements_3763.diff (32 lines)
[INFO] [2021-04-19 15:21:35] Loading measurements diff file into memory (32 /app/public/diff/morehouse_et_al__measurements_3763.diff lines)...
[INFO] [2021-04-19 15:21:35] Storing 1 ScientificNames
[INFO] [2021-04-19 15:21:35] Processing group of 1 in 1 groups of 1000
[INFO] [2021-04-19 15:21:35] Average Time: 0.0
[INFO] [2021-04-19 15:21:35] Total Time: 1s
[INFO] [2021-04-19 15:21:35] Storing 1 Nodes
[INFO] [2021-04-19 15:21:35] Processing group of 1 in 1 groups of 1000
[INFO] [2021-04-19 15:21:35] Average Time: 0.0
[INFO] [2021-04-19 15:21:35] Total Time: 1s
[INFO] [2021-04-19 15:21:35] Storing 1 Occurrences
[INFO] [2021-04-19 15:21:35] Processing group of 1 in 1 groups of 1000
[INFO] [2021-04-19 15:21:35] Average Time: 0.0
[INFO] [2021-04-19 15:21:35] Total Time: 1s
[INFO] [2021-04-19 15:21:35] Storing 30 Traits
[INFO] [2021-04-19 15:21:35] Processing group of 30 in 1 groups of 1000
[INFO] [2021-04-19 15:21:35] Average Time: 0.01
[INFO] [2021-04-19 15:21:35] Total Time: 1s
[INFO] [2021-04-19 15:21:35] Storing 4 MetaTraits
[INFO] [2021-04-19 15:21:35] Processing group of 4 in 1 groups of 1000
[INFO] [2021-04-19 15:21:35] Average Time: 0.0
[INFO] [2021-04-19 15:21:35] Total Time: 1s
[STOP] [2021-04-19 15:21:35] parse_diff_and_store
[START] [2021-04-19 15:21:35] resolve_keys
[INFO] [2021-04-19 15:21:41] Occurrences to nodes (through scientific_names)...
[INFO] [2021-04-19 15:21:41] traits to occurrences...
[INFO] [2021-04-19 15:21:41] traits to nodes (through occurrences)...
[INFO] [2021-04-19 15:21:41] Traits to sex term...
[INFO] [2021-04-19 15:21:41] Traits to lifestage term...
[INFO] [2021-04-19 15:21:41] MetaTraits to traits...
[INFO] [2021-04-19 15:21:41] MetaTraits (simple, measurement row refers to parent) to traits...
[INFO] [2021-04-19 15:21:41] Assocs to occurrences...
[INFO] [2021-04-19 15:21:41] Assocs to nodes...
[INFO] [2021-04-19 15:21:41] Assoc to sex term...
[INFO] [2021-04-19 15:21:41] Assoc to lifestage term...
[INFO] [2021-04-19 15:21:41] MetaAssoc to assocs...
[STOP] [2021-04-19 15:21:41] resolve_keys
[START] [2021-04-19 15:21:41] hold_for_later_1
[STOP] [2021-04-19 15:21:41] hold_for_later_1
[START] [2021-04-19 15:21:41] hold_for_later_2
[STOP] [2021-04-19 15:21:41] hold_for_later_2
[START] [2021-04-19 15:21:41] resolve_missing_parents
[STOP] [2021-04-19 15:21:41] resolve_missing_parents
[START] [2021-04-19 15:21:41] rebuild_nodes
[START] [2021-04-19 15:21:41] Flattener#flatten
[START] [2021-04-19 15:21:41] Flattener#study_resource
[START] [2021-04-19 15:21:41] Flattener#build_ancestry
[STOP] [2021-04-19 15:21:41] Flattener#build_ancestry
[INFO] [2021-04-19 15:21:41] 1 ancestry keys
[START] [2021-04-19 15:21:41] build_node_ancestors
[INFO] [2021-04-19 15:21:41] old ancestors deleted.
[STOP] [2021-04-19 15:21:41] build_node_ancestors
[WARN] [2021-04-19 15:21:41] Flattener: nothing to flatten! (Completely flat resource?)
[STOP] [2021-04-19 15:21:41] Flattener#flatten
[STOP] [2021-04-19 15:21:41] rebuild_nodes
[START] [2021-04-19 15:21:41] resolve_missing_media_owners
[STOP] [2021-04-19 15:21:41] resolve_missing_media_owners
[START] [2021-04-19 15:21:41] sanitize_media_verbatims
[STOP] [2021-04-19 15:21:41] sanitize_media_verbatims
[START] [2021-04-19 15:21:41] queue_downloads
[STOP] [2021-04-19 15:21:41] queue_downloads
[START] [2021-04-19 15:21:41] parse_names
[WARN] [2021-04-19 15:21:41] I see 1 names which still need to be parsed.
[STOP] [2021-04-19 15:21:42] parse_names
[START] [2021-04-19 15:21:42] denormalize_canonical_names_to_nodes
[STOP] [2021-04-19 15:21:42] denormalize_canonical_names_to_nodes
[START] [2021-04-19 15:21:42] match_nodes
[START] [2021-04-19 15:21:42] map_all_nodes_to_pages
[STOP] [2021-04-19 15:21:42] map_all_nodes_to_pages
[INFO] [2021-04-19 15:21:42] ZERO unmatched nodes (of 1)! Nicely done.
[START] [2021-04-19 15:21:42] update_nodes
[STOP] [2021-04-19 15:21:42] update_nodes
[STOP] [2021-04-19 15:21:42] match_nodes
[START] [2021-04-19 15:21:42] reindex_search
[STOP] [2021-04-19 15:21:42] reindex_search
[START] [2021-04-19 15:21:42] normalize_units
[STOP] [2021-04-19 15:21:42] normalize_units
[START] [2021-04-19 15:21:42] calculate_statistics
[2021-04-19 15:21:42] ZERO NODE ANCESTORS. Is this actually a completely flat resource?
[STOP] [2021-04-19 15:21:42] calculate_statistics
[START] [2021-04-19 15:21:42] complete_harvest_instance
[START] [2021-04-19 15:21:42] overall_tsv_creation
[INFO] [2021-04-19 15:21:42] Processing group of 1 in 1 batches of 10000
[INFO] [2021-04-19 15:22:18] 2 Traits (unfiltered)...
[INFO] [2021-04-19 15:22:50] 2 Traits (filtered)...
[INFO] [2021-04-19 15:22:50] 0 Associations (filtered)...
[INFO] [2021-04-19 15:22:50] 28 metadata added.
[INFO] [2021-04-19 15:22:50] 0 metadata added.
[INFO] [2021-04-19 15:23:16] Average Time: 70.01
[INFO] [2021-04-19 15:23:16] Total Time: 1m34s
[STOP] [2021-04-19 15:23:16] overall_tsv_creation
[INFO] [2021-04-19 15:23:16] Done. Check your files:
[INFO] [2021-04-19 15:23:16] (1 lines) /app/public/data/morehouse_et_al_/publish_nodes.tsv
[INFO] [2021-04-19 15:23:16] (1 lines) /app/public/data/morehouse_et_al_/publish_scientific_names.tsv
[INFO] [2021-04-19 15:23:16] (3 lines) /app/public/data/morehouse_et_al_/publish_traits.tsv
[INFO] [2021-04-19 15:23:16] (29 lines) /app/public/data/morehouse_et_al_/publish_metadata.tsv
[STOP] [2021-04-19 15:23:16] complete_harvest_instance
[START] [2021-04-19 15:23:16] completed
[STOP] [2021-04-19 15:23:16] completed
[STOP] [2021-04-19 15:23:16] logged process, took 101.68
Latest Process