Harvest for Bone and James 1993 Created 19 Apr 14:39

Stage: completed
Fetched: 19 Apr 14:39
Validated: 19 Apr 14:39
Deltas Created 19 Apr 14:39
Units Normalized: 19 Apr 14:39
Ancestry Built: 19 Apr 14:39
Nodes Matched: 19 Apr 14:39
Names Parsed: 19 Apr 14:39
New Models Stored: 19 Apr 14:39
Indexed: 19 Apr 14:39
Completed: 19 Apr 14:41
Time to Harvest: less than a minute

Harvesting Log

(908 lines)
# Logfile created on 2020-04-06 09:16:49 -0400 by logger.rb/56815
[START] [2020-04-06 09:16:49] logged process
[START] [2020-04-06 09:16:49] create_harvest_instance
[STOP] [2020-04-06 09:16:50] create_harvest_instance
[START] [2020-04-06 09:16:50] fetch_files
[STOP] [2020-04-06 09:16:50] fetch_files
[START] [2020-04-06 09:16:50] validate_each_file
[STOP] [2020-04-06 09:16:50] validate_each_file
[START] [2020-04-06 09:16:50] convert_to_csv
[CMD] [2020-04-06 09:16:50] /usr/bin/sort /app/public/converted_csv/bone_james_bone__nodes_20648.csv > /app/public/converted_csv/bone_james_bone__nodes_20648.csv_sorted
[CMD] [2020-04-06 09:16:50] /usr/bin/sort /app/public/converted_csv/bone_james_bone__occurrences_20649.csv > /app/public/converted_csv/bone_james_bone__occurrences_20649.csv_sorted
[CMD] [2020-04-06 09:16:50] /usr/bin/sort /app/public/converted_csv/bone_james_bone__measurements_20650.csv > /app/public/converted_csv/bone_james_bone__measurements_20650.csv_sorted
[STOP] [2020-04-06 09:16:50] convert_to_csv
[START] [2020-04-06 09:16:50] calculate_delta
[CMD] [2020-04-06 09:16:50] echo "0a" > /app/public/diff/bone_james_bone__nodes_20648.diff
[CMD] [2020-04-06 09:16:50] tail -n +1 /app/public/converted_csv/bone_james_bone__nodes_20648.csv >> /app/public/diff/bone_james_bone__nodes_20648.diff
[CMD] [2020-04-06 09:16:50] echo "." >> /app/public/diff/bone_james_bone__nodes_20648.diff
[CMD] [2020-04-06 09:16:50] echo "0a" > /app/public/diff/bone_james_bone__occurrences_20649.diff
[CMD] [2020-04-06 09:16:50] tail -n +1 /app/public/converted_csv/bone_james_bone__occurrences_20649.csv >> /app/public/diff/bone_james_bone__occurrences_20649.diff
[CMD] [2020-04-06 09:16:50] echo "." >> /app/public/diff/bone_james_bone__occurrences_20649.diff
[CMD] [2020-04-06 09:16:50] echo "0a" > /app/public/diff/bone_james_bone__measurements_20650.diff
[CMD] [2020-04-06 09:16:50] tail -n +1 /app/public/converted_csv/bone_james_bone__measurements_20650.csv >> /app/public/diff/bone_james_bone__measurements_20650.diff
[CMD] [2020-04-06 09:16:50] echo "." >> /app/public/diff/bone_james_bone__measurements_20650.diff
[STOP] [2020-04-06 09:16:50] calculate_delta
[START] [2020-04-06 09:16:50] parse_diff_and_store
[INFO] [2020-04-06 09:16:50] Loading nodes diff file into memory (true lines)...
[INFO] [2020-04-06 09:16:50] Loading occurrences diff file into memory (true lines)...
[INFO] [2020-04-06 09:16:50] Loading measurements diff file into memory (true lines)...
[INFO] [2020-04-06 09:16:50] Storing 1 ScientificNames
[INFO] [2020-04-06 09:16:50] Processing group of 1 in 1 groups of 1000
[INFO] [2020-04-06 09:16:50] Average Time: 0.0
[INFO] [2020-04-06 09:16:50] Total Time: 1s
[INFO] [2020-04-06 09:16:50] Storing 1 Nodes
[INFO] [2020-04-06 09:16:50] Processing group of 1 in 1 groups of 1000
[INFO] [2020-04-06 09:16:50] Average Time: 0.0
[INFO] [2020-04-06 09:16:50] Total Time: 1s
[INFO] [2020-04-06 09:16:50] Storing 1 Occurrences
[INFO] [2020-04-06 09:16:50] Processing group of 1 in 1 groups of 1000
[INFO] [2020-04-06 09:16:50] Average Time: 0.0
[INFO] [2020-04-06 09:16:50] Total Time: 1s
[INFO] [2020-04-06 09:16:50] Storing 3 Traits
[INFO] [2020-04-06 09:16:50] Processing group of 3 in 1 groups of 1000
[INFO] [2020-04-06 09:16:50] Average Time: 0.0
[INFO] [2020-04-06 09:16:50] Total Time: 1s
[INFO] [2020-04-06 09:16:50] Storing 3 MetaTraits
[INFO] [2020-04-06 09:16:50] Processing group of 3 in 1 groups of 1000
[INFO] [2020-04-06 09:16:50] Average Time: 0.0
[INFO] [2020-04-06 09:16:50] Total Time: 1s
[STOP] [2020-04-06 09:16:50] parse_diff_and_store
[START] [2020-04-06 09:16:50] resolve_keys
[INFO] [2020-04-06 09:16:55] Occurrences to nodes (through scientific_names)...
[INFO] [2020-04-06 09:16:55] traits to occurrences...
[INFO] [2020-04-06 09:16:55] traits to nodes (through occurrences)...
[INFO] [2020-04-06 09:16:55] Traits to sex term...
[INFO] [2020-04-06 09:16:55] Traits to lifestage term...
[INFO] [2020-04-06 09:16:55] MetaTraits to traits...
[INFO] [2020-04-06 09:16:55] MetaTraits (simple, measurement row refers to parent) to traits...
[INFO] [2020-04-06 09:16:55] Assocs to occurrences...
[INFO] [2020-04-06 09:16:55] Assocs to nodes...
[INFO] [2020-04-06 09:16:55] Assoc to sex term...
[INFO] [2020-04-06 09:16:55] Assoc to lifestage term...
[STOP] [2020-04-06 09:16:55] resolve_keys
[START] [2020-04-06 09:16:55] hold_for_later_1
[STOP] [2020-04-06 09:16:55] hold_for_later_1
[START] [2020-04-06 09:16:55] hold_for_later_2
[STOP] [2020-04-06 09:16:55] hold_for_later_2
[START] [2020-04-06 09:16:55] resolve_missing_parents
[STOP] [2020-04-06 09:16:55] resolve_missing_parents
[START] [2020-04-06 09:16:55] rebuild_nodes
[START] [2020-04-06 09:16:55] Flattener#flatten
[START] [2020-04-06 09:16:55] Flattener#study_resource
[START] [2020-04-06 09:16:55] Flattener#build_ancestry
[STOP] [2020-04-06 09:16:55] Flattener#build_ancestry
[INFO] [2020-04-06 09:16:55] 1 ancestry keys
[START] [2020-04-06 09:16:55] build_node_ancestors
[INFO] [2020-04-06 09:16:55] old ancestors deleted.
[STOP] [2020-04-06 09:16:55] build_node_ancestors
[WARN] [2020-04-06 09:16:55] Flattener: nothing to flatten! (Completely flat resource?)
[STOP] [2020-04-06 09:16:55] Flattener#flatten
[STOP] [2020-04-06 09:16:55] rebuild_nodes
[START] [2020-04-06 09:16:55] resolve_missing_media_owners
[STOP] [2020-04-06 09:16:55] resolve_missing_media_owners
[START] [2020-04-06 09:16:55] sanitize_media_verbatims
[STOP] [2020-04-06 09:16:55] sanitize_media_verbatims
[START] [2020-04-06 09:16:55] queue_downloads
[STOP] [2020-04-06 09:16:55] queue_downloads
[START] [2020-04-06 09:16:55] parse_names
[WARN] [2020-04-06 09:16:55] I see 1 names which still need to be parsed.
[STOP] [2020-04-06 09:16:56] parse_names
[START] [2020-04-06 09:16:56] denormalize_canonical_names_to_nodes
[STOP] [2020-04-06 09:16:56] denormalize_canonical_names_to_nodes
[START] [2020-04-06 09:16:56] match_nodes
[START] [2020-04-06 09:16:56] map_all_nodes_to_pages
[STOP] [2020-04-06 09:16:56] map_all_nodes_to_pages
[INFO] [2020-04-06 09:16:56] ZERO unmatched nodes (of 1)! Nicely done.
[START] [2020-04-06 09:16:56] update_nodes
[STOP] [2020-04-06 09:16:56] update_nodes
[STOP] [2020-04-06 09:16:56] match_nodes
[START] [2020-04-06 09:16:56] reindex_search
[STOP] [2020-04-06 09:16:57] reindex_search
[START] [2020-04-06 09:16:57] normalize_units
[STOP] [2020-04-06 09:16:57] normalize_units
[START] [2020-04-06 09:16:57] calculate_statistics
[2020-04-06 09:16:57] ZERO NODE ANCESTORS. Is this actually a completely flat resource?
[STOP] [2020-04-06 09:16:57] calculate_statistics
[START] [2020-04-06 09:16:57] complete_harvest_instance
[START] [2020-04-06 09:16:57] overall_tsv_creation
[INFO] [2020-04-06 09:16:57] Processing group of 1 in 1 batches of 10000
[INFO] [2020-04-06 09:18:15] 1 Traits (unfiltered)...
[INFO] [2020-04-06 09:18:26] 1 Traits (filtered)...
[INFO] [2020-04-06 09:18:26] 0 Associations (filtered)...
[INFO] [2020-04-06 09:19:00] 5 metadata added.
[INFO] [2020-04-06 09:19:00] 0 metadata added.
[INFO] [2020-04-06 09:19:00] Average Time: 66.19
[INFO] [2020-04-06 09:19:00] Total Time: 2m4s
[STOP] [2020-04-06 09:19:00] overall_tsv_creation
[INFO] [2020-04-06 09:19:00] Done. Check your files:
[INFO] [2020-04-06 09:19:00] (1 lines) /app/public/data/bone_james_bone_/publish_nodes.tsv
[INFO] [2020-04-06 09:19:00] (1 lines) /app/public/data/bone_james_bone_/publish_scientific_names.tsv
[INFO] [2020-04-06 09:19:00] (2 lines) /app/public/data/bone_james_bone_/publish_traits.tsv
[INFO] [2020-04-06 09:19:00] (6 lines) /app/public/data/bone_james_bone_/publish_metadata.tsv
[STOP] [2020-04-06 09:19:00] complete_harvest_instance
[START] [2020-04-06 09:19:00] completed
[STOP] [2020-04-06 09:19:00] completed
[STOP] [2020-04-06 09:19:00] logged process, took 131.28
[INFO] [2020-06-16 13:45:36] ## HARVEST: type = re_download_opendata_-harvest
[INFO] [2020-06-16 13:45:36] ## remove_type: ScientificName
[INFO] [2020-06-16 13:45:36] ++ Calling delete_all on 1 instances...
[INFO] [2020-06-16 13:45:36] [13:45:36.416] Removed 1 Scientificnames
[INFO] [2020-06-16 13:45:36] ## remove_type: Vernacular
[INFO] [2020-06-16 13:45:36] ++ Calling delete_all on 0 instances...
[INFO] [2020-06-16 13:45:36] [13:45:36.419] Removed 0 Vernaculars
[INFO] [2020-06-16 13:45:36] ## remove_type: Article
[INFO] [2020-06-16 13:45:36] ++ Calling delete_all on 0 instances...
[INFO] [2020-06-16 13:45:36] [13:45:36.422] Removed 0 Articles
[INFO] [2020-06-16 13:45:36] ## remove_type: Medium
[INFO] [2020-06-16 13:45:36] ++ Calling delete_all on 0 instances...
[INFO] [2020-06-16 13:45:36] [13:45:36.426] Removed 0 Media
[INFO] [2020-06-16 13:45:36] ## remove_type: Trait
[INFO] [2020-06-16 13:45:36] ++ Calling delete_all on 3 instances...
[INFO] [2020-06-16 13:45:36] [13:45:36.429] Removed 3 Traits
[INFO] [2020-06-16 13:45:36] ## remove_type: MetaTrait
[INFO] [2020-06-16 13:45:36] ++ Calling delete_all on 3 instances...
[INFO] [2020-06-16 13:45:36] [13:45:36.432] Removed 3 Metatraits
[INFO] [2020-06-16 13:45:36] ## remove_type: OccurrenceMetadatum
[INFO] [2020-06-16 13:45:36] ++ Calling delete_all on 0 instances...
[INFO] [2020-06-16 13:45:36] [13:45:36.434] Removed 0 Occurrencemetadata
[INFO] [2020-06-16 13:45:36] ## remove_type: Assoc
[INFO] [2020-06-16 13:45:36] ++ Calling delete_all on 0 instances...
[INFO] [2020-06-16 13:45:36] [13:45:36.437] Removed 0 Assocs
[INFO] [2020-06-16 13:45:36] ## remove_type: MetaAssoc
[INFO] [2020-06-16 13:45:36] ++ Calling delete_all on 0 instances...
[INFO] [2020-06-16 13:45:36] [13:45:36.439] Removed 0 Metaassocs
[INFO] [2020-06-16 13:45:36] ## remove_type: Identifier
[INFO] [2020-06-16 13:45:36] ++ Calling delete_all on 0 instances...
[INFO] [2020-06-16 13:45:36] [13:45:36.441] Removed 0 Identifiers
[INFO] [2020-06-16 13:45:36] ## remove_type: Reference
[INFO] [2020-06-16 13:45:36] ++ Calling delete_all on 0 instances...
[INFO] [2020-06-16 13:45:36] [13:45:36.444] Removed 0 References
[INFO] [2020-06-16 13:45:36] Starting batch with ID 68185453...
[INFO] [2020-06-16 13:45:36] Starting batch with ID 68185453...
[INFO] [2020-06-16 13:45:36] Starting batch with ID 68185453...
[INFO] [2020-06-16 13:45:36] Starting batch with ID 68185453...
[INFO] [2020-06-16 13:45:36] Starting batch with ID 68185453...
[INFO] [2020-06-16 13:45:36] Starting batch with ID 68185453...
[INFO] [2020-06-16 13:45:36] Starting batch with ID 68185453...
[INFO] [2020-06-16 13:45:36] Starting batch with ID 68185453...
[INFO] [2020-06-16 13:45:36] Starting batch with ID 68185453...
[INFO] [2020-06-16 13:45:36] Starting batch with ID 68185453...
[INFO] [2020-06-16 13:45:36] Starting batch with ID 68185453...
[INFO] [2020-06-16 13:45:36] Starting batch with ID 68185453...
[INFO] [2020-06-16 13:45:36] ## remove_type: Node
[INFO] [2020-06-16 13:45:36] ++ Calling delete_all on 1 instances...
[INFO] [2020-06-16 13:45:36] [13:45:36.634] Removed 1 Nodes
[START] [2020-06-16 13:45:37] logged process
[START] [2020-06-16 13:45:37] Creating resource from OpenData
[START] [2020-06-16 13:45:37] logged process
[START] [2020-06-16 13:45:37] Parse meta.xml file and create formats with fields
[STOP] [2020-06-16 13:45:37] Parse meta.xml file and create formats with fields
[STOP] [2020-06-16 13:45:37] Creating resource from OpenData
[START] [2020-06-16 13:45:37] logged process
[START] [2020-06-16 13:45:37] create_harvest_instance
[STOP] [2020-06-16 13:45:38] create_harvest_instance
[START] [2020-06-16 13:45:38] fetch_files
[STOP] [2020-06-16 13:45:38] fetch_files
[START] [2020-06-16 13:45:38] validate_each_file
[STOP] [2020-06-16 13:45:38] validate_each_file
[START] [2020-06-16 13:45:38] convert_to_csv
[CMD] [2020-06-16 13:45:38] /usr/bin/sort /app/public/converted_csv/bone_james_bone__nodes_21362.csv > /app/public/converted_csv/bone_james_bone__nodes_21362.csv_sorted
[CMD] [2020-06-16 13:45:38] /usr/bin/sort /app/public/converted_csv/bone_james_bone__occurrences_21363.csv > /app/public/converted_csv/bone_james_bone__occurrences_21363.csv_sorted
[CMD] [2020-06-16 13:45:38] /usr/bin/sort /app/public/converted_csv/bone_james_bone__measurements_21364.csv > /app/public/converted_csv/bone_james_bone__measurements_21364.csv_sorted
[STOP] [2020-06-16 13:45:38] convert_to_csv
[START] [2020-06-16 13:45:38] calculate_delta
[CMD] [2020-06-16 13:45:38] echo "0a" > /app/public/diff/bone_james_bone__nodes_21362.diff
[CMD] [2020-06-16 13:45:38] tail -n +1 /app/public/converted_csv/bone_james_bone__nodes_21362.csv >> /app/public/diff/bone_james_bone__nodes_21362.diff
[CMD] [2020-06-16 13:45:38] echo "." >> /app/public/diff/bone_james_bone__nodes_21362.diff
[CMD] [2020-06-16 13:45:38] echo "0a" > /app/public/diff/bone_james_bone__occurrences_21363.diff
[CMD] [2020-06-16 13:45:38] tail -n +1 /app/public/converted_csv/bone_james_bone__occurrences_21363.csv >> /app/public/diff/bone_james_bone__occurrences_21363.diff
[CMD] [2020-06-16 13:45:38] echo "." >> /app/public/diff/bone_james_bone__occurrences_21363.diff
[CMD] [2020-06-16 13:45:38] echo "0a" > /app/public/diff/bone_james_bone__measurements_21364.diff
[CMD] [2020-06-16 13:45:38] tail -n +1 /app/public/converted_csv/bone_james_bone__measurements_21364.csv >> /app/public/diff/bone_james_bone__measurements_21364.diff
[CMD] [2020-06-16 13:45:38] echo "." >> /app/public/diff/bone_james_bone__measurements_21364.diff
[STOP] [2020-06-16 13:45:38] calculate_delta
[START] [2020-06-16 13:45:38] parse_diff_and_store
[INFO] [2020-06-16 13:45:38] Loading nodes diff file into memory (true lines)...
[INFO] [2020-06-16 13:45:38] Loading occurrences diff file into memory (true lines)...
[INFO] [2020-06-16 13:45:38] Loading measurements diff file into memory (true lines)...
[INFO] [2020-06-16 13:45:38] Storing 1 ScientificNames
[INFO] [2020-06-16 13:45:38] Processing group of 1 in 1 groups of 1000
[INFO] [2020-06-16 13:45:38] Average Time: 0.0
[INFO] [2020-06-16 13:45:38] Total Time: 1s
[INFO] [2020-06-16 13:45:38] Storing 1 Nodes
[INFO] [2020-06-16 13:45:38] Processing group of 1 in 1 groups of 1000
[INFO] [2020-06-16 13:45:38] Average Time: 0.0
[INFO] [2020-06-16 13:45:38] Total Time: 1s
[INFO] [2020-06-16 13:45:38] Storing 1 Occurrences
[INFO] [2020-06-16 13:45:38] Processing group of 1 in 1 groups of 1000
[INFO] [2020-06-16 13:45:38] Average Time: 0.0
[INFO] [2020-06-16 13:45:38] Total Time: 1s
[INFO] [2020-06-16 13:45:38] Storing 3 Traits
[INFO] [2020-06-16 13:45:38] Processing group of 3 in 1 groups of 1000
[INFO] [2020-06-16 13:45:38] Average Time: 0.0
[INFO] [2020-06-16 13:45:38] Total Time: 1s
[INFO] [2020-06-16 13:45:38] Storing 3 MetaTraits
[INFO] [2020-06-16 13:45:38] Processing group of 3 in 1 groups of 1000
[INFO] [2020-06-16 13:45:38] Average Time: 0.0
[INFO] [2020-06-16 13:45:38] Total Time: 1s
[STOP] [2020-06-16 13:45:38] parse_diff_and_store
[START] [2020-06-16 13:45:38] resolve_keys
[INFO] [2020-06-16 13:45:45] Occurrences to nodes (through scientific_names)...
[INFO] [2020-06-16 13:45:45] traits to occurrences...
[INFO] [2020-06-16 13:45:45] traits to nodes (through occurrences)...
[INFO] [2020-06-16 13:45:45] Traits to sex term...
[INFO] [2020-06-16 13:45:45] Traits to lifestage term...
[INFO] [2020-06-16 13:45:45] MetaTraits to traits...
[INFO] [2020-06-16 13:45:45] MetaTraits (simple, measurement row refers to parent) to traits...
[INFO] [2020-06-16 13:45:45] Assocs to occurrences...
[INFO] [2020-06-16 13:45:45] Assocs to nodes...
[INFO] [2020-06-16 13:45:45] Assoc to sex term...
[INFO] [2020-06-16 13:45:45] Assoc to lifestage term...
[STOP] [2020-06-16 13:45:45] resolve_keys
[START] [2020-06-16 13:45:45] hold_for_later_1
[STOP] [2020-06-16 13:45:45] hold_for_later_1
[START] [2020-06-16 13:45:45] hold_for_later_2
[STOP] [2020-06-16 13:45:45] hold_for_later_2
[START] [2020-06-16 13:45:45] resolve_missing_parents
[STOP] [2020-06-16 13:45:45] resolve_missing_parents
[START] [2020-06-16 13:45:45] rebuild_nodes
[START] [2020-06-16 13:45:45] Flattener#flatten
[START] [2020-06-16 13:45:45] Flattener#study_resource
[START] [2020-06-16 13:45:45] Flattener#build_ancestry
[STOP] [2020-06-16 13:45:45] Flattener#build_ancestry
[INFO] [2020-06-16 13:45:45] 1 ancestry keys
[START] [2020-06-16 13:45:45] build_node_ancestors
[INFO] [2020-06-16 13:45:45] old ancestors deleted.
[STOP] [2020-06-16 13:45:45] build_node_ancestors
[WARN] [2020-06-16 13:45:45] Flattener: nothing to flatten! (Completely flat resource?)
[STOP] [2020-06-16 13:45:45] Flattener#flatten
[STOP] [2020-06-16 13:45:45] rebuild_nodes
[START] [2020-06-16 13:45:45] resolve_missing_media_owners
[STOP] [2020-06-16 13:45:45] resolve_missing_media_owners
[START] [2020-06-16 13:45:45] sanitize_media_verbatims
[STOP] [2020-06-16 13:45:45] sanitize_media_verbatims
[START] [2020-06-16 13:45:45] queue_downloads
[STOP] [2020-06-16 13:45:45] queue_downloads
[START] [2020-06-16 13:45:45] parse_names
[WARN] [2020-06-16 13:45:45] I see 1 names which still need to be parsed.
[STOP] [2020-06-16 13:45:46] parse_names
[START] [2020-06-16 13:45:46] denormalize_canonical_names_to_nodes
[STOP] [2020-06-16 13:45:46] denormalize_canonical_names_to_nodes
[START] [2020-06-16 13:45:46] match_nodes
[START] [2020-06-16 13:45:46] map_all_nodes_to_pages
[STOP] [2020-06-16 13:45:46] map_all_nodes_to_pages
[INFO] [2020-06-16 13:45:46] ZERO unmatched nodes (of 1)! Nicely done.
[START] [2020-06-16 13:45:46] update_nodes
[STOP] [2020-06-16 13:45:46] update_nodes
[STOP] [2020-06-16 13:45:46] match_nodes
[START] [2020-06-16 13:45:46] reindex_search
[STOP] [2020-06-16 13:45:46] reindex_search
[START] [2020-06-16 13:45:46] normalize_units
[STOP] [2020-06-16 13:45:46] normalize_units
[START] [2020-06-16 13:45:46] calculate_statistics
[2020-06-16 13:45:46] ZERO NODE ANCESTORS. Is this actually a completely flat resource?
[STOP] [2020-06-16 13:45:46] calculate_statistics
[START] [2020-06-16 13:45:46] complete_harvest_instance
[START] [2020-06-16 13:45:46] overall_tsv_creation
[INFO] [2020-06-16 13:45:46] Processing group of 1 in 1 batches of 10000
[INFO] [2020-06-16 13:46:37] 1 Traits (unfiltered)...
[INFO] [2020-06-16 13:46:48] 1 Traits (filtered)...
[INFO] [2020-06-16 13:46:48] 0 Associations (filtered)...
[INFO] [2020-06-16 13:47:24] 5 metadata added.
[INFO] [2020-06-16 13:47:24] 0 metadata added.
[INFO] [2020-06-16 13:47:24] Average Time: 69.09
[INFO] [2020-06-16 13:47:24] Total Time: 1m38s
[STOP] [2020-06-16 13:47:24] overall_tsv_creation
[INFO] [2020-06-16 13:47:24] Done. Check your files:
[INFO] [2020-06-16 13:47:24] (1 lines) /app/public/data/bone_james_bone_/publish_nodes.tsv
[INFO] [2020-06-16 13:47:24] (1 lines) /app/public/data/bone_james_bone_/publish_scientific_names.tsv
[INFO] [2020-06-16 13:47:24] (2 lines) /app/public/data/bone_james_bone_/publish_traits.tsv
[INFO] [2020-06-16 13:47:24] (6 lines) /app/public/data/bone_james_bone_/publish_metadata.tsv
[STOP] [2020-06-16 13:47:24] complete_harvest_instance
[START] [2020-06-16 13:47:24] completed
[STOP] [2020-06-16 13:47:24] completed
[STOP] [2020-06-16 13:47:24] logged process, took 107.16
[INFO] [2020-11-13 11:37:02] ## HARVEST: type = re_download_opendata_-harvest
[INFO] [2020-11-13 11:37:06] ## remove_type: ScientificName
[INFO] [2020-11-13 11:37:06] ++ Calling delete_all on 1 instances...
[INFO] [2020-11-13 11:37:06] [11:37:06.762] Removed 1 Scientificnames
[INFO] [2020-11-13 11:37:06] ## remove_type: Vernacular
[INFO] [2020-11-13 11:37:06] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-13 11:37:06] [11:37:06.764] Removed 0 Vernaculars
[INFO] [2020-11-13 11:37:06] ## remove_type: Article
[INFO] [2020-11-13 11:37:06] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-13 11:37:06] [11:37:06.767] Removed 0 Articles
[INFO] [2020-11-13 11:37:06] ## remove_type: Medium
[INFO] [2020-11-13 11:37:06] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-13 11:37:06] [11:37:06.770] Removed 0 Media
[INFO] [2020-11-13 11:37:06] ## remove_type: Trait
[INFO] [2020-11-13 11:37:06] ++ Calling delete_all on 3 instances...
[INFO] [2020-11-13 11:37:06] [11:37:06.774] Removed 3 Traits
[INFO] [2020-11-13 11:37:06] ## remove_type: MetaTrait
[INFO] [2020-11-13 11:37:06] ++ Calling delete_all on 3 instances...
[INFO] [2020-11-13 11:37:06] [11:37:06.777] Removed 3 Metatraits
[INFO] [2020-11-13 11:37:06] ## remove_type: OccurrenceMetadatum
[INFO] [2020-11-13 11:37:06] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-13 11:37:06] [11:37:06.780] Removed 0 Occurrencemetadata
[INFO] [2020-11-13 11:37:06] ## remove_type: Assoc
[INFO] [2020-11-13 11:37:06] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-13 11:37:06] [11:37:06.783] Removed 0 Assocs
[INFO] [2020-11-13 11:37:06] ## remove_type: MetaAssoc
[INFO] [2020-11-13 11:37:06] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-13 11:37:06] [11:37:06.785] Removed 0 Metaassocs
[INFO] [2020-11-13 11:37:06] ## remove_type: Identifier
[INFO] [2020-11-13 11:37:06] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-13 11:37:06] [11:37:06.788] Removed 0 Identifiers
[INFO] [2020-11-13 11:37:06] ## remove_type: Reference
[INFO] [2020-11-13 11:37:06] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-13 11:37:06] [11:37:06.790] Removed 0 References
[INFO] [2020-11-13 11:37:06] Starting batch with ID 80104443...
[INFO] [2020-11-13 11:37:06] Starting batch with ID 80104443...
[INFO] [2020-11-13 11:37:06] Starting batch with ID 80104443...
[INFO] [2020-11-13 11:37:06] Starting batch with ID 80104443...
[INFO] [2020-11-13 11:37:06] Starting batch with ID 80104443...
[INFO] [2020-11-13 11:37:06] Starting batch with ID 80104443...
[INFO] [2020-11-13 11:37:06] Starting batch with ID 80104443...
[INFO] [2020-11-13 11:37:06] Starting batch with ID 80104443...
[INFO] [2020-11-13 11:37:07] Starting batch with ID 80104443...
[INFO] [2020-11-13 11:37:07] Starting batch with ID 80104443...
[INFO] [2020-11-13 11:37:07] Starting batch with ID 80104443...
[INFO] [2020-11-13 11:37:07] Starting batch with ID 80104443...
[INFO] [2020-11-13 11:37:07] Starting batch with ID 80104443...
[INFO] [2020-11-13 11:37:07] Starting batch with ID 80104443...
[INFO] [2020-11-13 11:37:07] Starting batch with ID 80104443...
[INFO] [2020-11-13 11:37:07] Starting batch with ID 80104443...
[INFO] [2020-11-13 11:37:07] Starting batch with ID 80104443...
[INFO] [2020-11-13 11:37:07] Starting batch with ID 80104443...
[INFO] [2020-11-13 11:37:07] Starting batch with ID 80104443...
[INFO] [2020-11-13 11:37:07] Starting batch with ID 80104443...
[INFO] [2020-11-13 11:37:07] Starting batch with ID 80104443...
[INFO] [2020-11-13 11:37:07] Starting batch with ID 80104443...
[INFO] [2020-11-13 11:37:07] Starting batch with ID 80104443...
[INFO] [2020-11-13 11:37:07] Starting batch with ID 80104443...
[INFO] [2020-11-13 11:37:07] Starting batch with ID 80104443...
[INFO] [2020-11-13 11:37:07] Starting batch with ID 80104443...
[INFO] [2020-11-13 11:37:07] Starting batch with ID 80104443...
[INFO] [2020-11-13 11:37:07] Starting batch with ID 80104443...
[INFO] [2020-11-13 11:37:07] Starting batch with ID 80104443...
[INFO] [2020-11-13 11:37:07] Starting batch with ID 80104443...
[INFO] [2020-11-13 11:37:07] Starting batch with ID 80104443...
[INFO] [2020-11-13 11:37:07] Starting batch with ID 80104443...
[INFO] [2020-11-13 11:37:07] Starting batch with ID 80104443...
[INFO] [2020-11-13 11:37:07] Starting batch with ID 80104443...
[INFO] [2020-11-13 11:37:07] ## remove_type: Node
[INFO] [2020-11-13 11:37:07] ++ Calling delete_all on 1 instances...
[INFO] [2020-11-13 11:37:07] [11:37:07.871] Removed 1 Nodes
[START] [2020-11-13 11:37:08] logged process: 58bbc42b01abb4c1b2698de049792ffb4b63b979

[START] [2020-11-13 11:37:08] Creating resource from OpenData
[START] [2020-11-13 11:37:08] logged process: 58bbc42b01abb4c1b2698de049792ffb4b63b979

[START] [2020-11-13 11:37:08] Parse meta.xml file and create formats with fields
[STOP] [2020-11-13 11:37:08] Parse meta.xml file and create formats with fields
[STOP] [2020-11-13 11:37:08] Creating resource from OpenData
[START] [2020-11-13 11:37:08] logged process: 58bbc42b01abb4c1b2698de049792ffb4b63b979

[START] [2020-11-13 11:37:08] create_harvest_instance
[STOP] [2020-11-13 11:37:10] create_harvest_instance
[START] [2020-11-13 11:37:10] fetch_files
[STOP] [2020-11-13 11:37:10] fetch_files
[START] [2020-11-13 11:37:10] validate_each_file
[STOP] [2020-11-13 11:37:10] validate_each_file
[START] [2020-11-13 11:37:10] convert_to_csv
[CMD] [2020-11-13 11:37:10] /usr/bin/sort /app/public/converted_csv/bone_james_bone__nodes_24169.csv > /app/public/converted_csv/bone_james_bone__nodes_24169.csv_sorted
[CMD] [2020-11-13 11:37:10] /usr/bin/sort /app/public/converted_csv/bone_james_bone__occurrences_24170.csv > /app/public/converted_csv/bone_james_bone__occurrences_24170.csv_sorted
[CMD] [2020-11-13 11:37:10] /usr/bin/sort /app/public/converted_csv/bone_james_bone__measurements_24171.csv > /app/public/converted_csv/bone_james_bone__measurements_24171.csv_sorted
[STOP] [2020-11-13 11:37:10] convert_to_csv
[START] [2020-11-13 11:37:10] calculate_delta
[CMD] [2020-11-13 11:37:10] echo "0a" > /app/public/diff/bone_james_bone__nodes_24169.diff
[CMD] [2020-11-13 11:37:10] tail -n +1 /app/public/converted_csv/bone_james_bone__nodes_24169.csv >> /app/public/diff/bone_james_bone__nodes_24169.diff
[CMD] [2020-11-13 11:37:10] echo "." >> /app/public/diff/bone_james_bone__nodes_24169.diff
[CMD] [2020-11-13 11:37:10] echo "0a" > /app/public/diff/bone_james_bone__occurrences_24170.diff
[CMD] [2020-11-13 11:37:10] tail -n +1 /app/public/converted_csv/bone_james_bone__occurrences_24170.csv >> /app/public/diff/bone_james_bone__occurrences_24170.diff
[CMD] [2020-11-13 11:37:10] echo "." >> /app/public/diff/bone_james_bone__occurrences_24170.diff
[CMD] [2020-11-13 11:37:10] echo "0a" > /app/public/diff/bone_james_bone__measurements_24171.diff
[CMD] [2020-11-13 11:37:10] tail -n +1 /app/public/converted_csv/bone_james_bone__measurements_24171.csv >> /app/public/diff/bone_james_bone__measurements_24171.diff
[CMD] [2020-11-13 11:37:10] echo "." >> /app/public/diff/bone_james_bone__measurements_24171.diff
[STOP] [2020-11-13 11:37:10] calculate_delta
[START] [2020-11-13 11:37:10] parse_diff_and_store
[INFO] [2020-11-13 11:37:10] Loading nodes diff file into memory (true lines)...
[INFO] [2020-11-13 11:37:10] Loading occurrences diff file into memory (true lines)...
[INFO] [2020-11-13 11:37:10] Loading measurements diff file into memory (true lines)...
[INFO] [2020-11-13 11:37:11] Storing 1 ScientificNames
[INFO] [2020-11-13 11:37:11] Processing group of 1 in 1 groups of 1000
[INFO] [2020-11-13 11:37:11] Average Time: 0.0
[INFO] [2020-11-13 11:37:11] Total Time: 1s
[INFO] [2020-11-13 11:37:11] Storing 1 Nodes
[INFO] [2020-11-13 11:37:11] Processing group of 1 in 1 groups of 1000
[INFO] [2020-11-13 11:37:11] Average Time: 0.0
[INFO] [2020-11-13 11:37:11] Total Time: 1s
[INFO] [2020-11-13 11:37:11] Storing 1 Occurrences
[INFO] [2020-11-13 11:37:11] Processing group of 1 in 1 groups of 1000
[INFO] [2020-11-13 11:37:11] Average Time: 0.0
[INFO] [2020-11-13 11:37:11] Total Time: 1s
[INFO] [2020-11-13 11:37:11] Storing 3 Traits
[INFO] [2020-11-13 11:37:11] Processing group of 3 in 1 groups of 1000
[INFO] [2020-11-13 11:37:11] Average Time: 0.0
[INFO] [2020-11-13 11:37:11] Total Time: 1s
[INFO] [2020-11-13 11:37:11] Storing 2 MetaTraits
[INFO] [2020-11-13 11:37:11] Processing group of 2 in 1 groups of 1000
[INFO] [2020-11-13 11:37:11] Average Time: 0.0
[INFO] [2020-11-13 11:37:11] Total Time: 1s
[STOP] [2020-11-13 11:37:11] parse_diff_and_store
[START] [2020-11-13 11:37:11] resolve_keys
[INFO] [2020-11-13 11:37:17] Occurrences to nodes (through scientific_names)...
[INFO] [2020-11-13 11:37:17] traits to occurrences...
[INFO] [2020-11-13 11:37:17] traits to nodes (through occurrences)...
[INFO] [2020-11-13 11:37:17] Traits to sex term...
[INFO] [2020-11-13 11:37:17] Traits to lifestage term...
[INFO] [2020-11-13 11:37:17] MetaTraits to traits...
[INFO] [2020-11-13 11:37:17] MetaTraits (simple, measurement row refers to parent) to traits...
[INFO] [2020-11-13 11:37:17] Assocs to occurrences...
[INFO] [2020-11-13 11:37:17] Assocs to nodes...
[INFO] [2020-11-13 11:37:17] Assoc to sex term...
[INFO] [2020-11-13 11:37:17] Assoc to lifestage term...
[INFO] [2020-11-13 11:37:17] MetaAssoc to assocs...
[STOP] [2020-11-13 11:37:17] resolve_keys
[START] [2020-11-13 11:37:17] hold_for_later_1
[STOP] [2020-11-13 11:37:17] hold_for_later_1
[START] [2020-11-13 11:37:17] hold_for_later_2
[STOP] [2020-11-13 11:37:17] hold_for_later_2
[START] [2020-11-13 11:37:17] resolve_missing_parents
[STOP] [2020-11-13 11:37:17] resolve_missing_parents
[START] [2020-11-13 11:37:17] rebuild_nodes
[START] [2020-11-13 11:37:17] Flattener#flatten
[START] [2020-11-13 11:37:17] Flattener#study_resource
[START] [2020-11-13 11:37:17] Flattener#build_ancestry
[STOP] [2020-11-13 11:37:17] Flattener#build_ancestry
[INFO] [2020-11-13 11:37:17] 1 ancestry keys
[START] [2020-11-13 11:37:17] build_node_ancestors
[INFO] [2020-11-13 11:37:17] old ancestors deleted.
[STOP] [2020-11-13 11:37:17] build_node_ancestors
[WARN] [2020-11-13 11:37:17] Flattener: nothing to flatten! (Completely flat resource?)
[STOP] [2020-11-13 11:37:17] Flattener#flatten
[STOP] [2020-11-13 11:37:17] rebuild_nodes
[START] [2020-11-13 11:37:17] resolve_missing_media_owners
[STOP] [2020-11-13 11:37:17] resolve_missing_media_owners
[START] [2020-11-13 11:37:17] sanitize_media_verbatims
[STOP] [2020-11-13 11:37:17] sanitize_media_verbatims
[START] [2020-11-13 11:37:17] queue_downloads
[STOP] [2020-11-13 11:37:17] queue_downloads
[START] [2020-11-13 11:37:17] parse_names
[WARN] [2020-11-13 11:37:17] I see 1 names which still need to be parsed.
[STOP] [2020-11-13 11:37:18] parse_names
[START] [2020-11-13 11:37:18] denormalize_canonical_names_to_nodes
[STOP] [2020-11-13 11:37:18] denormalize_canonical_names_to_nodes
[START] [2020-11-13 11:37:18] match_nodes
[START] [2020-11-13 11:37:18] map_all_nodes_to_pages
[STOP] [2020-11-13 11:37:18] map_all_nodes_to_pages
[INFO] [2020-11-13 11:37:18] ZERO unmatched nodes (of 1)! Nicely done.
[START] [2020-11-13 11:37:18] update_nodes
[STOP] [2020-11-13 11:37:18] update_nodes
[STOP] [2020-11-13 11:37:18] match_nodes
[START] [2020-11-13 11:37:18] reindex_search
[STOP] [2020-11-13 11:37:18] reindex_search
[START] [2020-11-13 11:37:18] normalize_units
[STOP] [2020-11-13 11:37:18] normalize_units
[START] [2020-11-13 11:37:18] calculate_statistics
[2020-11-13 11:37:18] ZERO NODE ANCESTORS. Is this actually a completely flat resource?
[STOP] [2020-11-13 11:37:18] calculate_statistics
[START] [2020-11-13 11:37:18] complete_harvest_instance
[START] [2020-11-13 11:37:18] overall_tsv_creation
[INFO] [2020-11-13 11:37:18] Processing group of 1 in 1 batches of 10000
[INFO] [2020-11-13 11:37:57] 1 Traits (unfiltered)...
[INFO] [2020-11-13 11:38:34] 1 Traits (filtered)...
[INFO] [2020-11-13 11:38:34] 0 Associations (filtered)...
[INFO] [2020-11-13 11:38:34] 4 metadata added.
[INFO] [2020-11-13 11:38:34] 0 metadata added.
[INFO] [2020-11-13 11:38:34] Average Time: 51.36
[INFO] [2020-11-13 11:38:34] Total Time: 1m16s
[STOP] [2020-11-13 11:38:34] overall_tsv_creation
[INFO] [2020-11-13 11:38:34] Done. Check your files:
[INFO] [2020-11-13 11:38:34] (1 lines) /app/public/data/bone_james_bone_/publish_nodes.tsv
[INFO] [2020-11-13 11:38:34] (1 lines) /app/public/data/bone_james_bone_/publish_scientific_names.tsv
[INFO] [2020-11-13 11:38:34] (2 lines) /app/public/data/bone_james_bone_/publish_traits.tsv
[INFO] [2020-11-13 11:38:34] (4 lines) /app/public/data/bone_james_bone_/publish_metadata.tsv
[STOP] [2020-11-13 11:38:34] complete_harvest_instance
[START] [2020-11-13 11:38:34] completed
[STOP] [2020-11-13 11:38:34] completed
[STOP] [2020-11-13 11:38:34] logged process, took 85.52
[INFO] [2020-11-13 13:18:37] ## HARVEST: type = re_download_opendata_-harvest
[INFO] [2020-11-13 13:18:42] ## remove_type: ScientificName
[INFO] [2020-11-13 13:18:42] ++ Calling delete_all on 1 instances...
[INFO] [2020-11-13 13:18:42] [13:18:42.256] Removed 1 Scientificnames
[INFO] [2020-11-13 13:18:42] ## remove_type: Vernacular
[INFO] [2020-11-13 13:18:42] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-13 13:18:42] [13:18:42.259] Removed 0 Vernaculars
[INFO] [2020-11-13 13:18:42] ## remove_type: Article
[INFO] [2020-11-13 13:18:42] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-13 13:18:42] [13:18:42.262] Removed 0 Articles
[INFO] [2020-11-13 13:18:42] ## remove_type: Medium
[INFO] [2020-11-13 13:18:42] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-13 13:18:42] [13:18:42.265] Removed 0 Media
[INFO] [2020-11-13 13:18:42] ## remove_type: Trait
[INFO] [2020-11-13 13:18:42] ++ Calling delete_all on 3 instances...
[INFO] [2020-11-13 13:18:42] [13:18:42.269] Removed 3 Traits
[INFO] [2020-11-13 13:18:42] ## remove_type: MetaTrait
[INFO] [2020-11-13 13:18:42] ++ Calling delete_all on 2 instances...
[INFO] [2020-11-13 13:18:42] [13:18:42.273] Removed 2 Metatraits
[INFO] [2020-11-13 13:18:42] ## remove_type: OccurrenceMetadatum
[INFO] [2020-11-13 13:18:42] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-13 13:18:42] [13:18:42.276] Removed 0 Occurrencemetadata
[INFO] [2020-11-13 13:18:42] ## remove_type: Assoc
[INFO] [2020-11-13 13:18:42] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-13 13:18:42] [13:18:42.279] Removed 0 Assocs
[INFO] [2020-11-13 13:18:42] ## remove_type: MetaAssoc
[INFO] [2020-11-13 13:18:42] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-13 13:18:42] [13:18:42.281] Removed 0 Metaassocs
[INFO] [2020-11-13 13:18:42] ## remove_type: Identifier
[INFO] [2020-11-13 13:18:42] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-13 13:18:42] [13:18:42.284] Removed 0 Identifiers
[INFO] [2020-11-13 13:18:42] ## remove_type: Reference
[INFO] [2020-11-13 13:18:42] ++ Calling delete_all on 0 instances...
[INFO] [2020-11-13 13:18:42] [13:18:42.286] Removed 0 References
[INFO] [2020-11-13 13:18:42] Starting batch with ID 81659581...
[INFO] [2020-11-13 13:18:42] Starting batch with ID 81659581...
[INFO] [2020-11-13 13:18:42] Starting batch with ID 81659581...
[INFO] [2020-11-13 13:18:42] ## remove_type: Node
[INFO] [2020-11-13 13:18:42] ++ Calling delete_all on 1 instances...
[INFO] [2020-11-13 13:18:42] [13:18:42.345] Removed 1 Nodes
[START] [2020-11-13 13:18:42] logged process: 58bbc42b01abb4c1b2698de049792ffb4b63b979

[START] [2020-11-13 13:18:42] Creating resource from OpenData
[START] [2020-11-13 13:18:42] logged process: 58bbc42b01abb4c1b2698de049792ffb4b63b979

[START] [2020-11-13 13:18:42] Parse meta.xml file and create formats with fields
[STOP] [2020-11-13 13:18:42] Parse meta.xml file and create formats with fields
[STOP] [2020-11-13 13:18:42] Creating resource from OpenData
[START] [2020-11-13 13:18:42] logged process: 58bbc42b01abb4c1b2698de049792ffb4b63b979

[START] [2020-11-13 13:18:42] create_harvest_instance
[STOP] [2020-11-13 13:18:44] create_harvest_instance
[START] [2020-11-13 13:18:44] fetch_files
[STOP] [2020-11-13 13:18:44] fetch_files
[START] [2020-11-13 13:18:44] validate_each_file
[STOP] [2020-11-13 13:18:44] validate_each_file
[START] [2020-11-13 13:18:44] convert_to_csv
[CMD] [2020-11-13 13:18:44] /usr/bin/sort /app/public/converted_csv/bone_james_bone__nodes_24201.csv > /app/public/converted_csv/bone_james_bone__nodes_24201.csv_sorted
[CMD] [2020-11-13 13:18:44] /usr/bin/sort /app/public/converted_csv/bone_james_bone__occurrences_24202.csv > /app/public/converted_csv/bone_james_bone__occurrences_24202.csv_sorted
[CMD] [2020-11-13 13:18:44] /usr/bin/sort /app/public/converted_csv/bone_james_bone__measurements_24203.csv > /app/public/converted_csv/bone_james_bone__measurements_24203.csv_sorted
[STOP] [2020-11-13 13:18:44] convert_to_csv
[START] [2020-11-13 13:18:44] calculate_delta
[CMD] [2020-11-13 13:18:44] echo "0a" > /app/public/diff/bone_james_bone__nodes_24201.diff
[CMD] [2020-11-13 13:18:44] tail -n +1 /app/public/converted_csv/bone_james_bone__nodes_24201.csv >> /app/public/diff/bone_james_bone__nodes_24201.diff
[CMD] [2020-11-13 13:18:44] echo "." >> /app/public/diff/bone_james_bone__nodes_24201.diff
[CMD] [2020-11-13 13:18:44] echo "0a" > /app/public/diff/bone_james_bone__occurrences_24202.diff
[CMD] [2020-11-13 13:18:44] tail -n +1 /app/public/converted_csv/bone_james_bone__occurrences_24202.csv >> /app/public/diff/bone_james_bone__occurrences_24202.diff
[CMD] [2020-11-13 13:18:44] echo "." >> /app/public/diff/bone_james_bone__occurrences_24202.diff
[CMD] [2020-11-13 13:18:44] echo "0a" > /app/public/diff/bone_james_bone__measurements_24203.diff
[CMD] [2020-11-13 13:18:44] tail -n +1 /app/public/converted_csv/bone_james_bone__measurements_24203.csv >> /app/public/diff/bone_james_bone__measurements_24203.diff
[CMD] [2020-11-13 13:18:44] echo "." >> /app/public/diff/bone_james_bone__measurements_24203.diff
[STOP] [2020-11-13 13:18:44] calculate_delta
[START] [2020-11-13 13:18:44] parse_diff_and_store
[INFO] [2020-11-13 13:18:44] Loading nodes diff file into memory (true lines)...
[INFO] [2020-11-13 13:18:44] Loading occurrences diff file into memory (true lines)...
[INFO] [2020-11-13 13:18:45] Loading measurements diff file into memory (true lines)...
[INFO] [2020-11-13 13:18:45] Storing 1 ScientificNames
[INFO] [2020-11-13 13:18:45] Processing group of 1 in 1 groups of 1000
[INFO] [2020-11-13 13:18:45] Average Time: 0.0
[INFO] [2020-11-13 13:18:45] Total Time: 1s
[INFO] [2020-11-13 13:18:45] Storing 1 Nodes
[INFO] [2020-11-13 13:18:45] Processing group of 1 in 1 groups of 1000
[INFO] [2020-11-13 13:18:45] Average Time: 0.0
[INFO] [2020-11-13 13:18:45] Total Time: 1s
[INFO] [2020-11-13 13:18:45] Storing 1 Occurrences
[INFO] [2020-11-13 13:18:45] Processing group of 1 in 1 groups of 1000
[INFO] [2020-11-13 13:18:45] Average Time: 0.0
[INFO] [2020-11-13 13:18:45] Total Time: 1s
[INFO] [2020-11-13 13:18:45] Storing 1 OccurrenceMetadata
[INFO] [2020-11-13 13:18:45] Processing group of 1 in 1 groups of 1000
[INFO] [2020-11-13 13:18:45] Average Time: 0.0
[INFO] [2020-11-13 13:18:45] Total Time: 1s
[INFO] [2020-11-13 13:18:45] Storing 3 Traits
[INFO] [2020-11-13 13:18:45] Processing group of 3 in 1 groups of 1000
[INFO] [2020-11-13 13:18:45] Average Time: 0.0
[INFO] [2020-11-13 13:18:45] Total Time: 1s
[INFO] [2020-11-13 13:18:45] Storing 1 MetaTraits
[INFO] [2020-11-13 13:18:45] Processing group of 1 in 1 groups of 1000
[INFO] [2020-11-13 13:18:45] Average Time: 0.0
[INFO] [2020-11-13 13:18:45] Total Time: 1s
[STOP] [2020-11-13 13:18:45] parse_diff_and_store
[START] [2020-11-13 13:18:45] resolve_keys
[INFO] [2020-11-13 13:18:51] Occurrences to nodes (through scientific_names)...
[INFO] [2020-11-13 13:18:51] traits to occurrences...
[INFO] [2020-11-13 13:18:51] traits to nodes (through occurrences)...
[INFO] [2020-11-13 13:18:51] Traits to sex term...
[INFO] [2020-11-13 13:18:51] Traits to lifestage term...
[INFO] [2020-11-13 13:18:51] MetaTraits to traits...
[INFO] [2020-11-13 13:18:51] MetaTraits (simple, measurement row refers to parent) to traits...
[INFO] [2020-11-13 13:18:51] Assocs to occurrences...
[INFO] [2020-11-13 13:18:51] Assocs to nodes...
[INFO] [2020-11-13 13:18:51] Assoc to sex term...
[INFO] [2020-11-13 13:18:51] Assoc to lifestage term...
[INFO] [2020-11-13 13:18:51] MetaAssoc to assocs...
[STOP] [2020-11-13 13:18:51] resolve_keys
[START] [2020-11-13 13:18:51] hold_for_later_1
[STOP] [2020-11-13 13:18:51] hold_for_later_1
[START] [2020-11-13 13:18:51] hold_for_later_2
[STOP] [2020-11-13 13:18:51] hold_for_later_2
[START] [2020-11-13 13:18:51] resolve_missing_parents
[STOP] [2020-11-13 13:18:51] resolve_missing_parents
[START] [2020-11-13 13:18:51] rebuild_nodes
[START] [2020-11-13 13:18:51] Flattener#flatten
[START] [2020-11-13 13:18:51] Flattener#study_resource
[START] [2020-11-13 13:18:51] Flattener#build_ancestry
[STOP] [2020-11-13 13:18:51] Flattener#build_ancestry
[INFO] [2020-11-13 13:18:51] 1 ancestry keys
[START] [2020-11-13 13:18:51] build_node_ancestors
[INFO] [2020-11-13 13:18:51] old ancestors deleted.
[STOP] [2020-11-13 13:18:51] build_node_ancestors
[WARN] [2020-11-13 13:18:51] Flattener: nothing to flatten! (Completely flat resource?)
[STOP] [2020-11-13 13:18:51] Flattener#flatten
[STOP] [2020-11-13 13:18:51] rebuild_nodes
[START] [2020-11-13 13:18:51] resolve_missing_media_owners
[STOP] [2020-11-13 13:18:51] resolve_missing_media_owners
[START] [2020-11-13 13:18:51] sanitize_media_verbatims
[STOP] [2020-11-13 13:18:51] sanitize_media_verbatims
[START] [2020-11-13 13:18:51] queue_downloads
[STOP] [2020-11-13 13:18:51] queue_downloads
[START] [2020-11-13 13:18:51] parse_names
[WARN] [2020-11-13 13:18:51] I see 1 names which still need to be parsed.
[STOP] [2020-11-13 13:18:52] parse_names
[START] [2020-11-13 13:18:52] denormalize_canonical_names_to_nodes
[STOP] [2020-11-13 13:18:52] denormalize_canonical_names_to_nodes
[START] [2020-11-13 13:18:52] match_nodes
[START] [2020-11-13 13:18:52] map_all_nodes_to_pages
[STOP] [2020-11-13 13:18:52] map_all_nodes_to_pages
[INFO] [2020-11-13 13:18:52] ZERO unmatched nodes (of 1)! Nicely done.
[START] [2020-11-13 13:18:52] update_nodes
[STOP] [2020-11-13 13:18:52] update_nodes
[STOP] [2020-11-13 13:18:52] match_nodes
[START] [2020-11-13 13:18:52] reindex_search
[STOP] [2020-11-13 13:18:52] reindex_search
[START] [2020-11-13 13:18:52] normalize_units
[STOP] [2020-11-13 13:18:52] normalize_units
[START] [2020-11-13 13:18:52] calculate_statistics
[2020-11-13 13:18:52] ZERO NODE ANCESTORS. Is this actually a completely flat resource?
[STOP] [2020-11-13 13:18:52] calculate_statistics
[START] [2020-11-13 13:18:52] complete_harvest_instance
[START] [2020-11-13 13:18:52] overall_tsv_creation
[INFO] [2020-11-13 13:18:52] Processing group of 1 in 1 batches of 10000
[INFO] [2020-11-13 13:19:31] 1 Traits (unfiltered)...
[INFO] [2020-11-13 13:20:06] 1 Traits (filtered)...
[INFO] [2020-11-13 13:20:06] 0 Associations (filtered)...
[INFO] [2020-11-13 13:20:06] 4 metadata added.
[INFO] [2020-11-13 13:20:06] 0 metadata added.
[INFO] [2020-11-13 13:20:06] Average Time: 49.77
[INFO] [2020-11-13 13:20:06] Total Time: 1m15s
[STOP] [2020-11-13 13:20:06] overall_tsv_creation
[INFO] [2020-11-13 13:20:06] Done. Check your files:
[INFO] [2020-11-13 13:20:06] (1 lines) /app/public/data/bone_james_bone_/publish_nodes.tsv
[INFO] [2020-11-13 13:20:06] (1 lines) /app/public/data/bone_james_bone_/publish_scientific_names.tsv
[INFO] [2020-11-13 13:20:06] (2 lines) /app/public/data/bone_james_bone_/publish_traits.tsv
[INFO] [2020-11-13 13:20:06] (4 lines) /app/public/data/bone_james_bone_/publish_metadata.tsv
[STOP] [2020-11-13 13:20:06] complete_harvest_instance
[START] [2020-11-13 13:20:06] completed
[STOP] [2020-11-13 13:20:06] completed
[STOP] [2020-11-13 13:20:06] logged process, took 83.82
[INFO] [2021-04-19 14:39:15] ## HARVEST: type = re_download_opendata_-harvest
[INFO] [2021-04-19 14:39:20] ## remove_type: ScientificName
[INFO] [2021-04-19 14:39:20] ++ Calling delete_all on 1 instances...
[INFO] [2021-04-19 14:39:20] [14:39:20.778] Removed 1 Scientificnames
[INFO] [2021-04-19 14:39:20] ## remove_type: Vernacular
[INFO] [2021-04-19 14:39:20] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-19 14:39:20] [14:39:20.780] Removed 0 Vernaculars
[INFO] [2021-04-19 14:39:20] ## remove_type: Article
[INFO] [2021-04-19 14:39:20] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-19 14:39:20] [14:39:20.781] Removed 0 Articles
[INFO] [2021-04-19 14:39:20] ## remove_type: Medium
[INFO] [2021-04-19 14:39:20] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-19 14:39:20] [14:39:20.783] Removed 0 Media
[INFO] [2021-04-19 14:39:20] ## remove_type: Trait
[INFO] [2021-04-19 14:39:20] ++ Calling delete_all on 3 instances...
[INFO] [2021-04-19 14:39:20] [14:39:20.785] Removed 3 Traits
[INFO] [2021-04-19 14:39:20] ## remove_type: MetaTrait
[INFO] [2021-04-19 14:39:20] ++ Calling delete_all on 1 instances...
[INFO] [2021-04-19 14:39:20] [14:39:20.788] Removed 1 Metatraits
[INFO] [2021-04-19 14:39:20] ## remove_type: OccurrenceMetadatum
[INFO] [2021-04-19 14:39:20] ++ Calling delete_all on 1 instances...
[INFO] [2021-04-19 14:39:20] [14:39:20.790] Removed 1 Occurrencemetadata
[INFO] [2021-04-19 14:39:20] ## remove_type: Assoc
[INFO] [2021-04-19 14:39:20] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-19 14:39:20] [14:39:20.791] Removed 0 Assocs
[INFO] [2021-04-19 14:39:20] ## remove_type: MetaAssoc
[INFO] [2021-04-19 14:39:20] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-19 14:39:20] [14:39:20.793] Removed 0 Metaassocs
[INFO] [2021-04-19 14:39:20] ## remove_type: Identifier
[INFO] [2021-04-19 14:39:20] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-19 14:39:20] [14:39:20.795] Removed 0 Identifiers
[INFO] [2021-04-19 14:39:20] ## remove_type: Reference
[INFO] [2021-04-19 14:39:20] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-19 14:39:20] [14:39:20.797] Removed 0 References
[INFO] [2021-04-19 14:39:20] Starting batch with ID 81659604...
[INFO] [2021-04-19 14:39:20] Starting batch with ID 81659604...
[INFO] [2021-04-19 14:39:20] Starting batch with ID 81659604...
[INFO] [2021-04-19 14:39:20] Starting batch with ID 81659604...
[INFO] [2021-04-19 14:39:20] Starting batch with ID 81659604...
[INFO] [2021-04-19 14:39:20] Starting batch with ID 81659604...
[INFO] [2021-04-19 14:39:20] Starting batch with ID 81659604...
[INFO] [2021-04-19 14:39:20] Starting batch with ID 81659604...
[INFO] [2021-04-19 14:39:20] Starting batch with ID 81659604...
[INFO] [2021-04-19 14:39:20] Starting batch with ID 81659604...
[INFO] [2021-04-19 14:39:20] Starting batch with ID 81659604...
[INFO] [2021-04-19 14:39:20] Starting batch with ID 81659604...
[INFO] [2021-04-19 14:39:20] Starting batch with ID 81659604...
[INFO] [2021-04-19 14:39:20] Starting batch with ID 81659604...
[INFO] [2021-04-19 14:39:20] Starting batch with ID 81659604...
[INFO] [2021-04-19 14:39:20] Starting batch with ID 81659604...
[INFO] [2021-04-19 14:39:20] Starting batch with ID 81659604...
[INFO] [2021-04-19 14:39:20] Starting batch with ID 81659604...
[INFO] [2021-04-19 14:39:20] Starting batch with ID 81659604...
[INFO] [2021-04-19 14:39:20] Starting batch with ID 81659604...
[INFO] [2021-04-19 14:39:20] Starting batch with ID 81659604...
[INFO] [2021-04-19 14:39:20] Starting batch with ID 81659604...
[INFO] [2021-04-19 14:39:20] Starting batch with ID 81659604...
[INFO] [2021-04-19 14:39:21] Starting batch with ID 81659604...
[INFO] [2021-04-19 14:39:21] ## remove_type: Node
[INFO] [2021-04-19 14:39:21] ++ Calling delete_all on 1 instances...
[INFO] [2021-04-19 14:39:21] [14:39:21.013] Removed 1 Nodes
[START] [2021-04-19 14:39:21] logged process: 5ecc716a6a5541910d0c854f5a0c8d1651b82ad0 Improved MetaXml.ignore and added publisher to media (ignored)
[START] [2021-04-19 14:39:21] Creating resource from OpenData
[START] [2021-04-19 14:39:21] logged process: 5ecc716a6a5541910d0c854f5a0c8d1651b82ad0 Improved MetaXml.ignore and added publisher to media (ignored)
[START] [2021-04-19 14:39:21] Parse meta.xml file and create formats with fields
[STOP] [2021-04-19 14:39:21] Parse meta.xml file and create formats with fields
[STOP] [2021-04-19 14:39:21] Creating resource from OpenData
[START] [2021-04-19 14:39:21] logged process: 5ecc716a6a5541910d0c854f5a0c8d1651b82ad0 Improved MetaXml.ignore and added publisher to media (ignored)
[START] [2021-04-19 14:39:21] create_harvest_instance
[INFO] [2021-04-19 14:39:21] Created harvest instance #3745
[STOP] [2021-04-19 14:39:21] create_harvest_instance
[START] [2021-04-19 14:39:21] fetch_files
[STOP] [2021-04-19 14:39:21] fetch_files
[START] [2021-04-19 14:39:21] validate_each_file
[INFO] [2021-04-19 14:39:21] Looping over 3 formats...
[INFO] [2021-04-19 14:39:21] ...nodes (/app/public/data/bone_james_bone_/taxa.txt)
[INFO] [2021-04-19 14:39:21] Valid: /app/public/converted_csv/bone_james_bone__nodes_3745.csv (1 lines)
[INFO] [2021-04-19 14:39:21] ...occurrences (/app/public/data/bone_james_bone_/occurrences.txt)
[INFO] [2021-04-19 14:39:21] Valid: /app/public/converted_csv/bone_james_bone__occurrences_3745.csv (1 lines)
[INFO] [2021-04-19 14:39:21] ...measurements (/app/public/data/bone_james_bone_/measurementsorfacts.txt)
[INFO] [2021-04-19 14:39:21] Valid: /app/public/converted_csv/bone_james_bone__measurements_3745.csv (3 lines)
[STOP] [2021-04-19 14:39:21] validate_each_file
[START] [2021-04-19 14:39:21] convert_to_csv
[INFO] [2021-04-19 14:39:21] Looping over 3 formats...
[INFO] [2021-04-19 14:39:21] ...nodes (/app/public/data/bone_james_bone_/taxa.txt)
[CMD] [2021-04-19 14:39:21] /usr/bin/sort /app/public/converted_csv/bone_james_bone__nodes_3745.csv > /app/public/converted_csv/bone_james_bone__nodes_3745.csv_sorted
[INFO] [2021-04-19 14:39:21] Converted: /app/public/converted_csv/bone_james_bone__nodes_3745.csv (1 lines)
[INFO] [2021-04-19 14:39:21] ...occurrences (/app/public/data/bone_james_bone_/occurrences.txt)
[CMD] [2021-04-19 14:39:21] /usr/bin/sort /app/public/converted_csv/bone_james_bone__occurrences_3745.csv > /app/public/converted_csv/bone_james_bone__occurrences_3745.csv_sorted
[INFO] [2021-04-19 14:39:21] Converted: /app/public/converted_csv/bone_james_bone__occurrences_3745.csv (1 lines)
[INFO] [2021-04-19 14:39:21] ...measurements (/app/public/data/bone_james_bone_/measurementsorfacts.txt)
[CMD] [2021-04-19 14:39:21] /usr/bin/sort /app/public/converted_csv/bone_james_bone__measurements_3745.csv > /app/public/converted_csv/bone_james_bone__measurements_3745.csv_sorted
[INFO] [2021-04-19 14:39:21] Converted: /app/public/converted_csv/bone_james_bone__measurements_3745.csv (3 lines)
[STOP] [2021-04-19 14:39:21] convert_to_csv
[START] [2021-04-19 14:39:21] calculate_delta
[INFO] [2021-04-19 14:39:21] Looping over 3 formats...
[INFO] [2021-04-19 14:39:21] ...nodes (/app/public/data/bone_james_bone_/taxa.txt)
[CMD] [2021-04-19 14:39:21] echo "0a" > /app/public/diff/bone_james_bone__nodes_3745.diff
[CMD] [2021-04-19 14:39:21] tail -n +1 /app/public/converted_csv/bone_james_bone__nodes_3745.csv >> /app/public/diff/bone_james_bone__nodes_3745.diff
[CMD] [2021-04-19 14:39:21] echo "." >> /app/public/diff/bone_james_bone__nodes_3745.diff
[INFO] [2021-04-19 14:39:21] Created diff: /app/public/diff/bone_james_bone__nodes_3745.diff (3 lines)
[INFO] [2021-04-19 14:39:21] ...occurrences (/app/public/data/bone_james_bone_/occurrences.txt)
[CMD] [2021-04-19 14:39:21] echo "0a" > /app/public/diff/bone_james_bone__occurrences_3745.diff
[CMD] [2021-04-19 14:39:21] tail -n +1 /app/public/converted_csv/bone_james_bone__occurrences_3745.csv >> /app/public/diff/bone_james_bone__occurrences_3745.diff
[CMD] [2021-04-19 14:39:21] echo "." >> /app/public/diff/bone_james_bone__occurrences_3745.diff
[INFO] [2021-04-19 14:39:21] Created diff: /app/public/diff/bone_james_bone__occurrences_3745.diff (3 lines)
[INFO] [2021-04-19 14:39:21] ...measurements (/app/public/data/bone_james_bone_/measurementsorfacts.txt)
[CMD] [2021-04-19 14:39:21] echo "0a" > /app/public/diff/bone_james_bone__measurements_3745.diff
[CMD] [2021-04-19 14:39:21] tail -n +1 /app/public/converted_csv/bone_james_bone__measurements_3745.csv >> /app/public/diff/bone_james_bone__measurements_3745.diff
[CMD] [2021-04-19 14:39:21] echo "." >> /app/public/diff/bone_james_bone__measurements_3745.diff
[INFO] [2021-04-19 14:39:21] Created diff: /app/public/diff/bone_james_bone__measurements_3745.diff (5 lines)
[STOP] [2021-04-19 14:39:21] calculate_delta
[START] [2021-04-19 14:39:21] parse_diff_and_store
[INFO] [2021-04-19 14:39:21] Handling diff: /app/public/diff/bone_james_bone__nodes_3745.diff (3 lines)
[INFO] [2021-04-19 14:39:21] Loading nodes diff file into memory (3 /app/public/diff/bone_james_bone__nodes_3745.diff lines)...
[INFO] [2021-04-19 14:39:21] Handling diff: /app/public/diff/bone_james_bone__occurrences_3745.diff (3 lines)
[INFO] [2021-04-19 14:39:21] Loading occurrences diff file into memory (3 /app/public/diff/bone_james_bone__occurrences_3745.diff lines)...
[INFO] [2021-04-19 14:39:21] Handling diff: /app/public/diff/bone_james_bone__measurements_3745.diff (5 lines)
[INFO] [2021-04-19 14:39:21] Loading measurements diff file into memory (5 /app/public/diff/bone_james_bone__measurements_3745.diff lines)...
[INFO] [2021-04-19 14:39:21] Storing 1 ScientificNames
[INFO] [2021-04-19 14:39:21] Processing group of 1 in 1 groups of 1000
[INFO] [2021-04-19 14:39:21] Average Time: 0.0
[INFO] [2021-04-19 14:39:21] Total Time: 1s
[INFO] [2021-04-19 14:39:21] Storing 1 Nodes
[INFO] [2021-04-19 14:39:21] Processing group of 1 in 1 groups of 1000
[INFO] [2021-04-19 14:39:21] Average Time: 0.0
[INFO] [2021-04-19 14:39:21] Total Time: 1s
[INFO] [2021-04-19 14:39:21] Storing 1 Occurrences
[INFO] [2021-04-19 14:39:21] Processing group of 1 in 1 groups of 1000
[INFO] [2021-04-19 14:39:21] Average Time: 0.0
[INFO] [2021-04-19 14:39:21] Total Time: 1s
[INFO] [2021-04-19 14:39:21] Storing 1 OccurrenceMetadata
[INFO] [2021-04-19 14:39:21] Processing group of 1 in 1 groups of 1000
[INFO] [2021-04-19 14:39:21] Average Time: 0.0
[INFO] [2021-04-19 14:39:21] Total Time: 1s
[INFO] [2021-04-19 14:39:21] Storing 3 Traits
[INFO] [2021-04-19 14:39:21] Processing group of 3 in 1 groups of 1000
[INFO] [2021-04-19 14:39:21] Average Time: 0.0
[INFO] [2021-04-19 14:39:21] Total Time: 1s
[INFO] [2021-04-19 14:39:21] Storing 1 MetaTraits
[INFO] [2021-04-19 14:39:21] Processing group of 1 in 1 groups of 1000
[INFO] [2021-04-19 14:39:21] Average Time: 0.0
[INFO] [2021-04-19 14:39:21] Total Time: 1s
[STOP] [2021-04-19 14:39:21] parse_diff_and_store
[START] [2021-04-19 14:39:21] resolve_keys
[INFO] [2021-04-19 14:39:27] Occurrences to nodes (through scientific_names)...
[INFO] [2021-04-19 14:39:27] traits to occurrences...
[INFO] [2021-04-19 14:39:27] traits to nodes (through occurrences)...
[INFO] [2021-04-19 14:39:27] Traits to sex term...
[INFO] [2021-04-19 14:39:27] Traits to lifestage term...
[INFO] [2021-04-19 14:39:27] MetaTraits to traits...
[INFO] [2021-04-19 14:39:27] MetaTraits (simple, measurement row refers to parent) to traits...
[INFO] [2021-04-19 14:39:27] Assocs to occurrences...
[INFO] [2021-04-19 14:39:27] Assocs to nodes...
[INFO] [2021-04-19 14:39:27] Assoc to sex term...
[INFO] [2021-04-19 14:39:27] Assoc to lifestage term...
[INFO] [2021-04-19 14:39:27] MetaAssoc to assocs...
[STOP] [2021-04-19 14:39:27] resolve_keys
[START] [2021-04-19 14:39:27] hold_for_later_1
[STOP] [2021-04-19 14:39:27] hold_for_later_1
[START] [2021-04-19 14:39:27] hold_for_later_2
[STOP] [2021-04-19 14:39:27] hold_for_later_2
[START] [2021-04-19 14:39:27] resolve_missing_parents
[STOP] [2021-04-19 14:39:27] resolve_missing_parents
[START] [2021-04-19 14:39:27] rebuild_nodes
[START] [2021-04-19 14:39:27] Flattener#flatten
[START] [2021-04-19 14:39:27] Flattener#study_resource
[START] [2021-04-19 14:39:27] Flattener#build_ancestry
[STOP] [2021-04-19 14:39:27] Flattener#build_ancestry
[INFO] [2021-04-19 14:39:27] 1 ancestry keys
[START] [2021-04-19 14:39:27] build_node_ancestors
[INFO] [2021-04-19 14:39:27] old ancestors deleted.
[STOP] [2021-04-19 14:39:27] build_node_ancestors
[WARN] [2021-04-19 14:39:27] Flattener: nothing to flatten! (Completely flat resource?)
[STOP] [2021-04-19 14:39:27] Flattener#flatten
[STOP] [2021-04-19 14:39:27] rebuild_nodes
[START] [2021-04-19 14:39:27] resolve_missing_media_owners
[STOP] [2021-04-19 14:39:27] resolve_missing_media_owners
[START] [2021-04-19 14:39:27] sanitize_media_verbatims
[STOP] [2021-04-19 14:39:27] sanitize_media_verbatims
[START] [2021-04-19 14:39:27] queue_downloads
[STOP] [2021-04-19 14:39:27] queue_downloads
[START] [2021-04-19 14:39:27] parse_names
[WARN] [2021-04-19 14:39:27] I see 1 names which still need to be parsed.
[STOP] [2021-04-19 14:39:28] parse_names
[START] [2021-04-19 14:39:28] denormalize_canonical_names_to_nodes
[STOP] [2021-04-19 14:39:28] denormalize_canonical_names_to_nodes
[START] [2021-04-19 14:39:28] match_nodes
[START] [2021-04-19 14:39:28] map_all_nodes_to_pages
[STOP] [2021-04-19 14:39:28] map_all_nodes_to_pages
[INFO] [2021-04-19 14:39:28] ZERO unmatched nodes (of 1)! Nicely done.
[START] [2021-04-19 14:39:28] update_nodes
[STOP] [2021-04-19 14:39:28] update_nodes
[STOP] [2021-04-19 14:39:28] match_nodes
[START] [2021-04-19 14:39:28] reindex_search
[STOP] [2021-04-19 14:39:28] reindex_search
[START] [2021-04-19 14:39:28] normalize_units
[STOP] [2021-04-19 14:39:28] normalize_units
[START] [2021-04-19 14:39:28] calculate_statistics
[2021-04-19 14:39:28] ZERO NODE ANCESTORS. Is this actually a completely flat resource?
[STOP] [2021-04-19 14:39:28] calculate_statistics
[START] [2021-04-19 14:39:28] complete_harvest_instance
[START] [2021-04-19 14:39:28] overall_tsv_creation
[INFO] [2021-04-19 14:39:28] Processing group of 1 in 1 batches of 10000
[INFO] [2021-04-19 14:40:04] 1 Traits (unfiltered)...
[INFO] [2021-04-19 14:40:37] 1 Traits (filtered)...
[INFO] [2021-04-19 14:40:37] 0 Associations (filtered)...
[INFO] [2021-04-19 14:40:37] 2 metadata added.
[INFO] [2021-04-19 14:40:37] 0 metadata added.
[INFO] [2021-04-19 14:41:03] Average Time: 70.85
[INFO] [2021-04-19 14:41:03] Total Time: 1m35s
[STOP] [2021-04-19 14:41:03] overall_tsv_creation
[INFO] [2021-04-19 14:41:03] Done. Check your files:
[INFO] [2021-04-19 14:41:03] (1 lines) /app/public/data/bone_james_bone_/publish_nodes.tsv
[INFO] [2021-04-19 14:41:03] (1 lines) /app/public/data/bone_james_bone_/publish_scientific_names.tsv
[INFO] [2021-04-19 14:41:03] (2 lines) /app/public/data/bone_james_bone_/publish_traits.tsv
[INFO] [2021-04-19 14:41:03] (3 lines) /app/public/data/bone_james_bone_/publish_metadata.tsv
[STOP] [2021-04-19 14:41:03] complete_harvest_instance
[START] [2021-04-19 14:41:03] completed
[STOP] [2021-04-19 14:41:03] completed
[STOP] [2021-04-19 14:41:03] logged process, took 102.37

Latest Process