harvested resources
sign in

Harvest for Fabre et al 2016 Created 06 Apr 17:59

Stage: completed
Fetched: 06 Apr 17:59
Validated: 06 Apr 17:59
Deltas Created 06 Apr 17:59
Units Normalized: 06 Apr 17:59
Ancestry Built: 06 Apr 17:59
Nodes Matched: 06 Apr 17:59
Names Parsed: 06 Apr 17:59
New Models Stored: 06 Apr 17:59
Indexed: 06 Apr 17:59
Completed: 06 Apr 18:01
Time to Harvest: less than a minute

Harvesting Log

(350 lines) 
# Logfile created on 2021-02-03 10:05:07 -0500 by logger.rb/v1.4.2
[START] [2021-02-03 10:05:07] logged process: 16b22834be7ac1492cba86047bb0f5dbfa370977

[START] [2021-02-03 10:05:07] Creating resource from OpenData
[START] [2021-02-03 10:05:07] logged process: 16b22834be7ac1492cba86047bb0f5dbfa370977

[START] [2021-02-03 10:05:07] Parse meta.xml file and create formats with fields
[STOP] [2021-02-03 10:05:07] Parse meta.xml file and create formats with fields
[STOP] [2021-02-03 10:05:07] Creating resource from OpenData
[INFO] [2021-02-03 10:06:28] ## HARVEST: type = -harvest
[START] [2021-02-03 10:06:32] logged process: 16b22834be7ac1492cba86047bb0f5dbfa370977

[START] [2021-02-03 10:06:32] create_harvest_instance
[STOP] [2021-02-03 10:06:34] create_harvest_instance
[START] [2021-02-03 10:06:34] fetch_files
[STOP] [2021-02-03 10:06:34] fetch_files
[START] [2021-02-03 10:06:34] validate_each_file
[STOP] [2021-02-03 10:06:34] validate_each_file
[START] [2021-02-03 10:06:34] convert_to_csv
[CMD] [2021-02-03 10:06:34] /usr/bin/sort /app/public/converted_csv/fabre_et_al_fabr_nodes_26510.csv > /app/public/converted_csv/fabre_et_al_fabr_nodes_26510.csv_sorted
[CMD] [2021-02-03 10:06:34] /usr/bin/sort /app/public/converted_csv/fabre_et_al_fabr_occurrences_26511.csv > /app/public/converted_csv/fabre_et_al_fabr_occurrences_26511.csv_sorted
[CMD] [2021-02-03 10:06:34] /usr/bin/sort /app/public/converted_csv/fabre_et_al_fabr_measurements_26512.csv > /app/public/converted_csv/fabre_et_al_fabr_measurements_26512.csv_sorted
[STOP] [2021-02-03 10:06:34] convert_to_csv
[START] [2021-02-03 10:06:34] calculate_delta
[CMD] [2021-02-03 10:06:34] echo "0a" > /app/public/diff/fabre_et_al_fabr_nodes_26510.diff
[CMD] [2021-02-03 10:06:34] tail -n +1 /app/public/converted_csv/fabre_et_al_fabr_nodes_26510.csv >> /app/public/diff/fabre_et_al_fabr_nodes_26510.diff
[CMD] [2021-02-03 10:06:34] echo "." >> /app/public/diff/fabre_et_al_fabr_nodes_26510.diff
[CMD] [2021-02-03 10:06:34] echo "0a" > /app/public/diff/fabre_et_al_fabr_occurrences_26511.diff
[CMD] [2021-02-03 10:06:34] tail -n +1 /app/public/converted_csv/fabre_et_al_fabr_occurrences_26511.csv >> /app/public/diff/fabre_et_al_fabr_occurrences_26511.diff
[CMD] [2021-02-03 10:06:34] echo "." >> /app/public/diff/fabre_et_al_fabr_occurrences_26511.diff
[CMD] [2021-02-03 10:06:34] echo "0a" > /app/public/diff/fabre_et_al_fabr_measurements_26512.diff
[CMD] [2021-02-03 10:06:34] tail -n +1 /app/public/converted_csv/fabre_et_al_fabr_measurements_26512.csv >> /app/public/diff/fabre_et_al_fabr_measurements_26512.diff
[CMD] [2021-02-03 10:06:34] echo "." >> /app/public/diff/fabre_et_al_fabr_measurements_26512.diff
[STOP] [2021-02-03 10:06:34] calculate_delta
[START] [2021-02-03 10:06:34] parse_diff_and_store
[INFO] [2021-02-03 10:06:34] Loading nodes diff file into memory (true lines)...
[INFO] [2021-02-03 10:06:34] Loading occurrences diff file into memory (true lines)...
[INFO] [2021-02-03 10:06:34] Loading measurements diff file into memory (true lines)...
[INFO] [2021-02-03 10:06:35] Storing 55 ScientificNames
[INFO] [2021-02-03 10:06:35] Processing group of 55 in 1 groups of 1000
[INFO] [2021-02-03 10:06:35] Average Time: 0.02
[INFO] [2021-02-03 10:06:35] Total Time: 1s
[INFO] [2021-02-03 10:06:35] Storing 55 Nodes
[INFO] [2021-02-03 10:06:35] Processing group of 55 in 1 groups of 1000
[INFO] [2021-02-03 10:06:35] Average Time: 0.02
[INFO] [2021-02-03 10:06:35] Total Time: 1s
[INFO] [2021-02-03 10:06:35] Storing 55 Occurrences
[INFO] [2021-02-03 10:06:35] Processing group of 55 in 1 groups of 1000
[INFO] [2021-02-03 10:06:35] Average Time: 0.01
[INFO] [2021-02-03 10:06:35] Total Time: 1s
[INFO] [2021-02-03 10:06:35] Storing 55 Traits
[INFO] [2021-02-03 10:06:35] Processing group of 55 in 1 groups of 1000
[INFO] [2021-02-03 10:06:35] Average Time: 0.02
[INFO] [2021-02-03 10:06:35] Total Time: 1s
[INFO] [2021-02-03 10:06:35] Storing 60 MetaTraits
[INFO] [2021-02-03 10:06:35] Processing group of 60 in 1 groups of 1000
[INFO] [2021-02-03 10:06:35] Average Time: 0.01
[INFO] [2021-02-03 10:06:35] Total Time: 1s
[STOP] [2021-02-03 10:06:35] parse_diff_and_store
[START] [2021-02-03 10:06:35] resolve_keys
[INFO] [2021-02-03 10:06:41] Occurrences to nodes (through scientific_names)...
[INFO] [2021-02-03 10:06:41] traits to occurrences...
[INFO] [2021-02-03 10:06:41] traits to nodes (through occurrences)...
[INFO] [2021-02-03 10:06:41] Traits to sex term...
[INFO] [2021-02-03 10:06:41] Traits to lifestage term...
[INFO] [2021-02-03 10:06:41] MetaTraits to traits...
[INFO] [2021-02-03 10:06:41] MetaTraits (simple, measurement row refers to parent) to traits...
[INFO] [2021-02-03 10:06:41] Assocs to occurrences...
[INFO] [2021-02-03 10:06:41] Assocs to nodes...
[INFO] [2021-02-03 10:06:41] Assoc to sex term...
[INFO] [2021-02-03 10:06:41] Assoc to lifestage term...
[INFO] [2021-02-03 10:06:41] MetaAssoc to assocs...
[STOP] [2021-02-03 10:06:41] resolve_keys
[START] [2021-02-03 10:06:41] hold_for_later_1
[STOP] [2021-02-03 10:06:41] hold_for_later_1
[START] [2021-02-03 10:06:41] hold_for_later_2
[STOP] [2021-02-03 10:06:41] hold_for_later_2
[START] [2021-02-03 10:06:41] resolve_missing_parents
[STOP] [2021-02-03 10:06:41] resolve_missing_parents
[START] [2021-02-03 10:06:41] rebuild_nodes
[START] [2021-02-03 10:06:41] Flattener#flatten
[START] [2021-02-03 10:06:41] Flattener#study_resource
[START] [2021-02-03 10:06:41] Flattener#build_ancestry
[STOP] [2021-02-03 10:06:41] Flattener#build_ancestry
[INFO] [2021-02-03 10:06:41] 55 ancestry keys
[START] [2021-02-03 10:06:41] build_node_ancestors
[INFO] [2021-02-03 10:06:41] old ancestors deleted.
[STOP] [2021-02-03 10:06:41] build_node_ancestors
[WARN] [2021-02-03 10:06:41] Flattener: nothing to flatten! (Completely flat resource?)
[STOP] [2021-02-03 10:06:41] Flattener#flatten
[STOP] [2021-02-03 10:06:41] rebuild_nodes
[START] [2021-02-03 10:06:41] resolve_missing_media_owners
[STOP] [2021-02-03 10:06:41] resolve_missing_media_owners
[START] [2021-02-03 10:06:41] sanitize_media_verbatims
[STOP] [2021-02-03 10:06:41] sanitize_media_verbatims
[START] [2021-02-03 10:06:41] queue_downloads
[STOP] [2021-02-03 10:06:41] queue_downloads
[START] [2021-02-03 10:06:41] parse_names
[WARN] [2021-02-03 10:06:41] I see 55 names which still need to be parsed.
[WARN] [2021-02-03 10:06:42] I see 41 names which still need to be parsed.
[WARN] [2021-02-03 10:06:43] I see 27 names which still need to be parsed.
[WARN] [2021-02-03 10:06:44] I see 15 names which still need to be parsed.
[WARN] [2021-02-03 10:06:45] I see 5 names which still need to be parsed.
[STOP] [2021-02-03 10:06:46] parse_names
[START] [2021-02-03 10:06:46] denormalize_canonical_names_to_nodes
[STOP] [2021-02-03 10:06:46] denormalize_canonical_names_to_nodes
[START] [2021-02-03 10:06:46] match_nodes
[START] [2021-02-03 10:06:46] map_all_nodes_to_pages
[STOP] [2021-02-03 10:06:47] map_all_nodes_to_pages
[INFO] [2021-02-03 10:06:47] ZERO unmatched nodes (of 55)! Nicely done.
[START] [2021-02-03 10:06:47] update_nodes
[STOP] [2021-02-03 10:06:47] update_nodes
[STOP] [2021-02-03 10:06:47] match_nodes
[START] [2021-02-03 10:06:47] reindex_search
[STOP] [2021-02-03 10:06:47] reindex_search
[START] [2021-02-03 10:06:47] normalize_units
[STOP] [2021-02-03 10:06:47] normalize_units
[START] [2021-02-03 10:06:47] calculate_statistics
[2021-02-03 10:06:47] ZERO NODE ANCESTORS. Is this actually a completely flat resource?
[STOP] [2021-02-03 10:06:47] calculate_statistics
[START] [2021-02-03 10:06:47] complete_harvest_instance
[START] [2021-02-03 10:06:47] overall_tsv_creation
[INFO] [2021-02-03 10:06:47] Processing group of 55 in 1 batches of 10000
[INFO] [2021-02-03 10:07:26] 55 Traits (unfiltered)...
[INFO] [2021-02-03 10:08:03] 55 Traits (filtered)...
[INFO] [2021-02-03 10:08:03] 0 Associations (filtered)...
[INFO] [2021-02-03 10:08:03] 60 metadata added.
[INFO] [2021-02-03 10:08:03] 0 metadata added.
[INFO] [2021-02-03 10:08:34] Average Time: 81.02
[INFO] [2021-02-03 10:08:34] Total Time: 1m47s
[STOP] [2021-02-03 10:08:34] overall_tsv_creation
[INFO] [2021-02-03 10:08:34] Done. Check your files:
[INFO] [2021-02-03 10:08:34] (14 lines) /app/public/data/fabre_et_al_fabr/publish_nodes.tsv
[INFO] [2021-02-03 10:08:34] (55 lines) /app/public/data/fabre_et_al_fabr/publish_scientific_names.tsv
[INFO] [2021-02-03 10:08:34] (56 lines) /app/public/data/fabre_et_al_fabr/publish_traits.tsv
[INFO] [2021-02-03 10:08:34] (61 lines) /app/public/data/fabre_et_al_fabr/publish_metadata.tsv
[STOP] [2021-02-03 10:08:34] complete_harvest_instance
[START] [2021-02-03 10:08:34] completed
[STOP] [2021-02-03 10:08:34] completed
[STOP] [2021-02-03 10:08:34] logged process, took 121.27
[INFO] [2021-04-06 17:52:47] ## HARVEST: type = re_download_opendata_-harvest
[INFO] [2021-04-06 17:59:42] ## remove_type: ScientificName
[INFO] [2021-04-06 17:59:42] ++ Calling delete_all on 55 instances...
[INFO] [2021-04-06 17:59:42] [17:59:42.446] Removed 55 Scientificnames
[INFO] [2021-04-06 17:59:42] ## remove_type: Vernacular
[INFO] [2021-04-06 17:59:42] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-06 17:59:42] [17:59:42.448] Removed 0 Vernaculars
[INFO] [2021-04-06 17:59:42] ## remove_type: Article
[INFO] [2021-04-06 17:59:42] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-06 17:59:42] [17:59:42.449] Removed 0 Articles
[INFO] [2021-04-06 17:59:42] ## remove_type: Medium
[INFO] [2021-04-06 17:59:42] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-06 17:59:42] [17:59:42.450] Removed 0 Media
[INFO] [2021-04-06 17:59:42] ## remove_type: Trait
[INFO] [2021-04-06 17:59:42] ++ Calling delete_all on 55 instances...
[INFO] [2021-04-06 17:59:42] [17:59:42.453] Removed 55 Traits
[INFO] [2021-04-06 17:59:42] ## remove_type: MetaTrait
[INFO] [2021-04-06 17:59:42] ++ Calling delete_all on 60 instances...
[INFO] [2021-04-06 17:59:42] [17:59:42.456] Removed 60 Metatraits
[INFO] [2021-04-06 17:59:42] ## remove_type: OccurrenceMetadatum
[INFO] [2021-04-06 17:59:42] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-06 17:59:42] [17:59:42.457] Removed 0 Occurrencemetadata
[INFO] [2021-04-06 17:59:42] ## remove_type: Assoc
[INFO] [2021-04-06 17:59:42] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-06 17:59:42] [17:59:42.458] Removed 0 Assocs
[INFO] [2021-04-06 17:59:42] ## remove_type: MetaAssoc
[INFO] [2021-04-06 17:59:42] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-06 17:59:42] [17:59:42.460] Removed 0 Metaassocs
[INFO] [2021-04-06 17:59:42] ## remove_type: Identifier
[INFO] [2021-04-06 17:59:42] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-06 17:59:42] [17:59:42.461] Removed 0 Identifiers
[INFO] [2021-04-06 17:59:42] ## remove_type: Reference
[INFO] [2021-04-06 17:59:42] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-06 17:59:42] [17:59:42.463] Removed 0 References
[INFO] [2021-04-06 17:59:42] Starting batch with ID 87728347...
[INFO] [2021-04-06 17:59:42] Starting batch with ID 87728347...
[INFO] [2021-04-06 17:59:42] Starting batch with ID 87728347...
[INFO] [2021-04-06 17:59:42] Starting batch with ID 87728347...
[INFO] [2021-04-06 17:59:42] Starting batch with ID 87728347...
[INFO] [2021-04-06 17:59:42] Starting batch with ID 87728347...
[INFO] [2021-04-06 17:59:42] Starting batch with ID 87728347...
[INFO] [2021-04-06 17:59:42] Starting batch with ID 87728347...
[INFO] [2021-04-06 17:59:42] Starting batch with ID 87728347...
[INFO] [2021-04-06 17:59:42] Starting batch with ID 87728347...
[INFO] [2021-04-06 17:59:42] Starting batch with ID 87728347...
[INFO] [2021-04-06 17:59:42] Starting batch with ID 87728347...
[INFO] [2021-04-06 17:59:42] Starting batch with ID 87728347...
[INFO] [2021-04-06 17:59:42] ## remove_type: Node
[INFO] [2021-04-06 17:59:42] ++ Calling delete_all on 55 instances...
[INFO] [2021-04-06 17:59:42] [17:59:42.619] Removed 55 Nodes
[START] [2021-04-06 17:59:42] logged process: 5ecc716a6a5541910d0c854f5a0c8d1651b82ad0 Improved MetaXml.ignore and added publisher to media (ignored)
[START] [2021-04-06 17:59:42] Creating resource from OpenData
[START] [2021-04-06 17:59:42] logged process: 5ecc716a6a5541910d0c854f5a0c8d1651b82ad0 Improved MetaXml.ignore and added publisher to media (ignored)
[START] [2021-04-06 17:59:42] Parse meta.xml file and create formats with fields
[STOP] [2021-04-06 17:59:43] Parse meta.xml file and create formats with fields
[STOP] [2021-04-06 17:59:43] Creating resource from OpenData
[START] [2021-04-06 17:59:43] logged process: 5ecc716a6a5541910d0c854f5a0c8d1651b82ad0 Improved MetaXml.ignore and added publisher to media (ignored)
[START] [2021-04-06 17:59:43] create_harvest_instance
[INFO] [2021-04-06 17:59:43] Created harvest instance #3656
[STOP] [2021-04-06 17:59:43] create_harvest_instance
[START] [2021-04-06 17:59:43] fetch_files
[STOP] [2021-04-06 17:59:43] fetch_files
[START] [2021-04-06 17:59:43] validate_each_file
[INFO] [2021-04-06 17:59:43] Looping over 3 formats...
[INFO] [2021-04-06 17:59:43] ...nodes (/app/public/data/fabre_et_al_fabr/taxa.txt)
[INFO] [2021-04-06 17:59:43] Valid: /app/public/converted_csv/fabre_et_al_fabr_nodes_3656.csv (55 lines)
[INFO] [2021-04-06 17:59:43] ...occurrences (/app/public/data/fabre_et_al_fabr/occurrences.txt)
[INFO] [2021-04-06 17:59:43] Valid: /app/public/converted_csv/fabre_et_al_fabr_occurrences_3656.csv (55 lines)
[INFO] [2021-04-06 17:59:43] ...measurements (/app/public/data/fabre_et_al_fabr/measurementsorfacts.txt)
[INFO] [2021-04-06 17:59:43] Valid: /app/public/converted_csv/fabre_et_al_fabr_measurements_3656.csv (55 lines)
[STOP] [2021-04-06 17:59:43] validate_each_file
[START] [2021-04-06 17:59:43] convert_to_csv
[INFO] [2021-04-06 17:59:43] Looping over 3 formats...
[INFO] [2021-04-06 17:59:43] ...nodes (/app/public/data/fabre_et_al_fabr/taxa.txt)
[CMD] [2021-04-06 17:59:43] /usr/bin/sort /app/public/converted_csv/fabre_et_al_fabr_nodes_3656.csv > /app/public/converted_csv/fabre_et_al_fabr_nodes_3656.csv_sorted
[INFO] [2021-04-06 17:59:43] Converted: /app/public/converted_csv/fabre_et_al_fabr_nodes_3656.csv (55 lines)
[INFO] [2021-04-06 17:59:43] ...occurrences (/app/public/data/fabre_et_al_fabr/occurrences.txt)
[CMD] [2021-04-06 17:59:43] /usr/bin/sort /app/public/converted_csv/fabre_et_al_fabr_occurrences_3656.csv > /app/public/converted_csv/fabre_et_al_fabr_occurrences_3656.csv_sorted
[INFO] [2021-04-06 17:59:43] Converted: /app/public/converted_csv/fabre_et_al_fabr_occurrences_3656.csv (55 lines)
[INFO] [2021-04-06 17:59:43] ...measurements (/app/public/data/fabre_et_al_fabr/measurementsorfacts.txt)
[CMD] [2021-04-06 17:59:43] /usr/bin/sort /app/public/converted_csv/fabre_et_al_fabr_measurements_3656.csv > /app/public/converted_csv/fabre_et_al_fabr_measurements_3656.csv_sorted
[INFO] [2021-04-06 17:59:43] Converted: /app/public/converted_csv/fabre_et_al_fabr_measurements_3656.csv (55 lines)
[STOP] [2021-04-06 17:59:43] convert_to_csv
[START] [2021-04-06 17:59:43] calculate_delta
[INFO] [2021-04-06 17:59:43] Looping over 3 formats...
[INFO] [2021-04-06 17:59:43] ...nodes (/app/public/data/fabre_et_al_fabr/taxa.txt)
[CMD] [2021-04-06 17:59:43] echo "0a" > /app/public/diff/fabre_et_al_fabr_nodes_3656.diff
[CMD] [2021-04-06 17:59:43] tail -n +1 /app/public/converted_csv/fabre_et_al_fabr_nodes_3656.csv >> /app/public/diff/fabre_et_al_fabr_nodes_3656.diff
[CMD] [2021-04-06 17:59:43] echo "." >> /app/public/diff/fabre_et_al_fabr_nodes_3656.diff
[INFO] [2021-04-06 17:59:43] Created diff: /app/public/diff/fabre_et_al_fabr_nodes_3656.diff (57 lines)
[INFO] [2021-04-06 17:59:43] ...occurrences (/app/public/data/fabre_et_al_fabr/occurrences.txt)
[CMD] [2021-04-06 17:59:43] echo "0a" > /app/public/diff/fabre_et_al_fabr_occurrences_3656.diff
[CMD] [2021-04-06 17:59:43] tail -n +1 /app/public/converted_csv/fabre_et_al_fabr_occurrences_3656.csv >> /app/public/diff/fabre_et_al_fabr_occurrences_3656.diff
[CMD] [2021-04-06 17:59:43] echo "." >> /app/public/diff/fabre_et_al_fabr_occurrences_3656.diff
[INFO] [2021-04-06 17:59:43] Created diff: /app/public/diff/fabre_et_al_fabr_occurrences_3656.diff (57 lines)
[INFO] [2021-04-06 17:59:43] ...measurements (/app/public/data/fabre_et_al_fabr/measurementsorfacts.txt)
[CMD] [2021-04-06 17:59:43] echo "0a" > /app/public/diff/fabre_et_al_fabr_measurements_3656.diff
[CMD] [2021-04-06 17:59:43] tail -n +1 /app/public/converted_csv/fabre_et_al_fabr_measurements_3656.csv >> /app/public/diff/fabre_et_al_fabr_measurements_3656.diff
[CMD] [2021-04-06 17:59:43] echo "." >> /app/public/diff/fabre_et_al_fabr_measurements_3656.diff
[INFO] [2021-04-06 17:59:43] Created diff: /app/public/diff/fabre_et_al_fabr_measurements_3656.diff (57 lines)
[STOP] [2021-04-06 17:59:43] calculate_delta
[START] [2021-04-06 17:59:43] parse_diff_and_store
[INFO] [2021-04-06 17:59:43] Handling diff: /app/public/diff/fabre_et_al_fabr_nodes_3656.diff (57 lines)
[INFO] [2021-04-06 17:59:43] Loading nodes diff file into memory (57 /app/public/diff/fabre_et_al_fabr_nodes_3656.diff lines)...
[INFO] [2021-04-06 17:59:43] Handling diff: /app/public/diff/fabre_et_al_fabr_occurrences_3656.diff (57 lines)
[INFO] [2021-04-06 17:59:43] Loading occurrences diff file into memory (57 /app/public/diff/fabre_et_al_fabr_occurrences_3656.diff lines)...
[INFO] [2021-04-06 17:59:43] Handling diff: /app/public/diff/fabre_et_al_fabr_measurements_3656.diff (57 lines)
[INFO] [2021-04-06 17:59:43] Loading measurements diff file into memory (57 /app/public/diff/fabre_et_al_fabr_measurements_3656.diff lines)...
[INFO] [2021-04-06 17:59:43] Storing 55 ScientificNames
[INFO] [2021-04-06 17:59:43] Processing group of 55 in 1 groups of 1000
[INFO] [2021-04-06 17:59:43] Average Time: 0.02
[INFO] [2021-04-06 17:59:43] Total Time: 1s
[INFO] [2021-04-06 17:59:43] Storing 55 Nodes
[INFO] [2021-04-06 17:59:43] Processing group of 55 in 1 groups of 1000
[INFO] [2021-04-06 17:59:43] Average Time: 0.02
[INFO] [2021-04-06 17:59:43] Total Time: 1s
[INFO] [2021-04-06 17:59:43] Storing 55 Occurrences
[INFO] [2021-04-06 17:59:43] Processing group of 55 in 1 groups of 1000
[INFO] [2021-04-06 17:59:43] Average Time: 0.01
[INFO] [2021-04-06 17:59:43] Total Time: 1s
[INFO] [2021-04-06 17:59:43] Storing 55 Traits
[INFO] [2021-04-06 17:59:43] Processing group of 55 in 1 groups of 1000
[INFO] [2021-04-06 17:59:43] Average Time: 0.02
[INFO] [2021-04-06 17:59:43] Total Time: 1s
[INFO] [2021-04-06 17:59:43] Storing 60 MetaTraits
[INFO] [2021-04-06 17:59:43] Processing group of 60 in 1 groups of 1000
[INFO] [2021-04-06 17:59:43] Average Time: 0.01
[INFO] [2021-04-06 17:59:43] Total Time: 1s
[STOP] [2021-04-06 17:59:43] parse_diff_and_store
[START] [2021-04-06 17:59:43] resolve_keys
[INFO] [2021-04-06 17:59:49] Occurrences to nodes (through scientific_names)...
[INFO] [2021-04-06 17:59:49] traits to occurrences...
[INFO] [2021-04-06 17:59:49] traits to nodes (through occurrences)...
[INFO] [2021-04-06 17:59:49] Traits to sex term...
[INFO] [2021-04-06 17:59:49] Traits to lifestage term...
[INFO] [2021-04-06 17:59:49] MetaTraits to traits...
[INFO] [2021-04-06 17:59:49] MetaTraits (simple, measurement row refers to parent) to traits...
[INFO] [2021-04-06 17:59:49] Assocs to occurrences...
[INFO] [2021-04-06 17:59:49] Assocs to nodes...
[INFO] [2021-04-06 17:59:49] Assoc to sex term...
[INFO] [2021-04-06 17:59:49] Assoc to lifestage term...
[INFO] [2021-04-06 17:59:49] MetaAssoc to assocs...
[STOP] [2021-04-06 17:59:49] resolve_keys
[START] [2021-04-06 17:59:49] hold_for_later_1
[STOP] [2021-04-06 17:59:49] hold_for_later_1
[START] [2021-04-06 17:59:49] hold_for_later_2
[STOP] [2021-04-06 17:59:49] hold_for_later_2
[START] [2021-04-06 17:59:49] resolve_missing_parents
[STOP] [2021-04-06 17:59:49] resolve_missing_parents
[START] [2021-04-06 17:59:49] rebuild_nodes
[START] [2021-04-06 17:59:49] Flattener#flatten
[START] [2021-04-06 17:59:49] Flattener#study_resource
[START] [2021-04-06 17:59:49] Flattener#build_ancestry
[STOP] [2021-04-06 17:59:49] Flattener#build_ancestry
[INFO] [2021-04-06 17:59:49] 55 ancestry keys
[START] [2021-04-06 17:59:49] build_node_ancestors
[INFO] [2021-04-06 17:59:49] old ancestors deleted.
[STOP] [2021-04-06 17:59:49] build_node_ancestors
[WARN] [2021-04-06 17:59:49] Flattener: nothing to flatten! (Completely flat resource?)
[STOP] [2021-04-06 17:59:49] Flattener#flatten
[STOP] [2021-04-06 17:59:49] rebuild_nodes
[START] [2021-04-06 17:59:49] resolve_missing_media_owners
[STOP] [2021-04-06 17:59:49] resolve_missing_media_owners
[START] [2021-04-06 17:59:49] sanitize_media_verbatims
[STOP] [2021-04-06 17:59:49] sanitize_media_verbatims
[START] [2021-04-06 17:59:49] queue_downloads
[STOP] [2021-04-06 17:59:49] queue_downloads
[START] [2021-04-06 17:59:49] parse_names
[WARN] [2021-04-06 17:59:49] I see 55 names which still need to be parsed.
[WARN] [2021-04-06 17:59:50] I see 41 names which still need to be parsed.
[WARN] [2021-04-06 17:59:51] I see 27 names which still need to be parsed.
[WARN] [2021-04-06 17:59:52] I see 15 names which still need to be parsed.
[WARN] [2021-04-06 17:59:54] I see 5 names which still need to be parsed.
[STOP] [2021-04-06 17:59:55] parse_names
[START] [2021-04-06 17:59:55] denormalize_canonical_names_to_nodes
[STOP] [2021-04-06 17:59:55] denormalize_canonical_names_to_nodes
[START] [2021-04-06 17:59:55] match_nodes
[START] [2021-04-06 17:59:55] map_all_nodes_to_pages
[STOP] [2021-04-06 17:59:55] map_all_nodes_to_pages
[INFO] [2021-04-06 17:59:55] ZERO unmatched nodes (of 55)! Nicely done.
[START] [2021-04-06 17:59:55] update_nodes
[STOP] [2021-04-06 17:59:55] update_nodes
[STOP] [2021-04-06 17:59:55] match_nodes
[START] [2021-04-06 17:59:55] reindex_search
[STOP] [2021-04-06 17:59:55] reindex_search
[START] [2021-04-06 17:59:55] normalize_units
[STOP] [2021-04-06 17:59:55] normalize_units
[START] [2021-04-06 17:59:55] calculate_statistics
[2021-04-06 17:59:55] ZERO NODE ANCESTORS. Is this actually a completely flat resource?
[STOP] [2021-04-06 17:59:55] calculate_statistics
[START] [2021-04-06 17:59:55] complete_harvest_instance
[START] [2021-04-06 17:59:55] overall_tsv_creation
[INFO] [2021-04-06 17:59:55] Processing group of 55 in 1 batches of 10000
[INFO] [2021-04-06 18:00:32] 55 Traits (unfiltered)...
[INFO] [2021-04-06 18:01:06] 55 Traits (filtered)...
[INFO] [2021-04-06 18:01:06] 0 Associations (filtered)...
[INFO] [2021-04-06 18:01:06] 0 metadata added.
[INFO] [2021-04-06 18:01:06] 0 metadata added.
[INFO] [2021-04-06 18:01:34] Average Time: 74.01
[INFO] [2021-04-06 18:01:34] Total Time: 1m39s
[STOP] [2021-04-06 18:01:34] overall_tsv_creation
[INFO] [2021-04-06 18:01:34] Done. Check your files:
[INFO] [2021-04-06 18:01:34] (14 lines) /app/public/data/fabre_et_al_fabr/publish_nodes.tsv
[INFO] [2021-04-06 18:01:34] (55 lines) /app/public/data/fabre_et_al_fabr/publish_scientific_names.tsv
[INFO] [2021-04-06 18:01:34] (56 lines) /app/public/data/fabre_et_al_fabr/publish_traits.tsv
[INFO] [2021-04-06 18:01:34] (1 lines) /app/public/data/fabre_et_al_fabr/publish_metadata.tsv
[STOP] [2021-04-06 18:01:34] complete_harvest_instance
[START] [2021-04-06 18:01:34] completed
[STOP] [2021-04-06 18:01:34] completed
[STOP] [2021-04-06 18:01:34] logged process, took 111.03

Latest Process