Harvest for Adriatic Sea Species List Created 04 Apr 13:04

Harvesting Log

!! Reset log via RemoveContentJob on 2021-04-04 12:59:20 -0400
[INFO] [2021-04-04 13:03:51] ## HARVEST: type = re_download_opendata_-harvest
[INFO] [2021-04-04 13:03:52] ## remove_type: ScientificName
[INFO] [2021-04-04 13:03:52] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-04 13:03:52] [13:03:52.234] Removed 0 Scientificnames
[INFO] [2021-04-04 13:03:52] ## remove_type: Vernacular
[INFO] [2021-04-04 13:03:52] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-04 13:03:52] [13:03:52.237] Removed 0 Vernaculars
[INFO] [2021-04-04 13:03:52] ## remove_type: Article
[INFO] [2021-04-04 13:03:52] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-04 13:03:52] [13:03:52.238] Removed 0 Articles
[INFO] [2021-04-04 13:03:52] ## remove_type: Medium
[INFO] [2021-04-04 13:03:52] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-04 13:03:52] [13:03:52.241] Removed 0 Media
[INFO] [2021-04-04 13:03:52] ## remove_type: Trait
[INFO] [2021-04-04 13:03:52] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-04 13:03:52] [13:03:52.244] Removed 0 Traits
[INFO] [2021-04-04 13:03:52] ## remove_type: MetaTrait
[INFO] [2021-04-04 13:03:52] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-04 13:03:52] [13:03:52.247] Removed 0 Metatraits
[INFO] [2021-04-04 13:03:52] ## remove_type: OccurrenceMetadatum
[INFO] [2021-04-04 13:03:52] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-04 13:03:52] [13:03:52.250] Removed 0 Occurrencemetadata
[INFO] [2021-04-04 13:03:52] ## remove_type: Assoc
[INFO] [2021-04-04 13:03:52] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-04 13:03:52] [13:03:52.253] Removed 0 Assocs
[INFO] [2021-04-04 13:03:52] ## remove_type: MetaAssoc
[INFO] [2021-04-04 13:03:52] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-04 13:03:52] [13:03:52.256] Removed 0 Metaassocs
[INFO] [2021-04-04 13:03:52] ## remove_type: Identifier
[INFO] [2021-04-04 13:03:52] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-04 13:03:52] [13:03:52.259] Removed 0 Identifiers
[INFO] [2021-04-04 13:03:52] ## remove_type: Reference
[INFO] [2021-04-04 13:03:52] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-04 13:03:52] [13:03:52.261] Removed 0 References
[INFO] [2021-04-04 13:03:52] ## remove_type: Node
[INFO] [2021-04-04 13:03:52] ++ Calling delete_all on 0 instances...
[INFO] [2021-04-04 13:03:52] [13:03:52.276] Removed 0 Nodes
[START] [2021-04-04 13:03:53] logged process: 5ecc716a6a5541910d0c854f5a0c8d1651b82ad0 Improved MetaXml.ignore and added publisher to media (ignored)
[START] [2021-04-04 13:03:53] Creating resource from OpenData
[START] [2021-04-04 13:03:55] logged process: 5ecc716a6a5541910d0c854f5a0c8d1651b82ad0 Improved MetaXml.ignore and added publisher to media (ignored)
[START] [2021-04-04 13:03:55] Parse meta.xml file and create formats with fields
[STOP] [2021-04-04 13:04:07] Parse meta.xml file and create formats with fields
[STOP] [2021-04-04 13:04:07] Creating resource from OpenData
[START] [2021-04-04 13:04:08] logged process: 5ecc716a6a5541910d0c854f5a0c8d1651b82ad0 Improved MetaXml.ignore and added publisher to media (ignored)
[START] [2021-04-04 13:04:08] create_harvest_instance
[INFO] [2021-04-04 13:04:08] Created harvest instance #3636
[STOP] [2021-04-04 13:04:08] create_harvest_instance
[START] [2021-04-04 13:04:08] fetch_files
[STOP] [2021-04-04 13:04:08] fetch_files
[START] [2021-04-04 13:04:08] validate_each_file
[INFO] [2021-04-04 13:04:08] Looping over 4 formats...
[INFO] [2021-04-04 13:04:08] ...refs (/app/public/data/adriatic_sea_sp2/reference.tab)
[INFO] [2021-04-04 13:04:08] Valid: /app/public/converted_csv/adriatic_sea_sp2_refs_3636.csv (0 lines)
[INFO] [2021-04-04 13:04:08] ...nodes (/app/public/data/adriatic_sea_sp2/taxon.tab)
[INFO] [2021-04-04 13:04:08] Valid: /app/public/converted_csv/adriatic_sea_sp2_nodes_3636.csv (0 lines)
[INFO] [2021-04-04 13:04:08] ...occurrences (/app/public/data/adriatic_sea_sp2/occurrence.tab)
[INFO] [2021-04-04 13:04:08] Valid: /app/public/converted_csv/adriatic_sea_sp2_occurrences_3636.csv (0 lines)
[INFO] [2021-04-04 13:04:08] ...measurements (/app/public/data/adriatic_sea_sp2/measurement_or_fact_specific.tab)
[INFO] [2021-04-04 13:04:08] Valid: /app/public/converted_csv/adriatic_sea_sp2_measurements_3636.csv (0 lines)
[STOP] [2021-04-04 13:04:08] validate_each_file
[START] [2021-04-04 13:04:08] convert_to_csv
[INFO] [2021-04-04 13:04:08] Looping over 4 formats...
[INFO] [2021-04-04 13:04:08] ...refs (/app/public/data/adriatic_sea_sp2/reference.tab)
[CMD] [2021-04-04 13:04:08] /usr/bin/sort /app/public/converted_csv/adriatic_sea_sp2_refs_3636.csv > /app/public/converted_csv/adriatic_sea_sp2_refs_3636.csv_sorted
[INFO] [2021-04-04 13:04:09] Converted: /app/public/converted_csv/adriatic_sea_sp2_refs_3636.csv (0 lines)
[INFO] [2021-04-04 13:04:09] ...nodes (/app/public/data/adriatic_sea_sp2/taxon.tab)
[CMD] [2021-04-04 13:04:09] /usr/bin/sort /app/public/converted_csv/adriatic_sea_sp2_nodes_3636.csv > /app/public/converted_csv/adriatic_sea_sp2_nodes_3636.csv_sorted
[INFO] [2021-04-04 13:04:10] Converted: /app/public/converted_csv/adriatic_sea_sp2_nodes_3636.csv (0 lines)
[INFO] [2021-04-04 13:04:10] ...occurrences (/app/public/data/adriatic_sea_sp2/occurrence.tab)
[CMD] [2021-04-04 13:04:10] /usr/bin/sort /app/public/converted_csv/adriatic_sea_sp2_occurrences_3636.csv > /app/public/converted_csv/adriatic_sea_sp2_occurrences_3636.csv_sorted
[INFO] [2021-04-04 13:04:11] Converted: /app/public/converted_csv/adriatic_sea_sp2_occurrences_3636.csv (0 lines)
[INFO] [2021-04-04 13:04:11] ...measurements (/app/public/data/adriatic_sea_sp2/measurement_or_fact_specific.tab)
[CMD] [2021-04-04 13:04:11] /usr/bin/sort /app/public/converted_csv/adriatic_sea_sp2_measurements_3636.csv > /app/public/converted_csv/adriatic_sea_sp2_measurements_3636.csv_sorted
[INFO] [2021-04-04 13:04:12] Converted: /app/public/converted_csv/adriatic_sea_sp2_measurements_3636.csv (0 lines)
[STOP] [2021-04-04 13:04:12] convert_to_csv
[START] [2021-04-04 13:04:12] calculate_delta
[INFO] [2021-04-04 13:04:12] Looping over 4 formats...
[INFO] [2021-04-04 13:04:12] ...refs (/app/public/data/adriatic_sea_sp2/reference.tab)
[CMD] [2021-04-04 13:04:12] echo "0a" > /app/public/diff/adriatic_sea_sp2_refs_3636.diff
[CMD] [2021-04-04 13:04:13] tail -n +1 /app/public/converted_csv/adriatic_sea_sp2_refs_3636.csv >> /app/public/diff/adriatic_sea_sp2_refs_3636.diff
[CMD] [2021-04-04 13:04:14] echo "." >> /app/public/diff/adriatic_sea_sp2_refs_3636.diff
[INFO] [2021-04-04 13:04:15] Created diff: /app/public/diff/adriatic_sea_sp2_refs_3636.diff (2 lines)
[INFO] [2021-04-04 13:04:15] ...nodes (/app/public/data/adriatic_sea_sp2/taxon.tab)
[CMD] [2021-04-04 13:04:15] echo "0a" > /app/public/diff/adriatic_sea_sp2_nodes_3636.diff
[CMD] [2021-04-04 13:04:16] tail -n +1 /app/public/converted_csv/adriatic_sea_sp2_nodes_3636.csv >> /app/public/diff/adriatic_sea_sp2_nodes_3636.diff
[CMD] [2021-04-04 13:04:17] echo "." >> /app/public/diff/adriatic_sea_sp2_nodes_3636.diff
[INFO] [2021-04-04 13:04:18] Created diff: /app/public/diff/adriatic_sea_sp2_nodes_3636.diff (2 lines)
[INFO] [2021-04-04 13:04:18] ...occurrences (/app/public/data/adriatic_sea_sp2/occurrence.tab)
[CMD] [2021-04-04 13:04:18] echo "0a" > /app/public/diff/adriatic_sea_sp2_occurrences_3636.diff
[CMD] [2021-04-04 13:04:19] tail -n +1 /app/public/converted_csv/adriatic_sea_sp2_occurrences_3636.csv >> /app/public/diff/adriatic_sea_sp2_occurrences_3636.diff
[CMD] [2021-04-04 13:04:20] echo "." >> /app/public/diff/adriatic_sea_sp2_occurrences_3636.diff
[INFO] [2021-04-04 13:04:21] Created diff: /app/public/diff/adriatic_sea_sp2_occurrences_3636.diff (2 lines)
[INFO] [2021-04-04 13:04:21] ...measurements (/app/public/data/adriatic_sea_sp2/measurement_or_fact_specific.tab)
[CMD] [2021-04-04 13:04:21] echo "0a" > /app/public/diff/adriatic_sea_sp2_measurements_3636.diff
[CMD] [2021-04-04 13:04:22] tail -n +1 /app/public/converted_csv/adriatic_sea_sp2_measurements_3636.csv >> /app/public/diff/adriatic_sea_sp2_measurements_3636.diff
[CMD] [2021-04-04 13:04:24] echo "." >> /app/public/diff/adriatic_sea_sp2_measurements_3636.diff
[INFO] [2021-04-04 13:04:25] Created diff: /app/public/diff/adriatic_sea_sp2_measurements_3636.diff (2 lines)
[STOP] [2021-04-04 13:04:25] calculate_delta
[START] [2021-04-04 13:04:25] parse_diff_and_store
[INFO] [2021-04-04 13:04:25] Handling diff: /app/public/diff/adriatic_sea_sp2_refs_3636.diff (2 lines)
[INFO] [2021-04-04 13:04:26] Loading refs diff file into memory (2 /app/public/diff/adriatic_sea_sp2_refs_3636.diff lines)...
[INFO] [2021-04-04 13:04:27] Handling diff: /app/public/diff/adriatic_sea_sp2_nodes_3636.diff (2 lines)
[INFO] [2021-04-04 13:04:28] Loading nodes diff file into memory (2 /app/public/diff/adriatic_sea_sp2_nodes_3636.diff lines)...
[INFO] [2021-04-04 13:04:29] Handling diff: /app/public/diff/adriatic_sea_sp2_occurrences_3636.diff (2 lines)
[INFO] [2021-04-04 13:04:30] Loading occurrences diff file into memory (2 /app/public/diff/adriatic_sea_sp2_occurrences_3636.diff lines)...
[INFO] [2021-04-04 13:04:31] Handling diff: /app/public/diff/adriatic_sea_sp2_measurements_3636.diff (2 lines)
[INFO] [2021-04-04 13:04:32] Loading measurements diff file into memory (2 /app/public/diff/adriatic_sea_sp2_measurements_3636.diff lines)...
[STOP] [2021-04-04 13:04:33] parse_diff_and_store
[START] [2021-04-04 13:04:33] resolve_keys
[INFO] [2021-04-04 13:04:38] Occurrences to nodes (through scientific_names)...
[INFO] [2021-04-04 13:04:38] traits to occurrences...
[INFO] [2021-04-04 13:04:38] traits to nodes (through occurrences)...
[INFO] [2021-04-04 13:04:38] Traits to sex term...
[INFO] [2021-04-04 13:04:38] Traits to lifestage term...
[INFO] [2021-04-04 13:04:38] MetaTraits to traits...
[INFO] [2021-04-04 13:04:38] MetaTraits (simple, measurement row refers to parent) to traits...
[INFO] [2021-04-04 13:04:38] Assocs to occurrences...
[INFO] [2021-04-04 13:04:38] Assocs to nodes...
[INFO] [2021-04-04 13:04:38] Assoc to sex term...
[INFO] [2021-04-04 13:04:38] Assoc to lifestage term...
[INFO] [2021-04-04 13:04:38] MetaAssoc to assocs...
[STOP] [2021-04-04 13:04:38] resolve_keys
[START] [2021-04-04 13:04:38] hold_for_later_1
[STOP] [2021-04-04 13:04:38] hold_for_later_1
[START] [2021-04-04 13:04:38] hold_for_later_2
[STOP] [2021-04-04 13:04:38] hold_for_later_2
[START] [2021-04-04 13:04:38] resolve_missing_parents
[STOP] [2021-04-04 13:04:38] resolve_missing_parents
[START] [2021-04-04 13:04:38] rebuild_nodes
[START] [2021-04-04 13:04:38] Flattener#flatten
[START] [2021-04-04 13:04:38] Flattener#study_resource
[WARN] [2021-04-04 13:04:38] Flattener: no children found, aborting flatten
[STOP] [2021-04-04 13:04:38] Flattener#flatten
[STOP] [2021-04-04 13:04:38] rebuild_nodes
[START] [2021-04-04 13:04:38] resolve_missing_media_owners
[STOP] [2021-04-04 13:04:38] resolve_missing_media_owners
[START] [2021-04-04 13:04:38] sanitize_media_verbatims
[STOP] [2021-04-04 13:04:38] sanitize_media_verbatims
[START] [2021-04-04 13:04:38] queue_downloads
[STOP] [2021-04-04 13:04:38] queue_downloads
[START] [2021-04-04 13:04:38] parse_names
[STOP] [2021-04-04 13:04:38] parse_names
[START] [2021-04-04 13:04:38] denormalize_canonical_names_to_nodes
[STOP] [2021-04-04 13:04:38] denormalize_canonical_names_to_nodes
[START] [2021-04-04 13:04:38] match_nodes
[START] [2021-04-04 13:04:38] map_all_nodes_to_pages
[STOP] [2021-04-04 13:04:38] map_all_nodes_to_pages
[INFO] [2021-04-04 13:04:38] ZERO unmatched nodes (of 0)! Nicely done.
[START] [2021-04-04 13:04:38] update_nodes
[STOP] [2021-04-04 13:04:38] update_nodes
[STOP] [2021-04-04 13:04:38] match_nodes
[START] [2021-04-04 13:04:39] reindex_search
[STOP] [2021-04-04 13:04:39] reindex_search
[START] [2021-04-04 13:04:39] normalize_units
[STOP] [2021-04-04 13:04:39] normalize_units
[START] [2021-04-04 13:04:39] calculate_statistics
[ERR] [2021-04-04 13:04:39] Exception
[ERR] [2021-04-04 13:04:39] ZERO NODES! That is probably... bad.
[STOP] [2021-04-04 13:04:39] calculate_statistics
[STOP] [2021-04-04 13:04:39] logged process, took 31.39

Latest Process

04 Apr 13:03 EXITED currently processing 0