Harvest for ECOCROP Created 31 May 18:35

Harvesting Log

[INFO] [2021-05-31 18:35:48] Created harvest instance #3945
[STOP] [2021-05-31 18:35:48] create_harvest_instance
[START] [2021-05-31 18:35:48] fetch_files
[STOP] [2021-05-31 18:35:48] fetch_files
[START] [2021-05-31 18:35:48] validate_each_file
[INFO] [2021-05-31 18:35:48] Looping over 3 formats...
[INFO] [2021-05-31 18:35:48] ...nodes (/app/public/data/ECOCROP/taxa.txt)
[INFO] [2021-05-31 18:35:48] Valid: /app/public/converted_csv/ECOCROP_nodes_3945.csv (2567 lines)
[INFO] [2021-05-31 18:35:48] ...occurrences (/app/public/data/ECOCROP/occurrences.txt)
[INFO] [2021-05-31 18:35:48] Valid: /app/public/converted_csv/ECOCROP_occurrences_3945.csv (2567 lines)
[INFO] [2021-05-31 18:35:48] ...measurements (/app/public/data/ECOCROP/measurementsorfacts.txt)
[INFO] [2021-05-31 18:35:51] Valid: /app/public/converted_csv/ECOCROP_measurements_3945.csv (46351 lines)
[STOP] [2021-05-31 18:35:51] validate_each_file
[START] [2021-05-31 18:35:51] convert_to_csv
[INFO] [2021-05-31 18:35:51] Looping over 3 formats...
[INFO] [2021-05-31 18:35:51] ...nodes (/app/public/data/ECOCROP/taxa.txt)
[CMD] [2021-05-31 18:35:51] /usr/bin/sort /app/public/converted_csv/ECOCROP_nodes_3945.csv > /app/public/converted_csv/ECOCROP_nodes_3945.csv_sorted
[INFO] [2021-05-31 18:35:51] Converted: /app/public/converted_csv/ECOCROP_nodes_3945.csv (2567 lines)
[INFO] [2021-05-31 18:35:51] ...occurrences (/app/public/data/ECOCROP/occurrences.txt)
[CMD] [2021-05-31 18:35:51] /usr/bin/sort /app/public/converted_csv/ECOCROP_occurrences_3945.csv > /app/public/converted_csv/ECOCROP_occurrences_3945.csv_sorted
[INFO] [2021-05-31 18:35:52] Converted: /app/public/converted_csv/ECOCROP_occurrences_3945.csv (2567 lines)
[INFO] [2021-05-31 18:35:52] ...measurements (/app/public/data/ECOCROP/measurementsorfacts.txt)
[CMD] [2021-05-31 18:35:52] /usr/bin/sort /app/public/converted_csv/ECOCROP_measurements_3945.csv > /app/public/converted_csv/ECOCROP_measurements_3945.csv_sorted
[INFO] [2021-05-31 18:35:52] Converted: /app/public/converted_csv/ECOCROP_measurements_3945.csv (46351 lines)
[STOP] [2021-05-31 18:35:52] convert_to_csv
[START] [2021-05-31 18:35:52] calculate_delta
[INFO] [2021-05-31 18:35:52] Looping over 3 formats...
[INFO] [2021-05-31 18:35:52] ...nodes (/app/public/data/ECOCROP/taxa.txt)
[CMD] [2021-05-31 18:35:52] echo "0a" > /app/public/diff/ECOCROP_nodes_3945.diff
[CMD] [2021-05-31 18:35:53] tail -n +1 /app/public/converted_csv/ECOCROP_nodes_3945.csv >> /app/public/diff/ECOCROP_nodes_3945.diff
[CMD] [2021-05-31 18:35:53] echo "." >> /app/public/diff/ECOCROP_nodes_3945.diff
[INFO] [2021-05-31 18:35:53] Created diff: /app/public/diff/ECOCROP_nodes_3945.diff (2569 lines)
[INFO] [2021-05-31 18:35:53] ...occurrences (/app/public/data/ECOCROP/occurrences.txt)
[CMD] [2021-05-31 18:35:53] echo "0a" > /app/public/diff/ECOCROP_occurrences_3945.diff
[CMD] [2021-05-31 18:35:54] tail -n +1 /app/public/converted_csv/ECOCROP_occurrences_3945.csv >> /app/public/diff/ECOCROP_occurrences_3945.diff
[CMD] [2021-05-31 18:35:54] echo "." >> /app/public/diff/ECOCROP_occurrences_3945.diff
[INFO] [2021-05-31 18:35:54] Created diff: /app/public/diff/ECOCROP_occurrences_3945.diff (2569 lines)
[INFO] [2021-05-31 18:35:54] ...measurements (/app/public/data/ECOCROP/measurementsorfacts.txt)
[CMD] [2021-05-31 18:35:54] echo "0a" > /app/public/diff/ECOCROP_measurements_3945.diff
[CMD] [2021-05-31 18:35:55] tail -n +1 /app/public/converted_csv/ECOCROP_measurements_3945.csv >> /app/public/diff/ECOCROP_measurements_3945.diff
[CMD] [2021-05-31 18:35:55] echo "." >> /app/public/diff/ECOCROP_measurements_3945.diff
[INFO] [2021-05-31 18:35:56] Created diff: /app/public/diff/ECOCROP_measurements_3945.diff (46353 lines)
[STOP] [2021-05-31 18:35:56] calculate_delta
[START] [2021-05-31 18:35:56] parse_diff_and_store
[INFO] [2021-05-31 18:35:56] Handling diff: /app/public/diff/ECOCROP_nodes_3945.diff (2569 lines)
[INFO] [2021-05-31 18:35:56] Loading nodes diff file into memory (2569 /app/public/diff/ECOCROP_nodes_3945.diff lines)...
[WARN] [2021-05-31 18:35:57] Filtered Scientific Name `Setaria sphacelata  var. sericea` to `Setaria sphacelata var. sericea`
[INFO] [2021-05-31 18:35:57] Handling diff: /app/public/diff/ECOCROP_occurrences_3945.diff (2569 lines)
[INFO] [2021-05-31 18:35:58] Loading occurrences diff file into memory (2569 /app/public/diff/ECOCROP_occurrences_3945.diff lines)...
[INFO] [2021-05-31 18:35:58] Handling diff: /app/public/diff/ECOCROP_measurements_3945.diff (46353 lines)
[INFO] [2021-05-31 18:35:59] Loading measurements diff file into memory (46353 /app/public/diff/ECOCROP_measurements_3945.diff lines)...
[INFO] [2021-05-31 18:36:12] Storing 2770 ScientificNames
[INFO] [2021-05-31 18:36:12] Processing group of 2770 in 3 groups of 1000
[INFO] [2021-05-31 18:36:13] Average Time: 0.283
[INFO] [2021-05-31 18:36:13] Total Time: 1s
[INFO] [2021-05-31 18:36:13] Storing 2770 Nodes
[INFO] [2021-05-31 18:36:13] Processing group of 2770 in 3 groups of 1000
[INFO] [2021-05-31 18:36:13] Average Time: 0.223
[INFO] [2021-05-31 18:36:13] Total Time: 1s
[INFO] [2021-05-31 18:36:13] Storing 2567 Occurrences
[INFO] [2021-05-31 18:36:13] Processing group of 2567 in 3 groups of 1000
[INFO] [2021-05-31 18:36:14] Average Time: 0.087
[INFO] [2021-05-31 18:36:14] Total Time: 1s
[INFO] [2021-05-31 18:36:14] Storing 46351 Traits
[INFO] [2021-05-31 18:36:14] Processing group of 46351 in 47 groups of 1000
[INFO] [2021-05-31 18:36:30] Average Time: 0.336
[INFO] [2021-05-31 18:36:30] Total Time: 16s
[INFO] [2021-05-31 18:36:30] last 3 / first 3: 1.65
[INFO] [2021-05-31 18:36:30] Std.Dev: 0.11832159566199232; Max: 0.81
[INFO] [2021-05-31 18:36:30] Storing 2781 MetaTraits
[INFO] [2021-05-31 18:36:30] Processing group of 2781 in 3 groups of 1000
[INFO] [2021-05-31 18:36:30] Average Time: 0.103
[INFO] [2021-05-31 18:36:30] Total Time: 1s
[STOP] [2021-05-31 18:36:30] parse_diff_and_store
[START] [2021-05-31 18:36:30] resolve_keys
[INFO] [2021-05-31 18:36:37] Occurrences to nodes (through scientific_names)...
[INFO] [2021-05-31 18:36:37] traits to occurrences...
[INFO] [2021-05-31 18:36:38] traits to nodes (through occurrences)...
[INFO] [2021-05-31 18:36:39] Traits to sex term...
[INFO] [2021-05-31 18:36:39] Traits to lifestage term...
[INFO] [2021-05-31 18:36:39] MetaTraits to traits...
[INFO] [2021-05-31 18:36:39] MetaTraits (simple, measurement row refers to parent) to traits...
[INFO] [2021-05-31 18:36:40] Assocs to occurrences...
[INFO] [2021-05-31 18:36:40] Assocs to nodes...
[INFO] [2021-05-31 18:36:40] Assoc to sex term...
[INFO] [2021-05-31 18:36:40] Assoc to lifestage term...
[INFO] [2021-05-31 18:36:40] MetaAssoc to assocs...
[STOP] [2021-05-31 18:36:40] resolve_keys
[START] [2021-05-31 18:36:40] hold_for_later_1
[STOP] [2021-05-31 18:36:40] hold_for_later_1
[START] [2021-05-31 18:36:40] hold_for_later_2
[STOP] [2021-05-31 18:36:40] hold_for_later_2
[START] [2021-05-31 18:36:40] resolve_missing_parents
[STOP] [2021-05-31 18:36:40] resolve_missing_parents
[START] [2021-05-31 18:36:40] rebuild_nodes
[START] [2021-05-31 18:36:40] Flattener#flatten
[START] [2021-05-31 18:36:40] Flattener#study_resource
[START] [2021-05-31 18:36:40] Flattener#build_ancestry
[STOP] [2021-05-31 18:36:41] Flattener#build_ancestry
[INFO] [2021-05-31 18:36:41] 2770 ancestry keys
[START] [2021-05-31 18:36:41] build_node_ancestors
[INFO] [2021-05-31 18:36:41] old ancestors deleted.
[STOP] [2021-05-31 18:36:41] build_node_ancestors
[START] [2021-05-31 18:36:41] Flattener#propagate_ancestor_ids
[STOP] [2021-05-31 18:36:42] Flattener#propagate_ancestor_ids
[STOP] [2021-05-31 18:36:42] Flattener#flatten
[STOP] [2021-05-31 18:36:42] rebuild_nodes
[START] [2021-05-31 18:36:42] resolve_missing_media_owners
[STOP] [2021-05-31 18:36:42] resolve_missing_media_owners
[START] [2021-05-31 18:36:42] sanitize_media_verbatims
[STOP] [2021-05-31 18:36:42] sanitize_media_verbatims
[START] [2021-05-31 18:36:42] queue_downloads
[STOP] [2021-05-31 18:36:42] queue_downloads
[START] [2021-05-31 18:36:42] parse_names
[WARN] [2021-05-31 18:36:42] I see 2770 names which still need to be parsed.
[STOP] [2021-05-31 18:36:45] parse_names
[START] [2021-05-31 18:36:45] denormalize_canonical_names_to_nodes
[STOP] [2021-05-31 18:36:45] denormalize_canonical_names_to_nodes
[START] [2021-05-31 18:36:45] match_nodes
[START] [2021-05-31 18:36:45] map_all_nodes_to_pages
[STOP] [2021-05-31 18:37:18] map_all_nodes_to_pages
[INFO] [2021-05-31 18:37:18] 169 Unmatched nodes (of 2770)! That's too many to output. Full list in /app/public/data/ECOCROP/unmatched_nodes.txt ; First 10: Canonical: Trifolium agrarium; Node#95126789; ResourceID: 10521; Canonical: Trifolium cherlei; Node#95126794; ResourceID: 10528; Canonical: Acacia anticeps; Node#95127561; ResourceID: 2580; Canonical: Acacia macdonnelliensis; Node#95127600; ResourceID: 2651; Canonical: Acacia millifera; Node#95127607; ResourceID: 2662; Canonical: Acacia parchcarpa; Node#95127615; ResourceID: 2677; Canonical: Acacia xiphohylla; Node#95127641; ResourceID: 2720; Canonical: Albizia lucida; Node#95127723; ResourceID: 2965; Canonical: Albizia stipulata; Node#95127871; ResourceID: 352; Canonical: Rhynchosia diversifolia prostata; Node#95128069; ResourceID: 376654
[START] [2021-05-31 18:37:18] update_nodes
[STOP] [2021-05-31 18:37:19] update_nodes
[STOP] [2021-05-31 18:37:19] match_nodes
[START] [2021-05-31 18:37:19] reindex_search
[STOP] [2021-05-31 18:37:21] reindex_search
[START] [2021-05-31 18:37:21] normalize_units
[STOP] [2021-05-31 18:37:54] normalize_units
[START] [2021-05-31 18:37:54] calculate_statistics
[2021-05-31 18:37:55] (NEAR) DUPLICATE TRAITS FOUND! There are only 29135 (of 30515 total) unique traits.
[2021-05-31 18:38:21] (Near) duplicate trait pairs (up to 100):
[2021-05-31 18:38:21] (resource_pk: 10000Q2095PO_0009005, id: 214859735), (resource_pk: 10000Q2095PO_0009010, id: 214859736)
[2021-05-31 18:38:21] (resource_pk: 10015FOODON_03301293PO_0009001, id: 214859739), (resource_pk: 10015FOODON_03301293PO_0025034, id: 214859740)
[2021-05-31 18:38:21] (resource_pk: 10015NCIT_C142605PO_0009001, id: 214859741), (resource_pk: 10015NCIT_C142605PO_0009047, id: 214859742)
[2021-05-31 18:38:21] (resource_pk: 10015NCIT_C142605PO_0009001, id: 214859741), (resource_pk: 10015NCIT_C142605PO_0025034, id: 214859743)
[2021-05-31 18:38:21] (resource_pk: 10015NCIT_C142605PO_0009047, id: 214859742), (resource_pk: 10015NCIT_C142605PO_0025034, id: 214859743)
[2021-05-31 18:38:21] (resource_pk: 10015Q2095PO_0009001, id: 214859745), (resource_pk: 10015Q2095PO_0025034, id: 214859746)
[2021-05-31 18:38:21] (resource_pk: 10015Q2095PO_0009001, id: 214859745), (resource_pk: 10015Q2095PO_0025356, id: 214859747)
[2021-05-31 18:38:21] (resource_pk: 10015Q2095PO_0025034, id: 214859746), (resource_pk: 10015Q2095PO_0025356, id: 214859747)
[2021-05-31 18:38:21] (resource_pk: 1002NCIT_C142605PO_0009001, id: 214859753), (resource_pk: 1002NCIT_C142605PO_0009046, id: 214859754)
[2021-05-31 18:38:21] (resource_pk: 1002NCIT_C142605PO_0009001, id: 214859753), (resource_pk: 1002NCIT_C142605PO_0025034, id: 214859755)
[2021-05-31 18:38:21] (resource_pk: 1002NCIT_C142605PO_0009046, id: 214859754), (resource_pk: 1002NCIT_C142605PO_0025034, id: 214859755)
[2021-05-31 18:38:21] (resource_pk: 1007NCIT_C142605PO_0009005, id: 214859770), (resource_pk: 1007NCIT_C142605PO_0009047, id: 214859771)
[2021-05-31 18:38:21] (resource_pk: 1007Q2095PO_0009046, id: 214859773), (resource_pk: 1007Q2095PO_0025034, id: 214859774)
[2021-05-31 18:38:21] (resource_pk: 10126NCIT_C142605PO_0009001, id: 214859799), (resource_pk: 10126NCIT_C142605PO_0009005, id: 214859800)
[2021-05-31 18:38:21] (resource_pk: 10126NCIT_C142605PO_0009001, id: 214859799), (resource_pk: 10126NCIT_C142605PO_0009047, id: 214859801)
[2021-05-31 18:38:21] (resource_pk: 10126NCIT_C142605PO_0009001, id: 214859799), (resource_pk: 10126NCIT_C142605PO_0025034, id: 214859802)
[2021-05-31 18:38:21] (resource_pk: 10126NCIT_C142605PO_0009005, id: 214859800), (resource_pk: 10126NCIT_C142605PO_0009047, id: 214859801)
[2021-05-31 18:38:21] (resource_pk: 10126NCIT_C142605PO_0009005, id: 214859800), (resource_pk: 10126NCIT_C142605PO_0025034, id: 214859802)
[2021-05-31 18:38:21] (resource_pk: 10126NCIT_C142605PO_0009047, id: 214859801), (resource_pk: 10126NCIT_C142605PO_0025034, id: 214859802)
[2021-05-31 18:38:21] (resource_pk: 10130NCIT_C142605PO_0009001, id: 214859811), (resource_pk: 10130NCIT_C142605PO_0009005, id: 214859812)
[2021-05-31 18:38:21] (resource_pk: 10130NCIT_C142605PO_0009001, id: 214859811), (resource_pk: 10130NCIT_C142605PO_0009047, id: 214859813)
[2021-05-31 18:38:21] (resource_pk: 10130NCIT_C142605PO_0009005, id: 214859812), (resource_pk: 10130NCIT_C142605PO_0009047, id: 214859813)
[2021-05-31 18:38:21] (resource_pk: 10135NCIT_C142605PO_0009001, id: 214859816), (resource_pk: 10135NCIT_C142605PO_0009005, id: 214859817)
[2021-05-31 18:38:21] (resource_pk: 10135NCIT_C142605PO_0009001, id: 214859816), (resource_pk: 10135NCIT_C142605PO_0009047, id: 214859818)
[2021-05-31 18:38:21] (resource_pk: 10135NCIT_C142605PO_0009001, id: 214859816), (resource_pk: 10135NCIT_C142605PO_0025034, id: 214859819)
[2021-05-31 18:38:21] (resource_pk: 10135NCIT_C142605PO_0009005, id: 214859817), (resource_pk: 10135NCIT_C142605PO_0009047, id: 214859818)
[2021-05-31 18:38:21] (resource_pk: 10135NCIT_C142605PO_0009005, id: 214859817), (resource_pk: 10135NCIT_C142605PO_0025034, id: 214859819)
[2021-05-31 18:38:21] (resource_pk: 10135NCIT_C142605PO_0009047, id: 214859818), (resource_pk: 10135NCIT_C142605PO_0025034, id: 214859819)
[2021-05-31 18:38:21] (resource_pk: 10135T131PO_0009001, id: 214859822), (resource_pk: 10135T131PO_0009010, id: 214859823)
[2021-05-31 18:38:21] (resource_pk: 10207FOODON_03301293PO_0009001, id: 214859863), (resource_pk: 10207FOODON_03301293PO_0025034, id: 214859864)
[2021-05-31 18:38:21] (resource_pk: 10207NCIT_C142605PO_0009005, id: 214859865), (resource_pk: 10207NCIT_C142605PO_0009047, id: 214859866)
[2021-05-31 18:38:21] (resource_pk: 10207NCIT_C142605PO_0009005, id: 214859865), (resource_pk: 10207NCIT_C142605PO_0025034, id: 214859867)
[2021-05-31 18:38:21] (resource_pk: 10207NCIT_C142605PO_0009047, id: 214859866), (resource_pk: 10207NCIT_C142605PO_0025034, id: 214859867)
[2021-05-31 18:38:21] (resource_pk: 10223NCIT_C142605PO_0009001, id: 214859881), (resource_pk: 10223NCIT_C142605PO_0009047, id: 214859882)
[2021-05-31 18:38:21] (resource_pk: 10291Q2095PO_0009005, id: 214859931), (resource_pk: 10291Q2095PO_0009046, id: 214859932)
[2021-05-31 18:38:21] (resource_pk: 10291Q2095PO_0009005, id: 214859931), (resource_pk: 10291Q2095PO_0025034, id: 214859933)
[2021-05-31 18:38:21] (resource_pk: 10291Q2095PO_0009046, id: 214859932), (resource_pk: 10291Q2095PO_0025034, id: 214859933)
[2021-05-31 18:38:21] (resource_pk: 10299T131PO_0009010, id: 214859939), (resource_pk: 10299T131PO_0025034, id: 214859940)
[2021-05-31 18:38:21] (resource_pk: 10307Q2095PO_0009010, id: 214859941), (resource_pk: 10307Q2095PO_0025034, id: 214859942)
[2021-05-31 18:38:21] (resource_pk: 10307Q2095PO_0009010, id: 214859941), (resource_pk: 10307Q2095PO_0025356, id: 214859943)
[2021-05-31 18:38:21] (resource_pk: 10307Q2095PO_0025034, id: 214859942), (resource_pk: 10307Q2095PO_0025356, id: 214859943)
[2021-05-31 18:38:21] (resource_pk: 1030NCIT_C142605PO_0009001, id: 214859947), (resource_pk: 1030NCIT_C142605PO_0009010, id: 214859948)
[2021-05-31 18:38:21] (resource_pk: 1030NCIT_C142605PO_0009001, id: 214859947), (resource_pk: 1030NCIT_C142605PO_0009047, id: 214859949)
[2021-05-31 18:38:21] (resource_pk: 1030NCIT_C142605PO_0009001, id: 214859947), (resource_pk: 1030NCIT_C142605PO_0025034, id: 214859950)
[2021-05-31 18:38:21] (resource_pk: 1030NCIT_C142605PO_0009010, id: 214859948), (resource_pk: 1030NCIT_C142605PO_0009047, id: 214859949)
[2021-05-31 18:38:21] (resource_pk: 1030NCIT_C142605PO_0009010, id: 214859948), (resource_pk: 1030NCIT_C142605PO_0025034, id: 214859950)
[2021-05-31 18:38:21] (resource_pk: 1030NCIT_C142605PO_0009047, id: 214859949), (resource_pk: 1030NCIT_C142605PO_0025034, id: 214859950)
[2021-05-31 18:38:21] (resource_pk: 1033NCIT_C142605PO_0009005, id: 214859974), (resource_pk: 1033NCIT_C142605PO_0009047, id: 214859975)
[2021-05-31 18:38:21] (resource_pk: 1033NCIT_C142605PO_0009005, id: 214859974), (resource_pk: 1033NCIT_C142605PO_0025034, id: 214859976)
[2021-05-31 18:38:21] (resource_pk: 1033NCIT_C142605PO_0009047, id: 214859975), (resource_pk: 1033NCIT_C142605PO_0025034, id: 214859976)
[2021-05-31 18:38:21] (resource_pk: 10370Q2095PO_0004518, id: 214859991), (resource_pk: 10370Q2095PO_0025034, id: 214859992)
[2021-05-31 18:38:21] (resource_pk: 10373NCIT_C142605PO_0009005, id: 214859995), (resource_pk: 10373NCIT_C142605PO_0009010, id: 214859996)
[2021-05-31 18:38:21] (resource_pk: 10373NCIT_C142605PO_0009005, id: 214859995), (resource_pk: 10373NCIT_C142605PO_0009047, id: 214859997)
[2021-05-31 18:38:21] (resource_pk: 10373NCIT_C142605PO_0009005, id: 214859995), (resource_pk: 10373NCIT_C142605PO_0025034, id: 214859998)
[2021-05-31 18:38:21] (resource_pk: 10373NCIT_C142605PO_0009010, id: 214859996), (resource_pk: 10373NCIT_C142605PO_0009047, id: 214859997)
[2021-05-31 18:38:21] (resource_pk: 10373NCIT_C142605PO_0009010, id: 214859996), (resource_pk: 10373NCIT_C142605PO_0025034, id: 214859998)
[2021-05-31 18:38:21] (resource_pk: 10373NCIT_C142605PO_0009047, id: 214859997), (resource_pk: 10373NCIT_C142605PO_0025034, id: 214859998)
[2021-05-31 18:38:21] (resource_pk: 10391Q2095PO_0009001, id: 214860005), (resource_pk: 10391Q2095PO_0009010, id: 214860006)
[2021-05-31 18:38:21] (resource_pk: 10394Q2095PO_0009001, id: 214860009), (resource_pk: 10394Q2095PO_0009010, id: 214860010)
[2021-05-31 18:38:21] (resource_pk: 10432FOODON_00001857PO_0009046, id: 214860014), (resource_pk: 10432FOODON_00001857PO_0025034, id: 214860015)
[2021-05-31 18:38:21] (resource_pk: 10473CHEBI_18059PO_0009001, id: 214860040), (resource_pk: 10473CHEBI_18059PO_0009010, id: 214860041)
[2021-05-31 18:38:21] (resource_pk: 10473NCIT_C142605PO_0009001, id: 214860043), (resource_pk: 10473NCIT_C142605PO_0009010, id: 214860044)
[2021-05-31 18:38:21] (resource_pk: 10478Q2095PO_0009005, id: 214860048), (resource_pk: 10478Q2095PO_0025034, id: 214860049)
[2021-05-31 18:38:21] (resource_pk: 10554Q2095PO_0025034, id: 214860094), (resource_pk: 10554Q2095PPO_0001007, id: 214860095)
[2021-05-31 18:38:21] (resource_pk: 1059T131PO_0000003, id: 214860115), (resource_pk: 1059T131Q1139400, id: 214860116)
[2021-05-31 18:38:21] (resource_pk: 10624NCIT_C142605PO_0009046, id: 214860118), (resource_pk: 10624NCIT_C142605PO_0025034, id: 214860119)
[2021-05-31 18:38:21] (resource_pk: 10626Q2095PO_0009005, id: 214860122), (resource_pk: 10626Q2095PO_0009010, id: 214860123)
[2021-05-31 18:38:21] (resource_pk: 10628Q2095PO_0000055, id: 214860128), (resource_pk: 10628Q2095PO_0004518, id: 214860129)
[2021-05-31 18:38:21] (resource_pk: 10628Q2095PO_0000055, id: 214860128), (resource_pk: 10628Q2095PO_0009010, id: 214860130)
[2021-05-31 18:38:21] (resource_pk: 10628Q2095PO_0000055, id: 214860128), (resource_pk: 10628Q2095PO_0009046, id: 214860131)
[2021-05-31 18:38:21] (resource_pk: 10628Q2095PO_0000055, id: 214860128), (resource_pk: 10628Q2095PO_0025356, id: 214860132)
[2021-05-31 18:38:21] (resource_pk: 10628Q2095PO_0004518, id: 214860129), (resource_pk: 10628Q2095PO_0009010, id: 214860130)
[2021-05-31 18:38:21] (resource_pk: 10628Q2095PO_0004518, id: 214860129), (resource_pk: 10628Q2095PO_0009046, id: 214860131)
[2021-05-31 18:38:21] (resource_pk: 10628Q2095PO_0004518, id: 214860129), (resource_pk: 10628Q2095PO_0025356, id: 214860132)
[2021-05-31 18:38:21] (resource_pk: 10628Q2095PO_0009010, id: 214860130), (resource_pk: 10628Q2095PO_0009046, id: 214860131)
[2021-05-31 18:38:21] (resource_pk: 10628Q2095PO_0009010, id: 214860130), (resource_pk: 10628Q2095PO_0025356, id: 214860132)
[2021-05-31 18:38:21] (resource_pk: 10628Q2095PO_0009046, id: 214860131), (resource_pk: 10628Q2095PO_0025356, id: 214860132)
[2021-05-31 18:38:21] (resource_pk: 10631CHEBI_33292PO_0004518, id: 214860133), (resource_pk: 10631CHEBI_33292PO_0025034, id: 214860134)
[2021-05-31 18:38:21] (resource_pk: 10631NCIT_C142605PO_0009010, id: 214860135), (resource_pk: 10631NCIT_C142605PO_0009046, id: 214860136)
[2021-05-31 18:38:21] (resource_pk: 10631NCIT_C142605PO_0009010, id: 214860135), (resource_pk: 10631NCIT_C142605PO_0025034, id: 214860137)
[2021-05-31 18:38:21] (resource_pk: 10631NCIT_C142605PO_0009046, id: 214860136), (resource_pk: 10631NCIT_C142605PO_0025034, id: 214860137)
[2021-05-31 18:38:21] (resource_pk: 10631Q161PO_0000055, id: 214860138), (resource_pk: 10631Q161PO_0025034, id: 214860139)
[2021-05-31 18:38:21] (resource_pk: 10631Q2095PO_0000055, id: 214860140), (resource_pk: 10631Q2095PO_0009010, id: 214860141)
[2021-05-31 18:38:21] (resource_pk: 10631Q2095PO_0000055, id: 214860140), (resource_pk: 10631Q2095PO_0025356, id: 214860142)
[2021-05-31 18:38:21] (resource_pk: 10631Q2095PO_0009010, id: 214860141), (resource_pk: 10631Q2095PO_0025356, id: 214860142)
[2021-05-31 18:38:21] (resource_pk: 10633NCIT_C142605PO_0004518, id: 214860147), (resource_pk: 10633NCIT_C142605PO_0009005, id: 214860148)
[2021-05-31 18:38:21] (resource_pk: 10633NCIT_C142605PO_0004518, id: 214860147), (resource_pk: 10633NCIT_C142605PO_0009010, id: 214860149)
[2021-05-31 18:38:21] (resource_pk: 10633NCIT_C142605PO_0004518, id: 214860147), (resource_pk: 10633NCIT_C142605PO_0009046, id: 214860150)
[2021-05-31 18:38:21] (resource_pk: 10633NCIT_C142605PO_0004518, id: 214860147), (resource_pk: 10633NCIT_C142605PO_0025034, id: 214860151)
[2021-05-31 18:38:21] (resource_pk: 10633NCIT_C142605PO_0009005, id: 214860148), (resource_pk: 10633NCIT_C142605PO_0009010, id: 214860149)
[2021-05-31 18:38:21] (resource_pk: 10633NCIT_C142605PO_0009005, id: 214860148), (resource_pk: 10633NCIT_C142605PO_0009046, id: 214860150)
[2021-05-31 18:38:21] (resource_pk: 10633NCIT_C142605PO_0009005, id: 214860148), (resource_pk: 10633NCIT_C142605PO_0025034, id: 214860151)
[2021-05-31 18:38:21] (resource_pk: 10633NCIT_C142605PO_0009010, id: 214860149), (resource_pk: 10633NCIT_C142605PO_0009046, id: 214860150)
[2021-05-31 18:38:21] (resource_pk: 10633NCIT_C142605PO_0009010, id: 214860149), (resource_pk: 10633NCIT_C142605PO_0025034, id: 214860151)
[2021-05-31 18:38:21] (resource_pk: 10633NCIT_C142605PO_0009046, id: 214860150), (resource_pk: 10633NCIT_C142605PO_0025034, id: 214860151)
[2021-05-31 18:38:21] (resource_pk: 10633Q2095PO_0000055, id: 214860154), (resource_pk: 10633Q2095PO_0004518, id: 214860155)
[2021-05-31 18:38:21] (resource_pk: 10633Q2095PO_0000055, id: 214860154), (resource_pk: 10633Q2095PO_0009010, id: 214860156)
[2021-05-31 18:38:21] (resource_pk: 10633Q2095PO_0000055, id: 214860154), (resource_pk: 10633Q2095PO_0009046, id: 214860157)
[2021-05-31 18:38:21] (resource_pk: 10633Q2095PO_0000055, id: 214860154), (resource_pk: 10633Q2095PO_0025356, id: 214860158)
[2021-05-31 18:38:21] (resource_pk: 10633Q2095PO_0004518, id: 214860155), (resource_pk: 10633Q2095PO_0009010, id: 214860156)
[STOP] [2021-05-31 18:38:22] calculate_statistics
[START] [2021-05-31 18:38:22] complete_harvest_instance
[START] [2021-05-31 18:38:22] overall_tsv_creation
[INFO] [2021-05-31 18:38:22] Processing group of 2770 in 1 batches of 10000
[INFO] [2021-05-31 18:39:02] 30492 Traits (unfiltered)...
[INFO] [2021-05-31 18:40:39] 30492 Traits (filtered)...
[INFO] [2021-05-31 18:40:39] 0 Associations (filtered)...
[INFO] [2021-05-31 18:40:42] 15767 metadata added.
[INFO] [2021-05-31 18:40:42] 0 metadata added.
[INFO] [2021-05-31 18:41:11] Average Time: 146.31
[INFO] [2021-05-31 18:41:11] Total Time: 2m50s
[STOP] [2021-05-31 18:41:11] overall_tsv_creation
[INFO] [2021-05-31 18:41:11] Done. Check your files:
[INFO] [2021-05-31 18:41:11] (2770 lines) /app/public/data/ECOCROP/publish_nodes.tsv
[INFO] [2021-05-31 18:41:11] (5155 lines) /app/public/data/ECOCROP/publish_node_ancestors.tsv
[INFO] [2021-05-31 18:41:12] (2770 lines) /app/public/data/ECOCROP/publish_scientific_names.tsv
[INFO] [2021-05-31 18:41:12] (30493 lines) /app/public/data/ECOCROP/publish_traits.tsv
[INFO] [2021-05-31 18:41:12] (15768 lines) /app/public/data/ECOCROP/publish_metadata.tsv
[STOP] [2021-05-31 18:41:13] complete_harvest_instance
[START] [2021-05-31 18:41:13] completed
[STOP] [2021-05-31 18:41:13] completed
[STOP] [2021-05-31 18:41:13] logged process, took 324.76

Latest Process

31 May 18:35 EXITED currently processing 0