Harvest for Milk Composition of Non Dairy Animals Created 31 May 13:31

Harvesting Log

[INFO] [2021-05-31 13:31:16] Created harvest instance #3916
[STOP] [2021-05-31 13:31:16] create_harvest_instance
[START] [2021-05-31 13:31:16] fetch_files
[STOP] [2021-05-31 13:31:16] fetch_files
[START] [2021-05-31 13:31:16] validate_each_file
[INFO] [2021-05-31 13:31:16] Looping over 4 formats...
[INFO] [2021-05-31 13:31:16] ...refs (/app/public/data/mcnda/reference.tab)
[INFO] [2021-05-31 13:31:16] Valid: /app/public/converted_csv/mcnda_refs_3916.csv (517 lines)
[INFO] [2021-05-31 13:31:16] ...nodes (/app/public/data/mcnda/taxon.tab)
[INFO] [2021-05-31 13:31:16] Valid: /app/public/converted_csv/mcnda_nodes_3916.csv (272 lines)
[INFO] [2021-05-31 13:31:16] ...occurrences (/app/public/data/mcnda/occurrence.txt)
[INFO] [2021-05-31 13:31:17] Valid: /app/public/converted_csv/mcnda_occurrences_3916.csv (3198 lines)
[INFO] [2021-05-31 13:31:17] ...measurements (/app/public/data/mcnda/measurement_or_fact.txt)
[INFO] [2021-05-31 13:31:20] Valid: /app/public/converted_csv/mcnda_measurements_3916.csv (40588 lines)
[STOP] [2021-05-31 13:31:20] validate_each_file
[START] [2021-05-31 13:31:20] convert_to_csv
[INFO] [2021-05-31 13:31:20] Looping over 4 formats...
[INFO] [2021-05-31 13:31:20] ...refs (/app/public/data/mcnda/reference.tab)
[CMD] [2021-05-31 13:31:20] /usr/bin/sort /app/public/converted_csv/mcnda_refs_3916.csv > /app/public/converted_csv/mcnda_refs_3916.csv_sorted
[INFO] [2021-05-31 13:31:21] Converted: /app/public/converted_csv/mcnda_refs_3916.csv (517 lines)
[INFO] [2021-05-31 13:31:21] ...nodes (/app/public/data/mcnda/taxon.tab)
[CMD] [2021-05-31 13:31:21] /usr/bin/sort /app/public/converted_csv/mcnda_nodes_3916.csv > /app/public/converted_csv/mcnda_nodes_3916.csv_sorted
[INFO] [2021-05-31 13:31:21] Converted: /app/public/converted_csv/mcnda_nodes_3916.csv (272 lines)
[INFO] [2021-05-31 13:31:21] ...occurrences (/app/public/data/mcnda/occurrence.txt)
[CMD] [2021-05-31 13:31:21] /usr/bin/sort /app/public/converted_csv/mcnda_occurrences_3916.csv > /app/public/converted_csv/mcnda_occurrences_3916.csv_sorted
[INFO] [2021-05-31 13:31:22] Converted: /app/public/converted_csv/mcnda_occurrences_3916.csv (3198 lines)
[INFO] [2021-05-31 13:31:22] ...measurements (/app/public/data/mcnda/measurement_or_fact.txt)
[CMD] [2021-05-31 13:31:22] /usr/bin/sort /app/public/converted_csv/mcnda_measurements_3916.csv > /app/public/converted_csv/mcnda_measurements_3916.csv_sorted
[INFO] [2021-05-31 13:31:22] Converted: /app/public/converted_csv/mcnda_measurements_3916.csv (40588 lines)
[STOP] [2021-05-31 13:31:22] convert_to_csv
[START] [2021-05-31 13:31:22] calculate_delta
[INFO] [2021-05-31 13:31:22] Looping over 4 formats...
[INFO] [2021-05-31 13:31:22] ...refs (/app/public/data/mcnda/reference.tab)
[CMD] [2021-05-31 13:31:22] echo "0a" > /app/public/diff/mcnda_refs_3916.diff
[CMD] [2021-05-31 13:31:22] tail -n +1 /app/public/converted_csv/mcnda_refs_3916.csv >> /app/public/diff/mcnda_refs_3916.diff
[CMD] [2021-05-31 13:31:23] echo "." >> /app/public/diff/mcnda_refs_3916.diff
[INFO] [2021-05-31 13:31:23] Created diff: /app/public/diff/mcnda_refs_3916.diff (519 lines)
[INFO] [2021-05-31 13:31:23] ...nodes (/app/public/data/mcnda/taxon.tab)
[CMD] [2021-05-31 13:31:23] echo "0a" > /app/public/diff/mcnda_nodes_3916.diff
[CMD] [2021-05-31 13:31:23] tail -n +1 /app/public/converted_csv/mcnda_nodes_3916.csv >> /app/public/diff/mcnda_nodes_3916.diff
[CMD] [2021-05-31 13:31:24] echo "." >> /app/public/diff/mcnda_nodes_3916.diff
[INFO] [2021-05-31 13:31:24] Created diff: /app/public/diff/mcnda_nodes_3916.diff (274 lines)
[INFO] [2021-05-31 13:31:24] ...occurrences (/app/public/data/mcnda/occurrence.txt)
[CMD] [2021-05-31 13:31:24] echo "0a" > /app/public/diff/mcnda_occurrences_3916.diff
[CMD] [2021-05-31 13:31:25] tail -n +1 /app/public/converted_csv/mcnda_occurrences_3916.csv >> /app/public/diff/mcnda_occurrences_3916.diff
[CMD] [2021-05-31 13:31:25] echo "." >> /app/public/diff/mcnda_occurrences_3916.diff
[INFO] [2021-05-31 13:31:25] Created diff: /app/public/diff/mcnda_occurrences_3916.diff (3200 lines)
[INFO] [2021-05-31 13:31:25] ...measurements (/app/public/data/mcnda/measurement_or_fact.txt)
[CMD] [2021-05-31 13:31:25] echo "0a" > /app/public/diff/mcnda_measurements_3916.diff
[CMD] [2021-05-31 13:31:26] tail -n +1 /app/public/converted_csv/mcnda_measurements_3916.csv >> /app/public/diff/mcnda_measurements_3916.diff
[CMD] [2021-05-31 13:31:26] echo "." >> /app/public/diff/mcnda_measurements_3916.diff
[INFO] [2021-05-31 13:31:27] Created diff: /app/public/diff/mcnda_measurements_3916.diff (40590 lines)
[STOP] [2021-05-31 13:31:27] calculate_delta
[START] [2021-05-31 13:31:27] parse_diff_and_store
[INFO] [2021-05-31 13:31:27] Handling diff: /app/public/diff/mcnda_refs_3916.diff (519 lines)
[INFO] [2021-05-31 13:31:27] Loading refs diff file into memory (519 /app/public/diff/mcnda_refs_3916.diff lines)...
[INFO] [2021-05-31 13:31:27] Handling diff: /app/public/diff/mcnda_nodes_3916.diff (274 lines)
[INFO] [2021-05-31 13:31:28] Loading nodes diff file into memory (274 /app/public/diff/mcnda_nodes_3916.diff lines)...
[INFO] [2021-05-31 13:31:28] Handling diff: /app/public/diff/mcnda_occurrences_3916.diff (3200 lines)
[INFO] [2021-05-31 13:31:29] Loading occurrences diff file into memory (3200 /app/public/diff/mcnda_occurrences_3916.diff lines)...
[INFO] [2021-05-31 13:31:29] Handling diff: /app/public/diff/mcnda_measurements_3916.diff (40590 lines)
[INFO] [2021-05-31 13:31:30] Loading measurements diff file into memory (40590 /app/public/diff/mcnda_measurements_3916.diff lines)...
[INFO] [2021-05-31 13:31:45] Storing 517 References
[INFO] [2021-05-31 13:31:45] Processing group of 517 in 1 groups of 1000
[INFO] [2021-05-31 13:31:45] Average Time: 0.09
[INFO] [2021-05-31 13:31:45] Total Time: 1s
[INFO] [2021-05-31 13:31:45] Storing 449 ScientificNames
[INFO] [2021-05-31 13:31:45] Processing group of 449 in 1 groups of 1000
[INFO] [2021-05-31 13:31:45] Average Time: 0.13
[INFO] [2021-05-31 13:31:45] Total Time: 1s
[INFO] [2021-05-31 13:31:45] Storing 449 Nodes
[INFO] [2021-05-31 13:31:45] Processing group of 449 in 1 groups of 1000
[INFO] [2021-05-31 13:31:45] Average Time: 0.13
[INFO] [2021-05-31 13:31:45] Total Time: 1s
[INFO] [2021-05-31 13:31:45] Storing 3198 Occurrences
[INFO] [2021-05-31 13:31:45] Processing group of 3198 in 4 groups of 1000
[INFO] [2021-05-31 13:31:45] Average Time: 0.08
[INFO] [2021-05-31 13:31:45] Total Time: 1s
[INFO] [2021-05-31 13:31:45] Storing 40588 Traits
[INFO] [2021-05-31 13:31:45] Processing group of 40588 in 41 groups of 1000
[INFO] [2021-05-31 13:31:57] Average Time: 0.288
[INFO] [2021-05-31 13:31:57] Total Time: 12s
[INFO] [2021-05-31 13:31:57] last 3 / first 3: 0.89
[INFO] [2021-05-31 13:31:57] Std.Dev: 0.03162277660168379; Max: 0.38
[INFO] [2021-05-31 13:31:57] Storing 25999 MetaTraits
[INFO] [2021-05-31 13:31:57] Processing group of 25999 in 26 groups of 1000
[INFO] [2021-05-31 13:32:01] Average Time: 0.151
[INFO] [2021-05-31 13:32:01] Total Time: 5s
[INFO] [2021-05-31 13:32:01] last 3 / first 3: 0.95
[INFO] [2021-05-31 13:32:01] Std.Dev: 0.03162277660168379; Max: 0.2
[INFO] [2021-05-31 13:32:01] Storing 15364 TraitsReferences
[INFO] [2021-05-31 13:32:01] Processing group of 15364 in 16 groups of 1000
[INFO] [2021-05-31 13:32:02] Average Time: 0.059
[INFO] [2021-05-31 13:32:02] Total Time: 2s
[INFO] [2021-05-31 13:32:02] last 3 / first 3: 0.83
[INFO] [2021-05-31 13:32:02] Std.Dev: 0.0; Max: 0.07
[STOP] [2021-05-31 13:32:02] parse_diff_and_store
[START] [2021-05-31 13:32:03] resolve_keys
[INFO] [2021-05-31 13:32:08] Occurrences to nodes (through scientific_names)...
[INFO] [2021-05-31 13:32:08] traits to occurrences...
[INFO] [2021-05-31 13:32:09] traits to nodes (through occurrences)...
[INFO] [2021-05-31 13:32:09] Traits to sex term...
[INFO] [2021-05-31 13:32:09] Traits to lifestage term...
[INFO] [2021-05-31 13:32:09] MetaTraits to traits...
[INFO] [2021-05-31 13:32:10] MetaTraits (simple, measurement row refers to parent) to traits...
[INFO] [2021-05-31 13:32:12] Assocs to occurrences...
[INFO] [2021-05-31 13:32:12] Assocs to nodes...
[INFO] [2021-05-31 13:32:12] Assoc to sex term...
[INFO] [2021-05-31 13:32:12] Assoc to lifestage term...
[INFO] [2021-05-31 13:32:12] MetaAssoc to assocs...
[STOP] [2021-05-31 13:32:12] resolve_keys
[START] [2021-05-31 13:32:12] hold_for_later_1
[STOP] [2021-05-31 13:32:12] hold_for_later_1
[START] [2021-05-31 13:32:12] hold_for_later_2
[STOP] [2021-05-31 13:32:12] hold_for_later_2
[START] [2021-05-31 13:32:12] resolve_missing_parents
[STOP] [2021-05-31 13:32:12] resolve_missing_parents
[START] [2021-05-31 13:32:12] rebuild_nodes
[START] [2021-05-31 13:32:12] Flattener#flatten
[START] [2021-05-31 13:32:12] Flattener#study_resource
[START] [2021-05-31 13:32:12] Flattener#build_ancestry
[STOP] [2021-05-31 13:32:12] Flattener#build_ancestry
[INFO] [2021-05-31 13:32:12] 449 ancestry keys
[START] [2021-05-31 13:32:12] build_node_ancestors
[INFO] [2021-05-31 13:32:12] old ancestors deleted.
[STOP] [2021-05-31 13:32:12] build_node_ancestors
[START] [2021-05-31 13:32:12] Flattener#propagate_ancestor_ids
[STOP] [2021-05-31 13:32:12] Flattener#propagate_ancestor_ids
[STOP] [2021-05-31 13:32:12] Flattener#flatten
[STOP] [2021-05-31 13:32:12] rebuild_nodes
[START] [2021-05-31 13:32:12] resolve_missing_media_owners
[STOP] [2021-05-31 13:32:12] resolve_missing_media_owners
[START] [2021-05-31 13:32:12] sanitize_media_verbatims
[STOP] [2021-05-31 13:32:12] sanitize_media_verbatims
[START] [2021-05-31 13:32:12] queue_downloads
[STOP] [2021-05-31 13:32:12] queue_downloads
[START] [2021-05-31 13:32:12] parse_names
[WARN] [2021-05-31 13:32:12] I see 449 names which still need to be parsed.
[WARN] [2021-05-31 13:32:13] I see 6 names which still need to be parsed.
[STOP] [2021-05-31 13:32:14] parse_names
[START] [2021-05-31 13:32:14] denormalize_canonical_names_to_nodes
[STOP] [2021-05-31 13:32:14] denormalize_canonical_names_to_nodes
[START] [2021-05-31 13:32:14] match_nodes
[START] [2021-05-31 13:32:14] map_all_nodes_to_pages
[STOP] [2021-05-31 13:32:40] map_all_nodes_to_pages
[INFO] [2021-05-31 13:32:40] 22 Unmatched nodes (of 449)! That's too many to output. Full list in /app/public/data/mcnda/unmatched_nodes.txt ; First 10: Canonical: Alopex; Node#95096570; ResourceID: Alopex; Canonical: Cercopithecus talapoin; Node#95096644; ResourceID: Cercopithecus_talapoin; Canonical: Cervus dama; Node#95096646; ResourceID: Cervus_dama; Canonical: Cynopterus horsfeldii; Node#95096659; ResourceID: Cynopterus_horsfeldii; Canonical: Eumetopias jubata; Node#95096706; ResourceID: Eumetopias_jubata; Canonical: Lemur fulvus; Node#95096754; ResourceID: Lemur_fulvus; Canonical: Lemur macaco; Node#95096755; ResourceID: Lemur_macaco; Canonical: Lemur mongoz; Node#95096756; ResourceID: Lemur_mongoz; Canonical: Lemur rubriventer; Node#95096757; ResourceID: Lemur_rubriventer; Canonical: Lepus townsendi; Node#95096768; ResourceID: Lepus_townsendi
[START] [2021-05-31 13:32:40] update_nodes
[STOP] [2021-05-31 13:32:41] update_nodes
[STOP] [2021-05-31 13:32:41] match_nodes
[START] [2021-05-31 13:32:41] reindex_search
[STOP] [2021-05-31 13:32:41] reindex_search
[START] [2021-05-31 13:32:41] normalize_units
[STOP] [2021-05-31 13:33:33] normalize_units
[START] [2021-05-31 13:33:33] calculate_statistics
[2021-05-31 13:33:33] (NEAR) DUPLICATE TRAITS FOUND! There are only 6094 (of 14563 total) unique traits.
[2021-05-31 13:33:36] (Near) duplicate trait pairs (up to 100):
[2021-05-31 13:33:36] (resource_pk: M_00000001, id: 214723587), (resource_pk: M_00000019, id: 214723605)
[2021-05-31 13:33:36] (resource_pk: M_00000001, id: 214723587), (resource_pk: M_00000036, id: 214723622)
[2021-05-31 13:33:36] (resource_pk: M_00000002, id: 214723588), (resource_pk: M_00000029, id: 214723615)
[2021-05-31 13:33:36] (resource_pk: M_00000002, id: 214723588), (resource_pk: M_00000039, id: 214723625)
[2021-05-31 13:33:36] (resource_pk: M_00000003, id: 214723589), (resource_pk: M_00000024, id: 214723610)
[2021-05-31 13:33:36] (resource_pk: M_00000003, id: 214723589), (resource_pk: M_00000038, id: 214723624)
[2021-05-31 13:33:36] (resource_pk: M_00000004, id: 214723590), (resource_pk: M_00000027, id: 214723613)
[2021-05-31 13:33:36] (resource_pk: M_00000004, id: 214723590), (resource_pk: M_00000037, id: 214723623)
[2021-05-31 13:33:36] (resource_pk: M_00000005, id: 214723591), (resource_pk: M_00000022, id: 214723608)
[2021-05-31 13:33:36] (resource_pk: M_00000005, id: 214723591), (resource_pk: M_00000040, id: 214723626)
[2021-05-31 13:33:36] (resource_pk: M_00000006, id: 214723592), (resource_pk: M_00000025, id: 214723611)
[2021-05-31 13:33:36] (resource_pk: M_00000006, id: 214723592), (resource_pk: M_00000049, id: 214723635)
[2021-05-31 13:33:36] (resource_pk: M_00000008, id: 214723594), (resource_pk: M_00000023, id: 214723609)
[2021-05-31 13:33:36] (resource_pk: M_00000008, id: 214723594), (resource_pk: M_00000041, id: 214723627)
[2021-05-31 13:33:36] (resource_pk: M_00000009, id: 214723595), (resource_pk: M_00000028, id: 214723614)
[2021-05-31 13:33:36] (resource_pk: M_00000009, id: 214723595), (resource_pk: M_00000042, id: 214723628)
[2021-05-31 13:33:36] (resource_pk: M_00000010, id: 214723596), (resource_pk: M_00000021, id: 214723607)
[2021-05-31 13:33:36] (resource_pk: M_00000010, id: 214723596), (resource_pk: M_00000044, id: 214723630)
[2021-05-31 13:33:36] (resource_pk: M_00000011, id: 214723597), (resource_pk: M_00000026, id: 214723612)
[2021-05-31 13:33:36] (resource_pk: M_00000011, id: 214723597), (resource_pk: M_00000043, id: 214723629)
[2021-05-31 13:33:36] (resource_pk: M_00000012, id: 214723598), (resource_pk: M_00000020, id: 214723606)
[2021-05-31 13:33:36] (resource_pk: M_00000012, id: 214723598), (resource_pk: M_00000046, id: 214723632)
[2021-05-31 13:33:36] (resource_pk: M_00000013, id: 214723599), (resource_pk: M_00000030, id: 214723616)
[2021-05-31 13:33:36] (resource_pk: M_00000013, id: 214723599), (resource_pk: M_00000047, id: 214723633)
[2021-05-31 13:33:36] (resource_pk: M_00000014, id: 214723600), (resource_pk: M_00000031, id: 214723617)
[2021-05-31 13:33:36] (resource_pk: M_00000014, id: 214723600), (resource_pk: M_00000050, id: 214723636)
[2021-05-31 13:33:36] (resource_pk: M_00000015, id: 214723601), (resource_pk: M_00000034, id: 214723620)
[2021-05-31 13:33:36] (resource_pk: M_00000015, id: 214723601), (resource_pk: M_00000045, id: 214723631)
[2021-05-31 13:33:36] (resource_pk: M_00000016, id: 214723602), (resource_pk: M_00000032, id: 214723618)
[2021-05-31 13:33:36] (resource_pk: M_00000016, id: 214723602), (resource_pk: M_00000048, id: 214723634)
[2021-05-31 13:33:36] (resource_pk: M_00000017, id: 214723603), (resource_pk: M_00000033, id: 214723619)
[2021-05-31 13:33:36] (resource_pk: M_00000017, id: 214723603), (resource_pk: M_00000051, id: 214723637)
[2021-05-31 13:33:36] (resource_pk: M_00000018, id: 214723604), (resource_pk: M_00000035, id: 214723621)
[2021-05-31 13:33:36] (resource_pk: M_00000018, id: 214723604), (resource_pk: M_00000052, id: 214723638)
[2021-05-31 13:33:36] (resource_pk: M_00000019, id: 214723605), (resource_pk: M_00000036, id: 214723622)
[2021-05-31 13:33:36] (resource_pk: M_00000020, id: 214723606), (resource_pk: M_00000046, id: 214723632)
[2021-05-31 13:33:36] (resource_pk: M_00000021, id: 214723607), (resource_pk: M_00000044, id: 214723630)
[2021-05-31 13:33:36] (resource_pk: M_00000022, id: 214723608), (resource_pk: M_00000040, id: 214723626)
[2021-05-31 13:33:36] (resource_pk: M_00000023, id: 214723609), (resource_pk: M_00000041, id: 214723627)
[2021-05-31 13:33:36] (resource_pk: M_00000024, id: 214723610), (resource_pk: M_00000038, id: 214723624)
[2021-05-31 13:33:36] (resource_pk: M_00000025, id: 214723611), (resource_pk: M_00000049, id: 214723635)
[2021-05-31 13:33:36] (resource_pk: M_00000026, id: 214723612), (resource_pk: M_00000043, id: 214723629)
[2021-05-31 13:33:36] (resource_pk: M_00000027, id: 214723613), (resource_pk: M_00000037, id: 214723623)
[2021-05-31 13:33:36] (resource_pk: M_00000028, id: 214723614), (resource_pk: M_00000042, id: 214723628)
[2021-05-31 13:33:36] (resource_pk: M_00000029, id: 214723615), (resource_pk: M_00000039, id: 214723625)
[2021-05-31 13:33:36] (resource_pk: M_00000030, id: 214723616), (resource_pk: M_00000047, id: 214723633)
[2021-05-31 13:33:36] (resource_pk: M_00000031, id: 214723617), (resource_pk: M_00000050, id: 214723636)
[2021-05-31 13:33:36] (resource_pk: M_00000032, id: 214723618), (resource_pk: M_00000048, id: 214723634)
[2021-05-31 13:33:36] (resource_pk: M_00000033, id: 214723619), (resource_pk: M_00000051, id: 214723637)
[2021-05-31 13:33:36] (resource_pk: M_00000034, id: 214723620), (resource_pk: M_00000045, id: 214723631)
[2021-05-31 13:33:36] (resource_pk: M_00000035, id: 214723621), (resource_pk: M_00000052, id: 214723638)
[2021-05-31 13:33:36] (resource_pk: M_00000053, id: 214723639), (resource_pk: M_00000070, id: 214723656)
[2021-05-31 13:33:36] (resource_pk: M_00000054, id: 214723640), (resource_pk: M_00000074, id: 214723660)
[2021-05-31 13:33:36] (resource_pk: M_00000055, id: 214723641), (resource_pk: M_00000082, id: 214723668)
[2021-05-31 13:33:36] (resource_pk: M_00000056, id: 214723642), (resource_pk: M_00000076, id: 214723662)
[2021-05-31 13:33:36] (resource_pk: M_00000057, id: 214723643), (resource_pk: M_00000071, id: 214723657)
[2021-05-31 13:33:36] (resource_pk: M_00000058, id: 214723644), (resource_pk: M_00000072, id: 214723658)
[2021-05-31 13:33:36] (resource_pk: M_00000059, id: 214723645), (resource_pk: M_00000077, id: 214723663)
[2021-05-31 13:33:36] (resource_pk: M_00000060, id: 214723646), (resource_pk: M_00000079, id: 214723665)
[2021-05-31 13:33:36] (resource_pk: M_00000061, id: 214723647), (resource_pk: M_00000078, id: 214723664)
[2021-05-31 13:33:36] (resource_pk: M_00000062, id: 214723648), (resource_pk: M_00000075, id: 214723661)
[2021-05-31 13:33:36] (resource_pk: M_00000063, id: 214723649), (resource_pk: M_00000073, id: 214723659)
[2021-05-31 13:33:36] (resource_pk: M_00000064, id: 214723650), (resource_pk: M_00000083, id: 214723669)
[2021-05-31 13:33:36] (resource_pk: M_00000065, id: 214723651), (resource_pk: M_00000080, id: 214723666)
[2021-05-31 13:33:36] (resource_pk: M_00000066, id: 214723652), (resource_pk: M_00000081, id: 214723667)
[2021-05-31 13:33:36] (resource_pk: M_00000067, id: 214723653), (resource_pk: M_00000084, id: 214723670)
[2021-05-31 13:33:36] (resource_pk: M_00000068, id: 214723654), (resource_pk: M_00000086, id: 214723672)
[2021-05-31 13:33:36] (resource_pk: M_00000069, id: 214723655), (resource_pk: M_00000085, id: 214723671)
[2021-05-31 13:33:36] (resource_pk: M_00000087, id: 214723673), (resource_pk: M_00000105, id: 214723691)
[2021-05-31 13:33:36] (resource_pk: M_00000088, id: 214723674), (resource_pk: M_00000107, id: 214723693)
[2021-05-31 13:33:36] (resource_pk: M_00000089, id: 214723675), (resource_pk: M_00000106, id: 214723692)
[2021-05-31 13:33:36] (resource_pk: M_00000090, id: 214723676), (resource_pk: M_00000114, id: 214723700)
[2021-05-31 13:33:36] (resource_pk: M_00000091, id: 214723677), (resource_pk: M_00000115, id: 214723701)
[2021-05-31 13:33:36] (resource_pk: M_00000092, id: 214723678), (resource_pk: M_00000110, id: 214723696)
[2021-05-31 13:33:36] (resource_pk: M_00000093, id: 214723679), (resource_pk: M_00000108, id: 214723694)
[2021-05-31 13:33:36] (resource_pk: M_00000094, id: 214723680), (resource_pk: M_00000113, id: 214723699)
[2021-05-31 13:33:36] (resource_pk: M_00000095, id: 214723681), (resource_pk: M_00000111, id: 214723697)
[2021-05-31 13:33:36] (resource_pk: M_00000096, id: 214723682), (resource_pk: M_00000109, id: 214723695)
[2021-05-31 13:33:36] (resource_pk: M_00000097, id: 214723683), (resource_pk: M_00000116, id: 214723702)
[2021-05-31 13:33:36] (resource_pk: M_00000098, id: 214723684), (resource_pk: M_00000118, id: 214723704)
[2021-05-31 13:33:36] (resource_pk: M_00000099, id: 214723685), (resource_pk: M_00000112, id: 214723698)
[2021-05-31 13:33:36] (resource_pk: M_00000100, id: 214723686), (resource_pk: M_00000121, id: 214723707)
[2021-05-31 13:33:36] (resource_pk: M_00000101, id: 214723687), (resource_pk: M_00000117, id: 214723703)
[2021-05-31 13:33:36] (resource_pk: M_00000102, id: 214723688), (resource_pk: M_00000119, id: 214723705)
[2021-05-31 13:33:36] (resource_pk: M_00000103, id: 214723689), (resource_pk: M_00000120, id: 214723706)
[2021-05-31 13:33:36] (resource_pk: M_00000104, id: 214723690), (resource_pk: M_00000122, id: 214723708)
[2021-05-31 13:33:36] (resource_pk: M_00000123, id: 214723709), (resource_pk: M_00000143, id: 214723729)
[2021-05-31 13:33:36] (resource_pk: M_00000123, id: 214723709), (resource_pk: M_00000162, id: 214723748)
[2021-05-31 13:33:36] (resource_pk: M_00000123, id: 214723709), (resource_pk: M_00000183, id: 214723769)
[2021-05-31 13:33:36] (resource_pk: M_00000123, id: 214723709), (resource_pk: M_00000200, id: 214723786)
[2021-05-31 13:33:36] (resource_pk: M_00000123, id: 214723709), (resource_pk: M_00000218, id: 214723804)
[2021-05-31 13:33:36] (resource_pk: M_00000123, id: 214723709), (resource_pk: M_00000239, id: 214723825)
[2021-05-31 13:33:36] (resource_pk: M_00000124, id: 214723710), (resource_pk: M_00000142, id: 214723728)
[2021-05-31 13:33:36] (resource_pk: M_00000124, id: 214723710), (resource_pk: M_00000160, id: 214723746)
[2021-05-31 13:33:36] (resource_pk: M_00000124, id: 214723710), (resource_pk: M_00000184, id: 214723770)
[2021-05-31 13:33:36] (resource_pk: M_00000124, id: 214723710), (resource_pk: M_00000209, id: 214723795)
[2021-05-31 13:33:36] (resource_pk: M_00000124, id: 214723710), (resource_pk: M_00000220, id: 214723806)
[2021-05-31 13:33:36] (resource_pk: M_00000124, id: 214723710), (resource_pk: M_00000230, id: 214723816)
[2021-05-31 13:33:36] (resource_pk: M_00000125, id: 214723711), (resource_pk: M_00000152, id: 214723738)
[2021-05-31 13:33:36] (resource_pk: M_00000125, id: 214723711), (resource_pk: M_00000172, id: 214723758)
[STOP] [2021-05-31 13:33:36] calculate_statistics
[START] [2021-05-31 13:33:36] complete_harvest_instance
[START] [2021-05-31 13:33:36] overall_tsv_creation
[INFO] [2021-05-31 13:33:36] Processing group of 449 in 1 batches of 10000
[INFO] [2021-05-31 13:34:11] 14563 Traits (unfiltered)...
[INFO] [2021-05-31 13:35:13] 14563 Traits (filtered)...
[INFO] [2021-05-31 13:35:13] 0 Associations (filtered)...
[INFO] [2021-05-31 13:35:16] 29699 metadata added.
[INFO] [2021-05-31 13:35:16] 0 metadata added.
[INFO] [2021-05-31 13:35:42] Average Time: 103.97
[INFO] [2021-05-31 13:35:42] Total Time: 2m7s
[STOP] [2021-05-31 13:35:42] overall_tsv_creation
[INFO] [2021-05-31 13:35:42] Done. Check your files:
[INFO] [2021-05-31 13:35:43] (449 lines) /app/public/data/mcnda/publish_nodes.tsv
[INFO] [2021-05-31 13:35:43] (720 lines) /app/public/data/mcnda/publish_node_ancestors.tsv
[INFO] [2021-05-31 13:35:44] (449 lines) /app/public/data/mcnda/publish_scientific_names.tsv
[INFO] [2021-05-31 13:35:44] (14564 lines) /app/public/data/mcnda/publish_traits.tsv
[INFO] [2021-05-31 13:35:44] (29700 lines) /app/public/data/mcnda/publish_metadata.tsv
[STOP] [2021-05-31 13:35:44] complete_harvest_instance
[START] [2021-05-31 13:35:44] completed
[STOP] [2021-05-31 13:35:44] completed
[STOP] [2021-05-31 13:35:44] logged process, took 268.52

Latest Process

31 May 13:31 EXITED currently processing 0