Harvest for The US National Vegetation Classification USNVC Created 22 Jul 12:20

Harvesting Log

[INFO] [2021-07-22 12:20:32] Created harvest instance #4044
[STOP] [2021-07-22 12:20:32] create_harvest_instance
[START] [2021-07-22 12:20:32] fetch_files
[STOP] [2021-07-22 12:20:32] fetch_files
[START] [2021-07-22 12:20:32] validate_each_file
[INFO] [2021-07-22 12:20:32] Created new folder: /app/public/converted_csv
[INFO] [2021-07-22 12:20:32] Looping over 3 formats...
[INFO] [2021-07-22 12:20:32] ...nodes (/app/public/data/tnvct/taxa.txt)
[INFO] [2021-07-22 12:20:32] Valid: /app/public/converted_csv/tnvct_nodes_4044.csv (3208 lines)
[INFO] [2021-07-22 12:20:32] ...occurrences (/app/public/data/tnvct/occurrences.txt)
[INFO] [2021-07-22 12:20:32] Valid: /app/public/converted_csv/tnvct_occurrences_4044.csv (3208 lines)
[INFO] [2021-07-22 12:20:32] ...measurements (/app/public/data/tnvct/measurementsorfacts.txt)
[INFO] [2021-07-22 12:20:34] Valid: /app/public/converted_csv/tnvct_measurements_4044.csv (44442 lines)
[STOP] [2021-07-22 12:20:34] validate_each_file
[START] [2021-07-22 12:20:34] convert_to_csv
[INFO] [2021-07-22 12:20:34] Looping over 3 formats...
[INFO] [2021-07-22 12:20:34] ...nodes (/app/public/data/tnvct/taxa.txt)
[CMD] [2021-07-22 12:20:34] /usr/bin/sort /app/public/converted_csv/tnvct_nodes_4044.csv > /app/public/converted_csv/tnvct_nodes_4044.csv_sorted
[INFO] [2021-07-22 12:20:34] Converted: /app/public/converted_csv/tnvct_nodes_4044.csv (3208 lines)
[INFO] [2021-07-22 12:20:34] ...occurrences (/app/public/data/tnvct/occurrences.txt)
[CMD] [2021-07-22 12:20:34] /usr/bin/sort /app/public/converted_csv/tnvct_occurrences_4044.csv > /app/public/converted_csv/tnvct_occurrences_4044.csv_sorted
[INFO] [2021-07-22 12:20:34] Converted: /app/public/converted_csv/tnvct_occurrences_4044.csv (3208 lines)
[INFO] [2021-07-22 12:20:34] ...measurements (/app/public/data/tnvct/measurementsorfacts.txt)
[CMD] [2021-07-22 12:20:34] /usr/bin/sort /app/public/converted_csv/tnvct_measurements_4044.csv > /app/public/converted_csv/tnvct_measurements_4044.csv_sorted
[INFO] [2021-07-22 12:20:34] Converted: /app/public/converted_csv/tnvct_measurements_4044.csv (44442 lines)
[STOP] [2021-07-22 12:20:34] convert_to_csv
[START] [2021-07-22 12:20:34] calculate_delta
[INFO] [2021-07-22 12:20:34] Created diff dir: /app/public/diff
[INFO] [2021-07-22 12:20:34] Looping over 3 formats...
[INFO] [2021-07-22 12:20:34] ...nodes (/app/public/data/tnvct/taxa.txt)
[CMD] [2021-07-22 12:20:34] echo "0a" > /app/public/diff/tnvct_nodes_4044.diff
[CMD] [2021-07-22 12:20:34] tail -n +1 /app/public/converted_csv/tnvct_nodes_4044.csv >> /app/public/diff/tnvct_nodes_4044.diff
[CMD] [2021-07-22 12:20:34] echo "." >> /app/public/diff/tnvct_nodes_4044.diff
[INFO] [2021-07-22 12:20:34] Created diff: /app/public/diff/tnvct_nodes_4044.diff (3210 lines)
[INFO] [2021-07-22 12:20:34] ...occurrences (/app/public/data/tnvct/occurrences.txt)
[CMD] [2021-07-22 12:20:34] echo "0a" > /app/public/diff/tnvct_occurrences_4044.diff
[CMD] [2021-07-22 12:20:34] tail -n +1 /app/public/converted_csv/tnvct_occurrences_4044.csv >> /app/public/diff/tnvct_occurrences_4044.diff
[CMD] [2021-07-22 12:20:34] echo "." >> /app/public/diff/tnvct_occurrences_4044.diff
[INFO] [2021-07-22 12:20:34] Created diff: /app/public/diff/tnvct_occurrences_4044.diff (3210 lines)
[INFO] [2021-07-22 12:20:34] ...measurements (/app/public/data/tnvct/measurementsorfacts.txt)
[CMD] [2021-07-22 12:20:34] echo "0a" > /app/public/diff/tnvct_measurements_4044.diff
[CMD] [2021-07-22 12:20:34] tail -n +1 /app/public/converted_csv/tnvct_measurements_4044.csv >> /app/public/diff/tnvct_measurements_4044.diff
[CMD] [2021-07-22 12:20:34] echo "." >> /app/public/diff/tnvct_measurements_4044.diff
[INFO] [2021-07-22 12:20:34] Created diff: /app/public/diff/tnvct_measurements_4044.diff (44444 lines)
[STOP] [2021-07-22 12:20:34] calculate_delta
[START] [2021-07-22 12:20:34] parse_diff_and_store
[INFO] [2021-07-22 12:20:34] Handling diff: /app/public/diff/tnvct_nodes_4044.diff (3210 lines)
[INFO] [2021-07-22 12:20:34] Loading nodes diff file into memory (3210 /app/public/diff/tnvct_nodes_4044.diff lines)...
[INFO] [2021-07-22 12:20:35] Handling diff: /app/public/diff/tnvct_occurrences_4044.diff (3210 lines)
[INFO] [2021-07-22 12:20:35] Loading occurrences diff file into memory (3210 /app/public/diff/tnvct_occurrences_4044.diff lines)...
[INFO] [2021-07-22 12:20:35] Handling diff: /app/public/diff/tnvct_measurements_4044.diff (44444 lines)
[INFO] [2021-07-22 12:20:35] Loading measurements diff file into memory (44444 /app/public/diff/tnvct_measurements_4044.diff lines)...
[INFO] [2021-07-22 12:20:52] Storing 3225 ScientificNames
[INFO] [2021-07-22 12:20:52] Processing group of 3225 in 4 groups of 1000
[INFO] [2021-07-22 12:20:53] Average Time: 0.22
[INFO] [2021-07-22 12:20:53] Total Time: 1s
[INFO] [2021-07-22 12:20:53] Storing 3225 Nodes
[INFO] [2021-07-22 12:20:53] Processing group of 3225 in 4 groups of 1000
[INFO] [2021-07-22 12:20:54] Average Time: 0.205
[INFO] [2021-07-22 12:20:54] Total Time: 1s
[INFO] [2021-07-22 12:20:54] Storing 3208 Occurrences
[INFO] [2021-07-22 12:20:54] Processing group of 3208 in 4 groups of 1000
[INFO] [2021-07-22 12:20:55] Average Time: 0.085
[INFO] [2021-07-22 12:20:55] Total Time: 1s
[INFO] [2021-07-22 12:20:55] Storing 44442 Traits
[INFO] [2021-07-22 12:20:55] Processing group of 44442 in 45 groups of 1000
[INFO] [2021-07-22 12:21:09] Average Time: 0.312
[INFO] [2021-07-22 12:21:09] Total Time: 15s
[INFO] [2021-07-22 12:21:09] last 3 / first 3: 0.62
[INFO] [2021-07-22 12:21:09] Std.Dev: 0.07071067811865475; Max: 0.51
[INFO] [2021-07-22 12:21:09] Storing 20222 MetaTraits
[INFO] [2021-07-22 12:21:09] Processing group of 20222 in 21 groups of 1000
[INFO] [2021-07-22 12:21:11] Average Time: 0.116
[INFO] [2021-07-22 12:21:11] Total Time: 3s
[INFO] [2021-07-22 12:21:11] last 3 / first 3: 0.71
[INFO] [2021-07-22 12:21:11] Std.Dev: 0.03162277660168379; Max: 0.17
[STOP] [2021-07-22 12:21:11] parse_diff_and_store
[START] [2021-07-22 12:21:11] resolve_keys
[INFO] [2021-07-22 12:21:18] Occurrences to nodes (through scientific_names)...
[INFO] [2021-07-22 12:21:18] traits to occurrences...
[INFO] [2021-07-22 12:21:19] traits to nodes (through occurrences)...
[INFO] [2021-07-22 12:21:19] Traits to sex term...
[INFO] [2021-07-22 12:21:20] Traits to lifestage term...
[INFO] [2021-07-22 12:21:20] MetaTraits to traits...
[INFO] [2021-07-22 12:21:20] MetaTraits (simple, measurement row refers to parent) to traits...
[INFO] [2021-07-22 12:21:21] Assocs to occurrences...
[INFO] [2021-07-22 12:21:21] Assocs to nodes...
[INFO] [2021-07-22 12:21:21] Assoc to sex term...
[INFO] [2021-07-22 12:21:21] Assoc to lifestage term...
[INFO] [2021-07-22 12:21:21] MetaAssoc to assocs...
[STOP] [2021-07-22 12:21:21] resolve_keys
[START] [2021-07-22 12:21:21] hold_for_later_1
[STOP] [2021-07-22 12:21:21] hold_for_later_1
[START] [2021-07-22 12:21:21] hold_for_later_2
[STOP] [2021-07-22 12:21:21] hold_for_later_2
[START] [2021-07-22 12:21:21] resolve_missing_parents
[STOP] [2021-07-22 12:21:21] resolve_missing_parents
[START] [2021-07-22 12:21:21] rebuild_nodes
[START] [2021-07-22 12:21:21] Flattener#flatten
[START] [2021-07-22 12:21:21] Flattener#study_resource
[START] [2021-07-22 12:21:21] Flattener#build_ancestry
[STOP] [2021-07-22 12:21:22] Flattener#build_ancestry
[INFO] [2021-07-22 12:21:22] 3225 ancestry keys
[START] [2021-07-22 12:21:22] build_node_ancestors
[INFO] [2021-07-22 12:21:22] old ancestors deleted.
[STOP] [2021-07-22 12:21:22] build_node_ancestors
[START] [2021-07-22 12:21:22] Flattener#propagate_ancestor_ids
[STOP] [2021-07-22 12:21:22] Flattener#propagate_ancestor_ids
[STOP] [2021-07-22 12:21:22] Flattener#flatten
[STOP] [2021-07-22 12:21:22] rebuild_nodes
[START] [2021-07-22 12:21:22] resolve_missing_media_owners
[STOP] [2021-07-22 12:21:22] resolve_missing_media_owners
[START] [2021-07-22 12:21:22] sanitize_media_verbatims
[STOP] [2021-07-22 12:21:22] sanitize_media_verbatims
[START] [2021-07-22 12:21:22] queue_downloads
[STOP] [2021-07-22 12:21:22] queue_downloads
[START] [2021-07-22 12:21:22] parse_names
[WARN] [2021-07-22 12:21:22] I see 3225 names which still need to be parsed.
[WARN] [2021-07-22 12:21:25] I see 2 names which still need to be parsed.
[WARN] [2021-07-22 12:21:27] I see 1 names which still need to be parsed.
[STOP] [2021-07-22 12:21:28] parse_names
[START] [2021-07-22 12:21:28] denormalize_canonical_names_to_nodes
[STOP] [2021-07-22 12:21:28] denormalize_canonical_names_to_nodes
[START] [2021-07-22 12:21:28] match_nodes
[START] [2021-07-22 12:21:28] map_all_nodes_to_pages
[STOP] [2021-07-22 12:22:42] map_all_nodes_to_pages
[INFO] [2021-07-22 12:22:42] 92 Unmatched nodes (of 3225)! That's too many to output. Full list in /app/public/data/tnvct/unmatched_nodes.txt ; First 10: Canonical: Abies lowiana; Node#97218932; ResourceID: Abies lowiana; Canonical: Abies magnifica; Node#97218933; ResourceID: Abies magnifica; Canonical: Acacia neovernicosa; Node#97218943; ResourceID: Acacia neovernicosa; Canonical: Acoelorraphe wrightii; Node#97218980; ResourceID: Acoelorraphe wrightii; Canonical: Adiantum pedatum; Node#97218991; ResourceID: Adiantum pedatum; Canonical: Agave lechuguilla; Node#97219005; ResourceID: Agave lechuguilla; Canonical: Alnus viridis; Node#97219045; ResourceID: Alnus viridis; Canonical: Andropogon glaucopsis; Node#97219082; ResourceID: Andropogon glaucopsis; Canonical: Annual; Node#97219096; ResourceID: Annual Grass-Herb; Canonical: Argentina egedii egedii; Node#97219140; ResourceID: Argentina egedii ssp. egedii
[START] [2021-07-22 12:22:42] update_nodes
[STOP] [2021-07-22 12:22:43] update_nodes
[STOP] [2021-07-22 12:22:43] match_nodes
[START] [2021-07-22 12:22:43] reindex_search
[STOP] [2021-07-22 12:22:47] reindex_search
[START] [2021-07-22 12:22:47] normalize_units
[STOP] [2021-07-22 12:22:47] normalize_units
[START] [2021-07-22 12:22:47] calculate_statistics
[2021-07-22 12:22:47] (NEAR) DUPLICATE TRAITS FOUND! There are only 10654 (of 24220 total) unique traits.
[2021-07-22 12:22:48] (Near) duplicate trait pairs (up to 100):
[2021-07-22 12:22:48] (resource_pk: 0, id: 223510779), (resource_pk: 13394, id: 223514553)
[2021-07-22 12:22:48] (resource_pk: 0, id: 223510779), (resource_pk: 13397, id: 223514556)
[2021-07-22 12:22:48] (resource_pk: 0, id: 223510779), (resource_pk: 14462, id: 223515739)
[2021-07-22 12:22:48] (resource_pk: 0, id: 223510779), (resource_pk: 16347, id: 223517833)
[2021-07-22 12:22:48] (resource_pk: 0, id: 223510779), (resource_pk: 2832, id: 223527037)
[2021-07-22 12:22:48] (resource_pk: 0, id: 223510779), (resource_pk: 2868, id: 223527076)
[2021-07-22 12:22:48] (resource_pk: 0, id: 223510779), (resource_pk: 3878, id: 223528198)
[2021-07-22 12:22:48] (resource_pk: 0, id: 223510779), (resource_pk: 5003, id: 223538984)
[2021-07-22 12:22:48] (resource_pk: 0, id: 223510779), (resource_pk: 9593, id: 223554770)
[2021-07-22 12:22:48] (resource_pk: 1, id: 223510780), (resource_pk: 6462, id: 223551292)
[2021-07-22 12:22:48] (resource_pk: 10, id: 223510781), (resource_pk: 4065, id: 223528561)
[2021-07-22 12:22:48] (resource_pk: 100, id: 223510782), (resource_pk: 15201, id: 223516561)
[2021-07-22 12:22:48] (resource_pk: 100, id: 223510782), (resource_pk: 15204, id: 223516564)
[2021-07-22 12:22:48] (resource_pk: 100, id: 223510782), (resource_pk: 15206, id: 223516566)
[2021-07-22 12:22:48] (resource_pk: 100, id: 223510782), (resource_pk: 15208, id: 223516568)
[2021-07-22 12:22:48] (resource_pk: 100, id: 223510782), (resource_pk: 15401, id: 223516783)
[2021-07-22 12:22:48] (resource_pk: 100, id: 223510782), (resource_pk: 15408, id: 223516790)
[2021-07-22 12:22:48] (resource_pk: 100, id: 223510782), (resource_pk: 1708, id: 223518647)
[2021-07-22 12:22:48] (resource_pk: 100, id: 223510782), (resource_pk: 19053, id: 223520840)
[2021-07-22 12:22:48] (resource_pk: 1000, id: 223510783), (resource_pk: 16586, id: 223518098)
[2021-07-22 12:22:48] (resource_pk: 1000, id: 223510783), (resource_pk: 18949, id: 223520723)
[2021-07-22 12:22:48] (resource_pk: 1000, id: 223510783), (resource_pk: 19138, id: 223520934)
[2021-07-22 12:22:48] (resource_pk: 1000, id: 223510783), (resource_pk: 6304, id: 223551117)
[2021-07-22 12:22:48] (resource_pk: 1000, id: 223510783), (resource_pk: 746, id: 223552400)
[2021-07-22 12:22:48] (resource_pk: 1000, id: 223510783), (resource_pk: 8318, id: 223553354)
[2021-07-22 12:22:48] (resource_pk: 1000, id: 223510783), (resource_pk: 9758, id: 223554953)
[2021-07-22 12:22:48] (resource_pk: 1000, id: 223510783), (resource_pk: 995, id: 223555166)
[2021-07-22 12:22:48] (resource_pk: 10000, id: 223510784), (resource_pk: 13685, id: 223514876)
[2021-07-22 12:22:48] (resource_pk: 10000, id: 223510784), (resource_pk: 2071, id: 223522681)
[2021-07-22 12:22:48] (resource_pk: 10000, id: 223510784), (resource_pk: 2075, id: 223522725)
[2021-07-22 12:22:48] (resource_pk: 10000, id: 223510784), (resource_pk: 2079, id: 223522769)
[2021-07-22 12:22:48] (resource_pk: 10000, id: 223510784), (resource_pk: 2083, id: 223522814)
[2021-07-22 12:22:48] (resource_pk: 10000, id: 223510784), (resource_pk: 2101, id: 223523015)
[2021-07-22 12:22:48] (resource_pk: 10000, id: 223510784), (resource_pk: 4806, id: 223536794)
[2021-07-22 12:22:48] (resource_pk: 10000, id: 223510784), (resource_pk: 9087, id: 223554208)
[2021-07-22 12:22:48] (resource_pk: 10001, id: 223510785), (resource_pk: 11694, id: 223512664)
[2021-07-22 12:22:48] (resource_pk: 10001, id: 223510785), (resource_pk: 13687, id: 223514878)
[2021-07-22 12:22:48] (resource_pk: 10001, id: 223510785), (resource_pk: 15371, id: 223516749)
[2021-07-22 12:22:48] (resource_pk: 10001, id: 223510785), (resource_pk: 15374, id: 223516752)
[2021-07-22 12:22:48] (resource_pk: 10001, id: 223510785), (resource_pk: 15392, id: 223516772)
[2021-07-22 12:22:48] (resource_pk: 10001, id: 223510785), (resource_pk: 19543, id: 223521384)
[2021-07-22 12:22:48] (resource_pk: 10001, id: 223510785), (resource_pk: 19743, id: 223521606)
[2021-07-22 12:22:48] (resource_pk: 10001, id: 223510785), (resource_pk: 2072, id: 223522692)
[2021-07-22 12:22:48] (resource_pk: 10001, id: 223510785), (resource_pk: 2283, id: 223525036)
[2021-07-22 12:22:48] (resource_pk: 10001, id: 223510785), (resource_pk: 4971, id: 223538627)
[2021-07-22 12:22:48] (resource_pk: 10001, id: 223510785), (resource_pk: 5029, id: 223539272)
[2021-07-22 12:22:48] (resource_pk: 10001, id: 223510785), (resource_pk: 6876, id: 223551751)
[2021-07-22 12:22:48] (resource_pk: 10001, id: 223510785), (resource_pk: 7001, id: 223551892)
[2021-07-22 12:22:48] (resource_pk: 10001, id: 223510785), (resource_pk: 7213, id: 223552127)
[2021-07-22 12:22:48] (resource_pk: 10001, id: 223510785), (resource_pk: 7625, id: 223552584)
[2021-07-22 12:22:48] (resource_pk: 10001, id: 223510785), (resource_pk: 7639, id: 223552599)
[2021-07-22 12:22:48] (resource_pk: 10001, id: 223510785), (resource_pk: 7918, id: 223552909)
[2021-07-22 12:22:48] (resource_pk: 10001, id: 223510785), (resource_pk: 8174, id: 223553194)
[2021-07-22 12:22:48] (resource_pk: 10001, id: 223510785), (resource_pk: 8177, id: 223553197)
[2021-07-22 12:22:48] (resource_pk: 10001, id: 223510785), (resource_pk: 9229, id: 223554366)
[2021-07-22 12:22:48] (resource_pk: 10001, id: 223510785), (resource_pk: 9884, id: 223555093)
[2021-07-22 12:22:48] (resource_pk: 10001, id: 223510785), (resource_pk: 9949, id: 223555165)
[2021-07-22 12:22:48] (resource_pk: 10002, id: 223510786), (resource_pk: 10226, id: 223511034)
[2021-07-22 12:22:48] (resource_pk: 10002, id: 223510786), (resource_pk: 13417, id: 223514579)
[2021-07-22 12:22:48] (resource_pk: 10002, id: 223510786), (resource_pk: 13799, id: 223515002)
[2021-07-22 12:22:48] (resource_pk: 10002, id: 223510786), (resource_pk: 14923, id: 223516251)
[2021-07-22 12:22:48] (resource_pk: 10002, id: 223510786), (resource_pk: 15690, id: 223517103)
[2021-07-22 12:22:48] (resource_pk: 10002, id: 223510786), (resource_pk: 18076, id: 223519754)
[2021-07-22 12:22:48] (resource_pk: 10002, id: 223510786), (resource_pk: 19744, id: 223521607)
[2021-07-22 12:22:48] (resource_pk: 10002, id: 223510786), (resource_pk: 20199, id: 223522113)
[2021-07-22 12:22:48] (resource_pk: 10002, id: 223510786), (resource_pk: 2073, id: 223522703)
[2021-07-22 12:22:48] (resource_pk: 10002, id: 223510786), (resource_pk: 2077, id: 223522747)
[2021-07-22 12:22:48] (resource_pk: 10002, id: 223510786), (resource_pk: 2085, id: 223522836)
[2021-07-22 12:22:48] (resource_pk: 10002, id: 223510786), (resource_pk: 2102, id: 223523026)
[2021-07-22 12:22:48] (resource_pk: 10002, id: 223510786), (resource_pk: 2105, id: 223523059)
[2021-07-22 12:22:48] (resource_pk: 10002, id: 223510786), (resource_pk: 2116, id: 223523181)
[2021-07-22 12:22:48] (resource_pk: 10002, id: 223510786), (resource_pk: 2300, id: 223525226)
[2021-07-22 12:22:48] (resource_pk: 10002, id: 223510786), (resource_pk: 3, id: 223527222)
[2021-07-22 12:22:48] (resource_pk: 10002, id: 223510786), (resource_pk: 3948, id: 223528276)
[2021-07-22 12:22:48] (resource_pk: 10002, id: 223510786), (resource_pk: 4776, id: 223536460)
[2021-07-22 12:22:48] (resource_pk: 10002, id: 223510786), (resource_pk: 4804, id: 223536772)
[2021-07-22 12:22:48] (resource_pk: 10002, id: 223510786), (resource_pk: 4972, id: 223538638)
[2021-07-22 12:22:48] (resource_pk: 10002, id: 223510786), (resource_pk: 5030, id: 223539284)
[2021-07-22 12:22:48] (resource_pk: 10002, id: 223510786), (resource_pk: 7002, id: 223551893)
[2021-07-22 12:22:48] (resource_pk: 10002, id: 223510786), (resource_pk: 7292, id: 223552214)
[2021-07-22 12:22:48] (resource_pk: 10002, id: 223510786), (resource_pk: 7938, id: 223552931)
[2021-07-22 12:22:48] (resource_pk: 10002, id: 223510786), (resource_pk: 8221, id: 223553247)
[2021-07-22 12:22:48] (resource_pk: 10002, id: 223510786), (resource_pk: 9230, id: 223554368)
[2021-07-22 12:22:48] (resource_pk: 10002, id: 223510786), (resource_pk: 9950, id: 223555167)
[2021-07-22 12:22:48] (resource_pk: 10003, id: 223510787), (resource_pk: 13421, id: 223514584)
[2021-07-22 12:22:48] (resource_pk: 10003, id: 223510787), (resource_pk: 13800, id: 223515005)
[2021-07-22 12:22:48] (resource_pk: 10003, id: 223510787), (resource_pk: 15326, id: 223516699)
[2021-07-22 12:22:48] (resource_pk: 10003, id: 223510787), (resource_pk: 19587, id: 223521432)
[2021-07-22 12:22:48] (resource_pk: 10003, id: 223510787), (resource_pk: 3500, id: 223527780)
[2021-07-22 12:22:48] (resource_pk: 10003, id: 223510787), (resource_pk: 3943, id: 223528271)
[2021-07-22 12:22:48] (resource_pk: 10003, id: 223510787), (resource_pk: 3951, id: 223528280)
[2021-07-22 12:22:48] (resource_pk: 10003, id: 223510787), (resource_pk: 4008, id: 223528344)
[2021-07-22 12:22:48] (resource_pk: 10003, id: 223510787), (resource_pk: 4277, id: 223530916)
[2021-07-22 12:22:48] (resource_pk: 10003, id: 223510787), (resource_pk: 4285, id: 223531005)
[2021-07-22 12:22:48] (resource_pk: 10003, id: 223510787), (resource_pk: 4289, id: 223531049)
[2021-07-22 12:22:48] (resource_pk: 10003, id: 223510787), (resource_pk: 4300, id: 223531173)
[2021-07-22 12:22:48] (resource_pk: 10003, id: 223510787), (resource_pk: 4303, id: 223531206)
[2021-07-22 12:22:48] (resource_pk: 10003, id: 223510787), (resource_pk: 4306, id: 223531239)
[2021-07-22 12:22:48] (resource_pk: 10003, id: 223510787), (resource_pk: 4311, id: 223531295)
[2021-07-22 12:22:48] (resource_pk: 10003, id: 223510787), (resource_pk: 4314, id: 223531328)
[STOP] [2021-07-22 12:22:48] calculate_statistics
[START] [2021-07-22 12:22:48] complete_harvest_instance
[START] [2021-07-22 12:22:48] overall_tsv_creation
[INFO] [2021-07-22 12:22:48] Processing group of 3225 in 1 batches of 10000
[INFO] [2021-07-22 12:24:58] 24220 Traits (unfiltered)...
[INFO] [2021-07-22 12:28:37] 24220 Traits (filtered)...
[INFO] [2021-07-22 12:28:40] 0 Associations (filtered)...
[INFO] [2021-07-22 12:28:45] 20222 metadata added.
[INFO] [2021-07-22 12:28:45] 0 metadata added.
[INFO] [2021-07-22 12:29:21] Average Time: 287.78
[INFO] [2021-07-22 12:29:21] Total Time: 6m34s
[STOP] [2021-07-22 12:29:21] overall_tsv_creation
[INFO] [2021-07-22 12:29:21] Done. Check your files:
[INFO] [2021-07-22 12:29:21] (3223 lines) /app/public/data/tnvct/publish_nodes.tsv
[INFO] [2021-07-22 12:29:21] (6458 lines) /app/public/data/tnvct/publish_node_ancestors.tsv
[INFO] [2021-07-22 12:29:21] (3225 lines) /app/public/data/tnvct/publish_scientific_names.tsv
[INFO] [2021-07-22 12:29:21] (24221 lines) /app/public/data/tnvct/publish_traits.tsv
[INFO] [2021-07-22 12:29:21] (20223 lines) /app/public/data/tnvct/publish_metadata.tsv
[STOP] [2021-07-22 12:29:21] complete_harvest_instance
[START] [2021-07-22 12:29:21] completed
[STOP] [2021-07-22 12:29:21] completed
[STOP] [2021-07-22 12:29:21] logged process, took 529.65

Latest Process

22 Jul 12:20 EXITED currently processing 0