Harvest for North American Flora Created 10 Sep 10:29

Stage: completed
Fetched: 10 Sep 10:29
Validated: 10 Sep 10:29
Deltas Created 10 Sep 10:29
Units Normalized: 10 Sep 10:35
Ancestry Built: 10 Sep 10:32
Nodes Matched: 10 Sep 10:35
Names Parsed: 10 Sep 10:32
New Models Stored: 10 Sep 10:31
Indexed: 10 Sep 10:35
Completed: 10 Sep 10:51
Time to Harvest: less than a minute

Harvesting Log (most recent first)

[INFO] [2021-09-10 10:29:49] Created harvest instance #4054
[STOP] [2021-09-10 10:29:49] create_harvest_instance
[START] [2021-09-10 10:29:49] fetch_files
[STOP] [2021-09-10 10:29:49] fetch_files
[START] [2021-09-10 10:29:49] validate_each_file
[INFO] [2021-09-10 10:29:49] Looping over 4 formats...
[INFO] [2021-09-10 10:29:49] ...nodes (/app/public/data/n_am_flora/taxon.tab)
[INFO] [2021-09-10 10:29:49] Valid: /app/public/converted_csv/n_am_flora_nodes_4054.csv (11299 lines)
[INFO] [2021-09-10 10:29:49] ...media (/app/public/data/n_am_flora/media_resource.tab)
[INFO] [2021-09-10 10:29:51] Valid: /app/public/converted_csv/n_am_flora_media_4054.csv (11438 lines)
[INFO] [2021-09-10 10:29:51] ...occurrences (/app/public/data/n_am_flora/occurrence_specific.tab)
[INFO] [2021-09-10 10:29:52] Valid: /app/public/converted_csv/n_am_flora_occurrences_4054.csv (45013 lines)
[INFO] [2021-09-10 10:29:52] ...measurements (/app/public/data/n_am_flora/measurement_or_fact_specific.tab)
[INFO] [2021-09-10 10:29:54] Valid: /app/public/converted_csv/n_am_flora_measurements_4054.csv (45013 lines)
[STOP] [2021-09-10 10:29:54] validate_each_file
[START] [2021-09-10 10:29:54] convert_to_csv
[INFO] [2021-09-10 10:29:54] Looping over 4 formats...
[INFO] [2021-09-10 10:29:54] ...nodes (/app/public/data/n_am_flora/taxon.tab)
[CMD] [2021-09-10 10:29:54] /usr/bin/sort /app/public/converted_csv/n_am_flora_nodes_4054.csv > /app/public/converted_csv/n_am_flora_nodes_4054.csv_sorted
[INFO] [2021-09-10 10:29:54] Converted: /app/public/converted_csv/n_am_flora_nodes_4054.csv (11299 lines)
[INFO] [2021-09-10 10:29:54] ...media (/app/public/data/n_am_flora/media_resource.tab)
[CMD] [2021-09-10 10:29:54] /usr/bin/sort /app/public/converted_csv/n_am_flora_media_4054.csv > /app/public/converted_csv/n_am_flora_media_4054.csv_sorted
[INFO] [2021-09-10 10:29:54] Converted: /app/public/converted_csv/n_am_flora_media_4054.csv (11438 lines)
[INFO] [2021-09-10 10:29:54] ...occurrences (/app/public/data/n_am_flora/occurrence_specific.tab)
[CMD] [2021-09-10 10:29:54] /usr/bin/sort /app/public/converted_csv/n_am_flora_occurrences_4054.csv > /app/public/converted_csv/n_am_flora_occurrences_4054.csv_sorted
[INFO] [2021-09-10 10:29:54] Converted: /app/public/converted_csv/n_am_flora_occurrences_4054.csv (45013 lines)
[INFO] [2021-09-10 10:29:54] ...measurements (/app/public/data/n_am_flora/measurement_or_fact_specific.tab)
[CMD] [2021-09-10 10:29:54] /usr/bin/sort /app/public/converted_csv/n_am_flora_measurements_4054.csv > /app/public/converted_csv/n_am_flora_measurements_4054.csv_sorted
[INFO] [2021-09-10 10:29:55] Converted: /app/public/converted_csv/n_am_flora_measurements_4054.csv (45013 lines)
[STOP] [2021-09-10 10:29:55] convert_to_csv
[START] [2021-09-10 10:29:55] calculate_delta
[INFO] [2021-09-10 10:29:55] Looping over 4 formats...
[INFO] [2021-09-10 10:29:55] ...nodes (/app/public/data/n_am_flora/taxon.tab)
[CMD] [2021-09-10 10:29:55] echo "0a" > /app/public/diff/n_am_flora_nodes_4054.diff
[CMD] [2021-09-10 10:29:55] tail -n +1 /app/public/converted_csv/n_am_flora_nodes_4054.csv >> /app/public/diff/n_am_flora_nodes_4054.diff
[CMD] [2021-09-10 10:29:55] echo "." >> /app/public/diff/n_am_flora_nodes_4054.diff
[INFO] [2021-09-10 10:29:55] Created diff: /app/public/diff/n_am_flora_nodes_4054.diff (11301 lines)
[INFO] [2021-09-10 10:29:55] ...media (/app/public/data/n_am_flora/media_resource.tab)
[CMD] [2021-09-10 10:29:55] echo "0a" > /app/public/diff/n_am_flora_media_4054.diff
[CMD] [2021-09-10 10:29:55] tail -n +1 /app/public/converted_csv/n_am_flora_media_4054.csv >> /app/public/diff/n_am_flora_media_4054.diff
[CMD] [2021-09-10 10:29:55] echo "." >> /app/public/diff/n_am_flora_media_4054.diff
[INFO] [2021-09-10 10:29:55] Created diff: /app/public/diff/n_am_flora_media_4054.diff (11440 lines)
[INFO] [2021-09-10 10:29:55] ...occurrences (/app/public/data/n_am_flora/occurrence_specific.tab)
[CMD] [2021-09-10 10:29:55] echo "0a" > /app/public/diff/n_am_flora_occurrences_4054.diff
[CMD] [2021-09-10 10:29:56] tail -n +1 /app/public/converted_csv/n_am_flora_occurrences_4054.csv >> /app/public/diff/n_am_flora_occurrences_4054.diff
[CMD] [2021-09-10 10:29:56] echo "." >> /app/public/diff/n_am_flora_occurrences_4054.diff
[INFO] [2021-09-10 10:29:56] Created diff: /app/public/diff/n_am_flora_occurrences_4054.diff (45015 lines)
[INFO] [2021-09-10 10:29:56] ...measurements (/app/public/data/n_am_flora/measurement_or_fact_specific.tab)
[CMD] [2021-09-10 10:29:56] echo "0a" > /app/public/diff/n_am_flora_measurements_4054.diff
[CMD] [2021-09-10 10:29:56] tail -n +1 /app/public/converted_csv/n_am_flora_measurements_4054.csv >> /app/public/diff/n_am_flora_measurements_4054.diff
[CMD] [2021-09-10 10:29:56] echo "." >> /app/public/diff/n_am_flora_measurements_4054.diff
[INFO] [2021-09-10 10:29:56] Created diff: /app/public/diff/n_am_flora_measurements_4054.diff (45015 lines)
[STOP] [2021-09-10 10:29:56] calculate_delta
[START] [2021-09-10 10:29:56] parse_diff_and_store
[INFO] [2021-09-10 10:29:56] Handling diff: /app/public/diff/n_am_flora_nodes_4054.diff (11301 lines)
[INFO] [2021-09-10 10:29:56] Loading nodes diff file into memory (11301 /app/public/diff/n_am_flora_nodes_4054.diff lines)...
[INFO] [2021-09-10 10:29:59] Handling diff: /app/public/diff/n_am_flora_media_4054.diff (11440 lines)
[INFO] [2021-09-10 10:29:59] Loading media diff file into memory (11440 /app/public/diff/n_am_flora_media_4054.diff lines)...
[INFO] [2021-09-10 10:30:11] Handling diff: /app/public/diff/n_am_flora_occurrences_4054.diff (45015 lines)
[INFO] [2021-09-10 10:30:11] Loading occurrences diff file into memory (45015 /app/public/diff/n_am_flora_occurrences_4054.diff lines)...
[INFO] [2021-09-10 10:30:16] Handling diff: /app/public/diff/n_am_flora_measurements_4054.diff (45015 lines)
[INFO] [2021-09-10 10:30:16] Loading measurements diff file into memory (45015 /app/public/diff/n_am_flora_measurements_4054.diff lines)...
[INFO] [2021-09-10 10:30:42] Storing 11299 ScientificNames
[INFO] [2021-09-10 10:30:42] Processing group of 11299 in 12 groups of 1000
[INFO] [2021-09-10 10:30:47] Average Time: 0.449
[INFO] [2021-09-10 10:30:47] Total Time: 6s
[INFO] [2021-09-10 10:30:47] last 3 / first 3: 0.35
[INFO] [2021-09-10 10:30:47] Std.Dev: 0.23664319132398465; Max: 1.03
[INFO] [2021-09-10 10:30:47] Storing 11299 Nodes
[INFO] [2021-09-10 10:30:47] Processing group of 11299 in 12 groups of 1000
[INFO] [2021-09-10 10:30:51] Average Time: 0.315
[INFO] [2021-09-10 10:30:51] Total Time: 4s
[INFO] [2021-09-10 10:30:51] last 3 / first 3: 0.55
[INFO] [2021-09-10 10:30:51] Std.Dev: 0.08944271909999159; Max: 0.44
[INFO] [2021-09-10 10:30:51] Storing 11438 BibliographicCitations
[INFO] [2021-09-10 10:30:51] Processing group of 11438 in 12 groups of 1000
[INFO] [2021-09-10 10:30:53] Average Time: 0.153
[INFO] [2021-09-10 10:30:53] Total Time: 2s
[INFO] [2021-09-10 10:30:53] last 3 / first 3: 0.76
[INFO] [2021-09-10 10:30:53] Std.Dev: 0.03162277660168379; Max: 0.22
[INFO] [2021-09-10 10:30:53] Storing 11438 ArticlesSections
[INFO] [2021-09-10 10:30:53] Processing group of 11438 in 12 groups of 1000
[INFO] [2021-09-10 10:30:54] Average Time: 0.055
[INFO] [2021-09-10 10:30:54] Total Time: 1s
[INFO] [2021-09-10 10:30:54] last 3 / first 3: 0.75
[INFO] [2021-09-10 10:30:54] Std.Dev: 0.03162277660168379; Max: 0.13
[INFO] [2021-09-10 10:30:54] Storing 11438 Articles
[INFO] [2021-09-10 10:30:54] Processing group of 11438 in 12 groups of 1000
[INFO] [2021-09-10 10:30:59] Average Time: 0.461
[INFO] [2021-09-10 10:30:59] Total Time: 6s
[INFO] [2021-09-10 10:30:59] last 3 / first 3: 0.71
[INFO] [2021-09-10 10:30:59] Std.Dev: 0.1224744871391589; Max: 0.64
[INFO] [2021-09-10 10:30:59] Storing 45013 Occurrences
[INFO] [2021-09-10 10:30:59] Processing group of 45013 in 46 groups of 1000
[INFO] [2021-09-10 10:31:05] Average Time: 0.118
[INFO] [2021-09-10 10:31:05] Total Time: 6s
[INFO] [2021-09-10 10:31:05] last 3 / first 3: 0.51
[INFO] [2021-09-10 10:31:05] Std.Dev: 0.03162277660168379; Max: 0.2
[INFO] [2021-09-10 10:31:05] Storing 45013 Traits
[INFO] [2021-09-10 10:31:05] Processing group of 45013 in 46 groups of 1000
[INFO] [2021-09-10 10:31:20] Average Time: 0.335
[INFO] [2021-09-10 10:31:20] Total Time: 16s
[INFO] [2021-09-10 10:31:20] last 3 / first 3: 0.5
[INFO] [2021-09-10 10:31:20] Std.Dev: 0.08944271909999159; Max: 0.62
[INFO] [2021-09-10 10:31:20] Storing 90026 MetaTraits
[INFO] [2021-09-10 10:31:20] Processing group of 90026 in 91 groups of 1000
[INFO] [2021-09-10 10:31:33] Average Time: 0.136
[INFO] [2021-09-10 10:31:33] Total Time: 13s
[INFO] [2021-09-10 10:31:33] last 3 / first 3: 0.45
[INFO] [2021-09-10 10:31:33] Std.Dev: 0.06324555320336758; Max: 0.39
[STOP] [2021-09-10 10:31:33] parse_diff_and_store
[START] [2021-09-10 10:31:33] resolve_keys
[INFO] [2021-09-10 10:32:05] Occurrences to nodes (through scientific_names)...
[INFO] [2021-09-10 10:32:06] traits to occurrences...
[INFO] [2021-09-10 10:32:17] traits to nodes (through occurrences)...
[INFO] [2021-09-10 10:32:18] Traits to sex term...
[INFO] [2021-09-10 10:32:19] Traits to lifestage term...
[INFO] [2021-09-10 10:32:20] MetaTraits to traits...
[INFO] [2021-09-10 10:32:23] MetaTraits (simple, measurement row refers to parent) to traits...
[INFO] [2021-09-10 10:32:24] Assocs to occurrences...
[INFO] [2021-09-10 10:32:24] Assocs to nodes...
[INFO] [2021-09-10 10:32:24] Assoc to sex term...
[INFO] [2021-09-10 10:32:24] Assoc to lifestage term...
[INFO] [2021-09-10 10:32:24] MetaAssoc to assocs...
[STOP] [2021-09-10 10:32:24] resolve_keys
[START] [2021-09-10 10:32:24] hold_for_later_1
[STOP] [2021-09-10 10:32:24] hold_for_later_1
[START] [2021-09-10 10:32:24] hold_for_later_2
[STOP] [2021-09-10 10:32:24] hold_for_later_2
[START] [2021-09-10 10:32:24] resolve_missing_parents
[STOP] [2021-09-10 10:32:24] resolve_missing_parents
[START] [2021-09-10 10:32:24] rebuild_nodes
[START] [2021-09-10 10:32:24] Flattener#flatten
[START] [2021-09-10 10:32:24] Flattener#study_resource
[START] [2021-09-10 10:32:24] Flattener#build_ancestry
[STOP] [2021-09-10 10:32:24] Flattener#build_ancestry
[INFO] [2021-09-10 10:32:24] 11299 ancestry keys
[START] [2021-09-10 10:32:24] build_node_ancestors
[INFO] [2021-09-10 10:32:24] old ancestors deleted.
[STOP] [2021-09-10 10:32:24] build_node_ancestors
[WARN] [2021-09-10 10:32:24] Flattener: nothing to flatten! (Completely flat resource?)
[STOP] [2021-09-10 10:32:24] Flattener#flatten
[STOP] [2021-09-10 10:32:24] rebuild_nodes
[START] [2021-09-10 10:32:24] resolve_missing_media_owners
[STOP] [2021-09-10 10:32:24] resolve_missing_media_owners
[START] [2021-09-10 10:32:24] sanitize_media_verbatims
[STOP] [2021-09-10 10:32:24] sanitize_media_verbatims
[START] [2021-09-10 10:32:24] queue_downloads
[STOP] [2021-09-10 10:32:24] queue_downloads
[START] [2021-09-10 10:32:24] parse_names
[WARN] [2021-09-10 10:32:24] I see 11299 names which still need to be parsed.
[STOP] [2021-09-10 10:32:33] parse_names
[START] [2021-09-10 10:32:33] denormalize_canonical_names_to_nodes
[STOP] [2021-09-10 10:32:33] denormalize_canonical_names_to_nodes
[START] [2021-09-10 10:32:33] match_nodes
[START] [2021-09-10 10:32:33] map_all_nodes_to_pages
[STOP] [2021-09-10 10:35:01] map_all_nodes_to_pages
[INFO] [2021-09-10 10:35:01] 1554 Unmatched nodes (of 11299)! That's too many to output. Full list in /app/public/data/n_am_flora/unmatched_nodes.txt ; First 10: Canonical: Nesothamnus incanus; Node#98072069; ResourceID: 0086fbeedb032790a304e821a1becfa1; Canonical: Carex boliviensis in; Node#98072070; ResourceID: 0088d7697b299d714f350b95643b0f97; Canonical: Rauwolfia macrocarpa; Node#98072072; ResourceID: 008c3ccbd843b6b3b5f82b7ac6f27c32; Canonical: Phaca juliana; Node#98072073; ResourceID: 008ef7389e03f8779fbb9537d0e7111c; Canonical: Acaciella filicioides; Node#98072081; ResourceID: 00d33a8a328caf7c56564a1406767450; Canonical: Psorobatus benthami; Node#98072082; ResourceID: 00dff7f23982b87acb49d5b173b00f54; Canonical: Xylophacos musiniensis; Node#98072090; ResourceID: 00f4352e8896e867e600def361130bb5; Canonical: Xylophacos watsonianus; Node#98072122; ResourceID: 01b8ef783b462921dff705ef20e936e2; Canonical: Elsota sylvestris; Node#98072131; ResourceID: 01e64f78faa94b8c8875514369c1a4be; Canonical: Psorodendron johnsoni; Node#98072135; ResourceID: 020d056147ad6693298712982696f2e0
[START] [2021-09-10 10:35:01] update_nodes
[STOP] [2021-09-10 10:35:05] update_nodes
[STOP] [2021-09-10 10:35:05] match_nodes
[START] [2021-09-10 10:35:05] reindex_search
[STOP] [2021-09-10 10:35:13] reindex_search
[START] [2021-09-10 10:35:13] normalize_units
[STOP] [2021-09-10 10:35:13] normalize_units
[START] [2021-09-10 10:35:13] calculate_statistics
[2021-09-10 10:35:13] ZERO NODE ANCESTORS. Is this actually a completely flat resource?
[2021-09-10 10:35:14] (NEAR) DUPLICATE TRAITS FOUND! There are only 44582 (of 45013 total) unique traits.
[2021-09-10 10:41:28] (Near) duplicate trait pairs (up to 100):
[2021-09-10 10:41:28] (resource_pk: 016be9e25f5ca6ffa275a3a700e83b1a_15441_ENV, id: 223880619), (resource_pk: 8300f9c45a7e7ecf3e68730d06a4d8b4_15441_ENV, id: 223903443)
[2021-09-10 10:41:28] (resource_pk: 018e56bc66a69d8067ab019edd664056_15441_ENV, id: 223880638), (resource_pk: fc621a34c3e35b0017e129f8e213581c_15441_ENV, id: 223924753)
[2021-09-10 10:41:28] (resource_pk: 022500e91552c7c2c55b587b5808dfb8_15441_ENV, id: 223880740), (resource_pk: 4df9eedd934871cbdefc3112898e73a8_15441_ENV, id: 223894043)
[2021-09-10 10:41:28] (resource_pk: 02bc77d5cb7abd0d0f8864fe2a4ba343_91535_ENV, id: 223880823), (resource_pk: 5841ea2bfeac4a669df9e28f224ba866_91535_ENV, id: 223895880)
[2021-09-10 10:41:28] (resource_pk: 02ff0dbc09b62576c10d02b6e4e18013_91357_ENV, id: 223880860), (resource_pk: 75ac90136c8d43034e2a272d3323021b_91357_ENV, id: 223901094)
[2021-09-10 10:41:28] (resource_pk: 039a07b7f2388d422aa597b8d3926631_15441_ENV, id: 223880974), (resource_pk: f0975d24bc210887e4df071973c06be5_15441_ENV, id: 223922733)
[2021-09-10 10:41:28] (resource_pk: 041d2075d0ad19ed912fd0b7d71c9a24_15441_ENV, id: 223881059), (resource_pk: 666ee5591de813eb248584973529d8c9_15441_ENV, id: 223898440)
[2021-09-10 10:41:28] (resource_pk: 042cdd244aca951c906d3aa6f9769693_15441_ENV, id: 223881070), (resource_pk: 24c9b79a36c485418b8fe89a5594f767_15441_ENV, id: 223886776)
[2021-09-10 10:41:28] (resource_pk: 0450d23543043ba7f5afe8a8dcdc49c0_15441_ENV, id: 223881100), (resource_pk: 635055fced8087d59900a7e0ebf7be9c_15441_ENV, id: 223897877)
[2021-09-10 10:41:28] (resource_pk: 049f8c8a851205e2596f3187eb78e6de_15441_ENV, id: 223881150), (resource_pk: 46dbad9bbfd1ebd18f06f46fab9ec4bc_15441_ENV, id: 223892723)
[2021-09-10 10:41:28] (resource_pk: 04acb820620d06f67295355080195ccd_15441_ENV, id: 223881161), (resource_pk: d9d232322f25741718c1097e795fca4d_15441_ENV, id: 223918797)
[2021-09-10 10:41:28] (resource_pk: 04fecb27282871155d1347f18296c159_15441_ENV, id: 223881219), (resource_pk: 54047e6d40db71d5c3ec49f8328b31d6_15441_ENV, id: 223895151)
[2021-09-10 10:41:28] (resource_pk: 050853cedcd23f8f3a2b901fb3bb1101_15441_ENV, id: 223881226), (resource_pk: 8d8ffbc782dde6b1427d3e3d71749ae3_15441_ENV, id: 223905302)
[2021-09-10 10:41:28] (resource_pk: 05373c4b9951cf821261b970cb2e0e6e_15441_ENV, id: 223881260), (resource_pk: b569f81410e1b567c7e9ddb7c4cebe19_15441_ENV, id: 223912341)
[2021-09-10 10:41:28] (resource_pk: 0559fc918dedcf88422fe20dbb25f0e1_15441_ENV, id: 223881274), (resource_pk: 9f8ebce20bfcadfd5fa8975308d60e2e_15441_ENV, id: 223908503)
[2021-09-10 10:41:28] (resource_pk: 059c000f5a820ef74ddf46c6b33fefec_15441_ENV, id: 223881318), (resource_pk: e6a2a77ff38d145584a37cecf26b944e_15441_ENV, id: 223921064)
[2021-09-10 10:41:28] (resource_pk: 05b5288e31982c2ebf616d8447a9084b_15441_ENV, id: 223881330), (resource_pk: b782684592052a9393ea47f662614351_15441_ENV, id: 223912733)
[2021-09-10 10:41:28] (resource_pk: 05ec85513cf19e0a1f539c4958ca642d_15441_ENV, id: 223881376), (resource_pk: 594942fcf02ceecf3f823588ffea7b69_15441_ENV, id: 223896085)
[2021-09-10 10:41:28] (resource_pk: 0622f85bf182344e6a72ffe0758b7b27_15441_ENV, id: 223881414), (resource_pk: c58f36f9394e8ffbb6c376afaeca8fba_15441_ENV, id: 223915243)
[2021-09-10 10:41:28] (resource_pk: 065bb2f5543c74299fe9855d6c993353_15441_ENV, id: 223881462), (resource_pk: 3306afc546368543ed1b4a8aadb35727_15441_ENV, id: 223889281)
[2021-09-10 10:41:28] (resource_pk: 06694dcff6896e805be31f0fa51168fa_15441_ENV, id: 223881472), (resource_pk: 9ca764ec4e44ed04d2f91d173aeb7623_15441_ENV, id: 223907998)
[2021-09-10 10:41:28] (resource_pk: 066ee533ff81597e08db9e7185e33b98_15441_ENV, id: 223881476), (resource_pk: fb5067d045ff8a87fd618fb9b02c78d9_15441_ENV, id: 223924574)
[2021-09-10 10:41:28] (resource_pk: 06a4624238f80c282be9c5a5848cf055_15441_ENV, id: 223881512), (resource_pk: 913b8e4e0f10b0e1a875c171ccf81e4a_15441_ENV, id: 223905942)
[2021-09-10 10:41:28] (resource_pk: 077a92ca953ab5216754624351faa583_15441_ENV, id: 223881650), (resource_pk: dfa78f523a008e4bdafe99aecf9108e4_15441_ENV, id: 223919831)
[2021-09-10 10:41:28] (resource_pk: 07c5fed032163fdc5a7def5bcedbe5da_15441_ENV, id: 223881708), (resource_pk: 94f83ff2da2790ebd69c36b7c08f7071_15441_ENV, id: 223906550)
[2021-09-10 10:41:28] (resource_pk: 087b8d51bcc6c91c0cd0d442632fd39f_15441_ENV, id: 223881824), (resource_pk: 7aa443bcc2a628441dbfea71f5750e3f_15441_ENV, id: 223901943)
[2021-09-10 10:41:28] (resource_pk: 08b29dc29f1d66e85e3e5440b02994b3_15441_ENV, id: 223881861), (resource_pk: 74c5c6eff8e5d2085265d5d919827f05_15441_ENV, id: 223900949)
[2021-09-10 10:41:28] (resource_pk: 08c0a3fb935f0ab7235efcbf9ef8e189_15429_ENV, id: 223881869), (resource_pk: 228e86ad343ebe3aa8bbdaed5d73aa00_15429_ENV, id: 223886398)
[2021-09-10 10:41:28] (resource_pk: 08c5fc08fe0843aeec35514c535a2954_15441_ENV, id: 223881875), (resource_pk: 89828e23a02c2facd4436653aee7cfdc_15441_ENV, id: 223904618)
[2021-09-10 10:41:28] (resource_pk: 08ce68b3743282645f5da9bbe17a1fff_15441_ENV, id: 223881883), (resource_pk: 6d260d1a3ae1153ffc9a5d68adb6c7fe_15441_ENV, id: 223899619)
[2021-09-10 10:41:28] (resource_pk: 0911e6a6e4fdc153c74c9f2fb249e26c_15441_ENV, id: 223881932), (resource_pk: ea56aba204e634f70bbcb9fe2716a7a0_15441_ENV, id: 223921663)
[2021-09-10 10:41:28] (resource_pk: 0923b2c5efa7ce549d5efe012ed59061_15441_ENV, id: 223881949), (resource_pk: 4a76b9c25d6c12fc65b7ae02b375f7e3_15441_ENV, id: 223893359)
[2021-09-10 10:41:28] (resource_pk: 0a04cf618757744230dee42ec0c9e0eb_15441_ENV, id: 223882097), (resource_pk: 1acac16d35e366b8f63b497930b2b858_15441_ENV, id: 223885071)
[2021-09-10 10:41:28] (resource_pk: 0a07ff661dd5dc6ce0373ceebbf8393c_15441_ENV, id: 223882101), (resource_pk: f3301306b11878a2479b0311208d3964_15441_ENV, id: 223923209)
[2021-09-10 10:41:28] (resource_pk: 0a18d003f803d1500c65751d6008be18_15441_ENV, id: 223882118), (resource_pk: 3caa1bcbff48ab5147d5d66c245b160a_15441_ENV, id: 223890942)
[2021-09-10 10:41:28] (resource_pk: 0a6c78dc84ddb4ba7e33c05be1474408_15436_ENV, id: 223882171), (resource_pk: 3636ed643577287f87b50238ba56eebf_15436_ENV, id: 223889813)
[2021-09-10 10:41:28] (resource_pk: 0a9ad729e61ac3579d99f23af0cef813_15441_ENV, id: 223882209), (resource_pk: 0fa5a187f1a5fb82ae5fa1d4944ab7f1_15441_ENV, id: 223883146)
[2021-09-10 10:41:28] (resource_pk: 0b3106b8af8b63a34cb7c23832c29416_15441_ENV, id: 223882307), (resource_pk: fc3cb4874cace52e3f4c0803841b8ee9_15441_ENV, id: 223924728)
[2021-09-10 10:41:28] (resource_pk: 0b436b3ba1dbc696b83ac5ffd1bc022a_15441_ENV, id: 223882316), (resource_pk: 47b3c1097ff28a1466774a8b71f40b40_15441_ENV, id: 223892862)
[2021-09-10 10:41:28] (resource_pk: 0b75592ddda47d3f9175d60fa261fbd5_15441_ENV, id: 223882338), (resource_pk: eb216b9cc17b60f2dfd1484536fbbdea_15441_ENV, id: 223921784)
[2021-09-10 10:41:28] (resource_pk: 0ba481d662f121366491ce0b76a3808a_15441_ENV, id: 223882374), (resource_pk: b4ad8684be5bf3cf146ec1eb50e5c51d_15441_ENV, id: 223912225)
[2021-09-10 10:41:28] (resource_pk: 0bbd472fe5d669010c2210a7bda91542_15441_ENV, id: 223882388), (resource_pk: f07377f4b01ac59975a5863f27cf7501_15441_ENV, id: 223922708)
[2021-09-10 10:41:28] (resource_pk: 0c108d2cde12caeb0888553ef68e45ce_15438_ENV, id: 223882466), (resource_pk: 4b525755ce98265e3906d36d50e03187_15438_ENV, id: 223893515)
[2021-09-10 10:41:28] (resource_pk: 0c40e138011259e1ddce182160ccb68a_15441_ENV, id: 223882503), (resource_pk: 6ed1d4b73b3780ab776d6874585255c3_15441_ENV, id: 223899903)
[2021-09-10 10:41:28] (resource_pk: 0c4fcc1fa5d96c585647759f049a3446_15441_ENV, id: 223882518), (resource_pk: cb90f9b66b5ffc41da5d2f2c37965be8_15441_ENV, id: 223916293)
[2021-09-10 10:41:28] (resource_pk: 0c7c093317e4734563d218a01965974b_15441_ENV, id: 223882545), (resource_pk: dfd31c0403d345f86103dad923f1092a_15441_ENV, id: 223919868)
[2021-09-10 10:41:28] (resource_pk: 0cb9428c4c5f134494211e5b28bf8713_15441_ENV, id: 223882595), (resource_pk: 1290c1a00e5172a66c4e8827cd6427a5_15441_ENV, id: 223883645)
[2021-09-10 10:41:28] (resource_pk: 0d1d6ee14e928ee25dad58f35c91e753_15441_ENV, id: 223882682), (resource_pk: bec657d83a0219f5f014cc42962fc842_15441_ENV, id: 223914055)
[2021-09-10 10:41:28] (resource_pk: 0dd5031e2ac96f6ff6a79aa630814599_91538_ENV, id: 223882806), (resource_pk: 934d229993c7d7110d6e2fa5bd673de1_91538_ENV, id: 223906286)
[2021-09-10 10:41:28] (resource_pk: 0f383b868c26643a8b525dcda8f68c7d_91535_ENV, id: 223883068), (resource_pk: 9f5c33f171dc76899ae583b062cb69d4_91535_ENV, id: 223908460)
[2021-09-10 10:41:28] (resource_pk: 0fa681c4884a9c7d054ced4ff8f998d3_15441_ENV, id: 223883148), (resource_pk: f8e6e2274c8c7d06b79e2d360b2eed1c_15441_ENV, id: 223924157)
[2021-09-10 10:41:28] (resource_pk: 0fe46272e440109fa0409a3e5af55c5e_15441_ENV, id: 223883186), (resource_pk: dc143a2b56fc8bee3e1a48447794acad_15441_ENV, id: 223919203)
[2021-09-10 10:41:28] (resource_pk: 0ff3abe46e83e0a13d592f7a14e805b1_15441_ENV, id: 223883197), (resource_pk: 56483cfb5a51940a7d35810222476f95_15441_ENV, id: 223895542)
[2021-09-10 10:41:28] (resource_pk: 0ffe6f5fbe81f6a59d94502f2bf87e7c_15441_ENV, id: 223883216), (resource_pk: acf85a72e0aba7132a9af2ededd4a637_15441_ENV, id: 223910837)
[2021-09-10 10:41:28] (resource_pk: 1002c9b38c110130da7dbaef8d8bfebb_15441_ENV, id: 223883218), (resource_pk: 271af55693400b23996007a846a74772_15441_ENV, id: 223887200)
[2021-09-10 10:41:28] (resource_pk: 100e6b55f05674c0b553930779233823_15441_ENV, id: 223883230), (resource_pk: c7ab1026a03905f70efdd932b514f624_15441_ENV, id: 223915606)
[2021-09-10 10:41:28] (resource_pk: 109fb33f19ac28f10cf7554ff1fbbe38_15441_ENV, id: 223883317), (resource_pk: b6857a6e132f05eff0fc59bc5fc87d20_15441_ENV, id: 223912539)
[2021-09-10 10:41:28] (resource_pk: 11045b77731fa03f4acdb0c1c0d5eb58_15441_ENV, id: 223883379), (resource_pk: e46de33ace1bc0594be68e3b223c6f49_15441_ENV, id: 223920719)
[2021-09-10 10:41:28] (resource_pk: 113b00204405593cfaeaad1ed130988f_15441_ENV, id: 223883421), (resource_pk: e8583ad07d77c7fbaa2e88b62f604e51_15441_ENV, id: 223921318)
[2021-09-10 10:41:28] (resource_pk: 126c859fb30379883dfd210c69aebfd3_91538_ENV, id: 223883626), (resource_pk: d8e58d7a2345317b6edea4671641b86e_91538_ENV, id: 223918636)
[2021-09-10 10:41:28] (resource_pk: 1301a0eed5820214d60a743c4d988daf_15441_ENV, id: 223883707), (resource_pk: e75331d18bd80111ce3453cfd8cca0a9_15441_ENV, id: 223921161)
[2021-09-10 10:41:28] (resource_pk: 13793331ca6ee82be5e3507863af58e5_15441_ENV, id: 223883785), (resource_pk: a3d10b11b34c8b02cd364db47cfccae2_15441_ENV, id: 223909276)
[2021-09-10 10:41:28] (resource_pk: 13a0a2bfaa21449772cb8927fafed162_15441_ENV, id: 223883806), (resource_pk: e964ba3c38d5c70708e36dd9032d0172_15441_ENV, id: 223921502)
[2021-09-10 10:41:28] (resource_pk: 13f287b3dab5c603d816741466d94e67_15441_ENV, id: 223883862), (resource_pk: 36340a4aca3a3eb01f111286e3399c92_15441_ENV, id: 223889812)
[2021-09-10 10:41:28] (resource_pk: 1505bd6038a68cfdeb8fcdc1c5510e51_15441_ENV, id: 223884042), (resource_pk: 7732e1b2ddcaf29224046526f5c31bbe_15441_ENV, id: 223901352)
[2021-09-10 10:41:28] (resource_pk: 1543b2c0fcbdbfcc614f3babab80c2cd_15441_ENV, id: 223884077), (resource_pk: 7002c6e62ed33df950408742a2a34d56_15441_ENV, id: 223900108)
[2021-09-10 10:41:28] (resource_pk: 1576616c6826d039a7a116c53c39f1e7_15441_ENV, id: 223884111), (resource_pk: 5f5549b27581f1522f37484f6d8c8acf_15441_ENV, id: 223897156)
[2021-09-10 10:41:28] (resource_pk: 1592dc60a0f950c46a911c2b76370491_15441_ENV, id: 223884142), (resource_pk: 44fc82d0569b20d8d840ef488ad46da1_15441_ENV, id: 223892380)
[2021-09-10 10:41:28] (resource_pk: 15dca6518d42c9d28944eba6ceffb5a8_15438_ENV, id: 223884183), (resource_pk: ec0d922c834745d07b9bc204895c8316_15438_ENV, id: 223921937)
[2021-09-10 10:41:28] (resource_pk: 1647dcf683211393f30503cc39d4c903_15441_ENV, id: 223884263), (resource_pk: ba8e3ab496712b9651d836c9ee4e24c6_15441_ENV, id: 223913308)
[2021-09-10 10:41:28] (resource_pk: 16dc5e18ba40d627361229ebb1193e48_15441_ENV, id: 223884373), (resource_pk: 8bee1c3ef2e7f87564da640a42335e5f_15441_ENV, id: 223905031)
[2021-09-10 10:41:28] (resource_pk: 16ff0d394fd75158936a3b03c595dee4_15441_ENV, id: 223884398), (resource_pk: d8f4bb8703994261218a4f73ba395b67_15441_ENV, id: 223918646)
[2021-09-10 10:41:28] (resource_pk: 17503fc1b50b7d8c69d25700ae4aea8c_91336_ENV, id: 223884471), (resource_pk: eba013cbbf03d62fbbfdd7c537816254_91336_ENV, id: 223921860)
[2021-09-10 10:41:28] (resource_pk: 17534be185d2ec0aed759feb01884647_15441_ENV, id: 223884472), (resource_pk: 18a9d9b082e22451aaa7219c25d19e81_15441_ENV, id: 223884684)
[2021-09-10 10:41:28] (resource_pk: 17ca1f0aeef26fd4cff50457ac7b12fc_91538_ENV, id: 223884559), (resource_pk: e7c6387916a4eea320121ed04a2dc9f8_91538_ENV, id: 223921229)
[2021-09-10 10:41:28] (resource_pk: 17ed45979772e1644347d80ab5950e8b_15441_ENV, id: 223884580), (resource_pk: 44ed8a3a112387c81c1c8ff19f8e105b_15441_ENV, id: 223892373)
[2021-09-10 10:41:28] (resource_pk: 18c09098b8c7b39f3e5faf5bf61c8e76_15441_ENV, id: 223884699), (resource_pk: ceb9b9fe425e352f1cac2ffc6c92be60_15441_ENV, id: 223916826)
[2021-09-10 10:41:28] (resource_pk: 18de541f60290ea09c9d4bedf7641453_15441_ENV, id: 223884721), (resource_pk: 2ed2683aa7821361f0d99d2f749f6eba_15441_ENV, id: 223888536)
[2021-09-10 10:41:28] (resource_pk: 190e51aa4698899d928f9f9feadc112f_15441_ENV, id: 223884762), (resource_pk: 50d39a3d62022a6b0825066292353bba_15441_ENV, id: 223894566)
[2021-09-10 10:41:28] (resource_pk: 193cfdb0c5863ab0db24bd1cbff02287_15437_ENV, id: 223884800), (resource_pk: dac6451300c242fd213e6c32ed940c45_15437_ENV, id: 223918989)
[2021-09-10 10:41:28] (resource_pk: 198ad69c29cc74d5d7ba2077cb645b51_15438_ENV, id: 223884853), (resource_pk: 7c87e6e0d64dd6cef105bb53d0453913_15438_ENV, id: 223902278)
[2021-09-10 10:41:28] (resource_pk: 1a43084be7ffd4f47e7b59c4ffc01a1b_15441_ENV, id: 223884969), (resource_pk: 8c80cee30e8f16b3515f34bf27ff6db0_15441_ENV, id: 223905120)
[2021-09-10 10:41:28] (resource_pk: 1af6835c54fe3cd8a0688caf5e0f41fe_15441_ENV, id: 223885097), (resource_pk: 55eefc910a3695f8a50685b5896fad2b_15441_ENV, id: 223895480)
[2021-09-10 10:41:28] (resource_pk: 1b6f51288761f8e0c9f16c223adbd6af_15441_ENV, id: 223885196), (resource_pk: 703f6db344f7b22fd76a49df291e5bcc_15441_ENV, id: 223900160)
[2021-09-10 10:41:28] (resource_pk: 1c9bea559a5531eaf165396d54af6a37_15441_ENV, id: 223885371), (resource_pk: 8dc53bfb94ca3b457acdbb76ba2bfaec_15441_ENV, id: 223905330)
[2021-09-10 10:41:28] (resource_pk: 1cd98029092b352dab1540f69390c53a_15441_ENV, id: 223885416), (resource_pk: 9a7b49a4a7f74be86255391ac7979435_15441_ENV, id: 223907581)
[2021-09-10 10:41:28] (resource_pk: 1d068b08f45c5faba9498c7663563b07_15441_ENV, id: 223885446), (resource_pk: 1fbd1c6eb7c9bdc82725ad8901847e97_15441_ENV, id: 223885915)
[2021-09-10 10:41:28] (resource_pk: 1d1f448da8e35062e344a1470e7ab4f0_15441_ENV, id: 223885468), (resource_pk: c8571b975b0c4e0c70db8a0f30fd9027_15441_ENV, id: 223915722)
[2021-09-10 10:41:28] (resource_pk: 1d23ee44b32e53d410f87dcd90565176_91336_ENV, id: 223885472), (resource_pk: 1e7633ac6196f7d6bb0f8ae2cdcbf2ef_91336_ENV, id: 223885686)
[2021-09-10 10:41:28] (resource_pk: 1d31a63914f903a141492d0ed64d8b51_15436_ENV, id: 223885479), (resource_pk: 76326b3054530f60c7df03584caf28ad_15436_ENV, id: 223901188)
[2021-09-10 10:41:28] (resource_pk: 1dacada9bd0001bb7b4f41a1cb10942e_15441_ENV, id: 223885552), (resource_pk: 91062674ab595d026b05a025b6d37685_15441_ENV, id: 223905894)
[2021-09-10 10:41:28] (resource_pk: 1de34de2543fdb162098a496f4ef3ab6_15441_ENV, id: 223885592), (resource_pk: 2c64e6ab3897d21f5416126f3bde9f02_15441_ENV, id: 223888113)
[2021-09-10 10:41:28] (resource_pk: 1e0e74fdb6011fdb21159f57bebae444_15441_ENV, id: 223885614), (resource_pk: a4f43e2ffaa189b023727357967f88d4_15441_ENV, id: 223909479)
[2021-09-10 10:41:28] (resource_pk: 1e19efea8fde3119fade76d7fea9c07f_15441_ENV, id: 223885625), (resource_pk: 9b268b586eceff00c6fec9e10af273d6_15441_ENV, id: 223907701)
[2021-09-10 10:41:28] (resource_pk: 1e88db601ef5bfb8727c44b5fbafb251_15441_ENV, id: 223885696), (resource_pk: 245dac2ad3b96743eca21e1e8a4e4488_15441_ENV, id: 223886714)
[2021-09-10 10:41:28] (resource_pk: 1e8c7d5c0f9bcad538bb8035c449debe_15441_ENV, id: 223885700), (resource_pk: 5acb8ae787354b3c5c84f333de6b9694_15441_ENV, id: 223896349)
[2021-09-10 10:41:28] (resource_pk: 1fceb5e158d5ecc8c433ac171453c7f2_15441_ENV, id: 223885923), (resource_pk: e5cfd591fc8b48705977fdeee2cc40f3_15441_ENV, id: 223920926)
[2021-09-10 10:41:28] (resource_pk: 2093abeb3e48233e3f42cd293ab745fc_15441_ENV, id: 223886058), (resource_pk: 375517a723363565036062edbf6839b8_15441_ENV, id: 223890008)
[2021-09-10 10:41:28] (resource_pk: 20adf15b33073d0467bccfe61ab3020d_15441_ENV, id: 223886075), (resource_pk: 5a9756288161ca83ce34d8c99037eae4_15441_ENV, id: 223896311)
[2021-09-10 10:41:28] (resource_pk: 21506d94d13ecf5923410056f477d0fc_15441_ENV, id: 223886193), (resource_pk: 90886eb1a94ef5fe79b7d702e9caedde_15441_ENV, id: 223905797)
[STOP] [2021-09-10 10:41:28] calculate_statistics
[START] [2021-09-10 10:41:28] complete_harvest_instance
[START] [2021-09-10 10:41:28] overall_tsv_creation
[INFO] [2021-09-10 10:41:28] Processing group of 11299 in 2 batches of 10000
[INFO] [2021-09-10 10:42:46] 39983 Traits (unfiltered)...
[INFO] [2021-09-10 10:48:15] 39983 Traits (filtered)...
[INFO] [2021-09-10 10:48:18] 0 Associations (filtered)...
[INFO] [2021-09-10 10:48:22] 0 metadata added.
[INFO] [2021-09-10 10:48:22] 0 metadata added.
[INFO] [2021-09-10 10:49:51] 5016 Traits (unfiltered)...
[INFO] [2021-09-10 10:51:09] 5016 Traits (filtered)...
[INFO] [2021-09-10 10:51:09] 0 Associations (filtered)...
[INFO] [2021-09-10 10:51:09] 0 metadata added.
[INFO] [2021-09-10 10:51:09] 0 metadata added.
[INFO] [2021-09-10 10:51:43] Average Time: 273.71
[INFO] [2021-09-10 10:51:43] Total Time: 10m15s
[STOP] [2021-09-10 10:51:43] overall_tsv_creation
[INFO] [2021-09-10 10:51:43] Done. Check your files:
[INFO] [2021-09-10 10:51:43] (11438 lines) /app/public/data/n_am_flora/publish_bibliographic_citations.tsv
[INFO] [2021-09-10 10:51:43] (11299 lines) /app/public/data/n_am_flora/publish_nodes.tsv
[INFO] [2021-09-10 10:51:43] (11299 lines) /app/public/data/n_am_flora/publish_scientific_names.tsv
[INFO] [2021-09-10 10:51:43] (11438 lines) /app/public/data/n_am_flora/publish_articles.tsv
[INFO] [2021-09-10 10:51:43] (11438 lines) /app/public/data/n_am_flora/publish_content_sections.tsv
[INFO] [2021-09-10 10:51:43] (45000 lines) /app/public/data/n_am_flora/publish_traits.tsv
[INFO] [2021-09-10 10:51:43] (1 lines) /app/public/data/n_am_flora/publish_metadata.tsv
[STOP] [2021-09-10 10:51:44] complete_harvest_instance
[START] [2021-09-10 10:51:44] completed
[STOP] [2021-09-10 10:51:44] completed
[STOP] [2021-09-10 10:51:44] logged process, took 1314.82

Latest Process