Harvest for BROT plant trait database for Mediterranean Created 31 May 14:01

Stage: completed
Fetched: 31 May 14:01
Validated: 31 May 14:01
Deltas Created 31 May 14:01
Units Normalized: 31 May 14:02
Ancestry Built: 31 May 14:02
Nodes Matched: 31 May 14:02
Names Parsed: 31 May 14:02
New Models Stored: 31 May 14:02
Indexed: 31 May 14:02
Completed: 31 May 14:05
Time to Harvest: less than a minute

Harvesting Log

(285 lines)
[INFO] [2021-05-31 14:01:50] Created harvest instance #3924
[STOP] [2021-05-31 14:01:50] create_harvest_instance
[START] [2021-05-31 14:01:50] fetch_files
[STOP] [2021-05-31 14:01:50] fetch_files
[START] [2021-05-31 14:01:50] validate_each_file
[INFO] [2021-05-31 14:01:50] Looping over 4 formats...
[INFO] [2021-05-31 14:01:50] ...refs (/app/public/data/BROT/references.txt)
[INFO] [2021-05-31 14:01:50] Valid: /app/public/converted_csv/BROT_refs_3924.csv (295 lines)
[INFO] [2021-05-31 14:01:50] ...nodes (/app/public/data/BROT/taxa.txt)
[INFO] [2021-05-31 14:01:50] Valid: /app/public/converted_csv/BROT_nodes_3924.csv (950 lines)
[INFO] [2021-05-31 14:01:50] ...occurrences (/app/public/data/BROT/occurrences.txt)
[INFO] [2021-05-31 14:01:51] Valid: /app/public/converted_csv/BROT_occurrences_3924.csv (11482 lines)
[INFO] [2021-05-31 14:01:51] ...measurements (/app/public/data/BROT/measurements or facts.txt)
[INFO] [2021-05-31 14:01:53] Valid: /app/public/converted_csv/BROT_measurements_3924.csv (11449 lines)
[STOP] [2021-05-31 14:01:53] validate_each_file
[START] [2021-05-31 14:01:53] convert_to_csv
[INFO] [2021-05-31 14:01:53] Looping over 4 formats...
[INFO] [2021-05-31 14:01:53] ...refs (/app/public/data/BROT/references.txt)
[CMD] [2021-05-31 14:01:53] /usr/bin/sort /app/public/converted_csv/BROT_refs_3924.csv > /app/public/converted_csv/BROT_refs_3924.csv_sorted
[INFO] [2021-05-31 14:01:53] Converted: /app/public/converted_csv/BROT_refs_3924.csv (295 lines)
[INFO] [2021-05-31 14:01:53] ...nodes (/app/public/data/BROT/taxa.txt)
[CMD] [2021-05-31 14:01:53] /usr/bin/sort /app/public/converted_csv/BROT_nodes_3924.csv > /app/public/converted_csv/BROT_nodes_3924.csv_sorted
[INFO] [2021-05-31 14:01:53] Converted: /app/public/converted_csv/BROT_nodes_3924.csv (950 lines)
[INFO] [2021-05-31 14:01:53] ...occurrences (/app/public/data/BROT/occurrences.txt)
[CMD] [2021-05-31 14:01:53] /usr/bin/sort /app/public/converted_csv/BROT_occurrences_3924.csv > /app/public/converted_csv/BROT_occurrences_3924.csv_sorted
[INFO] [2021-05-31 14:01:54] Converted: /app/public/converted_csv/BROT_occurrences_3924.csv (11482 lines)
[INFO] [2021-05-31 14:01:54] ...measurements (/app/public/data/BROT/measurements or facts.txt)
[CMD] [2021-05-31 14:01:54] /usr/bin/sort /app/public/converted_csv/BROT_measurements_3924.csv > /app/public/converted_csv/BROT_measurements_3924.csv_sorted
[INFO] [2021-05-31 14:01:54] Converted: /app/public/converted_csv/BROT_measurements_3924.csv (11449 lines)
[STOP] [2021-05-31 14:01:54] convert_to_csv
[START] [2021-05-31 14:01:54] calculate_delta
[INFO] [2021-05-31 14:01:54] Looping over 4 formats...
[INFO] [2021-05-31 14:01:54] ...refs (/app/public/data/BROT/references.txt)
[CMD] [2021-05-31 14:01:54] echo "0a" > /app/public/diff/BROT_refs_3924.diff
[CMD] [2021-05-31 14:01:55] tail -n +1 /app/public/converted_csv/BROT_refs_3924.csv >> /app/public/diff/BROT_refs_3924.diff
[CMD] [2021-05-31 14:01:55] echo "." >> /app/public/diff/BROT_refs_3924.diff
[INFO] [2021-05-31 14:01:55] Created diff: /app/public/diff/BROT_refs_3924.diff (297 lines)
[INFO] [2021-05-31 14:01:55] ...nodes (/app/public/data/BROT/taxa.txt)
[CMD] [2021-05-31 14:01:55] echo "0a" > /app/public/diff/BROT_nodes_3924.diff
[CMD] [2021-05-31 14:01:56] tail -n +1 /app/public/converted_csv/BROT_nodes_3924.csv >> /app/public/diff/BROT_nodes_3924.diff
[CMD] [2021-05-31 14:01:56] echo "." >> /app/public/diff/BROT_nodes_3924.diff
[INFO] [2021-05-31 14:01:56] Created diff: /app/public/diff/BROT_nodes_3924.diff (952 lines)
[INFO] [2021-05-31 14:01:56] ...occurrences (/app/public/data/BROT/occurrences.txt)
[CMD] [2021-05-31 14:01:56] echo "0a" > /app/public/diff/BROT_occurrences_3924.diff
[CMD] [2021-05-31 14:01:57] tail -n +1 /app/public/converted_csv/BROT_occurrences_3924.csv >> /app/public/diff/BROT_occurrences_3924.diff
[CMD] [2021-05-31 14:01:57] echo "." >> /app/public/diff/BROT_occurrences_3924.diff
[INFO] [2021-05-31 14:01:58] Created diff: /app/public/diff/BROT_occurrences_3924.diff (11484 lines)
[INFO] [2021-05-31 14:01:58] ...measurements (/app/public/data/BROT/measurements or facts.txt)
[CMD] [2021-05-31 14:01:58] echo "0a" > /app/public/diff/BROT_measurements_3924.diff
[CMD] [2021-05-31 14:01:58] tail -n +1 /app/public/converted_csv/BROT_measurements_3924.csv >> /app/public/diff/BROT_measurements_3924.diff
[CMD] [2021-05-31 14:01:59] echo "." >> /app/public/diff/BROT_measurements_3924.diff
[INFO] [2021-05-31 14:01:59] Created diff: /app/public/diff/BROT_measurements_3924.diff (11451 lines)
[STOP] [2021-05-31 14:01:59] calculate_delta
[START] [2021-05-31 14:01:59] parse_diff_and_store
[INFO] [2021-05-31 14:01:59] Handling diff: /app/public/diff/BROT_refs_3924.diff (297 lines)
[INFO] [2021-05-31 14:01:59] Loading refs diff file into memory (297 /app/public/diff/BROT_refs_3924.diff lines)...
[INFO] [2021-05-31 14:02:00] Handling diff: /app/public/diff/BROT_nodes_3924.diff (952 lines)
[INFO] [2021-05-31 14:02:00] Loading nodes diff file into memory (952 /app/public/diff/BROT_nodes_3924.diff lines)...
[INFO] [2021-05-31 14:02:01] Handling diff: /app/public/diff/BROT_occurrences_3924.diff (11484 lines)
[INFO] [2021-05-31 14:02:01] Loading occurrences diff file into memory (11484 /app/public/diff/BROT_occurrences_3924.diff lines)...
[INFO] [2021-05-31 14:02:04] Handling diff: /app/public/diff/BROT_measurements_3924.diff (11451 lines)
[INFO] [2021-05-31 14:02:05] Loading measurements diff file into memory (11451 /app/public/diff/BROT_measurements_3924.diff lines)...
[INFO] [2021-05-31 14:02:14] Storing 295 References
[INFO] [2021-05-31 14:02:14] Processing group of 295 in 1 groups of 1000
[INFO] [2021-05-31 14:02:14] Average Time: 0.06
[INFO] [2021-05-31 14:02:14] Total Time: 1s
[INFO] [2021-05-31 14:02:14] Storing 1030 ScientificNames
[INFO] [2021-05-31 14:02:14] Processing group of 1030 in 2 groups of 1000
[INFO] [2021-05-31 14:02:14] Average Time: 0.15
[INFO] [2021-05-31 14:02:14] Total Time: 1s
[INFO] [2021-05-31 14:02:14] Storing 1030 Nodes
[INFO] [2021-05-31 14:02:14] Processing group of 1030 in 2 groups of 1000
[INFO] [2021-05-31 14:02:14] Average Time: 0.145
[INFO] [2021-05-31 14:02:14] Total Time: 1s
[INFO] [2021-05-31 14:02:14] Storing 11482 Occurrences
[INFO] [2021-05-31 14:02:14] Processing group of 11482 in 12 groups of 1000
[INFO] [2021-05-31 14:02:16] Average Time: 0.149
[INFO] [2021-05-31 14:02:16] Total Time: 2s
[INFO] [2021-05-31 14:02:16] last 3 / first 3: 1.35
[INFO] [2021-05-31 14:02:16] Std.Dev: 0.06324555320336758; Max: 0.28
[INFO] [2021-05-31 14:02:16] Storing 9724 OccurrenceMetadata
[INFO] [2021-05-31 14:02:16] Processing group of 9724 in 10 groups of 1000
[INFO] [2021-05-31 14:02:17] Average Time: 0.115
[INFO] [2021-05-31 14:02:17] Total Time: 2s
[INFO] [2021-05-31 14:02:17] last 3 / first 3: 0.89
[INFO] [2021-05-31 14:02:17] Std.Dev: 0.0; Max: 0.12
[INFO] [2021-05-31 14:02:17] Storing 10004 TraitsReferences
[INFO] [2021-05-31 14:02:17] Processing group of 10004 in 11 groups of 1000
[INFO] [2021-05-31 14:02:18] Average Time: 0.061
[INFO] [2021-05-31 14:02:18] Total Time: 1s
[INFO] [2021-05-31 14:02:18] last 3 / first 3: 0.67
[INFO] [2021-05-31 14:02:18] Std.Dev: 0.03162277660168379; Max: 0.13
[INFO] [2021-05-31 14:02:18] Storing 11449 Traits
[INFO] [2021-05-31 14:02:18] Processing group of 11449 in 12 groups of 1000
[INFO] [2021-05-31 14:02:22] Average Time: 0.292
[INFO] [2021-05-31 14:02:22] Total Time: 4s
[INFO] [2021-05-31 14:02:22] last 3 / first 3: 0.9
[INFO] [2021-05-31 14:02:22] Std.Dev: 0.05477225575051661; Max: 0.37
[INFO] [2021-05-31 14:02:22] Storing 35193 MetaTraits
[INFO] [2021-05-31 14:02:22] Processing group of 35193 in 36 groups of 1000
[INFO] [2021-05-31 14:02:26] Average Time: 0.122
[INFO] [2021-05-31 14:02:26] Total Time: 5s
[INFO] [2021-05-31 14:02:26] last 3 / first 3: 0.79
[INFO] [2021-05-31 14:02:26] Std.Dev: 0.03162277660168379; Max: 0.18
[STOP] [2021-05-31 14:02:26] parse_diff_and_store
[START] [2021-05-31 14:02:26] resolve_keys
[INFO] [2021-05-31 14:02:34] Occurrences to nodes (through scientific_names)...
[INFO] [2021-05-31 14:02:34] traits to occurrences...
[INFO] [2021-05-31 14:02:34] traits to nodes (through occurrences)...
[INFO] [2021-05-31 14:02:35] Traits to sex term...
[INFO] [2021-05-31 14:02:35] Traits to lifestage term...
[INFO] [2021-05-31 14:02:35] MetaTraits to traits...
[INFO] [2021-05-31 14:02:36] MetaTraits (simple, measurement row refers to parent) to traits...
[INFO] [2021-05-31 14:02:36] Assocs to occurrences...
[INFO] [2021-05-31 14:02:36] Assocs to nodes...
[INFO] [2021-05-31 14:02:36] Assoc to sex term...
[INFO] [2021-05-31 14:02:36] Assoc to lifestage term...
[INFO] [2021-05-31 14:02:36] MetaAssoc to assocs...
[STOP] [2021-05-31 14:02:36] resolve_keys
[START] [2021-05-31 14:02:36] hold_for_later_1
[STOP] [2021-05-31 14:02:36] hold_for_later_1
[START] [2021-05-31 14:02:36] hold_for_later_2
[STOP] [2021-05-31 14:02:36] hold_for_later_2
[START] [2021-05-31 14:02:36] resolve_missing_parents
[STOP] [2021-05-31 14:02:36] resolve_missing_parents
[START] [2021-05-31 14:02:36] rebuild_nodes
[START] [2021-05-31 14:02:36] Flattener#flatten
[START] [2021-05-31 14:02:36] Flattener#study_resource
[START] [2021-05-31 14:02:36] Flattener#build_ancestry
[STOP] [2021-05-31 14:02:36] Flattener#build_ancestry
[INFO] [2021-05-31 14:02:36] 1030 ancestry keys
[START] [2021-05-31 14:02:36] build_node_ancestors
[INFO] [2021-05-31 14:02:36] old ancestors deleted.
[STOP] [2021-05-31 14:02:36] build_node_ancestors
[START] [2021-05-31 14:02:36] Flattener#propagate_ancestor_ids
[STOP] [2021-05-31 14:02:36] Flattener#propagate_ancestor_ids
[STOP] [2021-05-31 14:02:36] Flattener#flatten
[STOP] [2021-05-31 14:02:36] rebuild_nodes
[START] [2021-05-31 14:02:36] resolve_missing_media_owners
[STOP] [2021-05-31 14:02:36] resolve_missing_media_owners
[START] [2021-05-31 14:02:36] sanitize_media_verbatims
[STOP] [2021-05-31 14:02:36] sanitize_media_verbatims
[START] [2021-05-31 14:02:36] queue_downloads
[STOP] [2021-05-31 14:02:36] queue_downloads
[START] [2021-05-31 14:02:36] parse_names
[WARN] [2021-05-31 14:02:36] I see 1030 names which still need to be parsed.
[STOP] [2021-05-31 14:02:38] parse_names
[START] [2021-05-31 14:02:38] denormalize_canonical_names_to_nodes
[STOP] [2021-05-31 14:02:38] denormalize_canonical_names_to_nodes
[START] [2021-05-31 14:02:38] match_nodes
[START] [2021-05-31 14:02:38] map_all_nodes_to_pages
[STOP] [2021-05-31 14:02:53] map_all_nodes_to_pages
[INFO] [2021-05-31 14:02:53] 24 Unmatched nodes (of 1030)! That's too many to output. Full list in /app/public/data/BROT/unmatched_nodes.txt ; First 10: Canonical: Centaurea antennata; Node#95112940; ResourceID: Centaurea antennata; Canonical: Centaurea boissieri duffourii; Node#95112943; ResourceID: Centaurea boissieri ssp. duffourii; Canonical: Centaurea linifolia linifolia; Node#95112947; ResourceID: Centaurea linifolia ssp. linifolia; Canonical: Centaurea pectinata pectinata; Node#95112950; ResourceID: Centaurea pectinata ssp. pectinata; Canonical: Chrysanthemum segetum; Node#95112978; ResourceID: Chrysanthemum segetum; Canonical: Inula conyza; Node#95113288; ResourceID: Inula conyza; Canonical: Inula montana; Node#95113289; ResourceID: Inula montana; Canonical: Senecio jacobaea; Node#95113614; ResourceID: Senecio jacobaea; Canonical: Calamintha sylvatica ascendens; Node#95112906; ResourceID: Calamintha sylvatica ssp. ascendens; Canonical: Teucrium expansum; Node#95113668; ResourceID: Teucrium expassum
[START] [2021-05-31 14:02:53] update_nodes
[STOP] [2021-05-31 14:02:53] update_nodes
[STOP] [2021-05-31 14:02:53] match_nodes
[START] [2021-05-31 14:02:53] reindex_search
[STOP] [2021-05-31 14:02:54] reindex_search
[START] [2021-05-31 14:02:54] normalize_units
[STOP] [2021-05-31 14:02:58] normalize_units
[START] [2021-05-31 14:02:58] calculate_statistics
[2021-05-31 14:02:59] (NEAR) DUPLICATE TRAITS FOUND! There are only 7910 (of 11449 total) unique traits.
[2021-05-31 14:03:02] (Near) duplicate trait pairs (up to 100):
[2021-05-31 14:03:02] (resource_pk: M-C00199, id: 214763158), (resource_pk: M-E02855, id: 214768998)
[2021-05-31 14:03:02] (resource_pk: M-C00201, id: 214763160), (resource_pk: M-E00070, id: 214767391)
[2021-05-31 14:03:02] (resource_pk: M-C00203, id: 214763162), (resource_pk: M-E02817, id: 214768975)
[2021-05-31 14:03:02] (resource_pk: M-C00206, id: 214763165), (resource_pk: M-E04188, id: 214769828)
[2021-05-31 14:03:02] (resource_pk: M-C00207, id: 214763166), (resource_pk: M-E02893, id: 214769023)
[2021-05-31 14:03:02] (resource_pk: M-C00213, id: 214763172), (resource_pk: M-E02681, id: 214768890)
[2021-05-31 14:03:02] (resource_pk: M-C00216, id: 214763175), (resource_pk: M-C04850, id: 214765353)
[2021-05-31 14:03:02] (resource_pk: M-C00216, id: 214763175), (resource_pk: M-E01318, id: 214768097)
[2021-05-31 14:03:02] (resource_pk: M-C00216, id: 214763175), (resource_pk: M-E02473, id: 214768750)
[2021-05-31 14:03:02] (resource_pk: M-C00219, id: 214763178), (resource_pk: M-E04180, id: 214769820)
[2021-05-31 14:03:02] (resource_pk: M-C00226, id: 214763185), (resource_pk: M-E01452, id: 214768158)
[2021-05-31 14:03:02] (resource_pk: M-C00226, id: 214763185), (resource_pk: M-E01459, id: 214768165)
[2021-05-31 14:03:02] (resource_pk: M-C00235, id: 214763194), (resource_pk: M-E04209, id: 214769849)
[2021-05-31 14:03:02] (resource_pk: M-C00252, id: 214763211), (resource_pk: M-E04197, id: 214769837)
[2021-05-31 14:03:02] (resource_pk: M-C00261, id: 214763220), (resource_pk: M-E04192, id: 214769832)
[2021-05-31 14:03:02] (resource_pk: M-C00264, id: 214763223), (resource_pk: M-E01103, id: 214767977)
[2021-05-31 14:03:02] (resource_pk: M-C00280, id: 214763237), (resource_pk: M-E01323, id: 214768102)
[2021-05-31 14:03:02] (resource_pk: M-C00516, id: 214763282), (resource_pk: M-C00599, id: 214763362)
[2021-05-31 14:03:02] (resource_pk: M-C00516, id: 214763282), (resource_pk: M-E03785, id: 214769575)
[2021-05-31 14:03:02] (resource_pk: M-C00518, id: 214763284), (resource_pk: M-E00463, id: 214767613)
[2021-05-31 14:03:02] (resource_pk: M-C00523, id: 214763288), (resource_pk: M-E02857, id: 214768999)
[2021-05-31 14:03:02] (resource_pk: M-C00525, id: 214763290), (resource_pk: M-E00072, id: 214767392)
[2021-05-31 14:03:02] (resource_pk: M-C00527, id: 214763292), (resource_pk: M-E02819, id: 214768976)
[2021-05-31 14:03:02] (resource_pk: M-C00529, id: 214763294), (resource_pk: M-E01851, id: 214768379)
[2021-05-31 14:03:02] (resource_pk: M-C00530, id: 214763295), (resource_pk: M-E01871, id: 214768390)
[2021-05-31 14:03:02] (resource_pk: M-C00530, id: 214763295), (resource_pk: M-E04298, id: 214769883)
[2021-05-31 14:03:02] (resource_pk: M-C00531, id: 214763296), (resource_pk: M-E01909, id: 214768410)
[2021-05-31 14:03:02] (resource_pk: M-C00532, id: 214763297), (resource_pk: M-E01929, id: 214768421)
[2021-05-31 14:03:02] (resource_pk: M-C00533, id: 214763298), (resource_pk: M-E04306, id: 214769891)
[2021-05-31 14:03:02] (resource_pk: M-C00536, id: 214763301), (resource_pk: M-E03859, id: 214769625)
[2021-05-31 14:03:02] (resource_pk: M-C00539, id: 214763304), (resource_pk: M-C00607, id: 214763370)
[2021-05-31 14:03:02] (resource_pk: M-C00540, id: 214763305), (resource_pk: M-E00872, id: 214767846)
[2021-05-31 14:03:02] (resource_pk: M-C00540, id: 214763305), (resource_pk: M-E01336, id: 214768106)
[2021-05-31 14:03:02] (resource_pk: M-C00540, id: 214763305), (resource_pk: M-E01949, id: 214768433)
[2021-05-31 14:03:02] (resource_pk: M-C00540, id: 214763305), (resource_pk: M-E02476, id: 214768751)
[2021-05-31 14:03:02] (resource_pk: M-C00541, id: 214763306), (resource_pk: M-E00258, id: 214767498)
[2021-05-31 14:03:02] (resource_pk: M-C00541, id: 214763306), (resource_pk: M-E02545, id: 214768798)
[2021-05-31 14:03:02] (resource_pk: M-C00542, id: 214763307), (resource_pk: M-C00609, id: 214763372)
[2021-05-31 14:03:02] (resource_pk: M-C00542, id: 214763307), (resource_pk: M-E00722, id: 214767760)
[2021-05-31 14:03:02] (resource_pk: M-C00542, id: 214763307), (resource_pk: M-E01969, id: 214768445)
[2021-05-31 14:03:02] (resource_pk: M-C00542, id: 214763307), (resource_pk: M-E02580, id: 214768822)
[2021-05-31 14:03:02] (resource_pk: M-C00542, id: 214763307), (resource_pk: M-E04291, id: 214769876)
[2021-05-31 14:03:02] (resource_pk: M-C00543, id: 214763308), (resource_pk: M-C00610, id: 214763373)
[2021-05-31 14:03:02] (resource_pk: M-C00543, id: 214763308), (resource_pk: M-E00732, id: 214767766)
[2021-05-31 14:03:02] (resource_pk: M-C00543, id: 214763308), (resource_pk: M-E04290, id: 214769875)
[2021-05-31 14:03:02] (resource_pk: M-C00544, id: 214763309), (resource_pk: M-E03834, id: 214769608)
[2021-05-31 14:03:02] (resource_pk: M-C00544, id: 214763309), (resource_pk: M-E04292, id: 214769877)
[2021-05-31 14:03:02] (resource_pk: M-C00546, id: 214763311), (resource_pk: M-E01989, id: 214768455)
[2021-05-31 14:03:02] (resource_pk: M-C00546, id: 214763311), (resource_pk: M-E04296, id: 214769881)
[2021-05-31 14:03:02] (resource_pk: M-C00547, id: 214763312), (resource_pk: M-E02651, id: 214768869)
[2021-05-31 14:03:02] (resource_pk: M-C00549, id: 214763314), (resource_pk: M-E00817, id: 214767815)
[2021-05-31 14:03:02] (resource_pk: M-C00549, id: 214763314), (resource_pk: M-E04322, id: 214769907)
[2021-05-31 14:03:02] (resource_pk: M-C00550, id: 214763315), (resource_pk: M-E01342, id: 214768112)
[2021-05-31 14:03:02] (resource_pk: M-C00550, id: 214763315), (resource_pk: M-E01466, id: 214768168)
[2021-05-31 14:03:02] (resource_pk: M-C00553, id: 214763318), (resource_pk: M-E04319, id: 214769904)
[2021-05-31 14:03:02] (resource_pk: M-C00560, id: 214763325), (resource_pk: M-C00598, id: 214763361)
[2021-05-31 14:03:02] (resource_pk: M-C00561, id: 214763326), (resource_pk: M-E04295, id: 214769880)
[2021-05-31 14:03:02] (resource_pk: M-C00564, id: 214763329), (resource_pk: M-E04307, id: 214769892)
[2021-05-31 14:03:02] (resource_pk: M-C00568, id: 214763333), (resource_pk: M-E00564, id: 214767670)
[2021-05-31 14:03:02] (resource_pk: M-C00570, id: 214763335), (resource_pk: M-E04302, id: 214769887)
[2021-05-31 14:03:02] (resource_pk: M-C00571, id: 214763336), (resource_pk: M-C00595, id: 214763358)
[2021-05-31 14:03:02] (resource_pk: M-C00572, id: 214763337), (resource_pk: M-C00596, id: 214763359)
[2021-05-31 14:03:02] (resource_pk: M-C00572, id: 214763337), (resource_pk: M-E01133, id: 214767991)
[2021-05-31 14:03:02] (resource_pk: M-C00573, id: 214763338), (resource_pk: M-C00597, id: 214763360)
[2021-05-31 14:03:02] (resource_pk: M-C00573, id: 214763338), (resource_pk: M-E01135, id: 214767993)
[2021-05-31 14:03:02] (resource_pk: M-C00574, id: 214763339), (resource_pk: M-C00600, id: 214763363)
[2021-05-31 14:03:02] (resource_pk: M-C00575, id: 214763340), (resource_pk: M-C00601, id: 214763364)
[2021-05-31 14:03:02] (resource_pk: M-C00576, id: 214763341), (resource_pk: M-C00602, id: 214763365)
[2021-05-31 14:03:02] (resource_pk: M-C00577, id: 214763342), (resource_pk: M-C00604, id: 214763367)
[2021-05-31 14:03:02] (resource_pk: M-C00578, id: 214763343), (resource_pk: M-C00605, id: 214763368)
[2021-05-31 14:03:02] (resource_pk: M-C00582, id: 214763345), (resource_pk: M-C00606, id: 214763369)
[2021-05-31 14:03:02] (resource_pk: M-C00583, id: 214763346), (resource_pk: M-C00608, id: 214763371)
[2021-05-31 14:03:02] (resource_pk: M-C00583, id: 214763346), (resource_pk: M-E03822, id: 214769600)
[2021-05-31 14:03:02] (resource_pk: M-C00585, id: 214763348), (resource_pk: M-E01340, id: 214768110)
[2021-05-31 14:03:02] (resource_pk: M-C00586, id: 214763349), (resource_pk: M-E01341, id: 214768111)
[2021-05-31 14:03:02] (resource_pk: M-C00588, id: 214763351), (resource_pk: M-C00611, id: 214763374)
[2021-05-31 14:03:02] (resource_pk: M-C00588, id: 214763351), (resource_pk: M-E01343, id: 214768113)
[2021-05-31 14:03:02] (resource_pk: M-C00590, id: 214763353), (resource_pk: M-C00612, id: 214763375)
[2021-05-31 14:03:02] (resource_pk: M-C00590, id: 214763353), (resource_pk: M-E01143, id: 214768001)
[2021-05-31 14:03:02] (resource_pk: M-C00591, id: 214763354), (resource_pk: M-C00613, id: 214763376)
[2021-05-31 14:03:02] (resource_pk: M-C00591, id: 214763354), (resource_pk: M-E01144, id: 214768002)
[2021-05-31 14:03:02] (resource_pk: M-C00592, id: 214763355), (resource_pk: M-C00614, id: 214763377)
[2021-05-31 14:03:02] (resource_pk: M-C00592, id: 214763355), (resource_pk: M-E01147, id: 214768005)
[2021-05-31 14:03:02] (resource_pk: M-C00596, id: 214763359), (resource_pk: M-E01133, id: 214767991)
[2021-05-31 14:03:02] (resource_pk: M-C00597, id: 214763360), (resource_pk: M-E01135, id: 214767993)
[2021-05-31 14:03:02] (resource_pk: M-C00599, id: 214763362), (resource_pk: M-E03785, id: 214769575)
[2021-05-31 14:03:02] (resource_pk: M-C00603, id: 214763366), (resource_pk: M-E01137, id: 214767995)
[2021-05-31 14:03:02] (resource_pk: M-C00603, id: 214763366), (resource_pk: M-E02529, id: 214768787)
[2021-05-31 14:03:02] (resource_pk: M-C00608, id: 214763371), (resource_pk: M-E03822, id: 214769600)
[2021-05-31 14:03:02] (resource_pk: M-C00609, id: 214763372), (resource_pk: M-E00722, id: 214767760)
[2021-05-31 14:03:02] (resource_pk: M-C00609, id: 214763372), (resource_pk: M-E01969, id: 214768445)
[2021-05-31 14:03:02] (resource_pk: M-C00609, id: 214763372), (resource_pk: M-E02580, id: 214768822)
[2021-05-31 14:03:02] (resource_pk: M-C00609, id: 214763372), (resource_pk: M-E04291, id: 214769876)
[2021-05-31 14:03:02] (resource_pk: M-C00610, id: 214763373), (resource_pk: M-E00732, id: 214767766)
[2021-05-31 14:03:02] (resource_pk: M-C00610, id: 214763373), (resource_pk: M-E04290, id: 214769875)
[2021-05-31 14:03:02] (resource_pk: M-C00611, id: 214763374), (resource_pk: M-E01343, id: 214768113)
[2021-05-31 14:03:02] (resource_pk: M-C00612, id: 214763375), (resource_pk: M-E01143, id: 214768001)
[2021-05-31 14:03:02] (resource_pk: M-C00613, id: 214763376), (resource_pk: M-E01144, id: 214768002)
[2021-05-31 14:03:02] (resource_pk: M-C00614, id: 214763377), (resource_pk: M-E01147, id: 214768005)
[2021-05-31 14:03:02] (resource_pk: M-C00747, id: 214763380), (resource_pk: M-E00464, id: 214767614)
[STOP] [2021-05-31 14:03:02] calculate_statistics
[START] [2021-05-31 14:03:02] complete_harvest_instance
[START] [2021-05-31 14:03:02] overall_tsv_creation
[INFO] [2021-05-31 14:03:02] Processing group of 1030 in 1 batches of 10000
[INFO] [2021-05-31 14:03:38] 11449 Traits (unfiltered)...
[INFO] [2021-05-31 14:04:33] 11449 Traits (filtered)...
[INFO] [2021-05-31 14:04:33] 0 Associations (filtered)...
[INFO] [2021-05-31 14:04:36] 19418 metadata added.
[INFO] [2021-05-31 14:04:36] 0 metadata added.
[INFO] [2021-05-31 14:05:03] Average Time: 97.99
[INFO] [2021-05-31 14:05:03] Total Time: 2m1s
[STOP] [2021-05-31 14:05:03] overall_tsv_creation
[INFO] [2021-05-31 14:05:03] Done. Check your files:
[INFO] [2021-05-31 14:05:03] (1030 lines) /app/public/data/BROT/publish_nodes.tsv
[INFO] [2021-05-31 14:05:03] (1979 lines) /app/public/data/BROT/publish_node_ancestors.tsv
[INFO] [2021-05-31 14:05:04] (1030 lines) /app/public/data/BROT/publish_scientific_names.tsv
[INFO] [2021-05-31 14:05:04] (11450 lines) /app/public/data/BROT/publish_traits.tsv
[INFO] [2021-05-31 14:05:05] (19419 lines) /app/public/data/BROT/publish_metadata.tsv
[STOP] [2021-05-31 14:05:05] complete_harvest_instance
[START] [2021-05-31 14:05:05] completed
[STOP] [2021-05-31 14:05:05] completed
[STOP] [2021-05-31 14:05:05] logged process, took 195.08

Latest Process