Harvest for Pantheria Created 28 May 09:41

Harvesting Log

[INFO] [2021-05-28 09:41:32] Created harvest instance #3902
[STOP] [2021-05-28 09:41:32] create_harvest_instance
[START] [2021-05-28 09:41:32] fetch_files
[STOP] [2021-05-28 09:41:32] fetch_files
[START] [2021-05-28 09:41:32] validate_each_file
[INFO] [2021-05-28 09:41:32] Looping over 4 formats...
[INFO] [2021-05-28 09:41:32] ...refs (/app/public/data/Pantheria/reference.tab)
[INFO] [2021-05-28 09:41:33] Valid: /app/public/converted_csv/Pantheria_refs_3902.csv (3143 lines)
[INFO] [2021-05-28 09:41:33] ...nodes (/app/public/data/Pantheria/taxon.tab)
[INFO] [2021-05-28 09:41:33] Valid: /app/public/converted_csv/Pantheria_nodes_3902.csv (5416 lines)
[INFO] [2021-05-28 09:41:33] ...occurrences (/app/public/data/Pantheria/occurrence.tab)
[INFO] [2021-05-28 09:41:45] Valid: /app/public/converted_csv/Pantheria_occurrences_3902.csv (103733 lines)
[INFO] [2021-05-28 09:41:45] ...measurements (/app/public/data/Pantheria/measurement_or_fact.tab)
[INFO] [2021-05-28 09:41:57] Valid: /app/public/converted_csv/Pantheria_measurements_3902.csv (104306 lines)
[STOP] [2021-05-28 09:41:57] validate_each_file
[START] [2021-05-28 09:41:57] convert_to_csv
[INFO] [2021-05-28 09:41:57] Looping over 4 formats...
[INFO] [2021-05-28 09:41:57] ...refs (/app/public/data/Pantheria/reference.tab)
[CMD] [2021-05-28 09:41:57] /usr/bin/sort /app/public/converted_csv/Pantheria_refs_3902.csv > /app/public/converted_csv/Pantheria_refs_3902.csv_sorted
[INFO] [2021-05-28 09:41:57] Converted: /app/public/converted_csv/Pantheria_refs_3902.csv (3143 lines)
[INFO] [2021-05-28 09:41:57] ...nodes (/app/public/data/Pantheria/taxon.tab)
[CMD] [2021-05-28 09:41:57] /usr/bin/sort /app/public/converted_csv/Pantheria_nodes_3902.csv > /app/public/converted_csv/Pantheria_nodes_3902.csv_sorted
[INFO] [2021-05-28 09:41:58] Converted: /app/public/converted_csv/Pantheria_nodes_3902.csv (5416 lines)
[INFO] [2021-05-28 09:41:58] ...occurrences (/app/public/data/Pantheria/occurrence.tab)
[CMD] [2021-05-28 09:41:58] /usr/bin/sort /app/public/converted_csv/Pantheria_occurrences_3902.csv > /app/public/converted_csv/Pantheria_occurrences_3902.csv_sorted
[INFO] [2021-05-28 09:41:58] Converted: /app/public/converted_csv/Pantheria_occurrences_3902.csv (103733 lines)
[INFO] [2021-05-28 09:41:58] ...measurements (/app/public/data/Pantheria/measurement_or_fact.tab)
[CMD] [2021-05-28 09:41:58] /usr/bin/sort /app/public/converted_csv/Pantheria_measurements_3902.csv > /app/public/converted_csv/Pantheria_measurements_3902.csv_sorted
[INFO] [2021-05-28 09:41:59] Converted: /app/public/converted_csv/Pantheria_measurements_3902.csv (104306 lines)
[STOP] [2021-05-28 09:41:59] convert_to_csv
[START] [2021-05-28 09:41:59] calculate_delta
[INFO] [2021-05-28 09:41:59] Looping over 4 formats...
[INFO] [2021-05-28 09:41:59] ...refs (/app/public/data/Pantheria/reference.tab)
[CMD] [2021-05-28 09:41:59] echo "0a" > /app/public/diff/Pantheria_refs_3902.diff
[CMD] [2021-05-28 09:41:59] tail -n +1 /app/public/converted_csv/Pantheria_refs_3902.csv >> /app/public/diff/Pantheria_refs_3902.diff
[CMD] [2021-05-28 09:42:00] echo "." >> /app/public/diff/Pantheria_refs_3902.diff
[INFO] [2021-05-28 09:42:00] Created diff: /app/public/diff/Pantheria_refs_3902.diff (3145 lines)
[INFO] [2021-05-28 09:42:00] ...nodes (/app/public/data/Pantheria/taxon.tab)
[CMD] [2021-05-28 09:42:00] echo "0a" > /app/public/diff/Pantheria_nodes_3902.diff
[CMD] [2021-05-28 09:42:00] tail -n +1 /app/public/converted_csv/Pantheria_nodes_3902.csv >> /app/public/diff/Pantheria_nodes_3902.diff
[CMD] [2021-05-28 09:42:00] echo "." >> /app/public/diff/Pantheria_nodes_3902.diff
[INFO] [2021-05-28 09:42:01] Created diff: /app/public/diff/Pantheria_nodes_3902.diff (5418 lines)
[INFO] [2021-05-28 09:42:01] ...occurrences (/app/public/data/Pantheria/occurrence.tab)
[CMD] [2021-05-28 09:42:01] echo "0a" > /app/public/diff/Pantheria_occurrences_3902.diff
[CMD] [2021-05-28 09:42:01] tail -n +1 /app/public/converted_csv/Pantheria_occurrences_3902.csv >> /app/public/diff/Pantheria_occurrences_3902.diff
[CMD] [2021-05-28 09:42:01] echo "." >> /app/public/diff/Pantheria_occurrences_3902.diff
[INFO] [2021-05-28 09:42:02] Created diff: /app/public/diff/Pantheria_occurrences_3902.diff (103735 lines)
[INFO] [2021-05-28 09:42:02] ...measurements (/app/public/data/Pantheria/measurement_or_fact.tab)
[CMD] [2021-05-28 09:42:02] echo "0a" > /app/public/diff/Pantheria_measurements_3902.diff
[CMD] [2021-05-28 09:42:02] tail -n +1 /app/public/converted_csv/Pantheria_measurements_3902.csv >> /app/public/diff/Pantheria_measurements_3902.diff
[CMD] [2021-05-28 09:42:02] echo "." >> /app/public/diff/Pantheria_measurements_3902.diff
[INFO] [2021-05-28 09:42:03] Created diff: /app/public/diff/Pantheria_measurements_3902.diff (104308 lines)
[STOP] [2021-05-28 09:42:03] calculate_delta
[START] [2021-05-28 09:42:03] parse_diff_and_store
[INFO] [2021-05-28 09:42:03] Handling diff: /app/public/diff/Pantheria_refs_3902.diff (3145 lines)
[INFO] [2021-05-28 09:42:03] Loading refs diff file into memory (3145 /app/public/diff/Pantheria_refs_3902.diff lines)...
[INFO] [2021-05-28 09:42:04] Handling diff: /app/public/diff/Pantheria_nodes_3902.diff (5418 lines)
[INFO] [2021-05-28 09:42:04] Loading nodes diff file into memory (5418 /app/public/diff/Pantheria_nodes_3902.diff lines)...
[INFO] [2021-05-28 09:42:08] Handling diff: /app/public/diff/Pantheria_occurrences_3902.diff (103735 lines)
[INFO] [2021-05-28 09:42:09] Loading occurrences diff file into memory (103735 /app/public/diff/Pantheria_occurrences_3902.diff lines)...
[INFO] [2021-05-28 09:42:24] Handling diff: /app/public/diff/Pantheria_measurements_3902.diff (104308 lines)
[INFO] [2021-05-28 09:42:24] Loading measurements diff file into memory (104308 /app/public/diff/Pantheria_measurements_3902.diff lines)...
[INFO] [2021-05-28 09:43:30] Storing 3143 References
[INFO] [2021-05-28 09:43:30] Processing group of 3143 in 4 groups of 1000
[INFO] [2021-05-28 09:43:33] Average Time: 0.75
[INFO] [2021-05-28 09:43:33] Total Time: 4s
[INFO] [2021-05-28 09:43:33] Storing 6831 ScientificNames
[INFO] [2021-05-28 09:43:33] Processing group of 6831 in 7 groups of 1000
[INFO] [2021-05-28 09:43:38] Average Time: 0.629
[INFO] [2021-05-28 09:43:38] Total Time: 5s
[INFO] [2021-05-28 09:43:38] last 3 / first 3: 0.34
[INFO] [2021-05-28 09:43:38] Std.Dev: 0.47644516998286385; Max: 1.5
[INFO] [2021-05-28 09:43:38] Storing 6831 Nodes
[INFO] [2021-05-28 09:43:38] Processing group of 6831 in 7 groups of 1000
[INFO] [2021-05-28 09:43:39] Average Time: 0.257
[INFO] [2021-05-28 09:43:39] Total Time: 2s
[INFO] [2021-05-28 09:43:39] last 3 / first 3: 0.93
[INFO] [2021-05-28 09:43:39] Std.Dev: 0.044721359549995794; Max: 0.35
[INFO] [2021-05-28 09:43:39] Storing 24605 NodesReferences
[INFO] [2021-05-28 09:43:39] Processing group of 24605 in 25 groups of 1000
[INFO] [2021-05-28 09:43:41] Average Time: 0.066
[INFO] [2021-05-28 09:43:41] Total Time: 2s
[INFO] [2021-05-28 09:43:41] last 3 / first 3: 0.62
[INFO] [2021-05-28 09:43:41] Std.Dev: 0.03162277660168379; Max: 0.16
[INFO] [2021-05-28 09:43:41] Storing 103733 Occurrences
[INFO] [2021-05-28 09:43:41] Processing group of 103733 in 104 groups of 1000
[INFO] [2021-05-28 09:43:56] Average Time: 0.135
[INFO] [2021-05-28 09:43:56] Total Time: 15s
[INFO] [2021-05-28 09:43:56] last 3 / first 3: 0.9
[INFO] [2021-05-28 09:43:56] Std.Dev: 0.08366600265340755; Max: 0.47
[INFO] [2021-05-28 09:43:56] Storing 11854 OccurrenceMetadata
[INFO] [2021-05-28 09:43:56] Processing group of 11854 in 12 groups of 1000
[INFO] [2021-05-28 09:43:57] Average Time: 0.12
[INFO] [2021-05-28 09:43:57] Total Time: 2s
[INFO] [2021-05-28 09:43:57] last 3 / first 3: 0.81
[INFO] [2021-05-28 09:43:57] Std.Dev: 0.03162277660168379; Max: 0.21
[INFO] [2021-05-28 09:43:57] Storing 103733 Traits
[INFO] [2021-05-28 09:43:57] Processing group of 103733 in 104 groups of 1000
[INFO] [2021-05-28 09:44:32] Average Time: 0.33
[INFO] [2021-05-28 09:44:32] Total Time: 35s
[INFO] [2021-05-28 09:44:32] last 3 / first 3: 0.91
[INFO] [2021-05-28 09:44:32] Std.Dev: 0.14491376746189438; Max: 1.15
[INFO] [2021-05-28 09:44:32] Storing 207466 MetaTraits
[INFO] [2021-05-28 09:44:32] Processing group of 207466 in 208 groups of 1000
[INFO] [2021-05-28 09:45:11] Average Time: 0.181
[INFO] [2021-05-28 09:45:11] Total Time: 39s
[INFO] [2021-05-28 09:45:11] last 3 / first 3: 0.72
[INFO] [2021-05-28 09:45:11] Std.Dev: 0.13784048752090222; Max: 1.35
[STOP] [2021-05-28 09:45:11] parse_diff_and_store
[START] [2021-05-28 09:45:11] resolve_keys
[INFO] [2021-05-28 09:45:22] Occurrences to nodes (through scientific_names)...
[INFO] [2021-05-28 09:45:25] traits to occurrences...
[INFO] [2021-05-28 09:45:28] traits to nodes (through occurrences)...
[INFO] [2021-05-28 09:45:31] Traits to sex term...
[INFO] [2021-05-28 09:45:32] Traits to lifestage term...
[INFO] [2021-05-28 09:45:33] MetaTraits to traits...
[INFO] [2021-05-28 09:45:38] MetaTraits (simple, measurement row refers to parent) to traits...
[INFO] [2021-05-28 09:45:39] Assocs to occurrences...
[INFO] [2021-05-28 09:45:39] Assocs to nodes...
[INFO] [2021-05-28 09:45:39] Assoc to sex term...
[INFO] [2021-05-28 09:45:39] Assoc to lifestage term...
[INFO] [2021-05-28 09:45:39] MetaAssoc to assocs...
[STOP] [2021-05-28 09:45:39] resolve_keys
[START] [2021-05-28 09:45:39] hold_for_later_1
[STOP] [2021-05-28 09:45:39] hold_for_later_1
[START] [2021-05-28 09:45:39] hold_for_later_2
[STOP] [2021-05-28 09:45:39] hold_for_later_2
[START] [2021-05-28 09:45:39] resolve_missing_parents
[STOP] [2021-05-28 09:45:39] resolve_missing_parents
[START] [2021-05-28 09:45:39] rebuild_nodes
[START] [2021-05-28 09:45:39] Flattener#flatten
[START] [2021-05-28 09:45:39] Flattener#study_resource
[START] [2021-05-28 09:45:39] Flattener#build_ancestry
[STOP] [2021-05-28 09:45:39] Flattener#build_ancestry
[INFO] [2021-05-28 09:45:39] 6831 ancestry keys
[START] [2021-05-28 09:45:39] build_node_ancestors
[INFO] [2021-05-28 09:45:39] old ancestors deleted.
[STOP] [2021-05-28 09:45:41] build_node_ancestors
[START] [2021-05-28 09:45:43] Flattener#propagate_ancestor_ids
[STOP] [2021-05-28 09:45:44] Flattener#propagate_ancestor_ids
[STOP] [2021-05-28 09:45:44] Flattener#flatten
[STOP] [2021-05-28 09:45:44] rebuild_nodes
[START] [2021-05-28 09:45:44] resolve_missing_media_owners
[STOP] [2021-05-28 09:45:44] resolve_missing_media_owners
[START] [2021-05-28 09:45:44] sanitize_media_verbatims
[STOP] [2021-05-28 09:45:44] sanitize_media_verbatims
[START] [2021-05-28 09:45:44] queue_downloads
[STOP] [2021-05-28 09:45:44] queue_downloads
[START] [2021-05-28 09:45:44] parse_names
[WARN] [2021-05-28 09:45:44] I see 6831 names which still need to be parsed.
[STOP] [2021-05-28 09:45:49] parse_names
[START] [2021-05-28 09:45:49] denormalize_canonical_names_to_nodes
[STOP] [2021-05-28 09:45:49] denormalize_canonical_names_to_nodes
[START] [2021-05-28 09:45:49] match_nodes
[START] [2021-05-28 09:45:49] map_all_nodes_to_pages
[STOP] [2021-05-28 09:51:33] map_all_nodes_to_pages
[INFO] [2021-05-28 09:51:33] Unmatched nodes (10 of 6831): Canonical: Myocastoridae; Node#95004770; ResourceID: Myocastoridae; Canonical: Artibeus aztecus; Node#95001323; ResourceID: Artibeus aztecus; Canonical: Paracoelops; Node#95005564; ResourceID: Paracoelops; Canonical: Cercopithecus preussi; Node#95001872; ResourceID: Cercopithecus preussi; Canonical: Oreonax; Node#95005381; ResourceID: Oreonax; Canonical: Pseudopotto; Node#95006236; ResourceID: Pseudopotto; Canonical: Pseudopotto martini; Node#95006237; ResourceID: Pseudopotto martini; Canonical: Bunopithecus; Node#95001575; ResourceID: Bunopithecus; Canonical: Callithrix pygmaea; Node#95001656; ResourceID: Callithrix pygmaea; Canonical: Erinaceomorpha; Node#95001357; ResourceID: Erinaceomorpha
[START] [2021-05-28 09:51:33] update_nodes
[STOP] [2021-05-28 09:51:36] update_nodes
[STOP] [2021-05-28 09:51:36] match_nodes
[START] [2021-05-28 09:51:36] reindex_search
[STOP] [2021-05-28 09:51:42] reindex_search
[START] [2021-05-28 09:51:42] normalize_units
[STOP] [2021-05-28 09:56:53] normalize_units
[START] [2021-05-28 09:56:53] calculate_statistics
[2021-05-28 09:56:54] (NEAR) DUPLICATE TRAITS FOUND! There are only 103662 (of 103733 total) unique traits.
[2021-05-28 12:33:39] (Near) duplicate trait pairs (up to 100):
[2021-05-28 12:33:39] (resource_pk: HomeRange184, id: 213838532), (resource_pk: HomeRangeIndiv159, id: 213839211)
[2021-05-28 12:33:39] (resource_pk: HomeRange234, id: 213838588), (resource_pk: HomeRangeIndiv200, id: 213839258)
[2021-05-28 12:33:39] (resource_pk: HomeRange321, id: 213838685), (resource_pk: HomeRangeIndiv268, id: 213839332)
[2021-05-28 12:33:39] (resource_pk: HomeRange322, id: 213838686), (resource_pk: HomeRangeIndiv273, id: 213839338)
[2021-05-28 12:33:39] (resource_pk: HomeRange324, id: 213838688), (resource_pk: HomeRangeIndiv274, id: 213839339)
[2021-05-28 12:33:39] (resource_pk: HomeRange332, id: 213838697), (resource_pk: HomeRangeIndiv277, id: 213839342)
[2021-05-28 12:33:39] (resource_pk: HomeRange334, id: 213838699), (resource_pk: HomeRangeIndiv279, id: 213839344)
[2021-05-28 12:33:39] (resource_pk: HomeRange337, id: 213838702), (resource_pk: HomeRangeIndiv281, id: 213839347)
[2021-05-28 12:33:39] (resource_pk: HomeRange344, id: 213838710), (resource_pk: HomeRangeIndiv289, id: 213839355)
[2021-05-28 12:33:39] (resource_pk: HomeRange347, id: 213838713), (resource_pk: HomeRangeIndiv290, id: 213839357)
[2021-05-28 12:33:39] (resource_pk: HomeRange348, id: 213838714), (resource_pk: HomeRangeIndiv292, id: 213839359)
[2021-05-28 12:33:39] (resource_pk: HomeRange362, id: 213838730), (resource_pk: HomeRangeIndiv299, id: 213839366)
[2021-05-28 12:33:39] (resource_pk: HomeRange364, id: 213838732), (resource_pk: HomeRangeIndiv300, id: 213839369)
[2021-05-28 12:33:39] (resource_pk: HomeRange368, id: 213838736), (resource_pk: HomeRangeIndiv304, id: 213839373)
[2021-05-28 12:33:39] (resource_pk: HomeRange371, id: 213838740), (resource_pk: HomeRangeIndiv305, id: 213839374)
[2021-05-28 12:33:39] (resource_pk: HomeRange374, id: 213838743), (resource_pk: HomeRangeIndiv311, id: 213839381)
[2021-05-28 12:33:39] (resource_pk: HomeRange377, id: 213838746), (resource_pk: HomeRangeIndiv315, id: 213839385)
[2021-05-28 12:33:39] (resource_pk: HomeRange380, id: 213838750), (resource_pk: HomeRangeIndiv319, id: 213839389)
[2021-05-28 12:33:39] (resource_pk: HomeRange382, id: 213838752), (resource_pk: HomeRangeIndiv323, id: 213839394)
[2021-05-28 12:33:39] (resource_pk: HomeRange384, id: 213838754), (resource_pk: HomeRangeIndiv325, id: 213839396)
[2021-05-28 12:33:39] (resource_pk: HomeRange386, id: 213838756), (resource_pk: HomeRangeIndiv327, id: 213839398)
[2021-05-28 12:33:39] (resource_pk: HomeRange394, id: 213838765), (resource_pk: HomeRangeIndiv330, id: 213839402)
[2021-05-28 12:33:39] (resource_pk: HomeRange395, id: 213838766), (resource_pk: HomeRangeIndiv332, id: 213839404)
[2021-05-28 12:33:39] (resource_pk: HomeRange396, id: 213838767), (resource_pk: HomeRangeIndiv333, id: 213839405)
[2021-05-28 12:33:39] (resource_pk: HomeRange401, id: 213838774), (resource_pk: HomeRangeIndiv334, id: 213839406)
[2021-05-28 12:33:39] (resource_pk: HomeRange408, id: 213838781), (resource_pk: HomeRangeIndiv339, id: 213839411)
[2021-05-28 12:33:39] (resource_pk: HomeRange409, id: 213838782), (resource_pk: HomeRangeIndiv342, id: 213839415)
[2021-05-28 12:33:39] (resource_pk: HomeRange410, id: 213838784), (resource_pk: HomeRangeIndiv343, id: 213839416)
[2021-05-28 12:33:39] (resource_pk: HomeRange412, id: 213838786), (resource_pk: HomeRangeIndiv344, id: 213839417)
[2021-05-28 12:33:39] (resource_pk: HomeRange413, id: 213838787), (resource_pk: HomeRangeIndiv345, id: 213839418)
[2021-05-28 12:33:39] (resource_pk: HomeRange414, id: 213838788), (resource_pk: HomeRangeIndiv346, id: 213839419)
[2021-05-28 12:33:39] (resource_pk: HomeRange415, id: 213838789), (resource_pk: HomeRangeIndiv347, id: 213839420)
[2021-05-28 12:33:39] (resource_pk: HomeRange418, id: 213838792), (resource_pk: HomeRangeIndiv348, id: 213839421)
[2021-05-28 12:33:39] (resource_pk: HomeRange419, id: 213838793), (resource_pk: HomeRangeIndiv350, id: 213839424)
[2021-05-28 12:33:39] (resource_pk: HomeRange420, id: 213838795), (resource_pk: HomeRangeIndiv351, id: 213839425)
[2021-05-28 12:33:39] (resource_pk: HomeRange422, id: 213838797), (resource_pk: HomeRangeIndiv354, id: 213839428)
[2021-05-28 12:33:39] (resource_pk: HomeRange423, id: 213838798), (resource_pk: HomeRangeIndiv355, id: 213839429)
[2021-05-28 12:33:39] (resource_pk: HomeRange425, id: 213838800), (resource_pk: HomeRangeIndiv356, id: 213839430)
[2021-05-28 12:33:39] (resource_pk: HomeRange426, id: 213838801), (resource_pk: HomeRangeIndiv357, id: 213839431)
[2021-05-28 12:33:39] (resource_pk: HomeRange427, id: 213838802), (resource_pk: HomeRangeIndiv358, id: 213839432)
[2021-05-28 12:33:39] (resource_pk: HomeRange430, id: 213838806), (resource_pk: HomeRangeIndiv359, id: 213839433)
[2021-05-28 12:33:39] (resource_pk: HomeRange431, id: 213838807), (resource_pk: HomeRangeIndiv360, id: 213839435)
[2021-05-28 12:33:39] (resource_pk: HomeRange432, id: 213838808), (resource_pk: HomeRangeIndiv362, id: 213839437)
[2021-05-28 12:33:39] (resource_pk: HomeRange435, id: 213838811), (resource_pk: HomeRangeIndiv363, id: 213839438)
[2021-05-28 12:33:39] (resource_pk: HomeRange436, id: 213838812), (resource_pk: HomeRangeIndiv364, id: 213839439)
[2021-05-28 12:33:39] (resource_pk: HomeRange437, id: 213838813), (resource_pk: HomeRangeIndiv365, id: 213839440)
[2021-05-28 12:33:39] (resource_pk: HomeRange441, id: 213838818), (resource_pk: HomeRangeIndiv366, id: 213839441)
[2021-05-28 12:33:39] (resource_pk: HomeRange443, id: 213838820), (resource_pk: HomeRangeIndiv367, id: 213839442)
[2021-05-28 12:33:39] (resource_pk: HomeRange446, id: 213838823), (resource_pk: HomeRangeIndiv368, id: 213839443)
[2021-05-28 12:33:39] (resource_pk: HomeRange447, id: 213838824), (resource_pk: HomeRangeIndiv369, id: 213839444)
[2021-05-28 12:33:39] (resource_pk: HomeRange448, id: 213838825), (resource_pk: HomeRangeIndiv370, id: 213839446)
[2021-05-28 12:33:39] (resource_pk: HomeRange449, id: 213838826), (resource_pk: HomeRangeIndiv371, id: 213839447)
[2021-05-28 12:33:39] (resource_pk: HomeRange450, id: 213838828), (resource_pk: HomeRangeIndiv372, id: 213839448)
[2021-05-28 12:33:39] (resource_pk: HomeRange451, id: 213838829), (resource_pk: HomeRangeIndiv374, id: 213839450)
[2021-05-28 12:33:39] (resource_pk: HomeRange452, id: 213838830), (resource_pk: HomeRangeIndiv376, id: 213839452)
[2021-05-28 12:33:39] (resource_pk: HomeRange453, id: 213838831), (resource_pk: HomeRangeIndiv377, id: 213839453)
[2021-05-28 12:33:39] (resource_pk: HomeRange454, id: 213838832), (resource_pk: HomeRangeIndiv378, id: 213839454)
[2021-05-28 12:33:39] (resource_pk: HomeRange455, id: 213838833), (resource_pk: HomeRangeIndiv379, id: 213839455)
[2021-05-28 12:33:39] (resource_pk: HomeRange456, id: 213838834), (resource_pk: HomeRangeIndiv380, id: 213839457)
[2021-05-28 12:33:39] (resource_pk: HomeRange458, id: 213838836), (resource_pk: HomeRangeIndiv381, id: 213839458)
[2021-05-28 12:33:39] (resource_pk: HomeRange459, id: 213838837), (resource_pk: HomeRangeIndiv383, id: 213839460)
[2021-05-28 12:33:39] (resource_pk: HomeRange460, id: 213838839), (resource_pk: HomeRangeIndiv385, id: 213839462)
[2021-05-28 12:33:39] (resource_pk: HomeRange461, id: 213838840), (resource_pk: HomeRangeIndiv387, id: 213839464)
[2021-05-28 12:33:39] (resource_pk: HomeRange462, id: 213838841), (resource_pk: HomeRangeIndiv388, id: 213839465)
[2021-05-28 12:33:39] (resource_pk: HomeRange465, id: 213838844), (resource_pk: HomeRangeIndiv389, id: 213839466)
[2021-05-28 12:33:39] (resource_pk: HomeRange466, id: 213838845), (resource_pk: HomeRangeIndiv390, id: 213839468)
[2021-05-28 12:33:39] (resource_pk: HomeRange467, id: 213838846), (resource_pk: HomeRangeIndiv392, id: 213839470)
[2021-05-28 12:33:39] (resource_pk: HomeRange468, id: 213838847), (resource_pk: HomeRangeIndiv393, id: 213839471)
[2021-05-28 12:33:39] (resource_pk: HomeRange469, id: 213838848), (resource_pk: HomeRangeIndiv394, id: 213839472)
[2021-05-28 12:33:39] (resource_pk: HomeRange470, id: 213838850), (resource_pk: HomeRangeIndiv395, id: 213839473)
[2021-05-28 12:33:39] (resource_pk: HomeRange471, id: 213838851), (resource_pk: HomeRangeIndiv398, id: 213839476)
[STOP] [2021-05-28 12:33:39] calculate_statistics
[START] [2021-05-28 12:33:39] complete_harvest_instance
[START] [2021-05-28 12:33:39] overall_tsv_creation
[INFO] [2021-05-28 12:33:39] Processing group of 6831 in 1 batches of 10000
[INFO] [2021-05-28 12:34:39] 103733 Traits (unfiltered)...
[INFO] [2021-05-28 12:39:37] 103733 Traits (filtered)...
[INFO] [2021-05-28 12:39:40] 0 Associations (filtered)...
[INFO] [2021-05-28 12:39:52] 573 metadata added.
[INFO] [2021-05-28 12:39:52] 0 metadata added.
[INFO] [2021-05-28 12:40:46] Average Time: 394.91
[INFO] [2021-05-28 12:40:46] Total Time: 7m8s
[STOP] [2021-05-28 12:40:46] overall_tsv_creation
[INFO] [2021-05-28 12:40:46] Done. Check your files:
[INFO] [2021-05-28 12:40:46] (6831 lines) /app/public/data/Pantheria/publish_nodes.tsv
[INFO] [2021-05-28 12:40:47] (39348 lines) /app/public/data/Pantheria/publish_node_ancestors.tsv
[INFO] [2021-05-28 12:40:47] (6831 lines) /app/public/data/Pantheria/publish_scientific_names.tsv
[INFO] [2021-05-28 12:40:47] (3087 lines) /app/public/data/Pantheria/publish_references.tsv
[INFO] [2021-05-28 12:40:48] (3087 lines) /app/public/data/Pantheria/publish_referents.tsv
[INFO] [2021-05-28 12:40:48] (103734 lines) /app/public/data/Pantheria/publish_traits.tsv
[INFO] [2021-05-28 12:40:48] (574 lines) /app/public/data/Pantheria/publish_metadata.tsv
[STOP] [2021-05-28 12:40:48] complete_harvest_instance
[START] [2021-05-28 12:40:48] completed
[STOP] [2021-05-28 12:40:48] completed
[STOP] [2021-05-28 12:40:48] logged process, took 10756.35

Latest Process

28 May 09:41 EXITED currently processing 0