Harvest for Shelled animal body mass Created 13 Oct 09:37

Harvesting Log

[INFO] [2023-10-13 09:37:20] Created harvest instance #4423
[STOP] [2023-10-13 09:37:20] create_harvest_instance
[START] [2023-10-13 09:37:20] fetch_files
[STOP] [2023-10-13 09:37:20] fetch_files
[START] [2023-10-13 09:37:20] validate_each_file
[INFO] [2023-10-13 09:37:20] Looping over 4 formats...
[INFO] [2023-10-13 09:37:20] ...refs (/app/public/data/Shelled_animal_b/reference.tab)
[INFO] [2023-10-13 09:37:21] Valid: /app/public/data/Shelled_animal_b/converted_csv/Shelled_animal_b_refs_30650.csv (22153 lines)
[INFO] [2023-10-13 09:37:21] ...nodes (/app/public/data/Shelled_animal_b/taxon.tab)
[INFO] [2023-10-13 09:37:23] Valid: /app/public/data/Shelled_animal_b/converted_csv/Shelled_animal_b_nodes_30649.csv (22173 lines)
[INFO] [2023-10-13 09:37:23] ...occurrences (/app/public/data/Shelled_animal_b/occurrence.tab)
[INFO] [2023-10-13 09:37:25] Valid: /app/public/data/Shelled_animal_b/converted_csv/Shelled_animal_b_occurrences_30651.csv (22170 lines)
[INFO] [2023-10-13 09:37:25] ...measurements (/app/public/data/Shelled_animal_b/measurement_or_fact.tab)
[INFO] [2023-10-13 09:37:32] Valid: /app/public/data/Shelled_animal_b/converted_csv/Shelled_animal_b_measurements_30652.csv (66510 lines)
[STOP] [2023-10-13 09:37:32] validate_each_file
[START] [2023-10-13 09:37:32] convert_to_csv
[INFO] [2023-10-13 09:37:32] Looping over 4 formats...
[INFO] [2023-10-13 09:37:32] ...refs (/app/public/data/Shelled_animal_b/reference.tab)
[CMD] [2023-10-13 09:37:32] /usr/bin/sort /app/public/data/Shelled_animal_b/converted_csv/Shelled_animal_b_refs_30650.csv > /app/public/data/Shelled_animal_b/converted_csv/Shelled_animal_b_refs_30650.csv_sorted
[INFO] [2023-10-13 09:37:32] Converted: /app/public/data/Shelled_animal_b/converted_csv/Shelled_animal_b_refs_30650.csv (22153 lines)
[INFO] [2023-10-13 09:37:32] ...nodes (/app/public/data/Shelled_animal_b/taxon.tab)
[CMD] [2023-10-13 09:37:32] /usr/bin/sort /app/public/data/Shelled_animal_b/converted_csv/Shelled_animal_b_nodes_30649.csv > /app/public/data/Shelled_animal_b/converted_csv/Shelled_animal_b_nodes_30649.csv_sorted
[INFO] [2023-10-13 09:37:32] Converted: /app/public/data/Shelled_animal_b/converted_csv/Shelled_animal_b_nodes_30649.csv (22173 lines)
[INFO] [2023-10-13 09:37:32] ...occurrences (/app/public/data/Shelled_animal_b/occurrence.tab)
[CMD] [2023-10-13 09:37:32] /usr/bin/sort /app/public/data/Shelled_animal_b/converted_csv/Shelled_animal_b_occurrences_30651.csv > /app/public/data/Shelled_animal_b/converted_csv/Shelled_animal_b_occurrences_30651.csv_sorted
[INFO] [2023-10-13 09:37:32] Converted: /app/public/data/Shelled_animal_b/converted_csv/Shelled_animal_b_occurrences_30651.csv (22170 lines)
[INFO] [2023-10-13 09:37:32] ...measurements (/app/public/data/Shelled_animal_b/measurement_or_fact.tab)
[CMD] [2023-10-13 09:37:32] /usr/bin/sort /app/public/data/Shelled_animal_b/converted_csv/Shelled_animal_b_measurements_30652.csv > /app/public/data/Shelled_animal_b/converted_csv/Shelled_animal_b_measurements_30652.csv_sorted
[INFO] [2023-10-13 09:37:33] Converted: /app/public/data/Shelled_animal_b/converted_csv/Shelled_animal_b_measurements_30652.csv (66510 lines)
[STOP] [2023-10-13 09:37:33] convert_to_csv
[START] [2023-10-13 09:37:33] calculate_delta
[INFO] [2023-10-13 09:37:33] Looping over 4 formats...
[INFO] [2023-10-13 09:37:33] ...refs (/app/public/data/Shelled_animal_b/reference.tab)
[CMD] [2023-10-13 09:37:33] echo "0a" > /app/public/data/Shelled_animal_b/diff/Shelled_animal_b_refs_30650.diff
[CMD] [2023-10-13 09:37:33] tail -n +1 /app/public/data/Shelled_animal_b/converted_csv/Shelled_animal_b_refs_30650.csv >> /app/public/data/Shelled_animal_b/diff/Shelled_animal_b_refs_30650.diff
[CMD] [2023-10-13 09:37:33] echo "." >> /app/public/data/Shelled_animal_b/diff/Shelled_animal_b_refs_30650.diff
[INFO] [2023-10-13 09:37:33] Created diff: /app/public/data/Shelled_animal_b/diff/Shelled_animal_b_refs_30650.diff (22155 lines)
[INFO] [2023-10-13 09:37:33] ...nodes (/app/public/data/Shelled_animal_b/taxon.tab)
[CMD] [2023-10-13 09:37:33] echo "0a" > /app/public/data/Shelled_animal_b/diff/Shelled_animal_b_nodes_30649.diff
[CMD] [2023-10-13 09:37:33] tail -n +1 /app/public/data/Shelled_animal_b/converted_csv/Shelled_animal_b_nodes_30649.csv >> /app/public/data/Shelled_animal_b/diff/Shelled_animal_b_nodes_30649.diff
[CMD] [2023-10-13 09:37:33] echo "." >> /app/public/data/Shelled_animal_b/diff/Shelled_animal_b_nodes_30649.diff
[INFO] [2023-10-13 09:37:33] Created diff: /app/public/data/Shelled_animal_b/diff/Shelled_animal_b_nodes_30649.diff (22175 lines)
[INFO] [2023-10-13 09:37:33] ...occurrences (/app/public/data/Shelled_animal_b/occurrence.tab)
[CMD] [2023-10-13 09:37:33] echo "0a" > /app/public/data/Shelled_animal_b/diff/Shelled_animal_b_occurrences_30651.diff
[CMD] [2023-10-13 09:37:33] tail -n +1 /app/public/data/Shelled_animal_b/converted_csv/Shelled_animal_b_occurrences_30651.csv >> /app/public/data/Shelled_animal_b/diff/Shelled_animal_b_occurrences_30651.diff
[CMD] [2023-10-13 09:37:34] echo "." >> /app/public/data/Shelled_animal_b/diff/Shelled_animal_b_occurrences_30651.diff
[INFO] [2023-10-13 09:37:34] Created diff: /app/public/data/Shelled_animal_b/diff/Shelled_animal_b_occurrences_30651.diff (22172 lines)
[INFO] [2023-10-13 09:37:34] ...measurements (/app/public/data/Shelled_animal_b/measurement_or_fact.tab)
[CMD] [2023-10-13 09:37:34] echo "0a" > /app/public/data/Shelled_animal_b/diff/Shelled_animal_b_measurements_30652.diff
[CMD] [2023-10-13 09:37:34] tail -n +1 /app/public/data/Shelled_animal_b/converted_csv/Shelled_animal_b_measurements_30652.csv >> /app/public/data/Shelled_animal_b/diff/Shelled_animal_b_measurements_30652.diff
[CMD] [2023-10-13 09:37:34] echo "." >> /app/public/data/Shelled_animal_b/diff/Shelled_animal_b_measurements_30652.diff
[INFO] [2023-10-13 09:37:34] Created diff: /app/public/data/Shelled_animal_b/diff/Shelled_animal_b_measurements_30652.diff (66512 lines)
[STOP] [2023-10-13 09:37:34] calculate_delta
[START] [2023-10-13 09:37:34] parse_diff_and_store
[INFO] [2023-10-13 09:37:34] Handling diff: /app/public/data/Shelled_animal_b/diff/Shelled_animal_b_refs_30650.diff (22155 lines)
[INFO] [2023-10-13 09:37:35] Loading refs diff file into memory (22155 lines)...
[INFO] [2023-10-13 09:37:36] Storing 9999 References (9999/10000/22155)
[INFO] [2023-10-13 09:37:39] Storing 10000 References (19999/20000/22155)
[INFO] [2023-10-13 09:37:42] Storing 2154 References (22153/22153/22155)
[INFO] [2023-10-13 09:37:42] Handling diff: /app/public/data/Shelled_animal_b/diff/Shelled_animal_b_nodes_30649.diff (22175 lines)
[INFO] [2023-10-13 09:37:43] Loading nodes diff file into memory (22175 lines)...
[WARN] [2023-10-13 09:37:43] Filtered Scientific Name `Vespericola megasoma ("Dall" Pilsbry, 1928)` to `Vespericola megasoma (Dall Pilsbry, 1928)`
[WARN] [2023-10-13 09:37:43] Filtered Scientific Name `Helicostyla gilva (Sowerby "Pfeiffer", 1845)` to `Helicostyla gilva (Sowerby Pfeiffer, 1845)`
[WARN] [2023-10-13 09:37:43] Filtered Scientific Name `Mutela bourguignati "Ancey" Bourguinat, 1885` to `Mutela bourguignati Ancey Bourguinat, 1885`
[WARN] [2023-10-13 09:37:43] Filtered Scientific Name `Hexaplex  kuesterianus (Tapparone-Canepi, 1875)` to `Hexaplex kuesterianus (Tapparone-Canepi, 1875)`
[WARN] [2023-10-13 09:37:44] Filtered Scientific Name `Vermetus alli  Hadfield & Kay in Hadfield, Kay, Gillette & Lloyd, 1972` to `Vermetus alli Hadfield & Kay in Hadfield, Kay, Gillette & Lloyd, 1972`
[WARN] [2023-10-13 09:37:45] Filtered Scientific Name `Capulus  galeus Dall, 1889` to `Capulus galeus Dall, 1889`
[WARN] [2023-10-13 09:37:45] Filtered Scientific Name `Cypraea carneola propinqua,  sulcidentata and hybrid` to `Cypraea carneola propinqua, sulcidentata and hybrid`
[WARN] [2023-10-13 09:37:46] Filtered Scientific Name `Conus amadis aurantia "Lamarck" Dautzenberg, 1937` to `Conus amadis aurantia Lamarck Dautzenberg, 1937`
[INFO] [2023-10-13 09:37:46] Storing 10305 ScientificNames (20610/10000/22175)
[INFO] [2023-10-13 09:37:50] Storing 10305 Nodes (20610/10000/22175)
[WARN] [2023-10-13 09:37:53] Filtered Scientific Name `Cypraea nebrites "galapaginensis"` to `Cypraea nebrites galapaginensis`
[WARN] [2023-10-13 09:37:54] Filtered Scientific Name `Amphidromus sinistralis lutea "Martens" Fulton, 1896` to `Amphidromus sinistralis lutea Martens Fulton, 1896`
[WARN] [2023-10-13 09:37:54] Filtered Scientific Name `Neptunea polycostata aino Fraussen &  Terryn, 2007` to `Neptunea polycostata aino Fraussen & Terryn, 2007`
[WARN] [2023-10-13 09:37:54] Filtered Scientific Name `Scaevatula pellisserpentis Gofas,\n1990` to `Scaevatula pellisserpentis Gofas,n1990`
[WARN] [2023-10-13 09:37:56] Filtered Scientific Name `Cypraea pantherina albonitens \nMelvill, 1888` to `Cypraea pantherina albonitens nMelvill, 1888`
[WARN] [2023-10-13 09:37:56] Filtered Scientific Name `Scaeochlamys squamea  Dijkstra & Maestrati, 2009` to `Scaeochlamys squamea Dijkstra & Maestrati, 2009`
[WARN] [2023-10-13 09:37:56] SKIPPED 277 Scientific names (41242/20000/22175) with resource_pks already be in the database!
[WARN] [2023-10-13 09:37:56] SKIPPED 277 Nodes (41242/20000/22175) with resource_pks already be in the database!
[INFO] [2023-10-13 09:37:56] Storing 10039 ScientificNames (41242/20000/22175)
[INFO] [2023-10-13 09:37:59] Storing 10039 Nodes (41242/20000/22175)
[WARN] [2023-10-13 09:38:03] Filtered Scientific Name `Porphyrobaphe grevillei ("Sowerby" Pfeiffer, 1876)` to `Porphyrobaphe grevillei (Sowerby Pfeiffer, 1876)`
[WARN] [2023-10-13 09:38:03] SKIPPED 223 Scientific names (46044/22173/22175) with resource_pks already be in the database!
[WARN] [2023-10-13 09:38:04] SKIPPED 223 Nodes (46044/22173/22175) with resource_pks already be in the database!
[INFO] [2023-10-13 09:38:04] Storing 2178 ScientificNames (46044/22173/22175)
[INFO] [2023-10-13 09:38:04] Storing 2178 Nodes (46044/22173/22175)
[INFO] [2023-10-13 09:38:05] Handling diff: /app/public/data/Shelled_animal_b/diff/Shelled_animal_b_occurrences_30651.diff (22172 lines)
[INFO] [2023-10-13 09:38:05] Loading occurrences diff file into memory (22172 lines)...
[INFO] [2023-10-13 09:38:08] Storing 9999 Occurrences (19998/10000/22172)
[INFO] [2023-10-13 09:38:10] Storing 9999 OccurrenceMetadata (19998/10000/22172)
[WARN] [2023-10-13 09:38:14] SKIPPED 10000 Occurrence metadata (39998/20000/22172) with resource_pks already be in the database!
[INFO] [2023-10-13 09:38:14] Storing 10000 Occurrences (39998/20000/22172)
[INFO] [2023-10-13 09:38:16] Storing 0 OccurrenceMetadata (39998/20000/22172)
[WARN] [2023-10-13 09:38:16] No models to import, skipping!
[WARN] [2023-10-13 09:38:17] SKIPPED 2171 Occurrence metadata (44340/22170/22172) with resource_pks already be in the database!
[INFO] [2023-10-13 09:38:17] Storing 2171 Occurrences (44340/22170/22172)
[INFO] [2023-10-13 09:38:17] Storing 0 OccurrenceMetadata (44340/22170/22172)
[WARN] [2023-10-13 09:38:17] No models to import, skipping!
[INFO] [2023-10-13 09:38:17] Handling diff: /app/public/data/Shelled_animal_b/diff/Shelled_animal_b_measurements_30652.diff (66512 lines)
[INFO] [2023-10-13 09:38:17] Loading measurements diff file into memory (66512 lines)...
[INFO] [2023-10-13 09:38:22] Storing 9999 TraitsReferences (29997/10000/66512)
[INFO] [2023-10-13 09:38:23] Storing 9999 Traits (29997/10000/66512)
[INFO] [2023-10-13 09:38:26] Storing 9999 MetaTraits (29997/10000/66512)
[INFO] [2023-10-13 09:38:33] Storing 15828 TraitsReferences (71653/20000/66512)
[INFO] [2023-10-13 09:38:34] Storing 10000 Traits (71653/20000/66512)
[INFO] [2023-10-13 09:38:38] Storing 15828 MetaTraits (71653/20000/66512)
[INFO] [2023-10-13 09:38:46] Storing 19000 TraitsReferences (119653/30000/66512)
[INFO] [2023-10-13 09:38:48] Storing 10000 Traits (119653/30000/66512)
[INFO] [2023-10-13 09:38:51] Storing 19000 MetaTraits (119653/30000/66512)
[INFO] [2023-10-13 09:39:00] Storing 27290 TraitsReferences (175943/40000/66512)
[INFO] [2023-10-13 09:39:02] Storing 10000 Traits (175943/40000/66512)
[INFO] [2023-10-13 09:39:05] Storing 19000 MetaTraits (175943/40000/66512)
[INFO] [2023-10-13 09:39:16] Storing 64000 TraitsReferences (268943/50000/66512)
[INFO] [2023-10-13 09:39:22] Storing 10000 Traits (268943/50000/66512)
[INFO] [2023-10-13 09:39:25] Storing 19000 MetaTraits (268943/50000/66512)
[INFO] [2023-10-13 09:39:38] Storing 64000 TraitsReferences (361943/60000/66512)
[INFO] [2023-10-13 09:39:43] Storing 10000 Traits (361943/60000/66512)
[INFO] [2023-10-13 09:39:46] Storing 19000 MetaTraits (361943/60000/66512)
[INFO] [2023-10-13 09:39:53] Storing 21583 TraitsReferences (399060/66510/66512)
[INFO] [2023-10-13 09:39:55] Storing 6511 Traits (399060/66510/66512)
[INFO] [2023-10-13 09:39:57] Storing 9023 MetaTraits (399060/66510/66512)
[STOP] [2023-10-13 09:39:58] parse_diff_and_store
[START] [2023-10-13 09:39:58] resolve_keys
[2023-10-13 09:40:02] Resolving downloaded urls (this is not actually downloading them yet)
[INFO] [2023-10-13 09:40:09] Occurrences to nodes (through scientific_names)...
[INFO] [2023-10-13 09:40:11] traits to occurrences...
[INFO] [2023-10-13 09:40:15] traits to nodes (through occurrences)...
[INFO] [2023-10-13 09:40:17] Traits to sex term...
[INFO] [2023-10-13 09:40:18] Traits to lifestage term...
[INFO] [2023-10-13 09:40:19] MetaTraits to traits...
[INFO] [2023-10-13 09:40:22] MetaTraits (simple, measurement row refers to parent) to traits...
[INFO] [2023-10-13 09:40:33] Assocs to occurrences...
[INFO] [2023-10-13 09:40:33] Assocs to nodes...
[INFO] [2023-10-13 09:40:33] Assoc to sex term...
[INFO] [2023-10-13 09:40:33] Assoc to lifestage term...
[INFO] [2023-10-13 09:40:33] MetaAssoc to assocs...
[STOP] [2023-10-13 09:40:33] resolve_keys
[START] [2023-10-13 09:40:33] hold_for_later_1
[STOP] [2023-10-13 09:40:33] hold_for_later_1
[START] [2023-10-13 09:40:33] hold_for_later_2
[STOP] [2023-10-13 09:40:33] hold_for_later_2
[START] [2023-10-13 09:40:33] resolve_missing_parents
[STOP] [2023-10-13 09:40:33] resolve_missing_parents
[START] [2023-10-13 09:40:33] rebuild_nodes
[START] [2023-10-13 09:40:33] Flattener#flatten
[START] [2023-10-13 09:40:33] Flattener#study_resource
[START] [2023-10-13 09:40:33] Flattener#build_ancestry
[STOP] [2023-10-13 09:40:35] Flattener#build_ancestry
[INFO] [2023-10-13 09:40:35] 22522 ancestry keys
[START] [2023-10-13 09:40:35] build_node_ancestors
[INFO] [2023-10-13 09:40:35] old ancestors deleted.
[STOP] [2023-10-13 09:40:37] build_node_ancestors
[START] [2023-10-13 09:40:39] Flattener#propagate_ancestor_ids
[STOP] [2023-10-13 09:40:40] Flattener#propagate_ancestor_ids
[STOP] [2023-10-13 09:40:40] Flattener#flatten
[STOP] [2023-10-13 09:40:40] rebuild_nodes
[START] [2023-10-13 09:40:40] resolve_missing_media_owners
[STOP] [2023-10-13 09:40:40] resolve_missing_media_owners
[START] [2023-10-13 09:40:40] sanitize_media_verbatims
[STOP] [2023-10-13 09:40:40] sanitize_media_verbatims
[START] [2023-10-13 09:40:40] queue_downloads
[STOP] [2023-10-13 09:40:40] queue_downloads
[START] [2023-10-13 09:40:40] parse_names
[WARN] [2023-10-13 09:40:40] I see 22522 names which still need to be parsed.
[WARN] [2023-10-13 09:40:42] Names to parse: 10000 formatted: 10000 learned: 9735 parsed: 10000
[WARN] [2023-10-13 09:40:50] Names to parse: 10000 formatted: 10000 learned: 9702 parsed: 10000
[WARN] [2023-10-13 09:40:56] Names to parse: 2522 formatted: 2522 learned: 2502 parsed: 2522
[STOP] [2023-10-13 09:40:59] parse_names
[START] [2023-10-13 09:40:59] denormalize_canonical_names_to_nodes
[STOP] [2023-10-13 09:40:59] denormalize_canonical_names_to_nodes
[START] [2023-10-13 09:40:59] match_nodes
[START] [2023-10-13 09:40:59] map_all_nodes_to_pages
[STOP] [2023-10-13 09:57:18] map_all_nodes_to_pages
[INFO] [2023-10-13 09:57:18] 4755 Unmatched nodes (of 22522)! That's too many to output. Full list in /app/public/data/Shelled_animal_b/unmatched_nodes.txt ; First 10: Canonical: Helicostyla gilva; Node#136994747; ResourceID: 011ce2d5265f4280f7c900920f41034b; Canonical: Aegista tenuissima; Node#136994838; ResourceID: 01c5ecb58259d9484dbf3a8957b145a8; Canonical: Aegista awajiensis; Node#136994978; ResourceID: 030ce6a17d6f8f152c1458cc0f82c869; Canonical: Phoenicobius campanulus; Node#136995037; ResourceID: 03aa705f4b6bd0fda9221034c35f8fad; Canonical: Plectotropis lancasteri; Node#136995984; ResourceID: 0dde97c3ba961a160a3ba32bf5ea9bab; Canonical: Helicostyla nobilis; Node#136996342; ResourceID: 11aae65a212852623cb6b8a5d8c04f61; Canonical: Bradybaena tourannensis rhodostoma; Node#136996706; ResourceID: 15ae0223055ca1b338eb71384dcd418b; Canonical: Helicostyla phaeostyla; Node#136996818; ResourceID: 16b132fe635df918de9e2da7789d1c39; Canonical: Euhadra cecillei; Node#136997659; ResourceID: 207cf31d1ac1c5957e2745b08e404c58; Canonical: Aegista proba goniosoma; Node#136998445; ResourceID: 292fc24f9ef653d1a740a9fdd3022bc6
[START] [2023-10-13 09:57:18] update_nodes
[STOP] [2023-10-13 09:57:29] update_nodes
[STOP] [2023-10-13 09:57:29] match_nodes
[START] [2023-10-13 09:57:29] reindex_search
[STOP] [2023-10-13 09:57:46] reindex_search
[START] [2023-10-13 09:57:46] normalize_units
[STOP] [2023-10-13 10:01:38] normalize_units
[START] [2023-10-13 10:01:38] calculate_statistics
[INFO] [2023-10-13 10:01:46] Duplicate page_id count: 0
[2023-10-13 10:01:46] WARNING: 35 trait(s) without source found! Please confirm that this is intentional.
[2023-10-13 10:01:47] traits w/o source (up to 100):
[2023-10-13 10:01:47] (resource_pk: 12945, id: 291104185)
[2023-10-13 10:01:47] (resource_pk: 19297, id: 291111242)
[2023-10-13 10:01:47] (resource_pk: 25957, id: 291118642)
[2023-10-13 10:01:47] (resource_pk: 25957, id: 291118642)
[2023-10-13 10:01:47] (resource_pk: 29218, id: 291122266)
[2023-10-13 10:01:47] (resource_pk: 29218, id: 291122266)
[2023-10-13 10:01:47] (resource_pk: 29671, id: 291122769)
[2023-10-13 10:01:47] (resource_pk: 29671, id: 291122769)
[2023-10-13 10:01:47] (resource_pk: 30062, id: 291123205)
[2023-10-13 10:01:47] (resource_pk: 30062, id: 291123205)
[2023-10-13 10:01:47] (resource_pk: 30824, id: 291124051)
[2023-10-13 10:01:47] (resource_pk: 30824, id: 291124051)
[2023-10-13 10:01:47] (resource_pk: 35115, id: 291128819)
[2023-10-13 10:01:47] (resource_pk: 35115, id: 291128819)
[2023-10-13 10:01:47] (resource_pk: 3787, id: 291131878)
[2023-10-13 10:01:47] (resource_pk: 41467, id: 291135876)
[2023-10-13 10:01:47] (resource_pk: 41467, id: 291135876)
[2023-10-13 10:01:47] (resource_pk: 48127, id: 291143276)
[2023-10-13 10:01:47] (resource_pk: 48127, id: 291143276)
[2023-10-13 10:01:47] (resource_pk: 51388, id: 291146899)
[2023-10-13 10:01:47] (resource_pk: 51388, id: 291146899)
[2023-10-13 10:01:47] (resource_pk: 51841, id: 291147403)
[2023-10-13 10:01:47] (resource_pk: 51841, id: 291147403)
[2023-10-13 10:01:47] (resource_pk: 52232, id: 291147838)
[2023-10-13 10:01:47] (resource_pk: 52232, id: 291147838)
[2023-10-13 10:01:47] (resource_pk: 52994, id: 291148683)
[2023-10-13 10:01:47] (resource_pk: 52994, id: 291148683)
[2023-10-13 10:01:47] (resource_pk: 57285, id: 291153451)
[2023-10-13 10:01:47] (resource_pk: 57285, id: 291153451)
[2023-10-13 10:01:47] (resource_pk: 63637, id: 291160509)
[2023-10-13 10:01:47] (resource_pk: 63637, id: 291160509)
[2023-10-13 10:01:47] (resource_pk: 7048, id: 291164143)
[2023-10-13 10:01:47] (resource_pk: 7501, id: 291164647)
[2023-10-13 10:01:47] (resource_pk: 7892, id: 291165080)
[2023-10-13 10:01:47] (resource_pk: 8654, id: 291165927)
[STOP] [2023-10-13 10:01:47] calculate_statistics
[START] [2023-10-13 10:01:47] complete_harvest_instance
[START] [2023-10-13 10:01:47] overall_tsv_creation
[INFO] [2023-10-13 10:01:47] Exporting 22522 nodes as TSV in batches of 10000...
[INFO] [2023-10-13 10:01:47] Processing group of 22522 in 3 batches of 10000
[INFO] [2023-10-13 10:02:04] 29091 Traits (unfiltered) and 0 associations...
[INFO] [2023-10-13 10:02:04] Building Traits map for 10000 nodes (this can take a while)...
[INFO] [2023-10-13 10:02:41] Mapped 29091 traits (48485 meta) for 10000 nodes.
[INFO] [2023-10-13 10:02:41] Building Associations map (this can take a while)...
[INFO] [2023-10-13 10:02:44] Done. 0 assocs mapped (0 meta).
[INFO] [2023-10-13 10:02:44] Adding 29091 traits...
[INFO] [2023-10-13 10:02:54] 116344 metadata added.
[INFO] [2023-10-13 10:02:54] Adding 0 assocs...
[INFO] [2023-10-13 10:02:54] 0 metadata added.
[INFO] [2023-10-13 10:03:46] Processed 10000/22522 nodes
[INFO] [2023-10-13 10:04:06] 29868 Traits (unfiltered) and 0 associations...
[INFO] [2023-10-13 10:04:06] Building Traits map for 10000 nodes (this can take a while)...
[INFO] [2023-10-13 10:04:44] Mapped 29868 traits (49780 meta) for 10000 nodes.
[INFO] [2023-10-13 10:04:44] Building Associations map (this can take a while)...
[INFO] [2023-10-13 10:04:47] Done. 0 assocs mapped (0 meta).
[INFO] [2023-10-13 10:04:47] Adding 29868 traits...
[INFO] [2023-10-13 10:05:00] 119462 metadata added.
[INFO] [2023-10-13 10:05:00] Adding 0 assocs...
[INFO] [2023-10-13 10:05:00] 0 metadata added.
[INFO] [2023-10-13 10:05:50] Processed 20000/22522 nodes
[INFO] [2023-10-13 10:05:58] 7551 Traits (unfiltered) and 0 associations...
[INFO] [2023-10-13 10:05:58] Building Traits map for 2522 nodes (this can take a while)...
[INFO] [2023-10-13 10:06:03] Mapped 7551 traits (12585 meta) for 2522 nodes.
[INFO] [2023-10-13 10:06:03] Building Associations map (this can take a while)...
[INFO] [2023-10-13 10:06:03] Done. 0 assocs mapped (0 meta).
[INFO] [2023-10-13 10:06:03] Adding 7551 traits...
[INFO] [2023-10-13 10:06:06] 30199 metadata added.
[INFO] [2023-10-13 10:06:06] Adding 0 assocs...
[INFO] [2023-10-13 10:06:06] 0 metadata added.
[INFO] [2023-10-13 10:06:53] Processed 22522/22522 nodes
[INFO] [2023-10-13 10:06:53] Average Time: 99.083
[INFO] [2023-10-13 10:06:53] Total Time: 5m7s
[STOP] [2023-10-13 10:06:53] overall_tsv_creation
[INFO] [2023-10-13 10:06:53] Done. Check your files:
[INFO] [2023-10-13 10:06:53] (22522 lines) /app/public/data/Shelled_animal_b/publish_nodes.tsv
[INFO] [2023-10-13 10:06:53] (44694 lines) /app/public/data/Shelled_animal_b/publish_node_ancestors.tsv
[INFO] [2023-10-13 10:06:53] (22522 lines) /app/public/data/Shelled_animal_b/publish_scientific_names.tsv
[INFO] [2023-10-13 10:06:53] (66511 lines) /app/public/data/Shelled_animal_b/publish_traits.tsv
[INFO] [2023-10-13 10:06:53] (266006 lines) /app/public/data/Shelled_animal_b/publish_metadata.tsv
[STOP] [2023-10-13 10:06:54] complete_harvest_instance
[START] [2023-10-13 10:06:54] completed
[STOP] [2023-10-13 10:06:54] completed
[STOP] [2023-10-13 10:06:54] logged process, took 1774.14

Latest Process

13 Oct 09:37 EXITED currently processing 0