Harvest for Plant Forms, Habitat and Distribution Created 13 Oct 10:14

Harvesting Log

[INFO] [2023-10-13 10:14:07] Created harvest instance #4426
[STOP] [2023-10-13 10:14:07] create_harvest_instance
[START] [2023-10-13 10:14:07] fetch_files
[STOP] [2023-10-13 10:14:07] fetch_files
[START] [2023-10-13 10:14:07] validate_each_file
[INFO] [2023-10-13 10:14:07] Looping over 4 formats...
[INFO] [2023-10-13 10:14:07] ...refs (/app/public/data/plant_fhd/reference.tab)
[INFO] [2023-10-13 10:14:07] Valid: /app/public/data/plant_fhd/converted_csv/plant_fhd_refs_30666.csv (16 lines)
[INFO] [2023-10-13 10:14:07] ...nodes (/app/public/data/plant_fhd/taxon.tab)
[INFO] [2023-10-13 10:14:15] Valid: /app/public/data/plant_fhd/converted_csv/plant_fhd_nodes_30665.csv (125731 lines)
[INFO] [2023-10-13 10:14:15] ...occurrences (/app/public/data/plant_fhd/occurrences.txt)
[INFO] [2023-10-13 10:14:19] Valid: /app/public/data/plant_fhd/converted_csv/plant_fhd_occurrences_30667.csv (144215 lines)
[INFO] [2023-10-13 10:14:19] ...measurements (/app/public/data/plant_fhd/measurement_or_fact.txt)
[INFO] [2023-10-13 10:14:39] Valid: /app/public/data/plant_fhd/converted_csv/plant_fhd_measurements_30668.csv (251526 lines)
[STOP] [2023-10-13 10:14:39] validate_each_file
[START] [2023-10-13 10:14:39] convert_to_csv
[INFO] [2023-10-13 10:14:39] Looping over 4 formats...
[INFO] [2023-10-13 10:14:39] ...refs (/app/public/data/plant_fhd/reference.tab)
[CMD] [2023-10-13 10:14:39] /usr/bin/sort /app/public/data/plant_fhd/converted_csv/plant_fhd_refs_30666.csv > /app/public/data/plant_fhd/converted_csv/plant_fhd_refs_30666.csv_sorted
[INFO] [2023-10-13 10:14:39] Converted: /app/public/data/plant_fhd/converted_csv/plant_fhd_refs_30666.csv (16 lines)
[INFO] [2023-10-13 10:14:39] ...nodes (/app/public/data/plant_fhd/taxon.tab)
[CMD] [2023-10-13 10:14:39] /usr/bin/sort /app/public/data/plant_fhd/converted_csv/plant_fhd_nodes_30665.csv > /app/public/data/plant_fhd/converted_csv/plant_fhd_nodes_30665.csv_sorted
[INFO] [2023-10-13 10:14:40] Converted: /app/public/data/plant_fhd/converted_csv/plant_fhd_nodes_30665.csv (125731 lines)
[INFO] [2023-10-13 10:14:40] ...occurrences (/app/public/data/plant_fhd/occurrences.txt)
[CMD] [2023-10-13 10:14:40] /usr/bin/sort /app/public/data/plant_fhd/converted_csv/plant_fhd_occurrences_30667.csv > /app/public/data/plant_fhd/converted_csv/plant_fhd_occurrences_30667.csv_sorted
[INFO] [2023-10-13 10:14:40] Converted: /app/public/data/plant_fhd/converted_csv/plant_fhd_occurrences_30667.csv (144215 lines)
[INFO] [2023-10-13 10:14:40] ...measurements (/app/public/data/plant_fhd/measurement_or_fact.txt)
[CMD] [2023-10-13 10:14:40] /usr/bin/sort /app/public/data/plant_fhd/converted_csv/plant_fhd_measurements_30668.csv > /app/public/data/plant_fhd/converted_csv/plant_fhd_measurements_30668.csv_sorted
[INFO] [2023-10-13 10:14:42] Converted: /app/public/data/plant_fhd/converted_csv/plant_fhd_measurements_30668.csv (251526 lines)
[STOP] [2023-10-13 10:14:42] convert_to_csv
[START] [2023-10-13 10:14:42] calculate_delta
[INFO] [2023-10-13 10:14:42] Looping over 4 formats...
[INFO] [2023-10-13 10:14:42] ...refs (/app/public/data/plant_fhd/reference.tab)
[CMD] [2023-10-13 10:14:42] echo "0a" > /app/public/data/plant_fhd/diff/plant_fhd_refs_30666.diff
[CMD] [2023-10-13 10:14:42] tail -n +1 /app/public/data/plant_fhd/converted_csv/plant_fhd_refs_30666.csv >> /app/public/data/plant_fhd/diff/plant_fhd_refs_30666.diff
[CMD] [2023-10-13 10:14:42] echo "." >> /app/public/data/plant_fhd/diff/plant_fhd_refs_30666.diff
[INFO] [2023-10-13 10:14:42] Created diff: /app/public/data/plant_fhd/diff/plant_fhd_refs_30666.diff (18 lines)
[INFO] [2023-10-13 10:14:42] ...nodes (/app/public/data/plant_fhd/taxon.tab)
[CMD] [2023-10-13 10:14:42] echo "0a" > /app/public/data/plant_fhd/diff/plant_fhd_nodes_30665.diff
[CMD] [2023-10-13 10:14:42] tail -n +1 /app/public/data/plant_fhd/converted_csv/plant_fhd_nodes_30665.csv >> /app/public/data/plant_fhd/diff/plant_fhd_nodes_30665.diff
[CMD] [2023-10-13 10:14:42] echo "." >> /app/public/data/plant_fhd/diff/plant_fhd_nodes_30665.diff
[INFO] [2023-10-13 10:14:43] Created diff: /app/public/data/plant_fhd/diff/plant_fhd_nodes_30665.diff (125733 lines)
[INFO] [2023-10-13 10:14:43] ...occurrences (/app/public/data/plant_fhd/occurrences.txt)
[CMD] [2023-10-13 10:14:43] echo "0a" > /app/public/data/plant_fhd/diff/plant_fhd_occurrences_30667.diff
[CMD] [2023-10-13 10:14:43] tail -n +1 /app/public/data/plant_fhd/converted_csv/plant_fhd_occurrences_30667.csv >> /app/public/data/plant_fhd/diff/plant_fhd_occurrences_30667.diff
[CMD] [2023-10-13 10:14:43] echo "." >> /app/public/data/plant_fhd/diff/plant_fhd_occurrences_30667.diff
[INFO] [2023-10-13 10:14:43] Created diff: /app/public/data/plant_fhd/diff/plant_fhd_occurrences_30667.diff (144217 lines)
[INFO] [2023-10-13 10:14:43] ...measurements (/app/public/data/plant_fhd/measurement_or_fact.txt)
[CMD] [2023-10-13 10:14:43] echo "0a" > /app/public/data/plant_fhd/diff/plant_fhd_measurements_30668.diff
[CMD] [2023-10-13 10:14:43] tail -n +1 /app/public/data/plant_fhd/converted_csv/plant_fhd_measurements_30668.csv >> /app/public/data/plant_fhd/diff/plant_fhd_measurements_30668.diff
[CMD] [2023-10-13 10:14:44] echo "." >> /app/public/data/plant_fhd/diff/plant_fhd_measurements_30668.diff
[INFO] [2023-10-13 10:14:44] Created diff: /app/public/data/plant_fhd/diff/plant_fhd_measurements_30668.diff (251528 lines)
[STOP] [2023-10-13 10:14:44] calculate_delta
[START] [2023-10-13 10:14:44] parse_diff_and_store
[INFO] [2023-10-13 10:14:44] Handling diff: /app/public/data/plant_fhd/diff/plant_fhd_refs_30666.diff (18 lines)
[INFO] [2023-10-13 10:14:44] Loading refs diff file into memory (18 lines)...
[INFO] [2023-10-13 10:14:44] Storing 16 References (16/16/18)
[INFO] [2023-10-13 10:14:45] Handling diff: /app/public/data/plant_fhd/diff/plant_fhd_nodes_30665.diff (125733 lines)
[INFO] [2023-10-13 10:14:45] Loading nodes diff file into memory (125733 lines)...
[WARN] [2023-10-13 10:14:45] Filtered Scientific Name `Abrahamia deflexa (H. Perrier) Randrian. & Lowry       ` to `Abrahamia deflexa (H. Perrier) Randrian. & Lowry `
[WARN] [2023-10-13 10:14:45] Filtered Scientific Name `Abrahamia humbertii (H. Perrier) Randrian. & Lowry       ` to `Abrahamia humbertii (H. Perrier) Randrian. & Lowry `
[WARN] [2023-10-13 10:14:45] Filtered Scientific Name `Abrahamia latifolia (Engl.) Randrian. & Lowry        ` to `Abrahamia latifolia (Engl.) Randrian. & Lowry `
[WARN] [2023-10-13 10:14:45] Filtered Scientific Name `Abrahamia lecomtei (H. Perrier) Randrian. & Lowry       ` to `Abrahamia lecomtei (H. Perrier) Randrian. & Lowry `
[WARN] [2023-10-13 10:14:45] Filtered Scientific Name `Abrahamia oblongifolia (Engl.) Randrian. & Lowry        ` to `Abrahamia oblongifolia (Engl.) Randrian. & Lowry `
[WARN] [2023-10-13 10:14:45] Filtered Scientific Name `Abrahamia viguieri (H. Perrier) Randrian. & Lowry       ` to `Abrahamia viguieri (H. Perrier) Randrian. & Lowry `
[WARN] [2023-10-13 10:14:45] Filtered Scientific Name `Abutilon grandidentatum Fryxell        ` to `Abutilon grandidentatum Fryxell `
[WARN] [2023-10-13 10:14:45] Filtered Scientific Name `Acacia mathuataensis A.C.Sm.        ` to `Acacia mathuataensis A.C.Sm. `
[WARN] [2023-10-13 10:14:45] Filtered Scientific Name `Acalypha stellata Cardiel        ` to `Acalypha stellata Cardiel `
[WARN] [2023-10-13 10:14:45] Filtered Scientific Name `Acritopappus pintoi Bautista & D.J.N.Hind      ` to `Acritopappus pintoi Bautista & D.J.N.Hind `
[WARN] [2023-10-13 10:14:45] Filtered Scientific Name `Adelobotrys panamensis Almeda        ` to `Adelobotrys panamensis Almeda `
[WARN] [2023-10-13 10:14:45] Filtered Scientific Name `Aequatorium asterotrichum B.Nord.        ` to `Aequatorium asterotrichum B.Nord. `
[WARN] [2023-10-13 10:14:45] Filtered Scientific Name `Aequatorium cajamarcense H.Rob. & Cuatrec.      ` to `Aequatorium cajamarcense H.Rob. & Cuatrec. `
[WARN] [2023-10-13 10:14:46] Filtered Scientific Name `Aiouea obscura van der Werff      ` to `Aiouea obscura van der Werff `
[WARN] [2023-10-13 10:14:46] Filtered Scientific Name `Almeidea caerulea St.Hil. ex G.Don      ` to `Almeidea caerulea St.Hil. ex G.Don `
[WARN] [2023-10-13 10:14:46] Filtered Scientific Name `Ambavia capuronii (Cavaco & Keraudren) Le Thomas                 ` to `Ambavia capuronii (Cavaco & Keraudren) Le Thomas `
[WARN] [2023-10-13 10:14:46] Filtered Scientific Name `Ambavia gerrardii (Baill.) Le Thomas         ` to `Ambavia gerrardii (Baill.) Le Thomas `
[WARN] [2023-10-13 10:14:47] Filtered Scientific Name `Annona conica Ruiz & Pav.      ` to `Annona conica Ruiz & Pav. `
[WARN] [2023-10-13 10:14:47] Filtered Scientific Name `Annona cordifolia (Szyszy\0x0142.) R.E.Fr.` to `Annona cordifolia (Szyszy0x0142.) R.E.Fr.`
[WARN] [2023-10-13 10:14:47] Filtered Scientific Name `Annona neoecuadorensis H. Rainer  ` to `Annona neoecuadorensis H. Rainer `
[WARN] [2023-10-13 10:14:47] Filtered Scientific Name `Anthonotha leptorrhachis (Harms) J.Leonard       ` to `Anthonotha leptorrhachis (Harms) J.Leonard `
[WARN] [2023-10-13 10:14:47] Filtered Scientific Name `Apeiba schomburgkii Szyszy\0x0142. ` to `Apeiba schomburgkii Szyszy0x0142. `
[WARN] [2023-10-13 10:14:47] Filtered Scientific Name `Ardisia awarum Ricketson & Pipoly      ` to `Ardisia awarum Ricketson & Pipoly `
[WARN] [2023-10-13 10:14:47] Filtered Scientific Name `Ardisia capuronii Pipoly    ` to `Ardisia capuronii Pipoly `
[WARN] [2023-10-13 10:14:47] Filtered Scientific Name `Ardisia eugenioides Lundell        ` to `Ardisia eugenioides Lundell `
[WARN] [2023-10-13 10:14:47] Filtered Scientific Name `Ardisia martinensis Lundell        ` to `Ardisia martinensis Lundell `
[WARN] [2023-10-13 10:14:47] Filtered Scientific Name `Argomuellera perrieri (Leandri) J. Leonard         ` to `Argomuellera perrieri (Leandri) J. Leonard `
[INFO] [2023-10-13 10:14:48] Storing 10728 ScientificNames (21456/10000/125733)
[INFO] [2023-10-13 10:14:51] Storing 10728 Nodes (21456/10000/125733)
[WARN] [2023-10-13 10:14:54] Filtered Scientific Name `Astrocaryum carnosum F.Kahn & B.Milla\0x0301n` to `Astrocaryum carnosum F.Kahn & B.Milla0x0301n`
[WARN] [2023-10-13 10:14:54] Filtered Scientific Name `Astrocaryum ferrugineum F.Kahn & B.Milla\0x0301n ` to `Astrocaryum ferrugineum F.Kahn & B.Milla0x0301n `
[WARN] [2023-10-13 10:14:54] (Reached filtered-name limit; supressing further warnings.)
[WARN] [2023-10-13 10:14:57] SKIPPED 169 Scientific names (42966/20000/125733) with resource_pks already be in the database!
[WARN] [2023-10-13 10:14:57] SKIPPED 169 Nodes (42966/20000/125733) with resource_pks already be in the database!
[INFO] [2023-10-13 10:14:57] Storing 10586 ScientificNames (42966/20000/125733)
[INFO] [2023-10-13 10:15:00] Storing 10586 Nodes (42966/20000/125733)
[WARN] [2023-10-13 10:15:06] SKIPPED 189 Scientific names (64286/30000/125733) with resource_pks already be in the database!
[WARN] [2023-10-13 10:15:06] SKIPPED 189 Nodes (64286/30000/125733) with resource_pks already be in the database!
[INFO] [2023-10-13 10:15:06] Storing 10471 ScientificNames (64286/30000/125733)
[INFO] [2023-10-13 10:15:09] Storing 10471 Nodes (64286/30000/125733)
[WARN] [2023-10-13 10:15:15] SKIPPED 194 Scientific names (85656/40000/125733) with resource_pks already be in the database!
[WARN] [2023-10-13 10:15:15] SKIPPED 194 Nodes (85656/40000/125733) with resource_pks already be in the database!
[INFO] [2023-10-13 10:15:15] Storing 10491 ScientificNames (85656/40000/125733)
[INFO] [2023-10-13 10:15:18] Storing 10491 Nodes (85656/40000/125733)
[WARN] [2023-10-13 10:15:24] SKIPPED 172 Scientific names (106568/50000/125733) with resource_pks already be in the database!
[WARN] [2023-10-13 10:15:24] SKIPPED 172 Nodes (106568/50000/125733) with resource_pks already be in the database!
[INFO] [2023-10-13 10:15:24] Storing 10284 ScientificNames (106568/50000/125733)
[INFO] [2023-10-13 10:15:27] Storing 10284 Nodes (106568/50000/125733)
[WARN] [2023-10-13 10:15:33] SKIPPED 235 Scientific names (128100/60000/125733) with resource_pks already be in the database!
[WARN] [2023-10-13 10:15:33] SKIPPED 235 Nodes (128100/60000/125733) with resource_pks already be in the database!
[INFO] [2023-10-13 10:15:33] Storing 10531 ScientificNames (128100/60000/125733)
[INFO] [2023-10-13 10:15:36] Storing 10531 Nodes (128100/60000/125733)
[WARN] [2023-10-13 10:15:42] SKIPPED 231 Scientific names (149490/70000/125733) with resource_pks already be in the database!
[WARN] [2023-10-13 10:15:43] SKIPPED 231 Nodes (149490/70000/125733) with resource_pks already be in the database!
[INFO] [2023-10-13 10:15:43] Storing 10464 ScientificNames (149490/70000/125733)
[INFO] [2023-10-13 10:15:46] Storing 10464 Nodes (149490/70000/125733)
[WARN] [2023-10-13 10:15:52] SKIPPED 221 Scientific names (170826/80000/125733) with resource_pks already be in the database!
[WARN] [2023-10-13 10:15:52] SKIPPED 221 Nodes (170826/80000/125733) with resource_pks already be in the database!
[INFO] [2023-10-13 10:15:52] Storing 10447 ScientificNames (170826/80000/125733)
[INFO] [2023-10-13 10:15:55] Storing 10447 Nodes (170826/80000/125733)
[WARN] [2023-10-13 10:16:02] SKIPPED 228 Scientific names (192226/90000/125733) with resource_pks already be in the database!
[WARN] [2023-10-13 10:16:02] SKIPPED 228 Nodes (192226/90000/125733) with resource_pks already be in the database!
[INFO] [2023-10-13 10:16:02] Storing 10472 ScientificNames (192226/90000/125733)
[INFO] [2023-10-13 10:16:05] Storing 10472 Nodes (192226/90000/125733)
[WARN] [2023-10-13 10:16:11] SKIPPED 244 Scientific names (213578/100000/125733) with resource_pks already be in the database!
[WARN] [2023-10-13 10:16:11] SKIPPED 244 Nodes (213578/100000/125733) with resource_pks already be in the database!
[INFO] [2023-10-13 10:16:11] Storing 10432 ScientificNames (213578/100000/125733)
[INFO] [2023-10-13 10:16:14] Storing 10432 Nodes (213578/100000/125733)
[WARN] [2023-10-13 10:16:21] SKIPPED 240 Scientific names (235060/110000/125733) with resource_pks already be in the database!
[WARN] [2023-10-13 10:16:21] SKIPPED 240 Nodes (235060/110000/125733) with resource_pks already be in the database!
[INFO] [2023-10-13 10:16:21] Storing 10501 ScientificNames (235060/110000/125733)
[INFO] [2023-10-13 10:16:24] Storing 10501 Nodes (235060/110000/125733)
[WARN] [2023-10-13 10:16:30] SKIPPED 247 Scientific names (256508/120000/125733) with resource_pks already be in the database!
[WARN] [2023-10-13 10:16:31] SKIPPED 247 Nodes (256508/120000/125733) with resource_pks already be in the database!
[INFO] [2023-10-13 10:16:31] Storing 10477 ScientificNames (256508/120000/125733)
[INFO] [2023-10-13 10:16:34] Storing 10477 Nodes (256508/120000/125733)
[WARN] [2023-10-13 10:16:39] SKIPPED 201 Scientific names (268972/125731/125733) with resource_pks already be in the database!
[WARN] [2023-10-13 10:16:39] SKIPPED 201 Nodes (268972/125731/125733) with resource_pks already be in the database!
[INFO] [2023-10-13 10:16:40] Storing 6031 ScientificNames (268972/125731/125733)
[INFO] [2023-10-13 10:16:42] Storing 6031 Nodes (268972/125731/125733)
[INFO] [2023-10-13 10:16:43] Handling diff: /app/public/data/plant_fhd/diff/plant_fhd_occurrences_30667.diff (144217 lines)
[INFO] [2023-10-13 10:16:43] Loading occurrences diff file into memory (144217 lines)...
[INFO] [2023-10-13 10:16:44] Storing 9999 Occurrences (10715/10000/144217)
[INFO] [2023-10-13 10:16:45] Storing 716 OccurrenceMetadata (10715/10000/144217)
[WARN] [2023-10-13 10:16:46] SKIPPED 660 Occurrence metadata (21375/20000/144217) with resource_pks already be in the database!
[INFO] [2023-10-13 10:16:46] Storing 10000 Occurrences (21375/20000/144217)
[INFO] [2023-10-13 10:16:48] Storing 0 OccurrenceMetadata (21375/20000/144217)
[WARN] [2023-10-13 10:16:48] No models to import, skipping!
[WARN] [2023-10-13 10:16:49] SKIPPED 797 Occurrence metadata (32172/30000/144217) with resource_pks already be in the database!
[INFO] [2023-10-13 10:16:49] Storing 10000 Occurrences (32172/30000/144217)
[INFO] [2023-10-13 10:16:50] Storing 0 OccurrenceMetadata (32172/30000/144217)
[WARN] [2023-10-13 10:16:50] No models to import, skipping!
[WARN] [2023-10-13 10:16:51] SKIPPED 685 Occurrence metadata (42857/40000/144217) with resource_pks already be in the database!
[INFO] [2023-10-13 10:16:51] Storing 10000 Occurrences (42857/40000/144217)
[INFO] [2023-10-13 10:16:52] Storing 0 OccurrenceMetadata (42857/40000/144217)
[WARN] [2023-10-13 10:16:52] No models to import, skipping!
[WARN] [2023-10-13 10:16:53] SKIPPED 528 Occurrence metadata (53385/50000/144217) with resource_pks already be in the database!
[INFO] [2023-10-13 10:16:53] Storing 10000 Occurrences (53385/50000/144217)
[INFO] [2023-10-13 10:16:54] Storing 0 OccurrenceMetadata (53385/50000/144217)
[WARN] [2023-10-13 10:16:54] No models to import, skipping!
[WARN] [2023-10-13 10:16:56] SKIPPED 541 Occurrence metadata (63926/60000/144217) with resource_pks already be in the database!
[INFO] [2023-10-13 10:16:56] Storing 10000 Occurrences (63926/60000/144217)
[INFO] [2023-10-13 10:16:57] Storing 0 OccurrenceMetadata (63926/60000/144217)
[WARN] [2023-10-13 10:16:57] No models to import, skipping!
[WARN] [2023-10-13 10:16:58] SKIPPED 774 Occurrence metadata (74700/70000/144217) with resource_pks already be in the database!
[INFO] [2023-10-13 10:16:58] Storing 10000 Occurrences (74700/70000/144217)
[INFO] [2023-10-13 10:16:59] Storing 0 OccurrenceMetadata (74700/70000/144217)
[WARN] [2023-10-13 10:16:59] No models to import, skipping!
[WARN] [2023-10-13 10:17:00] SKIPPED 536 Occurrence metadata (85236/80000/144217) with resource_pks already be in the database!
[INFO] [2023-10-13 10:17:00] Storing 10000 Occurrences (85236/80000/144217)
[INFO] [2023-10-13 10:17:01] Storing 0 OccurrenceMetadata (85236/80000/144217)
[WARN] [2023-10-13 10:17:01] No models to import, skipping!
[WARN] [2023-10-13 10:17:03] SKIPPED 1065 Occurrence metadata (96301/90000/144217) with resource_pks already be in the database!
[INFO] [2023-10-13 10:17:03] Storing 10000 Occurrences (96301/90000/144217)
[INFO] [2023-10-13 10:17:04] Storing 0 OccurrenceMetadata (96301/90000/144217)
[WARN] [2023-10-13 10:17:04] No models to import, skipping!
[WARN] [2023-10-13 10:17:05] SKIPPED 1002 Occurrence metadata (107303/100000/144217) with resource_pks already be in the database!
[INFO] [2023-10-13 10:17:05] Storing 10000 Occurrences (107303/100000/144217)
[INFO] [2023-10-13 10:17:06] Storing 0 OccurrenceMetadata (107303/100000/144217)
[WARN] [2023-10-13 10:17:06] No models to import, skipping!
[WARN] [2023-10-13 10:17:08] SKIPPED 705 Occurrence metadata (118008/110000/144217) with resource_pks already be in the database!
[INFO] [2023-10-13 10:17:08] Storing 10000 Occurrences (118008/110000/144217)
[INFO] [2023-10-13 10:17:09] Storing 0 OccurrenceMetadata (118008/110000/144217)
[WARN] [2023-10-13 10:17:09] No models to import, skipping!
[WARN] [2023-10-13 10:17:10] SKIPPED 801 Occurrence metadata (128809/120000/144217) with resource_pks already be in the database!
[INFO] [2023-10-13 10:17:10] Storing 10000 Occurrences (128809/120000/144217)
[INFO] [2023-10-13 10:17:11] Storing 0 OccurrenceMetadata (128809/120000/144217)
[WARN] [2023-10-13 10:17:11] No models to import, skipping!
[WARN] [2023-10-13 10:17:13] SKIPPED 681 Occurrence metadata (139490/130000/144217) with resource_pks already be in the database!
[INFO] [2023-10-13 10:17:13] Storing 10000 Occurrences (139490/130000/144217)
[INFO] [2023-10-13 10:17:14] Storing 0 OccurrenceMetadata (139490/130000/144217)
[WARN] [2023-10-13 10:17:14] No models to import, skipping!
[WARN] [2023-10-13 10:17:15] SKIPPED 674 Occurrence metadata (150164/140000/144217) with resource_pks already be in the database!
[INFO] [2023-10-13 10:17:15] Storing 10000 Occurrences (150164/140000/144217)
[INFO] [2023-10-13 10:17:16] Storing 0 OccurrenceMetadata (150164/140000/144217)
[WARN] [2023-10-13 10:17:16] No models to import, skipping!
[WARN] [2023-10-13 10:17:18] SKIPPED 390 Occurrence metadata (154770/144215/144217) with resource_pks already be in the database!
[INFO] [2023-10-13 10:17:18] Storing 4216 Occurrences (154770/144215/144217)
[INFO] [2023-10-13 10:17:18] Storing 0 OccurrenceMetadata (154770/144215/144217)
[WARN] [2023-10-13 10:17:18] No models to import, skipping!
[INFO] [2023-10-13 10:17:18] Handling diff: /app/public/data/plant_fhd/diff/plant_fhd_measurements_30668.diff (251528 lines)
[INFO] [2023-10-13 10:17:18] Loading measurements diff file into memory (251528 lines)...
[INFO] [2023-10-13 10:17:23] Storing 9999 Traits (19998/10000/251528)
[INFO] [2023-10-13 10:17:26] Storing 9999 MetaTraits (19998/10000/251528)
[INFO] [2023-10-13 10:17:31] Storing 10000 Traits (39998/20000/251528)
[INFO] [2023-10-13 10:17:34] Storing 10000 MetaTraits (39998/20000/251528)
[INFO] [2023-10-13 10:17:41] Storing 10000 Traits (65793/30000/251528)
[INFO] [2023-10-13 10:17:44] Storing 10000 MetaTraits (65793/30000/251528)
[INFO] [2023-10-13 10:17:45] Storing 5795 TraitsReferences (65793/30000/251528)
[INFO] [2023-10-13 10:17:52] Storing 22789 TraitsReferences (108582/40000/251528)
[INFO] [2023-10-13 10:17:53] Storing 10000 Traits (108582/40000/251528)
[INFO] [2023-10-13 10:17:56] Storing 10000 MetaTraits (108582/40000/251528)
[INFO] [2023-10-13 10:18:05] Storing 45000 TraitsReferences (173582/50000/251528)
[INFO] [2023-10-13 10:18:08] Storing 10000 Traits (173582/50000/251528)
[INFO] [2023-10-13 10:18:11] Storing 10000 MetaTraits (173582/50000/251528)
[INFO] [2023-10-13 10:18:20] Storing 45000 TraitsReferences (238582/60000/251528)
[INFO] [2023-10-13 10:18:23] Storing 10000 Traits (238582/60000/251528)
[INFO] [2023-10-13 10:18:26] Storing 10000 MetaTraits (238582/60000/251528)
[INFO] [2023-10-13 10:18:34] Storing 45000 TraitsReferences (303582/70000/251528)
[INFO] [2023-10-13 10:18:38] Storing 10000 Traits (303582/70000/251528)
[INFO] [2023-10-13 10:18:40] Storing 10000 MetaTraits (303582/70000/251528)
[INFO] [2023-10-13 10:18:47] Storing 1030 TraitsReferences (324840/80000/251528)
[INFO] [2023-10-13 10:18:47] Storing 10000 Traits (324840/80000/251528)
[INFO] [2023-10-13 10:18:50] Storing 10228 MetaTraits (324840/80000/251528)
[INFO] [2023-10-13 10:18:55] Storing 10000 Traits (344840/90000/251528)
[INFO] [2023-10-13 10:18:58] Storing 10000 MetaTraits (344840/90000/251528)
[INFO] [2023-10-13 10:19:04] Storing 10000 Traits (364840/100000/251528)
[INFO] [2023-10-13 10:19:07] Storing 10000 MetaTraits (364840/100000/251528)
[INFO] [2023-10-13 10:19:13] Storing 10000 Traits (384840/110000/251528)
[INFO] [2023-10-13 10:19:16] Storing 10000 MetaTraits (384840/110000/251528)
[INFO] [2023-10-13 10:19:22] Storing 10000 Traits (404840/120000/251528)
[INFO] [2023-10-13 10:19:25] Storing 10000 MetaTraits (404840/120000/251528)
[INFO] [2023-10-13 10:19:31] Storing 10000 Traits (424840/130000/251528)
[INFO] [2023-10-13 10:19:34] Storing 10000 MetaTraits (424840/130000/251528)
[INFO] [2023-10-13 10:19:40] Storing 10000 Traits (444840/140000/251528)
[INFO] [2023-10-13 10:19:43] Storing 10000 MetaTraits (444840/140000/251528)
[INFO] [2023-10-13 10:19:49] Storing 10000 Traits (465902/150000/251528)
[INFO] [2023-10-13 10:19:52] Storing 11062 MetaTraits (465902/150000/251528)
[INFO] [2023-10-13 10:19:59] Storing 10000 Traits (489913/160000/251528)
[INFO] [2023-10-13 10:20:02] Storing 14011 MetaTraits (489913/160000/251528)
[INFO] [2023-10-13 10:20:10] Storing 10000 Traits (517442/170000/251528)
[INFO] [2023-10-13 10:20:13] Storing 17529 MetaTraits (517442/170000/251528)
[INFO] [2023-10-13 10:20:20] Storing 10000 Traits (537442/180000/251528)
[INFO] [2023-10-13 10:20:23] Storing 10000 MetaTraits (537442/180000/251528)
[INFO] [2023-10-13 10:20:29] Storing 10000 Traits (557442/190000/251528)
[INFO] [2023-10-13 10:20:32] Storing 10000 MetaTraits (557442/190000/251528)
[INFO] [2023-10-13 10:20:38] Storing 10000 Traits (577442/200000/251528)
[INFO] [2023-10-13 10:20:41] Storing 10000 MetaTraits (577442/200000/251528)
[INFO] [2023-10-13 10:20:48] Storing 10000 Traits (597442/210000/251528)
[INFO] [2023-10-13 10:20:51] Storing 10000 MetaTraits (597442/210000/251528)
[INFO] [2023-10-13 10:20:57] Storing 10000 Traits (617442/220000/251528)
[INFO] [2023-10-13 10:21:00] Storing 10000 MetaTraits (617442/220000/251528)
[INFO] [2023-10-13 10:21:07] Storing 10000 Traits (637442/230000/251528)
[INFO] [2023-10-13 10:21:10] Storing 10000 MetaTraits (637442/230000/251528)
[INFO] [2023-10-13 10:21:16] Storing 10000 Traits (657442/240000/251528)
[INFO] [2023-10-13 10:21:19] Storing 10000 MetaTraits (657442/240000/251528)
[INFO] [2023-10-13 10:21:25] Storing 10000 Traits (677442/250000/251528)
[INFO] [2023-10-13 10:21:28] Storing 10000 MetaTraits (677442/250000/251528)
[INFO] [2023-10-13 10:21:31] Storing 1527 Traits (680496/251526/251528)
[INFO] [2023-10-13 10:21:32] Storing 1527 MetaTraits (680496/251526/251528)
[STOP] [2023-10-13 10:21:32] parse_diff_and_store
[START] [2023-10-13 10:21:32] resolve_keys
[2023-10-13 10:21:54] Resolving downloaded urls (this is not actually downloading them yet)
[INFO] [2023-10-13 10:22:01] Occurrences to nodes (through scientific_names)...
[INFO] [2023-10-13 10:22:05] traits to occurrences...
[INFO] [2023-10-13 10:22:14] traits to nodes (through occurrences)...
[INFO] [2023-10-13 10:22:21] Traits to sex term...
[INFO] [2023-10-13 10:22:24] Traits to lifestage term...
[INFO] [2023-10-13 10:22:26] MetaTraits to traits...
[INFO] [2023-10-13 10:22:33] MetaTraits (simple, measurement row refers to parent) to traits...
[INFO] [2023-10-13 10:22:41] Assocs to occurrences...
[INFO] [2023-10-13 10:22:41] Assocs to nodes...
[INFO] [2023-10-13 10:22:41] Assoc to sex term...
[INFO] [2023-10-13 10:22:41] Assoc to lifestage term...
[INFO] [2023-10-13 10:22:41] MetaAssoc to assocs...
[STOP] [2023-10-13 10:22:41] resolve_keys
[START] [2023-10-13 10:22:41] hold_for_later_1
[STOP] [2023-10-13 10:22:41] hold_for_later_1
[START] [2023-10-13 10:22:41] hold_for_later_2
[STOP] [2023-10-13 10:22:41] hold_for_later_2
[START] [2023-10-13 10:22:41] resolve_missing_parents
[STOP] [2023-10-13 10:22:48] resolve_missing_parents
[START] [2023-10-13 10:22:48] rebuild_nodes
[START] [2023-10-13 10:22:48] Flattener#flatten
[START] [2023-10-13 10:22:48] Flattener#study_resource
[START] [2023-10-13 10:22:48] Flattener#build_ancestry
[STOP] [2023-10-13 10:23:20] Flattener#build_ancestry
[INFO] [2023-10-13 10:23:20] 131915 ancestry keys
[START] [2023-10-13 10:23:20] build_node_ancestors
[INFO] [2023-10-13 10:23:20] old ancestors deleted.
[STOP] [2023-10-13 10:23:43] build_node_ancestors
[START] [2023-10-13 10:23:48] Flattener#propagate_ancestor_ids
[STOP] [2023-10-13 10:23:55] Flattener#propagate_ancestor_ids
[STOP] [2023-10-13 10:23:55] Flattener#flatten
[STOP] [2023-10-13 10:23:55] rebuild_nodes
[START] [2023-10-13 10:23:55] resolve_missing_media_owners
[STOP] [2023-10-13 10:23:55] resolve_missing_media_owners
[START] [2023-10-13 10:23:55] sanitize_media_verbatims
[STOP] [2023-10-13 10:23:55] sanitize_media_verbatims
[START] [2023-10-13 10:23:55] queue_downloads
[STOP] [2023-10-13 10:23:55] queue_downloads
[START] [2023-10-13 10:23:55] parse_names
[WARN] [2023-10-13 10:23:55] I see 131915 names which still need to be parsed.
[WARN] [2023-10-13 10:23:56] Names to parse: 10000 formatted: 10000 learned: 9822 parsed: 10000
[WARN] [2023-10-13 10:24:04] Names to parse: 10000 formatted: 10000 learned: 9845 parsed: 10000
[WARN] [2023-10-13 10:24:11] Names to parse: 10000 formatted: 10000 learned: 9856 parsed: 10000
[WARN] [2023-10-13 10:24:18] Names to parse: 10000 formatted: 10000 learned: 9864 parsed: 10000
[WARN] [2023-10-13 10:24:25] Names to parse: 10000 formatted: 10000 learned: 9881 parsed: 10000
[WARN] [2023-10-13 10:24:32] Names to parse: 10000 formatted: 10000 learned: 9843 parsed: 10000
[WARN] [2023-10-13 10:24:39] Names to parse: 10000 formatted: 10000 learned: 9872 parsed: 10000
[WARN] [2023-10-13 10:24:46] Names to parse: 10000 formatted: 10000 learned: 9841 parsed: 10000
[WARN] [2023-10-13 10:24:54] Names to parse: 10000 formatted: 10000 learned: 9910 parsed: 10000
[WARN] [2023-10-13 10:25:01] Names to parse: 10000 formatted: 10000 learned: 9903 parsed: 10000
[WARN] [2023-10-13 10:25:08] Names to parse: 10000 formatted: 10000 learned: 9855 parsed: 10000
[WARN] [2023-10-13 10:25:15] Names to parse: 10000 formatted: 10000 learned: 9867 parsed: 10000
[WARN] [2023-10-13 10:25:22] Names to parse: 10000 formatted: 10000 learned: 9877 parsed: 10000
[WARN] [2023-10-13 10:25:29] Names to parse: 1915 formatted: 1915 learned: 1880 parsed: 1915
[STOP] [2023-10-13 10:25:31] parse_names
[START] [2023-10-13 10:25:31] denormalize_canonical_names_to_nodes
[STOP] [2023-10-13 10:25:33] denormalize_canonical_names_to_nodes
[START] [2023-10-13 10:25:33] match_nodes
[START] [2023-10-13 10:25:33] map_all_nodes_to_pages
[STOP] [2023-10-13 10:50:51] map_all_nodes_to_pages
[INFO] [2023-10-13 10:50:51] 17900 Unmatched nodes (of 131915)! That's too many to output. Full list in /app/public/data/plant_fhd/unmatched_nodes.txt ; First 10: Canonical: Acampe praemorsa; Node#137019349; ResourceID: Acampe praemorsa (Roxb.) Blatter & McCann; Canonical: Acanthephippium sylhetense; Node#137019357; ResourceID: Acanthephippium sylhetense Lindl.; Canonical: Aerides falcatum; Node#137021230; ResourceID: Aerides falcatum Lindl. & Paxton; Canonical: Aerides jansoni; Node#137021256; ResourceID: Aerides x jansoni Rolfe; Canonical: Bulbophyllum burkilli; Node#137035471; ResourceID: Bulbophyllum burkilli Gage; Canonical: Bulbophyllum cretaceum; Node#137035479; ResourceID: Bulbophyllum cretaceum Lindl.; Canonical: Calanthe whiteana; Node#137036385; ResourceID: Calanthe whiteana King & Pantl.; Canonical: Campylocentrum matrogrossense; Node#137037738; ResourceID: Campylocentrum matogrossense; Canonical: Catasetum discolor; Node#137039634; ResourceID: Catasetum discolor; Canonical: Christensonella; Node#137042292; ResourceID: Plantae/Orchidaceae/Christensonella
[START] [2023-10-13 10:50:51] update_nodes
[STOP] [2023-10-13 10:51:05] update_nodes
[STOP] [2023-10-13 10:51:05] match_nodes
[START] [2023-10-13 10:51:05] reindex_search
[STOP] [2023-10-13 10:52:43] reindex_search
[START] [2023-10-13 10:52:43] normalize_units
[STOP] [2023-10-13 10:53:28] normalize_units
[START] [2023-10-13 10:53:28] calculate_statistics
[INFO] [2023-10-13 10:53:34] Duplicate page_id count: 0
[STOP] [2023-10-13 10:53:35] calculate_statistics
[START] [2023-10-13 10:53:35] complete_harvest_instance
[START] [2023-10-13 10:53:35] overall_tsv_creation
[INFO] [2023-10-13 10:53:35] Exporting 131915 nodes as TSV in batches of 10000...
[INFO] [2023-10-13 10:53:35] Processing group of 131915 in 14 batches of 10000
[INFO] [2023-10-13 10:53:53] 15336 Traits (unfiltered) and 0 associations...
[INFO] [2023-10-13 10:53:53] Building Traits map for 10000 nodes (this can take a while)...
[INFO] [2023-10-13 10:54:08] Mapped 15336 traits (16187 meta) for 10000 nodes.
[INFO] [2023-10-13 10:54:08] Building Associations map (this can take a while)...
[INFO] [2023-10-13 10:54:10] Done. 0 assocs mapped (0 meta).
[INFO] [2023-10-13 10:54:10] Adding 15336 traits...
[INFO] [2023-10-13 10:54:12] 13186 metadata added.
[INFO] [2023-10-13 10:54:12] Adding 0 assocs...
[INFO] [2023-10-13 10:54:12] 0 metadata added.
[INFO] [2023-10-13 10:54:59] Processed 10000/131915 nodes
[INFO] [2023-10-13 10:55:16] 16193 Traits (unfiltered) and 0 associations...
[INFO] [2023-10-13 10:55:16] Building Traits map for 10000 nodes (this can take a while)...
[INFO] [2023-10-13 10:55:33] Mapped 16193 traits (16830 meta) for 10000 nodes.
[INFO] [2023-10-13 10:55:33] Building Associations map (this can take a while)...
[INFO] [2023-10-13 10:55:36] Done. 0 assocs mapped (0 meta).
[INFO] [2023-10-13 10:55:36] Adding 16193 traits...
[INFO] [2023-10-13 10:55:37] 10186 metadata added.
[INFO] [2023-10-13 10:55:37] Adding 0 assocs...
[INFO] [2023-10-13 10:55:37] 0 metadata added.
[INFO] [2023-10-13 10:56:25] Processed 20000/131915 nodes
[INFO] [2023-10-13 10:56:44] 17863 Traits (unfiltered) and 0 associations...
[INFO] [2023-10-13 10:56:44] Building Traits map for 10000 nodes (this can take a while)...
[INFO] [2023-10-13 10:57:02] Mapped 17863 traits (18575 meta) for 10000 nodes.
[INFO] [2023-10-13 10:57:02] Building Associations map (this can take a while)...
[INFO] [2023-10-13 10:57:05] Done. 0 assocs mapped (0 meta).
[INFO] [2023-10-13 10:57:05] Adding 17863 traits...
[INFO] [2023-10-13 10:57:07] 14748 metadata added.
[INFO] [2023-10-13 10:57:07] Adding 0 assocs...
[INFO] [2023-10-13 10:57:07] 0 metadata added.
[INFO] [2023-10-13 10:57:54] Processed 30000/131915 nodes
[INFO] [2023-10-13 10:58:14] 15942 Traits (unfiltered) and 0 associations...
[INFO] [2023-10-13 10:58:14] Building Traits map for 10000 nodes (this can take a while)...
[INFO] [2023-10-13 10:58:31] Mapped 15942 traits (16652 meta) for 10000 nodes.
[INFO] [2023-10-13 10:58:31] Building Associations map (this can take a while)...
[INFO] [2023-10-13 10:58:33] Done. 0 assocs mapped (0 meta).
[INFO] [2023-10-13 10:58:33] Adding 15942 traits...
[INFO] [2023-10-13 10:58:35] 8627 metadata added.
[INFO] [2023-10-13 10:58:35] Adding 0 assocs...
[INFO] [2023-10-13 10:58:35] 0 metadata added.
[INFO] [2023-10-13 10:59:22] Processed 40000/131915 nodes
[INFO] [2023-10-13 10:59:42] 16729 Traits (unfiltered) and 0 associations...
[INFO] [2023-10-13 10:59:42] Building Traits map for 10000 nodes (this can take a while)...
[INFO] [2023-10-13 11:00:00] Mapped 16729 traits (16985 meta) for 10000 nodes.
[INFO] [2023-10-13 11:00:00] Building Associations map (this can take a while)...
[INFO] [2023-10-13 11:00:02] Done. 0 assocs mapped (0 meta).
[INFO] [2023-10-13 11:00:02] Adding 16729 traits...
[INFO] [2023-10-13 11:00:04] 11925 metadata added.
[INFO] [2023-10-13 11:00:04] Adding 0 assocs...
[INFO] [2023-10-13 11:00:04] 0 metadata added.
[INFO] [2023-10-13 11:00:51] Processed 50000/131915 nodes
[INFO] [2023-10-13 11:01:11] 16195 Traits (unfiltered) and 0 associations...
[INFO] [2023-10-13 11:01:11] Building Traits map for 10000 nodes (this can take a while)...
[INFO] [2023-10-13 11:01:29] Mapped 16195 traits (16679 meta) for 10000 nodes.
[INFO] [2023-10-13 11:01:29] Building Associations map (this can take a while)...
[INFO] [2023-10-13 11:01:31] Done. 0 assocs mapped (0 meta).
[INFO] [2023-10-13 11:01:31] Adding 16195 traits...
[INFO] [2023-10-13 11:01:33] 9699 metadata added.
[INFO] [2023-10-13 11:01:33] Adding 0 assocs...
[INFO] [2023-10-13 11:01:33] 0 metadata added.
[INFO] [2023-10-13 11:02:20] Processed 60000/131915 nodes
[INFO] [2023-10-13 11:02:41] 16450 Traits (unfiltered) and 0 associations...
[INFO] [2023-10-13 11:02:41] Building Traits map for 10000 nodes (this can take a while)...
[INFO] [2023-10-13 11:02:59] Mapped 16450 traits (18004 meta) for 10000 nodes.
[INFO] [2023-10-13 11:02:59] Building Associations map (this can take a while)...
[INFO] [2023-10-13 11:03:01] Done. 0 assocs mapped (0 meta).
[INFO] [2023-10-13 11:03:01] Adding 16450 traits...
[INFO] [2023-10-13 11:03:03] 12417 metadata added.
[INFO] [2023-10-13 11:03:03] Adding 0 assocs...
[INFO] [2023-10-13 11:03:03] 0 metadata added.
[INFO] [2023-10-13 11:03:50] Processed 70000/131915 nodes
[INFO] [2023-10-13 11:04:11] 19437 Traits (unfiltered) and 0 associations...
[INFO] [2023-10-13 11:04:11] Building Traits map for 10000 nodes (this can take a while)...
[INFO] [2023-10-13 11:04:32] Mapped 19437 traits (20060 meta) for 10000 nodes.
[INFO] [2023-10-13 11:04:32] Building Associations map (this can take a while)...
[INFO] [2023-10-13 11:04:34] Done. 0 assocs mapped (0 meta).
[INFO] [2023-10-13 11:04:34] Adding 19437 traits...
[INFO] [2023-10-13 11:04:37] 17655 metadata added.
[INFO] [2023-10-13 11:04:37] Adding 0 assocs...
[INFO] [2023-10-13 11:04:37] 0 metadata added.
[INFO] [2023-10-13 11:05:24] Processed 80000/131915 nodes
[INFO] [2023-10-13 11:05:45] 18004 Traits (unfiltered) and 0 associations...
[INFO] [2023-10-13 11:05:45] Building Traits map for 10000 nodes (this can take a while)...
[INFO] [2023-10-13 11:06:06] Mapped 18004 traits (18671 meta) for 10000 nodes.
[INFO] [2023-10-13 11:06:06] Building Associations map (this can take a while)...
[INFO] [2023-10-13 11:06:08] Done. 0 assocs mapped (0 meta).
[INFO] [2023-10-13 11:06:08] Adding 18004 traits...
[INFO] [2023-10-13 11:06:10] 14571 metadata added.
[INFO] [2023-10-13 11:06:10] Adding 0 assocs...
[INFO] [2023-10-13 11:06:10] 0 metadata added.
[INFO] [2023-10-13 11:06:58] Processed 90000/131915 nodes
[INFO] [2023-10-13 11:07:21] 17709 Traits (unfiltered) and 0 associations...
[INFO] [2023-10-13 11:07:21] Building Traits map for 10000 nodes (this can take a while)...
[INFO] [2023-10-13 11:07:38] Mapped 17709 traits (19095 meta) for 10000 nodes.
[INFO] [2023-10-13 11:07:38] Building Associations map (this can take a while)...
[INFO] [2023-10-13 11:07:41] Done. 0 assocs mapped (0 meta).
[INFO] [2023-10-13 11:07:41] Adding 17709 traits...
[INFO] [2023-10-13 11:07:43] 10859 metadata added.
[INFO] [2023-10-13 11:07:43] Adding 0 assocs...
[INFO] [2023-10-13 11:07:43] 0 metadata added.
[INFO] [2023-10-13 11:08:30] Processed 100000/131915 nodes
[INFO] [2023-10-13 11:08:51] 18187 Traits (unfiltered) and 0 associations...
[INFO] [2023-10-13 11:08:51] Building Traits map for 10000 nodes (this can take a while)...
[INFO] [2023-10-13 11:09:11] Mapped 18187 traits (18790 meta) for 10000 nodes.
[INFO] [2023-10-13 11:09:11] Building Associations map (this can take a while)...
[INFO] [2023-10-13 11:09:13] Done. 0 assocs mapped (0 meta).
[INFO] [2023-10-13 11:09:13] Adding 18187 traits...
[INFO] [2023-10-13 11:09:15] 15530 metadata added.
[INFO] [2023-10-13 11:09:15] Adding 0 assocs...
[INFO] [2023-10-13 11:09:15] 0 metadata added.
[INFO] [2023-10-13 11:10:02] Processed 110000/131915 nodes
[INFO] [2023-10-13 11:10:23] 17627 Traits (unfiltered) and 0 associations...
[INFO] [2023-10-13 11:10:23] Building Traits map for 10000 nodes (this can take a while)...
[INFO] [2023-10-13 11:10:41] Mapped 17627 traits (18243 meta) for 10000 nodes.
[INFO] [2023-10-13 11:10:41] Building Associations map (this can take a while)...
[INFO] [2023-10-13 11:10:43] Done. 0 assocs mapped (0 meta).
[INFO] [2023-10-13 11:10:43] Adding 17627 traits...
[INFO] [2023-10-13 11:10:46] 12730 metadata added.
[INFO] [2023-10-13 11:10:46] Adding 0 assocs...
[INFO] [2023-10-13 11:10:46] 0 metadata added.
[INFO] [2023-10-13 11:11:34] Processed 120000/131915 nodes
[INFO] [2023-10-13 11:11:56] 17534 Traits (unfiltered) and 0 associations...
[INFO] [2023-10-13 11:11:56] Building Traits map for 10000 nodes (this can take a while)...
[INFO] [2023-10-13 11:12:15] Mapped 17534 traits (18910 meta) for 10000 nodes.
[INFO] [2023-10-13 11:12:15] Building Associations map (this can take a while)...
[INFO] [2023-10-13 11:12:17] Done. 0 assocs mapped (0 meta).
[INFO] [2023-10-13 11:12:17] Adding 17534 traits...
[INFO] [2023-10-13 11:12:19] 13956 metadata added.
[INFO] [2023-10-13 11:12:19] Adding 0 assocs...
[INFO] [2023-10-13 11:12:19] 0 metadata added.
[INFO] [2023-10-13 11:13:06] Processed 130000/131915 nodes
[INFO] [2023-10-13 11:13:12] 3377 Traits (unfiltered) and 0 associations...
[INFO] [2023-10-13 11:13:12] Building Traits map for 1915 nodes (this can take a while)...
[INFO] [2023-10-13 11:13:13] Mapped 3377 traits (3500 meta) for 1915 nodes.
[INFO] [2023-10-13 11:13:13] Building Associations map (this can take a while)...
[INFO] [2023-10-13 11:13:13] Done. 0 assocs mapped (0 meta).
[INFO] [2023-10-13 11:13:13] Adding 3377 traits...
[INFO] [2023-10-13 11:13:14] 2107 metadata added.
[INFO] [2023-10-13 11:13:14] Adding 0 assocs...
[INFO] [2023-10-13 11:13:14] 0 metadata added.
[INFO] [2023-10-13 11:13:58] Processed 131915/131915 nodes
[INFO] [2023-10-13 11:13:58] Average Time: 82.758
[INFO] [2023-10-13 11:13:58] Total Time: 20m24s
[INFO] [2023-10-13 11:13:58] last 3 / first 3: 0.9
[INFO] [2023-10-13 11:13:58] Std.Dev: 9.408; Max: 89.96
[STOP] [2023-10-13 11:13:58] overall_tsv_creation
[INFO] [2023-10-13 11:13:58] Done. Check your files:
[INFO] [2023-10-13 11:13:58] (131913 lines) /app/public/data/plant_fhd/publish_nodes.tsv
[INFO] [2023-10-13 11:13:59] (274694 lines) /app/public/data/plant_fhd/publish_node_ancestors.tsv
[INFO] [2023-10-13 11:13:59] (131915 lines) /app/public/data/plant_fhd/publish_scientific_names.tsv
[INFO] [2023-10-13 11:13:59] (226584 lines) /app/public/data/plant_fhd/publish_traits.tsv
[INFO] [2023-10-13 11:13:59] (168197 lines) /app/public/data/plant_fhd/publish_metadata.tsv
[STOP] [2023-10-13 11:13:59] complete_harvest_instance
[START] [2023-10-13 11:13:59] completed
[STOP] [2023-10-13 11:13:59] completed
[STOP] [2023-10-13 11:13:59] logged process, took 3592.32

Latest Process

13 Oct 10:14 EXITED currently processing 0