Harvest for
Flora do Brasil
Created
07 Mar 12:06
Stage:
completed
Fetched:
07 Mar 12:06
Validated:
07 Mar 12:07
Deltas Created
07 Mar 12:07
Units Normalized:
07 Mar 12:44
Ancestry Built:
07 Mar 12:23
Nodes Matched:
07 Mar 12:43
Names Parsed:
07 Mar 12:25
New Models Stored:
07 Mar 12:20
Indexed:
07 Mar 12:44
Completed:
07 Mar 13:06
Time to Harvest:
1 minute
Harvesting Log
(502 lines)
[INFO] [2023-03-07 12:06:54] Created harvest instance #4309
[STOP] [2023-03-07 12:06:54] create_harvest_instance
[START] [2023-03-07 12:06:54] fetch_files
[STOP] [2023-03-07 12:06:54] fetch_files
[START] [2023-03-07 12:06:54] validate_each_file
[INFO] [2023-03-07 12:06:54] Looping over 5 formats...
[INFO] [2023-03-07 12:06:54] ...refs (/app/public/data/brazilian_flora_/reference.tab)
[INFO] [2023-03-07 12:06:55] Valid: /app/public/data/brazilian_flora_/converted_csv/brazilian_flora__refs_30228.csv (55234 lines)
[INFO] [2023-03-07 12:06:55] ...nodes (/app/public/data/brazilian_flora_/taxon.tab)
[INFO] [2023-03-07 12:07:03] Valid: /app/public/data/brazilian_flora_/converted_csv/brazilian_flora__nodes_30230.csv (122658 lines)
[INFO] [2023-03-07 12:07:03] ...vernaculars (/app/public/data/brazilian_flora_/vernacular_name.tab)
[INFO] [2023-03-07 12:07:03] Valid: /app/public/data/brazilian_flora_/converted_csv/brazilian_flora__vernaculars_30227.csv (9124 lines)
[INFO] [2023-03-07 12:07:03] ...occurrences (/app/public/data/brazilian_flora_/occurrence.tab)
[INFO] [2023-03-07 12:07:08] Valid: /app/public/data/brazilian_flora_/converted_csv/brazilian_flora__occurrences_30229.csv (404891 lines)
[INFO] [2023-03-07 12:07:08] ...measurements (/app/public/data/brazilian_flora_/measurement_or_fact_specific.tab)
[INFO] [2023-03-07 12:07:27] Valid: /app/public/data/brazilian_flora_/converted_csv/brazilian_flora__measurements_30231.csv (413020 lines)
[STOP] [2023-03-07 12:07:27] validate_each_file
[START] [2023-03-07 12:07:27] convert_to_csv
[INFO] [2023-03-07 12:07:27] Looping over 5 formats...
[INFO] [2023-03-07 12:07:27] ...refs (/app/public/data/brazilian_flora_/reference.tab)
[CMD] [2023-03-07 12:07:27] /usr/bin/sort /app/public/data/brazilian_flora_/converted_csv/brazilian_flora__refs_30228.csv > /app/public/data/brazilian_flora_/converted_csv/brazilian_flora__refs_30228.csv_sorted
[INFO] [2023-03-07 12:07:27] Converted: /app/public/data/brazilian_flora_/converted_csv/brazilian_flora__refs_30228.csv (55234 lines)
[INFO] [2023-03-07 12:07:27] ...nodes (/app/public/data/brazilian_flora_/taxon.tab)
[CMD] [2023-03-07 12:07:27] /usr/bin/sort /app/public/data/brazilian_flora_/converted_csv/brazilian_flora__nodes_30230.csv > /app/public/data/brazilian_flora_/converted_csv/brazilian_flora__nodes_30230.csv_sorted
[INFO] [2023-03-07 12:07:27] Converted: /app/public/data/brazilian_flora_/converted_csv/brazilian_flora__nodes_30230.csv (122658 lines)
[INFO] [2023-03-07 12:07:27] ...vernaculars (/app/public/data/brazilian_flora_/vernacular_name.tab)
[CMD] [2023-03-07 12:07:27] /usr/bin/sort /app/public/data/brazilian_flora_/converted_csv/brazilian_flora__vernaculars_30227.csv > /app/public/data/brazilian_flora_/converted_csv/brazilian_flora__vernaculars_30227.csv_sorted
[INFO] [2023-03-07 12:07:27] Converted: /app/public/data/brazilian_flora_/converted_csv/brazilian_flora__vernaculars_30227.csv (9124 lines)
[INFO] [2023-03-07 12:07:27] ...occurrences (/app/public/data/brazilian_flora_/occurrence.tab)
[CMD] [2023-03-07 12:07:27] /usr/bin/sort /app/public/data/brazilian_flora_/converted_csv/brazilian_flora__occurrences_30229.csv > /app/public/data/brazilian_flora_/converted_csv/brazilian_flora__occurrences_30229.csv_sorted
[INFO] [2023-03-07 12:07:28] Converted: /app/public/data/brazilian_flora_/converted_csv/brazilian_flora__occurrences_30229.csv (404891 lines)
[INFO] [2023-03-07 12:07:28] ...measurements (/app/public/data/brazilian_flora_/measurement_or_fact_specific.tab)
[CMD] [2023-03-07 12:07:28] /usr/bin/sort /app/public/data/brazilian_flora_/converted_csv/brazilian_flora__measurements_30231.csv > /app/public/data/brazilian_flora_/converted_csv/brazilian_flora__measurements_30231.csv_sorted
[INFO] [2023-03-07 12:07:32] Converted: /app/public/data/brazilian_flora_/converted_csv/brazilian_flora__measurements_30231.csv (413020 lines)
[STOP] [2023-03-07 12:07:32] convert_to_csv
[START] [2023-03-07 12:07:32] calculate_delta
[INFO] [2023-03-07 12:07:32] Looping over 5 formats...
[INFO] [2023-03-07 12:07:32] ...refs (/app/public/data/brazilian_flora_/reference.tab)
[CMD] [2023-03-07 12:07:32] echo "0a" > /app/public/data/brazilian_flora_/diff/brazilian_flora__refs_30228.diff
[CMD] [2023-03-07 12:07:32] tail -n +1 /app/public/data/brazilian_flora_/converted_csv/brazilian_flora__refs_30228.csv >> /app/public/data/brazilian_flora_/diff/brazilian_flora__refs_30228.diff
[CMD] [2023-03-07 12:07:32] echo "." >> /app/public/data/brazilian_flora_/diff/brazilian_flora__refs_30228.diff
[INFO] [2023-03-07 12:07:32] Created diff: /app/public/data/brazilian_flora_/diff/brazilian_flora__refs_30228.diff (55236 lines)
[INFO] [2023-03-07 12:07:32] ...nodes (/app/public/data/brazilian_flora_/taxon.tab)
[CMD] [2023-03-07 12:07:32] echo "0a" > /app/public/data/brazilian_flora_/diff/brazilian_flora__nodes_30230.diff
[CMD] [2023-03-07 12:07:32] tail -n +1 /app/public/data/brazilian_flora_/converted_csv/brazilian_flora__nodes_30230.csv >> /app/public/data/brazilian_flora_/diff/brazilian_flora__nodes_30230.diff
[CMD] [2023-03-07 12:07:32] echo "." >> /app/public/data/brazilian_flora_/diff/brazilian_flora__nodes_30230.diff
[INFO] [2023-03-07 12:07:32] Created diff: /app/public/data/brazilian_flora_/diff/brazilian_flora__nodes_30230.diff (122660 lines)
[INFO] [2023-03-07 12:07:32] ...vernaculars (/app/public/data/brazilian_flora_/vernacular_name.tab)
[CMD] [2023-03-07 12:07:32] echo "0a" > /app/public/data/brazilian_flora_/diff/brazilian_flora__vernaculars_30227.diff
[CMD] [2023-03-07 12:07:32] tail -n +1 /app/public/data/brazilian_flora_/converted_csv/brazilian_flora__vernaculars_30227.csv >> /app/public/data/brazilian_flora_/diff/brazilian_flora__vernaculars_30227.diff
[CMD] [2023-03-07 12:07:32] echo "." >> /app/public/data/brazilian_flora_/diff/brazilian_flora__vernaculars_30227.diff
[INFO] [2023-03-07 12:07:32] Created diff: /app/public/data/brazilian_flora_/diff/brazilian_flora__vernaculars_30227.diff (9126 lines)
[INFO] [2023-03-07 12:07:32] ...occurrences (/app/public/data/brazilian_flora_/occurrence.tab)
[CMD] [2023-03-07 12:07:32] echo "0a" > /app/public/data/brazilian_flora_/diff/brazilian_flora__occurrences_30229.diff
[CMD] [2023-03-07 12:07:32] tail -n +1 /app/public/data/brazilian_flora_/converted_csv/brazilian_flora__occurrences_30229.csv >> /app/public/data/brazilian_flora_/diff/brazilian_flora__occurrences_30229.diff
[CMD] [2023-03-07 12:07:33] echo "." >> /app/public/data/brazilian_flora_/diff/brazilian_flora__occurrences_30229.diff
[INFO] [2023-03-07 12:07:33] Created diff: /app/public/data/brazilian_flora_/diff/brazilian_flora__occurrences_30229.diff (404893 lines)
[INFO] [2023-03-07 12:07:33] ...measurements (/app/public/data/brazilian_flora_/measurement_or_fact_specific.tab)
[CMD] [2023-03-07 12:07:33] echo "0a" > /app/public/data/brazilian_flora_/diff/brazilian_flora__measurements_30231.diff
[CMD] [2023-03-07 12:07:33] tail -n +1 /app/public/data/brazilian_flora_/converted_csv/brazilian_flora__measurements_30231.csv >> /app/public/data/brazilian_flora_/diff/brazilian_flora__measurements_30231.diff
[CMD] [2023-03-07 12:07:35] echo "." >> /app/public/data/brazilian_flora_/diff/brazilian_flora__measurements_30231.diff
[INFO] [2023-03-07 12:07:35] Created diff: /app/public/data/brazilian_flora_/diff/brazilian_flora__measurements_30231.diff (413022 lines)
[STOP] [2023-03-07 12:07:35] calculate_delta
[START] [2023-03-07 12:07:35] parse_diff_and_store
[INFO] [2023-03-07 12:07:35] Handling diff: /app/public/data/brazilian_flora_/diff/brazilian_flora__refs_30228.diff (55236 lines)
[INFO] [2023-03-07 12:07:35] Loading refs diff file into memory (55236 lines)...
[INFO] [2023-03-07 12:07:42] Storing 9999 References (9999/10000/55236)
[INFO] [2023-03-07 12:07:50] Storing 10000 References (19999/20000/55236)
[INFO] [2023-03-07 12:07:58] Storing 10000 References (29999/30000/55236)
[INFO] [2023-03-07 12:08:06] Storing 10000 References (39999/40000/55236)
[INFO] [2023-03-07 12:08:15] Storing 10000 References (49999/50000/55236)
[INFO] [2023-03-07 12:08:20] Storing 5235 References (55234/55234/55236)
[INFO] [2023-03-07 12:08:21] Handling diff: /app/public/data/brazilian_flora_/diff/brazilian_flora__nodes_30230.diff (122660 lines)
[INFO] [2023-03-07 12:08:21] Loading nodes diff file into memory (122660 lines)...
[INFO] [2023-03-07 12:08:24] Storing 11928 ScientificNames (20599/10000/122660)
[INFO] [2023-03-07 12:08:28] Storing 8671 Nodes (20599/10000/122660)
[WARN] [2023-03-07 12:08:34] SKIPPED 443 Scientific names (39941/20000/122660) with resource_pks already be in the database!
[WARN] [2023-03-07 12:08:34] SKIPPED 443 Nodes (39941/20000/122660) with resource_pks already be in the database!
[INFO] [2023-03-07 12:08:34] Storing 11426 ScientificNames (39941/20000/122660)
[INFO] [2023-03-07 12:08:37] Storing 7030 Nodes (39941/20000/122660)
[WARN] [2023-03-07 12:08:39] Filtered Scientific Name `Peridinium gatunense var. madagascariensis Lefèvre` to `Peridinium gatunense var. madagascariensis Lefèvre`
[WARN] [2023-03-07 12:08:42] SKIPPED 917 Scientific names (60229/30000/122660) with resource_pks already be in the database!
[WARN] [2023-03-07 12:08:42] SKIPPED 917 Nodes (60229/30000/122660) with resource_pks already be in the database!
[INFO] [2023-03-07 12:08:42] Storing 11174 ScientificNames (60229/30000/122660)
[INFO] [2023-03-07 12:08:46] Storing 7280 Nodes (60229/30000/122660)
[WARN] [2023-03-07 12:08:50] Filtered Scientific Name `Glomus pachycaule (C.G. Wu & Z.C. Chen) Sieverd. & Oehl` to `Glomus pachycaule (C.G. Wu & Z.C. Chen) Sieverd. & Oehl`
[WARN] [2023-03-07 12:08:51] SKIPPED 1198 Scientific names (79683/40000/122660) with resource_pks already be in the database!
[WARN] [2023-03-07 12:08:51] SKIPPED 1198 Nodes (79683/40000/122660) with resource_pks already be in the database!
[INFO] [2023-03-07 12:08:51] Storing 11075 ScientificNames (79683/40000/122660)
[INFO] [2023-03-07 12:08:54] Storing 5983 Nodes (79683/40000/122660)
[WARN] [2023-03-07 12:08:59] Filtered Scientific Name `Poraqueiba Aubl.` to `Poraqueiba Aubl.`
[WARN] [2023-03-07 12:09:00] SKIPPED 1195 Scientific names (101221/50000/122660) with resource_pks already be in the database!
[WARN] [2023-03-07 12:09:00] SKIPPED 1195 Nodes (101221/50000/122660) with resource_pks already be in the database!
[INFO] [2023-03-07 12:09:00] Storing 10589 ScientificNames (101221/50000/122660)
[INFO] [2023-03-07 12:09:03] Storing 8559 Nodes (101221/50000/122660)
[WARN] [2023-03-07 12:09:09] SKIPPED 1087 Scientific names (120937/60000/122660) with resource_pks already be in the database!
[WARN] [2023-03-07 12:09:09] SKIPPED 1087 Nodes (120937/60000/122660) with resource_pks already be in the database!
[INFO] [2023-03-07 12:09:09] Storing 10213 ScientificNames (120937/60000/122660)
[INFO] [2023-03-07 12:09:12] Storing 7329 Nodes (120937/60000/122660)
[WARN] [2023-03-07 12:09:16] Filtered Scientific Name `Leopoldia Herb.` to `Leopoldia Herb.`
[WARN] [2023-03-07 12:09:17] SKIPPED 482 Scientific names (135634/70000/122660) with resource_pks already be in the database!
[WARN] [2023-03-07 12:09:17] SKIPPED 482 Nodes (135634/70000/122660) with resource_pks already be in the database!
[INFO] [2023-03-07 12:09:17] Storing 10765 ScientificNames (135634/70000/122660)
[INFO] [2023-03-07 12:09:20] Storing 2968 Nodes (135634/70000/122660)
[WARN] [2023-03-07 12:09:24] Filtered Scientific Name `Lejeunea ulicina (Taylor) Gottsche et al.` to `Lejeunea ulicina (Taylor) Gottsche et al.`
[WARN] [2023-03-07 12:09:24] SKIPPED 510 Scientific names (149579/80000/122660) with resource_pks already be in the database!
[WARN] [2023-03-07 12:09:25] SKIPPED 510 Nodes (149579/80000/122660) with resource_pks already be in the database!
[INFO] [2023-03-07 12:09:25] Storing 10775 ScientificNames (149579/80000/122660)
[INFO] [2023-03-07 12:09:28] Storing 2150 Nodes (149579/80000/122660)
[WARN] [2023-03-07 12:09:28] Filtered Scientific Name `Psilotum triquetrum Sw.` to `Psilotum triquetrum Sw.`
[WARN] [2023-03-07 12:09:28] Filtered Scientific Name `Perichaena calongei Lado, D. Wrigley & Estrada` to `Perichaena calongei Lado, D. Wrigley & Estrada`
[WARN] [2023-03-07 12:09:29] Filtered Scientific Name `Oncidium clovesianum (V.P.Castro & M.M.L.L.Cardoso) J.M.H.Shaw` to `Oncidium clovesianum (V.P.Castro & M.M.L.L.Cardoso) J.M.H.Shaw`
[WARN] [2023-03-07 12:09:29] Filtered Scientific Name `Craspedaria Link` to `Craspedaria Link`
[WARN] [2023-03-07 12:09:29] Filtered Scientific Name `Neolacis Wedd.` to `Neolacis Wedd.`
[WARN] [2023-03-07 12:09:29] Filtered Scientific Name `Croton verbenaefolius Müll.Arg.` to `Croton verbenaefolius Müll.Arg.`
[WARN] [2023-03-07 12:09:29] Filtered Scientific Name `Xylosma ciliatifolium (Clos) Eichler` to `Xylosma ciliatifolium (Clos) Eichler`
[WARN] [2023-03-07 12:09:29] Filtered Scientific Name `Physalis linkiana Nees` to `Physalis linkiana Nees`
[WARN] [2023-03-07 12:09:29] Filtered Scientific Name `Guatteria laevigata Mart.` to `Guatteria laevigata Mart.`
[WARN] [2023-03-07 12:09:29] Filtered Scientific Name `Lejeunea maxonii (A. Evans) X.-L. He` to `Lejeunea maxonii (A. Evans) X.-L. He`
[WARN] [2023-03-07 12:09:29] Filtered Scientific Name `Coccoloba fastigata Meisn.` to `Coccoloba fastigata Meisn.`
[WARN] [2023-03-07 12:09:29] Filtered Scientific Name `Colliguaya brasiliensis Klotzsch ex Baill.` to `Colliguaya brasiliensis Klotzsch ex Baill.`
[WARN] [2023-03-07 12:09:29] Filtered Scientific Name `Polypodium pendulum Sw.` to `Polypodium pendulum Sw.`
[WARN] [2023-03-07 12:09:29] Filtered Scientific Name `Miconia saldanhaei Cogn.` to `Miconia saldanhaei Cogn.`
[WARN] [2023-03-07 12:09:29] Filtered Scientific Name `Coccoloba martii Meisn.` to `Coccoloba martii Meisn.`
[WARN] [2023-03-07 12:09:29] Filtered Scientific Name `Trembleya selloana Cogn.` to `Trembleya selloana Cogn.`
[WARN] [2023-03-07 12:09:29] Filtered Scientific Name `Croton muscicapa Müll.Arg.` to `Croton muscicapa Müll.Arg.`
[WARN] [2023-03-07 12:09:29] Filtered Scientific Name `Coccoloba sticticaulis Wedd.` to `Coccoloba sticticaulis Wedd.`
[WARN] [2023-03-07 12:09:29] Filtered Scientific Name `Coccoloba polystachya Wedd.` to `Coccoloba polystachya Wedd.`
[WARN] [2023-03-07 12:09:29] Filtered Scientific Name `Xylosma tweedianum (Clos) Eichler` to `Xylosma tweedianum (Clos) Eichler`
[WARN] [2023-03-07 12:09:29] Filtered Scientific Name `Coccoloba crescentiaefolia Meisn.` to `Coccoloba crescentiaefolia Meisn.`
[WARN] [2023-03-07 12:09:29] Filtered Scientific Name `Celtis aculeata Sw.` to `Celtis aculeata Sw.`
[WARN] [2023-03-07 12:09:29] Filtered Scientific Name `Polypodium diversifolium Sw.` to `Polypodium diversifolium Sw.`
[WARN] [2023-03-07 12:09:29] Filtered Scientific Name `Croton rufo-argenteus Müll.Arg.` to `Croton rufo-argenteus Müll.Arg.`
[WARN] [2023-03-07 12:09:29] (Reached filtered-name limit; supressing further warnings.)
[WARN] [2023-03-07 12:09:32] SKIPPED 1242 Scientific names (166537/90000/122660) with resource_pks already be in the database!
[WARN] [2023-03-07 12:09:32] SKIPPED 1242 Nodes (166537/90000/122660) with resource_pks already be in the database!
[INFO] [2023-03-07 12:09:32] Storing 10750 ScientificNames (166537/90000/122660)
[INFO] [2023-03-07 12:09:35] Storing 3724 Nodes (166537/90000/122660)
[WARN] [2023-03-07 12:09:39] SKIPPED 462 Scientific names (180059/100000/122660) with resource_pks already be in the database!
[WARN] [2023-03-07 12:09:39] SKIPPED 462 Nodes (180059/100000/122660) with resource_pks already be in the database!
[INFO] [2023-03-07 12:09:39] Storing 10727 ScientificNames (180059/100000/122660)
[INFO] [2023-03-07 12:09:42] Storing 1871 Nodes (180059/100000/122660)
[WARN] [2023-03-07 12:09:46] SKIPPED 1043 Scientific names (196352/110000/122660) with resource_pks already be in the database!
[WARN] [2023-03-07 12:09:46] SKIPPED 1043 Nodes (196352/110000/122660) with resource_pks already be in the database!
[INFO] [2023-03-07 12:09:46] Storing 10572 ScientificNames (196352/110000/122660)
[INFO] [2023-03-07 12:09:49] Storing 3635 Nodes (196352/110000/122660)
[WARN] [2023-03-07 12:09:54] SKIPPED 1333 Scientific names (218447/120000/122660) with resource_pks already be in the database!
[WARN] [2023-03-07 12:09:54] SKIPPED 1333 Nodes (218447/120000/122660) with resource_pks already be in the database!
[INFO] [2023-03-07 12:09:54] Storing 10746 ScientificNames (218447/120000/122660)
[INFO] [2023-03-07 12:09:57] Storing 8683 Nodes (218447/120000/122660)
[WARN] [2023-03-07 12:10:01] SKIPPED 301 Scientific names (224211/122658/122660) with resource_pks already be in the database!
[WARN] [2023-03-07 12:10:01] SKIPPED 301 Nodes (224211/122658/122660) with resource_pks already be in the database!
[INFO] [2023-03-07 12:10:01] Storing 2886 ScientificNames (224211/122658/122660)
[INFO] [2023-03-07 12:10:02] Storing 2276 Nodes (224211/122658/122660)
[INFO] [2023-03-07 12:10:03] Handling diff: /app/public/data/brazilian_flora_/diff/brazilian_flora__vernaculars_30227.diff (9126 lines)
[INFO] [2023-03-07 12:10:03] Loading vernaculars diff file into memory (9126 lines)...
[INFO] [2023-03-07 12:10:05] Storing 9124 Vernaculars (9124/9124/9126)
[INFO] [2023-03-07 12:10:07] Handling diff: /app/public/data/brazilian_flora_/diff/brazilian_flora__occurrences_30229.diff (404893 lines)
[INFO] [2023-03-07 12:10:07] Loading occurrences diff file into memory (404893 lines)...
[INFO] [2023-03-07 12:10:08] Storing 9999 Occurrences (9999/10000/404893)
[INFO] [2023-03-07 12:10:10] Storing 10000 Occurrences (19999/20000/404893)
[INFO] [2023-03-07 12:10:12] Storing 10000 Occurrences (29999/30000/404893)
[INFO] [2023-03-07 12:10:14] Storing 10000 Occurrences (39999/40000/404893)
[INFO] [2023-03-07 12:10:16] Storing 10000 Occurrences (49999/50000/404893)
[INFO] [2023-03-07 12:10:18] Storing 10000 Occurrences (59999/60000/404893)
[INFO] [2023-03-07 12:10:20] Storing 10000 Occurrences (69999/70000/404893)
[INFO] [2023-03-07 12:10:22] Storing 10000 Occurrences (79999/80000/404893)
[INFO] [2023-03-07 12:10:25] Storing 10000 Occurrences (89999/90000/404893)
[INFO] [2023-03-07 12:10:27] Storing 10000 Occurrences (99999/100000/404893)
[INFO] [2023-03-07 12:10:29] Storing 10000 Occurrences (109999/110000/404893)
[INFO] [2023-03-07 12:10:31] Storing 10000 Occurrences (119999/120000/404893)
[INFO] [2023-03-07 12:10:34] Storing 10000 Occurrences (129999/130000/404893)
[INFO] [2023-03-07 12:10:36] Storing 10000 Occurrences (139999/140000/404893)
[INFO] [2023-03-07 12:10:39] Storing 10000 Occurrences (149999/150000/404893)
[INFO] [2023-03-07 12:10:41] Storing 10000 Occurrences (159999/160000/404893)
[INFO] [2023-03-07 12:10:44] Storing 10000 Occurrences (169999/170000/404893)
[INFO] [2023-03-07 12:10:47] Storing 10000 Occurrences (179999/180000/404893)
[INFO] [2023-03-07 12:10:49] Storing 10000 Occurrences (189999/190000/404893)
[INFO] [2023-03-07 12:10:52] Storing 10000 Occurrences (199999/200000/404893)
[INFO] [2023-03-07 12:10:55] Storing 10000 Occurrences (209999/210000/404893)
[INFO] [2023-03-07 12:10:58] Storing 10000 Occurrences (219999/220000/404893)
[INFO] [2023-03-07 12:11:01] Storing 10000 Occurrences (229999/230000/404893)
[INFO] [2023-03-07 12:11:03] Storing 10000 Occurrences (239999/240000/404893)
[INFO] [2023-03-07 12:11:06] Storing 10000 Occurrences (249999/250000/404893)
[INFO] [2023-03-07 12:11:09] Storing 10000 Occurrences (259999/260000/404893)
[INFO] [2023-03-07 12:11:12] Storing 10000 Occurrences (269999/270000/404893)
[INFO] [2023-03-07 12:11:15] Storing 10000 Occurrences (279999/280000/404893)
[INFO] [2023-03-07 12:11:18] Storing 10000 Occurrences (289999/290000/404893)
[INFO] [2023-03-07 12:11:21] Storing 10000 Occurrences (299999/300000/404893)
[INFO] [2023-03-07 12:11:24] Storing 10000 Occurrences (309999/310000/404893)
[INFO] [2023-03-07 12:11:27] Storing 10000 Occurrences (319999/320000/404893)
[INFO] [2023-03-07 12:11:30] Storing 10000 Occurrences (329999/330000/404893)
[INFO] [2023-03-07 12:11:34] Storing 10000 Occurrences (339999/340000/404893)
[INFO] [2023-03-07 12:11:37] Storing 10000 Occurrences (349999/350000/404893)
[INFO] [2023-03-07 12:11:40] Storing 10000 Occurrences (359999/360000/404893)
[INFO] [2023-03-07 12:11:44] Storing 10000 Occurrences (369999/370000/404893)
[INFO] [2023-03-07 12:11:47] Storing 10000 Occurrences (379999/380000/404893)
[INFO] [2023-03-07 12:11:50] Storing 10000 Occurrences (389999/390000/404893)
[INFO] [2023-03-07 12:11:54] Storing 10000 Occurrences (399999/400000/404893)
[INFO] [2023-03-07 12:11:57] Storing 4892 Occurrences (404891/404891/404893)
[INFO] [2023-03-07 12:11:58] Handling diff: /app/public/data/brazilian_flora_/diff/brazilian_flora__measurements_30231.diff (413022 lines)
[INFO] [2023-03-07 12:11:58] Loading measurements diff file into memory (413022 lines)...
[INFO] [2023-03-07 12:12:04] Storing 7421 TraitsReferences (37418/10000/413022)
[INFO] [2023-03-07 12:12:05] Storing 9999 Traits (37418/10000/413022)
[INFO] [2023-03-07 12:12:08] Storing 19998 MetaTraits (37418/10000/413022)
[INFO] [2023-03-07 12:12:16] Storing 7485 TraitsReferences (74903/20000/413022)
[INFO] [2023-03-07 12:12:16] Storing 10000 Traits (74903/20000/413022)
[INFO] [2023-03-07 12:12:19] Storing 20000 MetaTraits (74903/20000/413022)
[INFO] [2023-03-07 12:12:27] Storing 7386 TraitsReferences (112289/30000/413022)
[INFO] [2023-03-07 12:12:28] Storing 10000 Traits (112289/30000/413022)
[INFO] [2023-03-07 12:12:31] Storing 20000 MetaTraits (112289/30000/413022)
[INFO] [2023-03-07 12:12:39] Storing 10000 Traits (149717/40000/413022)
[INFO] [2023-03-07 12:12:42] Storing 20000 MetaTraits (149717/40000/413022)
[INFO] [2023-03-07 12:12:44] Storing 7428 TraitsReferences (149717/40000/413022)
[INFO] [2023-03-07 12:12:50] Storing 10000 Traits (187044/50000/413022)
[INFO] [2023-03-07 12:12:55] Storing 20000 MetaTraits (187044/50000/413022)
[INFO] [2023-03-07 12:12:58] Storing 7327 TraitsReferences (187044/50000/413022)
[INFO] [2023-03-07 12:13:04] Storing 10000 Traits (224522/60000/413022)
[INFO] [2023-03-07 12:13:07] Storing 20000 MetaTraits (224522/60000/413022)
[INFO] [2023-03-07 12:13:10] Storing 7478 TraitsReferences (224522/60000/413022)
[INFO] [2023-03-07 12:13:16] Storing 10000 Traits (261834/70000/413022)
[INFO] [2023-03-07 12:13:19] Storing 20000 MetaTraits (261834/70000/413022)
[INFO] [2023-03-07 12:13:22] Storing 7312 TraitsReferences (261834/70000/413022)
[INFO] [2023-03-07 12:13:28] Storing 10000 Traits (299233/80000/413022)
[INFO] [2023-03-07 12:13:31] Storing 20000 MetaTraits (299233/80000/413022)
[INFO] [2023-03-07 12:13:33] Storing 7399 TraitsReferences (299233/80000/413022)
[INFO] [2023-03-07 12:13:40] Storing 7442 TraitsReferences (336675/90000/413022)
[INFO] [2023-03-07 12:13:40] Storing 10000 Traits (336675/90000/413022)
[INFO] [2023-03-07 12:13:44] Storing 20000 MetaTraits (336675/90000/413022)
[INFO] [2023-03-07 12:13:52] Storing 7510 TraitsReferences (374185/100000/413022)
[INFO] [2023-03-07 12:13:52] Storing 10000 Traits (374185/100000/413022)
[INFO] [2023-03-07 12:13:55] Storing 20000 MetaTraits (374185/100000/413022)
[INFO] [2023-03-07 12:14:04] Storing 10000 Traits (411474/110000/413022)
[INFO] [2023-03-07 12:14:07] Storing 20000 MetaTraits (411474/110000/413022)
[INFO] [2023-03-07 12:14:09] Storing 7289 TraitsReferences (411474/110000/413022)
[INFO] [2023-03-07 12:14:16] Storing 10000 Traits (448992/120000/413022)
[INFO] [2023-03-07 12:14:19] Storing 20000 MetaTraits (448992/120000/413022)
[INFO] [2023-03-07 12:14:21] Storing 7518 TraitsReferences (448992/120000/413022)
[INFO] [2023-03-07 12:14:28] Storing 7319 TraitsReferences (486311/130000/413022)
[INFO] [2023-03-07 12:14:28] Storing 10000 Traits (486311/130000/413022)
[INFO] [2023-03-07 12:14:31] Storing 20000 MetaTraits (486311/130000/413022)
[INFO] [2023-03-07 12:14:40] Storing 10000 Traits (523543/140000/413022)
[INFO] [2023-03-07 12:14:43] Storing 20000 MetaTraits (523543/140000/413022)
[INFO] [2023-03-07 12:14:45] Storing 7232 TraitsReferences (523543/140000/413022)
[INFO] [2023-03-07 12:14:52] Storing 7395 TraitsReferences (560938/150000/413022)
[INFO] [2023-03-07 12:14:52] Storing 10000 Traits (560938/150000/413022)
[INFO] [2023-03-07 12:14:55] Storing 20000 MetaTraits (560938/150000/413022)
[INFO] [2023-03-07 12:15:04] Storing 10000 Traits (598391/160000/413022)
[INFO] [2023-03-07 12:15:07] Storing 20000 MetaTraits (598391/160000/413022)
[INFO] [2023-03-07 12:15:10] Storing 7453 TraitsReferences (598391/160000/413022)
[INFO] [2023-03-07 12:15:16] Storing 7423 TraitsReferences (635814/170000/413022)
[INFO] [2023-03-07 12:15:17] Storing 10000 Traits (635814/170000/413022)
[INFO] [2023-03-07 12:15:20] Storing 20000 MetaTraits (635814/170000/413022)
[INFO] [2023-03-07 12:15:28] Storing 7381 TraitsReferences (673195/180000/413022)
[INFO] [2023-03-07 12:15:29] Storing 10000 Traits (673195/180000/413022)
[INFO] [2023-03-07 12:15:32] Storing 20000 MetaTraits (673195/180000/413022)
[INFO] [2023-03-07 12:15:41] Storing 7418 TraitsReferences (710613/190000/413022)
[INFO] [2023-03-07 12:15:42] Storing 10000 Traits (710613/190000/413022)
[INFO] [2023-03-07 12:15:45] Storing 20000 MetaTraits (710613/190000/413022)
[INFO] [2023-03-07 12:15:54] Storing 7399 TraitsReferences (748012/200000/413022)
[INFO] [2023-03-07 12:15:54] Storing 10000 Traits (748012/200000/413022)
[INFO] [2023-03-07 12:15:57] Storing 20000 MetaTraits (748012/200000/413022)
[INFO] [2023-03-07 12:16:06] Storing 7403 TraitsReferences (785415/210000/413022)
[INFO] [2023-03-07 12:16:07] Storing 10000 Traits (785415/210000/413022)
[INFO] [2023-03-07 12:16:10] Storing 20000 MetaTraits (785415/210000/413022)
[INFO] [2023-03-07 12:16:18] Storing 7314 TraitsReferences (822729/220000/413022)
[INFO] [2023-03-07 12:16:19] Storing 10000 Traits (822729/220000/413022)
[INFO] [2023-03-07 12:16:22] Storing 20000 MetaTraits (822729/220000/413022)
[INFO] [2023-03-07 12:16:31] Storing 10000 Traits (860189/230000/413022)
[INFO] [2023-03-07 12:16:34] Storing 20000 MetaTraits (860189/230000/413022)
[INFO] [2023-03-07 12:16:37] Storing 7460 TraitsReferences (860189/230000/413022)
[INFO] [2023-03-07 12:16:43] Storing 7525 TraitsReferences (897714/240000/413022)
[INFO] [2023-03-07 12:16:44] Storing 10000 Traits (897714/240000/413022)
[INFO] [2023-03-07 12:16:47] Storing 20000 MetaTraits (897714/240000/413022)
[INFO] [2023-03-07 12:16:57] Storing 7509 TraitsReferences (935223/250000/413022)
[INFO] [2023-03-07 12:16:57] Storing 10000 Traits (935223/250000/413022)
[INFO] [2023-03-07 12:17:00] Storing 20000 MetaTraits (935223/250000/413022)
[INFO] [2023-03-07 12:17:09] Storing 7281 TraitsReferences (972504/260000/413022)
[INFO] [2023-03-07 12:17:10] Storing 10000 Traits (972504/260000/413022)
[INFO] [2023-03-07 12:17:13] Storing 20000 MetaTraits (972504/260000/413022)
[INFO] [2023-03-07 12:17:22] Storing 10000 Traits (1009912/270000/413022)
[INFO] [2023-03-07 12:17:25] Storing 20000 MetaTraits (1009912/270000/413022)
[INFO] [2023-03-07 12:17:28] Storing 7408 TraitsReferences (1009912/270000/413022)
[INFO] [2023-03-07 12:17:34] Storing 7328 TraitsReferences (1047240/280000/413022)
[INFO] [2023-03-07 12:17:35] Storing 10000 Traits (1047240/280000/413022)
[INFO] [2023-03-07 12:17:38] Storing 20000 MetaTraits (1047240/280000/413022)
[INFO] [2023-03-07 12:17:47] Storing 7406 TraitsReferences (1084646/290000/413022)
[INFO] [2023-03-07 12:17:49] Storing 10000 Traits (1084646/290000/413022)
[INFO] [2023-03-07 12:17:53] Storing 20000 MetaTraits (1084646/290000/413022)
[INFO] [2023-03-07 12:18:02] Storing 7510 TraitsReferences (1122156/300000/413022)
[INFO] [2023-03-07 12:18:02] Storing 10000 Traits (1122156/300000/413022)
[INFO] [2023-03-07 12:18:05] Storing 20000 MetaTraits (1122156/300000/413022)
[INFO] [2023-03-07 12:18:14] Storing 7345 TraitsReferences (1159501/310000/413022)
[INFO] [2023-03-07 12:18:15] Storing 10000 Traits (1159501/310000/413022)
[INFO] [2023-03-07 12:18:18] Storing 20000 MetaTraits (1159501/310000/413022)
[INFO] [2023-03-07 12:18:27] Storing 7392 TraitsReferences (1196893/320000/413022)
[INFO] [2023-03-07 12:18:28] Storing 10000 Traits (1196893/320000/413022)
[INFO] [2023-03-07 12:18:31] Storing 20000 MetaTraits (1196893/320000/413022)
[INFO] [2023-03-07 12:18:40] Storing 10000 Traits (1234101/330000/413022)
[INFO] [2023-03-07 12:18:44] Storing 20000 MetaTraits (1234101/330000/413022)
[INFO] [2023-03-07 12:18:46] Storing 7208 TraitsReferences (1234101/330000/413022)
[INFO] [2023-03-07 12:18:53] Storing 10000 Traits (1271453/340000/413022)
[INFO] [2023-03-07 12:18:56] Storing 20000 MetaTraits (1271453/340000/413022)
[INFO] [2023-03-07 12:18:59] Storing 7352 TraitsReferences (1271453/340000/413022)
[INFO] [2023-03-07 12:19:06] Storing 10000 Traits (1308759/350000/413022)
[INFO] [2023-03-07 12:19:09] Storing 20000 MetaTraits (1308759/350000/413022)
[INFO] [2023-03-07 12:19:12] Storing 7306 TraitsReferences (1308759/350000/413022)
[INFO] [2023-03-07 12:19:19] Storing 10000 Traits (1346155/360000/413022)
[INFO] [2023-03-07 12:19:22] Storing 20000 MetaTraits (1346155/360000/413022)
[INFO] [2023-03-07 12:19:25] Storing 7396 TraitsReferences (1346155/360000/413022)
[INFO] [2023-03-07 12:19:32] Storing 10000 Traits (1383630/370000/413022)
[INFO] [2023-03-07 12:19:35] Storing 20000 MetaTraits (1383630/370000/413022)
[INFO] [2023-03-07 12:19:38] Storing 7475 TraitsReferences (1383630/370000/413022)
[INFO] [2023-03-07 12:19:45] Storing 7420 TraitsReferences (1421050/380000/413022)
[INFO] [2023-03-07 12:19:46] Storing 10000 Traits (1421050/380000/413022)
[INFO] [2023-03-07 12:19:49] Storing 20000 MetaTraits (1421050/380000/413022)
[INFO] [2023-03-07 12:19:58] Storing 7313 TraitsReferences (1458363/390000/413022)
[INFO] [2023-03-07 12:19:59] Storing 10000 Traits (1458363/390000/413022)
[INFO] [2023-03-07 12:20:03] Storing 20000 MetaTraits (1458363/390000/413022)
[INFO] [2023-03-07 12:20:13] Storing 10000 Traits (1495902/400000/413022)
[INFO] [2023-03-07 12:20:16] Storing 20000 MetaTraits (1495902/400000/413022)
[INFO] [2023-03-07 12:20:19] Storing 7539 TraitsReferences (1495902/400000/413022)
[INFO] [2023-03-07 12:20:26] Storing 10000 Traits (1533249/410000/413022)
[INFO] [2023-03-07 12:20:29] Storing 20000 MetaTraits (1533249/410000/413022)
[INFO] [2023-03-07 12:20:31] Storing 7347 TraitsReferences (1533249/410000/413022)
[INFO] [2023-03-07 12:20:35] Storing 3021 Traits (1544531/413020/413022)
[INFO] [2023-03-07 12:20:36] Storing 6042 MetaTraits (1544531/413020/413022)
[INFO] [2023-03-07 12:20:37] Storing 2219 TraitsReferences (1544531/413020/413022)
[STOP] [2023-03-07 12:20:37] parse_diff_and_store
[START] [2023-03-07 12:20:37] resolve_keys
[2023-03-07 12:21:03] Resolving downloaded urls (this is not actually downloading them yet)
[INFO] [2023-03-07 12:21:14] Occurrences to nodes (through scientific_names)...
[INFO] [2023-03-07 12:21:32] traits to occurrences...
[INFO] [2023-03-07 12:21:48] traits to nodes (through occurrences)...
[INFO] [2023-03-07 12:22:01] Traits to sex term...
[INFO] [2023-03-07 12:22:10] Traits to lifestage term...
[INFO] [2023-03-07 12:22:18] MetaTraits to traits...
[INFO] [2023-03-07 12:22:38] MetaTraits (simple, measurement row refers to parent) to traits...
[INFO] [2023-03-07 12:22:57] Assocs to occurrences...
[INFO] [2023-03-07 12:22:57] Assocs to nodes...
[INFO] [2023-03-07 12:22:57] Assoc to sex term...
[INFO] [2023-03-07 12:22:57] Assoc to lifestage term...
[INFO] [2023-03-07 12:22:57] MetaAssoc to assocs...
[STOP] [2023-03-07 12:22:57] resolve_keys
[START] [2023-03-07 12:22:57] hold_for_later_1
[STOP] [2023-03-07 12:22:57] hold_for_later_1
[START] [2023-03-07 12:22:57] hold_for_later_2
[STOP] [2023-03-07 12:22:57] hold_for_later_2
[START] [2023-03-07 12:22:57] resolve_missing_parents
[STOP] [2023-03-07 12:22:57] resolve_missing_parents
[START] [2023-03-07 12:22:57] rebuild_nodes
[START] [2023-03-07 12:22:58] Flattener#flatten
[START] [2023-03-07 12:22:58] Flattener#study_resource
[START] [2023-03-07 12:22:58] Flattener#build_ancestry
[STOP] [2023-03-07 12:23:16] Flattener#build_ancestry
[INFO] [2023-03-07 12:23:16] 70159 ancestry keys
[START] [2023-03-07 12:23:16] build_node_ancestors
[INFO] [2023-03-07 12:23:16] old ancestors deleted.
[STOP] [2023-03-07 12:23:36] build_node_ancestors
[START] [2023-03-07 12:23:37] Flattener#propagate_ancestor_ids
[STOP] [2023-03-07 12:23:42] Flattener#propagate_ancestor_ids
[STOP] [2023-03-07 12:23:42] Flattener#flatten
[STOP] [2023-03-07 12:23:42] rebuild_nodes
[START] [2023-03-07 12:23:42] resolve_missing_media_owners
[STOP] [2023-03-07 12:23:42] resolve_missing_media_owners
[START] [2023-03-07 12:23:42] sanitize_media_verbatims
[STOP] [2023-03-07 12:23:42] sanitize_media_verbatims
[START] [2023-03-07 12:23:42] queue_downloads
[STOP] [2023-03-07 12:23:42] queue_downloads
[START] [2023-03-07 12:23:42] parse_names
[WARN] [2023-03-07 12:23:42] I see 133626 names which still need to be parsed.
[WARN] [2023-03-07 12:23:43] Names to parse: 10000 formatted: 10000 learned: 9988 parsed: 10000
[WARN] [2023-03-07 12:23:51] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2023-03-07 12:23:58] Names to parse: 10000 formatted: 10000 learned: 9976 parsed: 10000
[WARN] [2023-03-07 12:24:05] Names to parse: 10000 formatted: 10000 learned: 9981 parsed: 10000
[WARN] [2023-03-07 12:24:13] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2023-03-07 12:24:21] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2023-03-07 12:24:29] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2023-03-07 12:24:36] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2023-03-07 12:24:43] Names to parse: 10000 formatted: 10000 learned: 9998 parsed: 10000
[WARN] [2023-03-07 12:24:51] Names to parse: 10000 formatted: 10000 learned: 9981 parsed: 10000
[WARN] [2023-03-07 12:24:58] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2023-03-07 12:25:06] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2023-03-07 12:25:14] Names to parse: 10000 formatted: 10000 learned: 9999 parsed: 10000
[WARN] [2023-03-07 12:25:20] Names to parse: 3626 formatted: 3626 learned: 3626 parsed: 3626
[STOP] [2023-03-07 12:25:24] parse_names
[START] [2023-03-07 12:25:24] denormalize_canonical_names_to_nodes
[STOP] [2023-03-07 12:25:26] denormalize_canonical_names_to_nodes
[START] [2023-03-07 12:25:26] match_nodes
[START] [2023-03-07 12:25:26] map_all_nodes_to_pages
[STOP] [2023-03-07 12:42:36] map_all_nodes_to_pages
[INFO] [2023-03-07 12:42:36] 11792 Unmatched nodes (of 70159)! That's too many to output. Full list in /app/public/data/brazilian_flora_/unmatched_nodes.txt ; First 10: Canonical: Cyperaceae; Node#124963298; ResourceID: 100; Canonical: Cyperus mundtii; Node#124988440; ResourceID: 133952; Canonical: Cyperus cylindricus; Node#125011100; ResourceID: 49438; Canonical: Cyperus; Node#125018770; ResourceID: 7171; Canonical: Bisboeckelera; Node#125006016; ResourceID: 34873; Canonical: Carex; Node#125018750; ResourceID: 7161; Canonical: Chlorocyperus; Node#124967002; ResourceID: Plantae/Cyperaceae/Chlorocyperus; Canonical: Rhynchospora umbraticola; Node#124998106; ResourceID: 22628; Canonical: Rhynchospora elatior hirsuta; Node#125002136; ResourceID: 28551; Canonical: Rhynchospora; Node#125018949; ResourceID: 7258
[START] [2023-03-07 12:42:36] update_nodes
[STOP] [2023-03-07 12:43:14] update_nodes
[STOP] [2023-03-07 12:43:14] match_nodes
[START] [2023-03-07 12:43:14] reindex_search
[STOP] [2023-03-07 12:44:14] reindex_search
[START] [2023-03-07 12:44:14] normalize_units
[STOP] [2023-03-07 12:44:14] normalize_units
[START] [2023-03-07 12:44:14] calculate_statistics
[INFO] [2023-03-07 12:44:21] Duplicate page_id count: 0
[STOP] [2023-03-07 12:44:22] calculate_statistics
[START] [2023-03-07 12:44:22] complete_harvest_instance
[START] [2023-03-07 12:44:22] overall_tsv_creation
[INFO] [2023-03-07 12:44:22] Exporting 70159 nodes as TSV in batches of 10000...
[INFO] [2023-03-07 12:44:22] Processing group of 70159 in 8 batches of 10000
[INFO] [2023-03-07 12:44:43] 46729 Traits (unfiltered) and 0 associations...
[INFO] [2023-03-07 12:44:43] Building Traits map for 10000 nodes (this can take a while)...
[INFO] [2023-03-07 12:45:39] Mapped 46729 traits (93458 meta) for 10000 nodes.
[INFO] [2023-03-07 12:45:39] Building Associations map (this can take a while)...
[INFO] [2023-03-07 12:45:42] Done. 0 assocs mapped (0 meta).
[INFO] [2023-03-07 12:45:42] Adding 46729 traits...
[INFO] [2023-03-07 12:45:51] 36563 metadata added.
[INFO] [2023-03-07 12:45:51] Adding 0 assocs...
[INFO] [2023-03-07 12:45:51] 0 metadata added.
[INFO] [2023-03-07 12:46:47] Processed 10000/70159 nodes
[INFO] [2023-03-07 12:47:17] 38149 Traits (unfiltered) and 0 associations...
[INFO] [2023-03-07 12:47:17] Building Traits map for 10000 nodes (this can take a while)...
[INFO] [2023-03-07 12:48:05] Mapped 38149 traits (76298 meta) for 10000 nodes.
[INFO] [2023-03-07 12:48:05] Building Associations map (this can take a while)...
[INFO] [2023-03-07 12:48:08] Done. 0 assocs mapped (0 meta).
[INFO] [2023-03-07 12:48:08] Adding 38149 traits...
[INFO] [2023-03-07 12:48:13] 27649 metadata added.
[INFO] [2023-03-07 12:48:13] Adding 0 assocs...
[INFO] [2023-03-07 12:48:13] 0 metadata added.
[INFO] [2023-03-07 12:49:10] Processed 20000/70159 nodes
[INFO] [2023-03-07 12:49:44] 54930 Traits (unfiltered) and 0 associations...
[INFO] [2023-03-07 12:49:44] Building Traits map for 10000 nodes (this can take a while)...
[INFO] [2023-03-07 12:50:55] Mapped 54930 traits (109860 meta) for 10000 nodes.
[INFO] [2023-03-07 12:50:55] Building Associations map (this can take a while)...
[INFO] [2023-03-07 12:50:57] Done. 0 assocs mapped (0 meta).
[INFO] [2023-03-07 12:50:57] Adding 54930 traits...
[INFO] [2023-03-07 12:51:07] 33015 metadata added.
[INFO] [2023-03-07 12:51:07] Adding 0 assocs...
[INFO] [2023-03-07 12:51:07] 0 metadata added.
[INFO] [2023-03-07 12:52:05] Processed 30000/70159 nodes
[INFO] [2023-03-07 12:52:45] 91514 Traits (unfiltered) and 0 associations...
[INFO] [2023-03-07 12:52:45] Building Traits map for 10000 nodes (this can take a while)...
[INFO] [2023-03-07 12:54:58] Mapped 91514 traits (183028 meta) for 10000 nodes.
[INFO] [2023-03-07 12:54:58] Building Associations map (this can take a while)...
[INFO] [2023-03-07 12:55:00] Done. 0 assocs mapped (0 meta).
[INFO] [2023-03-07 12:55:00] Adding 91514 traits...
[INFO] [2023-03-07 12:55:14] 59047 metadata added.
[INFO] [2023-03-07 12:55:14] Adding 0 assocs...
[INFO] [2023-03-07 12:55:14] 0 metadata added.
[INFO] [2023-03-07 12:56:24] Processed 40000/70159 nodes
[INFO] [2023-03-07 12:56:58] 60210 Traits (unfiltered) and 0 associations...
[INFO] [2023-03-07 12:56:58] Building Traits map for 10000 nodes (this can take a while)...
[INFO] [2023-03-07 12:58:26] Mapped 60210 traits (120420 meta) for 10000 nodes.
[INFO] [2023-03-07 12:58:26] Building Associations map (this can take a while)...
[INFO] [2023-03-07 12:58:29] Done. 0 assocs mapped (0 meta).
[INFO] [2023-03-07 12:58:29] Adding 60210 traits...
[INFO] [2023-03-07 12:58:38] 47200 metadata added.
[INFO] [2023-03-07 12:58:38] Adding 0 assocs...
[INFO] [2023-03-07 12:58:38] 0 metadata added.
[INFO] [2023-03-07 12:59:36] Processed 50000/70159 nodes
[INFO] [2023-03-07 13:00:17] 59533 Traits (unfiltered) and 0 associations...
[INFO] [2023-03-07 13:00:17] Building Traits map for 10000 nodes (this can take a while)...
[INFO] [2023-03-07 13:01:40] Mapped 59533 traits (119066 meta) for 10000 nodes.
[INFO] [2023-03-07 13:01:40] Building Associations map (this can take a while)...
[INFO] [2023-03-07 13:01:42] Done. 0 assocs mapped (0 meta).
[INFO] [2023-03-07 13:01:42] Adding 59533 traits...
[INFO] [2023-03-07 13:01:51] 43896 metadata added.
[INFO] [2023-03-07 13:01:51] Adding 0 assocs...
[INFO] [2023-03-07 13:01:51] 0 metadata added.
[INFO] [2023-03-07 13:02:54] Processed 60000/70159 nodes
[INFO] [2023-03-07 13:03:26] 58778 Traits (unfiltered) and 0 associations...
[INFO] [2023-03-07 13:03:26] Building Traits map for 10000 nodes (this can take a while)...
[INFO] [2023-03-07 13:04:46] Mapped 58778 traits (117556 meta) for 10000 nodes.
[INFO] [2023-03-07 13:04:46] Building Associations map (this can take a while)...
[INFO] [2023-03-07 13:04:48] Done. 0 assocs mapped (0 meta).
[INFO] [2023-03-07 13:04:48] Adding 58778 traits...
[INFO] [2023-03-07 13:05:03] 55722 metadata added.
[INFO] [2023-03-07 13:05:03] Adding 0 assocs...
[INFO] [2023-03-07 13:05:03] 0 metadata added.
[INFO] [2023-03-07 13:06:03] Processed 70000/70159 nodes
[INFO] [2023-03-07 13:06:04] 933 Traits (unfiltered) and 0 associations...
[INFO] [2023-03-07 13:06:04] Building Traits map for 159 nodes (this can take a while)...
[INFO] [2023-03-07 13:06:04] Mapped 933 traits (1866 meta) for 159 nodes.
[INFO] [2023-03-07 13:06:04] Building Associations map (this can take a while)...
[INFO] [2023-03-07 13:06:04] Done. 0 assocs mapped (0 meta).
[INFO] [2023-03-07 13:06:04] Adding 933 traits...
[INFO] [2023-03-07 13:06:04] 596 metadata added.
[INFO] [2023-03-07 13:06:04] Adding 0 assocs...
[INFO] [2023-03-07 13:06:04] 0 metadata added.
[INFO] [2023-03-07 13:06:51] Processed 70159/70159 nodes
[INFO] [2023-03-07 13:06:51] Average Time: 161.324
[INFO] [2023-03-07 13:06:51] Total Time: 22m30s
[INFO] [2023-03-07 13:06:51] last 3 / first 3: 0.93
[INFO] [2023-03-07 13:06:51] Std.Dev: 57.764; Max: 250.21
[STOP] [2023-03-07 13:06:51] overall_tsv_creation
[INFO] [2023-03-07 13:06:51] Done. Check your files:
[INFO] [2023-03-07 13:06:51] (70159 lines) /app/public/data/brazilian_flora_/publish_nodes.tsv
[INFO] [2023-03-07 13:06:51] (207032 lines) /app/public/data/brazilian_flora_/publish_node_ancestors.tsv
[INFO] [2023-03-07 13:06:51] (116838 lines) /app/public/data/brazilian_flora_/publish_scientific_names.tsv
[INFO] [2023-03-07 13:06:51] (8548 lines) /app/public/data/brazilian_flora_/publish_vernaculars.tsv
[INFO] [2023-03-07 13:06:52] (410777 lines) /app/public/data/brazilian_flora_/publish_traits.tsv
[INFO] [2023-03-07 13:06:52] (303689 lines) /app/public/data/brazilian_flora_/publish_metadata.tsv
[STOP] [2023-03-07 13:06:52] complete_harvest_instance
[START] [2023-03-07 13:06:52] completed
[STOP] [2023-03-07 13:06:52] completed
[STOP] [2023-03-07 13:06:52] logged process, took 3598.04
Latest Process