Harvest for IUCN Created 05 Jan 10:39

Harvesting Log

[INFO] [2022-01-05 10:39:51] Created harvest instance #4088
[STOP] [2022-01-05 10:39:51] create_harvest_instance
[START] [2022-01-05 10:39:51] fetch_files
[STOP] [2022-01-05 10:39:51] fetch_files
[START] [2022-01-05 10:39:51] validate_each_file
[INFO] [2022-01-05 10:39:51] Looping over 3 formats...
[INFO] [2022-01-05 10:39:51] ...nodes (/app/public/data/IUCN-SD/taxon.tab)
[INFO] [2022-01-05 10:39:56] Valid: /app/public/converted_csv/IUCN-SD_nodes_4088.csv (127110 lines)
[INFO] [2022-01-05 10:39:56] ...occurrences (/app/public/data/IUCN-SD/occurrence.tab)
[INFO] [2022-01-05 10:40:02] Valid: /app/public/converted_csv/IUCN-SD_occurrences_4088.csv (385458 lines)
[INFO] [2022-01-05 10:40:02] ...measurements (/app/public/data/IUCN-SD/measurement_or_fact.tab)
[INFO] [2022-01-05 10:40:39] Valid: /app/public/converted_csv/IUCN-SD_measurements_4088.csv (990810 lines)
[STOP] [2022-01-05 10:40:39] validate_each_file
[START] [2022-01-05 10:40:39] convert_to_csv
[INFO] [2022-01-05 10:40:39] Looping over 3 formats...
[INFO] [2022-01-05 10:40:39] ...nodes (/app/public/data/IUCN-SD/taxon.tab)
[CMD] [2022-01-05 10:40:39] /usr/bin/sort /app/public/converted_csv/IUCN-SD_nodes_4088.csv > /app/public/converted_csv/IUCN-SD_nodes_4088.csv_sorted
[INFO] [2022-01-05 10:40:39] Converted: /app/public/converted_csv/IUCN-SD_nodes_4088.csv (127110 lines)
[INFO] [2022-01-05 10:40:39] ...occurrences (/app/public/data/IUCN-SD/occurrence.tab)
[CMD] [2022-01-05 10:40:39] /usr/bin/sort /app/public/converted_csv/IUCN-SD_occurrences_4088.csv > /app/public/converted_csv/IUCN-SD_occurrences_4088.csv_sorted
[INFO] [2022-01-05 10:40:40] Converted: /app/public/converted_csv/IUCN-SD_occurrences_4088.csv (385458 lines)
[INFO] [2022-01-05 10:40:40] ...measurements (/app/public/data/IUCN-SD/measurement_or_fact.tab)
[CMD] [2022-01-05 10:40:40] /usr/bin/sort /app/public/converted_csv/IUCN-SD_measurements_4088.csv > /app/public/converted_csv/IUCN-SD_measurements_4088.csv_sorted
[INFO] [2022-01-05 10:40:41] Converted: /app/public/converted_csv/IUCN-SD_measurements_4088.csv (990810 lines)
[STOP] [2022-01-05 10:40:41] convert_to_csv
[START] [2022-01-05 10:40:41] calculate_delta
[INFO] [2022-01-05 10:40:41] Looping over 3 formats...
[INFO] [2022-01-05 10:40:41] ...nodes (/app/public/data/IUCN-SD/taxon.tab)
[CMD] [2022-01-05 10:40:41] echo "0a" > /app/public/diff/IUCN-SD_nodes_4088.diff
[CMD] [2022-01-05 10:40:41] tail -n +1 /app/public/converted_csv/IUCN-SD_nodes_4088.csv >> /app/public/diff/IUCN-SD_nodes_4088.diff
[CMD] [2022-01-05 10:40:41] echo "." >> /app/public/diff/IUCN-SD_nodes_4088.diff
[INFO] [2022-01-05 10:40:41] Created diff: /app/public/diff/IUCN-SD_nodes_4088.diff (127112 lines)
[INFO] [2022-01-05 10:40:41] ...occurrences (/app/public/data/IUCN-SD/occurrence.tab)
[CMD] [2022-01-05 10:40:41] echo "0a" > /app/public/diff/IUCN-SD_occurrences_4088.diff
[CMD] [2022-01-05 10:40:41] tail -n +1 /app/public/converted_csv/IUCN-SD_occurrences_4088.csv >> /app/public/diff/IUCN-SD_occurrences_4088.diff
[CMD] [2022-01-05 10:40:41] echo "." >> /app/public/diff/IUCN-SD_occurrences_4088.diff
[INFO] [2022-01-05 10:40:41] Created diff: /app/public/diff/IUCN-SD_occurrences_4088.diff (385460 lines)
[INFO] [2022-01-05 10:40:41] ...measurements (/app/public/data/IUCN-SD/measurement_or_fact.tab)
[CMD] [2022-01-05 10:40:41] echo "0a" > /app/public/diff/IUCN-SD_measurements_4088.diff
[CMD] [2022-01-05 10:40:41] tail -n +1 /app/public/converted_csv/IUCN-SD_measurements_4088.csv >> /app/public/diff/IUCN-SD_measurements_4088.diff
[CMD] [2022-01-05 10:40:42] echo "." >> /app/public/diff/IUCN-SD_measurements_4088.diff
[INFO] [2022-01-05 10:40:42] Created diff: /app/public/diff/IUCN-SD_measurements_4088.diff (990812 lines)
[STOP] [2022-01-05 10:40:42] calculate_delta
[START] [2022-01-05 10:40:42] parse_diff_and_store
[INFO] [2022-01-05 10:40:42] Handling diff: /app/public/diff/IUCN-SD_nodes_4088.diff (127112 lines)
[INFO] [2022-01-05 10:40:42] Loading nodes diff file into memory (127112 /app/public/diff/IUCN-SD_nodes_4088.diff lines)...
[WARN] [2022-01-05 10:40:43] Filtered Scientific Name `Stictophaula coco  Gorochov & Voltshenkova, 2009` to `Stictophaula coco Gorochov & Voltshenkova, 2009`
[WARN] [2022-01-05 10:40:43] Filtered Scientific Name `Lamellidea microstoma (Quadras  and von Möllendorff, 1894)` to `Lamellidea microstoma (Quadras and von Möllendorff, 1894)`
[WARN] [2022-01-05 10:40:43] Filtered Scientific Name `Lamprocystis denticulata (Quadras  and von Möllendorff, 1894)` to `Lamprocystis denticulata (Quadras and von Möllendorff, 1894)`
[WARN] [2022-01-05 10:40:44] Filtered Scientific Name `Diplolophium buchananii (Benth. ex Oliv.)  C.Norman` to `Diplolophium buchananii (Benth. ex Oliv.) C.Norman`
[WARN] [2022-01-05 10:40:44] Filtered Scientific Name `Bothriocline moramballae (Oliv. & Hiern)  O.Hoffm.` to `Bothriocline moramballae (Oliv. & Hiern) O.Hoffm.`
[WARN] [2022-01-05 10:40:47] Filtered Scientific Name `Latopotamon xuanweiense Naruse, Yeo &  Zhou, 2008` to `Latopotamon xuanweiense Naruse, Yeo & Zhou, 2008`
[WARN] [2022-01-05 10:40:47] Filtered Scientific Name `Parathelphusa linduensis  (Roux, 1904)` to `Parathelphusa linduensis (Roux, 1904)`
[WARN] [2022-01-05 10:40:47] Filtered Scientific Name `Snaha aruna  Bahir & Yeo, 2007` to `Snaha aruna Bahir & Yeo, 2007`
[WARN] [2022-01-05 10:40:47] Filtered Scientific Name `Sinopotamon styxum  Dai, 1990` to `Sinopotamon styxum Dai, 1990`
[WARN] [2022-01-05 10:40:47] Filtered Scientific Name `Oncorhynchus nerka TRANSBOUNDARY FJORDS: Petersburg Lk/N Kupreanof Is (Walbaum, 1792)` to `Oncorhynchus nerka TRANSBOUNDARY FJORDS: Petersburg LkN Kupreanof Is (Walbaum, 1792)`
[WARN] [2022-01-05 10:40:47] Filtered Scientific Name `Oncorhynchus nerka COLUMBIA RIVER: Okanogan R/Osoyoos Lk (Walbaum, 1792)` to `Oncorhynchus nerka COLUMBIA RIVER: Okanogan ROsoyoos Lk (Walbaum, 1792)`
[WARN] [2022-01-05 10:40:47] Filtered Scientific Name `Oncorhynchus nerka PUGET SOUND-GEORGIA BASIN: Baker Lk/Puget Sound (Walbaum, 1792)` to `Oncorhynchus nerka PUGET SOUND-GEORGIA BASIN: Baker LkPuget Sound (Walbaum, 1792)`
[WARN] [2022-01-05 10:40:47] Filtered Scientific Name `Oncorhynchus nerka PUGET SOUND-GEORGIA BASIN: Schoen Ck/Davie R (Walbaum, 1792)` to `Oncorhynchus nerka PUGET SOUND-GEORGIA BASIN: Schoen CkDavie R (Walbaum, 1792)`
[WARN] [2022-01-05 10:40:47] Filtered Scientific Name `Oncorhynchus nerka NASS-SKEENA ESTUARY: Hugh Smith Lk/Boca de Quadra (Walbaum, 1792)` to `Oncorhynchus nerka NASS-SKEENA ESTUARY: Hugh Smith LkBoca de Quadra (Walbaum, 1792)`
[WARN] [2022-01-05 10:40:47] Filtered Scientific Name `Oncorhynchus nerka BERING SLOPE/KAMCHATKA CURRENTS (Walbaum, 1792)` to `Oncorhynchus nerka BERING SLOPEKAMCHATKA CURRENTS (Walbaum, 1792)`
[WARN] [2022-01-05 10:40:47] Filtered Scientific Name `Oncorhynchus nerka HECATE STRAIT-Q.C. SOUND: Lowe Lk/Granville Ch (Walbaum, 1792)` to `Oncorhynchus nerka HECATE STRAIT-Q.C. SOUND: Lowe LkGranville Ch (Walbaum, 1792)`
[WARN] [2022-01-05 10:40:47] Filtered Scientific Name `Oncorhynchus nerka COLUMBIA RIVER: Suttle Lk/Deschutes R (Walbaum, 1792)` to `Oncorhynchus nerka COLUMBIA RIVER: Suttle LkDeschutes R (Walbaum, 1792)`
[WARN] [2022-01-05 10:40:47] Filtered Scientific Name `Oncorhynchus nerka PUGET SOUND-GEORGIA BASIN: Glendale Ck/Knight Inl (Walbaum, 1792)` to `Oncorhynchus nerka PUGET SOUND-GEORGIA BASIN: Glendale CkKnight Inl (Walbaum, 1792)`
[WARN] [2022-01-05 10:40:47] Filtered Scientific Name `Oncorhynchus nerka SKEENA R, LOWER: McDonnel Lk/Zymoetz R (Walbaum, 1792)` to `Oncorhynchus nerka SKEENA R, LOWER: McDonnel LkZymoetz R (Walbaum, 1792)`
[WARN] [2022-01-05 10:40:47] Filtered Scientific Name `Oncorhynchus nerka TRANSBOUNDARY FJORDS: Kah Sheets Lk/S Kupreanof Is (Walbaum, 1792)` to `Oncorhynchus nerka TRANSBOUNDARY FJORDS: Kah Sheets LkS Kupreanof Is (Walbaum, 1792)`
[WARN] [2022-01-05 10:40:47] Filtered Scientific Name `Oncorhynchus nerka TRANSBOUNDARY FJORDS: Shipley Lk/N Kosciusko Is (Walbaum, 1792)` to `Oncorhynchus nerka TRANSBOUNDARY FJORDS: Shipley LkN Kosciusko Is (Walbaum, 1792)`
[WARN] [2022-01-05 10:40:47] Filtered Scientific Name `Oncorhynchus nerka TRANSBOUNDARY FJORDS: Kunk Lk/E Etolin Is (Walbaum, 1792)` to `Oncorhynchus nerka TRANSBOUNDARY FJORDS: Kunk LkE Etolin Is (Walbaum, 1792)`
[WARN] [2022-01-05 10:40:47] Filtered Scientific Name `Oncorhynchus nerka COLUMBIA RIVER: Upper, Headwater/Arrow, Whatsan Lk (Walbaum, 1792)` to `Oncorhynchus nerka COLUMBIA RIVER: Upper, HeadwaterArrow, Whatsan Lk (Walbaum, 1792)`
[WARN] [2022-01-05 10:40:47] Filtered Scientific Name `Oncorhynchus nerka SKEENA R, MIDDLE: Nanika Lk/Morice R (Walbaum, 1792)` to `Oncorhynchus nerka SKEENA R, MIDDLE: Nanika LkMorice R (Walbaum, 1792)`
[WARN] [2022-01-05 10:40:47] Filtered Scientific Name `Oncorhynchus nerka PUGET SOUND-GEORGIA BASIN: Quatse R/N Vancouver I (Walbaum, 1792)` to `Oncorhynchus nerka PUGET SOUND-GEORGIA BASIN: Quatse RN Vancouver I (Walbaum, 1792)`
[WARN] [2022-01-05 10:40:48] Filtered Scientific Name `Brachycephalus ferruginus Alves, Ribeiro, Haddad  & dos Reis, 2006` to `Brachycephalus ferruginus Alves, Ribeiro, Haddad & dos Reis, 2006`
[WARN] [2022-01-05 10:40:48] Filtered Scientific Name `Megophrys synoria (Stuart, Sok &  Neang, 2006)` to `Megophrys synoria (Stuart, Sok & Neang, 2006)`
[WARN] [2022-01-05 10:40:48] Filtered Scientific Name `Madhuca glabrescens  H.J.Lam` to `Madhuca glabrescens H.J.Lam`
[WARN] [2022-01-05 10:40:48] Filtered Scientific Name `Colobus guereza ssp. occidentalis  (de Rochebrune, 1887)` to `Colobus guereza ssp. occidentalis (de Rochebrune, 1887)`
[WARN] [2022-01-05 10:40:49] (Reached filtered-name limit; supressing further warnings.)
[INFO] [2022-01-05 10:41:30] Handling diff: /app/public/diff/IUCN-SD_occurrences_4088.diff (385460 lines)
[INFO] [2022-01-05 10:41:30] Loading occurrences diff file into memory (385460 /app/public/diff/IUCN-SD_occurrences_4088.diff lines)...
[INFO] [2022-01-05 10:42:19] Handling diff: /app/public/diff/IUCN-SD_measurements_4088.diff (990812 lines)
[INFO] [2022-01-05 10:42:19] Loading measurements diff file into memory (990812 /app/public/diff/IUCN-SD_measurements_4088.diff lines)...
[INFO] [2022-01-05 10:50:14] Storing 129954 ScientificNames
[INFO] [2022-01-05 10:50:14] Processing group of 129954 in 130 groups of 1000
[INFO] [2022-01-05 10:50:53] Average Time: 0.291
[INFO] [2022-01-05 10:50:53] Total Time: 39s
[INFO] [2022-01-05 10:50:53] last 3 / first 3: 0.74
[INFO] [2022-01-05 10:50:53] Std.Dev: 0.05477225575051661; Max: 0.48
[INFO] [2022-01-05 10:50:53] Storing 129954 Nodes
[INFO] [2022-01-05 10:50:53] Processing group of 129954 in 130 groups of 1000
[INFO] [2022-01-05 10:51:53] Average Time: 0.456
[INFO] [2022-01-05 10:51:53] Total Time: 1m1s
[INFO] [2022-01-05 10:51:53] last 3 / first 3: 0.86
[INFO] [2022-01-05 10:51:53] Std.Dev: 1.0936178491593853; Max: 11.38
[INFO] [2022-01-05 10:51:53] Storing 385458 Occurrences
[INFO] [2022-01-05 10:51:53] Processing group of 385458 in 386 groups of 1000
[INFO] [2022-01-05 10:53:17] Average Time: 0.213
[INFO] [2022-01-05 10:53:17] Total Time: 1m25s
[INFO] [2022-01-05 10:53:17] last 3 / first 3: 0.49
[INFO] [2022-01-05 10:53:17] Std.Dev: 0.3987480407475377; Max: 5.8
[INFO] [2022-01-05 10:53:17] Storing 990810 Traits
[INFO] [2022-01-05 10:53:17] Processing group of 990810 in 991 groups of 1000
[INFO] [2022-01-05 11:00:27] Average Time: 0.427
[INFO] [2022-01-05 11:00:27] Total Time: 7m10s
[INFO] [2022-01-05 11:00:27] last 3 / first 3: 1.18
[INFO] [2022-01-05 11:00:27] Std.Dev: 0.5412947441089743; Max: 5.12
[INFO] [2022-01-05 11:00:27] Storing 70618 MetaTraits
[INFO] [2022-01-05 11:00:27] Processing group of 70618 in 71 groups of 1000
[INFO] [2022-01-05 11:00:54] Average Time: 0.362
[INFO] [2022-01-05 11:00:54] Total Time: 27s
[INFO] [2022-01-05 11:00:54] last 3 / first 3: 0.56
[INFO] [2022-01-05 11:00:54] Std.Dev: 0.6107372593840988; Max: 2.67
[STOP] [2022-01-05 11:00:54] parse_diff_and_store
[START] [2022-01-05 11:00:54] resolve_keys
[INFO] [2022-01-05 11:02:12] Occurrences to nodes (through scientific_names)...
[INFO] [2022-01-05 11:02:22] traits to occurrences...
[INFO] [2022-01-05 11:02:39] traits to nodes (through occurrences)...
[INFO] [2022-01-05 11:02:48] Traits to sex term...
[INFO] [2022-01-05 11:02:57] Traits to lifestage term...
[INFO] [2022-01-05 11:03:04] MetaTraits to traits...
[INFO] [2022-01-05 11:03:06] MetaTraits (simple, measurement row refers to parent) to traits...
[INFO] [2022-01-05 11:03:27] Assocs to occurrences...
[INFO] [2022-01-05 11:03:27] Assocs to nodes...
[INFO] [2022-01-05 11:03:27] Assoc to sex term...
[INFO] [2022-01-05 11:03:27] Assoc to lifestage term...
[INFO] [2022-01-05 11:03:27] MetaAssoc to assocs...
[STOP] [2022-01-05 11:03:27] resolve_keys
[START] [2022-01-05 11:03:27] hold_for_later_1
[STOP] [2022-01-05 11:03:27] hold_for_later_1
[START] [2022-01-05 11:03:27] hold_for_later_2
[STOP] [2022-01-05 11:03:27] hold_for_later_2
[START] [2022-01-05 11:03:27] resolve_missing_parents
[STOP] [2022-01-05 11:03:35] resolve_missing_parents
[START] [2022-01-05 11:03:35] rebuild_nodes
[START] [2022-01-05 11:03:35] Flattener#flatten
[START] [2022-01-05 11:03:35] Flattener#study_resource
[START] [2022-01-05 11:03:36] Flattener#build_ancestry
[STOP] [2022-01-05 11:07:01] Flattener#build_ancestry
[INFO] [2022-01-05 11:07:01] 129954 ancestry keys
[START] [2022-01-05 11:07:01] build_node_ancestors
[INFO] [2022-01-05 11:07:01] old ancestors deleted.
[STOP] [2022-01-05 11:08:19] build_node_ancestors
[START] [2022-01-05 11:08:22] Flattener#propagate_ancestor_ids
[STOP] [2022-01-05 11:08:37] Flattener#propagate_ancestor_ids
[STOP] [2022-01-05 11:08:37] Flattener#flatten
[STOP] [2022-01-05 11:08:37] rebuild_nodes
[START] [2022-01-05 11:08:37] resolve_missing_media_owners
[STOP] [2022-01-05 11:08:37] resolve_missing_media_owners
[START] [2022-01-05 11:08:37] sanitize_media_verbatims
[STOP] [2022-01-05 11:08:37] sanitize_media_verbatims
[START] [2022-01-05 11:08:37] queue_downloads
[STOP] [2022-01-05 11:08:37] queue_downloads
[START] [2022-01-05 11:08:37] parse_names
[WARN] [2022-01-05 11:08:37] I see 129954 names which still need to be parsed.
[WARN] [2022-01-05 11:08:39] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2022-01-05 11:08:47] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2022-01-05 11:08:54] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2022-01-05 11:09:02] Names to parse: 10000 formatted: 10000 learned: 9997 parsed: 10000
[WARN] [2022-01-05 11:09:10] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2022-01-05 11:09:18] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2022-01-05 11:09:25] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2022-01-05 11:09:33] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2022-01-05 11:09:41] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2022-01-05 11:09:49] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2022-01-05 11:09:57] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2022-01-05 11:10:05] Names to parse: 10000 formatted: 10000 learned: 10000 parsed: 10000
[WARN] [2022-01-05 11:10:13] Names to parse: 9954 formatted: 9954 learned: 9951 parsed: 9954
[STOP] [2022-01-05 11:10:21] parse_names
[START] [2022-01-05 11:10:21] denormalize_canonical_names_to_nodes
[STOP] [2022-01-05 11:10:30] denormalize_canonical_names_to_nodes
[START] [2022-01-05 11:10:30] match_nodes
[START] [2022-01-05 11:10:30] map_all_nodes_to_pages
[STOP] [2022-01-05 12:26:26] map_all_nodes_to_pages
[INFO] [2022-01-05 12:26:26] 6077 Unmatched nodes (of 129954)! That's too many to output. Full list in /app/public/data/IUCN-SD/unmatched_nodes.txt ; First 10: Canonical: Laminella sanguinea; Node#101668545; ResourceID: 11198; Canonical: Carelia bicolor; Node#101767859; ResourceID: 3876; Canonical: Carelia cochlea; Node#101767870; ResourceID: 3877; Canonical: Carelia cumingiana; Node#101767881; ResourceID: 3878; Canonical: Carelia dolei; Node#101767892; ResourceID: 3879; Canonical: Carelia evelynae; Node#101767903; ResourceID: 3880; Canonical: Carelia necra; Node#101767989; ResourceID: 3888; Canonical: Carelia olivacea; Node#101767999; ResourceID: 3889; Canonical: Carelia paradoxa; Node#101768010; ResourceID: 3890; Canonical: Carelia pilsbryi; Node#101768030; ResourceID: 3892
[START] [2022-01-05 12:26:26] update_nodes
[STOP] [2022-01-05 12:26:39] update_nodes
[STOP] [2022-01-05 12:26:39] match_nodes
[START] [2022-01-05 12:26:39] reindex_search
[STOP] [2022-01-05 12:28:36] reindex_search
[START] [2022-01-05 12:28:36] normalize_units
[STOP] [2022-01-05 12:29:13] normalize_units
[START] [2022-01-05 12:29:13] calculate_statistics
[STOP] [2022-01-05 12:29:20] calculate_statistics
[START] [2022-01-05 12:29:20] complete_harvest_instance
[START] [2022-01-05 12:29:20] overall_tsv_creation
[INFO] [2022-01-05 12:29:21] Processing group of 129954 in 13 batches of 10000
[INFO] [2022-01-05 12:31:41] 27903 Traits (unfiltered)...
[INFO] [2022-01-05 12:36:35] 27903 Traits (filtered)...
[INFO] [2022-01-05 12:36:39] 0 Associations (filtered)...
[INFO] [2022-01-05 12:36:45] 43753 metadata added.
[INFO] [2022-01-05 12:36:45] 0 metadata added.
[INFO] [2022-01-05 12:38:54] 29686 Traits (unfiltered)...
[INFO] [2022-01-05 12:44:06] 29686 Traits (filtered)...
[INFO] [2022-01-05 12:44:10] 0 Associations (filtered)...
[INFO] [2022-01-05 12:44:16] 51922 metadata added.
[INFO] [2022-01-05 12:44:16] 0 metadata added.
[INFO] [2022-01-05 12:46:17] 30319 Traits (unfiltered)...
[INFO] [2022-01-05 12:51:35] 30319 Traits (filtered)...
[INFO] [2022-01-05 12:51:39] 0 Associations (filtered)...
[INFO] [2022-01-05 12:51:44] 40754 metadata added.
[INFO] [2022-01-05 12:51:44] 0 metadata added.
[INFO] [2022-01-05 12:53:46] 29293 Traits (unfiltered)...
[INFO] [2022-01-05 12:58:59] 29293 Traits (filtered)...
[INFO] [2022-01-05 12:59:03] 0 Associations (filtered)...
[INFO] [2022-01-05 12:59:09] 57843 metadata added.
[INFO] [2022-01-05 12:59:09] 0 metadata added.
[INFO] [2022-01-05 13:01:14] 30683 Traits (unfiltered)...
[INFO] [2022-01-05 13:06:38] 30683 Traits (filtered)...
[INFO] [2022-01-05 13:06:41] 0 Associations (filtered)...
[INFO] [2022-01-05 13:06:47] 49585 metadata added.
[INFO] [2022-01-05 13:06:47] 0 metadata added.
[INFO] [2022-01-05 13:08:51] 30575 Traits (unfiltered)...
[INFO] [2022-01-05 13:14:14] 30575 Traits (filtered)...
[INFO] [2022-01-05 13:14:18] 0 Associations (filtered)...
[INFO] [2022-01-05 13:14:24] 54662 metadata added.
[INFO] [2022-01-05 13:14:24] 0 metadata added.
[INFO] [2022-01-05 13:16:28] 30199 Traits (unfiltered)...
[INFO] [2022-01-05 13:21:51] 30199 Traits (filtered)...
[INFO] [2022-01-05 13:21:55] 0 Associations (filtered)...
[INFO] [2022-01-05 13:22:01] 57971 metadata added.
[INFO] [2022-01-05 13:22:01] 0 metadata added.
[INFO] [2022-01-05 13:24:07] 30054 Traits (unfiltered)...
[INFO] [2022-01-05 13:29:30] 30054 Traits (filtered)...
[INFO] [2022-01-05 13:29:33] 0 Associations (filtered)...
[INFO] [2022-01-05 13:29:39] 49862 metadata added.
[INFO] [2022-01-05 13:29:39] 0 metadata added.
[INFO] [2022-01-05 13:31:43] 31122 Traits (unfiltered)...
[INFO] [2022-01-05 13:37:09] 31122 Traits (filtered)...
[INFO] [2022-01-05 13:37:13] 0 Associations (filtered)...
[INFO] [2022-01-05 13:37:21] 33450 metadata added.
[INFO] [2022-01-05 13:37:21] 0 metadata added.
[INFO] [2022-01-05 13:39:23] 26312 Traits (unfiltered)...
[INFO] [2022-01-05 13:44:08] 26312 Traits (filtered)...
[INFO] [2022-01-05 13:44:11] 0 Associations (filtered)...
[INFO] [2022-01-05 13:44:15] 33555 metadata added.
[INFO] [2022-01-05 13:44:15] 0 metadata added.
[INFO] [2022-01-05 13:46:20] 26644 Traits (unfiltered)...
[INFO] [2022-01-05 13:51:05] 26644 Traits (filtered)...
[INFO] [2022-01-05 13:51:08] 0 Associations (filtered)...
[INFO] [2022-01-05 13:51:18] 48087 metadata added.
[INFO] [2022-01-05 13:51:18] 0 metadata added.
[INFO] [2022-01-05 13:53:21] 33004 Traits (unfiltered)...
[INFO] [2022-01-05 13:59:04] 33004 Traits (filtered)...
[INFO] [2022-01-05 13:59:08] 0 Associations (filtered)...
[INFO] [2022-01-05 13:59:17] 37373 metadata added.
[INFO] [2022-01-05 13:59:17] 0 metadata added.
[INFO] [2022-01-05 14:01:21] 29664 Traits (unfiltered)...
[INFO] [2022-01-05 14:06:33] 29664 Traits (filtered)...
[INFO] [2022-01-05 14:06:37] 0 Associations (filtered)...
[INFO] [2022-01-05 14:06:44] 46535 metadata added.
[INFO] [2022-01-05 14:06:44] 0 metadata added.
[INFO] [2022-01-05 14:07:26] Average Time: 413.141
[INFO] [2022-01-05 14:07:26] Total Time: 1h38m6s
[INFO] [2022-01-05 14:07:26] last 3 / first 3: 1.03
[INFO] [2022-01-05 14:07:26] Std.Dev: 18.109251779132123; Max: 446.93
[STOP] [2022-01-05 14:07:26] overall_tsv_creation
[INFO] [2022-01-05 14:07:26] Done. Check your files:
[INFO] [2022-01-05 14:07:26] (129948 lines) /app/public/data/IUCN-SD/publish_nodes.tsv
[INFO] [2022-01-05 14:07:27] (646280 lines) /app/public/data/IUCN-SD/publish_node_ancestors.tsv
[INFO] [2022-01-05 14:07:27] (129954 lines) /app/public/data/IUCN-SD/publish_scientific_names.tsv
[INFO] [2022-01-05 14:07:28] (385459 lines) /app/public/data/IUCN-SD/publish_traits.tsv
[INFO] [2022-01-05 14:07:28] (605353 lines) /app/public/data/IUCN-SD/publish_metadata.tsv
[STOP] [2022-01-05 14:07:28] complete_harvest_instance
[START] [2022-01-05 14:07:28] completed
[STOP] [2022-01-05 14:07:28] completed
[STOP] [2022-01-05 14:07:28] logged process, took 12457.64

Latest Process

05 Jan 10:39 EXITED currently processing 0