Harvest for United States Species List Created 12 Dec 13:23

Stage: completed
Fetched: 12 Dec 13:23
Validated: 12 Dec 13:24
Deltas Created 12 Dec 13:24
Units Normalized: 13 Dec 10:18
Ancestry Built: 12 Dec 14:10
Nodes Matched: 13 Dec 07:08
Names Parsed: 12 Dec 14:13
New Models Stored: 12 Dec 13:55
Indexed: 13 Dec 10:18
Completed: 13 Dec 11:16
Time to Harvest: 1 minute

Expected File Format Definitions

Harvesting Log (most recent first)

# Logfile created on 2019-12-12 13:23:43 -0500 by logger.rb/56815
[START] [2019-12-12 13:23:43] logged process
[START] [2019-12-12 13:23:44] create_harvest_instance
[STOP] [2019-12-12 13:23:44] create_harvest_instance
[START] [2019-12-12 13:23:44] fetch_files
[STOP] [2019-12-12 13:23:44] fetch_files
[START] [2019-12-12 13:23:44] validate_each_file
[STOP] [2019-12-12 13:24:10] validate_each_file
[START] [2019-12-12 13:24:10] convert_to_csv
[CMD] [2019-12-12 13:24:10] /usr/bin/sort /app/public/converted_csv/u_s_sp_list_refs_18934.csv > /app/public/converted_csv/u_s_sp_list_refs_18934.csv_sorted
[CMD] [2019-12-12 13:24:10] /usr/bin/sort /app/public/converted_csv/u_s_sp_list_nodes_18935.csv > /app/public/converted_csv/u_s_sp_list_nodes_18935.csv_sorted
[CMD] [2019-12-12 13:24:10] /usr/bin/sort /app/public/converted_csv/u_s_sp_list_occurrences_18936.csv > /app/public/converted_csv/u_s_sp_list_occurrences_18936.csv_sorted
[CMD] [2019-12-12 13:24:10] /usr/bin/sort /app/public/converted_csv/u_s_sp_list_measurements_18937.csv > /app/public/converted_csv/u_s_sp_list_measurements_18937.csv_sorted
[STOP] [2019-12-12 13:24:10] convert_to_csv
[START] [2019-12-12 13:24:10] calculate_delta
[CMD] [2019-12-12 13:24:10] echo "0a" > /app/public/diff/u_s_sp_list_refs_18934.diff
[CMD] [2019-12-12 13:24:10] tail -n +1 /app/public/converted_csv/u_s_sp_list_refs_18934.csv >> /app/public/diff/u_s_sp_list_refs_18934.diff
[CMD] [2019-12-12 13:24:10] echo "." >> /app/public/diff/u_s_sp_list_refs_18934.diff
[CMD] [2019-12-12 13:24:10] echo "0a" > /app/public/diff/u_s_sp_list_nodes_18935.diff
[CMD] [2019-12-12 13:24:10] tail -n +1 /app/public/converted_csv/u_s_sp_list_nodes_18935.csv >> /app/public/diff/u_s_sp_list_nodes_18935.diff
[CMD] [2019-12-12 13:24:10] echo "." >> /app/public/diff/u_s_sp_list_nodes_18935.diff
[CMD] [2019-12-12 13:24:10] echo "0a" > /app/public/diff/u_s_sp_list_occurrences_18936.diff
[CMD] [2019-12-12 13:24:10] tail -n +1 /app/public/converted_csv/u_s_sp_list_occurrences_18936.csv >> /app/public/diff/u_s_sp_list_occurrences_18936.diff
[CMD] [2019-12-12 13:24:10] echo "." >> /app/public/diff/u_s_sp_list_occurrences_18936.diff
[CMD] [2019-12-12 13:24:10] echo "0a" > /app/public/diff/u_s_sp_list_measurements_18937.diff
[CMD] [2019-12-12 13:24:10] tail -n +1 /app/public/converted_csv/u_s_sp_list_measurements_18937.csv >> /app/public/diff/u_s_sp_list_measurements_18937.diff
[CMD] [2019-12-12 13:24:10] echo "." >> /app/public/diff/u_s_sp_list_measurements_18937.diff
[STOP] [2019-12-12 13:24:10] calculate_delta
[START] [2019-12-12 13:24:10] parse_diff_and_store
[INFO] [2019-12-12 13:24:10] Loading refs diff file into memory (true lines)...
[INFO] [2019-12-12 13:24:10] Loading nodes diff file into memory (true lines)...
[WARN] [2019-12-12 13:24:24] Filtered Scientific Name `Stenopelmatus ""mahogany"""` to `Stenopelmatus mahogany`
[WARN] [2019-12-12 13:24:41] Filtered Scientific Name `Neoprocris  floridana` to `Neoprocris floridana`
[WARN] [2019-12-12 13:24:42] Filtered Scientific Name `Entzia  macrescens` to `Entzia macrescens`
[WARN] [2019-12-12 13:24:47] Filtered Scientific Name `Ollarianus  armus` to `Ollarianus armus`
[WARN] [2019-12-12 13:24:53] Filtered Scientific Name `Punctularia  strigosozonata` to `Punctularia strigosozonata`
[WARN] [2019-12-12 13:24:55] Filtered Scientific Name `Magnoliopsida ""` to `Magnoliopsida `
[WARN] [2019-12-12 13:24:56] Filtered Scientific Name `Tetragonocephala  flava` to `Tetragonocephala flava`
[WARN] [2019-12-12 13:24:57] Filtered Scientific Name `Craneiobia  tuba` to `Craneiobia tuba`
[WARN] [2019-12-12 13:24:57] Filtered Scientific Name `Odonata Nick."` to `Odonata Nick.`
[WARN] [2019-12-12 13:25:00] Filtered Scientific Name `"" Then That Looks Complete` to ` Then That Looks Complete`
[WARN] [2019-12-12 13:25:00] Filtered Scientific Name `Stenopelmatus "mahogany"` to `Stenopelmatus mahogany`
[WARN] [2019-12-12 13:25:00] Filtered Scientific Name `Isopoda rounded."` to `Isopoda rounded.`
[WARN] [2019-12-12 13:25:00] Filtered Scientific Name `Ovaticoccus  agavium` to `Ovaticoccus agavium`
[WARN] [2019-12-12 13:25:03] Filtered Scientific Name `Squamata etc."""` to `Squamata etc.`
[WARN] [2019-12-12 13:25:04] Filtered Scientific Name `Megasyrphus  laxus` to `Megasyrphus laxus`
[WARN] [2019-12-12 13:25:15] Filtered Scientific Name `Aphalaroida  pithecolobia` to `Aphalaroida pithecolobia`
[WARN] [2019-12-12 13:25:17] Filtered Scientific Name `Asterales rwp84"` to `Asterales rwp84`
[WARN] [2019-12-12 13:25:20] Filtered Scientific Name `Aphalaroida  inermis` to `Aphalaroida inermis`
[INFO] [2019-12-12 13:25:32] Loading occurrences diff file into memory (true lines)...
[INFO] [2019-12-12 13:25:57] Loading measurements diff file into memory (true lines)...
[INFO] [2019-12-12 13:44:20] Storing 2 References
[INFO] [2019-12-12 13:44:20] Processing group of 2 in 1 groups of 1000
[INFO] [2019-12-12 13:44:20] Average Time: 0.0
[INFO] [2019-12-12 13:44:20] Total Time: 1s
[INFO] [2019-12-12 13:44:20] Storing 215560 ScientificNames
[INFO] [2019-12-12 13:44:20] Processing group of 215560 in 216 groups of 1000
[INFO] [2019-12-12 13:46:39] Average Time: 0.639
[INFO] [2019-12-12 13:46:39] Total Time: 2m19s
[INFO] [2019-12-12 13:46:39] last 3 / first 3: 0.84
[INFO] [2019-12-12 13:46:39] Std.Dev: 1.401427843308388; Max: 8.02
[INFO] [2019-12-12 13:46:39] Storing 215560 Nodes
[INFO] [2019-12-12 13:46:39] Processing group of 215560 in 216 groups of 1000
[INFO] [2019-12-12 13:48:54] Average Time: 0.623
[INFO] [2019-12-12 13:48:54] Total Time: 2m16s
[INFO] [2019-12-12 13:48:54] last 3 / first 3: 0.75
[INFO] [2019-12-12 13:48:54] Std.Dev: 1.5937377450509227; Max: 9.03
[INFO] [2019-12-12 13:48:54] Storing 168817 Occurrences
[INFO] [2019-12-12 13:48:54] Processing group of 168817 in 169 groups of 1000
[INFO] [2019-12-12 13:49:32] Average Time: 0.223
[INFO] [2019-12-12 13:49:32] Total Time: 39s
[INFO] [2019-12-12 13:49:32] last 3 / first 3: 0.77
[INFO] [2019-12-12 13:49:32] Std.Dev: 0.9736529155710468; Max: 9.12
[INFO] [2019-12-12 13:49:32] Storing 337634 TraitsReferences
[INFO] [2019-12-12 13:49:32] Processing group of 337634 in 338 groups of 1000
[INFO] [2019-12-12 13:50:18] Average Time: 0.129
[INFO] [2019-12-12 13:50:18] Total Time: 46s
[INFO] [2019-12-12 13:50:18] last 3 / first 3: 0.4
[INFO] [2019-12-12 13:50:18] Std.Dev: 0.7286974680894672; Max: 9.65
[INFO] [2019-12-12 13:50:18] Storing 337650 Traits
[INFO] [2019-12-12 13:50:18] Processing group of 337650 in 338 groups of 1000
[INFO] [2019-12-12 13:53:41] Average Time: 0.599
[INFO] [2019-12-12 13:53:41] Total Time: 3m24s
[INFO] [2019-12-12 13:53:41] last 3 / first 3: 0.8
[INFO] [2019-12-12 13:53:41] Std.Dev: 1.7392527130926085; Max: 11.07
[INFO] [2019-12-12 13:53:41] Storing 337094 MetaTraits
[INFO] [2019-12-12 13:53:41] Processing group of 337094 in 338 groups of 1000
[INFO] [2019-12-12 13:55:06] Average Time: 0.247
[INFO] [2019-12-12 13:55:06] Total Time: 1m26s
[INFO] [2019-12-12 13:55:06] last 3 / first 3: 0.58
[INFO] [2019-12-12 13:55:06] Std.Dev: 1.2078907235342111; Max: 11.41
[STOP] [2019-12-12 13:55:06] parse_diff_and_store
[START] [2019-12-12 13:55:06] resolve_keys
[INFO] [2019-12-12 13:59:36] Occurrences to nodes (through scientific_names)...
[INFO] [2019-12-12 13:59:48] traits to occurrences...
[INFO] [2019-12-12 14:00:07] traits to nodes (through occurrences)...
[INFO] [2019-12-12 14:00:11] Traits to sex term...
[INFO] [2019-12-12 14:00:20] Traits to lifestage term...
[INFO] [2019-12-12 14:00:29] MetaTraits to traits...
[INFO] [2019-12-12 14:00:49] MetaTraits (simple, measurement row refers to parent) to traits...
[INFO] [2019-12-12 14:01:37] Assocs to occurrences...
[INFO] [2019-12-12 14:01:37] Assocs to nodes...
[INFO] [2019-12-12 14:01:37] Assoc to sex term...
[INFO] [2019-12-12 14:01:37] Assoc to lifestage term...
[STOP] [2019-12-12 14:01:37] resolve_keys
[START] [2019-12-12 14:01:37] hold_for_later_1
[STOP] [2019-12-12 14:01:37] hold_for_later_1
[START] [2019-12-12 14:01:37] hold_for_later_2
[STOP] [2019-12-12 14:01:37] hold_for_later_2
[START] [2019-12-12 14:01:37] resolve_missing_parents
[STOP] [2019-12-12 14:04:54] resolve_missing_parents
[START] [2019-12-12 14:04:54] rebuild_nodes
[START] [2019-12-12 14:04:54] Flattener#flatten
[START] [2019-12-12 14:04:54] Flattener#study_resource
[START] [2019-12-12 14:05:12] Flattener#build_ancestry
[STOP] [2019-12-12 14:06:46] Flattener#build_ancestry
[INFO] [2019-12-12 14:06:46] 215560 ancestry keys
[START] [2019-12-12 14:06:46] build_node_ancestors
[INFO] [2019-12-12 14:06:46] old ancestors deleted.
[STOP] [2019-12-12 14:09:43] build_node_ancestors
[START] [2019-12-12 14:09:43] Flattener#propagate_ancestor_ids
[STOP] [2019-12-12 14:10:42] Flattener#propagate_ancestor_ids
[STOP] [2019-12-12 14:10:42] Flattener#flatten
[STOP] [2019-12-12 14:10:42] rebuild_nodes
[START] [2019-12-12 14:10:42] resolve_missing_media_owners
[STOP] [2019-12-12 14:10:42] resolve_missing_media_owners
[START] [2019-12-12 14:10:42] sanitize_media_verbatims
[STOP] [2019-12-12 14:10:42] sanitize_media_verbatims
[START] [2019-12-12 14:10:42] queue_downloads
[STOP] [2019-12-12 14:10:42] queue_downloads
[START] [2019-12-12 14:10:42] parse_names
[WARN] [2019-12-12 14:10:43] I see 215560 names which still need to be parsed.
[STOP] [2019-12-12 14:13:28] parse_names
[START] [2019-12-12 14:13:28] denormalize_canonical_names_to_nodes
[STOP] [2019-12-12 14:13:31] denormalize_canonical_names_to_nodes
[START] [2019-12-12 14:13:31] match_nodes
[START] [2019-12-12 14:13:31] map_all_nodes_to_pages
[STOP] [2019-12-13 07:08:43] map_all_nodes_to_pages
[INFO] [2019-12-13 07:08:43] 23756 Unmatched nodes (of 215560)! That's too many to output. First 10: Ptilogonatidae (#60749386); Ptilogonys (#60792607); Ptilogonys caudatus (#60792606); Ptilogonys cinereus (#60817376); Mycetozoa (#60750309); Protosteliomycetes (#60750308); Protosteliales (#60750307); Protosteliaceae (#60750306); Protostelium pyriforme (#60754276); Schizoplasmodiopsis micropunctata (#60786879)
[START] [2019-12-13 07:08:43] update_nodes
[STOP] [2019-12-13 07:08:48] update_nodes
[STOP] [2019-12-13 07:08:48] match_nodes
[START] [2019-12-13 07:08:48] reindex_search
[STOP] [2019-12-13 07:09:25] reindex_search
[ERR] [2019-12-13 07:09:25] Faraday::ConnectionFailed
[ERR] [2019-12-13 07:09:25] Failed to open TCP connection to elasticsearch:9200 (Connection refused - connect(2) for "elasticsearch" port 9200)
[ERR] [2019-12-13 07:09:25] ../models/resource_harvester.rb:615:in `reindex_search'
[ERR] [2019-12-13 07:09:25] ../models/resource_harvester.rb:86:in `block (3 levels) in start'
[ERR] [2019-12-13 07:09:25] ../models/logged_process.rb:19:in `run_step'
[ERR] [2019-12-13 07:09:25] ../models/resource_harvester.rb:86:in `block (2 levels) in start'
[ERR] [2019-12-13 07:09:25] ../models/resource_harvester.rb:75:in `each_key'
[ERR] [2019-12-13 07:09:25] ../models/resource_harvester.rb:75:in `block in start'
[ERR] [2019-12-13 07:09:25] ../models/resource.rb:139:in `lock'
[ERR] [2019-12-13 07:09:25] ../models/resource_harvester.rb:72:in `start'
[ERR] [2019-12-13 07:09:25] ../models/resource.rb:223:in `harvest'
[ERR] [2019-12-13 07:09:25] ../models/resource.rb:199:in `re_download_opendata_and_harvest'
[STOP] [2019-12-13 07:09:25] logged process, took 63941.36
[START] [2019-12-13 10:13:08] logged process
[INFO] [2019-12-13 10:13:08] Already completed stage create_harvest_instance, skipping...
[INFO] [2019-12-13 10:13:08] Already completed stage fetch_files, skipping...
[INFO] [2019-12-13 10:13:08] Already completed stage validate_each_file, skipping...
[INFO] [2019-12-13 10:13:08] Already completed stage convert_to_csv, skipping...
[INFO] [2019-12-13 10:13:08] Already completed stage calculate_delta, skipping...
[INFO] [2019-12-13 10:13:08] Already completed stage parse_diff_and_store, skipping...
[INFO] [2019-12-13 10:13:08] Already completed stage resolve_keys, skipping...
[INFO] [2019-12-13 10:13:08] Already completed stage hold_for_later_1, skipping...
[INFO] [2019-12-13 10:13:08] Already completed stage hold_for_later_2, skipping...
[INFO] [2019-12-13 10:13:08] Already completed stage resolve_missing_parents, skipping...
[INFO] [2019-12-13 10:13:08] Already completed stage rebuild_nodes, skipping...
[INFO] [2019-12-13 10:13:08] Already completed stage resolve_missing_media_owners, skipping...
[INFO] [2019-12-13 10:13:08] Already completed stage sanitize_media_verbatims, skipping...
[INFO] [2019-12-13 10:13:08] Already completed stage queue_downloads, skipping...
[INFO] [2019-12-13 10:13:08] Already completed stage parse_names, skipping...
[INFO] [2019-12-13 10:13:08] Already completed stage denormalize_canonical_names_to_nodes, skipping...
[INFO] [2019-12-13 10:13:08] Already completed stage match_nodes, skipping...
[START] [2019-12-13 10:13:08] reindex_search
[STOP] [2019-12-13 10:18:31] reindex_search
[START] [2019-12-13 10:18:31] normalize_units
[STOP] [2019-12-13 10:18:32] normalize_units
[START] [2019-12-13 10:18:32] calculate_statistics
[STOP] [2019-12-13 10:18:33] calculate_statistics
[START] [2019-12-13 10:18:33] complete_harvest_instance
[START] [2019-12-13 10:18:33] overall_tsv_creation
[INFO] [2019-12-13 10:18:33] Processing group of 215560 in 22 batches of 10000
[INFO] [2019-12-13 10:20:21] 5904 Traits (unfiltered)...
[INFO] [2019-12-13 10:20:34] 5904 Traits (filtered)...
[INFO] [2019-12-13 10:20:34] 0 Associations (filtered)...
[INFO] [2019-12-13 10:21:22] 29518 metadata added.
[INFO] [2019-12-13 10:21:22] 0 metadata added.
[INFO] [2019-12-13 10:22:53] 6889 Traits (unfiltered)...
[INFO] [2019-12-13 10:23:06] 6889 Traits (filtered)...
[INFO] [2019-12-13 10:23:06] 0 Associations (filtered)...
[INFO] [2019-12-13 10:23:56] 34441 metadata added.
[INFO] [2019-12-13 10:23:56] 0 metadata added.
[INFO] [2019-12-13 10:25:29] 7225 Traits (unfiltered)...
[INFO] [2019-12-13 10:25:42] 7225 Traits (filtered)...
[INFO] [2019-12-13 10:25:42] 0 Associations (filtered)...
[INFO] [2019-12-13 10:26:36] 36117 metadata added.
[INFO] [2019-12-13 10:26:36] 0 metadata added.
[INFO] [2019-12-13 10:28:09] 7420 Traits (unfiltered)...
[INFO] [2019-12-13 10:28:22] 7420 Traits (filtered)...
[INFO] [2019-12-13 10:28:22] 0 Associations (filtered)...
[INFO] [2019-12-13 10:29:15] 37090 metadata added.
[INFO] [2019-12-13 10:29:15] 0 metadata added.
[INFO] [2019-12-13 10:30:45] 7613 Traits (unfiltered)...
[INFO] [2019-12-13 10:30:58] 7613 Traits (filtered)...
[INFO] [2019-12-13 10:30:59] 0 Associations (filtered)...
[INFO] [2019-12-13 10:31:50] 38052 metadata added.
[INFO] [2019-12-13 10:31:50] 0 metadata added.
[INFO] [2019-12-13 10:33:25] 7698 Traits (unfiltered)...
[INFO] [2019-12-13 10:33:38] 7698 Traits (filtered)...
[INFO] [2019-12-13 10:33:38] 0 Associations (filtered)...
[INFO] [2019-12-13 10:34:31] 38475 metadata added.
[INFO] [2019-12-13 10:34:31] 0 metadata added.
[INFO] [2019-12-13 10:36:04] 7801 Traits (unfiltered)...
[INFO] [2019-12-13 10:36:17] 7801 Traits (filtered)...
[INFO] [2019-12-13 10:36:17] 0 Associations (filtered)...
[INFO] [2019-12-13 10:37:10] 38994 metadata added.
[INFO] [2019-12-13 10:37:10] 0 metadata added.
[INFO] [2019-12-13 10:38:41] 7815 Traits (unfiltered)...
[INFO] [2019-12-13 10:38:54] 7815 Traits (filtered)...
[INFO] [2019-12-13 10:38:54] 0 Associations (filtered)...
[INFO] [2019-12-13 10:39:48] 39063 metadata added.
[INFO] [2019-12-13 10:39:48] 0 metadata added.
[INFO] [2019-12-13 10:41:19] 7826 Traits (unfiltered)...
[INFO] [2019-12-13 10:41:32] 7826 Traits (filtered)...
[INFO] [2019-12-13 10:41:32] 0 Associations (filtered)...
[INFO] [2019-12-13 10:42:25] 39118 metadata added.
[INFO] [2019-12-13 10:42:25] 0 metadata added.
[INFO] [2019-12-13 10:43:56] 7866 Traits (unfiltered)...
[INFO] [2019-12-13 10:44:09] 7866 Traits (filtered)...
[INFO] [2019-12-13 10:44:09] 0 Associations (filtered)...
[INFO] [2019-12-13 10:45:03] 39315 metadata added.
[INFO] [2019-12-13 10:45:03] 0 metadata added.
[INFO] [2019-12-13 10:46:34] 7943 Traits (unfiltered)...
[INFO] [2019-12-13 10:46:47] 7943 Traits (filtered)...
[INFO] [2019-12-13 10:46:47] 0 Associations (filtered)...
[INFO] [2019-12-13 10:47:41] 39697 metadata added.
[INFO] [2019-12-13 10:47:41] 0 metadata added.
[INFO] [2019-12-13 10:49:11] 7975 Traits (unfiltered)...
[INFO] [2019-12-13 10:49:25] 7975 Traits (filtered)...
[INFO] [2019-12-13 10:49:25] 0 Associations (filtered)...
[INFO] [2019-12-13 10:50:18] 39861 metadata added.
[INFO] [2019-12-13 10:50:18] 0 metadata added.
[INFO] [2019-12-13 10:51:49] 8015 Traits (unfiltered)...
[INFO] [2019-12-13 10:52:02] 8015 Traits (filtered)...
[INFO] [2019-12-13 10:52:03] 0 Associations (filtered)...
[INFO] [2019-12-13 10:52:56] 40047 metadata added.
[INFO] [2019-12-13 10:52:56] 0 metadata added.
[INFO] [2019-12-13 10:54:27] 8004 Traits (unfiltered)...
[INFO] [2019-12-13 10:54:40] 8004 Traits (filtered)...
[INFO] [2019-12-13 10:54:40] 0 Associations (filtered)...
[INFO] [2019-12-13 10:55:34] 39994 metadata added.
[INFO] [2019-12-13 10:55:34] 0 metadata added.
[INFO] [2019-12-13 10:57:05] 8074 Traits (unfiltered)...
[INFO] [2019-12-13 10:57:19] 8074 Traits (filtered)...
[INFO] [2019-12-13 10:57:19] 0 Associations (filtered)...
[INFO] [2019-12-13 10:58:13] 40340 metadata added.
[INFO] [2019-12-13 10:58:13] 0 metadata added.
[INFO] [2019-12-13 10:59:44] 8137 Traits (unfiltered)...
[INFO] [2019-12-13 10:59:58] 8137 Traits (filtered)...
[INFO] [2019-12-13 10:59:58] 0 Associations (filtered)...
[INFO] [2019-12-13 11:00:52] 40641 metadata added.
[INFO] [2019-12-13 11:00:52] 0 metadata added.
[INFO] [2019-12-13 11:02:23] 8285 Traits (unfiltered)...
[INFO] [2019-12-13 11:02:36] 8285 Traits (filtered)...
[INFO] [2019-12-13 11:02:36] 0 Associations (filtered)...
[INFO] [2019-12-13 11:03:30] 41393 metadata added.
[INFO] [2019-12-13 11:03:30] 0 metadata added.
[INFO] [2019-12-13 11:05:02] 8298 Traits (unfiltered)...
[INFO] [2019-12-13 11:05:15] 8298 Traits (filtered)...
[INFO] [2019-12-13 11:05:15] 0 Associations (filtered)...
[INFO] [2019-12-13 11:06:09] 41452 metadata added.
[INFO] [2019-12-13 11:06:09] 0 metadata added.
[INFO] [2019-12-13 11:07:40] 8276 Traits (unfiltered)...
[INFO] [2019-12-13 11:07:53] 8276 Traits (filtered)...
[INFO] [2019-12-13 11:07:54] 0 Associations (filtered)...
[INFO] [2019-12-13 11:08:48] 41331 metadata added.
[INFO] [2019-12-13 11:08:48] 0 metadata added.
[INFO] [2019-12-13 11:10:20] 8446 Traits (unfiltered)...
[INFO] [2019-12-13 11:10:33] 8446 Traits (filtered)...
[INFO] [2019-12-13 11:10:33] 0 Associations (filtered)...
[INFO] [2019-12-13 11:11:28] 42173 metadata added.
[INFO] [2019-12-13 11:11:28] 0 metadata added.
[INFO] [2019-12-13 11:13:00] 8526 Traits (unfiltered)...
[INFO] [2019-12-13 11:13:14] 8526 Traits (filtered)...
[INFO] [2019-12-13 11:13:14] 0 Associations (filtered)...
[INFO] [2019-12-13 11:14:09] 42573 metadata added.
[INFO] [2019-12-13 11:14:09] 0 metadata added.
[INFO] [2019-12-13 11:15:20] 4781 Traits (unfiltered)...
[INFO] [2019-12-13 11:15:33] 4781 Traits (filtered)...
[INFO] [2019-12-13 11:15:33] 0 Associations (filtered)...
[INFO] [2019-12-13 11:16:18] 23860 metadata added.
[INFO] [2019-12-13 11:16:18] 0 metadata added.
[INFO] [2019-12-13 11:16:18] Average Time: 128.7
[INFO] [2019-12-13 11:16:18] Total Time: 57m46s
[INFO] [2019-12-13 11:16:18] last 3 / first 3: 0.98
[INFO] [2019-12-13 11:16:18] Std.Dev: 5.934896797754785; Max: 132.33
[STOP] [2019-12-13 11:16:18] overall_tsv_creation
[INFO] [2019-12-13 11:16:18] Done. Check your files:
[INFO] [2019-12-13 11:16:18] (215560 lines) /app/public/data/u_s_sp_list/publish_nodes.tsv
[INFO] [2019-12-13 11:16:18] (1208618 lines) /app/public/data/u_s_sp_list/publish_node_ancestors.tsv
[INFO] [2019-12-13 11:16:18] (215560 lines) /app/public/data/u_s_sp_list/publish_scientific_names.tsv
[INFO] [2019-12-13 11:16:18] (168818 lines) /app/public/data/u_s_sp_list/publish_traits.tsv
[INFO] [2019-12-13 11:16:18] (843546 lines) /app/public/data/u_s_sp_list/publish_metadata.tsv
[STOP] [2019-12-13 11:16:19] complete_harvest_instance
[START] [2019-12-13 11:16:19] completed
[STOP] [2019-12-13 11:16:19] completed
[STOP] [2019-12-13 11:16:19] logged process, took 3790.35

Latest Process