Harvest for wikipedia emerging languages Created 28 May 12:36

Harvesting Log

# Logfile created on 2020-04-16 10:54:56 -0400 by logger.rb/v1.4.2
[INFO] [2020-04-16 10:54:56] ## HARVEST: type = -harvest
[START] [2020-04-16 10:54:59] logged process
[START] [2020-04-16 10:54:59] create_harvest_instance
[STOP] [2020-04-16 10:55:00] create_harvest_instance
[START] [2020-04-16 10:55:00] fetch_files
[STOP] [2020-04-16 10:55:00] fetch_files
[START] [2020-04-16 10:55:00] validate_each_file
[STOP] [2020-04-16 10:55:13] validate_each_file
[START] [2020-04-16 10:55:13] convert_to_csv
[CMD] [2020-04-16 10:55:13] /usr/bin/sort /app/public/converted_csv/wiki_combined_la_nodes_20819.csv > /app/public/converted_csv/wiki_combined_la_nodes_20819.csv_sorted
[CMD] [2020-04-16 10:55:15] /usr/bin/sort /app/public/converted_csv/wiki_combined_la_media_20820.csv > /app/public/converted_csv/wiki_combined_la_media_20820.csv_sorted
[STOP] [2020-04-16 10:55:17] convert_to_csv
[START] [2020-04-16 10:55:17] calculate_delta
[CMD] [2020-04-16 10:55:17] echo "0a" > /app/public/diff/wiki_combined_la_nodes_20819.diff
[CMD] [2020-04-16 10:55:18] tail -n +1 /app/public/converted_csv/wiki_combined_la_nodes_20819.csv >> /app/public/diff/wiki_combined_la_nodes_20819.diff
[CMD] [2020-04-16 10:55:19] echo "." >> /app/public/diff/wiki_combined_la_nodes_20819.diff
[CMD] [2020-04-16 10:55:20] echo "0a" > /app/public/diff/wiki_combined_la_media_20820.diff
[CMD] [2020-04-16 10:55:22] tail -n +1 /app/public/converted_csv/wiki_combined_la_media_20820.csv >> /app/public/diff/wiki_combined_la_media_20820.diff
[CMD] [2020-04-16 10:55:23] echo "." >> /app/public/diff/wiki_combined_la_media_20820.diff
[STOP] [2020-04-16 10:55:25] calculate_delta
[START] [2020-04-16 10:55:25] parse_diff_and_store
[INFO] [2020-04-16 10:55:26] Loading nodes diff file into memory (true lines)...
[WARN] [2020-04-16 10:55:29] Filtered Scientific Name `Visna/maedi virus` to `Visnamaedi virus`
[WARN] [2020-04-16 10:55:30] Filtered Scientific Name `Ambystoma  hakié jas jas` to `Ambystoma hakié jas jas`
[INFO] [2020-04-16 10:55:37] Loading media diff file into memory (true lines)...
[INFO] [2020-04-16 11:03:24] Storing 25535 ScientificNames
[INFO] [2020-04-16 11:03:24] Processing group of 25535 in 26 groups of 1000
[INFO] [2020-04-16 11:03:32] Average Time: 0.307
[INFO] [2020-04-16 11:03:32] Total Time: 9s
[INFO] [2020-04-16 11:03:32] last 3 / first 3: 0.67
[INFO] [2020-04-16 11:03:32] Std.Dev: 0.05477225575051661; Max: 0.44
[INFO] [2020-04-16 11:03:32] Storing 25535 Identifiers
[INFO] [2020-04-16 11:03:32] Processing group of 25535 in 26 groups of 1000
[INFO] [2020-04-16 11:03:35] Average Time: 0.113
[INFO] [2020-04-16 11:03:35] Total Time: 4s
[INFO] [2020-04-16 11:03:35] last 3 / first 3: 0.71
[INFO] [2020-04-16 11:03:35] Std.Dev: 0.0; Max: 0.14
[INFO] [2020-04-16 11:03:35] Storing 25535 Nodes
[INFO] [2020-04-16 11:03:35] Processing group of 25535 in 26 groups of 1000
[INFO] [2020-04-16 11:03:43] Average Time: 0.288
[INFO] [2020-04-16 11:03:43] Total Time: 8s
[INFO] [2020-04-16 11:03:43] last 3 / first 3: 0.91
[INFO] [2020-04-16 11:03:43] Std.Dev: 0.044721359549995794; Max: 0.45
[INFO] [2020-04-16 11:03:43] Storing 93555 ArticlesSections
[INFO] [2020-04-16 11:03:43] Processing group of 93555 in 94 groups of 1000
[INFO] [2020-04-16 11:03:50] Average Time: 0.071
[INFO] [2020-04-16 11:03:50] Total Time: 8s
[INFO] [2020-04-16 11:03:50] last 3 / first 3: 0.59
[INFO] [2020-04-16 11:03:50] Std.Dev: 0.1414213562373095; Max: 1.41
[INFO] [2020-04-16 11:03:50] Storing 93555 Articles
[INFO] [2020-04-16 11:03:50] Processing group of 93555 in 94 groups of 1000
[INFO] [2020-04-16 11:04:56] Average Time: 0.691
[INFO] [2020-04-16 11:04:56] Total Time: 1m6s
[INFO] [2020-04-16 11:04:56] last 3 / first 3: 0.78
[INFO] [2020-04-16 11:04:56] Std.Dev: 0.2683281572999748; Max: 2.48
[STOP] [2020-04-16 11:04:56] parse_diff_and_store
[START] [2020-04-16 11:04:56] resolve_keys
[INFO] [2020-04-16 11:13:16] Occurrences to nodes (through scientific_names)...
[INFO] [2020-04-16 11:13:16] traits to occurrences...
[INFO] [2020-04-16 11:13:16] traits to nodes (through occurrences)...
[INFO] [2020-04-16 11:13:16] Traits to sex term...
[INFO] [2020-04-16 11:13:16] Traits to lifestage term...
[INFO] [2020-04-16 11:13:16] MetaTraits to traits...
[INFO] [2020-04-16 11:13:16] MetaTraits (simple, measurement row refers to parent) to traits...
[INFO] [2020-04-16 11:13:16] Assocs to occurrences...
[INFO] [2020-04-16 11:13:16] Assocs to nodes...
[INFO] [2020-04-16 11:13:16] Assoc to sex term...
[INFO] [2020-04-16 11:13:16] Assoc to lifestage term...
[STOP] [2020-04-16 11:13:16] resolve_keys
[START] [2020-04-16 11:13:16] hold_for_later_1
[STOP] [2020-04-16 11:13:16] hold_for_later_1
[START] [2020-04-16 11:13:16] hold_for_later_2
[STOP] [2020-04-16 11:13:16] hold_for_later_2
[START] [2020-04-16 11:13:16] resolve_missing_parents
[STOP] [2020-04-16 11:13:25] resolve_missing_parents
[START] [2020-04-16 11:13:25] rebuild_nodes
[START] [2020-04-16 11:13:25] Flattener#flatten
[START] [2020-04-16 11:13:25] Flattener#study_resource
[START] [2020-04-16 11:13:26] Flattener#build_ancestry
[STOP] [2020-04-16 11:13:28] Flattener#build_ancestry
[INFO] [2020-04-16 11:13:28] 25535 ancestry keys
[START] [2020-04-16 11:13:28] build_node_ancestors
[INFO] [2020-04-16 11:13:28] old ancestors deleted.
[STOP] [2020-04-16 11:14:16] build_node_ancestors
[START] [2020-04-16 11:14:21] Flattener#propagate_ancestor_ids
[STOP] [2020-04-16 11:14:33] Flattener#propagate_ancestor_ids
[STOP] [2020-04-16 11:14:33] Flattener#flatten
[STOP] [2020-04-16 11:14:33] rebuild_nodes
[START] [2020-04-16 11:14:33] resolve_missing_media_owners
[STOP] [2020-04-16 11:14:33] resolve_missing_media_owners
[START] [2020-04-16 11:14:33] sanitize_media_verbatims
[STOP] [2020-04-16 11:14:33] sanitize_media_verbatims
[START] [2020-04-16 11:14:33] queue_downloads
[STOP] [2020-04-16 11:14:34] queue_downloads
[START] [2020-04-16 11:14:34] parse_names
[WARN] [2020-04-16 11:14:34] I see 25535 names which still need to be parsed.
[WARN] [2020-04-16 11:14:52] I see 26 names which still need to be parsed.
[STOP] [2020-04-16 11:14:53] parse_names
[START] [2020-04-16 11:14:53] denormalize_canonical_names_to_nodes
[STOP] [2020-04-16 11:14:54] denormalize_canonical_names_to_nodes
[START] [2020-04-16 11:14:54] match_nodes
[START] [2020-04-16 11:14:54] map_all_nodes_to_pages
[STOP] [2020-04-16 11:55:55] map_all_nodes_to_pages
[INFO] [2020-04-16 11:55:55] 2791 Unmatched nodes (of 25535)! That's too many to output. First 10: Biota (#68888420); Acytota (#68884147); Prokaryota (#68885560); Proteoarchaeota (#68887213); DPANN (#68888870); Aigarchaeota (#68894393); Bacteria (#68876726); Eubacteria (#68888854); Melainabacteria (#68886714); Negibacteria (#68893051)
[START] [2020-04-16 11:55:55] update_nodes
[STOP] [2020-04-16 11:56:05] update_nodes
[STOP] [2020-04-16 11:56:05] match_nodes
[START] [2020-04-16 11:56:05] reindex_search
[STOP] [2020-04-16 11:57:18] reindex_search
[START] [2020-04-16 11:57:18] normalize_units
[STOP] [2020-04-16 11:57:18] normalize_units
[START] [2020-04-16 11:57:18] calculate_statistics
[STOP] [2020-04-16 11:57:19] calculate_statistics
[START] [2020-04-16 11:57:19] complete_harvest_instance
[START] [2020-04-16 11:57:19] overall_tsv_creation
[INFO] [2020-04-16 11:57:19] Processing group of 25535 in 3 batches of 10000
[INFO] [2020-04-16 12:03:51] Average Time: 71.513
[INFO] [2020-04-16 12:03:51] Total Time: 6m32s
[STOP] [2020-04-16 12:03:51] overall_tsv_creation
[INFO] [2020-04-16 12:03:51] Done. Check your files:
[INFO] [2020-04-16 12:03:52] (25535 lines) /app/public/data/wiki_combined_la/publish_nodes.tsv
[INFO] [2020-04-16 12:03:53] (25535 lines) /app/public/data/wiki_combined_la/publish_identifiers.tsv
[INFO] [2020-04-16 12:03:54] (471184 lines) /app/public/data/wiki_combined_la/publish_node_ancestors.tsv
[INFO] [2020-04-16 12:03:56] (25535 lines) /app/public/data/wiki_combined_la/publish_scientific_names.tsv
[INFO] [2020-04-16 12:03:57] (1079978 lines) /app/public/data/wiki_combined_la/publish_articles.tsv
[INFO] [2020-04-16 12:03:58] (93555 lines) /app/public/data/wiki_combined_la/publish_content_sections.tsv
[STOP] [2020-04-16 12:03:59] complete_harvest_instance
[START] [2020-04-16 12:03:59] completed
[STOP] [2020-04-16 12:03:59] completed
[STOP] [2020-04-16 12:03:59] logged process, took 4140.08
[INFO] [2020-05-28 12:33:54] ## HARVEST: type = re_download_opendata_-harvest
[INFO] [2020-05-28 12:33:56] ## remove_type: ScientificName
[INFO] [2020-05-28 12:33:56] ++ Calling delete_all on 25535 instances...
[INFO] [2020-05-28 12:33:59] [12:33:59.430] Removed 25535 Scientificnames
[INFO] [2020-05-28 12:33:59] ## remove_type: Vernacular
[INFO] [2020-05-28 12:33:59] ++ Calling delete_all on 0 instances...
[INFO] [2020-05-28 12:33:59] [12:33:59.433] Removed 0 Vernaculars
[INFO] [2020-05-28 12:33:59] ## remove_type: Article
[INFO] [2020-05-28 12:33:59] ++ Calling delete_all on 93555 instances...
[INFO] [2020-05-28 12:34:29] [12:34:29.918] Removed 93555 Articles
[INFO] [2020-05-28 12:34:29] ## remove_type: Medium
[INFO] [2020-05-28 12:34:29] ++ Calling delete_all on 0 instances...
[INFO] [2020-05-28 12:34:29] [12:34:29.922] Removed 0 Media
[INFO] [2020-05-28 12:34:29] ## remove_type: Trait
[INFO] [2020-05-28 12:34:29] ++ Calling delete_all on 0 instances...
[INFO] [2020-05-28 12:34:29] [12:34:29.925] Removed 0 Traits
[INFO] [2020-05-28 12:34:29] ## remove_type: MetaTrait
[INFO] [2020-05-28 12:34:29] ++ Calling delete_all on 0 instances...
[INFO] [2020-05-28 12:34:29] [12:34:29.955] Removed 0 Metatraits
[INFO] [2020-05-28 12:34:29] ## remove_type: OccurrenceMetadatum
[INFO] [2020-05-28 12:34:30] ++ Calling delete_all on 0 instances...
[INFO] [2020-05-28 12:34:30] [12:34:30.089] Removed 0 Occurrencemetadata
[INFO] [2020-05-28 12:34:30] ## remove_type: Assoc
[INFO] [2020-05-28 12:34:30] ++ Calling delete_all on 0 instances...
[INFO] [2020-05-28 12:34:30] [12:34:30.092] Removed 0 Assocs
[INFO] [2020-05-28 12:34:30] ## remove_type: MetaAssoc
[INFO] [2020-05-28 12:34:30] ++ Calling delete_all on 0 instances...
[INFO] [2020-05-28 12:34:30] [12:34:30.103] Removed 0 Metaassocs
[INFO] [2020-05-28 12:34:30] ## remove_type: Identifier
[INFO] [2020-05-28 12:34:30] ++ Calling delete_all on 25535 instances...
[INFO] [2020-05-28 12:34:34] [12:34:34.110] Removed 25535 Identifiers
[INFO] [2020-05-28 12:34:34] ## remove_type: Reference
[INFO] [2020-05-28 12:34:34] ++ Calling delete_all on 0 instances...
[INFO] [2020-05-28 12:34:34] [12:34:34.114] Removed 0 References
[INFO] [2020-05-28 12:34:36] Starting batch with ID 68875836...
[INFO] [2020-05-28 12:34:38] Starting batch with ID 68886742...
[INFO] [2020-05-28 12:34:40] Starting batch with ID 68890800...
[INFO] [2020-05-28 12:34:41] Starting batch with ID 68876581...
[INFO] [2020-05-28 12:34:43] Starting batch with ID 68895805...
[INFO] [2020-05-28 12:34:44] Starting batch with ID 68895805...
[INFO] [2020-05-28 12:34:45] Starting batch with ID 68896525...
[INFO] [2020-05-28 12:34:46] Starting batch with ID 68884636...
[INFO] [2020-05-28 12:34:47] Starting batch with ID 68896841...
[INFO] [2020-05-28 12:34:47] Starting batch with ID 68900677...
[INFO] [2020-05-28 12:34:48] Starting batch with ID 68900677...
[INFO] [2020-05-28 12:34:48] Starting batch with ID 68900677...
[INFO] [2020-05-28 12:34:48] Starting batch with ID 68900677...
[INFO] [2020-05-28 12:34:48] Starting batch with ID 68900677...
[INFO] [2020-05-28 12:34:48] ## remove_type: Node
[INFO] [2020-05-28 12:34:48] ++ Calling delete_all on 25535 instances...
[INFO] [2020-05-28 12:34:52] [12:34:52.502] Removed 25535 Nodes
[START] [2020-05-28 12:35:58] logged process
[START] [2020-05-28 12:35:58] Creating resource from OpenData
[START] [2020-05-28 12:36:07] logged process
[START] [2020-05-28 12:36:07] Parse meta.xml file and create formats with fields
[STOP] [2020-05-28 12:36:09] Parse meta.xml file and create formats with fields
[STOP] [2020-05-28 12:36:09] Creating resource from OpenData
[START] [2020-05-28 12:36:09] logged process
[START] [2020-05-28 12:36:09] create_harvest_instance
[STOP] [2020-05-28 12:36:10] create_harvest_instance
[START] [2020-05-28 12:36:10] fetch_files
[STOP] [2020-05-28 12:36:10] fetch_files
[START] [2020-05-28 12:36:10] validate_each_file
[STOP] [2020-05-28 12:36:23] validate_each_file
[START] [2020-05-28 12:36:23] convert_to_csv
[CMD] [2020-05-28 12:36:23] /usr/bin/sort /app/public/converted_csv/wiki_combined_la_nodes_21016.csv > /app/public/converted_csv/wiki_combined_la_nodes_21016.csv_sorted
[CMD] [2020-05-28 12:36:24] /usr/bin/sort /app/public/converted_csv/wiki_combined_la_media_21017.csv > /app/public/converted_csv/wiki_combined_la_media_21017.csv_sorted
[STOP] [2020-05-28 12:36:25] convert_to_csv
[START] [2020-05-28 12:36:25] calculate_delta
[CMD] [2020-05-28 12:36:25] echo "0a" > /app/public/diff/wiki_combined_la_nodes_21016.diff
[CMD] [2020-05-28 12:36:25] tail -n +1 /app/public/converted_csv/wiki_combined_la_nodes_21016.csv >> /app/public/diff/wiki_combined_la_nodes_21016.diff
[CMD] [2020-05-28 12:36:25] echo "." >> /app/public/diff/wiki_combined_la_nodes_21016.diff
[CMD] [2020-05-28 12:36:25] echo "0a" > /app/public/diff/wiki_combined_la_media_21017.diff
[CMD] [2020-05-28 12:36:25] tail -n +1 /app/public/converted_csv/wiki_combined_la_media_21017.csv >> /app/public/diff/wiki_combined_la_media_21017.diff
[CMD] [2020-05-28 12:36:25] echo "." >> /app/public/diff/wiki_combined_la_media_21017.diff
[STOP] [2020-05-28 12:36:25] calculate_delta
[START] [2020-05-28 12:36:25] parse_diff_and_store
[INFO] [2020-05-28 12:36:25] Loading nodes diff file into memory (true lines)...
[INFO] [2020-05-28 12:36:36] Loading media diff file into memory (true lines)...
[INFO] [2020-05-28 12:44:36] Storing 25798 ScientificNames
[INFO] [2020-05-28 12:44:36] Processing group of 25798 in 26 groups of 1000
[INFO] [2020-05-28 12:44:44] Average Time: 0.301
[INFO] [2020-05-28 12:44:44] Total Time: 8s
[INFO] [2020-05-28 12:44:44] last 3 / first 3: 0.83
[INFO] [2020-05-28 12:44:44] Std.Dev: 0.03162277660168379; Max: 0.4
[INFO] [2020-05-28 12:44:44] Storing 25798 Identifiers
[INFO] [2020-05-28 12:44:44] Processing group of 25798 in 26 groups of 1000
[INFO] [2020-05-28 12:44:47] Average Time: 0.109
[INFO] [2020-05-28 12:44:47] Total Time: 3s
[INFO] [2020-05-28 12:44:47] last 3 / first 3: 0.86
[INFO] [2020-05-28 12:44:47] Std.Dev: 0.0; Max: 0.15
[INFO] [2020-05-28 12:44:47] Storing 25798 Nodes
[INFO] [2020-05-28 12:44:47] Processing group of 25798 in 26 groups of 1000
[INFO] [2020-05-28 12:44:56] Average Time: 0.34
[INFO] [2020-05-28 12:44:56] Total Time: 9s
[INFO] [2020-05-28 12:44:56] last 3 / first 3: 0.52
[INFO] [2020-05-28 12:44:56] Std.Dev: 0.1224744871391589; Max: 0.63
[INFO] [2020-05-28 12:44:56] Storing 96109 ArticlesSections
[INFO] [2020-05-28 12:44:56] Processing group of 96109 in 97 groups of 1000
[INFO] [2020-05-28 12:45:01] Average Time: 0.052
[INFO] [2020-05-28 12:45:01] Total Time: 6s
[INFO] [2020-05-28 12:45:01] last 3 / first 3: 0.65
[INFO] [2020-05-28 12:45:01] Std.Dev: 0.0; Max: 0.2
[INFO] [2020-05-28 12:45:01] Storing 96109 Articles
[INFO] [2020-05-28 12:45:01] Processing group of 96109 in 97 groups of 1000
[INFO] [2020-05-28 12:46:06] Average Time: 0.664
[INFO] [2020-05-28 12:46:06] Total Time: 1m5s
[INFO] [2020-05-28 12:46:06] last 3 / first 3: 0.65
[INFO] [2020-05-28 12:46:06] Std.Dev: 0.2024845673131659; Max: 2.45
[STOP] [2020-05-28 12:46:06] parse_diff_and_store
[START] [2020-05-28 12:46:06] resolve_keys
[INFO] [2020-05-28 12:48:00] Occurrences to nodes (through scientific_names)...
[INFO] [2020-05-28 12:48:00] traits to occurrences...
[INFO] [2020-05-28 12:48:00] traits to nodes (through occurrences)...
[INFO] [2020-05-28 12:48:00] Traits to sex term...
[INFO] [2020-05-28 12:48:00] Traits to lifestage term...
[INFO] [2020-05-28 12:48:00] MetaTraits to traits...
[INFO] [2020-05-28 12:48:00] MetaTraits (simple, measurement row refers to parent) to traits...
[INFO] [2020-05-28 12:48:00] Assocs to occurrences...
[INFO] [2020-05-28 12:48:00] Assocs to nodes...
[INFO] [2020-05-28 12:48:00] Assoc to sex term...
[INFO] [2020-05-28 12:48:00] Assoc to lifestage term...
[STOP] [2020-05-28 12:48:00] resolve_keys
[START] [2020-05-28 12:48:00] hold_for_later_1
[STOP] [2020-05-28 12:48:00] hold_for_later_1
[START] [2020-05-28 12:48:00] hold_for_later_2
[STOP] [2020-05-28 12:48:00] hold_for_later_2
[START] [2020-05-28 12:48:01] resolve_missing_parents
[STOP] [2020-05-28 12:48:12] resolve_missing_parents
[START] [2020-05-28 12:48:12] rebuild_nodes
[START] [2020-05-28 12:48:12] Flattener#flatten
[START] [2020-05-28 12:48:12] Flattener#study_resource
[START] [2020-05-28 12:48:12] Flattener#build_ancestry
[STOP] [2020-05-28 12:48:15] Flattener#build_ancestry
[INFO] [2020-05-28 12:48:15] 25798 ancestry keys
[START] [2020-05-28 12:48:15] build_node_ancestors
[INFO] [2020-05-28 12:48:15] old ancestors deleted.
[STOP] [2020-05-28 12:49:10] build_node_ancestors
[START] [2020-05-28 12:49:11] Flattener#propagate_ancestor_ids
[STOP] [2020-05-28 12:49:32] Flattener#propagate_ancestor_ids
[STOP] [2020-05-28 12:49:32] Flattener#flatten
[STOP] [2020-05-28 12:49:32] rebuild_nodes
[START] [2020-05-28 12:49:32] resolve_missing_media_owners
[STOP] [2020-05-28 12:49:32] resolve_missing_media_owners
[START] [2020-05-28 12:49:32] sanitize_media_verbatims
[STOP] [2020-05-28 12:49:32] sanitize_media_verbatims
[START] [2020-05-28 12:49:32] queue_downloads
[STOP] [2020-05-28 12:49:32] queue_downloads
[START] [2020-05-28 12:49:32] parse_names
[WARN] [2020-05-28 12:49:32] I see 25798 names which still need to be parsed.
[WARN] [2020-05-28 12:49:51] I see 29 names which still need to be parsed.
[STOP] [2020-05-28 12:49:52] parse_names
[START] [2020-05-28 12:49:52] denormalize_canonical_names_to_nodes
[STOP] [2020-05-28 12:49:52] denormalize_canonical_names_to_nodes
[START] [2020-05-28 12:49:52] match_nodes
[START] [2020-05-28 12:49:52] map_all_nodes_to_pages
[STOP] [2020-05-28 14:11:13] map_all_nodes_to_pages
[INFO] [2020-05-28 14:11:13] 2898 Unmatched nodes (of 25798)! That's too many to output. First 10: Biota (#78576802); Acytota (#78572353); Prokaryota (#78573785); Proteoarchaeota (#78575554); DPANN (#78577250); Aigarchaeota (#78582726); Bacteria (#78564848); Eubacteria (#78577234); Melainabacteria (#78575036); Negibacteria (#78581418)
[START] [2020-05-28 14:11:13] update_nodes
[STOP] [2020-05-28 14:11:24] update_nodes
[STOP] [2020-05-28 14:11:24] match_nodes
[START] [2020-05-28 14:11:24] reindex_search
[STOP] [2020-05-28 14:13:31] reindex_search
[START] [2020-05-28 14:13:31] normalize_units
[STOP] [2020-05-28 14:13:31] normalize_units
[START] [2020-05-28 14:13:31] calculate_statistics
[STOP] [2020-05-28 14:13:32] calculate_statistics
[START] [2020-05-28 14:13:32] complete_harvest_instance
[START] [2020-05-28 14:13:32] overall_tsv_creation
[INFO] [2020-05-28 14:13:32] Processing group of 25798 in 3 batches of 10000
[INFO] [2020-05-28 14:20:00] Average Time: 73.687
[INFO] [2020-05-28 14:20:00] Total Time: 6m29s
[STOP] [2020-05-28 14:20:00] overall_tsv_creation
[INFO] [2020-05-28 14:20:00] Done. Check your files:
[INFO] [2020-05-28 14:20:01] (25798 lines) /app/public/data/wiki_combined_la/publish_nodes.tsv
[INFO] [2020-05-28 14:20:01] (25798 lines) /app/public/data/wiki_combined_la/publish_identifiers.tsv
[INFO] [2020-05-28 14:20:01] (530455 lines) /app/public/data/wiki_combined_la/publish_node_ancestors.tsv
[INFO] [2020-05-28 14:20:01] (25798 lines) /app/public/data/wiki_combined_la/publish_scientific_names.tsv
[INFO] [2020-05-28 14:20:01] (1141223 lines) /app/public/data/wiki_combined_la/publish_articles.tsv
[INFO] [2020-05-28 14:20:01] (96109 lines) /app/public/data/wiki_combined_la/publish_content_sections.tsv
[STOP] [2020-05-28 14:20:02] complete_harvest_instance
[START] [2020-05-28 14:20:02] completed
[STOP] [2020-05-28 14:20:02] completed
[STOP] [2020-05-28 14:20:02] logged process, took 6232.67

Latest Process

28 May 12:36 EXITED currently processing 0