Stage:
completed
Fetched:
15 Dec 19:05
Validated:
15 Dec 19:05
Deltas Created
15 Dec 19:05
Units Normalized:
15 Dec 19:07
Ancestry Built:
15 Dec 19:06
Nodes Matched:
15 Dec 19:07
Names Parsed:
15 Dec 19:06
New Models Stored:
15 Dec 19:05
Indexed:
15 Dec 19:07
Completed:
15 Dec 19:13
Time to Harvest:
less than a minute
Harvesting Log
(285 lines)
[INFO] [2021-12-15 19:05:14] Created harvest instance #4081
[STOP] [2021-12-15 19:05:14] create_harvest_instance
[START] [2021-12-15 19:05:14] fetch_files
[STOP] [2021-12-15 19:05:14] fetch_files
[START] [2021-12-15 19:05:14] validate_each_file
[INFO] [2021-12-15 19:05:14] Looping over 4 formats...
[INFO] [2021-12-15 19:05:14] ...nodes (/app/public/data/memoirs_am_entom/taxon.tab)
[INFO] [2021-12-15 19:05:14] Valid: /app/public/converted_csv/memoirs_am_entom_nodes_4081.csv (5967 lines)
[INFO] [2021-12-15 19:05:14] ...media (/app/public/data/memoirs_am_entom/media_resource.tab)
[INFO] [2021-12-15 19:05:15] Valid: /app/public/converted_csv/memoirs_am_entom_media_4081.csv (1935 lines)
[INFO] [2021-12-15 19:05:15] ...occurrences (/app/public/data/memoirs_am_entom/occurrence_specific.tab)
[INFO] [2021-12-15 19:05:15] Valid: /app/public/converted_csv/memoirs_am_entom_occurrences_4081.csv (12385 lines)
[INFO] [2021-12-15 19:05:15] ...measurements (/app/public/data/memoirs_am_entom/measurement_or_fact_specific.tab)
[INFO] [2021-12-15 19:05:16] Valid: /app/public/converted_csv/memoirs_am_entom_measurements_4081.csv (12385 lines)
[STOP] [2021-12-15 19:05:16] validate_each_file
[START] [2021-12-15 19:05:16] convert_to_csv
[INFO] [2021-12-15 19:05:16] Looping over 4 formats...
[INFO] [2021-12-15 19:05:16] ...nodes (/app/public/data/memoirs_am_entom/taxon.tab)
[CMD] [2021-12-15 19:05:16] /usr/bin/sort /app/public/converted_csv/memoirs_am_entom_nodes_4081.csv > /app/public/converted_csv/memoirs_am_entom_nodes_4081.csv_sorted
[INFO] [2021-12-15 19:05:16] Converted: /app/public/converted_csv/memoirs_am_entom_nodes_4081.csv (5967 lines)
[INFO] [2021-12-15 19:05:16] ...media (/app/public/data/memoirs_am_entom/media_resource.tab)
[CMD] [2021-12-15 19:05:16] /usr/bin/sort /app/public/converted_csv/memoirs_am_entom_media_4081.csv > /app/public/converted_csv/memoirs_am_entom_media_4081.csv_sorted
[INFO] [2021-12-15 19:05:16] Converted: /app/public/converted_csv/memoirs_am_entom_media_4081.csv (1935 lines)
[INFO] [2021-12-15 19:05:16] ...occurrences (/app/public/data/memoirs_am_entom/occurrence_specific.tab)
[CMD] [2021-12-15 19:05:16] /usr/bin/sort /app/public/converted_csv/memoirs_am_entom_occurrences_4081.csv > /app/public/converted_csv/memoirs_am_entom_occurrences_4081.csv_sorted
[INFO] [2021-12-15 19:05:16] Converted: /app/public/converted_csv/memoirs_am_entom_occurrences_4081.csv (12385 lines)
[INFO] [2021-12-15 19:05:16] ...measurements (/app/public/data/memoirs_am_entom/measurement_or_fact_specific.tab)
[CMD] [2021-12-15 19:05:16] /usr/bin/sort /app/public/converted_csv/memoirs_am_entom_measurements_4081.csv > /app/public/converted_csv/memoirs_am_entom_measurements_4081.csv_sorted
[INFO] [2021-12-15 19:05:16] Converted: /app/public/converted_csv/memoirs_am_entom_measurements_4081.csv (12385 lines)
[STOP] [2021-12-15 19:05:16] convert_to_csv
[START] [2021-12-15 19:05:16] calculate_delta
[INFO] [2021-12-15 19:05:16] Looping over 4 formats...
[INFO] [2021-12-15 19:05:16] ...nodes (/app/public/data/memoirs_am_entom/taxon.tab)
[CMD] [2021-12-15 19:05:16] echo "0a" > /app/public/diff/memoirs_am_entom_nodes_4081.diff
[CMD] [2021-12-15 19:05:16] tail -n +1 /app/public/converted_csv/memoirs_am_entom_nodes_4081.csv >> /app/public/diff/memoirs_am_entom_nodes_4081.diff
[CMD] [2021-12-15 19:05:16] echo "." >> /app/public/diff/memoirs_am_entom_nodes_4081.diff
[INFO] [2021-12-15 19:05:16] Created diff: /app/public/diff/memoirs_am_entom_nodes_4081.diff (5969 lines)
[INFO] [2021-12-15 19:05:16] ...media (/app/public/data/memoirs_am_entom/media_resource.tab)
[CMD] [2021-12-15 19:05:16] echo "0a" > /app/public/diff/memoirs_am_entom_media_4081.diff
[CMD] [2021-12-15 19:05:17] tail -n +1 /app/public/converted_csv/memoirs_am_entom_media_4081.csv >> /app/public/diff/memoirs_am_entom_media_4081.diff
[CMD] [2021-12-15 19:05:17] echo "." >> /app/public/diff/memoirs_am_entom_media_4081.diff
[INFO] [2021-12-15 19:05:17] Created diff: /app/public/diff/memoirs_am_entom_media_4081.diff (1937 lines)
[INFO] [2021-12-15 19:05:17] ...occurrences (/app/public/data/memoirs_am_entom/occurrence_specific.tab)
[CMD] [2021-12-15 19:05:17] echo "0a" > /app/public/diff/memoirs_am_entom_occurrences_4081.diff
[CMD] [2021-12-15 19:05:17] tail -n +1 /app/public/converted_csv/memoirs_am_entom_occurrences_4081.csv >> /app/public/diff/memoirs_am_entom_occurrences_4081.diff
[CMD] [2021-12-15 19:05:17] echo "." >> /app/public/diff/memoirs_am_entom_occurrences_4081.diff
[INFO] [2021-12-15 19:05:17] Created diff: /app/public/diff/memoirs_am_entom_occurrences_4081.diff (12387 lines)
[INFO] [2021-12-15 19:05:17] ...measurements (/app/public/data/memoirs_am_entom/measurement_or_fact_specific.tab)
[CMD] [2021-12-15 19:05:17] echo "0a" > /app/public/diff/memoirs_am_entom_measurements_4081.diff
[CMD] [2021-12-15 19:05:17] tail -n +1 /app/public/converted_csv/memoirs_am_entom_measurements_4081.csv >> /app/public/diff/memoirs_am_entom_measurements_4081.diff
[CMD] [2021-12-15 19:05:17] echo "." >> /app/public/diff/memoirs_am_entom_measurements_4081.diff
[INFO] [2021-12-15 19:05:17] Created diff: /app/public/diff/memoirs_am_entom_measurements_4081.diff (12387 lines)
[STOP] [2021-12-15 19:05:17] calculate_delta
[START] [2021-12-15 19:05:17] parse_diff_and_store
[INFO] [2021-12-15 19:05:17] Handling diff: /app/public/diff/memoirs_am_entom_nodes_4081.diff (5969 lines)
[INFO] [2021-12-15 19:05:17] Loading nodes diff file into memory (5969 /app/public/diff/memoirs_am_entom_nodes_4081.diff lines)...
[WARN] [2021-12-15 19:05:18] Filtered Scientific Name `i\leiodes abdominalis` to `ileiodes abdominalis`
[INFO] [2021-12-15 19:05:19] Handling diff: /app/public/diff/memoirs_am_entom_media_4081.diff (1937 lines)
[INFO] [2021-12-15 19:05:19] Loading media diff file into memory (1937 /app/public/diff/memoirs_am_entom_media_4081.diff lines)...
[INFO] [2021-12-15 19:05:22] Handling diff: /app/public/diff/memoirs_am_entom_occurrences_4081.diff (12387 lines)
[INFO] [2021-12-15 19:05:23] Loading occurrences diff file into memory (12387 /app/public/diff/memoirs_am_entom_occurrences_4081.diff lines)...
[INFO] [2021-12-15 19:05:24] Handling diff: /app/public/diff/memoirs_am_entom_measurements_4081.diff (12387 lines)
[INFO] [2021-12-15 19:05:24] Loading measurements diff file into memory (12387 /app/public/diff/memoirs_am_entom_measurements_4081.diff lines)...
[INFO] [2021-12-15 19:05:32] Storing 5967 ScientificNames
[INFO] [2021-12-15 19:05:32] Processing group of 5967 in 6 groups of 1000
[INFO] [2021-12-15 19:05:34] Average Time: 0.338
[INFO] [2021-12-15 19:05:34] Total Time: 3s
[INFO] [2021-12-15 19:05:34] Storing 5967 Nodes
[INFO] [2021-12-15 19:05:34] Processing group of 5967 in 6 groups of 1000
[INFO] [2021-12-15 19:05:36] Average Time: 0.295
[INFO] [2021-12-15 19:05:36] Total Time: 2s
[INFO] [2021-12-15 19:05:36] Storing 1935 BibliographicCitations
[INFO] [2021-12-15 19:05:36] Processing group of 1935 in 2 groups of 1000
[INFO] [2021-12-15 19:05:36] Average Time: 0.145
[INFO] [2021-12-15 19:05:36] Total Time: 1s
[INFO] [2021-12-15 19:05:36] Storing 1935 ArticlesSections
[INFO] [2021-12-15 19:05:36] Processing group of 1935 in 2 groups of 1000
[INFO] [2021-12-15 19:05:36] Average Time: 0.05
[INFO] [2021-12-15 19:05:36] Total Time: 1s
[INFO] [2021-12-15 19:05:36] Storing 1935 Articles
[INFO] [2021-12-15 19:05:36] Processing group of 1935 in 2 groups of 1000
[INFO] [2021-12-15 19:05:37] Average Time: 0.63
[INFO] [2021-12-15 19:05:37] Total Time: 2s
[INFO] [2021-12-15 19:05:37] Storing 12385 Occurrences
[INFO] [2021-12-15 19:05:37] Processing group of 12385 in 13 groups of 1000
[INFO] [2021-12-15 19:05:40] Average Time: 0.16
[INFO] [2021-12-15 19:05:40] Total Time: 3s
[INFO] [2021-12-15 19:05:40] last 3 / first 3: 0.7
[INFO] [2021-12-15 19:05:40] Std.Dev: 0.130384048104053; Max: 0.51
[INFO] [2021-12-15 19:05:40] Storing 12385 Traits
[INFO] [2021-12-15 19:05:40] Processing group of 12385 in 13 groups of 1000
[INFO] [2021-12-15 19:05:45] Average Time: 0.362
[INFO] [2021-12-15 19:05:45] Total Time: 5s
[INFO] [2021-12-15 19:05:45] last 3 / first 3: 0.49
[INFO] [2021-12-15 19:05:45] Std.Dev: 0.1673320053068151; Max: 0.77
[INFO] [2021-12-15 19:05:45] Storing 24770 MetaTraits
[INFO] [2021-12-15 19:05:45] Processing group of 24770 in 25 groups of 1000
[INFO] [2021-12-15 19:05:48] Average Time: 0.124
[INFO] [2021-12-15 19:05:48] Total Time: 4s
[INFO] [2021-12-15 19:05:48] last 3 / first 3: 0.78
[INFO] [2021-12-15 19:05:48] Std.Dev: 0.03162277660168379; Max: 0.2
[STOP] [2021-12-15 19:05:48] parse_diff_and_store
[START] [2021-12-15 19:05:48] resolve_keys
[INFO] [2021-12-15 19:06:10] Occurrences to nodes (through scientific_names)...
[INFO] [2021-12-15 19:06:10] traits to occurrences...
[INFO] [2021-12-15 19:06:10] traits to nodes (through occurrences)...
[INFO] [2021-12-15 19:06:11] Traits to sex term...
[INFO] [2021-12-15 19:06:11] Traits to lifestage term...
[INFO] [2021-12-15 19:06:12] MetaTraits to traits...
[INFO] [2021-12-15 19:06:16] MetaTraits (simple, measurement row refers to parent) to traits...
[INFO] [2021-12-15 19:06:16] Assocs to occurrences...
[INFO] [2021-12-15 19:06:16] Assocs to nodes...
[INFO] [2021-12-15 19:06:16] Assoc to sex term...
[INFO] [2021-12-15 19:06:16] Assoc to lifestage term...
[INFO] [2021-12-15 19:06:16] MetaAssoc to assocs...
[STOP] [2021-12-15 19:06:16] resolve_keys
[START] [2021-12-15 19:06:16] hold_for_later_1
[STOP] [2021-12-15 19:06:16] hold_for_later_1
[START] [2021-12-15 19:06:16] hold_for_later_2
[STOP] [2021-12-15 19:06:16] hold_for_later_2
[START] [2021-12-15 19:06:16] resolve_missing_parents
[STOP] [2021-12-15 19:06:16] resolve_missing_parents
[START] [2021-12-15 19:06:16] rebuild_nodes
[START] [2021-12-15 19:06:16] Flattener#flatten
[START] [2021-12-15 19:06:16] Flattener#study_resource
[START] [2021-12-15 19:06:16] Flattener#build_ancestry
[STOP] [2021-12-15 19:06:16] Flattener#build_ancestry
[INFO] [2021-12-15 19:06:16] 5967 ancestry keys
[START] [2021-12-15 19:06:16] build_node_ancestors
[INFO] [2021-12-15 19:06:16] old ancestors deleted.
[STOP] [2021-12-15 19:06:16] build_node_ancestors
[WARN] [2021-12-15 19:06:16] Flattener: nothing to flatten! (Completely flat resource?)
[STOP] [2021-12-15 19:06:16] Flattener#flatten
[STOP] [2021-12-15 19:06:16] rebuild_nodes
[START] [2021-12-15 19:06:16] resolve_missing_media_owners
[STOP] [2021-12-15 19:06:16] resolve_missing_media_owners
[START] [2021-12-15 19:06:16] sanitize_media_verbatims
[STOP] [2021-12-15 19:06:16] sanitize_media_verbatims
[START] [2021-12-15 19:06:16] queue_downloads
[STOP] [2021-12-15 19:06:16] queue_downloads
[START] [2021-12-15 19:06:16] parse_names
[WARN] [2021-12-15 19:06:16] I see 5967 names which still need to be parsed.
[WARN] [2021-12-15 19:06:17] Names to parse: 5967 formatted: 5967 learned: 5967 parsed: 5967
[STOP] [2021-12-15 19:06:22] parse_names
[START] [2021-12-15 19:06:22] denormalize_canonical_names_to_nodes
[STOP] [2021-12-15 19:06:22] denormalize_canonical_names_to_nodes
[START] [2021-12-15 19:06:22] match_nodes
[START] [2021-12-15 19:06:22] map_all_nodes_to_pages
[STOP] [2021-12-15 19:07:33] map_all_nodes_to_pages
[INFO] [2021-12-15 19:07:33] 2221 Unmatched nodes (of 5967)! That's too many to output. Full list in /app/public/data/memoirs_am_entom/unmatched_nodes.txt ; First 10: Canonical: Precis sophia; Node#101293796; ResourceID: 0013002e010e8d39a5eed731175414b7; Canonical: Ichneumon luctus; Node#101293799; ResourceID: 001eac025a20b512433d25b6ce4cab07; Canonical: Tryphon croceiventris; Node#101293801; ResourceID: 001fe4c3e50d7e27e420631243882bd2; Canonical: Sphacophilus defius; Node#101293802; ResourceID: 002f65a3a8da6856716dbfa6f78395b2; Canonical: Porizon agilis; Node#101293803; ResourceID: 00466cbf2be4a423337a6399f4b010f2; Canonical: Amauris egialea egialea; Node#101293809; ResourceID: 008b73b0914feeaa798540fd71e4f624; Canonical: Ichneumon expunctus; Node#101293812; ResourceID: 0092b031ad1ff1eb1fa122adf7d512aa; Canonical: Hoplismenus limatus; Node#101293816; ResourceID: 00e26d09eeec7d520b56149dd156aed0; Canonical: Ichneumon infidelis; Node#101293817; ResourceID: 00f05d4b0a6e81e24db3c0de7b64d4cf; Canonical: Mutilla pacata; Node#101293818; ResourceID: 00fc05fcde274c2a8373fe5367700677
[START] [2021-12-15 19:07:33] update_nodes
[STOP] [2021-12-15 19:07:35] update_nodes
[STOP] [2021-12-15 19:07:35] match_nodes
[START] [2021-12-15 19:07:35] reindex_search
[STOP] [2021-12-15 19:07:40] reindex_search
[START] [2021-12-15 19:07:40] normalize_units
[STOP] [2021-12-15 19:07:40] normalize_units
[START] [2021-12-15 19:07:40] calculate_statistics
[2021-12-15 19:07:41] ZERO NODE ANCESTORS. Is this actually a completely flat resource?
[2021-12-15 19:07:41] (NEAR) DUPLICATE TRAITS FOUND! There are only 12266 (of 12385 total) unique traits.
[2021-12-15 19:09:10] (Near) duplicate trait pairs (up to 100):
[2021-12-15 19:09:10] (resource_pk: 000975219d1963d4b989bcd3c1b957cd_120083_ENV, id: 241136841), (resource_pk: b9e6c83a7b20c9752108b4dba0371d27_120083_ENV, id: 241145833)
[2021-12-15 19:09:10] (resource_pk: 0127c23c383c888d1812ec2833337d8f_118986_ENV, id: 241136892), (resource_pk: 20bba02f770d96244758d8c977fd4299_118986_ENV, id: 241138424)
[2021-12-15 19:09:10] (resource_pk: 01e4a248f5d47354a8430ad1e51ff153_118978_ENV, id: 241136935), (resource_pk: 907dc836c31a8b5adb6dade168a26769_118978_ENV, id: 241143875)
[2021-12-15 19:09:10] (resource_pk: 02f8199ecb0db4725b23d1e26b920947_120083_ENV, id: 241136987), (resource_pk: c49e78b7424b8a1ea0b4222eeda92b77_120083_ENV, id: 241146326)
[2021-12-15 19:09:10] (resource_pk: 04f3bb356d5c2fa953fcc4ce787aa338_118237_ENV, id: 241137078), (resource_pk: d1db3256d888551388480359fff97275_118237_ENV, id: 241146963)
[2021-12-15 19:09:10] (resource_pk: 050d740f323edceff9b3f3c26604634b_118237_ENV, id: 241137084), (resource_pk: f70fa700c0f68372ee329ca80f14a3d0_118237_ENV, id: 241148767)
[2021-12-15 19:09:10] (resource_pk: 0557ac485b788b428afca4beefb1bf1c_120083_ENV, id: 241137106), (resource_pk: 18ca52f11fab9eaa0c23ee67cb96151f_120083_ENV, id: 241138039)
[2021-12-15 19:09:10] (resource_pk: 073d2a90135d9eb0eb5cfec281a35d83_118978_ENV, id: 241137201), (resource_pk: 9d18bf5540dec3a6b194ffcd0c10c0ac_118978_ENV, id: 241144489)
[2021-12-15 19:09:10] (resource_pk: 084914fefc1bd9ab413cef36c33beed7_118986_ENV, id: 241137248), (resource_pk: 196680e9c3efd4d1c705ed9ad1dba411_118986_ENV, id: 241138068)
[2021-12-15 19:09:10] (resource_pk: 0b6af1f5e04845e0909018c8cc8d5e3b_120083_ENV, id: 241137384), (resource_pk: 84cdfb57ae56d122c9ba31b48b713c98_120083_ENV, id: 241143320)
[2021-12-15 19:09:10] (resource_pk: 0bda220ef45e74542acfed6b292faf98_120083_ENV, id: 241137403), (resource_pk: 46b10f960e5f0af7cacfd7c97bfb0418_120083_ENV, id: 241140280)
[2021-12-15 19:09:10] (resource_pk: 0e526e59529d8a33cb10b318f4403abe_120083_ENV, id: 241137515), (resource_pk: 215956f19b0d84746c2d36c70f7fe428_120083_ENV, id: 241138454)
[2021-12-15 19:09:10] (resource_pk: 0ec228f9ab9eed7bac0e15059a046f43_120083_ENV, id: 241137534), (resource_pk: 8cdbb0650be77da6539b4d0d04b1f8b9_120083_ENV, id: 241143687)
[2021-12-15 19:09:10] (resource_pk: 0f10b333817ae540fce53c7c972ab9ef_119188_ENV, id: 241137546), (resource_pk: 84af1b5072b8c0014c4d4f78c4f33dcf_119188_ENV, id: 241143312)
[2021-12-15 19:09:10] (resource_pk: 0f5355f5e146798f3069740ee8407b84_120083_ENV, id: 241137557), (resource_pk: 1d7859b2c0563a282c0c5128820d3447_120083_ENV, id: 241138251)
[2021-12-15 19:09:10] (resource_pk: 14473b1674b564694704880cb968dc87_118986_ENV, id: 241137800), (resource_pk: 5195f15296df3eae0376f4253fb92d6f_118986_ENV, id: 241140762)
[2021-12-15 19:09:10] (resource_pk: 184ca86e03c20ec8f1f795eff9e5c503_120083_ENV, id: 241138009), (resource_pk: 7d8acd7fc618265177ab37c042af717f_120083_ENV, id: 241143010)
[2021-12-15 19:09:10] (resource_pk: 189f19209ecd04123ea5a003310e95af_118986_ENV, id: 241138030), (resource_pk: aeeb33a20edf407c5de2882cb910512a_118986_ENV, id: 241145310)
[2021-12-15 19:09:10] (resource_pk: 1a86b5f34ec0894ed36b1afdf16d4faa_120083_ENV, id: 241138126), (resource_pk: 7fe02ebc0b79440b2d92c59dd17312b3_120083_ENV, id: 241143110)
[2021-12-15 19:09:10] (resource_pk: 1b22f640e78ca49c106d799b5bd1ec1d_120083_ENV, id: 241138154), (resource_pk: 6ac7bb6f800e8720021b39657caaf0d5_120083_ENV, id: 241142073)
[2021-12-15 19:09:10] (resource_pk: 1bc170cfa03be0c203bf6c815146f7b4_119188_ENV, id: 241138181), (resource_pk: e745fe8550b22bbff791a9b80709d160_119188_ENV, id: 241148031)
[2021-12-15 19:09:10] (resource_pk: 1c241e4943aaeea45ea6a4a96234945a_120083_ENV, id: 241138197), (resource_pk: b475f8f25d6b488cba9ca0eedfedf4c6_120083_ENV, id: 241145567)
[2021-12-15 19:09:10] (resource_pk: 1da1e69a53b746b6e51f07f6c86c5330_118936_ENV, id: 241138265), (resource_pk: d1177fc8ad18ee0d6956bc32d9c79130_118936_ENV, id: 241146928)
[2021-12-15 19:09:10] (resource_pk: 1eb2382bf65a34d67bfa157a40d56de3_118978_ENV, id: 241138320), (resource_pk: 3e7bf9324abae4e8a2f99ded68eac373_118978_ENV, id: 241139841)
[2021-12-15 19:09:10] (resource_pk: 1fd9b768f2ac2b719d9b27bde549492a_120083_ENV, id: 241138382), (resource_pk: dd9f6fb80b97b67452257b1ef0f446cc_120083_ENV, id: 241147559)
[2021-12-15 19:09:10] (resource_pk: 2030279c3c82b8619ec34b469bfa9965_120083_ENV, id: 241138402), (resource_pk: dfaa759344ae2f82a23e27d6db0a687c_120083_ENV, id: 241147660)
[2021-12-15 19:09:10] (resource_pk: 21171dc3153ea0f965934ec6766de7c9_118978_ENV, id: 241138441), (resource_pk: 407fdbabe337d6e3babe98a344e4cbaf_118978_ENV, id: 241139961)
[2021-12-15 19:09:10] (resource_pk: 21246ed4d11870df183c2a00dfa9e9b0_118978_ENV, id: 241138445), (resource_pk: 7d460e010339f8d40529fdc3aaac82c0_118978_ENV, id: 241143000)
[2021-12-15 19:09:10] (resource_pk: 23d07902ed53ae26db59a855399983e4_120083_ENV, id: 241138588), (resource_pk: 401da195865883b8e24318def8573aa0_120083_ENV, id: 241139935)
[2021-12-15 19:09:10] (resource_pk: 24ae11c553b11ba4628d74c735ea5151_118237_ENV, id: 241138626), (resource_pk: 685d8d051bb78ae8cf2f223ed391a5a4_118237_ENV, id: 241141949)
[2021-12-15 19:09:10] (resource_pk: 2a6df71bd96d6c8a6b6aa0320a917a76_118978_ENV, id: 241138890), (resource_pk: c216e03c6c3b47a757a7e6146850f848_118978_ENV, id: 241146201)
[2021-12-15 19:09:10] (resource_pk: 2bc0d51f778cd144be0b28f6bfb73f50_120083_ENV, id: 241138948), (resource_pk: c0a45140e110a2af67f7a16e70787fca_120083_ENV, id: 241146129)
[2021-12-15 19:09:10] (resource_pk: 2bf15d20eb4b1f5ad98b82401af7b58f_120083_ENV, id: 241138958), (resource_pk: 64d5a29f5c45d261965aa1c5c4317d0e_120083_ENV, id: 241141744)
[2021-12-15 19:09:10] (resource_pk: 2c86d5513c52af5e98833e02d0b238f8_120081_ENV, id: 241138993), (resource_pk: 94622c755d4035ea89e37765e7ac6ce4_120081_ENV, id: 241144052)
[2021-12-15 19:09:10] (resource_pk: 2d70d77ddf54b8088f39da33d0295cef_118237_ENV, id: 241139039), (resource_pk: 71bbaa12e61aec2278429cf7ff36ae29_118237_ENV, id: 241142404)
[2021-12-15 19:09:10] (resource_pk: 2d74004fca0ae85ff595082b66759d52_120083_ENV, id: 241139041), (resource_pk: ce33f94db3035e930735fd620e99b368_120083_ENV, id: 241146785)
[2021-12-15 19:09:10] (resource_pk: 2dab193a7f5a2b93021638701276721e_118986_ENV, id: 241139051), (resource_pk: b6c2aab9ec25b70dd3d89eee17cebdbf_118986_ENV, id: 241145671)
[2021-12-15 19:09:10] (resource_pk: 2dc2935e9712c557e4f44be2ce915af0_118978_ENV, id: 241139055), (resource_pk: 66f72cf33d6eb24886deff914237ad5c_118978_ENV, id: 241141872)
[2021-12-15 19:09:10] (resource_pk: 2e27ddb0c6252c452e561af08af2fe22_118941_ENV, id: 241139070), (resource_pk: ff5467591876e1dab473f569d142f48a_118941_ENV, id: 241149188)
[2021-12-15 19:09:10] (resource_pk: 2fc5dc62fc6756730ee3fdf05d1a7d2a_120083_ENV, id: 241139153), (resource_pk: f8ff0281c5274bf3a22feba001d09776_120083_ENV, id: 241148867)
[2021-12-15 19:09:10] (resource_pk: 3625df2c9571b8db35583d550dff298c_118978_ENV, id: 241139429), (resource_pk: 7d5a18da04bbc6a3859c8202f10ca693_118978_ENV, id: 241143002)
[2021-12-15 19:09:10] (resource_pk: 3704fbfd36258a35c53b7a1aa703a019_120083_ENV, id: 241139476), (resource_pk: f9912b1057f6a7a3e4df6d6bca344507_120083_ENV, id: 241148891)
[2021-12-15 19:09:10] (resource_pk: 388384cb45da09b5b20229d5e7221f9d_120083_ENV, id: 241139547), (resource_pk: 5c1ea85e06463540a09308a9fbf92d0c_120083_ENV, id: 241141298)
[2021-12-15 19:09:10] (resource_pk: 3e25ab04e06123b7986e5c4f79dbf205_118978_ENV, id: 241139822), (resource_pk: b44af0e39c7ee5330daeb0dc013538f7_118978_ENV, id: 241145561)
[2021-12-15 19:09:10] (resource_pk: 3eaa471f70b681415d72624c47d90911_118978_ENV, id: 241139854), (resource_pk: f15e0e6b81ed8b3883c1328d94e34b73_118978_ENV, id: 241148487)
[2021-12-15 19:09:10] (resource_pk: 41f17d73c6a969ffc16510f43942f733_118946_ENV, id: 241140029), (resource_pk: 6c34f72a51436e091d845a371a730cae_118946_ENV, id: 241142141)
[2021-12-15 19:09:10] (resource_pk: 4265bac05cc55b36a9fb7b2f9817e6a4_118978_ENV, id: 241140058), (resource_pk: 631e90476674a5681ec774d658748cdd_118978_ENV, id: 241141655)
[2021-12-15 19:09:10] (resource_pk: 44c0e5f5c50099837b9e92047c2ad746_120083_ENV, id: 241140177), (resource_pk: f0ef8dd6f9094893211300e0bbb433ff_120083_ENV, id: 241148466)
[2021-12-15 19:09:10] (resource_pk: 48c8235203ff004d7b7da13709b23562_118946_ENV, id: 241140372), (resource_pk: a7def2af90f2824ec4a0918895b14213_118946_ENV, id: 241144992)
[2021-12-15 19:09:10] (resource_pk: 49409e77be6e53b573dd88d99cfeebde_120083_ENV, id: 241140391), (resource_pk: 77636694120ff2fd18af0402f14ce067_120083_ENV, id: 241142709)
[2021-12-15 19:09:10] (resource_pk: 4951e94ba51172e57748ca7d9de5d9df_118986_ENV, id: 241140395), (resource_pk: 812035e26bcb2637def3c00508eecd1f_118986_ENV, id: 241143172)
[2021-12-15 19:09:10] (resource_pk: 4a1fe5c4e076438dad3adef07651ae4c_120083_ENV, id: 241140434), (resource_pk: c044b85647a04eb1d5223e62022f9675_120083_ENV, id: 241146113)
[2021-12-15 19:09:10] (resource_pk: 4c0c084da9d69e911c8c1329fa2bcc11_120083_ENV, id: 241140518), (resource_pk: c940ba3b36bec09d990d741994b56394_120083_ENV, id: 241146562)
[2021-12-15 19:09:10] (resource_pk: 4c4fcc0ca3c1aae6da342404d898f7f5_118978_ENV, id: 241140534), (resource_pk: f1fd29bc176ff0a80521dc294814c7b8_118978_ENV, id: 241148526)
[2021-12-15 19:09:10] (resource_pk: 4cc08bce4a3ffeb5a575903abfeb03e9_118978_ENV, id: 241140557), (resource_pk: f60eb0fda8b366cfa37dbe8d42d7dfd1_118978_ENV, id: 241148713)
[2021-12-15 19:09:10] (resource_pk: 4d449e6031e8ca1f914513dccc0d3036_120083_ENV, id: 241140579), (resource_pk: a5469eb473a138d1315853642d7b3a9f_120083_ENV, id: 241144862)
[2021-12-15 19:09:10] (resource_pk: 4d4f49820a7fc4b54f00b8ca493b05f8_120083_ENV, id: 241140580), (resource_pk: 82efdfada2726cea54250412718a1777_120083_ENV, id: 241143245)
[2021-12-15 19:09:10] (resource_pk: 4fecb03ec525b7cca431ee6c109febc3_118936_ENV, id: 241140702), (resource_pk: c319635ad88ae482830fb7847e103823_118936_ENV, id: 241146252)
[2021-12-15 19:09:10] (resource_pk: 51ee2623afd32dc5cd6e092ad070ad1c_119520_ENV, id: 241140786), (resource_pk: 58fa13456be49b0abd7c2d069bc2910e_119520_ENV, id: 241141144)
[2021-12-15 19:09:10] (resource_pk: 51f2e3dba0b86af035bf3d7b3e9cd3cd_118978_ENV, id: 241140787), (resource_pk: 54d23af8aa029f5fc5b9e9fdeb4c1614_118978_ENV, id: 241140928)
[2021-12-15 19:09:10] (resource_pk: 52526f1d4a3d7c6318650a9228b6ba70_120083_ENV, id: 241140804), (resource_pk: bde8374806a54facc27bca01002d7ec5_120083_ENV, id: 241146018)
[2021-12-15 19:09:10] (resource_pk: 52a5e1e636d1d3dd726dea6024d560d2_119187_ENV, id: 241140816), (resource_pk: f4521c877ab09432f99a9412dc9844f4_119187_ENV, id: 241148629)
[2021-12-15 19:09:10] (resource_pk: 53472923aee81ddb8a8481ace1513115_118978_ENV, id: 241140860), (resource_pk: 8112880f26efb4f3ecca4b881b84c413_118978_ENV, id: 241143167)
[2021-12-15 19:09:10] (resource_pk: 539de78b2a63575a943382e6b65c8843_120083_ENV, id: 241140875), (resource_pk: ed8904903b190457cef6af451de35c1f_120083_ENV, id: 241148297)
[2021-12-15 19:09:10] (resource_pk: 547e9906edf832f4e60921c146ed0403_27822_ENV, id: 241140910), (resource_pk: 70c922f8840695d0bf1af24cbb13b0f1_27822_ENV, id: 241142355)
[2021-12-15 19:09:10] (resource_pk: 56634f1e1e7fe7dee89e426f70b4b73d_120083_ENV, id: 241141005), (resource_pk: f78994a2f1ff3abac64bc641c94cf78c_120083_ENV, id: 241148792)
[2021-12-15 19:09:10] (resource_pk: 56903fd7276eecfc4e5be12f408fe997_119187_ENV, id: 241141011), (resource_pk: 7893bbdbff9cae16e1373bd35a705fd9_119187_ENV, id: 241142764)
[2021-12-15 19:09:10] (resource_pk: 56903fd7276eecfc4e5be12f408fe997_119187_ENV, id: 241141011), (resource_pk: dcc61cd1fc193898b887c0bf3f29a2cb_119187_ENV, id: 241147512)
[2021-12-15 19:09:10] (resource_pk: 574ab0b5496e530bc862ea2289279131_118978_ENV, id: 241141049), (resource_pk: df60ae793f76efc3d80097a7439e8d5a_118978_ENV, id: 241147646)
[2021-12-15 19:09:10] (resource_pk: 5809dd4435068e167eb7f98a0ce36d95_118237_ENV, id: 241141086), (resource_pk: 8473a2d62b14266fde4bf862bff8b0f3_118237_ENV, id: 241143303)
[2021-12-15 19:09:10] (resource_pk: 580f758d5830346684e55baa2d2a8944_120083_ENV, id: 241141089), (resource_pk: 599f102d019aba9f201fd097ad2ee298_120083_ENV, id: 241141173)
[2021-12-15 19:09:10] (resource_pk: 5aca678f757ff0c80eaeb09aade88c7c_120083_ENV, id: 241141231), (resource_pk: 62c3742b16e88162388f6c261f99a4c7_120083_ENV, id: 241141642)
[2021-12-15 19:09:10] (resource_pk: 5b7da39168e5cc8611adaf391d6b3f12_118946_ENV, id: 241141265), (resource_pk: d9c87d5f8ffe229883a36f3a14331c6e_118946_ENV, id: 241147351)
[2021-12-15 19:09:10] (resource_pk: 5e34008a02f93ee2130be411825aee4a_119520_ENV, id: 241141416), (resource_pk: d397ea51771612cb927b253776e92e35_119520_ENV, id: 241147050)
[2021-12-15 19:09:10] (resource_pk: 5ebe7cb564fd99f70795fda02b49f8ef_120083_ENV, id: 241141451), (resource_pk: d46a69903b401fa10db4e315e9a58ae3_120083_ENV, id: 241147091)
[2021-12-15 19:09:10] (resource_pk: 61dd53c6d5ea21d76f92775f69f9060d_120083_ENV, id: 241141587), (resource_pk: d3d32da1c967389470c9c3a1e46671e8_120083_ENV, id: 241147066)
[2021-12-15 19:09:10] (resource_pk: 61f7461f6a4778b5e818bd4e75a0c1aa_118978_ENV, id: 241141596), (resource_pk: 7ebeba940194a3c739152c0b797c8b65_118978_ENV, id: 241143055)
[2021-12-15 19:09:10] (resource_pk: 644c4bbedea5a19cb00c108c33f5bada_118978_ENV, id: 241141707), (resource_pk: d9566ae468e8120449e4bd70c1b411d5_118978_ENV, id: 241147325)
[2021-12-15 19:09:10] (resource_pk: 644e823111c437bd102a76cce0780bcd_119187_ENV, id: 241141708), (resource_pk: 83e0e111d529267ac0cf636128ceb13a_119187_ENV, id: 241143279)
[2021-12-15 19:09:10] (resource_pk: 64e63ad74a6e8befe17605991eca58df_120083_ENV, id: 241141750), (resource_pk: 9e8481854bca2a28770cc8cbe9ab0d03_120083_ENV, id: 241144543)
[2021-12-15 19:09:10] (resource_pk: 66722ce6125b28983f82c196f2351dc6_120083_ENV, id: 241141842), (resource_pk: a8105103404347b9c6ef9ee907e09202_120083_ENV, id: 241145004)
[2021-12-15 19:09:10] (resource_pk: 67abde62afd1afb7e050ddab073832e8_118946_ENV, id: 241141910), (resource_pk: c526d48c5fb50fe8a05b34341d4330f9_118946_ENV, id: 241146342)
[2021-12-15 19:09:10] (resource_pk: 68ad90929105d30682e66fb5b19622cb_120083_ENV, id: 241141961), (resource_pk: f02a34d00d4cb208d71deda1a26a2aec_120083_ENV, id: 241148424)
[2021-12-15 19:09:10] (resource_pk: 69e1546cba4331e44a13e268ca4b8ad4_118978_ENV, id: 241142032), (resource_pk: c894594389c752e1da52497e08d51d04_118978_ENV, id: 241146528)
[2021-12-15 19:09:10] (resource_pk: 6cdbb7000f1301c36f8a37504e537ca8_119188_ENV, id: 241142175), (resource_pk: a77df7b1c523e77b60dd4f45f03854f9_119188_ENV, id: 241144976)
[2021-12-15 19:09:10] (resource_pk: 75661b53a3eba9aa1e87cb206e9771d5_120083_ENV, id: 241142598), (resource_pk: 90c33247d53ceb59c5f6e8e683453053_120083_ENV, id: 241143886)
[2021-12-15 19:09:10] (resource_pk: 777c946fb3e69b2dc60a57891ff49b40_118946_ENV, id: 241142716), (resource_pk: 8d0057dfe858e736da2c363ce2b1fed8_118946_ENV, id: 241143694)
[2021-12-15 19:09:10] (resource_pk: 779ac0399a1683d4b6280e95eb3ad626_27822_ENV, id: 241142722), (resource_pk: 8f15446c36fa5c85fa3cdae7aac40d6a_27822_ENV, id: 241143796)
[2021-12-15 19:09:10] (resource_pk: 7893bbdbff9cae16e1373bd35a705fd9_119187_ENV, id: 241142764), (resource_pk: dcc61cd1fc193898b887c0bf3f29a2cb_119187_ENV, id: 241147512)
[2021-12-15 19:09:10] (resource_pk: 7c0410de07d61e76c92cb28aef3926ec_120081_ENV, id: 241142932), (resource_pk: bc86bb9ad251aae8e61fd0e52fe0e271_120081_ENV, id: 241145956)
[2021-12-15 19:09:10] (resource_pk: 7d402d8f8766012d99a879a77c8de3eb_120083_ENV, id: 241142998), (resource_pk: 8039670f9fd31221d64e1536a9ecebec_120083_ENV, id: 241143128)
[2021-12-15 19:09:10] (resource_pk: 80fe40f9c946ff276310e4656816483d_120083_ENV, id: 241143166), (resource_pk: 8d870601aaf7be3bd5ecb085d247c00b_120083_ENV, id: 241143719)
[2021-12-15 19:09:10] (resource_pk: 82d1366f9487c2976942f31ee16f3b37_120083_ENV, id: 241143240), (resource_pk: d9fd4aa40ecfd4cc99b153382359548f_120083_ENV, id: 241147365)
[2021-12-15 19:09:10] (resource_pk: 86a7a2ee640cf06ae5660dcbdfc20cc9_120083_ENV, id: 241143398), (resource_pk: a4e7143ef8eb4ac3e57875aa95e54f73_120083_ENV, id: 241144850)
[2021-12-15 19:09:10] (resource_pk: 876d5fb1b7c313e870bfc90c6aba50eb_120083_ENV, id: 241143432), (resource_pk: e082841a4ece9a9bfcc07188db587ee2_120083_ENV, id: 241147711)
[2021-12-15 19:09:10] (resource_pk: 8861d44ffd8dd542af9b9df6ae8decdc_118237_ENV, id: 241143479), (resource_pk: e1284c70a900f62676372401a93d4d29_118237_ENV, id: 241147731)
[2021-12-15 19:09:10] (resource_pk: 887a04ee8e471fb9f85131919b675ba1_120083_ENV, id: 241143486), (resource_pk: bab57df1f264ea7d5cddf69167f96762_120083_ENV, id: 241145868)
[2021-12-15 19:09:10] (resource_pk: 8c0696cc799b3bd0250e85bb724ca80b_120083_ENV, id: 241143651), (resource_pk: a6e9131ebaebb90078c71446faa2c3d6_120083_ENV, id: 241144942)
[2021-12-15 19:09:10] (resource_pk: 92dbf894ec90e17f6aa175d92a05aa43_120083_ENV, id: 241143979), (resource_pk: a5c588a93d757f86d7256243335144f8_120083_ENV, id: 241144884)
[2021-12-15 19:09:10] (resource_pk: 939b1383d0da847c0293f741cf5fb374_120083_ENV, id: 241144012), (resource_pk: a7fd6cfcc447f2df92d4d2c953c89803_120083_ENV, id: 241144998)
[STOP] [2021-12-15 19:09:10] calculate_statistics
[START] [2021-12-15 19:09:10] complete_harvest_instance
[START] [2021-12-15 19:09:10] overall_tsv_creation
[INFO] [2021-12-15 19:09:10] Processing group of 5967 in 1 batches of 10000
[INFO] [2021-12-15 19:10:11] 12385 Traits (unfiltered)...
[INFO] [2021-12-15 19:12:25] 12385 Traits (filtered)...
[INFO] [2021-12-15 19:12:28] 0 Associations (filtered)...
[INFO] [2021-12-15 19:12:29] 0 metadata added.
[INFO] [2021-12-15 19:12:29] 0 metadata added.
[INFO] [2021-12-15 19:13:03] Average Time: 202.46
[INFO] [2021-12-15 19:13:03] Total Time: 3m54s
[STOP] [2021-12-15 19:13:03] overall_tsv_creation
[INFO] [2021-12-15 19:13:03] Done. Check your files:
[INFO] [2021-12-15 19:13:03] (1935 lines) /app/public/data/memoirs_am_entom/publish_bibliographic_citations.tsv
[INFO] [2021-12-15 19:13:04] (5967 lines) /app/public/data/memoirs_am_entom/publish_nodes.tsv
[INFO] [2021-12-15 19:13:04] (5967 lines) /app/public/data/memoirs_am_entom/publish_scientific_names.tsv
[INFO] [2021-12-15 19:13:04] (1935 lines) /app/public/data/memoirs_am_entom/publish_articles.tsv
[INFO] [2021-12-15 19:13:04] (1935 lines) /app/public/data/memoirs_am_entom/publish_content_sections.tsv
[INFO] [2021-12-15 19:13:04] (12386 lines) /app/public/data/memoirs_am_entom/publish_traits.tsv
[INFO] [2021-12-15 19:13:04] (1 lines) /app/public/data/memoirs_am_entom/publish_metadata.tsv
[STOP] [2021-12-15 19:13:04] complete_harvest_instance
[START] [2021-12-15 19:13:04] completed
[STOP] [2021-12-15 19:13:04] completed
[STOP] [2021-12-15 19:13:04] logged process, took 470.0
Latest Process