Harvest for OBIS environmental data records Created 02 Dec 11:59

Stage: completed
Fetched: 02 Dec 11:59
Validated: 02 Dec 12:01
Deltas Created 02 Dec 12:01
Units Normalized: 03 Dec 02:05
Ancestry Built: 02 Dec 20:04
Nodes Matched: 02 Dec 23:59
Names Parsed: 02 Dec 20:06
New Models Stored: 02 Dec 19:49
Indexed: 03 Dec 00:02
Completed: 03 Dec 04:13
Time to Harvest: 16 minutes

Expected File Format Definitions

Harvesting Log (most recent first)

# Logfile created on 2019-12-02 11:59:22 -0500 by logger.rb/56815
[START] [2019-12-02 11:59:22] logged process
[START] [2019-12-02 11:59:22] create_harvest_instance
[STOP] [2019-12-02 11:59:24] create_harvest_instance
[START] [2019-12-02 11:59:24] fetch_files
[STOP] [2019-12-02 11:59:24] fetch_files
[START] [2019-12-02 11:59:24] validate_each_file
[STOP] [2019-12-02 12:01:05] validate_each_file
[START] [2019-12-02 12:01:05] convert_to_csv
[CMD] [2019-12-02 12:01:05] /usr/bin/sort /app/public/converted_csv/obis_environment_nodes_18824.csv > /app/public/converted_csv/obis_environment_nodes_18824.csv_sorted
[CMD] [2019-12-02 12:01:06] /usr/bin/sort /app/public/converted_csv/obis_environment_occurrences_18825.csv > /app/public/converted_csv/obis_environment_occurrences_18825.csv_sorted
[CMD] [2019-12-02 12:01:08] /usr/bin/sort /app/public/converted_csv/obis_environment_measurements_18826.csv > /app/public/converted_csv/obis_environment_measurements_18826.csv_sorted
[STOP] [2019-12-02 12:01:11] convert_to_csv
[START] [2019-12-02 12:01:11] calculate_delta
[CMD] [2019-12-02 12:01:11] echo "0a" > /app/public/diff/obis_environment_nodes_18824.diff
[CMD] [2019-12-02 12:01:13] tail -n +1 /app/public/converted_csv/obis_environment_nodes_18824.csv >> /app/public/diff/obis_environment_nodes_18824.diff
[CMD] [2019-12-02 12:01:14] echo "." >> /app/public/diff/obis_environment_nodes_18824.diff
[CMD] [2019-12-02 12:01:16] echo "0a" > /app/public/diff/obis_environment_occurrences_18825.diff
[CMD] [2019-12-02 12:01:18] tail -n +1 /app/public/converted_csv/obis_environment_occurrences_18825.csv >> /app/public/diff/obis_environment_occurrences_18825.diff
[CMD] [2019-12-02 12:01:20] echo "." >> /app/public/diff/obis_environment_occurrences_18825.diff
[CMD] [2019-12-02 12:01:21] echo "0a" > /app/public/diff/obis_environment_measurements_18826.diff
[CMD] [2019-12-02 12:01:23] tail -n +1 /app/public/converted_csv/obis_environment_measurements_18826.csv >> /app/public/diff/obis_environment_measurements_18826.diff
[CMD] [2019-12-02 12:01:25] echo "." >> /app/public/diff/obis_environment_measurements_18826.diff
[STOP] [2019-12-02 12:01:27] calculate_delta
[START] [2019-12-02 12:01:27] parse_diff_and_store
[INFO] [2019-12-02 12:01:28] Loading nodes diff file into memory (true lines)...
[WARN] [2019-12-02 12:01:31] Filtered Scientific Name `Campylosira Grunow <I>ex</I> Van Heurck, 1885` to `Campylosira Grunow <I>ex<I> Van Heurck, 1885`
[WARN] [2019-12-02 12:01:31] Filtered Scientific Name `Campylosira cymbelliformis (A. Schmidt) Grunow <I>ex</I> Van Heurck, 1885` to `Campylosira cymbelliformis (A. Schmidt) Grunow <I>ex<I> Van Heurck, 1885`
[WARN] [2019-12-02 12:01:31] Filtered Scientific Name `Chaetoceros subtilis P.T.  Cleve, 1896` to `Chaetoceros subtilis P.T. Cleve, 1896`
[WARN] [2019-12-02 12:01:32] Filtered Scientific Name `Dictyosporium pelagicum (Linder) G.C. Hughes <i>ex</i> E.B.G. Jones` to `Dictyosporium pelagicum (Linder) G.C. Hughes <i>ex<i> E.B.G. Jones`
[WARN] [2019-12-02 12:01:36] Filtered Scientific Name `Lyngbya aestuarii (Mertens in Jürgens) Liebman <i>ex</i> Gomont, 1892` to `Lyngbya aestuarii (Mertens in Jürgens) Liebman <i>ex<i> Gomont, 1892`
[WARN] [2019-12-02 12:01:37] Filtered Scientific Name `Nemertopsis flavida (McIntosh, 1873/74)` to `Nemertopsis flavida (McIntosh, 187374)`
[WARN] [2019-12-02 12:01:37] Filtered Scientific Name `Oscillatoria bonnemaisonii Crouan <i>in</i> Desmazières <i>ex</i> Gomont, 1892` to `Oscillatoria bonnemaisonii Crouan <i>in<i> Desmazières <i>ex<i> Gomont, 1892`
[WARN] [2019-12-02 12:01:37] Filtered Scientific Name `Oscillatoria curviceps C.A. Agardh <i>ex</i> Gomont, 1892` to `Oscillatoria curviceps C.A. Agardh <i>ex<i> Gomont, 1892`
[WARN] [2019-12-02 12:01:37] Filtered Scientific Name `Oscillatoria margaritifera Kützing <i>ex</i> Gomont, 1892` to `Oscillatoria margaritifera Kützing <i>ex<i> Gomont, 1892`
[WARN] [2019-12-02 12:01:37] Filtered Scientific Name `Oscillatoria princeps Vaucher <i>ex</i> Gomont, 1892` to `Oscillatoria princeps Vaucher <i>ex<i> Gomont, 1892`
[WARN] [2019-12-02 12:01:37] Filtered Scientific Name `Oscillatoria sancta Kützing <i>ex</i> Gomont, 1892` to `Oscillatoria sancta Kützing <i>ex<i> Gomont, 1892`
[WARN] [2019-12-02 12:01:38] Filtered Scientific Name `Phacus pyrum (Ehrenberg)  F. Stein, 1878` to `Phacus pyrum (Ehrenberg) F. Stein, 1878`
[WARN] [2019-12-02 12:01:40] Filtered Scientific Name `Spirulina Turpin <i>ex</i> Gomont, 1893` to `Spirulina Turpin <i>ex<i> Gomont, 1893`
[WARN] [2019-12-02 12:01:40] Filtered Scientific Name `Spirulina meneghiniana Zanardini <i>ex</i> Gomont, 1892` to `Spirulina meneghiniana Zanardini <i>ex<i> Gomont, 1892`
[WARN] [2019-12-02 12:01:52] Filtered Scientific Name `Nosopsyllus fasciatus Bosc d"" Antic` to `Nosopsyllus fasciatus Bosc d Antic`
[WARN] [2019-12-02 12:01:56] Filtered Scientific Name `Arthrospira Stizenberger <i>ex</i> Gomont, 1892` to `Arthrospira Stizenberger <i>ex<i> Gomont, 1892`
[WARN] [2019-12-02 12:01:56] Filtered Scientific Name `Trichodesmium Ehrenberg <i>ex</i> Gomont, 1892` to `Trichodesmium Ehrenberg <i>ex<i> Gomont, 1892`
[WARN] [2019-12-02 12:01:57] Filtered Scientific Name `Ampithoe (Pleonexes) (Bate,  1857)` to `Ampithoe (Pleonexes) (Bate, 1857)`
[WARN] [2019-12-02 12:01:59] Filtered Scientific Name `Enteroctopus magnificus (Villanueva/Sanchez/Compagno, 1992)` to `Enteroctopus magnificus (VillanuevaSanchezCompagno, 1992)`
[WARN] [2019-12-02 12:02:04] Filtered Scientific Name `Phormidium breve (Kützing <i>ex</i> Gomont) Anagnostidis & Komárek, 1988` to `Phormidium breve (Kützing <i>ex<i> Gomont) Anagnostidis & Komárek, 1988`
[WARN] [2019-12-02 12:02:04] Filtered Scientific Name `Phormidium formosum (Bory de Saint-Vincent <i>ex</i> Gomont) Anagnostidis & Komárek, 1988` to `Phormidium formosum (Bory de Saint-Vincent <i>ex<i> Gomont) Anagnostidis & Komárek, 1988`
[WARN] [2019-12-02 12:02:04] Filtered Scientific Name `Freneixicardia victor  (Angas, 1872)` to `Freneixicardia victor (Angas, 1872)`
[WARN] [2019-12-02 12:02:04] Filtered Scientific Name `Anabaena circinalis Rabenhorst <i>ex</i> Bornet & Flahault, 1886` to `Anabaena circinalis Rabenhorst <i>ex<i> Bornet & Flahault, 1886`
[WARN] [2019-12-02 12:02:04] Filtered Scientific Name `Phormidium terebriforme (C.A. Agardh <i>ex</i> Gomont) Anagnostidis & Komárek, 1988` to `Phormidium terebriforme (C.A. Agardh <i>ex<i> Gomont) Anagnostidis & Komárek, 1988`
[WARN] [2019-12-02 12:02:04] Filtered Scientific Name `Fulvia (Fulvia) laevigata  (Linnaeus, 1758)` to `Fulvia (Fulvia) laevigata (Linnaeus, 1758)`
[WARN] [2019-12-02 12:02:04] Filtered Scientific Name `Fragum scruposum  (Deshayes, 1855)` to `Fragum scruposum (Deshayes, 1855)`
[WARN] [2019-12-02 12:02:04] Filtered Scientific Name `Dinematomonas P.C.  Silva, 1960` to `Dinematomonas P.C. Silva, 1960`
[WARN] [2019-12-02 12:02:05] Filtered Scientific Name `Dyspanopeus Martin &  Abele, 1986` to `Dyspanopeus Martin & Abele, 1986`
[WARN] [2019-12-02 12:02:05] Filtered Scientific Name `Minyichthys Herald &  Randall, 1972` to `Minyichthys Herald & Randall, 1972`
[WARN] [2019-12-02 12:02:05] Filtered Scientific Name `Boodlea (Dickie) G. Murray & De Toni  in G.Murray, 1889` to `Boodlea (Dickie) G. Murray & De Toni in G.Murray, 1889`
[WARN] [2019-12-02 12:02:05] Filtered Scientific Name `Polyxenus Latreille, 1802/1803` to `Polyxenus Latreille, 18021803`
[WARN] [2019-12-02 12:02:06] Filtered Scientific Name `Cyclostomatida  incertae sedis` to `Cyclostomatida incertae sedis`
[WARN] [2019-12-02 12:02:07] Filtered Scientific Name `Augophyllum S.-M. Lin, S. Fredericq &M.H.  Hommersand, 2003` to `Augophyllum S.-M. Lin, S. Fredericq &M.H. Hommersand, 2003`
[WARN] [2019-12-02 12:02:08] Filtered Scientific Name `Fulgorariinae  Pilsbry & Olsson, 1954` to `Fulgorariinae Pilsbry & Olsson, 1954`
[WARN] [2019-12-02 12:02:10] Filtered Scientific Name `Neosiphonia harveyi (Bailey) Kim, Choi, Guiry &  Saunders, 2001` to `Neosiphonia harveyi (Bailey) Kim, Choi, Guiry & Saunders, 2001`
[WARN] [2019-12-02 12:02:11] Filtered Scientific Name `Ceratophyllanae Takht. ex Reveal & Doweld,  1999` to `Ceratophyllanae Takht. ex Reveal & Doweld, 1999`
[WARN] [2019-12-02 12:02:12] Filtered Scientific Name `Mazzaella parksii (Postels & Ruprecht)  J.R. Hughey, P.C. Silva & Hommerstand, 2001` to `Mazzaella parksii (Postels & Ruprecht) J.R. Hughey, P.C. Silva & Hommerstand, 2001`
[WARN] [2019-12-02 12:02:13] Filtered Scientific Name `Ulva simplex  (K.l.vinogradova) h.s. hayden & al.` to `Ulva simplex (K.l.vinogradova) h.s. hayden & al.`
[WARN] [2019-12-02 12:02:15] Filtered Scientific Name `Chrysochromulina acantha Leadbeater &  Manton` to `Chrysochromulina acantha Leadbeater & Manton`
[WARN] [2019-12-02 12:02:15] Filtered Scientific Name `Physonecta (Bate,  1857)` to `Physonecta (Bate, 1857)`
[WARN] [2019-12-02 12:02:15] Filtered Scientific Name `Sphacelaria radicans (Bate,  1857)` to `Sphacelaria radicans (Bate, 1857)`
[WARN] [2019-12-02 12:02:15] Filtered Scientific Name `Lichens (Bate,  1857)` to `Lichens (Bate, 1857)`
[WARN] [2019-12-02 12:02:15] Filtered Scientific Name `Cancridea (Bate,  1857)` to `Cancridea (Bate, 1857)`
[WARN] [2019-12-02 12:02:15] Filtered Scientific Name `Protodriloidae (Bate,  1857)` to `Protodriloidae (Bate, 1857)`
[WARN] [2019-12-02 12:02:15] Filtered Scientific Name `Lenticulina quenstedti/ouachensis` to `Lenticulina quenstedtiouachensis`
[WARN] [2019-12-02 12:02:15] Filtered Scientific Name `Rhodiniinae (Bate,  1857)` to `Rhodiniinae (Bate, 1857)`
[WARN] [2019-12-02 12:02:16] Filtered Scientific Name `Typhlotanais pulcher (Bate,  1857)` to `Typhlotanais pulcher (Bate, 1857)`
[WARN] [2019-12-02 12:02:16] Filtered Scientific Name `Porifera upright/erect` to `Porifera uprighterect`
[WARN] [2019-12-02 12:02:16] Filtered Scientific Name `Vauntompsonia (Bate,  1857)` to `Vauntompsonia (Bate, 1857)`
[WARN] [2019-12-02 12:02:16] Filtered Scientific Name `Pachymatisma johnstoni (Bate,  1857)` to `Pachymatisma johnstoni (Bate, 1857)`
[WARN] [2019-12-02 12:02:16] Filtered Scientific Name `Sunamphithoe pelagica (Bate,  1857)` to `Sunamphithoe pelagica (Bate, 1857)`
[WARN] [2019-12-02 12:02:16] Filtered Scientific Name `Anthoathecatae (Bate,  1857)` to `Anthoathecatae (Bate, 1857)`
[WARN] [2019-12-02 12:02:16] Filtered Scientific Name `Zostera marina (Bate,  1857)` to `Zostera marina (Bate, 1857)`
[WARN] [2019-12-02 12:02:17] Filtered Scientific Name `Sargassum muticum (Bate,  1857)` to `Sargassum muticum (Bate, 1857)`
[WARN] [2019-12-02 12:02:17] Filtered Scientific Name `Schottera nicaeensisres (Bate,  1857)` to `Schottera nicaeensisres (Bate, 1857)`
[WARN] [2019-12-02 12:02:17] Filtered Scientific Name `Jacullela/rhabdammina/hyperamminia fragments` to `Jacullelarhabdamminahyperamminia fragments`
[WARN] [2019-12-02 12:02:17] Filtered Scientific Name `Podocopina (Bate,  1857)` to `Podocopina (Bate, 1857)`
[WARN] [2019-12-02 12:02:17] Filtered Scientific Name `Chironomida (Bate,  1857)` to `Chironomida (Bate, 1857)`
[WARN] [2019-12-02 12:02:17] Filtered Scientific Name `Odostomia striata (A.E.  Verrill, 1880)` to `Odostomia striata (A.E. Verrill, 1880)`
[WARN] [2019-12-02 12:02:19] Filtered Scientific Name `Sigmophoranema  Hope & Murphy, 1972` to `Sigmophoranema Hope & Murphy, 1972`
[WARN] [2019-12-02 12:02:19] Filtered Scientific Name `Spirinia  Gerlach, 1963` to `Spirinia Gerlach, 1963`
[WARN] [2019-12-02 12:02:19] Filtered Scientific Name `Spilophorella  Filipjev, 1917` to `Spilophorella Filipjev, 1917`
[WARN] [2019-12-02 12:02:19] Filtered Scientific Name `Kappaphycus striatum (Schmitz) Doty ex  P. Silva` to `Kappaphycus striatum (Schmitz) Doty ex P. Silva`
[INFO] [2019-12-02 12:02:22] Loading occurrences diff file into memory (true lines)...
[INFO] [2019-12-02 12:20:12] Loading measurements diff file into memory (true lines)...
[INFO] [2019-12-02 18:02:50] Storing 162187 ScientificNames
[INFO] [2019-12-02 18:02:50] Processing group of 162187 in 163 groups of 1000
[INFO] [2019-12-02 18:07:07] Average Time: 1.57
[INFO] [2019-12-02 18:07:07] Total Time: 4m17s
[INFO] [2019-12-02 18:07:07] last 3 / first 3: 1.09
[INFO] [2019-12-02 18:07:07] Std.Dev: 5.739163702143371; Max: 31.18
[INFO] [2019-12-02 18:07:07] Storing 162187 Nodes
[INFO] [2019-12-02 18:07:07] Processing group of 162187 in 163 groups of 1000
[INFO] [2019-12-02 18:11:13] Average Time: 1.314
[INFO] [2019-12-02 18:11:13] Total Time: 4m7s
[INFO] [2019-12-02 18:11:13] last 3 / first 3: 0.76
[INFO] [2019-12-02 18:11:13] Std.Dev: 5.3985183152416925; Max: 31.7
[INFO] [2019-12-02 18:11:13] Storing 486561 Occurrences
[INFO] [2019-12-02 18:11:13] Processing group of 486561 in 487 groups of 1000
[INFO] [2019-12-02 18:16:08] Average Time: 0.6
[INFO] [2019-12-02 18:16:08] Total Time: 4m55s
[INFO] [2019-12-02 18:16:08] last 3 / first 3: 0.89
[INFO] [2019-12-02 18:16:08] Std.Dev: 3.795391942869669; Max: 32.31
[INFO] [2019-12-02 18:16:08] Storing 786867 OccurrenceMetadata
[INFO] [2019-12-02 18:16:08] Processing group of 786867 in 787 groups of 1000
[INFO] [2019-12-02 18:23:31] Average Time: 0.474
[INFO] [2019-12-02 18:23:31] Total Time: 7m23s
[INFO] [2019-12-02 18:23:31] last 3 / first 3: 0.87
[INFO] [2019-12-02 18:23:31] Std.Dev: 3.315569332708939; Max: 33.66
[INFO] [2019-12-02 18:23:31] Storing 1924554 Traits
[INFO] [2019-12-02 18:23:31] Processing group of 1924554 in 1925 groups of 1000
[INFO] [2019-12-02 19:08:16] Average Time: 1.39
[INFO] [2019-12-02 19:08:16] Total Time: 44m45s
[INFO] [2019-12-02 19:08:16] last 3 / first 3: 0.7
[INFO] [2019-12-02 19:08:16] Std.Dev: 6.19443298454346; Max: 42.11
[INFO] [2019-12-02 19:08:16] Storing 3849108 MetaTraits
[INFO] [2019-12-02 19:08:16] Processing group of 3849108 in 3850 groups of 1000
[INFO] [2019-12-02 19:49:47] Average Time: 0.618
[INFO] [2019-12-02 19:49:47] Total Time: 41m31s
[INFO] [2019-12-02 19:49:47] last 3 / first 3: 0.6
[INFO] [2019-12-02 19:49:47] Std.Dev: 4.576680019402711; Max: 47.31
[STOP] [2019-12-02 19:49:47] parse_diff_and_store
[START] [2019-12-02 19:49:47] resolve_keys
[INFO] [2019-12-02 19:50:12] Occurrences to nodes (through scientific_names)...
[INFO] [2019-12-02 19:51:01] traits to occurrences...
[INFO] [2019-12-02 19:54:23] traits to nodes (through occurrences)...
[INFO] [2019-12-02 19:55:15] Traits to sex term...
[INFO] [2019-12-02 19:56:00] Traits to lifestage term...
[INFO] [2019-12-02 19:56:45] MetaTraits to traits...
[INFO] [2019-12-02 20:01:21] MetaTraits (simple, measurement row refers to parent) to traits...
[INFO] [2019-12-02 20:01:23] Assocs to occurrences...
[INFO] [2019-12-02 20:01:23] Assocs to nodes...
[INFO] [2019-12-02 20:01:23] Assoc to sex term...
[INFO] [2019-12-02 20:01:23] Assoc to lifestage term...
[STOP] [2019-12-02 20:01:23] resolve_keys
[START] [2019-12-02 20:01:23] hold_for_later_1
[STOP] [2019-12-02 20:01:23] hold_for_later_1
[START] [2019-12-02 20:01:23] hold_for_later_2
[STOP] [2019-12-02 20:01:23] hold_for_later_2
[START] [2019-12-02 20:01:23] resolve_missing_parents
[STOP] [2019-12-02 20:01:23] resolve_missing_parents
[START] [2019-12-02 20:01:23] rebuild_nodes
[START] [2019-12-02 20:01:23] Flattener#flatten
[START] [2019-12-02 20:01:23] Flattener#study_resource
[START] [2019-12-02 20:01:35] Flattener#build_ancestry
[STOP] [2019-12-02 20:04:13] Flattener#build_ancestry
[INFO] [2019-12-02 20:04:13] 162187 ancestry keys
[START] [2019-12-02 20:04:13] build_node_ancestors
[INFO] [2019-12-02 20:04:13] old ancestors deleted.
[STOP] [2019-12-02 20:04:13] build_node_ancestors
[WARN] [2019-12-02 20:04:13] Flattener: nothing to flatten! (Completely flat resource?)
[STOP] [2019-12-02 20:04:13] Flattener#flatten
[STOP] [2019-12-02 20:04:13] rebuild_nodes
[START] [2019-12-02 20:04:13] resolve_missing_media_owners
[STOP] [2019-12-02 20:04:13] resolve_missing_media_owners
[START] [2019-12-02 20:04:13] sanitize_media_verbatims
[STOP] [2019-12-02 20:04:13] sanitize_media_verbatims
[START] [2019-12-02 20:04:14] queue_downloads
[STOP] [2019-12-02 20:04:14] queue_downloads
[START] [2019-12-02 20:04:14] parse_names
[WARN] [2019-12-02 20:04:14] I see 162187 names which still need to be parsed.
[STOP] [2019-12-02 20:06:33] parse_names
[START] [2019-12-02 20:06:33] denormalize_canonical_names_to_nodes
[STOP] [2019-12-02 20:06:36] denormalize_canonical_names_to_nodes
[START] [2019-12-02 20:06:36] match_nodes
[START] [2019-12-02 20:06:36] map_all_nodes_to_pages
[STOP] [2019-12-02 23:58:35] map_all_nodes_to_pages
[INFO] [2019-12-02 23:58:35] 50283 Unmatched nodes (of 162187)! That's too many to output. First 10: Conus labrolineata (#59267128); Aaptolasma nolearia (#59267130); Abarenicola (#59267136); Abarenicola claparedii oceanica (#59267138); Abasia (#59267140); Abathescalpellum fissum (#59267143); Abatus (#59267145); Aberranta (#59267155); Abietinaria (#59267157); Ablennes (#59267171)
[START] [2019-12-02 23:58:35] update_nodes
[STOP] [2019-12-02 23:59:03] update_nodes
[STOP] [2019-12-02 23:59:03] match_nodes
[START] [2019-12-02 23:59:03] reindex_search
[STOP] [2019-12-03 00:02:50] reindex_search
[START] [2019-12-03 00:02:50] normalize_units
[STOP] [2019-12-03 02:05:53] normalize_units
[START] [2019-12-03 02:05:53] calculate_statistics
[2019-12-03 02:05:53] ZERO NODE ANCESTORS. Is this actually a completely flat resource?
[STOP] [2019-12-03 02:05:53] calculate_statistics
[START] [2019-12-03 02:05:53] complete_harvest_instance
[START] [2019-12-03 02:05:53] overall_tsv_creation
[INFO] [2019-12-03 02:05:54] Processing group of 162187 in 17 batches of 10000
[INFO] [2019-12-03 02:07:35] 137705 Traits (unfiltered)...
[INFO] [2019-12-03 02:07:49] 137705 Traits (filtered)...
[INFO] [2019-12-03 02:07:49] 0 Associations (filtered)...
[INFO] [2019-12-03 02:14:12] 563894 metadata added.
[INFO] [2019-12-03 02:14:12] 0 metadata added.
[INFO] [2019-12-03 02:15:49] 137261 Traits (unfiltered)...
[INFO] [2019-12-03 02:16:02] 137261 Traits (filtered)...
[INFO] [2019-12-03 02:16:03] 0 Associations (filtered)...
[INFO] [2019-12-03 02:22:44] 563953 metadata added.
[INFO] [2019-12-03 02:22:44] 0 metadata added.
[INFO] [2019-12-03 02:24:19] 137127 Traits (unfiltered)...
[INFO] [2019-12-03 02:24:33] 137127 Traits (filtered)...
[INFO] [2019-12-03 02:24:33] 0 Associations (filtered)...
[INFO] [2019-12-03 02:31:14] 560220 metadata added.
[INFO] [2019-12-03 02:31:14] 0 metadata added.
[INFO] [2019-12-03 02:32:52] 134683 Traits (unfiltered)...
[INFO] [2019-12-03 02:33:06] 134683 Traits (filtered)...
[INFO] [2019-12-03 02:33:06] 0 Associations (filtered)...
[INFO] [2019-12-03 02:39:42] 550907 metadata added.
[INFO] [2019-12-03 02:39:42] 0 metadata added.
[INFO] [2019-12-03 02:41:19] 126968 Traits (unfiltered)...
[INFO] [2019-12-03 02:41:32] 126968 Traits (filtered)...
[INFO] [2019-12-03 02:41:33] 0 Associations (filtered)...
[INFO] [2019-12-03 02:47:52] 509050 metadata added.
[INFO] [2019-12-03 02:47:52] 0 metadata added.
[INFO] [2019-12-03 02:49:29] 111055 Traits (unfiltered)...
[INFO] [2019-12-03 02:49:42] 111055 Traits (filtered)...
[INFO] [2019-12-03 02:49:42] 0 Associations (filtered)...
[INFO] [2019-12-03 02:55:23] 430488 metadata added.
[INFO] [2019-12-03 02:55:23] 0 metadata added.
[INFO] [2019-12-03 02:57:00] 109092 Traits (unfiltered)...
[INFO] [2019-12-03 02:57:13] 109092 Traits (filtered)...
[INFO] [2019-12-03 02:57:13] 0 Associations (filtered)...
[INFO] [2019-12-03 03:02:55] 425374 metadata added.
[INFO] [2019-12-03 03:02:55] 0 metadata added.
[INFO] [2019-12-03 03:04:33] 108880 Traits (unfiltered)...
[INFO] [2019-12-03 03:04:47] 108880 Traits (filtered)...
[INFO] [2019-12-03 03:04:47] 0 Associations (filtered)...
[INFO] [2019-12-03 03:10:24] 426016 metadata added.
[INFO] [2019-12-03 03:10:24] 0 metadata added.
[INFO] [2019-12-03 03:12:03] 118108 Traits (unfiltered)...
[INFO] [2019-12-03 03:12:16] 118108 Traits (filtered)...
[INFO] [2019-12-03 03:12:16] 0 Associations (filtered)...
[INFO] [2019-12-03 03:18:02] 480843 metadata added.
[INFO] [2019-12-03 03:18:02] 0 metadata added.
[INFO] [2019-12-03 03:19:43] 98851 Traits (unfiltered)...
[INFO] [2019-12-03 03:19:56] 98851 Traits (filtered)...
[INFO] [2019-12-03 03:19:56] 0 Associations (filtered)...
[INFO] [2019-12-03 03:25:00] 407501 metadata added.
[INFO] [2019-12-03 03:25:00] 0 metadata added.
[INFO] [2019-12-03 03:26:40] 102345 Traits (unfiltered)...
[INFO] [2019-12-03 03:26:53] 102345 Traits (filtered)...
[INFO] [2019-12-03 03:26:53] 0 Associations (filtered)...
[INFO] [2019-12-03 03:32:14] 423021 metadata added.
[INFO] [2019-12-03 03:32:14] 0 metadata added.
[INFO] [2019-12-03 03:33:56] 136615 Traits (unfiltered)...
[INFO] [2019-12-03 03:34:10] 136615 Traits (filtered)...
[INFO] [2019-12-03 03:34:10] 0 Associations (filtered)...
[INFO] [2019-12-03 03:40:55] 620713 metadata added.
[INFO] [2019-12-03 03:40:55] 0 metadata added.
[INFO] [2019-12-03 03:42:37] 99643 Traits (unfiltered)...
[INFO] [2019-12-03 03:42:50] 99643 Traits (filtered)...
[INFO] [2019-12-03 03:42:50] 0 Associations (filtered)...
[INFO] [2019-12-03 03:48:02] 412177 metadata added.
[INFO] [2019-12-03 03:48:02] 0 metadata added.
[INFO] [2019-12-03 03:49:39] 109365 Traits (unfiltered)...
[INFO] [2019-12-03 03:49:52] 109365 Traits (filtered)...
[INFO] [2019-12-03 03:49:52] 0 Associations (filtered)...
[INFO] [2019-12-03 03:55:26] 435821 metadata added.
[INFO] [2019-12-03 03:55:26] 0 metadata added.
[INFO] [2019-12-03 03:57:03] 144002 Traits (unfiltered)...
[INFO] [2019-12-03 03:57:16] 144002 Traits (filtered)...
[INFO] [2019-12-03 03:57:16] 0 Associations (filtered)...
[INFO] [2019-12-03 04:04:16] 584648 metadata added.
[INFO] [2019-12-03 04:04:16] 0 metadata added.
[INFO] [2019-12-03 04:05:56] 94890 Traits (unfiltered)...
[INFO] [2019-12-03 04:06:09] 94890 Traits (filtered)...
[INFO] [2019-12-03 04:06:09] 0 Associations (filtered)...
[INFO] [2019-12-03 04:11:08] 373572 metadata added.
[INFO] [2019-12-03 04:11:08] 0 metadata added.
[INFO] [2019-12-03 04:12:08] 17964 Traits (unfiltered)...
[INFO] [2019-12-03 04:12:22] 17964 Traits (filtered)...
[INFO] [2019-12-03 04:12:22] 0 Associations (filtered)...
[INFO] [2019-12-03 04:13:28] 70320 metadata added.
[INFO] [2019-12-03 04:13:28] 0 metadata added.
[INFO] [2019-12-03 04:13:28] Average Time: 420.515
[INFO] [2019-12-03 04:13:28] Total Time: 2h7m35s
[INFO] [2019-12-03 04:13:28] last 3 / first 3: 0.7
[INFO] [2019-12-03 04:13:28] Std.Dev: 87.67041120013069; Max: 504.24
[STOP] [2019-12-03 04:13:28] overall_tsv_creation
[INFO] [2019-12-03 04:13:28] Done. Check your files:
[INFO] [2019-12-03 04:13:29] (162187 lines) /app/public/data/obis_environment/publish_nodes.tsv
[INFO] [2019-12-03 04:13:31] (162187 lines) /app/public/data/obis_environment/publish_scientific_names.tsv
[INFO] [2019-12-03 04:13:33] (1924555 lines) /app/public/data/obis_environment/publish_traits.tsv
[INFO] [2019-12-03 04:13:35] (7838519 lines) /app/public/data/obis_environment/publish_metadata.tsv
[STOP] [2019-12-03 04:13:36] complete_harvest_instance
[START] [2019-12-03 04:13:36] completed
[STOP] [2019-12-03 04:13:36] completed
[STOP] [2019-12-03 04:13:36] logged process, took 58453.33

Latest Process