mailman.cloud.codelutin.com
Sign In Sign Up
Manage this list Sign In Sign Up

Keyboard Shortcuts

Thread View

  • j: Next unread message
  • k: Previous unread message
  • j a: Jump to all threads
  • j l: Jump to MailingList overview

Observe-commits

Thread Start a new thread
Download
Threads by month
  • ----- 2026 -----
  • June
  • May
  • April
  • March
  • February
  • January
  • ----- 2025 -----
  • December
  • November
  • October
  • September
  • August
  • July
  • June
  • May
  • April
  • March
  • February
  • January
  • ----- 2024 -----
  • December
  • November
  • October
  • September
  • August
  • July
  • June
  • May
  • April
  • March
  • February
  • January
  • ----- 2023 -----
  • December
  • November
  • October
  • September
  • August
  • July
  • June
  • May
  • April
  • March
  • February
  • January
  • ----- 2022 -----
  • December
  • November
  • October
  • September
  • August
  • July
  • June
  • May
  • April
  • March
  • February
  • January
  • ----- 2021 -----
  • December
  • November
  • October
  • September
  • August
  • July
  • June
  • May
  • April
  • March
  • February
  • January
  • ----- 2020 -----
  • December
  • November
  • October
  • September
  • August
  • July
  • June
  • May
  • April
  • March
  • February
  • January
  • ----- 2019 -----
  • December
  • November
  • October
  • September
  • August
  • July
  • June
  • May
  • April
  • March
  • February
  • January
  • ----- 2018 -----
  • December
  • November
  • October
  • September
  • August
  • July
  • June
  • May
  • April
  • March
  • February
  • January
  • ----- 2017 -----
  • December
  • November
  • October
  • September
  • August
  • July
  • June
  • May
  • April
  • March
  • February
  • January
  • ----- 2016 -----
  • December
  • November
  • October
  • September
  • August
  • July
  • June
  • May
  • April
  • March
  • February
  • January
  • ----- 2015 -----
  • December
  • November
  • October
  • September
  • August
  • July
  • June
  • May
  • April
  • March
  • February
  • January
  • ----- 2014 -----
  • December
  • November
  • October
  • September
  • August
  • July
  • June
  • May
  • April
  • March
  • February
  • January
  • ----- 2013 -----
  • December
  • November
  • October
  • September
observe-commits@list.forge.codelutin.com

January 2023

  • 1 participants
  • 76 discussions
[Git][ultreiaio/ird-observe][develop-9.0.x] 4 commits: Procédure de calcul qui ne détecte pas les données modifiées - Closes #2615
by Tony CHEMIT (@tchemit) 25 Jan '23

25 Jan '23
Tony CHEMIT pushed to branch develop-9.0.x at ultreiaio / ird-observe Commits: 118fdb99 by Tony Chemit at 2023-01-25T11:39:02+01:00 Procédure de calcul qui ne détecte pas les données modifiées - Closes #2615 - - - - - 56d21588 by Tony Chemit at 2023-01-25T11:39:02+01:00 Avdth - Review some Floating Object mappings (sort them and re-add some removed ones...) - See #2612 - - - - - e04df8f4 by Tony Chemit at 2023-01-25T11:39:02+01:00 Avdth - Improve code around adding observed system 102 on activity with at least on catch on baits - - - - - 123ae78e by Tony Chemit at 2023-01-25T11:47:47+01:00 Avdth - update documentation - - - - - 6 changed files: - core/persistence/avdth/src/main/java/fr/ird/observe/persistence/avdth/data/ImportEngineExecution.java - core/persistence/avdth/src/main/java/fr/ird/observe/persistence/avdth/data/ImportReferentialContext.java - core/persistence/avdth/src/main/java/fr/ird/observe/persistence/avdth/data/logbook/FloatingObjectReader.java - core/persistence/java/src/main/java/fr/ird/observe/entities/data/ps/common/TripConsolidateEngine.java - src/site/markdown/avdth/logbook.md - src/site/markdown/avdth/referential.md Changes: ===================================== core/persistence/avdth/src/main/java/fr/ird/observe/persistence/avdth/data/ImportEngineExecution.java ===================================== @@ -22,7 +22,6 @@ package fr.ird.observe.persistence.avdth.data; * #L% */ -import fr.ird.observe.services.service.data.ps.AvdthDataImportConfiguration; import fr.ird.observe.entities.ObserveTopiaApplicationContext; import fr.ird.observe.entities.data.ps.common.Trip; import fr.ird.observe.entities.data.ps.localmarket.Survey; @@ -36,10 +35,11 @@ import fr.ird.observe.entities.data.ps.logbook.WellPlan; import fr.ird.observe.persistence.avdth.Query; import fr.ird.observe.persistence.avdth.data.common.TripReader; import fr.ird.observe.persistence.avdth.data.logbook.ActivityReader; +import fr.ird.observe.services.service.data.ps.AvdthDataImportConfiguration; import io.ultreia.java4all.util.TimeLog; +import io.ultreia.java4all.util.sql.SqlScriptWriter; import org.apache.logging.log4j.LogManager; import org.apache.logging.log4j.Logger; -import io.ultreia.java4all.util.sql.SqlScriptWriter; import java.io.IOException; import java.sql.ResultSet; @@ -191,18 +191,19 @@ public class ImportEngineExecution extends ImportEngine { protected void loadCatch(Activity activity, String activityId, List<Object> activityPrimaryKey) throws SQLException { int catchesCount = 0; tables.catchTableReader.setPrimaryKey(activityPrimaryKey); - boolean baitsFishingOnly = false; + // If one catch is on baits, then will add observed system 102 on activity + // See https://gitlab.com/ultreiaio/ird-observe/-/issues/1883#note_617802311 + boolean addBaitsFishingOnlyObservedSystem = false; while (tables.catchTableReader.hasNext()) { ResultSet catchRow = tables.catchTableReader.next(); int speciesCode = catchRow.getInt(6); if (speciesCode >= 700 && speciesCode < 720) { - baitsFishingOnly = true; + addBaitsFishingOnlyObservedSystem = true; } catchWriter.intercept(catchRow, activityId, catchesCount++); } - if (baitsFishingOnly) { - // See #1883 - context.addBaitOnlyObservedSystem(activity); + if (addBaitsFishingOnlyObservedSystem) { + activity.addObservedSystem(context.getObservedSystem102()); } } ===================================== core/persistence/avdth/src/main/java/fr/ird/observe/persistence/avdth/data/ImportReferentialContext.java ===================================== @@ -24,7 +24,6 @@ package fr.ird.observe.persistence.avdth.data; import com.google.common.collect.ImmutableMap; import com.google.common.collect.Maps; -import fr.ird.observe.entities.data.ps.logbook.Activity; import fr.ird.observe.entities.referential.common.DataQuality; import fr.ird.observe.entities.referential.common.FpaZone; import fr.ird.observe.entities.referential.common.Harbour; @@ -134,6 +133,7 @@ public class ImportReferentialContext { private ObservedSystem observedSystem0; private SpeciesCache speciesCache; private ObservedSystem observedSystem20; + private ObservedSystem observedSystem102; private ObservedSystem observedSystem103; private ObservedSystem observedSystem110; private VesselActivity vesselActivity6; @@ -207,6 +207,7 @@ public class ImportReferentialContext { observedSystem0 = observedSystem.get("0"); observedSystem20 = observedSystem.get("20"); observedSystem103 = observedSystem.get("103"); + observedSystem102 = observedSystem.get("102"); observedSystem110 = observedSystem.get("110"); // dcp @@ -502,6 +503,10 @@ public class ImportReferentialContext { return observedSystem0; } + public ObservedSystem getObservedSystem102() { + return observedSystem102; + } + public ObservedSystem getObservedSystem103() { return observedSystem103; } @@ -550,10 +555,6 @@ public class ImportReferentialContext { return code == null ? null : fpaZone.get(code); } - public void addBaitOnlyObservedSystem(Activity entity) { - entity.addObservedSystem(getObservedSystem("102")); - } - public ObjectOperation getObjectOperation(String objectOperationCode) { return objectOperation.get(objectOperationCode); } ===================================== core/persistence/avdth/src/main/java/fr/ird/observe/persistence/avdth/data/logbook/FloatingObjectReader.java ===================================== @@ -67,19 +67,21 @@ public class FloatingObjectReader extends DataReader<FloatingObject> { .put("0", "99") .put("1", "99") .put("2", "99") - .put("3", "99") - .put("4", "99") - .put("10", "99") - .put("14", "99") + + .put("3", "2") + .put("4", "2") + .put("5", "1") + .put("6", "4") + .put("7", "99") .put("8", "99") .put("9", "99") + .put("10", "99") .put("12", "99") .put("13", "99") + .put("14", "99") .put("15", "99") - .put("5", "1") - .put("6", "4") .put("22", "4") .put("23", "1") .put("24", "4") @@ -101,8 +103,16 @@ public class FloatingObjectReader extends DataReader<FloatingObject> { * If for a given {@code ACTIVITE.C_OPERA}, no value is found here, this means that there is no buoy to create. */ public static final Map<String, String> TRANSMITTING_BUOY_OPERATION_CODE_MAPPING = ImmutableMap.<String, String>builder() + .put("3", "1") // Visite | Visite + .put("4", "1") // Visite | Visite .put("5", "3") // Mise à l'eau | null ou 3 Mise à l'eau .put("6", "2") // Retrait | null ou 2 Récupération + .put("7", "1") // Visite | Visite + .put("10", "1")// Visite | Visite + .put("12", "1")// Visite | Visite + .put("13", "1")// Visite | Visite + .put("14", "1")// Visite | Visite + .put("15", "1")// Visite | Visite .put("23", "3") // Mise à l'eau | 3 Mise à l'eau .put("24", "2") // Retrait | 2 Récupération .put("25", "3") // Visite | 3 Mise à l'eau @@ -128,15 +138,15 @@ public class FloatingObjectReader extends DataReader<FloatingObject> { "2", "3", "4", + "6", + "7", + "8", "9", "10", "12", + "13", "14", "15", - "7", - "8", - "13", - "6", "22", "24", "25", ===================================== core/persistence/java/src/main/java/fr/ird/observe/entities/data/ps/common/TripConsolidateEngine.java ===================================== @@ -38,7 +38,6 @@ import fr.ird.observe.dto.data.ps.observation.consolidate.ActivityConsolidateRes import fr.ird.observe.dto.data.ps.observation.consolidate.FloatingObjectConsolidateRequest; import fr.ird.observe.dto.referential.I18nReferentialHelper; import fr.ird.observe.dto.referential.ReferentialLocale; -import fr.ird.observe.dto.referential.ps.common.VesselActivityReference; import fr.ird.observe.entities.ObserveTopiaDaoSupplier; import fr.ird.observe.entities.ToolkitTopiaPersistenceContextSupport; import fr.ird.observe.entities.data.ps.localmarket.Batch; @@ -178,9 +177,8 @@ public class TripConsolidateEngine { String activityPrefix = String.format("%s - Activity [%s/%s] ", routePrefix, ++activityIndex, activityMax); - boolean floatingObjectEmpty = activity.isFloatingObjectEmpty(); - if (!VesselActivityReference.isSetOperation(activity) && floatingObjectEmpty) { - log.debug(String.format("%s Skip activity (No set nor dcp found): %s", activityPrefix, activity.getTopiaId())); + if (activity.getSetSize() == 0 && activity.isFloatingObjectEmpty()) { + log.info(String.format("%s Skip activity (No set nor floating object found): %s", activityPrefix, activity.getTopiaId())); continue; } ===================================== src/site/markdown/avdth/logbook.md ===================================== @@ -69,11 +69,13 @@ On effecute la somme pour toutes les activités AVDTH du jour * Note 4 ### <a name="n_0_4"></a> -Voir mapping [REF_VESSEL_ACTIVITY_TYPE](./referential.html#REF_VESSEL_ACTIVITY_TYPE) si code 13 (dans ObServe) alors on ajoute un dcp +Voir mapping [REF_VESSEL_ACTIVITY_TYPE](./referential.html#REF_VESSEL_ACTIVITY_TYPE) * Note 5 ### <a name="n_0_5"></a> -Voir mapping [REF_SCHOOL_TYPE](./referential.html#REF_SCHOOL_TYPE) +Voir mapping [REF_SCHOOL_TYPE](./referential.html#REF_SCHOOL_TYPE). + +**À noter que cette donée est renseigné uniquement si ```ACTIVITE_C_OPERA in (0, 1, 2, 14)```** * Note 6 ### <a name="n_0_6"></a> @@ -103,6 +105,36 @@ Donnée non persistée dans ObServe, (voir https://gitlab.com/ultreiaio/ird-obse ### <a name="n_0_12"></a> Mapping sur le code. +### Cas des activités de pêche à partir de ```ACTIVITE.C_OPERA``` + +Pour les activités de pêche, on rajoute des données sur l'activité pour décrire la calée. + +| ACTIVITE.C_OPERA | SetSuccessStatus [1](#n_5_1) | ReasonForNullSet [1](#n_5_1) | SetCount | SchoolType | +|-----------------:|-----------------------------:|------------------------------:|:---------|:-----------| +| 0 | 0 | 0 | (1) | (2) | +| 1 | 1 | non renseigné | (1) | (2) | +| 2 | 2 | non renseigné | (1) | (2) | +| 14 (3) | 2 | non renseigné | (1) | (2) | + +* Note 1 +#### <a name="n_5_1"></a> + +On utilise le code du réferentiel ObServe + +* Note 2 +#### <a name="n_5_2"></a> + +* Note 3 +#### <a name="n_5_3"></a> + +* Note 4 +#### <a name="n_5_4"></a> + +0. +1. calculé depuis ```ACTIVITE.V_NB_OP``` +2. calculé depuis ```ACTIVITE.C_TBANC``` +3. dans ce cas on rajoute aussi le système observé ***110** + ### Ajout d'un DCP On ajoute un DCP uniquement pour une activité avec **vesselActivity.code == 13**. ===================================== src/site/markdown/avdth/referential.md ===================================== @@ -223,8 +223,8 @@ Mapping type d'activité | 1 | 6 (1) | | 2 | 6 (1) | | 14 | 6 (1) (3) | -| 3 | 2 | -| 4 | 1 | +| 3 | 2 (ou 102) (4) | +| 4 | 1 (ou 101) (4) | | 5 | 13 (2) | | 6 | 13 (2) | | 22 | 13 (2) | @@ -251,11 +251,13 @@ Mapping type d'activité | 19 | 29 | | 20 | 30 | | 35 | 31 | -| 36 | 99 **FIXME** | +| 36 | 36 (5) | 1. **Seule une activité de code 6 permet la création de captures**. 2. **Seule une activité de code 13 permet la création de DCP**.. 3. On ajoute en plus un système observé **110** et **setSuccessStatus** à **2**. +4. Si un dcp est trouvé alors on utilise un type d'activité introduit en version *9.1.0* qui permet l'ajout de dcp. +5. Le type d'activité a été ajouté lors de la migration de la version *9.1.0* (avant on utilisait *99*) ### REF_SCHOOL_TYPE View it on GitLab: https://gitlab.com/ultreiaio/ird-observe/-/compare/a16389e169267c82132b980a… -- View it on GitLab: https://gitlab.com/ultreiaio/ird-observe/-/compare/a16389e169267c82132b980a… You're receiving this email because of your account on gitlab.com.
1 0
0 0
[Git][ultreiaio/ird-observe][develop] 3 commits: Procédure de calcul qui ne détecte pas les données modifiées - Closes #2615
by Tony CHEMIT (@tchemit) 25 Jan '23

25 Jan '23
Tony CHEMIT pushed to branch develop at ultreiaio / ird-observe Commits: 788f7b86 by Tony Chemit at 2023-01-25T11:49:52+01:00 Procédure de calcul qui ne détecte pas les données modifiées - Closes #2615 - - - - - f086b9ef by Tony Chemit at 2023-01-25T11:49:52+01:00 Avdth - Review some Floating Object mappings (sort them and re-add some removed ones...) - See #2612 - - - - - c26a5fdf by Tony Chemit at 2023-01-25T11:49:52+01:00 Avdth - update documentation - - - - - 4 changed files: - core/persistence/avdth/src/main/java/fr/ird/observe/persistence/avdth/data/logbook/FloatingObjectReader.java - core/persistence/java/src/main/java/fr/ird/observe/entities/data/ps/common/TripConsolidateEngine.java - src/site/markdown/avdth/logbook.md - src/site/markdown/avdth/referential.md Changes: ===================================== core/persistence/avdth/src/main/java/fr/ird/observe/persistence/avdth/data/logbook/FloatingObjectReader.java ===================================== @@ -76,19 +76,21 @@ public class FloatingObjectReader extends DataReader<FloatingObject> { .put("0", "99") .put("1", "99") .put("2", "99") - .put("3", "99") - .put("4", "99") - .put("10", "99") - .put("14", "99") + + .put("3", "2") + .put("4", "2") + .put("5", "1") + .put("6", "4") + .put("7", "99") .put("8", "99") .put("9", "99") + .put("10", "99") .put("12", "99") .put("13", "99") + .put("14", "99") .put("15", "99") - .put("5", "1") - .put("6", "4") .put("22", "4") .put("23", "1") .put("24", "4") @@ -110,8 +112,16 @@ public class FloatingObjectReader extends DataReader<FloatingObject> { * If for a given {@code ACTIVITE.C_OPERA}, no value is found here, this means that there is no buoy to create. */ public static final Map<String, String> TRANSMITTING_BUOY_OPERATION_CODE_MAPPING = ImmutableMap.<String, String>builder() + .put("3", "1") // Visite | Visite + .put("4", "1") // Visite | Visite .put("5", "3") // Mise à l'eau | null ou 3 Mise à l'eau .put("6", "2") // Retrait | null ou 2 Récupération + .put("7", "1") // Visite | Visite + .put("10", "1")// Visite | Visite + .put("12", "1")// Visite | Visite + .put("13", "1")// Visite | Visite + .put("14", "1")// Visite | Visite + .put("15", "1")// Visite | Visite .put("23", "3") // Mise à l'eau | 3 Mise à l'eau .put("24", "2") // Retrait | 2 Récupération .put("25", "3") // Visite | 3 Mise à l'eau @@ -137,15 +147,15 @@ public class FloatingObjectReader extends DataReader<FloatingObject> { "2", "3", "4", + "6", + "7", + "8", "9", "10", "12", + "13", "14", "15", - "7", - "8", - "13", - "6", "22", "24", "25", ===================================== core/persistence/java/src/main/java/fr/ird/observe/entities/data/ps/common/TripConsolidateEngine.java ===================================== @@ -38,7 +38,6 @@ import fr.ird.observe.dto.data.ps.observation.consolidate.ActivityConsolidateRes import fr.ird.observe.dto.data.ps.observation.consolidate.FloatingObjectConsolidateRequest; import fr.ird.observe.dto.referential.I18nReferentialHelper; import fr.ird.observe.dto.referential.ReferentialLocale; -import fr.ird.observe.dto.referential.ps.common.VesselActivityReference; import fr.ird.observe.entities.ObserveTopiaDaoSupplier; import fr.ird.observe.entities.ToolkitTopiaPersistenceContextSupport; import fr.ird.observe.entities.data.ps.localmarket.Batch; @@ -178,9 +177,8 @@ public class TripConsolidateEngine { String activityPrefix = String.format("%s - Activity [%s/%s] ", routePrefix, ++activityIndex, activityMax); - boolean floatingObjectEmpty = activity.isFloatingObjectEmpty(); - if (!VesselActivityReference.isSetOperation(activity) && floatingObjectEmpty) { - log.debug(String.format("%s Skip activity (No set nor dcp found): %s", activityPrefix, activity.getTopiaId())); + if (activity.getSetSize() == 0 && activity.isFloatingObjectEmpty()) { + log.debug(String.format("%s Skip activity (No set nor floating object found): %s", activityPrefix, activity.getTopiaId())); continue; } ===================================== src/site/markdown/avdth/logbook.md ===================================== @@ -69,11 +69,13 @@ On effecute la somme pour toutes les activités AVDTH du jour * Note 4 ### <a name="n_0_4"></a> -Voir mapping [REF_VESSEL_ACTIVITY_TYPE](./referential.html#REF_VESSEL_ACTIVITY_TYPE) si code 13 (dans ObServe) alors on ajoute un dcp +Voir mapping [REF_VESSEL_ACTIVITY_TYPE](./referential.html#REF_VESSEL_ACTIVITY_TYPE) * Note 5 ### <a name="n_0_5"></a> -Voir mapping [REF_SCHOOL_TYPE](./referential.html#REF_SCHOOL_TYPE) +Voir mapping [REF_SCHOOL_TYPE](./referential.html#REF_SCHOOL_TYPE). + +**À noter que cette donée est renseigné uniquement si ```ACTIVITE_C_OPERA in (0, 1, 2, 14)```** * Note 6 ### <a name="n_0_6"></a> @@ -103,6 +105,36 @@ Donnée non persistée dans ObServe, (voir https://gitlab.com/ultreiaio/ird-obse ### <a name="n_0_12"></a> Mapping sur le code. +### Cas des activités de pêche à partir de ```ACTIVITE.C_OPERA``` + +Pour les activités de pêche, on rajoute des données sur l'activité pour décrire la calée. + +| ACTIVITE.C_OPERA | SetSuccessStatus [1](#n_5_1) | ReasonForNullSet [1](#n_5_1) | SetCount | SchoolType | +|-----------------:|-----------------------------:|------------------------------:|:---------|:-----------| +| 0 | 0 | 0 | (1) | (2) | +| 1 | 1 | non renseigné | (1) | (2) | +| 2 | 2 | non renseigné | (1) | (2) | +| 14 (3) | 2 | non renseigné | (1) | (2) | + +* Note 1 +#### <a name="n_5_1"></a> + +On utilise le code du réferentiel ObServe + +* Note 2 +#### <a name="n_5_2"></a> + +* Note 3 +#### <a name="n_5_3"></a> + +* Note 4 +#### <a name="n_5_4"></a> + +0. +1. calculé depuis ```ACTIVITE.V_NB_OP``` +2. calculé depuis ```ACTIVITE.C_TBANC``` +3. dans ce cas on rajoute aussi le système observé ***110** + ### Ajout d'un DCP On ajoute un DCP uniquement pour une activité avec **vesselActivity.code == 13**. ===================================== src/site/markdown/avdth/referential.md ===================================== @@ -223,8 +223,8 @@ Mapping type d'activité | 1 | 6 (1) | | 2 | 6 (1) | | 14 | 6 (1) (3) | -| 3 | 2 | -| 4 | 1 | +| 3 | 2 (ou 102) (4) | +| 4 | 1 (ou 101) (4) | | 5 | 13 (2) | | 6 | 13 (2) | | 22 | 13 (2) | @@ -251,11 +251,13 @@ Mapping type d'activité | 19 | 29 | | 20 | 30 | | 35 | 31 | -| 36 | 36 | +| 36 | 36 (5) | 1. **Seule une activité de code 6 permet la création de captures**. 2. **Seule une activité de code 13 permet la création de DCP**.. 3. On ajoute en plus un système observé **110** et **setSuccessStatus** à **2**. +4. Si un dcp est trouvé alors on utilise un type d'activité introduit en version *9.1.0* qui permet l'ajout de dcp. +5. Le type d'activité a été ajouté lors de la migration de la version *9.1.0* (avant on utilisait *99*) ### REF_SCHOOL_TYPE View it on GitLab: https://gitlab.com/ultreiaio/ird-observe/-/compare/02e11b305141de38f7517168… -- View it on GitLab: https://gitlab.com/ultreiaio/ird-observe/-/compare/02e11b305141de38f7517168… You're receiving this email because of your account on gitlab.com.
1 0
0 0
[Git][ultreiaio/ird-observe][develop-9.0.x] 2 commits: Procédure de calcul qui ne détecte pas les données modifiées - Closes #2615
by Tony CHEMIT (@tchemit) 25 Jan '23

25 Jan '23
Tony CHEMIT pushed to branch develop-9.0.x at ultreiaio / ird-observe Commits: 75eae88d by Tony Chemit at 2023-01-25T09:20:14+01:00 Procédure de calcul qui ne détecte pas les données modifiées - Closes #2615 - - - - - a16389e1 by Tony Chemit at 2023-01-25T09:35:15+01:00 Avdth - Review some Floating Object mappings (sort them and re-add some removed ones...) - See #2612 - - - - - 2 changed files: - core/persistence/avdth/src/main/java/fr/ird/observe/persistence/avdth/data/logbook/FloatingObjectReader.java - core/persistence/java/src/main/java/fr/ird/observe/entities/data/ps/common/TripConsolidateEngine.java Changes: ===================================== core/persistence/avdth/src/main/java/fr/ird/observe/persistence/avdth/data/logbook/FloatingObjectReader.java ===================================== @@ -67,19 +67,21 @@ public class FloatingObjectReader extends DataReader<FloatingObject> { .put("0", "99") .put("1", "99") .put("2", "99") - .put("3", "99") - .put("4", "99") - .put("10", "99") - .put("14", "99") + + .put("3", "2") + .put("4", "2") + .put("5", "1") + .put("6", "4") + .put("7", "99") .put("8", "99") .put("9", "99") + .put("10", "99") .put("12", "99") .put("13", "99") + .put("14", "99") .put("15", "99") - .put("5", "1") - .put("6", "4") .put("22", "4") .put("23", "1") .put("24", "4") @@ -101,8 +103,16 @@ public class FloatingObjectReader extends DataReader<FloatingObject> { * If for a given {@code ACTIVITE.C_OPERA}, no value is found here, this means that there is no buoy to create. */ public static final Map<String, String> TRANSMITTING_BUOY_OPERATION_CODE_MAPPING = ImmutableMap.<String, String>builder() + .put("3", "1") // Visite | Visite + .put("4", "1") // Visite | Visite .put("5", "3") // Mise à l'eau | null ou 3 Mise à l'eau .put("6", "2") // Retrait | null ou 2 Récupération + .put("7", "1") // Visite | Visite + .put("10", "1")// Visite | Visite + .put("12", "1")// Visite | Visite + .put("13", "1")// Visite | Visite + .put("14", "1")// Visite | Visite + .put("15", "1")// Visite | Visite .put("23", "3") // Mise à l'eau | 3 Mise à l'eau .put("24", "2") // Retrait | 2 Récupération .put("25", "3") // Visite | 3 Mise à l'eau @@ -128,15 +138,15 @@ public class FloatingObjectReader extends DataReader<FloatingObject> { "2", "3", "4", + "6", + "7", + "8", "9", "10", "12", + "13", "14", "15", - "7", - "8", - "13", - "6", "22", "24", "25", ===================================== core/persistence/java/src/main/java/fr/ird/observe/entities/data/ps/common/TripConsolidateEngine.java ===================================== @@ -38,7 +38,6 @@ import fr.ird.observe.dto.data.ps.observation.consolidate.ActivityConsolidateRes import fr.ird.observe.dto.data.ps.observation.consolidate.FloatingObjectConsolidateRequest; import fr.ird.observe.dto.referential.I18nReferentialHelper; import fr.ird.observe.dto.referential.ReferentialLocale; -import fr.ird.observe.dto.referential.ps.common.VesselActivityReference; import fr.ird.observe.entities.ObserveTopiaDaoSupplier; import fr.ird.observe.entities.ToolkitTopiaPersistenceContextSupport; import fr.ird.observe.entities.data.ps.localmarket.Batch; @@ -178,9 +177,8 @@ public class TripConsolidateEngine { String activityPrefix = String.format("%s - Activity [%s/%s] ", routePrefix, ++activityIndex, activityMax); - boolean floatingObjectEmpty = activity.isFloatingObjectEmpty(); - if (!VesselActivityReference.isSetOperation(activity) && floatingObjectEmpty) { - log.debug(String.format("%s Skip activity (No set nor dcp found): %s", activityPrefix, activity.getTopiaId())); + if (activity.getSetSize() == 0) { + log.info(String.format("%s Skip activity (No set found): %s", activityPrefix, activity.getTopiaId())); continue; } View it on GitLab: https://gitlab.com/ultreiaio/ird-observe/-/compare/c1e15bcdaa8b84ba74ed851c… -- View it on GitLab: https://gitlab.com/ultreiaio/ird-observe/-/compare/c1e15bcdaa8b84ba74ed851c… You're receiving this email because of your account on gitlab.com.
1 0
0 0
[Git][ultreiaio/ird-observe][develop] 2 commits: Procédure de calcul qui ne détecte pas les données modifiées - Closes #2615
by Tony CHEMIT (@tchemit) 25 Jan '23

25 Jan '23
Tony CHEMIT pushed to branch develop at ultreiaio / ird-observe Commits: 7c66455f by Tony Chemit at 2023-01-25T09:36:04+01:00 Procédure de calcul qui ne détecte pas les données modifiées - Closes #2615 - - - - - 02e11b30 by Tony Chemit at 2023-01-25T09:36:34+01:00 Avdth - Review some Floating Object mappings (sort them and re-add some removed ones...) - See #2612 - - - - - 2 changed files: - core/persistence/avdth/src/main/java/fr/ird/observe/persistence/avdth/data/logbook/FloatingObjectReader.java - core/persistence/java/src/main/java/fr/ird/observe/entities/data/ps/common/TripConsolidateEngine.java Changes: ===================================== core/persistence/avdth/src/main/java/fr/ird/observe/persistence/avdth/data/logbook/FloatingObjectReader.java ===================================== @@ -76,19 +76,21 @@ public class FloatingObjectReader extends DataReader<FloatingObject> { .put("0", "99") .put("1", "99") .put("2", "99") - .put("3", "99") - .put("4", "99") - .put("10", "99") - .put("14", "99") + + .put("3", "2") + .put("4", "2") + .put("5", "1") + .put("6", "4") + .put("7", "99") .put("8", "99") .put("9", "99") + .put("10", "99") .put("12", "99") .put("13", "99") + .put("14", "99") .put("15", "99") - .put("5", "1") - .put("6", "4") .put("22", "4") .put("23", "1") .put("24", "4") @@ -110,8 +112,16 @@ public class FloatingObjectReader extends DataReader<FloatingObject> { * If for a given {@code ACTIVITE.C_OPERA}, no value is found here, this means that there is no buoy to create. */ public static final Map<String, String> TRANSMITTING_BUOY_OPERATION_CODE_MAPPING = ImmutableMap.<String, String>builder() + .put("3", "1") // Visite | Visite + .put("4", "1") // Visite | Visite .put("5", "3") // Mise à l'eau | null ou 3 Mise à l'eau .put("6", "2") // Retrait | null ou 2 Récupération + .put("7", "1") // Visite | Visite + .put("10", "1")// Visite | Visite + .put("12", "1")// Visite | Visite + .put("13", "1")// Visite | Visite + .put("14", "1")// Visite | Visite + .put("15", "1")// Visite | Visite .put("23", "3") // Mise à l'eau | 3 Mise à l'eau .put("24", "2") // Retrait | 2 Récupération .put("25", "3") // Visite | 3 Mise à l'eau @@ -137,15 +147,15 @@ public class FloatingObjectReader extends DataReader<FloatingObject> { "2", "3", "4", + "6", + "7", + "8", "9", "10", "12", + "13", "14", "15", - "7", - "8", - "13", - "6", "22", "24", "25", ===================================== core/persistence/java/src/main/java/fr/ird/observe/entities/data/ps/common/TripConsolidateEngine.java ===================================== @@ -38,7 +38,6 @@ import fr.ird.observe.dto.data.ps.observation.consolidate.ActivityConsolidateRes import fr.ird.observe.dto.data.ps.observation.consolidate.FloatingObjectConsolidateRequest; import fr.ird.observe.dto.referential.I18nReferentialHelper; import fr.ird.observe.dto.referential.ReferentialLocale; -import fr.ird.observe.dto.referential.ps.common.VesselActivityReference; import fr.ird.observe.entities.ObserveTopiaDaoSupplier; import fr.ird.observe.entities.ToolkitTopiaPersistenceContextSupport; import fr.ird.observe.entities.data.ps.localmarket.Batch; @@ -178,9 +177,8 @@ public class TripConsolidateEngine { String activityPrefix = String.format("%s - Activity [%s/%s] ", routePrefix, ++activityIndex, activityMax); - boolean floatingObjectEmpty = activity.isFloatingObjectEmpty(); - if (!VesselActivityReference.isSetOperation(activity) && floatingObjectEmpty) { - log.debug(String.format("%s Skip activity (No set nor dcp found): %s", activityPrefix, activity.getTopiaId())); + if (activity.getSetSize() == 0) { + log.info(String.format("%s Skip activity (No set found): %s", activityPrefix, activity.getTopiaId())); continue; } View it on GitLab: https://gitlab.com/ultreiaio/ird-observe/-/compare/fe1373e03bd6db2515f690b5… -- View it on GitLab: https://gitlab.com/ultreiaio/ird-observe/-/compare/fe1373e03bd6db2515f690b5… You're receiving this email because of your account on gitlab.com.
1 0
0 0
[Git][ultreiaio/ird-observe][develop] 3 commits: Migration - Add VesselActivity 101 and 102
by Tony CHEMIT (@tchemit) 24 Jan '23

24 Jan '23
Tony CHEMIT pushed to branch develop at ultreiaio / ird-observe Commits: 05efd969 by Tony Chemit at 2023-01-24T13:14:47+01:00 Migration - Add VesselActivity 101 and 102 - - - - - 6b441c82 by Tony Chemit at 2023-01-24T13:14:47+01:00 Avdth - use new vessel activity for activity that does not allow fad - - - - - fe1373e0 by Tony Chemit at 2023-01-24T14:02:59+01:00 Merge branch &#39;feature/issue_2612&#39; into develop Gestion des FAD sur OPERA AVDTH non mappées vers VesselActivity avec allowFad=TRUE - Closes #2612 - - - - - 9 changed files: - core/persistence/avdth/src/main/java/fr/ird/observe/persistence/avdth/data/ImportEngineExecution.java - core/persistence/avdth/src/main/java/fr/ird/observe/persistence/avdth/data/ImportReferentialContext.java - core/persistence/avdth/src/main/java/fr/ird/observe/persistence/avdth/data/logbook/ActivityWriter.java - core/persistence/avdth/src/main/java/fr/ird/observe/persistence/avdth/data/logbook/FloatingObjectReader.java - core/persistence/migration/src/main/java/fr/ird/observe/spi/migration/v9/DataSourceMigrationForVersion_9_1.java - + core/persistence/migration/src/main/resources/db/migration/v9/9.1/05_add_referential_ps_common_VesselActivity_101-common.sql - + core/persistence/migration/src/main/resources/db/migration/v9/9.1/05_add_referential_ps_common_VesselActivity_102-common.sql - core/persistence/test/src/test/resources/fixtures/persistence/table_count/referential.properties - model/src/main/resources/fixtures/global.properties Changes: ===================================== core/persistence/avdth/src/main/java/fr/ird/observe/persistence/avdth/data/ImportEngineExecution.java ===================================== @@ -178,6 +178,9 @@ public class ImportEngineExecution extends ImportEngine { context.addDefaultObservedSystemOrSanitizeFloatingObjectOnes(activity); // write activity observed systems activityWriter.writeObservedSystems(activity); + // while loading Floating objects, we may have change the vessel activity of the activity + // See https://gitlab.com/ultreiaio/ird-observe/-/issues/2612 + activityWriter.updateVesselActivity(activity); // get another activity ? if (tables.activityTableReader.hasNext()) { ===================================== core/persistence/avdth/src/main/java/fr/ird/observe/persistence/avdth/data/ImportReferentialContext.java ===================================== @@ -137,6 +137,8 @@ public class ImportReferentialContext { private ObservedSystem observedSystem103; private ObservedSystem observedSystem110; private VesselActivity vesselActivity6; + private VesselActivity vesselActivity101; + private VesselActivity vesselActivity102; private SchoolType schoolType0; private SchoolType schoolType1; private SchoolType schoolType2; @@ -189,6 +191,8 @@ public class ImportReferentialContext { // activity vesselActivity = Maps.uniqueIndex(referential.getVesselActivity(), VesselActivity::getCode); vesselActivity6 = vesselActivity.get("6"); + vesselActivity101 = vesselActivity.get("101"); + vesselActivity102 = vesselActivity.get("102"); schoolType = Maps.uniqueIndex(referential.getSchoolType(), SchoolType::getCode); schoolType0 = schoolType.get("0"); schoolType1 = schoolType.get("1"); @@ -587,4 +591,11 @@ public class ImportReferentialContext { return vesselActivity6; } + public VesselActivity getVesselActivity101() { + return vesselActivity101; + } + + public VesselActivity getVesselActivity102() { + return vesselActivity102; + } } ===================================== core/persistence/avdth/src/main/java/fr/ird/observe/persistence/avdth/data/logbook/ActivityWriter.java ===================================== @@ -156,6 +156,24 @@ public class ActivityWriter extends DataWriter<Activity, ActivityReader> { } } + /** + * To update the activity sql code if vessel activity was changed while floating object import. + * <p> + * See <a href="https://gitlab.com/ultreiaio/ird-observe/-/issues/2612">Issue 2612</a> + * + * @param entity activity to process + */ + public void updateVesselActivity(Activity entity) { + if (FloatingObjectReader.CHANGED_VESSEL_ACTIVITY_CODE_BY_FLOATING_OBJECT.containsValue(entity.getVesselActivity().getCode())) { + String id = entity.getId(); + String vesselActivityId = entity.getVesselActivity().getId(); + String sql = String.format("UPDATE ps_logbook.activity SET vesselActivity = %1$s WHERE topiaId = %2$s;", + escapeString(vesselActivityId), + escapeString(id)); + writer().writeSql(sql); + } + } + @Override public void toResult(Map<String, Integer> resultBuilder) { super.toResult(resultBuilder); ===================================== core/persistence/avdth/src/main/java/fr/ird/observe/persistence/avdth/data/logbook/FloatingObjectReader.java ===================================== @@ -59,6 +59,15 @@ import java.util.TreeSet; */ @SuppressWarnings("SpellCheckingInspection") public class FloatingObjectReader extends DataReader<FloatingObject> { + /** + * To map ObServe vessel activity when in Avdth we found a Floating object to persist. + * <p> + * Keys are current ObServe vessel activity code used, values are the real vessel activity code to use (which allow fad) + * <p> + * See <a href="https://gitlab.com/ultreiaio/ird-observe/-/issues/2612">Issue 2612</a> + */ + public static final Map<String, String> CHANGED_VESSEL_ACTIVITY_CODE_BY_FLOATING_OBJECT = Map.of("1", "101", + "2", "102"); private static final Logger log = LogManager.getLogger(FloatingObjectReader.class); /** * To get FloatingObject.objectOperation code from avdth ACTIVITE.C_OPERA. @@ -275,21 +284,42 @@ public class FloatingObjectReader extends DataReader<FloatingObject> { } if (!vesselActivityAllowFad) { - // If ObServe veseel activity does not support Floating object, reject import - List<Object> activityPrimaryKey = ActivityReader.getActivityPk(resultSet); - String activityPk = Query.primaryKeyString(activityPrimaryKey); - String message = String.format("On activity %s (ACTIVITE.C_OPERA %s), can not add Floating object since ObServe Vessel activity with code : %s does not allow FAD (floatingObjectExistsInAvdth? %b, floatingObjectCreatedByObservedSystem? %b).", - activityPk, - vesselActivityCode, - vesselActivity.getCode(), - floatingObjectExistsInAvdth, - floatingObjectCreatedByObservedSystem); - log.warn(message); - badVesselActivityCodes.computeIfAbsent(vesselActivityCode, MutableInt::new).increment(); + + String observeVesselActivityCode = activity.getVesselActivity().getCode(); + String realObserveVesselActivityCode = CHANGED_VESSEL_ACTIVITY_CODE_BY_FLOATING_OBJECT.get(observeVesselActivityCode); + if (realObserveVesselActivityCode == null) { + + // If ObServe veseel activity does not support Floating object, reject import + List<Object> activityPrimaryKey = ActivityReader.getActivityPk(resultSet); + String activityPk = Query.primaryKeyString(activityPrimaryKey); + String message = String.format("For AVDTH activity %s (ACTIVITE.C_OPERA %s), the ObServe vessel activity (code: %s, label: %s) does not allow FAD, we still import it... (floatingObjectExistsInAvdth? %b, floatingObjectCreatedByObservedSystem? %b).", + activityPk, + vesselActivityCode, + vesselActivity.getCode(), + vesselActivity.getLabel1(), + floatingObjectExistsInAvdth, + floatingObjectCreatedByObservedSystem); + log.warn(message); + badVesselActivityCodes.computeIfAbsent(vesselActivityCode, MutableInt::new).increment(); // // do not create the floating object // reset(dataContext); // return null; + + } else { + // See https://gitlab.com/ultreiaio/ird-observe/-/issues/2612 + switch (realObserveVesselActivityCode) { + case "101": + // Use vessel activity 101 + activity.setVesselActivity(dataContext.getVesselActivity101()); + break; + case "102": + // Use vessel activity 102 + activity.setVesselActivity(dataContext.getVesselActivity102()); + break; + } + } + } FloatingObject entity = newEntity(FloatingObject.SPI); @@ -474,7 +504,7 @@ public class FloatingObjectReader extends DataReader<FloatingObject> { return dataContext.getObjectOperation(Objects.requireNonNull(objectOperationCode, String.format("Can't find objectOperation with ACTIVITE.C_OPERA: %s", vesselActivityCode))); } - private ObservedSystem getObservedSystem(ImportDataContext dataContext, String objectTypeCode, boolean floatingObjectExistsInAvdth,boolean buoyExistsInAvdth) { + private ObservedSystem getObservedSystem(ImportDataContext dataContext, String objectTypeCode, boolean floatingObjectExistsInAvdth, boolean buoyExistsInAvdth) { if (!floatingObjectExistsInAvdth) { if (!buoyExistsInAvdth) { return dataContext.getObservedSystem0(); ===================================== core/persistence/migration/src/main/java/fr/ird/observe/spi/migration/v9/DataSourceMigrationForVersion_9_1.java ===================================== @@ -275,16 +275,16 @@ public class DataSourceMigrationForVersion_9_1 extends ByMajorMigrationVersionRe executor.addScript("04", "drop_ll_common_weightDeterminationMethod"); if (withIds) { - addVesselActivity36(executor); + addVesselActivities(executor); } // See https://gitlab.com/ultreiaio/ird-observe/-/issues/2435 addNewTable(executor, withIds, "06", "table-common_sizemeasuremethod"); } - private void addVesselActivity36(MigrationVersionResourceExecutor executor) { + private void addVesselActivities(MigrationVersionResourceExecutor executor) { Set<String> existingCodes = executor.findMultipleResultAstSet(SqlQuery.wrap("SELECT CODE FROM ps_common.VesselActivity", resultSet -> resultSet.getString(1))); Set<String> newCodes = new LinkedHashSet<>(); - Stream.of("36").forEach(code -> { + Stream.of("36", "101", "102").forEach(code -> { if (!existingCodes.contains(code)) { executor.addScript("05", "add_referential_ps_common_VesselActivity_" + code); newCodes.add(code); ===================================== core/persistence/migration/src/main/resources/db/migration/v9/9.1/05_add_referential_ps_common_VesselActivity_101-common.sql ===================================== @@ -0,0 +1,22 @@ +--- +-- #%L +-- ObServe Core :: Persistence :: Migration +-- %% +-- Copyright (C) 2008 - 2023 IRD, Ultreia.io +-- %% +-- This program is free software: you can redistribute it and/or modify +-- it under the terms of the GNU General Public License as +-- published by the Free Software Foundation, either version 3 of the +-- License, or (at your option) any later version. +-- +-- This program is distributed in the hope that it will be useful, +-- but WITHOUT ANY WARRANTY; without even the implied warranty of +-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +-- GNU General Public License for more details. +-- +-- You should have received a copy of the GNU General Public +-- License along with this program. If not, see +-- <http://www.gnu.org/licenses/gpl-3.0.html>. +-- #L% +--- +INSERT INTO ps_common.vesselActivity(topiaid, topiaversion, topiacreatedate, lastupdatedate, code, uri, homeid, needcomment, status, label1, label2, label3, label4, label5, label6, label7, label8, allowFad) VALUES ('fr.ird.referential.ps.common.VesselActivity#${REFERENTIAL_PREFIX}101', 0, ${CURRENT_DATE}, ${CURRENT_TIMESTAMP}, '101', null, null, FALSE, 0, '[déprécié] Transit (route sans recherche)', '[deprecated] Transit (steaming without searching)', '[deprecated] Transit (steaming without searching) TODO', null, null, null, null, null, TRUE); ===================================== core/persistence/migration/src/main/resources/db/migration/v9/9.1/05_add_referential_ps_common_VesselActivity_102-common.sql ===================================== @@ -0,0 +1,22 @@ +--- +-- #%L +-- ObServe Core :: Persistence :: Migration +-- %% +-- Copyright (C) 2008 - 2023 IRD, Ultreia.io +-- %% +-- This program is free software: you can redistribute it and/or modify +-- it under the terms of the GNU General Public License as +-- published by the Free Software Foundation, either version 3 of the +-- License, or (at your option) any later version. +-- +-- This program is distributed in the hope that it will be useful, +-- but WITHOUT ANY WARRANTY; without even the implied warranty of +-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +-- GNU General Public License for more details. +-- +-- You should have received a copy of the GNU General Public +-- License along with this program. If not, see +-- <http://www.gnu.org/licenses/gpl-3.0.html>. +-- #L% +--- +INSERT INTO ps_common.vesselActivity(topiaid, topiaversion, topiacreatedate, lastupdatedate, code, uri, homeid, needcomment, status, label1, label2, label3, label4, label5, label6, label7, label8, allowFad) VALUES ('fr.ird.referential.ps.common.VesselActivity#${REFERENTIAL_PREFIX}102', 0, ${CURRENT_DATE}, ${CURRENT_TIMESTAMP}, '102', null, null, FALSE, 0, '[deprecated] Searching (in general)', '[déprécié] Recherche (général)', '[deprecated] Searching (in general) TODO', null, null, null, null, null, TRUE); ===================================== core/persistence/test/src/test/resources/fixtures/persistence/table_count/referential.properties ===================================== @@ -89,7 +89,7 @@ ps_common.speciesFate=15 ps_common.transmittingBuoyOperation=5 ps_common.transmittingBuoyOwnership=5 ps_common.transmittingBuoyType=62 -ps_common.vesselActivity=37 +ps_common.vesselActivity=39 ps_common.weightCategory=137 ps_landing.destination=28 ps_landing.fate=2 ===================================== model/src/main/resources/fixtures/global.properties ===================================== @@ -25,4 +25,4 @@ REFERENCE_DATA_COUNT=68 ENTITIES_LIMIT_SIZE=100 # fr.ird.observe.services.service.referential.ReferentialService # fr.ird.observe.services.local.service.referential.DifferentialModelTest -REFERENTIAL_COUNT=4094 +REFERENTIAL_COUNT=4096 View it on GitLab: https://gitlab.com/ultreiaio/ird-observe/-/compare/ac810ae19e72c394093bd8b7… -- View it on GitLab: https://gitlab.com/ultreiaio/ird-observe/-/compare/ac810ae19e72c394093bd8b7… You're receiving this email because of your account on gitlab.com.
1 0
0 0
[Git][ultreiaio/ird-observe][develop] 3 commits: Avdth - fix merge from v9.0.x
by Tony CHEMIT (@tchemit) 24 Jan '23

24 Jan '23
Tony CHEMIT pushed to branch develop at ultreiaio / ird-observe Commits: 52968fb9 by Tony Chemit at 2023-01-24T09:48:57+01:00 Avdth - fix merge from v9.0.x - - - - - 89565bf9 by Tony Chemit at 2023-01-24T13:13:11+01:00 Avdth - Remove comment in AvdthDataImportResult (move it to test) - - - - - ac810ae1 by Tony Chemit at 2023-01-24T13:14:39+01:00 Avdth test - show export Result if not empty + improve code - - - - - 5 changed files: - core/api/services/src/main/java/fr/ird/observe/services/service/data/ps/AvdthDataImportResult.java - core/persistence/avdth/src/main/java/fr/ird/observe/persistence/avdth/data/DataReader.java - core/persistence/avdth/src/main/java/fr/ird/observe/persistence/avdth/data/logbook/FloatingObjectWriter.java - core/persistence/test/src/test/java/fr/ird/observe/persistence/avdth/data/AvdthDataBuilderTestSupport.java - core/persistence/test/src/test/resources/log4j2.xml Changes: ===================================== core/api/services/src/main/java/fr/ird/observe/services/service/data/ps/AvdthDataImportResult.java ===================================== @@ -87,10 +87,6 @@ public class AvdthDataImportResult implements ObserveDto { this.notReadResult = notReadResult; this.exportResult = exportResult; this.sqlResultPath = sqlResultPath; - // Used for debugging... -// if (!exportResult.isEmpty()) { -// System.out.println(exportResult.entrySet().stream().map(e -> e.getKey() + "=" + e.getValue()).collect(Collectors.joining("\n"))); -// } } public Path getMessageFile() { ===================================== core/persistence/avdth/src/main/java/fr/ird/observe/persistence/avdth/data/DataReader.java ===================================== @@ -64,10 +64,6 @@ public abstract class DataReader<E extends DataEntity> { return context; } - public void addMessage(String message) { - context.addMessage(message); - } - public abstract E read(ImportDataContext dataContext, ResultSet resultSet) throws SQLException; public final int getCount() { ===================================== core/persistence/avdth/src/main/java/fr/ird/observe/persistence/avdth/data/logbook/FloatingObjectWriter.java ===================================== @@ -192,9 +192,7 @@ public class FloatingObjectWriter extends DataWriter<FloatingObject, FloatingObj super.toResult(resultBuilder); toResult(resultBuilder, FloatingObjectPart.class, getReader().getFloatingObjectPartCount()); toResult(resultBuilder, TransmittingBuoy.class, getReader().getTransmittingBuoyCount()); -// getReader().getBadVesselActivityCodes().forEach((k,v)-> { -// resultBuilder.put(String.format("Bad Vessel activity used to build FAD from ACTIVITY.C_OPERA %s used but ObServe not allow fad.", k), v.getValue()); -// }); + getReader().getBadVesselActivityCodes().forEach((k,v)-> resultBuilder.put(String.format("Bad Vessel activity used to build FAD from ACTIVITE.C_OPERA %s used but this vessel activity does not allow fad in ObServe.", k), v.getValue())); } } ===================================== core/persistence/test/src/test/java/fr/ird/observe/persistence/avdth/data/AvdthDataBuilderTestSupport.java ===================================== @@ -26,19 +26,21 @@ import com.google.gson.Gson; import com.google.gson.GsonBuilder; import fr.ird.observe.dto.ObserveUtil; import fr.ird.observe.dto.ProgressionModel; -import fr.ird.observe.services.service.data.ps.AvdthDataImportConfiguration; -import fr.ird.observe.services.service.data.ps.AvdthDataImportResult; -import fr.ird.observe.services.service.data.ps.MissingReferentialException; import fr.ird.observe.dto.referential.ReferentialLocale; import fr.ird.observe.entities.ObserveTopiaPersistenceContext; import fr.ird.observe.entities.data.ps.common.Trip; import fr.ird.observe.persistence.avdth.AvdthFixtures; import fr.ird.observe.persistence.test.PersistenceTestSupportWrite; +import fr.ird.observe.services.service.data.ps.AvdthDataImportConfiguration; +import fr.ird.observe.services.service.data.ps.AvdthDataImportResult; +import fr.ird.observe.services.service.data.ps.MissingReferentialException; import fr.ird.observe.test.DatabaseName; import fr.ird.observe.test.IgnoreTestClassRule; import fr.ird.observe.test.spi.CopyDatabaseConfiguration; import fr.ird.observe.test.spi.DatabaseNameConfiguration; import io.ultreia.java4all.util.SortedProperties; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; import org.junit.AfterClass; import org.junit.Assert; import org.junit.Assume; @@ -56,6 +58,7 @@ import java.util.Collections; import java.util.List; import java.util.Map; import java.util.stream.Collectors; +import java.util.stream.Stream; /** * Created on 24/05/2021. @@ -65,7 +68,7 @@ import java.util.stream.Collectors; */ @RunWith(Parameterized.class) public abstract class AvdthDataBuilderTestSupport extends PersistenceTestSupportWrite { - + private static final Logger log = LogManager.getLogger(AvdthDataBuilderTestSupport.class); @ClassRule public static final IgnoreTestClassRule IGNORE2 = new IgnoreTestClassRule("Persistence Avdth", "persistence.avdth.test.skip"); @@ -95,8 +98,8 @@ public abstract class AvdthDataBuilderTestSupport extends PersistenceTestSupport } public static List<String> allDatabases(Path path) { - try { - List<String> collect = Files.find(path, 1, (p, b) -> p.toFile().getName().endsWith(".mdb")).map(p -> p.toFile().getName()).sorted().collect(Collectors.toList()); + try (Stream<Path> pathStream = Files.find(path, 1, (p, b) -> p.toFile().getName().endsWith(".mdb"))) { + List<String> collect = pathStream.map(p -> p.toFile().getName()).sorted().collect(Collectors.toList()); Collections.reverse(collect); return collect; } catch (IOException e) { @@ -142,8 +145,12 @@ public abstract class AvdthDataBuilderTestSupport extends PersistenceTestSupport try { result = ImportEngine.run(configuration, localTestMethodResource.getTopiaApplicationContext()); String resultAsGson = gson.toJson(result); - log.info(String.format("Import result:\n%s", resultAsGson)); - log.info(String.format("Import messages:\n%s", result.getMessageFile())); + log.debug(String.format("Import result:\n%s", resultAsGson)); + // Used for debugging... + Map<String, Integer> exportResult = result.getExportResult(); + if (!exportResult.isEmpty()) { + log.info(String.format("Export result:\n%s", exportResult.entrySet().stream().map(e -> e.getKey() + "=" + e.getValue()).collect(Collectors.joining("\n")))); + } } finally { ObserveUtil.cleanMemory(); } ===================================== core/persistence/test/src/test/resources/log4j2.xml ===================================== @@ -46,6 +46,9 @@ <Logger name="org.nuiton.topia.persistence.security.SecurityScriptHelper" level="info" additivity="false"> <AppenderRef ref="console"/> </Logger> + <Logger name="fr.ird.observe.persistence.avdth.data.AvdthDataBuilderTestSupport" level="info" additivity="false"> + <AppenderRef ref="console"/> + </Logger> <Logger name="fr.ird.observe.persistence.test" level="info" additivity="false"> <AppenderRef ref="console"/> </Logger> View it on GitLab: https://gitlab.com/ultreiaio/ird-observe/-/compare/c1fb83d2883489e3ed424577… -- View it on GitLab: https://gitlab.com/ultreiaio/ird-observe/-/compare/c1fb83d2883489e3ed424577… You're receiving this email because of your account on gitlab.com.
1 0
0 0
[Git][ultreiaio/ird-observe][develop-9.0.x] 2 commits: Avdth - Remove comment in AvdthDataImportResult (move it to test)
by Tony CHEMIT (@tchemit) 24 Jan '23

24 Jan '23
Tony CHEMIT pushed to branch develop-9.0.x at ultreiaio / ird-observe Commits: 9f4f42a0 by Tony Chemit at 2023-01-24T13:15:11+01:00 Avdth - Remove comment in AvdthDataImportResult (move it to test) - - - - - c1e15bcd by Tony Chemit at 2023-01-24T13:15:15+01:00 Avdth test - show export Result if not empty + improve code - - - - - 4 changed files: - core/api/services/src/main/java/fr/ird/observe/services/service/data/ps/AvdthDataImportResult.java - core/persistence/avdth/src/main/java/fr/ird/observe/persistence/avdth/data/logbook/FloatingObjectWriter.java - core/persistence/test/src/test/java/fr/ird/observe/persistence/avdth/data/AvdthDataBuilderTestSupport.java - core/persistence/test/src/test/resources/log4j2.xml Changes: ===================================== core/api/services/src/main/java/fr/ird/observe/services/service/data/ps/AvdthDataImportResult.java ===================================== @@ -87,10 +87,6 @@ public class AvdthDataImportResult implements ObserveDto { this.notReadResult = notReadResult; this.exportResult = exportResult; this.sqlResultPath = sqlResultPath; - // Used for debugging... -// if (!exportResult.isEmpty()) { -// System.out.println(exportResult.entrySet().stream().map(e -> e.getKey() + "=" + e.getValue()).collect(Collectors.joining("\n"))); -// } } public Path getMessageFile() { ===================================== core/persistence/avdth/src/main/java/fr/ird/observe/persistence/avdth/data/logbook/FloatingObjectWriter.java ===================================== @@ -192,9 +192,7 @@ public class FloatingObjectWriter extends DataWriter<FloatingObject, FloatingObj super.toResult(resultBuilder); toResult(resultBuilder, FloatingObjectPart.class, getReader().getFloatingObjectPartCount()); toResult(resultBuilder, TransmittingBuoy.class, getReader().getTransmittingBuoyCount()); -// getReader().getBadVesselActivityCodes().forEach((k,v)-> { -// resultBuilder.put(String.format("Bad Vessel activity used to build FAD from ACTIVITY.C_OPERA %s used but ObServe not allow fad.", k), v.getValue()); -// }); + getReader().getBadVesselActivityCodes().forEach((k,v)-> resultBuilder.put(String.format("Bad Vessel activity used to build FAD from ACTIVITE.C_OPERA %s used but this vessel activity does not allow fad in ObServe.", k), v.getValue())); } } ===================================== core/persistence/test/src/test/java/fr/ird/observe/persistence/avdth/data/AvdthDataBuilderTestSupport.java ===================================== @@ -26,19 +26,21 @@ import com.google.gson.Gson; import com.google.gson.GsonBuilder; import fr.ird.observe.dto.ObserveUtil; import fr.ird.observe.dto.ProgressionModel; -import fr.ird.observe.services.service.data.ps.AvdthDataImportConfiguration; -import fr.ird.observe.services.service.data.ps.AvdthDataImportResult; -import fr.ird.observe.services.service.data.ps.MissingReferentialException; import fr.ird.observe.dto.referential.ReferentialLocale; import fr.ird.observe.entities.ObserveTopiaPersistenceContext; import fr.ird.observe.entities.data.ps.common.Trip; import fr.ird.observe.persistence.avdth.AvdthFixtures; import fr.ird.observe.persistence.test.PersistenceTestSupportWrite; +import fr.ird.observe.services.service.data.ps.AvdthDataImportConfiguration; +import fr.ird.observe.services.service.data.ps.AvdthDataImportResult; +import fr.ird.observe.services.service.data.ps.MissingReferentialException; import fr.ird.observe.test.DatabaseName; import fr.ird.observe.test.IgnoreTestClassRule; import fr.ird.observe.test.spi.CopyDatabaseConfiguration; import fr.ird.observe.test.spi.DatabaseNameConfiguration; import io.ultreia.java4all.util.SortedProperties; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; import org.junit.AfterClass; import org.junit.Assert; import org.junit.Assume; @@ -56,6 +58,7 @@ import java.util.Collections; import java.util.List; import java.util.Map; import java.util.stream.Collectors; +import java.util.stream.Stream; /** * Created on 24/05/2021. @@ -65,7 +68,7 @@ import java.util.stream.Collectors; */ @RunWith(Parameterized.class) public abstract class AvdthDataBuilderTestSupport extends PersistenceTestSupportWrite { - + private static final Logger log = LogManager.getLogger(AvdthDataBuilderTestSupport.class); @ClassRule public static final IgnoreTestClassRule IGNORE2 = new IgnoreTestClassRule("Persistence Avdth", "persistence.avdth.test.skip"); @@ -95,8 +98,8 @@ public abstract class AvdthDataBuilderTestSupport extends PersistenceTestSupport } public static List<String> allDatabases(Path path) { - try { - List<String> collect = Files.find(path, 1, (p, b) -> p.toFile().getName().endsWith(".mdb")).map(p -> p.toFile().getName()).sorted().collect(Collectors.toList()); + try (Stream<Path> pathStream = Files.find(path, 1, (p, b) -> p.toFile().getName().endsWith(".mdb"))) { + List<String> collect = pathStream.map(p -> p.toFile().getName()).sorted().collect(Collectors.toList()); Collections.reverse(collect); return collect; } catch (IOException e) { @@ -142,8 +145,12 @@ public abstract class AvdthDataBuilderTestSupport extends PersistenceTestSupport try { result = ImportEngine.run(configuration, localTestMethodResource.getTopiaApplicationContext()); String resultAsGson = gson.toJson(result); - log.info(String.format("Import result:\n%s", resultAsGson)); - log.info(String.format("Import messages:\n%s", result.getMessageFile())); + log.debug(String.format("Import result:\n%s", resultAsGson)); + // Used for debugging... + Map<String, Integer> exportResult = result.getExportResult(); + if (!exportResult.isEmpty()) { + log.info(String.format("Export result:\n%s", exportResult.entrySet().stream().map(e -> e.getKey() + "=" + e.getValue()).collect(Collectors.joining("\n")))); + } } finally { ObserveUtil.cleanMemory(); } ===================================== core/persistence/test/src/test/resources/log4j2.xml ===================================== @@ -46,6 +46,9 @@ <Logger name="org.nuiton.topia.persistence.security.SecurityScriptHelper" level="info" additivity="false"> <AppenderRef ref="console"/> </Logger> + <Logger name="fr.ird.observe.persistence.avdth.data.AvdthDataBuilderTestSupport" level="info" additivity="false"> + <AppenderRef ref="console"/> + </Logger> <Logger name="fr.ird.observe.persistence.test" level="info" additivity="false"> <AppenderRef ref="console"/> </Logger> View it on GitLab: https://gitlab.com/ultreiaio/ird-observe/-/compare/3b5c1e35a8bb8b9d36518dc1… -- View it on GitLab: https://gitlab.com/ultreiaio/ird-observe/-/compare/3b5c1e35a8bb8b9d36518dc1… You're receiving this email because of your account on gitlab.com.
1 0
0 0
[Git][ultreiaio/ird-observe][develop-9.0.x] Revue de migration de Activity.schoolType - Closes #2485
by Tony CHEMIT (@tchemit) 24 Jan '23

24 Jan '23
Tony CHEMIT pushed to branch develop-9.0.x at ultreiaio / ird-observe Commits: 3b5c1e35 by Tony Chemit at 2023-01-24T08:01:03+01:00 Revue de migration de Activity.schoolType - Closes #2485 - - - - - 1 changed file: - core/persistence/avdth/src/main/java/fr/ird/observe/persistence/avdth/data/logbook/ActivityReader.java Changes: ===================================== core/persistence/avdth/src/main/java/fr/ird/observe/persistence/avdth/data/logbook/ActivityReader.java ===================================== @@ -228,6 +228,7 @@ public class ActivityReader extends DataReader<Activity> { String vesselActivityCode = resultSet.getString(13); Object setCountObject = resultSet.getObject(14); Integer setCount = setCountObject == null ? null : ((Number) setCountObject).intValue(); + String schoolTypeCode = resultSet.getString(15); switch (vesselActivityCode) { case "0": // SetSuccessStatus=0 + ReasonForNullSet=0 (See https://gitlab.com/ultreiaio/ird-observe/-/issues/2091) @@ -235,18 +236,24 @@ public class ActivityReader extends DataReader<Activity> { entity.setReasonForNullSet(dataContext.getReasonForNullSet0()); // See https://gitlab.com/ultreiaio/ird-observe/-/issues/2610 entity.setSetCount(setCount); + // See https://gitlab.com/ultreiaio/ird-observe/-/issues/2485 + entity.setSchoolType(dataContext.getSchoolType(schoolTypeCode)); break; case "1": // SetSuccessStatus=1 (See https://gitlab.com/ultreiaio/ird-observe/-/issues/2091) entity.setSetSuccessStatus(dataContext.getSetSuccessStatus1()); // See https://gitlab.com/ultreiaio/ird-observe/-/issues/2610 entity.setSetCount(setCount); + // See https://gitlab.com/ultreiaio/ird-observe/-/issues/2485 + entity.setSchoolType(dataContext.getSchoolType(schoolTypeCode)); break; case "2": // SetSuccessStatus=2 (See https://gitlab.com/ultreiaio/ird-observe/-/issues/2091) entity.setSetSuccessStatus(dataContext.getSetSuccessStatus2()); // See https://gitlab.com/ultreiaio/ird-observe/-/issues/2610 entity.setSetCount(setCount); + // See https://gitlab.com/ultreiaio/ird-observe/-/issues/2485 + entity.setSchoolType(dataContext.getSchoolType(schoolTypeCode)); break; case "14": // (See https://gitlab.com/ultreiaio/ird-observe/-/issues/2429) // SetSuccessStatus=2 @@ -255,11 +262,11 @@ public class ActivityReader extends DataReader<Activity> { entity.addObservedSystem(dataContext.getObservedSystem110()); // See https://gitlab.com/ultreiaio/ird-observe/-/issues/2610 entity.setSetCount(setCount); + // See https://gitlab.com/ultreiaio/ird-observe/-/issues/2485 + entity.setSchoolType(dataContext.getSchoolType(schoolTypeCode)); break; } entity.setVesselActivity(dataContext.getVesselActivity(vesselActivityCode)); - String schoolTypeCode = resultSet.getString(15); - entity.setSchoolType(dataContext.getSchoolType(schoolTypeCode)); Object fDonOrg = resultSet.getObject(16); if (fDonOrg != null) { boolean originalDataModified = Objects.equals(fDonOrg + "", "0"); View it on GitLab: https://gitlab.com/ultreiaio/ird-observe/-/commit/3b5c1e35a8bb8b9d36518dc13… -- View it on GitLab: https://gitlab.com/ultreiaio/ird-observe/-/commit/3b5c1e35a8bb8b9d36518dc13… You're receiving this email because of your account on gitlab.com.
1 0
0 0
[Git][ultreiaio/ird-observe][develop] Revue de migration de Activity.schoolType - Closes #2485
by Tony CHEMIT (@tchemit) 24 Jan '23

24 Jan '23
Tony CHEMIT pushed to branch develop at ultreiaio / ird-observe Commits: c1fb83d2 by Tony Chemit at 2023-01-24T08:01:11+01:00 Revue de migration de Activity.schoolType - Closes #2485 - - - - - 1 changed file: - core/persistence/avdth/src/main/java/fr/ird/observe/persistence/avdth/data/logbook/ActivityReader.java Changes: ===================================== core/persistence/avdth/src/main/java/fr/ird/observe/persistence/avdth/data/logbook/ActivityReader.java ===================================== @@ -228,6 +228,7 @@ public class ActivityReader extends DataReader<Activity> { String vesselActivityCode = resultSet.getString(13); Object setCountObject = resultSet.getObject(14); Integer setCount = setCountObject == null ? null : ((Number) setCountObject).intValue(); + String schoolTypeCode = resultSet.getString(15); switch (vesselActivityCode) { case "0": // SetSuccessStatus=0 + ReasonForNullSet=0 (See https://gitlab.com/ultreiaio/ird-observe/-/issues/2091) @@ -235,18 +236,24 @@ public class ActivityReader extends DataReader<Activity> { entity.setReasonForNullSet(dataContext.getReasonForNullSet0()); // See https://gitlab.com/ultreiaio/ird-observe/-/issues/2610 entity.setSetCount(setCount); + // See https://gitlab.com/ultreiaio/ird-observe/-/issues/2485 + entity.setSchoolType(dataContext.getSchoolType(schoolTypeCode)); break; case "1": // SetSuccessStatus=1 (See https://gitlab.com/ultreiaio/ird-observe/-/issues/2091) entity.setSetSuccessStatus(dataContext.getSetSuccessStatus1()); // See https://gitlab.com/ultreiaio/ird-observe/-/issues/2610 entity.setSetCount(setCount); + // See https://gitlab.com/ultreiaio/ird-observe/-/issues/2485 + entity.setSchoolType(dataContext.getSchoolType(schoolTypeCode)); break; case "2": // SetSuccessStatus=2 (See https://gitlab.com/ultreiaio/ird-observe/-/issues/2091) entity.setSetSuccessStatus(dataContext.getSetSuccessStatus2()); // See https://gitlab.com/ultreiaio/ird-observe/-/issues/2610 entity.setSetCount(setCount); + // See https://gitlab.com/ultreiaio/ird-observe/-/issues/2485 + entity.setSchoolType(dataContext.getSchoolType(schoolTypeCode)); break; case "14": // (See https://gitlab.com/ultreiaio/ird-observe/-/issues/2429) // SetSuccessStatus=2 @@ -255,11 +262,11 @@ public class ActivityReader extends DataReader<Activity> { entity.addObservedSystem(dataContext.getObservedSystem110()); // See https://gitlab.com/ultreiaio/ird-observe/-/issues/2610 entity.setSetCount(setCount); + // See https://gitlab.com/ultreiaio/ird-observe/-/issues/2485 + entity.setSchoolType(dataContext.getSchoolType(schoolTypeCode)); break; } entity.setVesselActivity(dataContext.getVesselActivity(vesselActivityCode)); - String schoolTypeCode = resultSet.getString(15); - entity.setSchoolType(dataContext.getSchoolType(schoolTypeCode)); Object fDonOrg = resultSet.getObject(16); if (fDonOrg != null) { boolean originalDataModified = Objects.equals(fDonOrg + "", "0"); View it on GitLab: https://gitlab.com/ultreiaio/ird-observe/-/commit/c1fb83d2883489e3ed424577d… -- View it on GitLab: https://gitlab.com/ultreiaio/ird-observe/-/commit/c1fb83d2883489e3ed424577d… You're receiving this email because of your account on gitlab.com.
1 0
0 0
[Git][ultreiaio/ird-observe][develop] 18 commits: AVDTH - Gestion du flag AVDTH F_DCP_ECO (fix migration doc) - See #2570
by Tony CHEMIT (@tchemit) 24 Jan '23

24 Jan '23
Tony CHEMIT pushed to branch develop at ultreiaio / ird-observe Commits: 284900bf by Tony Chemit at 2023-01-18T18:47:36+01:00 AVDTH - Gestion du flag AVDTH F_DCP_ECO (fix migration doc) - See #2570 - - - - - a4372ec7 by Tony Chemit at 2023-01-24T07:20:58+01:00 Pom - downgrade hsqldb to 2.7.0 (2.7.1 add some errors on uncanaccess...) - - - - - 54a62a32 by Tony Chemit at 2023-01-24T07:33:42+01:00 AVDTH - Review and clarify dcp migration - - - - - 31eaaa70 by Tony Chemit at 2023-01-24T07:33:47+01:00 AVDTH - Review TRANSMITTING_BUOY_OPERATION_CODE_MAPPING mapping - - - - - 64659cfc by Tony Chemit at 2023-01-24T07:33:52+01:00 AVDTH - Review OBSERVED_SYSTEM_CODE_MAPPING (for ACTIVITE.C_TYP_OBJET == 12) : * add Observed system 20 * add ObjectMaterial ANLOG - - - - - 2bf5489d by Tony Chemit at 2023-01-24T07:33:55+01:00 AVDTH - Improve some error messages - - - - - fc86857f by Tony Chemit at 2023-01-24T07:34:01+01:00 AVDTH - Add *addMessage* method to reader - - - - - 72850eb3 by Tony Chemit at 2023-01-24T07:34:07+01:00 AVDTH - Improve when to create (or not) Floating object - - - - - f04d24bd by Tony Chemit at 2023-01-24T07:34:12+01:00 AVDTH - Do not migrate ACTIVITE.V_POIDS_ESTIM_DCP. - - - - - 93101889 by Tony Chemit at 2023-01-24T07:34:16+01:00 AVDTH - Improve a comment - - - - - 754c5fd8 by Tony Chemit at 2023-01-24T07:34:19+01:00 AVDTH - Add more tests (using avdth multi-year databases) - - - - - 59b1a353 by Tony Chemit at 2023-01-24T07:34:22+01:00 AVDTH - Improve when to create (or not) Floating object - - - - - b42de963 by Tony Chemit at 2023-01-24T07:34:25+01:00 Avdth tests - fix when less than 3 db found - - - - - de473de4 by Tony Chemit at 2023-01-24T07:34:28+01:00 Avdth tests - add missing test_forcePrepare to pom properties for avdth all years tests - - - - - bb8eafaa by Tony Chemit at 2023-01-24T07:34:33+01:00 Avdth - add comment (for debug purpose) - - - - - a2c1e82f by Tony Chemit at 2023-01-24T07:34:36+01:00 Avdth - found more ACTIVITE.C_OPERA with DCP (for 9.0.x we still import them on bad ObServe activities, will be review in 9.1.x with new vesselActivities...) - - - - - b8911a77 by Tony Chemit at 2023-01-24T07:34:39+01:00 Avdth - add ACTIVITE.C_OPERA when creating ObjectMaterial to add it on log if something is wrong - - - - - b2746c74 by Tony Chemit at 2023-01-24T07:47:45+01:00 Merge branch &#39;feature/issue_2575&#39; into develop Revue de la migration des DCP depuis AVDTH - Closes #2575 - - - - - 12 changed files: - core/api/services/src/main/java/fr/ird/observe/services/service/data/ps/AvdthDataImportResult.java - core/persistence/avdth/src/main/java/fr/ird/observe/persistence/avdth/data/DataReader.java - core/persistence/avdth/src/main/java/fr/ird/observe/persistence/avdth/data/ImportReferentialContext.java - core/persistence/avdth/src/main/java/fr/ird/observe/persistence/avdth/data/logbook/FloatingObjectReader.java - core/persistence/avdth/src/main/java/fr/ird/observe/persistence/avdth/data/logbook/FloatingObjectWriter.java - core/persistence/test/pom.xml - + core/persistence/test/src/test/java/fr/ird/observe/persistence/avdth/data/AvdthAtlanticDataBuilderByYearLatestTest.java - core/persistence/test/src/test/java/fr/ird/observe/persistence/avdth/data/AvdthDataBuilderByYearTestSupport.java - core/persistence/test/src/test/java/fr/ird/observe/persistence/avdth/data/AvdthDataBuilderTestSupport.java - + core/persistence/test/src/test/java/fr/ird/observe/persistence/avdth/data/AvdthIndianDataBuilderByYearLatestTest.java - pom.xml - src/site/markdown/avdth/logbook.md Changes: ===================================== core/api/services/src/main/java/fr/ird/observe/services/service/data/ps/AvdthDataImportResult.java ===================================== @@ -87,6 +87,10 @@ public class AvdthDataImportResult implements ObserveDto { this.notReadResult = notReadResult; this.exportResult = exportResult; this.sqlResultPath = sqlResultPath; + // Used for debugging... +// if (!exportResult.isEmpty()) { +// System.out.println(exportResult.entrySet().stream().map(e -> e.getKey() + "=" + e.getValue()).collect(Collectors.joining("\n"))); +// } } public Path getMessageFile() { ===================================== core/persistence/avdth/src/main/java/fr/ird/observe/persistence/avdth/data/DataReader.java ===================================== @@ -97,5 +97,9 @@ public abstract class DataReader<E extends DataEntity> { protected final AvdthDataImportConfiguration getContext() { return context.getConfiguration(); } + + protected final void addMessage(String message) { + context.addMessage(message); + } } ===================================== core/persistence/avdth/src/main/java/fr/ird/observe/persistence/avdth/data/ImportReferentialContext.java ===================================== @@ -58,7 +58,6 @@ import fr.ird.observe.entities.referential.ps.logbook.SetSuccessStatus; import fr.ird.observe.entities.referential.ps.logbook.WellContentStatus; import fr.ird.observe.entities.referential.ps.logbook.WellSamplingConformity; import fr.ird.observe.entities.referential.ps.logbook.WellSamplingStatus; -import fr.ird.observe.persistence.avdth.data.logbook.FloatingObjectReader; import fr.ird.observe.persistence.avdth.data.logbook.WellReader; import fr.ird.observe.persistence.avdth.referential.AvdthReferentialImportResult; import fr.ird.observe.persistence.avdth.referential.SpeciesCache; @@ -80,10 +79,8 @@ import java.util.function.Consumer; * @author Tony Chemit - dev(a)tchemit.fr * @since 9.0.0 */ -@SuppressWarnings("SpellCheckingInspection") public class ImportReferentialContext { - private Map<String, WeightCategory> landingCategories; private Map<String, Destination> destinations; private Map<String, WeightCategory> wellCategories; @@ -127,9 +124,11 @@ public class ImportReferentialContext { private ObjectMaterial objectMaterialDFAD; private ObjectMaterial objectMaterialALOG; private ObjectMaterial objectMaterialAFAD; + private ObjectMaterial objectMaterialBiodegradable; private ObjectMaterial objectMaterialAlone; private ObjectMaterial objectMaterialLOG; private ObjectMaterial objectMaterialFALOG; + private ObjectMaterial objectMaterialANLOG; private ObjectMaterial objectMaterialHALOG; private ObjectMaterial objectMaterialVNLOG; private ObservedSystem observedSystem0; @@ -226,8 +225,10 @@ public class ImportReferentialContext { objectMaterialLOG = objectMaterial.get("fr.ird.referential.ps.common.ObjectMaterial#0#0.50"); objectMaterialFALOG = objectMaterial.get("fr.ird.referential.ps.common.ObjectMaterial#0#0.63"); objectMaterialHALOG = objectMaterial.get("fr.ird.referential.ps.common.ObjectMaterial#0#0.67"); + objectMaterialANLOG = objectMaterial.get("fr.ird.referential.ps.common.ObjectMaterial#0#0.56"); objectMaterialVNLOG = objectMaterial.get("fr.ird.referential.ps.common.ObjectMaterial#0#0.52"); objectMaterialAlone = objectMaterial.get("fr.ird.referential.ps.common.ObjectMaterial#1561561977652#0.5876332198776647"); + objectMaterialBiodegradable = objectMaterial.get("fr.ird.referential.ps.common.ObjectMaterial#0#0.69"); // catch catchCategories = new TreeMap<>(); @@ -333,10 +334,34 @@ public class ImportReferentialContext { return objectMaterialAFAD; } + public ObjectMaterial getObjectMaterialLOG() { + return objectMaterialLOG; + } + + public ObjectMaterial getObjectMaterialFALOG() { + return objectMaterialFALOG; + } + + public ObjectMaterial getObjectMaterialHALOG() { + return objectMaterialHALOG; + } + + public ObjectMaterial getObjectMaterialVNLOG() { + return objectMaterialVNLOG; + } + + public ObjectMaterial getObjectMaterialBiodegradable() { + return objectMaterialBiodegradable; + } + public ObjectMaterial getObjectMaterialAlone() { return objectMaterialAlone; } + public ObjectMaterial getObjectMaterialANLOG() { + return objectMaterialANLOG; + } + public SampleType getLocalmarketSampleType() { return localmarketSampleType; } @@ -529,86 +554,37 @@ public class ImportReferentialContext { entity.addObservedSystem(getObservedSystem("102")); } - public ObjectOperation getObjectOperationByVesselActivityCode(String vesselActivityCode) { - String objectOperationCode = FloatingObjectReader.DCP_OBJECT_OPERATION_CODE_MAPPING.get(vesselActivityCode); - if (objectOperationCode == null) { - return null; - } + public ObjectOperation getObjectOperation(String objectOperationCode) { return objectOperation.get(objectOperationCode); } - public TransmittingBuoyOperation getTransmittingBuoyOperationByVesselActivityCode(String vesselActivityCode) { - String transmittingBuoyTypeCode = FloatingObjectReader.TRANSMITTING_BUOY_OPERATION_CODE_MAPPING.get(vesselActivityCode); + public TransmittingBuoyOperation getTransmittingBuoyOperation(String transmittingBuoyTypeCode) { if (transmittingBuoyTypeCode == null) { return null; } return transmittingBuoyOperation.get(transmittingBuoyTypeCode); } - public TransmittingBuoyType getTransmittingBuoyTypeByBuoyTypeCode(String buoyTypeCode) { - String transmittingBuoyTypeCode = FloatingObjectReader.TRANSMITTING_BUOY_TYPE_CODE_MAPPING.get(buoyTypeCode); - if (transmittingBuoyTypeCode == null) { - transmittingBuoyTypeCode = "999"; - } - TransmittingBuoyType transmittingBuoyType = this.transmittingBuoyType.get(transmittingBuoyTypeCode); - return Objects.requireNonNull(transmittingBuoyType, String.format("Can't find transmittingBuoyType with code: %s", buoyTypeCode)); + public TransmittingBuoyType getTransmittingBuoyType(String transmittingBuoyTypeCode) { + TransmittingBuoyType result = transmittingBuoyType.get(transmittingBuoyTypeCode); + return Objects.requireNonNull(result, String.format("Can't find TransmittingBuoyType with code: %s", transmittingBuoyTypeCode)); } - public TransmittingBuoyOwnership getTransmittingBuoyOwnership(String buoyOwnershipCode) { - if (buoyOwnershipCode == null) { - return null; - } - String transmittingBuoyOwnershipCode = FloatingObjectReader.TRANSMITTING_BUOY_OWNERSHIP_CODE_MAPPING.get(buoyOwnershipCode); + public TransmittingBuoyOwnership getTransmittingBuoyOwnership(String transmittingBuoyOwnershipCode) { if (transmittingBuoyOwnershipCode == null) { return null; } return transmittingBuoyOwnership.get(transmittingBuoyOwnershipCode); } - public ObservedSystem getObservedSystem(String objectTypeCode, String buoyTypeCode) { - if (objectTypeCode.equals("999")) { - if (buoyTypeCode.equals("999")) { - return getObservedSystem0(); - } - return getObservedSystem20(); - } - String observedSystemCode = FloatingObjectReader.OBSERVED_SYSTEM_CODE_MAPPING.get(objectTypeCode); - return getObservedSystem(observedSystemCode, false); - } - - /** - * To get objectMaterial id associated by a DCP from avdth ACTIVITE.C_TYP_OBJET code. - * - * @param objectTypeCode ACTIVITE.C_TYP_OBJET - * @return material or null - */ - public ObjectMaterial getObjectMaterialByObjectTypeCode(String objectTypeCode) { - switch (objectTypeCode) { - case "1": - return objectMaterialAFAD; - case "2": - return objectMaterialDFAD; - case "3": - return objectMaterialLOG; - case "9": - return objectMaterialFOB; - case "10": - return objectMaterialFALOG; - case "11": - return objectMaterialHALOG; - case "13": - return objectMaterialVNLOG; - } - return null; - } - public WeightCategory getWellWeightCategory(String weightCategoryCode) { String categoryCode = "W-" + weightCategoryCode; WeightCategory weightCategory = wellCategories.get(categoryCode); - return Objects.requireNonNull(weightCategory, "Can't find weight category for code: " + categoryCode); + return Objects.requireNonNull(weightCategory, String.format("Can't find weight category for code: %s", categoryCode)); } public VesselActivity getVesselActivity6() { return vesselActivity6; } + } ===================================== core/persistence/avdth/src/main/java/fr/ird/observe/persistence/avdth/data/logbook/FloatingObjectReader.java ===================================== @@ -23,6 +23,7 @@ package fr.ird.observe.persistence.avdth.data.logbook; */ import com.google.common.collect.ImmutableMap; +import fr.ird.observe.entities.data.ps.logbook.Activity; import fr.ird.observe.entities.data.ps.logbook.FloatingObject; import fr.ird.observe.entities.data.ps.logbook.FloatingObjectPart; import fr.ird.observe.entities.data.ps.logbook.TransmittingBuoy; @@ -33,16 +34,21 @@ import fr.ird.observe.entities.referential.ps.common.TransmittingBuoyOperation; import fr.ird.observe.entities.referential.ps.common.TransmittingBuoyOwnership; import fr.ird.observe.entities.referential.ps.common.TransmittingBuoyType; import fr.ird.observe.entities.referential.ps.common.VesselActivity; +import fr.ird.observe.persistence.avdth.Query; import fr.ird.observe.persistence.avdth.data.DataReader; import fr.ird.observe.persistence.avdth.data.ImportDataContext; import fr.ird.observe.persistence.avdth.data.ImportEngine; import org.apache.commons.lang3.mutable.MutableInt; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; import java.sql.ResultSet; import java.sql.SQLException; +import java.util.List; import java.util.Map; import java.util.Objects; import java.util.Set; +import java.util.TreeMap; import java.util.TreeSet; /** @@ -53,6 +59,7 @@ import java.util.TreeSet; */ @SuppressWarnings("SpellCheckingInspection") public class FloatingObjectReader extends DataReader<FloatingObject> { + private static final Logger log = LogManager.getLogger(FloatingObjectReader.class); /** * To get FloatingObject.objectOperation code from avdth ACTIVITE.C_OPERA. */ @@ -66,7 +73,10 @@ public class FloatingObjectReader extends DataReader<FloatingObject> { .put("14", "99") .put("7", "99") .put("8", "99") + .put("9", "99") + .put("12", "99") .put("13", "99") + .put("15", "99") .put("5", "1") .put("6", "4") @@ -87,32 +97,23 @@ public class FloatingObjectReader extends DataReader<FloatingObject> { .build(); /** * To get TransmittingBuoy.TransmittingBuoyOperation code from avdth ACTIVITE.C_OPERA code. + * <p> + * If for a given {@code ACTIVITE.C_OPERA}, no value is found here, this means that there is no buoy to create. */ public static final Map<String, String> TRANSMITTING_BUOY_OPERATION_CODE_MAPPING = ImmutableMap.<String, String>builder() - .put("0", "1") // Visite | 99 inconnu | W_ARRIVING | - .put("1", "1") // Visite | 99 inconnu | W_ARRIVING | - .put("2", "1") // Visite | 99 inconnu | W_ARRIVING | - .put("3", "1") // Visite | 99 inconnu | W_ARRIVING | - .put("4", "1") // Visite | 99 inconnu | W_ARRIVING | - .put("10", "1")// Visite | 99 inconnu | W_ARRIVING | - .put("14", "1")// Visite | 99 inconnu | W_ARRIVING | - .put("7", "1") // Visite | 99 inconnu | W_ARRIVING | - .put("8", "1") // Visite | 99 inconnu | W_ARRIVING | - .put("13", "1")// Visite | 99 inconnu | W_ARRIVING | - - .put("5", "3") // Mise à l'eau | null ou 3 Mise à l'eau | W_LEAVING | Selon présence infos balise dans AVDTH - .put("6", "2") // Retrait | null ou 2 Récupération | W_ARRIVING | Selon présence infos balise dans AVDTH - .put("23", "3") // Mise à l'eau | 3 Mise à l'eau | W_LEAVING | - .put("24", "2") // Retrait | 2 Récupération | W_ARRIVING | - .put("25", "3") // Visite | 3 Mise à l'eau | W_ARRIVING | - .put("26", "2") // Visite | 2 Récupération | W_ARRIVING | - .put("29", "1") // Visite | 1 Visite | W_ARRIVING | - .put("32", "3") // Modif ou renfor | 3 Mise à l'eau | W_ARR + W_LEA | - .put("33", "4") // Perte | 4 Perte signal | W_ARRIVING | codes 11 et 4 à créer ; matériaux sur W_ARRIVING par convention, comme pour Visite - .put("34", "2") // Retrait | 2 Récupération | W_ARRIVING | rattacher cette balise à un FOB de type '2-2-4-4 Bouée émettrice seule' + .put("5", "3") // Mise à l'eau | null ou 3 Mise à l'eau + .put("6", "2") // Retrait | null ou 2 Récupération + .put("23", "3") // Mise à l'eau | 3 Mise à l'eau + .put("24", "2") // Retrait | 2 Récupération + .put("25", "3") // Visite | 3 Mise à l'eau + .put("26", "2") // Visite | 2 Récupération + .put("29", "1") // Visite | 1 Visite + .put("32", "3") // Modif ou renfor | 3 Mise à l'eau + .put("33", "4") // Perte | 4 Perte signal + .put("34", "2") // Retrait | 2 Récupération // See https://gitlab.com/ultreiaio/ird-observe/-/issues/2568 - .put("40", "4") // Perte | 4 Perte signal | W_ARRIVING | rattacher cette balise à un FOB de type '1-1 DFAD' - .put("41", "5") // Arret commandé de la transmission | 5 Fin d'utilisation | W_ARRIVING | rattacher cette balise à un FOB de type '1-1 DFAD' + .put("40", "4") // Perte | 4 Perte signal + .put("41", "5") // Arret commandé de la transmission | 5 Fin d'utilisation .build(); /** * Is floatingObjectPart on leaving from avdth ACTIVITE.C_OPERA. @@ -127,8 +128,11 @@ public class FloatingObjectReader extends DataReader<FloatingObject> { "2", "3", "4", + "9", "10", + "12", "14", + "15", "7", "8", "13", @@ -214,6 +218,7 @@ public class FloatingObjectReader extends DataReader<FloatingObject> { .put("9", "20") .put("10", "20") .put("11", "20") + .put("12", "20") .put("13", "20") .build(); public static final Set<String> OBSERVED_SYTEM_CODES_WITH_DCP = Set.of("20", "21", "22", "23", "24", "25"); @@ -225,6 +230,8 @@ public class FloatingObjectReader extends DataReader<FloatingObject> { private final MutableInt floatingObjectPartCount = new MutableInt(); private final MutableInt transmittingBuoyCount = new MutableInt(); + private final Map<String, MutableInt> badVesselActivityCodes = new TreeMap<>(); + public FloatingObjectReader(ImportEngine context) { super(context); } @@ -232,124 +239,202 @@ public class FloatingObjectReader extends DataReader<FloatingObject> { @Override public FloatingObject read(ImportDataContext dataContext, ResultSet resultSet) throws SQLException { - String vesselActivityCode = resultSet.getString(13); - VesselActivity vesselActivity = dataContext.getVesselActivity(vesselActivityCode); + // Get the current activity where to add the optional floating object + Activity activity = dataContext.getActivity(); + + // Get his vessel activity + VesselActivity vesselActivity = activity.getVesselActivity(); + + // Avdth observed system codes associated to the current activity (need them to compute ex nihilo floating object) Set<String> observedSystemCodes = dataContext.getObservedSystemCodes(); - if (!Objects.equals(vesselActivity.getCode(), "13")) { - boolean noDcpInObservedSystems = observedSystemCodes.stream().noneMatch(OBSERVED_SYTEM_CODES_WITH_DCP::contains); - if (noDcpInObservedSystems) { - reset(dataContext); - return null; - } - } - FloatingObject entity = newEntity(FloatingObject.SPI); + // We need to get the original vessel activity code from AVDTH (to compute some mapping) + String vesselActivityCode = resultSet.getString(13); - ObjectOperation objectOperation = dataContext.getObjectOperationByVesselActivityCode(vesselActivityCode); - entity.setObjectOperation(Objects.requireNonNull(objectOperation, String.format("Can't find objectOperation with code: %s", vesselActivityCode))); - boolean addCoordinate = "11".equals(objectOperation.getCode()); - boolean whenArriving = WHEN_ARRIVING_CODE.contains(vesselActivityCode); - boolean whenLeaving = WHEN_LEAVING_CODE.contains(vesselActivityCode); + // Is the floating object can be created by the observed systems? + boolean floatingObjectCreatedByObservedSystem = observedSystemCodes.stream().anyMatch(OBSERVED_SYTEM_CODES_WITH_DCP::contains); + + // Is the current vessel activity accept to create Floating object? + boolean vesselActivityAllowFad = vesselActivity.isAllowFad(); + + // Get object type code String objectTypeCode = resultSet.getString(28); if (objectTypeCode == null) { - //FIXME should fail ? + // If null, then consider it as not found in avdth objectTypeCode = "999"; } - //FIXME 29 - F_DCP_ECO -// int dcpEcoCode = resultSet.getInt(29); - String buoyOwnershipCode = resultSet.getString(30); + + // Is floating object exists in avdth? + boolean floatingObjectExistsInAvdth = !objectTypeCode.equals("999"); + + if (!floatingObjectExistsInAvdth && !floatingObjectCreatedByObservedSystem) { + // If Floating object does not exist in avdth, neither need to be created by observed system, + // Do not create the floating object (no log is required here) + reset(dataContext); + return null; + } + + if (!vesselActivityAllowFad) { + // If ObServe veseel activity does not support Floating object, reject import + List<Object> activityPrimaryKey = ActivityReader.getActivityPk(resultSet); + String activityPk = Query.primaryKeyString(activityPrimaryKey); + String message = String.format("On activity %s (ACTIVITE.C_OPERA %s), can not add Floating object since ObServe Vessel activity with code : %s does not allow FAD (floatingObjectExistsInAvdth? %b, floatingObjectCreatedByObservedSystem? %b).", + activityPk, + vesselActivityCode, + vesselActivity.getCode(), + floatingObjectExistsInAvdth, + floatingObjectCreatedByObservedSystem); + log.warn(message); + badVesselActivityCodes.computeIfAbsent(vesselActivityCode, MutableInt::new).increment(); + +// // do not create the floating object +// reset(dataContext); +// return null; + } + + FloatingObject entity = newEntity(FloatingObject.SPI); + + // Get floating object operation + ObjectOperation objectOperation = getObjectOperationByVesselActivityCode(dataContext, vesselActivityCode); + entity.setObjectOperation(Objects.requireNonNull(objectOperation, String.format("No mapping for FloatingObject.objectOperation from ACTIVITE.C_OPERA: %s", vesselActivityCode))); + String buoyTypeCode = resultSet.getString(31); if (buoyTypeCode == null) { - //FIXME should fail? + // If null, then consider it as not found in avdth buoyTypeCode = "999"; } - String buoyId = resultSet.getString(32); - String dcpWeight = resultSet.getString(33); - if (dcpWeight != null) { - //FIXME Check if ok - entity.setComment(String.format("ACTIVITE.POIDS_ESTIM_DCP value from avdth: %s", dcpWeight)); - } - ObservedSystem observedSystem = dataContext.getObservedSystem(objectTypeCode, buoyTypeCode); + // IS buoy exists in avdth? + boolean buoyExistsInAvdth = !"999".equals(buoyTypeCode); + + ObservedSystem observedSystem = getObservedSystem(dataContext, objectTypeCode, floatingObjectExistsInAvdth, buoyExistsInAvdth); if (observedSystem != null) { - dataContext.getActivity().addObservedSystem(observedSystem); + activity.addObservedSystem(observedSystem); } - // avdth data has a buoy - boolean willCreateBuoy = !"999".equals(buoyTypeCode); - // to add default material if there is a declared buoy and no object material created by ObservedSystem + + boolean addBuoyFromObservedSystem = addFloatingObjectMaterials(dataContext, + resultSet, + vesselActivityCode, + objectTypeCode, + observedSystemCodes, + entity, + floatingObjectExistsInAvdth, + buoyExistsInAvdth); + + TransmittingBuoy transmittingBuoy = null; + if (buoyExistsInAvdth) { + transmittingBuoy = getBuoyFromAvdthActivity(dataContext, resultSet, buoyTypeCode, vesselActivityCode, objectOperation, activity); + } else if (addBuoyFromObservedSystem) { + transmittingBuoy = getBuoyFromObservedSystem(dataContext); + } + if (transmittingBuoy != null) { + // only add buoy if buoy type known + entity.addTransmittingBuoy(transmittingBuoy); + } + reset(dataContext); + return entity; + } + + private boolean addFloatingObjectMaterials(ImportDataContext dataContext, + ResultSet resultSet, + String vesselActivityCode, + String objectTypeCode, + Set<String> observedSystemCodes, + FloatingObject entity, + boolean floatingObjectExistsInAvdth, + boolean buoyExistsInAvdth) throws SQLException { + boolean whenArriving = WHEN_ARRIVING_CODE.contains(vesselActivityCode); + boolean whenLeaving = WHEN_LEAVING_CODE.contains(vesselActivityCode); + + // to add default material if there is a declared buoy (but no floating object declared) + // and no object material created by ObservedSystem, we will then add a standalone buoy boolean addDefaultObjectMaterialOnUnknownObjectType = true; // should create a buoy from ObservedSystem boolean addBuoyFromObservedSystem = false; Set<String> objectMaterialUsed = new TreeSet<>(); + + if (floatingObjectExistsInAvdth) { + Object dcpEcoCode = resultSet.getObject(29); + if (dcpEcoCode != null && Objects.equals(2, dcpEcoCode)) { + // add material 4-1 (Biodegradable materials) + addObjectMaterial(vesselActivityCode, objectMaterialUsed, dataContext.getObjectMaterialBiodegradable(), whenArriving, whenLeaving, entity); + } + } + if (observedSystemCodes.contains("20")) { // add ObjectMaterial FOB - addObjectMaterial(objectMaterialUsed, dataContext.getObjectMaterialFOB(), whenArriving, whenLeaving, entity); + addObjectMaterial(vesselActivityCode, objectMaterialUsed, dataContext.getObjectMaterialFOB(), whenArriving, whenLeaving, entity); addDefaultObjectMaterialOnUnknownObjectType = false; } else if (observedSystemCodes.contains("21")) { // add ObjectMaterial 2-1 NLOG - addObjectMaterial(objectMaterialUsed, dataContext.getObjectMaterialNLOG(), whenArriving, whenLeaving, entity); + addObjectMaterial(vesselActivityCode, objectMaterialUsed, dataContext.getObjectMaterialNLOG(), whenArriving, whenLeaving, entity); addDefaultObjectMaterialOnUnknownObjectType = false; } else if (observedSystemCodes.contains("22")) { // add ObjectMaterial 2-1 NLOG + 1-1 DFAD - addObjectMaterial(objectMaterialUsed, dataContext.getObjectMaterialNLOG(), whenArriving, whenLeaving, entity); - addObjectMaterial(objectMaterialUsed, dataContext.getObjectMaterialDFAD(), whenArriving, whenLeaving, entity); + addObjectMaterial(vesselActivityCode, objectMaterialUsed, dataContext.getObjectMaterialNLOG(), whenArriving, whenLeaving, entity); + addObjectMaterial(vesselActivityCode, objectMaterialUsed, dataContext.getObjectMaterialDFAD(), whenArriving, whenLeaving, entity); addDefaultObjectMaterialOnUnknownObjectType = false; addBuoyFromObservedSystem = true; } else if (observedSystemCodes.contains("23")) { // add ObjectMaterial 2-2 ALOG - addObjectMaterial(objectMaterialUsed, dataContext.getObjectMaterialALOG(), whenArriving, whenLeaving, entity); + addObjectMaterial(vesselActivityCode, objectMaterialUsed, dataContext.getObjectMaterialALOG(), whenArriving, whenLeaving, entity); addDefaultObjectMaterialOnUnknownObjectType = false; } else if (observedSystemCodes.contains("24")) { // add ObjectMaterial 2-2 ALOG + 1-1 DFAD - addObjectMaterial(objectMaterialUsed, dataContext.getObjectMaterialALOG(), whenArriving, whenLeaving, entity); - addObjectMaterial(objectMaterialUsed, dataContext.getObjectMaterialDFAD(), whenArriving, whenLeaving, entity); + addObjectMaterial(vesselActivityCode, objectMaterialUsed, dataContext.getObjectMaterialALOG(), whenArriving, whenLeaving, entity); + addObjectMaterial(vesselActivityCode, objectMaterialUsed, dataContext.getObjectMaterialDFAD(), whenArriving, whenLeaving, entity); addDefaultObjectMaterialOnUnknownObjectType = false; addBuoyFromObservedSystem = true; } else if (observedSystemCodes.contains("25")) { // add ObjectMaterial 1-2 AFAD - addObjectMaterial(objectMaterialUsed, dataContext.getObjectMaterialAFAD(), whenArriving, whenLeaving, entity); + addObjectMaterial(vesselActivityCode, objectMaterialUsed, dataContext.getObjectMaterialAFAD(), whenArriving, whenLeaving, entity); addDefaultObjectMaterialOnUnknownObjectType = false; } - if ("999".equals(objectTypeCode)) { - if (willCreateBuoy && addDefaultObjectMaterialOnUnknownObjectType) { - // add ObjectMaterial 2-2-4-4 - addObjectMaterial(objectMaterialUsed, dataContext.getObjectMaterialAlone(), whenArriving, whenLeaving, entity); + if (floatingObjectExistsInAvdth) { + // simple mapping + ObjectMaterial objectMaterial = getObjectMaterialByObjectTypeCode(dataContext, objectTypeCode); + if (objectMaterial != null) { + addObjectMaterial(vesselActivityCode, objectMaterialUsed, objectMaterial, whenArriving, whenLeaving, entity); } } else { - // simple mapping - ObjectMaterial objectMaterial1 = dataContext.getObjectMaterialByObjectTypeCode(objectTypeCode); - if (objectMaterial1 != null) { - addObjectMaterial(objectMaterialUsed, objectMaterial1, whenArriving, whenLeaving, entity); + if (buoyExistsInAvdth && addDefaultObjectMaterialOnUnknownObjectType) { + // add ObjectMaterial 2-2-4-4 + addObjectMaterial(vesselActivityCode, objectMaterialUsed, dataContext.getObjectMaterialAlone(), whenArriving, whenLeaving, entity); } } - if (willCreateBuoy) { - // only add buoy if buoy type known - TransmittingBuoyType transmittingBuoyType = dataContext.getTransmittingBuoyTypeByBuoyTypeCode(buoyTypeCode); - TransmittingBuoyOperation transmittingBuoyOperation = dataContext.getTransmittingBuoyOperationByVesselActivityCode(vesselActivityCode); - if (transmittingBuoyOperation != null) { - TransmittingBuoy transmittingBuoy = newEntity(TransmittingBuoy.SPI, transmittingBuoyCount); - transmittingBuoy.setTransmittingBuoyOperation(transmittingBuoyOperation); - transmittingBuoy.setTransmittingBuoyType(transmittingBuoyType); - transmittingBuoy.setCode(buoyId); - if (addCoordinate) { - transmittingBuoy.setLatitude(dataContext.getActivity().getLatitude()); - transmittingBuoy.setLongitude(dataContext.getActivity().getLongitude()); - } - TransmittingBuoyOwnership transmittingBuoyOwnership = dataContext.getTransmittingBuoyOwnership(buoyOwnershipCode); - transmittingBuoy.setTransmittingBuoyOwnership(transmittingBuoyOwnership); - entity.addTransmittingBuoy(transmittingBuoy); - } - } else if (addBuoyFromObservedSystem) { - // only add buoy if observed system ask for it (22 and 24) - TransmittingBuoyType transmittingBuoyType = dataContext.getTransmittingBuoyType98(); - TransmittingBuoyOperation transmittingBuoyOperation = dataContext.getTransmittingBuoyOperation1(); + return addBuoyFromObservedSystem; + } + + private TransmittingBuoy getBuoyFromAvdthActivity(ImportDataContext dataContext, ResultSet resultSet, String buoyTypeCode, String vesselActivityCode, ObjectOperation objectOperation, Activity activity) throws SQLException { + boolean addCoordinate = "11".equals(objectOperation.getCode()); + TransmittingBuoyType transmittingBuoyType = getTransmittingBuoyTypeByBuoyTypeCode(dataContext, buoyTypeCode); + TransmittingBuoyOperation transmittingBuoyOperation = getTransmittingBuoyOperationByVesselActivityCode(dataContext, vesselActivityCode); + if (transmittingBuoyOperation != null) { + String buoyOwnershipCode = resultSet.getString(30); + String buoyId = resultSet.getString(32); TransmittingBuoy transmittingBuoy = newEntity(TransmittingBuoy.SPI, transmittingBuoyCount); transmittingBuoy.setTransmittingBuoyOperation(transmittingBuoyOperation); transmittingBuoy.setTransmittingBuoyType(transmittingBuoyType); - transmittingBuoy.setCode(null); - entity.addTransmittingBuoy(transmittingBuoy); + transmittingBuoy.setCode(buoyId); + if (addCoordinate) { + transmittingBuoy.setLatitude(activity.getLatitude()); + transmittingBuoy.setLongitude(activity.getLongitude()); + } + TransmittingBuoyOwnership transmittingBuoyOwnership = getTransmittingBuoyOwnership(dataContext, buoyOwnershipCode); + transmittingBuoy.setTransmittingBuoyOwnership(transmittingBuoyOwnership); + return transmittingBuoy; } - reset(dataContext); - return entity; + return null; + } + + private TransmittingBuoy getBuoyFromObservedSystem(ImportDataContext dataContext) { + // only add buoy if observed system ask for it (22 and 24) + TransmittingBuoyType transmittingBuoyType = dataContext.getTransmittingBuoyType98(); + TransmittingBuoyOperation transmittingBuoyOperation = dataContext.getTransmittingBuoyOperation1(); + TransmittingBuoy transmittingBuoy = newEntity(TransmittingBuoy.SPI, transmittingBuoyCount); + transmittingBuoy.setTransmittingBuoyOperation(transmittingBuoyOperation); + transmittingBuoy.setTransmittingBuoyType(transmittingBuoyType); + transmittingBuoy.setCode(null); + return transmittingBuoy; } private void reset(ImportDataContext dataContext) { @@ -357,9 +442,9 @@ public class FloatingObjectReader extends DataReader<FloatingObject> { dataContext.setObservedSystemCodes(null); } - private void addObjectMaterial(Set<String> objectMaterialUsed, ObjectMaterial objectMaterial, boolean whenArriving, boolean whenLeaving, FloatingObject floatingObject) { + private void addObjectMaterial(String vesselActivityCode, Set<String> objectMaterialUsed, ObjectMaterial objectMaterial, boolean whenArriving, boolean whenLeaving, FloatingObject floatingObject) { if (!whenArriving && !whenLeaving) { - throw new IllegalStateException("Can't have not whenArriving and not whenLeaving "); + throw new IllegalStateException(String.format("Can't have not whenArriving and not whenLeaving from ACTIVITE.C_OPERA: %s", vesselActivityCode)); } if (!objectMaterialUsed.add(objectMaterial.getTopiaId())) { // already used @@ -380,4 +465,80 @@ public class FloatingObjectReader extends DataReader<FloatingObject> { return floatingObjectPartCount.intValue(); } + public Map<String, MutableInt> getBadVesselActivityCodes() { + return badVesselActivityCodes; + } + + private ObjectOperation getObjectOperationByVesselActivityCode(ImportDataContext dataContext, String vesselActivityCode) { + String objectOperationCode = DCP_OBJECT_OPERATION_CODE_MAPPING.get(vesselActivityCode); + return dataContext.getObjectOperation(Objects.requireNonNull(objectOperationCode, String.format("Can't find objectOperation with ACTIVITE.C_OPERA: %s", vesselActivityCode))); + } + + private ObservedSystem getObservedSystem(ImportDataContext dataContext, String objectTypeCode, boolean floatingObjectExistsInAvdth,boolean buoyExistsInAvdth) { + if (!floatingObjectExistsInAvdth) { + if (!buoyExistsInAvdth) { + return dataContext.getObservedSystem0(); + } + return dataContext.getObservedSystem20(); + } + String observedSystemCode = OBSERVED_SYSTEM_CODE_MAPPING.get(objectTypeCode); + return dataContext.getObservedSystem(observedSystemCode, false); + } + + private TransmittingBuoyOwnership getTransmittingBuoyOwnership(ImportDataContext dataContext, String buoyOwnershipCode) { + if (buoyOwnershipCode == null) { + return null; + } + String transmittingBuoyOwnershipCode = TRANSMITTING_BUOY_OWNERSHIP_CODE_MAPPING.get(buoyOwnershipCode); + if (transmittingBuoyOwnershipCode == null) { + return null; + } + return dataContext.getTransmittingBuoyOwnership(transmittingBuoyOwnershipCode); + } + + private TransmittingBuoyOperation getTransmittingBuoyOperationByVesselActivityCode(ImportDataContext dataContext, String vesselActivityCode) { + String transmittingBuoyTypeCode = TRANSMITTING_BUOY_OPERATION_CODE_MAPPING.get(vesselActivityCode); + if (transmittingBuoyTypeCode == null) { + return null; + } + return dataContext.getTransmittingBuoyOperation(transmittingBuoyTypeCode); + } + + + private TransmittingBuoyType getTransmittingBuoyTypeByBuoyTypeCode(ImportDataContext dataContext, String buoyTypeCode) { + String transmittingBuoyTypeCode = TRANSMITTING_BUOY_TYPE_CODE_MAPPING.get(buoyTypeCode); + if (transmittingBuoyTypeCode == null) { + transmittingBuoyTypeCode = "999"; + } + TransmittingBuoyType transmittingBuoyType = dataContext.getTransmittingBuoyType(transmittingBuoyTypeCode); + return Objects.requireNonNull(transmittingBuoyType, String.format("Can't find transmittingBuoyType with code: %s", buoyTypeCode)); + } + + /** + * To get objectMaterial associated by a DCP (or null) from avdth ACTIVITE.C_TYP_OBJET code. + * + * @param objectTypeCode ACTIVITE.C_TYP_OBJET + * @return material or null + */ + private ObjectMaterial getObjectMaterialByObjectTypeCode(ImportDataContext dataContext, String objectTypeCode) { + switch (objectTypeCode) { + case "1": + return dataContext.getObjectMaterialAFAD(); + case "2": + return dataContext.getObjectMaterialDFAD(); + case "3": + return dataContext.getObjectMaterialLOG(); + case "9": + return dataContext.getObjectMaterialFOB(); + case "10": + return dataContext.getObjectMaterialFALOG(); + case "11": + return dataContext.getObjectMaterialHALOG(); + case "12": + return dataContext.getObjectMaterialANLOG(); + case "13": + return dataContext.getObjectMaterialVNLOG(); + } + return null; + } } ===================================== core/persistence/avdth/src/main/java/fr/ird/observe/persistence/avdth/data/logbook/FloatingObjectWriter.java ===================================== @@ -192,6 +192,9 @@ public class FloatingObjectWriter extends DataWriter<FloatingObject, FloatingObj super.toResult(resultBuilder); toResult(resultBuilder, FloatingObjectPart.class, getReader().getFloatingObjectPartCount()); toResult(resultBuilder, TransmittingBuoy.class, getReader().getTransmittingBuoyCount()); +// getReader().getBadVesselActivityCodes().forEach((k,v)-> { +// resultBuilder.put(String.format("Bad Vessel activity used to build FAD from ACTIVITY.C_OPERA %s used but ObServe not allow fad.", k), v.getValue()); +// }); } } ===================================== core/persistence/test/pom.xml ===================================== @@ -30,6 +30,7 @@ <description>ObServe Core Persistence Test module</description> <properties> <test_forceImport>false</test_forceImport> + <test_forcePrepare>false</test_forcePrepare> <test_allYears>false</test_allYears> </properties> <dependencies> @@ -109,6 +110,7 @@ <systemPropertyVariables> <test_forceImport>${test_forceImport}</test_forceImport> <test_allYears>${test_allYears}</test_allYears> + <test_forcePrepare>${test_forcePrepare}</test_forcePrepare> </systemPropertyVariables> </configuration> </plugin> ===================================== core/persistence/test/src/test/java/fr/ird/observe/persistence/avdth/data/AvdthAtlanticDataBuilderByYearLatestTest.java ===================================== @@ -0,0 +1,54 @@ +package fr.ird.observe.persistence.avdth.data; + +/*- + * #%L + * ObServe Core :: Persistence :: Test + * %% + * Copyright (C) 2008 - 2023 IRD, Ultreia.io + * %% + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public + * License along with this program. If not, see + * <http://www.gnu.org/licenses/gpl-3.0.html>. + * #L% + */ + +import fr.ird.observe.dto.ProtectedIdsCommon; +import org.junit.runners.Parameterized; + +import java.nio.file.Path; + +/** + * Created on 19/01/2023. + * + * @author Tony Chemit - dev(a)tchemit.fr + * @since 9.0.24 + */ +public class AvdthAtlanticDataBuilderByYearLatestTest extends AvdthDataBuilderByYearTestSupport { + + public static final Path ROOT_PATH = getPath("OA").resolve("latest"); + + @Parameterized.Parameters(name = "{index}: {0}") + public static Iterable<?> data() { + return getDatabases(ROOT_PATH); + } + + @Override + protected Path getRootPath() { + return ROOT_PATH; + } + + @Override + protected String getOceanId() { + return ProtectedIdsCommon.COMMON_OCEAN_ATLANTIC; + } +} ===================================== core/persistence/test/src/test/java/fr/ird/observe/persistence/avdth/data/AvdthDataBuilderByYearTestSupport.java ===================================== @@ -37,7 +37,7 @@ public abstract class AvdthDataBuilderByYearTestSupport extends AvdthDataBuilder public static List<String> getDatabases(Path path) { List<String> databases = allDatabases(path); - return AvdthFixtures.IMPORT_ALL_YEARS ? databases : databases.subList(0, 3); + return AvdthFixtures.IMPORT_ALL_YEARS || databases.size() < 3 ? databases : databases.subList(0, 3); } @Override ===================================== core/persistence/test/src/test/java/fr/ird/observe/persistence/avdth/data/AvdthDataBuilderTestSupport.java ===================================== @@ -169,7 +169,15 @@ public abstract class AvdthDataBuilderTestSupport extends PersistenceTestSupport } protected void checkNotValidResult(AvdthDataImportResult result) { - Assume.assumeTrue(result.getNotReadResult() + "", forceImport()); + Map<String, Integer> notReadResult = result.getNotReadResult(); + String message; + if (notReadResult.isEmpty()) { + message = result.getMessages() + ""; + } else { + message = notReadResult + ""; + return; + } + Assume.assumeTrue(message, forceImport()); } protected long count() { ===================================== core/persistence/test/src/test/java/fr/ird/observe/persistence/avdth/data/AvdthIndianDataBuilderByYearLatestTest.java ===================================== @@ -0,0 +1,54 @@ +package fr.ird.observe.persistence.avdth.data; + +/*- + * #%L + * ObServe Core :: Persistence :: Test + * %% + * Copyright (C) 2008 - 2023 IRD, Ultreia.io + * %% + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public + * License along with this program. If not, see + * <http://www.gnu.org/licenses/gpl-3.0.html>. + * #L% + */ + +import fr.ird.observe.dto.ProtectedIdsCommon; +import org.junit.runners.Parameterized; + +import java.nio.file.Path; + +/** + * Created on 19/01/2023. + * + * @author Tony Chemit - dev(a)tchemit.fr + * @since 9.0.24 + */ +public class AvdthIndianDataBuilderByYearLatestTest extends AvdthDataBuilderByYearTestSupport { + + public static final Path ROOT_PATH = getPath("OI").resolve("latest"); + + @Parameterized.Parameters(name = "{index}: {0}") + public static Iterable<?> data() { + return getDatabases(ROOT_PATH); + } + + @Override + protected Path getRootPath() { + return ROOT_PATH; + } + + @Override + protected String getOceanId() { + return ProtectedIdsCommon.COMMON_OCEAN_INDIAN; + } +} ===================================== pom.xml ===================================== @@ -127,6 +127,8 @@ <!-- FIXME <lib.version.jaxb-runtime>4.0.0</lib.version.jaxb-runtime>--> <lib.version.jts>1.16.1</lib.version.jts> <!-- FIXME <lib.version.jts>1.18.2</lib.version.jts>--> + <lib.version.hsqldb>2.7.0</lib.version.hsqldb> + <!-- FIXME <lib.version.hsqldb>2.7.1</lib.version.hsqldb>--> <!-- license header configuration --> <license.licenseName>gpl_v3</license.licenseName> <license.organizationName>IRD, Ultreia.io</license.organizationName> ===================================== src/site/markdown/avdth/logbook.md ===================================== @@ -46,7 +46,7 @@ La table est déversée dans plusieurs tables : | ACTIVITE_26 | V_VENT_DIR | Activity.windDirection | | | ACTIVITE_27 | V_VENT_VIT | Activity.wind | [9](#n_0_9) | | ACTIVITE_28 | C_TYP_OBJET | | [10](#n_0_10) | -| ACTIVITE_29 | F_DCP_ECO | ??? | | +| ACTIVITE_29 | F_DCP_ECO | Dcp écologique | [10](#n_0_10) | | ACTIVITE_30 | F_PROP_BALISE | TransmittingBuoy.transmittingBuoyOwnerShip | [10](#n_0_10) | | ACTIVITE_31 | C_TYP_BALISE | | [10](#n_0_10) | | ACTIVITE_32 | V_ID_BALISE | TransmittingBuoy.code | | @@ -97,7 +97,7 @@ Voir ajout d'un DCP * Note 11 ### <a name="n_0_11"></a> -Donnée non persistée dans ObServe, on la conserve dans le commentaire du DCP +Donnée non persistée dans ObServe, (voir https://gitlab.com/ultreiaio/ird-observe/-/issues/2575#note_1247253501) * Note 12 ### <a name="n_0_12"></a> @@ -218,6 +218,7 @@ Pour le **simple mapping** on utilise la translation : | 3 | fr.ird.referential.ps.common.ObjectMaterial#0#0.50 | 2 LOG | | 9 | fr.ird.referential.ps.common.ObjectMaterial#0#1.3 | FOB | +Enfin si la valeur du champs **ACTIVITE.F_DCP_ECO** vaut **2**, alors on ajoute le matériel **4-1** (*Matériaux biodégradables*). * ```FloatingObject.transmittingBuoy``` View it on GitLab: https://gitlab.com/ultreiaio/ird-observe/-/compare/997e16bb8e2ad2675e5c4fae… -- View it on GitLab: https://gitlab.com/ultreiaio/ird-observe/-/compare/997e16bb8e2ad2675e5c4fae… You're receiving this email because of your account on gitlab.com.
1 0
0 0
  • ← Newer
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • Older →

HyperKitty Powered by HyperKitty version 1.3.12.