Nuiton-utils-commits
Threads by month
- ----- 2026 -----
- June
- May
- April
- March
- February
- January
- ----- 2025 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2024 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2023 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2022 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2021 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2020 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2019 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2018 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2017 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2016 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2015 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2014 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2013 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2012 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2011 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2010 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2009 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2008 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2007 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2006 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2005 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2004 -----
- December
- November
- October
- September
- August
December 2011
- 4 participants
- 18 discussions
r2256 - in trunk: nuiton-csv/src nuiton-csv/src/site nuiton-csv/src/site/apt src/site/apt
by tchemit@users.nuiton.org 14 Dec '11
by tchemit@users.nuiton.org 14 Dec '11
14 Dec '11
Author: tchemit
Date: 2011-12-14 08:38:25 +0100 (Wed, 14 Dec 2011)
New Revision: 2256
Url: http://nuiton.org/repositories/revision/nuiton-utils/2256
Log:
begin of doc for nuiton-csv
Added:
trunk/nuiton-csv/src/site/
trunk/nuiton-csv/src/site/apt/
trunk/nuiton-csv/src/site/apt/index.apt
trunk/nuiton-csv/src/site/site_fr.xml
Modified:
trunk/src/site/apt/index.apt
Added: trunk/nuiton-csv/src/site/apt/index.apt
===================================================================
--- trunk/nuiton-csv/src/site/apt/index.apt (rev 0)
+++ trunk/nuiton-csv/src/site/apt/index.apt 2011-12-14 07:38:25 UTC (rev 2256)
@@ -0,0 +1,147 @@
+~~~
+~~ #%L
+~~ Nuiton Utils :: Nuiton Csv
+~~
+~~ $Id$
+~~ $HeadURL$
+~~ %%
+~~ Copyright (C) 2011 CodeLutin
+~~ %%
+~~ This program is free software: you can redistribute it and/or modify
+~~ it under the terms of the GNU Lesser General Public License as
+~~ published by the Free Software Foundation, either version 3 of the
+~~ License, or (at your option) any later version.
+~~
+~~ This program is distributed in the hope that it will be useful,
+~~ but WITHOUT ANY WARRANTY; without even the implied warranty of
+~~ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+~~ GNU General Lesser Public License for more details.
+~~
+~~ You should have received a copy of the GNU General Lesser Public
+~~ License along with this program. If not, see
+~~ <http://www.gnu.org/licenses/lgpl-3.0.html>.
+~~ #L%
+~~~
+ ----
+ Nuiton csv
+ ----
+ ----
+ 2011-12-14
+ ----
+
+Présentation
+
+ nuiton-csv définit une api simple d'import-export au format csv. Cette api
+ permet de construire un modèle d'import (et/ou d'export) où les différentes
+ préocupations sont bien séparées.
+
+ Pour un import, et pour chaque ligne à importer, on a trois choses à faire :
+
+ - lire la donnée depuis la source d'entrée
+
+ - convertir la donnée en objet
+
+ - persister la donnée convertie dans un objet
+
+ La classe <<org.nuiton.util.csv.Import>> permet simplement d'effectuer des
+ imports, une fois le modèle crée.
+
+ Pour un export et pour chaque objet à persister dans le fichier csv, on a
+ aussi trois choses :
+
+ - lire la donnée depuis l'objet à persister
+
+ - convertir la donnée au format texte
+
+ - persister la donnée convertie dans le flux de sortie
+
+ La classe <<org.nuiton.util.csv.Export>> permet simplement d'effectuer un
+ export, une fois le modèle d'export crée et les données à persister récupérées.
+
+Api
+
+* org.nuiton.util.csv.ValueFormatter
+
+ A faire.
+
+* org.nuiton.util.csv.ValueParser
+
+ A faire.
+
+* org.nuiton.util.csv.ValueParserFormatter
+
+ A faire.
+
+* org.nuiton.util.csv.ImportModel
+
+ A faire.
+
+* org.nuiton.util.csv.ExportModel
+
+ A faire.
+
+* org.nuiton.util.csv.ImportExportModel
+
+ A faire.
+
+* org.nuiton.util.csv.ModelBuilder
+
+ A faire.
+
+* org.nuiton.util.csv.Import
+
+ A faire.
+
+* org.nuiton.util.csv.Export
+
+ A faire.
+
+Exemple
+
+* Construire un modèle d'import/export
+
+--------------------------------------------------------------------------------
+A faire...
+--------------------------------------------------------------------------------
+
+* Importer des données
+
+--------------------------------------------------------------------------------
+
+// création du modèle d'import csv
+ImportModel<E> csvModel = null;
+
+// creation d'un importer à partir d'un modèle et d'un reader sur fichier csv
+Import<E> importer = Import.newImport(csvModel, reader);
+
+try {
+
+ // parcours des objets crées à partir de chaque ligne du fichier csv
+ for (E entity : importer) {
+
+ // A vous :)
+
+ }
+} finally {
+
+ // fermeture de l'importer (ne ferme pas le flux d'entrée)
+ importer.close();
+}
+--------------------------------------------------------------------------------
+
+* Exporter des données
+
+--------------------------------------------------------------------------------
+
+// création du modèle d'export csv
+ExportModel<E> csvModel = null;
+
+// les données à exporter
+Iterable<E> datas = null;
+
+// création d'un exporter
+Export<E> exporter = Export.newExport(csvModel, datas);
+
+// lancement de l'export vers le fichier
+exporter.exportToFile(new File("output.csv"));
+--------------------------------------------------------------------------------
Property changes on: trunk/nuiton-csv/src/site/apt/index.apt
___________________________________________________________________
Added: svn:keywords
+ Author Date Id Revision HeadURL
Added: svn:eol-style
+ native
Added: trunk/nuiton-csv/src/site/site_fr.xml
===================================================================
--- trunk/nuiton-csv/src/site/site_fr.xml (rev 0)
+++ trunk/nuiton-csv/src/site/site_fr.xml 2011-12-14 07:38:25 UTC (rev 2256)
@@ -0,0 +1,56 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ #%L
+ Nuiton Utils :: Nuiton Csv
+
+ $Id$
+ $HeadURL$
+ %%
+ Copyright (C) 2011 CodeLutin
+ %%
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU Lesser General Public License as
+ published by the Free Software Foundation, either version 3 of the
+ License, or (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Lesser Public License for more details.
+
+ You should have received a copy of the GNU General Lesser Public
+ License along with this program. If not, see
+ <http://www.gnu.org/licenses/lgpl-3.0.html>.
+ #L%
+ -->
+
+
+
+<project name="${project.name}">
+
+ <bannerLeft>
+ <name>${project.name}</name>
+ <href>index.html</href>
+ </bannerLeft>
+
+ <poweredBy>
+ <logo href="http://maven.apache.org" name="Maven"
+ img="images/logos/maven-feather.png"/>
+ </poweredBy>
+
+ <body>
+
+ <breadcrumbs>
+ <item name="${project.name}" href="http://maven-site.nuiton.org/nuiton-utils/nuiton-csv/index.html"/>
+ </breadcrumbs>
+
+ <menu ref="modules"/>
+
+ <menu name="Utilisateur">
+ <item name="Accueil" href="index.html"/>
+ </menu>
+
+ <menu ref="reports"/>
+
+ </body>
+</project>
Property changes on: trunk/nuiton-csv/src/site/site_fr.xml
___________________________________________________________________
Added: svn:keywords
+ Author Date Id Revision HeadURL
Added: svn:eol-style
+ native
Modified: trunk/src/site/apt/index.apt
===================================================================
--- trunk/src/site/apt/index.apt 2011-12-11 18:07:30 UTC (rev 2255)
+++ trunk/src/site/apt/index.apt 2011-12-14 07:38:25 UTC (rev 2256)
@@ -63,3 +63,9 @@
permettant nottament de préciser des scopes (info, erreur, warning, ...)
{{{./nuiton-validator}Plus d'infos}}
+
+Librairie Nuiton-csv
+
+ Cette librairie propose une Api simple d'import export au format csv.
+
+ {{{./nuiton-csv}Plus d'infos}}
1
0
11 Dec '11
Author: tchemit
Date: 2011-12-11 19:07:30 +0100 (Sun, 11 Dec 2011)
New Revision: 2255
Url: http://nuiton.org/repositories/revision/nuiton-utils/2255
Log:
Add some usefull methods on Export + reformat
Modified:
trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/Export.java
trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/Import.java
Modified: trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/Export.java
===================================================================
--- trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/Export.java 2011-12-09 10:55:25 UTC (rev 2254)
+++ trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/Export.java 2011-12-11 18:07:30 UTC (rev 2255)
@@ -29,13 +29,36 @@
import org.apache.commons.logging.LogFactory;
import org.nuiton.util.StringUtil;
+import java.io.BufferedWriter;
+import java.io.File;
+import java.io.FileWriter;
import java.io.InputStream;
+import java.io.StringWriter;
+import java.io.Writer;
import java.util.Collection;
import java.util.LinkedList;
import java.util.List;
/**
- * TODO
+ * Object to realize a export from a {@link ExportModel} and some datas.
+ * <h2>Usefull static methods</h2>
+ * There is some usefull methods here to do operation in one line :
+ * <h3>To export to a file</h3>
+ * <pre>
+ * Export.exportToFile(model, data, file);
+ * </pre>
+ * <h3>To export to a writer</h3>
+ * <pre>
+ * Export.exportToWriter(model, data, writer);
+ * </pre>
+ * <h3>To export as a string</h3>
+ * <pre>
+ * String exportcontent = Export.exportToString(model, data);
+ * </pre>
+ * <h3>To obtain a new instance of an exporter</h3>
+ * <pre>
+ * Export<E> exporter = Export.newExport(model, data);
+ * </pre>
*
* @author bleny <leny(a)codelutin.com>
* @author tchemit <chemit(a)codelutin.com>
@@ -55,14 +78,42 @@
return new Export<E>(model, data);
}
+ public static <E> void exportToFile(ExportModel<E> model,
+ Iterable<E> data,
+ File file) throws Exception {
+ Export<E> exporter = newExport(model, data);
+ exporter.exportToFile(file);
+ }
+
+ public static <E> String exportToString(ExportModel<E> model,
+ Iterable<E> data) throws Exception {
+ Export<E> exporter = newExport(model, data);
+ return exporter.startExportAsString();
+ }
+
+ public static <E> void exportToWriter(ExportModel<E> model,
+ Iterable<E> data,
+ Writer writer) throws Exception {
+ Export<E> exporter = newExport(model, data);
+ exporter.startExport(writer);
+ }
+
protected Export(ExportModel<E> model, Iterable<E> data) {
this.model = model;
this.data = data;
}
- public String startExportAsString() throws Exception {
+ public void exportToFile(File file) throws Exception {
+ Writer writer = new BufferedWriter(new FileWriter(file));
+ try {
+ startExport(writer);
+ } finally {
+ writer.close();
+ }
+ }
+
+ public void startExport(Writer writer) throws Exception {
String separator = String.valueOf(model.getSeparator());
- StringBuilder csv = new StringBuilder();
// add headers
List<String> headerNames = new LinkedList<String>();
@@ -70,12 +121,12 @@
headerNames.add(column.getHeaderName());
}
String headersLine = StringUtil.join(headerNames, separator, true);
- csv.append(headersLine).append('\n');
-
+ writer.write(headersLine);
+ writer.write('\n');
if (log.isDebugEnabled()) {
log.debug("headers for export are '" + headersLine + "'");
if (data instanceof Collection) {
- log.debug("will export " + ((Collection<E>)data).size() + " lines");
+ log.debug("will export " + ((Collection<E>) data).size() + " lines");
}
}
@@ -90,17 +141,28 @@
if (formattedValue == null) {
throw new NullPointerException(
"column for header " + column.getHeaderName() +
- " returned a null value." + column.toString());
+ " returned a null value." + column.toString());
}
formattedValue =
StringUtil.escapeCsvValue(formattedValue, separator);
- csv.append(formattedValue).append(separator);
+ writer.write(formattedValue);
+ writer.write(separator);
}
- csv.append('\n');
+ writer.write('\n');
}
- return csv.toString();
}
+ public String startExportAsString() throws Exception {
+ StringWriter writer = new StringWriter();
+ try {
+ startExport(writer);
+ String result = writer.toString();
+ return result;
+ } finally {
+ writer.close();
+ }
+ }
+
public InputStream startExport() throws Exception {
String content = startExportAsString();
return IOUtils.toInputStream(content);
Modified: trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/Import.java
===================================================================
--- trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/Import.java 2011-12-09 10:55:25 UTC (rev 2254)
+++ trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/Import.java 2011-12-11 18:07:30 UTC (rev 2255)
@@ -96,7 +96,6 @@
*
* @return the Iterator used for csv iteration
* @see #prepareAndValidate()
- * @see Iterable#iterator()
*/
@Override
public Iterator<E> iterator() {
@@ -135,10 +134,10 @@
// read value from csv cell
String value = readValue(field, lineNumber);
-
+
// contravariance ftw
Object parsedValue = parseValue(field, lineNumber, value);
-
+
// set value to element
setValue(field, lineNumber, element, parsedValue);
}
@@ -166,7 +165,7 @@
* model. This will check if headers are unique, known by the model and
* if mandatory headers exist in the file. During this phase, the model
* will retrieve headers value with {@link
- * ImportModel#pushCsvHeaderNames(java.util.List)} call.
+ * ImportModel#pushCsvHeaderNames(List)} call.
*
* @since 2.4.1
*/
@@ -185,7 +184,8 @@
String[] headers = getHeaders();
if (log.isTraceEnabled()) {
- log.trace("headers of the CSV file are : " + Arrays.toString(headers));
+ log.trace("headers of the CSV file are : " +
+ Arrays.toString(headers));
}
// hook to do some stuff from the model
@@ -219,7 +219,7 @@
String value) {
try {
T parsedValue = field.parseValue(value);
- return parsedValue;
+ return parsedValue;
} catch (Exception e) {
String message = _("csv.import.error.unableToParseValue",
value, field.getHeaderName(), lineNumber)
@@ -259,9 +259,10 @@
model.getColumnsForImport()) {
validHeaderNames.add(importableColumn.getHeaderName());
}
- String validationMessage = _("csv.import.error.unrecognizedHeaders",
- StringUtil.join(csvHeaders, ", ", true),
- StringUtil.join(validHeaderNames, ", ", true));
+ String validationMessage =
+ _("csv.import.error.unrecognizedHeaders",
+ StringUtil.join(csvHeaders, ", ", true),
+ StringUtil.join(validHeaderNames, ", ", true));
throw new ImportRuntimeException(validationMessage);
}
}
@@ -298,45 +299,37 @@
mandatoryHeadersNames.removeAll(csvHeaders);
if (!mandatoryHeadersNames.isEmpty()) {
- String validationMessage = _("csv.import.error.missingMandatoryHeaders",
- StringUtil.join(mandatoryHeadersNames, ", ", true));
+ String validationMessage =
+ _("csv.import.error.missingMandatoryHeaders",
+ StringUtil.join(mandatoryHeadersNames, ", ", true));
throw new ImportRuntimeException(validationMessage);
}
-
- // XXX-fdesbois-2011-12-08 : this code is redundant with checkUniqueModelColumnNames() method
-// Set<String> headerNames = new HashSet<String>();
-// for (ImportableColumn<E, ?> importableColumn :
-// model.getColumnsForImport()) {
-// String headerName = importableColumn.getHeaderName();
-// boolean alreadyUsed = !headerNames.add(headerName);
-// if (alreadyUsed) {
-// throw new ImportRuntimeException(
-// "model contains multiple columnsForImport named '" +
-// headerName + "'");
-// }
-// }
}
protected String[] getHeaders() throws ImportRuntimeException {
try {
boolean canReadHeaders = reader.readHeaders();
if (!canReadHeaders) {
- throw new ImportRuntimeException(_("csv.import.error.unableToReadHeaders"));
+ throw new ImportRuntimeException(
+ _("csv.import.error.unableToReadHeaders"));
}
} catch (IOException e) {
- throw new ImportRuntimeException(_("csv.import.error.unableToReadHeaders"), e);
+ throw new ImportRuntimeException(
+ _("csv.import.error.unableToReadHeaders"), e);
}
try {
String[] result = reader.getHeaders();
return result;
} catch (IOException e) {
- throw new ImportRuntimeException(_("csv.import.error.unableToReadHeaders"), e);
+ throw new ImportRuntimeException(
+ _("csv.import.error.unableToReadHeaders"), e);
}
}
protected List<ImportableColumn<E, Object>> getNonIgnoredHeaders() {
- List<ImportableColumn<E, Object>> nonIgnoredHeaders = new ArrayList<ImportableColumn<E, Object>>();
+ List<ImportableColumn<E, Object>> nonIgnoredHeaders =
+ new ArrayList<ImportableColumn<E, Object>>();
for (ImportableColumn<E, Object> field : model.getColumnsForImport()) {
if (!field.isIgnored()) {
nonIgnoredHeaders.add(field);
@@ -346,7 +339,8 @@
}
protected List<ImportableColumn<E, ?>> getAllMandatoryHeaders() {
- List<ImportableColumn<E, ?>> allMandatoryHeaders = new ArrayList<ImportableColumn<E, ?>>();
+ List<ImportableColumn<E, ?>> allMandatoryHeaders =
+ new ArrayList<ImportableColumn<E, ?>>();
for (ImportableColumn<E, ?> field : model.getColumnsForImport()) {
if (field.isMandatory()) {
allMandatoryHeaders.add(field);
@@ -355,8 +349,7 @@
return allMandatoryHeaders;
}
- protected Import(ImportModel<E> model,
- InputStream inputStream) {
+ protected Import(ImportModel<E> model, InputStream inputStream) {
if (inputStream == null) {
throw new NullPointerException("inputStream is null");
}
@@ -365,8 +358,7 @@
reader.setTrimWhitespace(true);
}
- protected Import(ImportModel<E> model,
- Reader reader) {
+ protected Import(ImportModel<E> model, Reader reader) {
if (reader == null) {
throw new NullPointerException("reader is null");
}
@@ -380,7 +372,7 @@
* was successfully read.
*
* @return {@code true} if line was successfully read, says in fact there is
- * something after this line.
+ * something after this line.
* @throws ImportRuntimeException if could not read line
*/
protected boolean readRow() throws ImportRuntimeException {
1
0
r2254 - in trunk/nuiton-csv/src: main/java/org/nuiton/util/csv test/java/org/nuiton/util/csv test/resources
by tchemit@users.nuiton.org 09 Dec '11
by tchemit@users.nuiton.org 09 Dec '11
09 Dec '11
Author: tchemit
Date: 2011-12-09 11:55:25 +0100 (Fri, 09 Dec 2011)
New Revision: 2254
Url: http://nuiton.org/repositories/revision/nuiton-utils/2254
Log:
- clean unique test (avoid use external test resources for a one snapshot)
- use Iterable instead of Collection in that is called Model classes :)
Removed:
trunk/nuiton-csv/src/test/resources/org/
Modified:
trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/Export.java
trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/ExportModel.java
trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/ImportModel.java
trunk/nuiton-csv/src/test/java/org/nuiton/util/csv/ImportTest.java
Modified: trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/Export.java
===================================================================
--- trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/Export.java 2011-12-08 10:06:54 UTC (rev 2253)
+++ trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/Export.java 2011-12-09 10:55:25 UTC (rev 2254)
@@ -48,14 +48,14 @@
protected ExportModel<E> model;
- protected Collection<E> data;
+ protected Iterable<E> data;
public static <E> Export<E> newExport(ExportModel<E> model,
- Collection<E> data) {
+ Iterable<E> data) {
return new Export<E>(model, data);
}
- protected Export(ExportModel<E> model, Collection<E> data) {
+ protected Export(ExportModel<E> model, Iterable<E> data) {
this.model = model;
this.data = data;
}
@@ -74,10 +74,12 @@
if (log.isDebugEnabled()) {
log.debug("headers for export are '" + headersLine + "'");
- log.debug("will export " + data.size() + " lines");
+ if (data instanceof Collection) {
+ log.debug("will export " + ((Collection<E>)data).size() + " lines");
+ }
}
- Collection<ExportableColumn<E, Object>> columnsForExport =
+ Iterable<ExportableColumn<E, Object>> columnsForExport =
model.getColumnsForExport();
for (E object : data) {
Modified: trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/ExportModel.java
===================================================================
--- trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/ExportModel.java 2011-12-08 10:06:54 UTC (rev 2253)
+++ trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/ExportModel.java 2011-12-09 10:55:25 UTC (rev 2254)
@@ -24,8 +24,6 @@
*/
package org.nuiton.util.csv;
-import java.util.Collection;
-
/**
* TODO
*
@@ -37,6 +35,6 @@
char getSeparator();
- Collection<ExportableColumn<E, Object>> getColumnsForExport();
+ Iterable<ExportableColumn<E, Object>> getColumnsForExport();
}
Modified: trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/ImportModel.java
===================================================================
--- trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/ImportModel.java 2011-12-08 10:06:54 UTC (rev 2253)
+++ trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/ImportModel.java 2011-12-09 10:55:25 UTC (rev 2254)
@@ -24,7 +24,6 @@
*/
package org.nuiton.util.csv;
-import java.util.Collection;
import java.util.List;
/**
@@ -42,5 +41,5 @@
E newEmptyInstance();
- Collection<ImportableColumn<E, Object>> getColumnsForImport();
+ Iterable<ImportableColumn<E, Object>> getColumnsForImport();
}
Modified: trunk/nuiton-csv/src/test/java/org/nuiton/util/csv/ImportTest.java
===================================================================
--- trunk/nuiton-csv/src/test/java/org/nuiton/util/csv/ImportTest.java 2011-12-08 10:06:54 UTC (rev 2253)
+++ trunk/nuiton-csv/src/test/java/org/nuiton/util/csv/ImportTest.java 2011-12-09 10:55:25 UTC (rev 2254)
@@ -24,6 +24,7 @@
*/
package org.nuiton.util.csv;
+import org.apache.commons.lang3.time.DateUtils;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.junit.Assert;
@@ -32,10 +33,9 @@
import org.nuiton.util.csv.Common.BeanProperty;
import java.io.IOException;
-import java.io.InputStream;
-import java.text.DateFormat;
+import java.io.Reader;
+import java.io.StringReader;
import java.text.ParseException;
-import java.text.SimpleDateFormat;
import java.util.ArrayList;
import java.util.Collection;
import java.util.Date;
@@ -50,32 +50,27 @@
*/
public class ImportTest {
- private DateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd");
- /**
- * Test with {@link SimpleImportModel} that directly implements necessary {@link ImportableColumn}
- * with a empty file (says the file has only the header definition but no row
- *
- * @throws Exception for errors
- */
@Test
- public void testSimpleImportWithEmptyFile() throws Exception {
- ImportModel<Row> model = new SimpleImportModel(dateFormat);
- List<Row> rows = importFile(model, "emptyImportTest.csv");
+ public void testSimpleImportWithNoData() throws Exception {
+ String content = "DATE;NUMBER;TITLE";
+
+ List<Row> rows;
+ rows = importContent(new SimpleImportModel(), content);
Assert.assertEquals(0, rows.size());
+
+ content = "DATE;NUMBER;TITLE\n";
+ rows = importContent(new SimpleImportModel(), content);
+ Assert.assertEquals(0, rows.size());
}
- /**
- * Test with {@link SimpleImportModel} that directly implements necessary {@link ImportableColumn}
- * with a empty file (says the file has only the header definition but no row
- *
- * @throws Exception for errors
- */
@Test
- public void testSimpleImportWithEmptyFile2() throws Exception {
- ImportModel<Row> model = new SimpleImportModel(dateFormat);
- List<Row> rows = importFile(model, "emptyImport2Test.csv");
- Assert.assertEquals(0, rows.size());
+ public void testSimpleImportWithOneLine() throws Exception {
+ String content = "DATE;NUMBER;TITLE\n2011-12-05;18;\"1ère ligne\"";
+
+ List<Row> rows = importContent(new SimpleImportModel(), content);
+ Assert.assertEquals(1, rows.size());
+ assertRowEquals(rows.get(0), DateUtil.createDate(5, 12, 2011), 18, "1ère ligne");
}
/**
@@ -85,9 +80,14 @@
*/
@Test
public void testSimpleImport() throws Exception {
- ImportModel<Row> model = new SimpleImportModel(dateFormat);
- List<Row> rows = importFile(model, "importTest.csv");
- checkImportTestResult(rows);
+ String content = "DATE;NUMBER;TITLE\n" +
+ "2011-12-05;18;\"1ère ligne\"\n" +
+ "2011-12-06;19;\"2ème ligne\"\n" +
+ "2011-12-07;21;\"3ème ligne\"";
+ List<Row> rows = importContent(new SimpleImportModel(), content);
+ assertRowEquals(rows.get(0), DateUtil.createDate(5, 12, 2011), 18, "1ère ligne");
+ assertRowEquals(rows.get(1), DateUtil.createDate(6, 12, 2011), 19, "2ème ligne");
+ assertRowEquals(rows.get(2), DateUtil.createDate(7, 12, 2011), 21, "3ème ligne");
}
/**
@@ -97,15 +97,22 @@
*/
@Test
public void testColumnImport() throws Exception {
- ImportModel<Row> model = new ColumnImportModel(dateFormat);
- List<Row> rows = importFile(model, "importTest.csv");
- checkImportTestResult(rows);
+ String content = "DATE;NUMBER;TITLE\n" +
+ "2011-12-05;18;\"1ère ligne\"\n" +
+ "2011-12-06;19;\"2ème ligne\"\n" +
+ "2011-12-07;21;\"3ème ligne\"";
+ List<Row> rows = importContent(new ColumnImportModel(), content);
+ assertRowEquals(rows.get(0), DateUtil.createDate(5, 12, 2011), 18, "1ère ligne");
+ assertRowEquals(rows.get(1), DateUtil.createDate(6, 12, 2011), 19, "2ème ligne");
+ assertRowEquals(rows.get(2), DateUtil.createDate(7, 12, 2011), 21, "3ème ligne");
+
}
- protected List<Row> importFile(ImportModel<Row> model, String filePath) throws IOException {
- InputStream csvStream = getClass().getResourceAsStream(filePath);
+ protected List<Row> importContent(ImportModel<Row> model,
+ String content) throws IOException {
+ Reader reader = new StringReader(content);
try {
- Import<Row> rowImport = Import.newImport(model, csvStream);
+ Import<Row> rowImport = Import.newImport(model, reader);
try {
List<Row> result = new ArrayList<Row>();
for (Row row : rowImport) {
@@ -116,25 +123,14 @@
rowImport.close();
}
} finally {
- csvStream.close();
+ reader.close();
}
}
- private void checkImportTestResult(List<Row> rows) {
-
- Assert.assertEquals(3, rows.size());
- Row row1 = rows.get(0);
- Assert.assertEquals(DateUtil.createDate(5, 12, 2011), row1.getDate());
- Assert.assertEquals(new Integer(18), row1.getNumber());
- Assert.assertEquals("1ère ligne", row1.getTitle());
- Row row2 = rows.get(1);
- Assert.assertEquals(DateUtil.createDate(6, 12, 2011), row2.getDate());
- Assert.assertEquals(new Integer(19), row2.getNumber());
- Assert.assertEquals("2ème ligne", row2.getTitle());
- Row row3 = rows.get(2);
- Assert.assertEquals(DateUtil.createDate(7, 12, 2011), row3.getDate());
- Assert.assertEquals(new Integer(21), row3.getNumber());
- Assert.assertEquals("3ème ligne", row3.getTitle());
+ private void assertRowEquals(Row row, Date date, Integer number, String title) {
+ Assert.assertEquals(date, row.getDate());
+ Assert.assertEquals(number, row.getNumber());
+ Assert.assertEquals(title, row.getTitle());
}
public static class Row {
@@ -145,6 +141,15 @@
private Integer number;
+ public Row() {
+ }
+
+ public Row(Date date, String title, Integer number) {
+ this.date = date;
+ this.title = title;
+ this.number = number;
+ }
+
public Date getDate() {
return date;
}
@@ -168,17 +173,32 @@
public void setNumber(Integer number) {
this.number = number;
}
- }
- private static class ColumnImportModel implements ImportModel<Row> {
+ @Override
+ public boolean equals(Object o) {
+ if (this == o) return true;
+ if (!(o instanceof Row)) return false;
- private DateFormat dateFormat;
+ Row row = (Row) o;
- private ColumnImportModel(DateFormat dateFormat) {
- this.dateFormat = dateFormat;
+ if (!date.equals(row.date)) return false;
+ if (!number.equals(row.number)) return false;
+ return title.equals(row.title);
+
}
@Override
+ public int hashCode() {
+ int result = date.hashCode();
+ result = 31 * result + title.hashCode();
+ result = 31 * result + number.hashCode();
+ return result;
+ }
+ }
+
+ private static class ColumnImportModel implements ImportModel<Row> {
+
+ @Override
public char getSeparator() {
return ';';
}
@@ -242,7 +262,7 @@
@Override
public Date parse(String value)
throws ParseException {
- return dateFormat.parse(value);
+ return DateUtils.parseDate(value, "yyyy-MM-dd");
}
};
}
@@ -252,12 +272,6 @@
private Log log = LogFactory.getLog(SimpleImportModel.class);
- private DateFormat dateFormat;
-
- private SimpleImportModel(DateFormat dateFormat) {
- this.dateFormat = dateFormat;
- }
-
@Override
public char getSeparator() {
return ';';
@@ -337,7 +351,7 @@
@Override
public Date parseValue(String value)
throws ParseException {
- return dateFormat.parse(value);
+ return DateUtils.parseDate(value, "yyyy-MM-dd");
}
@Override
1
0
08 Dec '11
Author: fdesbois
Date: 2011-12-08 11:06:54 +0100 (Thu, 08 Dec 2011)
New Revision: 2253
Url: http://nuiton.org/repositories/revision/nuiton-utils/2253
Log:
#1846 : Little improvements
Modified:
trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/Import.java
Modified: trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/Import.java
===================================================================
--- trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/Import.java 2011-12-08 10:04:13 UTC (rev 2252)
+++ trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/Import.java 2011-12-08 10:06:54 UTC (rev 2253)
@@ -376,10 +376,10 @@
}
/**
- * Reads a next row from the reader and return {@code true} if line
- * was sucessfull read.
+ * Read the next row from the reader and return {@code true} if line
+ * was successfully read.
*
- * @return {@code true} if line was sucessfull read, says in fact there is
+ * @return {@code true} if line was successfully read, says in fact there is
* something after this line.
* @throws ImportRuntimeException if could not read line
*/
1
0
r2252 - in trunk/nuiton-csv/src: main/java/org/nuiton/util/csv main/resources/i18n test/java/org/nuiton/util/csv
by fdesbois@users.nuiton.org 08 Dec '11
by fdesbois@users.nuiton.org 08 Dec '11
08 Dec '11
Author: fdesbois
Date: 2011-12-08 11:04:13 +0100 (Thu, 08 Dec 2011)
New Revision: 2252
Url: http://nuiton.org/repositories/revision/nuiton-utils/2252
Log:
#1846 : Little improvements
Modified:
trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/Import.java
trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/ImportToMap.java
trunk/nuiton-csv/src/main/resources/i18n/nuiton-csv_en_GB.properties
trunk/nuiton-csv/src/main/resources/i18n/nuiton-csv_es_ES.properties
trunk/nuiton-csv/src/main/resources/i18n/nuiton-csv_fr_FR.properties
trunk/nuiton-csv/src/test/java/org/nuiton/util/csv/ImportTest.java
Modified: trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/Import.java
===================================================================
--- trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/Import.java 2011-12-08 02:59:10 UTC (rev 2251)
+++ trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/Import.java 2011-12-08 10:04:13 UTC (rev 2252)
@@ -84,10 +84,20 @@
return new Import<E>(model, inputStream);
}
- public static <E> Import<E> newImport(ImportModel<E> model, Reader reader) {
+ public static <E> Import<E> newImport(ImportModel<E> model,
+ Reader reader) {
return new Import<E>(model, reader);
}
+ /**
+ * Define iterator over import. First of all, the input stream will be
+ * validated based on defined model. Iteration will be done on all csv
+ * rows except first headers line.
+ *
+ * @return the Iterator used for csv iteration
+ * @see #prepareAndValidate()
+ * @see Iterable#iterator()
+ */
@Override
public Iterator<E> iterator() {
@@ -127,7 +137,7 @@
String value = readValue(field, lineNumber);
// contravariance ftw
- Object parsedValue = parserValue(field, lineNumber, value);
+ Object parsedValue = parseValue(field, lineNumber, value);
// set value to element
setValue(field, lineNumber, element, parsedValue);
@@ -151,6 +161,15 @@
reader.close();
}
+ /**
+ * Read the first mandatory headers line and validate it with the input
+ * model. This will check if headers are unique, known by the model and
+ * if mandatory headers exist in the file. During this phase, the model
+ * will retrieve headers value with {@link
+ * ImportModel#pushCsvHeaderNames(java.util.List)} call.
+ *
+ * @since 2.4.1
+ */
public void prepareAndValidate() {
if (validate) {
@@ -176,7 +195,6 @@
checkUniqueModelColumnNames();
// check that given headers from csv file are all known
-
checkHeaderNamesAreAllKnown(headers);
// check all mandatories column are on csv header
@@ -184,7 +202,7 @@
}
protected <T> String readValue(ImportableColumn<E, T> field,
- int lineNumber) {
+ int lineNumber) {
try {
String value = reader.get(field.getHeaderName());
return value;
@@ -196,8 +214,9 @@
}
}
- protected <T> T parserValue(ImportableColumn<E, T> field,
- int lineNumber, String value) {
+ protected <T> T parseValue(ImportableColumn<E, T> field,
+ int lineNumber,
+ String value) {
try {
T parsedValue = field.parseValue(value);
return parsedValue;
@@ -284,34 +303,35 @@
throw new ImportRuntimeException(validationMessage);
}
- Set<String> headerNames = new HashSet<String>();
- for (ImportableColumn<E, ?> importableColumn :
- model.getColumnsForImport()) {
- String headerName = importableColumn.getHeaderName();
- boolean alreadyUsed = !headerNames.add(headerName);
- if (alreadyUsed) {
- throw new ImportRuntimeException(
- "model contains multiple columnsForImport named '" +
- headerName + "'");
- }
- }
+ // XXX-fdesbois-2011-12-08 : this code is redundant with checkUniqueModelColumnNames() method
+// Set<String> headerNames = new HashSet<String>();
+// for (ImportableColumn<E, ?> importableColumn :
+// model.getColumnsForImport()) {
+// String headerName = importableColumn.getHeaderName();
+// boolean alreadyUsed = !headerNames.add(headerName);
+// if (alreadyUsed) {
+// throw new ImportRuntimeException(
+// "model contains multiple columnsForImport named '" +
+// headerName + "'");
+// }
+// }
}
protected String[] getHeaders() throws ImportRuntimeException {
try {
boolean canReadHeaders = reader.readHeaders();
if (!canReadHeaders) {
- throw new ImportRuntimeException("can't read headers");
+ throw new ImportRuntimeException(_("csv.import.error.unableToReadHeaders"));
}
} catch (IOException e) {
- throw new ImportRuntimeException("can't read headers");
+ throw new ImportRuntimeException(_("csv.import.error.unableToReadHeaders"), e);
}
try {
String[] result = reader.getHeaders();
return result;
- } catch (IOException eee) {
- throw new ImportRuntimeException("can't get headers", eee);
+ } catch (IOException e) {
+ throw new ImportRuntimeException(_("csv.import.error.unableToReadHeaders"), e);
}
}
@@ -335,7 +355,8 @@
return allMandatoryHeaders;
}
- protected Import(ImportModel<E> model, InputStream inputStream) {
+ protected Import(ImportModel<E> model,
+ InputStream inputStream) {
if (inputStream == null) {
throw new NullPointerException("inputStream is null");
}
@@ -344,7 +365,8 @@
reader.setTrimWhitespace(true);
}
- protected Import(ImportModel<E> model, Reader reader) {
+ protected Import(ImportModel<E> model,
+ Reader reader) {
if (reader == null) {
throw new NullPointerException("reader is null");
}
@@ -365,9 +387,6 @@
try {
boolean hasNext = reader.readRecord();
return hasNext;
-// if (emptyFile) {
-// throw new ImportRuntimeException("CSV file has no line");
-// }
} catch (IOException e) {
reader.close();
throw new ImportRuntimeException(_("csv.import.error.unableToReadLine", 1), e);
Modified: trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/ImportToMap.java
===================================================================
--- trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/ImportToMap.java 2011-12-08 02:59:10 UTC (rev 2251)
+++ trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/ImportToMap.java 2011-12-08 10:04:13 UTC (rev 2252)
@@ -99,7 +99,7 @@
String value = readValue(field, lineNumber);
// contravariance ftw
- Object parsedValue = parserValue(field, lineNumber, value);
+ Object parsedValue = parseValue(field, lineNumber, value);
// set value to element
setValue(field, lineNumber, element, parsedValue);
Modified: trunk/nuiton-csv/src/main/resources/i18n/nuiton-csv_en_GB.properties
===================================================================
--- trunk/nuiton-csv/src/main/resources/i18n/nuiton-csv_en_GB.properties 2011-12-08 02:59:10 UTC (rev 2251)
+++ trunk/nuiton-csv/src/main/resources/i18n/nuiton-csv_en_GB.properties 2011-12-08 10:04:13 UTC (rev 2252)
@@ -2,6 +2,7 @@
csv.import.error.missingMandatoryHeaders=The mandatory fields %s are missing
csv.import.error.unableToParseValue=Unable to parse value '%s' (column '%s', line %s)
csv.import.error.unableToReadField=Unable to read value of column '%s' at line %s
+csv.import.error.unableToReadHeaders=Unable to read headers
csv.import.error.unableToReadLine=Unable to read line %s
csv.import.error.unableToSetValue=Unable to set value '%s' (object '%s', line %s, column '%s')
csv.import.error.unrecognizedHeaders=Fields %s are not recognized. Accepted fields are %s.
Modified: trunk/nuiton-csv/src/main/resources/i18n/nuiton-csv_es_ES.properties
===================================================================
--- trunk/nuiton-csv/src/main/resources/i18n/nuiton-csv_es_ES.properties 2011-12-08 02:59:10 UTC (rev 2251)
+++ trunk/nuiton-csv/src/main/resources/i18n/nuiton-csv_es_ES.properties 2011-12-08 10:04:13 UTC (rev 2252)
@@ -2,6 +2,7 @@
csv.import.error.missingMandatoryHeaders=The mandatory fields %s are missing
csv.import.error.unableToParseValue=Unable to parse value '%s' (column '%s', line %s)
csv.import.error.unableToReadField=Unable to read value of column '%s' at line %s
+csv.import.error.unableToReadHeaders=Unable to read headers
csv.import.error.unableToReadLine=Unable to read line %s
csv.import.error.unableToSetValue=Unable to set value '%s' (object'%s', line %s, column '%s')
csv.import.error.unrecognizedHeaders=Fields %s are not recognized. Accepted fields are %s.
Modified: trunk/nuiton-csv/src/main/resources/i18n/nuiton-csv_fr_FR.properties
===================================================================
--- trunk/nuiton-csv/src/main/resources/i18n/nuiton-csv_fr_FR.properties 2011-12-08 02:59:10 UTC (rev 2251)
+++ trunk/nuiton-csv/src/main/resources/i18n/nuiton-csv_fr_FR.properties 2011-12-08 10:04:13 UTC (rev 2252)
@@ -2,6 +2,7 @@
csv.import.error.missingMandatoryHeaders=Les champs obligatoires %s sont manquants
csv.import.error.unableToParseValue=Erreur lors de l'interprétation de la valeur '%s' (colonne '%s', ligne %s)
csv.import.error.unableToReadField=Impossible de lire la colonne '%s' à la ligne %s
+csv.import.error.unableToReadHeaders=Impossible de lire les en-têtes de colonnes
csv.import.error.unableToReadLine=Impossible de lire la ligne %s
csv.import.error.unableToSetValue=Impossible d'enregistrer la valeur '%s' (objet '%s', ligne %s, column '%s')
csv.import.error.unrecognizedHeaders=Les champs %s ne sont pas reconnus. Les champs possibles sont %s.
Modified: trunk/nuiton-csv/src/test/java/org/nuiton/util/csv/ImportTest.java
===================================================================
--- trunk/nuiton-csv/src/test/java/org/nuiton/util/csv/ImportTest.java 2011-12-08 02:59:10 UTC (rev 2251)
+++ trunk/nuiton-csv/src/test/java/org/nuiton/util/csv/ImportTest.java 2011-12-08 10:04:13 UTC (rev 2252)
@@ -39,7 +39,6 @@
import java.util.ArrayList;
import java.util.Collection;
import java.util.Date;
-import java.util.Iterator;
import java.util.List;
/**
@@ -108,10 +107,9 @@
try {
Import<Row> rowImport = Import.newImport(model, csvStream);
try {
- Iterator<Row> it = rowImport.iterator();
List<Row> result = new ArrayList<Row>();
- while (it.hasNext()) {
- result.add(it.next());
+ for (Row row : rowImport) {
+ result.add(row);
}
return result;
} finally {
1
0
08 Dec '11
Author: tchemit
Date: 2011-12-08 03:59:10 +0100 (Thu, 08 Dec 2011)
New Revision: 2251
Url: http://nuiton.org/repositories/revision/nuiton-utils/2251
Log:
fix some javadoc
Modified:
trunk/nuiton-utils/src/main/java/org/nuiton/util/ApplicationConfig.java
Modified: trunk/nuiton-utils/src/main/java/org/nuiton/util/ApplicationConfig.java
===================================================================
--- trunk/nuiton-utils/src/main/java/org/nuiton/util/ApplicationConfig.java 2011-12-07 19:51:32 UTC (rev 2250)
+++ trunk/nuiton-utils/src/main/java/org/nuiton/util/ApplicationConfig.java 2011-12-08 02:59:10 UTC (rev 2251)
@@ -678,6 +678,7 @@
* example: 0,1,5,6
*
* @since 2.4
+ * @return ordered action step number
*/
public List<Integer> getActionStep() {
List<Integer> result = new ArrayList<Integer>(actions.keySet());
@@ -688,8 +689,6 @@
/**
* Do all action in specified order step (first 0).
*
- * @param step do action only defined in this step
- *
* @throws IllegalAccessException if action invocation failed
* @throws IllegalArgumentException if action invocation failed
* @throws InvocationTargetException if action invocation failed
1
0
r2250 - in trunk/nuiton-csv/src: main/java/org/nuiton/util/csv test/java/org/nuiton/util/csv test/resources/org/nuiton/util/csv
by tchemit@users.nuiton.org 07 Dec '11
by tchemit@users.nuiton.org 07 Dec '11
07 Dec '11
Author: tchemit
Date: 2011-12-07 20:51:32 +0100 (Wed, 07 Dec 2011)
New Revision: 2250
Url: http://nuiton.org/repositories/revision/nuiton-utils/2250
Log:
Anomalie #1845: Can not import empty file
Evolution #1846: Improve Import class code
Added:
trunk/nuiton-csv/src/test/resources/org/nuiton/util/csv/emptyImport2Test.csv
trunk/nuiton-csv/src/test/resources/org/nuiton/util/csv/emptyImportTest.csv
Modified:
trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/Import.java
trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/ImportToMap.java
trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/ModelBuilder.java
trunk/nuiton-csv/src/test/java/org/nuiton/util/csv/ImportTest.java
Modified: trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/Import.java
===================================================================
--- trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/Import.java 2011-12-01 15:22:54 UTC (rev 2249)
+++ trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/Import.java 2011-12-07 19:51:32 UTC (rev 2250)
@@ -48,9 +48,9 @@
/**
* Import engine for a given import model.
- *
+ * <p/>
* It acts as an {@link Iterable}, you can use directly inside a foreach.
- *
+ * <p/>
* The method {@link #prepareAndValidate()} will be invoked before all and
* only once. It mainly obtain header from the csv input, pass it to the model
* and then validate the model.
@@ -84,8 +84,7 @@
return new Import<E>(model, inputStream);
}
- public static <E> Import<E> newImport(ImportModel<E> model,
- Reader reader) {
+ public static <E> Import<E> newImport(ImportModel<E> model, Reader reader) {
return new Import<E>(model, reader);
}
@@ -94,16 +93,17 @@
prepareAndValidate();
- readFirstLine();
-
return new Iterator<E>() {
- boolean hasNext = true;
+ // read first line since first line is header
+ boolean hasNext = readRow();
+ // get once for all columns to import
+ List<ImportableColumn<E, Object>> columns = getNonIgnoredHeaders();
+
+ // to stock the current line number
int lineNumber;
- E lastElement;
-
@Override
public boolean hasNext() {
return hasNext;
@@ -121,57 +121,22 @@
E element = model.newEmptyInstance();
- for (ImportableColumn<E, Object> field : getNonIgnoredHeaders()) {
+ for (ImportableColumn<E, Object> field : columns) {
// read value from csv cell
- String value;
- try {
- value = reader.get(field.getHeaderName());
- } catch (Exception e) {
- reader.close();
- throw new ImportRuntimeException(
- _("csv.import.error.unableToReadField",
- field.getHeaderName(), lineNumber), e);
- }
-
+ String value = readValue(field, lineNumber);
+
// contravariance ftw
- Object parsedValue;
- try {
- parsedValue = field.parseValue(value);
- } catch (Exception e) {
- String message = _("csv.import.error.unableToParseValue",
- value, field.getHeaderName(), lineNumber)
- + "\n" + e.getMessage();
- throw new ImportRuntimeException(message, e);
- }
-
+ Object parsedValue = parserValue(field, lineNumber, value);
+
// set value to element
- try {
- field.setValue(element, parsedValue);
- } catch (Exception e) {
- String message = _("csv.import.error.unableToSetValue",
- parsedValue,
- element.toString(),
- lineNumber, field.getHeaderName());
- if (log.isErrorEnabled()) {
- log.error(message);
- }
- throw new ImportRuntimeException(message, e);
- }
-
- lastElement = element;
+ setValue(field, lineNumber, element, parsedValue);
}
- try {
- hasNext = reader.readRecord();
- } catch (IOException e) {
- reader.close();
- throw new ImportRuntimeException(
- _("csv.import.error.unableToReadLine",
- lineNumber + 1), e);
- }
+ // check if there is a next row to read
+ hasNext = readRow();
- return lastElement;
+ return element;
}
@Override
@@ -201,8 +166,7 @@
String[] headers = getHeaders();
if (log.isTraceEnabled()) {
- log.trace("headers of the CSV file are : " +
- Arrays.toString(headers));
+ log.trace("headers of the CSV file are : " + Arrays.toString(headers));
}
// hook to do some stuff from the model
@@ -219,6 +183,50 @@
checkAllMandatoryHeadersArePresent(headers);
}
+ protected <T> String readValue(ImportableColumn<E, T> field,
+ int lineNumber) {
+ try {
+ String value = reader.get(field.getHeaderName());
+ return value;
+ } catch (Exception e) {
+ reader.close();
+ throw new ImportRuntimeException(
+ _("csv.import.error.unableToReadField",
+ field.getHeaderName(), lineNumber), e);
+ }
+ }
+
+ protected <T> T parserValue(ImportableColumn<E, T> field,
+ int lineNumber, String value) {
+ try {
+ T parsedValue = field.parseValue(value);
+ return parsedValue;
+ } catch (Exception e) {
+ String message = _("csv.import.error.unableToParseValue",
+ value, field.getHeaderName(), lineNumber)
+ + "\n" + e.getMessage();
+ throw new ImportRuntimeException(message, e);
+ }
+ }
+
+ protected <T> void setValue(ImportableColumn<E, T> field,
+ int lineNumber,
+ E element,
+ T parsedValue) {
+ try {
+ field.setValue(element, parsedValue);
+ } catch (Exception e) {
+ String message = _("csv.import.error.unableToSetValue",
+ parsedValue,
+ element.toString(),
+ lineNumber, field.getHeaderName());
+ if (log.isErrorEnabled()) {
+ log.error(message);
+ }
+ throw new ImportRuntimeException(message, e);
+ }
+ }
+
protected void checkHeaderNamesAreAllKnown(String[] headers) {
List<String> csvHeaders = new ArrayList<String>();
Collections.addAll(csvHeaders, headers);
@@ -284,7 +292,7 @@
if (alreadyUsed) {
throw new ImportRuntimeException(
"model contains multiple columnsForImport named '" +
- headerName + "'");
+ headerName + "'");
}
}
}
@@ -345,13 +353,21 @@
this.reader.setTrimWhitespace(true);
}
- protected void readFirstLine() throws ImportRuntimeException {
+ /**
+ * Reads a next row from the reader and return {@code true} if line
+ * was sucessfull read.
+ *
+ * @return {@code true} if line was sucessfull read, says in fact there is
+ * something after this line.
+ * @throws ImportRuntimeException if could not read line
+ */
+ protected boolean readRow() throws ImportRuntimeException {
try {
- boolean emptyFile = !reader.readRecord();
-
- if (emptyFile) {
- throw new ImportRuntimeException("CSV file has no line");
- }
+ boolean hasNext = reader.readRecord();
+ return hasNext;
+// if (emptyFile) {
+// throw new ImportRuntimeException("CSV file has no line");
+// }
} catch (IOException e) {
reader.close();
throw new ImportRuntimeException(_("csv.import.error.unableToReadLine", 1), e);
Modified: trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/ImportToMap.java
===================================================================
--- trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/ImportToMap.java 2011-12-01 15:22:54 UTC (rev 2249)
+++ trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/ImportToMap.java 2011-12-07 19:51:32 UTC (rev 2250)
@@ -24,19 +24,14 @@
*/
package org.nuiton.util.csv;
-import org.apache.commons.logging.Log;
-import org.apache.commons.logging.LogFactory;
-
-import java.io.IOException;
import java.io.InputStream;
import java.io.Reader;
import java.util.HashMap;
import java.util.Iterator;
+import java.util.List;
import java.util.Map;
import java.util.NoSuchElementException;
-import static org.nuiton.i18n.I18n._;
-
/**
* A extended {@link Import} to read csv lines into a single map.
* <p/>
@@ -48,9 +43,6 @@
*/
public class ImportToMap extends Import<Map<String, Object>> {
- /** Logger. */
- private static final Log log = LogFactory.getLog(ImportToMap.class);
-
public static ImportToMap newImport(ImportModel<Map<String, Object>> model,
InputStream inputStream) {
return new ImportToMap(model, inputStream);
@@ -67,16 +59,20 @@
// obtain headers from csv input and validate the model
prepareAndValidate();
- // read first line since first line is header
- readFirstLine();
-
return new Iterator<Map<String, Object>>() {
- boolean hasNext = true;
+ // read first line since first line is header
+ boolean hasNext = readRow();
+ // get once for all columns to import
+ List<ImportableColumn<Map<String, Object>, Object>> columns =
+ getNonIgnoredHeaders();
+
+ // to stock the current line number
int lineNumber;
- Map<String, Object> element = new HashMap<String, Object>();
+ // the map where to object of a row
+ final Map<String, Object> element = new HashMap<String, Object>();
@Override
public boolean hasNext() {
@@ -97,51 +93,20 @@
element.clear();
for (ImportableColumn<Map<String, Object>, Object> field :
- getNonIgnoredHeaders()) {
+ columns) {
// read value from csv cell
- String value;
- try {
- value = reader.get(field.getHeaderName());
- } catch (Exception e) {
- reader.close();
- throw new ImportRuntimeException(_("csv.import.error.unableToReadField",
- field.getHeaderName(), lineNumber), e);
- }
+ String value = readValue(field, lineNumber);
// contravariance ftw
- Object parsedValue;
- try {
- parsedValue = field.parseValue(value);
- } catch (Exception e) {
- String message = _("csv.import.error.unableToParseValue",
- value, field.getHeaderName(), lineNumber)
- + "\n" + e.getMessage();
- throw new ImportRuntimeException(message, e);
- }
+ Object parsedValue = parserValue(field, lineNumber, value);
// set value to element
- try {
- field.setValue(element, parsedValue);
- } catch (Exception e) {
- String message = _("csv.import.error.unableToSetValue",
- parsedValue,
- element.toString(),
- lineNumber, field.getHeaderName());
- if (log.isErrorEnabled()) {
- log.error(message);
- }
- throw new ImportRuntimeException(message, e);
- }
+ setValue(field, lineNumber, element, parsedValue);
}
- try {
- hasNext = reader.readRecord();
- } catch (IOException e) {
- reader.close();
- throw new ImportRuntimeException(
- _("csv.import.error.unableToReadLine", lineNumber + 1), e);
- }
+ // check if there is a next row to read
+ hasNext = readRow();
return element;
}
Modified: trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/ModelBuilder.java
===================================================================
--- trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/ModelBuilder.java 2011-12-01 15:22:54 UTC (rev 2249)
+++ trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/ModelBuilder.java 2011-12-07 19:51:32 UTC (rev 2250)
@@ -28,8 +28,8 @@
import java.util.LinkedList;
/**
- * TODO
*
+ *
* @author bleny <leny(a)codelutin.com>
* @author tchemit <chemit(a)codelutin.com>
* @since 2.4
Modified: trunk/nuiton-csv/src/test/java/org/nuiton/util/csv/ImportTest.java
===================================================================
--- trunk/nuiton-csv/src/test/java/org/nuiton/util/csv/ImportTest.java 2011-12-01 15:22:54 UTC (rev 2249)
+++ trunk/nuiton-csv/src/test/java/org/nuiton/util/csv/ImportTest.java 2011-12-07 19:51:32 UTC (rev 2250)
@@ -1,6 +1,9 @@
/*
* #%L
* Nuiton Utils :: Nuiton Csv
+ *
+ * $Id$
+ * $HeadURL$
* %%
* Copyright (C) 2011 CodeLutin
* %%
@@ -24,11 +27,11 @@
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.junit.Assert;
-import org.junit.Before;
import org.junit.Test;
import org.nuiton.util.DateUtil;
import org.nuiton.util.csv.Common.BeanProperty;
+import java.io.IOException;
import java.io.InputStream;
import java.text.DateFormat;
import java.text.ParseException;
@@ -42,25 +45,50 @@
/**
* Created on 28/11/11
*
- * @author fdesbois <florian.desbois(a)wiztivi.com>
+ * @author fdesbois <desbois(a)codelutin.com>
+ * @author tchemit <chemit(a)codelutin.com>
+ * @since 2.4
*/
public class ImportTest {
- private DateFormat dateFormat;
+ private DateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd");
- @Before
- public void setUp() {
- dateFormat = new SimpleDateFormat("yyyy-MM-dd");
+ /**
+ * Test with {@link SimpleImportModel} that directly implements necessary {@link ImportableColumn}
+ * with a empty file (says the file has only the header definition but no row
+ *
+ * @throws Exception for errors
+ */
+ @Test
+ public void testSimpleImportWithEmptyFile() throws Exception {
+ ImportModel<Row> model = new SimpleImportModel(dateFormat);
+ List<Row> rows = importFile(model, "emptyImportTest.csv");
+ Assert.assertEquals(0, rows.size());
}
/**
+ * Test with {@link SimpleImportModel} that directly implements necessary {@link ImportableColumn}
+ * with a empty file (says the file has only the header definition but no row
+ *
+ * @throws Exception for errors
+ */
+ @Test
+ public void testSimpleImportWithEmptyFile2() throws Exception {
+ ImportModel<Row> model = new SimpleImportModel(dateFormat);
+ List<Row> rows = importFile(model, "emptyImport2Test.csv");
+ Assert.assertEquals(0, rows.size());
+ }
+
+ /**
* Test with {@link SimpleImportModel} that directly implements necessary {@link ImportableColumn}.
*
* @throws Exception for errors
*/
@Test
public void testSimpleImport() throws Exception {
- execute(new SimpleImportModel(dateFormat));
+ ImportModel<Row> model = new SimpleImportModel(dateFormat);
+ List<Row> rows = importFile(model, "importTest.csv");
+ checkImportTestResult(rows);
}
/**
@@ -70,40 +98,45 @@
*/
@Test
public void testColumnImport() throws Exception {
- execute(new ColumnImportModel(dateFormat));
+ ImportModel<Row> model = new ColumnImportModel(dateFormat);
+ List<Row> rows = importFile(model, "importTest.csv");
+ checkImportTestResult(rows);
}
- private void execute(ImportModel<Row> model) {
-
- InputStream csvStream = getClass().getResourceAsStream("importTest.csv");
- Import<Row> rowImport = Import.newImport(model, csvStream);
+ protected List<Row> importFile(ImportModel<Row> model, String filePath) throws IOException {
+ InputStream csvStream = getClass().getResourceAsStream(filePath);
try {
- Iterator<Row> it = rowImport.iterator();
- List<Row> rows = newList(it);
- Assert.assertEquals(3, rows.size());
- Row row1 = rows.get(0);
- Assert.assertEquals(DateUtil.createDate(5, 12, 2011), row1.getDate());
- Assert.assertEquals(new Integer(18), row1.getNumber());
- Assert.assertEquals("1ère ligne", row1.getTitle());
- Row row2 = rows.get(1);
- Assert.assertEquals(DateUtil.createDate(6, 12, 2011), row2.getDate());
- Assert.assertEquals(new Integer(19), row2.getNumber());
- Assert.assertEquals("2ème ligne", row2.getTitle());
- Row row3 = rows.get(2);
- Assert.assertEquals(DateUtil.createDate(7, 12, 2011), row3.getDate());
- Assert.assertEquals(new Integer(21), row3.getNumber());
- Assert.assertEquals("3ème ligne", row3.getTitle());
+ Import<Row> rowImport = Import.newImport(model, csvStream);
+ try {
+ Iterator<Row> it = rowImport.iterator();
+ List<Row> result = new ArrayList<Row>();
+ while (it.hasNext()) {
+ result.add(it.next());
+ }
+ return result;
+ } finally {
+ rowImport.close();
+ }
} finally {
- rowImport.close();
+ csvStream.close();
}
}
- private static <T> List<T> newList(Iterator<T> iterator) {
- List<T> result = new ArrayList<T>();
- while (iterator.hasNext()) {
- result.add(iterator.next());
- }
- return result;
+ private void checkImportTestResult(List<Row> rows) {
+
+ Assert.assertEquals(3, rows.size());
+ Row row1 = rows.get(0);
+ Assert.assertEquals(DateUtil.createDate(5, 12, 2011), row1.getDate());
+ Assert.assertEquals(new Integer(18), row1.getNumber());
+ Assert.assertEquals("1ère ligne", row1.getTitle());
+ Row row2 = rows.get(1);
+ Assert.assertEquals(DateUtil.createDate(6, 12, 2011), row2.getDate());
+ Assert.assertEquals(new Integer(19), row2.getNumber());
+ Assert.assertEquals("2ème ligne", row2.getTitle());
+ Row row3 = rows.get(2);
+ Assert.assertEquals(DateUtil.createDate(7, 12, 2011), row3.getDate());
+ Assert.assertEquals(new Integer(21), row3.getNumber());
+ Assert.assertEquals("3ème ligne", row3.getTitle());
}
public static class Row {
@@ -234,7 +267,9 @@
@Override
public void pushCsvHeaderNames(List<String> headerNames) {
- log.info("Headers are : " + headerNames);
+ if (log.isDebugEnabled()) {
+ log.debug("Headers are : " + headerNames);
+ }
}
@Override
Added: trunk/nuiton-csv/src/test/resources/org/nuiton/util/csv/emptyImport2Test.csv
===================================================================
--- trunk/nuiton-csv/src/test/resources/org/nuiton/util/csv/emptyImport2Test.csv (rev 0)
+++ trunk/nuiton-csv/src/test/resources/org/nuiton/util/csv/emptyImport2Test.csv 2011-12-07 19:51:32 UTC (rev 2250)
@@ -0,0 +1 @@
+DATE;NUMBER;TITLE
Property changes on: trunk/nuiton-csv/src/test/resources/org/nuiton/util/csv/emptyImport2Test.csv
___________________________________________________________________
Added: svn:keywords
+ Author Date Id Revision HeadURL
Added: svn:eol-style
+ native
Added: trunk/nuiton-csv/src/test/resources/org/nuiton/util/csv/emptyImportTest.csv
===================================================================
--- trunk/nuiton-csv/src/test/resources/org/nuiton/util/csv/emptyImportTest.csv (rev 0)
+++ trunk/nuiton-csv/src/test/resources/org/nuiton/util/csv/emptyImportTest.csv 2011-12-07 19:51:32 UTC (rev 2250)
@@ -0,0 +1 @@
+DATE;NUMBER;TITLE
\ No newline at end of file
Property changes on: trunk/nuiton-csv/src/test/resources/org/nuiton/util/csv/emptyImportTest.csv
___________________________________________________________________
Added: svn:keywords
+ Author Date Id Revision HeadURL
Added: svn:eol-style
+ native
1
0
01 Dec '11
Author: kmorin
Date: 2011-12-01 16:22:54 +0100 (Thu, 01 Dec 2011)
New Revision: 2249
Url: http://nuiton.org/repositories/revision/nuiton-utils/2249
Log:
debug test (was using Calendar.roll instead of Calendar.add)
Modified:
trunk/nuiton-utils/src/test/java/org/nuiton/util/DateUtilTest.java
Modified: trunk/nuiton-utils/src/test/java/org/nuiton/util/DateUtilTest.java
===================================================================
--- trunk/nuiton-utils/src/test/java/org/nuiton/util/DateUtilTest.java 2011-11-30 17:26:45 UTC (rev 2248)
+++ trunk/nuiton-utils/src/test/java/org/nuiton/util/DateUtilTest.java 2011-12-01 15:22:54 UTC (rev 2249)
@@ -262,8 +262,8 @@
log.info("getAge");
Calendar calendar = Calendar.getInstance();
- calendar.roll(Calendar.YEAR, -20);
- calendar.roll(Calendar.DATE, -1);
+ calendar.add(Calendar.YEAR, -20);
+ calendar.add(Calendar.DAY_OF_YEAR, -1);
int result = DateUtil.getAge(calendar.getTime());
log.info("result1 : " + result);
1
0