November 2011 - Nuiton-utils-commits - mailman.cloud.codelutin.com

r2248 - trunk/nuiton-csv/src/main/java/org/nuiton/util/csv
by bleny＠users.nuiton.org Nov. 30, 2011

Nov. 30, 2011

Author: bleny Date: 2011-11-30 18:26:45 +0100 (Wed, 30 Nov 2011) New Revision: 2248 Url: http://nuiton.org/repositories/revision/nuiton-utils/2248 Log: fix typo Modified: trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/Import.java Modified: trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/Import.java =================================================================== --- trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/Import.java 2011-11-30 17:19:54 UTC (rev 2247) +++ trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/Import.java 2011-11-30 17:26:45 UTC (rev 2248) @@ -216,7 +216,7 @@ checkHeaderNamesAreAllKnown(headers); // check all mandatories column are on csv header - checkAllManadatoryHeadersArePresent(headers); + checkAllMandatoryHeadersArePresent(headers); } protected void checkHeaderNamesAreAllKnown(String[] headers) { @@ -259,7 +259,7 @@ } } - protected void checkAllManadatoryHeadersArePresent(String[] headers) { + protected void checkAllMandatoryHeadersArePresent(String[] headers) { List<String> csvHeaders = new ArrayList<String>(); Collections.addAll(csvHeaders, headers);

1 0

r2247 - in trunk: . nuiton-csv nuiton-profiling nuiton-utils nuiton-validator
by maven-release＠users.nuiton.org Nov. 30, 2011

Nov. 30, 2011

Author: maven-release Date: 2011-11-30 18:19:54 +0100 (Wed, 30 Nov 2011) New Revision: 2247 Url: http://nuiton.org/repositories/revision/nuiton-utils/2247 Log: [maven-release-plugin] prepare for next development iteration Modified: trunk/nuiton-csv/pom.xml trunk/nuiton-profiling/pom.xml trunk/nuiton-utils/pom.xml trunk/nuiton-validator/pom.xml trunk/pom.xml Modified: trunk/nuiton-csv/pom.xml =================================================================== --- trunk/nuiton-csv/pom.xml 2011-11-30 17:19:53 UTC (rev 2246) +++ trunk/nuiton-csv/pom.xml 2011-11-30 17:19:54 UTC (rev 2247) @@ -31,7 +31,7 @@ <parent> <groupId>org.nuiton</groupId> <artifactId>nuiton-utils-parent</artifactId> - <version>2.4</version> + <version>2.4.1-SNAPSHOT</version> </parent> <artifactId>nuiton-csv</artifactId> Modified: trunk/nuiton-profiling/pom.xml =================================================================== --- trunk/nuiton-profiling/pom.xml 2011-11-30 17:19:53 UTC (rev 2246) +++ trunk/nuiton-profiling/pom.xml 2011-11-30 17:19:54 UTC (rev 2247) @@ -33,7 +33,7 @@ <parent> <groupId>org.nuiton</groupId> <artifactId>nuiton-utils-parent</artifactId> - <version>2.4</version> + <version>2.4.1-SNAPSHOT</version> </parent> <artifactId>nuiton-profiling</artifactId> Modified: trunk/nuiton-utils/pom.xml =================================================================== --- trunk/nuiton-utils/pom.xml 2011-11-30 17:19:53 UTC (rev 2246) +++ trunk/nuiton-utils/pom.xml 2011-11-30 17:19:54 UTC (rev 2247) @@ -33,7 +33,7 @@ <parent> <groupId>org.nuiton</groupId> <artifactId>nuiton-utils-parent</artifactId> - <version>2.4</version> + <version>2.4.1-SNAPSHOT</version> </parent> <artifactId>nuiton-utils</artifactId> Modified: trunk/nuiton-validator/pom.xml =================================================================== --- trunk/nuiton-validator/pom.xml 2011-11-30 17:19:53 UTC (rev 2246) +++ trunk/nuiton-validator/pom.xml 2011-11-30 17:19:54 UTC (rev 2247) @@ -33,7 +33,7 @@ <parent> <groupId>org.nuiton</groupId> <artifactId>nuiton-utils-parent</artifactId> - <version>2.4</version> + <version>2.4.1-SNAPSHOT</version> </parent> <artifactId>nuiton-validator</artifactId> Modified: trunk/pom.xml =================================================================== --- trunk/pom.xml 2011-11-30 17:19:53 UTC (rev 2246) +++ trunk/pom.xml 2011-11-30 17:19:54 UTC (rev 2247) @@ -35,7 +35,7 @@ </parent> <artifactId>nuiton-utils-parent</artifactId> - <version>2.4</version> + <version>2.4.1-SNAPSHOT</version> <modules> <module>nuiton-utils</module> @@ -211,12 +211,12 @@  <scm> <connection> - scm:svn:http://svn.nuiton.org/svn/nuiton-utils/tags/nuiton-utils-2.4 + scm:svn:http://svn.nuiton.org/svn/nuiton-utils/trunk </connection> <developerConnection> - scm:svn:http://svn.nuiton.org/svn/nuiton-utils/tags/nuiton-utils-2.4 + scm:svn:http://svn.nuiton.org/svn/nuiton-utils/trunk </developerConnection> - <url>http://www.nuiton.org/repositories/browse/nuiton-utils/tags/nuiton-utils-2.4</url> + <url>http://www.nuiton.org/repositories/browse/nuiton-utils/trunk</url> </scm> <distributionManagement>

1 0

r2246 - in tags: . nuiton-utils-2.4
by maven-release＠users.nuiton.org Nov. 30, 2011

Nov. 30, 2011

Author: maven-release Date: 2011-11-30 18:19:53 +0100 (Wed, 30 Nov 2011) New Revision: 2246 Url: http://nuiton.org/repositories/revision/nuiton-utils/2246 Log: [maven-release-plugin] copy for tag nuiton-utils-2.4 Added: tags/nuiton-utils-2.4/ Property changes on: tags/nuiton-utils-2.4 ___________________________________________________________________ Added: svn:ignore + target velocity.log maven.log .eclipse .classpath *.iml *.ipr *.iws .settings .project testdir Added: svn:mergeinfo +

1 0

r2245 - in trunk: . nuiton-csv nuiton-profiling nuiton-utils nuiton-validator
by maven-release＠users.nuiton.org Nov. 30, 2011

Nov. 30, 2011

Author: maven-release Date: 2011-11-30 18:19:52 +0100 (Wed, 30 Nov 2011) New Revision: 2245 Url: http://nuiton.org/repositories/revision/nuiton-utils/2245 Log: [maven-release-plugin] prepare release nuiton-utils-2.4 Modified: trunk/nuiton-csv/pom.xml trunk/nuiton-profiling/pom.xml trunk/nuiton-utils/pom.xml trunk/nuiton-validator/pom.xml trunk/pom.xml Modified: trunk/nuiton-csv/pom.xml =================================================================== --- trunk/nuiton-csv/pom.xml 2011-11-30 17:12:49 UTC (rev 2244) +++ trunk/nuiton-csv/pom.xml 2011-11-30 17:19:52 UTC (rev 2245) @@ -31,7 +31,7 @@ <parent> <groupId>org.nuiton</groupId> <artifactId>nuiton-utils-parent</artifactId> - <version>2.4-SNAPSHOT</version> + <version>2.4</version> </parent> <artifactId>nuiton-csv</artifactId> Modified: trunk/nuiton-profiling/pom.xml =================================================================== --- trunk/nuiton-profiling/pom.xml 2011-11-30 17:12:49 UTC (rev 2244) +++ trunk/nuiton-profiling/pom.xml 2011-11-30 17:19:52 UTC (rev 2245) @@ -33,7 +33,7 @@ <parent> <groupId>org.nuiton</groupId> <artifactId>nuiton-utils-parent</artifactId> - <version>2.4-SNAPSHOT</version> + <version>2.4</version> </parent> <artifactId>nuiton-profiling</artifactId> Modified: trunk/nuiton-utils/pom.xml =================================================================== --- trunk/nuiton-utils/pom.xml 2011-11-30 17:12:49 UTC (rev 2244) +++ trunk/nuiton-utils/pom.xml 2011-11-30 17:19:52 UTC (rev 2245) @@ -33,7 +33,7 @@ <parent> <groupId>org.nuiton</groupId> <artifactId>nuiton-utils-parent</artifactId> - <version>2.4-SNAPSHOT</version> + <version>2.4</version> </parent> <artifactId>nuiton-utils</artifactId> Modified: trunk/nuiton-validator/pom.xml =================================================================== --- trunk/nuiton-validator/pom.xml 2011-11-30 17:12:49 UTC (rev 2244) +++ trunk/nuiton-validator/pom.xml 2011-11-30 17:19:52 UTC (rev 2245) @@ -33,7 +33,7 @@ <parent> <groupId>org.nuiton</groupId> <artifactId>nuiton-utils-parent</artifactId> - <version>2.4-SNAPSHOT</version> + <version>2.4</version> </parent> <artifactId>nuiton-validator</artifactId> Modified: trunk/pom.xml =================================================================== --- trunk/pom.xml 2011-11-30 17:12:49 UTC (rev 2244) +++ trunk/pom.xml 2011-11-30 17:19:52 UTC (rev 2245) @@ -35,7 +35,7 @@ </parent> <artifactId>nuiton-utils-parent</artifactId> - <version>2.4-SNAPSHOT</version> + <version>2.4</version> <modules> <module>nuiton-utils</module> @@ -211,12 +211,12 @@  <scm> <connection> - scm:svn:http://svn.nuiton.org/svn/nuiton-utils/trunk + scm:svn:http://svn.nuiton.org/svn/nuiton-utils/tags/nuiton-utils-2.4 </connection> <developerConnection> - scm:svn:http://svn.nuiton.org/svn/nuiton-utils/trunk + scm:svn:http://svn.nuiton.org/svn/nuiton-utils/tags/nuiton-utils-2.4 </developerConnection> - <url>http://www.nuiton.org/repositories/browse/nuiton-utils/trunk</url> + <url>http://www.nuiton.org/repositories/browse/nuiton-utils/tags/nuiton-utils-2.4</url> </scm> <distributionManagement>

1 0

r2244 - in trunk/nuiton-csv/src: main/java/org/nuiton/util/csv main/resources/i18n test/java/org/nuiton/util/csv test/resources/org/nuiton/util/csv
by tchemit＠users.nuiton.org Nov. 30, 2011

Nov. 30, 2011

Author: tchemit Date: 2011-11-30 18:12:49 +0100 (Wed, 30 Nov 2011) New Revision: 2244 Url: http://nuiton.org/repositories/revision/nuiton-utils/2244 Log: - improve the Import design - introduce a ImportMap to import into a map (usefull when there is no direct bean for each line of a csv import) Added: trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/ImportToMap.java Modified: trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/Common.java trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/Import.java trunk/nuiton-csv/src/main/resources/i18n/nuiton-csv_en_GB.properties trunk/nuiton-csv/src/main/resources/i18n/nuiton-csv_es_ES.properties trunk/nuiton-csv/src/main/resources/i18n/nuiton-csv_fr_FR.properties trunk/nuiton-csv/src/test/java/org/nuiton/util/csv/ImportTest.java trunk/nuiton-csv/src/test/resources/org/nuiton/util/csv/importTest.csv Modified: trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/Common.java =================================================================== --- trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/Common.java 2011-11-30 09:00:26 UTC (rev 2243) +++ trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/Common.java 2011-11-30 17:12:49 UTC (rev 2244) @@ -50,7 +50,7 @@ public static final ValueParserFormatter<String> STRING = new StringValueParser(); - + public static final ValueFormatter<?> TO_STRING_FORMATTER = new ToStringValueFormatter(); @@ -100,6 +100,14 @@ public static final ValueParserFormatter<Date> WEEK = new DateValue("w/yyyy"); + public static <E extends Map<String, Object>, T> MapProperty<E, T> newMapProperty(String propertyName) { + return new MapProperty<E, T>(propertyName); + } + + public static <E , T> BeanProperty<E, T> newBeanProperty(String propertyName) { + return new BeanProperty<E, T>(propertyName); + } + public static class StringValueParser implements ValueParserFormatter<String> { @Override @@ -117,7 +125,7 @@ @Override public String format(Object value) { - return value==null?"":value.toString(); + return value == null ? "" : value.toString(); } } @@ -141,6 +149,26 @@ } } + public static class MapProperty<E extends Map<String, Object>, T> implements ValueGetterSetter<E, T> { + + protected String propertyName; + + MapProperty(String propertyName) { + this.propertyName = propertyName; + } + + @Override + public void set(E bean, T value) throws Exception { + bean.put(propertyName, value); + } + + @Override + public T get(E object) throws Exception { + T value = (T) object.get(propertyName); + return value; + } + } + public static class DateValue implements ValueParserFormatter<Date> { protected DateFormat dateFormat; Modified: trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/Import.java =================================================================== --- trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/Import.java 2011-11-30 09:00:26 UTC (rev 2243) +++ trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/Import.java 2011-11-30 17:12:49 UTC (rev 2244) @@ -24,6 +24,12 @@ */ package org.nuiton.util.csv; +import com.csvreader.CsvReader; +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.nuiton.util.StringUtil; + +import java.io.Closeable; import java.io.IOException; import java.io.InputStream; import java.io.Reader; @@ -38,31 +44,40 @@ import java.util.NoSuchElementException; import java.util.Set; -import com.csvreader.CsvReader; -import org.apache.commons.logging.Log; -import org.apache.commons.logging.LogFactory; -import org.nuiton.util.StringUtil; - import static org.nuiton.i18n.I18n._; /** - * TODO + * Import engine for a given import model. * + * It acts as an {@link Iterable}, you can use directly inside a foreach. + * + * The method {@link #prepareAndValidate()} will be invoked before all and + * only once. It mainly obtain header from the csv input, pass it to the model + * and then validate the model. + * * @author bleny <leny(a)codelutin.com> * @author tchemit <chemit(a)codelutin.com> + * @author fdesbois <desbois(a)codelutin.com> * @since 2.4 */ -public class Import<E> implements Iterable<E> { +public class Import<E> implements Iterable<E>, Closeable { /** Logger. */ private static final Log log = LogFactory.getLog(Import.class); + /** Csv import model. */ protected ImportModel<E> model; + /** Csv reader (this is the input). */ protected CsvReader reader; - /** save once done to prevent multiple headers read leading to consider first lines as headers */ - protected ValidationResult csvValidationResult; + /** + * A flag to know if model was already validated. + * + * Save once done to prevent multiple headers read leading to consider + * first lines as headers. + */ + protected boolean validate; public static <E> Import<E> newImport(ImportModel<E> model, InputStream inputStream) { @@ -74,87 +89,13 @@ return new Import<E>(model, reader); } - public ValidationResult validateCsv() throws IOException { + @Override + public Iterator<E> iterator() { - if (csvValidationResult == null) { + prepareAndValidate(); - boolean canReadHeaders = reader.readHeaders(); - if (!canReadHeaders) { - throw new IOException("can't read headers"); - } + readFirstLine(); - if (log.isTraceEnabled()) { - log.trace("headers of the CSV file are : " + Arrays.toString(reader.getHeaders())); - } - - model.pushCsvHeaderNames(Arrays.asList(reader.getHeaders())); - - validateModel(); - - csvValidationResult = new ValidationResult(); - csvValidationResult.setSuccess(true); - - List<String> csvHeaders = new ArrayList<String>(); - Collections.addAll(csvHeaders, reader.getHeaders()); - - for (ImportableColumn<E, ?> field : model.getColumnsForImport()) { - csvHeaders.remove(field.getHeaderName()); - } - - if (!csvHeaders.isEmpty()) { - csvValidationResult.setSuccess(false); - List<String> validHeaderNames = new LinkedList<String>(); - for (ImportableColumn<E, ?> importableColumn : model.getColumnsForImport()) { - validHeaderNames.add(importableColumn.getHeaderName()); - } - String validationMessage = _( - "csv.import.error.unrecognizedHeaders", - StringUtil.join(csvHeaders, ", ", true), - StringUtil.join(validHeaderNames, ", ", true)); - csvValidationResult.setMessage(validationMessage); - } - - csvHeaders = new ArrayList<String>(); - Collections.addAll(csvHeaders, reader.getHeaders()); - - List<String> mandatoryHeadersNames = new ArrayList<String>(); - for (ImportableColumn<E, ?> field : getAllMandatoryHeaders()) { - mandatoryHeadersNames.add(field.getHeaderName()); - } - mandatoryHeadersNames.removeAll(csvHeaders); - - if (!mandatoryHeadersNames.isEmpty()) { - csvValidationResult.setSuccess(false); - String validationMessage = _( - "csv.import.error.missingMandatoryHeaders", - StringUtil.join(mandatoryHeadersNames, ", ", true)); - csvValidationResult.setMessage(validationMessage); - } - } - - return csvValidationResult; - } - - public Iterator<E> startImport() throws IOException { - validateHeaders(); - Iterator<E> result = iterator(); - return result; - } - - public void validateHeaders() - throws IOException { - - ValidationResult validationResult = validateCsv(); - - if (!validationResult.isSuccess()) { - throw new IOException(validationResult.getMessage()); - } - } - - @Override - public Iterator<E> iterator() { - // XXX-fdesbois-2011-11-28 : it will be better to have a flag in ImportModel to know if headers are present or not - readFirstLine(); return new Iterator<E>() { boolean hasNext = true; @@ -188,7 +129,9 @@ value = reader.get(field.getHeaderName()); } catch (Exception e) { reader.close(); - throw new ImportRuntimeException(_("csv.import.error.unableToReadField", field.getHeaderName(), lineNumber), e); + throw new ImportRuntimeException( + _("csv.import.error.unableToReadField", + field.getHeaderName(), lineNumber), e); } // contravariance ftw @@ -196,8 +139,9 @@ try { parsedValue = field.parseValue(value); } catch (Exception e) { - String message = _("csv.import.error.unableToParseValue", value, field.getHeaderName(), lineNumber) - + "\n" + e.getMessage(); + String message = _("csv.import.error.unableToParseValue", + value, field.getHeaderName(), lineNumber) + + "\n" + e.getMessage(); throw new ImportRuntimeException(message, e); } @@ -206,9 +150,9 @@ field.setValue(element, parsedValue); } catch (Exception e) { String message = _("csv.import.error.unableToSetValue", - parsedValue, - element.toString(), - lineNumber, field.getHeaderName()); + parsedValue, + element.toString(), + lineNumber, field.getHeaderName()); if (log.isErrorEnabled()) { log.error(message); } @@ -223,7 +167,8 @@ } catch (IOException e) { reader.close(); throw new ImportRuntimeException( - _("csv.import.error.unableToReadLine", lineNumber + 1), e); + _("csv.import.error.unableToReadLine", + lineNumber + 1), e); } return lastElement; @@ -233,14 +178,135 @@ public void remove() { throw new UnsupportedOperationException(); } - }; } - public void stopImport() { + @Override + public void close() { reader.close(); } + public void prepareAndValidate() { + + if (validate) { + + // was already validated + return; + } + + // mark as validated + validate = true; + + // obtains headers + String[] headers = getHeaders(); + + if (log.isTraceEnabled()) { + log.trace("headers of the CSV file are : " + + Arrays.toString(headers)); + } + + // hook to do some stuff from the model + model.pushCsvHeaderNames(Arrays.asList(headers)); + + // check model columns name are unique + checkUniqueModelColumnNames(); + + // check that given headers from csv file are all known + + checkHeaderNamesAreAllKnown(headers); + + // check all mandatories column are on csv header + checkAllManadatoryHeadersArePresent(headers); + } + + protected void checkHeaderNamesAreAllKnown(String[] headers) { + List<String> csvHeaders = new ArrayList<String>(); + Collections.addAll(csvHeaders, headers); + + for (ImportableColumn<E, ?> field : model.getColumnsForImport()) { + csvHeaders.remove(field.getHeaderName()); + } + if (!csvHeaders.isEmpty()) { + List<String> validHeaderNames = new LinkedList<String>(); + for (ImportableColumn<E, ?> importableColumn : + model.getColumnsForImport()) { + validHeaderNames.add(importableColumn.getHeaderName()); + } + String validationMessage = _("csv.import.error.unrecognizedHeaders", + StringUtil.join(csvHeaders, ", ", true), + StringUtil.join(validHeaderNames, ", ", true)); + throw new ImportRuntimeException(validationMessage); + } + } + + protected void checkUniqueModelColumnNames() { + Set<String> headerNames = new HashSet<String>(); + Set<String> doubleHeaderNames = new HashSet<String>(); + for (ImportableColumn<E, ?> importableColumn : + model.getColumnsForImport()) { + String headerName = importableColumn.getHeaderName(); + boolean alreadyUsed = !headerNames.add(headerName); + if (alreadyUsed) { + doubleHeaderNames.add(headerName); + } + } + if (!doubleHeaderNames.isEmpty()) { + String message = _("csv.import.error.duplicatedHeaders", + StringUtil.join(doubleHeaderNames, ", ", true)); + + throw new ImportRuntimeException( + message); + } + } + + protected void checkAllManadatoryHeadersArePresent(String[] headers) { + + List<String> csvHeaders = new ArrayList<String>(); + Collections.addAll(csvHeaders, headers); + + List<String> mandatoryHeadersNames = new ArrayList<String>(); + for (ImportableColumn<E, ?> field : getAllMandatoryHeaders()) { + mandatoryHeadersNames.add(field.getHeaderName()); + } + mandatoryHeadersNames.removeAll(csvHeaders); + + if (!mandatoryHeadersNames.isEmpty()) { + String validationMessage = _("csv.import.error.missingMandatoryHeaders", + StringUtil.join(mandatoryHeadersNames, ", ", true)); + throw new ImportRuntimeException(validationMessage); + } + + Set<String> headerNames = new HashSet<String>(); + for (ImportableColumn<E, ?> importableColumn : + model.getColumnsForImport()) { + String headerName = importableColumn.getHeaderName(); + boolean alreadyUsed = !headerNames.add(headerName); + if (alreadyUsed) { + throw new ImportRuntimeException( + "model contains multiple columnsForImport named '" + + headerName + "'"); + } + } + } + + protected String[] getHeaders() throws ImportRuntimeException { + try { + boolean canReadHeaders = reader.readHeaders(); + if (!canReadHeaders) { + throw new ImportRuntimeException("can't read headers"); + } + } catch (IOException e) { + throw new ImportRuntimeException("can't read headers"); + } + + try { + String[] result = reader.getHeaders(); + return result; + } catch (IOException eee) { + throw new ImportRuntimeException("can't get headers", eee); + } + } + protected List<ImportableColumn<E, Object>> getNonIgnoredHeaders() { List<ImportableColumn<E, Object>> nonIgnoredHeaders = new ArrayList<ImportableColumn<E, Object>>(); for (ImportableColumn<E, Object> field : model.getColumnsForImport()) { @@ -279,18 +345,6 @@ this.reader.setTrimWhitespace(true); } - protected boolean validateModel() { - Set<String> headerNames = new HashSet<String>(); - for (ImportableColumn<E, ?> importableColumn : model.getColumnsForImport()) { - String headerName = importableColumn.getHeaderName(); - boolean alreadyUsed = !headerNames.add(headerName); - if (alreadyUsed) { - throw new IllegalArgumentException("model contains multiple columnsForImport named '" + headerName + "'"); - } - } - return true; - } - protected void readFirstLine() throws ImportRuntimeException { try { boolean emptyFile = !reader.readRecord(); @@ -304,5 +358,4 @@ } } - } Added: trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/ImportToMap.java =================================================================== --- trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/ImportToMap.java (rev 0) +++ trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/ImportToMap.java 2011-11-30 17:12:49 UTC (rev 2244) @@ -0,0 +1,165 @@ +/* + * #%L + * Nuiton Utils :: Nuiton Csv + * + * $Id$ + * $HeadURL$ + * %% + * Copyright (C) 2011 CodeLutin + * %% + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Lesser Public License for more details. + * + * You should have received a copy of the GNU General Lesser Public + * License along with this program. If not, see + * <http://www.gnu.org/licenses/lgpl-3.0.html>. + * #L% + */ +package org.nuiton.util.csv; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; + +import java.io.IOException; +import java.io.InputStream; +import java.io.Reader; +import java.util.HashMap; +import java.util.Iterator; +import java.util.Map; +import java.util.NoSuchElementException; + +import static org.nuiton.i18n.I18n._; + +/** + * A extended {@link Import} to read csv lines into a single map. + * + * Warning: The map used to push values for a csv line is the + * same for all lines, it means you have to copy to your own object. + * + * @author tchemit <chemit(a)codelutin.com> + * @since 2.4 + */ +public class ImportToMap extends Import<Map<String, Object>> { + + /** Logger. */ + private static final Log log = LogFactory.getLog(ImportToMap.class); + + public static ImportToMap newImport(ImportModel<Map<String, Object>> model, + InputStream inputStream) { + return new ImportToMap(model, inputStream); + } + + public static ImportToMap newImport(ImportModel<Map<String, Object>> model, + Reader reader) { + return new ImportToMap(model, reader); + } + + @Override + public Iterator<Map<String, Object>> iterator() { + + // obtain headers from csv input and validate the model + prepareAndValidate(); + + // read first line since first line is header + readFirstLine(); + + return new Iterator<Map<String, Object>>() { + + boolean hasNext = true; + + int lineNumber; + + Map<String, Object> element = new HashMap<String, Object>(); + + @Override + public boolean hasNext() { + return hasNext; + } + + @Override + public Map<String, Object> next() + throws NoSuchElementException, ImportRuntimeException { + + if (!hasNext) { + throw new NoSuchElementException(); + } + + lineNumber += 1; + + // clean all values from the element + element.clear(); + + for (ImportableColumn<Map<String, Object>, Object> field : + getNonIgnoredHeaders()) { + + // read value from csv cell + String value; + try { + value = reader.get(field.getHeaderName()); + } catch (Exception e) { + reader.close(); + throw new ImportRuntimeException(_("csv.import.error.unableToReadField", + field.getHeaderName(), lineNumber), e); + } + + // contravariance ftw + Object parsedValue; + try { + parsedValue = field.parseValue(value); + } catch (Exception e) { + String message = _("csv.import.error.unableToParseValue", + value, field.getHeaderName(), lineNumber) + + "\n" + e.getMessage(); + throw new ImportRuntimeException(message, e); + } + + // set value to element + try { + field.setValue(element, parsedValue); + } catch (Exception e) { + String message = _("csv.import.error.unableToSetValue", + parsedValue, + element.toString(), + lineNumber, field.getHeaderName()); + if (log.isErrorEnabled()) { + log.error(message); + } + throw new ImportRuntimeException(message, e); + } + } + + try { + hasNext = reader.readRecord(); + } catch (IOException e) { + reader.close(); + throw new ImportRuntimeException( + _("csv.import.error.unableToReadLine", lineNumber + 1), e); + } + + return element; + } + + @Override + public void remove() { + throw new UnsupportedOperationException(); + } + }; + } + + protected ImportToMap(ImportModel<Map<String, Object>> mapImportModel, + InputStream inputStream) { + super(mapImportModel, inputStream); + } + + protected ImportToMap(ImportModel<Map<String, Object>> mapImportModel, + Reader reader) { + super(mapImportModel, reader); + } +} Property changes on: trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/ImportToMap.java ___________________________________________________________________ Added: svn:keywords + Author Date Id Revision HeadURL Added: svn:eol-style + native Modified: trunk/nuiton-csv/src/main/resources/i18n/nuiton-csv_en_GB.properties =================================================================== --- trunk/nuiton-csv/src/main/resources/i18n/nuiton-csv_en_GB.properties 2011-11-30 09:00:26 UTC (rev 2243) +++ trunk/nuiton-csv/src/main/resources/i18n/nuiton-csv_en_GB.properties 2011-11-30 17:12:49 UTC (rev 2244) @@ -1,4 +1,7 @@ +csv.import.error.duplicatedHeaders=Fields %s are duplicated. +csv.import.error.missingMandatoryHeaders=The mandatory fields %s are missing csv.import.error.unableToParseValue=Unable to parse value '%s' (column '%s', line %s) csv.import.error.unableToReadField=Unable to read value of column '%s' at line %s csv.import.error.unableToReadLine=Unable to read line %s -csv.import.error.unableToSetValue=Unable to set value '%s' (entity '%s', line %s, column '%s') +csv.import.error.unableToSetValue=Unable to set value '%s' (object '%s', line %s, column '%s') +csv.import.error.unrecognizedHeaders=Fields %s are not recognized. Accepted fields are %s. Modified: trunk/nuiton-csv/src/main/resources/i18n/nuiton-csv_es_ES.properties =================================================================== --- trunk/nuiton-csv/src/main/resources/i18n/nuiton-csv_es_ES.properties 2011-11-30 09:00:26 UTC (rev 2243) +++ trunk/nuiton-csv/src/main/resources/i18n/nuiton-csv_es_ES.properties 2011-11-30 17:12:49 UTC (rev 2244) @@ -1,4 +1,7 @@ +csv.import.error.duplicatedHeaders=Fields %s are duplicated. +csv.import.error.missingMandatoryHeaders=The mandatory fields %s are missing csv.import.error.unableToParseValue=Unable to parse value '%s' (column '%s', line %s) csv.import.error.unableToReadField=Unable to read value of column '%s' at line %s csv.import.error.unableToReadLine=Unable to read line %s -csv.import.error.unableToSetValue=Unable to set value '%s' (entity '%s', line %s, column '%s') +csv.import.error.unableToSetValue=Unable to set value '%s' (object'%s', line %s, column '%s') +csv.import.error.unrecognizedHeaders=Fields %s are not recognized. Accepted fields are %s. Modified: trunk/nuiton-csv/src/main/resources/i18n/nuiton-csv_fr_FR.properties =================================================================== --- trunk/nuiton-csv/src/main/resources/i18n/nuiton-csv_fr_FR.properties 2011-11-30 09:00:26 UTC (rev 2243) +++ trunk/nuiton-csv/src/main/resources/i18n/nuiton-csv_fr_FR.properties 2011-11-30 17:12:49 UTC (rev 2244) @@ -1,4 +1,7 @@ +csv.import.error.duplicatedHeaders=Les champs %s sont dupliqués. +csv.import.error.missingMandatoryHeaders=Les champs obligatoires %s sont manquants csv.import.error.unableToParseValue=Erreur lors de l'interprétation de la valeur '%s' (colonne '%s', ligne %s) csv.import.error.unableToReadField=Impossible de lire la colonne '%s' à la ligne %s csv.import.error.unableToReadLine=Impossible de lire la ligne %s -csv.import.error.unableToSetValue=Impossible d'enregistrer la valeur '%s' (entité '%s', ligne %s, column '%s') +csv.import.error.unableToSetValue=Impossible d'enregistrer la valeur '%s' (objet '%s', ligne %s, column '%s') +csv.import.error.unrecognizedHeaders=Les champs %s ne sont pas reconnus. Les champs possibles sont %s. Modified: trunk/nuiton-csv/src/test/java/org/nuiton/util/csv/ImportTest.java =================================================================== --- trunk/nuiton-csv/src/test/java/org/nuiton/util/csv/ImportTest.java 2011-11-30 09:00:26 UTC (rev 2243) +++ trunk/nuiton-csv/src/test/java/org/nuiton/util/csv/ImportTest.java 2011-11-30 17:12:49 UTC (rev 2244) @@ -21,7 +21,14 @@ */ package org.nuiton.util.csv; -import java.io.IOException; +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.junit.Assert; +import org.junit.Before; +import org.junit.Test; +import org.nuiton.util.DateUtil; +import org.nuiton.util.csv.Common.BeanProperty; + import java.io.InputStream; import java.text.DateFormat; import java.text.ParseException; @@ -32,14 +39,6 @@ import java.util.Iterator; import java.util.List; -import org.apache.commons.logging.Log; -import org.apache.commons.logging.LogFactory; -import org.junit.Assert; -import org.junit.Before; -import org.junit.Test; -import org.nuiton.util.DateUtil; -import org.nuiton.util.csv.Common.BeanProperty; - /** * Created on 28/11/11 * @@ -60,8 +59,7 @@ * @throws Exception for errors */ @Test - public void testSimpleImport() - throws Exception { + public void testSimpleImport() throws Exception { execute(new SimpleImportModel(dateFormat)); } @@ -71,18 +69,16 @@ * @throws Exception for errors */ @Test - public void testColumnImport() - throws Exception { + public void testColumnImport() throws Exception { execute(new ColumnImportModel(dateFormat)); } - private void execute(ImportModel<Row> model) - throws IOException { + private void execute(ImportModel<Row> model) { InputStream csvStream = getClass().getResourceAsStream("importTest.csv"); Import<Row> rowImport = Import.newImport(model, csvStream); try { - Iterator<Row> it = rowImport.startImport(); + Iterator<Row> it = rowImport.iterator(); List<Row> rows = newList(it); Assert.assertEquals(3, rows.size()); Row row1 = rows.get(0); @@ -98,13 +94,13 @@ Assert.assertEquals(new Integer(21), row3.getNumber()); Assert.assertEquals("3ème ligne", row3.getTitle()); } finally { - rowImport.stopImport(); + rowImport.close(); } } private static <T> List<T> newList(Iterator<T> iterator) { List<T> result = new ArrayList<T>(); - while(iterator.hasNext()) { + while (iterator.hasNext()) { result.add(iterator.next()); } return result; @@ -274,14 +270,13 @@ } @Override - public String parseValue(String value) - throws ParseException { + public String parseValue(String value) { return value; } @Override public void update(Row object, - String value) + String value) throws Exception { object.setTitle(value); } @@ -340,8 +335,7 @@ } @Override - public Integer parseValue(String value) - throws ParseException { + public Integer parseValue(String value) { return Integer.parseInt(value); } @@ -365,7 +359,7 @@ } protected abstract void update(E object, F value) - throws Exception; + throws Exception; } } // $Id$ Property changes on: trunk/nuiton-csv/src/test/java/org/nuiton/util/csv/ImportTest.java ___________________________________________________________________ Added: svn:keywords + Author Date Id Revision HeadURL Added: svn:eol-style + native Property changes on: trunk/nuiton-csv/src/test/resources/org/nuiton/util/csv/importTest.csv ___________________________________________________________________ Added: svn:keywords + Author Date Id Revision HeadURL Added: svn:eol-style + native

1 0

r2243 - trunk/nuiton-utils/src/main/java/org/nuiton/util
by athimel＠users.nuiton.org Nov. 30, 2011

Nov. 30, 2011

Author: athimel Date: 2011-11-30 10:00:26 +0100 (Wed, 30 Nov 2011) New Revision: 2243 Url: http://nuiton.org/repositories/revision/nuiton-utils/2243 Log: Improve Javadoc on RecursiveProperties Modified: trunk/nuiton-utils/src/main/java/org/nuiton/util/RecursiveProperties.java Modified: trunk/nuiton-utils/src/main/java/org/nuiton/util/RecursiveProperties.java =================================================================== --- trunk/nuiton-utils/src/main/java/org/nuiton/util/RecursiveProperties.java 2011-11-30 08:59:00 UTC (rev 2242) +++ trunk/nuiton-utils/src/main/java/org/nuiton/util/RecursiveProperties.java 2011-11-30 09:00:26 UTC (rev 2243) @@ -28,27 +28,28 @@ import java.util.Properties; /** - * Surcharge {@link Properties} de manière à aller chercher dans les - * properties la valeur de la propriété si celle ci est encadrée par "${...}". + * Overrides {@link Properties} in order to check if the expected value + * contains another property key like "${...}". It that case, the key + * will be replaced by its value if possible. * - * Exemple : + * Example : * <pre> * myFirstName=Arnaud * myName=Thimel * org.nuiton.topia.userInfo.fullName=${fullName} * fullName=${myFirstName} ${myName} * namePhrase=My name is ${myName}. - * instruction=Placez votre texte comme ceci : ${monTexte} + * instruction=Put your text like this : ${myText} * </pre> * * Dans ce cas, * <ul> * <li>getProperty("org.nuiton.topia.userInfo.fullName") renverra "Arnaud Thimel" * <li>getProperty("namePhrase") renverra "My name is Thimel." - * <li>getProperty("instruction") renverra "Placez votre texte comme ceci : ${monTexte}" + * <li>getProperty("instruction") renverra "Put your text like this : ${myText}" * </ul> * - * @author thimel + * @author Arnaud Thimel <thimel(a)codelutin.com> */ public class RecursiveProperties extends Properties {

1 0

r2242 - trunk/nuiton-validator/src/test/java/org/nuiton/validator/xwork2/field
by athimel＠users.nuiton.org Nov. 30, 2011

Nov. 30, 2011

Author: athimel Date: 2011-11-30 09:59:00 +0100 (Wed, 30 Nov 2011) New Revision: 2242 Url: http://nuiton.org/repositories/revision/nuiton-utils/2242 Log: Anomalie #1838 Improve tests and fix test comments Modified: trunk/nuiton-validator/src/test/java/org/nuiton/validator/xwork2/field/VATIdentificationNumberFieldValidatorTest.java Modified: trunk/nuiton-validator/src/test/java/org/nuiton/validator/xwork2/field/VATIdentificationNumberFieldValidatorTest.java =================================================================== --- trunk/nuiton-validator/src/test/java/org/nuiton/validator/xwork2/field/VATIdentificationNumberFieldValidatorTest.java 2011-11-30 08:56:34 UTC (rev 2241) +++ trunk/nuiton-validator/src/test/java/org/nuiton/validator/xwork2/field/VATIdentificationNumberFieldValidatorTest.java 2011-11-30 08:59:00 UTC (rev 2242) @@ -58,11 +58,16 @@ assertFieldInError(Company.PROPERTY_VAT_NUMBER, "company.vat.format", true); - // Not valid vat, spaces in it + // Valid vat, with spaces in it bean.setVat("FR 57 442116703"); assertFieldInError(Company.PROPERTY_VAT_NUMBER, "company.vat.format", false); + // Real vat, with spaces in it + bean.setVat("FR 39 535 198 188"); + assertFieldInError(Company.PROPERTY_VAT_NUMBER, "company.vat.format", + false); + // Use required string validator for that bean.setVat(""); assertFieldInError(Company.PROPERTY_VAT_NUMBER, "company.vat.format",

1 0

r2241 - in trunk/nuiton-validator/src: main/java/org/nuiton/validator/xwork2/field test/java/org/nuiton/validator/xwork2/field
by athimel＠users.nuiton.org Nov. 30, 2011

Nov. 30, 2011

Author: athimel Date: 2011-11-30 09:56:34 +0100 (Wed, 30 Nov 2011) New Revision: 2241 Url: http://nuiton.org/repositories/revision/nuiton-utils/2241 Log: Anomalie #1838 VAT identification number validator does not support spaces Modified: trunk/nuiton-validator/src/main/java/org/nuiton/validator/xwork2/field/VATIdentificationNumberFieldValidator.java trunk/nuiton-validator/src/test/java/org/nuiton/validator/xwork2/field/VATIdentificationNumberFieldValidatorTest.java Modified: trunk/nuiton-validator/src/main/java/org/nuiton/validator/xwork2/field/VATIdentificationNumberFieldValidator.java =================================================================== --- trunk/nuiton-validator/src/main/java/org/nuiton/validator/xwork2/field/VATIdentificationNumberFieldValidator.java 2011-11-30 08:52:51 UTC (rev 2240) +++ trunk/nuiton-validator/src/main/java/org/nuiton/validator/xwork2/field/VATIdentificationNumberFieldValidator.java 2011-11-30 08:56:34 UTC (rev 2241) @@ -53,11 +53,17 @@ return; } if (value instanceof String) { - if (StringUtils.isEmpty((String) value)) { + + String vatNumber = (String) value; + if (StringUtils.isEmpty(vatNumber)) { // no value defined return; } - Matcher m = p.matcher((String) value); + + // Remove any space + vatNumber = vatNumber.replaceAll(" ", ""); + + Matcher m = p.matcher(vatNumber); if (!m.matches()) { addFieldError(fieldName, object); } Modified: trunk/nuiton-validator/src/test/java/org/nuiton/validator/xwork2/field/VATIdentificationNumberFieldValidatorTest.java =================================================================== --- trunk/nuiton-validator/src/test/java/org/nuiton/validator/xwork2/field/VATIdentificationNumberFieldValidatorTest.java 2011-11-30 08:52:51 UTC (rev 2240) +++ trunk/nuiton-validator/src/test/java/org/nuiton/validator/xwork2/field/VATIdentificationNumberFieldValidatorTest.java 2011-11-30 08:56:34 UTC (rev 2241) @@ -61,7 +61,7 @@ // Not valid vat, spaces in it bean.setVat("FR 57 442116703"); assertFieldInError(Company.PROPERTY_VAT_NUMBER, "company.vat.format", - true); + false); // Use required string validator for that bean.setVat("");

1 0

r2240 - in trunk/nuiton-validator/src: main/java/org/nuiton/validator/xwork2/field test/java/org/nuiton/validator/xwork2/field
by athimel＠users.nuiton.org Nov. 30, 2011

Nov. 30, 2011

Author: athimel Date: 2011-11-30 09:52:51 +0100 (Wed, 30 Nov 2011) New Revision: 2240 Url: http://nuiton.org/repositories/revision/nuiton-utils/2240 Log: Anomalie #1837 FrenchSiret validator does not support spaces Modified: trunk/nuiton-validator/src/main/java/org/nuiton/validator/xwork2/field/FrenchSiretFieldValidator.java trunk/nuiton-validator/src/test/java/org/nuiton/validator/xwork2/field/FrenchSiretFieldValidatorTest.java Modified: trunk/nuiton-validator/src/main/java/org/nuiton/validator/xwork2/field/FrenchSiretFieldValidator.java =================================================================== --- trunk/nuiton-validator/src/main/java/org/nuiton/validator/xwork2/field/FrenchSiretFieldValidator.java 2011-11-29 14:34:18 UTC (rev 2239) +++ trunk/nuiton-validator/src/main/java/org/nuiton/validator/xwork2/field/FrenchSiretFieldValidator.java 2011-11-30 08:52:51 UTC (rev 2240) @@ -86,6 +86,9 @@ return; } + // Remove any space + siret = siret.replaceAll(" ", ""); + Matcher m = p.matcher(siret); if (!m.matches() || !luhnChecksum(siret)) { addFieldError(fieldName, object); Modified: trunk/nuiton-validator/src/test/java/org/nuiton/validator/xwork2/field/FrenchSiretFieldValidatorTest.java =================================================================== --- trunk/nuiton-validator/src/test/java/org/nuiton/validator/xwork2/field/FrenchSiretFieldValidatorTest.java 2011-11-29 14:34:18 UTC (rev 2239) +++ trunk/nuiton-validator/src/test/java/org/nuiton/validator/xwork2/field/FrenchSiretFieldValidatorTest.java 2011-11-30 08:52:51 UTC (rev 2240) @@ -94,5 +94,11 @@ bean.setSiret(null); assertFieldInError(Company.PROPERTY_SIRET, "company.siret.format", false); + + // Valid siret with spaces + bean.setSiret("535 198 188 00018 "); + assertFieldInError(Company.PROPERTY_SIRET, "company.siret.format", + false); + } }

1 0

r2239 - in trunk/nuiton-csv/src: main/java/org/nuiton/util/csv test/java/org/nuiton/util test/java/org/nuiton/util/csv test/resources test/resources/org test/resources/org/nuiton test/resources/org/nuiton/util test/resources/org/nuiton/util/csv
by fdesbois＠users.nuiton.org Nov. 29, 2011

Nov. 29, 2011

Author: fdesbois Date: 2011-11-29 15:34:18 +0100 (Tue, 29 Nov 2011) New Revision: 2239 Url: http://nuiton.org/repositories/revision/nuiton-utils/2239 Log: #1818: Add unit tests for Import Added: trunk/nuiton-csv/src/test/java/org/nuiton/util/csv/ trunk/nuiton-csv/src/test/java/org/nuiton/util/csv/ImportTest.java trunk/nuiton-csv/src/test/resources/log4j.properties trunk/nuiton-csv/src/test/resources/org/ trunk/nuiton-csv/src/test/resources/org/nuiton/ trunk/nuiton-csv/src/test/resources/org/nuiton/util/ trunk/nuiton-csv/src/test/resources/org/nuiton/util/csv/ trunk/nuiton-csv/src/test/resources/org/nuiton/util/csv/importTest.csv Modified: trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/Import.java Modified: trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/Import.java =================================================================== --- trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/Import.java 2011-11-28 22:21:43 UTC (rev 2238) +++ trunk/nuiton-csv/src/main/java/org/nuiton/util/csv/Import.java 2011-11-29 14:34:18 UTC (rev 2239) @@ -24,11 +24,6 @@ */ package org.nuiton.util.csv; -import com.csvreader.CsvReader; -import org.apache.commons.logging.Log; -import org.apache.commons.logging.LogFactory; -import org.nuiton.util.StringUtil; - import java.io.IOException; import java.io.InputStream; import java.io.Reader; @@ -43,6 +38,11 @@ import java.util.NoSuchElementException; import java.util.Set; +import com.csvreader.CsvReader; +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.nuiton.util.StringUtil; + import static org.nuiton.i18n.I18n._; /** @@ -52,7 +52,7 @@ * @author tchemit <chemit(a)codelutin.com> * @since 2.4 */ -public class Import<E> { +public class Import<E> implements Iterable<E> { /** Logger. */ private static final Log log = LogFactory.getLog(Import.class); @@ -136,17 +136,27 @@ } public Iterator<E> startImport() throws IOException { + validateHeaders(); + Iterator<E> result = iterator(); + return result; + } + public void validateHeaders() + throws IOException { + ValidationResult validationResult = validateCsv(); if (!validationResult.isSuccess()) { throw new IOException(validationResult.getMessage()); } + } + @Override + public Iterator<E> iterator() { + // XXX-fdesbois-2011-11-28 : it will be better to have a flag in ImportModel to know if headers are present or not readFirstLine(); + return new Iterator<E>() { - Iterator<E> result = new Iterator<E>() { - boolean hasNext = true; int lineNumber; @@ -159,7 +169,8 @@ } @Override - public E next() throws NoSuchElementException, ImportRuntimeException { + public E next() + throws NoSuchElementException, ImportRuntimeException { if (!hasNext) { throw new NoSuchElementException(); @@ -186,7 +197,7 @@ parsedValue = field.parseValue(value); } catch (Exception e) { String message = _("csv.import.error.unableToParseValue", value, field.getHeaderName(), lineNumber) - + "\n" + e.getMessage(); + + "\n" + e.getMessage(); throw new ImportRuntimeException(message, e); } @@ -195,9 +206,9 @@ field.setValue(element, parsedValue); } catch (Exception e) { String message = _("csv.import.error.unableToSetValue", - parsedValue, - element.toString(), - lineNumber, field.getHeaderName()); + parsedValue, + element.toString(), + lineNumber, field.getHeaderName()); if (log.isErrorEnabled()) { log.error(message); } @@ -222,8 +233,8 @@ public void remove() { throw new UnsupportedOperationException(); } + }; - return result; } public void stopImport() { Added: trunk/nuiton-csv/src/test/java/org/nuiton/util/csv/ImportTest.java =================================================================== --- trunk/nuiton-csv/src/test/java/org/nuiton/util/csv/ImportTest.java (rev 0) +++ trunk/nuiton-csv/src/test/java/org/nuiton/util/csv/ImportTest.java 2011-11-29 14:34:18 UTC (rev 2239) @@ -0,0 +1,371 @@ +/* + * #%L + * Nuiton Utils :: Nuiton Csv + * %% + * Copyright (C) 2011 CodeLutin + * %% + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Lesser Public License for more details. + * + * You should have received a copy of the GNU General Lesser Public + * License along with this program. If not, see + * <http://www.gnu.org/licenses/lgpl-3.0.html>. + * #L% + */ +package org.nuiton.util.csv; + +import java.io.IOException; +import java.io.InputStream; +import java.text.DateFormat; +import java.text.ParseException; +import java.text.SimpleDateFormat; +import java.util.ArrayList; +import java.util.Collection; +import java.util.Date; +import java.util.Iterator; +import java.util.List; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.junit.Assert; +import org.junit.Before; +import org.junit.Test; +import org.nuiton.util.DateUtil; +import org.nuiton.util.csv.Common.BeanProperty; + +/** + * Created on 28/11/11 + * + * @author fdesbois <florian.desbois(a)wiztivi.com> + */ +public class ImportTest { + + private DateFormat dateFormat; + + @Before + public void setUp() { + dateFormat = new SimpleDateFormat("yyyy-MM-dd"); + } + + /** + * Test with {@link SimpleImportModel} that directly implements necessary {@link ImportableColumn}. + * + * @throws Exception for errors + */ + @Test + public void testSimpleImport() + throws Exception { + execute(new SimpleImportModel(dateFormat)); + } + + /** + * Test with {@link ColumnImportModel} that uses {@link Column} and {@link Common} tools. + * + * @throws Exception for errors + */ + @Test + public void testColumnImport() + throws Exception { + execute(new ColumnImportModel(dateFormat)); + } + + private void execute(ImportModel<Row> model) + throws IOException { + + InputStream csvStream = getClass().getResourceAsStream("importTest.csv"); + Import<Row> rowImport = Import.newImport(model, csvStream); + try { + Iterator<Row> it = rowImport.startImport(); + List<Row> rows = newList(it); + Assert.assertEquals(3, rows.size()); + Row row1 = rows.get(0); + Assert.assertEquals(DateUtil.createDate(5, 12, 2011), row1.getDate()); + Assert.assertEquals(new Integer(18), row1.getNumber()); + Assert.assertEquals("1ère ligne", row1.getTitle()); + Row row2 = rows.get(1); + Assert.assertEquals(DateUtil.createDate(6, 12, 2011), row2.getDate()); + Assert.assertEquals(new Integer(19), row2.getNumber()); + Assert.assertEquals("2ème ligne", row2.getTitle()); + Row row3 = rows.get(2); + Assert.assertEquals(DateUtil.createDate(7, 12, 2011), row3.getDate()); + Assert.assertEquals(new Integer(21), row3.getNumber()); + Assert.assertEquals("3ème ligne", row3.getTitle()); + } finally { + rowImport.stopImport(); + } + } + + private static <T> List<T> newList(Iterator<T> iterator) { + List<T> result = new ArrayList<T>(); + while(iterator.hasNext()) { + result.add(iterator.next()); + } + return result; + } + + public static class Row { + + private Date date; + + private String title; + + private Integer number; + + public Date getDate() { + return date; + } + + public void setDate(Date date) { + this.date = date; + } + + public String getTitle() { + return title; + } + + public void setTitle(String title) { + this.title = title; + } + + public Integer getNumber() { + return number; + } + + public void setNumber(Integer number) { + this.number = number; + } + } + + private static class ColumnImportModel implements ImportModel<Row> { + + private DateFormat dateFormat; + + private ColumnImportModel(DateFormat dateFormat) { + this.dateFormat = dateFormat; + } + + @Override + public char getSeparator() { + return ';'; + } + + @Override + public void pushCsvHeaderNames(List<String> headerNames) { + } + + @Override + public Row newEmptyInstance() { + return new Row(); + } + + @SuppressWarnings({"unchecked"}) + @Override + public Collection<ImportableColumn<Row, Object>> getColumnsForImport() { + List<ImportableColumn<Row, Object>> result = new ArrayList<ImportableColumn<Row, Object>>(); + // Column types are not checked but safe because Object is necessary, and element type is always Row + result.add(getTitleColumn()); + result.add(getNumberColumn()); + result.add(getDateColumn()); + return result; + } + + private Column getTitleColumn() { + return Column.newImportableColumn( + "TITLE", + Common.STRING, + withBeanSetter("title", String.class), + false + ); + } + + private Column getDateColumn() { + return Column.newImportableColumn( + "DATE", + withDateParser(), + withBeanSetter("date", Date.class), + false + ); + } + + private Column getNumberColumn() { + return Column.newImportableColumn( + "NUMBER", + Common.INTEGER, + withBeanSetter("number", Integer.class), + false + ); + } + + @SuppressWarnings({"UnusedParameters"}) + private <T> ValueSetter<Row, T> withBeanSetter(String propertyName, Class<T> propertyClass) { + // propertyClass is not used but useful to check type + return new BeanProperty<Row, T>(propertyName); + } + + private ValueParser<Date> withDateParser() { + return new ValueParser<Date>() { + + @Override + public Date parse(String value) + throws ParseException { + return dateFormat.parse(value); + } + }; + } + } + + private static class SimpleImportModel implements ImportModel<Row> { + + private Log log = LogFactory.getLog(SimpleImportModel.class); + + private DateFormat dateFormat; + + private SimpleImportModel(DateFormat dateFormat) { + this.dateFormat = dateFormat; + } + + @Override + public char getSeparator() { + return ';'; + } + + @Override + public void pushCsvHeaderNames(List<String> headerNames) { + log.info("Headers are : " + headerNames); + } + + @Override + public Row newEmptyInstance() { + return new Row(); + } + + @Override + public Collection<ImportableColumn<Row, Object>> getColumnsForImport() { + List<ImportableColumn<Row, Object>> result = new ArrayList<ImportableColumn<Row, Object>>(); + result.add(getTitleImportable()); + result.add(getNumberImportable()); + result.add(getDateImportable()); + return result; + } + + private ImportableColumn<Row, Object> getTitleImportable() { + return new AbstractImportableColumn<Row, String>() { + + @Override + public String getHeaderName() { + return "TITLE"; + } + + @Override + public boolean isMandatory() { + return true; + } + + @Override + public boolean isIgnored() { + return false; + } + + @Override + public String parseValue(String value) + throws ParseException { + return value; + } + + @Override + public void update(Row object, + String value) + throws Exception { + object.setTitle(value); + } + }; + } + + private ImportableColumn<Row, Object> getDateImportable() { + return new AbstractImportableColumn<Row, Date>() { + + @Override + public String getHeaderName() { + return "DATE"; + } + + @Override + public boolean isMandatory() { + return true; + } + + @Override + public boolean isIgnored() { + return false; + } + + @Override + public Date parseValue(String value) + throws ParseException { + return dateFormat.parse(value); + } + + @Override + public void update(Row object, + Date value) + throws Exception { + object.setDate(value); + } + }; + } + + private ImportableColumn<Row, Object> getNumberImportable() { + return new AbstractImportableColumn<Row, Integer>() { + + @Override + public String getHeaderName() { + return "NUMBER"; + } + + @Override + public boolean isMandatory() { + return true; + } + + @Override + public boolean isIgnored() { + return false; + } + + @Override + public Integer parseValue(String value) + throws ParseException { + return Integer.parseInt(value); + } + + @Override + public void update(Row object, + Integer value) + throws Exception { + object.setNumber(value); + } + }; + } + } + + private static abstract class AbstractImportableColumn<E, F> implements ImportableColumn<E, Object> { + + @Override + public void setValue(E object, + Object value) + throws Exception { + update(object, (F) value); + } + + protected abstract void update(E object, F value) + throws Exception; + } + +} // $Id$ Copied: trunk/nuiton-csv/src/test/resources/log4j.properties (from rev 2237, trunk/nuiton-utils/src/test/resources/log4j.properties) =================================================================== --- trunk/nuiton-csv/src/test/resources/log4j.properties (rev 0) +++ trunk/nuiton-csv/src/test/resources/log4j.properties 2011-11-29 14:34:18 UTC (rev 2239) @@ -0,0 +1,34 @@ +### +# #%L +# Nuiton Utils +# +# $Id$ +# $HeadURL$ +# %% +# Copyright (C) 2004 - 2011 CodeLutin, Chatellier Eric +# %% +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU Lesser General Public License as +# published by the Free Software Foundation, either version 3 of the +# License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Lesser Public License for more details. +# +# You should have received a copy of the GNU General Lesser Public +# License along with this program. If not, see +# <http://www.gnu.org/licenses/lgpl-3.0.html>. +# #L% +### +# Global logging configuration +log4j.rootLogger=ERROR, stdout + +# Console output... +log4j.appender.stdout=org.apache.log4j.ConsoleAppender +log4j.appender.stdout.layout=org.apache.log4j.PatternLayout +log4j.appender.stdout.layout.ConversionPattern=%5p [%t] (%F:%L) %M - %m%n + +# package level +log4j.logger.org.nuiton.util=INFO Added: trunk/nuiton-csv/src/test/resources/org/nuiton/util/csv/importTest.csv =================================================================== --- trunk/nuiton-csv/src/test/resources/org/nuiton/util/csv/importTest.csv (rev 0) +++ trunk/nuiton-csv/src/test/resources/org/nuiton/util/csv/importTest.csv 2011-11-29 14:34:18 UTC (rev 2239) @@ -0,0 +1,4 @@ +DATE;NUMBER;TITLE +2011-12-05;18;"1ère ligne" +2011-12-06;19;"2ème ligne" +2011-12-07;21;"3ème ligne" \ No newline at end of file

1 0