SOLUTIONS
Consulting and Innovative SolutionsINDUSTRY CHALLENGES
Data is the heart of digital transformation and has become critical and strategic for companies.
A challenge of Data integration relies in the capacity to integrate and analyze an increasing broad range of data with a minimum effort and time.
EASYREC MODULES
EasyRec Modules were released for our clients who need to retrieve data from different sources and formats.
They provide high flexibility in the usage and configuration, allowing users to reconcile data from different sources and different format (e.g.: CSV versus Database, …). It reduces and simplify the amount of work required to integrate data in EasyRec.
Data is loaded in real time into an in-memory database (or file database) and converted to structured SQL tables which enable end-users to easily manipulate data according to their need.
These modules use standard JDBC drivers approach for connecting to source data.
What are the advantages of our JDBC modules:
- Standard SQL Syntax
- Leverage existing in-house SQL knowledge
- Minimal training required
- Ease of maintenance
- Use simple SQL query instead of confusing proprietary scripting language
- Write your extraction query in a SQL syntax for greater flexibility
- Allows any SQL construct, including joins and unions
- No need to preload data
- Data are loaded in real time from their existing location
- Robustness
- High performance
- Volume: No archiving capacity issue with large data
EASYREC MODULES
Databases
EasyRec can connect to any Database providing that a JDBC driver is available
- Traditional databases (Sybase, Oracle, MySQL, MariaDB, PostgreSQL, Teradata …)
- In memory databases
- NoSQL databases
CSV Files
- Support single or multiple CSV files
- Ability to join files
- Possibility to convert columns to their expected format (number, date)
- Support for large data (tested with files of 2Go)
Fixed column width
- Support single or multiple files
- Ability to join files
- Possibility to convert columns to their expected format (number, date)
- Support for large data (tested with files of 2Go)
Excel files
- Support for Excel .xlsx and .xls formats
- Support for merged cells
- Possibility to read data from a specific location (in the excel report)
Properties
- Support single or multiple files
- Ability to join files
JSON
- Support single or multiple files
- Ability to join files
XML
- Use of XML Schema to map data
- High performance (SAX or DOM parser)
- High flexibility
SWIFT
- Support single or multiple files
- Ability to join files
- Support single or multiple files
- Ability to join files
- Possibility to convert columns to their expected format (number, date)
CONTINUOUS INTEGRATION CONTINOUS DEPLOYMENT (CI/CD)
In addition to the above modules, some specific modules were requested by our clients in the context of Configuration Management and Continuous Integration Continuous Deployment (CI/CD). Below modules can be used for file-based reconciliations.
One of our main challenge is that the majority of files which need to be reconciled by our clients are XML files. Common issues are that XML nodes are not always exported in the same order, full-path, timestamp or unique IDs are also exported in the XML. Filename can also include a unique sequence ID in their label. Due to these constraints, existing basic file-based reconciliation solutions cannot be used (reconciliation output contains many false breaks and therefore is not reliable).
ZIP files
Used to reconcile content of ZIP files
- Support for recursive ZIP content
- Flexible filename matching rule
- Fast and optimized algorithm to compare files based on their checksum
- Advanced comparison
- Orderless XML compare
- Ignore blanks, header, DOCTYPE, indentation
- Ability to execute customizable transformations (e.g. get rid of unique IDs)
- Support for regular expressions
- Embedded diff tool to show relevant differences
Local repository
Used to reconcile content of local directories
- Flexible filename matching rule
- Fast and optimized algorithm to compare files based on their checksum
- Advanced comparison
- Orderless XML compare
- Ignore blanks, header, DOCTYPE, indentation
- Ability to execute customizable transformations (e.g. get rid of unique IDs)
- Support for regular expressions
- Embedded diff tool to show relevant differences
GIT repository
Used to reconcile content of GIT repository
- Flexible filename matching rule
- Fast and optimized algorithm to compare files based on their checksum
- Advanced comparison
- Orderless XML compare
- Ignore blanks, header, DOCTYPE, indentation
- Ability to execute customizable transformations (e.g. get rid of unique IDs)
- Support for regular expressions
- Embedded diff tool to show relevant differences
SSH reconciliation
Used to reconcile content of directory using SSH connexion (ex: from a remote unix server)
- Flexible filename matching rule
- Flexible inclusion/exclusion filtering
- Fast and optimized algorithm to compare files based on their checksum
- Advanced comparison
- Orderless XML compare
- Ignore blanks, header, DOCTYPE, indentation
- Ability to execute customizable transformations (e.g. get rid of unique IDs)
- Support for regular expressions
- Embedded diff tool to show relevant differences
Manipulating structured data like database is easier than manipulating non structured data. Being able to easily load non structured data into a database is a key feature for data integration specialists.
These JDBC drivers reduce and simplify dramatically the amount of work and effort required to integrate and analyze data.
All above drivers are included in our reconciliation solution EasyRec providing high flexibility in the usage and configuration.