I3S Final meeting
Session 3
VTL Tools
What is VTL?
The Validation and Transformation Language (VTL) allows the formal expression of treatments to:
- validate statistical data
- calculate derived data
What is VTL?
- an SDMX standard
- grammar is formally defined (EBNF)
- strong support by Eurostat, the ECB and others
What is VTL?
VTL is user oriented and provides a technology-neutral and standard view of statistical processes at the business level
What tools for VTL?
- VTL is a language, so it has to be edited (-> editor)
- VTL expressions or rules can be managed and shared (-> rule manager)
- VTL is technology-neutral, so has to be translated to an executable language (-> execution engines)
EBNF helps hugely to build tools for VTL thanks to Antlr
What was done in I3S?
Context
- Eurostat developing a rule manager and editor
- SSB had a Java execution engine for VTL 1.1
- Insee developing JavaScript engine and editor
What was done in I3S?
Achievements
- Eurostat-Insee meet for technical alignment of editors
- Eurostat open-sources its VTL editor
- Insee drops own editor & contributes to Eurostat's
- Insee develops Trevas with SSB contribution
- Trevas added to the I3S deliverables
Zoom on Trevas
- a Java engine for VTL 2.0
- provides an abstract definition of a Java VTL engine
- also two concrete implementations
- in-memory engine for relatively small data
- Apache Spark engine for Big Data production environments
Zoom on Trevas
- Work in progress (coverage improving)
- Open source (MIT) on GitHub
- Published on Maven
Zoom on Trevas
Demo
(long version available on video)