Scientific discoveries rely increasingly on the ability to efficiently grind massive amounts of experimental data using database technologies. Current database management systems, however, lack the essential components required to reach this goal at reasonable cost. They do not support the daily aggregation of tera-bytes of data efficiently, nor can they provide near real time response to summarize tera-bytes of results for ad hoc queries. Neither are the systems designed for energy conservation. Major innovations in this domain, overhauling their architecture is widely recognized as a grand challenge and called for in the database research community
The SciLens mission is accomplished with the MonetDB system as its base. MonetDB excells as the most advanced open-source column store database system. Its multi-layered software stack: query-, optimizer-, and storage-layer, sets it apart from the monolithic alternatives. It has a proven track record in performance, innovation and breath of deployment.
Together with domain experts we develop real-life demonstrators to elicit the challenges posed in science. We belief that remarkable results will emerge by extending MonetDB with the missing components uncovered this way. In particular, the MonetDB system will be enriched with a query language for science, called SciQL, to capialize upon a symbiosis of the relational and array paradigms. It will provide "load free" integration with existing science repositories based on FITS, NETCDF, MSEED,etc. to saveguard investments made hitherto.