Archive for February, 2010

DIAS : The Future in Delivering Knowledge Based Applications

Tuesday, February 9th, 2010

What is DIAS

DIAS is a rapid software delivery framework which empowers knowledge based applications to be delivered into the business, without the need for a formal software development lifecycle. DIAS stands for Demand Intelligence Analytical Server, which primarily focuses at transforming demand intelligence (read data management, analysis and modelling) , into useful business applications, in an open standards, extensible, robust way.

Whether you need a short term forecasting engine, an application to track the electricity consumption sensitivity to temperature variations, how pricing signals are modifying electricity customer behaviour, or simply need to generate typical daily, weekly or monthly consumption profiles from noisy SCADA data, moving from the analytical model to the business application has never been easier, or quicker.

Where did it all start

The Demand Intelligence Group (DIG) is a niche business unit , part of Enerweb, a South African niche IT services provider. The DIG has several load research, analysis and modelling contracts for its biggest customer, Eskom, focussing on the understanding and quantification of electricity consumer behaviour. Although this work has been highly successful, a continuous challenge existed in deploying the work performed (typically in the form of analysis, models and data processing using R), into the business, and empowering business users to actually use the models to transform the way they do business on a day to day basis.

Why is a DIAS required

In December 2009, the concept of a Load Research Appliance, or software on a memory stick, that could address the problem of rapidly, and easily deploying knowledge based application into a large enterprise, was born. It was proposed, that such a device, could potentially solve the following traditional challenges that the enterprise faces :

  • Empowering and educating business users, so they can deploy, modify and update knowledge based models/applications themselves
  • Ensuring continuity – so that when a specific consultant/developer of a specific knowledge based application is no longer available, others can simple carry on
  • Re-usability of previous modules/applications, typically the know-how to re-use models/applications developed by multiple organisations /consultants , so at to make an every bigger growing ball of knowledge
  • Standardized models and development frameworks are required (also to be open source), so across multiple analysts, multiple consultants, all use the same formats (code & data)
  • Enabling and facilitating the sharing of models and data, between analysts and business users & consultants & organisations – no islanding or vendor or application lockin
  • Being able to easily handover models to the users/business, i.e. Move away from the consultant having to provide answers, to the user being able to run the model on his own machine

In terms of delivering knowledge based applications to a business, the following IT challenges are typically faced, namely :

  • Converting a model into an application typically requires slow, expensive IT projects (SDLC)
  • The pace of change in knowledge based models/structure/assumptions typically means that traditional IT applications cannot maintain the rate of change required for Knowledge Based Applications
  • Provide an open platform, that can be modified and extended, to include, for example gis/spatial calculators, transactional engine capability, content management control, RDBMS capability, etc
  • Deliver in a “portable” fashion with no dependence of host operating system (e.g. which Windows Version) , or deploy on Linux/Ubuntu
  • No installation required – e.g. run off a memory stick, so does not install on the machine, totally portable
  • Allow the same instance to run in desktop, or in eEnterprise server mode, , thus having advanced features such as load balancing, queues , security, transaction management / recovery)

The DIAS Proof of Concept

The initial technology design was to deploy a virtual computing environment, and thus to deliver a “stand alone” virtual machine. This approach however is resource intensive, as it requires hardware resources to be allocated in such a way that it effectively splits resources between the host machine, and the virtual environment. The concept however rapidly evolved in the “Portable Apps” concept, where the applications is packaged in such a way that it can execute without installation or leaving settings files on the host computer.  This results in a situation where advanced applications can be deployed from a memory stick (portable hard-drive, potentially even a CD), and run portably, thus not modifying the host operating system, or any of its settings (e.g. register files). Using this concept, together with the open source R statistical/analytical engine, the Jetty application server, as well as Eebaar, an open source component based development framework, DIAS was born. The first version of DIAS, demonstrated on the 20 January 2010, thus sported a simple task manager (enabling different tasks to be executed), an embedded R engine, and web front ends for input and output rendering of the tasks (i.e. R scripts). The web rendering is configurable from within the R scripts, so little more than a working knowledge of R is required, in order to do magical things with DIAS.

Where can I find out more

DIAS is currently under rapid development, but will have multiple “task-runner” support (i.e. not only limited to R), with multiple front end for rending input and output. DIAS is being released under Gnu Public License (GPL3), and the intention is to build a strong community of Demand Intelligence (to develop tasks), and Java developers (to build on the core framework) , across the globe, that can use the DIAS platform to deliver useful business applications into their perspective communities. Both development and general content generation is currently in rapid progress, and we expect to make the first official release of DIAS, before the end of February 2010 If you require further information or details about DIAS, please contact Schalk Heunis

Protected: SSH-Tunnels, Socks5, HTTP/S Proxy – by ipsocks -

Monday, February 8th, 2010

This content is password protected. To view it please enter your password below: