Blog Entries

Back in Action

oliver@xlmpp.com — Mon, 29 Apr 2013 08:00:00 +0100

Time to get this blog back into activity.

The past few years have been so busy with projects the size of 10s of PB, that I did not get a chance to do any public blogging. We started this site when #BigData was not a term but an over simplification. How times have changed.

We are working on brand new massive scale projects and are turning more and more into real time analytical processing.

Time to writeup what we have been up to. Check back over the next weeks as I will update the site [...]

Analytics as a Service - Social SQL

oliver@xlmpp.com — Sat, 03 Sep 2011 08:00:00 +0100

The past 12 months had us move Analytics as a Service (A3S) to new maturity levels. For the very first time we have a single point interface for all of our A3S services: the DataHub. I recently presented an overview and demo to a group of industry analysts and the feedback has been overwhelmingly positive.

The combination of Social, private Cloud, Analytics as a Service based on an Open Source built (joomla + kunena) social portal is turning into a killer application for the global [...]

Project Singularity

oliver@xlmpp.com — Sat, 12 Feb 2011 08:00:00 +0100

It has been a while since I actively blogged on this personal site of ours. It has been a busy couple of years and our teams have pushed the boundaries of pretty much any technology out there that deals with Data and Analytics.

Some 4-5 years ago we started an internal project and based on Ray Kurzweil's - The SIngularity is Near - we dubbed it Singularity.

We are only weeks away from launching V3 of our Singularity platform and its nothing short of amazing. We set out to scal [...]

Agile Business Analytics, not just Agile DW development

michael@xlmpp.com — Sat, 15 Nov 2008 08:00:00 +0100

Today I ran across some discussion about Agile Development in Data Warehousing, and note that we talk about this in the context of the DW development, but not in relation to the Business. I believe there is a need to discriminate some of these processes quite differently. Most simply put - One is applying Agile to DW development; the other is applying Agile to Business Analysis.

Core DW foundations involve modeling root components of business data needs and implementing a data model wh [...]

CCA08 - Cloud Computing and its Applications

oliver@xlmpp.com — Thu, 23 Oct 2008 08:00:00 +0100

Just got back from Chicago, where over the past 2 days a small group of scientists, academia and industry discussed various aspects of cloud computing and related topics.

One of the topics was about comparing extreme large scale analytical problems and the systems leverage to solve them. In order to compare classes of super computers, Alex Szalay (John Hopkins University) explained a simple yet interesting figure: The AMDAHL number (Amdahl's Law Bell, Gray and Szalay 2006)

A [...]

New software licensing is needed

michael@xlmpp.com — Thu, 23 Oct 2008 08:00:00 +0100

Over the last year I have spent a good amount of time thinking about the cost of analytics, and a few things worry me about our industry and how vendors price in this industry. In the Data Warehousing and BI industry, we're starting to see pricing models based on data volume or size. I know of one vendor which prices by specint, so - get a bigger system or virtualize your systems - get a big bill.

The problem with these licensing schemes is that they actually make the [...]

Analytics as a Service

oliver@xlmpp.com — Mon, 21 Apr 2008 08:00:00 +0100

Analytics as a Service

What Do you think about Agile Analytics? Every heard about it? Well, here are a couple thoughts from the guys who deal with it on a daily basis.

Analytics as a Service

Looking forward to seeing your comments on this one.

Science - DB Research Meeting

oliver@xlmpp.com — Sat, 22 Mar 2008 08:00:00 +0100

Next week I will be attending the next iteration of the xldb group events organized around eXtreme Large Database Applications. xldb workshop

With 100s of Peta Bytes of information waiting to be captured and analyzed, new concepts are required to scale today's platforms by 1-3 orders of magnitudes.

Today we 'limit' ourselves to 'only' capture 40TB/day of incremental incoming data volumes, next [...]

TACC Ranger goes live

oliver@xlmpp.com — Fri, 21 Mar 2008 08:00:00 +0100

On February 22nd 2008 TACC formally introduced the go-live of RANGER - a massive scale supercomputer. While not a traditional relational processing system, the design shared many components and basic principles of large scale processing platform.

Of particular interest is the multi terabit infiniband interconnect that allows the system to (re)distribute massive amounts of data.

One of the early learnings from the system is that loading massive amounts of data can at times be a [...]

A Systems overview

oliver@xlmpp.com — Fri, 21 Mar 2008 08:00:00 +0100

Finally I got to complete a high level systems overview. I realize it does not contain too much detail, but as you can imagine, we are bound by pretty strict NDAs.

Nevertheless, it should give you a good feel for how much data we process any given day. The stats are pretty much going into 2008 figures and are growing rapidly.

Here is a link to the article: Our Systems

Enjoy the reading and post your comments!

Oliver

Welcome to the blog @ xlmpp

oliver@xlmpp.com — Sat, 16 Feb 2008 08:00:00 +0100

Welcome to the xlmpp blog!

Its time to kick off an exciting blog about ultra large scale information processing architecture and real world analytical systems.

Over the next weeks and months Michael, Darren and myself will be blogging about the largest data processing systems, common issues, scalability of large scale mpp clusters, time to market, analytics, ultra high data volumes and lots more.

Be sure you bookmark the site and subscribe to our news feed.