January 4, 2008 1:11 PM
The Data Just Keeps Pouring In, This Time From the Heavens
I've come across another angle through which to consider the amount of bandwidth needed to support a 21st century society.
This article highlights how upgraded telescope sensors are providing the SETI (Search for Extraterrestrial Intelligence) project with a 500-fold increase in data from their search of the cosmos for people to talk to.
I think this is an important trend to note, that the increase in data isn't only coming from having more users come online and the introduction of more bandwidth-intensive applications, there's also the push by most anyone collecting data for research from digital instruments to improve those instruments and therefore collect more data.
I mean, what SETI's doing is nothing new, they're just now able to record and process data that's many magnitude's of order bigger and more detailed than they were in the past.
And this is data that's likely to go directly out over the Internet as the SETI project has long relied on the power of distributed computing through its SETI@home initiative to process the giant data sets their research creates. (As a review, distributed computing relies on using the idle processing power of many lower powered, networked computers rather than one ultrapowerful supercomputer.)
The simple truth is that we're smack dab in the middle of an exponential growth curve in the amount of data that needs to be stored and processed. And while it's been ramping up for a while, we're now entering a time of truly epic growth, especially as new instruments and technologies allow us to gather more data than ever before.