The definition of Big Data is simple -- itís the collection of large amounts of information. Going deeper, we include the ability to manipulate this data through analysis. Itís not a storage issue; itís a transaction and analytics issue. If storing massive data were the point, we wouldnít be obsessing about big data. The point is using data from a wide range of sources -- sensor data, demographic info, physical qualities -- to detect patterns and make decisions based on the knowledge derived from those patterns.
The ability to process massive data changes our behavior. Data analysis of stress on new materials allows the automotive and aerospace industries to bring strong, lightweight, sustainable materials into production. Big-data analytics allows plants to become ultra-optimized, greatly reducing energy consumption and reducing the overall cost of manufacturing. Big data is helping sustainable energy sources compete against fossil fuels. Ultimately, big data will keep our cars from bumping into each other.
Click on the image below to start the slideshow:
In a few short years, we will all feel the effect of big data in our cars. They wonít bump into each other as often, and quality problems with deadly accidents and massive recalls will begin to taper off. Google's autonomous car is the icon for big data in automotive. But it will be more. It will include intelligent design, intelligent manufacturing, and intelligent sensors that will reduce accidents dramatically. (Source: thebigdatainsightgroup.com)
Rob, what you describe are the three V's of Big Data. These are Volume, Velocity and Variety. I put them in this order because this is how they appeared in the database market. First there were very large databases. These typically very expensive to store and process, thus they were confined to very high value applications. Next was Velocity. In the early 1990s some telcos were using a DBMS to collect data on calls in real time to manage their network. Most recently we have seen the addition of Variety. Since storage has become so cheap and dense (both are important) and processing has become very cheap, we now keep stuff we could not afford to store before. In addition, we keep information that has a low information density, such as social media data. This can still be very useful even to industrial companies. It allows them to find out things they could not in other ways.
Interesting slideshow. I always sort of wondered what "big data" meant--it's one of those technical yet abstract concepts that can baffle someone if you don't know exactly what it means. This gives me a much better idea--sometimes visualizing a concept helps. Obviously it still can mean a lot of things depending on the application, and is an interesting concept for the future.
I appreciated the reference to the movie Money Ball because "Big Data" is really a huge paradigm change and can be hard to accept - just like the resistance for the system in Money Ball:
"For forty-one million, you built a playoff team. You lost Damon, Giambi, Isringhausen, Pena and you won more games without them than you did with them. You won the exact same number of games that the Yankees won, but the Yankees spent one point four million per win and you paid two hundred and sixty thousand. I know you've taken it in the teeth out there, but the first guy through the wall. It always gets bloody, always. It's the threat of not just the way of doing business, but in their minds it's threatening the game. But really what it's threatening is their livelihoods, it's threatening their jobs, it's threatening the way that they do things. And every time that happens, whether it's the government or a way of doing business or whatever it is, the people are holding the reins, have their hands on the switch. They go crazy. I mean, anybody who's not building a team right and rebuilding it using your model, they're dinosaurs."
Thanks for the quote, Nancy. In the case of Money Ball, as in the case with much of Big Data, is to use calculations to discover true efficiencies -- and to discover those efficiencies free from prejudice.
Of course Big Data does other things as well -- physical calculations that used to take months can be done in a few hours. That's helping with composites and with creating materials that are lighter than steel, yet stronger.
Ultimately, Big Data will be used to keep cars from bumping into each other. That could save 30,000 lives each year in the US alone.
Big Data is a big deal. I loved Money Ball for its demonstration of the value of data over "gut" feelings about ball players.
I loved hearing the logic behind the kid's explanations in Money Ball...
I'm still trying to grasp the overall concept of "Big Data" Rob, but anyone in the semiconductor industry can tell you of the value of data - lots and lots of data. Pre-stress testing, post- stress testing qual data. Production test data...A product engineer is often a master at detecting trends that can vastly effect production and design decisions. Now stepping that task up to the level of "Big Data" and the analysis task required is mind boggling.
With major product releases coming from big names like Sony, Microsoft, and Samsung, and big investments by companies like Facebook, 2015 could be the year that virtual reality (VR) and augmented reality (AR) finally pop. Here's take a look back at some of the technologies that got us here (for better and worse).
Good engineering designs are those that work in the real world; bad designs are those that donít. If we agree to set our egos aside and let the real world be our guide, we can resolve nearly any disagreement.
Focus on Fundamentals consists of 45-minute on-line classes that cover a host of technologies. You learn without leaving the comfort of your desk. All classes are taught by subject-matter experts and all are archived. So if you can't attend live, attend at your convenience.