"Big Data" has leapt rapidly into one of the most hyped terms in our industry, yet the hype should not blind people to the fact that this is a genuinely important shift about the role of data in the world. The amount, speed, and value of data sources is rapidly increasing.
Note: The following is an excerpt from Martin Fowler's Info Deck on Big Data.
Data management has to change in five broad areas:
- Extraction of data from a wider range of sources,
- Changes to the logistics of data management with new database and integration approaches,
- The use of agile principles in running analytics projects,
- An emphasis on techniques for data interpretation to separate signal from noise, and
- The importance of well-designed visualization to make that signal more comprehensible.
Summing up this means we don't need big analytics projects, instead we want the new data thinking to permeate our regular work.
- There is a significant change in the role that data plays in our activities.
- Although the term big data is often used to describe this change, it's not just about how much data we are looking to use.
- You may want to think of "big" applying to the importance of data - that data is playing a bigger role in our lives. Or just don't try to read "big data" literally.
Big data is term that's generated a lot of hype. But I think it's important to resist our usual aversion to hype in this case - there is a significant change in thinking that's happening. This shift forces us to change many long-held assumptions about data. It opens up new opportunities, but also calls for new thinking and new skills.
Disclaimer: The statements and opinions expressed in this article are those of the author(s) and do not necessarily reflect the positions of Thoughtworks.