big data

Understand Your Data

If your CSV, JSON, XML, database table, NoSQL store or Java Object has “20160704000000” in it as a String or even a number then you are part of the Big Data problem and a long way from Fast Data. The space is occupies is one thing but performance is another.

Avoid Big Data Antipatterns

Over the last five years, the amount of data in our data centers has exploded, and today there are numerous NoSQL, streaming, and batch systems that promise to scale with your data. These new technologies bring with them a difficult question that needs to be answered: which tool is the best fit for my use …

Avoid Big Data Antipatterns Read More »

Big Data Beyond Hadoop

Apache Hadoop is the de facto open source standard for big data solutions. A challenging task is to send all data to Hadoop for processing and storage (and getting it back to your application later), because data comes from many different applications (SAP, salesforce.com, Oracle’s Siebel product family, and so on) and databases (file, SQL, …

Big Data Beyond Hadoop Read More »

Big Data Architectural Patterns and Best Practices

The world is producing an ever increasing volume, velocity, and variety of big data. Consumers and businesses are demanding up-to-the-second (or even millisecond) analytics on their fast-moving data, in addition to classic batch processing. AWS delivers many technologies for solving big data problems. But what services should you use, why, when, and how?

Big Data, Bad Analogies

Data lakes, data exhaust, web scale, data is the new oil. Vendors are throwing new terms and analogies at us to convince us to buy their products as the market around data technologies grows. We change data persistence and transaction layers because “databases don’t scale” or because data is “unstructured”. If data had no structure …

Big Data, Bad Analogies Read More »