In the past, excessive data volume posed a storage dilemma that made discarding data pretty much the only solution for private business. Garbage-in-garbage-out was the prevailing mindset. Then technology reached a point where it became cheaper to store and analyze data than to discard it. At that point the big data industry was born.
Welcome to the future. It’s a brave new world; one in which analysts can now predict things like fluctuations in the Dow Jones Industrial Average, using statistical models that analyze, you guessed it, Big Data. Big data is big news in pretty much every corner of the economy. Some of its major advocates are even predicting such things as disease prevention and reduction of government bureaucracy. The possibilities are pretty captivating. But for now lets talk about the more day-to-day uses of big data.
We’ll get to the definition of the term ‘big data’ in a second but for now, what the usefulness of big data boils down to essentially, is that elements of business logistics that used to be a lot of mysticism and guesswork — too nebulous for most people to even think about — are now reduced to simple analytical decision making processes. Even though there’s still some debate on the best definition of the phrase ‘big data,’ from a general, semantic standpoint, we can at least think of it as a catch-all term that describes any data set too large to process using conventional desktop methods like spreadsheets.
A Cyberspace Odyssey …
Early in the year 2001, tech-industry analyst Doug Laney published what is now considered the mainstream definition of big data. It’s really more like the key elements of big data that help businesses definite the term for themselves case-by-case. In any case, Laney’s breakdown is known definitively as the three Vs: volume, velocity and variety. As mentioned previously, the definition of big data varies, industry-by-industry, depending on which of those elements takes precedence and how that information is applied. Narrowing down the answer to those questions depends on the specific goals businesses want big data to help them accomplish, whether it’s to increase revenue, reduce costs, optimize transit time, increase efficiency at various levels, or whatever else. So let’s get a little more in-depth with the three-V’s to start the conversation off and then we’ll move on to a few ways e-commerce companies are leveraging all those stockpiles of information.
Many factors contributed to the increase in data volume. Transaction-based data stored through the years. Unstructured data streaming in from social media. Increasing amounts of sensor and machine-to-machine data being collected. With decreasing storage costs, other issues emerged, including how to determine relevance within large data volumes and how to use analytics to create value from relevant data.
Data streams in now at unprecedented speeds and must be digested quickly. RFID tags, sensors and smart metering drive the need to deal with torrents of data in nearly-real time. Reactions executed quickly enough to deal with data velocity poses a challenge for most organizations.
Data today comes in all types of formats. Structured, numeric data in traditional databases, information created from line-of-business applications, unstructured text documents, email, video, audio, stock ticker data and financial transactions. Managing, merging and governing different varieties of data is something many organizations still grapple with.
So what are e-commerce companies doing to utilize big data?
1. Optimizing Shipping
Shipping companies make their living managing transit times, and so by the associative property, e-commerce companies need to as well. Think of it this way: transit time equates to time spent with engines running fuel burning, and man-hours accumulating. Those are all costs that eventually trickle down to shippers themselves. But really those are just the traditional lookback data that help companies determine where inefficiencies may exist. It forms a small part of the big data picture. Other non-traditional data relevant to the equation might include weather and traffic delays, port strikes, and/or unexpected repairs on those endlessly running engines. Data indicating trajectories within those systems may come from GPS devices, RFID tags, and traffic management systems, and possibly social media monitoring. Those sources are what businesses can compare against lookback data to identify patterns then forecast potential problems, take measures to avoid them, and presumably improve their bottom line. Learn more about shipping systems.
2. Supply Chain Management
According to an IndustryWeek report 97 percent of executives claim some understanding of how big data will improve their supply chain, while only 17 percent claim to have implemented it. On its face those statistics may seem like a lot of talk around the latest trendy buzzword, but all that talk is leading to understanding, and understanding is slowly leading to action. Nearly 40 percent of the same executives surveyed claim to have initiatives underway to implement big data analytics into their supply chain. The same IndustryWeek report emphasized the effectiveness over enterprise-wide adoption of big data as opposed to the processed-focused approach. What’s interesting is that those who have adopted the enterprise-wide approach report marked improvements at the process level. For example, 61 percent of those who have adopted enterprise-wide big data analytics have shortened their order-to-delivery cycle times.
3. Business Tracking
Amazon is using their own big data analytics tool, EMR, based on the open-source Hadoop platform to track expenses, income, and human resource information. Considering that some of Amazon’s warehouses are larger in both square area and employee numbers than some small towns in the U.S., that’s no small task. It’s a big data task. UPS is using big data to track up to the minutes speeds and direction of over 46,000 delivery vehicles. The company is reporting millions of gallons of fuel saved, and hundreds of millions of miles shaved off daily delivery routes after optimizing shipping systems, delivery routes, and configuring drivers’ pick-ups and drop-offs, all using big data.
4. Tracking Customer Preferences
Restaurant chains are using big data to track customer preferences in all sorts of categories. McDonald’s for example, is compiling trend-analytics to optimize drive-thru experience based on types of customers passing through, menu design, and information provided on the menu. The hope is that spotting trends in demand will inform measures to improve efficiency. But informing businesses what to do is not the only application to customer preference tracking. It also helps them know what not to do. According to an article posted on Wired, we can all thank big data of nipping that whole ‘bacon makes everything better,’ trend in the bud. It may seem a little silly, but it was big data that informed restaurateurs to put bacon back on sandwiches and take it off dessert items, and out of beverages.