What Is Big Data? A Layperson's Guide

Big data refers to large data sets that can be studied to reveal patterns, trends, and associations. The vast number of data collection avenues means that data can now come in larger quantities, be gathered much more quickly, and exist in a greater variety of formats than ever before. This new, larger, and more complex data is collectively called big data.

What makes big data “big”?

Though there is no threshold that separates big data from traditional data, big data is generally considered to be “big” because it cannot be processed effectively and quickly enough by older data analysis tools.

The three Vs of big data

Big data is broadly defined by the three Vs: volume, velocity, and variety.

• Volume refers to the amount of data. Big data deals with high volumes of data.

• Velocity refers to the rate at which the data is received. Big data streams at a high velocity, often directly into memory rather than being stored on a disk.

• Variety refers to the wide range of data formats. Big data may be structured, semi-structured, or unstructured and can be presented as numbers, text, images, audio, and more.

Companies that process big data may also focus on other Vs, such as value, veracity, and variability.

What’s driving big data growth?

Emerging information technology has allowed data to be collected, stored, and analyzed at unprecedented scales. The internet continues to be adopted by new users in the US and across the globe, and developing technologies have allowed the internet to be integrated into many different products, creating numerous new sources of data. The millions of people watching Netflix, using Google, and buying products online daily contribute to the increasing volume and sophistication of big data.

Examples of big data:

• Smart (Internet of Things) devices: A connection to the internet enables companies to collect data through devices like smart home systems, robotic vacuum cleaners, smart TVs, mobile devices, and wearable fitness trackers that log files.

• Social media: Likes, shares, posts, comments, how long you spend looking at a post—all of this information is considered insightful data about people’s behavior, sentiment, and preferences.

• Websites: Companies or other website owners can track page visits and general locations of visitors, see how long audiences spend on a page, what links are most clicked, and cursor movement.

• Business transactions: Data can come from customers purchasing products online and in person. Price, time of purchase, payment methods, and other details can inform a business about customer demand for its products.

• Machinery: Even without an internet connection, road cameras, sensors, and medical equipment can record information.

• Health care: The health care system is full of data. Data analysts can use aggregated information on health care records, insurance, and patient summaries to drive new insights and enhance patient care.

• Government: City, state, and federal governments can use data from many sources—road traffic information, agricultural yields, weather tracking systems, demographic information from censuses, to name a few—to make policy decisions.

How is big data used?

Big data can be used by almost any entity to gain valuable insights and make decisions about its operations. A business, for example, can analyze the data it collects to understand customer preferences better and devise impactful business strategies.

Big data in health care systems can be used to find common symptoms of diseases or decide how much staff to put on a hospital floor at any given time. Governments may use traffic data to plan new roads or track crime rates or terrorism risks to adjust their response accordingly.

Data analysts and other professionals who work with big data may use the following tools and methods:

Predictive analytics: Analysts can use data to predict the likelihood of events or trends in the future by using predictive models and machine learning technology.

Real-time analytics: Real-time analytics is the process of analyzing and using data the moment it enters a database to make decisions quickly, such as when a banking system flags a payment as potentially fraudulent when it is made out of the country.

Data mining: Data mining refers to a process that combs through huge amounts of data to find patterns, trends, and correlations. Finding relationships between data points is key to helping organizations make decisions.

Machine learning: Machine learning—a form of artificial intelligence that learns and improves itself continuously—helps predict trends and find patterns in large data sets. Machine learning can be useful in adapting to new data influxes. Find out how to scale data science and machine learning for big data using Apache Spark.

Deep learning: Deep learning is a subset of machine learning based on artificial neural networks that mimics the human brain's learning process. It is often used in speech and text recognition, as well as computer vision technology.

Data warehouses: Data warehouses store massive amounts of historical data. The data is typically cleaned and organized and can be accessed later to be analyzed.

Hadoop: Hadoop is a software framework used to store and process vast amounts of data that can work across several clusters of computers. Hadoop’s capacity to be scaled easily and ability to store various types of data at once have made it the go-to platform for processing big data. Learn to navigate your way around big data and get a grasp on Hadoop with UC San Diego’s course on Big Data.

Apache Spark: Apache Spark is a software framework that combines data analysis with artificial intelligence. It can often perform analyses on large sets of data more quickly than Hadoop.

I had never really thought of myself as a data-oriented person, but the way that the program was taught was easy to understand.

— Emma S., on completing the IBM Data Science Professional Certificate

Learn more: What Is Big Data Analytics? Definition, Benefits, and More

Big data careers

Data-related professions were among the top ten positions in the World Economic Forum’s list of job roles with increasing demand across industries in 2023, including AI and machine learning specialists, business intelligence analysts, information security analysts, data analysts and data scientists, and big data specialists [1]. Take a closer look at some jobs that use big data in different capacities. 

Data analyst: A data analyst gathers, cleans, interprets, and creates data models. Data analysts can work in various industries, including business, science, and health care.

Data engineer: Data engineers create and maintain data infrastructure, including data warehouses, pipelines, and other forms of data organization that analysts can use to make predictions or other interpretations. Big data engineers do this with software that allows them to maneuver large volumes of data.

Data scientist: A data scientist generally uses mathematical or statistical knowledge to build algorithms, models, and other analytical tools to help organize and interpret data. 

Business intelligence analyst: Business intelligence analysts parse business data, such as sales information or customer engagement metrics, to form actionable insights into a business's performance. 

Operations analyst: Operations analysts gather data about operational issues in businesses or other organizations. Operations analysts can use data to find business insights and solutions to issues in production, staffing, or any other related aspect.

Marketing analyst: Marketing researcher analysts harvest information about current or potential customers, market conditions, or competitor activities. The data collected is then used to understand how a business can respond through marketing tactics or product adjustments.

Learn how to use big data on Coursera.

Organizations can gain valuable insights, improve operations, and make data-driven health care, finance, and marketing decisions by analyzing big data. Incorporating big data into your career can bring fresh insights into your work, and data will likely continue growing in importance.

In Google's Data Analytics Professional Certificate, you'll learn key analytical tools and skills as you build in-demand skills at your own pace. Already have a strong data analytics skills foundation? Consider the Google Advanced Data Analytics Professional Certificate to grow your knowledge and open new job opportunities.