Too Big to Ignore

Can data really tell you how to run a business, where to invest, whom to hire, or even what kind of music you should listen to? In Too Big to Ignore, author and technologist Phil Simon argues that data has become so vast, so varied, and so fast-moving that ignoring it is no longer an option. Big Data isn’t just a buzzword—Simon contends it’s a global shift that’s reshaping how organizations make decisions, solve problems, and even understand the world.

Simon’s central claim is simple yet profound: we are living through a revolution, a “Data Deluge” that rivals the industrial revolution in its potential to transform business, science, and society. Organizations that learn to harness data creatively—like Target predicting pregnancies or Boston crowdsourcing pothole reports—gain extraordinary insight and efficiency. Those that don’t will fall behind. The volume, velocity, and variety of modern information, now streaming from phones, sensors, social networks, and cloud systems, are too great and too valuable to ignore.

The New Data Reality

Simon opens with a vivid reminder that Big Data isn’t theoretical—it’s here, quietly influencing everyday life. A baseball manager uses obscure statistics to build a winning team. Insurance companies track drivers in real time to price policies fairly. Governments crowdsource infrastructure problems through citizen smartphones. These examples show that the old modes of decision-making—intuition, limited reports, small samples—are obsolete. The real differentiator now is how well you capture and interpret unprecedented quantities of data to make smarter choices.

The Three V’s and Why They Matter

Simon organizes Big Data around Douglas Laney’s famous framework: volume (huge quantities of information), velocity (data moving faster than traditional systems can handle), and variety (multiple types, from tweets to photos to transactions). These qualities mean old tools—spreadsheets, relational databases—are no longer enough. We need new technologies like Hadoop, NoSQL, and machine learning that can process unstructured data and extract meaning from it. But Simon emphasizes that technology is only half the story; culture, mindset, and curiosity determine whether tools become transformative or merely decorative.

An Accessible Revolution

What sets Simon apart from other data evangelists (like Viktor Mayer-Schönberger or Nate Silver) is his accessibility. He doesn’t expect you to be a scientist or a mathematician. The point of Big Data is not that every company needs a PhD statistician—it’s that everyday professionals can now access insights once unimaginable. Free and open-source tools like Hadoop or crowdsourcing platforms make it possible for small startups, schools, and governments to compete with tech giants on intelligence.

Why “Too Big to Ignore” Matters

For Simon, Big Data is more than technical progress—it’s a paradigm shift. Like electricity or the internet, it forces every sector to reinvent itself. Healthcare firms use data to track disease patterns. Retailers personalize promotions by predicting behavior. Scientists collaborate across global databases that make previously invisible patterns clear. Ignoring these trends means forfeiting innovation, efficiency, and foresight. Simon advises readers—from CEOs to educators—to embrace data literacy as a core professional skill. Big Data, he insists, isn’t optional; it’s the new basis of competitive advantage and collective understanding.

What You’ll Learn

Throughout the summary, you’ll explore how data evolved from early spreadsheets to today’s unstructured flood; how organizations can demystify Big Data without drowning in it; and what tools, techniques, and cultural shifts make it work. You’ll see case studies—from Quantcast’s advertising algorithms to NASA’s crowdsourced innovation contests—showing the practical rewards of embracing Big Data. You’ll also confront critical issues: privacy, ethics, and the need to balance automation with human judgment. In the end, Simon’s argument is both pragmatic and visionary: Big Data will only grow larger and more intelligent, and stasis is not an option.

Phil Simon spends much of the book demystifying what Big Data actually is and, just as importantly, what it’s not. Instead of a single technology or dataset, Big Data is a convergence—a massive collection of information characterized by volume, variety, and velocity. These three dimensions create both extraordinary possibilities and daunting challenges for organizations trying to extract meaning from chaos.

Volume: The Scale That Changes Everything

We’ve entered a world where Kansas-sized server farms store data measured in petabytes and exabytes. Walmart’s systems track millions of transactions per hour. Social media sites generate billions of posts, likes, and images daily. Simon explains that these quantities alone force a rethinking of storage and processing. You can’t manage this with Excel or Oracle alone. Technologies such as Hadoop distribute the workload across clusters, enabling analysis of unimaginable quantities of information.

Velocity: The Speed of Modern Insight

Data isn’t static anymore—it’s streaming. Every second, mobile sensors, financial markets, and web traffic generate signals that organizations must interpret in real time. Simon notes that Progressive Insurance’s “Snapshot” program exemplifies this: drivers’ behaviors are uploaded instantly, allowing pricing to adjust dynamically. In politics, campaigns like Barack Obama’s 2012 victory used rapid data feedback loops to target messages to voters with pinpoint precision.

Variety: Beyond Rows and Columns

Older data models dealt mainly with tidy, structured information—numbers, customer IDs, dates. But modern data arrives as tweets, photos, GPS coordinates, audio recordings, and video logs. Simon calls this “poly-structured,” meaning it has inconsistent formats but can still reveal meaningful patterns. Technologies such as NoSQL and text analytics can take fragmented data and turn it into usable insights, from tracking sentiment on social media to predicting product failures.

The Anti-Definition: What Big Data Is Not

Simon warns against confusing Big Data with business intelligence (BI) or data mining. BI tools process structured information—sales reports, invoices, employee records—but cannot handle the unstructured flood of social media or sensor data. Big Data complements BI by adding a richer layer of analysis, combining both structured and unstructured information to see the full picture. (In contrast, BI looks backward; Big Data helps predict forward.)

Big Data isn’t just for giants like Google or Facebook. Thanks to open-source projects and affordable cloud computing, smaller firms and even government agencies can build systems once unimaginable. Simon highlights this democratization as the key distinction: the Big Data revolution isn’t about elite technologists—it’s about empowering everyone to ask smarter questions.

After establishing the nature of Big Data, Simon walks through the analytical techniques that allow organizations to transform noise into knowledge. These methods—many centuries old—become newly powerful when applied to massive, diverse datasets. He calls them the elements of persuasion, since they help persuade businesses to act on data-driven reasoning rather than gut instinct.

Statistical Foundations

Techniques such as regression analysis remain at the core. For instance, CapitalOne’s A/B tests measure how different credit card offers affect response rates, isolating independent variables to maximize profits. The same logic applies to HR analytics predicting employee turnover or marketing models forecasting product sales. Big Data amplifies these methods by expanding sample sizes—making predictions dramatically more accurate under the laws of large numbers.

Data Visualization and Storytelling

Visualization tools like Tableau and heat maps make Big Data human. You can’t read a table with millions of rows, but you can understand patterns in color and motion. Amazon used heat maps to reveal that political book purchases skewed conservative nationwide, offering insights into consumer identity. It’s a reminder that visualizing trends often reveals what raw numbers hide.

Automation and Machine Learning

From Google’s predictive search to Netflix recommendations, machines learn by detecting recurring patterns. Algorithms automatically improve from experience, helping businesses spot fraud, predict demand, or personalize content. NASA even applies gamified machine-learning contests through TopCoder to solve technical challenges faster than its own experts could.

Semantics and Sentiment

Natural Language Processing (NLP) and text analytics let computers interpret human emotion in words. Simon describes healthcare systems using NLP to read thousands of doctors’ notes to identify disease trends instantly—work that once took months of manual coding. Sentiment analysis extends the idea to social media, gauging whether consumers love or hate a brand. These tools bridge structure and meaning, turning messy language into clear intelligence.

Predictive Analytics: Seeing Tomorrow Today

Predictive analytics, Simon writes, is Big Data’s Holy Grail. It uses historical and real-time data to forecast future behavior, from Google Flu Trends predicting outbreaks to Amazon anticipating purchases. But Simon cautions against expecting perfect foresight. Data reduces uncertainty—it doesn’t eliminate it. The smartest organizations learn to make probability, not certainty, their compass.

Once you understand Big Data’s techniques, Simon equips you with knowledge of the technologies that make them possible. These aren’t abstract concepts but evolving ecosystems—platforms, databases, and infrastructures transforming every industry. He guides readers through the essential toolkit for turning massive datasets into usable intelligence.

Hadoop: The Backbone of Big Data

At the heart of the revolution is Hadoop, an open-source framework that distributes data across clusters of machines. Built on Google’s MapReduce architecture, Hadoop breaks giant problems into smaller chunks, processes them in parallel, and reassembles results. Organizations from Facebook to Quantcast use it to handle petabytes daily. Its flexibility and fault tolerance made Big Data feasible well beyond Silicon Valley.

NoSQL and Columnar Databases

Traditional relational databases (SQL) falter when confronted with unstructured information. Simon introduces NoSQL databases like Cassandra, MongoDB, and CouchDB, designed to handle non-tabular data. Columnar databases such as Vertica process queries faster by storing data vertically, increasing efficiency and compression. Together, these technologies form the backbone of flexible data architecture.

The Rise of Hybrid Systems

Innovators like Hadapt and RainStor blend the best of Hadoop with relational databases, merging structured and unstructured data in one system. The result is agility—organizations can run SQL-like queries while still handling multimedia and text. Companies like Cloudera and Hortonworks have built entire businesses supporting this hybrid ecosystem.

Cloud Platforms and Democratization

Amazon’s Web Services (AWS) and Google’s BigQuery make enterprise-scale computing accessible to everyone. A startup can analyze billions of records without owning a single server. Simon compares this to the “information electricity” model—just plug in and draw power as needed. It’s the essence of Big Data as an open, democratized resource.

The takeaway: technology removes the physical limits of data, but mindset determines success. Simon reminds readers that tools evolve rapidly—Hadoop today could become obsolete tomorrow—but curiosity and adaptability are timeless assets for anyone navigating the data frontier.

Simon brings his argument to life through vibrant case studies showing Big Data in action—from advertising to healthcare to aerospace. These stories illustrate how embracing data fosters innovation, efficiency, and even social good.

Quantcast: Advertising with Precision

Quantcast transformed marketing by using real-time analytics to match advertisers with exactly the right audiences. Its algorithms process 20+ petabytes daily to target consumers based on actual online behavior, not guesses. Simon explains how building its own file system (QFS) doubled speed and halved storage cost—a masterclass in engineering adaptation. For Quantcast, data isn’t noise; it’s business fuel.

Explorys: Healthcare Data for Better Lives

In healthcare, Explorys uses Cloudera’s Hadoop architecture to integrate millions of medical records across hospitals. Its platform helps identify disease trends, optimize treatment, and prevent costly ER visits. A doctor can query vast population datasets instantly—something impossible with legacy systems. By merging clinical and operational data, Explorys delivers both savings and healthier outcomes.

NASA and TopCoder: Crowdsourcing Innovation

NASA’s partnership with TopCoder turns data challenges into global competitions. Engineers and citizens worldwide submit algorithms to analyze space imagery or design satellite software. Solutions that once took years now emerge in weeks, with rewards as modest as $1,000. These contests show Simon’s favorite theme: innovation thrives when data becomes communal, not proprietary.

Together, these examples reveal Big Data’s universality. It scales from startups to governments, generating not just profits but breakthroughs. Whether predicting fraud, detecting illness, or optimizing ads, data is reshaping every field—and the organizations willing to experiment reap the biggest rewards.

After showing what’s possible, Simon offers practical guidance for leaders ready to dive into Big Data. His advice balances enthusiasm with realism: you must prepare the ground before planting seeds. Success depends less on software and more on mindset, culture, and strategy.

Start Small, Learn Fast

Simon warns against expensive, top-down implementations. Big Data thrives when it grows organically—begin with small pilot projects that deliver clear wins. Sears, for example, used Hadoop modestly to streamline its analytics, later expanding after seeing a 163% increase in profits in one quarter. These “little victories” build credibility and momentum.

Invest in People, Not Just Tools

Data scientists are in short supply, and the best combine math with business acumen. Simon suggests training high-potential employees or hiring hybrids—professionals who bridge IT and business. Schools like Carnegie Mellon now teach “Very Large Information Systems,” signaling how critical data literacy has become. (This mirrors Thomas Davenport’s idea of analytics as a competitive advantage.)

Make Data a Culture, Not a Project

Big Data isn’t a one-off initiative—it’s a cultural shift. Simon likens data governance to exercise: sporadic effort doesn’t work. Organizations must embed data-driven thinking at every level, from executives to interns. Success requires curiosity, openness, and humility—the willingness to question old habits and accept counterintuitive insights, like Walmart discovering beer and Pop-Tarts surge before hurricanes.

Avoid Common Pitfalls

Simon highlights key traps: treating Big Data as pure IT territory (it belongs to the business), confusing open-source “free” tools with cost-free success, rushing without readiness, or believing there’s a fixed Big Data checklist. Instead, see it as iterative experimentation—a long-term capability built through learning and adjustment.

The message is empowering: anyone can begin, but no one can master it overnight. Big Data rewards persistence, collaboration, and continual curiosity. Being data-driven isn’t about the size of your servers—it’s about the breadth of your imagination.

Simon doesn’t shy away from Big Data’s dark side. With great insight comes great responsibility. He examines the rising concerns around privacy, security, and human resistance to a world increasingly governed by algorithms.

Privacy and the Big Brother Effect

When companies predict pregnancies before parents know or track web behavior down to location, ethical boundaries blur. Google’s Street View data collection scandal and Facebook’s opaque data policies illustrate how easily innovation veers toward intrusion. Simon reminds readers: just because companies can capture data doesn’t mean they should. Transparency, consent, and accountability are essential safeguards.

Security and Vulnerability

Big Data makes big targets. Breaches like the Zappos and LinkedIn hacks show that as organizations accumulate more information, risk grows exponentially. Simon urges leaders to treat cybersecurity as integral to data strategy, not an afterthought. Without trust, the data economy crumbles.

Resistance and the Human Element

Beyond privacy lies another obstacle: emotional resistance. Many professionals fear being replaced by machines or losing autonomy to algorithms. Simon calls this the "Big Paradox"—as complexity rises, people prefer intuition over analysis. The cure isn’t more data but better communication. Organizations must show that analytics complements human judgment, rather than replacing it.

The Ethics of Innovation

Echoing Supreme Court Justice Potter Stewart’s line—“knowing the difference between what you have a right to do and what is right to do”—Simon argues that ethical data use will define leadership in the coming decades. Big Data isn’t neutral; it magnifies the values of those who wield it. Companies must choose transparency, fairness, and empathy if they want Big Data’s promise to outweigh its peril.

In the book’s conclusion, Simon looks ahead to the evolution of Big Data. He predicts that data will become not just bigger, but more intelligent, interconnected, and personal—driving both business innovation and societal transformation.

From Active to Passive Data Generation

The next frontier is the Internet of Things. Machines will soon outpace humans in data creation. Smart thermostats, RFID-tagged goods, and connected cars will continuously stream information, powering predictive models for everything from energy use to product design. Simon’s example of the Nest thermostat captures this shift from manual to automatic data capture—where every object becomes a sensor.

Democratization and Movements for Data Transparency

Simon spotlights grassroots efforts like Google’s Data Liberation Front and the Vibrant Data Project, which fight for user control and portability of personal data. Open metadata standards from the Open Data Foundation will improve global interoperability. The future, Simon believes, won’t belong to those who hoard data but to those who share and contextualize it.

The Economic Imperative

Governments and businesses alike face fiscal crises and shrinking resources. In such a world, Big Data becomes not a luxury but a necessity. Simon highlights Boston’s Mayor Thomas Menino, whose use of data-driven apps like Street Bump improved efficiency and citizen engagement. When stasis isn’t an option, data becomes the lifeline of innovation.

The Human Future of Big Data

Ultimately, Simon insists that Big Data’s future depends on values as much as algorithms. Technological power must serve human insight, creativity, and empathy. The organizations that thrive will be those that combine computational intelligence with moral intelligence—using data not just to predict behavior, but to improve life. As he closes, Simon reminds you that the revolution isn’t coming—it’s already here, and its impact will be too big to ignore.