Turning Big Data into Big Decisions Delivering on the High Demand for Data

Transcription

1 Turning Big Data into Big Decisions Delivering on the High Demand for Data Michael Ho, Vice President of Professional Services Digital Government Institute s Government Big Data Conference, October 31, 2013, Washington, DC

2 What is big data? 2

3 Structured Valuable Large Complex What is big data? Various Unstructured Fast-moving Variable 3

4 Structured Valuable Large Complex What is big data? Various Unstructured Fast-moving Variable 4

5 What does Structured Valuable Large Zettabytes? What is big data? Exabytes? Various Complex Fast-moving Petabytes? Terabytes? Gigabytes? Variable Unstructured mean? 5

6 What does Structured Valuable Large Complex What is big data? Various Unstructured Fast-moving Variable mean? 6

7 What does Valuable Large Complex Structured What is big data? Various Fast-moving Unstructured Variable mean? 7

8 So what actually makes data big? When the amount of data is too large to utilize within its window of relevance. 8

9 There is no formula. Data sources Data velocity Amount of data Window of relevance 9

10 There is no formula. Window of relevance Data sources Amount of data Data velocity are constantly shifting. 10

11 Why the hype, why now? Demand 2008 The number of internet-enabled devices exceeds the total world population An estimated 2.8 zettabytes, or over 3 trillion gigabytes, of data is collected. Experts predict 40 zettabytes of data will be created by the year

12 Why the hype, why now? Cost effectiveness Cost per gigabyte of RAM

13 The big [data] picture 13

14 The big [data] picture Getting the right data 14

15 The big [data] picture Getting the right data In front of the right people 15

16 The big [data] picture Getting the right data In front of the right people At the right time 16

17 The big [data] picture Saves time 17

18 The big [data] picture Saves time Saves money 18

19 The big [data] picture Saves time Saves money Saves lives 19

20 The big [data] picture Revolutionary or Evolutionary? 20

21 The big [data] picture and Revolutionary or Evolutionary 21

22 Big data technologies are the natural evolution of data collection and processing. 1970s - First RDBMS and SQL developed 2003 Ehcache created 1991 WWW becomes publicly available 1995 GPS operationalized 2005 Hadoop created 2009 Terracotta creates Enterprise Ehcache 1998 Google founded 2003 Wikipedia started

23 Big data technologies are the natural evolution of data collection and processing. Shrinking Decision Windows Decision Time ENTERPRISE EHCACHE Data Size Data Growth

24 Technology evolving to meet demand ENTERPRISE EHCACHE Traditional database or data warehouse Parallel computing, distributed processing Faster on demand access, interaction, and analytics Better, faster decision making 24

25 We ve seen this before. Personal computer Parallel computing, distributed processing Faster on demand access, interaction, and analytics Better, faster decision making 25

26 Evolving with your enterprise A good big data solution builds upon the solid technology foundation you have already laid. 26

27 The best big data solution will revolutionize the way you and your enterprise interact with data and each other. 27

28 Use cases revolutionized by big data solutions Real-time Credit & Market Risk in Banks A/B Testing Fraud Detection (Credit Card) & Financial Crimes (AML) in Banks Event-based Marketing in Financial Services and Telecomms Household Scoring Data Velocity Claims and Tax Fraud in Public Sector Predictive Maintenance in Aerospace Social Media Sentiment Analysis Demand Forecasting in Manufacturing Traditional Data Warehousing Disease Analysis on Electronic Health Records Market Sizing Video Surveillance & Analysis Batch Structured Semi-structured Unstructured Data Variety 28

29 Use cases revolutionized by big data solutions Real-time Credit & Market Risk in Banks A/B Testing Fraud Detection (Credit Card) & Financial Crimes (AML) in Banks Event-based Marketing in Financial Services and Telecomms Household Scoring Data Velocity Claims and Tax Fraud in Public Sector Predictive Maintenance in Aerospace Social Media Sentiment Analysis Demand Forecasting in Manufacturing Traditional Data Warehousing Disease Analysis on Electronic Health Records Market Sizing Video Surveillance & Analysis Batch Structured Semi-structured Unstructured Data Variety 29

30 A real-time data revolution Consume Analyze Visualize ENTERPRISE EHCACHE Volume Velocity Variety Historical Data Big Data Live Data 30

31 Visual tools facilitate real time analysis of disparate data Step 1 Step 2 Step 3 Consume User- Driven Analyze User- Driven Visualize 31

32 Real-time Analytics allows you to get the right data in front of the right people at the right time in ways you couldn t before. 32

33 Real-time Analytics Use Case #1: Situational Awareness 33

34 Real-time Analytics Use Case #2: Cyber Defense 34

35 Real-time Analytics Use Case #3: Financial Planning 35

36 Real-time Analytics Use Case #3: Financial Planning 36

37 Real-time Analytics Use Case #4: Data Transparency 37

38

39 Real-time Analytics Use Case #1: Situational Awareness 39

40 Real-time Analytics Use Case #1: Situational Awareness 40