INTRODUCTION TO DATA MINING SAS ENTERPRISE MINER

Transcription

1 INTRODUCTION TO DATA MINING SAS ENTERPRISE MINER Mary-Elizabeth ( M-E ) Eddlestone Principal Systems Engineer, Analytics SAS Customer Loyalty, SAS Institute, Inc.

2 AGENDA Overview/Introduction to Data Mining and Predictive Modeling Building Models Using SAS Enterprise Miner Walk through example Essential steps: Sample, Explore, Modify, Model, Assess, Score Show selection of tools, how to change their properties and surface results Building Automated Models using Excel or SAS Enterprise Guide (Rapid Predictive Modeler)

3 INTRODUCTION TO DATA MINING

4

5 DATA MINING GOALS INSIGHT AGILE or DYNAMIC PERSONALIZATION SPEED PRECISION IMPROVED PROFITABILITY Better Decisions

6

7 ANALYTICS INFERENTIAL Inferential Statistics Uses patterns in the sample data to draw inferences about the population represented, accounting for randomness Answering yes/no questions about the data (hypothesis testing) Describing associations within the data (correlation) Modeling relationships within the data (regression) Source: Wikipedia

8 ANALYTICS PREDICTIVE Predictive Analytics Encompasses a variety of techniques from statistics, modeling, machine learning, and data mining that analyze current and historical facts to make predictions about future, or otherwise unknown, events. Include: Data Mining Forecasting Source: Wikipedia

9 ANALYTICS DATA MINING VERSUS FORECASTING Both are predictive and both model past behavior. DATA MINING Time independent Casual (relationship) focused Categorical, Continuous, Discrete Seldom weight more recent observations FORECASTING Time dependent Interval oriented Continuity assumed Frequently weights more recent phenomena

10 DATA MINING Descriptive Data Mining Predictive Data Mining

11 DATA MINING Descriptive Data Mining Clustering (Segmentation) Associations and Sequences Predictive Data Mining Classification Models to predict class membership Regression Models to predict a number

12 THE GOAL? SCORING! Scoring is the act of applying what we ve learned from data mining to new cases. Keep this goal in mind and use it to help formulate the questions and the data needed for data mining and scoring.

13 THE ULTIMATE GOAL? BETTER DECISIONS The ultimate goal of data mining is to improve decision making. As you formulate your problem, also keep in mind how and when model scores will be used.

14 EXAMPLE DEVELOPING A CLASSIFICATION MODEL Models are developed using historical data in which the behavior is observed or known. Indicates the behavior was observed in this subject Information about each subject, in this case an individual, is used as inputs to the model to see how well the model can distinguish between the people who exhibit the behavior and those who do not. For example, age, gender, previous behaviors, etc.

15 EXAMPLE DATA

16 WHY? Consider a group of subjects whose relevant behavior is unknown. The same information is available for each of these subjects (age, gender, etc.) as is available for the individuals with known behavior. We would like to know which individuals are most likely to have the relevant behavior.

17 EXAMPLE NEW DATA?

18 SCORING The output of a predictive classification model output is typically an equation. Models are applied to new cases to calculate the predicted behavior through a process called scoring. Scoring, using the equation, calculates each subject s likelihood to have the relevant behavior. (It also calculates the likelihood to not have the behavior.)

19 EXAMPLE SCORED DATA

20 THE ANALYTICS LIFECYCLE BUSINESS MANAGER Domain Expert Makes Decisions Evaluates Processes and ROI EVALUATE / MONITOR RESULTS IDENTIFY / FORMULATE PROBLEM DATA PREPARATION BUSINESS ANALYST Data Exploration Data Visualization Report Creation DEPLOY MODEL DATA EXPLORATION IT SYSTEMS / MANAGEMENT Data Preparation Model Validation Model Deployment Model Monitoring VALIDATE MODEL BUILD MODEL TRANSFORM & SELECT DATA MINER / STATISTICIAN Exploratory Analysis Descriptive Segmentation Predictive Modeling

21 THE ANALYTICS LIFECYCLE EVALUATE / MONITOR RESULTS IDENTIFY / FORMULATE PROBLEM DATA PREPARATION DEPLOY MODEL DATA EXPLORATION VALIDATE MODEL BUILD MODEL TRANSFORM & SELECT

22 MAIN TYPES OF DATA MARTS One-Row-per- Subject Data Mart Multiple-Row-per- Subject Data Mart Longitudinal Data Mart

23 THE ANALYTICS LIFECYCLE SAS Enterprise Miner focuses on these aspects of the process. DEPLOY MODEL EVALUATE / MONITOR RESULTS IDENTIFY / FORMULATE PROBLEM DATA PREPARATION DATA EXPLORATION VALIDATE MODEL BUILD MODEL TRANSFORM & SELECT DATA MINER / STATISTICIAN Exploratory Analysis Descriptive Segmentation Predictive Modeling

24 SAS ENTERPRISE MINER

25 SAS ENTERPRISE MINER

26 SAS ENTERPRISE MINER Organized and logical GUI for data mining success Unmatched suite of modeling techniques and methods Sophisticated set of data preparation, summarization and exploration tools Business-based model comparisons, reporting and management

27 SAS ENTERPRISE MINER Automated scoring process delivers faster results High-performance gridenabled workbench Modern, distributable data mining system suited for large enterprises Open, extensible design for ultimate flexibility

28 WHAT IS SAS ENTERPRISE MINER? SAS Enterprise Miner is a sophisticated graphical user interface, designed with the specific needs of data miners in mind. SAS Enterprise Miner is a data miner s workbench that manages the process and provides a comprehensive set of tools to aid the data miner throughout the essential steps, known by the acronym, SEMMA: Sample, Explore, Modify, Model, Assess. SAS Enterprise Miner streamlines the data mining process to create highly accurate predictive and descriptive models based on analysis of vast amounts of data from across an enterprise.

29 DATA MINING WITH SAS ENTERPRISE MINER

30 SAS ENTERPRISE MINER 7.1 AND 12.1 MODEL DEVELOPMENT PROCESS (SEMMA) Sample Explore Modify Model Assess Utility

31 SAS ENTERPRISE MINER

32 SAS ENTERPRISE MINER Use the desired tools to define a logical process (SEMMA) Sample Explore Modify Model Assess

33 SAS ENTERPRISE MINER Modify settings (properties) for the tools.

34 SAS ENTERPRISE MINER Run the flow and check results. Refine as needed.

35 DEMONSTRATION

36 AUTOMATED PREDICTIVE MODELING

37

38 SAS RAPID PREDICTIVE MODELER KEY DRIVERS (BUSINESS USERS) Need to generate numerous models to solve a variety of business problems in a credible manner Models need to be developed in a quick timeframe using a self-service approach Does not want to always rely on analytic professionals (e.g. statistician or modeler or data miner)

39 SAS RAPID PREDICTIVE MODELER KEY DRIVERS (ANALYTIC PROFESSIONALS) Solving more complex issues on hand to gain incremental value Further customize or refine models for better results

40 RAPID PREDICTIVE MODELER

41 Open your data in SAS Enterprise Guide or Microsoft Excel Use the Rapid Predictive Modeler task and modify settings Review results

42 Microsoft Excel

43 SAS Enterprise Guide

44

45

46 RAPID PREDICTIVE MODELER BASIC

47

48 RAPID PREDICTIVE MODELER INTERMEDIATE

49

50 RAPID PREDICTIVE MODELER ADVANCED

51 RAPID PREDICTIVE MODELER: SAMPLE OUTPUT

52 Rapid Predictive Modeler: Sample Output

53 Rapid Predictive Modeler: Sample Output

54 Rapid Predictive Modeler: Sample Output

55 DEMONSTRATION

56 IN CONCLUSION

57 SAS ENTERPRISE MINER BENEFITS Support the entire data mining process with a broad set of tools. Build more models faster with an easy-to-use Graphical User Interface. Enhance accuracy of predictions Surface business information and easily share results through the unique model repository

58 RESOURCES SAS Rapid Predictive Modeler Website Product brief, Press release, Brief product demo, etc. SAS Enterprise Miner Web Site SAS Enterprise Miner Technical Support Web Site SAS Enterprise Miner Technical Forum (Join Today!) SAS Enterprise Miner Training Rapid Predictive Modeling for Customer Intelligence SAS Global Forum 2010 paper written by Wayne Thompson and David Duling, SAS Institute Inc., Cary, NC

59 POTENTIAL NEXT STEPS Work through the example in Getting Started with SAS Enterprise Miner - Both the data and the documentation are available on support.sas.com Contact SAS Technical Support if you get stuck There is no charge for this it is included in your SAS software license.

60

61 THANK YOU FOR USING SAS!