Course Forum

You are welcome to use it, but it is important to realize what it is doing and make some informed choices.  It can make quite a few changes to your dataset.  It can:

1. Convert dates into elapsed time in, say, days.
2. Extract months or days from a date field.
3. Exclude variables with too much missing data.
4. Exclude categorical variables with too many categories (defaults are >100 for nominal and >10 for ordinal variables) or too much data concentrated in one category.
5. Exclude numeric variables with fewer than 5 categories (or change the type).
6. Replace outliers with other values.
7. Replace missing data with imputed values.
8. Rescale numeric variables to have mean 0 and standard deviation 1.
9. Search for transformations of the outcome variable to make it more bell-shaped.
10. Combine categories of categorical variables when they have too few values.
11. Exclude variables from the model building if they are not statistically significantly related to the target.

All of the above are commonly used and frequently sensible steps in a data analysis.   But not always.  For example, in the NEISS dataset a key predictor variable for whether someone was admitted to the hospital was product type.  In fact, that is the main reason the database was created in the first case – to monitor product related injuries.  But because product type has >100 categories, it would automatically be dropped from an analysis if you used the Automatic Data Processing node with its defaults.  Used more sensibly, you can increase the maximum number of categories in the Exclude Input Fields tab to make sure it does not get excluded.