DM: Can you trust the splits produced by classification trees?

The answer may be NO! I'm going to discuss the case where all attributes
are categorical. Exhaustive search algorithm (described in Breiman,
Friedman, Olshen & Stone, 1984) tends to select categorical attribute with
many levels as the split variable. For a categorical attribute with c
levels, you need to evaluate up to 2^{c-1} - 1 possible splits. So, the
more levels the attribute has, the more likely the attribute is selected as
the split variable just by chance.

On the other hand, CHAID and its derivatives (Kass, 1980; Hawkins & Kass,
1982; Biggs, de Ville & Suen, 1991) tend to select categorical attribute
with few levels. The algorithm penalizes categorical attributes with many
levels too severely. The adjustment proposed by Biggs, et al. (1991) seems
to be the least conservative, however.

QUEST, CRUISE, and PLUS also tend to select categorical attribute with few
levels when all categorical attributes are "equally informative" with
respect to the dependent variable. This is an artifact of the Pearson's
chi-square test for independence in a 2-way contingency table.

Hence, users of classification tree methods should exercise caution in
interpreting the resulting tree diagram when the categorical attributes
have varying levels. The selection bias won't occur when all categorical
attributes have the same number of levels. There won't be any serious bias
when all attributes are numerical and they have roughly comparable numbers
of distinct values.

The case of mixed attributes (numerical and categorical) is more
complicated and I haven't studied it deeply. My preliminary simulation
results (not for citation yet) can be downloaded from

    http://www.recursive-partitioning.com/plus/split.pdf

Thank your for your attention. I'd welcome any discussion/comment.

--
Tjen-Sien Lim
tslim@recursive-partitioning.com
www.Recursive-Partitioning.com
______________________________________________________________________
Get paid to write a review! http://recursive-partitioning.epinions.com

• Prev by Date: DM: ICEIS 2000 last cfp: deadline 31-Jan-2000
• Next by Date: DM: Datamining on the ineternet
• Prev by thread: DM: Datamining on the ineternet
• Next by thread: DM: ICEIS 2000 last cfp: deadline 31-Jan-2000
• Index(es):
  • Date
  • Thread