Non-parametric Conditional Independence Testing for Mixed Continuous-Categorical Variables: A Novel Method and Numerical Evaluation

Research output: Contribution to book/Conference proceedings/Anthology/ReportConference contributionContributedpeer-review

Contributors

Abstract

Conditional independence testing (CIT) is a common task in machine learning, e.g., for variable selection, and a main component of constraint-based causal discovery. While most current CIT approaches assume that all variables in a dataset are of the same type, either numerical or categorical, many real-world applications involve mixed-type datasets that include both numerical and categorical variables. Non-parametric CIT can be conducted using conditional mutual information (CMI) estimators combined with a local permutation scheme. Recently, two novel CMI estimators for mixed-type datasets based on k-nearest-neighbors (k-NN) have been proposed. As with any k-NN method, these estimators rely on the definition of a distance metric. One approach computes distances by a one-hot encoding of the categorical variables, essentially treating categorical variables as discrete-numerical, while the other expresses CMI by entropy terms where the categorical variables appear as conditions only. In this work, we study these estimators and propose a variation of the former approach that does not treat categorical variables as numeric. Extensive numerical experiments show that our variant detects dependencies more robustly across different data distributions and preprocessing types.

Details

Original languageEnglish
Title of host publicationProceedings of the Fourth Conference on Causal Learning and Reasoning
EditorsBiwei Huang, Mathias Drton
Pages406-450
Number of pages45
Publication statusPublished - 2025
Peer-reviewedYes

Publication series

SeriesProceedings of Machine Learning Research
Volume275

Conference

Title4th Conference on Causal Learning and Reasoning
Abbreviated titleCLeaR 2025
Conference number4
Duration7 - 9 May 2025
Website
LocationSwissTech Convention Center
CityLausanne
CountrySwitzerland

Keywords

Keywords

  • conditional independence testing, conditional mutual information, mixed-type data