Examples

To get a better insight of how to use CARLA for the different purposes, we will provide here some short example implementations.

To benchmark an arbitrary recourse method, we provide an example implementation based on Quickstart, in the section Benchmarking

Quickstart

In this case, we want to use a recourse method to generate counterfactual examples with pre-implemented dataset and black-box-model.

 1 from carla import DataCatalog, MLModelCatalog
 2 from carla.recourse_methods import GrowingSpheres
 3
 4 # 1. Load data set from the DataCatalog
 5 data_name = "adult"
 6 dataset = DataCatalog(data_name)
 7
 8 # 2. Load pre-trained black-box model from the MLModelCatalog
 9 model = MLModelCatalog(dataset, "ann")
10
11 # 3. Load recourse model with model specific hyperparameters
12 gs = GrowingSpheres(model)
13
14 # 4. Generate counterfactual examples
15 factuals = dataset.raw.sample(10)
16 counterfactuals = gs.get_counterfactuals(factuals)

Customization

The following examples will contain some pseudo-code to visualize how a Data, Black-Box-Model, or Recourse Method implementation would look like. The structure to generate counterfactuals with these user specific implementations would still resemble Quickstart.

Data

 1from carla import Data
 2
 3# Custom data set implementations need to inherit from the Data interface
 4class MyOwnDataSet(Data):
 5    def __init__(self):
 6        # The data set can e.g. be loaded in the constructor
 7        self._dataset = load_dataset_from_disk()
 8
 9    # List of all categorical features
10    @property
11    def categoricals(self):
12        return [...]
13
14    # List of all continous features
15    @property
16    def continous(self):
17        return [...]
18
19    # List of all immutable features which
20    # should not be changed by the recourse method
21    @property
22    def immutables(self):
23        return [...]
24
25    # Feature name of the target column
26    @property
27    def target(self):
28        return "label"
29
30    # Non-encoded and  non-normalized, raw data set
31    @property
32    def raw(self):
33        return self._dataset

Black-Box-Model

 1 from carla import MLModel
 2
 3 # Custom black-box models need to inherit from
 4 # the MLModel interface
 5 class MyOwnModel(MLModel):
 6     def __init__(self, data):
 7         super().__init__(data)
 8         # The constructor can be used to load or build an
 9         # arbitrary black-box-model
10         self._mymodel = load_model()
11
12         # Define a fitted sklearn scaler to normalize input data
13         self.scaler = MySklearnScaler().fit()
14
15         # Define a fitted sklearn encoder for binary input data
16         self.encoder = MySklearnEncoder.fit()
17
18     # List of the feature order the ml model was trained on
19     @property
20     def feature_input_order(self):
21         return [...]
22
23     # The ML framework the model was trained on
24     @property
25     def backend(self):
26         return "pytorch"
27
28     # The black-box model object
29     @property
30     def raw_model(self):
31         return self._mymodel
32
33     # The predict function outputs
34     # the continous prediction of the model
35     def predict(self, x):
36         return self._mymodel.predict(x)
37
38     # The predict_proba method outputs
39     # the prediction as class probabilities
40     def predict_proba(self, x):
41         return self._mymodel.predict_proba(x)

Recourse Method

 1from carla import RecourseMethod
 2
 3 # Custom recourse implementations need to
 4 # inherit from the RecourseMethod interface
 5 class MyRecourseMethod(RecourseMethod):
 6     def __init__(self, mlmodel):
 7         super().__init__(mlmodel)
 8
 9     # Generate and return encoded and
10     # scaled counterfactual examples
11     def get_counterfactuals(self, factuals: pd.DataFrame):
12             [...]
13             return counterfactual_examples

Benchmarking

 1 from carla import Benchmark
 2
 3 # 1. Initilize the benchmarking class by passing
 4 # black-box-model, recourse method, and factuals into it
 5 benchmark = Benchmark(model, gs, factuals)
 6
 7 # 2. Either only compute the distance measures
 8 distances = benchmark.compute_distances()
 9
10 # 3. Or run all implemented measurements and create a
11 # DataFrame which consists of all results
12 results = benchmark.run_benchmark()