mirror of
https://github.com/IBM/ai-privacy-toolkit.git
synced 2026-04-25 04:46:21 +02:00
2b2dab6bef
* Squashed commit of wrappers:
Wrapper minimizer
* apply dataset wrapper on minimizer
* apply changes on minimization notebook
* add black_box_access and unlimited_queries params
Dataset wrapper anonymizer
Add features_names to ArrayDataset
and allow providing features names in QI and Cat features not just indexes
update notebooks
categorical features and QI passed by indexes
dataset include feature names and is_pandas param
add pytorch Dataset
Remove redundant code.
Use data wrappers in model wrapper APIs.
add generic dataset components
Create initial version of wrappers for models
* Fix handling of categorical features
448 lines
No EOL
15 KiB
Text
448 lines
No EOL
15 KiB
Text
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Using ML anonymization to defend against membership inference attacks"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"In this tutorial we will show how to anonymize models using the ML anonymization module. \n",
|
|
"\n",
|
|
"We will demonstrate running inference attacks both on a vanilla model, and then on an anonymized version of the model. We will run a black-box membership inference attack using ART's inference module (https://github.com/Trusted-AI/adversarial-robustness-toolbox/tree/main/art/attacks/inference). \n",
|
|
"\n",
|
|
"This will be demonstarted using the Adult dataset (original dataset can be found here: https://archive.ics.uci.edu/ml/datasets/adult). \n",
|
|
"\n",
|
|
"For simplicity, we used only the numerical features in the dataset."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Load data"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 6,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"[[ 39. 13. 2174. 0. 40.]\n",
|
|
" [ 50. 13. 0. 0. 13.]\n",
|
|
" [ 38. 9. 0. 0. 40.]\n",
|
|
" ...\n",
|
|
" [ 27. 13. 0. 0. 40.]\n",
|
|
" [ 26. 11. 0. 0. 48.]\n",
|
|
" [ 27. 9. 0. 0. 40.]]\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stderr",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"/var/folders/9b/qbtw28w53355cvpjs4qn83yc0000gn/T/ipykernel_85828/3975777015.py:22: DeprecationWarning: `np.int` is a deprecated alias for the builtin `int`. To silence this warning, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.\n",
|
|
"Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations\n",
|
|
" y_train = y_train.astype(np.int)\n",
|
|
"/var/folders/9b/qbtw28w53355cvpjs4qn83yc0000gn/T/ipykernel_85828/3975777015.py:26: DeprecationWarning: `np.int` is a deprecated alias for the builtin `int`. To silence this warning, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.\n",
|
|
"Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations\n",
|
|
" y_test = y_test.astype(np.int)\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"import numpy as np\n",
|
|
"\n",
|
|
"# Use only numeric features (age, education-num, capital-gain, capital-loss, hours-per-week)\n",
|
|
"x_train = np.loadtxt(\"https://archive.ics.uci.edu/ml/machine-learning-databases/adult/adult.data\",\n",
|
|
" usecols=(0, 4, 10, 11, 12), delimiter=\", \")\n",
|
|
"\n",
|
|
"y_train = np.loadtxt(\"https://archive.ics.uci.edu/ml/machine-learning-databases/adult/adult.data\",\n",
|
|
" usecols=14, dtype=str, delimiter=\", \")\n",
|
|
"\n",
|
|
"\n",
|
|
"x_test = np.loadtxt(\"https://archive.ics.uci.edu/ml/machine-learning-databases/adult/adult.test\",\n",
|
|
" usecols=(0, 4, 10, 11, 12), delimiter=\", \", skiprows=1)\n",
|
|
"\n",
|
|
"y_test = np.loadtxt(\"https://archive.ics.uci.edu/ml/machine-learning-databases/adult/adult.test\",\n",
|
|
" usecols=14, dtype=str, delimiter=\", \", skiprows=1)\n",
|
|
"\n",
|
|
"# Trim trailing period \".\" from label\n",
|
|
"y_test = np.array([a[:-1] for a in y_test])\n",
|
|
"\n",
|
|
"y_train[y_train == '<=50K'] = 0\n",
|
|
"y_train[y_train == '>50K'] = 1\n",
|
|
"y_train = y_train.astype(np.int)\n",
|
|
"\n",
|
|
"y_test[y_test == '<=50K'] = 0\n",
|
|
"y_test[y_test == '>50K'] = 1\n",
|
|
"y_test = y_test.astype(np.int)\n",
|
|
"\n",
|
|
"# get balanced dataset\n",
|
|
"x_train = x_train[:x_test.shape[0]]\n",
|
|
"y_train = y_train[:y_test.shape[0]]\n",
|
|
"\n",
|
|
"print(x_train)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Train decision tree model"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 7,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Base model accuracy: 0.8074442601805786\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"from sklearn.tree import DecisionTreeClassifier\n",
|
|
"from art.estimators.classification.scikitlearn import ScikitlearnDecisionTreeClassifier\n",
|
|
"\n",
|
|
"model = DecisionTreeClassifier()\n",
|
|
"model.fit(x_train, y_train)\n",
|
|
"\n",
|
|
"art_classifier = ScikitlearnDecisionTreeClassifier(model)\n",
|
|
"\n",
|
|
"print('Base model accuracy: ', model.score(x_test, y_test))\n",
|
|
"\n",
|
|
"x_train_predictions = np.array([np.argmax(arr) for arr in art_classifier.predict(x_train)]).reshape(-1,1)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Attack\n",
|
|
"The black-box attack basically trains an additional classifier (called the attack model) to predict the membership status of a sample.\n",
|
|
"#### Train attack model"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 8,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stderr",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"/Users/olasaadi/PycharmProjects/ai-privacy-toolkit-internal/venv/lib/python3.8/site-packages/art/attacks/inference/membership_inference/black_box.py:262: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples,), for example using ravel().\n",
|
|
" self.attack_model.fit(np.c_[x_1, x_2], y_ready) # type: ignore\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"from art.attacks.inference.membership_inference import MembershipInferenceBlackBox\n",
|
|
"\n",
|
|
"# attack_model_type can be nn (neural network), rf (randon forest) or gb (gradient boosting)\n",
|
|
"bb_attack = MembershipInferenceBlackBox(art_classifier, attack_model_type='rf')\n",
|
|
"\n",
|
|
"# use half of each dataset for training the attack\n",
|
|
"attack_train_ratio = 0.5\n",
|
|
"attack_train_size = int(len(x_train) * attack_train_ratio)\n",
|
|
"attack_test_size = int(len(x_test) * attack_train_ratio)\n",
|
|
"\n",
|
|
"# train attack model\n",
|
|
"bb_attack.fit(x_train[:attack_train_size], y_train[:attack_train_size],\n",
|
|
" x_test[:attack_test_size], y_test[:attack_test_size])"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"#### Infer sensitive feature and check accuracy"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 9,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"0.545264709495148\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"# get inferred values for remaining half\n",
|
|
"inferred_train_bb = bb_attack.infer(x_train[attack_train_size:], y_train[attack_train_size:])\n",
|
|
"inferred_test_bb = bb_attack.infer(x_test[attack_test_size:], y_test[attack_test_size:])\n",
|
|
"# check accuracy\n",
|
|
"train_acc = np.sum(inferred_train_bb) / len(inferred_train_bb)\n",
|
|
"test_acc = 1 - (np.sum(inferred_test_bb) / len(inferred_test_bb))\n",
|
|
"acc = (train_acc * len(inferred_train_bb) + test_acc * len(inferred_test_bb)) / (len(inferred_train_bb) + len(inferred_test_bb))\n",
|
|
"print(acc)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"This means that for 54% of the data, membership is inferred correctly using this attack."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Anonymized data\n",
|
|
"## k=100\n",
|
|
"\n",
|
|
"Now we will apply the same attacks on an anonymized version of the same dataset (k=100). The data is anonymized on the quasi-identifiers: age, education-num, capital-gain, hours-per-week.\n",
|
|
"\n",
|
|
"k=100 means that each record in the anonymized dataset is identical to 99 others on the quasi-identifier values (i.e., when looking only at those features, the records are indistinguishable)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 10,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"[[38. 13. 0. 0. 40.]\n",
|
|
" [57. 13. 0. 0. 30.]\n",
|
|
" [37. 9. 0. 0. 40.]\n",
|
|
" ...\n",
|
|
" [26. 13. 0. 0. 40.]\n",
|
|
" [29. 10. 0. 0. 50.]\n",
|
|
" [25. 9. 0. 0. 40.]]\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"from apt.utils.datasets import ArrayDataset\n",
|
|
"import os\n",
|
|
"import sys\n",
|
|
"sys.path.insert(0, os.path.abspath('..'))\n",
|
|
"from apt.anonymization import Anonymize\n",
|
|
"\n",
|
|
"# QI = (age, education-num, capital-gain, hours-per-week)\n",
|
|
"QI = [0, 1, 2, 4]\n",
|
|
"anonymizer = Anonymize(100, QI)\n",
|
|
"anon = anonymizer.anonymize(ArrayDataset(x_train, x_train_predictions))\n",
|
|
"print(anon)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 11,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": "6739"
|
|
},
|
|
"execution_count": 11,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"# number of distinct rows in original data\n",
|
|
"len(np.unique(x_train, axis=0))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 12,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": "658"
|
|
},
|
|
"execution_count": 12,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"# number of distinct rows in anonymized data\n",
|
|
"len(np.unique(anon, axis=0))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Train decision tree model"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 13,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Anonymized model accuracy: 0.83078434985566\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"anon_model = DecisionTreeClassifier()\n",
|
|
"anon_model.fit(anon, y_train)\n",
|
|
"\n",
|
|
"anon_art_classifier = ScikitlearnDecisionTreeClassifier(anon_model)\n",
|
|
"\n",
|
|
"print('Anonymized model accuracy: ', anon_model.score(x_test, y_test))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Attack\n",
|
|
"### Black-box attack"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 14,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stderr",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"/Users/olasaadi/PycharmProjects/ai-privacy-toolkit-internal/venv/lib/python3.8/site-packages/art/attacks/inference/membership_inference/black_box.py:262: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples,), for example using ravel().\n",
|
|
" self.attack_model.fit(np.c_[x_1, x_2], y_ready) # type: ignore\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"0.5047291487532244\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"anon_bb_attack = MembershipInferenceBlackBox(anon_art_classifier, attack_model_type='rf')\n",
|
|
"\n",
|
|
"# train attack model\n",
|
|
"anon_bb_attack.fit(x_train[:attack_train_size], y_train[:attack_train_size],\n",
|
|
" x_test[:attack_test_size], y_test[:attack_test_size])\n",
|
|
"\n",
|
|
"# get inferred values\n",
|
|
"anon_inferred_train_bb = anon_bb_attack.infer(x_train[attack_train_size:], y_train[attack_train_size:])\n",
|
|
"anon_inferred_test_bb = anon_bb_attack.infer(x_test[attack_test_size:], y_test[attack_test_size:])\n",
|
|
"# check accuracy\n",
|
|
"anon_train_acc = np.sum(anon_inferred_train_bb) / len(anon_inferred_train_bb)\n",
|
|
"anon_test_acc = 1 - (np.sum(anon_inferred_test_bb) / len(anon_inferred_test_bb))\n",
|
|
"anon_acc = (anon_train_acc * len(anon_inferred_train_bb) + anon_test_acc * len(anon_inferred_test_bb)) / (len(anon_inferred_train_bb) + len(anon_inferred_test_bb))\n",
|
|
"print(anon_acc)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Attack accuracy is reduced to 50% (eqiuvalent to random guessing)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 15,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"(0.5312420517168291, 0.7696843139663432)\n",
|
|
"(0.5048372911169745, 0.4935511607910576)\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"def calc_precision_recall(predicted, actual, positive_value=1):\n",
|
|
" score = 0 # both predicted and actual are positive\n",
|
|
" num_positive_predicted = 0 # predicted positive\n",
|
|
" num_positive_actual = 0 # actual positive\n",
|
|
" for i in range(len(predicted)):\n",
|
|
" if predicted[i] == positive_value:\n",
|
|
" num_positive_predicted += 1\n",
|
|
" if actual[i] == positive_value:\n",
|
|
" num_positive_actual += 1\n",
|
|
" if predicted[i] == actual[i]:\n",
|
|
" if predicted[i] == positive_value:\n",
|
|
" score += 1\n",
|
|
" \n",
|
|
" if num_positive_predicted == 0:\n",
|
|
" precision = 1\n",
|
|
" else:\n",
|
|
" precision = score / num_positive_predicted # the fraction of predicted “Yes” responses that are correct\n",
|
|
" if num_positive_actual == 0:\n",
|
|
" recall = 1\n",
|
|
" else:\n",
|
|
" recall = score / num_positive_actual # the fraction of “Yes” responses that are predicted correctly\n",
|
|
"\n",
|
|
" return precision, recall\n",
|
|
"\n",
|
|
"# regular\n",
|
|
"print(calc_precision_recall(np.concatenate((inferred_train_bb, inferred_test_bb)), \n",
|
|
" np.concatenate((np.ones(len(inferred_train_bb)), np.zeros(len(inferred_test_bb))))))\n",
|
|
"# anon\n",
|
|
"print(calc_precision_recall(np.concatenate((anon_inferred_train_bb, anon_inferred_test_bb)), \n",
|
|
" np.concatenate((np.ones(len(anon_inferred_train_bb)), np.zeros(len(anon_inferred_test_bb))))))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Precision and recall are also reduced."
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.8.3"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 2
|
|
} |