Commit graph

103 commits

Author SHA1 Message Date
olasaadi
7539ca0ead save checkpoints 2022-05-19 04:29:55 +03:00
olasaadi
521c8ce041 fix 2022-05-17 16:57:21 +03:00
abigailgold
fe676fa426
New model wrappers (#32)
* keras wrapper + blackbox classifier wrapper (fix #7)

* fix error in NCP calculation

* Update notebooks

* Fix #25 (incorrect attack_feature indexes for social feature in notebook)

* Consistent naming of internal parameters
2022-05-12 15:44:29 +03:00
olasaadi
f484135d84 add classes and implement some functions 2022-05-08 13:50:58 +03:00
abigailgold
fd6be8e778
Documentation updates (#29)
* Bump version to 0.1.0 (breaking changes to some APIs)

* Update documentation

* Update requirements

* gitignore
2022-05-02 11:46:18 +03:00
abigailgold
adc6b44350
Bump version and update documentation (#27)
* Bump version to 0.1.0 (breaking changes to some APIs)
* Update documentation
2022-05-01 15:18:07 +03:00
abigailgold
2b2dab6bef
Data and Model wrappers (#26)
* Squashed commit of wrappers:

    Wrapper minimizer

    * apply dataset wrapper on minimizer
    * apply changes on minimization notebook
    * add black_box_access and unlimited_queries params

    Dataset wrapper anonymizer

    Add features_names to ArrayDataset
    and allow providing features names in QI and Cat features not just indexes

    update notebooks

    categorical features and QI passed by indexes
    dataset include feature names and is_pandas param

    add pytorch Dataset

    Remove redundant code.
    Use data wrappers in model wrapper APIs.

    add generic dataset components 

    Create initial version of wrappers for models

* Fix handling of categorical features
2022-04-27 12:33:27 +03:00
abigailt
b8d2535231 Fix handling of categorical features 2022-04-25 17:41:15 +03:00
abigailt
a37ff06df8 Squashed commit of the following:
commit d53818644e
Author: olasaadi <92303887+olasaadi@users.noreply.github.com>
Date:   Mon Mar 7 20:12:55 2022 +0200

    Build the dt on all features anon (#23)

    * add param to build the DT on all features and not just on QI
    * one-hot encoding only for categorical features

commit c47819a031
Author: abigailt <abigailt@il.ibm.com>
Date:   Wed Feb 23 19:40:11 2022 +0200

    Update docs

commit 7e2ce7fe96
Merge: 7fbd1e4 752871d
Author: abigailt <abigailt@il.ibm.com>
Date:   Wed Feb 23 19:26:44 2022 +0200

    Merge remote-tracking branch 'origin/main' into main

commit 7fbd1e4b90
Author: abigailt <abigailt@il.ibm.com>
Date:   Wed Feb 23 19:22:54 2022 +0200

    Update version and docs

commit 752871dd0c
Author: olasaadi <92303887+olasaadi@users.noreply.github.com>
Date:   Wed Feb 23 14:57:12 2022 +0200

    add minimization notebook (#22)

    * add german credit notebook to showcase new features (minimize only some features and categorical features)

    * add notebook to show minimization data on a regression problem
2022-04-25 17:39:30 +03:00
ABIGAIL GOLDSTEEN
fb2413c4aa Fix boolean property return types 2022-04-19 15:24:14 +03:00
Ola Saadi
ac5d82aab6 Wrapper minimizer (#20)
* apply dataset wrapper on minimizer
* apply changes on minimization notebook
* add black_box_access and unlimited_queries params
2022-04-18 13:14:49 +03:00
Ola Saadi
5f6a258f8f Merge branch 'wrappers' into dataset_wrapper_anonimizer 2022-03-28 17:11:41 +03:00
olasaadi
31e278fe60 fix bug 2022-03-28 00:07:54 +03:00
olasaadi
6172385d91 update docstring 2022-03-28 00:07:23 +03:00
olasaadi
3bc1341dad update docstring 2022-03-28 00:07:04 +03:00
olasaadi
b54f0a2382 fix tests 2022-03-24 19:35:26 +02:00
olasaadi
66c86dc595 fix notebook and add features_names to ArrayDataset
and allow providing features names in QI and Cat features not just indexes
2022-03-24 19:32:24 +02:00
olasaadi
06158c8508 update 2022-03-23 17:54:37 +02:00
olasaadi
312469212e fix docstring and fix assert in test 2022-03-22 13:59:28 +02:00
olasaadi
5b3476071f fix 2022-03-21 21:59:14 +02:00
olasaadi
8aa7bb8281 categorical features and QI passed by indexes
dataset include feature names and is_pandas param
2022-03-21 21:57:11 +02:00
olasaadi
3263f92bee anonymizer works with numpy and return numpy/pandas as original dataset 2022-03-19 18:06:10 +02:00
olasaadi
7b788b9018 using dataset wrapper on anonymizer 2022-03-19 17:03:53 +02:00
Ron Shmelkin
a432b8f5f9 add pytorch Dataset 2022-03-19 17:03:44 +02:00
abigailt
a9162fbc43 Add more to wrappers 2022-03-19 17:03:44 +02:00
Ron Shmelkin
f99bf31030 add pytorch Dataset 2022-03-15 15:33:14 +02:00
abigailt
45cc9180b8 Add more to wrappers 2022-03-15 11:42:57 +02:00
olasaadi
a2b560920f apply changes 2022-03-15 10:52:45 +02:00
olasaadi
b4eddabd37 apply changes after rebase with wrappers 2022-03-10 13:49:05 +02:00
olasaadi
6afb175d6f add dataset wrapper to docstring 2022-03-10 13:42:49 +02:00
olasaadi
eefad206cd fix bug and update test_model 2022-03-10 13:42:47 +02:00
olasaadi
1280b849f4 using dataset wrapper on anonymizer 2022-03-10 13:41:57 +02:00
olasaadi
d53818644e
Build the dt on all features anon (#23)
* add param to build the DT on all features and not just on QI
* one-hot encoding only for categorical features
2022-03-07 20:12:55 +02:00
abigailt
f2df2fcc8c Renaming 2022-03-07 19:09:31 +02:00
abigailt
3d82db80c4 Remove redundant code.
Use data wrappers in model wrapper APIs.
More typing.
2022-03-06 21:15:07 +02:00
abigailt
c47819a031 Update docs 2022-02-23 19:40:11 +02:00
abigailt
7e2ce7fe96 Merge remote-tracking branch 'origin/main' into main 2022-02-23 19:26:44 +02:00
abigailt
7fbd1e4b90 Update version and docs 2022-02-23 19:22:54 +02:00
RON SHMELKIN
9f4d649934 add generic dataset components (#7)
* add generic dataset components
2022-02-22 12:00:04 +02:00
ABIGAIL GOLDSTEEN
b0c6c4d28e Create initial version of wrappers for models (#1)
* New wrapper classes for models
2022-02-10 15:36:41 +02:00
abigailgold
9de078f937
Update readme's with paper citations (#21) 2022-02-01 12:27:22 +02:00
olasaadi
3feebe8973
Regression minimization (#20)
* support regression in minimization and add test

* fix #10
2022-01-27 15:57:55 +02:00
olasaadi
cb9278ddb5
Support regression models (#19)
* support DecisionTreeRegressor

* support regression models

* Update membership_inference_dp_diabetes_reg.ipynb
2022-01-26 14:30:58 +02:00
olasaadi
a9a93c8a3a
Train just on qi (#15)
* QI updates
* update code to support training ML on QI features
* fix code so features that are not from QI should not be part of generalizations
and add description
* merging two branches, training on QI and on all data
* adding tests and asserts
2022-01-12 17:01:27 +02:00
olasaadi
2eb626c00c
Sup cat features (#14)
* support categorical features

* update the documentation and readme
added a test for the case where cells are supplied as a param.

* add big tests (adult test and iris)
and fixed bugs

* update transform to return numpy if original data is numpy

* added nursery test

* break loop if there is an illegal level

* Stop pruning one step before passing accuracy threshold

* adding asserts and fix DecisionTreeClassifier init

* Fix tests

Co-authored-by: abigailt <abigailt@il.ibm.com>
2022-01-11 09:51:04 +02:00
abigailgold
43952e2332
Minimization fixes (#12)
* Fixes related to corner cases in calculating generalizations

* Fix print

* Fix corner cases in transform as well

* Improve prints + bug fixes in calculation of feature to remove

* Notebook demonstrating ai minimization
2021-08-17 21:19:48 +03:00
abigailt
c06e2180e9 Fix images 2021-07-12 16:06:32 +03:00
abigailt
797f575252 Fix image 2021-07-12 16:05:22 +03:00
abigailgold
f2e1364b43
Add data minimization functionality to the ai-privacy-toolkit (#3)
* Fix directory issue when running tests for first time

* Initial version of data minimization

* Update version and documentation

* Fix documentation
2021-07-12 15:56:42 +03:00
abigailgold
bcc3d67ba4
Small fix + unified approach to numpy and pandas categorical data (#2) 2021-07-11 17:42:48 +03:00