Synthetic Data Generation

Sarus synthetic generation package. It provides utilities and a model, the MetaLearner that can be trained to generate synthetic features from a given dataset.

Place in the Data Analysis

In the standard pipeline of reading the csv, converting etc, it sits at the very end of the process after data classification and transcoding:

• Given a table (read from a csv file for example), a first study has mapped each column to an atomic/semantic type (for example Credit Card, Text, Images, Category).
• Each column has been transcoded: i.e. its representation has been transformed into a simpler one. For example, a categorical column has been one_hot encoded; a date has been mapped to three integers giving year, month, day; a text has been tokenized…

Documentation

The MetaLearner structure and functioning is detailed here. A more detailed documentation will be provided soon. A script to run an example with categorical/real data and one with images is given in the examples folder.

Implemented so far

Models

• MetaLearner: main data generator
• ConditionalGAN: model for images generation
• GPT2Decoder: pre-trained model for text generation

Codecs

• OneHotCodec: handles all data that can be one hot encoded in a one dimensional way (category, continuous). Uses a dense layer as an encoder.
• ImageCodec: handles images. The encoder is a convolutional network and the decoder a GAN.
• TextCodec: handles text. The encoder and decoder are two pre-trained GPT models. More documentation can be found here.

Layers

• Transformer layers (TransformerBlock,Meta_Learner_Core)
• Layers to handle images (Generator, Critic and sub-layers...)