# Periodic exports

In the platform, you can setup daily exports to your server.

## Configuration

Go to:

* Settings
* Account
* Periodic exports
* + New

There are two modes:

* **Specific export**: outputs a daily [CSV](https://en.wikipedia.org/wiki/Comma-separated_values) file with either
  * all participations for the given campaigns & contents (you can select optins contained in those resources to append their values to each line)
  * all selected user optins

After that, pick the Collective Audience™ resources you want to export.

* **Global export**: outputs a daily [ndJSON](http://ndjson.org) file with all the participations and all the optins from all your creatives

### Constraints

For the moment,

* **Specific exports** are only in the CSV format, and are transferred to your distant server using **SFTP**
* **Global exports** are only in the ndJSON format, and are transferred to your bucket on **Google Cloud Storage** (GCS). We have [a help page](https://../google-cloud-storage) if you're unsure about your credentials key and bucket name.

## Specific exports

### File naming

Every day, a new CSV file will arrive in the configured directory on your server.

The file will be named as follows:

```
YYYYMMDD_TYPE_NAME_UUID.csv
```

* `TYPE`: `crm-optin` or `crm-participation`
* `NAME`: *Slugified* name (e.g. `My export` will become `my-export`)
* `YYYYMMDD`: the current date
* `UUID`: a random ID

> As you can chose to run **multiple exports** every day, we recommend that you pick a different path for each. Don't worry if it's not the case, the random ID will ensure there's no name clash so that all the data is correctly written!

> ⚠️ As rights strategies can be very different from one case to the other, we will **not** create the directory, it will need to exist prior to the export.

### File contents

The CSV file will start with the following fixed columns:

* `email` (optional) string
* `phone` (optional) string
* `firstname` (optional) string
* `lastname` (optional) string
* `gender` (optional) string
* `birthdate` (optional) string
* `address-line-1` (optional) string
* `address-line-2` (optional) string
* `address-zip-code` (optional) string
* `address-city` (optional) string
* `address-country` (optional) string
* `creative-id` string
* `creative-name` string
* `campaign-id` (optional) string
* `campaign-name` (optional) string

And will end with **one column per selected optin**, named like the optin:

* `OPTIN_NAME` (e.g. `My Newsletter`): (optional) `true` if the user opted in

## Global exports

### File naming

Every day, a new **compressed** ndJSON file will arrive in the configured directory on your server.

The file will be named as follows:

```
YYYY-MM-DD_username.jsond.gz
```

* `YYYY-MM-DD`: the current date
* `username`: your username

### File contents

A *newline delimited JSON* is a simple text file where each line is a valid json value, here each line corresponds to a participation.

```txt
{"questions":[{"question_id" ( ... )
{"questions":[{"question_id" ( ... )
{"questions":[{"question_id" ( ... )
{"questions":[{"question_id" ( ... )
( ... )
```

Note that if any text value contains a newline character (`\n`), the latter is escaped (`\\n`) so that the remainder of the json is not confused with an actual new json entry on a new line. `\\n` should be handled transparently by any standard json parser.

Each participation json, if one were to expand it on multiple lines for better readability, has the following structure:

```json
{
  "questions" : [
    {
      "question_id" : QUESTION_ID,
      "question_text" : QUESTION_TEXT,
      "anwser_text" : ANSWER_TEXT
    },
    { ... }
  ],
  "email" : EMAIL,
  "optins" : [
    {
      "optin_email": EMAIL,
      "optin_text" : OPTIN_TEXT,
      "optin_id" : OPTIN_ID,
      "optin_value" : true
    },
    { ... }
  ],
  "forms" : [
    {
      "form_id" : FORM_ID,
      "form_name" : FORM_NAME,
      "form_data" : [
        {
          "field_id" : FIELD_ID,
          "field_name" : FIELD_NAME,
          "field_type" : FIELD_TYPE,
          "field_value" : FIELD_VALUE,
        },
        { ... }
      ]
    },
    { ... }
  ],
  "page_url" : URL_PAGE,
  "tc_string" : TC_STRING,
  "timestamp" : TIMESTAMP,
  "publisher_name" : PUBLISHER_NAME
  "publisher_id" : PUBLISHER_ID,
  "content_id" : CONTENT_ID,
  "content_name" : CONTENT_NAME,
  "content_version" : CONTENT_VERSION
}
```

---

**Documents**

- [Google Cloud Storage](https://outline.collectiveaudience.co/s/public/doc/google-cloud-storage-s5P0XdkasE)
- [Google BigQuery](https://outline.collectiveaudience.co/s/public/doc/google-bigquery-kiKGAYmeRi)