Doc - Maskings - Forward port improvement from 3.3 (3.4) (#8069)

CHANGELOG

@@ -9,7 +9,8 @@ v3.4.3 (XXXX-XX-XX)
   usage of the executed query. In a cluster, the value contains the peak memory
   usage across all shards, but it is not summed up across shards.
 
-* fixes in data masking: better documentation, default phone number
+* data masking: better documentation, fixed default phone number,
+  changed default range to -100 and 100 for integer masking function
 
 
 v3.4.2 (2019-01-24)

Documentation/Books/Manual/Programs/Arangodump/Maskings.md

@@ -205,24 +205,23 @@ are not contained in an attribute value of which the attribute name is
 
 If you specify a path and the attribute value is an array then the
 masking decision is applied to each element of the array as if this
-was the value of the attribute.
+was the value of the attribute. This applies to arrays inside the array too.
 
 If the attribute value is an object, then it is ignored and the attribute
 does not get masked. To mask nested fields, specify the full path for each
 leaf attribute.
 
 {% hint 'tip' %}
-If some documents have an attribute `name` with a string as value, but other
+If some documents have an attribute `email` with a string as value, but other
 documents store a nested object under the same attribute name, then make sure
 to set up proper masking for the latter case, in which sub-attributes will not
-get masked if there is only a masking configured for the attribute `name`
+get masked if there is only a masking configured for the attribute `email`
 but not its nested attributes.
 {% endhint %}
 
-**Example**
+**Examples**
-
-Masking `email` will convert:
 
+Masking `email` with the _Xify Front_ function will convert:
 
 ```json
 {
@@ -244,7 +243,10 @@ because `email` is a leaf attribute. The document:
 {
   "email" : [
     "address one",
-    "address two"
+    "address two",
+    [
+      "address three"
+    ]
   ]
 }
 ```
@@ -255,12 +257,16 @@ because `email` is a leaf attribute. The document:
 {
   "email" : [
     "xxxxxss xne",
-    "xxxxxss xwo"
+    "xxxxxss xwo",
+    [
+      "xxxxxss xxxee"
+    ]
   ]
 }
 ```
 
-… because the array is "unfolded". The document:
+… because the masking is applied to each array element individually
+including the elements of the sub-array. The document:
 
 ```json
 {
@@ -271,6 +277,8 @@ because `email` is a leaf attribute. The document:
 ```
 
 … will not be changed because `email` is not a leaf attribute.
+To mask the email address, you could use the paths `email.address`
+or `.address`.
 
 
 Masking Functions
@@ -393,8 +401,8 @@ Masking settings:
 
 - `path` (string): which field to mask
 - `type` (string): masking function name `"xifyFront"`
-- `unmaskedLength` (integer<!--, _default: ???_-->): how many characters to
+- `unmaskedLength` (number, _default: `2`_): how many characters to
-  leave as-is on the right-hand side of each word
+  leave as-is on the right-hand side of each word as integer value
 - `hash` (bool, _default: `false`_): whether to append a hash value to the
   masked string to avoid possible unique constraint violations caused by
   the obfuscation
@@ -462,8 +470,19 @@ a number which must not be `0`.
 
 ### Zip
 
-This masking type replaces a zip code with a random one. If the
+This masking type replaces a zip code with a random one.
-attribute value is not a string then the default value is used.
+It uses the following rules:
+
+- If a character of the original zip code is a digit it will be replaced
+  by a random digit.
+- If a character of the original zip code is a letter it
+  will be replaced by a random letter keeping the case.
+- If the attribute value is not a string then the default value is used.
+
+Note that this will generate random zip codes. Therefore there is a
+chance that the same zip code value is generated multiple times, which can
+cause unique constraint violations if a unique index is or will be
+used on the zip code attribute.
 
 Masking settings:
 
@@ -481,10 +500,8 @@ Masking settings:
 }
 ```
 
-This will replace a real zip code with a random one. It uses the following
+This replaces real zip codes stored in fields called `code` at any level
-rule: If a character of the original zip code is a digit it will be replaced
+with random ones. `"12345"` is used as fallback value.
-by a random digit. If a character of the original zip code is a letter it
-will be replaced by a random letter keeping the case.
 
 ```json
 {
@@ -510,10 +527,8 @@ If the original zip code is:
 
     OW91-JI
 
-Note that this will generate random zip codes. Therefore there is a
+If the original zip code is `null`, `true`, `false` or a number, then the
-chance that the same zip code value is generated multiple times, which can
+user-defined default value of `"abcdef"` will be used.
-cause unique constraint violations if a unique index is or will be
-used on the zip code attribute.
 
 ### Datetime
 
@@ -524,12 +539,16 @@ Masking settings:
 
 - `path` (string): which field to mask
 - `type` (string): masking function name `"datetime"`
-- `begin` (string): earliest point in time to return.
+- `begin` (string, _default: `"1970-01-01T00:00:00.000"`_):
-  Date time string in ISO 8601 format
+  earliest point in time to return. Date time string in ISO 8601 format.
-- `end` (string): latest point in time to return.
+- `end` (string, _default: now_):
-  Date time string in ISO 8601 format
+  latest point in time to return. Date time string in ISO 8601 format.
-- `format` (string<!--, default: ???-->): the formatting string format is
+  In case a partial date time string is provided (e.g. `2010-06` without day
+  and time) the earliest date and time is assumed (`2010-06-01T00:00:00.000`).
+  The default value is the current system date and time.
+- `format` (string, _default: `""`_): the formatting string format is
   described in [DATE_FORMAT()](../../../AQL/Functions/Date.html#dateformat).
+  If no format is specified, then the result will be an empty string.
 
 **Example**
 
@@ -557,8 +576,8 @@ Masking settings:
 
 - `path` (string): which field to mask
 - `type` (string): masking function name `"integer"`
-- `lower` (number): smallest integer value to return
+- `lower` (number, _default: `-100`_): smallest integer value to return
-- `upper` (number): largest integer value to return
+- `upper` (number, _default: `100`_): largest integer value to return
 
 **Example**
 
@@ -584,8 +603,8 @@ Masking settings:
 
 - `path` (string): which field to mask
 - `type` (string): masking function name `"decimal"`
-- `lower` (number): smallest floating point value to return
+- `lower` (number, _default: `-1`_): smallest floating point value to return
-- `upper` (number): largest floating point value to return
+- `upper` (number, _default: `1`_): largest floating point value to return
 - `scale` (number, _default: `2`_): maximal amount of digits in the
   decimal fraction part
 
@@ -621,7 +640,7 @@ The configuration:
 ### Credit Card Number
 
 This masking type replaces the value of the attribute with a random
-credit card number.
+credit card number (as integer number).
 See [Luhn algorithm](https://en.wikipedia.org/wiki/Luhn_algorithm)
 for details.
 
@@ -639,7 +658,8 @@ Masking settings:
 }
 ```
 
-This generates a random credit card number to mask field `ccNumber`.
+This generates a random credit card number to mask field `ccNumber`,
+e.g. `4111111414443302`.
 
 ### Phone Number
 
@@ -688,6 +708,7 @@ phone number in case the input value is not a string.
 This masking type takes an email address, computes a hash value and
 splits it into three equal parts `AAAA`, `BBBB`, and `CCCC`. The
 resulting email address is in the format `AAAA.BBBB@CCCC.invalid`.
+The hash is based on a random secret that is different for each run.
 
 Masking settings:
 

Documentation/Examples/arangodump.json

@@ -490,7 +490,10 @@
     "dynamic" : false,
     "enterpriseOnly" : false,
     "hidden" : false,
-    "introducedIn" : null,
+    "introducedIn" : [
+      "v3.3.22",
+      "v3.4.2"
+    ],
     "obsolete" : false,
     "requiresValue" : true,
     "section" : "",

arangosh/Dump/DumpFeature.cpp

@@ -590,7 +590,8 @@ void DumpFeature::collectOptions(std::shared_ptr<options::ProgramOptions> option
                      new UInt64Parameter(&_options.tickEnd));
 
   options->addOption("--maskings", "file with maskings definition",
-                     new StringParameter(&_options.maskingsFile));
+                     new StringParameter(&_options.maskingsFile))
+                     .setIntroducedIn(30322).setIntroducedIn(30402);
 }
 
 void DumpFeature::validateOptions(std::shared_ptr<options::ProgramOptions> options) {