1
0
Fork 0
Commit Graph

18 Commits

Author SHA1 Message Date
Frank Celler ac9f375fb5 big reformat 2018-12-26 00:54:03 +01:00
Jan 7351423ad5
cleanup import messages, reuse StringBuffer for JSON import (#5569) 2018-06-11 09:47:26 +02:00
Matthew Von-Maszewski 3678dc7b68 latency histogram option for arangoimport (#5552)
* basic histogram thread using non-arangod threading ... soon to change.
* Add --latency flag.  start histogram thread if set.
* convert quick_hist to use arangodb thread and condition objects
* move histogram object to be within ImportStatistics structure.
* update the thread shutdown to 1. work and 2. post last data
* add text that values are in microseconds to command line help.
* CHANGELOG note for new import latency option
* quick_hist.h rename to QuickHistogram.h
* update include file name to QuickHistogram.h
* update variable names to match coding standard
* a couple more code standard changes.  try/catch around push_back
2018-06-08 11:17:25 +02:00
Matthew Von-Maszewski 9ff6a41236 Add pacing logic to arangoimp (#5238)
* initial check-in of working auto block size tuning for import.  Needs clean-up.
* partial fix in this branch to enable testing, awaiting better fix in different branch.
* move pacing code from findIdleSender to AutoTuneThread object.
* move pacing code from findIdleSender to AutoTuneThread object.  clean up dead code.  add comments about algorithm.
* edits to algorithm discussion
* correct logging level.  add CHANGELOG entry.
* update comment concerning proper usage of shutdown()
* initialize new static member MaxBatchSize.  Correct sendCsvBuffer() call to addPeriodByteCount() since _outputBuffer drained by sendData().
* remove redundant semicolons.  add new static member MaxBatchSize.
* move MaxBatchSize to ImportHelper so both ImportFeature object and AutoTuneThread object can access it.
* minor updates to comments and statics
* add pacing discussion to import manual page
* minor edit
2018-05-02 22:34:04 +02:00
Jan 6040b4ae9f
add ignore-missing option to arangoimport (#4129) 2017-12-22 11:29:10 +01:00
Wilfried Goesgens 5d3a1512a7 by @jsteemann: Don't busy wait while waiting for the next arangoimp thread to become idle again. (#3828) 2017-12-06 11:02:20 +01:00
Simon Grätzer d485db8bd1 removing unnecessary rate limiting (#3194) 2017-09-07 14:39:20 +02:00
Simon Grätzer 885ea3dd5f Remove attribute option for arangoimp (#3141)
* Adding a remove attribute option to arangoimp (only for csv)

* Adding documentation

* Fixed log statement

* Update CHANGELOG
2017-08-30 10:45:02 +02:00
Simon Grätzer f50a36181d Various authentication related issues (#2824) 2017-07-18 22:22:09 +02:00
Simon Grätzer 2f2d07ab9a Multihreaded import 2017-05-24 18:37:45 +02:00
jsteemann aab138a4e6 added input type jsonl 2017-03-06 17:39:44 +01:00
jsteemann 7c0c84c21d added `--translate` option for arangoimp to translate attribute names from the input files to attriubte names expected by ArangoDB
The `--translate` option can be specified multiple times (once per translation
to be executed). The following example renames the "id" column from the input
file to "_key", and the "from" column to "_from", and the "to" column to "_to":

    arangoimp --type csv --file data.csv --translate "id=_key" --translate "from=_from" --translate "to=_to"

`--translate` works for CSV and TSV inputs only.
2017-03-03 11:35:16 +01:00
baslr cb3756bc7f arangoimp: csv convert value in _key column to string 2017-03-01 22:17:42 +01:00
Jan Steemann 7b62d71c63 added --skip-lines for import 2016-07-04 11:18:15 +02:00
Jan Steemann 8e090e642e removed "createCollection" attribute from APIs 2016-05-12 15:17:45 +02:00
Frank Celler d6e8046992 updated cleanupCFiles 2016-04-23 16:34:57 -07:00
Jan Steemann bbfaee3bfe allow specifying collection name prefixes for `_from` and `_to` in arangoimp: 2016-04-15 16:18:23 +02:00
Frank Celler dfa9047484 switch arangosh to new ApplicationFeatures and ProgramOptions 2016-03-08 23:32:43 +01:00