1
0
Fork 0
Commit Graph

55 Commits

Author SHA1 Message Date
Dan Larkin-York a83c2323c9 Refactor ApplicationServer stack (#9965) 2019-09-25 17:31:59 +02:00
Frank Celler aa3d3f8e40
Feature/cleanup ccpcheck (#9665) 2019-08-12 11:11:49 +02:00
Dan Larkin-York 3d0246cb18 Decentralize includes (#9623) 2019-08-06 15:32:09 +02:00
Matthew Von-Maszewski 91b56a50a3
Feature: Add gzip and encryption to import/export (#9560)
* port of feature-3.4/mv-gzip-export to devel branch

* add explicit namespaces so gcc 6.3.0 would successfully compile

* add conditional cleanup of ENCRYPTION file from another branch

* use _lseek() for Windows build to avoid deprecated warning.

* change from ifdef for lseek variants to TRI_LSEEK.

* force Windows lseek to return Linux expected type.
2019-07-26 07:53:39 -04:00
Wilfried Goesgens ca0f2b8b86 All hail to the SI (#9445) 2019-07-19 13:52:12 +02:00
Matthew Von-Maszewski 40a5c4f8b8
Bugfix: Import would not report communication errors (#9494)
* correct handleResult() to report transmission errors and stop.  Add reporting of line numbers impacted if CSV

* always generate import summary, even if import stops due to error

* pass line numbers to senderThread for potential error reporting.

* move line count from zero based to one based in error report.

* forgot to update CHANGELOG
2019-07-17 18:29:07 +02:00
Simon 241d5d19ef Fuerte changes from comm-task-refactor branch (#9422) 2019-07-08 18:02:35 +02:00
Wilfried Goesgens 1907a7211b Bug fix/cleanup system includes (#8962) 2019-05-15 15:12:59 +02:00
Jan Christoph Uhde c3f7961b88 apply unique log ids (#8561) 2019-03-25 20:26:51 +01:00
Frank Celler ac9f375fb5 big reformat 2018-12-26 00:54:03 +01:00
Lars Maier 6e7baba9f5 Do not access the array of field names to search for removed attributes if the index is out of bounds. (i.e. malformed input data) (#7465) 2018-11-26 17:35:05 +01:00
Jan 976cc38e7c
remove OpenFilesTracker (#7189) 2018-11-02 11:38:38 +01:00
Jan 7351423ad5
cleanup import messages, reuse StringBuffer for JSON import (#5569) 2018-06-11 09:47:26 +02:00
Vasiliy 6a53154160 issue 389.2: use static strings for Index definition json attributes, use TRI_vocbase_t references instead of pointers in V8Context, use TRI_vocbase_t references instead of pointers in DatabaseInitialSyncer (#5344) 2018-05-14 19:06:24 +03:00
jsteemann 1f6ffc4166 fix clang compile error 2018-05-03 11:06:14 +02:00
Matthew Von-Maszewski 9ff6a41236 Add pacing logic to arangoimp (#5238)
* initial check-in of working auto block size tuning for import.  Needs clean-up.
* partial fix in this branch to enable testing, awaiting better fix in different branch.
* move pacing code from findIdleSender to AutoTuneThread object.
* move pacing code from findIdleSender to AutoTuneThread object.  clean up dead code.  add comments about algorithm.
* edits to algorithm discussion
* correct logging level.  add CHANGELOG entry.
* update comment concerning proper usage of shutdown()
* initialize new static member MaxBatchSize.  Correct sendCsvBuffer() call to addPeriodByteCount() since _outputBuffer drained by sendData().
* remove redundant semicolons.  add new static member MaxBatchSize.
* move MaxBatchSize to ImportHelper so both ImportFeature object and AutoTuneThread object can access it.
* minor updates to comments and statics
* add pacing discussion to import manual page
* minor edit
2018-05-02 22:34:04 +02:00
Simon 35136a89c0 Fix some problems with active failover (#4540) 2018-02-09 15:11:53 +01:00
Jan 6040b4ae9f
add ignore-missing option to arangoimport (#4129) 2017-12-22 11:29:10 +01:00
Jan 282be208cc
remove TRI_usleep and TRI_sleep, and use std::this_thread::sleep_for … (#3817) 2017-12-06 18:43:49 +01:00
Wilfried Goesgens 5d3a1512a7 by @jsteemann: Don't busy wait while waiting for the next arangoimp thread to become idle again. (#3828) 2017-12-06 11:02:20 +01:00
Jan 5165155ed1 Bug fix/fixes 0609 (#3227)
* do not use V8 variant of AQL functions in early optimization stage when a C++ variant is available

* additionally, simplify AQL function definitions and aliases

* warn when more than 90% of max mappings are in use

* added C++ variant of replication catchup

* added `--log.role` option

* updated CHANGELOG

* removed non-existing scheduler.threads option from config

* removed useless __FILE__, __LINE__ invocations

* updated CHANGELOG

* allow a priority V8 context

* remove TRI_CORE_MEM_ZONE

* try to fix Windows errors & warnings

* cleanup

* removed memory zones altogether

* exclude system collections from collection tests
2017-09-13 16:28:21 +02:00
Simon Grätzer d485db8bd1 removing unnecessary rate limiting (#3194) 2017-09-07 14:39:20 +02:00
Simon Grätzer 885ea3dd5f Remove attribute option for arangoimp (#3141)
* Adding a remove attribute option to arangoimp (only for csv)

* Adding documentation

* Fixed log statement

* Update CHANGELOG
2017-08-30 10:45:02 +02:00
Simon Grätzer 2bc6a71d48 Fixing error reporting in arangoimp (#3153) 2017-08-28 15:04:45 +02:00
Dan Larkin 02990540d9 Report proper error for arangoimp create collection failure. (#3127) 2017-08-28 14:54:23 +02:00
Simon Grätzer f50a36181d Various authentication related issues (#2824) 2017-07-18 22:22:09 +02:00
jsteemann a01716a2af fix threading problems in arangoimp 2017-06-02 13:58:18 +02:00
jsteemann 145ba83d04 fixed threading issues in import 2017-05-31 15:50:11 +02:00
jsteemann 7b57bdf5f6 fix superfluous re-creation attempts for collections 2017-05-30 18:20:37 +02:00
jsteemann cccb2dda8d Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2017-05-24 19:41:49 +02:00
jsteemann 9f856aeafb cppcheck 2017-05-24 19:41:33 +02:00
Simon Grätzer 0bf4467457 small fixes 2017-05-24 19:21:23 +02:00
Simon Grätzer 2f2d07ab9a Multihreaded import 2017-05-24 18:37:45 +02:00
jsteemann e3ca787992 warn before we run out of file descriptors 2017-03-29 02:35:46 +02:00
jsteemann 9278149b24 fixed --convert 2017-03-09 14:29:23 +01:00
jsteemann aab138a4e6 added input type jsonl 2017-03-06 17:39:44 +01:00
jsteemann 7c0c84c21d added `--translate` option for arangoimp to translate attribute names from the input files to attriubte names expected by ArangoDB
The `--translate` option can be specified multiple times (once per translation
to be executed). The following example renames the "id" column from the input
file to "_key", and the "from" column to "_from", and the "to" column to "_to":

    arangoimp --type csv --file data.csv --translate "id=_key" --translate "from=_from" --translate "to=_to"

`--translate` works for CSV and TSV inputs only.
2017-03-03 11:35:16 +01:00
jsteemann 6217a58559 protect against out-of-bounds read 2017-03-03 11:07:41 +01:00
jsteemann 7051f5d87a fixed --convert false in arangoimp 2017-03-02 15:05:06 +01:00
jsteemann 737698151d fixed issue #2342 2017-03-02 13:45:05 +01:00
baslr cb3756bc7f arangoimp: csv convert value in _key column to string 2017-03-01 22:17:42 +01:00
jsteemann d024a6d00a remove logging for non-topics 2017-02-10 09:32:50 +01:00
jsteemann a24d26d74f cppcheck 2016-08-25 14:04:42 +02:00
Jan Christoph Uhde 1febeadfd7 finish ResponseCode enum 2016-08-23 15:46:42 +02:00
Jan Christoph Uhde d3654d99d8 finish RequestType enum 2016-08-23 14:49:29 +02:00
Jan Steemann 9c98cb6cf3 added test for skipLines 2016-07-04 12:15:13 +02:00
Jan Steemann 7b62d71c63 added --skip-lines for import 2016-07-04 11:18:15 +02:00
Jan Steemann 1726c53805 fixed some import edge cases 2016-06-15 18:16:49 +02:00
jsteemann 7080f8293d various AQL changes
added behavior of TO_STRING() for arrays and objects
added C++ implementation for CONTAINS() function
added memmem implementation for Windows
2016-05-23 17:10:15 +02:00
Jan Steemann 8e090e642e removed "createCollection" attribute from APIs 2016-05-12 15:17:45 +02:00