1
0
Fork 0
Commit Graph

118 Commits

Author SHA1 Message Date
Jan c1a0274ad0 fixed issue #10183: Arangoimport imports on _system when you try to create a new database. (#10187) 2019-10-09 14:54:46 +03:00
Matthew Von-Maszewski 916cb42b16 Bugfix 3.5: Add object counts to arangoimport latency (#9937)
* port PR 9856 from 3.4 to 3.5. Add object counts to latency reporting.

* address code review issues
2019-09-09 21:37:08 +03:00
Matthew Von-Maszewski 0832c6d546 Feature 3.5: add gzip and encryption support to arangoexport and arangoimport (#9579)
* port gzip/encryption for arangoexport/import from devel branch

* revert Dump default use of gzip from true to false.

* Stylefix in changelog.
2019-07-27 00:06:33 +03:00
Matthew Von-Maszewski 4441a806de Bugfix 3.5: Import would not report communication errors (#9500)
* arangoimport was neither reporting nor stopping on communications errors.  Add reporting of impacted lines for CSV imports.

* correct warning from Windows build about size_t to int conversion
2019-07-18 16:40:01 +03:00
Simon 241d5d19ef Fuerte changes from comm-task-refactor branch (#9422) 2019-07-08 18:02:35 +02:00
Wilfried Goesgens 1907a7211b Bug fix/cleanup system includes (#8962) 2019-05-15 15:12:59 +02:00
Jan Christoph Uhde 677a79026c Foxx Security (#8845) 2019-04-25 09:56:29 +02:00
Jan Christoph Uhde c3f7961b88 apply unique log ids (#8561) 2019-03-25 20:26:51 +01:00
Jan a841d472be
emit a warning when connecting with the client tools to a DBSERVER (#8397) 2019-03-13 13:08:41 +01:00
Frank Celler ac9f375fb5 big reformat 2018-12-26 00:54:03 +01:00
Wilfried Goesgens c0f9e8125a Bug fix/allow tcp connection to finish (#7635) 2018-12-10 10:38:34 +01:00
Lars Maier 6e7baba9f5 Do not access the array of field names to search for removed attributes if the index is out of bounds. (i.e. malformed input data) (#7465) 2018-11-26 17:35:05 +01:00
Jan 976cc38e7c
remove OpenFilesTracker (#7189) 2018-11-02 11:38:38 +01:00
Wilfried Goesgens a477df49cf Feature/windows utf16 fileaccess (#6534) 2018-09-24 19:41:17 +02:00
Vasiliy 6fd541d110 issue 427.5: use ApplicationServer reference instead of pointer (#6145)
* issue 427.5: use ApplicationServer reference instead of pointer

* address MSVC build failure
2018-08-15 12:16:02 +03:00
Jan b278d6874a
allow master & slave to work in parallel for RocksDB WAL tailing (#6059) 2018-08-03 13:37:53 +02:00
Michael Hackstein 7a95c5e675
Feature/feature phases (#5272)
* Added feature phases

* BasicsPhase and DatabasePhase to the required files. Server now has Feature circles and does not boot. Will be sorted out later on.

* Added ClusterPhase to features

* Added V8Phase to the required features

* Added AQLPhase to the affected features

* Added ServerPhase to Features

* Added FoxxPhase to the relevant features

* Added AgencyPhase to the relevant features

* Moved registration from local variable SYS_SYSTEM_REPLICATION_FACTOR from cluster to V8 as their ordering is now vice versa

* Moved Bootstrap feature into FoxxPhase. It could be moved to ServerPhase easily if the FoxxQueue dependency would be removed

* Final movement of Startup Phases. Now solved all circles.

* Removed merge conflict

* Moved ReplicationTimeout into cluster phase and fixed cross-phase requirements

* Added greetings phase. This phase separates the Basics Phase and is the first to be run. Includes Logger and Hello/Goodbye

* Added the GreetingsPhase in the corresponding features. Now all BasicsPhase features start after greetings Phase. There is some issue in this branch which prevents the Agency from Gossipping right now. Will be fixed next

* Moved creation of the Agent into the prepare phase of the feature. THereby it is guaranteed that agents at least exists before the GeneralServer is activating endpoints

* Recovery needs to be started after the ServerID

* Moved log output of FeaturePhases to DEBUG instead of ERROR.

* Added feature phases for clients

* ClusterFeature now does not directly require AgencyFeature any more

* Added requirement of TravEngineRegistryFeature in AQL feature. Otherwise shutdown may be undefined

* The ApplicationServer can now handout the list of ordered features. Used for testing purposes

* Fixed IResearchVew Tests Setup to honor new feature ordering

* Fixed IResearchViewDBServer Tests Setup to honor new feature ordering

* Started fixing IResearchView Coordinator tests with startup ordering. Not finished yet

* Added startup phases to ViewCoordinator test

* Disabled expected logoutput in ClusterRepairsTest

* Fixed indention in test code

* LinkCoordinator now honors startup ordering

* Link meta now honors startup rdering

* Supress expected cluster logs in ViewTest

* Removed '#' accidentially added.
2018-07-16 14:09:36 +02:00
jsteemann 7538b6fde0 fix MSVC compile warning 2018-07-09 13:20:15 +02:00
Simon 3bec336aff TransactionState::addCollection refactoring (#5606) 2018-06-14 15:34:58 +02:00
Jan 7351423ad5
cleanup import messages, reuse StringBuffer for JSON import (#5569) 2018-06-11 09:47:26 +02:00
Matthew Von-Maszewski 3678dc7b68 latency histogram option for arangoimport (#5552)
* basic histogram thread using non-arangod threading ... soon to change.
* Add --latency flag.  start histogram thread if set.
* convert quick_hist to use arangodb thread and condition objects
* move histogram object to be within ImportStatistics structure.
* update the thread shutdown to 1. work and 2. post last data
* add text that values are in microseconds to command line help.
* CHANGELOG note for new import latency option
* quick_hist.h rename to QuickHistogram.h
* update include file name to QuickHistogram.h
* update variable names to match coding standard
* a couple more code standard changes.  try/catch around push_back
2018-06-08 11:17:25 +02:00
Vasiliy 6a53154160 issue 389.2: use static strings for Index definition json attributes, use TRI_vocbase_t references instead of pointers in V8Context, use TRI_vocbase_t references instead of pointers in DatabaseInitialSyncer (#5344) 2018-05-14 19:06:24 +03:00
jsteemann 52de92d334 add missing override specifiers, add final specifiers 2018-05-04 09:01:50 +02:00
jsteemann 1f6ffc4166 fix clang compile error 2018-05-03 11:06:14 +02:00
Matthew Von-Maszewski 9ff6a41236 Add pacing logic to arangoimp (#5238)
* initial check-in of working auto block size tuning for import.  Needs clean-up.
* partial fix in this branch to enable testing, awaiting better fix in different branch.
* move pacing code from findIdleSender to AutoTuneThread object.
* move pacing code from findIdleSender to AutoTuneThread object.  clean up dead code.  add comments about algorithm.
* edits to algorithm discussion
* correct logging level.  add CHANGELOG entry.
* update comment concerning proper usage of shutdown()
* initialize new static member MaxBatchSize.  Correct sendCsvBuffer() call to addPeriodByteCount() since _outputBuffer drained by sendData().
* remove redundant semicolons.  add new static member MaxBatchSize.
* move MaxBatchSize to ImportHelper so both ImportFeature object and AutoTuneThread object can access it.
* minor updates to comments and statics
* add pacing discussion to import manual page
* minor edit
2018-05-02 22:34:04 +02:00
Wilfried Goesgens 7d6e580780 Refactoring & code cleanup (#5138) (#5142) 2018-04-24 14:42:23 +02:00
Jan 8b9eb13925
forward port branch feature-3.3/allow-jwt-in-arangosh (#5009) 2018-04-03 17:37:01 +02:00
Dan Larkin-York 38f162e344 Parallelize arangodump (#4356) 2018-03-26 15:55:14 +02:00
Simon 35136a89c0 Fix some problems with active failover (#4540) 2018-02-09 15:11:53 +01:00
Jan Christoph Uhde bddadda0ee add 'create database' to 'import feature' (#4278) 2018-01-15 15:15:31 +01:00
Jan 6040b4ae9f
add ignore-missing option to arangoimport (#4129) 2017-12-22 11:29:10 +01:00
Dan Larkin 9cd8c47eb7 Renamed arangoimp to arangoimport (with alias for compatibility.) (#4040) 2017-12-14 21:31:21 +01:00
Jan 282be208cc
remove TRI_usleep and TRI_sleep, and use std::this_thread::sleep_for … (#3817) 2017-12-06 18:43:49 +01:00
Wilfried Goesgens 5d3a1512a7 by @jsteemann: Don't busy wait while waiting for the next arangoimp thread to become idle again. (#3828) 2017-12-06 11:02:20 +01:00
Jan 5165155ed1 Bug fix/fixes 0609 (#3227)
* do not use V8 variant of AQL functions in early optimization stage when a C++ variant is available

* additionally, simplify AQL function definitions and aliases

* warn when more than 90% of max mappings are in use

* added C++ variant of replication catchup

* added `--log.role` option

* updated CHANGELOG

* removed non-existing scheduler.threads option from config

* removed useless __FILE__, __LINE__ invocations

* updated CHANGELOG

* allow a priority V8 context

* remove TRI_CORE_MEM_ZONE

* try to fix Windows errors & warnings

* cleanup

* removed memory zones altogether

* exclude system collections from collection tests
2017-09-13 16:28:21 +02:00
Simon Grätzer d485db8bd1 removing unnecessary rate limiting (#3194) 2017-09-07 14:39:20 +02:00
Simon Grätzer 885ea3dd5f Remove attribute option for arangoimp (#3141)
* Adding a remove attribute option to arangoimp (only for csv)

* Adding documentation

* Fixed log statement

* Update CHANGELOG
2017-08-30 10:45:02 +02:00
Simon Grätzer 2bc6a71d48 Fixing error reporting in arangoimp (#3153) 2017-08-28 15:04:45 +02:00
Dan Larkin 02990540d9 Report proper error for arangoimp create collection failure. (#3127) 2017-08-28 14:54:23 +02:00
Kaveh Vahedipour 0b6d6d9287 Fixed distributeShardsLike bug in creating collections. numberOfShard… (#2895) 2017-08-03 19:38:16 +02:00
Simon Grätzer f50a36181d Various authentication related issues (#2824) 2017-07-18 22:22:09 +02:00
jsteemann 2275fc8174 better automatic file-type detection for arangoimp 2017-06-08 14:55:21 +02:00
jsteemann a01716a2af fix threading problems in arangoimp 2017-06-02 13:58:18 +02:00
jsteemann 145ba83d04 fixed threading issues in import 2017-05-31 15:50:11 +02:00
jsteemann 2cfd7460d7 fix output 2017-05-31 11:38:20 +02:00
jsteemann 7b57bdf5f6 fix superfluous re-creation attempts for collections 2017-05-30 18:20:37 +02:00
jsteemann cccb2dda8d Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2017-05-24 19:41:49 +02:00
jsteemann 9f856aeafb cppcheck 2017-05-24 19:41:33 +02:00
Simon Grätzer 0bf4467457 small fixes 2017-05-24 19:21:23 +02:00
Simon Grätzer 36503af230 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel
# Conflicts:
#	arangod/RocksDBEngine/RocksDBReplicationTailing.cpp
2017-05-24 18:38:34 +02:00