1
0
Fork 0
Commit Graph

102 Commits

Author SHA1 Message Date
Michael Hackstein 7a95c5e675
Feature/feature phases (#5272)
* Added feature phases

* BasicsPhase and DatabasePhase to the required files. Server now has Feature circles and does not boot. Will be sorted out later on.

* Added ClusterPhase to features

* Added V8Phase to the required features

* Added AQLPhase to the affected features

* Added ServerPhase to Features

* Added FoxxPhase to the relevant features

* Added AgencyPhase to the relevant features

* Moved registration from local variable SYS_SYSTEM_REPLICATION_FACTOR from cluster to V8 as their ordering is now vice versa

* Moved Bootstrap feature into FoxxPhase. It could be moved to ServerPhase easily if the FoxxQueue dependency would be removed

* Final movement of Startup Phases. Now solved all circles.

* Removed merge conflict

* Moved ReplicationTimeout into cluster phase and fixed cross-phase requirements

* Added greetings phase. This phase separates the Basics Phase and is the first to be run. Includes Logger and Hello/Goodbye

* Added the GreetingsPhase in the corresponding features. Now all BasicsPhase features start after greetings Phase. There is some issue in this branch which prevents the Agency from Gossipping right now. Will be fixed next

* Moved creation of the Agent into the prepare phase of the feature. THereby it is guaranteed that agents at least exists before the GeneralServer is activating endpoints

* Recovery needs to be started after the ServerID

* Moved log output of FeaturePhases to DEBUG instead of ERROR.

* Added feature phases for clients

* ClusterFeature now does not directly require AgencyFeature any more

* Added requirement of TravEngineRegistryFeature in AQL feature. Otherwise shutdown may be undefined

* The ApplicationServer can now handout the list of ordered features. Used for testing purposes

* Fixed IResearchVew Tests Setup to honor new feature ordering

* Fixed IResearchViewDBServer Tests Setup to honor new feature ordering

* Started fixing IResearchView Coordinator tests with startup ordering. Not finished yet

* Added startup phases to ViewCoordinator test

* Disabled expected logoutput in ClusterRepairsTest

* Fixed indention in test code

* LinkCoordinator now honors startup ordering

* Link meta now honors startup rdering

* Supress expected cluster logs in ViewTest

* Removed '#' accidentially added.
2018-07-16 14:09:36 +02:00
jsteemann 7538b6fde0 fix MSVC compile warning 2018-07-09 13:20:15 +02:00
Simon 3bec336aff TransactionState::addCollection refactoring (#5606) 2018-06-14 15:34:58 +02:00
Jan 7351423ad5
cleanup import messages, reuse StringBuffer for JSON import (#5569) 2018-06-11 09:47:26 +02:00
Matthew Von-Maszewski 3678dc7b68 latency histogram option for arangoimport (#5552)
* basic histogram thread using non-arangod threading ... soon to change.
* Add --latency flag.  start histogram thread if set.
* convert quick_hist to use arangodb thread and condition objects
* move histogram object to be within ImportStatistics structure.
* update the thread shutdown to 1. work and 2. post last data
* add text that values are in microseconds to command line help.
* CHANGELOG note for new import latency option
* quick_hist.h rename to QuickHistogram.h
* update include file name to QuickHistogram.h
* update variable names to match coding standard
* a couple more code standard changes.  try/catch around push_back
2018-06-08 11:17:25 +02:00
Vasiliy 6a53154160 issue 389.2: use static strings for Index definition json attributes, use TRI_vocbase_t references instead of pointers in V8Context, use TRI_vocbase_t references instead of pointers in DatabaseInitialSyncer (#5344) 2018-05-14 19:06:24 +03:00
jsteemann 52de92d334 add missing override specifiers, add final specifiers 2018-05-04 09:01:50 +02:00
jsteemann 1f6ffc4166 fix clang compile error 2018-05-03 11:06:14 +02:00
Matthew Von-Maszewski 9ff6a41236 Add pacing logic to arangoimp (#5238)
* initial check-in of working auto block size tuning for import.  Needs clean-up.
* partial fix in this branch to enable testing, awaiting better fix in different branch.
* move pacing code from findIdleSender to AutoTuneThread object.
* move pacing code from findIdleSender to AutoTuneThread object.  clean up dead code.  add comments about algorithm.
* edits to algorithm discussion
* correct logging level.  add CHANGELOG entry.
* update comment concerning proper usage of shutdown()
* initialize new static member MaxBatchSize.  Correct sendCsvBuffer() call to addPeriodByteCount() since _outputBuffer drained by sendData().
* remove redundant semicolons.  add new static member MaxBatchSize.
* move MaxBatchSize to ImportHelper so both ImportFeature object and AutoTuneThread object can access it.
* minor updates to comments and statics
* add pacing discussion to import manual page
* minor edit
2018-05-02 22:34:04 +02:00
Wilfried Goesgens 7d6e580780 Refactoring & code cleanup (#5138) (#5142) 2018-04-24 14:42:23 +02:00
Jan 8b9eb13925
forward port branch feature-3.3/allow-jwt-in-arangosh (#5009) 2018-04-03 17:37:01 +02:00
Dan Larkin-York 38f162e344 Parallelize arangodump (#4356) 2018-03-26 15:55:14 +02:00
Simon 35136a89c0 Fix some problems with active failover (#4540) 2018-02-09 15:11:53 +01:00
Jan Christoph Uhde bddadda0ee add 'create database' to 'import feature' (#4278) 2018-01-15 15:15:31 +01:00
Jan 6040b4ae9f
add ignore-missing option to arangoimport (#4129) 2017-12-22 11:29:10 +01:00
Dan Larkin 9cd8c47eb7 Renamed arangoimp to arangoimport (with alias for compatibility.) (#4040) 2017-12-14 21:31:21 +01:00
Jan 282be208cc
remove TRI_usleep and TRI_sleep, and use std::this_thread::sleep_for … (#3817) 2017-12-06 18:43:49 +01:00
Wilfried Goesgens 5d3a1512a7 by @jsteemann: Don't busy wait while waiting for the next arangoimp thread to become idle again. (#3828) 2017-12-06 11:02:20 +01:00
Jan 5165155ed1 Bug fix/fixes 0609 (#3227)
* do not use V8 variant of AQL functions in early optimization stage when a C++ variant is available

* additionally, simplify AQL function definitions and aliases

* warn when more than 90% of max mappings are in use

* added C++ variant of replication catchup

* added `--log.role` option

* updated CHANGELOG

* removed non-existing scheduler.threads option from config

* removed useless __FILE__, __LINE__ invocations

* updated CHANGELOG

* allow a priority V8 context

* remove TRI_CORE_MEM_ZONE

* try to fix Windows errors & warnings

* cleanup

* removed memory zones altogether

* exclude system collections from collection tests
2017-09-13 16:28:21 +02:00
Simon Grätzer d485db8bd1 removing unnecessary rate limiting (#3194) 2017-09-07 14:39:20 +02:00
Simon Grätzer 885ea3dd5f Remove attribute option for arangoimp (#3141)
* Adding a remove attribute option to arangoimp (only for csv)

* Adding documentation

* Fixed log statement

* Update CHANGELOG
2017-08-30 10:45:02 +02:00
Simon Grätzer 2bc6a71d48 Fixing error reporting in arangoimp (#3153) 2017-08-28 15:04:45 +02:00
Dan Larkin 02990540d9 Report proper error for arangoimp create collection failure. (#3127) 2017-08-28 14:54:23 +02:00
Kaveh Vahedipour 0b6d6d9287 Fixed distributeShardsLike bug in creating collections. numberOfShard… (#2895) 2017-08-03 19:38:16 +02:00
Simon Grätzer f50a36181d Various authentication related issues (#2824) 2017-07-18 22:22:09 +02:00
jsteemann 2275fc8174 better automatic file-type detection for arangoimp 2017-06-08 14:55:21 +02:00
jsteemann a01716a2af fix threading problems in arangoimp 2017-06-02 13:58:18 +02:00
jsteemann 145ba83d04 fixed threading issues in import 2017-05-31 15:50:11 +02:00
jsteemann 2cfd7460d7 fix output 2017-05-31 11:38:20 +02:00
jsteemann 7b57bdf5f6 fix superfluous re-creation attempts for collections 2017-05-30 18:20:37 +02:00
jsteemann cccb2dda8d Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2017-05-24 19:41:49 +02:00
jsteemann 9f856aeafb cppcheck 2017-05-24 19:41:33 +02:00
Simon Grätzer 0bf4467457 small fixes 2017-05-24 19:21:23 +02:00
Simon Grätzer 36503af230 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel
# Conflicts:
#	arangod/RocksDBEngine/RocksDBReplicationTailing.cpp
2017-05-24 18:38:34 +02:00
Simon Grätzer 2f2d07ab9a Multihreaded import 2017-05-24 18:37:45 +02:00
jsteemann 27a8adf9b8 moved shell colors into their own feature 2017-05-23 16:53:43 +02:00
jsteemann 53fa66e60d move parts from ProgramOptions from headers into cpp files 2017-05-23 10:30:36 +02:00
jsteemann e3ca787992 warn before we run out of file descriptors 2017-03-29 02:35:46 +02:00
jsteemann 97b5b2f5df fix issues reported by coverity scan 2017-03-23 19:40:39 +01:00
jsteemann 9278149b24 fixed --convert 2017-03-09 14:29:23 +01:00
jsteemann aab138a4e6 added input type jsonl 2017-03-06 17:39:44 +01:00
jsteemann 6478c6d043 updated CHANGELOG 2017-03-03 15:03:52 +01:00
jsteemann db9099f0f9 Merge branch 'devel' of https://github.com/arangodb/arangodb into engine-api 2017-03-03 12:07:50 +01:00
jsteemann 7c0c84c21d added `--translate` option for arangoimp to translate attribute names from the input files to attriubte names expected by ArangoDB
The `--translate` option can be specified multiple times (once per translation
to be executed). The following example renames the "id" column from the input
file to "_key", and the "from" column to "_from", and the "to" column to "_to":

    arangoimp --type csv --file data.csv --translate "id=_key" --translate "from=_from" --translate "to=_to"

`--translate` works for CSV and TSV inputs only.
2017-03-03 11:35:16 +01:00
jsteemann 6217a58559 protect against out-of-bounds read 2017-03-03 11:07:41 +01:00
jsteemann 6e4e29943d added 'auto' file type 2017-03-02 23:35:52 +01:00
jsteemann 7051f5d87a fixed --convert false in arangoimp 2017-03-02 15:05:06 +01:00
jsteemann 737698151d fixed issue #2342 2017-03-02 13:45:05 +01:00
baslr cb3756bc7f arangoimp: csv convert value in _key column to string 2017-03-01 22:17:42 +01:00
jsteemann 0ac29eb177 don't fail when using optional features as dependencies 2017-03-01 12:52:51 +01:00