1
0
Fork 0
Commit Graph

5533 Commits

Author SHA1 Message Date
Simon Grätzer d14710b683 Updating Replication Factor (#3528)
* Changing replication factor in 3.2 (#3513)

* Allow changing replicationFactor on coordinator

* Fixing logic

* Allowing change of replication factor

* Additional input validation

* grrr

* Testing invalid inputs

(cherry picked from commit 47600e1ea1)

* jslint

(cherry picked from commit 4d223597db)

* cherry-pick commits from 3.2: 47600e1ea1 4d223597db d684eaa7f8 a898af3723
2017-10-26 15:28:05 +02:00
Jan c569c1b816 fix test (#3524) 2017-10-26 14:04:41 +02:00
Max Neunhöffer 5e2f17c04f Fast path for get-in-sync of empty shards. (#3470)
* Fast path for get-in-sync of empty shards.

* Fix short cut in synchronization.
2017-10-26 09:55:00 +02:00
jsteemann 7f00bf3b7d next attempt 2017-10-26 09:34:48 +02:00
jsteemann 45f2c4aca7 try to fix non-deterministic test 2017-10-25 21:07:31 +02:00
Jan 9eb5e1545f Bug fix/fix cluster foxx queue startup (#3507)
* slightly clean up indexes

* proper stringification of error code

* do not run AQL queries during Foxx queue startup

* always create keyspace
2017-10-25 18:07:27 +02:00
Jan 9a4703dbde changed test expectations in cluster mode (#3491) 2017-10-25 18:05:37 +02:00
Manuel B 7d4b319c58 cpp aql substring, fixed v8 substring (#3489) 2017-10-23 17:12:45 +02:00
Jan feba30fcec attempt to make replication test more deterministic (#3482) 2017-10-23 14:21:46 +02:00
Manuel B be8c648620 cpp aql collections() and test (#3331)
* cpp aql collections() and test

* remview fixes

* fixed jslint

* fix for new ExecContext / auth api
2017-10-23 11:42:34 +02:00
Manuel B cf0578242f Feature/devel cpp aql matches (#3329)
* matches function header

* aql matches() C++ definition

* cpp aql matches() implementation

* review fixes

* test for v8 / cpp

* aql cpp matches matches behavior of js matches, more tests

* fix cpp matches()

* final fixes
2017-10-23 08:25:55 +02:00
Simon Grätzer fd3f9d99d9 Fixing webinterface access (#3464)
* intermediate commit

* Refactoring the ExecContext

* Fixing authentication

* Added start script

* some fixes

* fixed access to nullptr

* some c++

* fixed misleading message

* Made DatabaseGuard movable. Also adapted map insertions to _vocbase in Syncer classes, which failed to compile under older GCC versions

* added support for global flag to replication handler

* Started Refactoring in replication-static

* Fixing syncer code

* store applier configuration

* Static replication tests now test replication in a non system Database

* added flags to replication feature

* Adding some extra checks

* Fixing issue with rocksdb rest replication handler

* replication static now runs _system and otherdatabase replication tests.

* Fixing crash on startup

* Replication_sync now tests _system as well as other Database

* Fixing up heartbeat thread, adding global flag to rest handler

* Fixing wrong assert

* some cleanup, probably some tests are broken

* Made non-system db version of replication-ongoing tests

* fix determine-open-transaction

* Fixed ongoing tests. And added a test where we drop a database on slave while replication is still ongoing

* test fixes

* Activated ongoing other db tests. Also added a test that drops the DB on master, while the slave is still syncing.

* some better error reporting

* gradually switch to Result

* createCollection -> create

* re-activate using of collection ids for now

* enable auto-start

* Fixed create collection in replication ongoing test

* Added first draft of a test for global replication

* move to Result

* use system database for global applier

* improved error reporting

* fixed invalid URLs

* add test case filter

* load existing global applier configuration

* improve error reporting

* Added further tests for global replication

* Fixed global replication test, it now properly waits for replication. Timeouts after 10 seconds.

* Removed erronious assertion

* improve error reporting

* intermediate commit

* Added a test-case for global replication where the Master already has some data and the slave is clean

* fix deletion of replication contexts

* Fixed JSLint

* compiling code

* fix typo

* do not fail for global applier when no database is configured

* intermediate commit

* syncer supports switch for 3.3 / 3.2

* fixed errors

* Fixing some replication bugs

* Fixing some assertions

* Fixed missing commit markers

* Fixing assertion on database drop

* Attempt to fix deadlock in applier and assertion

* Fixing some stupid things

* Support for collection parameter

* Acidentally turned off some tests

* Grrr

* Fixing wrong method call

* Fixed startscript

* Fixed assignmet instead of equality check typo

* Added a test far interrupted replication. For now it justs tests basics on _system database.

* Improved index tests on replication.

* properly initialize variable

* fixed some replication problems

* MMFiles wal access support

* fix replication issues

* Started mmfiles replication support

* fixing a bug

* Fixing an issue

* fixing some mmfiles stuff

* fix test

* reload users

* prevent pure virtual method call

* intermediate commit

* Making from exclusive

* do not call getMasterState if child syncer

* some reformatting

* Adding global support for handleCommandSync

* Fixing assertion

* removing some debug logs

* Changing return codes

* Fixing some issues in the rest handler

* Make replication less susceptible to errors

* remove some debug output

* return last log tick

* remove waits from tests

* fix two tests

* changing header for open-transactions call

* some fixes

* fix test

* invalidate cached databases

* merging request and execcontext

* try to fix assertion error

* renamed method

* fix compile warning

* small changes

* Always use execcontext

* Fixing an assert

* fix replication issues

* try to fix collection lookups

* try to fix master/slave start

* Changing comments in heartbeat thread

* fix wrong signature of READ_LOCKER_EVENTUAL

* log server role in testing mode

* Fixed authentication, removed execContext in favor of request context

* Adding cluster rest api

* Fixing cluster rest handler

* Fixing cluster callback

* Some refactoring

* Queue creation is not a single operation

* Allowed for leader redirects

* Setting start of batch

* Disabling 2.8 compat tests

* fix start/stop bugs

* jslint

* various little changes

* add flag for exposing jwt

* indentation

* cleanup

* Some changed to guid

* fixing tcp to http, vst

* changed endpoint header

* small fixes

* Reorder servers by health status

* Higher timeout

* Changing error messages

* update the fromTick when fetching multiple batches from the coordinator

* more debug info

* Reducing copy pasted code

* change uid generation

* reducing logspam

* more exceptions for redirects

* more exceptions

* attempt to fix uniqids in cluster

* centralize printing of HTTP errors in replication

* debug output

* fix messages for authentication

* cleanup

* removing --cluster.my-id, --cluster.my-local-info

* Added leadership race to bootstrap, determine foxxmaster on boostrap, removing obsolete code

* improve error reporting in RestAqlHandler

* Changing heartbeat thread, fixing cluster_sync

* some more debug output

* added master

* attempt to make tests more deterministic

* added logging about indexes

* added some safety checks to the logger

* slighty better error messages

* fix location header for SSL

* fix error message

* try to make tests more deterministic

* change error code from TRI_ERROR_INTERNAL (which we want to avoid) to TRI_ERROR_FAILED

* Fixing broken webinterface access

* reverting groovy change

* Fixing read-only internal users

* Using superuser rights for dashboard now

* Adding mode field to _admin/server/role

* added mode TRYAGAIN

* remove inventory lock (does not seem necessary here)

* remove invalid assertion

* fixing agency bugs

* Removing debug output

* return proper errors in case of "method not allowed"

* Fixed up some info messages

* jslint
2017-10-20 18:06:59 +02:00
Jan 7840d3f824 Bug fix/fixes 1810 (#3460)
* improve error reporting in RestAqlHandler

* added logging about indexes

* added some safety checks to the logger

* slighty better error messages

* fix location header for SSL

* fix error message

* try to make tests more deterministic

* change error code from TRI_ERROR_INTERNAL (which we want to avoid) to TRI_ERROR_FAILED
2017-10-19 11:28:01 +02:00
Simon Grätzer 7c31960cf2 Feature/async failover (#3451) 2017-10-18 23:59:29 +02:00
Jan 451abcb23d added non-destructive DISTINCT (#3320)
* added non-destructive DISTINCT

* added tests

* update documentation
2017-10-18 12:52:33 +02:00
Jan 0eb59e9bec do not use `_doCount` attribute (which does nothing) (#3454) 2017-10-18 12:43:34 +02:00
Jan 2fee652e77 Bug fix/fixes 1610 (#3425)
* remove memory zone remainders

* remove useless calls

* attempt to fix non-deterministic CRC test

* attempt to fix non-deterministic replication tests

* fix document removal with multiple documents in cluster

* attempt to fix non-determinism of tests

* fix privilege checks for accessing collections by collection id, not by name
2017-10-18 12:23:39 +02:00
Simon Grätzer a7903495a5 Fix createQueue silent unique constraint violation, Fix asan detection in clang, (#3421) 2017-10-18 12:21:31 +02:00
Frank Celler 8f0f861cc4 disabled corrupted-crc-mmfiles.js 2017-10-15 11:26:02 +02:00
Jan f1497e0df7 fixed issue #3395 (#3399) 2017-10-13 12:25:46 +02:00
Jan dd32992121 fix potential overflow in CRC marker check when a corrupted CRC marke… (#3386)
* fix potential overflow in CRC marker check when a corrupted CRC marker is found at the very beginning of an MMFiles datafile

* added recovery test for marker corruption
2017-10-10 09:49:13 +02:00
Jan 2525a3a1bd Feature/mmfiles hash lookup performance (#3265)
* speed up full collection scans in mmfiles engine

* some API cleanup
2017-09-26 14:48:06 +02:00
Jan 43e0fd318e Bug fix/fixes 1909 (#3285)
* remove TRI_ERROR_INTERNAL usage

* prevent assertion failure from occurring

* ensure downwards-compatibility for explain handler

* allow schmutz to drop system collections as well - needed for arangojs tests
2017-09-19 21:56:32 +02:00
Wilfried Goesgens 0faf9be695 Feature/compaction configuration (#3217)
* make the different values influencing the compaction run configurable

* Compaction statistics handling

 - we mustn't keep the number of dead objects on the compacted datafiles statistics, else it will be compacted again.
 - keep statistics of the compaction runs on the DatafileStatistics object
 - add the new statistics on DatafileStatistics to the figures api
 - implement test that assures only one compaction is run, and the statistic values are maintained

* don't mention the version number

* Implement review

 - fix documentation
 - allow 0 maxfiles to enable users to disable combined of datafiles
 - add statistic element that counts the number of combined datafiles

* Implement review

 - fix documentation
 - use locks to make statistic values consistent.
 - fix typo in variable name

* fix temporary variable unnecessary.

* changelog
2017-09-15 14:37:30 +02:00
Wilfried Goesgens 2833bd6185 Bugfix/cleanup graphs (#3250)
* when running against an external server also setup the proper connection

* cleanup grahs after testing
2017-09-15 14:20:21 +02:00
Jan 5165155ed1 Bug fix/fixes 0609 (#3227)
* do not use V8 variant of AQL functions in early optimization stage when a C++ variant is available

* additionally, simplify AQL function definitions and aliases

* warn when more than 90% of max mappings are in use

* added C++ variant of replication catchup

* added `--log.role` option

* updated CHANGELOG

* removed non-existing scheduler.threads option from config

* removed useless __FILE__, __LINE__ invocations

* updated CHANGELOG

* allow a priority V8 context

* remove TRI_CORE_MEM_ZONE

* try to fix Windows errors & warnings

* cleanup

* removed memory zones altogether

* exclude system collections from collection tests
2017-09-13 16:28:21 +02:00
Jan dc46c9db07 more logging so we can find a specific bug (#3247) 2017-09-13 09:19:50 +02:00
jsteemann f8d8a561a4 fix various issues 2017-09-13 09:03:28 +02:00
Wilfried Goesgens b41cfac17c cleanup 2017-09-13 00:00:32 +02:00
Wilfried Goesgens 11437d20a9 fix cleanup 2017-09-12 15:57:33 +02:00
Wilfried Goesgens 0a9b9e6025 cleanup after testrun, lintify 2017-09-12 14:48:20 +02:00
Jan eacad99df7 Bug fix/fixes 0409 (#3199)
* some authentication optimizations...

don't query the permissions for one database three times, and don't build up tons of useless temporary strings for each request

* fix some floating point inaccuracies in AstNode value comparisons
2017-09-04 18:04:05 +02:00
Jan f49e0ca237 execute DOCUMENT function via CXX on coordinator in cluster mode (#3196) 2017-09-04 17:34:24 +02:00
jsteemann b1fd92c438 removed useless explain 2017-09-01 14:14:01 +02:00
Simon Grätzer 88d01b89b5 Optimizations for Caches and Graph Traversals (#3169) 2017-08-31 18:33:10 +02:00
Wilfried Goesgens c662dfc863 Cleanup after each testrun (#3109)
* Cleanup after each testrun

 - add a success testsuite next to the fail testsuite
 - create temporary directories in the fail & success testsuites
 - add a function to cleanup the last unshifted directory
 - don't say we're crashed when exiting with the help message

* Specify temp path for SUT so its files will be removed after cleanup.

 - cleanup
 - don't use getTempFile(), it will create another subdirectory that we don't clean up afterwards...

* Fix temp path usage

 - creating a 'bundles' directory wherein we would keep our temporary bundle files would collide if we'd clean it up.
 - create 'bundles-tmpxxxx' instead so its individual to us and we can remove it when done without problems.

* properly create & pass in a temporary directory - as we now require this.

* Remove intermediate directory recursive - for some unknown reason there may be temp files created.

* we need to unify tempfilenames here too.

* use individual suite names

* use the standart performTests to run the backup tests - so we have the proper directory and server handling from the infrastructure.

* fix temporary directory handling in the recovery tests.

* script to build debugging debian packages

* Improve test shutdown

 - handle a set of options globally per testsuite, so they don't need to clone them inside.
 - properly cleanup the slave instance of the replication tests after each run.
 - The per testsuite options vector will fix the behaviour that after one failed testsuite
   all other suites won't be cleaned up anymore.

* don't call the overal cleanup if anything failed.

* fix cleaning up of directories.

* cleanup, fix directory creation / deletion / dummy creation.

* cleanup, fix directory creation / deletion / dummy creation.

* fix tempdir handling for the recovery tests
2017-08-30 10:48:24 +02:00
Simon Grätzer 885ea3dd5f Remove attribute option for arangoimp (#3141)
* Adding a remove attribute option to arangoimp (only for csv)

* Adding documentation

* Fixed log statement

* Update CHANGELOG
2017-08-30 10:45:02 +02:00
Kaveh Vahedipour 00650e6a3f Bug fix/agency mt fixes (#3158)
* added debugging methods

* try to fix invalid access in case of error

* remove unused members

* bugfixes and comments

* all agency fixes in

* merge bug

* partially unguarded Agent::lead fixed

* all agency fixes in

* added nrBlocked to thread startup eval

* added nrBlocked to thread startup eval

* recombination of cases in State::get

* some maps replaced with unordered_maps

* optimized maps some
2017-08-30 10:43:51 +02:00
Wilfried Goesgens d0f159e848 Don't use global static variables as collection caches - this may cau… (#3150)
* Don't use global static variables as collection caches - this may cause troubles.

* cleanup: remove old cluster kickstarter UI remaints.

* cleanup - remove references to long gone tests
2017-08-28 15:04:05 +02:00
Frank Celler 6d08d4f4aa Bug fix/scheduler delete (#3077)
* removed delete call

* cleanup

* lower cpu activity of log thread too

* fix log messages

* do not enter threads into unordered_set, as it is unneeded

* do not compile in calls to disabled plan cache

* moved AQL regex cache from thread local variables to a class of its own

* more sensible thread creation and destruction
2017-08-25 12:00:17 +02:00
Wilfried Goesgens 0c2261445b use indices on our foxx queues queries; Migrate queries away from aqb… (#3101)
* use indices on our foxx queues queries; Migrate queries away from aqb; style guide cleanup.

* update CHANGELOG
2017-08-24 17:06:03 +02:00
Jan 39465193c2 fix replication (#3099) 2017-08-22 15:51:50 +02:00
Dan Larkin 84f11c223d Fixed issue with periodic syncing and WAL file retention. (#3098) 2017-08-22 15:51:12 +02:00
Dan Larkin 5708fc8fab Added test to ensure RocksDB recovers keygen data outside WAL. (#3072) 2017-08-22 10:42:46 +02:00
Kaveh Vahedipour 15a87b2982 removed database loop around rebalancing of shards; thus rebalance shards now only affects current database (#3091)
This is UI only
2017-08-21 17:13:27 +02:00
Jan 47e29e6e1f Bug fix/issues 1806 (#3069)
* fix buffer overruns in linenoise for long input lines

* don't make historian repeatedly print the same error messages that nothing can be done about

* make the implementations of the logging operator<<s not throw exceptions, so that logging does throw exceptions as an unintended side effect

* update CHANGELOG

* improve error message

* don't copy strings, but pass them by const reference
2017-08-18 22:58:09 +02:00
Mark 136faad9b8 Fix issue #3037: Foxx, internal server error when I try to add a new service (#3056) 2017-08-17 21:47:27 +02:00
Jan e612aad2a3 fixed issue #3044 (#3048) 2017-08-17 21:34:57 +02:00
Jan 19d872dcee fixed issue #3039 (#3045) 2017-08-17 21:33:20 +02:00
Jan 1d6b7abc80 fix and extend journal-related tests (#3043) 2017-08-16 17:39:56 +02:00
Jan 6180fcfdd1 Bug fix/prevent multiple journals (#3027)
* prevent multiple journals

* fix documentation

* remove _nrDesired, as it is not used anymore
2017-08-15 23:02:08 +02:00
Jan 79d76c89fb Bug fix/use count for checksum (#3033)
* Count as checksum

* Make readLockId optional as well so upgrades are possible

* fix option name in startup script

* fix some replication issues with RocksDB engine
2017-08-15 11:24:54 +02:00
Jan 8e4dac4fc4 fix fulltext index removal performance, simplified code (#3015)
* simplify index API a bit

* fix fulltext index removal performance, simplified code

* updated CHANGELOG

* fix hanging test

* try to fix shutdown problem

* improve fulltext query performance

* fixed duplicate var

* removed obsolete code

* fix some shutdown races

* do not call ensureIndex that often
2017-08-12 09:25:30 +02:00
Jan Christoph Uhde 0eaf4cabd9 Feature/js to cpp transaction handler (#2985)
* implement transaction handler in c++

* add fake request object

* move general code into extra file

* improve error messages for RestJobHandler

* support cancel operation for async jobs with RestTransactionHandler

* fix integration test and add some optional debugging output

* fix returning of context

* fix error type

* fix code in cluster test

* remove unused variable
2017-08-11 09:29:34 +02:00
Jan e8d6ff21d0 fix failing recovery test (called undefined function) (#2994) 2017-08-10 09:38:20 +02:00
Michael Hackstein 8b19d63ec7 Merge pull request #2809 from arangodb/bug-fix/trav-filter-optimizer-tests
Fixed Traverser Bug in edge-case where minDepth is lower than filter conditions depth.
2017-08-10 08:44:25 +02:00
Jan 40cbc5ebab Bug fix/fix issue 3011 (#3012) 2017-08-09 17:38:44 +02:00
mchacki 0b4374ead5 Fixed invalid optimisation for variables/references used in traversal 2017-08-09 17:01:18 +02:00
mchacki 3f5c97c8c0 Merge branch 'devel' of github.com:arangodb/arangodb into bug-fix/trav-filter-optimizer-tests 2017-08-09 14:58:52 +02:00
Michael Hackstein 7ab117f5f8 Bug fix/dump restore auth (#2917)
* Added a backup test suite. This suite is supposed to entirely drop an ArangoDB _system and restore it into a fresh one. This also includes system collections.

* Added more test cases for backup suite. Now tests several authorization/rights cenarios

* Fixed RestReplication Handlers to restore _user collections Properly.

* Updated Changelog

* Added special handling of _users in Restore for MMFiles as well.

* Added JWT secret for cluster execution of this test, also added JWT secret to shutdown call
2017-08-09 12:59:51 +02:00
Manuel B 9c268ec815 aql char length null returns now 0 (#2973)
* aql char length null returns now 0

* Update CHANGELOG
2017-08-08 09:33:31 +02:00
Michael Hackstein a885da6190 Added more test cases for associative filter conditions 2017-08-05 10:35:23 -06:00
Michael Hackstein 67e6b7d9b9 Merge branch 'devel' of github.com:arangodb/arangodb into bug-fix/trav-filter-optimizer-tests 2017-08-05 09:30:46 -06:00
Michael Hackstein 48bbc667f7 Added tests for optimization of mindepth. 2017-08-04 17:59:30 -05:00
Jan 52d5941897 fix potential deadlock errors in collector thread (#2933)
* fix potential deadlock errors in collector thread

* add test
2017-08-03 22:47:24 +02:00
Jan a10832d41e do not allow the creation of volatile collections in the RocksDB stor… (#2900)
* do not allow the creation of volatile collections in the RocksDB storage engine

* only create a volatile collection for mmfiles in test

* fix tests
2017-08-03 22:45:17 +02:00
Simon Grätzer 3f58a44e3b Fix cases where the root user has access to excluded collections (#2856) 2017-08-03 21:16:08 +02:00
Manuel B 9917a8d471 aql upper lower cpp (#2936)
* initial aql upper lower

* aql upper lower test for cpp and v8
2017-08-03 21:10:13 +02:00
Jan 80730cd5f4 fixed issue #2928 (#2934) 2017-08-03 13:39:01 +02:00
Jan 62f9231aa5 Bug fix/predictable results data modifcation multiple fors (#2921) 2017-08-01 17:08:35 +02:00
Manuel B f0a24218e3 Feature/cpp aql char length (#2883)
* redirect aql char_length to length

* aql function CHAR_LENGTH in C++, fixed test

* adding table

* fix crlf

* docu table

* remove alternative fact

* docu

* fix string length calculation for aql length
2017-08-01 15:39:25 +02:00
Frank Celler 8fa8bbe5fd Merge branch 'devel' of https://github.com/arangodb/arangodb into bug-fix/trav-filter-optimizer-tests 2017-07-28 15:20:55 +02:00
m0ppers 9a0bc716d0 Do not allow replication to create/drop collections (#2898)
In the cluster case the only one who is allowed to do this is the schmutz
2017-07-28 14:24:02 +02:00
Frank Celler 6613ecda72 Merge branch 'devel' of https://github.com/arangodb/arangodb into bug-fix/trav-filter-optimizer-tests 2017-07-28 09:30:26 +02:00
m0ppers 589ffd5c59 Feature/improve logging (#2881)
* Improve logging in various places

* Fix jslint
2017-07-28 09:23:18 +02:00
m0ppers 6fefe5ab31 In high concurrency situations double checking checksums is a must have (#2890) 2017-07-28 00:43:21 +02:00
Jan 56143bc186 fix issue #2879 (#2885) 2017-07-27 23:30:05 +02:00
Frank Celler 00a6cbab64 fixed jslint error messages 2017-07-27 16:01:45 +02:00
Simon Grätzer 9786b198c2 Allow pregel jobs to interrupt on shutdown (#2855) 2017-07-27 15:29:01 +02:00
Mark 3a59ea553b Add catch when propagateSelfHeal (#2839) 2017-07-19 17:03:52 +02:00
Simon Grätzer f50a36181d Various authentication related issues (#2824) 2017-07-18 22:22:09 +02:00
Frank Celler a692577f5b Feature/auth context (#2815) 2017-07-16 09:39:03 +02:00
mchacki dd13944264 Added some test cases where filter optimization in Traverser could have yield wrong results. 2017-07-15 17:49:00 +02:00
Max Neunhöffer 442be8faad 60s timeout for syncCollectionFinalize. (#2805) 2017-07-14 23:14:04 +02:00
Manuel B df7051f2cb server / client side db._parse added (#2789) 2017-07-14 14:31:51 +02:00
Frank Celler a5a25754ed Feature/reduce extraction to projection (#2792)
* reduce extractions to projections

* recycle string buffers in SocketTask

* micro optimizations for mmfiles indexes

* added special lookup function for _key

* moved function into the correct file

* speed up key buffer allocations a bit

* added noexcept specifier

* correctly name variable

* explicitly move bounds

* fix and speedup from/toPersistent functions

* reuse string from ManagedDocumentResult for multiple lookups

* use move-assign

* a bit less work for single server

* speedup AQL function HASH

* single fetch optimization

* performance optimization for the case when no documents need to be returned

* make reduce-extraction-to-projection a RocksDB-only optimizer rule

* cppcheck

* try to fix compile error on MacOS

* bug fix for MacOSX

* missing namespace (in Windows compile)
2017-07-14 08:40:29 +02:00
Max Neunhöffer 2f874249bb Bug fix/adjust agency comm timeouts (#2765)
* Take out 503 timeouts altogether.
* Overhaul of AgencyComm::sendWithFailover loop.
* Let performRequests optionally ignore 404 coll not found.
* Fix error message "database not found" when AgencyComm failed.
* Add log entries in Agency if locks are acquired too slowly.
* Reexecute the javascript cluster sync stuff even if there was no plan/current change...So failed sync jobs can retry later...
* Cover callbacks in Communicator by lock. This fixes https://github.com/arangodb/planning/issues/370
* Put in delay in waiting for leader in agency test.
* Schmutz logging to heartbeat topic.
* Add more lock time diagnostic in agent.
* Switch on agencycomm tracing in coordinator.
2017-07-13 00:44:28 +02:00
Frank Celler 40d73d5a8b Revert "[WIP] Feature/reduce extraction to projection (#2735)"
This reverts commit 5bfcff30cd.
2017-07-12 12:52:14 +02:00
Jan 5bfcff30cd [WIP] Feature/reduce extraction to projection (#2735)
* reduce extractions to projections

* recycle string buffers in SocketTask

* micro optimizations for mmfiles indexes

* added special lookup function for _key

* moved function into the correct file

* speed up key buffer allocations a bit

* added noexcept specifier

* correctly name variable

* explicitly move bounds

* fix and speedup from/toPersistent functions

* reuse string from ManagedDocumentResult for multiple lookups

* use move-assign

* a bit less work for single server

* speedup AQL function HASH

* single fetch optimization

* performance optimization for the case when no documents need to be returned

* make reduce-extraction-to-projection a RocksDB-only optimizer rule
2017-07-12 11:22:29 +02:00
Kaveh Vahedipour d98481f889 Merge pull request #2769 from arangodb/feature/fix-jslint-aql-graph-traverser.js
missing assertTrue entry in global
2017-07-11 16:04:37 +02:00
Kaveh Vahedipour 774ebae638 missing assertTrue entry in global 2017-07-11 16:02:02 +02:00
Kaveh Vahedipour 231a360b3b fixes for secondaries 2017-07-11 14:05:51 +02:00
Michael Hackstein 2cd48e614b Feature/optimizer rule remove filter covered by traversal (#2731)
* Added a first test for remove-filter-covered-by-traverser rule.

* initial try at adding the functions

* Initial try at fixing all cases

* Fixing optimizer rule order

* Changing rule

* excluding quantifier ANY

* Fixing assertion error

* quick fix for some failed traverser cases

* Fixing quantifier related tests

* Adding upper limit for scannedIndex in cluster

* Fixing the mmfiles asserts

* Allow the coordinator to do condition checking on edges again. Used in enterprise.

* Fix naming of optimizer rules (#2739)
2017-07-10 10:15:17 +02:00
Frank Celler 6b47544611 Revert "Feature/optimizer rule remove filter covered by traversal" (#2724)
* Revert "Feature/authorization query cache (#2720)"

This reverts commit 83712b7b4a.

* Revert "properly return index in case of unique constraint violation (#2716)"

This reverts commit c3f346e0a5.

* Revert "fix https://github.com/arangodb/planning/issues/388 (#2714)"

This reverts commit 1d944b97a4.

* Revert "fix typo (#2718)"

This reverts commit 61a80ed697.

* Revert "Feature/optimizer rule remove filter covered by traversal (#2676)"

This reverts commit c54b81fb69.
2017-07-04 11:13:36 +02:00
Simon Grätzer c54b81fb69 Feature/optimizer rule remove filter covered by traversal (#2676) 2017-07-03 18:39:32 +02:00
Frank Celler bbe7484521 Feature/auth context (#2704)
* added read-only users
2017-07-02 23:15:57 +02:00
Michael Hackstein b73c0dfda4 Merge pull request #2670 from arangodb/feature/issue-277-nextExtra
Feature/issue 277 next extra
2017-06-29 17:24:55 +02:00
Dan Larkin ee8cb749bb Added recovery tests for views and fixed a few related bugs. (#2682)
* Added recovery tests for views and fixed a few related bugs.

* Added more view recovery tests.

* Modified view recovery tests to add a waitForSync operation after.

* fixed usage of wrong view type

* fixed recovery of view change markers
2017-06-29 17:09:43 +02:00
Michael Hackstein 6901a9b53f Fixed a misplaced edge filter It checked for a condition on the wrong depth. 2017-06-29 17:03:05 +02:00
Jan Christoph Uhde 828ab6db23 update unittests 2017-06-28 13:45:17 +02:00
Mark a033eef12d Fix issue 2684: Legacy service UI not working (#2685) 2017-06-28 12:09:15 +02:00
Frank Celler 2807ef559c Feature/move shard fix (#2626)
Major overhaul of handling of synchronous replication.
2017-06-26 16:55:01 +02:00
Mark 50ad6fbd76 Foxx service CRUD optimizations (#2657)
* Pass config/deps on _install so setup can see them

* Only register service when install succeeded

* Don't throw after point of no return: breaky install is breaky

* Delete service files during install if service validation fail

* Delete service files during install if service execution fails

* Fix commitLocalState with invalide services

* Remove deprecated test
2017-06-26 09:28:59 +02:00
Jan cfbd8ed93c Bug fix/rocksdb autoincrement (#2648)
* Added autoincrement keygen support to RocksDB engine with test.

* fixed key generator state keeping for mmfiles engine
2017-06-23 15:27:09 +02:00
Simon Grätzer 52fa4f450a Bugfix for cluster bug (#2649) 2017-06-22 17:34:01 +02:00
Jan c77a6fd738 fix one more limit issue in traversal (#2634) 2017-06-22 12:32:47 +02:00
Jan ebaeb639b2 Bug fix/incremental replication syncs too much (#2631)
* fix too much synchronization
* fix incremental sync for RocksDB engine
2017-06-21 14:56:33 +02:00
Mark 7743db32b9 Fix #2613 (#2614)
Reduce log level when Foxx manager tries to self heal missing database
2017-06-20 12:41:45 +02:00
Simon Grätzer 492d832695 Refactored /_api/index and /_api/database (#2582)
* working documents rest handler

* fixed cluster tests

* Consolidating database APIs

* clang-format

* Fixing issue with user creation through db._createDatabase

* replaced and refactored api-index and index api

* fixed cluster

* renaming some files

* added user methods

* removed files intended for later

* Fixed CC build

* Fixed method signature

* Fixing shell_server, shell_client tests
2017-06-19 23:47:40 +02:00
Jan f38ba7c84d fix invalid results (too many) when a skipping LIMIT was used for a traversal (#2603)
`LIMIT x` or `LIMIT 0, x` were not affected, but `LIMIT s, x` may have returned too many results
2017-06-19 17:34:47 +02:00
Jan 9bebd33ba3 fix invalid first group results for sorted AQL COLLECT when LIMIT was used (#2577) 2017-06-16 09:44:14 +02:00
Max Neunhoeffer cc985becd3 One more logging line in resilience test output. 2017-06-09 10:46:26 +02:00
jsteemann 9b77aae857 fixed some tests 2017-06-08 17:56:58 +02:00
Wilfried Goesgens 7224e7515c Implement testcases, this fixes #1806 2017-06-08 15:59:05 +02:00
Andreas Streichardt a2f5871ee6 Keep retrying coordinator bootstrap 2017-06-07 18:22:08 +02:00
Simon Grätzer fba82fc516 Forgot test case 2017-06-07 17:20:37 +02:00
jsteemann 210a4e485b fix path to version file 2017-06-06 15:42:42 +02:00
Andreas Streichardt 07364e857d Remove useless debug 2017-06-02 11:47:05 +02:00
Michael Hackstein ce9812d8fa Added proper sorting in test code. Not deterministic otherwise 2017-06-02 09:33:30 +02:00
Mop 619eae9be5 Revert "Squashed commit of the following:"
This reverts commit 2252088572.
2017-06-01 18:37:45 +02:00
Andreas Streichardt 2252088572 Squashed commit of the following:
commit f3d0fd6584b0e451b8c97abcb4ba8d9f2fc6f560
Author: Andreas Streichardt <andreas@arangodb.com>
Date:   Thu Jun 1 17:31:36 2017 +0200

    fix unittest

commit 7cd3544a39e1b78af9d4175cb3b978799b9bbfff
Author: Andreas Streichardt <andreas@arangodb.com>
Date:   Thu Jun 1 17:10:00 2017 +0200

    Remove debug comment

commit fb6b10dac15be49a72dbff80030a7d22abdfc3e0
Merge: 055eb1d269 6b18cc64fe
Author: Andreas Streichardt <andreas@arangodb.com>
Date:   Thu Jun 1 17:00:21 2017 +0200

    Merge branch 'devel' into shardorganizer

commit 055eb1d2693a583d21ea59ec8b6ba95ab0db57ac
Merge: 1ff7998ebf 8ea89b7677
Author: Mop <andreas@arangodb.com>
Date:   Thu Jun 1 16:56:30 2017 +0200

    Merge branch 'shardorganizer' of https://github.com/arangodb/arangodb into shardorganizer

commit 8ea89b76777c75b6a77bf695c3f074a0c4643c29
Author: Andreas Streichardt <andreas@arangodb.com>
Date:   Thu Jun 1 16:55:41 2017 +0200

    Fix shardmapping bug

commit 1ff7998ebfd691598ec5b455ca5bc2bfd7020fb4
Author: Mop <andreas@arangodb.com>
Date:   Wed May 31 17:26:08 2017 +0200

    more output

commit 68e88aa0e14316c4929d05b2c151bee6421d754d
Merge: 0978ad1d9e 44a6a78ec3
Author: Mop <andreas@arangodb.com>
Date:   Wed May 31 17:03:33 2017 +0200

    Merge branch 'shardorganizer' of https://github.com/arangodb/arangodb into shardorganizer

commit 44a6a78ec338a1e7cabb15464500d96b84c68f1d
Author: Andreas Streichardt <andreas@arangodb.com>
Date:   Wed May 31 07:42:43 2017 -0700

    Fix namespace

commit 0978ad1d9e2f01b86204990e74b66958f25eba66
Merge: f98582ccff d74e5989ad
Author: Mop <andreas@arangodb.com>
Date:   Wed May 31 16:40:35 2017 +0200

    Merge branch 'shardorganizer' of https://github.com/arangodb/arangodb into shardorganizer

commit f98582ccff3448f6c2388dab4cc2dc38034271b0
Author: Mop <andreas@arangodb.com>
Date:   Wed May 31 16:39:03 2017 +0200

    Revert "Revert "Next attempt at merging ShardOrganizer...distributeShardsLike fixed""

    This reverts commit fed45b7b10.

commit d74e5989ad478efe7d66d196715c05f4f41c9c29
Author: Andreas Streichardt <andreas@arangodb.com>
Date:   Wed May 31 16:31:31 2017 +0200

    Make it an error

commit 0a6a9ef9464df4f24ad205bbab5b9f8ded50054f
Author: Andreas Streichardt <andreas@arangodb.com>
Date:   Wed May 31 12:42:51 2017 +0200

    distributeShardsLike has to be saved as a cidString
2017-06-01 17:32:40 +02:00
jsteemann d13144f389 fixed issue #2515 2017-06-01 13:45:47 +02:00
Alan Plum 4b4b9f8c55 Fix Foxx API error (#2517)
* Fix Foxx API error
* Manager should distinguish between errors and services
* Foxx API should not require service to exist for uninstall
  if force option is used
* Respect options.force in FM _install
2017-06-01 11:59:04 +02:00
jsteemann 3df0940c7e jslint 2017-06-01 10:31:42 +02:00
Akshesh Doshi 46b50c266c Add tests and docs for REGEX_REPLACE AQL function (#2518)
* Fixed #1902: Add tests and documentation for REGEX_REPLACE

* Cleanup
2017-06-01 10:29:43 +02:00
Mark f832dd0388 Implement endpoint "commit coordinator state" (#2513)
Implement endpoint "commit coordinator state" (also see #2479)
2017-05-31 18:21:50 +02:00
Alan Plum 7608cde674 Implement clusterfoxx mounting logic (#2514)
* Serve error when service is out of sync
* Don't check system services for bundles
* Use appropriate errorNum if Foxx blows up
* Only check checksum when file exists
* Don't expose stack traces
2017-05-31 18:21:40 +02:00
Michael Hackstein 52406567cd Replaced EdgeCursors callback to return cid,rev instead of _id. Does less costly copying and comparision internally 2017-05-31 17:49:46 +02:00
Alan Plum ca52aa17b3
Make legacy test less flakey 2017-05-31 11:27:31 +02:00
Alan Plum 83310c12d7
Move FM.listJson into aardvark 2017-05-30 20:19:58 +02:00
Alan Plum 5fc4a17b24
Make sure the bundle doesn't exist before loading it from db 2017-05-30 20:09:58 +02:00
Alan Plum c4305897cf
More realistic binary document test 2017-05-30 18:39:56 +02:00
Alan Plum cceccf59da Simplified Foxx self healing (#2511)
* Implement new self-heal
* Add error codes for 503, service missing/outdated
* Detect changes to service via rev
* Pretty print incoming response object in log
2017-05-30 18:27:32 +02:00
Michael Hackstein eecdcd4e96 Made shortest path OOM test deterministic 2017-05-30 17:41:07 +02:00
jsteemann ed21022f55 added test for shardids restriction 2017-05-29 17:02:16 +02:00
jsteemann a35f85bcc7 added proper docs for --query.fail-on-warning 2017-05-29 15:46:48 +02:00
Frank Celler ee1c60c67a added _binaryDocument and _binaryInsert (#2506) 2017-05-27 22:49:22 +02:00
Andreas Streichardt b5fcd15214 Fix linter 2017-05-24 14:53:06 +02:00
Andreas Streichardt f08cae2028 Explicitly wait for unloaded 2017-05-24 14:37:30 +02:00
jsteemann 0c6b3cfd46 fix non-deterministic behavior in cluster_sync test 2017-05-24 12:51:27 +02:00
Max Neunhoeffer 11e6b9ea7f Merge branch 'devel' of ssh://github.com/ArangoDB/ArangoDB into devel 2017-05-23 20:38:38 +02:00
Andreas Streichardt d8dc3c85e7 Call version instead of foxx status to save a v8 context 2017-05-22 19:07:24 +02:00
Max Neunhoeffer 4bddb0c1ad Handle errors in loadScript for JS, use in cluster-bootstrap.
This recognizes errors that happen during the global cluster bootstrap.
If an error happens, the bootstrap is tried again later. The cluster
does not start until the cluster bootstrap has been finished
successfully.
2017-05-22 15:34:41 +02:00
jsteemann a5070978da fixed issue #2488 2017-05-19 15:57:47 +02:00
Michael Hackstein ca737edd97 Merge branch 'devel' of github.com:arangodb/arangodb into devel 2017-05-17 18:53:35 +02:00
Michael Hackstein 0d18821705 Adapted the shell-index-figures test to honor that the primary index does not have a cache anymore. 2017-05-17 18:52:09 +02:00
Andreas Streichardt 83dff84be7 Merge branch 'devel' of github.com:arangodb/arangodb into devel 2017-05-17 18:19:26 +02:00
Andreas Streichardt 49328dcecb Add authorizedflag fixing cluster internal foxx requests 2017-05-17 18:14:11 +02:00
Alan Plum 9b8a641e77
loadServiceFromDisk was renamed 2017-05-17 16:51:28 +02:00
Alan Plum 8a1286ac1c
Service constructor changed a bit 2017-05-17 16:48:06 +02:00