1
0
Fork 0
Commit Graph

5439 Commits

Author SHA1 Message Date
Simon Grätzer ee8209943f Missing things for active / passive (#3578)
* Switching from ttl to supervision based failover mechanism

* Allowing canceling of ongoing actions

* refactored asyncjobmanager

* refactoring some code

* adding read-only flag

* catching some exceptions to reduce log pollution, removing unnecessary code, removing tests for _changeMode

* fixing "createsANewDatabaseWithAnInvalidUser"

* auth = off does not longer make everyone superuser

* Fixing cluster_sync and maybe resilience
2017-11-04 20:30:23 +01:00
m0ppers b552853909 Feature/optional replication factor enforcement (#3570) 2017-11-02 21:41:25 +01:00
Simon Grätzer 64e9377c05 Replacing /_api/collection with RestHandler (#3543) 2017-11-02 14:57:17 +01:00
Manuel B edba1e57ce Feature/cpp aql left trim right (#3004) (#3506)
* Feature/cpp left trim right (#3004)

* aql char length null returns now 0

* cpp aql trim and tests

* Update CHANGELOG
2017-10-26 20:21:41 +02:00
jsteemann 376cc7dcad fix replicationFactor value validation 2017-10-26 20:15:50 +02:00
jsteemann 21cbb85867 jslint 2017-10-26 15:46:40 +02:00
Simon Grätzer d14710b683 Updating Replication Factor (#3528)
* Changing replication factor in 3.2 (#3513)

* Allow changing replicationFactor on coordinator

* Fixing logic

* Allowing change of replication factor

* Additional input validation

* grrr

* Testing invalid inputs

(cherry picked from commit 47600e1ea1)

* jslint

(cherry picked from commit 4d223597db)

* cherry-pick commits from 3.2: 47600e1ea1 4d223597db d684eaa7f8 a898af3723
2017-10-26 15:28:05 +02:00
Jan c569c1b816 fix test (#3524) 2017-10-26 14:04:41 +02:00
Max Neunhöffer 5e2f17c04f Fast path for get-in-sync of empty shards. (#3470)
* Fast path for get-in-sync of empty shards.

* Fix short cut in synchronization.
2017-10-26 09:55:00 +02:00
jsteemann 7f00bf3b7d next attempt 2017-10-26 09:34:48 +02:00
jsteemann 45f2c4aca7 try to fix non-deterministic test 2017-10-25 21:07:31 +02:00
Jan 9eb5e1545f Bug fix/fix cluster foxx queue startup (#3507)
* slightly clean up indexes

* proper stringification of error code

* do not run AQL queries during Foxx queue startup

* always create keyspace
2017-10-25 18:07:27 +02:00
Jan 9a4703dbde changed test expectations in cluster mode (#3491) 2017-10-25 18:05:37 +02:00
Manuel B 7d4b319c58 cpp aql substring, fixed v8 substring (#3489) 2017-10-23 17:12:45 +02:00
Jan feba30fcec attempt to make replication test more deterministic (#3482) 2017-10-23 14:21:46 +02:00
Manuel B be8c648620 cpp aql collections() and test (#3331)
* cpp aql collections() and test

* remview fixes

* fixed jslint

* fix for new ExecContext / auth api
2017-10-23 11:42:34 +02:00
Manuel B cf0578242f Feature/devel cpp aql matches (#3329)
* matches function header

* aql matches() C++ definition

* cpp aql matches() implementation

* review fixes

* test for v8 / cpp

* aql cpp matches matches behavior of js matches, more tests

* fix cpp matches()

* final fixes
2017-10-23 08:25:55 +02:00
Simon Grätzer fd3f9d99d9 Fixing webinterface access (#3464)
* intermediate commit

* Refactoring the ExecContext

* Fixing authentication

* Added start script

* some fixes

* fixed access to nullptr

* some c++

* fixed misleading message

* Made DatabaseGuard movable. Also adapted map insertions to _vocbase in Syncer classes, which failed to compile under older GCC versions

* added support for global flag to replication handler

* Started Refactoring in replication-static

* Fixing syncer code

* store applier configuration

* Static replication tests now test replication in a non system Database

* added flags to replication feature

* Adding some extra checks

* Fixing issue with rocksdb rest replication handler

* replication static now runs _system and otherdatabase replication tests.

* Fixing crash on startup

* Replication_sync now tests _system as well as other Database

* Fixing up heartbeat thread, adding global flag to rest handler

* Fixing wrong assert

* some cleanup, probably some tests are broken

* Made non-system db version of replication-ongoing tests

* fix determine-open-transaction

* Fixed ongoing tests. And added a test where we drop a database on slave while replication is still ongoing

* test fixes

* Activated ongoing other db tests. Also added a test that drops the DB on master, while the slave is still syncing.

* some better error reporting

* gradually switch to Result

* createCollection -> create

* re-activate using of collection ids for now

* enable auto-start

* Fixed create collection in replication ongoing test

* Added first draft of a test for global replication

* move to Result

* use system database for global applier

* improved error reporting

* fixed invalid URLs

* add test case filter

* load existing global applier configuration

* improve error reporting

* Added further tests for global replication

* Fixed global replication test, it now properly waits for replication. Timeouts after 10 seconds.

* Removed erronious assertion

* improve error reporting

* intermediate commit

* Added a test-case for global replication where the Master already has some data and the slave is clean

* fix deletion of replication contexts

* Fixed JSLint

* compiling code

* fix typo

* do not fail for global applier when no database is configured

* intermediate commit

* syncer supports switch for 3.3 / 3.2

* fixed errors

* Fixing some replication bugs

* Fixing some assertions

* Fixed missing commit markers

* Fixing assertion on database drop

* Attempt to fix deadlock in applier and assertion

* Fixing some stupid things

* Support for collection parameter

* Acidentally turned off some tests

* Grrr

* Fixing wrong method call

* Fixed startscript

* Fixed assignmet instead of equality check typo

* Added a test far interrupted replication. For now it justs tests basics on _system database.

* Improved index tests on replication.

* properly initialize variable

* fixed some replication problems

* MMFiles wal access support

* fix replication issues

* Started mmfiles replication support

* fixing a bug

* Fixing an issue

* fixing some mmfiles stuff

* fix test

* reload users

* prevent pure virtual method call

* intermediate commit

* Making from exclusive

* do not call getMasterState if child syncer

* some reformatting

* Adding global support for handleCommandSync

* Fixing assertion

* removing some debug logs

* Changing return codes

* Fixing some issues in the rest handler

* Make replication less susceptible to errors

* remove some debug output

* return last log tick

* remove waits from tests

* fix two tests

* changing header for open-transactions call

* some fixes

* fix test

* invalidate cached databases

* merging request and execcontext

* try to fix assertion error

* renamed method

* fix compile warning

* small changes

* Always use execcontext

* Fixing an assert

* fix replication issues

* try to fix collection lookups

* try to fix master/slave start

* Changing comments in heartbeat thread

* fix wrong signature of READ_LOCKER_EVENTUAL

* log server role in testing mode

* Fixed authentication, removed execContext in favor of request context

* Adding cluster rest api

* Fixing cluster rest handler

* Fixing cluster callback

* Some refactoring

* Queue creation is not a single operation

* Allowed for leader redirects

* Setting start of batch

* Disabling 2.8 compat tests

* fix start/stop bugs

* jslint

* various little changes

* add flag for exposing jwt

* indentation

* cleanup

* Some changed to guid

* fixing tcp to http, vst

* changed endpoint header

* small fixes

* Reorder servers by health status

* Higher timeout

* Changing error messages

* update the fromTick when fetching multiple batches from the coordinator

* more debug info

* Reducing copy pasted code

* change uid generation

* reducing logspam

* more exceptions for redirects

* more exceptions

* attempt to fix uniqids in cluster

* centralize printing of HTTP errors in replication

* debug output

* fix messages for authentication

* cleanup

* removing --cluster.my-id, --cluster.my-local-info

* Added leadership race to bootstrap, determine foxxmaster on boostrap, removing obsolete code

* improve error reporting in RestAqlHandler

* Changing heartbeat thread, fixing cluster_sync

* some more debug output

* added master

* attempt to make tests more deterministic

* added logging about indexes

* added some safety checks to the logger

* slighty better error messages

* fix location header for SSL

* fix error message

* try to make tests more deterministic

* change error code from TRI_ERROR_INTERNAL (which we want to avoid) to TRI_ERROR_FAILED

* Fixing broken webinterface access

* reverting groovy change

* Fixing read-only internal users

* Using superuser rights for dashboard now

* Adding mode field to _admin/server/role

* added mode TRYAGAIN

* remove inventory lock (does not seem necessary here)

* remove invalid assertion

* fixing agency bugs

* Removing debug output

* return proper errors in case of "method not allowed"

* Fixed up some info messages

* jslint
2017-10-20 18:06:59 +02:00
Jan 7840d3f824 Bug fix/fixes 1810 (#3460)
* improve error reporting in RestAqlHandler

* added logging about indexes

* added some safety checks to the logger

* slighty better error messages

* fix location header for SSL

* fix error message

* try to make tests more deterministic

* change error code from TRI_ERROR_INTERNAL (which we want to avoid) to TRI_ERROR_FAILED
2017-10-19 11:28:01 +02:00
Simon Grätzer 7c31960cf2 Feature/async failover (#3451) 2017-10-18 23:59:29 +02:00
Jan 451abcb23d added non-destructive DISTINCT (#3320)
* added non-destructive DISTINCT

* added tests

* update documentation
2017-10-18 12:52:33 +02:00
Jan 0eb59e9bec do not use `_doCount` attribute (which does nothing) (#3454) 2017-10-18 12:43:34 +02:00
Jan 2fee652e77 Bug fix/fixes 1610 (#3425)
* remove memory zone remainders

* remove useless calls

* attempt to fix non-deterministic CRC test

* attempt to fix non-deterministic replication tests

* fix document removal with multiple documents in cluster

* attempt to fix non-determinism of tests

* fix privilege checks for accessing collections by collection id, not by name
2017-10-18 12:23:39 +02:00
Simon Grätzer a7903495a5 Fix createQueue silent unique constraint violation, Fix asan detection in clang, (#3421) 2017-10-18 12:21:31 +02:00
Frank Celler 8f0f861cc4 disabled corrupted-crc-mmfiles.js 2017-10-15 11:26:02 +02:00
Jan f1497e0df7 fixed issue #3395 (#3399) 2017-10-13 12:25:46 +02:00
Jan dd32992121 fix potential overflow in CRC marker check when a corrupted CRC marke… (#3386)
* fix potential overflow in CRC marker check when a corrupted CRC marker is found at the very beginning of an MMFiles datafile

* added recovery test for marker corruption
2017-10-10 09:49:13 +02:00
Jan 2525a3a1bd Feature/mmfiles hash lookup performance (#3265)
* speed up full collection scans in mmfiles engine

* some API cleanup
2017-09-26 14:48:06 +02:00
Jan 43e0fd318e Bug fix/fixes 1909 (#3285)
* remove TRI_ERROR_INTERNAL usage

* prevent assertion failure from occurring

* ensure downwards-compatibility for explain handler

* allow schmutz to drop system collections as well - needed for arangojs tests
2017-09-19 21:56:32 +02:00
Wilfried Goesgens 0faf9be695 Feature/compaction configuration (#3217)
* make the different values influencing the compaction run configurable

* Compaction statistics handling

 - we mustn't keep the number of dead objects on the compacted datafiles statistics, else it will be compacted again.
 - keep statistics of the compaction runs on the DatafileStatistics object
 - add the new statistics on DatafileStatistics to the figures api
 - implement test that assures only one compaction is run, and the statistic values are maintained

* don't mention the version number

* Implement review

 - fix documentation
 - allow 0 maxfiles to enable users to disable combined of datafiles
 - add statistic element that counts the number of combined datafiles

* Implement review

 - fix documentation
 - use locks to make statistic values consistent.
 - fix typo in variable name

* fix temporary variable unnecessary.

* changelog
2017-09-15 14:37:30 +02:00
Wilfried Goesgens 2833bd6185 Bugfix/cleanup graphs (#3250)
* when running against an external server also setup the proper connection

* cleanup grahs after testing
2017-09-15 14:20:21 +02:00
Jan 5165155ed1 Bug fix/fixes 0609 (#3227)
* do not use V8 variant of AQL functions in early optimization stage when a C++ variant is available

* additionally, simplify AQL function definitions and aliases

* warn when more than 90% of max mappings are in use

* added C++ variant of replication catchup

* added `--log.role` option

* updated CHANGELOG

* removed non-existing scheduler.threads option from config

* removed useless __FILE__, __LINE__ invocations

* updated CHANGELOG

* allow a priority V8 context

* remove TRI_CORE_MEM_ZONE

* try to fix Windows errors & warnings

* cleanup

* removed memory zones altogether

* exclude system collections from collection tests
2017-09-13 16:28:21 +02:00
Jan dc46c9db07 more logging so we can find a specific bug (#3247) 2017-09-13 09:19:50 +02:00
jsteemann f8d8a561a4 fix various issues 2017-09-13 09:03:28 +02:00
Wilfried Goesgens b41cfac17c cleanup 2017-09-13 00:00:32 +02:00
Wilfried Goesgens 11437d20a9 fix cleanup 2017-09-12 15:57:33 +02:00
Wilfried Goesgens 0a9b9e6025 cleanup after testrun, lintify 2017-09-12 14:48:20 +02:00
Jan eacad99df7 Bug fix/fixes 0409 (#3199)
* some authentication optimizations...

don't query the permissions for one database three times, and don't build up tons of useless temporary strings for each request

* fix some floating point inaccuracies in AstNode value comparisons
2017-09-04 18:04:05 +02:00
Jan f49e0ca237 execute DOCUMENT function via CXX on coordinator in cluster mode (#3196) 2017-09-04 17:34:24 +02:00
jsteemann b1fd92c438 removed useless explain 2017-09-01 14:14:01 +02:00
Simon Grätzer 88d01b89b5 Optimizations for Caches and Graph Traversals (#3169) 2017-08-31 18:33:10 +02:00
Wilfried Goesgens c662dfc863 Cleanup after each testrun (#3109)
* Cleanup after each testrun

 - add a success testsuite next to the fail testsuite
 - create temporary directories in the fail & success testsuites
 - add a function to cleanup the last unshifted directory
 - don't say we're crashed when exiting with the help message

* Specify temp path for SUT so its files will be removed after cleanup.

 - cleanup
 - don't use getTempFile(), it will create another subdirectory that we don't clean up afterwards...

* Fix temp path usage

 - creating a 'bundles' directory wherein we would keep our temporary bundle files would collide if we'd clean it up.
 - create 'bundles-tmpxxxx' instead so its individual to us and we can remove it when done without problems.

* properly create & pass in a temporary directory - as we now require this.

* Remove intermediate directory recursive - for some unknown reason there may be temp files created.

* we need to unify tempfilenames here too.

* use individual suite names

* use the standart performTests to run the backup tests - so we have the proper directory and server handling from the infrastructure.

* fix temporary directory handling in the recovery tests.

* script to build debugging debian packages

* Improve test shutdown

 - handle a set of options globally per testsuite, so they don't need to clone them inside.
 - properly cleanup the slave instance of the replication tests after each run.
 - The per testsuite options vector will fix the behaviour that after one failed testsuite
   all other suites won't be cleaned up anymore.

* don't call the overal cleanup if anything failed.

* fix cleaning up of directories.

* cleanup, fix directory creation / deletion / dummy creation.

* cleanup, fix directory creation / deletion / dummy creation.

* fix tempdir handling for the recovery tests
2017-08-30 10:48:24 +02:00
Simon Grätzer 885ea3dd5f Remove attribute option for arangoimp (#3141)
* Adding a remove attribute option to arangoimp (only for csv)

* Adding documentation

* Fixed log statement

* Update CHANGELOG
2017-08-30 10:45:02 +02:00
Kaveh Vahedipour 00650e6a3f Bug fix/agency mt fixes (#3158)
* added debugging methods

* try to fix invalid access in case of error

* remove unused members

* bugfixes and comments

* all agency fixes in

* merge bug

* partially unguarded Agent::lead fixed

* all agency fixes in

* added nrBlocked to thread startup eval

* added nrBlocked to thread startup eval

* recombination of cases in State::get

* some maps replaced with unordered_maps

* optimized maps some
2017-08-30 10:43:51 +02:00
Wilfried Goesgens d0f159e848 Don't use global static variables as collection caches - this may cau… (#3150)
* Don't use global static variables as collection caches - this may cause troubles.

* cleanup: remove old cluster kickstarter UI remaints.

* cleanup - remove references to long gone tests
2017-08-28 15:04:05 +02:00
Frank Celler 6d08d4f4aa Bug fix/scheduler delete (#3077)
* removed delete call

* cleanup

* lower cpu activity of log thread too

* fix log messages

* do not enter threads into unordered_set, as it is unneeded

* do not compile in calls to disabled plan cache

* moved AQL regex cache from thread local variables to a class of its own

* more sensible thread creation and destruction
2017-08-25 12:00:17 +02:00
Wilfried Goesgens 0c2261445b use indices on our foxx queues queries; Migrate queries away from aqb… (#3101)
* use indices on our foxx queues queries; Migrate queries away from aqb; style guide cleanup.

* update CHANGELOG
2017-08-24 17:06:03 +02:00
Jan 39465193c2 fix replication (#3099) 2017-08-22 15:51:50 +02:00
Dan Larkin 84f11c223d Fixed issue with periodic syncing and WAL file retention. (#3098) 2017-08-22 15:51:12 +02:00
Dan Larkin 5708fc8fab Added test to ensure RocksDB recovers keygen data outside WAL. (#3072) 2017-08-22 10:42:46 +02:00