* fix some deadlocks found by evil lock manager (tm)
* fix duplicate lock
* fix indentation
* ensure proper lock dependencies
* fix lock acquisition
* removed useless comment
* do not lock twice
* create either a V8 transaction context or a standalone transaction context, depending on if we are called from within V8 or not
* AQL micro optimizations
* use explicit constructor
* only use V8DealerFeature's ConditionLocker for acquiring a free V8 context
entering and exiting the selected context is then done later on without having to hold the ConditionLocker
* remove some recursive locks
* Disable custom deadlock detection when Thread Sanitizer is enabled
* Changing ifdef's
* grr
* broke gcc
* Using atomic for ApplicationServer::_server
* fix premature unlock
* add some asserts
* honor collection locking in cluster
* yet one more lock fix
* removed assertion
* some more bugfixes
* Fixing assert
(cherry picked from commit 1155df173bfb67303077fbe04ee8d909517bfd21)
* Fixing an issue with intermediate commits
(cherry picked from commit 7f61fb9be0f22904be21e9dd26c3870035a5b318)
* Only check collection counts without intermediate commits
* Fixing assert
* Add security check in AgencyComm::sendWithFailover.
* some cleanup
* added some more tests
* add typeName() to AgencyCommTransaction to make the transaction type printable in debug messages
* improve debuggability
* Special method to access configured access levels as opposed to only getting the effective access level
* Add integration test
* Allow grants on collection level
* Fixed object assignment operator for agency's key value store
* Node's toJson is now actually toJson. getString should be used for string extractions
* adjust agency's documentation (clarify precondition)
if yes, the server may dynamically adapt the size of the response, in order to ensure that
HTTP responses do not get out of hand size-wise. This is a new feature in devel, and this
commit now makes it optional so that older clients do not need to be changed.
* Removed unused active(...) method in Agent
* Inception's restart from persistence allows peer with empty active RAFT list to join
* Agency's UUID is persisted outside of the database comparable to coordinator and db server action.
* Publicized Methods to UUID stuff in ServerState
* Inception method documentation
* added --agency.disaster-recovery-id to allow for specification of known former agency id. this is a very dangerous option potentially.
* Delete a unused methods.
* separate _id and _recoveryId
* populating active list with entire pool
* Improve logging.
* reject gossip from unknown agent, if pool is complete
* Switching from ttl to supervision based failover mechanism
* Allowing canceling of ongoing actions
* refactored asyncjobmanager
* refactoring some code
* adding read-only flag
* Fixing some bugs
* Fixing tests
* Canceling ongoing operations
* removing some unused code + some asserts
* catching some exceptions to reduce log pollution, removing unnecessary code, removing tests for _changeMode
* fixing "createsANewDatabaseWithAnInvalidUser"
* Current work
* proper ifs
* Migrate resthandler to c++ and implement mode
* readonly server mode spec test
* Add changelog
* change code so it expects a full object and not just a string
* Fix jslint
* Added a more sophisticated test for shardDistribution format
* Updated shard distribution test to use request instead of download
* Added a cxx reporter for the shard distribuation. WIP
* Added some virtual functions/classes for Mocking
* Added a unittest for the new CXX ShardDistribution Reporter.
* The ShardDsitributionReporter now reports Plan and Current correctly. However it does not dare to find a good total/current value and just returns a default. Hence these tests are still red
* Shard distribution now uses the cxx variant
* The ShardDistribution reporter now tries to execute count on the shards
* Updated changelog
* Added error case tests. If the servers time out the mechanism will stop bothering after two seconds and just report default values.
* remove some non-unused V8 persistents
* do not throw that many bogus assertions
* do not rely on server role being defined
* slightly better debug output for V8 context debugging
* fix collection ids in inventory response
* simplify bootstrap a bit
* slightly better error handling
* make elapsed time a queryable value
* use less memory for stub collections
* added assertions that will always make sense
* added assertions
* do not garbage-collect while waiting
* less copying of parameters
* do not show "load indexes into memory" buttons for mmfiles engine
as all indexes are in memory anyway
* when a collection is truncated via the web interface, flush the WAL and rotate all active journals
this will make close all open journals on leader and followers and make them subject to compaction opportunities
* fix invalid server id values being passed from web interface to backend
* introduce afterTruncate method for indexes
* added test case for issue #3447
* updated CHANGELOG
* don't warn about replicationFactor for system collections
* check that the queries actually use the geo index and not some other index
* properly report error in web interface
* fix some internals checks that made truncate fail for bigger collections in maintainer mode
* also run a compact() operation after a serious truncate
in order to make iteration over the truncated range much faster
when the collection is next accessed
* increase default maximum number of V8 contexts to at least 16
* Switching from ttl to supervision based failover mechanism
* Allowing canceling of ongoing actions
* refactored asyncjobmanager
* refactoring some code
* adding read-only flag
* catching some exceptions to reduce log pollution, removing unnecessary code, removing tests for _changeMode
* fixing "createsANewDatabaseWithAnInvalidUser"
* auth = off does not longer make everyone superuser
* Fixing cluster_sync and maybe resilience
* properly initialize all properties
* use faster comparison
* properly detect and handle "method not allowed"
* code-style
* remove unused variable
* narrow variable scope
* handle non-existance of AuthenticationFeature
* remove dead code
* replace some C string handling with std::strings
* moved assertion to the correct place
* honor number of array members for IN operator
* slightly adjust error messages
* slighty adjust some error messages
* try to fix issue with lingering replication contexts on shutdown
* clean up heartbeat thread a little bit
* small fixes
* intermediate commit
* Refactoring the ExecContext
* Fixing authentication
* Added start script
* some fixes
* fixed access to nullptr
* some c++
* fixed misleading message
* Made DatabaseGuard movable. Also adapted map insertions to _vocbase in Syncer classes, which failed to compile under older GCC versions
* added support for global flag to replication handler
* Started Refactoring in replication-static
* Fixing syncer code
* store applier configuration
* Static replication tests now test replication in a non system Database
* added flags to replication feature
* Adding some extra checks
* Fixing issue with rocksdb rest replication handler
* replication static now runs _system and otherdatabase replication tests.
* Fixing crash on startup
* Replication_sync now tests _system as well as other Database
* Fixing up heartbeat thread, adding global flag to rest handler
* Fixing wrong assert
* some cleanup, probably some tests are broken
* Made non-system db version of replication-ongoing tests
* fix determine-open-transaction
* Fixed ongoing tests. And added a test where we drop a database on slave while replication is still ongoing
* test fixes
* Activated ongoing other db tests. Also added a test that drops the DB on master, while the slave is still syncing.
* some better error reporting
* gradually switch to Result
* createCollection -> create
* re-activate using of collection ids for now
* enable auto-start
* Fixed create collection in replication ongoing test
* Added first draft of a test for global replication
* move to Result
* use system database for global applier
* improved error reporting
* fixed invalid URLs
* add test case filter
* load existing global applier configuration
* improve error reporting
* Added further tests for global replication
* Fixed global replication test, it now properly waits for replication. Timeouts after 10 seconds.
* Removed erronious assertion
* improve error reporting
* intermediate commit
* Added a test-case for global replication where the Master already has some data and the slave is clean
* fix deletion of replication contexts
* Fixed JSLint
* compiling code
* fix typo
* do not fail for global applier when no database is configured
* intermediate commit
* syncer supports switch for 3.3 / 3.2
* fixed errors
* Fixing some replication bugs
* Fixing some assertions
* Fixed missing commit markers
* Fixing assertion on database drop
* Attempt to fix deadlock in applier and assertion
* Fixing some stupid things
* Support for collection parameter
* Acidentally turned off some tests
* Grrr
* Fixing wrong method call
* Fixed startscript
* Fixed assignmet instead of equality check typo
* Added a test far interrupted replication. For now it justs tests basics on _system database.
* Improved index tests on replication.
* properly initialize variable
* fixed some replication problems
* MMFiles wal access support
* fix replication issues
* Started mmfiles replication support
* fixing a bug
* Fixing an issue
* fixing some mmfiles stuff
* fix test
* reload users
* prevent pure virtual method call
* intermediate commit
* Making from exclusive
* do not call getMasterState if child syncer
* some reformatting
* Adding global support for handleCommandSync
* Fixing assertion
* removing some debug logs
* Changing return codes
* Fixing some issues in the rest handler
* Make replication less susceptible to errors
* remove some debug output
* return last log tick
* remove waits from tests
* fix two tests
* changing header for open-transactions call
* some fixes
* fix test
* invalidate cached databases
* merging request and execcontext
* try to fix assertion error
* renamed method
* fix compile warning
* small changes
* Always use execcontext
* Fixing an assert
* fix replication issues
* try to fix collection lookups
* try to fix master/slave start
* Changing comments in heartbeat thread
* fix wrong signature of READ_LOCKER_EVENTUAL
* log server role in testing mode
* Fixed authentication, removed execContext in favor of request context
* Adding cluster rest api
* Fixing cluster rest handler
* Fixing cluster callback
* Some refactoring
* Queue creation is not a single operation
* Allowed for leader redirects
* Setting start of batch
* Disabling 2.8 compat tests
* fix start/stop bugs
* jslint
* various little changes
* add flag for exposing jwt
* indentation
* cleanup
* Some changed to guid
* fixing tcp to http, vst
* changed endpoint header
* small fixes
* Reorder servers by health status
* Higher timeout
* Changing error messages
* update the fromTick when fetching multiple batches from the coordinator
* more debug info
* Reducing copy pasted code
* change uid generation
* reducing logspam
* more exceptions for redirects
* more exceptions
* attempt to fix uniqids in cluster
* centralize printing of HTTP errors in replication
* debug output
* fix messages for authentication
* cleanup
* removing --cluster.my-id, --cluster.my-local-info
* Added leadership race to bootstrap, determine foxxmaster on boostrap, removing obsolete code
* improve error reporting in RestAqlHandler
* Changing heartbeat thread, fixing cluster_sync
* some more debug output
* added master
* attempt to make tests more deterministic
* added logging about indexes
* added some safety checks to the logger
* slighty better error messages
* fix location header for SSL
* fix error message
* try to make tests more deterministic
* change error code from TRI_ERROR_INTERNAL (which we want to avoid) to TRI_ERROR_FAILED
* Fixing broken webinterface access
* reverting groovy change
* Fixing read-only internal users
* Using superuser rights for dashboard now
* Adding mode field to _admin/server/role
* added mode TRYAGAIN
* remove inventory lock (does not seem necessary here)
* remove invalid assertion
* fixing agency bugs
* Removing debug output
* return proper errors in case of "method not allowed"
* Fixed up some info messages
* jslint
* improve error reporting in RestAqlHandler
* added logging about indexes
* added some safety checks to the logger
* slighty better error messages
* fix location header for SSL
* fix error message
* try to make tests more deterministic
* change error code from TRI_ERROR_INTERNAL (which we want to avoid) to TRI_ERROR_FAILED
* remove memory zone remainders
* remove useless calls
* attempt to fix non-deterministic CRC test
* attempt to fix non-deterministic replication tests
* fix document removal with multiple documents in cluster
* attempt to fix non-determinism of tests
* fix privilege checks for accessing collections by collection id, not by name
* don't add items into the array if we have an empty list member.
* we mustn't display 100% here, since its a rounding error. If, the checkmark would be set.
* rather use string cutting to round so we don't get into the 100% trap.
- make sure the question is only output after log messages are sent to the console
- make sure error messages are sent before re-quoting for the next try
- inform the user whether the password has been sent
this is required to ensure the console gets one dedicated V8 context and all other
operations have at least one extra context. This requirement was not enforced
anymore.
* fix potential overflow in CRC marker check when a corrupted CRC marker is found at the very beginning of an MMFiles datafile
* added recovery test for marker corruption
* Send out empty heartbeats regardless of non-empty AppendEntriesRPC.
* Also improve logging:
Note if a log in the empty heartbeat sending takes > 0.01 s.
Clearly mark places where a leader resigns in logging.
Log if no empty heartbeat is sent out.
* Make leader more tolerant w.r.t. incoming AppendEntriesRPC responses.
* Add debug logging for _lastAcked and challengeLeadership.
* Remove some unused code. Do not count ourselves in challengeLeadership.
* Removal of entire activation/deactivation mechanisms in agency
* TRI_microtime up to c++11
* added term to response to sendAppendEntries.
[arangod/GeneralServer/AuthenticationFeature.cpp:42]: (warning) Member variable 'AuthenticationFeature::_authenticationTimeout' is not initialized in the constructor.
[arangod/Utils/Authentication.h:49]: (performance) Function parameter 'roles' should be passed by reference.
[arangod/VocBase/AuthInfo.h:47]: (warning) Member variable 'AuthResult::_expiry' is not initialized in the constructor.
* Make isRestore work in the cluster.
This covers sharded collections with default sharding and non-default
sharding.
* always use locally generate revision ids for storing and looking up documents
Also improve logging:
Note if a log in the empty heartbeat sending takes > 0.01 s.
Clearly mark places where a leader resigns in logging.
Log if no empty heartbeat is sent out.
* remove TRI_ERROR_INTERNAL usage
* prevent assertion failure from occurring
* ensure downwards-compatibility for explain handler
* allow schmutz to drop system collections as well - needed for arangojs tests
* Fixed ModifyDocument on coordinator. It uses the internally optimized function to extract _key values, which fails on Compact objects
* do not crash server when passing invalid document key type
* make the different values influencing the compaction run configurable
* Compaction statistics handling
- we mustn't keep the number of dead objects on the compacted datafiles statistics, else it will be compacted again.
- keep statistics of the compaction runs on the DatafileStatistics object
- add the new statistics on DatafileStatistics to the figures api
- implement test that assures only one compaction is run, and the statistic values are maintained
* don't mention the version number
* Implement review
- fix documentation
- allow 0 maxfiles to enable users to disable combined of datafiles
- add statistic element that counts the number of combined datafiles
* Implement review
- fix documentation
- use locks to make statistic values consistent.
- fix typo in variable name
* fix temporary variable unnecessary.
* changelog
* do not use V8 variant of AQL functions in early optimization stage when a C++ variant is available
* additionally, simplify AQL function definitions and aliases
* warn when more than 90% of max mappings are in use
* added C++ variant of replication catchup
* added `--log.role` option
* updated CHANGELOG
* removed non-existing scheduler.threads option from config
* removed useless __FILE__, __LINE__ invocations
* updated CHANGELOG
* allow a priority V8 context
* remove TRI_CORE_MEM_ZONE
* try to fix Windows errors & warnings
* cleanup
* removed memory zones altogether
* exclude system collections from collection tests
* fixed a bug, where when servers failed, when also agency leadership changes
* redid entire design of checkDBServers/checkCoordinators.
* comparison in supervision must be between oldPersisted and newHealth
* UI stuff
* UI stuff
* FailedServer test needed adjustment
* Hopefully final round
* fixed supervision failure detection
* FailedServer tests back to origin devel
* oldNot documented among preconditions in Agency HTTP API docs
* changed only look for status updated
* non action line in api-cluster
* some authentication optimizations...
don't query the permissions for one database three times, and don't build up tons of useless temporary strings for each request
* fix some floating point inaccuracies in AstNode value comparisons
* fixed issue #3075
* update the known issues document, as some issues have already been fixed
* speed up equality lookups in unique vpack indexes
* Allow for re-use of RocksDBKey objects through transaction leaser.
* Converted all RocksDBKey variants to non-static constructors.
* order of free and free2 changed with use of unordered_multimap vs multimap
* fixing order independance for maps in catch tests fixed?
* missing bits and pieces
* Get rid of a compiler warning in community edition.
* Teach /_api/replication/clusterInventory to report Plan/Version and readiness.
This is first implemented in the ClusterInfo library.
Then the clusterInventory code uses it and checks readiness.
Readiness of a collection means that it is created and all shards
and all replications have been created and are in sync.
* return different error codes for check-version
* Adds data file for exit codes that can be used by ArangoDB applications
* add some exit codes
* better return codes for centos
* add codes in arangod
* remove test code
* prevent re-downloading of starter and other small improvements
* fix some erros
* finish centos
* update urls
* add info about systemd detection
* move further towards a working systemd setup
* set service type to simple
* add arangodb-update-db script
* start work on debian packages
* a db dir that is already in place (former installation) should be reused now
* fix minor bugs
* Implement exit code translation in the windows installer.
* add autogenerated NSIS exit code mapping
* added debugging methods
* try to fix invalid access in case of error
* remove unused members
* bugfixes and comments
* all agency fixes in
* merge bug
* partially unguarded Agent::lead fixed
* all agency fixes in
* added nrBlocked to thread startup eval
* added nrBlocked to thread startup eval
* recombination of cases in State::get
* some maps replaced with unordered_maps
* optimized maps some
* added V8 context lifetime control options `--javascript.v8-contexts-max-invocations` and `--javascript.v8-contexts-max-age`
* make thread scheduling take into account most of the tasks dispatched via the io service
* Added checks everywhere for CachedValue::construct.
* Enforce 16MiB limit on keys and 4GiB limit on values.
* Protected members behind function calls in anticipation of format change.
* Shrunk CachedValue struct.
* Fixed alignment of CachedValues.
* Fixed improper comparison.
* Added virtual attributes for enterprise on Methods.cpp
* Working no access collections
* align a comment
* Documentation and test fix
* fixing community build
* removed delete call
* cleanup
* lower cpu activity of log thread too
* fix log messages
* do not enter threads into unordered_set, as it is unneeded
* do not compile in calls to disabled plan cache
* moved AQL regex cache from thread local variables to a class of its own
* more sensible thread creation and destruction
* remove API docs for endpoints that were deleted in 3.0
* remove unused variable
* set all scalar values in the ctor, so check tools do not complain
* move Result objects
* prevent potential nullptr access
* remove ambiguous boolean parameter with no clear meaning
* make code check tools happy
* don't allow copying cache Findings, allow moving Results
* the debug functionality for counting RocksDB key ranges was broken
this produced sporadic test failures
* remove superfluous iostream include
* fix range checks
* don't leak if an error happens during database removal
* try to simplify Result a bit
* try to deletion checks a bit more robust
* fixed issue #3075
* update the known issues document, as some issues have already been fixed
* speed up equality lookups in unique vpack indexes
* fixed wording
* add "cluster selectivity estimates" to CHANGELOG
* add some documentation to RocksDBRestReplicationHandler
* fix building with relative paths
* add some more doc
* add some tests for the replication api
* fix RocksDBRestReplicationHandler and add tests
* update documentation
* remove obsolete parameter
* fix error message
* Implementing logger-first-tick, logger-tick-ranges. Fixing dump `chunkSize` documentation
* fix buffer overruns in linenoise for long input lines
* don't make historian repeatedly print the same error messages that nothing can be done about
* make the implementations of the logging operator<<s not throw exceptions, so that logging does throw exceptions as an unintended side effect
* update CHANGELOG
* improve error message
* don't copy strings, but pass them by const reference
* added unique id to cluster, added access to Health
* added agents to health api
* added agents to health api
* added agents to health api
* transaction information for api
* agents listed like other servers
* missing line through merge conflict
* we must now ignore that datafiles are not sealed
this is because an unsealed datafile may have been produced by
renaming multiple journals to datafiles at server start
* acquire collection count after we have acquired the lock
* count the null byte as well
* fix count value acquisition
* send query fragments to the correct servers, even after failover or when a follower drops
the problem with using the previous shard-based approach is that responsibilities for shards may change at runtime
however, an AQL query must send all requests for the query to the initially used servers.
if there is a failover while the query is executing, we must still send all following requests to the same servers, and not the newly responsible servers
otherwise we potentially would try to get data from a query from server B while the query was only
instanciated on server A.
* extend linenoise character buffer size
* dont make it throw exceptions all the time
* set INSTANCE to nullptr at a later stage
otherwise a nullptr may be accessed during shutdown
Jan
Simon Grätzer
jsteemann
Wilfried Goesgens
Michael Hackstein
Matthew Von-Maszewski
Frank Celler
Kaveh Vahedipour
m0ppers
Jan Christoph Uhde
Max Neunhoeffer
Dan Larkin
Manuel B
Andreas Streichardt
Mark