1
0
Fork 0
Commit Graph

686 Commits

Author SHA1 Message Date
Frank Celler cb0aaa435f backport Scheduler from devel (#5533) 2018-06-28 13:26:04 +02:00
Tobias Gödderz a391088dbf [3.3] Fix inconsistent distributeShardsLike job (#5300) 2018-05-17 12:07:20 +02:00
Simon c954841a4f Backport supervision for active failover job + testsuite (#5181) 2018-04-23 16:38:11 +02:00
Frank Celler 46b7dbcf75
added /_admin/status (#5047)
* added /_admin/status
2018-04-07 21:24:43 +02:00
Simon 1d57a46168 Refactor Authentication Layer (3.3) (#4588)
* added tests for revokeCollection and revokeDatabase

* optimized user permission test

* ui selection bugfix

* fixed ldap ui login

* login view

* Authentication refactoring

* localstorage fallback if user config is not available

* Fixing permission resolution test

* Adding missing import

* local storage queries now supported

* disabled collection task check for ldap

* added internal ldapEnabled function and ldap config to the ui

* more db creation tests

* removed console logs

* render fix

* Various authentication related fixes

* exec ldap test also for cluster

* Adding support to refresh user rights from external auth sources

* ldap test howto comment

* Handling roles more correctly

* jwt

* login view

* First part of rework of LDAP documentation.

* test roles in a ldap environment

* Changing role handling

* Finish revision of the LDAP chapter in the manual.

* Fixing user header

* Fixing some slight issues with LDAP users and roles

* Removing unused code

* Removing unused code

* added ldap test

* fixing a bug in restuserhandler

* more ldap tests

* ldap tests

* ldap tests

* optimized ldap testing, added cluster support, fixed some auth tests

* ldap cleanup test

* ldap tests

* auth tests

* ldap test

* Changing permission defaults

* revert change

* updated user helper test

* rm of try catch block connection

* Removing FeatureCacheFeature

* Changed permission resolution according to discussion

* updated the docs to clarify the permission resolution and the intricacies of LDAP users

* Fixing wrong permissions check in handling of PUT in ResUserHandler

* Using revision ID when replacing users

* Adding basic replace test

* tests

* Fixing some outstanding issues

* Fixing test setup, optimizing some stuff

* Fixing permission resolution rules, testsuite setup, etc

* Fix deadlock

* Adding error message for keyspace, slightly changing test setup

* Removing remaining mentions of FeatureCacheFeature

* Fix jslint

* Fixing some failed tests

* Fixing cluster authentication issue, red tests

* Fixing ldap testsuite, adding trace logging

* Fixint ldap tesuite setup and LDAP recognition

* Fixing an assert

* Cleanup, adding changelog entry

* fix typo

* Fixing dump_authentication test

* improvements found during code review

* oops

* updated CHANGELOG

* Fixing broken handling, disallowing adding of local users when disabled

* added tests for ldap search mode

* Fixing testInvalidGrants

(cherry picked from commit bc7ea2aaa29a9ed0974898f487e8a318f24912f1)

* Removing undefined auth level externally

(cherry picked from commit 70859f43ae6fd694fdbf70f669fbfdafc58e7913)

* Fixing previous commit

(cherry picked from commit 2fbcffd2ed657862ef9fb5e6d45201a6ec8ada69)

* more use of sessionstorage

* intentionally removed `after` methods from tests

because they are executed before the tests start
no cleanup is performed right now after the authentication tests
however, a cleanup is done at start of every test

* ldap tests all modes

* fix LDAP test invocation

* Added roles transformation to ldap test suite

* Fix compilation of community version.

* Imrpved the ldap testsuites by unifying their options

* fix permission problems for system collections

* Improved LDAP configuration documentation.

* Grunt.

* fixed some ro/rw display issues

* fixed some ro/rw display issues part 2

* grunt build

* bump version number

* Fixed typos in LDAP manual
2018-02-28 13:24:18 +01:00
Simon e96e899fd3 Active Failover for Foxx Services (3.3) (#4593) 2018-02-15 09:36:25 +01:00
Max Neunhöffer 1ca1b89deb
Port static-build fixes to the 3.3 branch (libmusl). (#4481)
* Changes for libmusl compilation. (#4294)
* Further fixes for libmusl completely static build. (#4474)
* Make crc assembler code position independent.
* Fix PIE compilation for libmusl in alpine linux.
* Use backtrace library if found.
2018-02-07 10:49:32 +01:00
Matthew Von-Maszewski d35ebbe6a1 move from devel to 3.3 the dynamic chooseTimeout() feature. (#4166) 2017-12-27 16:36:42 +01:00
m0ppers 6cbf7159be Feature/server mode (#3590)
* Switching from ttl to supervision based failover mechanism

* Allowing canceling of ongoing actions

* refactored asyncjobmanager

* refactoring some code

* adding read-only flag

* Fixing some bugs

* Fixing tests

* Canceling ongoing operations

* removing some unused code + some asserts

* catching some exceptions to reduce log pollution, removing unnecessary code, removing tests for _changeMode

* fixing "createsANewDatabaseWithAnInvalidUser"

* Current work

* proper ifs

* Migrate resthandler to c++ and implement mode

* readonly server mode spec test

* Add changelog

* change code so it expects a full object and not just a string

* Fix jslint
2017-11-10 17:56:21 +01:00
Michael Hackstein 5c633f9fae Bug fix/speedup shard distribution (#3645)
* Added a more sophisticated test for shardDistribution format

* Updated shard distribution test to use request instead of download

* Added a cxx reporter for the shard distribuation. WIP

* Added some virtual functions/classes for Mocking

* Added a unittest for the new CXX ShardDistribution Reporter.

* The ShardDsitributionReporter now reports Plan and Current correctly. However it does not dare to find a good total/current value and just returns a default. Hence these tests are still red

* Shard distribution now uses the cxx variant

* The ShardDistribution reporter now tries to execute count on the shards

* Updated changelog

* Added error case tests. If the servers time out the mechanism will stop bothering after two seconds and just report default values.
2017-11-10 15:17:08 +01:00
Jan 7613bc4314 Bug fix/fixes 0211 (#3568)
* remove some non-unused V8 persistents

* do not throw that many bogus assertions

* do not rely on server role being defined

* slightly better debug output for V8 context debugging

* fix collection ids in inventory response

* simplify bootstrap a bit

* slightly better error handling

* make elapsed time a queryable value

* use less memory for stub collections

* added assertions that will always make sense

* added assertions

* do not garbage-collect while waiting

* less copying of parameters

* do not show "load indexes into memory" buttons for mmfiles engine

  as all indexes are in memory anyway

* when a collection is truncated via the web interface, flush the WAL and rotate all active journals

this will make close all open journals on leader and followers and make them subject to compaction opportunities

* fix invalid server id values being passed from web interface to backend

* introduce afterTruncate method for indexes

* added test case for issue #3447

* updated CHANGELOG

* don't warn about replicationFactor for system collections

* check that the queries actually use the geo index and not some other index

* properly report error in web interface

* fix some internals checks that made truncate fail for bigger collections in maintainer mode

* also run a compact() operation after a serious truncate

in order to make iteration over the truncated range much faster
when the collection is next accessed

* increase default maximum number of V8 contexts to at least 16
2017-11-09 12:48:15 +01:00
Simon Grätzer ee8209943f Missing things for active / passive (#3578)
* Switching from ttl to supervision based failover mechanism

* Allowing canceling of ongoing actions

* refactored asyncjobmanager

* refactoring some code

* adding read-only flag

* catching some exceptions to reduce log pollution, removing unnecessary code, removing tests for _changeMode

* fixing "createsANewDatabaseWithAnInvalidUser"

* auth = off does not longer make everyone superuser

* Fixing cluster_sync and maybe resilience
2017-11-04 20:30:23 +01:00
Simon Grätzer 64e9377c05 Replacing /_api/collection with RestHandler (#3543) 2017-11-02 14:57:17 +01:00
Simon Grätzer 7c31960cf2 Feature/async failover (#3451) 2017-10-18 23:59:29 +02:00
Max Neunhöffer d86f27bd19 Bug fix/agency leader timeouts (#3373)
* Send out empty heartbeats regardless of non-empty AppendEntriesRPC.
* Also improve logging:
  Note if a log in the empty heartbeat sending takes > 0.01 s.
  Clearly mark places where a leader resigns in logging.
  Log if no empty heartbeat is sent out.
* Make leader more tolerant w.r.t. incoming AppendEntriesRPC responses.
* Add debug logging for _lastAcked and challengeLeadership.
* Remove some unused code. Do not count ourselves in challengeLeadership.
* Removal of entire activation/deactivation mechanisms in agency
* TRI_microtime up to c++11
* added term to response to sendAppendEntries.
2017-10-06 10:11:51 +02:00
Jan 4ba38bf981 try to work around some assertions (#3296)
* remove obsolete values from relative config

* warn about using obsolete config parameters
2017-09-28 09:21:33 +02:00
Simon Grätzer ffc465433a No access collections Improvements (#3190)
* consolidated EdgeDocumentToken

* optimizing cluster traversal

* adding skip collection checks

* API cleanup

* copying AQLValue to avoid use-after-free bugs

* Fixing rocksdb SingleServerEdgeCursor

* Fixing a collection resolving issue
2017-09-07 14:55:07 +02:00
Kaveh Vahedipour ef59a347a9 fix broken enterprise build (#3181) 2017-09-01 10:32:18 +02:00
Andreas Streichardt dfb56301ac PUBLIC enterprise definitions 2017-08-31 20:09:20 +02:00
Simon Grätzer 88d01b89b5 Optimizations for Caches and Graph Traversals (#3169) 2017-08-31 18:33:10 +02:00
Andreas Streichardt 7d2ab6055f Merge with latest changes in improve-jenkins
stashing should work better now
2017-08-31 17:53:48 +02:00
Frank Celler 6d08d4f4aa Bug fix/scheduler delete (#3077)
* removed delete call

* cleanup

* lower cpu activity of log thread too

* fix log messages

* do not enter threads into unordered_set, as it is unneeded

* do not compile in calls to disabled plan cache

* moved AQL regex cache from thread local variables to a class of its own

* more sensible thread creation and destruction
2017-08-25 12:00:17 +02:00
Jan Christoph Uhde 0eaf4cabd9 Feature/js to cpp transaction handler (#2985)
* implement transaction handler in c++

* add fake request object

* move general code into extra file

* improve error messages for RestJobHandler

* support cancel operation for async jobs with RestTransactionHandler

* fix integration test and add some optional debugging output

* fix returning of context

* fix error type

* fix code in cluster test

* remove unused variable
2017-08-11 09:29:34 +02:00
Jan 2b791af3dd Feature/jemalloc 5.0.1 (#2991)
* updated CHANGELOG
2017-08-10 09:46:43 +02:00
Jan e6c6c5acda Bug fix/pre release woes (#2988)
* fixed issue #2982

* updated CHANGELOG
2017-08-10 09:39:28 +02:00
Michael Hackstein 7ab117f5f8 Bug fix/dump restore auth (#2917)
* Added a backup test suite. This suite is supposed to entirely drop an ArangoDB _system and restore it into a fresh one. This also includes system collections.

* Added more test cases for backup suite. Now tests several authorization/rights cenarios

* Fixed RestReplication Handlers to restore _user collections Properly.

* Updated Changelog

* Added special handling of _users in Restore for MMFiles as well.

* Added JWT secret for cluster execution of this test, also added JWT secret to shutdown call
2017-08-09 12:59:51 +02:00
Dan Larkin 5f772b09a7 Converted rest handler for explain from JS to C++. (#2907) 2017-08-01 22:10:39 +02:00
Dan Larkin 4cc46f9b96 Converted a portion of the admin routing API from JS to C++ (#2919)
* Added a partial C++ implementation for admin routing API.

* Finished C++ implementation for admin routing API.
2017-08-01 17:14:08 +02:00
Dan Larkin 750fc96ab4 Converted endpoint handler from JS to C++ (#2905)
* Added native C++ rest handler for endpoints.

* Added test of rest endpoint interface and removed JS handler.
2017-08-01 09:53:04 +02:00
Frank Celler a692577f5b Feature/auth context (#2815) 2017-07-16 09:39:03 +02:00
Frank Celler a5a25754ed Feature/reduce extraction to projection (#2792)
* reduce extractions to projections

* recycle string buffers in SocketTask

* micro optimizations for mmfiles indexes

* added special lookup function for _key

* moved function into the correct file

* speed up key buffer allocations a bit

* added noexcept specifier

* correctly name variable

* explicitly move bounds

* fix and speedup from/toPersistent functions

* reuse string from ManagedDocumentResult for multiple lookups

* use move-assign

* a bit less work for single server

* speedup AQL function HASH

* single fetch optimization

* performance optimization for the case when no documents need to be returned

* make reduce-extraction-to-projection a RocksDB-only optimizer rule

* cppcheck

* try to fix compile error on MacOS

* bug fix for MacOSX

* missing namespace (in Windows compile)
2017-07-14 08:40:29 +02:00
Frank Celler 40d73d5a8b Revert "[WIP] Feature/reduce extraction to projection (#2735)"
This reverts commit 5bfcff30cd.
2017-07-12 12:52:14 +02:00
Jan 5bfcff30cd [WIP] Feature/reduce extraction to projection (#2735)
* reduce extractions to projections

* recycle string buffers in SocketTask

* micro optimizations for mmfiles indexes

* added special lookup function for _key

* moved function into the correct file

* speed up key buffer allocations a bit

* added noexcept specifier

* correctly name variable

* explicitly move bounds

* fix and speedup from/toPersistent functions

* reuse string from ManagedDocumentResult for multiple lookups

* use move-assign

* a bit less work for single server

* speedup AQL function HASH

* single fetch optimization

* performance optimization for the case when no documents need to be returned

* make reduce-extraction-to-projection a RocksDB-only optimizer rule
2017-07-12 11:22:29 +02:00
Jan 418105b1f5 remove obsolete cmake commands (#2709) 2017-07-02 23:26:29 +02:00
Frank Celler bbe7484521 Feature/auth context (#2704)
* added read-only users
2017-07-02 23:15:57 +02:00
Simon Grätzer 492d832695 Refactored /_api/index and /_api/database (#2582)
* working documents rest handler

* fixed cluster tests

* Consolidating database APIs

* clang-format

* Fixing issue with user creation through db._createDatabase

* replaced and refactored api-index and index api

* fixed cluster

* renaming some files

* added user methods

* removed files intended for later

* Fixed CC build

* Fixed method signature

* Fixing shell_server, shell_client tests
2017-06-19 23:47:40 +02:00
Mop 619eae9be5 Revert "Squashed commit of the following:"
This reverts commit 2252088572.
2017-06-01 18:37:45 +02:00
Andreas Streichardt 2252088572 Squashed commit of the following:
commit f3d0fd6584b0e451b8c97abcb4ba8d9f2fc6f560
Author: Andreas Streichardt <andreas@arangodb.com>
Date:   Thu Jun 1 17:31:36 2017 +0200

    fix unittest

commit 7cd3544a39e1b78af9d4175cb3b978799b9bbfff
Author: Andreas Streichardt <andreas@arangodb.com>
Date:   Thu Jun 1 17:10:00 2017 +0200

    Remove debug comment

commit fb6b10dac15be49a72dbff80030a7d22abdfc3e0
Merge: 055eb1d269 6b18cc64fe
Author: Andreas Streichardt <andreas@arangodb.com>
Date:   Thu Jun 1 17:00:21 2017 +0200

    Merge branch 'devel' into shardorganizer

commit 055eb1d2693a583d21ea59ec8b6ba95ab0db57ac
Merge: 1ff7998ebf 8ea89b7677
Author: Mop <andreas@arangodb.com>
Date:   Thu Jun 1 16:56:30 2017 +0200

    Merge branch 'shardorganizer' of https://github.com/arangodb/arangodb into shardorganizer

commit 8ea89b76777c75b6a77bf695c3f074a0c4643c29
Author: Andreas Streichardt <andreas@arangodb.com>
Date:   Thu Jun 1 16:55:41 2017 +0200

    Fix shardmapping bug

commit 1ff7998ebfd691598ec5b455ca5bc2bfd7020fb4
Author: Mop <andreas@arangodb.com>
Date:   Wed May 31 17:26:08 2017 +0200

    more output

commit 68e88aa0e14316c4929d05b2c151bee6421d754d
Merge: 0978ad1d9e 44a6a78ec3
Author: Mop <andreas@arangodb.com>
Date:   Wed May 31 17:03:33 2017 +0200

    Merge branch 'shardorganizer' of https://github.com/arangodb/arangodb into shardorganizer

commit 44a6a78ec338a1e7cabb15464500d96b84c68f1d
Author: Andreas Streichardt <andreas@arangodb.com>
Date:   Wed May 31 07:42:43 2017 -0700

    Fix namespace

commit 0978ad1d9e2f01b86204990e74b66958f25eba66
Merge: f98582ccff d74e5989ad
Author: Mop <andreas@arangodb.com>
Date:   Wed May 31 16:40:35 2017 +0200

    Merge branch 'shardorganizer' of https://github.com/arangodb/arangodb into shardorganizer

commit f98582ccff3448f6c2388dab4cc2dc38034271b0
Author: Mop <andreas@arangodb.com>
Date:   Wed May 31 16:39:03 2017 +0200

    Revert "Revert "Next attempt at merging ShardOrganizer...distributeShardsLike fixed""

    This reverts commit fed45b7b10.

commit d74e5989ad478efe7d66d196715c05f4f41c9c29
Author: Andreas Streichardt <andreas@arangodb.com>
Date:   Wed May 31 16:31:31 2017 +0200

    Make it an error

commit 0a6a9ef9464df4f24ad205bbab5b9f8ded50054f
Author: Andreas Streichardt <andreas@arangodb.com>
Date:   Wed May 31 12:42:51 2017 +0200

    distributeShardsLike has to be saved as a cidString
2017-06-01 17:32:40 +02:00
Michael Hackstein 52406567cd Replaced EdgeCursors callback to return cid,rev instead of _id. Does less costly copying and comparision internally 2017-05-31 17:49:46 +02:00
Mop fed45b7b10 Revert "Next attempt at merging ShardOrganizer...distributeShardsLike fixed"
This reverts commit 16329a3b01.
2017-05-31 14:46:22 +02:00
Andreas Streichardt 16329a3b01 Next attempt at merging ShardOrganizer...distributeShardsLike fixed 2017-05-31 13:01:30 +02:00
Frank Celler 73a4203413 Revert "Implement shardorganizer...Cluster with 1 DBServer possible again"
This reverts commit 1b2d6dca83.
2017-05-30 23:40:01 +02:00
Andreas Streichardt dded5abb6c Merge remote-tracking branch 'origin' into shardorganizer 2017-05-30 20:17:07 +02:00
Andreas Streichardt 1b2d6dca83 Implement shardorganizer...Cluster with 1 DBServer possible again 2017-05-30 20:13:00 +02:00
jsteemann 53b150b206 added tests for intermediate commits 2017-05-30 12:03:58 +02:00
jsteemann a35f85bcc7 added proper docs for --query.fail-on-warning 2017-05-29 15:46:48 +02:00
Andreas Streichardt f2670f8040 Extract compareServerList and make it reuseable 2017-05-24 14:13:51 +02:00
jsteemann a4fde59fd2 some refactoring 2017-05-23 13:18:51 +02:00
Andreas Streichardt 4c2ce35538 Do not generate product version when skipping packages 2017-05-16 02:47:28 -07:00
Simon Grätzer 49cc12e6a9 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2017-05-13 01:08:28 +02:00