1
0
Fork 0
Commit Graph

428 Commits

Author SHA1 Message Date
jsteemann d1b4cfc9ef Merge branch 'devel' of github.com:arangodb/arangodb into devel 2019-10-29 10:44:41 +01:00
jsteemann 203333bfe4 more debug infos 2019-10-29 10:43:48 +01:00
Andrey Abramov 7541b4d5e6
Bug fix/internal issue #647 (#10292)
* extend replication tests

* ensure proper replication order

* fix tests

* address review comments

* address test failures

* extend dump tests

* fix analyzers tests

* more fixes

* extend tests

* enhance tests

* adjust tests

* use enum instead of flags (part 1)

* cleanup

* use enum instead of flags (part 2)

* get rid of flags for views

* get rid of flags for collections

* completely get rid of collection flags

* fix replication test

* refactor index flags

* fix tests and move AnalyzerPool out of class scope

* fix tests

* adjust log levels

* add tests

* remove debug logging

* remove noexcept from `equalAanalyzer`

* extend cluster tests

* fix cluster tests

* add tests for views and smart graphs

* address jslint errors
2019-10-29 12:09:04 +03:00
Jan 513151c010
try to make coordinator more resilient when database/collection does not (#10328) 2019-10-28 15:44:13 +01:00
Jan ae818e07d5
Feature/add cluster force one shard option (#10300) 2019-10-28 13:23:33 +01:00
Lars Maier b53ba3b907 Feature/force backup (#10265)
* abort write transactions
* Code layout for force backup.
* improve killing of queries
* added query killing test
* reactivate all other tests again
* added tests for killing queries
* Lock dbserver async when forced.
* make it even more killable
* added query status "killed"
* Fixed logids.
2019-10-25 16:39:55 +02:00
Jan 359ee03dd8
upgrade vpack library (#10314) 2019-10-25 11:04:16 +02:00
Jan 09134ee8d4
micro optimizations for case conversion (#10291) 2019-10-22 09:39:16 +02:00
Simon d526805e81 Bug fix/fix suspicous stuff (#10273) 2019-10-17 15:34:22 +02:00
Jan Christoph Uhde b16a5e6393 C++17 (#10178)
* Squashed commit of the following:

commit dd4b85fcaa634ed829e79e266ee547c0522e71f6
Merge: 9fc1548e54 fa7de56cf8
Author: Jan Christoph Uhde <Jan@UhdeJC.com>
Date:   Thu Jan 10 10:21:26 2019 +0100

    Merge branch 'devel' into feature/cpp17

    * devel:
      upgrade to boost 1.69.0 (#7910)
      Doc - Add a paragraph that JS JS trxs are excluded from intermediate commits. (#7919)
      issue 511.7.3: restore single-document optimization, implement call to update data-store options, add IResearchView upgrade step (#7918)
      improve assertions (#7830)
      fix MSVC warning
      remove unused function
      fix #7900 - null bind values do not change to empty string anymore (#7917)

commit 9fc1548e54f8cfd2990e7ae0b2b7ae3da55a2cd2
Merge: f2ecee8136 e7810e1956
Author: Jan Christoph Uhde <Jan@UhdeJC.com>
Date:   Wed Jan 9 09:36:19 2019 +0100

    Merge remote-tracking branch 'origin/devel' into feature/cpp17

    * origin/devel:
      Doc - Update remark about per-database replication (#7891)
      Doc - Known Issues - new links (#7908)
      support db._explain with all plans (#7895)
      JWT base64url encoding (#7899)
      cppcheck/Scheduler (#7909)
      multiplex REPLICATION-APPLIER-STATE files for RocksDB engine (#7898)
      fix compile warnings
      upgrade bundled curl library to version 7.63 (#7846)
      Feature/new server infra (#7733)
      Doc - check_function (#7885)
      simran (#7881)
      added AQL function CHECK_DOCUMENT (#7842)
      added arangorestore option `--cleanup-duplicate-attributes` (#7877)
      Doc - arangorestore improvements (#7878)
      added arangorestore options `--number-of-shards` and `--replication-f… (#7869)

commit f2ecee81360f0f0c19901accdbc9902304814bb2
Merge: 44adf97fa0 5c53ebceb0
Author: Jan Christoph Uhde <Jan@UhdeJC.com>
Date:   Thu Jan 3 10:51:20 2019 +0100

    Merge remote-tracking branch 'origin/devel' into feature/cpp17

    * origin/devel: (40 commits)
      Feature/jwt keyfile (#7863)
      prevent duplicate attributes being generated by AQL queries (#7837)
      fixed issue #7834 (#7843)
      Bug fix/issues 23122018 (#7847)
      Documents that arangorestore/arangodump threads option is from v3.4.0 on (#7701)
      Update ISSUE_TEMPLATE.md (#7103)
      Doc - Forwardports to devel - 2018-12-27 (#7858)
      Doc - Sync-external-repo-2018-12-25 (#7854)
      Doc - re-add switching storage engine section (#7781)
      Doc - Fix build (whitespaces, triple-backticks, comments) (#7857)
      Add http_server test for creating view with links (#7852)
      big reformat
      added new clang style and reformat script for clang-format 6.0
      Doc - Improve MacOS Installation page (#7744)
      Fix issue with geo iterator reset. (#7839)
      Doc - Single Instance vs. Cluster (diff + migration) (#7739)
      Rdb index background (preliminary) (#7644)
      iResearch asan issue (#7832)
      Feature/improve edgeindex covered (#7718)
      issue 511.6: update iresearch to 6e870904eae5eaf891359ed06e7332666c02cb19 (#7819)
      ...

commit 44adf97fa0bc9cc7dc649c98916a91dba456b956
Author: Jan Christoph Uhde <Jan@UhdeJC.com>
Date:   Tue Dec 18 13:39:03 2018 +0100

    Add COMPILER_* variables to VERSIONS file.

    The variables will be used to select a fitting compiler in the CI.
    Developers need to ensure themselves that their development environment
    supports the c++ standard.

commit 8124eb3b44fb2302cdd7250d1573b25135719e41
Merge: 127bb7d1cb 4423125a7f
Author: Jan Christoph Uhde <Jan@UhdeJC.com>
Date:   Tue Dec 18 11:09:42 2018 +0100

    Merge branch 'devel' of https://github.com/arangodb/arangodb into feature/cpp17

    * 'devel' of https://github.com/arangodb/arangodb: (381 commits)
      tools are now looking into the build to find the file (#7128)
      issue 511.3: remove IResearchViewDBServer and use the IResearchView directly on db-server (#7748)
      Fix S2 for aarch64. (#7770)
      equalising devel and 3.4 in agency/cluster (#7755)
      wording
      startLocalCluster in rr debugging mode cannot be started with --console on agents / db servers (#7776)
      fix cmake parameters coincidence (#7774)
      Fix a rare deadlock situation in Replication sync phase (#7759)
      add virtual dtor
      wait for procdump to exit too. (#7731)
      fix invalid handling of `_lastValue` in case of multiple coordinators (#7734)
      fixed item 3 of issue #7009 (#7747)
      updated 3rdparty licenses
      remove inclusion of valgrind.h
      remove now-unused file
      Bug fix/fix thread shutdown (#7728)
      Doc - fix build by adding missing SUMMARY entries (#7741)
      issue 511.2.1: use references instead of raw pointers for Index operations to avoid null pointer access (#7725)
      port 3.4 change to stopBackgroundThreads.  Move ClusterCommThread routines to same place in source file. (#7721)
      Ignore invaild geo coordinates when indexing. (#7724)
      ...

commit 127bb7d1cb56fb71b06613971c896aedc1817a98
Author: Jan Christoph Uhde <Jan@UhdeJC.com>
Date:   Mon Oct 15 22:06:27 2018 +0200

    fix invoke for c++17

commit 65a514c34472a611167cd01e14efd5dccc9ae2d1
Author: Frank Celler <frank@arangodb.com>
Date:   Mon Oct 15 18:58:31 2018 +0200

    trying to use 17

commit 8344d96e43522757813bb37e10b95ae83ff35abf
Merge: 2822dae6e1 c7933bcb31
Author: Frank Celler <frank@arangodb.com>
Date:   Mon Oct 15 18:08:52 2018 +0200

    Merge branch 'devel' of ssh://github.com/arangodb/ArangoDB into feature/cpp17

commit 2822dae6e175e866106c49ed8e1f5248dcef0b62
Merge: 4001d11afd af2f18fcbf
Author: Frank Celler <frank@arangodb.com>
Date:   Fri Oct 12 17:44:04 2018 +0200

    Merge branch 'feature/boost-168' of ssh://github.com/arangodb/ArangoDB into feature/cpp17

commit 4001d11afd6ed691ce9dcce34b1ee7b75a08be4b
Author: Frank Celler <frank@arangodb.com>
Date:   Fri Oct 12 17:43:46 2018 +0200

    fixed merge conflict

commit 94cb4b7d4c43c7c12db7ccc3c7cb1c1885e9fafa
Merge: cbc2f34bb4 d3c446156c
Author: Frank Celler <frank@arangodb.com>
Date:   Fri Oct 12 14:35:42 2018 +0200

    Merge branch 'devel' of https://github.com/arangodb/arangodb into feature/cpp17

commit af2f18fcbf709a86e76feb5165ea848f3d4eba67
Author: Jan Christoph Uhde <Jan@UhdeJC.com>
Date:   Fri Sep 28 14:44:16 2018 +0200

    add bcrypt.lib to make uuid work

commit 11efd1299a40a38ef92d717a8ee9e882fc77c82b
Author: Jan Christoph Uhde <Jan@UhdeJC.com>
Date:   Wed Sep 19 12:31:25 2018 +0200

    fix boost path for iresearch that is not able to use the provided
    information

commit 57ed116b6c7c5fda1180929c4f1d63895c25ea1f
Author: Jan Christoph Uhde <Jan@UhdeJC.com>
Date:   Wed Sep 19 10:24:10 2018 +0200

    add missing boost core includes

commit 5d52b322a0fa8fccb4c0d50f5677f7c4b0505284
Author: Jan Christoph Uhde <Jan@UhdeJC.com>
Date:   Wed Sep 19 09:22:43 2018 +0200

    upgrade boost to version 1.68.0

commit cbc2f34bb475f988fc407b2fd332d9984fc682db
Merge: 3de8d481e6 1b1142647a
Author: Jan Christoph Uhde <Jan@UhdeJC.com>
Date:   Wed Sep 19 12:08:13 2018 +0200

    Merge branch 'feature/cpp17' of https://github.com/arangodb/arangodb into feature/cpp17

    * 'feature/cpp17' of https://github.com/arangodb/arangodb:
      allow MSVC to user autopointer with c++17
      nightly frontend build
      Remove unused variable check from arangodump. (#6529)
      fixed JavaScript error
      added a reserve call
      Bug fix/simplify things (#6516)
      catch 501 (not implemented) in the query editor (ui) (#6523)
      return error codes in case a babies remove operation fails (#6527)
      Use physical collection directly (#6526)
      Feature/aql item block compression (#6514)
      move tests to the right location
      take over selectivity estimates (#6505)
      Properly check syncer erros, catch more exceptions (#6520)
      prevent out-of-bounds array access
      Fix deduplication in IndexBlock (#6508)

commit 3de8d481e6c40ab1b160ca931b23882fe36048c2
Author: Jan Christoph Uhde <Jan@UhdeJC.com>
Date:   Wed Sep 19 12:05:56 2018 +0200

    add autopointer and tr1

commit 1b1142647ac79764c55c0688c1b22a0227c7fad6
Merge: 4b38f0a7a0 0dd277d325
Author: Jan Christoph Uhde <Jan@UhdeJC.com>
Date:   Wed Sep 19 10:06:01 2018 +0200

    Merge branch 'devel' of https://github.com/arangodb/arangodb into feature/cpp17

    * 'devel' of https://github.com/arangodb/arangodb:
      nightly frontend build
      Remove unused variable check from arangodump. (#6529)
      fixed JavaScript error
      added a reserve call
      Bug fix/simplify things (#6516)
      catch 501 (not implemented) in the query editor (ui) (#6523)
      return error codes in case a babies remove operation fails (#6527)
      Use physical collection directly (#6526)
      Feature/aql item block compression (#6514)
      move tests to the right location
      take over selectivity estimates (#6505)
      Properly check syncer erros, catch more exceptions (#6520)
      prevent out-of-bounds array access
      Fix deduplication in IndexBlock (#6508)

commit 4b38f0a7a003a2da813a363b91dc289419e0210c
Author: Jan Christoph Uhde <Jan@UhdeJC.com>
Date:   Wed Sep 19 10:04:46 2018 +0200

    allow MSVC to user autopointer with c++17

commit 9c9102341245f2bc4373d11cd69c8e4c3d9fb129
Merge: 1b9e580f14 e51359b68e
Author: Jan Christoph Uhde <Jan@UhdeJC.com>
Date:   Mon Sep 17 12:53:03 2018 +0200

    Merge branch 'devel' of https://github.com/arangodb/arangodb into feature/cpp17

    * 'devel' of https://github.com/arangodb/arangodb:
      port startup fix
      Bugfix/early out invalid links in view creation (#6502)

commit 1b9e580f14db737f005b7eed9926c222fa047754
Merge: b08bc2e9ba 8bd834bcf7
Author: Jan Christoph Uhde <Jan@UhdeJC.com>
Date:   Mon Sep 17 06:38:24 2018 +0200

    Merge remote-tracking branch 'origin/devel' into feature/cpp17

    * origin/devel: (38 commits)
      Maintenance delayed by incomplete hashing maintenance actions (#6448)
      Do not hide indexes (#6507)
      Fix WebUI Login with read-only (#6500)
      Improve use of single shard restriction (#6182)
      make cid values in /_api/replication/logger-follow strings as advertised (#6499)
      less dynamic memory allocations on index operations (#6497)
      Fix several TSan warnings (#6473)
      Copy installation files on startup (#6491)
      nightly frontend build
      fix cluster selectivity estimates (#6488)
      this moves aql::Query* out of the call parameter to any aql::Function (#6485)
      Add advertised endpoints. (#6104)
      nightly frontend build
      fix leak in regex cache (#6480)
      Removing ClusterComm ClientTransactionID (#6294)
      pass variables by const reference
      remove unused variable
      remove unused variable
      fix init-order fiasco with static strings (#6475)
      minor issues (#6471)
      ...

commit b08bc2e9ba1553d608ae1bd5dc4a42290fa2cabd
Merge: 03fabcc509 2eb3c2dee4
Author: Jan Christoph Uhde <Jan@UhdeJC.com>
Date:   Tue Sep 11 12:46:51 2018 +0200

    Merge branch 'devel' of https://github.com/arangodb/arangodb into feature/cpp17

    * 'devel' of https://github.com/arangodb/arangodb:
      temporarily increase log verbosity for replication log topic
      Fix skipSome PR (#6455)
      bump version number to 3.5 (#6454)
      Feature/new devel versioning scheme (#6312)
      Bug fix/aql modify cleanup (#6380)
      Trace skipSome() calls (#6449)
      nightly frontend build
      Fix ArangoSearch authentication update task test (#6453)
      Forward port changelog changes and typo fixes from #6303 (#6409)
      fix some issues with sorted variant of COLLECT (#6433)
      added tests, updated docu (#6407)
      now that we are on c++14 and std::make_unique is available everywhere, we can safely remove our shim for std::make_unique (#6429)
      Fixed memory leak due to cyclic references of shared pointer. (#6442)
      do not create empty useless transaction data objects (#6430)
      add some debugging for unstable replication_sync test (#6436)
      Address leak sanitizer warnings (#6275)
      Fix Pregel Graph Loading Logic (#6419)
      remove unused variables
      fix profile output for nodes without calls to getSome (#6397)

commit 03fabcc5098f4e097d31546835fbadd373e854c1
Author: Jan Christoph Uhde <Jan@UhdeJC.com>
Date:   Tue Sep 11 07:19:21 2018 +0200

    switch more libs to c++17

commit 335942038f5d0d817554f59a5943e041d928ec40
Author: Jan Christoph Uhde <Jan@UhdeJC.com>
Date:   Mon Sep 10 07:31:49 2018 +0200

    fix static asserts in iresearch

commit 90eab952765c26d165e57dc8e01ee42bc1d2d8e6
Author: Jan Christoph Uhde <Jan@UhdeJC.com>
Date:   Mon Sep 10 07:10:48 2018 +0200

    move from c++14 to c++17

* specify required clang and gcc versions

* fix openssl 1.3 usage

* change random_shuffle to shuffle as the former has been removed from the c++ standard

https://meetingcpp.com/blog/items/stdrandom_shuffle-is-deprecated.html

* disable uncaught_exceptions in catch2 for darwin

In file included from /Users/jenkins/Git/cpp17/tests/main.cpp:2:
/Users/jenkins/Git/cpp17/3rdParty/catch/catch.hpp:10877:21: error: 'uncaught_exceptions' is unavailable: introduced in macOS 10.12
        return std::uncaught_exceptions() > 0;
                    ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/exception:130:63: note: 'uncaught_exceptions' has been explicitly marked unavailable here
_LIBCPP_FUNC_VIS _LIBCPP_AVAILABILITY_UNCAUGHT_EXCEPTIONS int uncaught_exceptions() _NOEXCEPT;
                                                              ^

* boost locale now depends on boost thread

* add fix for VS2019

* try to fix build for windows

* add some change required by older macos

* add miserable hack to circumvent std::stack implementation problem on older Xcode

* try to fix windows build

* remove unary function as in v8 upstream

* clean up CMakeLists.txt

* Remove emplace hack that was introduced for OSX

* Revert "Remove emplace hack that was introduced for OSX"

This reverts commit 2b0a4cdd7881baa9ea8fbd34f2e89ef6fa9b9b63.

* Remove emplace hack that was introduced on OSX to support C++17 (#10184)

* Revert "Revert "Remove emplace hack that was introduced for OSX""

This reverts commit 7df9edde4097a951e95fcb204113d9c5289bf4bf.

* fixed MACOSX_DEPLOYMENT_TARGET

* Fix quotes usage in conditions

* Added MACOS_MIN

* Adjust variables

* Fix compilation

* remove iresearch patches

* Update CMakeLists.txt

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>
2019-10-16 20:28:27 +03:00
Simon 4d8c909666 A mixed bag of goods (#10220) 2019-10-14 17:02:36 +02:00
Max Neunhöffer 4dc2f7b059 Fix dbserver locking and releasing for hotbackup. (#10177) 2019-10-14 10:16:44 +02:00
Max Neunhöffer 82ca8e0b52
Improve timings for hotbackup locking. (#10229) 2019-10-11 15:13:19 +02:00
Dan Larkin-York 13e24b2db9 Convert many uses of ClusterComm to Fuerte (#10154) 2019-10-10 14:03:33 +02:00
Simon 8be084dc71 Feature/fuerte cluster 9000 (#10137) 2019-10-08 21:06:38 +02:00
Lars Maier f6c163ae1a Added available field to indicate bad backups. (#10129)
* Added available field to indicate bad backups.
* Added nrPiecesPresent.
* Fixed logids...
* Make Windows compiler happy.
* Fix log ids.
2019-10-08 15:35:38 +02:00
Jan 21b0311d57
rename minReplicationFactor to writeConcern (#10118) 2019-10-07 15:12:15 +02:00
Jan 9898606223 fix logIds (#10135) 2019-10-01 20:19:38 +03:00
Kaveh Vahedipour a0be102d2c unitended multiple unlocks (#10114) 2019-10-01 16:21:44 +02:00
Max Neunhöffer ca9affc798
Add backup sizes. (#10104)
* Single server backup size in META and create.
* More size info in backup.
* Finish size and nrfiles for create and list.
* Add nrDBServers and potentiallyInconsistent to BackupMeta.
* Fix API documentation.
* Make Windows compiler happier.
* One more fix for Windows.
* Added creation date.
2019-10-01 13:31:17 +02:00
Dan Larkin-York 1d7225b289 Pass connection pool directly to network methods. (#10096) 2019-09-30 12:44:47 +02:00
Kaveh Vahedipour bbe7d424d5 corrected hot backup lock timings (#10076)
* corrected hot backup lock timings
* the lock timeout added to overall unlock timeout]
2019-09-30 10:22:24 +02:00
Simon cb7bf0314b Use fuerte in RemoteExecutor (#10077) 2019-09-27 16:20:38 +02:00
Dan Larkin-York dc23896a01 Make count, figures, revision, and index warmup use non-blocking communication (#10048) 2019-09-27 09:54:01 +02:00
Dan Larkin-York a83c2323c9 Refactor ApplicationServer stack (#9965) 2019-09-25 17:31:59 +02:00
Simon ac2158ee22 Async el cheapo (#10061) 2019-09-24 12:00:13 +02:00
Simon 2d1c76a55a Refactor Batch API docs (#10015) 2019-09-23 19:56:04 +02:00
Kaveh Vahedipour dd10909dfc rebootIds instead of boot stamps (#10050)
* rebootIds instead of boot stamps
* noexcept wrong as copies are done
2019-09-20 10:26:35 +02:00
Kaveh Vahedipour 49a01e14ff Bugfix/agency lock left behind (#10021)
* fix potentially left behind agency lock
* typo
* do not silently patronise the customer
2019-09-20 10:09:21 +02:00
Tobias Gödderz 7d091523e5 Fixed some problems found with IPO enabled (#10020) 2019-09-16 17:10:13 +02:00
Dan Larkin-York 8c573549b3 Make truncate use non-blocking communication. (#9980) 2019-09-16 10:46:49 +02:00
Simon 2e91f4fe67 Non block delete (#10005) 2019-09-12 21:44:35 +02:00
Simon 3d2952b23a Non block modify (#9963) 2019-09-11 15:37:02 +02:00
Jan aada04e75b
don't assert/crash when using an unknown collection/shard (#9959) 2019-09-11 12:03:13 +02:00
Kaveh Vahedipour 3011846025 broken hotbackup list with (#9956)
* fix broken list of non existing id
2019-09-10 10:17:24 +02:00
Simon 6b7fb0994e Non-Blocking reads (#9883) 2019-09-05 19:26:01 +02:00
Kaveh Vahedipour ad36adc354 Feature/hotbackup list retries (#9924)
* retry hot backup listing for 2 minutes in cluster before giving up
2019-09-05 16:44:43 +02:00
Markus Pfeiffer 753ff4aa67 Feature/atomic database creation 2 (#9826) 2019-09-05 12:38:07 +02:00
Tobias Gödderz b7f8f01a22 Feature/split libarangoserver (#9670)
* Made usage of named variables for targets a little more consistent

* Bumped minimum cmake version

- removed policies superseded by minimum cmake version
- fixed static linking due to new policy CMP0060

* Fixed library suffixes for windows

* Extracted libs arango_mmfiles and _rocksdb from arangoserver

* Replaced target variables by constants

* Extracted arango_cluster_engine from arangoserver

* Extracted llhttp from arangoserver

* First successful split of arangoserver

* Moved enterprise files to enterprise

* Again only optionally include RestTestHandler and AcceptorUnixDomain

* Cleaned source files from other libraries

* Removed old commented sources

* Split off a small third library

* Fixed boost dependency for cluster engine

* Added some missing dependencies

* Added arango_geo dep, and -J on windows

* Began to split off an arango_graph lib

* Moved more files to arango_graph

* Do not set /J globally for ATL

* Moved more files to arango_graph

* Moved graph-RestHandlers to arango_graph

* Added arango_geo dependency to _mmfiles and _rocksdb

* Updated graph dependencies

* Split off arango_pregel

* Split off arango_aql

* Added missing boost_system dependency to pregel

* Split off arango_vocbase

* Cleanup

* Added missing boost_system dependency for arango_vocbase

* Split off arango_v8server

* Split off arango_utils

* Minor cleanup

* Split of arango_storage_engine

* Split off arango_indexes and arango_cache

* Fixed some dependencies

* Split off arango_replication

* Resolved two todos

* Split off arango_agency

* Reordered some statements

* Ordered dependency definitions alphabetically

* Cleaned some deps

* Break one cycle, comment on another

* Merge the remaining arangoserver_part[123] sources

* Moved some utils to vocbase to break cycles

* Added missing backtrace dependency to iresearch-s

* Added missing boost dependency

* Added dependency arango_indexes -> arango_geo

* Added deps to arango_cluster_engine, cleaned duplicate deps

* Broke remaining dependency cycles

* Actually, missed one cycle...

* Re-added include for Mac

* arango_cache needs SharedPRNG
2019-09-05 09:37:12 +02:00
Kaveh Vahedipour cced54cbc7 forceBackup is now allowInconsistent (#9850)
* forceBackup is now allowInconsistent
2019-09-02 10:32:58 +02:00
Simon 2f9d1f8c51 Baby operations in cluster fix (#9870) 2019-09-02 09:36:37 +02:00
Simon 0ee0cebb11 Non-Blocking inserts (#9823) 2019-08-30 09:17:58 +02:00
Frank Celler 68ea717af4
Feature/set environment (#9688) 2019-08-13 09:18:16 +02:00
Frank Celler cf26b3a39e removed unused variable 2019-08-09 10:01:14 +02:00
Dan Larkin-York 3d0246cb18 Decentralize includes (#9623) 2019-08-06 15:32:09 +02:00
Lars Maier ed496fe5dd Feature/hotbackup devel (#9495)
Hotbackup
2019-08-02 11:39:46 +02:00
Michael Hackstein 987ad41364
Forward Port of changes in 3.5 review (#9544)
* Bug fix 3.5/min replication factor (#9524)

* Cherry-pick minReplicationFactor

* Bug fix/failover with min replication factor (#9486)

* Improve collection time of IResearchQueryOptimizationTest

* Added a minReplicationFactor field in Collections. It is not possible to modify it yet and noone cares for it

* Added some assertion son minReplicationFactor

* Transaction API will now reject writes as soon as minimal replication factor is NOT fulfilled

* added minReplicationFactor to the user interface, preparation for the collection api changes

* added minReplicationFactor to VocBaseCollection, RestReplicationHandler, RestCollectionHandler, ClusterMethods, ClusterInfo and ClusterCollectionCreationInfo

* added minReplicationFactor usage to tests

* TODO TEMOPORARY COMMIT FOR TESTING PLEASE REVERT ME

* minReplicationFactor now able to change via collection  properties route

* fixed wrongly assert

* added minReplicationFactor to the graph management ui

* added minReplicationFactor to the gharial api

* Fixed off-by-one error in minReplicationFactor. We actually enforced one more.

* adjusted description of minReplicationFactor

* FollowerInfo Refactoring

* added gharial api graph creation tests with minimal replication factor

* proper cleanup of shell collection tests, removed lots of duplicate code, preparation for some new tests

* added collection create tests using invalid/valid names, replicationFactor and minReplicationFactor

* Debug logging

* MORE Debug logging

* Included replication fast lane

* Use correct minreplicationfactor

* modified debug logging

* Fixed compileissues

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* Revert "MORE Debug logging"

This reverts commit dab5af28c0.

* Revert "MORE Debug logging"

This reverts commit 6134b664bd.

* Revert "MORE Debug logging"

This reverts commit 80160bdf3b.

* Revert "MORE Debug logging"

This reverts commit 06aabcdfe1.

* Removed debug output

* Added replication fast lane. Also refactored the commands as i cannot take it any more...

* Put some requests of RocksDBReplication onto CATCHUP Lane.

* Put some requests of MMFilesReplication onto CATCHUP Lane.

* Adjusted Fast and MED lane usage in Supervised scheduler

* Added changelog entry

* Added new features entry

* A new leader will now keep old followers in case of failover

* Update arangod/Cluster/ClusterCollectionCreationInfo.cpp

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Fixed JSLINT

* Unified lane handling of replication handlers

* Sorry forgotten in last commit

* replaced strings with static strings

* more use of static strings

* optimized min repl description in the ui

* decr initial loop variable

* clean up of the createWithId test

* more use of static strings

* Update js/apps/system/_admin/aardvark/APP/frontend/js/views/collectionsView.js

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Added some comments on condition, renamed variable as suggested in review

* Added check for min replicationFactor to be non-zero

* Added assertion

* Added function to modify min and max replication factor in one go

* added missing semicolon

* rm log devel

* Added a second information to follower info that can keep track of followers that have been in sync before a failover has taken place

* Maintenance reports previous version now to follower info. instead of lying by itself. The Follower Info now gets a failover save mode to report insync followers

* check replFactor against nr dbservers

* Add lie reporting in CURRENT

* Reverted most of my recent commits about Failover situation. The intended plan simply does not work out

* move replication checks from logical collection to rest collection handler

* added more replication tests

* Include assert only if we are not in gtest

* jslint

* set min repl factor to zero if satellite collection

* check replication attributes in v8 collection

* Initial commit, old plan, does not yet work

* fixed ires tests

* Included FailoverCandidates key. Not fully implemented

* fixed wrong assert

* unified in sync follower reporting

* fixed compiler errors

* Cleanup locking, and fixed potential deadlocks

* Comments about locking order in FollowerInfo.

* properly check uint

* Keep old leader as potential failover candidate

* Transaction methods now use followerInfo to check if the leader can write, this might have the sideeffect that 'failoverCandidates' are updated

* Let agency check failoverCandidates if possible

* Initialize member variables

* Use unified follower reporting in DBServerAgencySync

* Removed obsolete variable, collecting it somewhere else

* repl factor attr check

* Reimplemented previous followers, second attempt now. PhaseOne and PhaseTwo can now synchronize on current.

* Fixed assertion, forgot an off-by-one

* adjusted test to be more preciese now

* Fixed failove candidates list

* Disable write on dropping too many followers

* Allow to run updateFailoerCandidates multiple times with same leader.

* Final fixes, resilience tests now green, crossing fingers for jenkins

* Fixed race on atomics comparison

* Fixed invalid number type

* added nullptr handling

* added nullptr handling

* Removed invalid assert

* Make takeover of leadership an atomic operation

* Update tests/js/common/shell/shell-cluster-collection.js

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Review fixes

* Fixed creation code to use takeoverLeadership

* Update arangod/Cluster/FollowerInfo.h

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Applied review fixes

* There is no timeout

* Moved AQL + Pregel to INTERNAL_AQL lane, which is medium priority, to avoid deadlocks with Sync replication

* More review fixes

* Use difference if you want to compare two vectors...

* Use std::string ...

* Now check if we are in recovery mode

* Added documentation for minReplicationFactor

* Added readme update as well in documenation

* Removed merge conflict leftovers 0o, i should not trust the IDE

* Update js/apps/system/_admin/aardvark/APP/frontend/js/views/collectionsView.js

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update js/apps/system/_admin/aardvark/APP/frontend/js/views/collectionsView.js

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update Documentation/Books/Manual/Architecture/Replication/README.md

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update CHANGELOG

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update Documentation/Books/Manual/DataModeling/Collections/DatabaseMethods.md

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update Documentation/Books/Manual/ReleaseNotes/NewFeatures35.md

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update Documentation/DocuBlocks/Rest/Collections/1_structs.md

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update js/apps/system/_admin/aardvark/APP/frontend/js/views/graphManagementView.js

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update js/apps/system/_admin/aardvark/APP/frontend/js/views/graphManagementView.js

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update Documentation/DocuBlocks/Rest/Graph/1_structs.md

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Apply suggestions from code review

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Adepted review requests, thanks for finding!

* Removed unnecessary const

* Apply suggestions from code review

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Moved initilization of variable more downwards

* Apply lock before notify_all()

* Remove documentation except DocuBlocks, covered by PR in docs repo

* Remove accidental indent

* Removed leftover merge conflict in documentation block
2019-07-23 13:14:38 +02:00
Jan cdbe63fa6e
Bug fix/fix races in collection creation (#9506) 2019-07-19 15:11:08 +02:00
Michael Hackstein cbcf561450
Feature/min replication factor (#9433)
* Added a minReplicationFactor field in Collections. It is not possible to modify it yet and noone cares for it

* Added some assertion son minReplicationFactor

* Transaction API will now reject writes as soon as minimal replication factor is NOT fulfilled

* added minReplicationFactor to the user interface, preparation for the collection api changes

* added minReplicationFactor to VocBaseCollection, RestReplicationHandler, RestCollectionHandler, ClusterMethods, ClusterInfo and ClusterCollectionCreationInfo

* added minReplicationFactor usage to tests

* TODO TEMOPORARY COMMIT FOR TESTING PLEASE REVERT ME

* minReplicationFactor now able to change via collection  properties route

* fixed wrongly assert

* added minReplicationFactor to the graph management ui

* added minReplicationFactor to the gharial api

* Fixed off-by-one error in minReplicationFactor. We actually enforced one more.

* adjusted description of minReplicationFactor

* FollowerInfo Refactoring

* added gharial api graph creation tests with minimal replication factor

* proper cleanup of shell collection tests, removed lots of duplicate code, preparation for some new tests

* added collection create tests using invalid/valid names, replicationFactor and minReplicationFactor

* Debug logging

* MORE Debug logging

* Included replication fast lane

* Use correct minreplicationfactor

* modified debug logging

* Fixed compileissues

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* Revert "MORE Debug logging"

This reverts commit dab5af28c0.

* Revert "MORE Debug logging"

This reverts commit 6134b664bd.

* Revert "MORE Debug logging"

This reverts commit 80160bdf3b.

* Revert "MORE Debug logging"

This reverts commit 06aabcdfe1.

* Removed debug output

* Added replication fast lane. Also refactored the commands as i cannot take it any more...

* Put some requests of RocksDBReplication onto CATCHUP Lane.

* Put some requests of MMFilesReplication onto CATCHUP Lane.

* Adjusted Fast and MED lane usage in Supervised scheduler

* Added changelog entry

* Added new features entry

* A new leader will now keep old followers in case of failover

* Update arangod/Cluster/ClusterCollectionCreationInfo.cpp

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Fixed JSLINT

* Unified lane handling of replication handlers

* Sorry forgotten in last commit

* replaced strings with static strings

* more use of static strings

* optimized min repl description in the ui

* decr initial loop variable

* clean up of the createWithId test

* more use of static strings

* Update js/apps/system/_admin/aardvark/APP/frontend/js/views/collectionsView.js

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Added some comments on condition, renamed variable as suggested in review

* Added check for min replicationFactor to be non-zero

* Added assertion

* Added function to modify min and max replication factor in one go

* added missing semicolon

* rm log devel

* Added a second information to follower info that can keep track of followers that have been in sync before a failover has taken place

* Maintenance reports previous version now to follower info. instead of lying by itself. The Follower Info now gets a failover save mode to report insync followers

* check replFactor against nr dbservers

* Add lie reporting in CURRENT

* Reverted most of my recent commits about Failover situation. The intended plan simply does not work out

* move replication checks from logical collection to rest collection handler

* added more replication tests

* Include assert only if we are not in gtest

* jslint

* set min repl factor to zero if satellite collection

* check replication attributes in v8 collection

* fixed ires tests

* fixed wrong assert

* properly check uint

* repl factor attr check

* adjusted test to be more preciese now

* Fixed race on atomics comparison

* Fixed invalid number type

* Update tests/js/common/shell/shell-cluster-collection.js

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Review fixes

* More review fixes
2019-07-19 13:02:28 +02:00
Jan 300b8e58f4
Bug fix/fix duplicate actions (#9452) 2019-07-19 09:17:19 +02:00