1
0
Fork 0
Commit Graph

1461 Commits

Author SHA1 Message Date
Max Neunhöffer 328f46e3d6 This merges hotbackup and atomic-db-creation into 3.5. (#9968)
* Squashed commit of feature-3.5/hotbackup_devel.

This puts hotbackup into 3.5.

* Port atomic-database-creation-2 to 3.5.

* Remove some wrongly ported code.

* Fix compilation.

* Fix a manual merge error.

* Remove a feature from the mocks which does not exist in 3.5.

* Add some code which was forgotten in manual merge.

* Fix a problem introduced in a manual merge.

* reuse function

* Address some whitespace issues that came up in review

* aardvark should not create the frontend collection

* create _frontend collection from c++

* recheckAndUpdate Callback in CollectionWatcher

* Wrong author ;)

* rm outdated todo

* Update lib/Basics/VelocyPackHelper.h

Co-Authored-By: Michael Hackstein <michael@arangodb.com>

* use logger unique id, use startup logger

* not needed

* optimized vector shardid method

* do not create _modules collection lazy anymre

* Formatting.

* Assert instead of if/TRI_ASSERT(false)

* Don't use exceptions as control structure

* Re-add READ_LOCKER that got lost in translation

* Fix audit log in case database creation fails early.

* legacy sharding

* Add CHANGELOG entry.

* Retry database cancellation indefinitely

* Do not use exceptions in UpgradeTask

* DropCollection is a FAST_LANE action and should not need much time or else retry.

* Remove superflous addition of LdapFeature

Proudly brought to you by ASAN tests

* Fixed check for distributShardsLike sharding on _system database

* Fixed compile issue on tests

* Removed assertion that seems to be not correct yet on devel.

* Sort out google cloud storage as remote. (#9918)

* Add successful method to ClusterCommResult.
* Improve error forwarding for cluster internal communication.

* Feature/hotbackup list retries (#9924)

* retry hot backup listing for 2 minutes in cluster before giving up

* Enable api by default.

* fix broken list of non existing id (#9957)

* Fix compilation after manual merge.

* Fix another compilation problem.

* Yet more fixes for compilation.

* More compilation fixes.
2019-09-11 13:13:54 +03:00
Max Neunhoeffer a52ffb4e34
Add hot_backup to testing black list. 2019-07-15 11:49:52 +02:00
Jan Christoph Uhde 677a79026c Foxx Security (#8845) 2019-04-25 09:56:29 +02:00
KVS85 33caefea6e
Blacklist nonexistent "permissions_server" suite 2019-04-20 18:59:04 +03:00
jsteemann a500175e16 blacklist audit test 2019-04-15 18:13:01 +02:00
jsteemann 6e5ce95dad don't execute permissions tests (yet) 2019-03-27 17:04:22 +01:00
Wilfried Goesgens c5587f1c30 fix enterprise tests - we mustn't restore smart collections over exising collections (#8581)
* fix enterprise tests - we mustn't restore smart collections over existing collections

* un-blacklist test
2019-03-27 09:08:38 +01:00
jsteemann 269537e5f6 blacklist dump_multiple temporarily 2019-03-26 13:00:24 +01:00
KVS85 aa165b7674 Deblacklist ldap tests (#8483) 2019-03-20 17:14:50 +01:00
Wilfried Goesgens 164b074f1c Bug fix/stronger testing certificate (#8284) 2019-02-28 15:36:41 +01:00
KVS85 bbd8aa4e0b Remove procdump usage while taring archives (#7815) 2018-12-21 11:39:39 +01:00
Wilfried Goesgens 7acb3b3ec4 wait for procdump to exit too. (#7731)
* wait for procdump to exit too.

* move running process check up, propagate status. First check whether processes are still running before attempting to force kill them

* only list processes, don't change their state

* add stopping procdump in all places
2018-12-12 18:06:08 +01:00
Wilfried Goesgens 879c7bb479 slash all left over processes (if) and set status to bad. (#7557) 2018-11-30 16:24:07 +01:00
Frank Celler a6e4c844ad Feature-3.4/whiteblacklist (#7259)
* add blacklist

* added --skipGrey
2018-11-08 00:01:29 +01:00
Wilfried Goesgens c2ce154a15 initialize the testresult summary with an abort message, in case we don't finish successfully (#7046) 2018-10-24 16:02:17 +02:00
jsteemann 76504640f4 more failing tests again, back by popular demand! 2018-10-17 20:19:53 +02:00
jsteemann 4184144933 disable shell_client_aql tests until this is remedied 2018-10-17 16:25:43 +02:00
Wilfried Goesgens ad31b31096 print when we mark a build crashy (#6769)
LGTM
2018-10-09 16:47:16 +02:00
Frank Celler a669d5a03d disabled all ldap tests 2018-09-02 11:28:42 +02:00
Frank Celler 0bc8c88c12 disabled ldap tests in oskar for now 2018-09-02 10:29:19 +02:00
Jan 494764b95c
Bug fix/fiddle version string into test paths (#6267) 2018-08-28 14:58:35 +02:00
Vasiliy e04b5318b4 issue 450.5: rename consolidation-related configuration properties and remove locale (#6232) 2018-08-23 16:04:25 +03:00
Jan 102f15bece
removed several unused internal APIs (#6193) 2018-08-20 12:57:58 +02:00
Vasiliy c08ab01f67 issue 450.1: remove the "commit" configuration layer from ArangoSearch properties (#6185)
* issue 450.1: remove the "commit" configuration layer from ArangoSearch properties

* fix broken tests
2018-08-20 02:18:02 +03:00
Tobias Gödderz de4f5587ae Gharial rewrite in C++ (#5631)
* Built a C++ skeleton REST handler for gharial, with fallback to the JS handler

* Moved aql::Graph to graph::Graph

* Added complete edge definitions to Graph

Also:
- some cleanup
- used forward-declarations in headers
- use Graph in graph rest handler

* Handle graph lookup failures according to the test suite

* Added GET vertex

* Bugfixes in ResultT

- Added missing #include
- Fixed move semantics

* Move central code of readVertex to GraphOperations

* ResultT fixes and complements

* Implemented a graph cache

* Added and used graph cache to the rest handler

* Added GET edge

* Added DELETE edge

* Extracted some code

* Added PATCH and PUT for both edge and vertex

* Moved update/replace transaction code to GraphOperations

* Added stub routes for POST and a TODO note

* Added a test checking that deleting a vertex removes all incident edges as well

* Added a test checking that deleting a vertex does not remove edges in non-graph collections

* fixed compiler warnings and errors

* Began work on DELETE vertex

For this, added a V8Context to allow for AQL queries to use subtransactions

* Continued work on DELETE vertex (still WIP)

* prep for graph post routes

* fixed removeVertex operation (aql)

* added post vertex and post edge gharial routes

* wasSynchronoues flag changed

* gharial post c++ handler, naming conventions

* added gharial tests

* temporary disabled cache (because not completed), added graph property read functions

* added c++ gharial list vertex collections

* added c++ gharial graph config

* added c++ gharial list graphs

* added graph manager class

* first implementation of create graph in c++, WIP

* changed error messages

* added etag to create graph api, still multiple edge definition check missing

* finished POST /_api/gharial/<graph>

* WIP - DELETE /_api/gharial/<graphName>

* added DELETE /_api/gharial/<graphName> validation, still missing correct response

* gharial delete

* fixed delete gharial lock

* finished DELETE /_api/gharial/<graphName>

* added routes for graph based vertices and edge definitions

* improved delete route

* added add new edge definition to existing graph

* patch edge definition in a graph, still <WIP>

* finished edit edge definition route

* code changes due to devel code changes

* added remove edge definition route

* added vertex delete function

* added todo note regarding one drop collection issue

* add oprhan collection to graph route implemented

* Added a test

* Updated a comment

* Several minor changes

* Minor changes during review

* Changes during review

* Changes during review

* Bugfix: orphans may be null or omitted

* Bugfix: resolve externals

* minor code changes

* seperated graph class to independent component classes

* seperated graph class to independent component classes

* removed log output

* fixed create collection behaviour in a cluster environment

* fixed enterprise graphs behaviour in c++ gharial api

* removed log output

* formatting

* improved error handling, fixed a linux compile bug

* more result refactoring

* more result type cleanup

* fixed wrongly defined test

* result handling

* error handling

* more refactoring

* Bugfix: avoid race condition in cluster when creating collections

* updated graph documentation

* added graph related static strings

* static strings, new method to create options for gharial created collections

* Some minor cleanup

* more use of static strings

* minor code changes, review

* added missing parseint

* removed gharial foxx, added js common module, added v8 general graph module

* correct use of virtual method

* more v8, js general graph, broken state

* more v8 graph functions

* fixed editEdgeDefinition, added drop function

* fixed drop behaviour

* added _list, _exists

* added c++ rename graph collections, added v8 + graph module function

* Added a regression test

* added graph._deleteEdgeDefinition, v8, server

* more v8g

* added _removeVertexCollection

* added _extendEditDefinitions

* todo, need to add a helper sort method for a local defined relation

* fixed test

* fixed lots of tests, added more client functions, _addVertexCollection on client module is still broken

* added more client graph functions, all tests green

* more client functions

* add del edge def route

* Fix use after move

* Minor changes in client general-graph.js module

* Make a copy before sorting (don't touch the argument)

* Minor changes and some additional asserts in graph tests

* Consistently set parameter defaults

* Renamed static strings

* Remove superfluous function

* Made comment more verbose

* Minor changes in general-graph-common.js

* Added missing template arguments

* Fixed community build

* Cleanup in editEdgeDefinition

* Regression test & bugfix: comparison of edge definitions didn' order from and to

* Fixed errors introduced by merge

* Minor changes in v8-general-graph.cpp

* Fixed test failure due to wrong error code in CE

* added missing id field

* Added permission checks for graph._create

* Removed assertion that is no longer valid

* Moved removeGraph from GraphOperations to GraphManager

* Allow C++ implementation of graph._drop to handle smart graphs

* Flush js client db cache after creating/dropping collections via the general graph module

* Added _deleteEdgeDefinition to the general graph client module

* WIP: Added permission checks for drop graph

* Fixed permission checks for drop graph

* Added permission checks for other graph operations

* Bugfix: assert edge definitions are returned in order

* Some cleanup

* Removed unused method

* Minor improvements in GraphManager

* Fixed a type in general-graph common module

* Most useful fix of all times ever: Do not auto cast from bool to int and alternate error/noerror by this

* Added the initial keyword to StaticStrings

* Added a new error code, used whenever a user tries to inject a documentcollection as a relation into the graph, which is invalid

* Some GraphManager/Ops/Graph cleanup. Less Slice parsing, more usage of GraphObjects

* Test edgeDefinitions in graphs with a defined ordering

* GraphClass Layout cleanup

* Do not test error messages, use codes instead

* Recreated backwards compatibility of Graph Creation Permission errors

* Changed error-code if edgedefinition is used twice

* Added a StaticString for the GraphName

* Renamed graphToVpack => graphForClient

* Partly fixed graph-api test to work with better error messages. Still red: The edgeDefinitions are now sorted, the test is supposed to sort his own list, but appearently does not do so. Under investigation

* Added a new error code that rejects injection of differently sharded smart collection into smartgraph. Should be more helpful to our users

* graph createCollectionOptions now require an open object to be cross-called from enterprise. Made enterprise switch for creation of graph more elegant.

* Updated graphs.cpp

* Massive refactoring. Made Factories for graphs to make SmartGraph much more transparent. Also reduced amount of multiple implementations of the same stuff. Killed vocbase/graphs use GraphManager instead. Removed usage of GraphCache, was not completely implemented anyway and only partially used, which is bad at the moment. Option for later improvement never the less

* Adapted JS code to now really use c++ variants. ALso included 3 Classes: Graph, SmartGraph and GraphModule.

* Fixed undefined behaviour in Remove Vertex. Fixed smartgraph sharding if one collection already exists.

* Removed DEBUG output

* Removed DEBUG logs

* Removed dead code

* Fixed Graph EdgeDefinition test, they now have a different ordering.

* Added a test when adding a vertexCollection that it is actually valid in the graph

* Client Graph API now correctly sends `orphanCollections` and not `orphans`

* Let GraphOperations modify the graph in-place. It should now properly handle edgeDefinitions.

* Added initial cid StaticString

* Included the vocbase in fromPersistence creation of Graphs. Only required to enhance 3.3 SmartGraphs on the fly.

* Fixed internal error message

* Fixed compiler isses originiated from merging

* Removed unused imports

* Regenerated generated file
2018-08-09 09:30:04 +02:00
KVS85 9a82e50ea7 Remove shell_client_aql suite from blacklist (#6115) 2018-08-08 16:24:10 +03:00
Jan b278d6874a
allow master & slave to work in parallel for RocksDB WAL tailing (#6059) 2018-08-03 13:37:53 +02:00
Jan e4d7f1c5f0
Bug fix/wenn der shard mann 2mal klingelt (#5890) 2018-07-26 15:37:40 +02:00
Jan Christoph Uhde 4c0f881764 compatibilty 3.3 <-> 3.4 (#5963) 2018-07-25 09:05:34 +02:00
Vasiliy fae47715eb issue 410.3: remove the extra neting of ArangoSearch view definition into 'properties' (#5956)
* issue 410.3: remove the extra neting of ArangoSearch view definition into 'properties'

* address merge issue

* backport: address test failures

* backport: address dump test failure
2018-07-24 16:27:28 +03:00
Vasiliy 3c384bf3e8 issue 410.1: ensure both JavaScript and Rest API for views matches API for collections (#5903)
* issue 410.1: ensure both JavaScript and Rest API for views matches API for collections
* backport: address some FIXMEs
* backport: address test failures
* backport: address more test failures
* backport: fix some Ruby tests
* temporarily comment out broken Ruby tests
* backport: fix more broken Ruby tests
* temporarily comment out failing Ruby tests
* backport: match result structure for View instances with Collection instances
* backport: return different result structure for view listing and view definition
* backport: fix even more broken Ruby tests
* backport: do not depend on unordered_set stable order
* update documentation
2018-07-19 15:07:17 +02:00
Wilfried Goesgens d0ed06c033 more RFC compliance when parsing http headers - support quotes (#5860) 2018-07-13 11:14:06 +02:00
Vasiliy 64666d0edd issue 400.2.4: register IResearchView sync tasks with IResearchFeature, remove unused configuration parameter (#5813)
* issue 400.2.4: register IResearchView sync tasks with IResearchFeature, remove unused configuration parameter

* set flag before notify, fix failures in new tests from 'devel'

* revert set flag before notify, fix more failures in new tests from 'devel'

* fix even more failures in new tests from 'devel', backport: do not reset notification flag if pending tasks were seen

* fix still more failures in new tests from 'devel'

* try to work around buggy test failures
2018-07-10 18:15:01 +03:00
jsteemann 6c25718a85 temporarily turn off shell_client_aql tests 2018-07-10 16:05:44 +02:00
KVS85 18bf61c477 Add empty 3.4 tests blacklist for Oskar (#5801) 2018-07-09 14:52:52 +02:00
Jan b88fde9659
added key generators "uuid" and "padded" (#5740) 2018-07-03 15:23:47 +02:00
Vasiliy bf1f2eab29 issue 401.2: split IResearchViewMeta into the user configurable part and internal state (#5683)
* issue 401.2: split IResearchViewMeta into the user configurable part and internal state

* backport: update documentation and integration tests to reflect the IResearchViewMeta changes

* backport: addresss typo

* backport: address another typo
2018-06-27 16:40:51 +03:00
Simon 8c48fdf4ab Replace rocksdb export cursor with query (#5657) 2018-06-25 18:14:27 +02:00
Andrey Abramov 5eef6cd618
Feature/test iresearch (#5610)
* start implementing arangosearch cluster tests.

* backport: ensure view lookup is done via collectionNameResover, ensure updateProperties returns current view properties

* first attempt to fix failing tests

* refactor cluster wide view creation logic

* if view is not found in the new plan then check the old plan too

* ensure the cluster-wide view is looked up in vocbase as well on startup/recovery

* do not store cluster-wide IResearchView in vocbase

* move stale view cleanup to the shared pointer deleter, address test failures

* do not print warning

* enable arangosearch tests by default

* fix catch tests

* address icorrect return value for cluster-wide links

* address some issues with test failures due to cluster-view allocated within TRI_vocbase_t

* simplify per-cid view name, address 'catch' test failures

* ensure IResearchViewNode volatility is properly calculated in cluster

* invoke callbacks directly in AgencyMock instead of waiting for timeout

* ensure view updates via JavaScript always use the latest view definition

* pass a list of shards to `IResearchViewDBServer::snapshot`

* extend cluster aql tests

* fixes after merge

* fix class/struct inconsistencies

* comment failing tests

* remove debug logging

* add debug function

* tests cleanup

* simplify upcoming merge: pass resolver from a side

* backport: move all transaction status callback logic to Methods

* add changes missed from previous commit

* fix js and ruby tests

* more tests for IResearchViewNode

* pass transaction to IResearchViewDBServer::snapshot, address IResearchViewDBServer tests segfault

* pass transaction to IResearchView::snapshot instead of transaction state

* temporarily add trace log output to tests to try to find the cause of the core dump on Jenkins

* add more temporary debug output to trace down the segfault on Jenkins

* add even more temporary debug output to trace down the segfault on Jenkins

* ensure Vieew related maps are cleared during shutdown

* reset ClusterInfo::instance() before DatabaseFeature::unprepare()

* remove extraneous debug output

* missed line from previous commit

* uncomment required line

* add nullptr checks to RocksDBIndexFactory::prepareIndexes(...) similar to the ones in MMFilesIndexFactory::prepareIndexes(...)

* attempt to fix deadlock in tests

* add comment as per reviewer request

* fix aql test suite name

* add some debug logging

* address deadlock between ClusterInfo::loadPlan() and CollectionNameResolver::localNameLookup(...)

* eplicitly state which index definition failed in the log message

* use vocbase from shard-view isntead just in case

* explicitly state which index definition failed in the log message

* do not create shard-view instances from cluster-link instances (only register existing ones)

* add some tests
2018-06-21 20:35:04 +03:00
Jan 5ce22a3bc9
make API for POST /_api/aqlfunction downwards-compatible again (#5636) 2018-06-20 12:52:54 +02:00
jsteemann 50871dd87c fixed jslint warning 2018-06-13 19:25:16 +02:00
Wilfried Goesgens 0470f945fa Bug fix/improve test status handling (#5591)
* add missing documentation about argument handled in unittest.js

* improve error handling on fs.write

* fix option name

* improve result handling

- add exception handling
- write status bad even before we start the test so we really know when
  something went wrong.

* SYS exceptions will already handle system exceptions

* rethrow

* only return true if the file object has no error bit set

* don't double-close on error.

* add note that we disable exceptions explicitely
2018-06-13 14:10:44 +02:00
Wilfried Goesgens eb9bf93c98 Feature/cleanup rspec integration (#5517) 2018-06-07 11:47:46 +02:00
Wilfried Goesgens 83b22aee44 fix not condition (#5486) 2018-05-29 16:56:29 +02:00
Jan Christoph Uhde a2dcb6cc5d WIP - start adding optional overwrite to insert operation (RepSert) (#5268) 2018-05-24 19:47:15 +02:00
Simon 828f1d423c S2 based Geo-Spatial index (#5249) 2018-05-02 23:54:41 +02:00
Andrey Abramov 5fc89b315c adjust error message check 2018-03-20 19:09:07 +03:00
Andrey Abramov 29b7a7ea44 remove logger view (#4764) 2018-03-08 09:18:21 +01:00
Simon 345fc3c0b7 Refactor Authentication Layer (devel) (#4592)
* Cherry Picking LDAP changes

* Adding missing merges

* Fixing remaining mentions of FeatureCacheFeature

* Fix jslint

* Fixing some failed tests

* Fixing cluster authentication issue, red tests

* Fixing ldap testsuite, adding trace logging

* Fixint ldap tesuite setup and LDAP recognition

(cherry picked from commit 686d28a779)

* Fixing wrong assert

* Adding changelog entry, making requested changes from code review

* Fixing dump_authentication, fix typos

* improvements found during code review

* oops

* more use of sessionstorage

* fix tests

* Fixing broken handling, disallowing adding of local users when disabled

* Fixing testInvalidGrants

* Removing undefined auth level externally

* Fixing previous commit

* added tests for ldap search mode

* intentionally removed `after` methods from tests

because they are executed before the tests start
no cleanup is performed right now after the authentication tests
however, a cleanup is done at start of every test

* ldap tests all modes

* forward port changes from 3.3

* added generated files

* forward port missing changes for web UI

* added generated files

* added generated files
2018-02-28 13:24:28 +01:00
Simon 11a7bbf321 Replacing statistics handler with c++ (#4653) 2018-02-26 15:33:43 +01:00