1
0
Fork 0
Commit Graph

735 Commits

Author SHA1 Message Date
Jan d6a3b66e2a
micro optimizations (#6162) 2018-08-16 08:50:16 +02:00
Tobias Gödderz de4f5587ae Gharial rewrite in C++ (#5631)
* Built a C++ skeleton REST handler for gharial, with fallback to the JS handler

* Moved aql::Graph to graph::Graph

* Added complete edge definitions to Graph

Also:
- some cleanup
- used forward-declarations in headers
- use Graph in graph rest handler

* Handle graph lookup failures according to the test suite

* Added GET vertex

* Bugfixes in ResultT

- Added missing #include
- Fixed move semantics

* Move central code of readVertex to GraphOperations

* ResultT fixes and complements

* Implemented a graph cache

* Added and used graph cache to the rest handler

* Added GET edge

* Added DELETE edge

* Extracted some code

* Added PATCH and PUT for both edge and vertex

* Moved update/replace transaction code to GraphOperations

* Added stub routes for POST and a TODO note

* Added a test checking that deleting a vertex removes all incident edges as well

* Added a test checking that deleting a vertex does not remove edges in non-graph collections

* fixed compiler warnings and errors

* Began work on DELETE vertex

For this, added a V8Context to allow for AQL queries to use subtransactions

* Continued work on DELETE vertex (still WIP)

* prep for graph post routes

* fixed removeVertex operation (aql)

* added post vertex and post edge gharial routes

* wasSynchronoues flag changed

* gharial post c++ handler, naming conventions

* added gharial tests

* temporary disabled cache (because not completed), added graph property read functions

* added c++ gharial list vertex collections

* added c++ gharial graph config

* added c++ gharial list graphs

* added graph manager class

* first implementation of create graph in c++, WIP

* changed error messages

* added etag to create graph api, still multiple edge definition check missing

* finished POST /_api/gharial/<graph>

* WIP - DELETE /_api/gharial/<graphName>

* added DELETE /_api/gharial/<graphName> validation, still missing correct response

* gharial delete

* fixed delete gharial lock

* finished DELETE /_api/gharial/<graphName>

* added routes for graph based vertices and edge definitions

* improved delete route

* added add new edge definition to existing graph

* patch edge definition in a graph, still <WIP>

* finished edit edge definition route

* code changes due to devel code changes

* added remove edge definition route

* added vertex delete function

* added todo note regarding one drop collection issue

* add oprhan collection to graph route implemented

* Added a test

* Updated a comment

* Several minor changes

* Minor changes during review

* Changes during review

* Changes during review

* Bugfix: orphans may be null or omitted

* Bugfix: resolve externals

* minor code changes

* seperated graph class to independent component classes

* seperated graph class to independent component classes

* removed log output

* fixed create collection behaviour in a cluster environment

* fixed enterprise graphs behaviour in c++ gharial api

* removed log output

* formatting

* improved error handling, fixed a linux compile bug

* more result refactoring

* more result type cleanup

* fixed wrongly defined test

* result handling

* error handling

* more refactoring

* Bugfix: avoid race condition in cluster when creating collections

* updated graph documentation

* added graph related static strings

* static strings, new method to create options for gharial created collections

* Some minor cleanup

* more use of static strings

* minor code changes, review

* added missing parseint

* removed gharial foxx, added js common module, added v8 general graph module

* correct use of virtual method

* more v8, js general graph, broken state

* more v8 graph functions

* fixed editEdgeDefinition, added drop function

* fixed drop behaviour

* added _list, _exists

* added c++ rename graph collections, added v8 + graph module function

* Added a regression test

* added graph._deleteEdgeDefinition, v8, server

* more v8g

* added _removeVertexCollection

* added _extendEditDefinitions

* todo, need to add a helper sort method for a local defined relation

* fixed test

* fixed lots of tests, added more client functions, _addVertexCollection on client module is still broken

* added more client graph functions, all tests green

* more client functions

* add del edge def route

* Fix use after move

* Minor changes in client general-graph.js module

* Make a copy before sorting (don't touch the argument)

* Minor changes and some additional asserts in graph tests

* Consistently set parameter defaults

* Renamed static strings

* Remove superfluous function

* Made comment more verbose

* Minor changes in general-graph-common.js

* Added missing template arguments

* Fixed community build

* Cleanup in editEdgeDefinition

* Regression test & bugfix: comparison of edge definitions didn' order from and to

* Fixed errors introduced by merge

* Minor changes in v8-general-graph.cpp

* Fixed test failure due to wrong error code in CE

* added missing id field

* Added permission checks for graph._create

* Removed assertion that is no longer valid

* Moved removeGraph from GraphOperations to GraphManager

* Allow C++ implementation of graph._drop to handle smart graphs

* Flush js client db cache after creating/dropping collections via the general graph module

* Added _deleteEdgeDefinition to the general graph client module

* WIP: Added permission checks for drop graph

* Fixed permission checks for drop graph

* Added permission checks for other graph operations

* Bugfix: assert edge definitions are returned in order

* Some cleanup

* Removed unused method

* Minor improvements in GraphManager

* Fixed a type in general-graph common module

* Most useful fix of all times ever: Do not auto cast from bool to int and alternate error/noerror by this

* Added the initial keyword to StaticStrings

* Added a new error code, used whenever a user tries to inject a documentcollection as a relation into the graph, which is invalid

* Some GraphManager/Ops/Graph cleanup. Less Slice parsing, more usage of GraphObjects

* Test edgeDefinitions in graphs with a defined ordering

* GraphClass Layout cleanup

* Do not test error messages, use codes instead

* Recreated backwards compatibility of Graph Creation Permission errors

* Changed error-code if edgedefinition is used twice

* Added a StaticString for the GraphName

* Renamed graphToVpack => graphForClient

* Partly fixed graph-api test to work with better error messages. Still red: The edgeDefinitions are now sorted, the test is supposed to sort his own list, but appearently does not do so. Under investigation

* Added a new error code that rejects injection of differently sharded smart collection into smartgraph. Should be more helpful to our users

* graph createCollectionOptions now require an open object to be cross-called from enterprise. Made enterprise switch for creation of graph more elegant.

* Updated graphs.cpp

* Massive refactoring. Made Factories for graphs to make SmartGraph much more transparent. Also reduced amount of multiple implementations of the same stuff. Killed vocbase/graphs use GraphManager instead. Removed usage of GraphCache, was not completely implemented anyway and only partially used, which is bad at the moment. Option for later improvement never the less

* Adapted JS code to now really use c++ variants. ALso included 3 Classes: Graph, SmartGraph and GraphModule.

* Fixed undefined behaviour in Remove Vertex. Fixed smartgraph sharding if one collection already exists.

* Removed DEBUG output

* Removed DEBUG logs

* Removed dead code

* Fixed Graph EdgeDefinition test, they now have a different ordering.

* Added a test when adding a vertexCollection that it is actually valid in the graph

* Client Graph API now correctly sends `orphanCollections` and not `orphans`

* Let GraphOperations modify the graph in-place. It should now properly handle edgeDefinitions.

* Added initial cid StaticString

* Included the vocbase in fromPersistence creation of Graphs. Only required to enhance 3.3 SmartGraphs on the fly.

* Fixed internal error message

* Fixed compiler isses originiated from merging

* Removed unused imports

* Regenerated generated file
2018-08-09 09:30:04 +02:00
Vinay Pyati 5709034e3d getBatchSize() instead setBatchSize() in Get query options (#6116) 2018-08-08 14:07:04 +02:00
Simon 606c15ce48 VST support in arangosh (#6047) 2018-08-08 12:18:54 +02:00
Frank Celler 54f72cae1b added pretty print for views 2018-08-03 14:26:33 +02:00
Jan b278d6874a
allow master & slave to work in parallel for RocksDB WAL tailing (#6059) 2018-08-03 13:37:53 +02:00
Dan Larkin-York 54e607aa39 Added C++ implementation, load balancer support, and user restriction to Pregel API (#6060) 2018-08-03 12:55:28 +02:00
Frank Celler c37d12a790
restrict attributes (#6066) 2018-08-03 11:01:56 +02:00
Frank Celler 05ee214012 Revert "added pretty print for views"
uthor: Frank Celler <frank@

This reverts commit be3ae7a444.
2018-08-03 09:35:00 +02:00
Frank Celler be3ae7a444 added pretty print for views 2018-08-03 09:32:56 +02:00
jsteemann efa5e8faa3 better status information on shutdown 2018-07-30 19:27:19 +02:00
Jan 2a416f2e33
Bug fix/3007 (#6019) 2018-07-30 17:16:47 +02:00
jsteemann 89edebaeae also make RocksDB the default engine when testing 2018-07-26 18:25:43 +02:00
Vasiliy a291f6dcfa issue 431.1: remove error generation for empty jSON (#6007) 2018-07-26 19:03:35 +03:00
Jan e4d7f1c5f0
Bug fix/wenn der shard mann 2mal klingelt (#5890) 2018-07-26 15:37:40 +02:00
Wilfried Goesgens 93f9da4078 set the disableCores parameter properly (#5965) 2018-07-24 17:50:33 +02:00
jsteemann fb7ab6d0d0 fixed magic number 2018-07-24 17:48:45 +02:00
Wilfried Goesgens c463bda0d7 Rework shutdown & crash detection
- make sure we only check exactly _once_ for the exit status of a server
  that is no longer running
- don't do a separate loop about Coordinator/DB-Server shutdown
- output the instance type after naming it being shut down
- as before, shut down DBserver and Coordinators first, then the agency.
2018-07-24 11:37:42 +02:00
Wilfried Goesgens 0a3da65cfd Merge branch 'devel' of github.com:arangodb/ArangoDB into bug-fix/only-test-once-for-exit 2018-07-24 11:16:26 +02:00
Jan b1fab04296
issue a call to /_admin/cluster/maintenance before shutdown (#5945) 2018-07-24 09:34:57 +02:00
Dan Larkin-York 718e3f7bfa Add load balancing support and user restriction for async jobs API (#5906) 2018-07-23 15:23:18 +02:00
Wilfried Goesgens c1484d08a0 when shutdownArangod() already stopped arangods, don't look for the status again 2018-07-23 10:41:40 +02:00
jsteemann 7cd3c3e706 added facility to turn off specific endpoints tests
use `--skipEndpointsIpv6 true` to turn off Ipv6 endpoint tests
  use `--skipEndpointsUnix true` to turn off Unix domain socket endpoint tests

  additionally, use a random filename for the unix socket descriptor file
2018-07-20 16:20:06 +02:00
jsteemann ab6b539c03 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2018-07-19 15:59:02 +02:00
Vasiliy 3c384bf3e8 issue 410.1: ensure both JavaScript and Rest API for views matches API for collections (#5903)
* issue 410.1: ensure both JavaScript and Rest API for views matches API for collections
* backport: address some FIXMEs
* backport: address test failures
* backport: address more test failures
* backport: fix some Ruby tests
* temporarily comment out broken Ruby tests
* backport: fix more broken Ruby tests
* temporarily comment out failing Ruby tests
* backport: match result structure for View instances with Collection instances
* backport: return different result structure for view listing and view definition
* backport: fix even more broken Ruby tests
* backport: do not depend on unordered_set stable order
* update documentation
2018-07-19 15:07:17 +02:00
jsteemann f60fe97b28 fix `--database.check-version` 2018-07-18 12:18:42 +02:00
Jan f4b99bb5ed
Bug fix/meierloch (#5895) 2018-07-17 11:39:50 +02:00
Simran 34ec56d421 Feature/misc spelling corrections (#5164) 2018-07-13 13:06:20 +02:00
Wilfried Goesgens 5e296cb0c9 add ability to filter jsunity testsuites by testcase (#5807) 2018-07-12 13:35:29 +02:00
jsteemann dbad0f27b3 remove yet another hard-coded thread number issue 2018-07-11 11:36:38 +02:00
Jan 9ab26ba972
Bug fix/fix hangs (#5808) 2018-07-10 11:39:37 +02:00
KVS85 e2d1d1e898 Bug fix/convert arangosearch tests to common (#5772)
* Create shell_client_aql test alias and move aql-view-arangosearch-noncluster to it

*  Changes to be committed:
	modified:   js/client/modules/@arangodb/testsuites/aql.js

* Move remained AS AQL tests (aql-view-arangosearch-cluster, aql-view-arangosearch-ddl-noncluster, aql-view-arangosearch-ddl-cluster, aql-view-arangosearch-feature) to common and fix aql.js to really use arangosh in client execution

* Update aql.js of client modules

* 	modified:   aql-view-arangosearch-noncluster.js

* Remove toString() from assertions

* Minor typo fixes and changes to aql-view-arangosearch-noncluster tests from devel

* Added lost assertion for testInTokensFilterSortTFIDF

* Fix typos.

* Restored modified in devel tests to be common.

* Fix db._query() typo.

* Remove unsupported AQL query and shell_*_aql from shell_* groups of
tests.

* Fix conflict.

* Fix conflict.

* Remove moved tests.

* Revert "Remove moved tests."

This reverts commit bcdbde4baed610df2ae2ab4768a8c17b79ed3f64.

* test
2018-07-04 21:16:00 +03:00
Jan b88fde9659
added key generators "uuid" and "padded" (#5740) 2018-07-03 15:23:47 +02:00
Dan Larkin-York 21e16a8a24 Add load balancer awareness for cursor API (#5682) 2018-07-03 14:29:09 +02:00
Simon 545561e9a9 Read only server (#5652) 2018-07-03 09:58:16 +02:00
Jan 90d90f2cd2
added test suite for audit log (#5681) 2018-06-29 10:53:34 +02:00
jsteemann 0e59b674cc execute enterprise tests again 2018-06-22 19:27:00 +02:00
Andrey Abramov 5eef6cd618
Feature/test iresearch (#5610)
* start implementing arangosearch cluster tests.

* backport: ensure view lookup is done via collectionNameResover, ensure updateProperties returns current view properties

* first attempt to fix failing tests

* refactor cluster wide view creation logic

* if view is not found in the new plan then check the old plan too

* ensure the cluster-wide view is looked up in vocbase as well on startup/recovery

* do not store cluster-wide IResearchView in vocbase

* move stale view cleanup to the shared pointer deleter, address test failures

* do not print warning

* enable arangosearch tests by default

* fix catch tests

* address icorrect return value for cluster-wide links

* address some issues with test failures due to cluster-view allocated within TRI_vocbase_t

* simplify per-cid view name, address 'catch' test failures

* ensure IResearchViewNode volatility is properly calculated in cluster

* invoke callbacks directly in AgencyMock instead of waiting for timeout

* ensure view updates via JavaScript always use the latest view definition

* pass a list of shards to `IResearchViewDBServer::snapshot`

* extend cluster aql tests

* fixes after merge

* fix class/struct inconsistencies

* comment failing tests

* remove debug logging

* add debug function

* tests cleanup

* simplify upcoming merge: pass resolver from a side

* backport: move all transaction status callback logic to Methods

* add changes missed from previous commit

* fix js and ruby tests

* more tests for IResearchViewNode

* pass transaction to IResearchViewDBServer::snapshot, address IResearchViewDBServer tests segfault

* pass transaction to IResearchView::snapshot instead of transaction state

* temporarily add trace log output to tests to try to find the cause of the core dump on Jenkins

* add more temporary debug output to trace down the segfault on Jenkins

* add even more temporary debug output to trace down the segfault on Jenkins

* ensure Vieew related maps are cleared during shutdown

* reset ClusterInfo::instance() before DatabaseFeature::unprepare()

* remove extraneous debug output

* missed line from previous commit

* uncomment required line

* add nullptr checks to RocksDBIndexFactory::prepareIndexes(...) similar to the ones in MMFilesIndexFactory::prepareIndexes(...)

* attempt to fix deadlock in tests

* add comment as per reviewer request

* fix aql test suite name

* add some debug logging

* address deadlock between ClusterInfo::loadPlan() and CollectionNameResolver::localNameLookup(...)

* eplicitly state which index definition failed in the log message

* use vocbase from shard-view isntead just in case

* explicitly state which index definition failed in the log message

* do not create shard-view instances from cluster-link instances (only register existing ones)

* add some tests
2018-06-21 20:35:04 +03:00
Jan 5ce22a3bc9
make API for POST /_api/aqlfunction downwards-compatible again (#5636) 2018-06-20 12:52:54 +02:00
Wilfried Goesgens 0470f945fa Bug fix/improve test status handling (#5591)
* add missing documentation about argument handled in unittest.js

* improve error handling on fs.write

* fix option name

* improve result handling

- add exception handling
- write status bad even before we start the test so we really know when
  something went wrong.

* SYS exceptions will already handle system exceptions

* rethrow

* only return true if the file object has no error bit set

* don't double-close on error.

* add note that we disable exceptions explicitely
2018-06-13 14:10:44 +02:00
Wilfried Goesgens da3b222ebc disable circumventing of coredumps for windows (#5583)
* disable circumventing of coredumps for windows
* this is a local variable now.
* fix argument count
2018-06-13 13:54:24 +02:00
Wilfried Goesgens 31c708ce71 merge dump suites so they can use shared code as much as possible. (#5528) 2018-06-12 13:48:04 +02:00
Jan 8e28b03efd
do not invoke catch tests twice (#5570) 2018-06-11 10:24:07 +02:00
Jan 7351423ad5
cleanup import messages, reuse StringBuffer for JSON import (#5569) 2018-06-11 09:47:26 +02:00
jsteemann 20de7f04e6 fix "scripts/unittest arangosh" 2018-06-09 13:44:01 +02:00
Heiko 743d4430bf added replication ui (#5555) 2018-06-08 11:19:05 +02:00
Wilfried Goesgens a416d12fd8 forgot to add bucket splitting (#5561) 2018-06-08 10:54:10 +02:00
Frank Celler c5ac519d1c Bug fix for Read/Write race [WIP] (#5534)
* added wrapper, added asio_ns
* Temporarily fix condition variable bug in job queue.
* preparation for 3.3 back-port
* clang-format
* removed unecessary check, this is now fixed by stand
* added missing RequestStatistics::SET_READ_END
* cosmetics
2018-06-08 10:51:54 +02:00
Kaveh Vahedipour 1671201241 Bugfix/inspector thread (#5553) 2018-06-07 16:23:10 +02:00
Kaveh Vahedipour de04250fbb typos is api acquisitions (#5543) 2018-06-07 13:42:46 +02:00