1
0
Fork 0
Commit Graph

863 Commits

Author SHA1 Message Date
Simon 3ada15fc35 The Legendary El Cheapo (#8485) 2019-03-22 11:38:33 +01:00
jsteemann dc381a99df Revert "Feature/ncc1701 (#8440)"
This reverts commit 59ad583796.
2019-03-21 19:18:46 +01:00
Jan 59ad583796
Feature/ncc1701 (#8440) 2019-03-21 15:05:36 +01:00
Jan 6d107beeda
don't run compact() on a collection after a truncate() was done in th… (#8468)
* don't run compact() on a collection after a truncate() was done in the same transaction

running compact() in the same transaction will only increase the data size on disk due to RocksDB not being able to remove
any documents physically due to the snapshot we take at transaction start.
Decoupling the truncate transaction from the compact operation allows finishing the truncate transaction first, so we can
get rid of the snapshot. Running compact afterwards is then free to physically remove all the data.
As a nice side effect this change will also speed up the truncation of larger collections, because the compact will run
faster.

This change also exposes db.<collection>.compact() in the arangosh, in order to manually run a compaction on the data
range of a collection should it be needed for maintenance.

* fix documentation anchors
2019-03-20 16:27:54 +01:00
KVS85 a7090f30e9 fix typo (#8470) 2019-03-20 18:14:55 +03:00
KVS85 ea5e6aef05 Fix upgrade_data tests (#8459) 2019-03-20 11:05:39 +01:00
Wilfried Goesgens c2042c86e9 abort testing if we fail to get the currently available collections during server startup (#8442) 2019-03-18 15:14:08 +01:00
Dan Larkin-York 413e90508f Named indices (#8370) 2019-03-13 18:20:32 +01:00
Jan dd4938598c
use requireFromPresent and barriers in more cases (#8398) 2019-03-13 15:16:22 +01:00
Wilfried Goesgens 8bcddf1498 expect leader to have disappeared (#8326) 2019-03-06 11:20:49 +01:00
Wilfried Goesgens 4ae0cd42c3 when in activefailover mode detect leader changes, and abort tests since they wouldn't deliver good values anymore (#8306) 2019-03-05 11:10:20 +01:00
Wilfried Goesgens b1a1edb205 add default values and documentation missing for this parameter (#8303) 2019-03-01 14:11:48 +01:00
jsteemann 35311178fd try..catch to avoid useless failures 2019-01-29 18:00:56 +01:00
Jan f7d94daba2
port some changes from 3.4 (#8011) 2019-01-29 09:26:57 +01:00
Frank Celler 9927b3a281
Feature/maskings (#8006) 2019-01-22 22:23:25 +01:00
Wilfried Goesgens 457e14b970 add windows suspend/resume hooks as @char101 sugested for others (#7833)
* add windows suspend/resume hooks

* disable procdump
2019-01-11 12:41:27 +01:00
Simon a2a0b03f43 Rdb index background (preliminary) (#7644) 2018-12-21 19:24:10 +01:00
KVS85 bbd8aa4e0b Remove procdump usage while taring archives (#7815) 2018-12-21 11:39:39 +01:00
Wilfried Goesgens 7acb3b3ec4 wait for procdump to exit too. (#7731)
* wait for procdump to exit too.

* move running process check up, propagate status. First check whether processes are still running before attempting to force kill them

* only list processes, don't change their state

* add stopping procdump in all places
2018-12-12 18:06:08 +01:00
Michael Hackstein 170e3520be
Improved stability of dump restore test (#7593)
* Improved stability of dump restore test, the assumption tested was not correct

* Fixed options in recovery test suite
2018-12-10 15:48:34 +01:00
Dan Larkin-York 68ebf1d934 Update upgrade data tests for compatibility with Oskar. (#7485) 2018-12-10 12:51:51 +01:00
Jan fb04c1c602
Bug fix/forward port missing changes (#7706) 2018-12-10 12:51:20 +01:00
Tobias Gödderz 44b6257df9 Fix error messages, allow usage of rr with the recovery suite (#7539) 2018-12-10 12:43:45 +01:00
Wilfried Goesgens ea44f8394d try checking whether _admin/shutdown is received, print sockstats of arangod processes (#7677) 2018-12-06 15:52:46 +01:00
Wilfried Goesgens e890bdc01a add bucket to test summary, so you can determine which logfile to look at (#7661) 2018-12-06 14:23:42 +01:00
Wilfried Goesgens 76cf74cc83 add missing rootDir parameter (#7546) 2018-11-29 15:18:58 +01:00
Jan 683d45bd68
Feature/micro optimizations 28112018 (#7496) 2018-11-28 15:57:10 +01:00
Wilfried Goesgens 3cf8d2a2e7 make the recovering server output its log (#7467) 2018-11-26 19:16:51 +01:00
Wilfried Goesgens 346a0a15a4 fix test status (#7456) 2018-11-26 14:03:37 +01:00
Frank Celler 7a4cb9346e allow buckets for authentication 2018-11-23 22:13:22 +01:00
Heiko 3322cc5ae2 Bug fix/update foxx services after restore (#6774)
* triggers foxx queue update which leads to the internal selfHeal function of a foxx app to make foxx apps publicy visible after a foxx app is restored

* this will also add the recognition of imported foxx services

* Testing JS refactoring

* Added test case for Foxx dump -> restore cases

* Fixed check, sth- i tries to contact to the foxx service. Test is red now.

* Fixed tests for dump / restore using Foxx. Also first draft to fix the fox side. Blocked by sth. fixed in the Main Repositorx already

* Fixed typo in dump/restore testsuite

* Removed old version of the fix

* Fixed JSLint
2018-11-23 17:06:09 +01:00
Wilfried Goesgens ed4af5acf8 re-connect force terminate mechanism on test timeout (#7424) 2018-11-23 11:16:29 +01:00
Wilfried Goesgens a905b2ca35 Bugfix/fix recovery testsuite (#7396) 2018-11-22 11:18:28 +01:00
Wilfried Goesgens 51e020b0f7 make our tools output their log messages (#7403) 2018-11-21 16:38:03 +01:00
Simon c9e44046a2 add timeout (#7398)
(cherry picked from commit 800acadfcabc508cf9c65a18e7c0032b4bfb0c62)
2018-11-21 14:16:23 +01:00
Wilfried Goesgens 270301b312 when checking for test leftovers catch dead server connections and abort tests cleanly (#7283) 2018-11-20 10:42:55 +01:00
Wilfried Goesgens fdb1e85d8d When using `--test` to segment filter, we also need to add it to the database directory. (#6944) 2018-11-20 10:26:55 +01:00
Jan 0dd1776467
Make recovery more reliable (#7297) 2018-11-19 13:59:18 +01:00
Frank Celler a6e4c844ad Feature-3.4/whiteblacklist (#7259)
* add blacklist

* added --skipGrey
2018-11-08 00:01:29 +01:00
Wilfried Goesgens 6fe9b774f8 fix typo (#7254) 2018-11-07 17:37:12 +01:00
Dan Larkin-York b447f98a7a Upgrade old MMFiles collections for compatibility with ArangoSearch. (#7248) 2018-11-07 15:52:37 +01:00
Vasiliy 68953ae33a issue 496.4.1: move StorageEngine-specific flag out of the genric API and closer to the storage engine (#7212) 2018-11-04 16:52:28 +03:00
Wilfried Goesgens ecade47aaf don't spawn additional arangosh processes to run the tests (#7185) 2018-11-02 13:08:04 +01:00
Jan dc03e370f9
stop printing `connection class corrupted` in arangosh (#7180)
when just starting the arangosh without a connection to a server
and running code such as `require("internal")`, the shell always
printed "connection class corrupted", which is somewhat misleading.
just don't print this error here, as it is useless
2018-10-31 18:46:47 +01:00
Wilfried Goesgens 2cfe9126f2 Bug fix/windows process handling (#7012) 2018-10-23 15:38:24 +02:00
Simon 4c1e8819c2 Add engine specific collection APIs (#6977) 2018-10-19 17:46:33 +02:00
Wilfried Goesgens b0c91cf80c Don't listen to procdumps exit code (#6932) 2018-10-18 11:36:56 +02:00
Wilfried Goesgens 55a45f1a84 properly check wether the replication follower instance is still alive (#6938) 2018-10-17 14:14:09 +02:00
Wilfried Goesgens d1b3a1ee98 Don't talk to a server when we know its dead. Print current testsuite on Falure. (#6879) 2018-10-15 14:29:16 +02:00
Wilfried Goesgens 239771b5f3 Better result handling for recovery tests (#6826) 2018-10-15 13:17:45 +02:00
Kaveh Vahedipour 7f7e4e04fd fuerte move resulted in broken inspector 2018-10-12 19:52:24 +02:00
Wilfried Goesgens 676f61805e Make procdump abort on std::bad_cast; kill SUT if it occurs; Make procdump optional. (#6602) 2018-10-10 17:55:53 +02:00
Wilfried Goesgens ad31b31096 print when we mark a build crashy (#6769)
LGTM
2018-10-09 16:47:16 +02:00
Jan bd344f9f09
remove undocumented view.toArray() function (#6763) 2018-10-09 09:48:00 +02:00
Wilfried Goesgens 056ee7064c Feature/log timestamp on test start (#6759) 2018-10-08 16:18:04 +02:00
Jan ae85fbc4ea
much better diagnostics for AQL query results cache (#6580) 2018-09-25 09:49:11 +02:00
jsteemann a57640993f fixed JavaScript error 2018-09-18 20:13:46 +02:00
Simon 82aa24ad7e Copy installation files on startup (#6491) 2018-09-14 11:15:21 +02:00
Jan 7a3c2bf6af
add some debugging for unstable replication_sync test (#6436) 2018-09-10 14:46:50 +02:00
Simon 0661a4c1fe Hide Links from getIndexes() (#6325) 2018-09-03 15:17:24 +02:00
Frank Celler f17eac6b96 fixed typo in test setup 2018-09-02 10:21:39 +02:00
Frank Celler f696efe5ea removed auditlog 2018-09-02 08:54:04 +02:00
Jan e03f0daccd skip "dump_encrypted" test in non-enterprise mode (#6334) 2018-09-01 22:55:04 +02:00
Jan 3a3c4f7f8b fix failing readOnly test (#6333) 2018-09-01 22:54:33 +02:00
Frank Celler ca34956f3c fixed wrong result code 2018-09-01 16:16:29 +02:00
Jan 24ca3acf97
rename `tests/js/*/tests/*` to `tests/js/*/*` (#6280) 2018-08-31 19:42:21 +02:00
Jan 494764b95c
Bug fix/fiddle version string into test paths (#6267) 2018-08-28 14:58:35 +02:00
Jan 0a73e64b0b Bug fix/fiddle version string into test paths (#6266)
* fiddle version string into test paths
2018-08-28 09:27:32 +02:00
jsteemann 6fc8b43fe2 start the correct test 2018-08-28 01:01:02 +02:00
Jan 5022ccc24d
Bug fix/fixes 2508 (#6254) 2018-08-27 21:36:39 +02:00
Jan 791eaba873 Bug fix/planning 2865 (#6252)
* potential bugfix for planning/#2865

* speed up dump tests setup

* enable authentication for backup tests

* make arangodump provide a "serverId" to the server

this allows the server to track arangodump as an active dump client
so any data required for the dumping may be retained while the
dump is ongoing

* don't log binary stuff into the logfile
2018-08-27 10:21:34 +02:00
Kaveh Vahedipour 28754cbf15 Feature/schmutz plus plus (#5972)
- Schmutz now called "Maintenance" and completely implemented in C++
 - Fix index locking bug in mmfiles
 - Fix a bug in mmfiles with silent option and repsert
 - Slightly increase supervision okperiod and graceperiod
2018-08-24 12:15:35 +02:00
Wilfried Goesgens 2b063cada8 properly feed the crash analyzer function, so arangosh crashes will be analyzed too (if) (#6238) 2018-08-24 11:47:44 +02:00
Jan 03ea6b929b
fix "auto" test (#6224) 2018-08-24 10:45:31 +02:00
Jan 102f15bece
removed several unused internal APIs (#6193) 2018-08-20 12:57:58 +02:00
Vasiliy c08ab01f67 issue 450.1: remove the "commit" configuration layer from ArangoSearch properties (#6185)
* issue 450.1: remove the "commit" configuration layer from ArangoSearch properties

* fix broken tests
2018-08-20 02:18:02 +03:00
jsteemann 59db24c65e fixed jslint error 2018-08-17 20:24:47 +02:00
Vasiliy 320332c779 issue 433.1: make client JavaScript code for view DDL more similar to collection DDL (#6179)
* issue 433.1: make client JavaScript code for view DDL more similar to collection DDL

* remove invalid statement

* remove cached view
2018-08-17 17:14:57 +03:00
Jan d6a3b66e2a
micro optimizations (#6162) 2018-08-16 08:50:16 +02:00
Tobias Gödderz de4f5587ae Gharial rewrite in C++ (#5631)
* Built a C++ skeleton REST handler for gharial, with fallback to the JS handler

* Moved aql::Graph to graph::Graph

* Added complete edge definitions to Graph

Also:
- some cleanup
- used forward-declarations in headers
- use Graph in graph rest handler

* Handle graph lookup failures according to the test suite

* Added GET vertex

* Bugfixes in ResultT

- Added missing #include
- Fixed move semantics

* Move central code of readVertex to GraphOperations

* ResultT fixes and complements

* Implemented a graph cache

* Added and used graph cache to the rest handler

* Added GET edge

* Added DELETE edge

* Extracted some code

* Added PATCH and PUT for both edge and vertex

* Moved update/replace transaction code to GraphOperations

* Added stub routes for POST and a TODO note

* Added a test checking that deleting a vertex removes all incident edges as well

* Added a test checking that deleting a vertex does not remove edges in non-graph collections

* fixed compiler warnings and errors

* Began work on DELETE vertex

For this, added a V8Context to allow for AQL queries to use subtransactions

* Continued work on DELETE vertex (still WIP)

* prep for graph post routes

* fixed removeVertex operation (aql)

* added post vertex and post edge gharial routes

* wasSynchronoues flag changed

* gharial post c++ handler, naming conventions

* added gharial tests

* temporary disabled cache (because not completed), added graph property read functions

* added c++ gharial list vertex collections

* added c++ gharial graph config

* added c++ gharial list graphs

* added graph manager class

* first implementation of create graph in c++, WIP

* changed error messages

* added etag to create graph api, still multiple edge definition check missing

* finished POST /_api/gharial/<graph>

* WIP - DELETE /_api/gharial/<graphName>

* added DELETE /_api/gharial/<graphName> validation, still missing correct response

* gharial delete

* fixed delete gharial lock

* finished DELETE /_api/gharial/<graphName>

* added routes for graph based vertices and edge definitions

* improved delete route

* added add new edge definition to existing graph

* patch edge definition in a graph, still <WIP>

* finished edit edge definition route

* code changes due to devel code changes

* added remove edge definition route

* added vertex delete function

* added todo note regarding one drop collection issue

* add oprhan collection to graph route implemented

* Added a test

* Updated a comment

* Several minor changes

* Minor changes during review

* Changes during review

* Changes during review

* Bugfix: orphans may be null or omitted

* Bugfix: resolve externals

* minor code changes

* seperated graph class to independent component classes

* seperated graph class to independent component classes

* removed log output

* fixed create collection behaviour in a cluster environment

* fixed enterprise graphs behaviour in c++ gharial api

* removed log output

* formatting

* improved error handling, fixed a linux compile bug

* more result refactoring

* more result type cleanup

* fixed wrongly defined test

* result handling

* error handling

* more refactoring

* Bugfix: avoid race condition in cluster when creating collections

* updated graph documentation

* added graph related static strings

* static strings, new method to create options for gharial created collections

* Some minor cleanup

* more use of static strings

* minor code changes, review

* added missing parseint

* removed gharial foxx, added js common module, added v8 general graph module

* correct use of virtual method

* more v8, js general graph, broken state

* more v8 graph functions

* fixed editEdgeDefinition, added drop function

* fixed drop behaviour

* added _list, _exists

* added c++ rename graph collections, added v8 + graph module function

* Added a regression test

* added graph._deleteEdgeDefinition, v8, server

* more v8g

* added _removeVertexCollection

* added _extendEditDefinitions

* todo, need to add a helper sort method for a local defined relation

* fixed test

* fixed lots of tests, added more client functions, _addVertexCollection on client module is still broken

* added more client graph functions, all tests green

* more client functions

* add del edge def route

* Fix use after move

* Minor changes in client general-graph.js module

* Make a copy before sorting (don't touch the argument)

* Minor changes and some additional asserts in graph tests

* Consistently set parameter defaults

* Renamed static strings

* Remove superfluous function

* Made comment more verbose

* Minor changes in general-graph-common.js

* Added missing template arguments

* Fixed community build

* Cleanup in editEdgeDefinition

* Regression test & bugfix: comparison of edge definitions didn' order from and to

* Fixed errors introduced by merge

* Minor changes in v8-general-graph.cpp

* Fixed test failure due to wrong error code in CE

* added missing id field

* Added permission checks for graph._create

* Removed assertion that is no longer valid

* Moved removeGraph from GraphOperations to GraphManager

* Allow C++ implementation of graph._drop to handle smart graphs

* Flush js client db cache after creating/dropping collections via the general graph module

* Added _deleteEdgeDefinition to the general graph client module

* WIP: Added permission checks for drop graph

* Fixed permission checks for drop graph

* Added permission checks for other graph operations

* Bugfix: assert edge definitions are returned in order

* Some cleanup

* Removed unused method

* Minor improvements in GraphManager

* Fixed a type in general-graph common module

* Most useful fix of all times ever: Do not auto cast from bool to int and alternate error/noerror by this

* Added the initial keyword to StaticStrings

* Added a new error code, used whenever a user tries to inject a documentcollection as a relation into the graph, which is invalid

* Some GraphManager/Ops/Graph cleanup. Less Slice parsing, more usage of GraphObjects

* Test edgeDefinitions in graphs with a defined ordering

* GraphClass Layout cleanup

* Do not test error messages, use codes instead

* Recreated backwards compatibility of Graph Creation Permission errors

* Changed error-code if edgedefinition is used twice

* Added a StaticString for the GraphName

* Renamed graphToVpack => graphForClient

* Partly fixed graph-api test to work with better error messages. Still red: The edgeDefinitions are now sorted, the test is supposed to sort his own list, but appearently does not do so. Under investigation

* Added a new error code that rejects injection of differently sharded smart collection into smartgraph. Should be more helpful to our users

* graph createCollectionOptions now require an open object to be cross-called from enterprise. Made enterprise switch for creation of graph more elegant.

* Updated graphs.cpp

* Massive refactoring. Made Factories for graphs to make SmartGraph much more transparent. Also reduced amount of multiple implementations of the same stuff. Killed vocbase/graphs use GraphManager instead. Removed usage of GraphCache, was not completely implemented anyway and only partially used, which is bad at the moment. Option for later improvement never the less

* Adapted JS code to now really use c++ variants. ALso included 3 Classes: Graph, SmartGraph and GraphModule.

* Fixed undefined behaviour in Remove Vertex. Fixed smartgraph sharding if one collection already exists.

* Removed DEBUG output

* Removed DEBUG logs

* Removed dead code

* Fixed Graph EdgeDefinition test, they now have a different ordering.

* Added a test when adding a vertexCollection that it is actually valid in the graph

* Client Graph API now correctly sends `orphanCollections` and not `orphans`

* Let GraphOperations modify the graph in-place. It should now properly handle edgeDefinitions.

* Added initial cid StaticString

* Included the vocbase in fromPersistence creation of Graphs. Only required to enhance 3.3 SmartGraphs on the fly.

* Fixed internal error message

* Fixed compiler isses originiated from merging

* Removed unused imports

* Regenerated generated file
2018-08-09 09:30:04 +02:00
Vinay Pyati 5709034e3d getBatchSize() instead setBatchSize() in Get query options (#6116) 2018-08-08 14:07:04 +02:00
Simon 606c15ce48 VST support in arangosh (#6047) 2018-08-08 12:18:54 +02:00
Frank Celler 54f72cae1b added pretty print for views 2018-08-03 14:26:33 +02:00
Jan b278d6874a
allow master & slave to work in parallel for RocksDB WAL tailing (#6059) 2018-08-03 13:37:53 +02:00
Dan Larkin-York 54e607aa39 Added C++ implementation, load balancer support, and user restriction to Pregel API (#6060) 2018-08-03 12:55:28 +02:00
Frank Celler c37d12a790
restrict attributes (#6066) 2018-08-03 11:01:56 +02:00
Frank Celler 05ee214012 Revert "added pretty print for views"
uthor: Frank Celler <frank@

This reverts commit be3ae7a444.
2018-08-03 09:35:00 +02:00
Frank Celler be3ae7a444 added pretty print for views 2018-08-03 09:32:56 +02:00
jsteemann efa5e8faa3 better status information on shutdown 2018-07-30 19:27:19 +02:00
Jan 2a416f2e33
Bug fix/3007 (#6019) 2018-07-30 17:16:47 +02:00
jsteemann 89edebaeae also make RocksDB the default engine when testing 2018-07-26 18:25:43 +02:00
Vasiliy a291f6dcfa issue 431.1: remove error generation for empty jSON (#6007) 2018-07-26 19:03:35 +03:00
Jan e4d7f1c5f0
Bug fix/wenn der shard mann 2mal klingelt (#5890) 2018-07-26 15:37:40 +02:00
Wilfried Goesgens 93f9da4078 set the disableCores parameter properly (#5965) 2018-07-24 17:50:33 +02:00
jsteemann fb7ab6d0d0 fixed magic number 2018-07-24 17:48:45 +02:00
Wilfried Goesgens c463bda0d7 Rework shutdown & crash detection
- make sure we only check exactly _once_ for the exit status of a server
  that is no longer running
- don't do a separate loop about Coordinator/DB-Server shutdown
- output the instance type after naming it being shut down
- as before, shut down DBserver and Coordinators first, then the agency.
2018-07-24 11:37:42 +02:00
Wilfried Goesgens 0a3da65cfd Merge branch 'devel' of github.com:arangodb/ArangoDB into bug-fix/only-test-once-for-exit 2018-07-24 11:16:26 +02:00
Jan b1fab04296
issue a call to /_admin/cluster/maintenance before shutdown (#5945) 2018-07-24 09:34:57 +02:00
Dan Larkin-York 718e3f7bfa Add load balancing support and user restriction for async jobs API (#5906) 2018-07-23 15:23:18 +02:00
Wilfried Goesgens c1484d08a0 when shutdownArangod() already stopped arangods, don't look for the status again 2018-07-23 10:41:40 +02:00