1
0
Fork 0
Commit Graph

3357 Commits

Author SHA1 Message Date
Tobias Gödderz afe13fba57 Fix race condition between foxx install and selfHeal (#5273) 2018-05-11 12:27:56 +02:00
Tobias Gödderz 8c87f51429 Feature/fix inconsistent distribute shards like job (#4743) 2018-05-07 16:53:08 +02:00
Jan 8cd52ae95b
remove obsolete printBrowser function (#5259) 2018-05-04 11:30:39 +02:00
Jan 9c0460acff
Feature/aql optimizations (#5237) 2018-05-03 16:55:58 +02:00
Alan Plum 7a5139fe2c Add query, aql.literal (#5184)
* Add query, aql.literal

* Update ArangoDB.md
2018-05-03 14:18:20 +02:00
Alan Plum d5a93f5926 Teach executeTransaction about ArangoCollection objects (#5183)
* Teach executeTransaction about ArangoCollection objects

* Simplify logic

* Don't test with allowImplicit

* Update arango-database.js
2018-05-03 14:17:31 +02:00
Simon 828f1d423c S2 based Geo-Spatial index (#5249) 2018-05-02 23:54:41 +02:00
jsteemann c3efc6eab5 fix compilation 2018-05-02 22:01:11 +02:00
Michael Hackstein 43d0dcbbd7
Bug fix/remove unnecessary cmake replace (#5242)
* Bug fix 3.3/remove unnecessary cmake replace (#5233)

* Removed superflous comment
2018-05-02 12:11:53 +02:00
Andrey Abramov 4649b40b96
Coordinator ArangoSearch view + Execution nodes + AgencyMock (#5160)
* add initial implementation of scatter view rule and node

* add tests for `IResearchViewNode` and `IResearchViewScatterNode`

* add missing check

* modify IResearch execution nodes to use references instead of pointers

* use view id in searialized `ExecutionNode` representation instead of the name

* add cluster mode stubs and checks

* very first attempt to distribute IResearchViewNode

* further implementation of cluster-wide arangosearch views

* fix invalid json format

* add tests for coordinator iresearch view

* allow to retrieve a list of existing views on a coordinator

* more tests for coordinator iresearch view

* some fixes to enable query explanation

* remove Collection dependency from RemoteNode

* remove unnecessary remote ArangoSearch view scatter

* fix explanation appearance

* add some assertions

* minor fixes

* implement IResearchViewCoordinator::updateProperties

* fix view DDL issues

* handle link modifications in DDL operations

* add coordinator implementation of iresearch view links

* fix tests

* further coordinator based view DDL implementation

* further IResearchViewCoordinator implementation

* add initial implementation of AgencyMock

* fix some tests

* code cleanup

* extend test + some fixes

* more tests for IResearchViewCoordinator

* fix tests for IResearchLinkCoordinator

* some fixes after merge

* fix tests

* remove declaration of nonexistent (previously removed) method

* some fixes after review

* remove string duplication

* more tests and fixes

* more fixes and tests

* more tests

* one more test

* fix 'use-after-free' asan error

* fix non-enterprise tests issues
2018-05-02 00:15:11 +03:00
Dan Larkin-York b9d05351fb Parallelize arangorestore using the tools developed for arangodump (#5206) 2018-05-01 10:56:11 +02:00
Tobias Gödderz 192fc46486 Removed ERROR_ARANGO_COLLECTION_NOT_FOUND (#5231) 2018-04-30 18:34:02 +02:00
Jan 349e2c1681 forward-port AQL optimizations from 3.3 (#5020)
* forward-port AQL optimizations from 3.3

* fix restrict-to-single-shard rule

* added tests for specific cases

* ModificationNodes now can know if they are restricted or not

* OptimizerRule now updates the ModificationNode instead of setting ownName on remote node.

* The DBServerEngine container now only sends a subset of snippets (those which are relevant) It is  able to restrict any part of the query to a certain shard or even a list of shards

* ExecutionEngine CreateBlocks in ClusterMode now returns Result instead of throwing.

* Adapted Coordinator side to a bitter mapping of RemoteNode => QuerySnippets. This is still ongoing work.

* Fixed Coordinator enginecontainer test and finisehd implementation of EngineInfoContainerDBServer. Seems to work so far, need to adapt test to prove it.

* Adapted test for modification nodes and restrict to shard.

* restrictTo can now be de-/serialized on Modification Nodes

* Fixed Community Compile Error

* Added a test for Indexes/EnumerateCollection nodes

* EnumerateCollection and IndexNode now have options to be restricted to a single shard, opt rule not yet updated

* DBServer part of planning now checks Enumerate and IndexNodes for single shard optimization

* Added tests for simple single shard optimization on IndexNodes

* Adapted optimzierrule for SingleShard optimierung on IndexNodes

* Fixed an issue with externally restricted shard ids in combination with shard_id based optimization

* Added a test for multiple single shard optimizations on the same collection

* Removed collection/Shard duplicate check of optimize to single shard rules

* Updated Explainer to handle optimize to single shard

* Let more tests pass an optimized AQL variant

* Temporarily disabled a tests that can only be solved after some work in transaction
2018-04-30 11:25:09 +02:00
Heiko bcd33b8ea1 Bug fix/issue 3811 (#4984)
* now checking existence of _from and _to vertices during edge creation
2018-04-25 14:24:24 +02:00
Simon 95c8710dec Fixing subqueries in aql profiler (#5195)
* Fixing subqueries in aql profiler

* Adding additional testsuite

* Fix AQL profiling example name, add API example

Also use different query for explain and profiling with SLEEP() for
non-zero runtimes.

* New/updated examples for query explain/profiling

* Fix runtime calculation
2018-04-25 13:35:55 +02:00
Simon 468231efc5 AQL Profiling code (#5165)
* initial start of profiling

* adding profiling code

* Fixing remote block tracing, fixing width and units

* Fixing some tests

* Various fixes

* adressing review comments
2018-04-24 16:17:30 +02:00
Michael Hackstein 0d2d61ca9c
Bug fix 3.3/anonymize in debug dump (#5190) (#5191)
* Fixed the amount of Xs generated by debug dump

* Fixed typo in debugDump information
2018-04-24 13:26:21 +02:00
Michael Hackstein a8e737f16f
Feature/debugdump ui (#5187)
* Adeded a route to aardvark to export a debug dump, it can optinally contain anonymized example data

* Added the Download Debug Button to UI

* fixed zip file download of query dump file

* removed not needed vars

* Added JWT token authentication and error display on debug dump button

* Fixed Error reporting on download debug dump.

* Anonymization does not anonymize system attributes any more.

* Anonymize in explainer debugDump will now anonymize bindVars and will not anonymize system document attributes.
2018-04-24 09:43:04 +02:00
Simon 45fbed497b Supervision Job for Active Failover (#5066) 2018-04-23 12:49:41 +02:00
Alan Plum 41e9d6e585 Add @arangodb/locals module (#5145)
* Add @arangodb/locals module

Extending Node's module object in TS is tricky so let's provide a fake module for compatibility.

* Add testcase
2018-04-20 12:51:33 +02:00
Simon 8be273efb8 Replication cleanup (#5105) 2018-04-17 08:17:42 +02:00
Jan 2eb6d719ca
added ArangoShell helper function for packaging all information about an AQL query so it can be run and analyzed elsewhere: (#5023)
query = "FOR doc IN @@collection FILTER doc.value > @value RETURN doc";
    bind = { value: 42, "@collection": "mycollection" };
    options = { examples: 10, anonymize: true };
    require("@arangodb/aql/explainer").debugDump("/tmp/query-debug-info", query, bind, options);

Entitled users can send the generated file to the ArangoDB support to facilitate
reproduction and debugging.

The data from the generated file can be restored and analyzed via the *inspectDump*
function:

    require("@arangodb/aql/explainer").inspectDump("/tmp/query-debug-info");
2018-04-05 14:06:59 +02:00
Jan 167dfcdf7f
Feature/index only scans (#5005) 2018-04-03 17:34:28 +02:00
Jan c03bb12216
remove dysfunctional contrib files (#4976) 2018-04-03 17:32:13 +02:00
Jan 11cfbf5039
Feature/optimize cluster collect (#4897) 2018-04-03 17:28:07 +02:00
Max Neunhoeffer c7b77076a9
Merge remote-tracking branch 'origin/devel' into feature/arangosearch-cluster-views 2018-03-28 23:26:48 +02:00
Dan Larkin-York 38f162e344 Parallelize arangodump (#4356) 2018-03-26 15:55:14 +02:00
Max Neunhoeffer 790824fd68
Merge remote-tracking branch 'origin/devel' into feature/arangosearch-cluster-views 2018-03-26 10:50:23 +02:00
Wilfried Goesgens a16d4b0109 Feature/cpp aql reverse (#4911) 2018-03-23 09:54:37 +01:00
Andrey Abramov 294c36f778 use ERROR_ARANGO_DATA_SOURCE_NOT_FOUND instead of ERROR_ARANGO_COLLECTION_NOT_FOUND/ERROR_ARANGO_VIEW_NOT_FOUND in js part too 2018-03-19 13:55:25 +03:00
Max Neunhoeffer 0f46598200
Merge remote-tracking branch 'origin/devel' into feature/arangosearch-cluster-views 2018-03-14 23:24:41 +01:00
Max Neunhoeffer ce8db24975
Add methods in ClusterInfo to create and drop views. 2018-03-14 23:22:44 +01:00
Jan f1a2acebfb remove useless flush from test, bump wal flush default timeouts (#4845) 2018-03-14 21:59:48 +01:00
Wilfried Goesgens 33bc2f704f Doc - AQL improve examples (#4788) 2018-03-08 20:25:56 +01:00
Michael Hackstein 6f42d062be Fixed false http error codes in timeout case. Also reactivated retry in error case. (#4760) (#4770) 2018-03-08 09:19:16 +01:00
Simon 102998dd65 Streaming Query Cursor (#4769) 2018-03-08 09:18:41 +01:00
Andrey Abramov 29b7a7ea44 remove logger view (#4764) 2018-03-08 09:18:21 +01:00
Heiko a9ce6f8d9a fixed wrong behaviour when updatating foxx repository was not successful (#4763) 2018-03-06 12:07:45 +01:00
Wilfried Goesgens c7a9b41cf5 Doc - Interactively generated AQL examples (#4757)
* interactively generated AQL examples
2018-03-05 19:17:16 +01:00
Heiko b6df96fcc1 Feature/foxx store (#4600)
Vastly improved the Foxx Store UI
2018-02-28 14:39:36 +01:00
Simon 345fc3c0b7 Refactor Authentication Layer (devel) (#4592)
* Cherry Picking LDAP changes

* Adding missing merges

* Fixing remaining mentions of FeatureCacheFeature

* Fix jslint

* Fixing some failed tests

* Fixing cluster authentication issue, red tests

* Fixing ldap testsuite, adding trace logging

* Fixint ldap tesuite setup and LDAP recognition

(cherry picked from commit 686d28a779)

* Fixing wrong assert

* Adding changelog entry, making requested changes from code review

* Fixing dump_authentication, fix typos

* improvements found during code review

* oops

* more use of sessionstorage

* fix tests

* Fixing broken handling, disallowing adding of local users when disabled

* Fixing testInvalidGrants

* Removing undefined auth level externally

* Fixing previous commit

* added tests for ldap search mode

* intentionally removed `after` methods from tests

because they are executed before the tests start
no cleanup is performed right now after the authentication tests
however, a cleanup is done at start of every test

* ldap tests all modes

* forward port changes from 3.3

* added generated files

* forward port missing changes for web UI

* added generated files

* added generated files
2018-02-28 13:24:28 +01:00
Wilfried Goesgens b8e55ef022 fix https://github.com/arangodb/arangojs/issues/508 (#4664) 2018-02-27 14:54:51 +01:00
Jan Christoph Uhde 586a66ebbf Fix: #4583 - adds AQL ASSERT and WARN (#4584) 2018-02-26 14:25:24 +01:00
Wilfried Goesgens 619155a62d Feature/aqlfunction to cpp (#4573) 2018-02-13 11:50:49 +01:00
Dan Larkin 934e936bae Fix index selectivity estimate issues with RocksDB engine (#4429) 2018-02-09 16:05:33 +01:00
Alan Plum b00d169bcf More Foxx API tests (#4516) 2018-02-09 15:55:42 +01:00
Jan 42dd337152
allow using `returnOld` and `returnNew` attributes in HTTP REST APIs … (#4479) 2018-02-09 14:50:56 +01:00
Mark 599da158b5 Bugfix Foxx API tests (#4446) 2018-02-05 11:31:27 +01:00
Alan Plum 58cfd340cc Don't greedily delete legit app folders (#4437)
See #4395
2018-02-05 11:30:05 +01:00
Dan Larkin a25384fe6c Rename -iresearch files to -arangosearch for proper test selection. (#4386) 2018-01-31 17:25:59 +01:00
Jan fe0fca9029
Bug fix/restore unlock (#4387) 2018-01-25 15:56:27 +01:00
Wilfried Goesgens 9f7f03af44 Fix output of arangosh when run in testing.js (#4411) 2018-01-24 12:28:19 +01:00
Simon 3de120d915 Doc - Move cursor documentation out of deprecation folder (#4353) 2018-01-23 11:01:29 +00:00
Jan 41f1b31e58
added test for issue #4308 (#4372) 2018-01-22 09:22:48 +01:00
jsteemann 6bbdafb31d added test case 2018-01-19 15:43:30 +01:00
Andrey Abramov a1cfb3d72b Feature iresearch (#4105) 2018-01-19 14:23:58 +01:00
Michael Hackstein 40fa74f626
Bug fix/selectivity estimates on abort (#4337) 2018-01-16 18:21:55 +01:00
Michael Hackstein 4a5301fbf9
Bug fix/rocksdb truncate (#4328) 2018-01-16 15:21:43 +01:00
Michael Hackstein 802fda8e10 Revert "Bug fix/selectivity estimates on abort (#4314)"
This reverts commit d32f3b5fe9.
2018-01-16 14:39:02 +01:00
Michael Hackstein d32f3b5fe9
Bug fix/selectivity estimates on abort (#4314) 2018-01-16 12:39:03 +01:00
Michael Hackstein d32ac3b9c8
Bug fix/rocksdb truncate (#4060) 2018-01-16 09:01:04 +01:00
Jan 37da067059
fix missing events in RocksDB-based replication for transactions that… (#4277) 2018-01-16 00:27:11 +01:00
Jan Christoph Uhde 9c2ebb7bfc remove code that potentially throws from ~WBReader (#4282) 2018-01-15 15:14:55 +01:00
Jan 19ab240bf8
add wait to test (#4224) 2018-01-05 14:53:35 +01:00
Jan 6cb79b164e
change message from "no leader" to "not a leader" (#4134) 2018-01-05 14:50:15 +01:00
Jan bbfb8238b1
clean up generator code (#4191) 2018-01-02 15:26:06 +01:00
Jan 6040b4ae9f
add ignore-missing option to arangoimport (#4129) 2017-12-22 11:29:10 +01:00
Jan 2a96df5ca5
Feature iresearch (#4071) 2017-12-18 15:04:59 +01:00
Jan f5d71f42d2
added tests and updated documentation for Foxx queues (#3861) 2017-12-08 15:52:45 +01:00
jsteemann a005aca3e7 fix non-deterministic test 2017-11-30 09:53:25 +01:00
m0ppers ce537af092 Broken client API and server default was wrong (#3748) 2017-11-21 17:19:14 +01:00
m0ppers 75e1bf31cd Feature/allow access to configured 🔐 in readonly mode (#3698)
* Special method to access configured access levels as opposed to only getting the effective access level
* Add integration test
* Allow grants on collection level
2017-11-17 16:11:22 +01:00
Jan 4a199e2fe3 make truncate a bit less obstrusive (#3721) 2017-11-16 20:28:33 +01:00
Jan 5abf0c1185 Bug fix/fixes 1511 (#3711) 2017-11-16 14:18:51 +01:00
jsteemann 020e1b5b69 fix another test 2017-11-16 11:13:33 +01:00
jsteemann e3ecf833ce fix test 2017-11-15 13:15:57 +01:00
Jan 15ad2fbd99 do not complain about missing datafile statistics (#3688) 2017-11-14 16:42:31 +01:00
jsteemann caa5145482 fix failing test 2017-11-13 21:18:41 +01:00
Jan 0195ede1d8 potential fix for issue #3581: Unexpected "rocksdb unique constraint violated" with unique hash index (#3596) 2017-11-07 10:42:41 +01:00
Simon Grätzer ee8209943f Missing things for active / passive (#3578)
* Switching from ttl to supervision based failover mechanism

* Allowing canceling of ongoing actions

* refactored asyncjobmanager

* refactoring some code

* adding read-only flag

* catching some exceptions to reduce log pollution, removing unnecessary code, removing tests for _changeMode

* fixing "createsANewDatabaseWithAnInvalidUser"

* auth = off does not longer make everyone superuser

* Fixing cluster_sync and maybe resilience
2017-11-04 20:30:23 +01:00
Simon Grätzer 64e9377c05 Replacing /_api/collection with RestHandler (#3543) 2017-11-02 14:57:17 +01:00
Max Neunhöffer 5e2f17c04f Fast path for get-in-sync of empty shards. (#3470)
* Fast path for get-in-sync of empty shards.

* Fix short cut in synchronization.
2017-10-26 09:55:00 +02:00
Jan df5ea8ee2b try to fix another permissions problem when collections are accessed … (#3485)
* try to fix another permissions problem when collections are accessed by id

* added tests
2017-10-25 18:05:17 +02:00
Simon Grätzer f8260d20eb Fixing permission problem in createQueues (#3483)
* Fixing permission problem in createQueues

* Fixing queues test

* fixing jslint
2017-10-25 18:04:55 +02:00
Kaveh Vahedipour b1cf8903d4 test babies api in cluster with replicationFactor and numberOfShards set to 2 (#3457) 2017-10-18 23:59:53 +02:00
Simon Grätzer 7c31960cf2 Feature/async failover (#3451) 2017-10-18 23:59:29 +02:00
Frank Celler f240b8bc16 Feature/roles (#3364) 2017-10-03 10:31:18 +02:00
Frank Celler 70e206fc25 terminal codes 2017-09-28 22:23:54 +02:00
Jan 5165155ed1 Bug fix/fixes 0609 (#3227)
* do not use V8 variant of AQL functions in early optimization stage when a C++ variant is available

* additionally, simplify AQL function definitions and aliases

* warn when more than 90% of max mappings are in use

* added C++ variant of replication catchup

* added `--log.role` option

* updated CHANGELOG

* removed non-existing scheduler.threads option from config

* removed useless __FILE__, __LINE__ invocations

* updated CHANGELOG

* allow a priority V8 context

* remove TRI_CORE_MEM_ZONE

* try to fix Windows errors & warnings

* cleanup

* removed memory zones altogether

* exclude system collections from collection tests
2017-09-13 16:28:21 +02:00
Jan e1a1427c1e remove shell-database.js from git again, and apply modifications to shell-database.js.in (#3248) 2017-09-13 09:49:05 +02:00
jsteemann f8d8a561a4 fix various issues 2017-09-13 09:03:28 +02:00
Wilfried Goesgens c18d35dbf0 fix cleanup 2017-09-12 19:32:34 +02:00
Wilfried Goesgens 0a9b9e6025 cleanup after testrun, lintify 2017-09-12 14:48:20 +02:00
Wilfried Goesgens 1f9b322684 fix cleanup 2017-09-11 18:38:26 +02:00
Michael Hackstein da4d7966a8 Bug fix/truncate geo (#3173)
* Add geo truncate test

* added a specific truncate method to rocksdb geo index

* Fixing truncate on rocksdb geo-indexes with support for rollbacks
2017-09-04 17:48:04 +02:00
Simon Grätzer 88d01b89b5 Optimizations for Caches and Graph Traversals (#3169) 2017-08-31 18:33:10 +02:00
Wilfried Goesgens c662dfc863 Cleanup after each testrun (#3109)
* Cleanup after each testrun

 - add a success testsuite next to the fail testsuite
 - create temporary directories in the fail & success testsuites
 - add a function to cleanup the last unshifted directory
 - don't say we're crashed when exiting with the help message

* Specify temp path for SUT so its files will be removed after cleanup.

 - cleanup
 - don't use getTempFile(), it will create another subdirectory that we don't clean up afterwards...

* Fix temp path usage

 - creating a 'bundles' directory wherein we would keep our temporary bundle files would collide if we'd clean it up.
 - create 'bundles-tmpxxxx' instead so its individual to us and we can remove it when done without problems.

* properly create & pass in a temporary directory - as we now require this.

* Remove intermediate directory recursive - for some unknown reason there may be temp files created.

* we need to unify tempfilenames here too.

* use individual suite names

* use the standart performTests to run the backup tests - so we have the proper directory and server handling from the infrastructure.

* fix temporary directory handling in the recovery tests.

* script to build debugging debian packages

* Improve test shutdown

 - handle a set of options globally per testsuite, so they don't need to clone them inside.
 - properly cleanup the slave instance of the replication tests after each run.
 - The per testsuite options vector will fix the behaviour that after one failed testsuite
   all other suites won't be cleaned up anymore.

* don't call the overal cleanup if anything failed.

* fix cleaning up of directories.

* cleanup, fix directory creation / deletion / dummy creation.

* cleanup, fix directory creation / deletion / dummy creation.

* fix tempdir handling for the recovery tests
2017-08-30 10:48:24 +02:00
Jan Christoph Uhde 20e1a673fd Feature/planning 499 different error codes for version check (#3118)
* return different error codes for check-version

* Adds data file for exit codes that can be used by ArangoDB applications

* add some exit codes

* better return codes for centos

* add codes in arangod

* remove test code

* prevent re-downloading of starter and other small improvements

* fix some erros

* finish centos

* update urls

* add info about systemd detection

* move further towards a working systemd setup

* set service type to simple

* add arangodb-update-db script

* start work on debian packages

* a db dir that is already in place (former installation) should be reused now

* fix minor bugs

* Implement exit code translation in the windows installer.

* add autogenerated NSIS exit code mapping
2017-08-30 10:47:31 +02:00
Kaveh Vahedipour 00650e6a3f Bug fix/agency mt fixes (#3158)
* added debugging methods

* try to fix invalid access in case of error

* remove unused members

* bugfixes and comments

* all agency fixes in

* merge bug

* partially unguarded Agent::lead fixed

* all agency fixes in

* added nrBlocked to thread startup eval

* added nrBlocked to thread startup eval

* recombination of cases in State::get

* some maps replaced with unordered_maps

* optimized maps some
2017-08-30 10:43:51 +02:00
Michael Hackstein 0f1b6e32eb Bug fix/issue 3106 orphans in general graph (#3147)
* Added a test-case for #3106

* Added a fix for #3106
2017-08-28 15:02:15 +02:00
Jan 5c5b5e548d Feature/unique vpack index lookup performance improvements (#3087)
* fixed issue #3075

* update the known issues document, as some issues have already been fixed

* speed up equality lookups in unique vpack indexes

* fixed wording
2017-08-24 14:39:09 +02:00
Simon Grätzer a14ffcd26e Relaxing edge cache test asserts (#3103)
* fixing cache resizing bug

* reenabling test

* relaxing test asserts
2017-08-22 15:52:48 +02:00
Frank Celler 536625a50a Bug fix/foxx queue registration (#3097)
* Fix Foxx queue registration

Multiple registration of the same queue was possible. The queue document in _queue was saved multiple times with the same key/id

* Add additional foxx queue test
2017-08-22 11:56:07 +02:00
Simon Grätzer b76b53d87a Fix edge cache behaviour (#3079)
* fixing cache resizing bug

* reenabling test
2017-08-22 10:40:52 +02:00
m0ppers 0dfea58d54 Feature/remove manual zippery (#3036) 2017-08-17 23:30:27 +02:00
Jan 8e4dac4fc4 fix fulltext index removal performance, simplified code (#3015)
* simplify index API a bit

* fix fulltext index removal performance, simplified code

* updated CHANGELOG

* fix hanging test

* try to fix shutdown problem

* improve fulltext query performance

* fixed duplicate var

* removed obsolete code

* fix some shutdown races

* do not call ensureIndex that often
2017-08-12 09:25:30 +02:00
Jan e6c6c5acda Bug fix/pre release woes (#2988)
* fixed issue #2982

* updated CHANGELOG
2017-08-10 09:39:28 +02:00
Jan 0238d651fb Bug fix/3.2.1 issues (#3009)
* fix documentation

* fix crash in explain
2017-08-09 16:49:24 +02:00
Michael Hackstein 7ab117f5f8 Bug fix/dump restore auth (#2917)
* Added a backup test suite. This suite is supposed to entirely drop an ArangoDB _system and restore it into a fresh one. This also includes system collections.

* Added more test cases for backup suite. Now tests several authorization/rights cenarios

* Fixed RestReplication Handlers to restore _user collections Properly.

* Updated Changelog

* Added special handling of _users in Restore for MMFiles as well.

* Added JWT secret for cluster execution of this test, also added JWT secret to shutdown call
2017-08-09 12:59:51 +02:00
Jan bfa1051345 Feature/slow queries log bind vars (#2954)
* add bind variables when logging slow queries, and make this configurable

* added bind variable tracking
2017-08-04 20:45:12 +02:00
Jan 40c5b94d68 Bug fix/nightly fixes (#2946)
* fix internal documentation (thanks @Simran-B)

* fixed issue #2810

* potential fix for issue #2939

* updated CHANGELOG

* updated CHANGELOG
2017-08-04 13:34:12 +02:00
Jan a10832d41e do not allow the creation of volatile collections in the RocksDB stor… (#2900)
* do not allow the creation of volatile collections in the RocksDB storage engine

* only create a volatile collection for mmfiles in test

* fix tests
2017-08-03 22:45:17 +02:00
Simon Grätzer 3f58a44e3b Fix cases where the root user has access to excluded collections (#2856) 2017-08-03 21:16:08 +02:00
Kaveh Vahedipour 0b6d6d9287 Fixed distributeShardsLike bug in creating collections. numberOfShard… (#2895) 2017-08-03 19:38:16 +02:00
Jan Christoph Uhde ed8efe3566 Feature/issue 387 cluster index estimates (#2866) 2017-08-01 09:53:58 +02:00
Jan 6820ec3891 fixed issue #2868: cname missing from logger-follow results in rocksdb (#2901) 2017-07-28 18:52:30 +02:00
m0ppers 6fefe5ab31 In high concurrency situations double checking checksums is a must have (#2890) 2017-07-28 00:43:21 +02:00
Simon Grätzer 9786b198c2 Allow pregel jobs to interrupt on shutdown (#2855) 2017-07-27 15:29:01 +02:00
Wilfried Goesgens 7e03187123 When a transaction aborts, add the error messages to the returned sta… (#2848)
* When a transaction aborts, add the error messages to the returned status to aid the development process (as suggested by @DeShadow)

* If we are an arangoerror log out the ArangoError code

* don't send the stacktraces via the 'net, but have better error messages anyway

* static is wrong here - we want to use it with per threads individual values now.

* restructure source just to have one stacktrace processing function.

* Pull array from javascript logging functions up to c++

* js_log now can handle an array of v8-strings, converts it to a vector of std::strings, feeds them one by one into logging for now.

* the stacktraces now don't do individual calls to js_log, but one with an array of log lines.
2017-07-27 12:01:30 +02:00
Heiko 1bb9265ee3 fixed general graph module when options attribute is missing (#2842) 2017-07-19 18:18:30 +02:00
Heiko d5e85bf987 general graph module and UI now supporting sharded and replicated graph (#2825) 2017-07-19 12:49:04 +02:00
Frank Celler a5a25754ed Feature/reduce extraction to projection (#2792)
* reduce extractions to projections

* recycle string buffers in SocketTask

* micro optimizations for mmfiles indexes

* added special lookup function for _key

* moved function into the correct file

* speed up key buffer allocations a bit

* added noexcept specifier

* correctly name variable

* explicitly move bounds

* fix and speedup from/toPersistent functions

* reuse string from ManagedDocumentResult for multiple lookups

* use move-assign

* a bit less work for single server

* speedup AQL function HASH

* single fetch optimization

* performance optimization for the case when no documents need to be returned

* make reduce-extraction-to-projection a RocksDB-only optimizer rule

* cppcheck

* try to fix compile error on MacOS

* bug fix for MacOSX

* missing namespace (in Windows compile)
2017-07-14 08:40:29 +02:00
Max Neunhöffer 2f874249bb Bug fix/adjust agency comm timeouts (#2765)
* Take out 503 timeouts altogether.
* Overhaul of AgencyComm::sendWithFailover loop.
* Let performRequests optionally ignore 404 coll not found.
* Fix error message "database not found" when AgencyComm failed.
* Add log entries in Agency if locks are acquired too slowly.
* Reexecute the javascript cluster sync stuff even if there was no plan/current change...So failed sync jobs can retry later...
* Cover callbacks in Communicator by lock. This fixes https://github.com/arangodb/planning/issues/370
* Put in delay in waiting for leader in agency test.
* Schmutz logging to heartbeat topic.
* Add more lock time diagnostic in agent.
* Switch on agencycomm tracing in coordinator.
2017-07-13 00:44:28 +02:00
Frank Celler 40d73d5a8b Revert "[WIP] Feature/reduce extraction to projection (#2735)"
This reverts commit 5bfcff30cd.
2017-07-12 12:52:14 +02:00
Jan 5bfcff30cd [WIP] Feature/reduce extraction to projection (#2735)
* reduce extractions to projections

* recycle string buffers in SocketTask

* micro optimizations for mmfiles indexes

* added special lookup function for _key

* moved function into the correct file

* speed up key buffer allocations a bit

* added noexcept specifier

* correctly name variable

* explicitly move bounds

* fix and speedup from/toPersistent functions

* reuse string from ManagedDocumentResult for multiple lookups

* use move-assign

* a bit less work for single server

* speedup AQL function HASH

* single fetch optimization

* performance optimization for the case when no documents need to be returned

* make reduce-extraction-to-projection a RocksDB-only optimizer rule
2017-07-12 11:22:29 +02:00
Frank Celler 545e861829 Bug fix/agency prepare leading bug (#2752) 2017-07-08 17:08:30 +02:00
Jan Christoph Uhde 562cc77b59 Bug fix/issue 2499 insert 9 instead of 1 (#2727)
* Add testcase to demonstrate #2499

  - In subqueries inserts get pushed to the DB-Servers

  - these create documents documents on each DB-Server that don't respect the shard-key
  
- subsequent deletes will fail to detect the shard correctly and will also fail.

* fix for #2499
2017-07-06 09:53:39 +02:00
Jan 27dc53b451 Bug fix/cleanup stuff (#2734)
* cppcheck

* remove unused files

* cppcheck just sucks

* add _FORTIFY_SOURCE
2017-07-05 19:34:16 +02:00
Jan d334455bbf remove a non-deterministic part of a test (#2730) 2017-07-04 17:33:31 +02:00
Jan a5afdf9437 Bug fix/small issues (#2696)
* remove unused code

* print values of all options

* don't print "warning(s):" if there are none

* remove remainders of old file deletion functionality

* remove unused function
2017-07-01 00:30:45 +02:00
Jan 94b4a9ec4b added "deduplicate" attribute for array indexes (#2644) 2017-06-30 17:53:50 +02:00
Mark 2ba5ee87c2 Fix issue #2677 (#2678)
Foxx disabling development mode creates non-deterministic service bundle
2017-06-27 16:38:42 +02:00
Frank Celler 2807ef559c Feature/move shard fix (#2626)
Major overhaul of handling of synchronous replication.
2017-06-26 16:55:01 +02:00
Mark 50ad6fbd76 Foxx service CRUD optimizations (#2657)
* Pass config/deps on _install so setup can see them

* Only register service when install succeeded

* Don't throw after point of no return: breaky install is breaky

* Delete service files during install if service validation fail

* Delete service files during install if service execution fails

* Fix commitLocalState with invalide services

* Remove deprecated test
2017-06-26 09:28:59 +02:00
Mark 7743db32b9 Fix #2613 (#2614)
Reduce log level when Foxx manager tries to self heal missing database
2017-06-20 12:41:45 +02:00
Jan 9bebd33ba3 fix invalid first group results for sorted AQL COLLECT when LIMIT was used (#2577) 2017-06-16 09:44:14 +02:00
Michael Hackstein 830769dcb5 Fixed test file still calling warmup 2017-06-12 23:46:43 +02:00
Michael Hackstein 9edb884bc8 Feature/planning issue 188 (#2547)
* add warmup documentation - #188

* warn when relinking build directory

* add warmup documentation - #188

* warn when relinking build directory

* Renamed warmup to loadIndexesInMemory which is a better name for the current implementation of this feature.

* Adapted WebUI to state 'load indexes in memory' instead of 'warmup'

* Added loadIndexesInMemory documentation.

* Renamed loadIndexesInMemory => loadIndexesIntoMemory
2017-06-12 18:54:45 +02:00
Max Neunhoeffer 29921d32a8 Fix behaviour of synchronous replication in dropFollower case.
If we are already in shutdown, we do not drop a follower.
If we cannot drop a follower (no contact to agency), we error out.
2017-06-09 13:06:02 +02:00
Alan Plum 57487d6f4d Expose internal.time (#2530) 2017-06-09 10:44:29 +02:00
Andreas Streichardt 7250ba29ee Disable communicator and abort all requests when shutting down 2017-06-08 11:58:15 +02:00
Michael Hackstein db0dcd5579 Reverted API name change on _getConnectingEdges 2017-06-07 11:13:52 +02:00
Kaveh Vahedipour da0cc3490c Squashed commit of the following:
commit 3d9cf792912db1974b9ac5e00ca2b4c9245b7d34
Author: Kaveh Vahedipour <kaveh@vahedipour.de>
Date:   Fri Jun 2 15:32:43 2017 +0200

    optimise for single writes in agency log

commit 65056ab9026f9b4b211dda0f17c75602b978f2bf
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Fri Jun 2 15:01:15 2017 +0200

    More tests, taking agency log compaction interval into account.

commit 6600d707784e8fd5b62c0c75fd1826af09b8e13f
Merge: cf46882 02f00cc
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Fri Jun 2 14:50:38 2017 +0200

    Merge branch 'agency-log-compaction-overhaul' of ssh://github.com/ArangoDB/ArangoDB into agency-log-compaction-overhaul

commit 02f00cc271d027f02b0625afb76745bfa76bf833
Author: Kaveh Vahedipour <kaveh@vahedipour.de>
Date:   Fri Jun 2 14:34:41 2017 +0200

    compaction step and keep size defaults for 3.2

commit 03fc8fbff8f0ac701f7d7f94521c0c3152dd6f92
Author: Kaveh Vahedipour <kaveh@vahedipour.de>
Date:   Fri Jun 2 14:32:46 2017 +0200

    Constituent fatally exists if eletion ballot cannot be persisted

commit cf4688226fc897e74bb2d9ebdfca3ce4578c3b70
Merge: c727fc4 724bd1e
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Fri Jun 2 13:08:15 2017 +0200

    Merge branch 'agency-log-compaction-overhaul' of ssh://github.com/ArangoDB/ArangoDB into agency-log-compaction-overhaul

commit 724bd1efe19e2e9dbfc14cd819f180816b6d62d0
Author: Kaveh Vahedipour <kaveh@vahedipour.de>
Date:   Fri Jun 2 13:02:51 2017 +0200

    persistence success in agency state is properly evaluated

commit c727fc48bb93e7b135b3ca929c03221c7bcaddb9
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Fri Jun 2 12:04:55 2017 +0200

    Set default compaction step size in agency to 20000 and 10000 keep size.

commit ded16ae6945e9c1479e99bc2e7ccb4d6feca19a6
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Fri Jun 2 11:11:12 2017 +0200

    Fix help page in startStandAloneAgency.sh for --use-persistence.

commit 13ae9f40f649a8f92eeca4b16bbb5647b540722d
Merge: 834c7c9 aa3e8c1
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu Jun 1 23:41:34 2017 +0200

    Merge remote-tracking branch 'origin/devel' into agency-log-compaction-overhaul

commit 834c7c920d36db3579def66c38fb04870936f8bd
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu Jun 1 16:56:03 2017 +0200

    Handle error in recvAppendEntriesRPC properly.

commit bd9c8d03b76ad25d4078740b5bf994fdba3845d0
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu Jun 1 16:55:35 2017 +0200

    Handle errors in persist() and log() properly.

commit 5b4d2c3d9af078d6a1b5626af20dc9abf2546baa
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu Jun 1 16:25:22 2017 +0200

    Improve error reporting.

commit d60697c5f26d6592eecefc9b9a43e9b699d1773d
Author: Kaveh Vahedipour <kaveh@vahedipour.de>
Date:   Thu Jun 1 12:16:39 2017 +0200

    Agency must not responds to any requests after startup util leader has RAFT commited up to pre shutdown state

commit 92b8ede5fa022ace1596607abcf8fad1130504c8
Merge: 9340e74 d24455c
Author: Kaveh Vahedipour <kaveh@vahedipour.de>
Date:   Wed May 31 16:54:45 2017 +0200

    Merge remote-tracking branch 'origin/devel' into agency-log-compaction-overhaul

commit 9340e7461130a4783c09ad8d91e5a07f9500a045
Merge: 7b7ce9d 63a9d60
Author: Kaveh Vahedipour <kaveh@vahedipour.de>
Date:   Wed May 31 12:13:55 2017 +0200

    agency tests to cover compaction

commit 63a9d604c474eda4302032629dff1f0f69fa0813
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Wed May 31 11:59:11 2017 +0200

    Set agency.compaction-keep-size to at least 0.

commit ef842260968a4769d9502a701b7251da32647e52
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Wed May 31 11:49:34 2017 +0200

    Fix agent size 1 case (thread already gone).

commit 7b7ce9d79f6e8208c13f153b1b9a395b780d6ce1
Merge: 24e2e7e ff306bf
Author: Kaveh Vahedipour <kaveh@vahedipour.de>
Date:   Wed May 31 11:39:58 2017 +0200

    Merge branch 'agency-log-compaction-overhaul' of https://github.com/arangodb/arangodb into agency-log-compaction-overhaul

commit ff306bf547bc4f528c9b66e222271ac143029508
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Wed May 31 11:11:06 2017 +0200

    Move compaction into the future when we take a snapshot from leader.

commit 24e2e7e00f960928a79ce4008b8031d6b9b07fd9
Merge: 84034ac b3ea17a
Author: Kaveh Vahedipour <kaveh@vahedipour.de>
Date:   Wed May 31 11:01:13 2017 +0200

    Merge branch 'agency-log-compaction-overhaul' of https://github.com/arangodb/arangodb into agency-log-compaction-overhaul

commit b3ea17a219baa2abd5892819012fb59f440cdeb8
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Wed May 31 09:42:59 2017 +0200

    Get rid of double nonsense.

commit 035c8d1b34e1b73a381d5468422adf13b2ebc36a
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Wed May 31 09:25:28 2017 +0200

    Sort out Agent::load sequence.

commit 84034ac2809a77145d6b1d23bf44857b3a0c4651
Merge: eb34a2e 3180a9d
Author: Kaveh Vahedipour <kaveh@vahedipour.de>
Date:   Tue May 30 17:07:20 2017 +0200

    merging in

commit eb34a2e64e6ac8dc6571b92cb853c38b7022c833
Author: Kaveh Vahedipour <kaveh@vahedipour.de>
Date:   Tue May 30 16:58:05 2017 +0200

    keep persistence for restarts in standalone agency

commit 3180a9d9ce4a4401a55ef02606b020316d43cbe5
Merge: 5d60524 28b9580
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Tue May 30 16:56:56 2017 +0200

    Merge branch 'agency-log-compaction-overhaul' of ssh://github.com/ArangoDB/ArangoDB into agency-log-compaction-overhaul

commit 5d60524429d8ddda4491beecb931c3b9e3cc1d8a
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Tue May 30 16:56:36 2017 +0200

    Implement snapshot sending in AppendEntriesRCV.

commit 28b958054f51c9cb36706df4e4345aa0f726ed15
Author: Kaveh Vahedipour <kaveh@vahedipour.de>
Date:   Tue May 30 14:20:13 2017 +0200

    state machine should not advance _committed if empty

commit df18f326acea7f5bc2660a37e22f1503952e4b41
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Mon May 29 23:39:47 2017 +0200

    Store term with compaction snapshot, recover again.

commit 2551a48b6fb513c9ea934bce755f8c364dae2f05
Author: Kaveh Vahedipour <kaveh@vahedipour.de>
Date:   Mon May 29 17:45:26 2017 +0200

    indices renamed to closely match RAFT documentation

commit e62dcdecf6e8650cfa5725d91b809d05591b48a4
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Mon May 29 16:37:43 2017 +0200

    More cleanup.

commit 9f4787c46621375f0361138a8961431eb21ce5c0
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Mon May 29 14:50:25 2017 +0200

    Revise loadLastCompactedSnapshot to return 0 without a snapshot.

commit 13285e1d70c8a4ac8c79a08de6f8fbc0f8d242bf
Merge: 3393c43 6c5f23e
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Mon May 29 12:06:20 2017 +0200

    Merge branch 'agency-log-compaction-overhaul' of ssh://github.com/ArangoDB/ArangoDB into agency-log-compaction-overhaul

commit 3393c43c75520c74d20df09c74fbbbd8b1af5976
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Mon May 29 12:03:47 2017 +0200

    More cleanup around Store::apply and friends.

commit 4ccb41d1839748c98e11403fa04f6a7d6af5e95b
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Mon May 29 09:45:41 2017 +0200

    Document apply methods in Store with comments.

commit ea05c4880fedb6fe535e24761ac5cb3c26ccfc20
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Mon May 29 09:45:18 2017 +0200

    Intentionally keep one log entry more to prevent empty log.

commit 67fb62f2259cc3c6368319917c7257ebcc177d3f
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Mon May 29 09:44:42 2017 +0200

    Improve plausibility checks at compaction time.

commit 0bafc368785b15a94f8783c4c929f4208f87d09c
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu May 25 00:51:51 2017 +0200

    Sort out (re-)building of agency K/V store(s) from compaction snapshots.

    This is in case of (re-)start, becoming a leader and when serving
    /_api/agency/store.

commit 46b0750bc6c597ec388aac0cdca32082c0cc54b8
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu May 25 00:50:54 2017 +0200

    Set compaction interval to 200 and keep to 0.

    This is way to small but tests should run with it.
    Will later increase numbers again.

commit 024dc0846ae30248b464dd481a8bbc1134f56983
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Wed May 24 23:29:57 2017 +0200

    Add a trivial test for agency log compaction.

commit e12fd3b46833419d7b436eeadd7246304324b891
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Wed May 24 23:26:03 2017 +0200

    First part of cleanup of agency log compaction.

    Now the right compaction snapshots are taken and persisted.
    Furthermore, the right log entries and old snapshots are removed
    after compaction, both the volatile and the persisted ones.
    The readDB and spearHead stay unchanged at compaction time as it
    should be.

commit d59901aea0c3ca31ef253299d2adc3353b79e664
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Wed May 24 12:18:26 2017 +0200

    Remove unused member variable.

commit 6c5f23eb7b42d9f20d4dadb2932a63add99f9c76
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Mon May 29 09:45:41 2017 +0200

    Document apply methods in Store with comments.

commit 670899f72d215e0fcc0ca0389cea9250a291e83b
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Mon May 29 09:45:18 2017 +0200

    Intentionally keep one log entry more to prevent empty log.

commit 660f61029917bbc2ce1fae3e4fc903095b023297
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Mon May 29 09:44:42 2017 +0200

    Improve plausibility checks at compaction time.

commit e2802e4b36d1f67d8361c1d8b0c92fbff696f439
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu May 25 00:51:51 2017 +0200

    Sort out (re-)building of agency K/V store(s) from compaction snapshots.

    This is in case of (re-)start, becoming a leader and when serving
    /_api/agency/store.

commit 12b43f1b91284a1185390d6dcfbd1e838522d392
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Thu May 25 00:50:54 2017 +0200

    Set compaction interval to 200 and keep to 0.

    This is way to small but tests should run with it.
    Will later increase numbers again.

commit c8b9a37a690b8e7e8bfa1276a3f9ba4b6b5a9c27
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Wed May 24 23:29:57 2017 +0200

    Add a trivial test for agency log compaction.

commit cf0c8c1fff666f76411082f87efe685a412ecebb
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Wed May 24 23:26:03 2017 +0200

    First part of cleanup of agency log compaction.

    Now the right compaction snapshots are taken and persisted.
    Furthermore, the right log entries and old snapshots are removed
    after compaction, both the volatile and the persisted ones.
    The readDB and spearHead stay unchanged at compaction time as it
    should be.

commit 0a4255359a57b8686133e6014e2b82b8079f36fa
Author: Max Neunhoeffer <max@arangodb.com>
Date:   Wed May 24 12:18:26 2017 +0200

    Remove unused member variable.
2017-06-02 16:13:03 +02:00
Simon Grätzer e5386aef30 fixing compiler warnings and edge cache test 2017-06-02 16:00:41 +02:00
Simon Grätzer 634254b80f adding load and unload 2017-06-02 14:44:38 +02:00
Alan Plum 7608cde674 Implement clusterfoxx mounting logic (#2514)
* Serve error when service is out of sync
* Don't check system services for bundles
* Use appropriate errorNum if Foxx blows up
* Only check checksum when file exists
* Don't expose stack traces
2017-05-31 18:21:40 +02:00
jsteemann 62c944d8a8 added more tests for transaction parameters 2017-05-31 17:28:28 +02:00
Andreas Streichardt dded5abb6c Merge remote-tracking branch 'origin' into shardorganizer 2017-05-30 20:17:07 +02:00
Alan Plum 23ac54183c
More meaningful object logging
Don't log inherited attributes.
Do log constructor name for class instances.
Use _PRINT methods if available except when using console.dir.
2017-05-30 20:09:58 +02:00
Alan Plum ab177612a5
Response -> IncomingResponse 2017-05-30 20:09:58 +02:00
Alan Plum cceccf59da Simplified Foxx self healing (#2511)
* Implement new self-heal
* Add error codes for 503, service missing/outdated
* Detect changes to service via rev
* Pretty print incoming response object in log
2017-05-30 18:27:32 +02:00
jsteemann 53b150b206 added tests for intermediate commits 2017-05-30 12:03:58 +02:00
jsteemann a35f85bcc7 added proper docs for --query.fail-on-warning 2017-05-29 15:46:48 +02:00
Michael Hackstein b9d2faa7cc Added warmup function for indexes.
Squashed commit of the following:

commit bc0472d212b2efef0d7b1b8a36f2b43e8432ba86
Merge: eb47631 5f87674
Author: Michael Hackstein <michael@arangodb.com>
Date:   Mon May 29 09:39:31 2017 +0200

    Merge branch 'devel' of github.com:arangodb/arangodb into feature/rocks-index-warmup

commit eb476310b0ca6165be10b37b960933886c2757f5
Merge: d725f21 32149d2
Author: Michael Hackstein <michael@arangodb.com>
Date:   Mon May 29 09:00:41 2017 +0200

    Merge branch 'devel' of github.com:arangodb/arangodb into feature/rocks-index-warmup

commit d725f21f7d61f7c79ba31fc0338881e35f4d8f48
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 15:53:23 2017 +0200

    Activated collection warmup in Cluster Mode. Added a test for collection warmup (SingleServerOnly)

commit 132bf4f9565b0dcf4ec9f84e93897b482a67ce7f
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 12:46:20 2017 +0200

    Implemented client-side warmup. It can now be triggered via http and arangosh

commit 78ea449dff86118814a2f87bdb59dc16544d92b6
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 11:25:40 2017 +0200

    Fixed assertion.

commit dae80f6277dde1a52eadda506858cc36e235bd55
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 11:08:46 2017 +0200

    Improved the CuckooIndexEstimator computeEstimate function. It is now much better on collections with many different values

commit 7abf57876511ba369d7a577e1995d4575e98c7c8
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 11:06:43 2017 +0200

    Edge index warmup will now first check if a document is stored and if not will insert it. Furthermore it resizes the cache with an estimate of how many documents will be most likely inserted

commit 890d8ad4cdfd155249f060fedd5c798b9531d556
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 11:04:47 2017 +0200

    Adjusted thresholds in transactional cache. Wastly increased the amount of tries to get the lock. Lowered the fill-grade boundaries

commit 60972ab7151a3acb78e1aae4149de11d0da7aceb
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 10:45:38 2017 +0200

    Added new Logtpopic CACHE that should be used to log important information regarding caches, like debug ouput on evictions / resizing, or memory-pressure resizing going on

commit 2dfacb9aef6e3fde169032514baca386786d059c
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 10:44:21 2017 +0200

    Fixed an assertion in index iterator. It expected _from/_to to be cached but we modified to cache entire document

commit f05beccef65853c30eda1859a39c28ea2295bc71
Merge: 13c6abf 5c46430
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 07:56:58 2017 +0200

    Merge branch 'devel' of github.com:arangodb/arangodb into feature/rocks-index-warmup

commit 13c6abfbc2ddb451a689011110411df68247435e
Author: Michael Hackstein <michael@arangodb.com>
Date:   Wed May 24 09:52:01 2017 +0200

    Added a getExtra feature for the edgeIndex. It now returns the complete edge as extra (temporary). Modified the EdgeCache to cache token and edge document (temporary). Added a warmup Function to collections that is used to warmup the index caches on demand.
2017-05-29 09:40:58 +02:00
Simon Grätzer 36503af230 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel
# Conflicts:
#	arangod/RocksDBEngine/RocksDBReplicationTailing.cpp
2017-05-24 18:38:34 +02:00
Simon Grätzer 2f2d07ab9a Multihreaded import 2017-05-24 18:37:45 +02:00
Andreas Streichardt b70644440b Do not allow creation of user collection when there are not enough
servers
2017-05-24 18:04:35 +02:00
jsteemann e946a0c0b4 added diagnostic output 2017-05-24 11:52:33 +02:00
mpv1989 379521de53 Add adler32 test 2017-05-24 09:09:08 +02:00
mpv1989 cdfd41f9b9 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2017-05-22 13:54:08 +02:00
mpv1989 66fa98e14d Add alder32 tests 2017-05-22 13:54:02 +02:00
jsteemann bbdb191797 remove unused error code 2017-05-22 13:53:10 +02:00
Michael Hackstein ae3a1d6dcc Merge branch 'devel' of github.com:arangodb/arangodb into devel 2017-05-18 10:07:50 +02:00
Michael Hackstein 592508125b Deactivated index cache test im cluster. It is not supposed to work 2017-05-18 10:05:39 +02:00
jsteemann b68482bcaa Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2017-05-18 00:17:55 +02:00
jsteemann 226920d7fa optimizations 2017-05-17 23:41:16 +02:00
Michael Hackstein 0d18821705 Adapted the shell-index-figures test to honor that the primary index does not have a cache anymore. 2017-05-17 18:52:09 +02:00
Jan Christoph Uhde 02da6fc4ca fix getIndex(withStats) for arangosh 2017-05-17 13:50:03 +02:00
Jan Christoph Uhde 55175386e7 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel
* 'devel' of https://github.com/arangodb/arangodb:
  Lint
  No more need to wrap legacy errors in new Foxx HTTP API
  fixed initializing bug in dashboard ui at initial startup
  Correctly expose errors in old Foxx API
  Fix error expecations
2017-05-17 13:49:45 +02:00
Alan Plum 30524bbb84
No more need to wrap legacy errors in new Foxx HTTP API 2017-05-17 13:22:11 +02:00
Alan Plum 599ed214c6
Fix error expecations 2017-05-17 12:33:41 +02:00
Jan Christoph Uhde 0423b75137 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel
* 'devel' of https://github.com/arangodb/arangodb:
  Throw correct errors in Foxx management internals
  Make build flag easier to find in docs
2017-05-17 12:23:23 +02:00
Jan Christoph Uhde 415cbfea60 fix figures and add tests 2017-05-17 12:22:43 +02:00
Alan Plum 819a1194ba
Throw correct errors in Foxx management internals 2017-05-17 12:22:16 +02:00
Michael Hackstein 6a2ae1af8c Merge branch 'devel' of github.com:arangodb/arangodb into devel 2017-05-17 09:55:35 +02:00
Michael Hackstein b19549f5e4 RocksDB now properly reports EdgeIndex estimates to the user. Also reactivated edgeIndex tests for the estimate in RocksDB. 2017-05-17 09:52:25 +02:00
Jan Christoph Uhde 1f4e90ec17 add test for index figures 2017-05-17 09:51:50 +02:00
Michael Hackstein 9569b67e8d Reactivated hash-index tests on rocksdb. Mostly checking estimates. 2017-05-17 09:14:06 +02:00
Alan Plum 5d017d733c Improve Foxx cluster resilience (#2479)
* Improve Foxx cluster resilience

Fixes #2083
Fixes #2384
Fixes #2408
Addresses #1892

* Port old Foxx API

* Implement single-file services

* Add console.errorStack/warnStack/infoStack helpers

* Simplify serviceInfo validation

* Extract github/upload logic into Aardvark and old FM API
* Move generator logic into Aardvark
* Move zip/js buffer logic into FM core

* Add Foxxmanager tests

* Send empty response when no README

* Disambiguate script arg format

Historically we allow passing an array of positional arguments or an arbitrary first argument.
This is surprising behaviour, so we should just always treat the value as a first argument.

* Rebuild bundle in development mode

* Nicer HTTP docs formatting

* Create Foxx HTTP docs

* Simplify service upload handling

* Remove inline swagger docs

* Implement public download route

* Consistency

* Rebuild aardvark

* Move bundle route into /_api/foxx/_local

* Rebuild Swagger API docs

* Add changes to CHANGELOG

* More docs
2017-05-16 17:41:29 +02:00
jsteemann 499491bee9 use exclusive locks for some transactions 2017-05-15 22:47:50 +02:00
Jan Christoph Uhde c673391387 fix jslint warnings 2017-05-15 17:00:35 +02:00
Jan Christoph Uhde de6a21f4f2 reduce number of log messages 2017-05-15 16:27:52 +02:00
Jan Christoph Uhde b09b84dcad add test for equality 2017-05-15 15:29:38 +02:00
Jan Christoph Uhde 2bc60c24bd fix jslint erros 2017-05-15 13:45:13 +02:00
Jan Christoph Uhde 22e2b0fc7e fix edge index test 2017-05-15 13:36:17 +02:00
Jan Christoph Uhde ea386b605f renable edge-index tests 2017-05-15 12:30:34 +02:00
Jan Christoph Uhde e5db579b25 Merge branch 'devel' of https://github.com/arangodb/arangodb into feature/edge-index-cache
* 'devel' of https://github.com/arangodb/arangodb: (54 commits)
  attempt to fix races in replication
  grunt build
  use exclusive transaction in syncer
  change feature order around
  dashboard
  added api cluster api routes
  add selection of storage engine
  use some parallelism
  micro optimizations
  Fix truncate for geo
  added assertions
  remove unused include
  try to fix initial sync problem
  remove unnecessary includes
  Finally fix the error where suddenly an array of dbservers are being called
  Update queryView.js
  initialize bool variable
  remove debug output
  removed unused private field
  Wrong index number
  ...
2017-05-10 15:14:08 +02:00
Jan Christoph Uhde 4f552e7985 reactivate index tests 2017-05-10 14:48:05 +02:00
Simon Grätzer 3b7ef439c8 Merge branch 'devel' of https://github.com/arangodb/arangodb into feature/rocksdb-geoindex 2017-05-09 17:42:27 +02:00
jsteemann e2039d587b added debug output in case tests fail 2017-05-09 11:24:41 +02:00
Simon Grätzer 1e3368c76a Preliminary changes 2017-05-08 18:27:51 +02:00
Simon Grätzer 31b87de7b3 Adding full text index to tests, and fixing some error handling 2017-05-08 11:41:12 +02:00
Simon Grätzer 1948896e5c Broke a test file 2017-04-28 18:16:16 +02:00
Simon Grätzer 86ad31124d Fixing some shell_replication tests 2017-04-28 18:14:03 +02:00
Simon Grätzer 52058ac909 fixing replication errors 2017-04-28 16:28:35 +02:00
Simon Grätzer 84ed7ec100 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel
# Conflicts:
#	arangod/RocksDBEngine/RocksDBRestReplicationHandler.cpp
2017-04-28 14:40:17 +02:00
Simon Grätzer 0a00a7eb67 Fixed broken JSON output 2017-04-28 14:28:34 +02:00
jsteemann 8e95e1ee59 try to make task tests more robust and reliable 2017-04-28 09:40:08 +02:00
Frank Celler 33fda9275c Merge pull request #2451 from arangodb/feature/ldap-auth
Feature/ldap auth
2017-04-26 14:12:32 +02:00
Michael Hackstein 2e417a1576 db._explain should now be backwards-compatible if we access a 3.1 instance with a 3.2 arangosh 2017-04-26 11:01:05 +02:00
baslr f45a7f07b1 Merge remote-tracking branch 'origin/devel' into feature/ldap-auth 2017-04-26 10:52:50 +02:00
Wilfried Goesgens 05a46e4781 fix db._explain() to work with the new interface of the graph traverser 2017-04-25 20:33:17 +02:00
Michael Hackstein f21fd3c0fe Made dropping of graphs more resilient to errors 2017-04-25 15:56:39 +02:00
jsteemann 9ab0ee817a jslint 2017-04-25 15:17:33 +02:00
Michael Hackstein e5f7303dad Fixed drop in general-graph it now honors distributeShardsLike 2017-04-25 15:08:45 +02:00
baslr 54e2bd6386 Merge remote-tracking branch 'origin/devel' into feature/ldap-auth 2017-04-25 14:19:05 +02:00
Michael Hackstein 02eb704279 Merge branch 'devel' of github.com:arangodb/arangodb into devel 2017-04-25 12:58:53 +02:00
Michael Hackstein 5737dd00e0 Adapted replication applier tests to connect to 127.0.0.1:0 which is invalid instead of some random invalid IP. Causes the Mac to ignore the timeout. 2017-04-25 12:58:46 +02:00
baslr 579718ad37 Merge remote-tracking branch 'origin/devel' into feature/ldap-auth 2017-04-25 11:18:07 +02:00
Jan Christoph Uhde 5f31e2b692 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel
* 'devel' of https://github.com/arangodb/arangodb:
  Removed non-applicable test case.
  unused variable
  rocksdb options
  fixes for non-array IN lookups, added tests
  Add check for distributeShardsLike to dropCollectionCoordinator.
  fixed foxx test
  added missing return
  js request simplified if sslProtocol
  pass also sslProtocol from request to internal.download
  fixed issue #2450
  cppcheck
  Added a missing switch break.
  remove dead code
  updated CHANGELOG
  raised to 4
  added 'x-content-type-options: nosniff' to avoid MSIE bug
  fixed queue size
  added queue size to statistics
  added queue time and request tracing with timings
2017-04-25 08:38:51 +02:00
baslr 28461a24ca js request simplified if sslProtocol 2017-04-24 23:06:46 +02:00
baslr e01abcf1ed pass also sslProtocol from request to internal.download 2017-04-24 23:06:45 +02:00
Jan Christoph Uhde 37db67dee5 fix engine name 2017-04-24 19:01:31 +02:00
Michael Hackstein adf86751f1 Merge branch 'devel' of github.com:arangodb/arangodb into devel 2017-04-24 15:38:02 +02:00
Michael Hackstein 13b7bf5744 Fixed replication tests. They tried to drop system collections without FORCE => setup failures if the collection exists before. 2017-04-24 15:37:52 +02:00
baslr 9a78fc650b Merge remote-tracking branch 'origin/devel' into feature/ldap-auth 2017-04-24 15:37:18 +02:00
Kaveh Vahedipour ccc388a940 more dictributeShardsLike code mergedfrom 3.1 2017-04-24 15:13:40 +02:00
Kaveh Vahedipour c099c6daa9 more dictributeShardsLike code mergedfrom 3.1 2017-04-24 15:12:38 +02:00
baslr 3d9f58bb70 Merge remote-tracking branch 'origin/devel' into feature/ldap-auth 2017-04-24 12:45:35 +02:00
jsteemann 1df46f8923 Merge branch 'devel' of https://github.com/arangodb/arangodb into engine-api 2017-04-21 16:59:32 +02:00
Kaveh Vahedipour 1f81ce28b0 merge in cpp & js from 3.1.18 yet to do tests 2017-04-21 15:41:05 +02:00
jsteemann 054f1afeb0 fix README_maintainers, provide debug output for invalid invocations 2017-04-19 12:05:16 +02:00
Simon Grätzer a389dc0a92 Fixing geo + full text tests 2017-04-18 13:32:44 +02:00
Simon Grätzer bab71f6339 Fixing jslint errors 2017-04-18 12:15:26 +02:00
Dan Larkin 5852597080 Moved a test file. 2017-04-14 00:04:11 -04:00
Simon Grätzer 866d8b7451 Fixing tests 2017-04-13 17:45:16 +02:00
Simon Grätzer a0e25edac8 Fix a cluster test 2017-04-13 15:20:28 +02:00
baslr d4ae94f937 Merge remote-tracking branch 'origin/devel' into feature/ldap-auth 2017-04-10 17:14:37 +02:00
Jan Christoph Uhde 77da0794ce fix shell-foxx-manager-nightly-spec.js 2017-04-06 17:32:49 +02:00
Jan Christoph Uhde b7bd1d9e59 fix shell-foxx-manager-nightly-spec.js 2017-04-06 17:31:21 +02:00
jsteemann a8f8587504 fix a test 2017-04-06 14:16:35 +02:00
Simon Grätzer 5603e1a6da Renaming test files 2017-04-06 10:29:40 +02:00
Dan Larkin 1d9366b1f1 Moved/split more tests. 2017-04-05 16:09:46 -04:00
Dan Larkin ff0efe8de8 Split/moved more test files. 2017-04-05 12:16:54 -04:00
Dan Larkin 84be9365af Split a JS test. 2017-04-05 11:39:28 -04:00
jsteemann ca60cd691a remove dependency on fulltext index 2017-04-05 11:22:53 +02:00
Simon Grätzer 4e818e483a Marked hash index selectivity test as MMFiles only 2017-03-31 11:48:11 +02:00
Simon Grätzer 807d38784c Added some small fixes 2017-03-31 11:29:27 +02:00
baslr f669fd4b9e LDAP: more options 2017-03-30 11:03:13 +02:00
baslr 3b2ccf46f4 Merge remote-tracking branch 'origin/devel' into feature/ldap-auth 2017-03-30 01:55:08 +02:00
jsteemann 3ff4f15a19 Merge branch 'devel' of https://github.com/arangodb/arangodb into engine-api 2017-03-29 17:50:55 +02:00
mpv1989 e763233ca7 Foxx: Fix arangoUser not being set correctly 2017-03-29 17:33:16 +02:00
baslr 0713099acc Merge remote-tracking branch 'origin/devel' into feature/ldap-auth 2017-03-29 15:13:24 +02:00
mpv1989 59e000b743 Foxx: Fix CORS blowing up when headers are not set 2017-03-29 13:43:06 +02:00
jsteemann f728fcdbb8 Merge branch 'devel' of https://github.com/arangodb/arangodb into engine-api 2017-03-29 02:41:00 +02:00
baslr 7f92c26774 Merge remote-tracking branch 'origin/devel' into feature/ldap-auth 2017-03-29 00:29:16 +02:00
mpv1989 f3ebf247b0 automatically add CORS response headers 2017-03-28 19:48:46 +02:00
Jan Christoph Uhde f4e164ef78 fix compile errors 2017-03-27 08:57:51 +02:00
baslr fef40e3c9d merge with devel 2017-03-24 12:07:29 +01:00
baslr e78da3d85f LDAP: added additional errors 2017-03-23 23:51:48 +01:00