1
0
Fork 0
Commit Graph

5412 Commits

Author SHA1 Message Date
Frank Celler 8fa8bbe5fd Merge branch 'devel' of https://github.com/arangodb/arangodb into bug-fix/trav-filter-optimizer-tests 2017-07-28 15:20:55 +02:00
m0ppers 9a0bc716d0 Do not allow replication to create/drop collections (#2898)
In the cluster case the only one who is allowed to do this is the schmutz
2017-07-28 14:24:02 +02:00
Frank Celler 6613ecda72 Merge branch 'devel' of https://github.com/arangodb/arangodb into bug-fix/trav-filter-optimizer-tests 2017-07-28 09:30:26 +02:00
m0ppers 589ffd5c59 Feature/improve logging (#2881)
* Improve logging in various places

* Fix jslint
2017-07-28 09:23:18 +02:00
m0ppers 6fefe5ab31 In high concurrency situations double checking checksums is a must have (#2890) 2017-07-28 00:43:21 +02:00
Jan 56143bc186 fix issue #2879 (#2885) 2017-07-27 23:30:05 +02:00
Frank Celler 00a6cbab64 fixed jslint error messages 2017-07-27 16:01:45 +02:00
Simon Grätzer 9786b198c2 Allow pregel jobs to interrupt on shutdown (#2855) 2017-07-27 15:29:01 +02:00
Mark 3a59ea553b Add catch when propagateSelfHeal (#2839) 2017-07-19 17:03:52 +02:00
Simon Grätzer f50a36181d Various authentication related issues (#2824) 2017-07-18 22:22:09 +02:00
Frank Celler a692577f5b Feature/auth context (#2815) 2017-07-16 09:39:03 +02:00
mchacki dd13944264 Added some test cases where filter optimization in Traverser could have yield wrong results. 2017-07-15 17:49:00 +02:00
Max Neunhöffer 442be8faad 60s timeout for syncCollectionFinalize. (#2805) 2017-07-14 23:14:04 +02:00
Manuel B df7051f2cb server / client side db._parse added (#2789) 2017-07-14 14:31:51 +02:00
Frank Celler a5a25754ed Feature/reduce extraction to projection (#2792)
* reduce extractions to projections

* recycle string buffers in SocketTask

* micro optimizations for mmfiles indexes

* added special lookup function for _key

* moved function into the correct file

* speed up key buffer allocations a bit

* added noexcept specifier

* correctly name variable

* explicitly move bounds

* fix and speedup from/toPersistent functions

* reuse string from ManagedDocumentResult for multiple lookups

* use move-assign

* a bit less work for single server

* speedup AQL function HASH

* single fetch optimization

* performance optimization for the case when no documents need to be returned

* make reduce-extraction-to-projection a RocksDB-only optimizer rule

* cppcheck

* try to fix compile error on MacOS

* bug fix for MacOSX

* missing namespace (in Windows compile)
2017-07-14 08:40:29 +02:00
Max Neunhöffer 2f874249bb Bug fix/adjust agency comm timeouts (#2765)
* Take out 503 timeouts altogether.
* Overhaul of AgencyComm::sendWithFailover loop.
* Let performRequests optionally ignore 404 coll not found.
* Fix error message "database not found" when AgencyComm failed.
* Add log entries in Agency if locks are acquired too slowly.
* Reexecute the javascript cluster sync stuff even if there was no plan/current change...So failed sync jobs can retry later...
* Cover callbacks in Communicator by lock. This fixes https://github.com/arangodb/planning/issues/370
* Put in delay in waiting for leader in agency test.
* Schmutz logging to heartbeat topic.
* Add more lock time diagnostic in agent.
* Switch on agencycomm tracing in coordinator.
2017-07-13 00:44:28 +02:00
Frank Celler 40d73d5a8b Revert "[WIP] Feature/reduce extraction to projection (#2735)"
This reverts commit 5bfcff30cd.
2017-07-12 12:52:14 +02:00
Jan 5bfcff30cd [WIP] Feature/reduce extraction to projection (#2735)
* reduce extractions to projections

* recycle string buffers in SocketTask

* micro optimizations for mmfiles indexes

* added special lookup function for _key

* moved function into the correct file

* speed up key buffer allocations a bit

* added noexcept specifier

* correctly name variable

* explicitly move bounds

* fix and speedup from/toPersistent functions

* reuse string from ManagedDocumentResult for multiple lookups

* use move-assign

* a bit less work for single server

* speedup AQL function HASH

* single fetch optimization

* performance optimization for the case when no documents need to be returned

* make reduce-extraction-to-projection a RocksDB-only optimizer rule
2017-07-12 11:22:29 +02:00
Kaveh Vahedipour d98481f889 Merge pull request #2769 from arangodb/feature/fix-jslint-aql-graph-traverser.js
missing assertTrue entry in global
2017-07-11 16:04:37 +02:00
Kaveh Vahedipour 774ebae638 missing assertTrue entry in global 2017-07-11 16:02:02 +02:00
Kaveh Vahedipour 231a360b3b fixes for secondaries 2017-07-11 14:05:51 +02:00
Michael Hackstein 2cd48e614b Feature/optimizer rule remove filter covered by traversal (#2731)
* Added a first test for remove-filter-covered-by-traverser rule.

* initial try at adding the functions

* Initial try at fixing all cases

* Fixing optimizer rule order

* Changing rule

* excluding quantifier ANY

* Fixing assertion error

* quick fix for some failed traverser cases

* Fixing quantifier related tests

* Adding upper limit for scannedIndex in cluster

* Fixing the mmfiles asserts

* Allow the coordinator to do condition checking on edges again. Used in enterprise.

* Fix naming of optimizer rules (#2739)
2017-07-10 10:15:17 +02:00
Frank Celler 6b47544611 Revert "Feature/optimizer rule remove filter covered by traversal" (#2724)
* Revert "Feature/authorization query cache (#2720)"

This reverts commit 83712b7b4a.

* Revert "properly return index in case of unique constraint violation (#2716)"

This reverts commit c3f346e0a5.

* Revert "fix https://github.com/arangodb/planning/issues/388 (#2714)"

This reverts commit 1d944b97a4.

* Revert "fix typo (#2718)"

This reverts commit 61a80ed697.

* Revert "Feature/optimizer rule remove filter covered by traversal (#2676)"

This reverts commit c54b81fb69.
2017-07-04 11:13:36 +02:00
Simon Grätzer c54b81fb69 Feature/optimizer rule remove filter covered by traversal (#2676) 2017-07-03 18:39:32 +02:00
Frank Celler bbe7484521 Feature/auth context (#2704)
* added read-only users
2017-07-02 23:15:57 +02:00
Michael Hackstein b73c0dfda4 Merge pull request #2670 from arangodb/feature/issue-277-nextExtra
Feature/issue 277 next extra
2017-06-29 17:24:55 +02:00
Dan Larkin ee8cb749bb Added recovery tests for views and fixed a few related bugs. (#2682)
* Added recovery tests for views and fixed a few related bugs.

* Added more view recovery tests.

* Modified view recovery tests to add a waitForSync operation after.

* fixed usage of wrong view type

* fixed recovery of view change markers
2017-06-29 17:09:43 +02:00
Michael Hackstein 6901a9b53f Fixed a misplaced edge filter It checked for a condition on the wrong depth. 2017-06-29 17:03:05 +02:00
Jan Christoph Uhde 828ab6db23 update unittests 2017-06-28 13:45:17 +02:00
Mark a033eef12d Fix issue 2684: Legacy service UI not working (#2685) 2017-06-28 12:09:15 +02:00
Frank Celler 2807ef559c Feature/move shard fix (#2626)
Major overhaul of handling of synchronous replication.
2017-06-26 16:55:01 +02:00
Mark 50ad6fbd76 Foxx service CRUD optimizations (#2657)
* Pass config/deps on _install so setup can see them

* Only register service when install succeeded

* Don't throw after point of no return: breaky install is breaky

* Delete service files during install if service validation fail

* Delete service files during install if service execution fails

* Fix commitLocalState with invalide services

* Remove deprecated test
2017-06-26 09:28:59 +02:00
Jan cfbd8ed93c Bug fix/rocksdb autoincrement (#2648)
* Added autoincrement keygen support to RocksDB engine with test.

* fixed key generator state keeping for mmfiles engine
2017-06-23 15:27:09 +02:00
Simon Grätzer 52fa4f450a Bugfix for cluster bug (#2649) 2017-06-22 17:34:01 +02:00
Jan c77a6fd738 fix one more limit issue in traversal (#2634) 2017-06-22 12:32:47 +02:00
Jan ebaeb639b2 Bug fix/incremental replication syncs too much (#2631)
* fix too much synchronization
* fix incremental sync for RocksDB engine
2017-06-21 14:56:33 +02:00
Mark 7743db32b9 Fix #2613 (#2614)
Reduce log level when Foxx manager tries to self heal missing database
2017-06-20 12:41:45 +02:00
Simon Grätzer 492d832695 Refactored /_api/index and /_api/database (#2582)
* working documents rest handler

* fixed cluster tests

* Consolidating database APIs

* clang-format

* Fixing issue with user creation through db._createDatabase

* replaced and refactored api-index and index api

* fixed cluster

* renaming some files

* added user methods

* removed files intended for later

* Fixed CC build

* Fixed method signature

* Fixing shell_server, shell_client tests
2017-06-19 23:47:40 +02:00
Jan f38ba7c84d fix invalid results (too many) when a skipping LIMIT was used for a traversal (#2603)
`LIMIT x` or `LIMIT 0, x` were not affected, but `LIMIT s, x` may have returned too many results
2017-06-19 17:34:47 +02:00
Jan 9bebd33ba3 fix invalid first group results for sorted AQL COLLECT when LIMIT was used (#2577) 2017-06-16 09:44:14 +02:00
Max Neunhoeffer cc985becd3 One more logging line in resilience test output. 2017-06-09 10:46:26 +02:00
jsteemann 9b77aae857 fixed some tests 2017-06-08 17:56:58 +02:00
Wilfried Goesgens 7224e7515c Implement testcases, this fixes #1806 2017-06-08 15:59:05 +02:00
Andreas Streichardt a2f5871ee6 Keep retrying coordinator bootstrap 2017-06-07 18:22:08 +02:00
Simon Grätzer fba82fc516 Forgot test case 2017-06-07 17:20:37 +02:00
jsteemann 210a4e485b fix path to version file 2017-06-06 15:42:42 +02:00
Andreas Streichardt 07364e857d Remove useless debug 2017-06-02 11:47:05 +02:00
Michael Hackstein ce9812d8fa Added proper sorting in test code. Not deterministic otherwise 2017-06-02 09:33:30 +02:00
Mop 619eae9be5 Revert "Squashed commit of the following:"
This reverts commit 2252088572.
2017-06-01 18:37:45 +02:00
Andreas Streichardt 2252088572 Squashed commit of the following:
commit f3d0fd6584b0e451b8c97abcb4ba8d9f2fc6f560
Author: Andreas Streichardt <andreas@arangodb.com>
Date:   Thu Jun 1 17:31:36 2017 +0200

    fix unittest

commit 7cd3544a39e1b78af9d4175cb3b978799b9bbfff
Author: Andreas Streichardt <andreas@arangodb.com>
Date:   Thu Jun 1 17:10:00 2017 +0200

    Remove debug comment

commit fb6b10dac15be49a72dbff80030a7d22abdfc3e0
Merge: 055eb1d269 6b18cc64fe
Author: Andreas Streichardt <andreas@arangodb.com>
Date:   Thu Jun 1 17:00:21 2017 +0200

    Merge branch 'devel' into shardorganizer

commit 055eb1d2693a583d21ea59ec8b6ba95ab0db57ac
Merge: 1ff7998ebf 8ea89b7677
Author: Mop <andreas@arangodb.com>
Date:   Thu Jun 1 16:56:30 2017 +0200

    Merge branch 'shardorganizer' of https://github.com/arangodb/arangodb into shardorganizer

commit 8ea89b76777c75b6a77bf695c3f074a0c4643c29
Author: Andreas Streichardt <andreas@arangodb.com>
Date:   Thu Jun 1 16:55:41 2017 +0200

    Fix shardmapping bug

commit 1ff7998ebfd691598ec5b455ca5bc2bfd7020fb4
Author: Mop <andreas@arangodb.com>
Date:   Wed May 31 17:26:08 2017 +0200

    more output

commit 68e88aa0e14316c4929d05b2c151bee6421d754d
Merge: 0978ad1d9e 44a6a78ec3
Author: Mop <andreas@arangodb.com>
Date:   Wed May 31 17:03:33 2017 +0200

    Merge branch 'shardorganizer' of https://github.com/arangodb/arangodb into shardorganizer

commit 44a6a78ec338a1e7cabb15464500d96b84c68f1d
Author: Andreas Streichardt <andreas@arangodb.com>
Date:   Wed May 31 07:42:43 2017 -0700

    Fix namespace

commit 0978ad1d9e2f01b86204990e74b66958f25eba66
Merge: f98582ccff d74e5989ad
Author: Mop <andreas@arangodb.com>
Date:   Wed May 31 16:40:35 2017 +0200

    Merge branch 'shardorganizer' of https://github.com/arangodb/arangodb into shardorganizer

commit f98582ccff3448f6c2388dab4cc2dc38034271b0
Author: Mop <andreas@arangodb.com>
Date:   Wed May 31 16:39:03 2017 +0200

    Revert "Revert "Next attempt at merging ShardOrganizer...distributeShardsLike fixed""

    This reverts commit fed45b7b10.

commit d74e5989ad478efe7d66d196715c05f4f41c9c29
Author: Andreas Streichardt <andreas@arangodb.com>
Date:   Wed May 31 16:31:31 2017 +0200

    Make it an error

commit 0a6a9ef9464df4f24ad205bbab5b9f8ded50054f
Author: Andreas Streichardt <andreas@arangodb.com>
Date:   Wed May 31 12:42:51 2017 +0200

    distributeShardsLike has to be saved as a cidString
2017-06-01 17:32:40 +02:00
jsteemann d13144f389 fixed issue #2515 2017-06-01 13:45:47 +02:00
Alan Plum 4b4b9f8c55 Fix Foxx API error (#2517)
* Fix Foxx API error
* Manager should distinguish between errors and services
* Foxx API should not require service to exist for uninstall
  if force option is used
* Respect options.force in FM _install
2017-06-01 11:59:04 +02:00
jsteemann 3df0940c7e jslint 2017-06-01 10:31:42 +02:00
Akshesh Doshi 46b50c266c Add tests and docs for REGEX_REPLACE AQL function (#2518)
* Fixed #1902: Add tests and documentation for REGEX_REPLACE

* Cleanup
2017-06-01 10:29:43 +02:00
Mark f832dd0388 Implement endpoint "commit coordinator state" (#2513)
Implement endpoint "commit coordinator state" (also see #2479)
2017-05-31 18:21:50 +02:00
Alan Plum 7608cde674 Implement clusterfoxx mounting logic (#2514)
* Serve error when service is out of sync
* Don't check system services for bundles
* Use appropriate errorNum if Foxx blows up
* Only check checksum when file exists
* Don't expose stack traces
2017-05-31 18:21:40 +02:00
Michael Hackstein 52406567cd Replaced EdgeCursors callback to return cid,rev instead of _id. Does less costly copying and comparision internally 2017-05-31 17:49:46 +02:00
Alan Plum ca52aa17b3
Make legacy test less flakey 2017-05-31 11:27:31 +02:00
Alan Plum 83310c12d7
Move FM.listJson into aardvark 2017-05-30 20:19:58 +02:00
Alan Plum 5fc4a17b24
Make sure the bundle doesn't exist before loading it from db 2017-05-30 20:09:58 +02:00
Alan Plum c4305897cf
More realistic binary document test 2017-05-30 18:39:56 +02:00
Alan Plum cceccf59da Simplified Foxx self healing (#2511)
* Implement new self-heal
* Add error codes for 503, service missing/outdated
* Detect changes to service via rev
* Pretty print incoming response object in log
2017-05-30 18:27:32 +02:00
Michael Hackstein eecdcd4e96 Made shortest path OOM test deterministic 2017-05-30 17:41:07 +02:00
jsteemann ed21022f55 added test for shardids restriction 2017-05-29 17:02:16 +02:00
jsteemann a35f85bcc7 added proper docs for --query.fail-on-warning 2017-05-29 15:46:48 +02:00
Frank Celler ee1c60c67a added _binaryDocument and _binaryInsert (#2506) 2017-05-27 22:49:22 +02:00
Andreas Streichardt b5fcd15214 Fix linter 2017-05-24 14:53:06 +02:00
Andreas Streichardt f08cae2028 Explicitly wait for unloaded 2017-05-24 14:37:30 +02:00
jsteemann 0c6b3cfd46 fix non-deterministic behavior in cluster_sync test 2017-05-24 12:51:27 +02:00
Max Neunhoeffer 11e6b9ea7f Merge branch 'devel' of ssh://github.com/ArangoDB/ArangoDB into devel 2017-05-23 20:38:38 +02:00
Andreas Streichardt d8dc3c85e7 Call version instead of foxx status to save a v8 context 2017-05-22 19:07:24 +02:00
Max Neunhoeffer 4bddb0c1ad Handle errors in loadScript for JS, use in cluster-bootstrap.
This recognizes errors that happen during the global cluster bootstrap.
If an error happens, the bootstrap is tried again later. The cluster
does not start until the cluster bootstrap has been finished
successfully.
2017-05-22 15:34:41 +02:00
jsteemann a5070978da fixed issue #2488 2017-05-19 15:57:47 +02:00
Michael Hackstein ca737edd97 Merge branch 'devel' of github.com:arangodb/arangodb into devel 2017-05-17 18:53:35 +02:00
Michael Hackstein 0d18821705 Adapted the shell-index-figures test to honor that the primary index does not have a cache anymore. 2017-05-17 18:52:09 +02:00
Andreas Streichardt 83dff84be7 Merge branch 'devel' of github.com:arangodb/arangodb into devel 2017-05-17 18:19:26 +02:00
Andreas Streichardt 49328dcecb Add authorizedflag fixing cluster internal foxx requests 2017-05-17 18:14:11 +02:00
Alan Plum 9b8a641e77
loadServiceFromDisk was renamed 2017-05-17 16:51:28 +02:00
Alan Plum 8a1286ac1c
Service constructor changed a bit 2017-05-17 16:48:06 +02:00
Jan Christoph Uhde d7a1b98117 fix liniting errors 2017-05-17 12:47:37 +02:00
Alan Plum 819a1194ba
Throw correct errors in Foxx management internals 2017-05-17 12:22:16 +02:00
Michael Hackstein b19549f5e4 RocksDB now properly reports EdgeIndex estimates to the user. Also reactivated edgeIndex tests for the estimate in RocksDB. 2017-05-17 09:52:25 +02:00
Alan Plum 5d017d733c Improve Foxx cluster resilience (#2479)
* Improve Foxx cluster resilience

Fixes #2083
Fixes #2384
Fixes #2408
Addresses #1892

* Port old Foxx API

* Implement single-file services

* Add console.errorStack/warnStack/infoStack helpers

* Simplify serviceInfo validation

* Extract github/upload logic into Aardvark and old FM API
* Move generator logic into Aardvark
* Move zip/js buffer logic into FM core

* Add Foxxmanager tests

* Send empty response when no README

* Disambiguate script arg format

Historically we allow passing an array of positional arguments or an arbitrary first argument.
This is surprising behaviour, so we should just always treat the value as a first argument.

* Rebuild bundle in development mode

* Nicer HTTP docs formatting

* Create Foxx HTTP docs

* Simplify service upload handling

* Remove inline swagger docs

* Implement public download route

* Consistency

* Rebuild aardvark

* Move bundle route into /_api/foxx/_local

* Rebuild Swagger API docs

* Add changes to CHANGELOG

* More docs
2017-05-16 17:41:29 +02:00
jsteemann 985c3f4bfc also execute tasks in existing databases 2017-05-16 12:11:23 +02:00
jsteemann 499491bee9 use exclusive locks for some transactions 2017-05-15 22:47:50 +02:00
Michael Hackstein 301f5125cf Merge branch 'devel' of github.com:arangodb/arangodb into devel 2017-05-15 17:19:35 +02:00
Michael Hackstein d7d650e27d Implemented index estimates for RocksDB. Now the following indexes have
a proper estimate:
* HashIndex
* SkiplistIndex / Persistent
* EdgeIndex
2017-05-15 17:17:58 +02:00
Andreas Streichardt 4dd5e54dc6 reenable check 2017-05-15 16:49:08 +02:00
Andreas Streichardt 41f8185553 Add test for rolling back indexes 2017-05-15 16:49:07 +02:00
jsteemann a829fed1bc do not turn off foxx queues 2017-05-15 15:15:29 +02:00
jsteemann 787e71d60f disable foxx queues 2017-05-15 14:35:09 +02:00
jsteemann 422b06211f fix some replication issues 2017-05-11 19:00:32 +02:00
jsteemann c5a195d62c attempt to fix races in replication 2017-05-10 14:32:16 +02:00
Simon Grätzer fd9647af83 Wrong index number 2017-05-10 10:41:47 +02:00
Simon Grätzer 2bba69bafc Fixed dump test 2017-05-10 10:27:50 +02:00
Simon Grätzer 3b7ef439c8 Merge branch 'devel' of https://github.com/arangodb/arangodb into feature/rocksdb-geoindex 2017-05-09 17:42:27 +02:00
Simon Grätzer b0bc75c93e Working geo index 2017-05-09 14:47:13 +02:00
jsteemann d0ae7673b9 do not print errors on shutdown 2017-05-09 10:26:59 +02:00
Simon Grätzer 1e3368c76a Preliminary changes 2017-05-08 18:27:51 +02:00
Michael Hackstein 0b985cb9da Merge branch 'devel' of github.com:arangodb/arangodb into devel 2017-05-08 12:39:06 +02:00