1
0
Fork 0
Commit Graph

17491 Commits

Author SHA1 Message Date
Jan 1d15b50d22
Bug fix/applicationserver stop (#9414) 2019-07-08 20:30:05 +02:00
Jan 66d8c01ad6
Bug fix/fixes 08 07 2019 (#9423) 2019-07-08 20:29:29 +02:00
Andrey Abramov a793a4749f
Bug fix/rename iresearch analyzer feature (#9424)
* rename feature

* ensure 'MMFilesLogFileManager::SafeToUseInstance' is safe to use in multithreaded context

* address mac compilation issue
2019-07-08 15:46:57 +03:00
Tobias Gödderz f501e00e9d Bug fix/add shard id to replication client identifier (#9366) 2019-07-08 14:03:42 +02:00
Andrey Abramov 174b54b036
Bug fix/internal issue #605 (#9421)
* release tick before committing changes to arangosearch

* fix tests

* address review comments

* address review comments

* add comment
2019-07-08 14:07:11 +03:00
Dronplane 22d5df16e8 Bug fix/internal issue #606 (#9413)
* Fixed stem, norm, text analyzer creation without properties. Added tests.

* Removed non-determenistic test. Same functionality is tested by gtest (IResearchAnalyzerFeatureTest.test_remove)

* Added check for analyzer type existence.

* Made error messages for analyzer creation more specific and readable

* Fixed test

* Applied review suggestions
2019-07-08 13:11:22 +03:00
Max Neunhöffer 7aa0c19026
Leader updates Current precondition fixes. (#9410)
* Better logging and error reporting.
* Preconditions for FollowerInfo.
* Preconditions when updating Current as leader.
* Change a log level.
* Fix unit tests.
* CHANGELOG.
* LOG_TOPIC ids.
* Fix a log id.
* Fix Windows compilation.
2019-07-05 13:35:13 +02:00
Andrey Abramov 595728ec1b
Bug fix/internal issue #605 (#9405)
* fix recovery

* update iresearch

* more fixes

* address review comments
2019-07-05 13:14:37 +03:00
Dronplane b70d7372ef Bug fix/internal issue #586 (#9401)
* Removet lazy creation ob analyzers collection. Test fixes (added explicit creation of analyzers collection for test enviroments)

* Fixed test runs. Removed cluster tests for analyzer DDL

* Analyzers collection name moved to tests common

* Disbaled load_library in normalize calls

* Legacy code cleanup

* added db analyzers collection checks in analyzer creation test to cover functions removed from gtest

* Reverted analyzer properties comparsion to non-utf8 as that must be binary equal
2019-07-04 13:12:06 +03:00
Simon b3ffaff86c Pregel additional test & TSan error fix (#9357) 2019-07-04 09:36:40 +02:00
Simon 9eaeb6ce37 use a lock when calling unload (#9375) 2019-07-04 09:36:19 +02:00
Jan 801c957348
add initializeCursor back to DistinctCollectExecutor (#9386) 2019-07-03 16:15:15 +02:00
Lars Maier c4ff5ccce9 [Devel] Queue-Full-Logging (#9388) 2019-07-03 13:07:24 +02:00
Jan 1a58cc2213
add VelocyPackHelper::equal method (#9389) 2019-07-03 12:15:11 +02:00
Jan 2f90f5ec49
update velocypack version (#9379) 2019-07-03 12:14:52 +02:00
Jan 727bb39f1a
fix JSON statistics (#9385) 2019-07-02 18:24:04 +02:00
Jan 11f5f33659
make sure all error code names are prefixed with ERROR_ @fceller @kvs85 (#9384) 2019-07-02 18:07:33 +02:00
jsteemann b175853bae fix invalid logId 2019-07-02 16:48:06 +02:00
Jan be052fb614
use index projections when range-scanning the primary index and filtering on _key (#9372) 2019-07-02 11:26:27 +02:00
Dronplane 1f8099e295 Bug fix/fix cluster tests for analyzers (#9370)
* Fixed analyzer definition in test (definition should be object). Fixed proper reporting of invalid parameter error.

* Restored ability to parse string-encoded json object in analyzers rest hadler. Test was reverted to pass string again.

* Fixed test run

* Get rid of shared ptr to simplify code.
2019-07-01 16:52:06 +02:00
Jan 9cb08ded92
make the comparison functions unambiguous (#9349)
* make the comparison functions unambiguous

* added @kaveh's suggestion
2019-07-01 16:35:28 +02:00
Jan fe19b8aaae
don't unfairly favor the edge index when comparing it to more specialized indexes on `_from`/`_to` plus other attributes (#9362) 2019-07-01 16:32:42 +02:00
Andrey Abramov 671380b8fb
bug-fix/internal-issue-#604 (#9353)
* ensure flush subscriptions are being unsubscribed

* update tick even if no changes happened

* remove debug output

* fix test

* address review comments

* address test failures
2019-07-01 16:00:14 +02:00
Lars Maier c215e30299 Precs to check if collection exists (#9278)
* Adding preconditions for jobs to check that the collection still exists.

* Make it compile.

* Fixed tests.
2019-07-01 13:24:53 +02:00
Wilfried Goesgens 0afe3ddb5a Trap whether the statistics-Thread is asleep for too long. (#9364) 2019-07-01 11:32:31 +02:00
Jan d842b877a2
actually honor the return value of FollowerInfo::addFollower (#9358) 2019-06-28 18:31:15 +02:00
Jan 1653e9698a
remove unused functionality (#9360) 2019-06-28 18:28:11 +02:00
Dronplane d561221201 Bug fix/fixed analyzer removal from db (#9359)
* Added missing commit. Updated tests to check actual deletion from db.

* Added check for commit result
2019-06-28 17:29:49 +02:00
Jan 64c8079186
Forbid indexes on _id (#9337) 2019-06-28 11:11:22 +02:00
Andrey Abramov 40cce2cee0
bug-fix/internal-issue-#565.1 (#9257)
* validate analyzer properties, do not return `null` for identity analyzer properties

* fix some tests

* Fixed tests for new analyzer parameters validation. Added explicit test for analyzer parameter validation

* update iresearch, fix analyzer tests

* fix more tests

* fix tests

* store analyzer properties as vpack

* more compilation fixes

* Updated iresearch

* Tests compilation fixed

* Test run fixes

* Test run fixes

* Fix test run

* Fixed all IresearchTests

* Fixed V8Analyzers tests

* Added ngram and delimeter analyzers vpack config

* Added ngram and stem analyzers vpack parsers. Added tests

* Fixed internal issue #593 Fixed build issue

* Fixed tests

* Fixed Gtest tests

* Changed test run to fix Mac failure

* Mac tests debugging

* Fixed jslint errors

* test tracing added

* Taken account for VPackSlice operator== false negatives
2019-06-28 10:49:01 +02:00
Lars Maier eb1aa6e024 Response compression (#9300)
* First draft of response compression.

* Cleanup.

* Removed compression from /_api/version.

* Added ruby test for response compression.
2019-06-28 10:02:48 +02:00
Max Neunhöffer d6d362bd3b
Fix agency election lock step bug. (#9351)
* Fix agency election lockstep bug.

Reset the base point for the random election timeout to now whenever we have
cast a vote, be it for us or for some other server.

* CHANGELOG.

* Fix compilation.
2019-06-27 22:06:26 +02:00
Jan 161ce9607d
added test for object creation (#9355) 2019-06-27 21:13:03 +02:00
Jan 32ce797be4
make it possible to limit the number of parallel WAL tailing invocations (#9344) 2019-06-27 18:43:53 +02:00
Simon 683552ac13 Bug fix/simplify cache setup (#9341) 2019-06-27 15:44:39 +02:00
Jan 6f0c116130
clean up usage costs (#9237) 2019-06-26 17:01:22 +02:00
Kaveh Vahedipour 2fb159a8e2 compilation fix (#9340) 2019-06-26 17:00:33 +02:00
Kaveh Vahedipour 18fa84d619 if collection is gone in meantine ... (#9332) 2019-06-26 15:11:51 +02:00
Simon 739b6f273e fix windows warnings (#9326) 2019-06-26 09:24:43 +02:00
Andrey Abramov 33ce94379f
bug-fix/internal-issue-#597 (#9324)
* ensure immutable properties of ArangoSearch are properly handled in cluster

* address jslint warnings
2019-06-25 17:15:07 +02:00
Simon 505138b7d1 Fix issues with parallel database creation/dropping (#9272) 2019-06-25 11:09:26 +02:00
Simon cf7cf0131b Try to fix corruption error (#9258) 2019-06-25 10:18:26 +02:00
Jan Christoph Uhde ddb2e14e32 disable security checkes for statistics (#9282) 2019-06-24 15:07:39 +02:00
Michael Hackstein e61fb5a34e
Bug fix/create collections better preconditions (#9296)
* Fixed case where a SmartVertex collection could be available too early. Only possible if a SmartGraph is created only with this one collection.

* Now the TTL remove operation will properly check preconditions again.

* Second attempt, we only  say collection creation was success iff the plan for the collection has not been mdified during create.

* Disabled assertion in favor of tests.

* Removed debug output
2019-06-21 15:17:57 +02:00
Lars Maier 49fde75427 Added special lock for local data. Use read and write locking. Do not hold read lock during agency transactions. (#9277) 2019-06-21 14:27:20 +02:00
Markus Pfeiffer a6bf8028c7 Try improving error message (#9303)
When we try dropping a collection `A` for which there are other collections
that have distributeShardsLike set to `A`, mention this in the error message.
2019-06-21 14:23:42 +02:00
Tobias Gödderz f4baf0c436 Specialize scheduler pointer to allow devirtualization of method calls (#9225) 2019-06-21 11:15:18 +02:00
Frank Celler 54b0dc7fb5 removed old check 2019-06-20 17:30:41 +02:00
Tobias Gödderz 51254758f8 Fix warning about unused field (#9297) 2019-06-19 14:47:21 +02:00
Tobias Gödderz 0904e8c656 Fix compile warning with MSVC (#9291) 2019-06-18 18:14:49 +02:00
Tobias Gödderz da6e9da820 Measure time a request takes to get executed by the scheduler (#9224) 2019-06-18 17:34:33 +02:00
Tobias Gödderz c854d04864 Feature/limit executor passthrough (#9162)
* Some refactoring to implement helper methods iff necessary

* Updated comments

* Added static assertions

* Re-enabled check for maintainer mode

* Allow pass-through for LimitExecutor. This is not yet working with fullCount correctly!

* Get fullCount before returning the last row

* Adapted fullCount tests: LimitExecutor doesn't lie about the state of the last row anymore

* Use correct row

* Handle LimitExecutor-stats during fetchBlockForPassthrough

* Fixed LimitExecutorTest

* Removed _stats member

* Added more catch tests

* Bugfix for fullStats

* Removed an erroneous assertion

* Implemented LimitExecutor::skipRows

* Avoid name clash of class enum member to please MSVC

* Test MSVC

* Revert "Test MSVC"

This reverts commit d8325d95318bb83b79c8c4cc9f886d36e484c870.

* CREATE_NEW seems to be taken, too. Next try for MSVC

* Built VPack to ItemBlock helpers, and made SingleRowFetcherHelper more general

* Removed ostream operator because MSVC doesn't like it...

* Implemented SingleRowFetcherHelper::fetchBlockForPassthrough, plus a small framework to test executors

* Removed erroneous include

* Moved code in separate files

* Began writing the extended schema of runExecutor

* Added output operator for ExecutorCall

* Fixed removeWaiting

* Fixed expected output

* Added skipRows to runExecutor

* Added two tests

* Allow empty blocks

* Added two more tests

* Added another test

* Built two different parametrized classes for LimitExecutor

* Some cleanup

* Even more tests

* Fixed a bug found by the new tests

* Fix compile error on windows

* Use native matrix representation in tests

* Updated some comments
2019-06-18 17:33:58 +02:00
Wilfried Goesgens eadcedaa5e Bug fix/specify snappy dependency (#9289) 2019-06-18 17:33:23 +02:00
Wilfried Goesgens 200be8ec8b Bug fix/zero mac clang warning to errors (#9279) 2019-06-18 10:28:30 +02:00
Wilfried Goesgens 6fd3dd024c windows warning as errors, masquerade remaining unfixeable errors (#9249) 2019-06-17 16:21:29 +02:00
Lars Maier 50ce41e062 Release _to server when abort because of dropped collection or `something serious went wrong`. (#9267) 2019-06-17 15:04:44 +02:00
Matthew Von-Maszewski 80157a1267 update syncRequest() so that it aborts pending messages on shutdown. (#9260) 2019-06-13 21:47:45 +02:00
Frank Celler f021ea68f7 Merge branch 'devel' of github.com:arangodb/arangodb into 3.5 2019-06-13 20:47:41 +02:00
Jan Christoph Uhde 307880ffad Allow statistics for internal contexts. (#9262) 2019-06-13 19:27:29 +02:00
Michael Hackstein 2c78e2471b Bug fix/collection babies race timeout (#9185)
* Fixed include guard.

* Forward port of 3.4 bug-fix

* Removed lockers alltogether we are secured mutex already

* Fixed recursive lock gathering
2019-06-13 19:11:24 +02:00
Simon cc125b377c Use GetForUpdate on unique indexes (#9234)
* Use GetForUpdate on unique indexes

* add changelog

* fix loop

* use c++11 loops

* move some code

* fix error code check
2019-06-13 16:32:24 +02:00
Dan Larkin-York 1b5475a0c2 Revert change to shared_ptr in ClusterInfo. (#9256) 2019-06-12 16:19:18 +02:00
Heiko 56c969316f Bug fix/gharial edge from and tos verification (#9177)
* prepared new functions for more precise edge verification

* added tests

* more testing

* validate function added to replace edge functionality

* changelog

* applied requested mchackis refactor wishes

* adjustments for update and replace edge validation

* ruby testing adjustments due more prices update and replace edge verification mechanisms

* used wrong flag for the replacement of an edge process

* adjusted last failing tests

* rm print

* added edge to edge tests
2019-06-12 15:12:50 +02:00
Matthew Von-Maszewski 7119ccfdf1
copy logic of 3.4 scheduler that leaves threads available for FAST lane work (#9235) 2019-06-11 11:06:42 -04:00
Andrey Abramov 384c4310b1
Bug fix/internal issue #589 (#9244)
* update iresearch from upstream

* adjust tfidf/bm25 to original behaviour

* update iresearch from upstream

* fix compilation issues

* reduce number of virtual calls

* upgrade iresearch from upstream

* add js test
2019-06-11 17:42:30 +03:00
KVS85 7b2a11e0fb Remove accidentally forgotten 3.5 RC1 code 2019-06-10 16:03:11 +02:00
Dan Larkin-York 938e5fd36c Add drop-check for index creation in cluster (#9219)
* Add drop-check for index creation in cluster.

* Move check from callback to regular read.

* Add changelog entry.

* Incorporate review suggestion

Co-Authored-By: Simon <simon@graetzer.org>

* Convert to VPackArrayIterator.
2019-06-10 10:11:17 +02:00
Frank Celler 2d732be590
wait until task is shut down (#9239) 2019-06-10 10:07:31 +02:00
Jan 59d84e8301 attempt to remove expired WAL files on agents (#9231) 2019-06-09 19:46:45 +02:00
Tobias Gödderz 09e8aa8911 Fix two int size mismatches (#9226) 2019-06-09 07:35:03 +02:00
Jan 63eadb1d79
fix issues (#9236) 2019-06-08 19:40:42 +02:00
Jan 98e9197a3f properly downgrade enterprise features when restoring them into a com… (#9161) 2019-06-07 15:45:14 +02:00
Jan d1a6d23c36
normalize AQL filter conditions, so that attribute accesses are normally placed on the left hand side of the condition (#9217) 2019-06-07 13:36:22 +02:00
Jan Christoph Uhde 3f603f024f remove some containers from common.h (#9223)
* remove some containers from Common.h

* enterprise fixes
2019-06-07 13:27:24 +02:00
Tobias Gödderz b632d58c80 Fix shutdown deadlock regarding comm tasks (#9204)
* Wait for _commTasks in unprepare, that is after Cluster::stop

* Chose better method names

* Revert "Chose better method names"

This reverts commit 91e821348740c655f47207af7e570075f2241895.

* Revert "Wait for _commTasks in unprepare, that is after Cluster::stop"

This reverts commit 6551ae90d74fc046369fdb97cc5872706ce1a184.

* Next try, stop ClusterComm threads earlier
2019-06-07 13:23:33 +02:00
Jan eb22eac2d1
display reason for error in background listener, plus ignore shutdown (#9206) 2019-06-07 10:11:12 +02:00
Jan 6a07476c41
don't include the Logger in header files if it's not necessary (#9216) 2019-06-07 10:08:03 +02:00
Wilfried Goesgens 863257f1d5 fix windows warnings / compile errors (#9215) 2019-06-06 18:08:44 +02:00
Kaveh Vahedipour 8a511934df [devel] fix ttl handling for object assigments (#9207)
* fix ttl handling for object assigments
* completed test handling
* CHANGELOG.
2019-06-06 15:48:40 +02:00
Jan 074397d367
allow "stream" attribute on top level (#9195) 2019-06-05 15:48:58 +02:00
Jan Christoph Uhde 12ec86c2b1 add shardKeyAttribute to pregel start parameters (#9149) 2019-06-05 14:25:47 +02:00
Michael Hackstein 8a4d0bc068
Performance improvement on Sort (#9198)
* Modified AqlItemMatrix to use two uint32 values to identify for in Block instead of one uint64 value encoding the two and calculation. Also now throw an error as soon as we exceed the limit there

* Removed new error code, thanks jsteemann for pointing out.
2019-06-05 12:45:17 +02:00
Jan e0c94e64d7
fix path length checks when copying js files (#9191) 2019-06-05 10:39:52 +02:00
Andrey Abramov 0fb8b348db
bug-fix/internal-issue-#574 (#8991)
* restrict number of buckets to compare

* do not use more sort buckets than requested

* cleanup
2019-06-04 18:55:25 +03:00
Simon 1402395f1c Port fuerte changes from network pool branch (#9125) 2019-06-04 16:35:15 +02:00
Jan 1087c11fa4
Bug fix/pregel micro improvements (#9179) 2019-06-04 09:31:48 +02:00
Andrey Abramov b2a2bd100e
Bug fix/internal issue #590 (#9176)
* disallow creation of analyzer by a full name

* fix tests
2019-06-04 00:28:20 +03:00
Dronplane 1e24c79ca3 Bug fix/internal issue #565 (#9154)
* Merged latest iresearch. Removed creation of build-in analyzers from tests

* Updated tests for new analyzer parameter names

* Merged fix for format from iresearch

* Apply suggestions from code review. Corrected typo in variable name.

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* backport changes from IResearch upstream

* fix compilation issues

* backport fix from IResarch upstream

* Fixed analyzer parameter name in tests
2019-06-03 18:20:23 +03:00
Michael Hackstein d135d55d55
Bug fix/collection babies (#9124)
* Bug fix 3.4/collection babies (#9033)

* Prepare API to create multiple collections in a single request to ClusterMethods to improve speedup

* Added counter on how many collections are successfully created

* Allow multi collection creation one level higher

* CollectionMethods now allow batch createion of Collections

* Improved array size assertions

* Now a graph is createad within a single roundtrip in the agency.

* Added new header files

* Insert collections in the AGENCY with TTL and a isBuilding flag, collections with this flag should not be visisible in the coordinator

* Added forgotten C++ file

* Fixed a rare race condition, and the failing IResearch Tests

* readded callback on DONE, otherwise lists are out of sync

* Fixed assertions to let mocked tests pass...

* Fixed community cluster

* Started fixing IResearch analyzer test, catch-tests are failing ;(

* Solved missed merge-conflict

* Added helper functions in AnalyzerFeature-test

* Refactoring AnalyzerTest Section-Auth

* Refactoring AnalyzerTest Section-Emplace-Duplicates

* Refactoring AnalyzerTest Section-Emplace-Error-Cases. Recovery-Test is now red, it seemed to be green because of invalid test case before.

* Refactoring AnalyzerTest, split GET test into multiple parts, still left 'cluster simulation'.

* Attempt to extract Coordinator / DBServer tests a little bit. This commit starts to break all Coordinator tests. However i am convinced that earlier version did NOT test a cluster situation at all, but some hybrid of SingleServer with full local storage that got told to be a Coordinator from now on, but without any Coordinator setup...

* Temporarly disabled some tests in AnalyzerFeature, as discussed with @gnusi.

* Fixed include guard.

* Temporarily deactivated failing tests

* You shall save your files before you commit...

* Fixed test asserting on plan version, which is now higher than before
2019-06-03 17:11:22 +02:00
Jan a4a8bb37c2
improve error message (#9158) 2019-06-03 17:06:51 +02:00
jsteemann 67b751103d fix typo 2019-06-03 13:07:47 +02:00
jsteemann a0676dcf20 add commtask debugging again 2019-06-03 11:10:31 +02:00
Lars Maier 1e94ecf414 Bug fix/supervision fixes4 (#9016)
* Try to fix agency problems with snapshots.

* Abort MoveShards jobs that have the failed server as fromServer.

* Report aborts.

* CHANGELOG.
2019-05-31 17:20:06 +02:00
Dan Larkin-York 44a413a9af Miscellaneous fixes for named indices (#9100) 2019-05-31 17:00:56 +02:00
Tobias Gödderz 6a56130ff8 Reimplement ReturnExecutor with a simple pass-through (#8876) 2019-05-31 15:27:59 +02:00
Tobias Gödderz 79cd45f89c Wait for replication before inserting documents (#9151)
* Wait for replication before inserting documents

Also, increased some timeouts and fixed a log message

* Fixed some log levels and a log message

* Removed repair-distribute-shards-like-spec from greylisted tests
2019-05-31 16:09:20 +03:00
Jan 3364ea6273
refactor index APIs (#9145) 2019-05-31 11:12:39 +02:00
KVS85 fab06069db
Bug fix/adjust 3.5 rc3 (#9146)
* fix for windows

* Fix some uses of atomics (#9141)

* Remove std::move as it prevents copy elision (#9144)

* bug-fix/internal-issue-#549 (#9086)

* do not persist legacy analyzers into _analyzers table

* fix arangosearch upgrade in cluster

* get rid of Vasiliy's shit

* address review comments

* ensure link is synchronized after creation in upgrade

* fix compilation error

* minor cleanup

* fix tests

* distribute '_analyzers' collection as '_graphs'

* comment out Vasiliy's shit part 2

* Remove ArangoSearch views upgrade limit
2019-05-30 23:20:33 +03:00
Andrey Abramov e8b38bfa8e
bug-fix/internal-issue-#549 (#9086)
* do not persist legacy analyzers into _analyzers table

* fix arangosearch upgrade in cluster

* get rid of Vasiliy's shit

* address review comments

* ensure link is synchronized after creation in upgrade

* fix compilation error

* minor cleanup

* fix tests

* distribute '_analyzers' collection as '_graphs'

* comment out Vasiliy's shit part 2
2019-05-30 22:00:06 +03:00
Markus Pfeiffer 3323cdcc28 Fix some uses of atomics (#9141) 2019-05-30 15:50:36 +02:00
Frank Celler d2034289a6 fix for windows 2019-05-30 12:14:52 +02:00
Frank Celler 1a71cc6898 fix for windows 2019-05-30 12:10:54 +02:00
Frank Celler c3cc48e35f Merge branch 'devel' of github.com:arangodb/arangodb into 3.5 2019-05-30 07:32:27 +02:00
Jan c3d1d24b27 fix processing of AQL index hints when upgrading (#9138) 2019-05-30 04:28:19 +02:00
Kaveh Vahedipour 773f3c8422 [devel] fix state clientlookuptable (#9066) 2019-05-30 04:24:46 +02:00
Max Neunhöffer 4da1a6afdf
Coordinators do not unregister at every shutdown. (#9134)
* Coordinators do not unregister at every shutdown.

Instead they create a new short name with every start.
This is needed for the transactions.

* Always new short id for coordinators. Never for DBServers!
2019-05-29 22:55:22 +02:00
Jan d67513607d
fixed issue #9106 (#9116) 2019-05-29 18:27:38 +02:00
Markus Pfeiffer 73ac4f194d (K_)SHORTEST PATHS: Display correct directions in explainer/profiler (#9080) 2019-05-29 15:31:09 +02:00
Simon a5a72ef6d4 port Pregel segmented buffers (#9112) 2019-05-28 18:23:20 +02:00
Simon 27112a3486 Bug fix/jobs indexes (#9107) 2019-05-28 09:38:14 +02:00
Wilfried Goesgens b442f3f6ec add std:: (#9115) 2019-05-27 20:47:03 +02:00
Tobias Gödderz 4fbff3280f Added missing initialization; use alignas instead of padding; mark move constructor noexcept (#9111) 2019-05-27 18:50:20 +02:00
Dronplane 3d697d8584 Added filtering analyzers before adding them to view link - duplica… (#9094)
* Added filtering analyzers before adding them to view link   - duplicates are not allowed any more

* Updated test testAddDuplicateAnalyzers - now it tests analyzers with same names but in different databases
2019-05-27 13:20:51 +03:00
Jan c23c50c069
try to fix a race in TTL thread shutdown (#9096) 2019-05-27 11:25:49 +02:00
Jan 208f22c842
only execute requests directly if there is one client on the IoContext (#9090) 2019-05-24 14:29:58 +02:00
Jan 3d79491f36
exclude VST requests from direct execution (#9075) 2019-05-24 09:53:15 +02:00
Jan 59b67cad40
fix various small annoyances (#9079) 2019-05-23 17:36:38 +02:00
Andrey Abramov f5a14414e1 Bug fix/disable upgrade for views again 2 (#9084)
* temporary disable upgrade for views

* temporary comment out test for upgrade
2019-05-23 15:31:26 +02:00
Frank Celler 37a1d7521e Merge branch 'devel' of github.com:arangodb/arangodb into 3.5 2019-05-23 15:28:56 +02:00
Frank Celler d3fadedad0 temporary disable upgrade for views (#9083) 2019-05-23 15:22:23 +02:00
Frank Celler fdbaeec732 Revert "temporary disable upgrade for views (#9083)"
This reverts commit b459a85498.
2019-05-23 15:20:09 +02:00
Andrey Abramov b459a85498 temporary disable upgrade for views (#9083) 2019-05-23 15:18:55 +02:00
Jan 70cccc3c89 fix VERSION files on upgrade and database creation in cluster (#9076) 2019-05-22 22:42:48 +03:00
Dronplane 6b08aa95c4 Added helper method to DataStore in iresearch link for performing full data store reset. Fixed usage of data store reset. (#9071) 2019-05-22 19:18:08 +03:00
Michael Hackstein 1a6719e144
Revert "(K_)SHORTEST PATHS: Display correct directions in explainer/profiler (#9049)" (#9074)
This reverts commit 44d15db812.
2019-05-22 16:46:45 +02:00
Markus Pfeiffer 44d15db812 (K_)SHORTEST PATHS: Display correct directions in explainer/profiler (#9049) 2019-05-22 16:06:46 +02:00
Jan 79258e072a
Bug fix/remove io task (#9056) 2019-05-22 14:34:49 +02:00
jsteemann 0e24c18253 re-add optimization for single servers 2019-05-22 14:29:12 +02:00
jsteemann d0868e76f8 fix "random" scheduler hangs 2019-05-22 14:01:03 +02:00
Frank Celler 446a025960 Merge branch 'devel' of github.com:arangodb/arangodb into 3.5 2019-05-22 09:54:14 +02:00
Tobias Gödderz f1dd0659fa Fixed test usage of MultiDependencySingleRowFetcher (#9060)
* Fixed assertion to not have side effects

* init dependencies in test explicitly
2019-05-22 09:39:50 +02:00
Dronplane e8298aeb00 Update IResearchLink.cpp (#9061)
added proper release for recovery reader on  link  drop.
2019-05-21 22:02:16 +03:00
Jan de717924ac
fix crash (#9057) 2019-05-21 18:19:11 +02:00
Jan 82b1e26686
Bug fix/smart joins (#9050) 2019-05-21 17:49:12 +02:00
Jan 6cea7528fb
fix result of AQL function `MIN` in case the first input element is (#9053)
`null`
2019-05-21 15:43:33 +02:00
Jan a61dd2895e
only offer indexes of type "persistent" in the web UI for the RocksDB (#9046) 2019-05-21 10:11:07 +02:00
Dan Larkin-York d5ecdd143a Convert unit tests to googletest framework (#9034) 2019-05-21 09:17:46 +02:00
Jan 49babbc66b
handle incomplete read errors (#9021) 2019-05-20 14:13:41 +02:00
Lars Maier 4fc2790863 [devel] Direct Exec Scheduler (#9004) 2019-05-20 11:38:57 +02:00
Andrey Abramov 56696e3ea6 Bug fix/internal issue #577 (#9035)
* properly handle shorthand of system analyzer

* don't allow accessing local analyzer from within another database

* ensure local analyzer are not acessible from outside

* fix duplicate ids
2019-05-19 18:00:13 +02:00
Jan Christoph Uhde d194cfc31e bug-fix/iresearch-error-propagation (#9001)
* use result instead of bool

* WIP: use arangodb::Results in more places

* WIP: fix unittests

* fix tests
2019-05-17 19:00:29 +03:00
Tobias Gödderz 3ccaabc2f5 Make ExecutionEngine respect the batch size. (#9012)
Fixes a test failure in api-export-spec-timecritical-noncluster.rb
2019-05-17 16:41:48 +02:00
Markus Pfeiffer c1d6a88da2 Fix a crash in ShortestPathExecutor (#8985)
* Fix a crash in ShortestPathExecutor

* Add a test that checks whether ShortestPathFinder is passed valid inputs
2019-05-17 16:40:22 +02:00
Simon 394c070a4f Do not check isEnabled (#8997) 2019-05-17 16:33:17 +02:00
Simon 93b2e64f37 Port pregel fixes (#9022) 2019-05-17 16:32:58 +02:00
Jan Christoph Uhde 970f732dec add exceptions for tokens function. (#8984)
* Provide users of the TOKENS function with errors.

* remove newlines

* fix compare

* fix as asked for

* improve fix
2019-05-16 22:12:43 +03:00
jsteemann 0f6af587ad remove debug output 2019-05-16 19:49:22 +02:00
jsteemann 4fa87528c6 Merge branch 'devel' of github.com:arangodb/arangodb into bug-fix/debug-commtasks 2019-05-16 19:47:12 +02:00
jsteemann 01dad862a4 move stopping more down 2019-05-16 13:52:46 +02:00
Tobias Gödderz 83732bc369 Improve calculation executor performance (#8851)
* Avoid a copy in CalculationExecutor<CalculationType::Reference>::doEvaluation

* Hack: fixed compilation

* Removed accidentally committed broken assert

* Replaced const_cast hack

* Completed implementation of copyBlockInternalRegister

* Simplified some assert code

* Fixed maintainer compilation
2019-05-16 11:31:21 +02:00
Tobias Gödderz c9fa5fba11 Additional AQL skipSome tests for IndexExecutor (#8938)
* Began work on additional index skipSome tests

* Added more index skip tests
2019-05-16 11:30:24 +02:00
jsteemann 73a7990a31 add debugging for CommTasks creation & destruction 2019-05-16 10:37:12 +02:00
Andrey Abramov b7cfa24518
bug-fix/arangosearch-upgrade (#9010)
* fix upgrade

* fix catch tests

* Update arangod/IResearch/IResearchAnalyzerFeature.cpp

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update arangod/IResearch/IResearchAnalyzerFeature.cpp

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>
2019-05-16 10:38:39 +03:00
Wilfried Goesgens 1907a7211b Bug fix/cleanup system includes (#8962) 2019-05-15 15:12:59 +02:00
Jan d94f49947c
fix sort order in distributed RETURN DISTINCT (#8935) 2019-05-15 15:02:53 +02:00
Tobias Gödderz 84d3d5dc5b Let only on shard per collection, not per DBServer, be responsible for initialize cursor (and shutdown) (#8946)
* Let only on shard per collection, not per DBServer, be responsible for initialize cursor (and shutdown)

* Reverted assertion

* Changed parameter to const&

* Style correction
2019-05-15 11:21:13 +02:00
Lars Maier b26ed84a81 Retry updating user collection. (#9002) 2019-05-15 10:08:25 +02:00
Tobias Gödderz 82f5978f30 Bugfix: Do not call getSome on remote after DONE (#8978)
* Bugfix: Do not call getSome on remote after DONE

* Added profiler tests for Sorting- and UnsortingGather

* Fixed jslint error
2019-05-15 09:13:44 +02:00
Jan bacb0886dc Bug fix/fix analyzer upgrade (#8990) 2019-05-14 23:29:21 +02:00
Simon 1923d315e8 New tests for Transaction API (#8996) 2019-05-14 21:40:31 +02:00
Andrey Abramov 8358b978be disable arangosearch analyzers in agency (#8989)
* disable arangosearch analyzers

* do nothing in `stop()` if feature is disabled
2019-05-14 17:37:38 +02:00
Matthew Von-Maszewski fb157b89ac Add protective barrier to sendWithFailover() in case MANAGER is nullptr (#8998) 2019-05-14 17:35:55 +02:00
Jan Christoph Uhde 057ba7130a Feature/cluster comm logging (#8971) 2019-05-14 16:11:58 +02:00
Lars Maier d9ff6647d8 [devel] Do not skip the check if there is only a single agent. (#8943)
* Do not skip the check if there is only a single agent.
* Updated changelog.
2019-05-14 15:42:43 +02:00
Lars Maier 49c568e674 Added reason to job abort method. (#8877) 2019-05-14 15:39:53 +02:00
Wilfried Goesgens e36655dee7 Feature/have collection count type (#8942) 2019-05-14 13:19:09 +02:00
Vasiliy 12bd4c785b issue 555.1.1: issue 555.1.1: try loading previous data store snapshots if checkpoint file unavailable, ehnace error message, address some warnings (#8975) 2019-05-13 18:33:44 +03:00
Andrey Abramov 404bcc8b0c
switch to tiered consolidation by default (#8981)
* switch to tiered consolidation by default

* fix copy/paste error

* fix tests

* fix tests
2019-05-13 18:25:46 +03:00
Markus Pfeiffer f00a8e98a2 Fix ASAN failure (#8974) 2019-05-13 13:56:19 +02:00
Vasiliy bbd5973337 issue 562.1: represent analyzer properties as object when possible (#8929) 2019-05-11 18:39:25 +03:00
Andrey Abramov 4e12fdb30b
ArangoSearch sorted indexes (#8888)
* move IResearchViewSort into separate unit

* ensure sort is propagated to IResearchLink

* update iresearch and extend some tests

* implement insertion of sorted values, store value comparator in link

* add tests for VPackComparer

* minor cleanup

* fix issue with sort definiton propagation

* propagate error message from index factory

* add integration tests for sorted indexes, single server

* initial implementation of IResearchViewMergeExecutor

* move iresearch::VPackComparer to separate translation unit

* store collection in IndexReadBuffer for IResearchViewMergeExecutor

* set primary sort from optimizer rule

* add some tests for sorted view case

* extend IResearchViewNode tests

* simplify primary sort definition

* pretty print order condition covered by arangosearch in query explanation

* fix compilation with g++-8

* address catch test failures

* fix tests

* adjust arangosearch optimizer rule to optimize sort in case of cluster

* make tests for sorted indexes work in cluster too

* fix tests

* extend js tests for sorted indices

* fix keyword in query explainer

* ensure sort-limit rule works with views

* backport ngram analyzer fix from upstream

* address jslint errors

* modify IResearchViewSort to support boolean direction specification format

* add some tests

* add tests

* add skip functionality

* extend tests

* add more catch tests

* inline some methods

* address jslint errors

* extend catch tests to reproduce the failure

* add some tests for check fullcount

* add more tests

* extend tests

* temporarily comment out broken tests

* extend js tests

* address review comments

* address jslint errors

* update iresearch
2019-05-11 17:58:56 +03:00
Vasiliy 9f1340eeeb issue 563.1: include static analyzers in analyzer listing (#8925)
* issue 563.1: include static analyzers in analyzer listing

* address macos test failures
2019-05-11 14:52:48 +03:00
Vasiliy 42b830c7a9 issue 561.1: add pretty-printing to analyzers in JavaScript (#8964) 2019-05-10 21:50:15 +03:00
Jan c00442e31c
fix some issues found by cppcheck (#8959) 2019-05-10 16:31:22 +02:00
Simon d62afaf8ea do not allocate string for _id value temporary (#8961) 2019-05-10 10:18:38 +02:00
Simon 6075fec35e Forward port some changes (#8949) 2019-05-09 19:42:06 +02:00
Jan 162bcd205c
Bug fix/fix velocypack error (#8954) 2019-05-09 19:38:00 +02:00
Jan c1d48c0fef
honor index hints in optimizer rule (#8940) 2019-05-09 18:47:36 +02:00
Wilfried Goesgens c43ec49eae make id a string so all of it can be passed on to V8. (#8933) 2019-05-08 18:52:29 +02:00
jsteemann 3c7c54985b fix compile warnings about unused variables/functions 2019-05-08 16:56:01 +02:00
Jan 9c3fd26682
upgrade velocypack library (#8926) 2019-05-08 16:34:20 +02:00
Jan 30e9fbafe0
Bug fix/cleanup commtasks (#8893) 2019-05-08 16:33:19 +02:00
Markus Pfeiffer 65d090be9d Fix warning about incompatible types in KShortestPathsFinder (#8936) 2019-05-08 15:29:51 +02:00
Heiko 58db1023f3 Feature/aql skip some (#8771)
* testing skipSome implementation, default and passThrough

* added simple implementation for default and passthrough skipsome methods

* added skipRows function to enumerate collection executor

* added tests for the enumerate collection skipRows function

* implemented and tested enumeratecollection skipSome

* skipSome tests

* fixed a test, prepared iresearch view exec for skipping

* rm logs

* gcc bug workaround

* changed to original test code as it has been before

* iresearch skipping, added ires skip test

* added index executor skipRows

* added skip blockfetcher, quick exit index

* test

* const dummy and singlerowfetcher skip

* forgot return value

* input wrong initialized

* trying to remove dynamic cast and solve with a different approach

* cleanup

* const skip

* jslint

* Handle skipSome of subqueries correctly

* Removed unused code

* Removed unused member

* Simplified skip variants and enabled IndexExecutor skipping

* A little cleanup, fixed DependencyProxy::skipSome

* Reverted test change

* Tried to make testLimitBlock3 clearer

* Extended test suite

* Bugfix

* Added stats when skipping and fixed a few other things

* Bugfixes

* Moar bugfixes

* Update arangod/Aql/IResearchViewExecutor.cpp

Co-Authored-By: hkernbach <hkernbach@users.noreply.github.com>

* Update arangod/Aql/IndexExecutor.cpp

Co-Authored-By: hkernbach <hkernbach@users.noreply.github.com>

* Update arangod/Aql/IndexExecutor.cpp

Co-Authored-By: hkernbach <hkernbach@users.noreply.github.com>

* applied requested changes

* Fixed a bug in LimitExecutor::expectedNumberOfRows()

* Fix skipSome in catch test RowFetcherHelper

* Fixed a bug in the tests

* Two bugfixes in LimitExecutor::expectedNumberOfRows

* Avoid passing large batch sizes from skipSome to getSome

* Fixed Windows compile errors

* Fixed a skip bug with WAITING in unsorting gather blocks

* Make aql-skipping find some cluster bugs, like the last commit

* Bugfix and additional asserts

* Fixed skipSome counting in IndexExecutor

* Resolved merge conflicts
2019-05-08 15:02:43 +02:00
Simon 0502a97abb forwardport changes from 3.4 (#8894) 2019-05-08 14:34:25 +02:00
Jan 445d020bac
fix a rare recovery error, add diagnostic output (#8931) 2019-05-08 14:24:26 +02:00
Dan Larkin-York efa180c88f Suppress replication operations from audit output. (#8896) 2019-05-08 10:16:57 +02:00
Tobias Gödderz 20236845e2 Improve document callback performance (#8874)
* Experimental: avoid nested callback

* Still experimental: get rid of more nested callbacks

* Bugfix: count number of documents read correctly

* Rebuild callbacks whenever necessary

* Fixed now dissalowed call.

* Nono rebuild callback refactoring

* Fixed local tests.

* Fixed false compiler warning.

* Use tag dispatching, thanks for the suggestion @mpoeter.
2019-05-07 19:57:24 +02:00
jsteemann c234bf1304 Merge branch 'devel' of github.com:arangodb/arangodb into devel 2019-05-07 15:20:21 +02:00
jsteemann b3819d117d fix some zirkus 2019-05-07 15:19:58 +02:00
Vasiliy 6a5785be40 issue 568.1: restrict analyzer name to characters allowed for collection names (#8923) 2019-05-07 16:10:00 +03:00
Jan Christoph Uhde f3a2eaf6d9 Bug fix/permission tests (#8890) 2019-05-07 15:07:59 +02:00
Heiko 4837c7b864 Bug fix/collect aggregate undefined (#8853)
* allow 3.4 behaviour regarding aggregate collect and undefined variable

* added test to check undefined var

* rm print

* remove duplicate doCopyRow

* wrongly removed due merge

* initialize _allowSourceRowUninitialized always as false as default

* try to fix another case

* added missing parenthesis

* optimized if, also added compiler hint
2019-05-07 15:05:49 +02:00
Markus Pfeiffer d586ff8e95 Add K_SHORTEST_PATHS handling to optimizer and executor (#8895)
There were a few places where K_SHORTEST_PATHS was not handled. This
commit adds them.
2019-05-07 13:54:35 +02:00
Tobias Gödderz 9f09b2fb15 Avoid occasional query id not found errors (#8914) 2019-05-07 13:41:09 +02:00
Markus Pfeiffer c9a35ea2fa Improve k Shortest Paths performance (#8889)
* Only consider spur paths from after the branchpoint
   from the previous path
 * Cache neighbourhoods around vertices
2019-05-07 13:21:52 +02:00
Jan 976dc2b726
Bug fix/issues 2019 05 06 (#8913) 2019-05-07 12:17:16 +02:00
Jan eed6729789
fix an issue, add tests (#8920) 2019-05-07 12:15:43 +02:00
Simon 0a0cb41f04 do not acquire a snapshot for a single read op (#8916) 2019-05-07 09:50:14 +02:00
Jan b06c44472b
less locking for ClusterSelectivityEstimates (#8892) 2019-05-06 13:22:36 +02:00