1
0
Fork 0
Commit Graph

3613 Commits

Author SHA1 Message Date
jsteemann 1588c358b9 options cleanup 2018-07-17 12:33:10 +02:00
Jan 1a128799f2
finalize `--rocksdb.sync-interval` feature (#5856) 2018-07-17 11:09:57 +02:00
jsteemann f7a16f60ea updated changes 2018-07-16 22:12:38 +02:00
jsteemann eedfe80361 remove obviously unused DEPRECATED.md file 2018-07-16 21:48:55 +02:00
jsteemann 36f05c07e0 cleanup of server options 2018-07-16 21:38:35 +02:00
maxkernbach d0aeff6db3 Doc - sync from external repos (#5889) 2018-07-16 18:59:44 +02:00
jsteemann b6cd448c54 update CHANGELOG 2018-07-16 17:24:02 +02:00
sleto-it e0f934e705
Doc - fix build (#5887) 2018-07-16 11:26:27 +02:00
shivamdave24 c45a1b6ac1 Soundex function implementation with integration tests and minor bug fixes to feature/add-aql-tobase64-tohex-encodeuricomponent-uuid branch (#5851) 2018-07-13 17:47:25 +02:00
Wilfried Goesgens 35d87bdbd5 add APIs to the changelog (#5870) 2018-07-13 17:24:22 +02:00
Simran 34ec56d421 Feature/misc spelling corrections (#5164) 2018-07-13 13:06:20 +02:00
Jan Christoph Uhde bcf9e30a4d Feature/cluster single operations (#5585) 2018-07-12 20:18:36 +02:00
sleto-it 8714a9293c
Doc - Starter Recovery Procedure + external sync (#5853) 2018-07-12 14:55:33 +02:00
sleto-it ccde2a5ea2
Doc - Deployment - Missing commit (#5848) 2018-07-11 21:02:41 +02:00
sleto-it a75905b62d
Doc - Additional Deployment Refactor (#5776) 2018-07-11 20:46:19 +02:00
Simran ef6e5dd8df Doc - do not wrap links in Markdown, unclutter page ToC (#5820) 2018-07-11 19:34:38 +02:00
Wilfried Goesgens fca40ca87c Doc - improve error handling in case of empty json file (#5832)
* improve error handling in case of empty json file
* fresh examples
* Update generateMdFiles.py
2018-07-11 19:32:09 +02:00
Simran 3504b737d4 Doc - Re-add automatic pacing paragraph for arangoimport (#5840)
* This paragraph got lost in programs refactor
2018-07-11 19:30:53 +02:00
maxkernbach cf39008acb Doc - new page 'Remove vertex' in AQL examples (#5818) 2018-07-10 19:07:30 +02:00
maxkernbach d3870dc81e Doc - add driver link in the Manual readme.md (#5829) 2018-07-10 19:05:59 +02:00
maxkernbach fad5a0fcdf Doc - fix java driver integration (#5826) 2018-07-10 18:43:26 +02:00
Vasiliy 64666d0edd issue 400.2.4: register IResearchView sync tasks with IResearchFeature, remove unused configuration parameter (#5813)
* issue 400.2.4: register IResearchView sync tasks with IResearchFeature, remove unused configuration parameter

* set flag before notify, fix failures in new tests from 'devel'

* revert set flag before notify, fix more failures in new tests from 'devel'

* fix even more failures in new tests from 'devel', backport: do not reset notification flag if pending tasks were seen

* fix still more failures in new tests from 'devel'

* try to work around buggy test failures
2018-07-10 18:15:01 +03:00
Vinay Pyati bc55b61d04 Doc - Update java driver chapter (#5796) 2018-07-09 18:28:58 +02:00
sleto-it a6c3ce48a9
Doc - Upgrade Paths (#5795) 2018-07-09 18:27:39 +02:00
Tobias Gödderz fc3e11dbbc Async AQL (#5806)
* Modified header to new initializeCursor API

* Adapted initializeCursor to DONE/WAITING API. Compiles but not tested and no one reacts to WAITING state, it is not returned anywhere yet

* Subqueries now expect a WAITING return from initilize cursor. However they will just return a nullptr and pretend the query is empty, this will be fixed later

* First attempt to simulate thread waiting over information within the query

* Small fix to allow for isDirect handlers to go to sleep.

* Waiting in the necessary places now for the async request to be send.

* Thank you auto-casting compiler, you are totally right i absolutely wanted to use this bool value as an index in may Array. How could i possibly not want to use it here?

* Include cond-var header

* Fixed mutex/cond_var usage

* Added oldAPI wrappers in AQL Blocks for get/skip some variants. This Commit compiles but is NOT tested

* Let getSome now return unique_ptr of AqlItemsBlocks. Also implemented the async variant of getSome in subqueries.

* Removed all references to OLD implementations in AQL. only the base wrappers are allowed to call OLD functions from now on. Now the testing part starts

* Fixed endless virtual recursion

* Implemented new getOrSkip API in SortBlock

* Implemented new getOrSkip API in LimitBlock

* Initilaize all variables

* Fixed logic bug in SubqueryBlock

* getBlock in ExecutionBlock now returns a state. All blocks need to handle this properly!

* Createad a wrapper getBlockOld that servers the old sync api and is used now in AQL. To be replaced overtime.

* Added IndexBlock::skipSome and IndexBlock::getSome

* getBlock now returns its old return value along with the state

* Switch from getBlockOld to getBlock in IndexBlock::skipSome

* Switch from getBlockOld to getBlock in IndexBlock::getSome

* ShortestPathBlock::skipSome is not implemented! Added a regression test

* Attempt to fix SubQueryResult memory management

* Fixed LIMIT Block

* Moved from ShortestPathBlock::getSomeOld to ::getSome

* Implemented ASYNC api on SingletonBlock

* Adapted EnumerateCollectionBlock to new async API

* Fixed FilterBlock and adapted return block to async API

* Adapted NORESULTS block to async AQL api.

* Adapted Modification Blocks to async API

* Fixed some initialize cursor functions to reset values required during get/skipSome

* First steps to adapt ClusterNodes to Async AQL api. Not there yet, need to implement the core still

* Added asnyc implementation for xxxForShard in ClusterBlocks. This commit changes internal logic of _doneForShard. Needs additional testing as soon as everything is in place.

* Adapted CalculationBlock to async API

* Adapted TraversalBlocks to ASYNC Aql. This is not optimal yet, we need a better decission if we are DONE or not on RETURN

* Adapted EnumerateListBlock to Async AQL api

* Adapted RemoteBlock to ASYNC API in getSome/skipSome. The whole thing is now LIVE in the cluster. Exetensive testing to be started now

* Fixed IndexBlock WAITING behaviour if Waiting occurs during a index processing

* Adapted IReasearchViewBlock to ASYNC AQL API

* Fixed SortingGatherBlock in WAITING state.

* Adapted IResearch ExecutionBlockMock to Async API

* Unified the HASMORE/DONE distinction. Code is much more readable now and harder to get incorrect 👍

* Implemented tonly heoretically reachable function of non void function.

* Fixed last commit

* Added inline TODO comments

* fix warning

* Fixed a clearing logic bug in RemoveNodes

* Fixed Error Handling in RemoteBlocks. Also fixed a logic bug (true/false simply has a 50% chance of getting it wrong) in Distribute and Scatter.

* remove unused methods

* Fixed failure test

* implement skipping

* Moved the Query Waiting out of the ExecutionEngine.

* changed one of the collect blocks

* Removed _upstreamState from ExecutionBlockMock, that is in the base-class now

* Added a Test Mock for a an ExecutionBlock that simulates the WAITING/HASMORE/DONE api.

* do not check "hasMore" if not necessary

* Added DistinctCollectBlock::getOrSkipSome from ~Old and changed its return type

(still uses getBlockOld)

* Save state to resume in DistinctCollectBlock::getOrSkipSome

* Extracted redundant code

* fixed some ops

* added one more test

* fix endless blocking

* fix compile error

* fix test

* Refactored HashedCollectBlock::getOrSkipSome

* Return blocks to the manager

* Replaced usage of getBlockOld in HashedCollectBlock::getOrSkipSome

* remove unused shutdown calls, simplify ownership for expressions

* Removed superfluous variable

* Capture const variable by value

* Removed SortedCollectBlock::getOrSkipSomeOld in favour of getOrSkipSome

* Added a working version of SortedCollectBlock::getOrSkipSome

Has yet to be cleaned up

* Removed isTotalAggregation special treatment

* On no input, return a group of nulls (instead of no group at all)

* Bugfixes

* Simplified code

* Move return to the end, eliminate duplicate code

* Corrected skipped count in HashedCollectBlock

* Aligned getNextRow() implementations

* Added comments

* some cleanup

* fix potential memleak

* Bugfix

* Fixed failure tests

* Removed usage of getBlockOld in ExecutionBlock::getOrSkipSome

* Replaced hasMore with an async implementation (mostly)

* Removed getBlockOld()

* Added hasMoreState to the AQL API (and renamed hasMore methods to hasMoreState)

* RemoteBlock now uses the async hasMoreState route

* remove job queue

* options

* Bugfixes in the async implementation of LimitBlock

* LimitBlock::getOrSkipSome now always skips when calculating the fullcount

* fix compile warnings

* restrict threads

* Fixed api of Waiting ExecBlockMock. Unused yet

* Made SortedGatherBlock async-capable

* Removed nonEmptyIndex hack

* Removed duplicate traceGetSome~ calls, moved all to getSome

* Added asserts before replacing getNr*Registers

* Added a TODO note and a comment

* Removed getSomeWithoutRegisterClearoutOld()

* Removed skip()

* Removed common code by using getNr*Registers()

* Use getNr*Registers() in the TraversalBlock as well

* started to add lane

* started to add lane

* added lane

* completed lane

* removed debug output

* fixed merge

* Began working on a test suite for AQL tracing/profiling

* Added more tests and asserts in aql-profiler

* Made some ExecutionBlocks final

* Added a type enum to all blocks and the per-block stats

* Add block type to stats nodes when tracing AQL on block level

* Removed initializeCursor call from instantiateFromPlan

* Avoided additional getSome calls after DONE

* Added more profiler tests

* Refactored ExecutionBlock::getOrSkipSome and fixed two bugs

- set _upstreamState also when skipping
- explicitly use xecutionBlock::getHasMoreState()

* Bugfix: update state

* Reuse parent _skipped wherever possible; rename where not (LimitBlock)

* Simplified SortedCollectBlock::getOrSkipSome and reused general pattern & code

* Implemented missing virtual function (with USE_FAILURE)

* Reset neccessary values during initializeCursor

* Simplified code in EnumerateListBlock a little

* Added a test for DistinctCollectBlock in aql-profiler

* Avoid redundant getSome calls in DistinctCollectBlock

* fix compilation

* Fixed DistinctCollectBlock profiler test

* Added a second profiler test for the DistinctCollectBlock

* Added a profiler test for EnumerateCollectionBlock

* Bugfix in EnumerateListBlock

* added --server.fifoN-size

* Simplified EnumerateCollectionBlock::getSome

* Simplified EnumerateCollectionBlock::getSome, and return HASMORE less often when DONE

* Fix testEnumerateCollectionBlock1 for mmfiles

* do not pass by reference

* Fixed compile error

* fixed merge conflicts

* Added profiler tests for EnumerateCollectionBlock

* Test fix for mmfiles

* Fixed IResearch tests

* Bugfix in DistinctCollectBlock and a regression test

* Updated comment

* Bugfix for query statistics in cluster

* Check plan in distinct test

* Fix aql-profiler tests in cluster

* Removed unused line / bugfix for single server test runs

* This commit implements waking up of AQL queries. (#5651)

* Non-compiling intermediate commit for handover.

* Make branch compile again

* Started implementation of continueable rest cursor handler by moving the callbacks to the outer part. This is not yet fully tested!

* Made finalizeExecute noexcept. We cannot react to this errors as the response was potentially written before. Also introduced continueExecution in the RestHandler engine.

* First successful query wakeup.

* The wakeup callback now posts on the scheduler directly. A resthandler only needs to provide a callback that encapsulates the continueExecution call on this handler

* renamed finalizeExecute to shutdownExecute

* Added a differentiation between Handler and Callback in Query continuation. Handler will be posted in IO service. Callback will be executed directly

* fix audit log

* Removed callback from deleteQueryCursor. This cannot be waiting

* use CONDITION_LOCKER

* removed yet another thread-local variable

* Fixed forward declaration

* Made RestAqlHandler repeatable

* Use defer to close the query in RestAqlHandler. Now waiting will close the query as well.

* Added a mutex in the RestHandlers to make sure if the callback over network is too fast that there is only one Thread running in the RestHandler

* Captured the GeneralCommTask if it is posted to a RestHandler. This is necessary in the PAUSED case

* Refactoring of _noLockHeader responsibilities. Now the BaseHandler selects them and resets them after it is done. Only Coordinators are allowed to define them if a query is loaded.

* Removed reaction to existing nolockheaders in Coordinator Query Planning Phase

* Removed incorrect assertion.

* Further refactoring of NoLockHeaders. Now there is a wrapper class around it which allows for debugging and logging. The state now seems to be better. Also all non-rest-handler triggered queries clean up the NoLockHeaders properly.

* Fixed UserManager, now deletes nolock headers properly

* Swing to the Symphony of Destruction

* Forgot about community build...

* Fixed compiling of Catch tests

* Fixed community build

* need thread for size

* Made the restSimpleHndler repeatable

* Implemented dump and dumpSync in Cursors, Sync will block a thread, dump allows to wait, only relevant for Streaming cursor

* Reactivated StreamingCursors

* Removed debug output.

* Fixed false query continuation

* Reset thread output to non-debug

* Added missing return statements

* Allow some CollectionMethods to hand-in a context that may contain a transaction. This is meant to honor nolock headers.

* Fixed hidden merge conflict

* Bugfix in aql-profiler.js: use plan.nodes order, not stats

* Added two profiler tests for filter

* Avoid too many getBlock calls in the FilterBlock

* Removed debug output

* RemoteBlock API will now send a done(bool) flag whenever we request documents from remote Servers. It is possible that we are DONE and have a result. The pre 3.4.0 API uses exhausted which is exclusive to a result. This API is still implemented for beckwards compatibility.

* Implemented an executeSync function in AqlQuery. This will block the thread until query execution is complete

* Added another test for FILTER, and one test for the HashedCollectBlock

* Added more tests for HashedCollectBlock; avoid unneccessary getSome calls

* Added an profiler IndexBlock test

* IndexBlock: avoid redundant getSome calls, added missing traceGetSomeEnd calls

* Added a second test profiling IndexBlock

* Added a third test for IndexBlock

* Moved general code to module

* Moved noncluster tests into a separate file

* Split aql-profiler testsuite into three files

* Added profiler tests for LimitBlock

* Added a test for NoResultsBlock

* Added profiler tests for TraversalBlock

* Shutdown of an AQL query is now asynchronous. However in Error-Cases it will be executed in a blocking way still

* Optimized TraversalBlock getSome calls due to new (nightly) test results

* Fixed std::min calls I broke

* Let shutdown calls in AQL wait, if the query is executed successfully.

* Fixed queryResult going out of scope

* fix compile error through merge conflict with devel

* Fixed compiler warning "mismatching tags"

* Removed debug log output

* Added TODO notes

* Fixed test fail due to devel merge

* Fixed some invalid sync waiting implementations

* Added a profiler test for SortBlock

* Added profiler tests for SortedCollectBlock

* Fixed bug introduced by devel merge

* Fixed Remoteblocks ignoring errors!

* Added some more continue Callbacks in used places. And removed debug log

* Removed debug log output

* Suppress clang warnings

* Bugfix: use of invalid stack pointer

* Bugfix: RemoteBlock::shutdown now sends code as int, not string

* Revert "Suppress clang warnings"

This reverts commit 05591649c59743c992edd5e78814edc8ca2a83e0.

* Bugfix: cleanup state in RemoteBlock ::shutdown, ::getSome and ::skipSome

* Bugfix in Subquery shutdown: don't skip subquery shutdown when main query shutdown failed

* Allow copy elision
2018-07-09 14:24:10 +02:00
sleto-it edd5eba814
Doc - Cluster Administration Improvements (#5799) 2018-07-09 02:20:31 +02:00
Wilfried Goesgens bd8976956c Doc - Add example anchors (#5752) 2018-07-06 16:19:36 +02:00
jsteemann dd36442c47 added missing function parameters to documentation 2018-07-05 22:51:25 +02:00
Dan Larkin-York a5ef1e8e86 Finishing touches for cursor load balancer support. (#5787) 2018-07-05 22:48:32 +02:00
sleto-it 939b63cea3
Doc - Update ChangeRootPassword.md 2018-07-05 17:58:57 +02:00
Simon f47367a495 Allow dumping / restoring of views (#5774)
* allow dumping and restoring of views

* fix restore, adding some tests

* remove debug print

* fix global replication
2018-07-05 17:55:28 +02:00
Omar_A 97987d6c57 Doc - added EE note (#5784) 2018-07-05 16:18:57 +02:00
Omar_A 16f3a57014 Doc - Change Root Password page (#5771) 2018-07-05 10:44:44 +02:00
Jan f9dc610c1e
Bug fix/internal issue 2658 (#5760) (#5778) 2018-07-05 10:19:28 +02:00
Wilfried Goesgens 5ed2ca5a56 Doc - fix link layer1 (#5763) 2018-07-04 19:20:10 +02:00
jsteemann 2d622bcd9e updated CHANGELOG 2018-07-04 19:02:50 +02:00
Jan 0001fbae92 Doc - fix planning issue 2623 (#5725) 2018-07-04 18:16:56 +02:00
Simran eef962d619 Doc - Fix UPDATE example, add details about single document operations (#5222) 2018-07-04 18:07:33 +02:00
Jan Christoph Uhde 5bf28ee587 Doc - fix aql-insert-options (#5633)
* fix aql-insert-options
* Syntax, INTO over IN, be explicit about OPTIONS being optional
* partial work of a more wider effort
2018-07-04 16:57:10 +02:00
Simran 26814b9918 Doc - Add ArangoSearch as top-level chapter in Manual and AQL docs (#5702)
* Add ArangoSearch as top-level chapter in Manual and AQL docs
* Update SUMMARY.md
* `Views/README.md` already has content, so we shouldn't remove it. It acts as intro page to ArangoSearch, which is okay-ish. The actual ArangoSearch page appears as sub-chapter called _Usage_.
* Change ArangoSearch chapter names for Manual and AQL book
* Fix links
2018-07-04 14:46:26 +02:00
Simran 9fcab2fbe5 Docs - Add drivers integrations section (#5754) 2018-07-04 14:44:36 +02:00
Simran 2ebad67906 Doc - Startup parameter usage improvements (#5759) 2018-07-04 13:58:15 +02:00
Simran 38839c336e Doc - Remove Crawling Github and SailsJS recipes (#5764) 2018-07-04 13:55:47 +02:00
Andrey Abramov 17e955d837
ArangoSearch advanced AQL functions (#5756)
* extend filter checking

* extend IResearchViewNode tests

* distinguish scorer and filter arangosearch functions

* initial implementation

* add some tests for the recently added aql functions

* more tests

* force phrase function to use context-wise analyzer

* more tests

* ensure exists function uses analyzer from context

* fix query execution tests

* allow to override context analyzer in phrase function

* modify EXISTS signature to accept optional analyzer, fix tests

* more tests

* even moar tests

* more tests

* update documentation

* add testing scorer

* fixes after review

* attempt to fix msvc compilation issues

* add extra check

* add another check
2018-07-04 02:59:31 +03:00
maxkernbach 10a2709318 Doc - fix documentation build script + a few other things (#5751) 2018-07-03 18:12:36 +02:00
Jan b88fde9659
added key generators "uuid" and "padded" (#5740) 2018-07-03 15:23:47 +02:00
Simon 545561e9a9 Read only server (#5652) 2018-07-03 09:58:16 +02:00
Simran 0aa1ae24b3 Doc - Take whatever line endings and turn them into LF in generation (#5714) 2018-07-02 15:34:35 +02:00
Wilfried Goesgens 8c586a1a91 Doc - reference swagger on a prominent place (#5731) 2018-07-02 15:33:23 +02:00
Wilfried Goesgens 8c39f9f666 Doc - fix links (#5739) 2018-07-02 13:56:09 +02:00