1
0
Fork 0
Commit Graph

5753 Commits

Author SHA1 Message Date
Jan Christoph Uhde 8f4475524b escape internal REs (#8857) 2019-04-26 15:15:53 +02:00
Jan Christoph Uhde 89449df97d Avoid some exceptions in environment feature when reading files. (#8834)
The changes have been added to reduce the noise during startup.
2019-04-26 10:21:34 +02:00
Jan Christoph Uhde 677a79026c Foxx Security (#8845) 2019-04-25 09:56:29 +02:00
Kaveh Vahedipour fe65959a05 🍻 get rid of all specialised container output code 🍻 (#8798) 2019-04-18 17:50:35 +02:00
Dan Larkin-York 48fb15e94f Auth-related tests for audit logging (#8790) 2019-04-18 10:45:41 +02:00
Jan 6ee9206e46
clear traverser cache in-between (#8772) 2019-04-17 15:27:37 +02:00
Simon 5bb9d2dc17 transaction manager tests (#8759) 2019-04-16 21:26:40 +02:00
Jan 7fd456a83b
don't reveal information via logs or to JavaScript apps (#8763) 2019-04-15 19:10:03 +02:00
Jan c4d8091f45
remove unused attributeExcludeHandler (#8760) 2019-04-15 18:55:59 +02:00
Dan Larkin-York 777ba1a364 Improved audit logging (#8740) 2019-04-15 14:51:32 +02:00
Jan 449ab1ed8e
Bug fix/cppcheck 13042019 (#8752) 2019-04-15 10:13:56 +02:00
Simon 937d743ba6 Bug fix/pregel stuff (#8733) 2019-04-11 15:58:28 +02:00
Jan 5d83eb7442
this looks better (#8680) 2019-04-04 15:51:40 +02:00
Jan d1d1fe555b
Bug fix/better error message in case something goes wrong http wise (#8670) 2019-04-03 18:44:10 +02:00
Vasiliy f4919dc173 issue 526.6: implement REST and V8 handlers for the iresearch analyzer feature (#8626)
* issue 526.6: implement REST and V8 handlers for the iresearch analyzer feature

* address typo

* remove excess comments

* temporarily comment out tests failing on MacOS

* temporarily comment out more MacOS-only test failures
2019-04-02 19:23:28 +03:00
Jan 616ea94f24
Bug fix/cleanup 31032019 (#8632) 2019-04-01 17:14:11 +02:00
Jan 531b55f225
introduce option `--log.time-format` (#8604) 2019-03-29 17:15:33 +01:00
Jan 9ab9cc7857
disambiguate internal exceptions (#8623) 2019-03-29 15:59:37 +01:00
Vasiliy 3a3c360765 issue 526.5: add analyzer retrieval functionality (#8592) 2019-03-29 13:23:36 +01:00
KVS85 2d5bf58c94 Remove USE_IRESEARCH definition and usage (#8613) 2019-03-28 17:23:51 +01:00
Dan Larkin-York b40177cb30 Fix bug in thread creation failure handling. (#8601) 2019-03-28 16:36:24 +01:00
Jan 0265beeb06
reintroduce check for vm.overcommit_memory setting (#8603) 2019-03-28 12:49:57 +01:00
Jan 14720b85be
add AQL functions CRC32 and FNV64 (#8600) 2019-03-28 03:06:33 +01:00
Jan Christoph Uhde deb770caba fix generateLogIds.py (#8570)
* fix generateLogIds.py, update README_mainatiners

* Add missing `or` in doc.

* remove quotes before inserting into map
2019-03-26 09:19:16 +01:00
Jan dbcc9767fa Feature/turn on unique log ids (#8572)
* turn on unique log message IDs by default

* add missing feature documentation

* update CHANGELOG
2019-03-26 09:18:32 +01:00
Jan 80a6e621ee
don't allocate memory so often in ClusterComm requests (#8550) 2019-03-26 00:31:56 +01:00
Jan Christoph Uhde c3f7961b88 apply unique log ids (#8561) 2019-03-25 20:26:51 +01:00
Jan e078f35285
fixed typos, removed unneeded includes (#8547) 2019-03-25 12:09:37 +01:00
Jan 39a3f5bc4e
reintroduce smart joins after temporarily reverting them in devel (#8543) 2019-03-23 20:36:02 +01:00
Dan Larkin-York 44810a1218 Improved index error messages. (#8532) 2019-03-22 20:26:09 +01:00
Jan cfd3418115
microoptimizations (#8519) 2019-03-22 16:53:21 +01:00
Simran b442e68fed
Doc - setIntroducedIn for log.file-group/mode (#8486) 2019-03-22 12:28:09 +01:00
Simon 3ada15fc35 The Legendary El Cheapo (#8485) 2019-03-22 11:38:33 +01:00
Jan a35f6ccf10
fix type of hidden startup option `--rocksdb.recycle-log-file-num` (#8517) 2019-03-22 11:21:53 +01:00
jsteemann dc381a99df Revert "Feature/ncc1701 (#8440)"
This reverts commit 59ad583796.
2019-03-21 19:18:46 +01:00
Jan 59ad583796
Feature/ncc1701 (#8440) 2019-03-21 15:05:36 +01:00
Jan 00ac1db850
add RocksDB options `--rocksdb.allow-fallocate` and `--rocksdb.limit-open-files-at-startup` (#8493) 2019-03-21 14:21:35 +01:00
jsteemann 11efed7510 fix compile warnings 2019-03-20 14:28:19 +01:00
jsteemann 2de5ca2e67 pretend to use result marked as warn-unused 2019-03-20 14:21:51 +01:00
Frank Celler ea9be71998 added --log.file-mode and --log.file-group (#8450) 2019-03-20 14:21:18 +01:00
Manuel Pöter b6f2222140 Abort the process if a stopped thread does not terminate within 5min. (#8357) 2019-03-20 14:20:47 +01:00
Jan 2c4accdf34
micro improvements for rocksdb options (#8433) 2019-03-19 12:47:57 +01:00
Dan Larkin-York 2eadab33e7 Index hints (#8431) 2019-03-19 09:14:18 +01:00
Simon 49cc3bcd1e Refactorings from cluster trx improvement branch (#8391) 2019-03-14 23:13:17 +01:00
Jan a1897d4e84
expose ARANGODB_USE_CATCH_TESTS define, make TEST_VIRTUAL be based on it (#8410) 2019-03-14 12:53:57 +01:00
Dan Larkin-York 413e90508f Named indices (#8370) 2019-03-13 18:20:32 +01:00
Jan f6d817b8ba
remove some unused stuff (#8350) 2019-03-08 15:52:39 +01:00
Jan c82e0c2ca0
add option `--console.history` to arangosh (#8327) 2019-03-07 13:05:54 +01:00
jsteemann 6a828e7a11 show thread name in fatal error messages 2019-03-07 11:34:15 +01:00
Kaveh Vahedipour 68178ba165 [devel] supervision bug fix backports (#8314)
* back ports for supervision fixes from 3.4 part 1

* back ports for supervision fixes from 3.4 part 2
2019-03-04 19:27:24 +01:00
Frank Celler 2329dd1ee2
Feature/data masking improvments (#8129) 2019-03-04 10:47:26 +01:00
Jan 5d2ab0c901
port from 3.4 (#8275) 2019-02-28 14:36:29 +01:00
Manuel Pöter bed5a5f9a2 Set thread's state to `STOPPED` when the thread function finishes. (#8283) 2019-02-28 11:54:49 +01:00
Jan 1798036ea0
Bug fix/optimizations 18022019 (#8180) 2019-02-19 19:24:04 +01:00
Jan 44c6a2d732
Feature/ttl index (#8169) 2019-02-19 14:12:21 +01:00
Wilfried Goesgens 492d05c1f1 Feature/upgrade v8 7.1.302.28 (#8088) 2019-02-19 11:15:34 +01:00
Dan Larkin-York c755330db8 Make message generation lazy. (#8173) 2019-02-18 09:40:15 +01:00
Dan Larkin-York f4c2347fbd Make Result final (#8157) 2019-02-15 20:05:30 +01:00
Luschgy b7b84beebf Feature/aql blocks by line (#6553)
* Fixed hidden merge conflict

* Added BlockFetcher, added a minimal test for SingleRowFetcher

* move RegInfo in types.h

* fix getSomeLogic

* Added the AqlItemMatrix implementation. Not yet tested!

* Fixed nr called => Waiting ratio in AllBlocksFetcherMock

* fix compiling

* add RegInfo to produceRow

* BlockFetcherHelper now returns a matrix

* pass RegInfo as requested into AqlItemRow ctor

I would still keep it out as the ItemRow should be a thin view.

* allow to change the AqlItemRows cheaper / remove _sourceRow member

* implement copyRow for AqlItemRow

* organize cleanup for outputblock used in getsome / shrink if necessary

* Added tests for SingleRowFetcher, cleanup in AqlItemRow

* start mocking

* Added test helpers, began work on a more complex test for SingleRowFetcher

* Re-added RegInfo again to AqlItemRow

* helper, added test case for FilterExecutor

* merge fixed compile

* Removed trx name where it is actually not used

* Made some functions TEST_VIRTUAL for mocking.

* implemented Sort executor

* Fixed the BlockFetcherHelper API, to always return a matrix if it is not waiting

* Implemented a SortExecutor test with data

* disable borken test

* Fixed test on SortExecutor. Green now, needs some more testing

* change a bit how the AqlItemRow is used in a test

* Added another test for the SingleRowFetcher

* Added more tests for SingleRowFetcher

* fix getRegisterInfo

* Moved RegInfo into ExecutorInfos. Also now use the FilterExecutor implementation in production code instead of FilterBlock. Starting integration testing.

* add more asserts

* Fixed undefined behaviour in TestCode.

* Do not shrink an empty output block

* switch to block pointers in AqlItemRow

* return blocks in the row fetcher

* Updated grammar.cpp

* Updated (fixed) tests to verify returnBlock

* Improved const-ness

* Changed BlockFetcher constructor in order to make it mockable

* Implemented BlockFetcherMock and rewrote some tests with it

* Updated a comment

* Updated comments

* init commit - EnumerateListExecutor, including EnumerateListExecutorInfos

* add potential fix for ItemMatrix

* Replaced fakeit with BlockFetcherMock in another GIVEN block

* Replaced the last fakeit mocks with BlockFetcherMock and removed fakeit

* Updated function definition

* included missing arary included

* Renamed AqlItemRow to InputAqlItemRow

* Split AqlItemRow into Input~ and Output~ classes

* EnumerateListExecutor + Tests

* Added tests for AqlItemRows

* Added test file for AqlItemRows...

* Made the ostream operator for AqlItemBlocks available to Logger and Catch info

* build error

* Added a second test for AqlItemRows. Need to make input space invalid now, sth. acidential referencing will show up

* Introduce a block ID in the InputAqlItemRow

* Copy output values correctly, avoid InputAqlItemRow pointers

* Added a test for registerClearout

* Fixed tests

* Added a test for adding new values

* Added a test for an incomplete row write to not count as produced.

* Fixed getSome and a test

* Added a test for larger AQL values in AqlItemRows

* Fixed compile error in hidden merge conflict

* Added tests for Filter counter in FilterExecutor

* OutputAqlItemRow now takes responsibility of the block

* Handed responsibility of blocks in ExecutionBlockImpl to OutputAqlItemRow

* EnumerateListExecutor should be close to its finish. Tests need to be fixed, not ready yet.

* better swap

* OutputAqlItemRow now takes ExecutorInfos

- added several assert
- implemented a more strict and safe produced() logic

* enum list test

* Added a more specific error message

* Removed OutputAqlItemRow.changeRow()

* fix invalid block freeing

* changed order of input/output registers

* Fixed a test

* Fixed matrix width in test

* fix failure tests

* finished enumlist tests

* Implemented AllRowsFetcher and tests for it

* Bugfix in skipSome stub

* insert sort and enumerate list executors

into exectuion nodes' create functions

* fixed singlerowfetcherhelper, added tests

* small fixes

* add missing Infos

* restore accidentally deleted scorer

* make it compile

* fix executor info creation / usage

* make use of output var in EnumarateListNode

* Added Executor::Stats and FilterStats

* Extended FilterExecutor tests with stats, added node tracing to getSome

* Fixed Sort-&EnumerateList- Executor to use the new return type with stats

* Fixed compile error in tests

* Changed *ExecutorInfo constructors and classes to contain the necessary register informations

* Made an ExecutorInfo construction more readable

* Updated comments

* Fixed checks in OutputAqlItemRow::setValue()

* Add comparison method to SortRegister. Forbid construction without one.

* Implemented ExecutionBlockImpl::initializeCursor

* more mocking inside ExecutionBlockImplTest

* Minor changes

* Avoid AqlValueGuard destroying the value copied into the output block

* Disabled failure tests that do not apply to the new Sort implementation

* further mocking of test modules

* Removed constexpr for compatibility with outdated gcc versions

* add some failure tests - getsome works differently

therefor we do not test the old behavior

* fix sort node

* clean up alq item matrix

* Removed the transaction methods from read-only access to AqlValues (get and at) and repliced by CollectionNameResolver where necessary. Still left: some calls extract VPackOptions from the Transaction. Compare needs a BuilderLeaser only.

* textexecutionhelper, execimpl test

* Eliminated FilterBlock

* Eliminated the EnumerateListBlock

* first successful running test of executorblockimpl

* Eliminated SortBlock

* fixed testexecutor helper

* Made the ExecutorInfos private to the executors again.

* Implement and use AqlItemBlockShell (#6562)

* Added AqlItemBlockShell

* Minor changes

* Mostly replaced AqlItemBlock with ~Shell.

Current state:
- arangod compiles
- OutputAqlItemRow has yet to be changed
- catch tests have yet to be fixed

* Use AqlItemBlockShell in OutputAqlItemRow

* Added registersToKeep to AqlItemBlockShell

* Removed ExecutorInfos from OutputAqlItemRow

* Split AqlItemBlockShell into Input~ and Output~

* Changed tests so they compile again - they're still not yet working.

* Fixed tests

* fixed building, modified impl test

* Removed now superfluous BlockFetcherMock functions

* more execimpl testing

* Removed block IDs

* more impl testing

* Reduced dependencies of BlockFetcher construction

* Bugfix, cleared documentation

* Removed debug-output

* Adressed minor code review remarks

* Moved method definitions into .cpp files to avoid includes

* Added assertions

* Added comments

* Addressed several more review comments

* Added missing includes

* more testing, not finalized yet

* added missing header file

* added a test for execution block impl

* removed old comment

* Improved lookup algorithm to identify correct block/row combination in AqlItemMatrix

* added test for ExecutionBlockImpl

* move utility to the right position

* Bugfix and a TODO note

* Bugfix: use a uniform ResourceMonitor in tests

* Made OutputAqlItemRow::setValue clone by default and copy on move

* Replaced the getRow lookup with a binary search

*  enumerate tests

* Disable emplacing of AqlValues in ItemBlock

* Reenabled std::enable_if will warn until oskar is using c++17 as standard.

* Add helper function to create a shared pointer of an unordered_map<RegisterId>

* CalculationExecutor (#6570)

* add CalculationExecutor skeleton

* first shot of CalculationExecutor

* fix minor bugs

* handle references

* use new shared pointers

* commit to share work

* fix failure test

* reformat with clang-format

* add test for CalculationExecutor

* fix CalcualtionExecutor test

* clean up

* Removes unused code

* clean up includes

* fix/remove comments

* add changes requrested by hacki

* ReturnExecutor (#7932)

* add skeleton to CMakeLists.txt

* build most parts of the ReturnExecutor

* This commit removes ReturnBlock and adds some debug messages

removed inheritRegisters functionality

* finish ReturnExecutor

* add test for return executor

* remove unused code

* add stats and clean up includes

* verify result in ReturnExecutorTest and move *Stats to "Aql/Stats.h"

* Fixed algorithm to find correct row in a MultiRow Input

* fix full count and failure tests (#7969)

* fix full count and failure tests

* fix filter test

* WIP: fixes

* fixed

* NoResultsExecutor (#7953)

* add NoResultsExecutor

* clean up

* some review changes

* fix test

* fix

* fix WAIT -> DONE

* remove unused code (#7995)

* create skeleton for SingletonExecutor (#7957)

* create skeleton for SingletonExecutor

* WIP: continue implementation

* WIP: add fetcher skel

* WIP: add inject function to ConstFetcher

* rename SingletonExecutor to IdExecutor

* WIP: replace singleton node

* WIP: More Debugging on IdExecutor

* WIP: IdExecutor

* WIP: add failure tests

* make iresarch test work but not pass

* WIP: add ConstFetcherHelper

* fix test

* fix tests

* fix iresearch tests

* fix wording

* add review changes

* add review change to SingleRowFetcher as well

* fix test

* fix error when increasing index

* temporarily remove fold-expression

* clean up

* Fix gcc compile error

* Feature/aql limit executor (#7982)

* limit executor

* removed not needed tri assert

* rm _invariable in limitnode

* find out if we are in a subquery

* queryDepth in limit infos

* added missing queryDepth

* finalizing limit executor

* added more tests, fixed a condition

* add assert to test

* added missing fullcount check

* typo

* minor changes, fixed test

* test

* wrong check

* fixed tests

* false merge

* remove basic blocks

* remove basic block includes

* include sort order

* removed not needed basic blocks

* removed not needed basic blocks

* fix test

* Feature/remove input output shells (#8090)

* Added template parameters to allow for block pass-through to be chosen at compile time

* Adapted the tests to match the new template parameters

* Adapted ConstFetcher and IdExecutor to match the new templated fetcher

* Implemented pass-through, fixed SingleRowFetcherTest

* Fixed implicit merge conflicts

* Fix assumptions that are violated by block pass-through

* Added 2 regression tests

* Fixed test expected data

* Fix nullptr dereference

* Fix handling of non-invalidated input variables

* Fix nullptr dereference

* Fix handling of non-invalidated input variables

* Fixed compile error

* Fixed a useless and leaking allocation

* Try a less implicit fix

* Allow IdExecutor and ConstFetcher to be used with pass-through

* Performance patches from @mchacki

* Fixed merge conflicts, and bugs in ConstFetcher

* Fixed atMost pass-through, and blocks that call their parents too often

* Removed InputAqlItemBlockShell (except from tests)

* Removed OutputAqlItemBlockShell (except for tests)

* Moved implementations of OutputAqlItemRow methods in the header

* Moved SingleRowFetcher::fetchRow to the header to allow inlining

* Reenabled returnInheritedResults optimization

* Enable passThrough for ReturnExecutor with inherited results

* Specialized CalculationExecutor to avoid unneccessary conditional jumps

* Made tests compile again (but are currently broken)

* Fixed ReturnExecutor test

* Move created InputRows out of AqlItemMatrix

* Limit- and EnumerateList Executors no longer fetch from upstream after DONE

* Added comments

* Added experimental AqlItemBlock optimizations

* Removed _numEntries, added comments

* Allow executors to pass an "atMost"-limit to the SingleRowFetcher (for LIMIT)

* Let the LimitExecutor pass atMost upstream

* Bugfix for FullCount

* Feature/aql enumeratecollection executor (#8028)

* initial commit

* enum coll executor

* buildable version of enumeratecollectionexecutor

* wrong usage of outVariable

* Added template parameters to allow for block pass-through to be chosen at compile time

* Adapted the tests to match the new template parameters

* Adapted ConstFetcher and IdExecutor to match the new templated fetcher

* more fixes, still some tests failing

* Implemented pass-through, fixed SingleRowFetcherTest

* Fixed implicit merge conflicts

* adding missing set value (remember optimize todo), optimized constructor

* removed old enumerate collection blocks

* Fix assumptions that are violated by block pass-through

* Added 2 regression tests

* Fixed test expected data

* Fix nullptr dereference

* Fix handling of non-invalidated input variables

* Fix nullptr dereference

* Fix handling of non-invalidated input variables

* Fixed compile error

* Fixed a useless and leaking allocation

* added broken uncompilable executor test

* Try a less implicit fix

* fix mocking

* enumerate collection executor test

* remove class, use only static function.

* clean up test after merging with main branch

* Allow IdExecutor and ConstFetcher to be used with pass-through

* Performance patches from @mchacki

* Fixed merge conflicts, and bugs in ConstFetcher

* Fixed atMost pass-through, and blocks that call their parents too often

* Removed InputAqlItemBlockShell (except from tests)

* Removed OutputAqlItemBlockShell (except for tests)

* Moved implementations of OutputAqlItemRow methods in the header

* Moved SingleRowFetcher::fetchRow to the header to allow inlining

* Reenabled returnInheritedResults optimization

* Enable passThrough for ReturnExecutor with inherited results

* Specialized CalculationExecutor to avoid unneccessary conditional jumps

* Made tests compile again (but are currently broken)

* Fixed ReturnExecutor test

* Limit- and EnumerateList Executors no longer fetch from upstream after DONE

* Removed an assertion which seems to be incorrect during bootstrap.

* Inlined AQL ItemMatrix and removed unused elements in other Executors

* Now inline + Move in AqlItemMatrix, seems to be overall best performing

* Bugfix: Do not returnInheritedResults on DBServers

* Fixed LIMIT test, and made it more thorough.

* Fixed createion of NoResultBlock creation in IResearch Nodes. Also relaxed an assertion

* Reactivated failure point, now identical with anotherone in OutputRow, need to unified

* Bug fix: do not mutate the offset in the LimitExecutor (#8109)

* Introduced test

* Fixed LimitExecutor to not mutate its infos, so it works correctly in subqueries

* Fixed crashes in CatchTests

* Fixed non-maintainer build

* try to fix subqueries

* Fix isInSubQuery for non-singleton nodes.

* remove useless while true

* Feature/aql blocks by line dan sort (#8143)

* Add SortNode::SorterType to SortExecutorInfoss.

* prepare imple of sort executor

* WIP: add some code for dan sort

* WIP: working at pushRows

* WIP - try to add Dan's heap-sort

* re add missing test

* 2 fixes

* fix

* Dan's tests are working

* remove unused variable

* remove SortBlock.cpp from CMakeLists.txt

* remove hack and provide an extra method to set the _baseIndex

* Simplified ContrainedSortExecutor.

* fix forgotten variable

* update doc

* Assert that we do not use functions that might return wrong results.

Preconditions or invariants the functions rely on are broken. Therefore
the functions can not be used. This is asserted in maintainer mode.

* add function to restore row invariant

* do not emplace and pop_back if not necessary

* Use clear registers in case we can move the rows.

* WIndows opst zirkus

* Actually destroy AqlItemValue

* Feature/traversal executor by line (#8000)

* Added stubs for TraversalExecutor, no implementation, not compiled in yet

* Included production code of TraversalExecutor and made it compile. Now implement

* Implemented production code for trivial tests, if no row is reaching the traverser.

* Fixed TraversalExecutor code to pull all rows from above and added tests. No traversal yet, this is next

* Intermediate blind commit might not work

* Fixed compile issue in test code

* Created a shared place for Mocks in tests and moved the StorageEngineMock there, also started a shared place for mock servers, most test will need the same server in order to work.

* Fixed travesalExecutor tests. Now able to create tests with actual input.

* Added successful test in TraversalExecutor, that Traverser as started with correct start vertices (by inputRegister)

* Further test cases

* Added stubs for TraversalExecutor, no implementation, not compiled in yet

* Included production code of TraversalExecutor and made it compile. Now implement

* Implemented production code for trivial tests, if no row is reaching the traverser.

* Fixed TraversalExecutor code to pull all rows from above and added tests. No traversal yet, this is next

* Intermediate blind commit might not work

* Fixed compile issue in test code

* Created a shared place for Mocks in tests and moved the StorageEngineMock there, also started a shared place for mock servers, most test will need the same server in order to work.

* Fixed travesalExecutor tests. Now able to create tests with actual input.

* Added successful test in TraversalExecutor, that Traverser as started with correct start vertices (by inputRegister)

* Fixed leftovers from rebase

* Refixed mocks in different folder

* Fixed Mock import

* Fixed Calculators Exector Test to use the unified mocked server.

* Implemented logic for the TraversalExecutor. Also added tests for it. Not fully complete though

* Added validation of result values

* Added usage of constant register in traversal executor

* Included filter variables in traersal executor. Added test for const input. And moved MMDR into base traverser class for simplicity

* Added ostream operators on AQLValue and AqlInputRow. Not 100% done yet but good enough

* Added a shutdownable ExecutionBlock. Some blocks need to implement a specific shutdown still. I added this class to make it easier to eliminate them when the time is due.

* Removed ostream operator for INPUT row again, does for some reason not compile on Linux0o

* Fixed initialization order

* Removed classwrapper for shutdown.

* Applied review changes

* Refixed failing tests, API is not subject to change

* Fixed CatchTest (wrong usage of string literal in test case)

* Feature/shortest path executor (#8042)

* Added stubs for TraversalExecutor, no implementation, not compiled in yet

* Included production code of TraversalExecutor and made it compile. Now implement

* Implemented production code for trivial tests, if no row is reaching the traverser.

* Fixed TraversalExecutor code to pull all rows from above and added tests. No traversal yet, this is next

* Intermediate blind commit might not work

* Fixed compile issue in test code

* Created a shared place for Mocks in tests and moved the StorageEngineMock there, also started a shared place for mock servers, most test will need the same server in order to work.

* Fixed travesalExecutor tests. Now able to create tests with actual input.

* Added successful test in TraversalExecutor, that Traverser as started with correct start vertices (by inputRegister)

* Further test cases

* Added stubs for TraversalExecutor, no implementation, not compiled in yet

* Included production code of TraversalExecutor and made it compile. Now implement

* Implemented production code for trivial tests, if no row is reaching the traverser.

* Fixed TraversalExecutor code to pull all rows from above and added tests. No traversal yet, this is next

* Intermediate blind commit might not work

* Fixed compile issue in test code

* Created a shared place for Mocks in tests and moved the StorageEngineMock there, also started a shared place for mock servers, most test will need the same server in order to work.

* Fixed travesalExecutor tests. Now able to create tests with actual input.

* Added successful test in TraversalExecutor, that Traverser as started with correct start vertices (by inputRegister)

* Fixed leftovers from rebase

* Refixed mocks in different folder

* Fixed Mock import

* Fixed Calculators Exector Test to use the unified mocked server.

* Implemented logic for the TraversalExecutor. Also added tests for it. Not fully complete though

* Added validation of result values

* Added usage of constant register in traversal executor

* Included filter variables in traersal executor. Added test for const input. And moved MMDR into base traverser class for simplicity

* Added ostream operators on AQLValue and AqlInputRow. Not 100% done yet but good enough

* Added a shutdownable ExecutionBlock. Some blocks need to implement a specific shutdown still. I added this class to make it easier to eliminate them when the time is due.

* Removed ostream operator for INPUT row again, does for some reason not compile on Linux0o

* Fixed initialization order

* First draft of shortest path executor

* Fixed logic bugs, Local AQL tests succeed

* Added first version of ShortestPathExecutor tests. Edge output tests are missing yet.

* Added EDGE output tests

* Replaced TraversalStats with NoStats, they have never been tracked...

* Removed obsolete ShortestPathBlock.

* Removed classwrapper for shutdown.

* Applied review changes

* Refixed failing tests, API is not subject to change

* windows macro opst circus

* Clear registers per block, not per row (#8164)

* Fixed compilation under linux again...
2019-02-14 19:13:08 +01:00
Manuel Pöter a0975b4062 Switch to thread-local RandomDevice instances. (#8150) 2019-02-14 15:35:42 +01:00
Jan 820ba5dd7c
micro optimizations (#8156) 2019-02-13 16:14:28 +01:00
Frank Celler b9fae3e47e
added run-time openssl version (#8121) 2019-02-08 14:14:48 +01:00
Frank Celler e4c7b0e18e
added a warning if compiled in maintainer mode (#8116) 2019-02-06 19:55:01 +01:00
Simon 1123bc10c0 Check authentication for cluster APIs (#8051) 2019-02-04 14:47:44 +01:00
Jan 0053a72a6a update velocypack (#8076) 2019-01-31 17:31:14 +01:00
Jan d45906e569
Bug fix/31012019 (#8078) 2019-01-31 17:27:18 +01:00
Frank Celler 603612f1d2
Feature/maskings documentation (#8018) 2019-01-30 12:01:49 +01:00
Jan f7d94daba2
port some changes from 3.4 (#8011) 2019-01-29 09:26:57 +01:00
jsteemann 8c5e7423b2 Revert "Do not acquire snapshot for single read (#8041)"
This reverts commit 91931ace4d.
2019-01-28 16:05:27 +01:00
Manuel Pöter ecf4d9d62a Fix race conditions in thread management. (#8032) 2019-01-28 15:44:46 +01:00
Simon 91931ace4d Do not acquire snapshot for single read (#8041) 2019-01-28 13:15:51 +01:00
Frank Celler 9927b3a281
Feature/maskings (#8006) 2019-01-22 22:23:25 +01:00
Jan 5502ed2a3a
vadim (#7983) 2019-01-22 14:06:17 +01:00
Jan 6944115de6
various optimizations (#7994) 2019-01-21 13:40:39 +01:00
Jan 1e3b323a46 move some date functionality into lib (#7997) 2019-01-21 10:43:37 +01:00
Jan 8e7b0067c7
Bug fix/fix catch issues (#7970) 2019-01-17 14:24:09 +01:00
Simon f748aee240 Added collectAll, updated fuerte (#7949) 2019-01-16 11:31:08 +01:00
Tobias Gödderz a1d3bc3e94 Foxx queue jobs hanging after Foxxmaster crash (#7922)
* Fixed bug where the Foxxmaster doesn't reset jobs after a crash when it should, or a non-master coordinator removes jobs in progress during startup

* Added a regression test

* Updated CHANGELOG

* Fixed non-maintainer compile
2019-01-14 16:08:08 +01:00
Jan 5670698d7b
fix issue #7933: Regression on ISO8601 string compatibility in AQL (#7935) 2019-01-14 13:51:13 +01:00
Jan 9ce9312239
speed up containers that contain ExecutionNodes' variables (#7937) 2019-01-14 13:38:56 +01:00
Jan e42befdc52
do not simplify non-deterministic conditions (#7926) 2019-01-11 14:48:34 +01:00
Wilfried Goesgens 457e14b970 add windows suspend/resume hooks as @char101 sugested for others (#7833)
* add windows suspend/resume hooks

* disable procdump
2019-01-11 12:41:27 +01:00
jsteemann 63f5379115 fix OpenSSL warnings about deprecated functions 2019-01-11 09:37:48 +01:00
Jan Christoph Uhde aabb307295 fix openssl 1.3 usage 2019-01-10 15:27:27 +01:00
Jan fa7de56cf8
upgrade to boost 1.69.0 (#7910) 2019-01-09 17:17:33 +01:00
jsteemann 0fe3b61fc4 remove unused function 2019-01-09 10:21:13 +01:00
Lars Maier 423cf7a8d4 cppcheck/Scheduler (#7909) 2019-01-08 16:39:56 +01:00
Jan b40b88d73b
upgrade bundled curl library to version 7.63 (#7846) 2019-01-08 11:27:59 +01:00
Lars Maier 12eebb15fe Feature/new server infra (#7733)
* Decoupled IO handling from Scheduler.

* Fixed SSL start up bug.

* Replaced Scheduler with new worker farm implementation.

* Added minimal statistics and info string for Scheduler.

* Added support for timed submissions.

* Updated delayed submission api. Updated code that used timers.

* Extracted new Scheduler into a virtual parent class. The implementation can now depend on the usecase.

* Signal handler now working.

* Changed threads names, `_stop` is atomic, check for failure during thread start + exception handling like old scheduler did.

* Commented on source code and added TODOs.

* Played around with start-stop-conditions

* Play around with start stop condition.

* start stop cond

* Sart Stop Conditions

* Removed bad cv_status check.

* Bug fix: now compare the actual objects instead of pointer values. Setup t1 and t2 depending on the thread id.

* Moved most of the stuff now unrelated to the Scheduler to GeneralServer. Got rid of JobGuard.

* Instead of waiting for a thread to terminate, put it on a clean up list and check for its termination in each supervisor run.

* Allow detaching long running threads.

* Fixed test mock.

* Updated the WorkHandle logic. Removed post functions.

* Fixed crash when obtaining shared_ptr from this in destructor.

* Added lost mutex.

* Fixed memory leak.

* Fixed merge bug.

* Changed a lot of code to optimize the scheduler.

* Fixed bug of invalidated iterator. Dont remove task on shutdown at different places. Let scheduler threads run until queue is empty.

* Only by value calls to queue.

* Added options again.

* Clean up of code.

* UI Request Lane added.

* Bug fixes in Scheduler.

* Applied reformat.

* Use sigaction.
2019-01-08 10:12:02 +01:00
Jan bd5a291d41
simran (#7881) 2019-01-04 17:09:03 +01:00
Frank Celler ac9f375fb5 big reformat 2018-12-26 00:54:03 +01:00
Simon a2a0b03f43 Rdb index background (preliminary) (#7644) 2018-12-21 19:24:10 +01:00
Jan cc66e7f91f
copy enterprise files too when starting with `--javascript.copy-installation` (#7818) 2018-12-20 17:47:10 +01:00
Wilfried Goesgens c30947961f Fix Cluster shutdown handling of http connections (#7809) 2018-12-19 19:22:11 +01:00
Jan ea39dfa3d2
fixed issue #7757 (#7805)
* fixed issue #7757

* fix compilation error
2018-12-19 14:06:52 +01:00
Jan 4dc64cbab5
remove unused thread functionality (#7769) 2018-12-18 18:28:13 +01:00
jsteemann 489ff0aa94 add virtual dtor 2018-12-12 18:57:21 +01:00
Wilfried Goesgens 7acb3b3ec4 wait for procdump to exit too. (#7731)
* wait for procdump to exit too.

* move running process check up, propagate status. First check whether processes are still running before attempting to force kill them

* only list processes, don't change their state

* add stopping procdump in all places
2018-12-12 18:06:08 +01:00
jsteemann 2de632d2e7 remove inclusion of valgrind.h 2018-12-12 10:29:05 +01:00
Jan 465ab147bb
Bug fix/fix thread shutdown (#7728) 2018-12-11 22:18:33 +01:00
Jan fb04c1c602
Bug fix/forward port missing changes (#7706) 2018-12-10 12:51:20 +01:00
Wilfried Goesgens c0f9e8125a Bug fix/allow tcp connection to finish (#7635) 2018-12-10 10:38:34 +01:00
Jan 945922ad87
minor fixes (#7689) 2018-12-07 16:22:25 +01:00
Jan 5bae3742e5
Feature/internal 3306 (#7683) 2018-12-06 16:19:28 +01:00
Wilfried Goesgens ea44f8394d try checking whether _admin/shutdown is received, print sockstats of arangod processes (#7677) 2018-12-06 15:52:46 +01:00
jsteemann 67c98cb9c7 remove currently unused method 2018-12-04 20:05:05 +01:00
Jan 7c0d1e9349
don't make Logger write empty strings (#7636) 2018-12-04 19:24:10 +01:00
Jan 056363a3b3
cleanup (#7597) 2018-12-03 16:17:45 +01:00
Frank Celler 710ca04fb1
Feature/maskings [WIP] (#7347) 2018-11-30 18:21:37 +01:00
Dan Larkin-York 8449a185ec Port #7490. (#7559) 2018-11-30 17:26:29 +01:00
Jan 14c598c194
allow using UTF8 filenames for UUID directory (#7568) 2018-11-30 16:44:04 +01:00
Wilfried Goesgens 879c7bb479 slash all left over processes (if) and set status to bad. (#7557) 2018-11-30 16:24:07 +01:00
Jan 643b58a347
fix path normalization for Windows (#7566) 2018-11-30 16:21:30 +01:00
Jan be0f8b59fa
added option `--rocksdb.enforce-block-cache-size-limit` (#7505) 2018-11-28 21:02:45 +01:00
Jan 59868fdf04
Bug fix/use lock for pregel stats (#7499) 2018-11-28 18:39:27 +01:00
Jan 683d45bd68
Feature/micro optimizations 28112018 (#7496) 2018-11-28 15:57:10 +01:00
jsteemann b4e888ef13 revert Scheduler changes 2018-11-26 09:57:53 +01:00
Michael Hackstein 16d0874da5
Bug fix/synchronous replication catchup (#7146)
* merged fixes from 3.4

* odd fix

* Bug fix 3.4/sync repl release thread (#6784)

* First attempt to not block the thread that requires the EXCLUSIVE sync-up lock

* Fixed insertion of query into registry in rest replication handler.

* Removed unnecessary / false asserts as suggested in review. Fixed code comments.

* Replaced auto with a correct type as suggested in review

* Added a helper function to validate if a query is in use in the registry

* Fixed logic bug in usage of query registry

* Fixed compile issue

* Automaticly transfrom int -> bool in initializerlist sucks...

* Inverted boolen logic bug hidden due to int->bool beeing logically inverted.

* Today it seems that bools are too complicated for my brain.

* Removed failure point, didn't write a test for it, and it is hard to write it in the current test environment. Need to find a better solution in future

* Applied chenges required by @goedderz in review

* Bug fix 3.4/shorter foot in door (#7084)

* Implement `syncCollectionCatchup` in DatabaseTailingSyncer.

First stab, might not even compile.

* Fixed a typo.

* Fix a typo and a compilation problem.

* Further compilation fix.

* Implement two stage catchup.

* Two small corrections.

* Unified error messages in Synchronize shard job.

* Improved a code comment.

* Fixed autocasting bool->double and double->bool issue. That is truely one of the best features ever invented... </irony>

* Renamed doHardLock => toSoftLockOnly and inverted default value

* Merged soft/hard foot logic with Transaction splits

* Use scopeguards to cancel readlocks

* Bug fix 3.4/sync replication allow soft and hard lock (#6864)

* First attempt to not block the thread that requires the EXCLUSIVE sync-up lock

* Fixed insertion of query into registry in rest replication handler.

* Removed unnecessary / false asserts as suggested in review. Fixed code comments.

* Replaced auto with a correct type as suggested in review

* Added a helper function to validate if a query is in use in the registry

* Fixed logic bug in usage of query registry

* Fixed compile issue

* Implemented optional 'doHardLock' parameter in the replication acquire read-lock call. A hard-lock guarntees to stop all writes, a soft-lock may not.

* Fixed compile issue

* Automaticly transfrom int -> bool in initializerlist sucks...

* Inverted boolen logic bug hidden due to int->bool beeing logically inverted.

* Today it seems that bools are too complicated for my brain.

* Removed failure point, didn't write a test for it, and it is hard to write it in the current test environment. Need to find a better solution in future

* Applied chenges required by @goedderz in review

* Renamed doHardLock => toSoftLockOnly and inverted default value
2018-11-23 16:16:34 +01:00
Simon d5cb94d2d0 Minor refactoring (#7408) 2018-11-22 16:16:05 +01:00
Jan c7869f1c46
Bug fix/remove shutdown assertion (#7388) 2018-11-22 15:35:55 +01:00
Jan b360c21f32
remove some unused StringUtils functions (#7402)
* remove some unused StringUtils functions

* fix compilation
2018-11-22 09:41:47 +01:00
Jan 33f18942df
yet more micro optimizations (#7400) 2018-11-21 17:09:09 +01:00
Kaveh Vahedipour 9ec6619b84 Bug fix/index readiness (#6541)
* indexes are marked  while still missing in Current
* index handling getCollection
* supervision gets indexes from isbuilding, when coordinator is gone before finishing
* seems right now
* fixed broken views
* remove junk comments
* cleanup
* node / supervision adjustements
* supervision fixes
* neunhoef remarks part i
* neunhoef remarks part ii
* neunhoef remarks part ii
* neunhoef remarks part iiI
* collection's current version please
* no need to wait for current once again
* no longer necessary code
* clear comments
* delete left overs
* dead code revived
2018-11-21 14:42:58 +01:00
Jan bf2eeb16cd
Bug fix/aql speedup (#7379) 2018-11-20 16:07:40 +01:00
Jan 0dd1776467
Make recovery more reliable (#7297) 2018-11-19 13:59:18 +01:00
hellower 94626595cd Update messages.h (#7360)
Korean bye message
2018-11-19 09:17:33 +01:00
Markus Pfeiffer 39bdebf851 Port bug-fix-3.4/timeout-create-coll to devel (#7307)
* Fix loophole in error handling.
* Fix inquiry case of id not found: 404.
* Also handle correctly in AgencyComm.
* Fix agency tests.
* Fix error handling in dropCollectionOnCoordinator.
2018-11-14 10:03:55 +01:00
Jan 21979e6906
Bug fix/do not create routing collection (#7290) 2018-11-13 16:54:06 +01:00
Jan a5db298c92
fix buffer overrun, remove unused variable (#7302) 2018-11-13 14:18:50 +01:00
jsteemann 02cf9013e3 fix define name 2018-11-12 09:49:22 +01:00
jsteemann f90cb7455f add some debug information in case temp directory cannot be created 2018-11-10 23:13:36 +01:00
Wilfried Goesgens 0ce4c715b2 rather use null-pointers than try/catch for control flow (#7271) 2018-11-08 16:27:44 +01:00
Dan Larkin-York b447f98a7a Upgrade old MMFiles collections for compatibility with ArangoSearch. (#7248) 2018-11-07 15:52:37 +01:00
Jan 41a12626c7
turn off warnings about `overcommit_memory` settings (#7234) 2018-11-07 11:07:31 +01:00
Matthew Von-Maszewski 0d39ff66f5
Bugfix: backport defensive Communicator change and revert constant change in Scheduler (#7214)
* revert accidental change to MIN_SECONDS

* from bugfix-3.4/mv-communicator-defensive:  simplify lambda usage to static functions.
2018-11-05 15:18:31 -06:00
Simon 311fb717f2 Bug fix/refactor query optimizer (#7228) 2018-11-05 18:30:12 +01:00
Vasiliy 68953ae33a issue 496.4.1: move StorageEngine-specific flag out of the genric API and closer to the storage engine (#7212) 2018-11-04 16:52:28 +03:00
Jan 5853344237
add ge goodbye (#7205) 2018-11-03 20:17:03 +01:00
Jan c7d76c5da4
fix some issues reported by codescan (#7195) 2018-11-02 19:04:39 +01:00
Simon b883026f45 Server stream cursor (#7186) 2018-11-02 16:19:48 +01:00
Jan 976cc38e7c
remove OpenFilesTracker (#7189) 2018-11-02 11:38:38 +01:00
Jan 4bc646f207
increase the maximum allowed number of collections in an AQL query (#7154) 2018-10-31 17:20:30 +01:00
Matthew Von-Maszewski a054e31f73 port 3.4 changes that give libcurl time to retry a failed DNS query. Also add changes to vocbase.cpp that were missed in previous PR. (#7132) 2018-10-30 21:00:13 +01:00
Simon c72818a9dc Make ensureIndexOnCoordinator more robust (#7110) 2018-10-29 17:45:46 +01:00
Jan 7765a48bb6
speed up query string parsing (#7115) 2018-10-29 15:45:42 +01:00
Tobias Gödderz c7c838ed30 Stop curl from trying to POST stdin (#7098)
* Stop libcurl from trying to POST stdin

* Stop relocking every iteration in wait

* Restrict setting of empty POSTFIELDS to POST requests

* Revert locking change
2018-10-29 14:41:28 +01:00
jsteemann 446cb6d988 fix compilation error that I caused 2018-10-29 12:57:24 +01:00
Jan 5719a7fc99
remove unneeded nullptr checks (#7121) 2018-10-29 11:51:27 +01:00
jsteemann 64cbee76a5 report presence of AVX2 instruction set in `--version` 2018-10-27 19:18:12 +02:00
Matthew Von-Maszewski 97ba8ca2be Bugfix: More 3.4 scheduler changes backported (#7091) 2018-10-26 17:09:20 +02:00
Wilfried Goesgens 4a32fcd648 Bug fix/improve invalid parameter handler (#7077)
* don't use icu to convert utf16 here, it may already be de-initialized.

* don't use atexit handler, so we can use _exit on windows

* use _exit() here.
2018-10-25 17:26:30 +02:00
Simon 10dc287eb3 Silence Tsan warnings (#7075) 2018-10-25 15:50:39 +02:00
Jan 28e22d60be
add option `--rocksdb.total-write-buffer-size` (#7025) 2018-10-25 11:36:26 +02:00
Jan df54403db6
const-expressify some VPack functions (#6993) 2018-10-23 11:32:46 +02:00
Simon 4cfff35cef Use SSE42 for UTF-8 string validation (#6991) 2018-10-23 11:32:06 +02:00
Jan 87d5d29e5f
Feature/optimizations 211018 (#6994) 2018-10-23 09:57:24 +02:00
Jan a161fe78a0 don't fiddle with thread structs that were not initialized yet (#6980) 2018-10-23 09:53:11 +02:00
Jan 360888c3ef
do not copy node_modules directory on startup (#6945) 2018-10-22 16:43:13 +02:00
Simon 23b8e5c735 Propper commit Sequence Numbers (#6958) (#6964) 2018-10-19 14:19:57 +02:00
Simon 8b7a4099b8 Properly compare velocypack objects in Agency operations (#6921)
* Properly compare velocypack objects in Agency operations

* Add changelog

* added option for VPackDumper
2018-10-17 20:03:53 +02:00
Simon 63cfdad52e Fix SimpleHttpClient assertions (#6914)
(cherry picked from commit 0d36790cbaaf8b9f7492534e3c9e5318da909e0f)
2018-10-16 20:31:59 +02:00
Wilfried Goesgens 5b020e465f add missing include (#6896) 2018-10-15 16:31:12 +02:00
Simon 6628a4e55a Refactor stuff, add async batch extension task (#6875) (#6880) 2018-10-15 13:18:24 +02:00
Wilfried Goesgens 67439484e4 add file and linenumber to windows stacktraces (#6865) 2018-10-15 09:35:07 +02:00
Simon fd81f52ab4 Allow WAL logger to split up transactions (#6800) (#6866) 2018-10-12 17:50:58 +02:00
Jan a0980ed2f2
Bug fix/fixes 111018 (#6843) 2018-10-12 12:51:11 +02:00
Jan 96edc0f6ff
Bug fix/fixes 121018 (#6853) 2018-10-12 12:47:30 +02:00
Matthew Von-Maszewski e2bc7e10e3 port of 3.4 libcurl connection management to devel (#6775) 2018-10-10 09:10:34 +02:00
jsteemann 63136348b0 honor our coding guidelines, remove useless debug log message 2018-10-08 16:36:59 +02:00
Lars Maier fac7b48c74 [3.5] Feature/decoupled io (#6281)
* Decoupled IO from Scheduler.
* Fixed SSL start up bug.
* Updated messages and thread names. Fixed missing code from cherry-pick.
* Reintroduced checks for executing thread to be correct. Modifed default value for io-context depending on cores.
* Fixed memory leak caused by cyclic references.
* Actually distribute endpoints. Move handlers into function and do not copy them for each encapsulation.
* Inserted debug output.
* BUG FIXED! One has to call drain() on every queue as temporary work around.
* Added some flags and output for testing.
* More debug output!!!
* Manuel is right.
* Removed debug output.
2018-10-08 13:05:12 +02:00
Jan e78d1aa541
Bug fix/even more ldap debugging (#6736) 2018-10-08 09:42:11 +02:00
Lars Maier 090024daed Reset retry counter only when previously connected in order to not disable the retry check. (#6737)
Abort the doRequest loop when in shutdown.
2018-10-05 17:10:44 +02:00
Jan 3414825696
Feature/improvements 031018 (#6712) 2018-10-04 15:10:54 +02:00
Wilfried Goesgens c1b6426e92 Feature/drop before win7 support in compiler (#6681) 2018-10-03 18:33:52 +02:00
jsteemann 319c05c55d remove unused variables 2018-10-03 13:25:07 +02:00
jsteemann 56147843c2 Merge branch 'feature/additional-logging' of https://github.com/arangodb/arangodb into devel 2018-10-02 17:53:31 +02:00
Jan c06f2d77da
Feature/velocypack update (#6678) 2018-10-02 14:04:14 +02:00
Dan Larkin-York b922d260bc More cleanup and additional logging. 2018-10-02 07:50:26 -04:00
Wilfried Goesgens 73ee5cb40e Fix ICU path building, windows codepage correctness (#6679) 2018-10-02 12:38:24 +02:00
Wilfried Goesgens a1e32118db Bug fix/fix console codepage (#6646) 2018-10-01 17:25:07 +02:00
Jan 4b2588d74f
Bug fix/remove valgrind (#6644) 2018-09-28 17:51:53 +02:00
jsteemann eac5f3edfd better add this include 2018-09-28 15:21:50 +02:00
Jan dade5b1bb1 fix race on Logger shutdown (#6637) 2018-09-27 19:30:27 +02:00
Simon 25f5c841b9 Fix recovery tests on MacOS (#6610) 2018-09-27 18:52:28 +02:00
jsteemann 805a1c5d51 cache pid for logging 2018-09-27 17:38:41 +02:00
jsteemann 39cbd8adcc Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2018-09-25 15:57:36 +02:00
jsteemann 3e41f9ca2b dont reveal passwd 2018-09-25 15:57:22 +02:00
Jan 5a13afd96d
modify msync code so that only base address is page-aligned (#6585)
according to man msync, there is no need for the end address to be page-aligned
also use uintptr_t consistently instead of intptr_t
2018-09-25 09:37:03 +02:00
Wilfried Goesgens a477df49cf Feature/windows utf16 fileaccess (#6534) 2018-09-24 19:41:17 +02:00
Jan abb2ffd98f
centralize some redundant SSL options handling (#6576) 2018-09-24 13:03:48 +02:00
Simon 912f109968 Add simple Future library (#6464) 2018-09-21 16:14:17 +02:00
jsteemann 98a3fecb6a silence a compile warning about potential string value truncation 2018-09-20 01:18:57 +02:00
Simon 0fa7f01c66 Resilience test failure points (#6539) 2018-09-20 01:05:10 +02:00
Simon 82aa24ad7e Copy installation files on startup (#6491) 2018-09-14 11:15:21 +02:00
Max Neunhöffer 84735955ea Add advertised endpoints. (#6104) 2018-09-13 16:30:55 +02:00
Simon 22b9c31c13 Removing ClusterComm ClientTransactionID (#6294) 2018-09-12 22:15:16 +02:00
Jan 9e7fa8420d
align mmfiles index elements to at least pointer alignment (#6460)
this is required for systems with strict alignment requirements
2018-09-12 13:56:06 +02:00
Heiko 2ec1a5cc26 Feature/new devel versioning scheme (#6312) 2018-09-11 08:57:38 +02:00
Jan d1241b1deb
Bug fix/aql modify cleanup (#6380) 2018-09-11 08:54:45 +02:00
Jan d9f669e085
now that we are on c++14 and std::make_unique is available everywhere, we can safely remove our shim for std::make_unique (#6429) 2018-09-10 16:19:06 +02:00
Simon da4585ddad Address leak sanitizer warnings (#6275) 2018-09-10 12:10:55 +02:00
Vasiliy 5329f34771 issue 465.2.2: remove redudnant heap allocations and simplify API (#6349)
* issue 465.2.2: remove redudnant heap allocations and simplify API

* address merge issue

* address more merge issues

* address more merge issues

* address review comments

* do not deallocate non-allocated instances
2018-09-05 13:37:37 +03:00
Simon 0661a4c1fe Hide Links from getIndexes() (#6325) 2018-09-03 15:17:24 +02:00
Jan 07abfca588
Bug fix/cleanup 020918 (#6338) 2018-09-03 12:56:41 +02:00
Jan c8ff719665 Bug fix/always remove lock file (#6314)
* always remove LOCK file

* disable useless messages about environment for upgrade check and password reset
2018-09-01 12:09:23 +02:00
Vasiliy e862efdc3b issue 458.4: retrieve the system database via the SystemDatabaseFeature (#6299) 2018-08-31 19:45:10 +02:00
Jan 840ad8c20d
added option `--rocksdb.use-file-logging` (#6308) 2018-08-31 17:23:00 +02:00
jsteemann 1093c00cd1 add path to error message 2018-08-27 22:31:57 +02:00
jsteemann 1aa162d6d6 add error message 2018-08-27 22:20:34 +02:00
Jan 5022ccc24d
Bug fix/fixes 2508 (#6254) 2018-08-27 21:36:39 +02:00
Lars Maier 5555bd2fad Schmutz++ Improved (#6259)
* Fixed startup order. Don't start maintenance threads in single-server or agent.
Added range check for `--server.maintenance-threads`.
Fixed invalid array access, when shard exists locally but not in plan.
* Removed unused header imports.
* Added CHANGELOG entry
* Fixed shutdown bug. Startup fixed.
* Fixed catch test.
* Add Maintenance improvements to NewFeature34.md.
2018-08-27 20:25:09 +02:00
Jan 601e2ccbc6
Feature/aql 1808 (#6191) 2018-08-27 14:56:57 +02:00
Kaveh Vahedipour 28754cbf15 Feature/schmutz plus plus (#5972)
- Schmutz now called "Maintenance" and completely implemented in C++
 - Fix index locking bug in mmfiles
 - Fix a bug in mmfiles with silent option and repsert
 - Slightly increase supervision okperiod and graceperiod
2018-08-24 12:15:35 +02:00
Simon 948820e484 Various small changes (#6234) 2018-08-24 09:39:03 +02:00
Simon 247d41287b More fuerte changes (#6212) 2018-08-22 10:08:33 +02:00
Jan cb12be3e4e
fix crash when pressing CTRL+c in arangosh (#6187) 2018-08-21 11:12:11 +02:00
Simon 229c09d434 Allow dirty-reads from passive (#6136) 2018-08-20 16:26:14 +02:00
jsteemann eca02eaaf1 added "license" return attribute 2018-08-18 16:22:04 +02:00
Matthew Von-Maszewski 86ea784372 bugfix: establish unique function name & implementation for communication retry status (#6150)
* initial checkin of isRetryOK().  Includes fixes to known code that has previously hung shutdowns by performing infinite retries.

* slight help on getting out of a loop faster during shutdown.  not essential.
2018-08-17 14:57:12 +02:00
Wilfried Goesgens e5843d32f8 Improve windows installer (#6045) 2018-08-17 12:37:47 +02:00
Jan d4d0f44f87
add `enterpriseOnly` documentation sections (#6175) 2018-08-17 10:28:25 +02:00
jsteemann 34869f9324 add kernel ASLR detection 2018-08-16 15:13:44 +02:00
Jan d6a3b66e2a
micro optimizations (#6162) 2018-08-16 08:50:16 +02:00
Vasiliy 6fd541d110 issue 427.5: use ApplicationServer reference instead of pointer (#6145)
* issue 427.5: use ApplicationServer reference instead of pointer

* address MSVC build failure
2018-08-15 12:16:02 +03:00
Frank Celler a688dc0962
Feature/remove job queue thread (#5986)
limiting V8 calls in flight
2018-08-10 12:17:43 +02:00
Tobias Gödderz de4f5587ae Gharial rewrite in C++ (#5631)
* Built a C++ skeleton REST handler for gharial, with fallback to the JS handler

* Moved aql::Graph to graph::Graph

* Added complete edge definitions to Graph

Also:
- some cleanup
- used forward-declarations in headers
- use Graph in graph rest handler

* Handle graph lookup failures according to the test suite

* Added GET vertex

* Bugfixes in ResultT

- Added missing #include
- Fixed move semantics

* Move central code of readVertex to GraphOperations

* ResultT fixes and complements

* Implemented a graph cache

* Added and used graph cache to the rest handler

* Added GET edge

* Added DELETE edge

* Extracted some code

* Added PATCH and PUT for both edge and vertex

* Moved update/replace transaction code to GraphOperations

* Added stub routes for POST and a TODO note

* Added a test checking that deleting a vertex removes all incident edges as well

* Added a test checking that deleting a vertex does not remove edges in non-graph collections

* fixed compiler warnings and errors

* Began work on DELETE vertex

For this, added a V8Context to allow for AQL queries to use subtransactions

* Continued work on DELETE vertex (still WIP)

* prep for graph post routes

* fixed removeVertex operation (aql)

* added post vertex and post edge gharial routes

* wasSynchronoues flag changed

* gharial post c++ handler, naming conventions

* added gharial tests

* temporary disabled cache (because not completed), added graph property read functions

* added c++ gharial list vertex collections

* added c++ gharial graph config

* added c++ gharial list graphs

* added graph manager class

* first implementation of create graph in c++, WIP

* changed error messages

* added etag to create graph api, still multiple edge definition check missing

* finished POST /_api/gharial/<graph>

* WIP - DELETE /_api/gharial/<graphName>

* added DELETE /_api/gharial/<graphName> validation, still missing correct response

* gharial delete

* fixed delete gharial lock

* finished DELETE /_api/gharial/<graphName>

* added routes for graph based vertices and edge definitions

* improved delete route

* added add new edge definition to existing graph

* patch edge definition in a graph, still <WIP>

* finished edit edge definition route

* code changes due to devel code changes

* added remove edge definition route

* added vertex delete function

* added todo note regarding one drop collection issue

* add oprhan collection to graph route implemented

* Added a test

* Updated a comment

* Several minor changes

* Minor changes during review

* Changes during review

* Changes during review

* Bugfix: orphans may be null or omitted

* Bugfix: resolve externals

* minor code changes

* seperated graph class to independent component classes

* seperated graph class to independent component classes

* removed log output

* fixed create collection behaviour in a cluster environment

* fixed enterprise graphs behaviour in c++ gharial api

* removed log output

* formatting

* improved error handling, fixed a linux compile bug

* more result refactoring

* more result type cleanup

* fixed wrongly defined test

* result handling

* error handling

* more refactoring

* Bugfix: avoid race condition in cluster when creating collections

* updated graph documentation

* added graph related static strings

* static strings, new method to create options for gharial created collections

* Some minor cleanup

* more use of static strings

* minor code changes, review

* added missing parseint

* removed gharial foxx, added js common module, added v8 general graph module

* correct use of virtual method

* more v8, js general graph, broken state

* more v8 graph functions

* fixed editEdgeDefinition, added drop function

* fixed drop behaviour

* added _list, _exists

* added c++ rename graph collections, added v8 + graph module function

* Added a regression test

* added graph._deleteEdgeDefinition, v8, server

* more v8g

* added _removeVertexCollection

* added _extendEditDefinitions

* todo, need to add a helper sort method for a local defined relation

* fixed test

* fixed lots of tests, added more client functions, _addVertexCollection on client module is still broken

* added more client graph functions, all tests green

* more client functions

* add del edge def route

* Fix use after move

* Minor changes in client general-graph.js module

* Make a copy before sorting (don't touch the argument)

* Minor changes and some additional asserts in graph tests

* Consistently set parameter defaults

* Renamed static strings

* Remove superfluous function

* Made comment more verbose

* Minor changes in general-graph-common.js

* Added missing template arguments

* Fixed community build

* Cleanup in editEdgeDefinition

* Regression test & bugfix: comparison of edge definitions didn' order from and to

* Fixed errors introduced by merge

* Minor changes in v8-general-graph.cpp

* Fixed test failure due to wrong error code in CE

* added missing id field

* Added permission checks for graph._create

* Removed assertion that is no longer valid

* Moved removeGraph from GraphOperations to GraphManager

* Allow C++ implementation of graph._drop to handle smart graphs

* Flush js client db cache after creating/dropping collections via the general graph module

* Added _deleteEdgeDefinition to the general graph client module

* WIP: Added permission checks for drop graph

* Fixed permission checks for drop graph

* Added permission checks for other graph operations

* Bugfix: assert edge definitions are returned in order

* Some cleanup

* Removed unused method

* Minor improvements in GraphManager

* Fixed a type in general-graph common module

* Most useful fix of all times ever: Do not auto cast from bool to int and alternate error/noerror by this

* Added the initial keyword to StaticStrings

* Added a new error code, used whenever a user tries to inject a documentcollection as a relation into the graph, which is invalid

* Some GraphManager/Ops/Graph cleanup. Less Slice parsing, more usage of GraphObjects

* Test edgeDefinitions in graphs with a defined ordering

* GraphClass Layout cleanup

* Do not test error messages, use codes instead

* Recreated backwards compatibility of Graph Creation Permission errors

* Changed error-code if edgedefinition is used twice

* Added a StaticString for the GraphName

* Renamed graphToVpack => graphForClient

* Partly fixed graph-api test to work with better error messages. Still red: The edgeDefinitions are now sorted, the test is supposed to sort his own list, but appearently does not do so. Under investigation

* Added a new error code that rejects injection of differently sharded smart collection into smartgraph. Should be more helpful to our users

* graph createCollectionOptions now require an open object to be cross-called from enterprise. Made enterprise switch for creation of graph more elegant.

* Updated graphs.cpp

* Massive refactoring. Made Factories for graphs to make SmartGraph much more transparent. Also reduced amount of multiple implementations of the same stuff. Killed vocbase/graphs use GraphManager instead. Removed usage of GraphCache, was not completely implemented anyway and only partially used, which is bad at the moment. Option for later improvement never the less

* Adapted JS code to now really use c++ variants. ALso included 3 Classes: Graph, SmartGraph and GraphModule.

* Fixed undefined behaviour in Remove Vertex. Fixed smartgraph sharding if one collection already exists.

* Removed DEBUG output

* Removed DEBUG logs

* Removed dead code

* Fixed Graph EdgeDefinition test, they now have a different ordering.

* Added a test when adding a vertexCollection that it is actually valid in the graph

* Client Graph API now correctly sends `orphanCollections` and not `orphans`

* Let GraphOperations modify the graph in-place. It should now properly handle edgeDefinitions.

* Added initial cid StaticString

* Included the vocbase in fromPersistence creation of Graphs. Only required to enhance 3.3 SmartGraphs on the fly.

* Fixed internal error message

* Fixed compiler isses originiated from merging

* Removed unused imports

* Regenerated generated file
2018-08-09 09:30:04 +02:00
Simon 606c15ce48 VST support in arangosh (#6047) 2018-08-08 12:18:54 +02:00
Jan b278d6874a
allow master & slave to work in parallel for RocksDB WAL tailing (#6059) 2018-08-03 13:37:53 +02:00
Dan Larkin-York b2db010969 Register version and storage engine info in agency to facilitate rolling upgrades (#6061) 2018-08-03 09:46:04 +02:00
Jan dc2a6d3066
always look in etc/relative first for config files (#6049) 2018-08-03 09:38:08 +02:00
Frank Celler da7ef0a025
added __attribute__((unused)) and other minor tweaks to silence some of the warnings (#6035) 2018-08-01 13:22:45 +02:00
shivamdave24 d4c6431ee5 Regex Split Implementation (unfinished) (#5991) 2018-08-01 12:09:36 +02:00
jsteemann ac94de2e96 fix some replication issues 2018-07-27 19:03:43 +02:00
Jan 1f16898027
added more RocksDB options and stats (#6006) 2018-07-26 16:48:25 +02:00
Jan e4d7f1c5f0
Bug fix/wenn der shard mann 2mal klingelt (#5890) 2018-07-26 15:37:40 +02:00
Simon 2dd8593609 View Replication (#5915) 2018-07-26 10:28:46 +02:00
shivamdave24 7db28da251 Levenshtein Distance function implementation with documentation and i… (#5922) 2018-07-25 12:48:58 +02:00
Jan 21064144c8
Bug fix/replication improvements (#5962) 2018-07-25 09:04:50 +02:00
Jan a5bb50b0bf
remove methods from VelocyPackHelper that are also in VPackSlice (#5946) 2018-07-25 09:01:29 +02:00
jsteemann 7d57dead38 use pthread_attr_init instead of memset 2018-07-24 18:09:04 +02:00
Michael Hackstein f7373fe103
Bug fix/lock file cleanup (#5960)
* Added cleanup functions class that allows to execute functions on unexpected shutdown (like FATAL_EXIT). First use: delete the LOCK file. (#5218)

* Removed superfluous log message
2018-07-24 09:43:15 +02:00
Jan ac1d5aac9b
allow starting agency with --console again (requires V8 then) (#5927) 2018-07-24 09:34:22 +02:00
Jan 3319829432
do not run storage engine equality check too early (#5935) 2018-07-24 09:34:11 +02:00
Dan Larkin-York 718e3f7bfa Add load balancing support and user restriction for async jobs API (#5906) 2018-07-23 15:23:18 +02:00
jsteemann ed167ddcd4 fix warning 2018-07-23 14:59:05 +02:00
jsteemann 2138dc0479 Merge branch 'bug-fix/fixes-1707' of https://github.com/arangodb/arangodb into devel 2018-07-19 16:02:30 +02:00
jsteemann d5be86d9ce Merge branch 'bug-fix/rw-lock-changes' of https://github.com/arangodb/arangodb into devel 2018-07-19 11:17:00 +02:00
jsteemann 5c10fc24da fix typo in comment 2018-07-19 10:58:16 +02:00
Simon Grätzer 80baf2bb11 update copyright, use uin32_t everywhere 2018-07-18 23:43:44 +02:00
jsteemann c2ffc3d21d Merge branch 'devel' of https://github.com/arangodb/arangodb into bug-fix/fixes-1707 2018-07-17 13:30:18 +02:00