Hawkin
/
arangodb
mirror of https://gitee.com/bigwinds/arangodb
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
45,346 Commits 358 Branches 495 Tags 994 MiB
Commit Graph

1401 Commits

Author SHA1 Message Date
Simon 1145c7c053 Improve estimator apply algo + test (#8871) 2019-04-29 17:08:14 +02:00
jsteemann 3f4ea17b32 RocksDB returns certain statistics values as strings. honor that 2019-04-26 19:05:23 +02:00
Simon 6381659a53 fix assertion (#8860) 2019-04-26 18:51:34 +02:00
Simon 2f1cfa40e0 Fix tick overflow (#8840) 2019-04-26 11:07:04 +02:00
Simon 729e506876 Remove duplicate code (#8825) 2019-04-26 10:53:35 +02:00
Tobias Gödderz 1bf19cdc91 Read docs in EnumerateCollection in batches (#8823) 
* Read docs in EnumerateCollection in batches

* Read docs in IndexExecutor in batches

* Fixed stats with batching

* Fixed a bug in RocksDBPrimaryIndexRangeIterator::nextCovering
 2019-04-25 11:56:19 +02:00
Jan Christoph Uhde 677a79026c Foxx Security (#8845) 2019-04-25 09:56:29 +02:00
Simon 569198a089 Abort el-cheapo transactions if servers fail (#8799) 2019-04-22 19:31:24 +02:00
jsteemann 90f17e2ee1 ok, cannot simply make these SmallVectors :sad: 2019-04-20 11:15:06 +02:00
Dan Larkin-York 91ed0d6568 Fix issue with truncate and index selectivity estimates (#8805) 2019-04-20 10:14:15 +02:00
Jan 2669544944
Bug fix/fix skip inaccessible (#8779) 2019-04-17 15:54:15 +02:00
Jan 80b021f915
speed up projections with RocksDBPrimaryIndex if there is no FILTER condition (#8770) 2019-04-16 17:40:30 +02:00
Jan 7c0c01c401
use the primary index when full-scanning a collection (#8548) 2019-04-16 16:50:03 +02:00
Jan 1d175f446d
do not use the TTL index for queries which it does not support (#8766) 2019-04-15 18:55:19 +02:00
Dan Larkin-York 777ba1a364 Improved audit logging (#8740) 2019-04-15 14:51:32 +02:00
Jan 13b2d40aef
fix compile warnings (#8743) 2019-04-15 11:22:24 +02:00
Simon 937d743ba6 Bug fix/pregel stuff (#8733) 2019-04-11 15:58:28 +02:00
Simon 676012ca3a snafu (#8728) 2019-04-10 19:14:34 +02:00
Jan c6d3f8e052
Bug fix/pass on error messages (#8690) 2019-04-10 12:34:25 +02:00
Jan 892446ed23
keep g++ from complaining about potential nullptr dereference (#8721) 2019-04-10 11:03:41 +02:00
Simon 4e80e1b351 Simplify recovery code (#8699) 2019-04-09 21:48:04 +02:00
Jan 675b006ebc
this addresses an issue described in ES-260, and is a forward-port of… (#8696) 2019-04-09 11:53:20 +02:00
jsteemann ed1e0336cb fix memleaks 2019-04-09 11:24:03 +02:00
Simon 2b594bdab5 Reduce # of memcpy from storage engine layer (#8685) 2019-04-08 22:43:29 +02:00
Jan 4f7923a971
remove replication clients handling out of vocbase code (#8676) 2019-04-08 19:15:28 +02:00
Jan 5897baa984
added db.<collection>.getResponsibleShard() (#8683) 2019-04-08 16:10:45 +02:00
Dan Larkin-York 8f49fde98f Fix RocksDB recovery ticks (#8682) 2019-04-04 20:38:44 +02:00
Jan 5a19a0320b
remove/fix parameters in crud methods (#8675) 2019-04-03 19:55:15 +02:00
Simon 7cd84a785a Remove Obsolete code (#8657) 2019-04-03 13:40:44 +02:00
Jan ec6a52fb0e
fix smart join optimization when another optimizer rule (reduce-extraction-to-projection) turned full scans into index accesses (#8660) 2019-04-03 10:42:21 +02:00
Jan 616ea94f24
Bug fix/cleanup 31032019 (#8632) 2019-04-01 17:14:11 +02:00
Jan b6df220821
cleanup replication contexts (#8630) 2019-04-01 09:35:08 +02:00
Jan 9ab9cc7857
disambiguate internal exceptions (#8623) 2019-03-29 15:59:37 +01:00
Dan Larkin-York ffc05c05c0 Fix handling of index name conflicts during replication. (#8571) 2019-03-28 18:53:43 +01:00
KVS85 2d5bf58c94 Remove USE_IRESEARCH definition and usage (#8613) 2019-03-28 17:23:51 +01:00
Jan a5e80fa7ed
remove redundant parameter from update API (#8607) 2019-03-28 12:50:20 +01:00
Jan Christoph Uhde c3f7961b88 apply unique log ids (#8561) 2019-03-25 20:26:51 +01:00
Jan e078f35285
fixed typos, removed unneeded includes (#8547) 2019-03-25 12:09:37 +01:00
Jan 39a3f5bc4e
reintroduce smart joins after temporarily reverting them in devel (#8543) 2019-03-23 20:36:02 +01:00
Dan Larkin-York be2cccfa28 Add RocksDB background error listener. (#8535) 2019-03-22 20:25:04 +01:00
Jan cfd3418115
microoptimizations (#8519) 2019-03-22 16:53:21 +01:00
Simon f4ecf8cd9e Fix asan error in truncate (#8523) 2019-03-22 15:22:28 +01:00
Simon 3ada15fc35 The Legendary El Cheapo (#8485) 2019-03-22 11:38:33 +01:00
Jan a35f6ccf10
fix type of hidden startup option `--rocksdb.recycle-log-file-num` (#8517) 2019-03-22 11:21:53 +01:00
jsteemann dc381a99df Revert "Feature/ncc1701 (#8440)" 
This reverts commit 59ad583796.
 2019-03-21 19:18:46 +01:00
Jan 59ad583796
Feature/ncc1701 (#8440) 2019-03-21 15:05:36 +01:00
Jan 00ac1db850
add RocksDB options `--rocksdb.allow-fallocate` and `--rocksdb.limit-open-files-at-startup` (#8493) 2019-03-21 14:21:35 +01:00
Jan 0bef1c1098
dont attempt to remove non-existing WAL files (#8474) 2019-03-20 17:58:28 +01:00
Jan 6d107beeda
don't run compact() on a collection after a truncate() was done in th… (#8468) 
* don't run compact() on a collection after a truncate() was done in the same transaction

running compact() in the same transaction will only increase the data size on disk due to RocksDB not being able to remove
any documents physically due to the snapshot we take at transaction start.
Decoupling the truncate transaction from the compact operation allows finishing the truncate transaction first, so we can
get rid of the snapshot. Running compact afterwards is then free to physically remove all the data.
As a nice side effect this change will also speed up the truncation of larger collections, because the compact will run
faster.

This change also exposes db.<collection>.compact() in the arangosh, in order to manually run a compaction on the data
range of a collection should it be needed for maintenance.

* fix documentation anchors
 2019-03-20 16:27:54 +01:00
Jan 2c4accdf34
micro improvements for rocksdb options (#8433) 2019-03-19 12:47:57 +01:00
Simon 49cc3bcd1e Refactorings from cluster trx improvement branch (#8391) 2019-03-14 23:13:17 +01:00
Wilfried Goesgens d88bc376a6 When updating documents with array indices and unique values we mustn't disable indexing in rocksdb (#8404) 2019-03-14 13:35:38 +01:00
Dan Larkin-York 413e90508f Named indices (#8370) 2019-03-13 18:20:32 +01:00
Jan 9d3327c6ea
Bug fix/rearm cursors (#8363) 2019-03-12 15:28:33 +01:00
Vasiliy f5f572dd29 issue 527.1: ensure the cause of the failure is reported to the user (#8371) 2019-03-11 23:42:34 +03:00
Jan 12e11a5197
port of replication improvements from 3.4 (#8308) 2019-03-11 13:37:18 +01:00
Vasiliy e97cd8a0fb issue 526.1: initial support for vocbase-prefixed analyzer names (#8319) 
* issue 526.1: initial support for vocbase-prefixed analyzer names

* backport: add tests for new functionality, add temporary workaround for legacy analyzers

* address review comments

* fix typo
 2019-03-06 15:13:19 +03:00
Jan 08a6561adc
Bug fix/small issues 21022019 (#8229) 2019-03-01 17:20:10 +01:00
Vasiliy 7130ffa427 issue 525.1: ensure RocksDB CreateIndex/DropIndex WAL markers are properly written during recovery (#8282) 
* issue 525.1: ensure RocksDB CreateIndex/DropIndex WAL markers are properly writen during recovery

* account for changes in devel

* skip writing DropIndex marker in recovery
 2019-03-01 15:58:11 +03:00
Andrey Abramov d86fb726e6
ArangoSearch index encryption (#8214) 
* update iresearch

* add index encryption support

* update iresearch

* encrypt ArangoSearch index using rocksdb encryption provider

* fix merge issue

* update iresearch

* fix cmake

* minor fixes after update

* update iresearch

* partially revert changes from commit 492d05c1f1 since they cause iresearch configuration failures

* Update arangod/IResearch/IResearchRocksDBLink.cpp

Co-Authored-By: gnusi <gnusmas@kvolitek.ru>

* partially revert changes from commit 492d05c1f1 since they cause iresearch configuration failures

* fix cmake configuration for MAX

* another attempt to fix cmake on mac

* Fix 3rdparty cmake for Windows
 2019-03-01 15:56:40 +03:00
Jan 5d2ab0c901
port from 3.4 (#8275) 2019-02-28 14:36:29 +01:00
Simon dc24d7fa4e Sync Foxx Queues (#8250) 2019-02-25 17:13:14 +01:00
Jan d46acbb156
Bug fix/make ttl index work with datestrings too (#8208) 2019-02-22 16:09:51 +01:00
Jan 1798036ea0
Bug fix/optimizations 18022019 (#8180) 2019-02-19 19:24:04 +01:00
Jan 44c6a2d732
Feature/ttl index (#8169) 2019-02-19 14:12:21 +01:00
Wilfried Goesgens 492d05c1f1 Feature/upgrade v8 7.1.302.28 (#8088) 2019-02-19 11:15:34 +01:00
Dan Larkin-York f4c2347fbd Make Result final (#8157) 2019-02-15 20:05:30 +01:00
Jan 820ba5dd7c
micro optimizations (#8156) 2019-02-13 16:14:28 +01:00
Jan Christoph Uhde 77dc65ee89 fix while loop in in-iterator (#8153) 2019-02-12 18:00:58 +01:00
Simon 581f9f362b test for broken indexes (#8103) 2019-02-05 09:38:31 +01:00
Simon 7127b6edb7 fix error handking in V8Cursor::New (#8104) 2019-02-05 09:37:30 +01:00
Jan 6d467a9f9f
use PutUntracked to insert entries into non-unique indexes (#8098) 2019-02-05 09:34:06 +01:00
Dan Larkin-York efb8364d99 Optimize memory usage in AQL by using constrained heap sort when reasonable (#7990) 2019-01-31 22:46:57 +01:00
Jan d45906e569
Bug fix/31012019 (#8078) 2019-01-31 17:27:18 +01:00
Simon 655d594949 Fix background indexing (#8065) 2019-01-31 11:49:57 +01:00
Simon fd70b6fc34 Background Indexing (#8039) 2019-01-29 09:31:32 +01:00
Jan a578f2e82b
fix some cppcheck complaints (#8053) 2019-01-29 09:26:25 +01:00
jsteemann 8c5e7423b2 Revert "Do not acquire snapshot for single read (#8041)" 
This reverts commit 91931ace4d.
 2019-01-28 16:05:27 +01:00
Simon 91931ace4d Do not acquire snapshot for single read (#8041) 2019-01-28 13:15:51 +01:00
Simon b1dc8fb804 reduce timeout for write-lock (#8034) 2019-01-28 09:21:30 +01:00
Vasiliy 22ea8bc293 issue 466.2: update iresearch to revision 9f179cea9037c702115d0884c88e742a30921cd1 (#8010) 
* issue 466.2: update iresearch to revision ee15310f1450035c7d3ba664c6886ef7a39eb6bb

* update iresearch to commit 9f179cea9037c702115d0884c88e742a30921cd1
 2019-01-24 16:11:04 +03:00
Jan cabeb8fc7b
make use of projections if the projected data is used by an IndexNode's condition (#8001) 2019-01-21 23:11:17 +01:00
Vasiliy e09f932dd3 issue 466.1.1: initial commit of FlushFeature redesign with WAL markers (#7980) 
* issue 466.1.1: initial commit of FlushFeature redesign with WAL markers

* convert an std::shared_ptr destructor into a TRI_DEFER(..)
 2019-01-21 17:50:42 +03:00
Simon ba102323cc Refactoring index interface (#7971) 2019-01-17 16:33:46 +01:00
Vasiliy fac48ddbc1 issue 511.9.1: track DataSource instance instead of only its name in QueryCache (#7943) 
* issue 511.9.1: track DataSource instance instead of only its name in QueryCache

* address use of invalid function

* fix expected test result

* address merge issue
 2019-01-14 16:25:18 +03:00
Jan 9ce9312239
speed up containers that contain ExecutionNodes' variables (#7937) 2019-01-14 13:38:56 +01:00
Vasiliy 0fcf2c5784 issue 511.8: remove a redundant IResearchLinkDrop marker, it does the exact same thing as IndexDrop that is also written to the WAL (#7923) 
* issue 511.8: remove a redundant IResearchLinkDrop marker, it does the exact same thing as IndexDrop that is also written to the WAL

* add a comment for deprecated markers
 2019-01-11 18:09:23 +03:00
Jan Christoph Uhde f65853e30f make use of sortedness of rocksdb primary index (#7788) 2019-01-11 09:40:22 +01:00
Jan fe81007438
support db._explain with all plans (#7895) 2019-01-08 18:06:46 +01:00
Lars Maier 12eebb15fe Feature/new server infra (#7733) 
* Decoupled IO handling from Scheduler.

* Fixed SSL start up bug.

* Replaced Scheduler with new worker farm implementation.

* Added minimal statistics and info string for Scheduler.

* Added support for timed submissions.

* Updated delayed submission api. Updated code that used timers.

* Extracted new Scheduler into a virtual parent class. The implementation can now depend on the usecase.

* Signal handler now working.

* Changed threads names, `_stop` is atomic, check for failure during thread start + exception handling like old scheduler did.

* Commented on source code and added TODOs.

* Played around with start-stop-conditions

* Play around with start stop condition.

* start stop cond

* Sart Stop Conditions

* Removed bad cv_status check.

* Bug fix: now compare the actual objects instead of pointer values. Setup t1 and t2 depending on the thread id.

* Moved most of the stuff now unrelated to the Scheduler to GeneralServer. Got rid of JobGuard.

* Instead of waiting for a thread to terminate, put it on a clean up list and check for its termination in each supervisor run.

* Allow detaching long running threads.

* Fixed test mock.

* Updated the WorkHandle logic. Removed post functions.

* Fixed crash when obtaining shared_ptr from this in destructor.

* Added lost mutex.

* Fixed memory leak.

* Fixed merge bug.

* Changed a lot of code to optimize the scheduler.

* Fixed bug of invalidated iterator. Dont remove task on shutdown at different places. Let scheduler threads run until queue is empty.

* Only by value calls to queue.

* Added options again.

* Clean up of code.

* UI Request Lane added.

* Bug fixes in Scheduler.

* Applied reformat.

* Use sigaction.
 2019-01-08 10:12:02 +01:00
Jan f4f61e2947
Bug fix/issues 23122018 (#7847) 2019-01-02 11:11:03 +01:00
Frank Celler ac9f375fb5 big reformat 2018-12-26 00:54:03 +01:00
Dan Larkin-York 2ad638cb45 Fix issue with geo iterator reset. (#7839) 2018-12-23 00:41:12 +01:00
Simon a2a0b03f43 Rdb index background (preliminary) (#7644) 2018-12-21 19:24:10 +01:00
Michael Hackstein 4965cea454 Feature/improve edgeindex covered (#7718) 
* The Rocks Edge index now states that it covers _from and _to and can be used as a covering index for both.

* Moved the ExecutionState checkmark in query to a later point. It actually still included parts of Instantiation in cluster

* Updated test for the new feature, it now covers more states

* Updated changelog

* Updated ClusterIndex to support RocksDBEdgeIndex covering

* Removed assertion, during startup this assertion is not hold

* Replaced inplace builder by a leased builder

* Update CHANGELOG

Co-Authored-By: jsteemann <jsteemann@users.noreply.github.com>

* Update CHANGELOG

Co-Authored-By: jsteemann <jsteemann@users.noreply.github.com>
 2018-12-21 13:41:25 +01:00
Wilfried Goesgens c3e73ba45b upgrade rocksdb to V5.18.X (#7792) 2018-12-19 19:34:55 +01:00
Jan ea39dfa3d2
fixed issue #7757 (#7805) 
* fixed issue #7757

* fix compilation error
 2018-12-19 14:06:52 +01:00
Vasiliy f1bf5cc9cf issue 511.3: remove IResearchViewDBServer and use the IResearchView directly on db-server (#7748) 
* issue 511.3: remove IResearchViewDBServer and use the IResearchView directly on db-server

* ensure objectId is set in RocksDBLink
 2018-12-17 21:11:20 +03:00
Vasiliy f1245af554 issue 511.2.1: use references instead of raw pointers for Index operations to avoid null pointer access (#7725) 2018-12-11 14:40:49 +03:00
jsteemann 0f6512be82 added missing changes from 3.4 2018-12-10 15:29:44 +01:00