1
0
Fork 0
Commit Graph

101 Commits

Author SHA1 Message Date
Simon 8597708b3b Fix intermediate commits (#6039) 2018-08-06 17:15:02 +02:00
Jan dc5f6b89d6
replication improvements (#5958) 2018-07-25 12:49:24 +02:00
Jan 731925080d
Feature 3.3/aql optimizations (#5140) 2018-05-15 14:41:07 +02:00
jsteemann a3073bec2a adjust index selection slightly 2018-03-24 22:56:41 +01:00
jsteemann 2f46033ff6 adjust index selection slightly 2018-03-24 22:27:28 +01:00
jsteemann dbd05dab0b improve index selection a bit 2018-03-24 20:54:14 +01:00
Jan 50fef0f430
multiple cluster-related AQL optimizations (#4928)
* multiple cluster-related AQL optimizations

* do not apply single-shard optimization if _includedShards is already set from the outside

* implement changes requested by @mchacki
2018-03-23 14:52:58 +01:00
jsteemann 1b5384439e removed invalid assertion 2018-02-22 11:01:13 +01:00
Michael Hackstein ffbc7058cd
Bug fix 3.3/rocksdb truncate (#4338) 2018-01-16 19:52:27 +01:00
Jan 8b873bdf4a
prefer unique equality lookups, as they do not have O(log(n)) complexity (#4096) 2017-12-21 20:41:10 +01:00
Jan b4f6ee9273 Feature/improved index api for unique constraints and replication (#3715) 2017-11-16 21:02:01 +01:00
jsteemann 571d4587ca Revert "Return offending key for unique constraint violations (#3624)"
This reverts commit 68bd31ac99.
2017-11-10 16:40:56 +01:00
Dan Larkin 68bd31ac99 Return offending key for unique constraint violations (#3624) 2017-11-10 16:03:10 +01:00
Jan 7613bc4314 Bug fix/fixes 0211 (#3568)
* remove some non-unused V8 persistents

* do not throw that many bogus assertions

* do not rely on server role being defined

* slightly better debug output for V8 context debugging

* fix collection ids in inventory response

* simplify bootstrap a bit

* slightly better error handling

* make elapsed time a queryable value

* use less memory for stub collections

* added assertions that will always make sense

* added assertions

* do not garbage-collect while waiting

* less copying of parameters

* do not show "load indexes into memory" buttons for mmfiles engine

  as all indexes are in memory anyway

* when a collection is truncated via the web interface, flush the WAL and rotate all active journals

this will make close all open journals on leader and followers and make them subject to compaction opportunities

* fix invalid server id values being passed from web interface to backend

* introduce afterTruncate method for indexes

* added test case for issue #3447

* updated CHANGELOG

* don't warn about replicationFactor for system collections

* check that the queries actually use the geo index and not some other index

* properly report error in web interface

* fix some internals checks that made truncate fail for bigger collections in maintainer mode

* also run a compact() operation after a serious truncate

in order to make iteration over the truncated range much faster
when the collection is next accessed

* increase default maximum number of V8 contexts to at least 16
2017-11-09 12:48:15 +01:00
Jan 720e6df82e Bug fix/fixes 1910 (#3471)
* properly initialize all properties

* use faster comparison

* properly detect and handle "method not allowed"

* code-style

* remove unused variable

* narrow variable scope

* handle non-existance of AuthenticationFeature

* remove dead code

* replace some C string handling with std::strings

* moved assertion to the correct place

* honor number of array members for IN operator

* slightly adjust error messages

* slighty adjust some error messages

* try to fix issue with lingering replication contexts on shutdown

* clean up heartbeat thread a little bit

* small fixes
2017-10-23 09:17:36 +02:00
Jan 0561bf45ce Bug fix/isrestore (#3283)
* Make isRestore work in the cluster.

This covers sharded collections with default sharding and non-default
sharding.

* always use locally generate revision ids for storing and looking up documents
2017-10-03 11:53:49 +02:00
Jan b4e5132572 fix dropping of system collections and retrieving list of user databa… (#3304)
* fix dropping of system collections and retrieving list of user databases in cluster

* fix MSVC compile warnings
2017-09-21 10:05:46 +02:00
Jan 5165155ed1 Bug fix/fixes 0609 (#3227)
* do not use V8 variant of AQL functions in early optimization stage when a C++ variant is available

* additionally, simplify AQL function definitions and aliases

* warn when more than 90% of max mappings are in use

* added C++ variant of replication catchup

* added `--log.role` option

* updated CHANGELOG

* removed non-existing scheduler.threads option from config

* removed useless __FILE__, __LINE__ invocations

* updated CHANGELOG

* allow a priority V8 context

* remove TRI_CORE_MEM_ZONE

* try to fix Windows errors & warnings

* cleanup

* removed memory zones altogether

* exclude system collections from collection tests
2017-09-13 16:28:21 +02:00
Dan Larkin a775e37c76 Feature/unique vpack index lookup performance improvements with rocksdbkey optimizations (#3123)
* fixed issue #3075

* update the known issues document, as some issues have already been fixed

* speed up equality lookups in unique vpack indexes

* Allow for re-use of RocksDBKey objects through transaction leaser.

* Converted all RocksDBKey variants to non-static constructors.
2017-08-31 20:00:05 +02:00
Simon Grätzer 88d01b89b5 Optimizations for Caches and Graph Traversals (#3169) 2017-08-31 18:33:10 +02:00
Jan 5c5b5e548d Feature/unique vpack index lookup performance improvements (#3087)
* fixed issue #3075

* update the known issues document, as some issues have already been fixed

* speed up equality lookups in unique vpack indexes

* fixed wording
2017-08-24 14:39:09 +02:00
Simon Grätzer 4cc8686e82 RocksDB: Optimizing dropCollection / dropIndex / fillIndex (#2927)
* cleaning up several duplicate versions of Index::memory, Index::cleanup, Index::drop, integrating branch issue-383-ensure-delete

* fixing wrong column family
2017-08-05 21:09:41 +02:00
Jan 40c5b94d68 Bug fix/nightly fixes (#2946)
* fix internal documentation (thanks @Simran-B)

* fixed issue #2810

* potential fix for issue #2939

* updated CHANGELOG

* updated CHANGELOG
2017-08-04 13:34:12 +02:00
Jan Christoph Uhde ed8efe3566 Feature/issue 387 cluster index estimates (#2866) 2017-08-01 09:53:58 +02:00
Jan 87567c3c2a fixed issue #2876 (#2896) 2017-07-30 14:24:25 +02:00
Jan 634574ed9f don't mask errors with fake OOM messages (#2872) 2017-07-28 12:08:37 +02:00
Frank Celler a5a25754ed Feature/reduce extraction to projection (#2792)
* reduce extractions to projections

* recycle string buffers in SocketTask

* micro optimizations for mmfiles indexes

* added special lookup function for _key

* moved function into the correct file

* speed up key buffer allocations a bit

* added noexcept specifier

* correctly name variable

* explicitly move bounds

* fix and speedup from/toPersistent functions

* reuse string from ManagedDocumentResult for multiple lookups

* use move-assign

* a bit less work for single server

* speedup AQL function HASH

* single fetch optimization

* performance optimization for the case when no documents need to be returned

* make reduce-extraction-to-projection a RocksDB-only optimizer rule

* cppcheck

* try to fix compile error on MacOS

* bug fix for MacOSX

* missing namespace (in Windows compile)
2017-07-14 08:40:29 +02:00
Frank Celler 40d73d5a8b Revert "[WIP] Feature/reduce extraction to projection (#2735)"
This reverts commit 5bfcff30cd.
2017-07-12 12:52:14 +02:00
Jan 5bfcff30cd [WIP] Feature/reduce extraction to projection (#2735)
* reduce extractions to projections

* recycle string buffers in SocketTask

* micro optimizations for mmfiles indexes

* added special lookup function for _key

* moved function into the correct file

* speed up key buffer allocations a bit

* added noexcept specifier

* correctly name variable

* explicitly move bounds

* fix and speedup from/toPersistent functions

* reuse string from ManagedDocumentResult for multiple lookups

* use move-assign

* a bit less work for single server

* speedup AQL function HASH

* single fetch optimization

* performance optimization for the case when no documents need to be returned

* make reduce-extraction-to-projection a RocksDB-only optimizer rule
2017-07-12 11:22:29 +02:00
Jan bee32b6325 properly compact ranges so data actually gets deleted after drop etc. (#2771) 2017-07-11 21:47:16 +02:00
Simon Grätzer d8c57f2cc8 Feature/rocksdb hashsearch (#2713)
* Testing hash for documents

* fixing logspam error

* added base support for kHash in the edge index

* Current version of hash enabled edge index

* removed rocksdb key type byte, dynamic prefixes for hash-index

* Fixing issues with key classes

* Fixing merge conflict

* Fixing version with semi-working hash index

* fixing various issues

* Fixing hanger in EdgeIndexIterator nextExta, when _cache is disabled. Reenable cache

* Fixing geo index bounds, and fillIndexes error handling

* Adding back failure point

* Fixing RocksDBKey::vertexId

* Added version check on startup

* Fixing version check

* added one more check

* properly detect column families at startup

* Adding additional sanity checking

* Adding missing column family

* some fixes

* Turned off geoindex sync

* fixed error

* added asserts (untested)

* Adding more asserts
2017-07-05 10:18:52 +02:00
Jan c7c8910c7c make use of selectivity estimates in hash, skiplist and persistent indexes (#2703) 2017-06-30 18:11:58 +02:00
Jan 94b4a9ec4b added "deduplicate" attribute for array indexes (#2644) 2017-06-30 17:53:50 +02:00
Michael Hackstein 2c90b36bb1 The ArangoDB VPackIndex now does not attempt to compute an Estimate on Coordinator. Caused the coordinator to crash (#2629) 2017-06-22 12:37:12 +02:00
Jan cb8014dbdd use correct column families for size estimates (#2624) 2017-06-22 12:32:08 +02:00
Jan c176a11930 control RocksDB block cache filling for the primary and edge indexes (#2532) 2017-06-09 08:32:46 +02:00
Dan Larkin 8c7ba2aaa3 Changed several index/collection APIs to provide better context messages.
Addresses Issue #342.
2017-06-06 12:27:08 -04:00
Simon Grätzer 5509776503 Fixing various issues 2017-05-30 13:44:26 +02:00
Simon Grätzer ab292b2717 Fixing edge bounds 2017-05-30 11:30:48 +02:00
Simon Grätzer 3ad939ea41 changed storage format 2017-05-30 10:12:39 +02:00
Simon Grätzer f20a0a8c3d Merge branch 'devel' of https://github.com/arangodb/arangodb into feature/storage-format-refactoring 2017-05-29 16:01:06 +02:00
Simon Grätzer f8ab86e160 Begin storage format refactoring 2017-05-29 16:00:55 +02:00
Michael Hackstein b9d2faa7cc Added warmup function for indexes.
Squashed commit of the following:

commit bc0472d212b2efef0d7b1b8a36f2b43e8432ba86
Merge: eb47631 5f87674
Author: Michael Hackstein <michael@arangodb.com>
Date:   Mon May 29 09:39:31 2017 +0200

    Merge branch 'devel' of github.com:arangodb/arangodb into feature/rocks-index-warmup

commit eb476310b0ca6165be10b37b960933886c2757f5
Merge: d725f21 32149d2
Author: Michael Hackstein <michael@arangodb.com>
Date:   Mon May 29 09:00:41 2017 +0200

    Merge branch 'devel' of github.com:arangodb/arangodb into feature/rocks-index-warmup

commit d725f21f7d61f7c79ba31fc0338881e35f4d8f48
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 15:53:23 2017 +0200

    Activated collection warmup in Cluster Mode. Added a test for collection warmup (SingleServerOnly)

commit 132bf4f9565b0dcf4ec9f84e93897b482a67ce7f
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 12:46:20 2017 +0200

    Implemented client-side warmup. It can now be triggered via http and arangosh

commit 78ea449dff86118814a2f87bdb59dc16544d92b6
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 11:25:40 2017 +0200

    Fixed assertion.

commit dae80f6277dde1a52eadda506858cc36e235bd55
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 11:08:46 2017 +0200

    Improved the CuckooIndexEstimator computeEstimate function. It is now much better on collections with many different values

commit 7abf57876511ba369d7a577e1995d4575e98c7c8
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 11:06:43 2017 +0200

    Edge index warmup will now first check if a document is stored and if not will insert it. Furthermore it resizes the cache with an estimate of how many documents will be most likely inserted

commit 890d8ad4cdfd155249f060fedd5c798b9531d556
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 11:04:47 2017 +0200

    Adjusted thresholds in transactional cache. Wastly increased the amount of tries to get the lock. Lowered the fill-grade boundaries

commit 60972ab7151a3acb78e1aae4149de11d0da7aceb
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 10:45:38 2017 +0200

    Added new Logtpopic CACHE that should be used to log important information regarding caches, like debug ouput on evictions / resizing, or memory-pressure resizing going on

commit 2dfacb9aef6e3fde169032514baca386786d059c
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 10:44:21 2017 +0200

    Fixed an assertion in index iterator. It expected _from/_to to be cached but we modified to cache entire document

commit f05beccef65853c30eda1859a39c28ea2295bc71
Merge: 13c6abf 5c46430
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 07:56:58 2017 +0200

    Merge branch 'devel' of github.com:arangodb/arangodb into feature/rocks-index-warmup

commit 13c6abfbc2ddb451a689011110411df68247435e
Author: Michael Hackstein <michael@arangodb.com>
Date:   Wed May 24 09:52:01 2017 +0200

    Added a getExtra feature for the edgeIndex. It now returns the complete edge as extra (temporary). Modified the EdgeCache to cache token and edge document (temporary). Added a warmup Function to collections that is used to warmup the index caches on demand.
2017-05-29 09:40:58 +02:00
Simon Grätzer d5f85d5bbb Adding more colum families 2017-05-26 10:43:17 +02:00
Simon Grätzer 9679aecf53 Adding bounds back 2017-05-22 17:39:19 +02:00
Simon Grätzer a99611e7c0 Various changes 2017-05-19 16:43:00 +02:00
Simon Grätzer e73f446c5b Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2017-05-18 22:08:39 +02:00
Simon Grätzer 2bd2a15cb3 Column family support 2017-05-18 22:08:34 +02:00
jsteemann f2a5a6473e try to fix ADL issue 2017-05-18 21:07:14 +02:00
jsteemann b3c87b1f29 fix dropping indexes 2017-05-18 14:29:54 +02:00