Hawkin
/
arangodb
mirror of https://gitee.com/bigwinds/arangodb
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
45,782 Commits 358 Branches 495 Tags 994 MiB
Commit Graph

174 Commits

Author SHA1 Message Date
Jan b4f6ee9273 Feature/improved index api for unique constraints and replication (#3715) 2017-11-16 21:02:01 +01:00
jsteemann 571d4587ca Revert "Return offending key for unique constraint violations (#3624)" 
This reverts commit 68bd31ac99.
 2017-11-10 16:40:56 +01:00
Dan Larkin 68bd31ac99 Return offending key for unique constraint violations (#3624) 2017-11-10 16:03:10 +01:00
Jan 7613bc4314 Bug fix/fixes 0211 (#3568) 
* remove some non-unused V8 persistents

* do not throw that many bogus assertions

* do not rely on server role being defined

* slightly better debug output for V8 context debugging

* fix collection ids in inventory response

* simplify bootstrap a bit

* slightly better error handling

* make elapsed time a queryable value

* use less memory for stub collections

* added assertions that will always make sense

* added assertions

* do not garbage-collect while waiting

* less copying of parameters

* do not show "load indexes into memory" buttons for mmfiles engine

  as all indexes are in memory anyway

* when a collection is truncated via the web interface, flush the WAL and rotate all active journals

this will make close all open journals on leader and followers and make them subject to compaction opportunities

* fix invalid server id values being passed from web interface to backend

* introduce afterTruncate method for indexes

* added test case for issue #3447

* updated CHANGELOG

* don't warn about replicationFactor for system collections

* check that the queries actually use the geo index and not some other index

* properly report error in web interface

* fix some internals checks that made truncate fail for bigger collections in maintainer mode

* also run a compact() operation after a serious truncate

in order to make iteration over the truncated range much faster
when the collection is next accessed

* increase default maximum number of V8 contexts to at least 16
 2017-11-09 12:48:15 +01:00
Simon Grätzer 7c31960cf2 Feature/async failover (#3451) 2017-10-18 23:59:29 +02:00
Jan 0561bf45ce Bug fix/isrestore (#3283) 
* Make isRestore work in the cluster.

This covers sharded collections with default sharding and non-default
sharding.

* always use locally generate revision ids for storing and looking up documents
 2017-10-03 11:53:49 +02:00
Jan 2525a3a1bd Feature/mmfiles hash lookup performance (#3265) 
* speed up full collection scans in mmfiles engine

* some API cleanup
 2017-09-26 14:48:06 +02:00
Jan b4e5132572 fix dropping of system collections and retrieving list of user databa… (#3304) 
* fix dropping of system collections and retrieving list of user databases in cluster

* fix MSVC compile warnings
 2017-09-21 10:05:46 +02:00
Jan 5165155ed1 Bug fix/fixes 0609 (#3227) 
* do not use V8 variant of AQL functions in early optimization stage when a C++ variant is available

* additionally, simplify AQL function definitions and aliases

* warn when more than 90% of max mappings are in use

* added C++ variant of replication catchup

* added `--log.role` option

* updated CHANGELOG

* removed non-existing scheduler.threads option from config

* removed useless __FILE__, __LINE__ invocations

* updated CHANGELOG

* allow a priority V8 context

* remove TRI_CORE_MEM_ZONE

* try to fix Windows errors & warnings

* cleanup

* removed memory zones altogether

* exclude system collections from collection tests
 2017-09-13 16:28:21 +02:00
Jan 523d99b236 Bug fix/fixes 0109 (#3191) 
* fix double free

* simplfy conditions

* fix cppcheck complaint
 2017-09-04 17:32:25 +02:00
Dan Larkin a775e37c76 Feature/unique vpack index lookup performance improvements with rocksdbkey optimizations (#3123) 
* fixed issue #3075

* update the known issues document, as some issues have already been fixed

* speed up equality lookups in unique vpack indexes

* Allow for re-use of RocksDBKey objects through transaction leaser.

* Converted all RocksDBKey variants to non-static constructors.
 2017-08-31 20:00:05 +02:00
Simon Grätzer 88d01b89b5 Optimizations for Caches and Graph Traversals (#3169) 2017-08-31 18:33:10 +02:00
Dan Larkin 18e8c19271 Improvements to CachedValue internals and handling. (#3126) 
* Added checks everywhere for CachedValue::construct.

* Enforce 16MiB limit on keys and 4GiB limit on values.

* Protected members behind function calls in anticipation of format change.

* Shrunk CachedValue struct.

* Fixed alignment of CachedValues.

* Fixed improper comparison.
 2017-08-28 15:07:16 +02:00
Simon Grätzer 4cc8686e82 RocksDB: Optimizing dropCollection / dropIndex / fillIndex (#2927) 
* cleaning up several duplicate versions of Index::memory, Index::cleanup, Index::drop, integrating branch issue-383-ensure-delete

* fixing wrong column family
 2017-08-05 21:09:41 +02:00
Jan bee32b6325 properly compact ranges so data actually gets deleted after drop etc. (#2771) 2017-07-11 21:47:16 +02:00
Simon Grätzer d8c57f2cc8 Feature/rocksdb hashsearch (#2713) 
* Testing hash for documents

* fixing logspam error

* added base support for kHash in the edge index

* Current version of hash enabled edge index

* removed rocksdb key type byte, dynamic prefixes for hash-index

* Fixing issues with key classes

* Fixing merge conflict

* Fixing version with semi-working hash index

* fixing various issues

* Fixing hanger in EdgeIndexIterator nextExta, when _cache is disabled. Reenable cache

* Fixing geo index bounds, and fillIndexes error handling

* Adding back failure point

* Fixing RocksDBKey::vertexId

* Added version check on startup

* Fixing version check

* added one more check

* properly detect column families at startup

* Adding additional sanity checking

* Adding missing column family

* some fixes

* Turned off geoindex sync

* fixed error

* added asserts (untested)

* Adding more asserts
 2017-07-05 10:18:52 +02:00
Dan Larkin 05c8d3ae71 Converted Get calls to use PinnableSlice where possible. (#2683) 2017-07-01 00:27:27 +02:00
Jan cb8014dbdd use correct column families for size estimates (#2624) 2017-06-22 12:32:08 +02:00
Dan Larkin a3ae2b7191 Augmented cache API and fixed a potential deadlock scenario. (#2579) 
* Augmented cache API and fixed a potential deadlock scenario.

* The RocksDBEdgeIndex now tries to read up to 10 times form the cache on lock_timeout. Instead of rewriting everything after first attempt
 2017-06-19 17:28:04 +02:00
Jan c176a11930 control RocksDB block cache filling for the primary and edge indexes (#2532) 2017-06-09 08:32:46 +02:00
Dan Larkin 8c7ba2aaa3 Changed several index/collection APIs to provide better context messages. 
Addresses Issue #342.
 2017-06-06 12:27:08 -04:00
Simon Grätzer 53ece582a8 optimized all index iterator 2017-05-31 11:34:32 +02:00
Simon Grätzer d5f85d5bbb Adding more colum families 2017-05-26 10:43:17 +02:00
Simon Grätzer 88977d204a Merge branch 'devel' of https://github.com/arangodb/arangodb into column-family 
# Conflicts:
#	arangod/RocksDBEngine/RocksDBIndex.cpp
#	arangod/RocksDBEngine/RocksDBIndex.h
#	arangod/RocksDBEngine/RocksDBPrimaryIndex.cpp
#	arangod/RocksDBEngine/RocksDBPrimaryIndex.h
 2017-05-23 11:17:30 +02:00
jsteemann 2f3e4d0ba0 fix init order 2017-05-23 08:57:39 +02:00
jsteemann ad5e58b050 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2017-05-23 08:53:26 +02:00
jsteemann a7b3839e6c renamed "getMore" to "next" 2017-05-23 08:53:13 +02:00
Simon Grätzer ddfa15b8cc Adding bound checking back 2017-05-22 19:09:32 +02:00
Simon Grätzer 9679aecf53 Adding bounds back 2017-05-22 17:39:19 +02:00
Simon Grätzer 552894fa97 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 
# Conflicts:
#	arangod/RocksDBEngine/RocksDBPrimaryIndex.cpp
 2017-05-19 17:06:24 +02:00
Simon Grätzer bbb5f560ed Adding asserts 2017-05-19 17:05:43 +02:00
jsteemann 3459cbd15d re-activate fill_cache 2017-05-19 16:50:01 +02:00
Simon Grätzer a99611e7c0 Various changes 2017-05-19 16:43:00 +02:00
Simon Grätzer 7cd0bc3c62 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 
# Conflicts:
#	arangod/RocksDBEngine/RocksDBPrimaryIndex.cpp
 2017-05-19 11:15:36 +02:00
jsteemann c34ebf8aa1 added Equal function to Comparator 
also don't fill block cache when using all/any iterators
 2017-05-19 09:58:19 +02:00
Simon Grätzer 2bd2a15cb3 Column family support 2017-05-18 22:08:34 +02:00
Michael Hackstein b4d1664023 Disabled Collection Cache {_rev => doc} and PrimaryIndex Cache {_key => _rev}. This seems to not have a positive effect in our tests. 2017-05-17 17:50:15 +02:00
Jan Christoph Uhde 415cbfea60 fix figures and add tests 2017-05-17 12:22:43 +02:00
Simon Grätzer 5265d2a063 intermediate commits 2017-05-16 16:21:39 +02:00
Simon Grätzer df3eed2289 Various changes 2017-05-16 10:58:15 +02:00
Dan Larkin 1fc3480a6b Added document cache. 2017-05-11 14:39:00 -04:00
Dan Larkin f43e342c47 Merge branch 'devel' of github.com:arangodb/arangodb into devel 2017-05-10 19:57:27 -04:00
Dan Larkin 52711a2b00 Added proper truncation support for RocksDB indexes. 2017-05-10 19:57:22 -04:00
jsteemann 1e0ce29d5b remove debug log message 2017-05-10 23:36:51 +02:00
Dan Larkin ab815e6020 Merge branch 'devel' of github.com:arangodb/arangodb into devel 2017-05-10 17:28:27 -04:00
Dan Larkin 9276890e92 Fixed key bounds to be compatible with prefix seek mode. 2017-05-10 13:07:01 -04:00
Jan Christoph Uhde 847dd3ad5d Merge branch 'devel' of https://github.com/arangodb/arangodb into feature/edge-index-cache 
* 'devel' of https://github.com/arangodb/arangodb:
  geo index fixes
  adding transactions to the geo index
 2017-05-10 18:43:20 +02:00
Jan Christoph Uhde 3e25aec2f3 fix enabling of caches 2017-05-10 15:58:28 +02:00
Jan Christoph Uhde e5db579b25 Merge branch 'devel' of https://github.com/arangodb/arangodb into feature/edge-index-cache 
* 'devel' of https://github.com/arangodb/arangodb: (54 commits)
  attempt to fix races in replication
  grunt build
  use exclusive transaction in syncer
  change feature order around
  dashboard
  added api cluster api routes
  add selection of storage engine
  use some parallelism
  micro optimizations
  Fix truncate for geo
  added assertions
  remove unused include
  try to fix initial sync problem
  remove unnecessary includes
  Finally fix the error where suddenly an array of dbservers are being called
  Update queryView.js
  initialize bool variable
  remove debug output
  removed unused private field
  Wrong index number
  ...
 2017-05-10 15:14:08 +02:00
Simon Grätzer 5fa85761a2 adding transactions to the geo index 2017-05-10 14:54:39 +02:00
jsteemann e85cadb90d removed unused private field 2017-05-10 11:04:03 +02:00
Jan Christoph Uhde e6d41cac9a clean up edge-index code - handling of invalidated caches still missing 2017-05-10 09:03:21 +02:00
Dan Larkin ca91c0f7c9 Added some optimizations enabled by prefix extraction. 2017-05-09 18:36:12 -04:00
Jan Christoph Uhde 738989f9a3 add blacklisting for keys in rocksdb-edge-index on insert/remove 2017-05-09 18:33:05 +02:00
Simon Grätzer d9bff18814 Fulltext index 2017-05-05 13:58:11 +02:00
Simon Grätzer 804676ae65 Moving estimatedSize to memory() 2017-05-04 18:57:22 +02:00
Simon Grätzer 94e4437c3a Added some non transactional truncate logic 2017-05-04 18:07:47 +02:00
Simon Grätzer 1c82d4448c Adding compaction and estimateSize method for rocksdb 2017-05-04 13:08:02 +02:00
Simon Grätzer 0faa1adac1 fixed unique contraints 2017-05-03 17:45:21 +02:00
Simon Grätzer a2e2c00d62 fillIndex non transactional + tests 2017-05-03 16:07:29 +02:00
Simon Grätzer 953b017d37 Fixing issues with the seek incremental sync optimization 2017-05-02 17:24:49 +02:00
Simon Grätzer e335a29326 better incremental sync 2017-05-02 15:08:05 +02:00
Dan Larkin 5e0a60406a Fixed recovery bug. 2017-04-28 14:02:49 -04:00
jsteemann 81f1d3820f added assertions for transaction states 2017-04-28 14:28:45 +02:00
jsteemann 4300c77d3e fixes for non-array IN lookups, added tests 2017-04-25 01:47:07 +02:00
Michael Hackstein 6cfa55ac6c Merge branch 'engine-api' of github.com:arangodb/arangodb into devel 2017-04-24 13:54:42 +02:00
jsteemann fcc61115d8 try to fix feature startup order (Scheduler vs. CacheManager vs. DatabaseFeature) 2017-04-24 13:17:21 +02:00
Simon Grätzer e850dd635a Incremental sync 2017-04-21 18:10:10 +02:00
jsteemann 862ec7bae3 remove log garbage 2017-04-21 12:57:20 +02:00
Michael Hackstein 9d621d1ffd Merge branch 'engine-api' of github.com:arangodb/arangodb into engine-api 2017-04-20 18:08:53 +02:00
Michael Hackstein 24992be707 Removed an assertion of PrimaryIndex toVelocyPack. The case i was tackeling was writing the index-information as an attribtue into an object, which was not allowed by this assertion, but actually it is. 2017-04-20 17:49:26 +02:00
Simon Grätzer 502dc66697 Make some methods const 2017-04-20 16:39:18 +02:00
Simon Grätzer f389baf500 Fixed warning 2017-04-20 15:28:42 +02:00
jsteemann 8e779aab24 cppcheck 2017-04-19 13:20:46 +02:00
Dan Larkin 7ad0c75745 Fixed bug in index info persistence/recovery. 2017-04-18 14:55:58 -04:00
Dan Larkin 97d525e9b7 Revert "Fixed primary index cluster bug." 
This reverts commit 8c4214625c.
 2017-04-17 17:06:23 -04:00
Dan Larkin 841ef8b1a0 Changed index VPack building to include forPersistence flag. 2017-04-17 17:04:50 -04:00
Dan Larkin 8c4214625c Fixed primary index cluster bug. 2017-04-13 21:44:51 -04:00
Dan Larkin 2e0fdbf666 Fixed some bugs. 2017-04-13 20:06:25 -04:00
Jan Christoph Uhde e690e7fc80 Merge remote-tracking branch 'origin/feature/engine-api/edgeindex-report-as-single-index' into engine-api 
* origin/feature/engine-api/edgeindex-report-as-single-index:
  change mmfiles indexes to work with slightly changed API
  fix rocksdb engine index reporting but break report for mmfiles engine
  update error messages
 2017-04-13 08:46:54 +02:00
Jan Christoph Uhde 2700a6d79d fix rocksdb engine index reporting but break report for mmfiles engine 2017-04-12 14:50:55 +02:00
Dan Larkin b030cdb347 Fixed an edge index iterator bug. 2017-04-10 20:12:15 -04:00
Dan Larkin eaac6c013c Fixed issue with cache transactions. 2017-04-07 16:58:22 -04:00
jsteemann cb031de4fe disable cache in primary index 2017-04-07 14:56:11 +02:00
jsteemann 2b0673c3fe removed type "Index" 2017-04-07 12:05:56 +02:00
Dan Larkin c066b5c84c Fixed bug in document update code. 2017-04-06 18:34:11 -04:00
Dan Larkin aa24d452fe Fixed any iterator. 2017-04-06 14:23:15 -04:00
Simon Grätzer 63d4a34d3f Fixed Any iterator 2017-04-06 13:38:40 +02:00
jsteemann 0e98a32455 fix objectId handling 2017-04-06 11:19:27 +02:00
Dan Larkin 3c3d6ae52f Slightly improved any iterator. 2017-04-05 18:03:10 -04:00
Dan Larkin ae84e8ac84 Cache workaround to pass upgrade test. 2017-04-05 12:55:16 -04:00
jsteemann e5d733867e make more tests pass 2017-04-05 14:57:18 +02:00
Simon Grätzer 5822e77843 invokeOnAllElements 2017-04-05 13:38:43 +02:00
Simon Grätzer b69294641f Moving api-wal-noncluster and adding running trx 2017-04-05 11:54:30 +02:00
Simon Grätzer 57b0aa4a3d Any iterator 2017-04-05 10:48:08 +02:00
Simon Grätzer 96dc714139 Fixed some tests in shell_server 2017-04-04 15:10:17 +02:00
Simon Grätzer 245b4b87ea Fixing comparator use 2017-04-04 14:11:14 +02:00
Michael Hackstein 069394634c RocksDB Indexes do now destroy their cache as soon as they are dropped, instead of during deletion 2017-04-03 17:10:10 +02:00
Jan Christoph Uhde e49024f16e add drop collection to rocksdb engine 2017-04-03 10:41:02 +02:00
Dan Larkin 23038fdc3b Fixed uniqueness checking in primary index. 2017-03-31 18:03:50 -04:00