1
0
Fork 0
Commit Graph

276 Commits

Author SHA1 Message Date
Vasiliy 06eb8ade01 issue 344.7: remove more redundant functions (#4863)
* issue 344.7: remove more redundant functions

* backport: fix missed functions under USE_ENTERPRISE
2018-03-15 17:10:28 +01:00
Vasiliy 148bdb7158 issue 344.6: remove some redundant functions (#4842) 2018-03-15 11:03:35 +01:00
Max Neunhoeffer 33ce6acc2c
Merge remote-tracking branch 'origin/devel' into feature/arangosearch-cluster-views 2018-03-09 11:39:25 +01:00
Max Neunhoeffer 28be92ec52
Parse Views hierarchy in loadPlan. 2018-03-08 15:07:46 +01:00
Vasiliy daa1db659c manually-merge: differentiate data-source types by category e.g. LogicalCollection/LogicalView 2018-03-08 13:40:25 +03:00
Vasiliy a988cb19d7 Changes since last PR: create a common ancestor to LogicalCollection/LogicalView (#4719) 2018-03-02 09:02:09 +01:00
Jan 653f95f243
fixed issue #4185: On execution of FULLTEXT search / AQL query db is … (#4238) 2018-01-16 00:27:27 +01:00
Jan b2b6c06cbf
Feature/efficiency (#3736) 2018-01-05 16:51:31 +01:00
Max Neunhöffer 49082d9b0e
clusterInventory: distinguish between isReady and allInSync. (#4213) 2018-01-03 22:17:57 +01:00
Jan 2a96df5ca5
Feature iresearch (#4071) 2017-12-18 15:04:59 +01:00
Simon Grätzer 19f68df395 Use uuid in Initial sync to ensure proper collection mapping (#3965)
* Use uuid in initalsync to ensure proper collection mapping

* fix compile error after merge
2017-12-13 22:12:47 +01:00
Michael Hackstein 5c633f9fae Bug fix/speedup shard distribution (#3645)
* Added a more sophisticated test for shardDistribution format

* Updated shard distribution test to use request instead of download

* Added a cxx reporter for the shard distribuation. WIP

* Added some virtual functions/classes for Mocking

* Added a unittest for the new CXX ShardDistribution Reporter.

* The ShardDsitributionReporter now reports Plan and Current correctly. However it does not dare to find a good total/current value and just returns a default. Hence these tests are still red

* Shard distribution now uses the cxx variant

* The ShardDistribution reporter now tries to execute count on the shards

* Updated changelog

* Added error case tests. If the servers time out the mechanism will stop bothering after two seconds and just report default values.
2017-11-10 15:17:08 +01:00
Jan 7613bc4314 Bug fix/fixes 0211 (#3568)
* remove some non-unused V8 persistents

* do not throw that many bogus assertions

* do not rely on server role being defined

* slightly better debug output for V8 context debugging

* fix collection ids in inventory response

* simplify bootstrap a bit

* slightly better error handling

* make elapsed time a queryable value

* use less memory for stub collections

* added assertions that will always make sense

* added assertions

* do not garbage-collect while waiting

* less copying of parameters

* do not show "load indexes into memory" buttons for mmfiles engine

  as all indexes are in memory anyway

* when a collection is truncated via the web interface, flush the WAL and rotate all active journals

this will make close all open journals on leader and followers and make them subject to compaction opportunities

* fix invalid server id values being passed from web interface to backend

* introduce afterTruncate method for indexes

* added test case for issue #3447

* updated CHANGELOG

* don't warn about replicationFactor for system collections

* check that the queries actually use the geo index and not some other index

* properly report error in web interface

* fix some internals checks that made truncate fail for bigger collections in maintainer mode

* also run a compact() operation after a serious truncate

in order to make iteration over the truncated range much faster
when the collection is next accessed

* increase default maximum number of V8 contexts to at least 16
2017-11-09 12:48:15 +01:00
Simon Grätzer 64e9377c05 Replacing /_api/collection with RestHandler (#3543) 2017-11-02 14:57:17 +01:00
Simon Grätzer 7c31960cf2 Feature/async failover (#3451) 2017-10-18 23:59:29 +02:00
Frank Celler 4846104bf2 fixed LocalDocumentId (is now a class) 2017-10-03 22:18:07 +02:00
Jan 0561bf45ce Bug fix/isrestore (#3283)
* Make isRestore work in the cluster.

This covers sharded collections with default sharding and non-default
sharding.

* always use locally generate revision ids for storing and looking up documents
2017-10-03 11:53:49 +02:00
Simon Grätzer 88d01b89b5 Optimizations for Caches and Graph Traversals (#3169) 2017-08-31 18:33:10 +02:00
Max Neunhöffer f3acea797b Feature/cluster inventory version (#3152)
* Get rid of a compiler warning in community edition.

* Teach /_api/replication/clusterInventory to report Plan/Version and readiness.

This is first implemented in the ClusterInfo library.
Then the clusterInventory code uses it and checks readiness.
Readiness of a collection means that it is created and all shards
and all replications have been created and are in sync.
2017-08-30 13:34:23 +02:00
Jan a212f45a86 Bug fix/issues 2308 (#3117)
* remove API docs for endpoints that were deleted in 3.0

* remove unused variable

* set all scalar values in the ctor, so check tools do not complain

* move Result objects

* prevent potential nullptr access

* remove ambiguous boolean parameter with no clear meaning

* make code check tools happy

* don't allow copying cache Findings, allow moving Results

* the debug functionality for counting RocksDB key ranges was broken

this produced sporadic test failures

* remove superfluous iostream include

* fix range checks

* don't leak if an error happens during database removal

* try to simplify Result a bit

* try to deletion checks a bit more robust
2017-08-24 14:47:22 +02:00
Jan 79d76c89fb Bug fix/use count for checksum (#3033)
* Count as checksum

* Make readLockId optional as well so upgrades are possible

* fix option name in startup script

* fix some replication issues with RocksDB engine
2017-08-15 11:24:54 +02:00
Jan 8e4dac4fc4 fix fulltext index removal performance, simplified code (#3015)
* simplify index API a bit

* fix fulltext index removal performance, simplified code

* updated CHANGELOG

* fix hanging test

* try to fix shutdown problem

* improve fulltext query performance

* fixed duplicate var

* removed obsolete code

* fix some shutdown races

* do not call ensureIndex that often
2017-08-12 09:25:30 +02:00
Kaveh Vahedipour 0b6d6d9287 Fixed distributeShardsLike bug in creating collections. numberOfShard… (#2895) 2017-08-03 19:38:16 +02:00
Jan 62f9231aa5 Bug fix/predictable results data modifcation multiple fors (#2921) 2017-08-01 17:08:35 +02:00
Jan Christoph Uhde ed8efe3566 Feature/issue 387 cluster index estimates (#2866) 2017-08-01 09:53:58 +02:00
m0ppers 6fefe5ab31 In high concurrency situations double checking checksums is a must have (#2890) 2017-07-28 00:43:21 +02:00
Frank Celler a5a25754ed Feature/reduce extraction to projection (#2792)
* reduce extractions to projections

* recycle string buffers in SocketTask

* micro optimizations for mmfiles indexes

* added special lookup function for _key

* moved function into the correct file

* speed up key buffer allocations a bit

* added noexcept specifier

* correctly name variable

* explicitly move bounds

* fix and speedup from/toPersistent functions

* reuse string from ManagedDocumentResult for multiple lookups

* use move-assign

* a bit less work for single server

* speedup AQL function HASH

* single fetch optimization

* performance optimization for the case when no documents need to be returned

* make reduce-extraction-to-projection a RocksDB-only optimizer rule

* cppcheck

* try to fix compile error on MacOS

* bug fix for MacOSX

* missing namespace (in Windows compile)
2017-07-14 08:40:29 +02:00
Frank Celler 40d73d5a8b Revert "[WIP] Feature/reduce extraction to projection (#2735)"
This reverts commit 5bfcff30cd.
2017-07-12 12:52:14 +02:00
Jan 5bfcff30cd [WIP] Feature/reduce extraction to projection (#2735)
* reduce extractions to projections

* recycle string buffers in SocketTask

* micro optimizations for mmfiles indexes

* added special lookup function for _key

* moved function into the correct file

* speed up key buffer allocations a bit

* added noexcept specifier

* correctly name variable

* explicitly move bounds

* fix and speedup from/toPersistent functions

* reuse string from ManagedDocumentResult for multiple lookups

* use move-assign

* a bit less work for single server

* speedup AQL function HASH

* single fetch optimization

* performance optimization for the case when no documents need to be returned

* make reduce-extraction-to-projection a RocksDB-only optimizer rule
2017-07-12 11:22:29 +02:00
Dan Larkin 8c7ba2aaa3 Changed several index/collection APIs to provide better context messages.
Addresses Issue #342.
2017-06-06 12:27:08 -04:00
Simon Grätzer 634254b80f adding load and unload 2017-06-02 14:44:38 +02:00
jsteemann 8bc6b3d3a1 removed default parameters 2017-05-18 15:06:19 +02:00
Simon Grätzer c21bd1eeb8 Rocksb: index locking issue 2017-05-11 15:12:05 +02:00
Simon Grätzer 9950fa7245 removing unused and unimplemented methods 2017-05-10 18:25:41 +02:00
Dan Larkin 841ef8b1a0 Changed index VPack building to include forPersistence flag. 2017-04-17 17:04:50 -04:00
Jan Christoph Uhde cc39030042 move key options ot logical collection 2017-04-06 11:19:30 +02:00
jsteemann 5880daf2a5 Merge branch 'engine-api' of https://github.com/arangodb/arangodb into engine-api 2017-04-05 13:40:25 +02:00
jsteemann 1c43920bfc preparations for revision() 2017-04-05 13:40:18 +02:00
Simon Grätzer 5822e77843 invokeOnAllElements 2017-04-05 13:38:43 +02:00
jsteemann 81dcfbbe3d pass trx into numberDocuments 2017-03-31 11:33:16 +02:00
jsteemann 7bd6bd6e45 slightly adjusted index creation 2017-03-28 08:46:01 +02:00
jsteemann 2c54a6ac1e added some basic transaction handling 2017-03-24 17:23:32 +01:00
jsteemann 25a380f77c Merge branch 'devel' of https://github.com/arangodb/arangodb into views 2017-03-16 12:50:11 +01:00
jsteemann 0bb2711abb Merge branch 'engine-api' of https://github.com/arangodb/arangodb into views 2017-03-16 12:16:23 +01:00
Simon Grätzer 1c5c616875 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2017-03-15 14:12:18 +01:00
jsteemann 1a06f1e15e initial functionality for views - unstable 2017-03-14 16:35:45 +01:00
Michael Hackstein 1ebf562c4e Moved MMFiles out of vocbase.cpp 2017-03-14 10:29:17 +01:00
Michael Hackstein 1c33e77bd8 Moved unloadCollection callback into StorageEngine. 2017-03-13 17:28:08 +01:00
Simon Grätzer 7b35f75b23 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel
# Conflicts:
#	arangod/CMakeLists.txt
#	arangod/GeneralServer/GeneralServerFeature.cpp
2017-03-13 12:25:28 +01:00
Michael Hackstein 83663c47b2 Deleted CollectionResult. It is identical to arangodb::Result introduced later. 2017-03-10 17:18:12 +02:00
Simon Grätzer 67066c64ad Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2017-03-08 18:20:44 +01:00
jsteemann a2a29ab6b1 Merge branch 'engine-api' of https://github.com/arangodb/arangodb into engine-api 2017-03-08 15:07:33 +01:00
jsteemann 3ed19e291d added "shardIds" option for AQL queries to restrict queries to specific shards 2017-03-08 15:07:10 +01:00
Michael Hackstein 4701a40867 Cleanup LogicalCollection.h 2017-03-08 13:26:20 +01:00
Michael Hackstein 580d2a4336 Moved indexes down into the physical collections 2017-03-08 11:03:45 +01:00
Simon Grätzer 2c77f42486 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2017-03-07 15:08:06 +01:00
Michael Hackstein e6ab0afb50 Moved indexBuckets from Logical => MMFiles collection. 2017-03-07 14:07:32 +01:00
Michael Hackstein 31df7eb527 Fixed error in recovery for duplicate collection name 2017-03-06 17:34:54 +01:00
Simon Grätzer 27c617fe10 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel
# Conflicts:
#	3rdParty/V8/v8
#	arangod/Transaction/Methods.h
#	arangod/Utils/UserTransaction.h
#	arangod/V8Server/v8-collection.cpp
2017-03-01 14:52:35 +01:00
Michael Hackstein 7950270134 Removed obsolete code in LogicalCollection. 2017-02-28 16:08:54 +01:00
Michael Hackstein 900b15dd3a API cleanup in LogicalCollection. Removed some unused exposed toVelocypack functions. Removed toVelocyPack with maxTick variant. That is available in the StorageEngine instead 2017-02-28 15:35:31 +01:00
Michael Hackstein df4eadac75 Replaced several specialized toVelocyPack functions of LogicalCollection by toVelocyPackIgnore(). Fixed distributeLike bug that occured through this. 2017-02-28 14:54:32 +01:00
Michael Hackstein a884ee2bfb Intermediate commit. Moved many toVPack functions of LogicalCollection to a unified vpack + ignore-keys alternative. This commit is used to run all tests everywhere. May be red 2017-02-27 13:01:07 +01:00
Michael Hackstein c8f03d16e3 updateProperties now returns an CollectionResult container with errorMessage and code. Moved error reporting logic for cluster-case into this function. 2017-02-24 17:00:41 +01:00
Michael Hackstein 076198106d Now also the cluster variant of collection properties uses the shared toVPackVariant 2017-02-23 16:40:33 +01:00
Michael Hackstein cc78b0131f v8-collection-properties now reuses propertiesVPack on logical collection (SingleServerCase) 2017-02-23 15:31:13 +01:00
Michael Hackstein de31c68539 Moved isVolatile into MMfiles collection. 2017-02-23 14:56:54 +01:00
Michael Hackstein 853394da79 Moved keyGenerator into physicalCollection. Only used when actually storing data. 2017-02-23 11:10:58 +01:00
Michael Hackstein df59411710 Merge branch 'engine-api' of github.com:arangodb/arangodb into engine-api 2017-02-22 14:43:10 +01:00
Michael Hackstein a359424af0 Moved logic of write Marker out of the dropIndex API from Logical collection. Can be derived from the engine internally. 2017-02-22 12:32:54 +01:00
Jan Christoph Uhde 711a71ac4d add LogicalCollection::invokeOnAllElements()
and remove MMFilesPrimaryIndex.h from Transaction/Methods.cpp
2017-02-22 10:40:27 +01:00
Jan Christoph Uhde fcfe66e254 add getAll/AnyIterator to LogicalCollection 2017-02-22 09:24:38 +01:00
Jan Christoph Uhde 3467031cb9 remove transferMarkers from StorageEngine.h 2017-02-21 16:00:37 +01:00
Michael Hackstein 0f7d50765d Merge branch 'engine-api' of github.com:arangodb/arangodb into engine-api 2017-02-21 14:12:16 +01:00
Michael Hackstein fbdbfdcb6d Moved primaryIndex knowledge out of Logical Collection. Also moved logic for close into MMFiles 2017-02-21 14:12:07 +01:00
Jan Christoph Uhde 133961ff8d remove applyForTickRange from LogicalCollection 2017-02-21 13:42:59 +01:00
Jan Christoph Uhde e9555e4b10 remove rotateJournals from LogicalCollection 2017-02-21 11:06:53 +01:00
Jan Christoph Uhde 6e1f77be24 remove ditches from interface. the ditches concept needs fixing!! 2017-02-21 10:53:51 +01:00
Jan Christoph Uhde 0f86801645 remove updateStats from LogicalCollection interface 2017-02-21 10:06:51 +01:00
Jan Christoph Uhde cf970ed230 Squashed commit of the following:
commit 2644735fe06806ec8ffb78ceb2ede04037258952
Author: Jan Christoph Uhde <Jan@UhdeJC.com>
Date:   Tue Feb 21 09:10:55 2017 +0100

    failed ammend

commit c2b8604df5a60fd00e57b4f088fe8a2624d9ac3b
Author: Jan Christoph Uhde <Jan@UhdeJC.com>
Date:   Mon Feb 20 12:49:58 2017 +0100

    move - maxTick to Physical - RACE!
2017-02-21 09:19:49 +01:00
Jan Christoph Uhde 071137adf6 move doCompact into PhysicalColletion 2017-02-20 10:53:11 +01:00
Jan Christoph Uhde f3a1f128ee remove isFullyCollected from LogicalCollection 2017-02-20 10:28:28 +01:00
Jan Christoph Uhde 40a1345f76 Merge branch 'engine-api' of github.com:arangodb/arangodb into obi-engine-api
* 'engine-api' of github.com:arangodb/arangodb:
  Moved logic for drop index into MMFilesCollection
2017-02-20 09:50:17 +01:00
Michael Hackstein 17effdc065 Moved logic for drop index into MMFilesCollection 2017-02-20 09:06:41 +01:00
Jan Christoph Uhde 21cb8e3217 Merge branch 'engine-api' of github.com:arangodb/arangodb into obi-engine-api
* 'engine-api' of github.com:arangodb/arangodb:
  Removed readRevision from LogicalCollection. You only get readDocument with a token. Moved it's logic into MMFilesCollection. removed the ignoreWal parameter from readDocumentConditional. It was always true anyways...
  Moved pre-commit document modifications out of logical-collection into the physical collection. They are only relevant when actually writing to disc.
  Moved useSecondaryIndexes check from Logical to MMFiles collection
  Some cleanup in Logical collection imports
  Moved index filling into the MMFiles collection
  changelog
  grunt build
  css
  ui: queries now using cursors api - issue #2065
  remove crosscompile hack - it doesn't work anymore
  updated CHANGELOG
  add changlog entry for stacktrace log messages
  clean up write lockers a bit
  fix too early closing of sockets
  Moved restoreIndex to Physical collection.
  Fix a shell script used during build.
  Lower log level to debug in shutdown message.
  fix spelling
  Moved saveIndex logic into the physical collection
  fix potential deadlock

Conflicts:
	arangod/MMFiles/MMFilesCollection.cpp
	arangod/MMFiles/MMFilesCollection.h
	arangod/VocBase/LogicalCollection.cpp
2017-02-18 19:10:45 +01:00
Jan Christoph Uhde 2d5e74f7f6 move journaize() to PhysicalCollection
MMFilesCollection ctor might be buggy
2017-02-17 20:06:47 +01:00
Michael Hackstein 91f303e963 Removed readRevision from LogicalCollection. You only get readDocument with a token. Moved it's logic into MMFilesCollection. removed the ignoreWal parameter from readDocumentConditional. It was always true anyways... 2017-02-17 17:29:49 +01:00
Michael Hackstein b84b05244f Moved pre-commit document modifications out of logical-collection into the physical collection. They are only relevant when actually writing to disc. 2017-02-17 16:07:02 +01:00
Michael Hackstein 5366f09ec8 Moved useSecondaryIndexes check from Logical to MMFiles collection 2017-02-17 13:33:00 +01:00
Michael Hackstein e396446a17 Some cleanup in Logical collection imports 2017-02-17 13:16:22 +01:00
Michael Hackstein c543dd06a4 Moved index filling into the MMFiles collection 2017-02-17 12:56:54 +01:00
Michael Hackstein f0fbe78b2d Moved restoreIndex to Physical collection. 2017-02-16 15:01:00 +01:00
Michael Hackstein a3aae2e43b Moved saveIndex logic into the physical collection 2017-02-16 12:53:05 +01:00
Michael Hackstein 94fbfaa453 Moved the entire logic of open() to the Physical collection. It is different for each StorageEngine. 2017-02-15 16:26:55 +01:00
Michael Hackstein c3ff82f4e1 Moved version management to MMFiles collection. Logical does not need to know it. Removed PhysicalCollection inclode from logicical collection. Now we should rebuild less code whan changing sth in storage engine. 2017-02-15 15:48:33 +01:00
jsteemann 7ff2e19abb Merge branch 'devel' of https://github.com/arangodb/arangodb into engine-api 2017-02-15 12:46:06 +01:00
Jan Christoph Uhde be89cac445 move path from logical to physical 2017-02-15 11:43:38 +01:00
Michael Hackstein d49cbffc46 Moved revision handling from Logical to MMFiles collection. That is only relevant for MMFiles. 2017-02-15 11:26:58 +01:00
Michael Hackstein 9e41a7a6bb Merge branch 'engine-api' of github.com:arangodb/arangodb into engine-api 2017-02-15 10:09:55 +01:00
Michael Hackstein 089816a353 Moved Rollback Operation and InsertIntoIndexes from Logical to MMFilesCollection. This is marker/index specific code. 2017-02-15 10:09:51 +01:00