1
0
Fork 0
Commit Graph

546 Commits

Author SHA1 Message Date
Frank Celler 2807ef559c Feature/move shard fix (#2626)
Major overhaul of handling of synchronous replication.
2017-06-26 16:55:01 +02:00
Michael Hackstein 9edb884bc8 Feature/planning issue 188 (#2547)
* add warmup documentation - #188

* warn when relinking build directory

* add warmup documentation - #188

* warn when relinking build directory

* Renamed warmup to loadIndexesInMemory which is a better name for the current implementation of this feature.

* Adapted WebUI to state 'load indexes in memory' instead of 'warmup'

* Added loadIndexesInMemory documentation.

* Renamed loadIndexesInMemory => loadIndexesIntoMemory
2017-06-12 18:54:45 +02:00
Simon Grätzer 571356a6ae added pregel vertex / edge count checks 2017-06-07 17:18:59 +02:00
jsteemann 1952b75b75 fix potential nullptr access 2017-06-07 13:22:00 +02:00
Dan Larkin 8c7ba2aaa3 Changed several index/collection APIs to provide better context messages.
Addresses Issue #342.
2017-06-06 12:27:08 -04:00
Simon Grätzer f20a0a8c3d Merge branch 'devel' of https://github.com/arangodb/arangodb into feature/storage-format-refactoring 2017-05-29 16:01:06 +02:00
Simon Grätzer f8ab86e160 Begin storage format refactoring 2017-05-29 16:00:55 +02:00
Michael Hackstein b9d2faa7cc Added warmup function for indexes.
Squashed commit of the following:

commit bc0472d212b2efef0d7b1b8a36f2b43e8432ba86
Merge: eb47631 5f87674
Author: Michael Hackstein <michael@arangodb.com>
Date:   Mon May 29 09:39:31 2017 +0200

    Merge branch 'devel' of github.com:arangodb/arangodb into feature/rocks-index-warmup

commit eb476310b0ca6165be10b37b960933886c2757f5
Merge: d725f21 32149d2
Author: Michael Hackstein <michael@arangodb.com>
Date:   Mon May 29 09:00:41 2017 +0200

    Merge branch 'devel' of github.com:arangodb/arangodb into feature/rocks-index-warmup

commit d725f21f7d61f7c79ba31fc0338881e35f4d8f48
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 15:53:23 2017 +0200

    Activated collection warmup in Cluster Mode. Added a test for collection warmup (SingleServerOnly)

commit 132bf4f9565b0dcf4ec9f84e93897b482a67ce7f
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 12:46:20 2017 +0200

    Implemented client-side warmup. It can now be triggered via http and arangosh

commit 78ea449dff86118814a2f87bdb59dc16544d92b6
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 11:25:40 2017 +0200

    Fixed assertion.

commit dae80f6277dde1a52eadda506858cc36e235bd55
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 11:08:46 2017 +0200

    Improved the CuckooIndexEstimator computeEstimate function. It is now much better on collections with many different values

commit 7abf57876511ba369d7a577e1995d4575e98c7c8
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 11:06:43 2017 +0200

    Edge index warmup will now first check if a document is stored and if not will insert it. Furthermore it resizes the cache with an estimate of how many documents will be most likely inserted

commit 890d8ad4cdfd155249f060fedd5c798b9531d556
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 11:04:47 2017 +0200

    Adjusted thresholds in transactional cache. Wastly increased the amount of tries to get the lock. Lowered the fill-grade boundaries

commit 60972ab7151a3acb78e1aae4149de11d0da7aceb
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 10:45:38 2017 +0200

    Added new Logtpopic CACHE that should be used to log important information regarding caches, like debug ouput on evictions / resizing, or memory-pressure resizing going on

commit 2dfacb9aef6e3fde169032514baca386786d059c
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 10:44:21 2017 +0200

    Fixed an assertion in index iterator. It expected _from/_to to be cached but we modified to cache entire document

commit f05beccef65853c30eda1859a39c28ea2295bc71
Merge: 13c6abf 5c46430
Author: Michael Hackstein <michael@arangodb.com>
Date:   Fri May 26 07:56:58 2017 +0200

    Merge branch 'devel' of github.com:arangodb/arangodb into feature/rocks-index-warmup

commit 13c6abfbc2ddb451a689011110411df68247435e
Author: Michael Hackstein <michael@arangodb.com>
Date:   Wed May 24 09:52:01 2017 +0200

    Added a getExtra feature for the edgeIndex. It now returns the complete edge as extra (temporary). Modified the EdgeCache to cache token and edge document (temporary). Added a warmup Function to collections that is used to warmup the index caches on demand.
2017-05-29 09:40:58 +02:00
Frank Celler ee1c60c67a added _binaryDocument and _binaryInsert (#2506) 2017-05-27 22:49:22 +02:00
jsteemann 8bc6b3d3a1 removed default parameters 2017-05-18 15:06:19 +02:00
Simon Grätzer f101e21328 Geo index update, renaming 2017-05-11 13:19:51 +02:00
jsteemann 0cad04bdb3 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2017-05-04 15:09:39 +02:00
Simon Grätzer 40b8eb2824 Trigger compaction on drop and truncate 2017-05-04 13:45:30 +02:00
jsteemann d892f280ed fix cluster_sync test 2017-05-04 13:16:47 +02:00
jsteemann f1763e491d fix db._engineStats() 2017-05-04 00:33:36 +02:00
jsteemann 910f5680b0 moved files 2017-05-02 12:34:28 +02:00
Max Neunhoeffer c12878b1f0 Port 3.1 changes to devel, JS support AddFollower/RemoveFollower for tests. 2017-04-26 11:06:24 +02:00
Michael Hackstein 1087065ea5 Merge branch 'devel' of github.com:arangodb/arangodb into engine-api 2017-04-13 17:34:25 +02:00
Max Neunhoeffer 53b3a73486 Add a timeout option to <collection>.drop().
This is in particular used to allow the Schmutz to make progress
in the case of a long running transaction.
2017-04-07 15:52:19 +02:00
jsteemann f59c78b587 add engine capabilities 2017-04-06 14:50:27 +02:00
jsteemann 1c43920bfc preparations for revision() 2017-04-05 13:40:18 +02:00
jsteemann dac6aeab0b Merge branch 'devel' of https://github.com/arangodb/arangodb into engine-api 2017-04-05 12:10:40 +02:00
jsteemann b39935b711 fix conversion warning 2017-04-05 12:06:06 +02:00
Jan Christoph Uhde 95007fe28e Merge branch 'engine-api' of https://github.com/arangodb/arangodb into engine-api-arango-result
* 'engine-api' of https://github.com/arangodb/arangodb: (46 commits)
  Merged RocksDBPathBasedIndex into RocksDBVPackIndex.
  fix handling of externals
  fix toArray()
  fix returning documents
  Added hash-, skip list, persistent indexes
  harmonize filtering - filter on full filename.
  mini modifications
  standardize starting of servers for replication tests
  Edge index compliant with design doc
  another fix for #2404
  Moved „removeLargeRange“ method to commons
  Added length byte to IndexValue keys.
  fixed issue #2404
  Added drop to primary index
  Added drop() to indexes
  Fixes a coordinator crash in Clustered Traversal. In case of failover during query planning.
  Fixed edge index
  Expand on CORS documentation
  Let ArangoDB handle the allow-credentials header
  make the authentication tests run on directories
  ...
2017-03-30 19:46:17 +02:00
jsteemann 894666915d mini modifications 2017-03-30 15:50:23 +02:00
Jan Christoph Uhde b83ae2ab82 refactor some code to make use of arangodb::Result 2017-03-30 09:39:21 +02:00
jsteemann e1aef394d8 delete useless DEV_TIMERS, fix rwlockcpp11 2017-03-28 22:38:27 +02:00
jsteemann 71bf18dd12 renaming 2017-03-22 13:01:40 +01:00
jsteemann fbb14ceffc added db._engine() 2017-03-22 11:19:31 +01:00
jsteemann 25a380f77c Merge branch 'devel' of https://github.com/arangodb/arangodb into views 2017-03-16 12:50:11 +01:00
Simon Grätzer f2c133d5f6 Fix pregel test 2017-03-16 10:52:02 +01:00
Simon Grätzer 529cb4ae85 Integer ID for testing 2017-03-15 10:20:48 +01:00
jsteemann 1a06f1e15e initial functionality for views - unstable 2017-03-14 16:35:45 +01:00
Simon Grätzer 7b35f75b23 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel
# Conflicts:
#	arangod/CMakeLists.txt
#	arangod/GeneralServer/GeneralServerFeature.cpp
2017-03-13 12:25:28 +01:00
Michael Hackstein 83663c47b2 Deleted CollectionResult. It is identical to arangodb::Result introduced later. 2017-03-10 17:18:12 +02:00
Michael Hackstein 046793815c Merge branch 'engine-api' of github.com:arangodb/arangodb into engine-api 2017-03-09 16:37:51 +02:00
jsteemann 645889949c move engine-specific V8 functions into storage engine 2017-03-09 12:59:44 +01:00
Michael Hackstein 97824bb53e Include fixes. Update less todos. 2017-03-09 12:34:55 +01:00
Michael Hackstein a7965945d9 Just comment fixes 2017-03-09 12:20:22 +01:00
Michael Hackstein d0244b3d60 Usage of arangodb::Result above int return value. Moved marker logging out of v8-collection 2017-03-09 12:17:20 +01:00
Michael Hackstein 90b4876d3c Moved WriteMarker code Vocbase => MMFilesEngine. Also removed the decission to write a marker out of Vocbase-API 2017-03-09 11:12:39 +01:00
Simon Grätzer 67066c64ad Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2017-03-08 18:20:44 +01:00
Simon Grätzer a1752da492 Added single server support 2017-03-08 18:20:36 +01:00
Michael Hackstein 489f0d12b8 Moved the decision to write a marker into the databaseFeature, it is not triggered from outside any more. Furthermore renameCollection write marker is now done in MMFiles. 2017-03-08 14:54:25 +01:00
Simon Grätzer c049576c59 Added single server support 2017-03-07 15:07:51 +01:00
Simon Grätzer 5b1cb0ec6d Various fixes 2017-03-06 15:41:27 +01:00
Simon Grätzer 27c617fe10 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel
# Conflicts:
#	3rdParty/V8/v8
#	arangod/Transaction/Methods.h
#	arangod/Utils/UserTransaction.h
#	arangod/V8Server/v8-collection.cpp
2017-03-01 14:52:35 +01:00
jsteemann e37e94adbe Merge branch 'engine-api' of https://github.com/arangodb/arangodb into engine-api 2017-02-27 14:38:35 +01:00
jsteemann 666b2f8da9 renaming 2017-02-27 14:38:27 +01:00
Michael Hackstein a884ee2bfb Intermediate commit. Moved many toVPack functions of LogicalCollection to a unified vpack + ignore-keys alternative. This commit is used to run all tests everywhere. May be red 2017-02-27 13:01:07 +01:00
Michael Hackstein c8f03d16e3 updateProperties now returns an CollectionResult container with errorMessage and code. Moved error reporting logic for cluster-case into this function. 2017-02-24 17:00:41 +01:00
Michael Hackstein b6943d233b Fixed v8-collection properties cluster-variant. In some cases it did not contain correct information 2017-02-23 16:46:21 +01:00
Michael Hackstein 076198106d Now also the cluster variant of collection properties uses the shared toVPackVariant 2017-02-23 16:40:33 +01:00
Michael Hackstein cc78b0131f v8-collection-properties now reuses propertiesVPack on logical collection (SingleServerCase) 2017-02-23 15:31:13 +01:00
Michael Hackstein de31c68539 Moved isVolatile into MMfiles collection. 2017-02-23 14:56:54 +01:00
Michael Hackstein 853394da79 Moved keyGenerator into physicalCollection. Only used when actually storing data. 2017-02-23 11:10:58 +01:00
Jan Christoph Uhde 3a201b4362 add persistProperties() to PysicalCollection 2017-02-22 12:47:03 +01:00
Jan Christoph Uhde e9555e4b10 remove rotateJournals from LogicalCollection 2017-02-21 11:06:53 +01:00
Jan Christoph Uhde 071137adf6 move doCompact into PhysicalColletion 2017-02-20 10:53:11 +01:00
Jan Christoph Uhde 2d5e74f7f6 move journaize() to PhysicalCollection
MMFilesCollection ctor might be buggy
2017-02-17 20:06:47 +01:00
Michael Hackstein c3ff82f4e1 Moved version management to MMFiles collection. Logical does not need to know it. Removed PhysicalCollection inclode from logicical collection. Now we should rebuild less code whan changing sth in storage engine. 2017-02-15 15:48:33 +01:00
jsteemann 7ff2e19abb Merge branch 'devel' of https://github.com/arangodb/arangodb into engine-api 2017-02-15 12:46:06 +01:00
jsteemann d49c5c3fc0 simplified internal APIs a bit 2017-02-15 12:42:07 +01:00
Jan Christoph Uhde be89cac445 move path from logical to physical 2017-02-15 11:43:38 +01:00
Michael Hackstein 32bd15d098 Renamed collection->update() to update the properties of a collection to collection->updateProperties(). There is another update() which is used for the stored Documents. 2017-02-14 10:32:24 +01:00
jsteemann 16189aa969 moved TransactionHints.h 2017-02-13 14:36:27 +01:00
Simon Grätzer edab268572 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel
# Conflicts:
#	arangod/Aql/FunctionDefinitions.cpp
#	arangod/Aql/Functions.h
#	arangod/Utils/ExplicitTransaction.h
2017-02-10 15:21:24 +01:00
jsteemann d024a6d00a remove logging for non-topics 2017-02-10 09:32:50 +01:00
Simon Grätzer 9b9f91a81c Pegel: Adding support for "PREGEL_RESULT" in aql 2017-02-07 18:37:23 +01:00
Simon Grätzer 13f19fe51b Merge branch 'devel' of https://github.com/arangodb/arangodb into devel
# Conflicts:
#	arangod/CMakeLists.txt
#	arangod/RestServer/arangod.cpp
#	arangod/Utils/ExplicitTransaction.h
#	arangod/V8Server/v8-collection.cpp
2017-02-02 11:25:23 +01:00
Simon Grätzer 97473e3223 Spelling fixes, reduce log spam 2017-01-31 16:42:17 +01:00
Simon Grätzer cec711bc85 Fixed test 2017-01-31 14:13:58 +01:00
Simon Grätzer 64343cd579 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel
# Conflicts:
#	scripts/startLocalCluster.sh
2017-01-27 15:58:22 +01:00
jsteemann d7b131852c factored out transaction hints 2017-01-25 13:12:09 +01:00
jsteemann 00b1632ece factored out AccessMode from transaction.h 2017-01-25 11:57:21 +01:00
jsteemann a812e69ce7 Merge branch 'devel' of https://github.com/arangodb/arangodb into engine-api 2017-01-25 09:41:26 +01:00
Andreas Streichardt d91a6af1a7 Remove singletransaction and fetch collection from vocbase directly 2017-01-24 18:51:05 +01:00
jsteemann 7fedb4e956 moved files into engine directory 2017-01-23 13:52:57 +01:00
Michael Hackstein 92ad574c97 Moved Indexes to StorageEngine 2017-01-23 13:32:23 +01:00
jsteemann 952c3f848e factored out transaction manager from logfile manager 2017-01-23 12:31:45 +01:00
jsteemann f9fb701505 moved LogfileManager into engine 2017-01-23 10:30:55 +01:00
Simon Grätzer 1bd1a52163 Fixed pregel API 2017-01-21 19:00:37 +01:00
Simon Grätzer dd14b7b2dd Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2017-01-20 15:56:11 +01:00
Kaveh Vahedipour d2760f4ef1 pushing avoidServers property 2017-01-20 15:15:03 +01:00
Simon Grätzer d71ebf7c1d Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2017-01-20 14:42:19 +01:00
Kaveh Vahedipour 648160c68a Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2017-01-20 09:35:45 +01:00
Kaveh Vahedipour a372cf71b4 avoidServers getting there 2017-01-20 09:35:42 +01:00
Kaveh Vahedipour 40c6e2b48c unobserve should not create entries in agency, if empty before 2017-01-20 08:06:27 +01:00
Andreas Streichardt 5a101ed6f0 Fix unloading collections 2017-01-19 17:05:41 +01:00
Andreas Streichardt a9ccdf86c1 Several movement fixes 2017-01-18 17:48:01 +01:00
Simon Grätzer 1d66c418b6 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel
# Conflicts:
#	arangod/CMakeLists.txt
2017-01-16 17:56:44 +01:00
Andreas Streichardt 10d3bd2200 Merge branch 'devel' into schmutz-ng 2017-01-16 15:22:17 +01:00
jsteemann cdabaa931f renaming 2017-01-16 14:48:13 +01:00
Max Neunhoeffer fe9116f7fb More Schmutz-NG.
Does not trigger sync repl yet.
2017-01-12 16:33:57 +01:00
Simon Grätzer b7143906f5 Various changes 2017-01-10 11:15:05 +01:00
Simon Grätzer 59b9167728 Fixed aync mode 2016-12-23 00:20:38 +01:00
Simon Grätzer 97a219da76 Changed document counting 2016-12-21 23:21:23 +01:00
Simon Grätzer b92aef1b7f Small fixes 2016-12-21 11:56:24 +01:00
Simon Grätzer e4951835aa Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2016-12-14 10:40:03 +01:00
jsteemann 4d76ef962b added count(details = true) for sharded collections 2016-12-13 16:03:49 +01:00
Simon Grätzer 730ea4755e Merge branch 'devel' of https://github.com/arangodb/arangodb into devel
# Conflicts:
#	scripts/startLocalCluster.sh
2016-12-09 14:30:19 +01:00
Andreas Streichardt 82682f8d25 Wait for synchronous replication to settle 2016-12-07 18:38:15 +01:00
Andreas Streichardt 11bd9381d5 Add satellite collections 2016-12-06 16:40:50 +01:00
Simon Grätzer 7d7474f6fd Structural changes, Algorithm registry, begin with optimistic recovery 2016-12-01 14:10:52 +01:00
Simon Grätzer 9bcf8cd407 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2016-11-25 12:30:39 +01:00
Simon Grätzer 273c906415 New Features 2016-11-25 12:27:24 +01:00
jsteemann 817933c77b bugfixes for sync replication and collection status in cluster 2016-11-24 17:00:57 +01:00
Simon Grätzer 6c6c2a092d Multiple edge collections 2016-11-18 22:02:13 +01:00
Simon Grätzer 9b1a11e68f Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2016-11-18 14:44:29 +01:00
jsteemann f74fe77a40 fix startup 2016-11-17 17:18:18 +01:00
Simon Grätzer 8e16011c67 Adding RangeIterator 2016-11-16 14:47:30 +01:00
Simon Grätzer c97a0b825c Runtime Statistics, fix in javascript 2016-11-16 11:09:09 +01:00
Simon Grätzer 84ca321fe5 Fixed javascript API 2016-11-15 23:19:44 +01:00
Simon Grätzer 4b3a5a414e More js APIs 2016-11-15 21:19:36 +01:00
Simon Grätzer 524878d242 Fixed aggregators 2016-11-11 12:01:15 +01:00
Simon Grätzer ecc8e29314 Working SSSP 2016-11-09 16:50:31 +01:00
Simon Grätzer 66137e4789 Working multiple collections 2016-11-08 13:45:44 +01:00
Simon Grätzer 034bc41729 Aggregators continued 2016-11-08 11:34:40 +01:00
Simon Grätzer 64817db293 Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2016-11-04 11:55:17 +01:00
Michael Hackstein c6f31091bc In the properties of a collection now distributeShardsLike is contained. 2016-11-03 10:55:22 +01:00
Simon Grätzer 1c0253e5dc Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2016-11-03 00:12:56 +01:00
Simon Grätzer b2819b418a multiple collections 2016-11-03 00:10:36 +01:00
jsteemann cfc9ecd198 fix Visual Studio complaints 2016-10-31 09:59:18 +01:00
Simon Grätzer 8463c41540 Working template version 2016-10-28 20:01:06 +02:00
jsteemann 7efabd3706 fix revision id handling 2016-10-28 13:44:55 +02:00
jsteemann 09e1fda1a0 fix revision id handling 2016-10-24 17:24:27 +02:00
Simon Grätzer ba437cab71 Included support for multiple shards 2016-10-13 16:41:17 +02:00
Simon Grätzer f67acf39be Introduced PregelFeature 2016-10-13 13:31:19 +02:00
Simon Grätzer 7d07b4930e Zwischenstand 2016-10-12 14:01:21 +02:00
Simon Grätzer 1b351ba138 Further improvements 2016-10-08 00:56:24 +02:00
Simon Grätzer 742450f948 Some more fixes 2016-10-07 16:22:38 +02:00
Simon Grätzer d6dc8e776d Merge branch 'devel' of https://github.com/arangodb/arangodb into devel 2016-10-07 11:52:20 +02:00
Simon Grätzer a95ef5ed53 Finished implementation 2016-10-07 11:51:15 +02:00
Kaveh Vahedipour 9f1501e2e6 type in v8-collection 2016-10-05 13:49:02 +02:00
Kaveh Vahedipour ca92128104 typo in v8-collection 2016-10-05 13:44:13 +02:00
Kaveh Vahedipour a44f873cfa load() and unload() in smart graphs 2016-10-05 13:06:50 +02:00
Kaveh Vahedipour 9d3214c539 minor 2016-10-04 15:31:32 +02:00
Max Neunhoeffer 0698065d63 Fix memory leaks around LogicalCollection. 2016-10-04 11:56:35 +02:00
Simon Grätzer c927c3aa15 Initial Pregel try 2016-10-03 18:08:06 +02:00
Max Neunhoeffer 5cd913e003 Fix error handling for ClusterInfo::getCollection. 2016-09-30 14:11:46 +02:00
Max Neunhoeffer db021508da Fix virtual cloning of LogicalCollections and derived types. 2016-09-28 21:05:29 +02:00
Max Neunhoeffer dc706712b1 Silence a compiler warning in enterprise mode. 2016-09-28 16:23:58 +02:00
Kaveh Vahedipour da0f624088 Adding Smart Edge Collection drops to v8-vocbase 2016-09-28 15:50:53 +02:00
Frank Celler eed90fc588 fixed cppcheck warnings 2016-09-09 15:08:48 +02:00
jsteemann 68ebbabe17 delete unused collection directories 2016-09-06 14:40:31 +02:00
jsteemann 1823cff3a9 removed TRI_collection_t 2016-09-02 14:54:13 +02:00
jsteemann 3909103193 privatized some members 2016-09-02 13:10:17 +02:00
jsteemann 4839d65b36 moved things out of TRI_collection_t 2016-09-02 11:56:48 +02:00
jsteemann 5524b25800 removed VocbaseCollectionInfo 2016-09-02 09:41:36 +02:00
jsteemann e69a82a2a7 refactoring 2016-09-02 08:44:31 +02:00