1
0
Fork 0
Commit Graph

270 Commits

Author SHA1 Message Date
Simon 58c310fe7b fix docublocks parameter naming (#10255) 2019-10-16 17:58:18 +03:00
Simran e9019b1fb7 Double line-breaks break indention (#9821) 2019-10-08 15:41:31 +03:00
Kaveh Vahedipour 2584d69a33 [3.5] hotbackup path normalisation and list documentation updates (#10143)
* path normalisation upload

* path normalisation download

* added documentation to hot backup APIs

* added new parameters

* added new parameters

* Polish docublocks.
2019-10-02 19:04:53 +03:00
Max Neunhöffer c067e17bb6 Port backup size to 3.5. (#10117)
* Port backup size to 3.5.

* Lars' fixes from devel.

* Fix compilation one more time.

* Lars' datetime fix from devel.
2019-10-01 16:45:28 +03:00
Max Neunhöffer 328f46e3d6 This merges hotbackup and atomic-db-creation into 3.5. (#9968)
* Squashed commit of feature-3.5/hotbackup_devel.

This puts hotbackup into 3.5.

* Port atomic-database-creation-2 to 3.5.

* Remove some wrongly ported code.

* Fix compilation.

* Fix a manual merge error.

* Remove a feature from the mocks which does not exist in 3.5.

* Add some code which was forgotten in manual merge.

* Fix a problem introduced in a manual merge.

* reuse function

* Address some whitespace issues that came up in review

* aardvark should not create the frontend collection

* create _frontend collection from c++

* recheckAndUpdate Callback in CollectionWatcher

* Wrong author ;)

* rm outdated todo

* Update lib/Basics/VelocyPackHelper.h

Co-Authored-By: Michael Hackstein <michael@arangodb.com>

* use logger unique id, use startup logger

* not needed

* optimized vector shardid method

* do not create _modules collection lazy anymre

* Formatting.

* Assert instead of if/TRI_ASSERT(false)

* Don't use exceptions as control structure

* Re-add READ_LOCKER that got lost in translation

* Fix audit log in case database creation fails early.

* legacy sharding

* Add CHANGELOG entry.

* Retry database cancellation indefinitely

* Do not use exceptions in UpgradeTask

* DropCollection is a FAST_LANE action and should not need much time or else retry.

* Remove superflous addition of LdapFeature

Proudly brought to you by ASAN tests

* Fixed check for distributShardsLike sharding on _system database

* Fixed compile issue on tests

* Removed assertion that seems to be not correct yet on devel.

* Sort out google cloud storage as remote. (#9918)

* Add successful method to ClusterCommResult.
* Improve error forwarding for cluster internal communication.

* Feature/hotbackup list retries (#9924)

* retry hot backup listing for 2 minutes in cluster before giving up

* Enable api by default.

* fix broken list of non existing id (#9957)

* Fix compilation after manual merge.

* Fix another compilation problem.

* Yet more fixes for compilation.

* More compilation fixes.
2019-09-11 13:13:54 +03:00
Jan Christoph Uhde 44ab3f04b5 3.5 -- intermediate commit stats (#9780)
* count intermediate commits

add transactions statistics struct

add tests

put values in object / split test is multiple smaller tests

* fix broken condition

* fix test

* remove print

* fix windows build

* fix mmfiles

* fix cluster

* add structs in docublocks

* we do not know about intermediate commits on coordinators

* add documentation

* add isCluster function

* add serverStatistics to 3.5 client

* update documentation
2019-08-26 16:56:21 +03:00
Simran 73932a784b ArangoSearch refactor (3.5) (#9597)
* Correct View Creation object struct (WIP)

* 404 -> 409, remove old note

* Remove unnecessary DocuBlocks, uppercase Views

* field-name has object value

* Add missing View props, unify

* Remove line break

* Remove subtype for consolidationPolicy

* Indent list

* attempt to fix fields / field-name

* remove links @RESTSTRUCTs and point to docs
Note: this uses an absolute link in order to work in Swagger
2019-08-21 13:26:43 +03:00
Jan a608e38992 Bug fix 3.5/add swagger apis for shards (#9733)
* added swagger API examples for shards methods

* renamed examples

* arhhggg
2019-08-19 15:40:09 +03:00
Simran 7de07df802 Bug fix/fix swagger datatype (#9045) (#9602)
* Bug fix/fix swagger datatype (#9045)

* remove http so https arangos will work

* verify that query parameters are proper swagger data types, fix offending documentation files

* return the actual type - not the list of available ones

* check formats

* there is no uint64 in swagger

* Fresh Swagger
2019-08-06 14:44:58 +03:00
Wilfried Goesgens 18748882d8 Feature 3.5/add js isolate mem statistics (#9569)
* add statistics about isolate memory usage, add isolate numbers to logs

* add statistics about isolate memory usage, add isolate numbers to logs

* add documentation

* Update Documentation/DocuBlocks/Rest/Administration/get_admin_statistics.md

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update arangod/Statistics/Descriptions.cpp

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update arangod/Statistics/StatisticsWorker.cpp

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update lib/ApplicationFeatures/V8PlatformFeature.cpp

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update lib/V8/v8-globals.h

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update arangod/Statistics/Descriptions.cpp

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* fix time handling as sugested by @jsteemann

* cleanup

* Update Documentation/DocuBlocks/Rest/Administration/get_admin_statistics.md

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update Documentation/DocuBlocks/Rest/Administration/get_admin_statistics.md

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update arangod/Statistics/StatisticsWorker.cpp

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>
2019-07-26 16:36:20 +03:00
Jan 9777f53878 Bug fix 3.5/add db list transactions function (#9575)
* added missing function db._transactions(), and equivalent REST API route
GET /_api/transaction

* updated

* use performRequests

* updated CHANGELOG

* added tests, fixed segfault

* use throwNotRunning

* use read transactions for testing (mmfiles blocks with multiple write
transactions on the same collection)
2019-07-26 16:34:57 +03:00
Michael Hackstein d5840c125a Bug fix 3.5/min replication factor (#9524)
* Cherry-pick minReplicationFactor

* Bug fix/failover with min replication factor (#9486)

* Improve collection time of IResearchQueryOptimizationTest

* Added a minReplicationFactor field in Collections. It is not possible to modify it yet and noone cares for it

* Added some assertion son minReplicationFactor

* Transaction API will now reject writes as soon as minimal replication factor is NOT fulfilled

* added minReplicationFactor to the user interface, preparation for the collection api changes

* added minReplicationFactor to VocBaseCollection, RestReplicationHandler, RestCollectionHandler, ClusterMethods, ClusterInfo and ClusterCollectionCreationInfo

* added minReplicationFactor usage to tests

* TODO TEMOPORARY COMMIT FOR TESTING PLEASE REVERT ME

* minReplicationFactor now able to change via collection  properties route

* fixed wrongly assert

* added minReplicationFactor to the graph management ui

* added minReplicationFactor to the gharial api

* Fixed off-by-one error in minReplicationFactor. We actually enforced one more.

* adjusted description of minReplicationFactor

* FollowerInfo Refactoring

* added gharial api graph creation tests with minimal replication factor

* proper cleanup of shell collection tests, removed lots of duplicate code, preparation for some new tests

* added collection create tests using invalid/valid names, replicationFactor and minReplicationFactor

* Debug logging

* MORE Debug logging

* Included replication fast lane

* Use correct minreplicationfactor

* modified debug logging

* Fixed compileissues

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* MORE Debug logging

* Revert "MORE Debug logging"

This reverts commit dab5af28c0.

* Revert "MORE Debug logging"

This reverts commit 6134b664bd.

* Revert "MORE Debug logging"

This reverts commit 80160bdf3b.

* Revert "MORE Debug logging"

This reverts commit 06aabcdfe1.

* Removed debug output

* Added replication fast lane. Also refactored the commands as i cannot take it any more...

* Put some requests of RocksDBReplication onto CATCHUP Lane.

* Put some requests of MMFilesReplication onto CATCHUP Lane.

* Adjusted Fast and MED lane usage in Supervised scheduler

* Added changelog entry

* Added new features entry

* A new leader will now keep old followers in case of failover

* Update arangod/Cluster/ClusterCollectionCreationInfo.cpp

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Fixed JSLINT

* Unified lane handling of replication handlers

* Sorry forgotten in last commit

* replaced strings with static strings

* more use of static strings

* optimized min repl description in the ui

* decr initial loop variable

* clean up of the createWithId test

* more use of static strings

* Update js/apps/system/_admin/aardvark/APP/frontend/js/views/collectionsView.js

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Added some comments on condition, renamed variable as suggested in review

* Added check for min replicationFactor to be non-zero

* Added assertion

* Added function to modify min and max replication factor in one go

* added missing semicolon

* rm log devel

* Added a second information to follower info that can keep track of followers that have been in sync before a failover has taken place

* Maintenance reports previous version now to follower info. instead of lying by itself. The Follower Info now gets a failover save mode to report insync followers

* check replFactor against nr dbservers

* Add lie reporting in CURRENT

* Reverted most of my recent commits about Failover situation. The intended plan simply does not work out

* move replication checks from logical collection to rest collection handler

* added more replication tests

* Include assert only if we are not in gtest

* jslint

* set min repl factor to zero if satellite collection

* check replication attributes in v8 collection

* Initial commit, old plan, does not yet work

* fixed ires tests

* Included FailoverCandidates key. Not fully implemented

* fixed wrong assert

* unified in sync follower reporting

* fixed compiler errors

* Cleanup locking, and fixed potential deadlocks

* Comments about locking order in FollowerInfo.

* properly check uint

* Keep old leader as potential failover candidate

* Transaction methods now use followerInfo to check if the leader can write, this might have the sideeffect that 'failoverCandidates' are updated

* Let agency check failoverCandidates if possible

* Initialize member variables

* Use unified follower reporting in DBServerAgencySync

* Removed obsolete variable, collecting it somewhere else

* repl factor attr check

* Reimplemented previous followers, second attempt now. PhaseOne and PhaseTwo can now synchronize on current.

* Fixed assertion, forgot an off-by-one

* adjusted test to be more preciese now

* Fixed failove candidates list

* Disable write on dropping too many followers

* Allow to run updateFailoerCandidates multiple times with same leader.

* Final fixes, resilience tests now green, crossing fingers for jenkins

* Fixed race on atomics comparison

* Fixed invalid number type

* added nullptr handling

* added nullptr handling

* Removed invalid assert

* Make takeover of leadership an atomic operation

* Update tests/js/common/shell/shell-cluster-collection.js

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Review fixes

* Fixed creation code to use takeoverLeadership

* Update arangod/Cluster/FollowerInfo.h

Co-Authored-By: Tobias Gödderz <tobias@arangodb.com>

* Applied review fixes

* There is no timeout

* Moved AQL + Pregel to INTERNAL_AQL lane, which is medium priority, to avoid deadlocks with Sync replication

* More review fixes

* Use difference if you want to compare two vectors...

* Use std::string ...

* Now check if we are in recovery mode

* Added documentation for minReplicationFactor

* Added readme update as well in documenation

* Removed merge conflict leftovers 0o, i should not trust the IDE

* Update js/apps/system/_admin/aardvark/APP/frontend/js/views/collectionsView.js

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update js/apps/system/_admin/aardvark/APP/frontend/js/views/collectionsView.js

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update Documentation/Books/Manual/Architecture/Replication/README.md

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update CHANGELOG

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update Documentation/Books/Manual/DataModeling/Collections/DatabaseMethods.md

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update Documentation/Books/Manual/ReleaseNotes/NewFeatures35.md

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update Documentation/DocuBlocks/Rest/Collections/1_structs.md

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update js/apps/system/_admin/aardvark/APP/frontend/js/views/graphManagementView.js

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update js/apps/system/_admin/aardvark/APP/frontend/js/views/graphManagementView.js

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Update Documentation/DocuBlocks/Rest/Graph/1_structs.md

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Apply suggestions from code review

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Adepted review requests, thanks for finding!

* Removed unnecessary const

* Apply suggestions from code review

Co-Authored-By: Jan <jsteemann@users.noreply.github.com>

* Moved initilization of variable more downwards

* Apply lock before notify_all()

* Remove documentation except DocuBlocks, covered by PR in docs repo

* Remove accidental indent
2019-07-22 17:48:34 +03:00
Tobias Gödderz 87e5fe7dd2 Bug fix 3.5/clean replication api wal tracking (#9503)
* Use int type for server id

Change serverId to an int

Pass syncerId only for synchronous replication

Added UrlBuilder

structs to classes, reordering

Added Location class, cleanup

Fixed initialization order

Use Location class

Use string for large ints

Documentation

Added clientInfo to ReplicationClientProgressTracker and corresponding rest handlers

Pass clientInfo string in sync replication

Pass clientInfo in addFollower, too

Updated docu

Renamed UrlBuilder to UrlHelper

Updated docu

Try to fix compile error on windows

Fixed a bug and a test

* Implemented @jsteeman's comments
2019-07-18 19:38:31 +03:00
Simran ef3aeb00e5 Documentation 3.5/example generation improvements (#9498)
* Documentation/switch example generation to rocksdb (#9478)

* improve the formatting of the batch request documentation

* split to linewise, check whether we have a json and format it.

* Add missing asterisk

* Minor fixes / improvements
2019-07-18 14:45:59 +03:00
Tobias Gödderz f501e00e9d Bug fix/add shard id to replication client identifier (#9366) 2019-07-08 14:03:42 +02:00
KVS85 ff59b8e37b Fix ArangoSearch documentation examples 2019-07-02 13:07:34 +02:00
Dronplane 0e5185fd6b Fixed REST API examples for new analyzer naming (#9200) 2019-06-05 14:54:11 +03:00
Wilfried Goesgens 8ff54d3433 Doc - Feature/swagger operation ids (#8945)
Helps to grep for Swagger DocuBlocks based on method names in code
2019-05-15 12:23:56 +02:00
m0ppers 33da54d6ed 📝 Adjust link for new docs (#8869) 2019-05-06 11:13:00 +02:00
Simran e5703eba7c
Doc - Fix Views Markup (#7032) 2019-04-30 11:22:55 +02:00
Simon b110f5fc61 Stream Transaction HTTP docs (#8833) 2019-04-29 17:04:27 +02:00
Dan Larkin-York 48fb15e94f Auth-related tests for audit logging (#8790) 2019-04-18 10:45:41 +02:00
Vasiliy 1a22d1360c issue 526.9.1: implement swagger interface, add documentation (#8730)
* issue 526.9.1: implement swagger interface, add documentation

* address review comments

* add ngram

* Formatting

* Move REST description to new Analyzers top chapter in HTTP book

* Missed a DocuBlock

* Add Analyzers chapter to Manual SUMMARY.md

* Move REST API description back to Manual

Headlines were broken

* Add n-gram example
2019-04-16 18:54:30 +03:00
Dan Larkin-York 777ba1a364 Improved audit logging (#8740) 2019-04-15 14:51:32 +02:00
Wilfried Goesgens fafb40ee33 Feature/complete collection documentation (#8734) 2019-04-11 15:53:43 +02:00
Alan Plum 6144459dee Expose "filter" attribute for running Foxx tests (#8700)
* Expose foxx test "filter" option in HTTP API

* Typo sucessful -> successful

* CHANGELOG
2019-04-09 15:32:12 +02:00
Jan 5897baa984
added db.<collection>.getResponsibleShard() (#8683) 2019-04-08 16:10:45 +02:00
jsteemann 8e534ecb09 fix failing example 2019-04-04 16:01:46 +02:00
Jan 39a3f5bc4e
reintroduce smart joins after temporarily reverting them in devel (#8543) 2019-03-23 20:36:02 +01:00
jsteemann dc381a99df Revert "Feature/ncc1701 (#8440)"
This reverts commit 59ad583796.
2019-03-21 19:18:46 +01:00
Jan 59ad583796
Feature/ncc1701 (#8440) 2019-03-21 15:05:36 +01:00
Simran Brucherseifer 4c8cb6704d Doc - Fix cluster health indention 2019-03-20 12:53:40 +01:00
Simran fe67f4f93c
Doc - Make cleanOutServer step mandatory in drain documentation (#8462) 2019-03-20 02:15:27 +01:00
Dan Larkin-York 413e90508f Named indices (#8370) 2019-03-13 18:20:32 +01:00
Vasiliy b1e30937bb issue 520.1.1: decrease link commit interval to 1 second, add workaround for MMFiles not recovering document insertions via WAL (#8313) 2019-03-05 14:50:54 +03:00
Simran 3df86129e8 Doc - Use proper terms for editions (esp. shardingStrategy) (#8014) 2019-02-23 23:11:36 +01:00
Jan 44c6a2d732
Feature/ttl index (#8169) 2019-02-19 14:12:21 +01:00
Luigi Servini 3f2af07e78 Doc - Added default ttl for cursor (#8148) 2019-02-13 13:20:25 +01:00
Vasiliy 98b024d787 issue 466.6: remove unused code, remove hardcoded view meta values (#8130) 2019-02-11 16:32:04 +03:00
Simran f7b9733a84 Doc - Fix build (whitespaces, triple-backticks, comments) (#7857) 2018-12-26 21:41:19 +01:00
Simran dec65b904c Doc - deprecate simple all keys (#7657) 2018-12-06 11:19:31 +01:00
Lars Maier 908df47cd7 [devel] Bug fix/cluster health ui timestamp (#7562) 2018-11-30 16:26:21 +01:00
Lars Maier 52cff7ad55 Feature/engine version added to agent configuration (#7481) (#7524)
* agents' is obtained from leader's configuration
* corrections in Supervision for advertised endpoints
* change log
* Updated Documentation for cluster/health.
* Unified naming convention.
* Fixed missing update of volatile fields.
* Set version in right order.
* Removed debug output.
* Fixed jslint - missing ;
2018-11-29 14:25:40 +01:00
Vasiliy f088733420 issue 496.3: move more coordinator-related logic out of TRI_vocbase_t, rename some arangosearch view configuration parameters, remove some consolidation policies, update iresearch to revision 6fd9760d81b136f769e277ea5b8f53996ed7a1ca (#7166)
* issue 496.3: move more coordinator-related logic out of TRI_vocbase_t, rename some arangosearch view configuration parameters, remove some consolidation policies, update iresearch to revision 6fd9760d81b136f769e277ea5b8f53996ed7a1ca

* address potential deadlock between link creation and FlushThread

* remove code causing nullptr access

* add back lock around reader reopen

* revert: address potential deadlock between link creation and FlushThread

* invalidate payload for each field in FieldIterator before setting a value
2018-11-01 23:10:01 +03:00
Simran 496ab3af05 Doc - Deprecation notes 3.4 (#6405) 2018-10-23 17:45:07 +02:00
Alan Plum 4e25350f6b Fix Foxx API handling of broken services (#6459)
* Add tests for service failing on mount

* Don't install broken services

* Restore error page for broken services

* Prefer JSON error pages

* Expose service upsert in API

* Add Foxx fixes to CHANGELOG

* Update CHANGELOG

Typo
2018-10-23 13:05:03 +02:00
Simon 4c1e8819c2 Add engine specific collection APIs (#6977) 2018-10-19 17:46:33 +02:00
Vasiliy 78567bef09 update iresearch to codebase as of 20181011 (#6858)
* update iresearch to codebase as of 20181011

* backport: address cluster test failures

* backport: address dump test failures

* backport: address discrepency in view creation between single-server and cluster

* backport: address test failure on cluster (revert change)

* backport: address test failures

* backport: address MSVC build issues

* backport: address issue with LogicalDatasource destructing after TRI_vocbase_t

* Revert "backport: address issue with LogicalDatasource destructing after TRI_vocbase_t"

This reverts commit 4f9880bbaa22194dfbb604b5a54658de1d447ac1.
2018-10-12 21:07:12 +03:00
Simon fd81f52ab4 Allow WAL logger to split up transactions (#6800) (#6866) 2018-10-12 17:50:58 +02:00
Michael Hackstein 838e91bbe0
Documentation/improve gharial documentation (#6782)
* Minor update to gharial list call

* Updated documentation for graph creation API

* Updated graph get documentation

* Added a section for Query Parameters in Graph Create

* Improved gharial drop documentation

* Added ReturnBodies for graph creation results

* Added the response object description to graph list

* Minor typo

* Added response bodies to graph get documentation.

* Added Response Body documentation on Graph grop

* Added HTTP documentation for add graph vertex collection

* Added dots to finish one-line API description sentences.

* ReplyBodyNames have to be globally unique 😢

* Documented ResponseBodies for vertex-collection-add

* Documented ResponseBodies for vertex-collection-remove

* Added result bodies for create vertex in graph documentation

* started to extract structure in separate file

* Fixed orphan structure reference

* Reuse the same documentation structore in all Graph APIs

* Documented list edge collections in GraphAPI

* Unified vertex representation

* Documented edge definition creation API

* Documented modification of edge definition

* fix support for external structure definitions

* Unified edge definition struct

* Added documentation code for vertex CRUD in Gharial

* Removed unused struct in GraphRestDocu

* Added documentation for edge CRUD in gharial. Fixed wording in vertex CRUD part

* Added missing dots at end of sentences

* MOre dots at end of sentences

* Removed empty lines

* Fixed missing edge_definition remove documentation

* Fixed a typo
2018-10-11 15:18:11 +02:00