1
0
Fork 0
Commit Graph

474 Commits

Author SHA1 Message Date
Matthew Von-Maszewski 474f0cde31 Bug fix 3.4/scheduler empty reformat (#7872)
* added check for empty scheduler

* removed log, old is 1 not 0

* require running in this thread

* test

* added isDirect to callback

* signature fixed

* added drain

* added allowDirectHandling

* disabled for testing

* Add ExecContextScope object to direct call.

* try alternate initialization of ExecContextScope

* remove ExecContextScope, no help.  try _fifoSize as part of direct decision.

* strand management to minimize reuse of same strand per listen socket

* blind attempt to address Jenkins shutdown lock up.  may remove quickly.

* add filename and line to existing error log message

* Adjust queueOperation() to stop accepting items once isStopping() becomes true.

* revert previous check-in to MMFilesCollectorThread.cpp

* big reformat

* fixed merge conflicts

* Add CHANGELOG entry.
2019-01-08 20:39:42 +01:00
Lars Maier e1dcad0153 Feature 3.4/jwt keyfile (#7864)
* Added jwt-keyfile option and warning for old option.
* CHANGELOG
* Add trimming to --auth.jwt-secret-keyfile
* Adjust some comments.
2019-01-02 21:45:18 +01:00
Frank Celler 9477af198b big reformat 2018-12-26 00:57:05 +01:00
Jan f1b0a803eb
do not use an internal error for JSON parse errors (#7800) 2018-12-19 12:54:20 +01:00
Jan 677522991e
Feature/internal 3306 (#7683) (#7688) 2018-12-06 17:46:58 +01:00
Max Neunhöffer 3a7df19189 Fix super user JWT token behaviour with non-ex. db. (#7656) 2018-12-05 16:50:41 +01:00
Jan ffc823e1c8
Bug fix 3.4/backport optimizations (#7434) 2018-11-26 19:16:05 +01:00
Michael Hackstein 8098bb4eed
Bug fix 3.4/syncing of followers (#7377)
* Added some DEBUG output for replication rest handler

* Some more debug logging.

* Increased the priority of the ReplicationHandler. This way we will not get stuck with locks that cannot be canceled. Also cancel the lock on the correct database.

* Added extensive log output for replication thins

* Added tombstones to RestReplicationHandler. In a very unlikely case the cancel of a lock can be executed BEFORE the code that actually registers the lock, in this case we will now write a tombstone and do not lock.

* Revert "Added extensive log output for replication thins"

This reverts commit 6d4e37ea1e59e3b3457336019cc7dbc4c979504d.

* Added extensive log output for replication things, now in ERR level instead of MAINTAINER only

* Now actually use hours for synchronization

* React to errors under soft lock if they show up.

* Added a retry loop to increase the read-lock timer.

* Added more timeing output in RocksDB collection internals to figure out why the followers are dropped

* Tweaked RocksDB options

* Revert "Tweaked RocksDB options"

This reverts commit 2bf9c43280beda4792c47d079387fe5154cdd896.

* Removed debug output

* Applied all requested changes by goedderz

* Deleted unused variable
2018-11-23 16:08:27 +01:00
Lars Maier ea7b476ab6 [3.4] Endpoints via _system (#7344)
* Allow accessing _api/cluser/endpoints as authenticated user via the _system database.
* Removed start.sh
2018-11-16 10:01:42 +01:00
Jan fac788bb89
fix nullptr access to usermanager object (#7094) 2018-10-26 12:59:00 +02:00
Simon e87b42a0c3 Silence tsan warnings (#7051) 2018-10-24 23:58:47 +02:00
Simon c0455e9c60 Add engine specific collection APIs (#6962) 2018-10-19 15:23:55 +02:00
Lars Maier def04788c5 Fixed bad token generation when username is empty. (#6967) 2018-10-19 12:52:12 +02:00
Jan 18de63c7c8
Feature 3.4/medium priority (#6910) 2018-10-18 17:08:39 +02:00
Matthew Von-Maszewski a9ce39f85c Bugfix 3.4: Merge scheduler changes by Michael & Frank into recent overlapping code changes (#6928)
* manual recreation of bug-fix-3.4/scheduler-high-low within recent Scheduler changes.

* restore Documentation that was unintentionally deleted
2018-10-16 22:51:00 +02:00
Jan 5a347017da
backport some fixes to 3.4 (#6739) 2018-10-08 09:43:14 +02:00
Jan ac3ca432fa Bug fix 3.4/even more ldap debugging (#6735)
* forward-port changes from 3.3

* remove useless log output
2018-10-05 18:44:37 +02:00
Simon f79a7d1a8f Fix crash on Agency / DBserver with user JWT tokens (#6595) 2018-09-26 14:22:27 +02:00
jsteemann c2a642fdaf safety first: turn off sync/async mixing for HTTPS in general 2018-09-20 13:27:39 +02:00
Jan e37576e03c
Bug fix 3.4/fix ssl vst (#6548) 2018-09-19 23:22:40 +02:00
Kaveh Vahedipour 28754cbf15 Feature/schmutz plus plus (#5972)
- Schmutz now called "Maintenance" and completely implemented in C++
 - Fix index locking bug in mmfiles
 - Fix a bug in mmfiles with silent option and repsert
 - Slightly increase supervision okperiod and graceperiod
2018-08-24 12:15:35 +02:00
jsteemann 50be715aeb removed gcc-specific attribute 2018-08-24 12:05:39 +02:00
Simon 948820e484 Various small changes (#6234) 2018-08-24 09:39:03 +02:00
Lars Maier 0944245bfd RestTestHandler (#6204) 2018-08-22 14:54:04 +02:00
Jan cb12be3e4e
fix crash when pressing CTRL+c in arangosh (#6187) 2018-08-21 11:12:11 +02:00
Simon 229c09d434 Allow dirty-reads from passive (#6136) 2018-08-20 16:26:14 +02:00
Jan 102f15bece
removed several unused internal APIs (#6193) 2018-08-20 12:57:58 +02:00
Jan d6a3b66e2a
micro optimizations (#6162) 2018-08-16 08:50:16 +02:00
Jan 43b53bea57
document the RequestLanes, and "fix" the lanes of a few handlers (#6152) 2018-08-15 11:58:16 +02:00
Vasiliy 6fd541d110 issue 427.5: use ApplicationServer reference instead of pointer (#6145)
* issue 427.5: use ApplicationServer reference instead of pointer

* address MSVC build failure
2018-08-15 12:16:02 +03:00
Frank Celler a688dc0962
Feature/remove job queue thread (#5986)
limiting V8 calls in flight
2018-08-10 12:17:43 +02:00
Tobias Gödderz de4f5587ae Gharial rewrite in C++ (#5631)
* Built a C++ skeleton REST handler for gharial, with fallback to the JS handler

* Moved aql::Graph to graph::Graph

* Added complete edge definitions to Graph

Also:
- some cleanup
- used forward-declarations in headers
- use Graph in graph rest handler

* Handle graph lookup failures according to the test suite

* Added GET vertex

* Bugfixes in ResultT

- Added missing #include
- Fixed move semantics

* Move central code of readVertex to GraphOperations

* ResultT fixes and complements

* Implemented a graph cache

* Added and used graph cache to the rest handler

* Added GET edge

* Added DELETE edge

* Extracted some code

* Added PATCH and PUT for both edge and vertex

* Moved update/replace transaction code to GraphOperations

* Added stub routes for POST and a TODO note

* Added a test checking that deleting a vertex removes all incident edges as well

* Added a test checking that deleting a vertex does not remove edges in non-graph collections

* fixed compiler warnings and errors

* Began work on DELETE vertex

For this, added a V8Context to allow for AQL queries to use subtransactions

* Continued work on DELETE vertex (still WIP)

* prep for graph post routes

* fixed removeVertex operation (aql)

* added post vertex and post edge gharial routes

* wasSynchronoues flag changed

* gharial post c++ handler, naming conventions

* added gharial tests

* temporary disabled cache (because not completed), added graph property read functions

* added c++ gharial list vertex collections

* added c++ gharial graph config

* added c++ gharial list graphs

* added graph manager class

* first implementation of create graph in c++, WIP

* changed error messages

* added etag to create graph api, still multiple edge definition check missing

* finished POST /_api/gharial/<graph>

* WIP - DELETE /_api/gharial/<graphName>

* added DELETE /_api/gharial/<graphName> validation, still missing correct response

* gharial delete

* fixed delete gharial lock

* finished DELETE /_api/gharial/<graphName>

* added routes for graph based vertices and edge definitions

* improved delete route

* added add new edge definition to existing graph

* patch edge definition in a graph, still <WIP>

* finished edit edge definition route

* code changes due to devel code changes

* added remove edge definition route

* added vertex delete function

* added todo note regarding one drop collection issue

* add oprhan collection to graph route implemented

* Added a test

* Updated a comment

* Several minor changes

* Minor changes during review

* Changes during review

* Changes during review

* Bugfix: orphans may be null or omitted

* Bugfix: resolve externals

* minor code changes

* seperated graph class to independent component classes

* seperated graph class to independent component classes

* removed log output

* fixed create collection behaviour in a cluster environment

* fixed enterprise graphs behaviour in c++ gharial api

* removed log output

* formatting

* improved error handling, fixed a linux compile bug

* more result refactoring

* more result type cleanup

* fixed wrongly defined test

* result handling

* error handling

* more refactoring

* Bugfix: avoid race condition in cluster when creating collections

* updated graph documentation

* added graph related static strings

* static strings, new method to create options for gharial created collections

* Some minor cleanup

* more use of static strings

* minor code changes, review

* added missing parseint

* removed gharial foxx, added js common module, added v8 general graph module

* correct use of virtual method

* more v8, js general graph, broken state

* more v8 graph functions

* fixed editEdgeDefinition, added drop function

* fixed drop behaviour

* added _list, _exists

* added c++ rename graph collections, added v8 + graph module function

* Added a regression test

* added graph._deleteEdgeDefinition, v8, server

* more v8g

* added _removeVertexCollection

* added _extendEditDefinitions

* todo, need to add a helper sort method for a local defined relation

* fixed test

* fixed lots of tests, added more client functions, _addVertexCollection on client module is still broken

* added more client graph functions, all tests green

* more client functions

* add del edge def route

* Fix use after move

* Minor changes in client general-graph.js module

* Make a copy before sorting (don't touch the argument)

* Minor changes and some additional asserts in graph tests

* Consistently set parameter defaults

* Renamed static strings

* Remove superfluous function

* Made comment more verbose

* Minor changes in general-graph-common.js

* Added missing template arguments

* Fixed community build

* Cleanup in editEdgeDefinition

* Regression test & bugfix: comparison of edge definitions didn' order from and to

* Fixed errors introduced by merge

* Minor changes in v8-general-graph.cpp

* Fixed test failure due to wrong error code in CE

* added missing id field

* Added permission checks for graph._create

* Removed assertion that is no longer valid

* Moved removeGraph from GraphOperations to GraphManager

* Allow C++ implementation of graph._drop to handle smart graphs

* Flush js client db cache after creating/dropping collections via the general graph module

* Added _deleteEdgeDefinition to the general graph client module

* WIP: Added permission checks for drop graph

* Fixed permission checks for drop graph

* Added permission checks for other graph operations

* Bugfix: assert edge definitions are returned in order

* Some cleanup

* Removed unused method

* Minor improvements in GraphManager

* Fixed a type in general-graph common module

* Most useful fix of all times ever: Do not auto cast from bool to int and alternate error/noerror by this

* Added the initial keyword to StaticStrings

* Added a new error code, used whenever a user tries to inject a documentcollection as a relation into the graph, which is invalid

* Some GraphManager/Ops/Graph cleanup. Less Slice parsing, more usage of GraphObjects

* Test edgeDefinitions in graphs with a defined ordering

* GraphClass Layout cleanup

* Do not test error messages, use codes instead

* Recreated backwards compatibility of Graph Creation Permission errors

* Changed error-code if edgedefinition is used twice

* Added a StaticString for the GraphName

* Renamed graphToVpack => graphForClient

* Partly fixed graph-api test to work with better error messages. Still red: The edgeDefinitions are now sorted, the test is supposed to sort his own list, but appearently does not do so. Under investigation

* Added a new error code that rejects injection of differently sharded smart collection into smartgraph. Should be more helpful to our users

* graph createCollectionOptions now require an open object to be cross-called from enterprise. Made enterprise switch for creation of graph more elegant.

* Updated graphs.cpp

* Massive refactoring. Made Factories for graphs to make SmartGraph much more transparent. Also reduced amount of multiple implementations of the same stuff. Killed vocbase/graphs use GraphManager instead. Removed usage of GraphCache, was not completely implemented anyway and only partially used, which is bad at the moment. Option for later improvement never the less

* Adapted JS code to now really use c++ variants. ALso included 3 Classes: Graph, SmartGraph and GraphModule.

* Fixed undefined behaviour in Remove Vertex. Fixed smartgraph sharding if one collection already exists.

* Removed DEBUG output

* Removed DEBUG logs

* Removed dead code

* Fixed Graph EdgeDefinition test, they now have a different ordering.

* Added a test when adding a vertexCollection that it is actually valid in the graph

* Client Graph API now correctly sends `orphanCollections` and not `orphans`

* Let GraphOperations modify the graph in-place. It should now properly handle edgeDefinitions.

* Added initial cid StaticString

* Included the vocbase in fromPersistence creation of Graphs. Only required to enhance 3.3 SmartGraphs on the fly.

* Fixed internal error message

* Fixed compiler isses originiated from merging

* Removed unused imports

* Regenerated generated file
2018-08-09 09:30:04 +02:00
Simon 606c15ce48 VST support in arangosh (#6047) 2018-08-08 12:18:54 +02:00
Frank Celler 653557a678 fixed wrong logging test for http-request-body 2018-08-07 22:25:47 +02:00
jsteemann d80b62f4f9 fix typos 2018-08-07 11:53:51 +02:00
Jan 1c8f6a75dd
Bug fix/fix issue 6076 (#6082) 2018-08-06 14:27:25 +02:00
Jan b278d6874a
allow master & slave to work in parallel for RocksDB WAL tailing (#6059) 2018-08-03 13:37:53 +02:00
Dan Larkin-York 54e607aa39 Added C++ implementation, load balancer support, and user restriction to Pregel API (#6060) 2018-08-03 12:55:28 +02:00
Jan 29ab2fdbe3
Bug fix/3107 (#6046) 2018-08-01 12:50:55 +02:00
Wilfried Goesgens 74a2ed9b44 fix shared pointer handling in async jobs (on behalf of @jsteemann) (#5999) 2018-07-30 14:42:55 +02:00
Simon 2dd8593609 View Replication (#5915) 2018-07-26 10:28:46 +02:00
Dan Larkin-York 83f393500a Add C++ implementation, load-balancing support, and user restriction for tasks API (#5973) 2018-07-25 09:17:43 +02:00
Dan Larkin-York 718e3f7bfa Add load balancing support and user restriction for async jobs API (#5906) 2018-07-23 15:23:18 +02:00
jsteemann e461e34c5d removed unused code 2018-07-19 22:41:01 +02:00
jsteemann c2ffc3d21d Merge branch 'devel' of https://github.com/arangodb/arangodb into bug-fix/fixes-1707 2018-07-17 13:30:18 +02:00
jsteemann 1588c358b9 options cleanup 2018-07-17 12:33:10 +02:00
Jan f4b99bb5ed
Bug fix/meierloch (#5895) 2018-07-17 11:39:50 +02:00
Michael Hackstein 7a95c5e675
Feature/feature phases (#5272)
* Added feature phases

* BasicsPhase and DatabasePhase to the required files. Server now has Feature circles and does not boot. Will be sorted out later on.

* Added ClusterPhase to features

* Added V8Phase to the required features

* Added AQLPhase to the affected features

* Added ServerPhase to Features

* Added FoxxPhase to the relevant features

* Added AgencyPhase to the relevant features

* Moved registration from local variable SYS_SYSTEM_REPLICATION_FACTOR from cluster to V8 as their ordering is now vice versa

* Moved Bootstrap feature into FoxxPhase. It could be moved to ServerPhase easily if the FoxxQueue dependency would be removed

* Final movement of Startup Phases. Now solved all circles.

* Removed merge conflict

* Moved ReplicationTimeout into cluster phase and fixed cross-phase requirements

* Added greetings phase. This phase separates the Basics Phase and is the first to be run. Includes Logger and Hello/Goodbye

* Added the GreetingsPhase in the corresponding features. Now all BasicsPhase features start after greetings Phase. There is some issue in this branch which prevents the Agency from Gossipping right now. Will be fixed next

* Moved creation of the Agent into the prepare phase of the feature. THereby it is guaranteed that agents at least exists before the GeneralServer is activating endpoints

* Recovery needs to be started after the ServerID

* Moved log output of FeaturePhases to DEBUG instead of ERROR.

* Added feature phases for clients

* ClusterFeature now does not directly require AgencyFeature any more

* Added requirement of TravEngineRegistryFeature in AQL feature. Otherwise shutdown may be undefined

* The ApplicationServer can now handout the list of ordered features. Used for testing purposes

* Fixed IResearchVew Tests Setup to honor new feature ordering

* Fixed IResearchViewDBServer Tests Setup to honor new feature ordering

* Started fixing IResearchView Coordinator tests with startup ordering. Not finished yet

* Added startup phases to ViewCoordinator test

* Disabled expected logoutput in ClusterRepairsTest

* Fixed indention in test code

* LinkCoordinator now honors startup ordering

* Link meta now honors startup rdering

* Supress expected cluster logs in ViewTest

* Removed '#' accidentially added.
2018-07-16 14:09:36 +02:00
Tobias Gödderz 2fb9c2aeb0 Check early if authentication is turned off (#5822) 2018-07-10 16:07:55 +02:00
Jan ddd6b73d85
Bug fix/fixes 0907 (#5812) 2018-07-10 08:48:14 +02:00