1
0
Fork 0
Commit Graph

384 Commits

Author SHA1 Message Date
Jan Christoph Uhde 5b2e3f041b disable-foxx 3.4 (#9593) 2019-07-31 10:58:42 +02:00
Heiko abfe61de33 Bug fix 3.4/foxx auth req object (#8361)
* code now fits to the documentation of the arangoUser object inside the foxx request object

* more tests
2019-03-26 08:09:45 +01:00
Simon a52e6fa3d3 Sync Foxx Queues (#8254) 2019-02-25 17:16:26 +01:00
Tobias Gödderz d48495c195 [3.4] Foxx queue jobs hanging after Foxxmaster crash (#7921)
* Fixed bug where the Foxxmaster doesn't reset jobs after a crash when it should, or a non-master coordinator removes jobs in progress during startup

* Added a regression test

* Removed foxxmaster test from greylist

* Updated CHANGELOG

* Fixed non-maintainer compile
2019-01-14 16:06:48 +01:00
Alan Plum da4179af47 Fix TypeError for invalid manifests (3.4) (#7603)
* Fix TypeError for invalid manifests

* Remove unnecessary wrap

* Update CHANGELOG
2018-12-10 15:46:23 +01:00
Tobias Gödderz f61ccd4047 Reload Foxx routes during startup (#7531) 2018-11-29 15:31:40 +01:00
Michael Hackstein 75f89ed74b
Bug fix/update foxx services after restore (#6774) (#7455)
* Bug fix/update foxx services after restore (#6774)

* triggers foxx queue update which leads to the internal selfHeal function of a foxx app to make foxx apps publicy visible after a foxx app is restored

* this will also add the recognition of imported foxx services

* Testing JS refactoring

* Added test case for Foxx dump -> restore cases

* Fixed check, sth- i tries to contact to the foxx service. Test is red now.

* Fixed tests for dump / restore using Foxx. Also first draft to fix the fox side. Blocked by sth. fixed in the Main Repositorx already

* Fixed typo in dump/restore testsuite

* Removed old version of the fix

* Fixed JSLint

* Removed debug-log output from foxx manager
2018-11-26 13:22:50 +01:00
Alan Plum 4b8a72d1ed Fix Foxx complaining about valid manifest.json $schema value (3.4) (#7304)
* Fix manifest $schema schema

* Remove unused argument

* Add to CHANGELOG
2018-11-20 13:36:59 +01:00
Alan Plum 1ea6374511 Fix Foxx Queues infinite maxFailures (#7224) 2018-11-12 13:21:12 +01:00
Alan Plum e821ed6923 Fix Foxx API handling of broken services (#6459) (#7013)
* Add tests for service failing on mount

* Don't install broken services

* Restore error page for broken services

* Prefer JSON error pages

* Expose service upsert in API

* Add Foxx fixes to CHANGELOG

* Update CHANGELOG

Typo
2018-10-24 15:29:31 +02:00
Simon a8eabf95d4 Fix foxx test failures (#6725) 2018-10-05 12:49:17 +02:00
Alan Plum cfa0e0db55 Better foxx errors (#6264)
* Fix codeFrame formatting

Pushing the codeFrame first results in ugly line breaks or extra spaces.

* Include cause in MODULE_FAILURE message

The error is always a wrapper for an underlying cause so it should be easier to determine what caused it.

* Add a codeFrame for failed _install

We normally only want this in dev mode but service CRUD should always provide detailed info.

* Expose codeFrame in Foxx CRUD

The codeFrame can only come from a failed _install, so let's make it visible to Foxx CLI and friends.

* Add codeFrame to runScript

Scripts can also only be executed with access to the Foxx API so we can provide the same info as during CRUD.

* Trim codeFrame fileName to basePath

If we have a basePath, we don't need to expose the full path to the user for the codeFrame to be meaningful.

* Fix getReadableName

Previously 'catch-fire' would become 'Catch  Ire'.

* Add to changelog
2018-08-28 14:36:33 +02:00
Tobias Gödderz de4f5587ae Gharial rewrite in C++ (#5631)
* Built a C++ skeleton REST handler for gharial, with fallback to the JS handler

* Moved aql::Graph to graph::Graph

* Added complete edge definitions to Graph

Also:
- some cleanup
- used forward-declarations in headers
- use Graph in graph rest handler

* Handle graph lookup failures according to the test suite

* Added GET vertex

* Bugfixes in ResultT

- Added missing #include
- Fixed move semantics

* Move central code of readVertex to GraphOperations

* ResultT fixes and complements

* Implemented a graph cache

* Added and used graph cache to the rest handler

* Added GET edge

* Added DELETE edge

* Extracted some code

* Added PATCH and PUT for both edge and vertex

* Moved update/replace transaction code to GraphOperations

* Added stub routes for POST and a TODO note

* Added a test checking that deleting a vertex removes all incident edges as well

* Added a test checking that deleting a vertex does not remove edges in non-graph collections

* fixed compiler warnings and errors

* Began work on DELETE vertex

For this, added a V8Context to allow for AQL queries to use subtransactions

* Continued work on DELETE vertex (still WIP)

* prep for graph post routes

* fixed removeVertex operation (aql)

* added post vertex and post edge gharial routes

* wasSynchronoues flag changed

* gharial post c++ handler, naming conventions

* added gharial tests

* temporary disabled cache (because not completed), added graph property read functions

* added c++ gharial list vertex collections

* added c++ gharial graph config

* added c++ gharial list graphs

* added graph manager class

* first implementation of create graph in c++, WIP

* changed error messages

* added etag to create graph api, still multiple edge definition check missing

* finished POST /_api/gharial/<graph>

* WIP - DELETE /_api/gharial/<graphName>

* added DELETE /_api/gharial/<graphName> validation, still missing correct response

* gharial delete

* fixed delete gharial lock

* finished DELETE /_api/gharial/<graphName>

* added routes for graph based vertices and edge definitions

* improved delete route

* added add new edge definition to existing graph

* patch edge definition in a graph, still <WIP>

* finished edit edge definition route

* code changes due to devel code changes

* added remove edge definition route

* added vertex delete function

* added todo note regarding one drop collection issue

* add oprhan collection to graph route implemented

* Added a test

* Updated a comment

* Several minor changes

* Minor changes during review

* Changes during review

* Changes during review

* Bugfix: orphans may be null or omitted

* Bugfix: resolve externals

* minor code changes

* seperated graph class to independent component classes

* seperated graph class to independent component classes

* removed log output

* fixed create collection behaviour in a cluster environment

* fixed enterprise graphs behaviour in c++ gharial api

* removed log output

* formatting

* improved error handling, fixed a linux compile bug

* more result refactoring

* more result type cleanup

* fixed wrongly defined test

* result handling

* error handling

* more refactoring

* Bugfix: avoid race condition in cluster when creating collections

* updated graph documentation

* added graph related static strings

* static strings, new method to create options for gharial created collections

* Some minor cleanup

* more use of static strings

* minor code changes, review

* added missing parseint

* removed gharial foxx, added js common module, added v8 general graph module

* correct use of virtual method

* more v8, js general graph, broken state

* more v8 graph functions

* fixed editEdgeDefinition, added drop function

* fixed drop behaviour

* added _list, _exists

* added c++ rename graph collections, added v8 + graph module function

* Added a regression test

* added graph._deleteEdgeDefinition, v8, server

* more v8g

* added _removeVertexCollection

* added _extendEditDefinitions

* todo, need to add a helper sort method for a local defined relation

* fixed test

* fixed lots of tests, added more client functions, _addVertexCollection on client module is still broken

* added more client graph functions, all tests green

* more client functions

* add del edge def route

* Fix use after move

* Minor changes in client general-graph.js module

* Make a copy before sorting (don't touch the argument)

* Minor changes and some additional asserts in graph tests

* Consistently set parameter defaults

* Renamed static strings

* Remove superfluous function

* Made comment more verbose

* Minor changes in general-graph-common.js

* Added missing template arguments

* Fixed community build

* Cleanup in editEdgeDefinition

* Regression test & bugfix: comparison of edge definitions didn' order from and to

* Fixed errors introduced by merge

* Minor changes in v8-general-graph.cpp

* Fixed test failure due to wrong error code in CE

* added missing id field

* Added permission checks for graph._create

* Removed assertion that is no longer valid

* Moved removeGraph from GraphOperations to GraphManager

* Allow C++ implementation of graph._drop to handle smart graphs

* Flush js client db cache after creating/dropping collections via the general graph module

* Added _deleteEdgeDefinition to the general graph client module

* WIP: Added permission checks for drop graph

* Fixed permission checks for drop graph

* Added permission checks for other graph operations

* Bugfix: assert edge definitions are returned in order

* Some cleanup

* Removed unused method

* Minor improvements in GraphManager

* Fixed a type in general-graph common module

* Most useful fix of all times ever: Do not auto cast from bool to int and alternate error/noerror by this

* Added the initial keyword to StaticStrings

* Added a new error code, used whenever a user tries to inject a documentcollection as a relation into the graph, which is invalid

* Some GraphManager/Ops/Graph cleanup. Less Slice parsing, more usage of GraphObjects

* Test edgeDefinitions in graphs with a defined ordering

* GraphClass Layout cleanup

* Do not test error messages, use codes instead

* Recreated backwards compatibility of Graph Creation Permission errors

* Changed error-code if edgedefinition is used twice

* Added a StaticString for the GraphName

* Renamed graphToVpack => graphForClient

* Partly fixed graph-api test to work with better error messages. Still red: The edgeDefinitions are now sorted, the test is supposed to sort his own list, but appearently does not do so. Under investigation

* Added a new error code that rejects injection of differently sharded smart collection into smartgraph. Should be more helpful to our users

* graph createCollectionOptions now require an open object to be cross-called from enterprise. Made enterprise switch for creation of graph more elegant.

* Updated graphs.cpp

* Massive refactoring. Made Factories for graphs to make SmartGraph much more transparent. Also reduced amount of multiple implementations of the same stuff. Killed vocbase/graphs use GraphManager instead. Removed usage of GraphCache, was not completely implemented anyway and only partially used, which is bad at the moment. Option for later improvement never the less

* Adapted JS code to now really use c++ variants. ALso included 3 Classes: Graph, SmartGraph and GraphModule.

* Fixed undefined behaviour in Remove Vertex. Fixed smartgraph sharding if one collection already exists.

* Removed DEBUG output

* Removed DEBUG logs

* Removed dead code

* Fixed Graph EdgeDefinition test, they now have a different ordering.

* Added a test when adding a vertexCollection that it is actually valid in the graph

* Client Graph API now correctly sends `orphanCollections` and not `orphans`

* Let GraphOperations modify the graph in-place. It should now properly handle edgeDefinitions.

* Added initial cid StaticString

* Included the vocbase in fromPersistence creation of Graphs. Only required to enhance 3.3 SmartGraphs on the fly.

* Fixed internal error message

* Fixed compiler isses originiated from merging

* Removed unused imports

* Regenerated generated file
2018-08-09 09:30:04 +02:00
Alan Plum 046a90e8a3 Add pbkdf2 to Foxx auth module (#5667)
* Add pbkdf2 to Foxx auth module

* Add to CHANGELOG
2018-07-27 11:34:52 +02:00
Simran 34ec56d421 Feature/misc spelling corrections (#5164) 2018-07-13 13:06:20 +02:00
Mark 435523167f Bug-fix Foxx empty json request body (#5304) 2018-05-16 16:54:38 +02:00
Mark 3f557d64cc Fix Foxx queues (#5352) 2018-05-16 16:54:03 +02:00
Tobias Gödderz afe13fba57 Fix race condition between foxx install and selfHeal (#5273) 2018-05-11 12:27:56 +02:00
Alan Plum 2a6f8a21aa Allow specifying Foxx manifest.json value (#5157) 2018-04-20 12:55:18 +02:00
Simon 345fc3c0b7 Refactor Authentication Layer (devel) (#4592)
* Cherry Picking LDAP changes

* Adding missing merges

* Fixing remaining mentions of FeatureCacheFeature

* Fix jslint

* Fixing some failed tests

* Fixing cluster authentication issue, red tests

* Fixing ldap testsuite, adding trace logging

* Fixint ldap tesuite setup and LDAP recognition

(cherry picked from commit 686d28a779)

* Fixing wrong assert

* Adding changelog entry, making requested changes from code review

* Fixing dump_authentication, fix typos

* improvements found during code review

* oops

* more use of sessionstorage

* fix tests

* Fixing broken handling, disallowing adding of local users when disabled

* Fixing testInvalidGrants

* Removing undefined auth level externally

* Fixing previous commit

* added tests for ldap search mode

* intentionally removed `after` methods from tests

because they are executed before the tests start
no cleanup is performed right now after the authentication tests
however, a cleanup is done at start of every test

* ldap tests all modes

* forward port changes from 3.3

* added generated files

* forward port missing changes for web UI

* added generated files

* added generated files
2018-02-28 13:24:28 +01:00
Mark cd97787adf Fix Foxx API (#4614) 2018-02-20 10:22:27 +01:00
Simon 35136a89c0 Fix some problems with active failover (#4540) 2018-02-09 15:11:53 +01:00
Alan Plum 232abd4bd0 Fix legacy Foxx HTTP API compat layer (#4265) (#4449) 2018-02-05 12:35:51 +01:00
Mark 599da158b5 Bugfix Foxx API tests (#4446) 2018-02-05 11:31:27 +01:00
Alan Plum 58cfd340cc Don't greedily delete legit app folders (#4437)
See #4395
2018-02-05 11:30:05 +01:00
Alan Plum 05d0c81d7d More consistent config/deps output in Foxx API (devel) (#3792)
* Correctly expose parsed/raw config

This makes accessing the configuration more consistent by distinguishing between the raw (assigned) and parsed (validated/default) values.
Previously there was no way to recognise default values for unset config options and the "current" value did not actually reflect what the service sees.

* Add minimal option to config/deps routes

The default values are backwards compatible with the existing behaviour but can be overridden to consistently get more/less output.

* Actually merge warnings into non-mininal response
2018-01-30 16:50:36 +01:00
Alan Plum d37fd83a96 Warn about Foxx manifest names (#3899)
Via #3818
2018-01-05 14:49:05 +01:00
Jan f5d71f42d2
added tests and updated documentation for Foxx queues (#3861) 2017-12-08 15:52:45 +01:00
jsteemann 80ff79d0e4 fix issue #3858 2017-12-01 12:47:07 +01:00
Jan 9eb5e1545f Bug fix/fix cluster foxx queue startup (#3507)
* slightly clean up indexes

* proper stringification of error code

* do not run AQL queries during Foxx queue startup

* always create keyspace
2017-10-25 18:07:27 +02:00
Simon Grätzer 7c31960cf2 Feature/async failover (#3451) 2017-10-18 23:59:29 +02:00
Simon Grätzer a7903495a5 Fix createQueue silent unique constraint violation, Fix asan detection in clang, (#3421) 2017-10-18 12:21:31 +02:00
Wilfried Goesgens c662dfc863 Cleanup after each testrun (#3109)
* Cleanup after each testrun

 - add a success testsuite next to the fail testsuite
 - create temporary directories in the fail & success testsuites
 - add a function to cleanup the last unshifted directory
 - don't say we're crashed when exiting with the help message

* Specify temp path for SUT so its files will be removed after cleanup.

 - cleanup
 - don't use getTempFile(), it will create another subdirectory that we don't clean up afterwards...

* Fix temp path usage

 - creating a 'bundles' directory wherein we would keep our temporary bundle files would collide if we'd clean it up.
 - create 'bundles-tmpxxxx' instead so its individual to us and we can remove it when done without problems.

* properly create & pass in a temporary directory - as we now require this.

* Remove intermediate directory recursive - for some unknown reason there may be temp files created.

* we need to unify tempfilenames here too.

* use individual suite names

* use the standart performTests to run the backup tests - so we have the proper directory and server handling from the infrastructure.

* fix temporary directory handling in the recovery tests.

* script to build debugging debian packages

* Improve test shutdown

 - handle a set of options globally per testsuite, so they don't need to clone them inside.
 - properly cleanup the slave instance of the replication tests after each run.
 - The per testsuite options vector will fix the behaviour that after one failed testsuite
   all other suites won't be cleaned up anymore.

* don't call the overal cleanup if anything failed.

* fix cleaning up of directories.

* cleanup, fix directory creation / deletion / dummy creation.

* cleanup, fix directory creation / deletion / dummy creation.

* fix tempdir handling for the recovery tests
2017-08-30 10:48:24 +02:00
Wilfried Goesgens d0f159e848 Don't use global static variables as collection caches - this may cau… (#3150)
* Don't use global static variables as collection caches - this may cause troubles.

* cleanup: remove old cluster kickstarter UI remaints.

* cleanup - remove references to long gone tests
2017-08-28 15:04:05 +02:00
Wilfried Goesgens 0c2261445b use indices on our foxx queues queries; Migrate queries away from aqb… (#3101)
* use indices on our foxx queues queries; Migrate queries away from aqb; style guide cleanup.

* update CHANGELOG
2017-08-24 17:06:03 +02:00
Mark 136faad9b8 Fix issue #3037: Foxx, internal server error when I try to add a new service (#3056) 2017-08-17 21:47:27 +02:00
Simon Grätzer 3f58a44e3b Fix cases where the root user has access to excluded collections (#2856) 2017-08-03 21:16:08 +02:00
Mark 3a59ea553b Add catch when propagateSelfHeal (#2839) 2017-07-19 17:03:52 +02:00
Mark a033eef12d Fix issue 2684: Legacy service UI not working (#2685) 2017-06-28 12:09:15 +02:00
Mark 50ad6fbd76 Foxx service CRUD optimizations (#2657)
* Pass config/deps on _install so setup can see them

* Only register service when install succeeded

* Don't throw after point of no return: breaky install is breaky

* Delete service files during install if service validation fail

* Delete service files during install if service execution fails

* Fix commitLocalState with invalide services

* Remove deprecated test
2017-06-26 09:28:59 +02:00
Mark 7743db32b9 Fix #2613 (#2614)
Reduce log level when Foxx manager tries to self heal missing database
2017-06-20 12:41:45 +02:00
Alan Plum 4b4b9f8c55 Fix Foxx API error (#2517)
* Fix Foxx API error
* Manager should distinguish between errors and services
* Foxx API should not require service to exist for uninstall
  if force option is used
* Respect options.force in FM _install
2017-06-01 11:59:04 +02:00
Mark f832dd0388 Implement endpoint "commit coordinator state" (#2513)
Implement endpoint "commit coordinator state" (also see #2479)
2017-05-31 18:21:50 +02:00
Alan Plum 7608cde674 Implement clusterfoxx mounting logic (#2514)
* Serve error when service is out of sync
* Don't check system services for bundles
* Use appropriate errorNum if Foxx blows up
* Only check checksum when file exists
* Don't expose stack traces
2017-05-31 18:21:40 +02:00
Alan Plum 83310c12d7
Move FM.listJson into aardvark 2017-05-30 20:19:58 +02:00
Alan Plum 5fc4a17b24
Make sure the bundle doesn't exist before loading it from db 2017-05-30 20:09:58 +02:00
Alan Plum cceccf59da Simplified Foxx self healing (#2511)
* Implement new self-heal
* Add error codes for 503, service missing/outdated
* Detect changes to service via rev
* Pretty print incoming response object in log
2017-05-30 18:27:32 +02:00
Andreas Streichardt d8dc3c85e7 Call version instead of foxx status to save a v8 context 2017-05-22 19:07:24 +02:00
Andreas Streichardt 49328dcecb Add authorizedflag fixing cluster internal foxx requests 2017-05-17 18:14:11 +02:00
Jan Christoph Uhde d7a1b98117 fix liniting errors 2017-05-17 12:47:37 +02:00