always create statistics collections & indices & fishbowl (#9967)

* Squashed commit of feature-3.5/hotbackup_devel.

This puts hotbackup into 3.5.

* Port atomic-database-creation-2 to 3.5.

* Remove some wrongly ported code.

* Fix compilation.

* Fix a manual merge error.

* Remove a feature from the mocks which does not exist in 3.5.

* Add some code which was forgotten in manual merge.

* Fix a problem introduced in a manual merge.

* reuse function

* Address some whitespace issues that came up in review

* aardvark should not create the frontend collection

* create _frontend collection from c++

* recheckAndUpdate Callback in CollectionWatcher

* Wrong author ;)

* rm outdated todo

* Update lib/Basics/VelocyPackHelper.h

Co-Authored-By: Michael Hackstein <michael@arangodb.com>

* use logger unique id, use startup logger

* not needed

* optimized vector shardid method

* do not create _modules collection lazy anymre

* Formatting.

* Assert instead of if/TRI_ASSERT(false)

* Don't use exceptions as control structure

* Re-add READ_LOCKER that got lost in translation

* Fix audit log in case database creation fails early.

* legacy sharding

* Add CHANGELOG entry.

* Retry database cancellation indefinitely

* Do not use exceptions in UpgradeTask

* DropCollection is a FAST_LANE action and should not need much time or else retry.

* Remove superflous addition of LdapFeature

Proudly brought to you by ASAN tests

* Fixed check for distributShardsLike sharding on _system database

* Fixed compile issue on tests

* Removed assertion that seems to be not correct yet on devel.

* Sort out google cloud storage as remote. (#9918)

* Add successful method to ClusterCommResult.
* Improve error forwarding for cluster internal communication.

* Feature/hotbackup list retries (#9924)

* retry hot backup listing for 2 minutes in cluster before giving up

* Enable api by default.

* always create statistics collections & indices & fishbowl

* fix broken list of non existing id (#9957)

* Fix compilation after manual merge.

* Fix another compilation problem.

* Yet more fixes for compilation.

* More compilation fixes.

* Removed forgotten, illegal if.

* Removed obsolete include

arangod/VocBase/Methods/UpgradeTasks.cpp

@@ -164,11 +164,9 @@ Result createSystemCollections(TRI_vocbase_t& vocbase,
 
     createdCollections.push_back(colToDistributeShardsLike);
     systemCollections.push_back(StaticStrings::GraphsCollection);
-    if (StatisticsFeature::enabled()) {
+    systemCollections.push_back(StaticStrings::StatisticsCollection);
-      systemCollections.push_back(StaticStrings::StatisticsCollection);
+    systemCollections.push_back(StaticStrings::Statistics15Collection);
-      systemCollections.push_back(StaticStrings::Statistics15Collection);
+    systemCollections.push_back(StaticStrings::StatisticsRawCollection);
-      systemCollections.push_back(StaticStrings::StatisticsRawCollection);
-    }
   } else {
     // we will use GraphsCollection for distributeShardsLike
     // this is equal to older versions
@@ -190,6 +188,7 @@ Result createSystemCollections(TRI_vocbase_t& vocbase,
   systemCollections.push_back(StaticStrings::AppBundlesCollection);
   systemCollections.push_back(StaticStrings::FrontendCollection);
   systemCollections.push_back(StaticStrings::ModulesCollection);
+  systemCollections.push_back(StaticStrings::FishbowlCollection);
 
   TRI_IF_FAILURE("UpgradeTasks::CreateCollectionsExistsGraphAqlFunctions") {
     VPackBuilder testOptions;
@@ -217,14 +216,6 @@ Result createSystemCollections(TRI_vocbase_t& vocbase,
         });
   }
 
-  // check wether we need fishbowl collection, or not.
-  ServerSecurityFeature* security =
-      application_features::ApplicationServer::getFeature<ServerSecurityFeature>(
-          "ServerSecurity");
-  if (!security->isFoxxStoreDisabled()) {
-    systemCollections.push_back(StaticStrings::FishbowlCollection);
-  }
-
   std::vector<std::shared_ptr<VPackBuffer<uint8_t>>> buffers;
 
   for (auto const& collection : systemCollections) {
@@ -267,7 +258,7 @@ Result createSystemCollections(TRI_vocbase_t& vocbase,
 
 Result createSystemStatisticsCollections(TRI_vocbase_t& vocbase,
                                          std::vector<std::shared_ptr<LogicalCollection>>& createdCollections) {
-  if (vocbase.isSystem() && StatisticsFeature::enabled()) {
+  if (vocbase.isSystem()) {
     typedef std::function<void(std::shared_ptr<LogicalCollection> const&)> FuncCallback;
     FuncCallback const noop = [](std::shared_ptr<LogicalCollection> const&) -> void {};
 
@@ -345,7 +336,7 @@ static Result createIndex(std::string const name, Index::IndexType type,
 Result createSystemStatisticsIndices(TRI_vocbase_t& vocbase,
                                      std::vector<std::shared_ptr<LogicalCollection>>& collections) {
   Result res;
-  if (vocbase.isSystem() && StatisticsFeature::enabled()) {
+  if (vocbase.isSystem()) {
     res = ::createIndex(StaticStrings::StatisticsCollection,
                         arangodb::Index::TRI_IDX_TYPE_SKIPLIST_INDEX, {"time"},
                         false, false, collections);