Хранение SQL в Cosmos

Question

Мне нужно сохранить данные в Cosmos в таблицах SQL вместо файлов HDFS.

Я развернул виртуальную машину в облачном разделе FI-Lab, где я установил версию 0.14.0 Orion и 0.3 Cygnus. Я настроил Cygnus для хранения данных в HDFS и SQL ... Проблема в том, что сохранение в файлах HDFS прекрасно работает, но это невозможно в таблицах SQL, несмотря на то, что в прошлом я его получил. Вот почему я запутался

Я думаю, что если упорство HDFS работает, это должно быть проблемой файла cygnus.config. Так я показываю его ниже:

# APACHE_FLUME_HOME/conf/cygnus.conf 

# The next tree fields set the sources, sinks and channels used by Cygnus. You could use different names than the 
# ones suggested below, but in that case make sure you keep coherence in properties names along the configuration file. 
# Regarding sinks, you can use multiple ones at the same time; the only requirement is to provide a channel for each 
# one of them (this example shows how to configure 3 sinks at the same time). 
cygnusagent.sources = http-source 
cygnusagent.sinks = hdfs-sink mysql-sink 
cygnusagent.channels = hdfs-channel mysql-channel 

#============================================= 
# source configuration 
# channel name where to write the notification events 
cygnusagent.sources.http-source.channels = hdfs-channel mysql-channel 
# source class, must not be changed 
cygnusagent.sources.http-source.type = org.apache.flume.source.http.HTTPSource 
# listening port the Flume source will use for receiving incoming notifications 
cygnusagent.sources.http-source.port = 5050 
# Flume handler that will parse the notifications, must not be changed 
cygnusagent.sources.http-source.handler = es.tid.fiware.fiwareconnectors.cygnus.handlers.OrionRestHandler 
# URL target 
cygnusagent.sources.http-source.handler.notification_target = /notify 
# Default organization (organization semantic depend on the persistence sink) 
cygnusagent.sources.http-source.handler.default_organization = org42 

# ============================================ 
# OrionHDFSSink configuration 
# channel name from where to read notification events 
cygnusagent.sinks.hdfs-sink.channel = hdfs-channel 
# sink class, must not be changed 
cygnusagent.sinks.hdfs-sink.type = es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionHDFSSink 
# The FQDN/IP address of the Cosmos deployment where the notification events will be persisted 
cygnusagent.sinks.hdfs-sink.cosmos_host = 130.206.80.46 
# port of the Cosmos service listening for persistence operations; 14000 for httpfs, 50070 for webhdfs and free choice for inifinty 
cygnusagent.sinks.hdfs-sink.cosmos_port = 14000 
# default username allowed to write in HDFS 
cygnusagent.sinks.hdfs-sink.cosmos_default_username = quiquehz 
# default password for the default username 
cygnusagent.sinks.hdfs-sink.cosmos_default_password = 'password' 
# HDFS backend type (webhdfs, httpfs or infinity) 
cygnusagent.sinks.hdfs-sink.hdfs_api = httpfs 
# how the attributes are stored, either per row either per column (row, column) 
cygnusagent.sinks.hdfs-sink.attr_persistence = column 
# prefix for the database and table names, empty if no prefix is desired 
cygnusagent.sinks.hdfs-sink.naming_prefix = 
# Hive port for Hive external table provisioning 
cygnusagent.sinks.hdfs-sink.hive_port = 10000 

# ============================================ 
# OrionMySQLSink configuration 
# channel name from where to read notification events 
cygnusagent.sinks.mysql-sink.channel = mysql-channel 
# sink class, must not be changed 
cygnusagent.sinks.mysql-sink.type = es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionMySQLSink 
# the FQDN/IP address where the MySQL server runs 
cygnusagent.sinks.mysql-sink.mysql_host = 130.206.80.46 
# the port where the MySQL server listes for incomming connections 
cygnusagent.sinks.mysql-sink.mysql_port = 3306 
# a valid user in the MySQL server 
cygnusagent.sinks.mysql-sink.mysql_username = quiquehz 
# password for the user above 
cygnusagent.sinks.mysql-sink.mysql_password = 'password' 
# how the attributes are stored, either per row either per column (row, column) 
cygnusagent.sinks.mysql-sink.attr_persistence = column 
# prefix for the database and table names, empty if no prefix is desired 
cygnusagent.sinks.mysql-sink.naming_prefix = 

#============================================= 
# hdfs-channel configuration 
# channel type (must not be changed) 
cygnusagent.channels.hdfs-channel.type = memory 
# capacity of the channel 
cygnusagent.channels.hdfs-channel.capacity = 1000 
# amount of bytes that can be sent per transaction 
cygnusagent.channels.hdfs-channel.transactionCapacity = 100 


#============================================= 
# mysql-channel configuration 
# channel type (must not be changed) 
cygnusagent.channels.mysql-channel.type = memory 
# capacity of the channel 
cygnusagent.channels.mysql-channel.capacity = 1000 
# amount of bytes that can be sent per transaction 
cygnusagent.channels.mysql-channel.transactionCapacity = 100 

Answer 1

Имея взгляд к вашей конфигурации, вы указываете на сервер MySQL работает в Космосе NameNode. Я не знаю, является ли это ошибкой (и тогда это может быть причиной того, что она не работает должным образом), или это ваша реальная цель (в этом случае я должен предупредить вас, что внешнее соединение с базой данных будет автоматически отключено) , В любом случае сервер MySQL должен жить вне космоса, в вашей VM, поскольку Cosmos предназначен только для хранения HDFS и анализа MapReduce.