1
В настоящее время я использую SSTableLoader для загрузки данных из нескольких CSV-файлов в cassandra. Раньше, когда я запускал SSTableLoader на Cassandra 2.1.9, это успешно выполнялось без ошибок.Ошибка SSTableLoader в Cassandra 2.2.3 Слишком много открытых файлов
Однако, так как я повышен до Cassandra 2.2.3, я был встречая следующее сообщение об ошибке от SSTableLoader:
org.apache.cassandra.io.FSReadError: java.io.IOException: An existing connection was forcibly closed by the remote host
at org.apache.cassandra.io.util.ChannelProxy.transferTo(ChannelProxy.java:144) ~[apache-cassandra-2.2.3.jar:2.2.3]
at org.apache.cassandra.streaming.compress.CompressedStreamWriter$1.apply(CompressedStreamWriter.java:79) ~[apache-cassandra-2.2.3.jar:2.2.3]
at org.apache.cassandra.streaming.compress.CompressedStreamWriter$1.apply(CompressedStreamWriter.java:76) ~[apache-cassandra-2.2.3.jar:2.2.3]
at org.apache.cassandra.io.util.BufferedDataOutputStreamPlus.applyToChannel(BufferedDataOutputStreamPlus.java:293) ~[apache-cassandra-2.2.3.jar:2.2.3]
at org.apache.cassandra.streaming.compress.CompressedStreamWriter.write(CompressedStreamWriter.java:75) ~[apache-cassandra-2.2.3.jar:2.2.3]
at org.apache.cassandra.streaming.messages.OutgoingFileMessage.serialize(OutgoingFileMessage.java:96) ~[apache-cassandra-2.2.3.jar:2.2.3]
at org.apache.cassandra.streaming.messages.OutgoingFileMessage$1.serialize(OutgoingFileMessage.java:48) ~[apache-cassandra-2.2.3.jar:2.2.3]
at org.apache.cassandra.streaming.messages.OutgoingFileMessage$1.serialize(OutgoingFileMessage.java:40) ~[apache-cassandra-2.2.3.jar:2.2.3]
at org.apache.cassandra.streaming.messages.StreamMessage.serialize(StreamMessage.java:47) ~[apache-cassandra-2.2.3.jar:2.2.3]
at org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.sendMessage(ConnectionHandler.java:363) ~[apache-cassandra-2.2.3.jar:2.2.3]
at org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.run(ConnectionHandler.java:335) ~[apache-cassandra-2.2.3.jar:2.2.3]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
Caused by: java.io.IOException: An existing connection was forcibly closed by the remote host
at sun.nio.ch.SocketDispatcher.write0(Native Method) ~[na:1.8.0_45]
at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:51) ~[na:1.8.0_45]
at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93) ~[na:1.8.0_45]
at sun.nio.ch.IOUtil.write(IOUtil.java:51) ~[na:1.8.0_45]
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471) ~[na:1.8.0_45]
at sun.nio.ch.FileChannelImpl.transferToTrustedChannel(FileChannelImpl.java:483) ~[na:1.8.0_45]
at sun.nio.ch.FileChannelImpl.transferTo(FileChannelImpl.java:579) ~[na:1.8.0_45]
at org.apache.cassandra.io.util.ChannelProxy.transferTo(ChannelProxy.java:140) ~[apache-cassandra-2.2.3.jar:2.2.3]
... 11 common frames omitted
After examining the system.log file in "/var/log/cassandra", I see that the error was caused by "Too many open files".
Here is the error message in the system.log file:
ERROR [STREAM-IN-/169.254.122.80] 2015-12-02 11:37:48,235 StreamSession.java:524 - [Stream #d33ca5f0-98e8-11e5-b1fd-732c31b2403a] Streaming error occurred
java.nio.file.FileSystemException: /var/lib/cassandra/data/pricing/listings-088f7050945411e59780d168f1e8851b/tmp-la-968-big-Data.db: Too many open files
at sun.nio.fs.UnixException.translateToIOException(Unknown Source) ~[na:1.8.0_60]
at sun.nio.fs.UnixException.rethrowAsIOException(Unknown Source) ~[na:1.8.0_60]
at sun.nio.fs.UnixException.rethrowAsIOException(Unknown Source) ~[na:1.8.0_60]
at sun.nio.fs.UnixFileSystemProvider.newFileChannel(Unknown Source) ~[na:1.8.0_60]
at java.nio.channels.FileChannel.open(Unknown Source) ~[na:1.8.0_60]
at java.nio.channels.FileChannel.open(Unknown Source) ~[na:1.8.0_60]
at org.apache.cassandra.io.util.SequentialWriter.<init>(SequentialWriter.java:131) ~[apache-cassandra-2.2.3.jar:2.2.3]
at org.apache.cassandra.io.compress.CompressedSequentialWriter.<init>(CompressedSequentialWriter.java:66) ~[apache-cassandra-2.2.3.jar:2.2.3]
at org.apache.cassandra.io.util.SequentialWriter.open(SequentialWriter.java:168) ~[apache-cassandra-2.2.3.jar:2.2.3]
at org.apache.cassandra.io.sstable.format.big.BigTableWriter.<init>(BigTableWriter.java:75) ~[apache-cassandra-2.2.3.jar:2.2.3]
at org.apache.cassandra.io.sstable.format.big.BigFormat$WriterFactory.open(BigFormat.java:107) ~[apache-cassandra-2.2.3.jar:2.2.3]
at org.apache.cassandra.io.sstable.format.SSTableWriter.create(SSTableWriter.java:84) ~[apache-cassandra-2.2.3.jar:2.2.3]
at org.apache.cassandra.io.sstable.format.SSTableWriter.create(SSTableWriter.java:106) ~[apache-cassandra-2.2.3.jar:2.2.3]
at org.apache.cassandra.io.sstable.format.SSTableWriter.create(SSTableWriter.java:95) ~[apache-cassandra-2.2.3.jar:2.2.3]
at org.apache.cassandra.streaming.StreamReader.createWriter(StreamReader.java:129) ~[apache-cassandra-2.2.3.jar:2.2.3]
at org.apache.cassandra.streaming.compress.CompressedStreamReader.read(CompressedStreamReader.java:78) ~[apache-cassandra-2.2.3.jar:2.2.3]
at org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:49) ~[apache-cassandra-2.2.3.jar:2.2.3]
at org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:38) ~[apache-cassandra-2.2.3.jar:2.2.3]
at org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:58) ~[apache-cassandra-2.2.3.jar:2.2.3]
at org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:261) ~[apache-cassandra-2.2.3.jar:2.2.3]
at java.lang.Thread.run(Unknown Source) [na:1.8.0_60]
Изучив через предыдущие вопросы по этой теме, одна из вещей, которые Я попытался использовать команду ulimit, чтобы установить количество открытых файлов на 1000000. Однако после перезапуска службы cassandra я все равно получаю ошибку.
Есть ли что-нибудь еще, что мне нужно сделать, чтобы эти изменения вступили в силу? Будет ли параметр, который мне нужно установить в файле cassandra.yaml, чтобы это не произошло? Большое спасибо.