i am using following code for compression
Configuration conf = new Configuration();
conf.setBoolean("mapred.compress.map.output", true);
conf.set("mapred.map.output.compression.codec","org.apache.hadoop.io.compress.SnappyCodec");
using snappy algorithm. but while compressing an input file with some mb of data ( 70 to 100 mb) it compresses data with file size which is more than the input file and if i try it with input directory containing all types of file like(.jpg, .mp3, .mp4 etc..) with size 100 to 150 mb, it displays error as :
log4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.lib.MutableMetricsFactory).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Java HotSpot(TM) Server VM warning: INFO: os::commit_memory(0x930c0000, 105119744, 0) failed; error='Cannot allocate memory' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 105119744 bytes for committing reserved memory.
# An error report file with more information is saved as:
# /home/hduser/workspace/TestProject/hs_err_pid16619.log
As i am trying to compress and decompress data using snappy algorithm, please here suggest me , how do i compress data using sanppy algorithm with less space.
i am using
Ubuntu 13.10, 32 bit Jdk 7 32 bit. with hadoop-2.2.0