10

First the context, im trying to use Redis as an in-memory store backed with persistence. I need to store large number of objects (millions) in a Redis Hash.

At the same time, i don't want my redis instance to consume too much memory. So i've set the maxmemory property in redis.conf to 100mb. I've set maxmemory-policy as allkeys-random The persistece mode is AOF and fysnc is every second.

Now the issue iam facing is, every time i try to store more than two hundred thousand ojects in the hash, the hash gets reset (ie all the existing key values in the hash vanishes ). I confirm this by using the hlen command on the hash in the redis-cli.

Find below the object im trying to store

public class Employee implements Serializable {

private static final long serialVersionUID = 1L;
int id;
String name;
String department;
String address;

    /* Getters and Setters */

    /* Hashcode - Generates hashcode (key) for each object */
public int hashCode() {
    final int prime = 31;
    int result = 1;
    result = prime * result + ((address == null) ? 0 : address.hashCode());
    result = prime * result + ((department == null) ? 0 : department.hashCode());
    result = prime * result + id;
    result = prime * result + ((name == null) ? 0 : name.hashCode());
    return result;
}
}

Also, Find below the code that stores into redis (Im using Jedis to interact with Redis )

    JedisPool jedisPool = new JedisPool(new JedisPoolConfig(), "localhost");
    Jedis jedis = (Jedis) jedisPool.getResource();

    System.out.println("Starting....");
    for(int i=0;i<1000000;i++) {

             /* Converting object to byte array */
        Employee employee = new Employee(i, "Arun Jolly", "IT", "SomeCompany");
        ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
        ObjectOutputStream objectOutputStream = new ObjectOutputStream(byteArrayOutputStream);
        objectOutputStream.writeObject(employee);
        byte[] value = byteArrayOutputStream.toByteArray();

        /* Creating key in byte array format using hashCode() */
        ByteBuffer buffer = ByteBuffer.allocate(128);
        buffer.putInt(employee.hashCode());
        byte[] field = buffer.array();

            /* Specyfying the Redis Hash in byte array format */ 
        String tableName = "Employee_Details";
        byte[] key = tableName.getBytes();

        jedis.hset(key, field, value);
        System.out.println("Stored Employee "+i);
    }

Am i missing something ?

Does this mean that redis does not swap out to the disk once the maxmemory is reached ( Is it trying to hold all the key values in memory ? ) Does it mean that i have to incrementally increase the maxmemory limit according to the increase in the number of key-value pairs that i might have to store ?

4

1 回答 1

12

我错过了什么吗？

是的。Redis 是一个纯内存存储，具有持久性选项。一切都必须适合记忆。

这是否意味着一旦达到 maxmemory，redis 就不会换出到磁盘。

恰恰。

它是否试图将所有关键值保存在内存中？

键和值，是的。

这是否意味着我必须根据我可能必须存储的键值对数量的增加逐步增加最大内存限制？

您需要预先决定分配给 Redis 的内存量，是的。

如果您的内存受限，那么使用基于磁盘的存储会更好。

于 2013-07-23T12:30:57.947 回答