0

我有一个从 redis 中提取的字节数组。

r.set('a', '')
r.setbit('a', 0, 1)
r.setbit('a', 1, 1)
r.setbit('a', 12, 1)

a_raw = db.get('a')
# b'\xc0\x08'
a_bin = bin(int.from_bytes(a, byteorder="big")) 
# 0b1100000000001000

我想使用该字节数组从 ndarray 中选择行。

arr = np.arange(12)
arr[a_raw]
# array([0, 1, 12])

编辑两种解决方案都有效，但我发现@paul-panzer 更快

import timeit

setup = '''import numpy as np; a = b'\\xc0\\x08'; '''

t1 = timeit.timeit('idx = np.unpackbits(np.frombuffer(a, np.uint8)); np.where(idx)', 
              setup = setup, number=10000)

t2 = timeit.timeit('idx = np.array(list(bin(int.from_bytes(a, byteorder="big"))[2:])) == "1"; np.where(idx)',
              setup = setup, number=10000)

print(t1, t2)
#0.019560601096600294 0.054518797900527716

编辑 2实际上，该from_bytes方法不会返回我正在寻找的内容：

redis_db.delete('timeit_test')
redis_db.setbit('timeit_test', 12666, 1)
redis_db.setbit('timeit_test', 14379, 1)
by = redis_db.get('timeit_test')

idx = np.unpackbits(np.frombuffer(by, np.uint8))
indices = np.where(idx)

idx = np.array(list(bin(int.from_bytes(by, byteorder="big"))[2:])) == "1"
indices_2 = np.where(idx)

print(indices, indices_2)
#(array([12666, 14379]),) (array([   1, 1714]),)
4

2 回答 2

1

这是一种使用方法unpackbits：

>>> a = b'\xc0\x08'
>>> b = np.arange(32).reshape(16, 2)
>>> c = np.arange(40).reshape(20, 2)
>>> 
>>> idx = np.unpackbits(np.frombuffer(a, np.uint8))
>>> 
# if the sizes match boolen indexing can be used
>>> b[idx.view(bool)]
array([[ 0,  1],
       [ 2,  3],
       [24, 25]])
>>> 
# non matching sizes can be worked around using where
>>> c[np.where(idx)]
array([[ 0,  1],
       [ 2,  3],
       [24, 25]])
>>>
于 2018-03-29T02:08:00.493 回答
0

这是一种方法：

In [57]: b = 0b1100000000001000

In [58]: mask = np.array(list(bin(b)[2:])) == '1'

In [59]: arr = np.arange(13)

In [60]: arr[mask[:len(arr)]]
Out[60]: array([ 0,  1, 12])

此外，这是一个简单的检查，以证明 for 的__getitem__实现ndarray不支持直接在bytes对象上建立索引：

In [61]: by = b'\xc0\x08'

In [62]: arr[by]
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-111-6cd68003b176> in <module>()
----> 1 arr[by]

IndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis (`None`) 
and integer or boolean arrays are valid indices

因此，除非您子类ndarray化或创建具有自定义__getitem__行为的扩展模块，否则无法直接从bytes.

bytes下面是一个示例，比较了直接从原始对象工作的几种不同方法的时间：

In [171]: %timeit np.array(list(bin(int.from_bytes(by, byteorder='big'))[2:])) == '1'
3.51 µs ± 38 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

In [172]: %timeit np.unpackbits(np.frombuffer(by, np.uint8))
2.05 µs ± 29.59 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

In [173]: %timeit np.array(list(bin(struct.unpack('>H', by)[0])[2:])) == '1'
2.65 µs ± 6.8 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
于 2018-03-28T23:06:19.163 回答