(自 2015 年 9 月 4 日起更正以包括可重复的基准和结论)
当然values.clear()
比创建新对象更快(只需将最后一项索引设置为零)。
几乎可以肯定 avalues.clear()
会比创建一个新对象更快。在ArrayList
您最初使用的情况下,它只会将插入索引设置为零。
正如我在 PD#1 中评论的那样,对于这种元素是整数的情况, BitSet可能是最快的方法(假设值的范围不是太宽。但这可能对任何其他类型的元素都没有用。
还正如所说 正如我与 Assylias 的回答不谋而合, HashSet 是比 ArrayList 更好的选择(假设hashCode()
提供了一个不错的分布,不会导致我们达到 O(N) 性能)。
在这种HashSet
情况下,直觉还表明clear()
(基本上将HashSet#table
“鸽洞”数组设置为空)比构建一个全新的集合(在任何情况下都需要将同一个表初始化/重置为零)更快。但在这种特殊情况下,事情会反过来发生。Assylias 发表了他的研究结果。不幸的是,我不得不自己编写我的基准测试代码,以了解这怎么会发生。我在 PD#3 中讨论了这个问题
无论如何,主要的事情是,因为为每次迭代创建一个全新的 HashSet 并没有实质性的损失,这样做是有意义的(因为它更简单),除非我们必须更加关心性能和资源。
关于性能的另一个问题是I/O。System.out.println()
示例代码中的那一行可能会自动flush()
将瓶颈转移到 console/stdout。一种解决方法可能是添加到StringBuffer
. 除非有一个阅读器进程热切地等待该输出,否则将写入延迟到循环结束可能是有意义的。
这将是我的尝试:
Set<Integer> values = new HashSet<Integer>();
// (PD 1) Or new BitSet(max_x - min_x + 1);
// (PD 2) Or new HashSet((int)Math.ceil(n/0.75));
StringBuffer sb = new StringBuffer(); // appends row values for printing together.
for (int j = 0, x; j < m; j++) {
values.clear();
sb.setLength(0);
for (int i = 0; i < n; i++) {
x = X[i][j];
if (! values.contains(x)){
sb.append(x+"\n");
values.add(x);
}
}
System.out.print(sb);
}
PD1。另外,如果您可能考虑使用BitSet
. 它具有O(1)访问性能(即使在最坏的情况下,因为没有冲突)。它最适合范围从 0 开始的整数(否则可能需要转换)和在可能分布中足够密集的实际值群体。
- 例如,如果您检查 Unicode 代码点的出现,您将需要一个 139,264 字节长的数组 (17 (planes) * 2 16 (codepoints/plane) / 8),其中可能您在 100 个字符长中仅使用 40 个不同的字符短信,可能有点过头了。但是,如果您仅限于 ISO-Latin-1 中的 256 个可能值。(8 字节位集),这实际上是一个完美的选择。
PD2。此外,正如 Assylias 所说,为 HashSet 设置初始大小可能会有所帮助。作为threshold = (int)(capacity * loadFactor)
,您可能希望initialCapacity=(int)Math.ceil(n/0.75)
确保没有调整大小。 这个问题属于 Assylias 的帖子(我没有为自己使用),以这种方式讨论是不合适的
PD3(2015 年 9 月:3 年后)我碰巧重新审视了这个问题,我对 Assylas 结果非常感兴趣,我编写了自己的微基准(我包括在内,因此任何人都可以复制)。这些是我的结论:
- 我提出的
BitSet
(注意:不适合非整数和非常稀疏的分布)明显优于 HashSet 的所有风格(在密集分布中大约快 4 倍)
- 对大小为 1000的高度填充集的测试显示有利于创建新集合(7.7" vs 9.8")的轻微优势。但是,vs的“试运行”会产生相反的结果(9.5" vs 7.5")。我的猜测是,这是因为重置时缓存无效的惩罚(设置它不是)。
HashSet#clear()
new HashSet()
HashSet.table
null
null
- 事先知道最佳尺寸也是一个很大的优势(这可能并不总是可行)。该
HashSet.clear()
方法更具适应性,并且可以更好地低估大小。高估不会意味着太大的差异,但如果内存是一个问题,则可能不是一个好策略。
- 结果清楚地表明,如今创建对象和分配内存并不是一个大问题(参见Programmers.SE)。但是,重用对象仍然应该是一个需要考虑的选项。参见drmirror中的示例, 即使在 JDK 1.6 演进之后,重用实例 (CharBuffer) 也会使性能翻倍。
- 我还想知道 Assylias 使用 a 的影响是什么
loadFactor==1.0f
(HashSet 不会调整大小直到size > table.length*loadFactor
,这与我建议他的不同,但如果没有冲突就完美了)。大约loadFactor==0.75f
需要 1.33 倍的表空间以换取避免 25% 的冲突。我的测试没有显示这种情况下默认设置的任何优势。
这是我用于测试的课程。抱歉,如果它可能在某些方面过冲而在其他方面缺乏(没有热身,只需执行足够长的时间,以便实现有机会窒息他自己的垃圾)。
/**
* Messing around this StackOverflow question: https://stackoverflow.com/questions/11740013/fastest-way-to-recreate-the-arraylist-in-a-for-loop/ .
* Quite surprisingly new HashSet() (which should imply actual memory initialization) is faster than HashSet.clear() in the given scenario.
* Primary goal is to test this phenomenon (new vs clear) under different scenarios.
* Secondarily a bit about the BitSet and the HashSet loadFactor is tested.
* @author Javier
*/
public class TestSetClear2 {
public static interface MicroBenchmark {
public String getName();
/**
*
* @param dataSet Data set to insert in the collection
* @param initialSize Initial size for the collection. Can try to be optimal or try to fool.
* @param iterations Number of times to go through the dataSet over and over
*/
public void run(int[] dataSet, int initialSize, int iterations);
}
/** Bad initial case. Based in question code */
public static class MBList implements MicroBenchmark {
@Override public String getName() { return "ArrayList.clear()"; }
@Override public void run(int[] data, int initialSize, int n) {
// Not taking initial size into account may result in a resizing penalty in the first iteration
// But will have an adequate size in following iterations, and wont be fooled by bad estimations.
List<Integer> values = new ArrayList<Integer>();
for (int iter = 0; iter < n; iter++) {
values.clear();
for (int i = 0; i < data.length; i++) {
int x = data[i];
if (values.contains(x)) continue;
values.add(x);
}
}
}
}
/** new HashSet(N,1) for every iteration. Reported as best by assylias. */
public static class MBNewHashSetN1 implements MicroBenchmark {
@Override public String getName() { return "new HashSet(N,1)"; }
@Override public void run(int[] data, int initialSize, int n) {
for (int iter = 0; iter < n; iter++) {
Set<Integer> values = new HashSet<>(initialSize, 1.0f); // 1.0 loadfactor optimal if no collisions.
for (int i = 0; i < data.length; i++) {
int x = data[i];
if (values.contains(x)) continue;
values.add(x);
}
}
}
}
// No need to implement raw new HashSet() (reported as worse). Will be enough fooling to initialize to 16 so it succumbs to resizing.
/** HashsetClear for every iteration. Attempted by Assylias and Javier. Clear() does not perform as well as expected under basic tests. */
public static class MBHashSetClear implements MicroBenchmark {
private float loadFactor; // Allow loadFactor to check how much 1.0 factor affects if there are collisions.
private String name;
public MBHashSetClear(float loadFactor) {
this.loadFactor = loadFactor;
name = String.format(Locale.ENGLISH, "HashSet(N,%f).clear()", loadFactor);
}
@Override public String getName() { return name; }
@Override public void run(int[] data, int initialSize, int n) {
HashSet<Integer> values = new HashSet<>((int)Math.ceil(initialSize/loadFactor), loadFactor);// Just the size for loadfactor so it wont resize.
for (int iter = 0; iter < n; iter++) {
values.clear();
for (int i = 0; i < data.length; i++) {
int x = data[i];
if (values.contains(x)) continue;
values.add(x);
}
}
}
}
/** Javier BitSet. Might clearly outperform HashSet, but only on the very specific constraints of the test (non negative integers, not hugely big). */
public static class MBBitSet implements MicroBenchmark {
@Override public String getName() { return "BitSet.clear()"; }
@Override public void run(int[] data, int distributionSize, int n) {
BitSet values = new BitSet(distributionSize);
for (int iter = 0; iter < n; iter++) {
values.clear();
for (int i = 0; i < data.length; i++) {
int x = data[i];
if (values.get(x)) continue;
values.set(x);
}
}
}
}
public static void main(String[] args) {
final MicroBenchmark mbNew = new MBNewHashSetN1();
// Create with same loadFactor as MBNewHashSetN1. So we compare apples with apples (same size of handled table, same collisions).
final MicroBenchmark mbClear = new MBHashSetClear(1.0f);
final MicroBenchmark mbClear075 = new MBHashSetClear(0.75f);
final MicroBenchmark mbBitset = new MBBitSet();
final MicroBenchmark mbList = new MBList(); // Will have a taste of O(N) with a not too bit dataset.
// warmup. trigger the cpu high performance mode? Fill the heap with garbage?
//mbNew.run(dataSetE3xE3, 1000, (int)1e5); // Using new HS might give a bit advantage?
int timePerTest = 10000;
int distributionSize, initialCapacity, datasetLength;
// 1000 long and values 0..999 (1e3 x 1e3). Optimal initial capacity
distributionSize = 1000; datasetLength = 1000; initialCapacity = 1000;
final int[] dataSetE3xE3 = generateRandomSet(1000,1000);
runBenchmark("E3xE3", dataSetE3xE3, distributionSize, timePerTest, initialCapacity,
mbNew, mbClear, mbClear075, mbBitset);
// repeat with underestimated initial size. Will incur in resizing penalty
initialCapacity = 16; // Default initial
runBenchmark("E3xE3+underSize", dataSetE3xE3, distributionSize, timePerTest, initialCapacity,
mbNew, mbClear, mbBitset);
// repeat with overestimated initial size. larger garbage and clearing.
initialCapacity = 100000; // oversized will force to handle large tables filled with 0 / null.
runBenchmark("E3xE3+overSize", dataSetE3xE3, distributionSize, timePerTest, initialCapacity,
mbNew, mbClear, mbBitset);
// Dry run (not rum). what if we focus on the new and clear operations. Just 1 item so clear() is forced to traverse the table.
datasetLength = 1; distributionSize = 1000; initialCapacity = 1000;
runBenchmark("E3xE3-DryRun", generateRandomSet(datasetLength, distributionSize),
distributionSize, timePerTest, initialCapacity,
mbNew, mbClear);
// check for * 100 and / 100 sizes.
distributionSize = datasetLength = initialCapacity = 10;
runBenchmark("E1xE1",
generateRandomSet(datasetLength, distributionSize),
distributionSize, timePerTest, initialCapacity,
mbNew, mbClear, mbList);
distributionSize = datasetLength = initialCapacity = 100000;
runBenchmark("E5xE5", generateRandomSet(datasetLength, distributionSize),
distributionSize, timePerTest, initialCapacity,
mbNew, mbClear);
// Concentrated distributions might behave as with oversized?
datasetLength=10000; distributionSize=10; initialCapacity=Math.min(datasetLength, distributionSize);
runBenchmark("E4xE1",
generateRandomSet(datasetLength, distributionSize),
distributionSize, timePerTest, initialCapacity,
mbNew, mbClear);
// Sparse distributions might allow mild collision. Also adverse for BitSet.
// TODO Generate a greater/known amount of collisions
datasetLength=10000; distributionSize=(int)1e6; initialCapacity=Math.min(datasetLength, distributionSize);
runBenchmark("E4xE6",
generateRandomSet(datasetLength, distributionSize),
distributionSize, timePerTest, initialCapacity,
mbNew, mbClear, mbClear075);
}
private static void runBenchmark(String testName, int[] dataSet, int distributionSize, int timePerTest
, int initialCapacity, MicroBenchmark ... testees /* not testes */) {
// How many iterations? Will use first testee to callibrate.
MicroBenchmark curTest = testees[0];
long t0 = System.nanoTime();
long ellapsed = 0L;
final long minToCallibrate = (long)0.5e9; // half second
int iterations = 1;
while (ellapsed < minToCallibrate) {
curTest.run(dataSet, initialCapacity, iterations);
iterations *= 2; // same as <<= 1
ellapsed = System.nanoTime() - t0;
}
// calculation is not laser-sharp precise (actually executed iterations -1, and some extra initializations).
final int nIterations = (int) ((double)iterations * timePerTest * 1e6 /* nanos/millis */ / ellapsed);
// Do actual benchmark
System.out.printf(Locale.ENGLISH, "dataset:{name=%s,length:%d,distrib:%d,capacity0:%d,iterations:%d}\n",
testName, dataSet.length, distributionSize, initialCapacity, nIterations);
for (MicroBenchmark testee : testees) {
t0 = System.nanoTime();
testee.run(dataSet, initialCapacity, nIterations);
ellapsed = System.nanoTime() - t0;
System.out.printf(Locale.ENGLISH, "%s : %5.3f\n", testee.getName(), ellapsed/1e9 );
}
}
private static int[] generateRandomSet(int lengthOfSet, int distributionSize) {
Random r = new Random();
int[] result = new int[lengthOfSet];
for (int i = 0; i < lengthOfSet; i++) {
result[i] = r.nextInt(distributionSize);
}
return result;
}
}
这是我的结果(使用 JDK 1.8.0_31 - 64 bits - Windows 7 )
dataset:{name=E3xE3,length:1000,distrib:1000,capacity0:1000,iterations:514241}
new HashSet(N,1) : 7.688
HashSet(N,1.000000).clear() : 9.796
HashSet(N,0.750000).clear() : 9.923
BitSet.clear() : 1.990
dataset:{name=E3xE3+underSize,length:1000,distrib:1000,capacity0:16,iterations:420572}
new HashSet(N,1) : 9.735
HashSet(N,1.000000).clear() : 6.637
BitSet.clear() : 1.611
dataset:{name=E3xE3+overSize,length:1000,distrib:1000,capacity0:100000,iterations:143032}
new HashSet(N,1) : 9.948
HashSet(N,1.000000).clear() : 10.377
BitSet.clear() : 0.447
dataset:{name=E3xE3-DryRun,length:1,distrib:1000,capacity0:1000,iterations:18511486}
new HashSet(N,1) : 9.583
HashSet(N,1.000000).clear() : 7.523
dataset:{name=E1xE1,length:10,distrib:10,capacity0:10,iterations:76177852}
new HashSet(N,1) : 9.988
HashSet(N,1.000000).clear() : 10.521
ArrayList.clear() : 7.915
dataset:{name=E5xE5,length:100000,distrib:100000,capacity0:100000,iterations:2892}
new HashSet(N,1) : 9.764
HashSet(N,1.000000).clear() : 9.615
dataset:{name=E4xE1,length:10000,distrib:10,capacity0:10,iterations:170386}
new HashSet(N,1) : 9.843
HashSet(N,1.000000).clear() : 9.708
dataset:{name=E4xE6,length:10000,distrib:1000000,capacity0:10000,iterations:36497}
new HashSet(N,1) : 9.686
HashSet(N,1.000000).clear() : 10.079
HashSet(N,0.750000).clear() : 10.008