python - 无论参数如何，scipy.interpolate.UnivariateSpline 都不平滑

Question

我无法让 scipy.interpolate.UnivariateSpline 在插值时使用任何平滑。根据函数的页面以及之前的一些帖子，我相信它应该提供s参数平滑。

这是我的代码：

# Imports
import scipy
import pylab

# Set up and plot actual data
x = [0, 5024.2059124920379, 7933.1645067836089, 7990.4664106277542, 9879.9717114947653, 13738.60563208926, 15113.277958924193]
y = [0.0, 3072.5653360000988, 5477.2689107965398, 5851.6866463790966, 6056.3852496014106, 7895.2332350173638, 9154.2956175610598]
pylab.plot(x, y, "o", label="Actual")

# Plot estimates using splines with a range of degrees
for k in range(1, 4):
    mySpline = scipy.interpolate.UnivariateSpline(x=x, y=y, k=k, s=2)
    xi = range(0, 15100, 20)
    yi = mySpline(xi)
    pylab.plot(xi, yi, label="Predicted k=%d" % k)

# Show the plot
pylab.grid(True)
pylab.xticks(rotation=45)
pylab.legend( loc="lower right" )
pylab.show()

结果如下：

我已经尝试过使用一系列s值（0.01、0.1、1、2、5、50）以及显式权重，设置为相同的值（1.0）或随机设置。我仍然无法进行任何平滑处理，并且结的数量始终与数据点的数量相同。特别是，我正在寻找像第 4 点（7990.4664106277542、5851.6866463790966）这样的异常值来平滑处理。

是因为我没有足够的数据吗？如果是这样，我可以应用类似的样条函数或聚类技术来实现对这几个数据点的平滑吗？

Answer 1

简短的回答：您需要s更仔细地选择值。

UnivariateSpline的文档指出：

Positive smoothing factor used to choose the number of knots. Number of 
knots will be increased until the     smoothing condition is satisfied:
sum((w[i]*(y[i]-s(x[i])))**2,axis=0) <= s

从这个可以推断出平滑的“合理”值，如果你不传递明确的权重，那么数据点的数量和数据的方差在s = m * v哪里。在这种情况下，。mvs_good ~ 5e7

编辑： 的合理值s当然也取决于数据中的噪声水平。文档似乎建议在与要平滑的“噪声”相关的标准偏差s范围内(m - sqrt(2*m)) * std**2 <= s <= (m + sqrt(2*m)) * std**2进行选择。std

Answer 2

@Zhenya 在数据点之间手动设置节点的答案过于粗糙，无法在噪声数据中提供良好的结果，而无需选择如何应用该技术。然而，在他/她的建议的启发下，我在 scikit-learn 包中的Mean-Shift 聚类方面取得了成功。它执行集群计数的自动确定，并且似乎做了相当好的平滑工作（实际上非​​常平滑）。

# Imports
import numpy
import pylab
import scipy
import sklearn.cluster

# Set up original data - note that it's monotonically increasing by X value!
data = {}
data['original'] = {}
data['original']['x'] = [0, 5024.2059124920379, 7933.1645067836089, 7990.4664106277542, 9879.9717114947653, 13738.60563208926, 15113.277958924193]
data['original']['y'] = [0.0, 3072.5653360000988, 5477.2689107965398, 5851.6866463790966, 6056.3852496014106, 7895.2332350173638, 9154.2956175610598]

# Cluster data, sort it and and save
inputNumpy = numpy.array([[data['original']['x'][i], data['original']['y'][i]] for i in range(0, len(data['original']['x']))])
meanShift = sklearn.cluster.MeanShift()
meanShift.fit(inputNumpy)
clusteredData = [[pair[0], pair[1]] for pair in meanShift.cluster_centers_]
clusteredData.sort(lambda pair1, pair2: cmp(pair1[0],pair2[0]))
data['clustered'] = {}
data['clustered']['x'] = [pair[0] for pair in clusteredData]
data['clustered']['y'] = [pair[1] for pair in clusteredData]

# Build a spline using the clustered data and predict
mySpline = scipy.interpolate.UnivariateSpline(x=data['clustered']['x'], y=data['clustered']['y'], k=1)
xi = range(0, round(max(data['original']['x']), -3) + 3000, 20)
yi = mySpline(xi)

# Plot the datapoints
pylab.plot(data['clustered']['x'], data['clustered']['y'], "D", label="Datapoints (%s)" % 'clustered')
pylab.plot(xi, yi, label="Predicted (%s)" %  'clustered')
pylab.plot(data['original']['x'], data['original']['y'], "o", label="Datapoints (%s)" % 'original')

# Show the plot
pylab.grid(True)
pylab.xticks(rotation=45)
pylab.legend( loc="lower right" )
pylab.show()

Answer 3

虽然我不知道有任何库会为你做这件事，但我会尝试更多的 DIY 方法：我会从在原始数据点之间制作一个带有结的样条线开始，在x和y. 在您的特定示例中，在第 4 点和第 5 点之间有一个结应该可以解决问题，因为它会消除在 around 处的巨大导数x=8000。

Answer 4

我无法让 BigChef 的答案运行，这是适用于 python 3.6 的变体：

# Imports
import pylab
import scipy
import sklearn.cluster

# Set up original data - note that it's monotonically increasing by X value!
data = {}
data['original'] = {}
data['original']['x'] = [0, 5024.2059124920379, 7933.1645067836089, 7990.4664106277542, 9879.9717114947653, 13738.60563208926, 15113.277958924193]
data['original']['y'] = [0.0, 3072.5653360000988, 5477.2689107965398, 5851.6866463790966, 6056.3852496014106, 7895.2332350173638, 9154.2956175610598]

# Cluster data, sort it and and save
import numpy
inputNumpy = numpy.array([[data['original']['x'][i], data['original']['y'][i]] for i in range(0, len(data['original']['x']))])
meanShift = sklearn.cluster.MeanShift()
meanShift.fit(inputNumpy)
clusteredData = [[pair[0], pair[1]] for pair in meanShift.cluster_centers_]

clusteredData.sort(key=lambda li: li[0])
data['clustered'] = {}
data['clustered']['x'] = [pair[0] for pair in clusteredData]
data['clustered']['y'] = [pair[1] for pair in clusteredData]

# Build a spline using the clustered data and predict
mySpline = scipy.interpolate.UnivariateSpline(x=data['clustered']['x'], y=data['clustered']['y'], k=1)
xi = range(0, int(round(max(data['original']['x']), -3)) + 3000, 20)
yi = mySpline(xi)

# Plot the datapoints
pylab.plot(data['clustered']['x'], data['clustered']['y'], "D", label="Datapoints (%s)" % 'clustered')
pylab.plot(xi, yi, label="Predicted (%s)" %  'clustered')
pylab.plot(data['original']['x'], data['original']['y'], "o", label="Datapoints (%s)" % 'original')

# Show the plot
pylab.grid(True)
pylab.xticks(rotation=45)
pylab.show()