2

您好正在使用 statsmodel 运行以下模型，它工作正常。

from statsmodels.formula.api import ols
from statsmodels.iolib.summary2 import summary_col #for summary stats of large tables
time_FE_str = ' + C(hour_of_day) + C(day_of_week) + C(week_of_year)'
weather_2_str = ' +  C(weather_index) + rain + extreme_temperature + wind_speed'
model = ols("activity_count ~ C(city_id)"+weather_2_str+time_FE_str, data=df)
results = model.fit()
print summary_col(results).tables

print 'F-TEST:'
hypotheses = '(C(weather_index) = 0), (rain=0), (extreme_temperature=0), (wind_speed=0)'
f_test = results.f_test(hypotheses)

但是，如果我想包含分类变量，我不知道如何为 F 检验制定假设C(weather_index)。我为我尝试了所有可以想象的版本，但总是出错。

以前有人遇到过这个问题吗？

有任何想法吗？

F-TEST:
Traceback (most recent call last):
  File "C:/VK/scripts_python/predict_activity.py", line 95, in <module>
    f_test = results.f_test(hypotheses)
  File "C:\Users\Niko\Anaconda2\envs\gl-env\lib\site-packages\statsmodels\base\model.py", line 1375, in f_test
    invcov=invcov, use_f=True)
  File "C:\Users\Niko\Anaconda2\envs\gl-env\lib\site-packages\statsmodels\base\model.py", line 1437, in wald_test
    LC = DesignInfo(names).linear_constraint(r_matrix)
  File "C:\Users\Niko\Anaconda2\envs\gl-env\lib\site-packages\patsy\design_info.py", line 536, in linear_constraint
    return linear_constraint(constraint_likes, self.column_names)
  File "C:\Users\Niko\Anaconda2\envs\gl-env\lib\site-packages\patsy\constraint.py", line 391, in linear_constraint
    tree = parse_constraint(code, variable_names)
  File "C:\Users\Niko\Anaconda2\envs\gl-env\lib\site-packages\patsy\constraint.py", line 225, in parse_constraint
    return infix_parse(_tokenize_constraint(string, variable_names),
  File "C:\Users\Niko\Anaconda2\envs\gl-env\lib\site-packages\patsy\constraint.py", line 184, in _tokenize_constraint
    Origin(string, offset, offset + 1))
patsy.PatsyError: unrecognized token in constraint
   (C(weather_index) = 0), (rain=0), (extreme_temperature=0), (wind_speed=0)
    ^
4

2 回答 2

2

方法 t_test、wald_test 和 f_test 用于直接对参数进行假设检验，而不是用于整个分类或复合效应。

Results.summary() 显示了 patsy 为分类变量创建的参数名称。这些可用于为分类效果创建对比或限制。

作为替代 anova_lm 直接计算一个术语，例如一个分类变量没有影响的假设检验。

于 2018-01-04T20:53:54.170 回答
1

省略C()！

我尝试对这些数据进行分析。

Area    Clover_yield    Yarrow_stems
A   19.0    220
A   76.7    20
A   11.4    510
A   25.1    40
A   32.2    120
A   19.5    300
A   89.9    60
A   38.8    10
A   45.3    70
A   39.7    290
B   16.5    460
B   1.8 320
B   82.4    0
B   54.2    80
B   27.4    0
B   25.8    450
B   69.3    30
B   28.7    250
B   52.6    20
B   34.5    100
C   49.7    0
C   23.3    220
C   38.9    160
C   79.4    0
C   53.2    120
C   30.1    150
C   4.0 450
C   20.7    240
C   29.8    250
C   68.5    0

当我在代码中显示的第一次调用中使用线性模型时，ols我最终遇到了您遇到的障碍。但是，当我省略了Area假设离散水平这一事实时，我能够计算对比的 F 检验。

import pandas as pd
from statsmodels.formula.api import ols

df = pd.read_csv('clover.csv', sep='\s+')
model = ols('Clover_yield ~ C(Area) + Yarrow_stems', data=df)
model = ols('Clover_yield ~ Area + Yarrow_stems', data=df)

results = model.fit()
print (results.summary())

print (results.f_test(['Area[T.B] = Area[T.C], Yarrow_stems=150']))

这是输出。

请注意，在我们的例子中，摘要表示可用于制定因子对比​​的名称Area[T.B]和Area[T.C]。

                            OLS Regression Results                            
==============================================================================
Dep. Variable:           Clover_yield   R-squared:                       0.529
Model:                            OLS   Adj. R-squared:                  0.474
Method:                 Least Squares   F-statistic:                     9.726
Date:                Thu, 04 Jan 2018   Prob (F-statistic):           0.000177
Time:                        17:26:03   Log-Likelihood:                -125.61
No. Observations:                  30   AIC:                             259.2
Df Residuals:                      26   BIC:                             264.8
Df Model:                           3                                         
Covariance Type:            nonrobust                                         
================================================================================
                   coef    std err          t      P>|t|      [0.025      0.975]
--------------------------------------------------------------------------------
Intercept       57.5772      6.337      9.086      0.000      44.551      70.603
Area[T.B]        0.3205      7.653      0.042      0.967     -15.411      16.052
Area[T.C]       -0.5432      7.653     -0.071      0.944     -16.274      15.187
Yarrow_stems    -0.1086      0.020     -5.401      0.000      -0.150      -0.067
==============================================================================
Omnibus:                        0.459   Durbin-Watson:                   2.312
Prob(Omnibus):                  0.795   Jarque-Bera (JB):                0.449
Skew:                           0.260   Prob(JB):                        0.799
Kurtosis:                       2.702   Cond. No.                         766.
==============================================================================

Warnings:
[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.
<F test: F=array([[ 27873807.59795523]]), p=4.939796675253845e-83, df_denom=26, df_num=2>

正如我的评论中提到的，我不清楚你打算测试什么。

编辑：在 user333700 的评论提示下，我尝试再次运行此代码，使用两个略有不同的语句，model = ols('Clover_yield ~ C(Area) + Yarrow_stems', data=df)并且print (results.f_test(['C(Area)[T.B] = C(Area)[T.C], Yarrow_stems=150'])). 'C(Area)[TB]' 和 'C(Area)[TC]' 来自带有更改模型的摘要。因此，对于这种类型的分析，是否使用 C() 声明并不重要。如摘要中所述，您必须记住对虚拟变量使用适当的形式。

于 2018-01-04T22:36:21.110 回答