python - 如何从 Python 脚本中捕获 Curl 的输出

Question

我想使用 curl 查找有关网页的信息，但是在 Python 中，到目前为止我有这个：

os.system("curl --head www.google.com")

如果我运行它，它会打印出：

HTTP/1.1 200 OK
Date: Sun, 15 Apr 2012 00:50:13 GMT
Expires: -1
Cache-Control: private, max-age=0
Content-Type: text/html; charset=ISO-8859-1
Set-Cookie: PREF=ID=3e39ad65c9fa03f3:FF=0:TM=1334451013:LM=1334451013:S=IyFnmKZh0Ck4xfJ4; expires=Tue, 15-Apr-2014 00:50:13 GMT; path=/; domain=.google.com
Set-Cookie: NID=58=Giz8e5-6p4cDNmx9j9QLwCbqhRksc907LDDO6WYeeV-hRbugTLTLvyjswf6Vk1xd6FPAGi8VOPaJVXm14TBm-0Seu1_331zS6gPHfFp4u4rRkXtSR9Un0hg-smEqByZO; expires=Mon, 15-Oct-2012 00:50:13 GMT; path=/; domain=.google.com; HttpOnly
P3P: CP="This is not a P3P policy! See http://www.google.com/support/accounts/bin/answer.py?hl=en&answer=151657 for more info."
Server: gws
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
Transfer-Encoding: chunked

我想要做的是能够使用正则表达式匹配其中的 200（我不需要帮助），但是，我找不到将上面的所有文本转换为字符串的方法。我怎么做？我试过了：info = os.system("curl --head www.google.com")但info只是0.

Answer 1

出于某种原因...我需要使用 curl （没有 pycurl，httplib2 ...），也许这可以帮助某人：

import os
result = os.popen("curl http://google.es").read()
print result

Answer 2

试试这个，使用subprocess.Popen()：

import subprocess
proc = subprocess.Popen(["curl", "--head", "www.google.com"], stdout=subprocess.PIPE)
(out, err) = proc.communicate()
print out

如文档中所述：

subprocess 模块允许您生成新进程，连接到它们的输入/输出/错误管道，并获取它们的返回码。该模块旨在替换其他几个较旧的模块和功能，例如：

os.system
os.spawn*
os.popen*
popen2.*
commands.*

Answer 3

import os
cmd = 'curl https://randomuser.me/api/'
os.system(cmd)

---

结果

{"results":[{"gender":"male","name":{"title":"mr","first":"çetin","last":"nebioğlu"},"location":{"street":"5919 abanoz sk","city":"adana","state":"kayseri","postcode":53537},"email":"çetin.nebioğlu@example.com","login":{"username":"heavyleopard188","password":"forgot","salt":"91TJOXWX","md5":"2b1124732ed2716af7d87ff3b140d178","sha1":"cb13fddef0e2ce14fa08a1731b66f5a603e32abe","sha256":"cbc252db886cc20e13f1fe000af1762be9f05e4f6372c289f993b89f1013a68c"},"dob":"1977-05-10 18:26:56","registered":"2009-09-08 15:57:32","phone":"(518)-816-4122","cell":"(605)-165-1900","id":{"name":"","value":null},"picture":{"large":"https://randomuser.me/api/portraits/men/38.jpg","medium":"https://randomuser.me/api/portraits/med/men/38.jpg","thumbnail":"https://randomuser.me/api/portraits/thumb/men/38.jpg"},"nat":"TR"}],"info":{"seed":"0b38b702ef718e83","results":1,"page":1,"version":"1.1"}}

Answer 4

好吧，有一种更容易阅读但更麻烦的方法。这里是：

import os
outfile=''  #put your file path there
os.system("curl --head www.google.com>>{x}".format(x=str(outfile))  #Outputs command to log file (and creates it if it doesnt exist).
readOut=open("{z}".format(z=str(outfile),"r")  #Opens file in reading mode.
for line in readOut:
    print line  #Prints lines in file
readOut.close()  #Closes file
os.system("del {c}".format(c=str(outfile))  #This is optional, as it just deletes the log file after use.

这应该可以满足您的需求。:)

Answer 5

您可以在 Python 中使用 HTTP 库或 http 客户端库，而不是调用 curl 命令。事实上，您可以安装一个 curl 库（只要您的操作系统上有编译器）。

其他选择是 httplib2（推荐），它是一个相当完整的 http 协议客户端，也支持缓存，或者只是普通的 httplib 或名为 Request 的库。

如果您真的真的只想运行 curl 命令并捕获其输出，那么您可以在此处记录的内置子进程模块中使用 Popen 执行此操作：http: //docs.python.org/library/subprocess.html

Answer 6

尝试这个：

import httplib
conn = httplib.HTTPConnection("www.python.org")
conn.request("GET", "/index.html")
r1 = conn.getresponse()
print r1.status, r1.reason