测试套件增加后，Selenium 测试开始失败。通常，在达到 10 个通过的测试后测试失败，之后，所有其余测试都失败，没有触发重试的信息。所有这些测试都在本地成功通过，我尝试将它们混合并创建不同的订单批次，但仍然在 CI 中通过了前十个测试，其余的都失败了。这些失败并不是每次代码推送都会发生，它大约每 3 次代码推送就会发生一次。
尝试混合测试，在本地多次运行以确保没有失败。由于 dockerized 环境中的内存问题，我已经更改了 webdriver 实例的范围，现在它在新环境中运行每个测试，没有来自以前运行的测试的数据。
此外，当这些失败刚开始出现时，我还没有在日志中看到它，只有 10 个通过的测试和消息console log output exceeded timeout，所以我添加了一个参数circleci/config.yml
no_output_timeout: 20m
，开始显示其余测试的失败。
platform linux -- Python 3.6.4, pytest-4.3.1, py-1.8.0, pluggy-0.9.0 -- /usr/local/bin/python3
cachedir: .pytest_cache
metadata: {'Python': '3.6.4', 'Platform': 'Linux-4.4.0-96-generic-x86_64-with', 'Packages': {'pytest': '4.3.1', 'py': '1.8.0', 'pluggy': '0.9.0'}, 'Plugins': {'reportportal': '1.0.4', 'parallel': '0.0.9', 'metadata': '1.8.0', 'html': '1.20.0', 'flaky': '3.5.3'}}
rootdir: /home/robot/functional/retail, inifile:
plugins: reportportal-1.0.4, parallel-0.0.9, metadata-1.8.0, html-1.20.0, flaky-3.5.3
collecting ...
collected 36 items
tests/ui_workflows/test_ecom_publish.py::TestEcomPublish::test_create_new_item_and_publish_on_ecom[firefox] SKIPPED
tests/ui_workflows/test_retail_create.py::TestRetailCreate::test_create_new_item[firefox] Using Selenium grid at http://selenium-grid:4444/wd/hub
PASSED
tests/ui_workflows/test_retail_create.py::TestRetailCreate::test_create_new_customer_type[firefox] Using Selenium grid at http://selenium-grid:4444/wd/hub
PASSED
tests/ui_workflows/test_retail_create.py::TestRetailCreate::test_create_new_sales_tax[firefox] Using Selenium grid at http://selenium-grid:4444/wd/hub
PASSED
tests/ui_workflows/test_retail_import.py::TestRetailCreate::test_import_items_from_csv[firefox] Using Selenium grid at http://selenium-grid:4444/wd/hub
PASSED
tests/ui_workflows/test_retail_import.py::TestRetailCreate::test_import_items_from_xlsx[firefox] Using Selenium grid at http://selenium-grid:4444/wd/hub
PASSED
tests/ui_workflows/test_retail_import.py::TestRetailCreate::test_import_300_items_from_xlsx[firefox] Using Selenium grid at http://selenium-grid:4444/wd/hub
PASSED
tests/ui_workflows/test_retail_item.py::TestRetailItem::test_create_item_with_all_attributes[firefox] Using Selenium grid at http://selenium-grid:4444/wd/hub
PASSED
tests/ui_workflows/test_retail_item.py::TestRetailItem::test_create_non_taxable_item_and_make_sale[firefox] Using Selenium grid at http://selenium-grid:4444/wd/hub
PASSED
tests/ui_workflows/test_retail_item.py::TestRetailItem::test_create_non_discountable_item_and_make_sale[firefox] Using Selenium grid at http://selenium-grid:4444/wd/hub
PASSED
tests/ui_workflows/test_retail_item.py::TestRetailItem::test_create_non_discount_non_tax_item_and_make_sale[firefox] Using Selenium grid at http://selenium-grid:4444/wd/hub
PASSED
tests/ui_workflows/test_retail_item.py::TestRetailItem::test_create_item_and_quick_edit_all_attr[firefox] Using Selenium grid at http://selenium-grid:4444/wd/hub
ERROR
tests/ui_workflows/test_retail_item_matrix.py::TestRetailMatrix::test_create_color_size_attributes_matrix[firefox] Using Selenium grid at http://selenium-grid:4444/wd/hub
ERROR
tests/ui_workflows/test_retail_item_matrix.py::TestRetailMatrix::test_add_4th_attr_color_size_attr_matrix[firefox] Using Selenium grid at http://selenium-grid:4444/wd/hub
ERROR
tests/ui_workflows/test_retail_item_matrix.py::TestRetailMatrix::test_create_color_attributes_matrix[firefox] Using Selenium grid at http://selenium-grid:4444/wd/hub
ERROR
tests/ui_workflows/test_retail_item_matrix.py::TestRetailMatrix::test_add_4th_color_attribute_matrix[firefox] Using Selenium grid at http://selenium-grid:4444/wd/hub
ERROR
tests/ui_workflows/test_retail_item_matrix.py::TestRetailMatrix::test_create_size_attributes_matrix[firefox] Using Selenium grid at http://selenium-grid:4444/wd/hub
ERROR
==================================== ERRORS ====================================
_ ERROR at setup of TestRetailItem.test_create_item_and_quick_edit_all_attr[firefox] _
> lambda: ihook(item=item, **kwds),
when=when,
)
/usr/local/lib/python3.6/site-packages/flaky/flaky_pytest_plugin.py:306:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/ui_workflows/conftest.py:89: in driver
driver = webdriver.Remote(command_executor=selenium_grid_endpoint, desired_capabilities=capabilities)
/usr/local/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py:156: in __init__
self.start_session(capabilities, browser_profile)
/usr/local/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py:251: in start_session
response = self.execute(Command.NEW_SESSION, parameters)
/usr/local/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py:320: in execute
self.error_handler.check_response(response)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <selenium.webdriver.remote.errorhandler.ErrorHandler object at 0x7f30b61d3d68>
response = {'status': 500, 'value': '{\n "sessionId": null,\n "status": 13,\n "value": {\n "class": "org.openqa.grid.common...\tat org.openqa.grid.web.servlet.handler.RequestHandler.process(RequestHandler.java:112)\\n\\t... 38 more\\n"\n }\n}'}
def check_response(self, response):
"""
Checks that a JSON response from the WebDriver does not have an error.
:Args:
- response - The JSON response from the WebDriver server as a dictionary
object.
:Raises: If the response contains an error message.
"""
status = response.get('status', None)
if status is None or status == ErrorCode.SUCCESS:
return
value = None
message = response.get("message", "")
screen = response.get("screen", "")
stacktrace = None
if isinstance(status, int):
value_json = response.get('value', None)
if value_json and isinstance(value_json, basestring):
import json
try:
value = json.loads(value_json)
if len(value.keys()) == 1:
value = value['value']
status = value.get('error', None)
if status is None:
status = value["status"]
message = value["value"]
if not isinstance(message, basestring):
value = message
message = message.get('message')
else:
message = value.get('message', None)
except ValueError:
pass
exception_class = ErrorInResponseException
if status in ErrorCode.NO_SUCH_ELEMENT:
exception_class = NoSuchElementException
elif status in ErrorCode.NO_SUCH_FRAME:
exception_class = NoSuchFrameException
elif status in ErrorCode.NO_SUCH_WINDOW:
exception_class = NoSuchWindowException
elif status in ErrorCode.STALE_ELEMENT_REFERENCE:
exception_class = StaleElementReferenceException
elif status in ErrorCode.ELEMENT_NOT_VISIBLE:
exception_class = ElementNotVisibleException
elif status in ErrorCode.INVALID_ELEMENT_STATE:
exception_class = InvalidElementStateException
elif status in ErrorCode.INVALID_SELECTOR \
or status in ErrorCode.INVALID_XPATH_SELECTOR \
or status in ErrorCode.INVALID_XPATH_SELECTOR_RETURN_TYPER:
exception_class = InvalidSelectorException
elif status in ErrorCode.ELEMENT_IS_NOT_SELECTABLE:
exception_class = ElementNotSelectableException
elif status in ErrorCode.ELEMENT_NOT_INTERACTABLE:
exception_class = ElementNotInteractableException
elif status in ErrorCode.INVALID_COOKIE_DOMAIN:
exception_class = InvalidCookieDomainException
elif status in ErrorCode.UNABLE_TO_SET_COOKIE:
exception_class = UnableToSetCookieException
elif status in ErrorCode.TIMEOUT:
exception_class = TimeoutException
elif status in ErrorCode.SCRIPT_TIMEOUT:
exception_class = TimeoutException
elif status in ErrorCode.UNKNOWN_ERROR:
exception_class = WebDriverException
elif status in ErrorCode.UNEXPECTED_ALERT_OPEN:
exception_class = UnexpectedAlertPresentException
elif status in ErrorCode.NO_ALERT_OPEN:
exception_class = NoAlertPresentException
elif status in ErrorCode.IME_NOT_AVAILABLE:
exception_class = ImeNotAvailableException
elif status in ErrorCode.IME_ENGINE_ACTIVATION_FAILED:
exception_class = ImeActivationFailedException
elif status in ErrorCode.MOVE_TARGET_OUT_OF_BOUNDS:
exception_class = MoveTargetOutOfBoundsException
elif status in ErrorCode.JAVASCRIPT_ERROR:
exception_class = JavascriptException
elif status in ErrorCode.SESSION_NOT_CREATED:
exception_class = SessionNotCreatedException
elif status in ErrorCode.INVALID_ARGUMENT:
exception_class = InvalidArgumentException
elif status in ErrorCode.NO_SUCH_COOKIE:
exception_class = NoSuchCookieException
elif status in ErrorCode.UNABLE_TO_CAPTURE_SCREEN:
exception_class = ScreenshotException
elif status in ErrorCode.ELEMENT_CLICK_INTERCEPTED:
exception_class = ElementClickInterceptedException
elif status in ErrorCode.INSECURE_CERTIFICATE:
exception_class = InsecureCertificateException
elif status in ErrorCode.INVALID_COORDINATES:
exception_class = InvalidCoordinatesException
elif status in ErrorCode.INVALID_SESSION_ID:
exception_class = InvalidSessionIdException
elif status in ErrorCode.UNKNOWN_METHOD:
exception_class = UnknownMethodException
else:
exception_class = WebDriverException
if value == '' or value is None:
value = response['value']
if isinstance(value, basestring):
if exception_class == ErrorInResponseException:
raise exception_class(response, value)
raise exception_class(value)
if message == "" and 'message' in value:
message = value['message']
screen = None
if 'screen' in value:
screen = value['screen']
stacktrace = None
if 'stackTrace' in value and value['stackTrace']:
stacktrace = []
try:
for frame in value['stackTrace']:
line = self._value_or_default(frame, 'lineNumber', '')
file = self._value_or_default(frame, 'fileName', '<anonymous>')
if line:
file = "%s:%s" % (file, line)
meth = self._value_or_default(frame, 'methodName', '<anonymous>')
if 'className' in frame:
meth = "%s.%s" % (frame['className'], meth)
msg = " at %s (%s)"
msg = msg % (meth, file)
stacktrace.append(msg)
except TypeError:
pass
if exception_class == ErrorInResponseException:
raise exception_class(response, message)
elif exception_class == UnexpectedAlertPresentException:
alert_text = None
if 'data' in value:
alert_text = value['data'].get('text')
elif 'alert' in value:
alert_text = value['alert'].get('text')
raise exception_class(message, screen, stacktrace, alert_text)
> raise exception_class(message, screen, stacktrace)
E selenium.common.exceptions.WebDriverException: Message: Error forwarding the new session Request timed out waiting for a node to become available.
E Stacktrace:
E at org.openqa.grid.web.servlet.handler.RequestHandler.process (RequestHandler.java:118)
E at org.openqa.grid.web.servlet.DriverServlet.process (DriverServlet.java:85)
E at org.openqa.grid.web.servlet.DriverServlet.doPost (DriverServlet.java:69)
E at javax.servlet.http.HttpServlet.service (HttpServlet.java:707)
E at javax.servlet.http.HttpServlet.service (HttpServlet.java:790)
E at org.seleniumhq.jetty9.servlet.ServletHolder.handle (ServletHolder.java:865)
E at org.seleniumhq.jetty9.servlet.ServletHandler$CachedChain.doFilter (ServletHandler.java:1655)
E at io.prometheus.client.filter.MetricsFilter.doFilter (MetricsFilter.java:170)
E at org.seleniumhq.jetty9.servlet.ServletHandler$CachedChain.doFilter (ServletHandler.java:1642)
E at org.seleniumhq.jetty9.servlet.ServletHandler.doHandle (ServletHandler.java:533)
E at org.seleniumhq.jetty9.server.handler.ScopedHandler.handle (ScopedHandler.java:146)
E at org.seleniumhq.jetty9.security.SecurityHandler.handle (SecurityHandler.java:548)
E at org.seleniumhq.jetty9.server.handler.HandlerWrapper.handle (HandlerWrapper.java:132)
E at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextHandle (ScopedHandler.java:257)
E at org.seleniumhq.jetty9.server.session.SessionHandler.doHandle (SessionHandler.java:1595)
E at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextHandle (ScopedHandler.java:255)
E at org.seleniumhq.jetty9.server.handler.ContextHandler.doHandle (ContextHandler.java:1340)
E at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextScope (ScopedHandler.java:203)
E at org.seleniumhq.jetty9.servlet.ServletHandler.doScope (ServletHandler.java:473)
E at org.seleniumhq.jetty9.server.session.SessionHandler.doScope (SessionHandler.java:1564)
E at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextScope (ScopedHandler.java:201)
E at org.seleniumhq.jetty9.server.handler.ContextHandler.doScope (ContextHandler.java:1242)
E at org.seleniumhq.jetty9.server.handler.ScopedHandler.handle (ScopedHandler.java:144)
E at org.seleniumhq.jetty9.server.handler.StatisticsHandler.handle (StatisticsHandler.java:174)
E at org.seleniumhq.jetty9.server.handler.HandlerWrapper.handle (HandlerWrapper.java:132)
E at org.seleniumhq.jetty9.server.Server.handle (Server.java:503)
E at org.seleniumhq.jetty9.server.HttpChannel.handle (HttpChannel.java:364)
E at org.seleniumhq.jetty9.server.HttpConnection.onFillable (HttpConnection.java:260)
E at org.seleniumhq.jetty9.io.AbstractConnection$ReadCallback.succeeded (AbstractConnection.java:305)
E at org.seleniumhq.jetty9.io.FillInterest.fillable (FillInterest.java:103)
E at org.seleniumhq.jetty9.io.ChannelEndPoint$2.run (ChannelEndPoint.java:118)
E at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.runTask (EatWhatYouKill.java:333)
E at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.doProduce (EatWhatYouKill.java:310)
E at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.tryProduce (EatWhatYouKill.java:168)
E at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.run (EatWhatYouKill.java:126)
E at org.seleniumhq.jetty9.util.thread.ReservedThreadExecutor$ReservedThread.run (ReservedThreadExecutor.java:366)
E at org.seleniumhq.jetty9.util.thread.QueuedThreadPool.runJob (QueuedThreadPool.java:765)
E at org.seleniumhq.jetty9.util.thread.QueuedThreadPool$2.run (QueuedThreadPool.java:683)
E at java.lang.Thread.run (Thread.java:748)
/usr/local/lib/python3.6/site-packages/selenium/webdriver/remote/errorhandler.py:242: WebDriverException