You can try to override method
public void onCloseMenu(MenuBuilder menu, boolean allMenusAreClosing)
of PopupMenu class. Exactly this method is calling setOnDismissListener() internally in API 14+. But this is not easy to do. Problem is what class com.android.internal.view.menu.MenuBuilder is not included to Android SDK, so you will not be able to compile your code if you will trying to import this class.
However, you can create your own "fake" implementation of this class, place it in package com.android.internal.view.menu, make a .jar file with it, copy this "fake library" to libs folder and set scope of this library in Project Settings of IntelliJ IDEA as "provided". This means that classes in this jar file already have real devices (what is true) and you only need them to compile your code. Main idea here - your fake MenuBuilder should be in classpath for javac compilation, but should not be included in resulting .apk file.
This is not easy solution, but it should work.