0

我正在尝试在任务栏应用程序图标上侦听鼠标单击事件，以使用 python 在 Windows 中当前运行的应用程序。在下面的示例中，我使用 python 来启动 Internet Explorer 应用程序。应用程序启动后，我使用 FindWindow windows API 函数来获取窗口 ID/句柄。

有没有办法使用 Windows API 或其他方法来监听用户在 Internet Explorer 任务栏图标上的鼠标点击？单击任务栏中的 IE 图标时，我需要打印一条消息。

在此处输入图像描述

这是我目前正在使用的代码：

from win32gui import GetWindowText, GetForegroundWindow, FindWindow
import win32api, win32gui
import win32con
import sys, os

os.system('"C:\Program Files (x86)\Internet Explorer\iexplore.exe"') #open internet explorer application.. this will make the internet explorer icon appear in the taskbar..

window_id = FindWindow(0, 'Internet Explorer') #find the newly opened IE window's id/handle
print (window_id) 

class test:

    def __init__(self, **kwargs):
        super(test, self).__init__(**kwargs)
        self.oldWndProc = win32gui.SetWindowLong(window_id, win32con.GWL_WNDPROC, self.MyWndProc)
        self.msgdict = {} 
        for name in dir(win32con):
            if name.startswith("WM_"):
                value = getattr(win32con, name)
                self.msgdict[value] = name
        print (self.msgdict) #I can access all the win32 window events using this dictionary..

    #listener to track win32 window events related to the newly opened internet explorer window..
    #how do I track when the internet explorer icon is CLICKED in the task bar??
    def MyWndProc(self, hWnd, msg, wParam, lParam):
        print (self.msgdict.get(msg)) #none of the window events print here..
        return win32gui.CallWindowProc(self.oldWndProc, hWnd, msg, wParam, lParam)

t = test()
4

2 回答 2

1

尝试使用UI 自动化。

任务栏的窗口类名是MSTaskListWClass：

在此处输入图像描述

您还可以从以下位置查看 UIA 元素及其 BoundingRectangle inspect.exe：

在此处输入图像描述

您可以先获取桌面上的点击事件，然后获取任务图标的IUIAutomationElement，然后用于GetClickablePoint确定点击是否在元素上。

于 2020-01-28T08:31:32.520 回答
1

感谢 Drake Wu 为我指明了正确的方向，我能够使用 python 来获取有关单击的元素及其父元素的 UI 自动化信息。使用该信息，我可以判断当单击的元素的 ControlType 为“ButtonControl”且其父级的类名称为“MSTaskListWClass”时，单击了任务栏中的应用程序图标。您可以通过查看单击元素的自动化 ID 来确定单击了哪个应用程序。

对于此示例，您需要 pip install uiautomation。我不得不在 uiautomation.py 文件中添加一些额外的功能/方法，因为它缺少所需的功能。

以下是我添加到 uiautomation.py 文件中以使其对本示例更有用的方法（请参阅此处显示的新代码）：

def ControlFromPoint(x: int, y: int) -> Control:
    """
    Call IUIAutomation ElementFromPoint x,y. May return None if mouse is over cmd's title bar icon.
    Return `Control` subclass or None.
    """
    element = _AutomationClient.instance().IUIAutomation.ElementFromPoint(ctypes.wintypes.POINT(x, y))
    return Control.CreateControlFromElement(element)

#NEW CODE HERE
def GetElementFromPoint(x: int, y: int) -> Control: #gets the element from a point (x, y)
    element = _AutomationClient.instance().IUIAutomation.ElementFromPoint(ctypes.wintypes.POINT(x, y))
    return element

#NEW CODE HERE
def GetParentElement(element): #gets the parent of an element input
    parent_element = _AutomationClient.instance().ViewWalker.GetParentElement(element)
    return parent_element

#NEW CODE HERE
def GetElementInfo(element): #gets the property information about an element
    element_info = Control.CreateControlFromElement(element)
    return element_info

然后我在下面的主要示例中使用了这些新添加的 uiautomation 方法：

import sys, os
from pynput import mouse
from pynput.mouse import Controller
from pynput.mouse import Listener
import uiautomation
from uiautomation import GetElementFromPoint, GetParentElement, GetElementInfo #these are newly added methods to make this example work

mouse2 = mouse.Controller()

class test:

    last_mouse_pos_x = ''
    last_mouse_pos_y = ''

    def __init__(self, **kwargs):
        super(test, self).__init__(**kwargs)
        self.mouse_listener() #start mouse listener

    def mouse_listener(self):
        global element_clicked
        while True:
            if mouse2.position[0] != test.last_mouse_pos_x or mouse2.position[1] != test.last_mouse_pos_y:
                element_clicked = uiautomation.GetElementFromPoint(mouse2.position[0], mouse2.position[1]) #returns the element that you clicked (this has to run here or it errors)
                test.last_mouse_pos_x = mouse2.position[0]
                test.last_mouse_pos_y = mouse2.position[1]

    def on_click(x, y, button, pressed):
        if pressed:
            try:
                element_clicked_info = GetElementInfo(element_clicked) #returns info about the element you clicked
                parent_of_element_clicked = uiautomation.GetParentElement(element_clicked) #returns the parent of the element that you clicked
                parent_of_element_clicked_info = GetElementInfo(parent_of_element_clicked) #returns info about the parent of the element you clicked
                #print info about the element you clicked and its parent
                print ("ELEMENT CLICKED INFO: " + str(element_clicked_info))
                print ("PARENT ELEMENT INFO: " + str(parent_of_element_clicked_info)) 
            except:
                pass #skip over any errors

            #ANALYSIS: If ControlType of element clicked = 'ButtonControl' AND class name of its parent = 'MSTaskListWClass', then an app icon in the taskbar was clicked!

    listener = mouse.Listener(on_click=on_click)
    listener.start()

t = test()
于 2020-01-28T21:23:40.183 回答