3

I have written an extensive on-screen keyboard (OSK) in Java. This is intended for people with disabilities and needs to be able to do everything that a standard keyboard can do. One thing I've noticed that doesn't work is context menus - in (most) Windows programs. I want my OSK user to be able to click the 'Context Menu' key (normally beside right-Ctrl on standard keyboard) and use the arrow keys to navigate that menu. The context menu appears fine, but when I click back onto my Frame (to press an arrow key), the context menu disappears.

Anybody got any ideas as to how this might be achieved (probably a fairly monstrous 'hack' required I would think)? I'm pretty sure the context is disappearing because a mouse-click is occurring elsewhere on the screen (hard to get around that when you need to actually click something!).

Below is a test app. I can use it to navigate down through a context menu in NetBeans successfully, but not in any other application I've tried (e.g. Windows Explorer).

import java.awt.Robot;
import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;
import java.awt.event.KeyEvent;
import javax.swing.JButton;
import javax.swing.JFrame;
import javax.swing.JPanel;
import javax.swing.WindowConstants;

public class RobotContextMenuTest extends JFrame implements ActionListener
{
    Robot R;
    JPanel JP;
    JButton JBmenu, JBdown;

    public RobotContextMenuTest()
    {
        try
        {
            R = new Robot();
        }
        catch(Exception e)
        {
            e.printStackTrace();
        }

        JP = new JPanel();
        getContentPane().add(JP);

        JBmenu = new JButton("Menu");
        JBmenu.addActionListener(this);
        JP.add(JBmenu);

        JBdown = new JButton("Down");
        JBdown.addActionListener(this);
        JP.add(JBdown);

        setFocusableWindowState(false);
        setAlwaysOnTop(true);
        setSize(200,80);
        setDefaultCloseOperation(WindowConstants.EXIT_ON_CLOSE);
        setVisible(true);
    }
    public static void main(String args[])
    {
        new RobotContextMenuTest();
    }

    public void actionPerformed(ActionEvent e)
    {
        if(e.getActionCommand().equals("Menu"))
        {
            R.keyPress(KeyEvent.VK_CONTEXT_MENU);
            R.keyRelease(KeyEvent.VK_CONTEXT_MENU);
        }
        else if(e.getActionCommand().equals("Down"))
        {
            R.keyPress(KeyEvent.VK_DOWN);
            R.keyRelease(KeyEvent.VK_DOWN);
        }
    }
}

Finally, here's what I've tried:

  • Using MouseListener on the buttons instead of ActionListener, then 'consuming' the MouseEvent in the hope that the other application won't register it and hide the context menu.
  • I made a Threaded test application that is able to show any context menu and automatically scan down through it (once every second) but that doesn't really solve my problem! I want the context menu navigation to be controlled by the user's mouse clicks.
  • I've tried using JNativeHook as an alternative method of capturing mouse clicks but the result is the same.

Sorry for the long-winded question but it's a fairly complicated problem! Incidentally, the standard Windows 7 OSK can't do this either but WiViK can. Thanks.

4

2 回答 2

3

终于在朋友的帮助下破解了这个问题（感谢 Chris Gregan！）。经过多次失败的尝试，答案相对简单。我使用 JNA 和 Windows API 为低级鼠标事件创建了一个挂钩。这一切都用 'prunge' 的答案中给出的 2 个示例类进行了描述。注意：我必须包含 jna 版本 3.4.0 和 jna 平台版本 3.4.0 的 jars（这些 jars 的较新版本会导致错误！）。

使用上面链接的 MouseHook，您可以通过从 LowLevelMouseProc 回调返回 -1 来防止低级事件传播（到达其他应用程序）。

这是我添加到 MouseHook 中的回调方法中的一个片段，用于拦截/抑制鼠标左键按下（从而防止上下文菜单消失！）：

if(nCode >= 0 && wParam.intValue() == WinUserX.WM_LBUTTONDOWN)
    return new LRESULT(-1);

再次感谢 Alex Barker 抽出宝贵时间来研究这个问题，并为我指明了 JNA 和 Windows API 的方向。

于 2013-08-22T08:58:01.150 回答
2
  • 在按钮上使用 MouseListener 而不是 ActionListener，然后“使用” MouseEvent，希望其他应用程序不会注册它并隐藏上下文菜单。

这目前对您不起作用，因为在 Java 中使用事件只会影响您的应用程序，它不能阻止事件在本机系统上传播。JNativeHook 1.2 可能有一个未记录的方法允许您在 Windows 上执行此操作，尽管我不确定它是否真的能解决您的问题。如果您在本机系统上使用单击事件，jvm 可能不会收到 osk 按钮上的单击事件以发送向下箭头事件。在您需要使用本机事件之前，无法检测是否单击了 java 按钮。换言之，您将调用消费方法的 NativeInputEvent 可能会在 Java AWT InputEvent 之前交付。请参阅https://code.google.com/p/jnativehook/issues/detail?id=22了解更多信息。

我现在能想到的唯一好的解决方案是使用 JNA 调用 Windows API 并获取上下文菜单句柄，然后直接遍历它。有可用的 API 来做这种事情，我只是不记得它们是什么。也许从http://msdn.microsoft.com/en-us/library/windows/desktop/ff468865(v=vs.85).aspx开始，看看您是否可以创建一个小测试程序来操作菜单。

*小更新：在谷歌搜索之后，我找到了下面的帖子，它几乎概述了如何使用 C++ 做到这一点。将其移植到 JNA 应该相当简单。

于 2013-07-22T19:10:11.307 回答