v6 patch: fix VoiceOver double cursor + add typing echo
Fixes: 1. accessibilityFrame now returns view frame (not cursor rect) - fixes VoiceOver drawing duplicate cursor overlay 2. Added NSAccessibilityValueChangedNotification - enables VoiceOver typing echo (character-by-character feedback when typing) 3. Cursor position exposed only via accessibilityBoundsForRange: Full VoiceOver text protocol retained (accessibilityValue, accessibilitySelectedText, accessibilityStringForRange:, etc).
This commit is contained in:
@@ -1,27 +1,30 @@
|
|||||||
From: Martin Sukany <martin@sukany.cz>
|
From: Martin Sukany <martin@sukany.cz>
|
||||||
Date: Tue, 25 Feb 2026 19:00:00 +0100
|
Date: Tue, 25 Feb 2026 19:20:00 +0100
|
||||||
Subject: [PATCH] ns: implement macOS Zoom cursor tracking and VoiceOver
|
Subject: [PATCH] ns: implement macOS Zoom cursor tracking and VoiceOver
|
||||||
support via UAZoomChangeFocus + NSAccessibility
|
support via UAZoomChangeFocus + NSAccessibility
|
||||||
|
|
||||||
Add cursor tracking and screen reader support for macOS accessibility:
|
Add cursor tracking and screen reader support for macOS accessibility:
|
||||||
|
|
||||||
1. UAZoomChangeFocus() from ApplicationServices/UniversalAccess.h:
|
1. UAZoomChangeFocus() from ApplicationServices/UniversalAccess.h:
|
||||||
Directly tells macOS Zoom where the cursor is. This is Apple's
|
Directly tells macOS Zoom where the cursor is.
|
||||||
documented API for applications to control Zoom focus.
|
|
||||||
Ref: https://developer.apple.com/documentation/applicationservices/universalaccess_h
|
Ref: https://developer.apple.com/documentation/applicationservices/universalaccess_h
|
||||||
Ref: https://developer.apple.com/documentation/applicationservices/1458830-uazoomchangefocus
|
Ref: https://developer.apple.com/documentation/applicationservices/1458830-uazoomchangefocus
|
||||||
|
|
||||||
2. NSAccessibility protocol on EmacsView (macOS 10.10+):
|
2. NSAccessibility protocol on EmacsView (macOS 10.10+):
|
||||||
Full text area protocol: accessibilityValue (buffer text),
|
Full text area protocol for VoiceOver: accessibilityValue (buffer text),
|
||||||
accessibilitySelectedText, accessibilitySelectedTextRange,
|
accessibilitySelectedText, accessibilitySelectedTextRange,
|
||||||
accessibilityInsertionPointLineNumber, accessibilityVisibleCharacterRange,
|
accessibilityInsertionPointLineNumber, accessibilityVisibleCharacterRange,
|
||||||
accessibilityStringForRange:, accessibilityBoundsForRange:,
|
accessibilityStringForRange:, accessibilityBoundsForRange:.
|
||||||
accessibilityFrame. Serves VoiceOver and other screen readers.
|
Notifications: ValueChanged (typing echo), SelectedTextChanged (cursor
|
||||||
|
movement), FocusedUIElementChanged (window focus).
|
||||||
Ref: https://developer.apple.com/documentation/appkit/nsaccessibilityprotocol
|
Ref: https://developer.apple.com/documentation/appkit/nsaccessibilityprotocol
|
||||||
|
|
||||||
Both mechanisms are needed: UAZoomChangeFocus serves Zoom's "Follow
|
accessibilityFrame returns the view's frame (standard behavior) so
|
||||||
keyboard focus"; NSAccessibility serves VoiceOver and screen readers
|
VoiceOver draws its focus ring around the text area. Cursor position
|
||||||
that query the accessibility tree. Same dual pattern used by iTerm2.
|
is exposed via accessibilityBoundsForRange: only.
|
||||||
|
|
||||||
|
Both mechanisms are needed: UAZoomChangeFocus serves Zoom; NSAccessibility
|
||||||
|
serves VoiceOver. Same dual pattern used by iTerm2.
|
||||||
---
|
---
|
||||||
diff --git a/src/nsterm.h b/src/nsterm.h
|
diff --git a/src/nsterm.h b/src/nsterm.h
|
||||||
index 7c1ee4c..6c1ff34 100644
|
index 7c1ee4c..6c1ff34 100644
|
||||||
@@ -38,10 +41,10 @@ index 7c1ee4c..6c1ff34 100644
|
|||||||
|
|
||||||
/* AppKit-side interface. */
|
/* AppKit-side interface. */
|
||||||
diff --git a/src/nsterm.m b/src/nsterm.m
|
diff --git a/src/nsterm.m b/src/nsterm.m
|
||||||
index 932d209..6a06b2e 100644
|
index 932d209..2576c96 100644
|
||||||
--- a/src/nsterm.m
|
--- a/src/nsterm.m
|
||||||
+++ b/src/nsterm.m
|
+++ b/src/nsterm.m
|
||||||
@@ -3232,6 +3232,67 @@ Note that CURSOR_WIDTH is meaningful only for (h)bar cursors.
|
@@ -3232,6 +3232,72 @@ Note that CURSOR_WIDTH is meaningful only for (h)bar cursors.
|
||||||
/* Prevent the cursor from being drawn outside the text area. */
|
/* Prevent the cursor from being drawn outside the text area. */
|
||||||
r = NSIntersectionRect (r, ns_row_rect (w, glyph_row, TEXT_AREA));
|
r = NSIntersectionRect (r, ns_row_rect (w, glyph_row, TEXT_AREA));
|
||||||
|
|
||||||
@@ -53,32 +56,38 @@ index 932d209..6a06b2e 100644
|
|||||||
+ technology. Two complementary mechanisms are used:
|
+ technology. Two complementary mechanisms are used:
|
||||||
+
|
+
|
||||||
+ 1. NSAccessibility notifications:
|
+ 1. NSAccessibility notifications:
|
||||||
+ Post NSAccessibilitySelectedTextChangedNotification so that
|
+ - ValueChangedNotification: tells VoiceOver the buffer content
|
||||||
+ VoiceOver and other AT tools can query cursor position via
|
+ changed (triggers typing echo and re-read of accessibilityValue).
|
||||||
+ accessibilityBoundsForRange: / accessibilityFrame.
|
+ - SelectedTextChangedNotification: tells AT tools the cursor moved
|
||||||
|
+ (triggers re-query of accessibilitySelectedTextRange and
|
||||||
|
+ accessibilityBoundsForRange:).
|
||||||
+
|
+
|
||||||
+ 2. UAZoomChangeFocus() from ApplicationServices/UniversalAccess.h:
|
+ 2. UAZoomChangeFocus() from ApplicationServices/UniversalAccess.h:
|
||||||
+ Directly tells macOS Zoom where to move its viewport. This is
|
+ Directly tells macOS Zoom where to move its viewport. This is
|
||||||
+ the documented API for applications to control Zoom focus. Same
|
+ the documented API for applications to control Zoom focus.
|
||||||
+ approach used by iTerm2 (PTYTextView.m:refreshAccessibility).
|
+ Same approach used by iTerm2 (PTYTextView.m:refreshAccessibility).
|
||||||
+
|
+
|
||||||
+ Ref: https://developer.apple.com/documentation/applicationservices/universalaccess_h
|
+ Ref: https://developer.apple.com/documentation/applicationservices/universalaccess_h
|
||||||
+ Ref: https://developer.apple.com/documentation/applicationservices/1458830-uazoomchangefocus
|
+ Ref: https://developer.apple.com/documentation/applicationservices/1458830-uazoomchangefocus
|
||||||
+
|
+
|
||||||
+ Both mechanisms are needed: NSAccessibility serves VoiceOver and
|
+ Both mechanisms are needed: NSAccessibility serves VoiceOver and
|
||||||
+ screen readers (which query the accessibility tree); UAZoomChangeFocus
|
+ screen readers (which query the accessibility tree); UAZoomChangeFocus
|
||||||
+ serves macOS Zoom's "Follow keyboard focus" (which does not reliably
|
+ serves macOS Zoom's "Follow keyboard focus" feature. */
|
||||||
+ track custom views through NSAccessibility alone). */
|
|
||||||
+ {
|
+ {
|
||||||
+ EmacsView *view = FRAME_NS_VIEW (f);
|
+ EmacsView *view = FRAME_NS_VIEW (f);
|
||||||
+ if (view)
|
+ if (view)
|
||||||
+ {
|
+ {
|
||||||
+ /* Store cursor rect for accessibilityFrame and
|
+ /* Store cursor rect for accessibilityBoundsForRange: queries. */
|
||||||
+ accessibilityBoundsForRange: queries. */
|
|
||||||
+ view->lastAccessibilityCursorRect = r;
|
+ view->lastAccessibilityCursorRect = r;
|
||||||
+
|
+
|
||||||
+ /* Post NSAccessibility notification for VoiceOver and other
|
+ /* Notify VoiceOver that buffer content may have changed.
|
||||||
+ AT tools. */
|
+ This triggers typing echo (character-by-character feedback)
|
||||||
|
+ and re-read of accessibilityValue. iTerm2 posts this same
|
||||||
|
+ notification in its refreshAccessibility method. */
|
||||||
|
+ NSAccessibilityPostNotification (view,
|
||||||
|
+ NSAccessibilityValueChangedNotification);
|
||||||
|
+
|
||||||
|
+ /* Notify AT tools that the cursor (selection) moved. */
|
||||||
+ NSAccessibilityPostNotification (view,
|
+ NSAccessibilityPostNotification (view,
|
||||||
+ NSAccessibilitySelectedTextChangedNotification);
|
+ NSAccessibilitySelectedTextChangedNotification);
|
||||||
+
|
+
|
||||||
@@ -86,8 +95,7 @@ index 932d209..6a06b2e 100644
|
|||||||
+ expects coordinates with origin at the top-left of the
|
+ expects coordinates with origin at the top-left of the
|
||||||
+ primary screen (CG accessibility coordinate space).
|
+ primary screen (CG accessibility coordinate space).
|
||||||
+ convertRectToScreen: returns Quartz coordinates (origin at
|
+ convertRectToScreen: returns Quartz coordinates (origin at
|
||||||
+ bottom-left), so we flip the y axis. This coordinate
|
+ bottom-left), so we flip the y axis. */
|
||||||
+ conversion follows the same pattern used by iTerm2. */
|
|
||||||
+ if (UAZoomEnabled ())
|
+ if (UAZoomEnabled ())
|
||||||
+ {
|
+ {
|
||||||
+ NSRect windowRect = [view convertRect:r toView:nil];
|
+ NSRect windowRect = [view convertRect:r toView:nil];
|
||||||
@@ -109,7 +117,7 @@ index 932d209..6a06b2e 100644
|
|||||||
ns_focus (f, NULL, 0);
|
ns_focus (f, NULL, 0);
|
||||||
|
|
||||||
NSGraphicsContext *ctx = [NSGraphicsContext currentContext];
|
NSGraphicsContext *ctx = [NSGraphicsContext currentContext];
|
||||||
@@ -8237,6 +8298,15 @@ - (void)windowDidBecomeKey /* for direct calls */
|
@@ -8237,6 +8303,15 @@ - (void)windowDidBecomeKey /* for direct calls */
|
||||||
XSETFRAME (event.frame_or_window, emacsframe);
|
XSETFRAME (event.frame_or_window, emacsframe);
|
||||||
kbd_buffer_store_event (&event);
|
kbd_buffer_store_event (&event);
|
||||||
ns_send_appdefined (-1); // Kick main loop
|
ns_send_appdefined (-1); // Kick main loop
|
||||||
@@ -125,7 +133,7 @@ index 932d209..6a06b2e 100644
|
|||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
@@ -9474,6 +9544,326 @@ - (int) fullscreenState
|
@@ -9474,6 +9549,310 @@ - (int) fullscreenState
|
||||||
return fs_state;
|
return fs_state;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -136,31 +144,26 @@ index 932d209..6a06b2e 100644
|
|||||||
+
|
+
|
||||||
+ EmacsView implements the NSAccessibility protocol so that:
|
+ EmacsView implements the NSAccessibility protocol so that:
|
||||||
+ - macOS Zoom can query cursor position (accessibilityBoundsForRange:)
|
+ - macOS Zoom can query cursor position (accessibilityBoundsForRange:)
|
||||||
+ - VoiceOver can read buffer contents and track cursor
|
+ - VoiceOver can read buffer contents, track cursor, echo typing
|
||||||
+ - Accessibility Inspector shows correct element hierarchy
|
+ - Accessibility Inspector shows correct element hierarchy
|
||||||
+
|
+
|
||||||
+ The primary Zoom tracking mechanism is UAZoomChangeFocus() in
|
+ The primary Zoom tracking mechanism is UAZoomChangeFocus() in
|
||||||
+ ns_draw_window_cursor above. These methods serve VoiceOver and
|
+ ns_draw_window_cursor above. These methods serve VoiceOver and
|
||||||
+ other AT tools that query the accessibility tree directly.
|
+ other AT tools that query the accessibility tree directly.
|
||||||
+
|
+
|
||||||
+ Text content methods (accessibilityValue, accessibilitySelectedText,
|
+ IMPORTANT: accessibilityFrame returns the VIEW's frame (standard
|
||||||
+ accessibilityInsertionPointLineNumber, accessibilityVisibleCharacterRange,
|
+ behavior). VoiceOver uses this to draw its focus ring around the
|
||||||
+ accessibilityNumberOfCharacters) read from the current buffer, mirroring
|
+ entire text area. The CURSOR position is exposed via
|
||||||
+ the pattern in EmacsWindow's accessibilityAttributeValue: but at the
|
+ accessibilityBoundsForRange: which AT tools call with the
|
||||||
+ view level where VoiceOver expects to find them.
|
+ selectedTextRange to locate the insertion point. Returning the
|
||||||
|
+ cursor rect from accessibilityFrame causes VoiceOver to draw a
|
||||||
|
+ duplicate cursor overlay.
|
||||||
+
|
+
|
||||||
+ Ref: https://developer.apple.com/documentation/appkit/nsaccessibilityprotocol
|
+ Ref: https://developer.apple.com/documentation/appkit/nsaccessibilityprotocol
|
||||||
+ Ref: https://developer.apple.com/documentation/appkit/nsaccessibilityboundsforrangeparameterizedattribute
|
|
||||||
+
|
|
||||||
+ Note: EmacsWindow has a separate accessibilityAttributeValue: that
|
|
||||||
+ returns buffer text via the legacy API. These EmacsView methods use
|
|
||||||
+ the modern protocol API (10.10+) and operate on the view level where
|
|
||||||
+ VoiceOver resolves the text area role. No conflict.
|
|
||||||
+ ---------------------------------------------------------------- */
|
+ ---------------------------------------------------------------- */
|
||||||
+
|
+
|
||||||
+- (BOOL)accessibilityIsIgnored
|
+- (BOOL)accessibilityIsIgnored
|
||||||
+{
|
+{
|
||||||
+ /* EmacsView must participate in the accessibility hierarchy. */
|
|
||||||
+ return NO;
|
+ return NO;
|
||||||
+}
|
+}
|
||||||
+
|
+
|
||||||
@@ -171,7 +174,6 @@ index 932d209..6a06b2e 100644
|
|||||||
+
|
+
|
||||||
+- (id)accessibilityFocusedUIElement
|
+- (id)accessibilityFocusedUIElement
|
||||||
+{
|
+{
|
||||||
+ /* This view is the focused element -- it contains the text cursor. */
|
|
||||||
+ return self;
|
+ return self;
|
||||||
+}
|
+}
|
||||||
+
|
+
|
||||||
@@ -190,7 +192,7 @@ index 932d209..6a06b2e 100644
|
|||||||
+- (id)accessibilityValue
|
+- (id)accessibilityValue
|
||||||
+{
|
+{
|
||||||
+ /* Return visible buffer text. VoiceOver reads this when the user
|
+ /* Return visible buffer text. VoiceOver reads this when the user
|
||||||
+ navigates to the text area. */
|
+ navigates to the text area and after ValueChangedNotification. */
|
||||||
+ if (!emacsframe)
|
+ if (!emacsframe)
|
||||||
+ return @"";
|
+ return @"";
|
||||||
+
|
+
|
||||||
@@ -203,7 +205,6 @@ index 932d209..6a06b2e 100644
|
|||||||
+ ptrdiff_t byte_range = BUF_ZV_BYTE (curbuf) - start_byte;
|
+ ptrdiff_t byte_range = BUF_ZV_BYTE (curbuf) - start_byte;
|
||||||
+ ptrdiff_t range = BUF_ZV (curbuf) - BUF_BEGV (curbuf);
|
+ ptrdiff_t range = BUF_ZV (curbuf) - BUF_BEGV (curbuf);
|
||||||
+
|
+
|
||||||
+ /* Limit to 10000 chars to avoid performance issues with large buffers. */
|
|
||||||
+ if (range > 10000)
|
+ if (range > 10000)
|
||||||
+ {
|
+ {
|
||||||
+ range = 10000;
|
+ range = 10000;
|
||||||
@@ -267,7 +268,6 @@ index 932d209..6a06b2e 100644
|
|||||||
+ if (!curbuf)
|
+ if (!curbuf)
|
||||||
+ return NSMakeRange (0, 0);
|
+ return NSMakeRange (0, 0);
|
||||||
+
|
+
|
||||||
+ /* Return cursor position as collapsed selection. */
|
|
||||||
+ ptrdiff_t pt = BUF_PT (curbuf) - BUF_BEGV (curbuf);
|
+ ptrdiff_t pt = BUF_PT (curbuf) - BUF_BEGV (curbuf);
|
||||||
+ return NSMakeRange ((NSUInteger) pt, 0);
|
+ return NSMakeRange ((NSUInteger) pt, 0);
|
||||||
+}
|
+}
|
||||||
@@ -281,7 +281,6 @@ index 932d209..6a06b2e 100644
|
|||||||
+ if (!w)
|
+ if (!w)
|
||||||
+ return 0;
|
+ return 0;
|
||||||
+
|
+
|
||||||
+ /* Return cursor line relative to window start. */
|
|
||||||
+ return (NSInteger) (w->cursor.vpos);
|
+ return (NSInteger) (w->cursor.vpos);
|
||||||
+}
|
+}
|
||||||
+
|
+
|
||||||
@@ -332,26 +331,20 @@ index 932d209..6a06b2e 100644
|
|||||||
+ return [NSString stringWithLispString:str];
|
+ return [NSString stringWithLispString:str];
|
||||||
+}
|
+}
|
||||||
+
|
+
|
||||||
+/* ---- Cursor position methods for Zoom and VoiceOver ---- */
|
+/* ---- Cursor position for Zoom (via accessibilityBoundsForRange:) ----
|
||||||
|
+
|
||||||
|
+ accessibilityFrame intentionally returns the VIEW's frame (standard
|
||||||
|
+ behavior) so VoiceOver draws its focus ring around the entire text
|
||||||
|
+ area rather than at the cursor position. The cursor location is
|
||||||
|
+ exposed through accessibilityBoundsForRange: which AT tools query
|
||||||
|
+ using the selectedTextRange. */
|
||||||
+
|
+
|
||||||
+- (NSRect)accessibilityFrame
|
+- (NSRect)accessibilityFrame
|
||||||
+{
|
+{
|
||||||
+ /* Return the cursor's screen coordinates. This is the key method
|
+ /* Return the view's screen frame (standard NSView behavior).
|
||||||
+ that macOS Zoom reads after receiving a focus/selection notification.
|
+ VoiceOver uses this for its focus ring. Do NOT return the cursor
|
||||||
+
|
+ rect here -- that causes a duplicate cursor overlay. */
|
||||||
+ lastAccessibilityCursorRect is in EmacsView coordinates (flipped:
|
+ return [super accessibilityFrame];
|
||||||
+ origin top-left). convertRect:toView:nil handles the
|
|
||||||
+ flipped-to-unflipped conversion automatically (isFlipped=YES). */
|
|
||||||
+ NSRect viewRect = lastAccessibilityCursorRect;
|
|
||||||
+ if (NSIsEmptyRect (viewRect))
|
|
||||||
+ return [super accessibilityFrame];
|
|
||||||
+
|
|
||||||
+ NSWindow *win = [self window];
|
|
||||||
+ if (win == nil)
|
|
||||||
+ return [super accessibilityFrame];
|
|
||||||
+
|
|
||||||
+ NSRect windowRect = [self convertRect:viewRect toView:nil];
|
|
||||||
+ return [win convertRectToScreen:windowRect];
|
|
||||||
+}
|
+}
|
||||||
+
|
+
|
||||||
+- (NSArray *)accessibilityAttributeNames
|
+- (NSArray *)accessibilityAttributeNames
|
||||||
@@ -395,13 +388,13 @@ index 932d209..6a06b2e 100644
|
|||||||
+ return [super accessibilityAttributeValue:attribute];
|
+ return [super accessibilityAttributeValue:attribute];
|
||||||
+}
|
+}
|
||||||
+
|
+
|
||||||
+/* Modern NSAccessibilityProtocol (macOS 10.10+).
|
+/* Modern NSAccessibilityProtocol (macOS 10.10+). */
|
||||||
+ Ref: https://developer.apple.com/documentation/appkit/nsaccessibilityprotocol */
|
|
||||||
+- (NSRect)accessibilityBoundsForRange:(NSRange)range
|
+- (NSRect)accessibilityBoundsForRange:(NSRange)range
|
||||||
+{
|
+{
|
||||||
+ /* Return cursor screen rect regardless of requested range.
|
+ /* Return cursor screen rect. AT tools call this with the
|
||||||
+ Emacs does not expose a character-level geometry model to AppKit,
|
+ selectedTextRange to locate the insertion point. We return the
|
||||||
+ so we always return the cursor position. */
|
+ cursor position regardless of the requested range because Emacs
|
||||||
|
+ does not expose character-level geometry to AppKit. */
|
||||||
+ NSRect viewRect = lastAccessibilityCursorRect;
|
+ NSRect viewRect = lastAccessibilityCursorRect;
|
||||||
+
|
+
|
||||||
+ if (viewRect.size.width < 1)
|
+ if (viewRect.size.width < 1)
|
||||||
@@ -417,8 +410,7 @@ index 932d209..6a06b2e 100644
|
|||||||
+ return [win convertRectToScreen:windowRect];
|
+ return [win convertRectToScreen:windowRect];
|
||||||
+}
|
+}
|
||||||
+
|
+
|
||||||
+/* Legacy parameterized attribute API -- fallback for older AT tools.
|
+/* Legacy parameterized attribute API. */
|
||||||
+ Ref: https://developer.apple.com/documentation/appkit/nsaccessibilityboundsforrangeparameterizedattribute */
|
|
||||||
+- (NSArray *)accessibilityParameterizedAttributeNames
|
+- (NSArray *)accessibilityParameterizedAttributeNames
|
||||||
+{
|
+{
|
||||||
+ NSArray *superAttrs = [super accessibilityParameterizedAttributeNames];
|
+ NSArray *superAttrs = [super accessibilityParameterizedAttributeNames];
|
||||||
|
|||||||
Reference in New Issue
Block a user