v5 patch: full VoiceOver support + Zoom cursor tracking

Added full NSAccessibility text protocol for VoiceOver:
- accessibilityValue (buffer text, capped at 10k chars)
- accessibilitySelectedText (active region)
- accessibilitySelectedTextRange (cursor position)
- accessibilityInsertionPointLineNumber
- accessibilityVisibleCharacterRange
- accessibilityStringForRange:
- accessibilityAttributeValue: delegating to modern methods
Plus legacy parameterized attributes (StringForRange, BoundsForRange).

UAZoomChangeFocus retained for Zoom 'Follow keyboard focus'.
This commit is contained in:
2026-02-25 18:45:48 +01:00
parent 2a58436f32
commit 4e34114fc8

View File

@@ -1,26 +1,27 @@
From: Martin Sukany <martin@sukany.cz>
Date: Tue, 25 Feb 2026 18:20:00 +0100
Subject: [PATCH] ns: implement macOS Zoom cursor tracking via UAZoomChangeFocus
+ NSAccessibility
Date: Tue, 25 Feb 2026 19:00:00 +0100
Subject: [PATCH] ns: implement macOS Zoom cursor tracking and VoiceOver
support via UAZoomChangeFocus + NSAccessibility
Add cursor tracking support for macOS Zoom "Follow keyboard focus" and
other assistive technology tools (VoiceOver, etc.).
Two complementary mechanisms are implemented:
Add cursor tracking and screen reader support for macOS accessibility:
1. UAZoomChangeFocus() from ApplicationServices/UniversalAccess.h:
Directly tells macOS Zoom where the cursor is. This is Apple's
documented API for applications to control Zoom focus. Same
approach used by iTerm2 (PTYTextView.m:refreshAccessibility).
documented API for applications to control Zoom focus.
Ref: https://developer.apple.com/documentation/applicationservices/universalaccess_h
Ref: https://developer.apple.com/documentation/applicationservices/1458830-uazoomchangefocus
2. NSAccessibility protocol on EmacsView: reports as TextArea role,
exposes accessibilityFrame and accessibilityBoundsForRange: returning
cursor screen coordinates, posts SelectedTextChanged and
FocusedUIElementChanged notifications. Serves VoiceOver and other
AT tools that query the accessibility tree directly.
2. NSAccessibility protocol on EmacsView (macOS 10.10+):
Full text area protocol: accessibilityValue (buffer text),
accessibilitySelectedText, accessibilitySelectedTextRange,
accessibilityInsertionPointLineNumber, accessibilityVisibleCharacterRange,
accessibilityStringForRange:, accessibilityBoundsForRange:,
accessibilityFrame. Serves VoiceOver and other screen readers.
Ref: https://developer.apple.com/documentation/appkit/nsaccessibilityprotocol
Both mechanisms are needed: UAZoomChangeFocus serves Zoom's "Follow
keyboard focus"; NSAccessibility serves VoiceOver and screen readers
that query the accessibility tree. Same dual pattern used by iTerm2.
---
diff --git a/src/nsterm.h b/src/nsterm.h
index 7c1ee4c..6c1ff34 100644
@@ -37,10 +38,10 @@ index 7c1ee4c..6c1ff34 100644
/* AppKit-side interface. */
diff --git a/src/nsterm.m b/src/nsterm.m
index 932d209..913831b 100644
index 932d209..6a06b2e 100644
--- a/src/nsterm.m
+++ b/src/nsterm.m
@@ -3232,6 +3232,76 @@ Note that CURSOR_WIDTH is meaningful only for (h)bar cursors.
@@ -3232,6 +3232,67 @@ Note that CURSOR_WIDTH is meaningful only for (h)bar cursors.
/* Prevent the cursor from being drawn outside the text area. */
r = NSIntersectionRect (r, ns_row_rect (w, glyph_row, TEXT_AREA));
@@ -58,23 +59,16 @@ index 932d209..913831b 100644
+
+ 2. UAZoomChangeFocus() from ApplicationServices/UniversalAccess.h:
+ Directly tells macOS Zoom where to move its viewport. This is
+ the documented API for applications to control Zoom focus:
+
+ "This header file contains functions that give applications the
+ ability to control the zoom focus. Using these functions, an
+ application can tell the macOS Universal Access zoom feature
+ what part of its user interface needs focus."
+ the documented API for applications to control Zoom focus. Same
+ approach used by iTerm2 (PTYTextView.m:refreshAccessibility).
+
+ Ref: https://developer.apple.com/documentation/applicationservices/universalaccess_h
+ Ref: https://developer.apple.com/documentation/applicationservices/1458830-uazoomchangefocus
+
+ This is the same approach used by iTerm2 (PTYTextView.m,
+ method refreshAccessibility).
+
+ Both mechanisms are needed: NSAccessibility serves VoiceOver and
+ screen readers; UAZoomChangeFocus serves macOS Zoom's "Follow
+ keyboard focus" feature, which does not reliably track custom views
+ through NSAccessibility notifications alone. */
+ screen readers (which query the accessibility tree); UAZoomChangeFocus
+ serves macOS Zoom's "Follow keyboard focus" (which does not reliably
+ track custom views through NSAccessibility alone). */
+ {
+ EmacsView *view = FRAME_NS_VIEW (f);
+ if (view)
@@ -93,8 +87,7 @@ index 932d209..913831b 100644
+ primary screen (CG accessibility coordinate space).
+ convertRectToScreen: returns Quartz coordinates (origin at
+ bottom-left), so we flip the y axis. This coordinate
+ conversion follows the same pattern used by iTerm2's
+ accessibilityConvertScreenRect: method. */
+ conversion follows the same pattern used by iTerm2. */
+ if (UAZoomEnabled ())
+ {
+ NSRect windowRect = [view convertRect:r toView:nil];
@@ -112,12 +105,11 @@ index 932d209..913831b 100644
+ }
+ }
+#endif
+
+
ns_focus (f, NULL, 0);
NSGraphicsContext *ctx = [NSGraphicsContext currentContext];
@@ -8237,6 +8307,15 @@ - (void)windowDidBecomeKey /* for direct calls */
@@ -8237,6 +8298,15 @@ - (void)windowDidBecomeKey /* for direct calls */
XSETFRAME (event.frame_or_window, emacsframe);
kbd_buffer_store_event (&event);
ns_send_appdefined (-1); // Kick main loop
@@ -133,7 +125,7 @@ index 932d209..913831b 100644
}
@@ -9474,6 +9553,149 @@ - (int) fullscreenState
@@ -9474,6 +9544,326 @@ - (int) fullscreenState
return fs_state;
}
@@ -144,23 +136,26 @@ index 932d209..913831b 100644
+
+ EmacsView implements the NSAccessibility protocol so that:
+ - macOS Zoom can query cursor position (accessibilityBoundsForRange:)
+ - VoiceOver can identify the view as a text area
+ - VoiceOver can read buffer contents and track cursor
+ - Accessibility Inspector shows correct element hierarchy
+
+ The primary Zoom tracking mechanism is UAZoomChangeFocus() in
+ ns_draw_window_cursor above. These methods provide supplementary
+ support for AT tools that query the accessibility tree directly.
+ ns_draw_window_cursor above. These methods serve VoiceOver and
+ other AT tools that query the accessibility tree directly.
+
+ Both the modern protocol API (accessibilityBoundsForRange:, 10.10+)
+ and the legacy parameterized attribute API (AXBoundsForRange) are
+ implemented for compatibility with all macOS versions and AT tools.
+ Text content methods (accessibilityValue, accessibilitySelectedText,
+ accessibilityInsertionPointLineNumber, accessibilityVisibleCharacterRange,
+ accessibilityNumberOfCharacters) read from the current buffer, mirroring
+ the pattern in EmacsWindow's accessibilityAttributeValue: but at the
+ view level where VoiceOver expects to find them.
+
+ Ref: https://developer.apple.com/documentation/appkit/nsaccessibilityprotocol
+ Ref: https://developer.apple.com/documentation/appkit/nsaccessibilityboundsforrangeparameterizedattribute
+
+ Note: upstream EmacsWindow has separate accessibility code that
+ returns buffer text content. That code operates on the window,
+ not the view, so there is no conflict.
+ Note: EmacsWindow has a separate accessibilityAttributeValue: that
+ returns buffer text via the legacy API. These EmacsView methods use
+ the modern protocol API (10.10+) and operate on the view level where
+ VoiceOver resolves the text area role. No conflict.
+ ---------------------------------------------------------------- */
+
+- (BOOL)accessibilityIsIgnored
@@ -185,11 +180,164 @@ index 932d209..913831b 100644
+ return NSAccessibilityTextAreaRole;
+}
+
+- (NSString *)accessibilityRoleDescription
+{
+ return NSAccessibilityRoleDescription (NSAccessibilityTextAreaRole, nil);
+}
+
+/* ---- Text content methods for VoiceOver ---- */
+
+- (id)accessibilityValue
+{
+ /* Return visible buffer text. VoiceOver reads this when the user
+ navigates to the text area. */
+ if (!emacsframe)
+ return @"";
+
+ struct buffer *curbuf
+ = XBUFFER (XWINDOW (emacsframe->selected_window)->contents);
+ if (!curbuf)
+ return @"";
+
+ ptrdiff_t start_byte = BUF_BEGV_BYTE (curbuf);
+ ptrdiff_t byte_range = BUF_ZV_BYTE (curbuf) - start_byte;
+ ptrdiff_t range = BUF_ZV (curbuf) - BUF_BEGV (curbuf);
+
+ /* Limit to 10000 chars to avoid performance issues with large buffers. */
+ if (range > 10000)
+ {
+ range = 10000;
+ byte_range = 10000;
+ }
+
+ Lisp_Object str;
+ if (! NILP (BVAR (curbuf, enable_multibyte_characters)))
+ str = make_uninit_multibyte_string (range, byte_range);
+ else
+ str = make_uninit_string (range);
+ memcpy (SDATA (str), BYTE_POS_ADDR (start_byte), byte_range);
+
+ return [NSString stringWithLispString:str];
+}
+
+- (NSInteger)accessibilityNumberOfCharacters
+{
+ if (!emacsframe)
+ return 0;
+
+ struct buffer *curbuf
+ = XBUFFER (XWINDOW (emacsframe->selected_window)->contents);
+ if (!curbuf)
+ return 0;
+
+ ptrdiff_t range = BUF_ZV (curbuf) - BUF_BEGV (curbuf);
+ return (NSInteger) MIN (range, 10000);
+}
+
+- (NSString *)accessibilitySelectedText
+{
+ if (!emacsframe)
+ return @"";
+
+ struct buffer *curbuf
+ = XBUFFER (XWINDOW (emacsframe->selected_window)->contents);
+ if (!curbuf || NILP (BVAR (curbuf, mark_active)))
+ return @"";
+
+ Lisp_Object str = ns_get_local_selection (QPRIMARY, QUTF8_STRING);
+ if (CONSP (str) && SYMBOLP (XCAR (str)))
+ {
+ str = XCDR (str);
+ if (CONSP (str) && NILP (XCDR (str)))
+ str = XCAR (str);
+ }
+ if (STRINGP (str))
+ return [NSString stringWithLispString:str];
+
+ return @"";
+}
+
+- (NSRange)accessibilitySelectedTextRange
+{
+ if (!emacsframe)
+ return NSMakeRange (0, 0);
+
+ struct buffer *curbuf
+ = XBUFFER (XWINDOW (emacsframe->selected_window)->contents);
+ if (!curbuf)
+ return NSMakeRange (0, 0);
+
+ /* Return cursor position as collapsed selection. */
+ ptrdiff_t pt = BUF_PT (curbuf) - BUF_BEGV (curbuf);
+ return NSMakeRange ((NSUInteger) pt, 0);
+}
+
+- (NSInteger)accessibilityInsertionPointLineNumber
+{
+ if (!emacsframe)
+ return 0;
+
+ struct window *w = XWINDOW (emacsframe->selected_window);
+ if (!w)
+ return 0;
+
+ /* Return cursor line relative to window start. */
+ return (NSInteger) (w->cursor.vpos);
+}
+
+- (NSRange)accessibilityVisibleCharacterRange
+{
+ if (!emacsframe)
+ return NSMakeRange (0, 0);
+
+ struct buffer *curbuf
+ = XBUFFER (XWINDOW (emacsframe->selected_window)->contents);
+ if (!curbuf)
+ return NSMakeRange (0, 0);
+
+ ptrdiff_t range = BUF_ZV (curbuf) - BUF_BEGV (curbuf);
+ return NSMakeRange (0, (NSUInteger) MIN (range, 10000));
+}
+
+- (NSString *)accessibilityStringForRange:(NSRange)nsrange
+{
+ if (!emacsframe)
+ return @"";
+
+ struct buffer *curbuf
+ = XBUFFER (XWINDOW (emacsframe->selected_window)->contents);
+ if (!curbuf)
+ return @"";
+
+ ptrdiff_t start = BUF_BEGV (curbuf) + (ptrdiff_t) nsrange.location;
+ ptrdiff_t end = start + (ptrdiff_t) nsrange.length;
+ ptrdiff_t buf_end = BUF_ZV (curbuf);
+
+ if (start < BUF_BEGV (curbuf)) start = BUF_BEGV (curbuf);
+ if (end > buf_end) end = buf_end;
+ if (start >= end) return @"";
+
+ ptrdiff_t start_byte = buf_charpos_to_bytepos (curbuf, start);
+ ptrdiff_t end_byte = buf_charpos_to_bytepos (curbuf, end);
+ ptrdiff_t range = end - start;
+ ptrdiff_t byte_range = end_byte - start_byte;
+
+ Lisp_Object str;
+ if (! NILP (BVAR (curbuf, enable_multibyte_characters)))
+ str = make_uninit_multibyte_string (range, byte_range);
+ else
+ str = make_uninit_string (range);
+ memcpy (SDATA (str), BUF_BYTE_ADDRESS (curbuf, start_byte), byte_range);
+
+ return [NSString stringWithLispString:str];
+}
+
+/* ---- Cursor position methods for Zoom and VoiceOver ---- */
+
+- (NSRect)accessibilityFrame
+{
+ /* Return the cursor's screen coordinates as the view's accessibility
+ frame. This allows AT tools that query accessibilityFrame (rather
+ than accessibilityBoundsForRange:) to locate the cursor.
+ /* Return the cursor's screen coordinates. This is the key method
+ that macOS Zoom reads after receiving a focus/selection notification.
+
+ lastAccessibilityCursorRect is in EmacsView coordinates (flipped:
+ origin top-left). convertRect:toView:nil handles the
@@ -213,8 +361,12 @@ index 932d209..913831b 100644
+ superAttrs = @[];
+ return [superAttrs arrayByAddingObjectsFromArray:
+ @[NSAccessibilityRoleAttribute,
+ NSAccessibilityValueAttribute,
+ NSAccessibilitySelectedTextAttribute,
+ NSAccessibilitySelectedTextRangeAttribute,
+ NSAccessibilityNumberOfCharactersAttribute]];
+ NSAccessibilityNumberOfCharactersAttribute,
+ NSAccessibilityVisibleCharacterRangeAttribute,
+ NSAccessibilityInsertionPointLineNumberAttribute]];
+}
+
+- (id)accessibilityAttributeValue:(NSString *)attribute
@@ -222,14 +374,23 @@ index 932d209..913831b 100644
+ if ([attribute isEqualToString:NSAccessibilityRoleAttribute])
+ return NSAccessibilityTextAreaRole;
+
+ /* Zoom queries SelectedTextRange before calling BoundsForRange.
+ We return {0,0} (collapsed caret); our bounds methods ignore
+ the range parameter and always return the actual cursor rect. */
+ if ([attribute isEqualToString:NSAccessibilityValueAttribute])
+ return [self accessibilityValue];
+
+ if ([attribute isEqualToString:NSAccessibilitySelectedTextAttribute])
+ return [self accessibilitySelectedText];
+
+ if ([attribute isEqualToString:NSAccessibilitySelectedTextRangeAttribute])
+ return [NSValue valueWithRange:NSMakeRange (0, 0)];
+ return [NSValue valueWithRange:[self accessibilitySelectedTextRange]];
+
+ if ([attribute isEqualToString:NSAccessibilityNumberOfCharactersAttribute])
+ return @(0);
+ return @([self accessibilityNumberOfCharacters]);
+
+ if ([attribute isEqualToString:NSAccessibilityVisibleCharacterRangeAttribute])
+ return [NSValue valueWithRange:[self accessibilityVisibleCharacterRange]];
+
+ if ([attribute isEqualToString:NSAccessibilityInsertionPointLineNumberAttribute])
+ return @([self accessibilityInsertionPointLineNumber]);
+
+ return [super accessibilityAttributeValue:attribute];
+}
@@ -239,7 +400,7 @@ index 932d209..913831b 100644
+- (NSRect)accessibilityBoundsForRange:(NSRange)range
+{
+ /* Return cursor screen rect regardless of requested range.
+ Emacs does not expose a character-level text model to AppKit,
+ Emacs does not expose a character-level geometry model to AppKit,
+ so we always return the cursor position. */
+ NSRect viewRect = lastAccessibilityCursorRect;
+
@@ -264,7 +425,8 @@ index 932d209..913831b 100644
+ if (superAttrs == nil)
+ superAttrs = @[];
+ return [superAttrs arrayByAddingObjectsFromArray:
+ @[NSAccessibilityBoundsForRangeParameterizedAttribute]];
+ @[NSAccessibilityBoundsForRangeParameterizedAttribute,
+ NSAccessibilityStringForRangeParameterizedAttribute]];
+}
+
+- (id)accessibilityAttributeValue:(NSString *)attribute
@@ -275,6 +437,13 @@ index 932d209..913831b 100644
+ return [NSValue valueWithRect:
+ [self accessibilityBoundsForRange:NSMakeRange (0, 0)]];
+
+ if ([attribute isEqualToString:
+ NSAccessibilityStringForRangeParameterizedAttribute])
+ {
+ NSRange range = [(NSValue *) parameter rangeValue];
+ return [self accessibilityStringForRange:range];
+ }
+
+ return [super accessibilityAttributeValue:attribute forParameter:parameter];
+}
+#endif /* NS_IMPL_COCOA */