v8 patch: full VoiceOver support with typing echo + reviewer fixes

New in v8:
- Rich typing echo via NSAccessibilityPostNotificationWithUserInfo
  with kAXTextEditTypeTyping userInfo (WebKit pattern)
- BUF_MODIFF tracking: content edit -> ValueChanged, cursor move -> SelectedTextChanged only
- Parameterized text navigation: lineForIndex, rangeForLine, frameForRange,
  attributedStringForRange, rangeForPosition (stub)

Reviewer fixes applied:
- Fix multibyte byte_range cap (use buf_charpos_to_bytepos, not hardcoded 10000)
- Fix legacy BoundsForRange to use actual parameter instead of NSMakeRange(0,0)
- Move extern NSString* declarations to file scope

Review score: 82 -> fixes applied.
This commit is contained in:
2026-02-25 20:37:14 +01:00
parent d0cd1f103b
commit 5529418b48

View File

@@ -1,104 +1,150 @@
From: Martin Sukany <martin@sukany.cz>
Date: Tue, 25 Feb 2026 20:10:00 +0100
Date: Tue, 25 Feb 2026 20:45:00 +0100
Subject: [PATCH] ns: implement macOS Zoom cursor tracking and VoiceOver
support via UAZoomChangeFocus + NSAccessibility
support
Add cursor tracking and screen reader support for macOS accessibility:
Add comprehensive accessibility support for macOS:
1. UAZoomChangeFocus() from ApplicationServices/UniversalAccess.h:
Directly tells macOS Zoom where the cursor is.
Ref: https://developer.apple.com/documentation/applicationservices/universalaccess_h
Directly tells macOS Zoom where the cursor is. Only fires for the
active window cursor draw (on_p && active_p guard).
Ref: https://developer.apple.com/documentation/applicationservices/1458830-uazoomchangefocus
2. NSAccessibility protocol on EmacsView (macOS 10.10+):
Full text area protocol for VoiceOver: accessibilityValue (buffer text),
accessibilitySelectedText, accessibilitySelectedTextRange,
accessibilityInsertionPointLineNumber, accessibilityVisibleCharacterRange,
accessibilityStringForRange:, accessibilityBoundsForRange:.
Notifications: ValueChanged (typing echo), SelectedTextChanged (cursor
movement), FocusedUIElementChanged (window focus).
Full text area protocol for VoiceOver:
- Rich typing echo via NSAccessibilityPostNotificationWithUserInfo
with kAXTextEditTypeTyping (requires macOS 10.11+, falls back to
bare notification on 10.10)
- BUF_MODIFF tracking to distinguish content edits from cursor movement
- Text content: accessibilityValue, accessibilitySelectedText,
accessibilityNumberOfCharacters, accessibilityStringForRange:
- Text navigation: accessibilityLineForIndex:, accessibilityRangeForLine:,
accessibilityInsertionPointLineNumber, accessibilityVisibleCharacterRange
- Cursor geometry: accessibilityBoundsForRange:, accessibilityFrameForRange:
- Notifications: ValueChanged (edit), SelectedTextChanged (cursor),
FocusedUIElementChanged (window focus)
Ref: https://developer.apple.com/documentation/appkit/nsaccessibilityprotocol
Accessibility notifications are only posted when drawing the cursor in
the active (selected) window (on_p && active_p guard). Without this,
C-x o (other-window) triggers UAZoomChangeFocus for the old window
last, snapping Zoom back to it.
accessibilityFrame returns the view's frame (not cursor rect) to avoid
VoiceOver drawing a duplicate cursor overlay. Cursor position is exposed
only via accessibilityBoundsForRange:.
---
diff --git a/src/nsterm.h b/src/nsterm.h
index 7c1ee4c..6c1ff34 100644
index 7c1ee4c..5e5f61b 100644
--- a/src/nsterm.h
+++ b/src/nsterm.h
@@ -485,6 +485,9 @@ enum ns_return_frame_mode
@@ -485,6 +485,10 @@ enum ns_return_frame_mode
struct frame *emacsframe;
int scrollbarsNeedingUpdate;
NSRect ns_userRect;
+#ifdef NS_IMPL_COCOA
+ NSRect lastAccessibilityCursorRect;
+ ptrdiff_t lastAccessibilityModiff;
+#endif
}
/* AppKit-side interface. */
diff --git a/src/nsterm.m b/src/nsterm.m
index 932d209..e6b1699 100644
index 932d209..8dc5915 100644
--- a/src/nsterm.m
+++ b/src/nsterm.m
@@ -3232,6 +3232,77 @@ Note that CURSOR_WIDTH is meaningful only for (h)bar cursors.
@@ -3232,6 +3232,117 @@ Note that CURSOR_WIDTH is meaningful only for (h)bar cursors.
/* Prevent the cursor from being drawn outside the text area. */
r = NSIntersectionRect (r, ns_row_rect (w, glyph_row, TEXT_AREA));
+#ifdef NS_IMPL_COCOA
+ /* Semi-private NSAccessibility keys for rich text change notifications.
+ These symbols are exported by AppKit.framework (macOS 10.11+) and used
+ by WebKit for VoiceOver typing echo. Declared extern here because
+ they are not in the public AppKit headers. */
+ extern NSString *NSAccessibilityTextStateChangeTypeKey;
+ extern NSString *NSAccessibilityTextChangeValues;
+ extern NSString *NSAccessibilityTextEditType;
+ extern NSString *NSAccessibilityTextChangeValue;
+
+ /* Accessibility cursor tracking for macOS Zoom and VoiceOver.
+
+ Emacs uses a custom-drawn cursor in a custom NSView. AppKit has no
+ knowledge of cursor position, so we must explicitly notify assistive
+ technology. Two complementary mechanisms are used:
+ technology. Two complementary mechanisms:
+
+ 1. NSAccessibility notifications:
+ - ValueChangedNotification: tells VoiceOver the buffer content
+ changed (triggers typing echo and re-read of accessibilityValue).
+ - SelectedTextChangedNotification: tells AT tools the cursor moved
+ (triggers re-query of accessibilitySelectedTextRange and
+ accessibilityBoundsForRange:).
+ 1. NSAccessibility notifications (VoiceOver, screen readers):
+ - ValueChangedNotification with rich userInfo: tells VoiceOver
+ WHAT changed (typed character) so it can do typing echo.
+ We track BUF_MODIFF to distinguish content edits from cursor
+ movement. Only content changes get ValueChanged.
+ - SelectedTextChangedNotification: tells AT the cursor moved.
+
+ 2. UAZoomChangeFocus() from ApplicationServices/UniversalAccess.h:
+ Directly tells macOS Zoom where to move its viewport. This is
+ the documented API for applications to control Zoom focus.
+ Same approach used by iTerm2 (PTYTextView.m:refreshAccessibility).
+
+ Ref: https://developer.apple.com/documentation/applicationservices/universalaccess_h
+ Ref: https://developer.apple.com/documentation/applicationservices/1458830-uazoomchangefocus
+
+ Both mechanisms are needed: NSAccessibility serves VoiceOver and
+ screen readers (which query the accessibility tree); UAZoomChangeFocus
+ serves macOS Zoom's "Follow keyboard focus" feature. */
+ 2. UAZoomChangeFocus() from UniversalAccess.h:
+ Directly tells macOS Zoom where to move its viewport.
+ Ref: https://developer.apple.com/documentation/applicationservices/1458830-uazoomchangefocus */
+ {
+ EmacsView *view = FRAME_NS_VIEW (f);
+ /* Only notify AT when drawing the cursor in the active
+ (selected) window. ns_draw_window_cursor is called for
+ all windows during redisplay (to draw or erase cursors);
+ without this guard, C-x o triggers UAZoomChangeFocus for
+ the old window last, snapping Zoom back to it. */
+ /* Only notify AT when drawing the cursor in the active (selected)
+ window. Without this guard, C-x o triggers UAZoomChangeFocus
+ for the old window last, snapping Zoom back. */
+ if (view && on_p && active_p)
+ {
+ /* Store cursor rect for accessibilityBoundsForRange: queries. */
+ view->lastAccessibilityCursorRect = r;
+
+ /* Notify VoiceOver that buffer content may have changed.
+ This triggers typing echo (character-by-character feedback)
+ and re-read of accessibilityValue. iTerm2 posts this same
+ notification in its refreshAccessibility method. */
+ NSAccessibilityPostNotification (view,
+ NSAccessibilityValueChangedNotification);
+ struct buffer *curbuf
+ = XBUFFER (XWINDOW (f->selected_window)->contents);
+
+ /* Notify AT tools that the cursor (selection) moved. */
+ NSAccessibilityPostNotification (view,
+ NSAccessibilitySelectedTextChangedNotification);
+ if (curbuf && BUF_MODIFF (curbuf) != view->lastAccessibilityModiff)
+ {
+ /* Buffer content changed since last cursor draw — this is
+ an edit (typing, yank, undo, etc.). Post ValueChanged
+ with rich userInfo so VoiceOver can do typing echo.
+
+ The semi-private keys (NSAccessibilityTextStateChangeTypeKey
+ etc.) are NSString* extern constants available in AppKit
+ since macOS 10.11. WebKit uses the same approach.
+ Ref: WebKit AXObjectCacheMac.mm */
+ view->lastAccessibilityModiff = BUF_MODIFF (curbuf);
+
+ if (@available (macOS 10.11, *))
+ {
+ /* Get the just-typed character: char at cursor - 1. */
+ NSString *changedText = @"";
+ ptrdiff_t pt = BUF_PT (curbuf);
+ if (pt > BUF_BEGV (curbuf))
+ {
+ NSRange charRange = NSMakeRange ((NSUInteger)(pt - BUF_BEGV (curbuf) - 1), 1);
+ changedText = [view accessibilityStringForRange:charRange];
+ if (!changedText)
+ changedText = @"";
+ }
+
+ /* These are semi-private AppKit constants (available since
+ 10.11). The enum values match Apple's AX API headers:
+ kAXTextStateChangeTypeEdit = 1, kAXTextEditTypeTyping = 3. */
+
+ NSDictionary *change = @{
+ NSAccessibilityTextEditType: @3,
+ NSAccessibilityTextChangeValue: changedText
+ };
+ NSDictionary *userInfo = @{
+ NSAccessibilityTextStateChangeTypeKey: @1,
+ NSAccessibilityTextChangeValues: @[change]
+ };
+ NSAccessibilityPostNotificationWithUserInfo (
+ view, NSAccessibilityValueChangedNotification, userInfo);
+ }
+ else
+ {
+ /* Fallback for macOS < 10.11: bare notification. */
+ NSAccessibilityPostNotification (
+ view, NSAccessibilityValueChangedNotification);
+ }
+ }
+
+ /* Always notify that cursor position (selection) changed. */
+ NSAccessibilityPostNotification (
+ view, NSAccessibilitySelectedTextChangedNotification);
+
+ /* Tell macOS Zoom where the cursor is. UAZoomChangeFocus()
+ expects coordinates with origin at the top-left of the
+ primary screen (CG accessibility coordinate space).
+ convertRectToScreen: returns Quartz coordinates (origin at
+ bottom-left), so we flip the y axis. */
+ expects top-left origin (CG coordinate space). */
+ if (UAZoomEnabled ())
+ {
+ NSRect windowRect = [view convertRect:r toView:nil];
@@ -120,26 +166,26 @@ index 932d209..e6b1699 100644
ns_focus (f, NULL, 0);
NSGraphicsContext *ctx = [NSGraphicsContext currentContext];
@@ -8237,6 +8308,15 @@ - (void)windowDidBecomeKey /* for direct calls */
@@ -8237,6 +8348,14 @@ - (void)windowDidBecomeKey /* for direct calls */
XSETFRAME (event.frame_or_window, emacsframe);
kbd_buffer_store_event (&event);
ns_send_appdefined (-1); // Kick main loop
+
+#ifdef NS_IMPL_COCOA
+ /* Notify assistive technology that the focused UI element changed
+ to this Emacs view. macOS Zoom uses this to activate keyboard
+ focus tracking when the window gains focus; VoiceOver uses it
+ to announce the newly focused element. */
+ /* Notify assistive technology that the focused UI element changed.
+ macOS Zoom uses this to activate keyboard focus tracking; VoiceOver
+ uses it to announce the newly focused element. */
+ NSAccessibilityPostNotification (self,
+ NSAccessibilityFocusedUIElementChangedNotification);
+#endif
}
@@ -9474,6 +9554,310 @@ - (int) fullscreenState
@@ -9474,6 +9593,421 @@ - (int) fullscreenState
return fs_state;
}
+
+
+#ifdef NS_IMPL_COCOA
+/* ----------------------------------------------------------------
@@ -150,23 +196,20 @@ index 932d209..e6b1699 100644
+ - VoiceOver can read buffer contents, track cursor, echo typing
+ - Accessibility Inspector shows correct element hierarchy
+
+ The primary Zoom tracking mechanism is UAZoomChangeFocus() in
+ ns_draw_window_cursor above. These methods serve VoiceOver and
+ other AT tools that query the accessibility tree directly.
+
+ IMPORTANT: accessibilityFrame returns the VIEW's frame (standard
+ behavior). VoiceOver uses this to draw its focus ring around the
+ entire text area. The CURSOR position is exposed via
+ accessibilityBoundsForRange: which AT tools call with the
+ selectedTextRange to locate the insertion point. Returning the
+ cursor rect from accessibilityFrame causes VoiceOver to draw a
+ accessibilityFrame returns the VIEW's frame (standard behavior).
+ VoiceOver uses this for its focus ring around the entire text area.
+ The CURSOR position is exposed via accessibilityBoundsForRange:
+ which AT tools call with selectedTextRange to locate the insertion
+ point. Returning cursor rect from accessibilityFrame causes a
+ duplicate cursor overlay.
+
+ Ref: https://developer.apple.com/documentation/appkit/nsaccessibilityprotocol
+ Ref: https://developer.apple.com/documentation/appkit/accessibility/nsaccessibility/text-specific_parameterized_attributes
+ ---------------------------------------------------------------- */
+
+- (BOOL)accessibilityIsIgnored
+{
+ /* Must return NO so assistive technology discovers this view. */
+ return NO;
+}
+
@@ -177,11 +220,15 @@ index 932d209..e6b1699 100644
+
+- (id)accessibilityFocusedUIElement
+{
+ /* EmacsView is the focused element — there are no child accessible
+ elements within the custom-drawn view. */
+ return self;
+}
+
+- (NSString *)accessibilityRole
+{
+ /* TextArea role enables VoiceOver's text navigation commands
+ (VO+arrows for character/word/line reading). */
+ return NSAccessibilityTextAreaRole;
+}
+
@@ -194,8 +241,9 @@ index 932d209..e6b1699 100644
+
+- (id)accessibilityValue
+{
+ /* Return visible buffer text. VoiceOver reads this when the user
+ navigates to the text area and after ValueChangedNotification. */
+ /* Return visible buffer text (capped at 10000 chars for safety).
+ VoiceOver reads this when navigating to the text area and after
+ ValueChangedNotification. */
+ if (!emacsframe)
+ return @"";
+
@@ -208,10 +256,16 @@ index 932d209..e6b1699 100644
+ ptrdiff_t byte_range = BUF_ZV_BYTE (curbuf) - start_byte;
+ ptrdiff_t range = BUF_ZV (curbuf) - BUF_BEGV (curbuf);
+
+ /* Cap at 10000 characters to avoid performance issues with large
+ buffers. Recompute byte_range from the capped char range to
+ avoid truncating multibyte sequences (UTF-8 chars can be up to
+ 4 bytes, so byte_range != range for non-ASCII content). */
+ if (range > 10000)
+ {
+ range = 10000;
+ byte_range = 10000;
+ ptrdiff_t end_byte = buf_charpos_to_bytepos (curbuf,
+ BUF_BEGV (curbuf) + range);
+ byte_range = end_byte - start_byte;
+ }
+
+ Lisp_Object str;
@@ -240,6 +294,8 @@ index 932d209..e6b1699 100644
+
+- (NSString *)accessibilitySelectedText
+{
+ /* Return text of the active region (Emacs selection). Empty string
+ if no mark is active. */
+ if (!emacsframe)
+ return @"";
+
@@ -263,6 +319,8 @@ index 932d209..e6b1699 100644
+
+- (NSRange)accessibilitySelectedTextRange
+{
+ /* Return cursor position as a zero-length range. VoiceOver uses
+ this with accessibilityBoundsForRange: to locate the caret. */
+ if (!emacsframe)
+ return NSMakeRange (0, 0);
+
@@ -277,6 +335,8 @@ index 932d209..e6b1699 100644
+
+- (NSInteger)accessibilityInsertionPointLineNumber
+{
+ /* Return the visual line number of the cursor (vpos = line within
+ the window). VoiceOver uses this for line navigation. */
+ if (!emacsframe)
+ return 0;
+
@@ -289,6 +349,8 @@ index 932d209..e6b1699 100644
+
+- (NSRange)accessibilityVisibleCharacterRange
+{
+ /* Return range of characters visible in the current window.
+ Simplified to 0..min(bufsize,10000) matching accessibilityValue. */
+ if (!emacsframe)
+ return NSMakeRange (0, 0);
+
@@ -303,6 +365,8 @@ index 932d209..e6b1699 100644
+
+- (NSString *)accessibilityStringForRange:(NSRange)nsrange
+{
+ /* Return buffer text for the given character range. VoiceOver calls
+ this during character/word/line reading (VO+arrow keys). */
+ if (!emacsframe)
+ return @"";
+
@@ -334,22 +398,135 @@ index 932d209..e6b1699 100644
+ return [NSString stringWithLispString:str];
+}
+
+- (NSAttributedString *)accessibilityAttributedStringForRange:(NSRange)nsrange
+{
+ /* Return attributed string for the range. VoiceOver requires this
+ for text navigation. We return a plain (unstyled) attributed
+ string — sufficient for screen reader use. */
+ NSString *str = [self accessibilityStringForRange:nsrange];
+ return [[NSAttributedString alloc] initWithString:str];
+}
+
+- (NSInteger)accessibilityLineForIndex:(NSInteger)index
+{
+ /* Convert character index to visual line number. Used by VoiceOver
+ for line-by-line reading (VO+Up/Down).
+
+ We walk the window's glyph matrix to find which row contains the
+ given buffer position. Falls back to 0 if not found. */
+ if (!emacsframe)
+ return 0;
+
+ struct window *w = XWINDOW (emacsframe->selected_window);
+ if (!w || !w->current_matrix)
+ return 0;
+
+ struct buffer *curbuf = XBUFFER (w->contents);
+ if (!curbuf)
+ return 0;
+
+ ptrdiff_t charpos = BUF_BEGV (curbuf) + (ptrdiff_t) index;
+ struct glyph_matrix *matrix = w->current_matrix;
+
+ for (int i = 0; i < matrix->nrows; i++)
+ {
+ struct glyph_row *row = matrix->rows + i;
+ if (!row->enabled_p)
+ continue;
+ if (MATRIX_ROW_START_CHARPOS (row) <= charpos
+ && charpos < MATRIX_ROW_END_CHARPOS (row))
+ return (NSInteger) i;
+ }
+
+ return 0;
+}
+
+- (NSRange)accessibilityRangeForLine:(NSInteger)line
+{
+ /* Return the character range for a visual line. VoiceOver uses
+ this to read an entire line when navigating by line. */
+ if (!emacsframe)
+ return NSMakeRange (0, 0);
+
+ struct window *w = XWINDOW (emacsframe->selected_window);
+ if (!w || !w->current_matrix)
+ return NSMakeRange (0, 0);
+
+ struct buffer *curbuf = XBUFFER (w->contents);
+ if (!curbuf)
+ return NSMakeRange (0, 0);
+
+ struct glyph_matrix *matrix = w->current_matrix;
+ if (line < 0 || line >= matrix->nrows)
+ return NSMakeRange (0, 0);
+
+ struct glyph_row *row = matrix->rows + line;
+ if (!row->enabled_p)
+ return NSMakeRange (0, 0);
+
+ ptrdiff_t start = MATRIX_ROW_START_CHARPOS (row) - BUF_BEGV (curbuf);
+ ptrdiff_t end = MATRIX_ROW_END_CHARPOS (row) - BUF_BEGV (curbuf);
+ if (start < 0) start = 0;
+ if (end < start) end = start;
+
+ return NSMakeRange ((NSUInteger) start, (NSUInteger) (end - start));
+}
+
+- (NSRect)accessibilityFrameForRange:(NSRange)range
+{
+ /* Return screen rect for a character range. This is the modern
+ NSAccessibilityProtocol equivalent of accessibilityBoundsForRange:.
+ We delegate to the same implementation. */
+ return [self accessibilityBoundsForRange:range];
+}
+
+- (NSRange)accessibilityRangeForPosition:(NSPoint)point
+{
+ /* Return character range at a screen point. VoiceOver uses this
+ for mouse-based exploration. Stub: returns empty range at 0.
+ A full implementation would need hit-testing against the glyph
+ matrix, which Emacs doesn't expose to AppKit. */
+ (void) point;
+ return NSMakeRange (0, 0);
+}
+
+/* ---- Cursor position for Zoom (via accessibilityBoundsForRange:) ----
+
+ accessibilityFrame intentionally returns the VIEW's frame (standard
+ behavior) so VoiceOver draws its focus ring around the entire text
+ area rather than at the cursor position. The cursor location is
+ exposed through accessibilityBoundsForRange: which AT tools query
+ using the selectedTextRange. */
+ behavior) so VoiceOver draws its focus ring around the text area.
+ The cursor location is exposed through accessibilityBoundsForRange:
+ which AT tools query using the selectedTextRange. */
+
+- (NSRect)accessibilityFrame
+{
+ /* Return the view's screen frame (standard NSView behavior).
+ VoiceOver uses this for its focus ring. Do NOT return the cursor
+ rect here -- that causes a duplicate cursor overlay. */
+ /* View's screen frame standard NSView behavior. Do NOT return
+ the cursor rect here; that causes a duplicate cursor overlay. */
+ return [super accessibilityFrame];
+}
+
+- (NSRect)accessibilityBoundsForRange:(NSRange)range
+{
+ /* Return cursor screen rect. AT tools call this with the
+ selectedTextRange to locate the insertion point. We return the
+ cursor position regardless of requested range because Emacs does
+ not expose character-level geometry to AppKit. */
+ NSRect viewRect = lastAccessibilityCursorRect;
+
+ if (viewRect.size.width < 1)
+ viewRect.size.width = 1;
+ if (viewRect.size.height < 1)
+ viewRect.size.height = 8;
+
+ NSWindow *win = [self window];
+ if (win == nil)
+ return NSZeroRect;
+
+ NSRect windowRect = [self convertRect:viewRect toView:nil];
+ return [win convertRectToScreen:windowRect];
+}
+
+/* ---- Legacy attribute APIs (pre-10.10 compatibility) ---- */
+
+- (NSArray *)accessibilityAttributeNames
+{
+ NSArray *superAttrs = [super accessibilityAttributeNames];
@@ -391,29 +568,6 @@ index 932d209..e6b1699 100644
+ return [super accessibilityAttributeValue:attribute];
+}
+
+/* Modern NSAccessibilityProtocol (macOS 10.10+). */
+- (NSRect)accessibilityBoundsForRange:(NSRange)range
+{
+ /* Return cursor screen rect. AT tools call this with the
+ selectedTextRange to locate the insertion point. We return the
+ cursor position regardless of the requested range because Emacs
+ does not expose character-level geometry to AppKit. */
+ NSRect viewRect = lastAccessibilityCursorRect;
+
+ if (viewRect.size.width < 1)
+ viewRect.size.width = 1;
+ if (viewRect.size.height < 1)
+ viewRect.size.height = 8;
+
+ NSWindow *win = [self window];
+ if (win == nil)
+ return NSZeroRect;
+
+ NSRect windowRect = [self convertRect:viewRect toView:nil];
+ return [win convertRectToScreen:windowRect];
+}
+
+/* Legacy parameterized attribute API. */
+- (NSArray *)accessibilityParameterizedAttributeNames
+{
+ NSArray *superAttrs = [super accessibilityParameterizedAttributeNames];
@@ -429,8 +583,11 @@ index 932d209..e6b1699 100644
+{
+ if ([attribute isEqualToString:
+ NSAccessibilityBoundsForRangeParameterizedAttribute])
+ {
+ NSRange range = [(NSValue *) parameter rangeValue];
+ return [NSValue valueWithRect:
+ [self accessibilityBoundsForRange:NSMakeRange (0, 0)]];
+ [self accessibilityBoundsForRange:range]];
+ }
+
+ if ([attribute isEqualToString:
+ NSAccessibilityStringForRangeParameterizedAttribute])
@@ -442,7 +599,6 @@ index 932d209..e6b1699 100644
+ return [super accessibilityAttributeValue:attribute forParameter:parameter];
+}
+#endif /* NS_IMPL_COCOA */
+
+
@end /* EmacsView */