Merge pull request #125 from xcxmiku/feature/private-display-recorder

Use private display capture for recording and screenshots
This commit is contained in:
Lakr
2026-03-07 19:30:32 +08:00
4 changed files with 290 additions and 26 deletions

View File

@@ -177,3 +177,47 @@
| PCC 26.3 (`23D128`) | 14 | 59 |
| iOS 26.1 (`23B85`) | 14 | 59 |
| iOS 26.3 (`23D127`) | 14 | 59 |
## Automation Notes (2026-03-06)
- `scripts/setup_machine.sh` non-interactive flow fix: renamed local variable `status` to `boot_state` in first-boot log wait and boot-analysis wait helpers to avoid zsh `status` read-only special parameter collision.
- `scripts/setup_machine.sh` non-interactive first-boot wait fix: replaced `(( waited++ ))` with `(( ++waited ))` in `monitor_boot_log_until` to avoid `set -e` abort when arithmetic expression evaluates to `0`.
- `scripts/jb_patch_autotest.sh` loop fix for sweep stability under `set -e`: replaced `((idx++))` with `(( ++idx ))`.
- `scripts/jb_patch_autotest.sh` zsh compatibility fix: renamed per-case result variable `status` to `case_status` to avoid `status` read-only special parameter collision.
- `scripts/jb_patch_autotest.sh` selection logic update:
- default run now excludes methods listed in `KernelJBPatcher._DEV_SINGLE_WORKING_METHODS` (pending-only sweep).
- set `JB_AUTOTEST_INCLUDE_WORKING=1` to include already-working methods and run the full list.
- Sweep run record:
- `setup_logs/jb_patch_tests_20260306_114417` (2026-03-06): aborted at `[1/20]` with `read-only variable: status` in `jb_patch_autotest.sh`.
- `setup_logs/jb_patch_tests_20260306_115027` (2026-03-06): rerun after `status` fix, pending-only mode (`Total methods: 19`).
- Final run result from `jb_patch_tests_20260306_115027` at `2026-03-06 13:17`:
- Finished: 19/19 (`PASS=15`, `FAIL=4`, all fails `rc=2`).
- Failing methods at that time: `patch_bsd_init_auth`, `patch_io_secure_bsd_root`, `patch_vm_fault_enter_prepare`, `patch_cred_label_update_execve`.
- 2026-03-06 follow-up: `patch_io_secure_bsd_root` failure is now attributed to a wrong-site patch in `AppleARMPE::callPlatformFunction` (`"SecureRoot"` gate at `0xFFFFFE000836E1F0`), not the intended `"SecureRootName"` deny-return path. The code was retargeted the same day to `0xFFFFFE000836E464` and re-enabled for the next restore/boot check.
- 2026-03-06 follow-up: `patch_bsd_init_auth` was retargeted after confirming the old matcher was hitting unrelated code; keep disabled in default schedule until a fresh clean-baseline boot test passes.
- Final case: `[19/19] patch_syscallmask_apply_to_proc` (`PASS`).
- 2026-03-06 re-analysis: that historical `PASS` is now treated as a false positive for functionality, because the recorded bytes landed at `0xfffffe00093ae6e4`/`0xfffffe00093ae6e8` inside `_profile_syscallmask_destroy` underflow handling, not in `_proc_apply_syscall_masks`.
- 2026-03-06 code update: `scripts/patchers/kernel_jb_patch_syscallmask.py` was rebuilt to target the real syscallmask apply wrapper structurally and now dry-runs on `PCC-CloudOS-26.1-23B85 kernelcache.research.vphone600` with 3 writes: `0x02395530`, `0x023955E8`, and cave `0x00AB1720`. User-side boot validation succeeded the same day.
- 2026-03-06 follow-up: `patch_kcall10` was rebuilt from the old ABI-unsafe pseudo-10-arg design into an ABI-correct `sysent[439]` cave. Focused dry-run on `PCC-CloudOS-26.1-23B85 kernelcache.research.vphone600` now emits 4 writes: cave `0x00AB1720`, `sy_call` `0x0073E180`, `sy_arg_munge32` `0x0073E188`, and metadata `0x0073E190`; the method was re-enabled in `_GROUP_C_METHODS`.
- Observed failure symptom in current failing set: first boot panic before command injection (or boot process early exit).
- Post-run schedule change (per user request):
- commented out failing methods from default `KernelJBPatcher._PATCH_METHODS` schedule in `scripts/patchers/kernel_jb.py`:
- `patch_bsd_init_auth`
- `patch_io_secure_bsd_root`
- `patch_vm_fault_enter_prepare`
- `patch_cred_label_update_execve`
- 2026-03-06 re-research note for `patch_cred_label_update_execve`:
- old entry-time early-return strategy was identified as boot-unsafe because it skipped AMFI exec-time `csflags` and entitlement propagation entirely.
- implementation was reworked to a success-tail trampoline that preserves normal AMFI processing and only clears restrictive `csflags` bits on the success path.
- default JB schedule still keeps the method disabled until the reworked strategy is boot-validated.
- Manual DEV+single (`setup_machine` + `PATCH=<method>`) working set now includes:
- `patch_amfi_cdhash_in_trustcache`
- `patch_amfi_execve_kill_path`
- `patch_task_conversion_eval_internal`
- `patch_sandbox_hooks_extended`
- `patch_post_validation_additional`
- 2026-03-07 host-side note:
- reviewed private Virtualization.framework display APIs against the recorder pipeline in `sources/vphone-cli/VPhoneScreenRecorder.swift`.
- replaced the old AppKit-first recorder path with a private-display-only implementation built around hidden `VZGraphicsDisplay._takeScreenshotWithCompletionHandler:` capture.
- added still screenshot actions that can copy the captured image to the pasteboard or save a PNG to disk using the same private capture path.
- `make build` is used as the sanity check path; live VM validation is still needed to confirm the exact screenshot object type returned on macOS 15.

View File

@@ -9,6 +9,9 @@ extension VPhoneMenuController {
let toggle = makeItem("Start Recording", action: #selector(toggleRecording))
recordingItem = toggle
menu.addItem(toggle)
menu.addItem(NSMenuItem.separator())
menu.addItem(makeItem("Copy Screenshot to Clipboard", action: #selector(copyScreenshotToClipboard)))
menu.addItem(makeItem("Save Screenshot to File", action: #selector(saveScreenshotToFile)))
item.submenu = menu
return item
}
@@ -20,18 +23,54 @@ extension VPhoneMenuController {
recordingItem?.title = "Start Recording"
}
} else {
guard let window = NSApp.keyWindow,
let view = window.contentView
else {
print("[record] no active window")
guard let view = activeCaptureView() else {
showAlert(title: "Recording", message: "No active VM window.", style: .warning)
return
}
do {
try screenRecorder?.startRecording(view: view)
recordingItem?.title = "Stop Recording"
} catch {
print("[record] failed to start: \(error)")
showAlert(title: "Recording", message: "\(error)", style: .warning)
}
}
}
@objc func copyScreenshotToClipboard() {
guard let recorder = screenRecorder else { return }
guard let view = activeCaptureView() else {
showAlert(title: "Screenshot", message: "No active VM window.", style: .warning)
return
}
Task { @MainActor in
do {
try await recorder.copyScreenshotToPasteboard(view: view)
showAlert(title: "Screenshot", message: "Copied to clipboard.", style: .informational)
} catch {
showAlert(title: "Screenshot", message: "\(error)", style: .warning)
}
}
}
@objc func saveScreenshotToFile() {
guard let recorder = screenRecorder else { return }
guard let view = activeCaptureView() else {
showAlert(title: "Screenshot", message: "No active VM window.", style: .warning)
return
}
Task { @MainActor in
do {
let url = try await recorder.saveScreenshot(view: view)
showAlert(title: "Screenshot", message: "Saved to \(url.path)", style: .informational)
} catch {
showAlert(title: "Screenshot", message: "\(error)", style: .warning)
}
}
}
private func activeCaptureView() -> NSView? {
NSApp.keyWindow?.contentView ?? NSApp.mainWindow?.contentView
}
}

View File

@@ -1,18 +1,48 @@
import AppKit
import AVFoundation
import CoreVideo
import ObjectiveC.runtime
import Virtualization
// MARK: - Screen Recorder
@MainActor
class VPhoneScreenRecorder {
private enum CaptureError: LocalizedError {
case captureFailed
case clipboardWriteFailed
case pngEncodingFailed
var errorDescription: String? {
switch self {
case .captureFailed:
"Failed to capture a frame from the virtual machine."
case .clipboardWriteFailed:
"Failed to copy the screenshot to the pasteboard."
case .pngEncodingFailed:
"Failed to encode the screenshot as PNG."
}
}
}
private struct CaptureSource {
let graphicsDisplay: VZGraphicsDisplay
let description: String
}
private typealias ScreenshotCompletionBlock = @convention(block) (AnyObject?) -> Void
private typealias ScreenshotIMP = @convention(c) (AnyObject, Selector, AnyObject) -> Void
private var writer: AVAssetWriter?
private var videoInput: AVAssetWriterInput?
private var adaptor: AVAssetWriterInputPixelBufferAdaptor?
private var timer: Timer?
private var frameCount: Int64 = 0
private var outputURL: URL?
private weak var view: NSView?
private var graphicsDisplay: VZGraphicsDisplay?
private var captureModeDescription = "private VZGraphicsDisplay screenshots"
private var screenshotInFlight = false
private var didLogCaptureFailure = false
var isRecording: Bool {
writer?.status == .writing
@@ -21,15 +51,12 @@ class VPhoneScreenRecorder {
func startRecording(view: NSView) throws {
guard !isRecording else { return }
let backingSize = view.convertToBacking(view.bounds.size)
let width = Int(backingSize.width)
let height = Int(backingSize.height)
let source = try resolveCaptureSource(for: view)
let captureSize = source.graphicsDisplay.sizeInPixels
let width = max(Int(captureSize.width), 1)
let height = max(Int(captureSize.height), 1)
let timestamp = ISO8601DateFormatter().string(from: Date())
.replacingOccurrences(of: ":", with: "-")
let desktop = FileManager.default.homeDirectoryForCurrentUser
.appendingPathComponent("Desktop")
let url = desktop.appendingPathComponent("vphone-recording-\(timestamp).mov")
let url = recordingOutputURL()
outputURL = url
let writer = try AVAssetWriter(outputURL: url, fileType: .mov)
@@ -59,8 +86,11 @@ class VPhoneScreenRecorder {
self.writer = writer
videoInput = input
self.adaptor = adaptor
self.view = view
graphicsDisplay = source.graphicsDisplay
captureModeDescription = source.description
frameCount = 0
screenshotInFlight = false
didLogCaptureFailure = false
timer = Timer.scheduledTimer(withTimeInterval: 1.0 / 30.0, repeats: true) {
[weak self] _ in
@@ -69,7 +99,9 @@ class VPhoneScreenRecorder {
}
}
print("[record] started — \(url.lastPathComponent) (\(width)x\(height))")
print(
"[record] started — \(url.lastPathComponent) (\(width)x\(height), source: \(captureModeDescription))"
)
}
func stopRecording() async -> URL? {
@@ -86,7 +118,9 @@ class VPhoneScreenRecorder {
videoInput = nil
adaptor = nil
outputURL = nil
view = nil
graphicsDisplay = nil
screenshotInFlight = false
didLogCaptureFailure = false
if let url {
print("[record] saved — \(url.path)")
@@ -94,26 +128,91 @@ class VPhoneScreenRecorder {
return url
}
func copyScreenshotToPasteboard(view: NSView) async throws {
let cgImage = try await captureStillImage(from: view)
let image = NSImage(
cgImage: cgImage,
size: NSSize(width: cgImage.width, height: cgImage.height)
)
let pasteboard = NSPasteboard.general
pasteboard.clearContents()
guard pasteboard.writeObjects([image]) else {
throw CaptureError.clipboardWriteFailed
}
print("[record] screenshot copied to clipboard")
}
func saveScreenshot(view: NSView) async throws -> URL {
let cgImage = try await captureStillImage(from: view)
let bitmap = NSBitmapImageRep(cgImage: cgImage)
guard let pngData = bitmap.representation(using: .png, properties: [:]) else {
throw CaptureError.pngEncodingFailed
}
let url = screenshotOutputURL()
try pngData.write(to: url, options: .atomic)
print("[record] screenshot saved — \(url.path)")
return url
}
// MARK: - Frame Capture
private func captureFrame() {
guard let view, let adaptor, let input = videoInput,
input.isReadyForMoreMediaData
guard let adaptor, let input = videoInput,
input.isReadyForMoreMediaData,
let graphicsDisplay
else { return }
// Render view into bitmap at backing (retina) resolution
let bounds = view.bounds
guard let rep = view.bitmapImageRepForCachingDisplay(in: bounds) else { return }
view.cacheDisplay(in: bounds, to: rep)
guard let cgImage = rep.cgImage else { return }
captureGraphicsDisplayFrame(graphicsDisplay, adaptor: adaptor)
}
// Get pixel buffer from pool
private func captureGraphicsDisplayFrame(
_ graphicsDisplay: VZGraphicsDisplay,
adaptor: AVAssetWriterInputPixelBufferAdaptor
) {
guard !screenshotInFlight else { return }
screenshotInFlight = true
takeGraphicsScreenshot(from: graphicsDisplay) { [weak self] cgImage in
Task { @MainActor in
guard let self else { return }
self.screenshotInFlight = false
guard let input = self.videoInput, input.isReadyForMoreMediaData else { return }
guard let cgImage else {
if !self.didLogCaptureFailure {
print("[record] graphics screenshot returned no image")
self.didLogCaptureFailure = true
}
return
}
self.didLogCaptureFailure = false
self.appendFrame(from: adaptor, cgImage: cgImage)
}
}
}
private func captureStillImage(from view: NSView) async throws -> CGImage {
let source = try resolveCaptureSource(for: view)
guard let cgImage = await takeGraphicsScreenshot(from: source.graphicsDisplay) else {
throw CaptureError.captureFailed
}
return cgImage
}
private func appendFrame(from adaptor: AVAssetWriterInputPixelBufferAdaptor, cgImage: CGImage) {
guard let input = videoInput, input.isReadyForMoreMediaData else { return }
guard let pool = adaptor.pixelBufferPool else { return }
var pixelBuffer: CVPixelBuffer?
CVPixelBufferPoolCreatePixelBuffer(nil, pool, &pixelBuffer)
guard let pb = pixelBuffer else { return }
// Draw CGImage into pixel buffer
CVPixelBufferLockBaseAddress(pb, [])
let pbWidth = CVPixelBufferGetWidth(pb)
let pbHeight = CVPixelBufferGetHeight(pb)
@@ -135,4 +234,79 @@ class VPhoneScreenRecorder {
adaptor.append(pb, withPresentationTime: time)
frameCount += 1
}
private func resolveCaptureSource(for view: NSView) throws -> CaptureSource {
guard let vmView = view as? VPhoneVirtualMachineView,
let graphicsDisplay = vmView.recordingGraphicsDisplay
else {
throw CaptureError.captureFailed
}
return CaptureSource(
graphicsDisplay: graphicsDisplay,
description: "private VZGraphicsDisplay screenshots"
)
}
private func takeGraphicsScreenshot(
from graphicsDisplay: VZGraphicsDisplay,
completion: @escaping (CGImage?) -> Void
) {
let selector = NSSelectorFromString("_takeScreenshotWithCompletionHandler:")
guard graphicsDisplay.responds(to: selector),
let cls = object_getClass(graphicsDisplay),
let method = class_getInstanceMethod(cls, selector)
else {
completion(nil)
return
}
let implementation = method_getImplementation(method)
let function = unsafeBitCast(implementation, to: ScreenshotIMP.self)
let block: ScreenshotCompletionBlock = { [weak self] imageObject in
completion(self?.convertScreenshotObject(imageObject))
}
let blockObject = unsafeBitCast(block, to: AnyObject.self)
function(graphicsDisplay, selector, blockObject)
}
private func takeGraphicsScreenshot(from graphicsDisplay: VZGraphicsDisplay) async -> CGImage? {
await withCheckedContinuation { continuation in
takeGraphicsScreenshot(from: graphicsDisplay) { cgImage in
continuation.resume(returning: cgImage)
}
}
}
private func convertScreenshotObject(_ imageObject: AnyObject?) -> CGImage? {
guard let imageObject else { return nil }
if let nsImage = imageObject as? NSImage {
return nsImage.cgImage(forProposedRect: nil, context: nil, hints: nil)
}
let cfObject = unsafeBitCast(imageObject, to: CFTypeRef.self)
if CFGetTypeID(cfObject) == CGImage.typeID {
return unsafeBitCast(cfObject, to: CGImage.self)
}
return nil
}
private func recordingOutputURL() -> URL {
desktopDirectory().appendingPathComponent("vphone-recording-\(timestampString()).mov")
}
private func screenshotOutputURL() -> URL {
desktopDirectory().appendingPathComponent("vphone-screenshot-\(timestampString()).png")
}
private func timestampString() -> String {
ISO8601DateFormatter().string(from: Date()).replacingOccurrences(of: ":", with: "-")
}
private func desktopDirectory() -> URL {
FileManager.default.homeDirectoryForCurrentUser.appendingPathComponent("Desktop")
}
}

View File

@@ -21,6 +21,13 @@ class VPhoneVirtualMachineView: VZVirtualMachineView {
return devices.object(at: 0) as AnyObject
}
var recordingGraphicsDisplay: VZGraphicsDisplay? {
if let display = Dynamic(self)._graphicsDisplay.asObject as? VZGraphicsDisplay {
return display
}
return virtualMachine?.graphicsDevices.first?.displays.first
}
// MARK: - Event Handling
override var acceptsFirstResponder: Bool {