mirror of
https://github.com/Lakr233/vphone-cli.git
synced 2026-04-05 13:09:06 +08:00
Merge pull request #125 from xcxmiku/feature/private-display-recorder
Use private display capture for recording and screenshots
This commit is contained in:
@@ -177,3 +177,47 @@
|
||||
| PCC 26.3 (`23D128`) | 14 | 59 |
|
||||
| iOS 26.1 (`23B85`) | 14 | 59 |
|
||||
| iOS 26.3 (`23D127`) | 14 | 59 |
|
||||
|
||||
## Automation Notes (2026-03-06)
|
||||
|
||||
- `scripts/setup_machine.sh` non-interactive flow fix: renamed local variable `status` to `boot_state` in first-boot log wait and boot-analysis wait helpers to avoid zsh `status` read-only special parameter collision.
|
||||
- `scripts/setup_machine.sh` non-interactive first-boot wait fix: replaced `(( waited++ ))` with `(( ++waited ))` in `monitor_boot_log_until` to avoid `set -e` abort when arithmetic expression evaluates to `0`.
|
||||
- `scripts/jb_patch_autotest.sh` loop fix for sweep stability under `set -e`: replaced `((idx++))` with `(( ++idx ))`.
|
||||
- `scripts/jb_patch_autotest.sh` zsh compatibility fix: renamed per-case result variable `status` to `case_status` to avoid `status` read-only special parameter collision.
|
||||
- `scripts/jb_patch_autotest.sh` selection logic update:
|
||||
- default run now excludes methods listed in `KernelJBPatcher._DEV_SINGLE_WORKING_METHODS` (pending-only sweep).
|
||||
- set `JB_AUTOTEST_INCLUDE_WORKING=1` to include already-working methods and run the full list.
|
||||
- Sweep run record:
|
||||
- `setup_logs/jb_patch_tests_20260306_114417` (2026-03-06): aborted at `[1/20]` with `read-only variable: status` in `jb_patch_autotest.sh`.
|
||||
- `setup_logs/jb_patch_tests_20260306_115027` (2026-03-06): rerun after `status` fix, pending-only mode (`Total methods: 19`).
|
||||
- Final run result from `jb_patch_tests_20260306_115027` at `2026-03-06 13:17`:
|
||||
- Finished: 19/19 (`PASS=15`, `FAIL=4`, all fails `rc=2`).
|
||||
- Failing methods at that time: `patch_bsd_init_auth`, `patch_io_secure_bsd_root`, `patch_vm_fault_enter_prepare`, `patch_cred_label_update_execve`.
|
||||
- 2026-03-06 follow-up: `patch_io_secure_bsd_root` failure is now attributed to a wrong-site patch in `AppleARMPE::callPlatformFunction` (`"SecureRoot"` gate at `0xFFFFFE000836E1F0`), not the intended `"SecureRootName"` deny-return path. The code was retargeted the same day to `0xFFFFFE000836E464` and re-enabled for the next restore/boot check.
|
||||
- 2026-03-06 follow-up: `patch_bsd_init_auth` was retargeted after confirming the old matcher was hitting unrelated code; keep disabled in default schedule until a fresh clean-baseline boot test passes.
|
||||
- Final case: `[19/19] patch_syscallmask_apply_to_proc` (`PASS`).
|
||||
- 2026-03-06 re-analysis: that historical `PASS` is now treated as a false positive for functionality, because the recorded bytes landed at `0xfffffe00093ae6e4`/`0xfffffe00093ae6e8` inside `_profile_syscallmask_destroy` underflow handling, not in `_proc_apply_syscall_masks`.
|
||||
- 2026-03-06 code update: `scripts/patchers/kernel_jb_patch_syscallmask.py` was rebuilt to target the real syscallmask apply wrapper structurally and now dry-runs on `PCC-CloudOS-26.1-23B85 kernelcache.research.vphone600` with 3 writes: `0x02395530`, `0x023955E8`, and cave `0x00AB1720`. User-side boot validation succeeded the same day.
|
||||
- 2026-03-06 follow-up: `patch_kcall10` was rebuilt from the old ABI-unsafe pseudo-10-arg design into an ABI-correct `sysent[439]` cave. Focused dry-run on `PCC-CloudOS-26.1-23B85 kernelcache.research.vphone600` now emits 4 writes: cave `0x00AB1720`, `sy_call` `0x0073E180`, `sy_arg_munge32` `0x0073E188`, and metadata `0x0073E190`; the method was re-enabled in `_GROUP_C_METHODS`.
|
||||
- Observed failure symptom in current failing set: first boot panic before command injection (or boot process early exit).
|
||||
- Post-run schedule change (per user request):
|
||||
- commented out failing methods from default `KernelJBPatcher._PATCH_METHODS` schedule in `scripts/patchers/kernel_jb.py`:
|
||||
- `patch_bsd_init_auth`
|
||||
- `patch_io_secure_bsd_root`
|
||||
- `patch_vm_fault_enter_prepare`
|
||||
- `patch_cred_label_update_execve`
|
||||
- 2026-03-06 re-research note for `patch_cred_label_update_execve`:
|
||||
- old entry-time early-return strategy was identified as boot-unsafe because it skipped AMFI exec-time `csflags` and entitlement propagation entirely.
|
||||
- implementation was reworked to a success-tail trampoline that preserves normal AMFI processing and only clears restrictive `csflags` bits on the success path.
|
||||
- default JB schedule still keeps the method disabled until the reworked strategy is boot-validated.
|
||||
- Manual DEV+single (`setup_machine` + `PATCH=<method>`) working set now includes:
|
||||
- `patch_amfi_cdhash_in_trustcache`
|
||||
- `patch_amfi_execve_kill_path`
|
||||
- `patch_task_conversion_eval_internal`
|
||||
- `patch_sandbox_hooks_extended`
|
||||
- `patch_post_validation_additional`
|
||||
- 2026-03-07 host-side note:
|
||||
- reviewed private Virtualization.framework display APIs against the recorder pipeline in `sources/vphone-cli/VPhoneScreenRecorder.swift`.
|
||||
- replaced the old AppKit-first recorder path with a private-display-only implementation built around hidden `VZGraphicsDisplay._takeScreenshotWithCompletionHandler:` capture.
|
||||
- added still screenshot actions that can copy the captured image to the pasteboard or save a PNG to disk using the same private capture path.
|
||||
- `make build` is used as the sanity check path; live VM validation is still needed to confirm the exact screenshot object type returned on macOS 15.
|
||||
|
||||
@@ -9,6 +9,9 @@ extension VPhoneMenuController {
|
||||
let toggle = makeItem("Start Recording", action: #selector(toggleRecording))
|
||||
recordingItem = toggle
|
||||
menu.addItem(toggle)
|
||||
menu.addItem(NSMenuItem.separator())
|
||||
menu.addItem(makeItem("Copy Screenshot to Clipboard", action: #selector(copyScreenshotToClipboard)))
|
||||
menu.addItem(makeItem("Save Screenshot to File", action: #selector(saveScreenshotToFile)))
|
||||
item.submenu = menu
|
||||
return item
|
||||
}
|
||||
@@ -20,18 +23,54 @@ extension VPhoneMenuController {
|
||||
recordingItem?.title = "Start Recording"
|
||||
}
|
||||
} else {
|
||||
guard let window = NSApp.keyWindow,
|
||||
let view = window.contentView
|
||||
else {
|
||||
print("[record] no active window")
|
||||
guard let view = activeCaptureView() else {
|
||||
showAlert(title: "Recording", message: "No active VM window.", style: .warning)
|
||||
return
|
||||
}
|
||||
do {
|
||||
try screenRecorder?.startRecording(view: view)
|
||||
recordingItem?.title = "Stop Recording"
|
||||
} catch {
|
||||
print("[record] failed to start: \(error)")
|
||||
showAlert(title: "Recording", message: "\(error)", style: .warning)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@objc func copyScreenshotToClipboard() {
|
||||
guard let recorder = screenRecorder else { return }
|
||||
guard let view = activeCaptureView() else {
|
||||
showAlert(title: "Screenshot", message: "No active VM window.", style: .warning)
|
||||
return
|
||||
}
|
||||
|
||||
Task { @MainActor in
|
||||
do {
|
||||
try await recorder.copyScreenshotToPasteboard(view: view)
|
||||
showAlert(title: "Screenshot", message: "Copied to clipboard.", style: .informational)
|
||||
} catch {
|
||||
showAlert(title: "Screenshot", message: "\(error)", style: .warning)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@objc func saveScreenshotToFile() {
|
||||
guard let recorder = screenRecorder else { return }
|
||||
guard let view = activeCaptureView() else {
|
||||
showAlert(title: "Screenshot", message: "No active VM window.", style: .warning)
|
||||
return
|
||||
}
|
||||
|
||||
Task { @MainActor in
|
||||
do {
|
||||
let url = try await recorder.saveScreenshot(view: view)
|
||||
showAlert(title: "Screenshot", message: "Saved to \(url.path)", style: .informational)
|
||||
} catch {
|
||||
showAlert(title: "Screenshot", message: "\(error)", style: .warning)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private func activeCaptureView() -> NSView? {
|
||||
NSApp.keyWindow?.contentView ?? NSApp.mainWindow?.contentView
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,18 +1,48 @@
|
||||
import AppKit
|
||||
import AVFoundation
|
||||
import CoreVideo
|
||||
import ObjectiveC.runtime
|
||||
import Virtualization
|
||||
|
||||
// MARK: - Screen Recorder
|
||||
|
||||
@MainActor
|
||||
class VPhoneScreenRecorder {
|
||||
private enum CaptureError: LocalizedError {
|
||||
case captureFailed
|
||||
case clipboardWriteFailed
|
||||
case pngEncodingFailed
|
||||
|
||||
var errorDescription: String? {
|
||||
switch self {
|
||||
case .captureFailed:
|
||||
"Failed to capture a frame from the virtual machine."
|
||||
case .clipboardWriteFailed:
|
||||
"Failed to copy the screenshot to the pasteboard."
|
||||
case .pngEncodingFailed:
|
||||
"Failed to encode the screenshot as PNG."
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private struct CaptureSource {
|
||||
let graphicsDisplay: VZGraphicsDisplay
|
||||
let description: String
|
||||
}
|
||||
|
||||
private typealias ScreenshotCompletionBlock = @convention(block) (AnyObject?) -> Void
|
||||
private typealias ScreenshotIMP = @convention(c) (AnyObject, Selector, AnyObject) -> Void
|
||||
|
||||
private var writer: AVAssetWriter?
|
||||
private var videoInput: AVAssetWriterInput?
|
||||
private var adaptor: AVAssetWriterInputPixelBufferAdaptor?
|
||||
private var timer: Timer?
|
||||
private var frameCount: Int64 = 0
|
||||
private var outputURL: URL?
|
||||
private weak var view: NSView?
|
||||
private var graphicsDisplay: VZGraphicsDisplay?
|
||||
private var captureModeDescription = "private VZGraphicsDisplay screenshots"
|
||||
private var screenshotInFlight = false
|
||||
private var didLogCaptureFailure = false
|
||||
|
||||
var isRecording: Bool {
|
||||
writer?.status == .writing
|
||||
@@ -21,15 +51,12 @@ class VPhoneScreenRecorder {
|
||||
func startRecording(view: NSView) throws {
|
||||
guard !isRecording else { return }
|
||||
|
||||
let backingSize = view.convertToBacking(view.bounds.size)
|
||||
let width = Int(backingSize.width)
|
||||
let height = Int(backingSize.height)
|
||||
let source = try resolveCaptureSource(for: view)
|
||||
let captureSize = source.graphicsDisplay.sizeInPixels
|
||||
let width = max(Int(captureSize.width), 1)
|
||||
let height = max(Int(captureSize.height), 1)
|
||||
|
||||
let timestamp = ISO8601DateFormatter().string(from: Date())
|
||||
.replacingOccurrences(of: ":", with: "-")
|
||||
let desktop = FileManager.default.homeDirectoryForCurrentUser
|
||||
.appendingPathComponent("Desktop")
|
||||
let url = desktop.appendingPathComponent("vphone-recording-\(timestamp).mov")
|
||||
let url = recordingOutputURL()
|
||||
outputURL = url
|
||||
|
||||
let writer = try AVAssetWriter(outputURL: url, fileType: .mov)
|
||||
@@ -59,8 +86,11 @@ class VPhoneScreenRecorder {
|
||||
self.writer = writer
|
||||
videoInput = input
|
||||
self.adaptor = adaptor
|
||||
self.view = view
|
||||
graphicsDisplay = source.graphicsDisplay
|
||||
captureModeDescription = source.description
|
||||
frameCount = 0
|
||||
screenshotInFlight = false
|
||||
didLogCaptureFailure = false
|
||||
|
||||
timer = Timer.scheduledTimer(withTimeInterval: 1.0 / 30.0, repeats: true) {
|
||||
[weak self] _ in
|
||||
@@ -69,7 +99,9 @@ class VPhoneScreenRecorder {
|
||||
}
|
||||
}
|
||||
|
||||
print("[record] started — \(url.lastPathComponent) (\(width)x\(height))")
|
||||
print(
|
||||
"[record] started — \(url.lastPathComponent) (\(width)x\(height), source: \(captureModeDescription))"
|
||||
)
|
||||
}
|
||||
|
||||
func stopRecording() async -> URL? {
|
||||
@@ -86,7 +118,9 @@ class VPhoneScreenRecorder {
|
||||
videoInput = nil
|
||||
adaptor = nil
|
||||
outputURL = nil
|
||||
view = nil
|
||||
graphicsDisplay = nil
|
||||
screenshotInFlight = false
|
||||
didLogCaptureFailure = false
|
||||
|
||||
if let url {
|
||||
print("[record] saved — \(url.path)")
|
||||
@@ -94,26 +128,91 @@ class VPhoneScreenRecorder {
|
||||
return url
|
||||
}
|
||||
|
||||
func copyScreenshotToPasteboard(view: NSView) async throws {
|
||||
let cgImage = try await captureStillImage(from: view)
|
||||
let image = NSImage(
|
||||
cgImage: cgImage,
|
||||
size: NSSize(width: cgImage.width, height: cgImage.height)
|
||||
)
|
||||
|
||||
let pasteboard = NSPasteboard.general
|
||||
pasteboard.clearContents()
|
||||
guard pasteboard.writeObjects([image]) else {
|
||||
throw CaptureError.clipboardWriteFailed
|
||||
}
|
||||
|
||||
print("[record] screenshot copied to clipboard")
|
||||
}
|
||||
|
||||
func saveScreenshot(view: NSView) async throws -> URL {
|
||||
let cgImage = try await captureStillImage(from: view)
|
||||
let bitmap = NSBitmapImageRep(cgImage: cgImage)
|
||||
guard let pngData = bitmap.representation(using: .png, properties: [:]) else {
|
||||
throw CaptureError.pngEncodingFailed
|
||||
}
|
||||
|
||||
let url = screenshotOutputURL()
|
||||
try pngData.write(to: url, options: .atomic)
|
||||
print("[record] screenshot saved — \(url.path)")
|
||||
return url
|
||||
}
|
||||
|
||||
// MARK: - Frame Capture
|
||||
|
||||
private func captureFrame() {
|
||||
guard let view, let adaptor, let input = videoInput,
|
||||
input.isReadyForMoreMediaData
|
||||
guard let adaptor, let input = videoInput,
|
||||
input.isReadyForMoreMediaData,
|
||||
let graphicsDisplay
|
||||
else { return }
|
||||
|
||||
// Render view into bitmap at backing (retina) resolution
|
||||
let bounds = view.bounds
|
||||
guard let rep = view.bitmapImageRepForCachingDisplay(in: bounds) else { return }
|
||||
view.cacheDisplay(in: bounds, to: rep)
|
||||
guard let cgImage = rep.cgImage else { return }
|
||||
captureGraphicsDisplayFrame(graphicsDisplay, adaptor: adaptor)
|
||||
}
|
||||
|
||||
// Get pixel buffer from pool
|
||||
private func captureGraphicsDisplayFrame(
|
||||
_ graphicsDisplay: VZGraphicsDisplay,
|
||||
adaptor: AVAssetWriterInputPixelBufferAdaptor
|
||||
) {
|
||||
guard !screenshotInFlight else { return }
|
||||
|
||||
screenshotInFlight = true
|
||||
takeGraphicsScreenshot(from: graphicsDisplay) { [weak self] cgImage in
|
||||
Task { @MainActor in
|
||||
guard let self else { return }
|
||||
|
||||
self.screenshotInFlight = false
|
||||
|
||||
guard let input = self.videoInput, input.isReadyForMoreMediaData else { return }
|
||||
|
||||
guard let cgImage else {
|
||||
if !self.didLogCaptureFailure {
|
||||
print("[record] graphics screenshot returned no image")
|
||||
self.didLogCaptureFailure = true
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
self.didLogCaptureFailure = false
|
||||
self.appendFrame(from: adaptor, cgImage: cgImage)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private func captureStillImage(from view: NSView) async throws -> CGImage {
|
||||
let source = try resolveCaptureSource(for: view)
|
||||
guard let cgImage = await takeGraphicsScreenshot(from: source.graphicsDisplay) else {
|
||||
throw CaptureError.captureFailed
|
||||
}
|
||||
return cgImage
|
||||
}
|
||||
|
||||
private func appendFrame(from adaptor: AVAssetWriterInputPixelBufferAdaptor, cgImage: CGImage) {
|
||||
guard let input = videoInput, input.isReadyForMoreMediaData else { return }
|
||||
guard let pool = adaptor.pixelBufferPool else { return }
|
||||
|
||||
var pixelBuffer: CVPixelBuffer?
|
||||
CVPixelBufferPoolCreatePixelBuffer(nil, pool, &pixelBuffer)
|
||||
guard let pb = pixelBuffer else { return }
|
||||
|
||||
// Draw CGImage into pixel buffer
|
||||
CVPixelBufferLockBaseAddress(pb, [])
|
||||
let pbWidth = CVPixelBufferGetWidth(pb)
|
||||
let pbHeight = CVPixelBufferGetHeight(pb)
|
||||
@@ -135,4 +234,79 @@ class VPhoneScreenRecorder {
|
||||
adaptor.append(pb, withPresentationTime: time)
|
||||
frameCount += 1
|
||||
}
|
||||
|
||||
private func resolveCaptureSource(for view: NSView) throws -> CaptureSource {
|
||||
guard let vmView = view as? VPhoneVirtualMachineView,
|
||||
let graphicsDisplay = vmView.recordingGraphicsDisplay
|
||||
else {
|
||||
throw CaptureError.captureFailed
|
||||
}
|
||||
|
||||
return CaptureSource(
|
||||
graphicsDisplay: graphicsDisplay,
|
||||
description: "private VZGraphicsDisplay screenshots"
|
||||
)
|
||||
}
|
||||
|
||||
private func takeGraphicsScreenshot(
|
||||
from graphicsDisplay: VZGraphicsDisplay,
|
||||
completion: @escaping (CGImage?) -> Void
|
||||
) {
|
||||
let selector = NSSelectorFromString("_takeScreenshotWithCompletionHandler:")
|
||||
guard graphicsDisplay.responds(to: selector),
|
||||
let cls = object_getClass(graphicsDisplay),
|
||||
let method = class_getInstanceMethod(cls, selector)
|
||||
else {
|
||||
completion(nil)
|
||||
return
|
||||
}
|
||||
|
||||
let implementation = method_getImplementation(method)
|
||||
let function = unsafeBitCast(implementation, to: ScreenshotIMP.self)
|
||||
|
||||
let block: ScreenshotCompletionBlock = { [weak self] imageObject in
|
||||
completion(self?.convertScreenshotObject(imageObject))
|
||||
}
|
||||
let blockObject = unsafeBitCast(block, to: AnyObject.self)
|
||||
function(graphicsDisplay, selector, blockObject)
|
||||
}
|
||||
|
||||
private func takeGraphicsScreenshot(from graphicsDisplay: VZGraphicsDisplay) async -> CGImage? {
|
||||
await withCheckedContinuation { continuation in
|
||||
takeGraphicsScreenshot(from: graphicsDisplay) { cgImage in
|
||||
continuation.resume(returning: cgImage)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private func convertScreenshotObject(_ imageObject: AnyObject?) -> CGImage? {
|
||||
guard let imageObject else { return nil }
|
||||
|
||||
if let nsImage = imageObject as? NSImage {
|
||||
return nsImage.cgImage(forProposedRect: nil, context: nil, hints: nil)
|
||||
}
|
||||
|
||||
let cfObject = unsafeBitCast(imageObject, to: CFTypeRef.self)
|
||||
if CFGetTypeID(cfObject) == CGImage.typeID {
|
||||
return unsafeBitCast(cfObject, to: CGImage.self)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
private func recordingOutputURL() -> URL {
|
||||
desktopDirectory().appendingPathComponent("vphone-recording-\(timestampString()).mov")
|
||||
}
|
||||
|
||||
private func screenshotOutputURL() -> URL {
|
||||
desktopDirectory().appendingPathComponent("vphone-screenshot-\(timestampString()).png")
|
||||
}
|
||||
|
||||
private func timestampString() -> String {
|
||||
ISO8601DateFormatter().string(from: Date()).replacingOccurrences(of: ":", with: "-")
|
||||
}
|
||||
|
||||
private func desktopDirectory() -> URL {
|
||||
FileManager.default.homeDirectoryForCurrentUser.appendingPathComponent("Desktop")
|
||||
}
|
||||
}
|
||||
|
||||
@@ -21,6 +21,13 @@ class VPhoneVirtualMachineView: VZVirtualMachineView {
|
||||
return devices.object(at: 0) as AnyObject
|
||||
}
|
||||
|
||||
var recordingGraphicsDisplay: VZGraphicsDisplay? {
|
||||
if let display = Dynamic(self)._graphicsDisplay.asObject as? VZGraphicsDisplay {
|
||||
return display
|
||||
}
|
||||
return virtualMachine?.graphicsDevices.first?.displays.first
|
||||
}
|
||||
|
||||
// MARK: - Event Handling
|
||||
|
||||
override var acceptsFirstResponder: Bool {
|
||||
|
||||
Reference in New Issue
Block a user