mirror of
https://github.com/YusufB5/ASCILINE.git
synced 2026-06-14 22:25:13 +02:00
feat: Core engine V2 (A/V Sync, Zero-Copy Pixel, FPS Decimation) & CLI shortcuts
This commit is contained in:
parent
522ba756c8
commit
e758423338
7 changed files with 257 additions and 65 deletions
32
ARCHITECTURE_NOTES - Kopya.md
Normal file
32
ARCHITECTURE_NOTES - Kopya.md
Normal file
|
|
@ -0,0 +1,32 @@
|
|||
# ASCILINE Architecture Notes
|
||||
*A turning point in performance and synchronization.*
|
||||
|
||||
## The Philosophy
|
||||
As the creator of ASCILINE, I believe in taking calculated, deliberate steps to ensure every component of this engine is fundamentally solid before moving forward. This document records the most critical architectural turning point in the project's history: **The transition from a naive frame loop to a professional-grade, self-healing media engine.**
|
||||
|
||||
## 1. The Audio Master Clock (A/V Sync)
|
||||
**The Problem:** Originally, video frames and audio were streaming independently. If the computer lagged, the video slowed down, but the audio kept playing. This caused irreversible desynchronization.
|
||||
**The Turning Point:** We shifted the paradigm. The Audio is now the absolute "Master Clock".
|
||||
- The server mathematically calculates the exact timestamp each frame belongs to.
|
||||
- The browser checks the audio track's current time.
|
||||
- If the video is lagging, the system instantly drops frames to catch up.
|
||||
- If the video is ahead, it pauses and waits for the audio.
|
||||
**Result:** The stream is perfectly self-correcting. Even if you minimize the browser or experience a heavy CPU spike, the video will flawlessly snap back into sync with the audio the moment performance returns.
|
||||
|
||||
## 2. Zero-Copy Pipeline (The Direct Canvas Transfer)
|
||||
**The Problem:** In our early pixel-mode implementation, Python was doing too much "heavy lifting". It was receiving BGR data from the video, copying it to flip it to RGB, appending invisible ASCII characters to every pixel, and packing it into a massive array before sending it.
|
||||
**The Turning Point:** We eliminated the middleman.
|
||||
- **Is the stream directly transferring to the web canvas now?** **YES.**
|
||||
- In Pixel Mode (`--pixel`), Python takes the raw, untouched BGR byte array directly from the OpenCV video decoder and shoots it straight down the WebSocket.
|
||||
- There is no memory copying, no array flipping, and no invisible characters.
|
||||
- On the receiving end, the JavaScript V8 engine takes those raw bytes and maps them directly into the HTML5 Canvas `ImageData` memory buffer.
|
||||
|
||||
**Result:** CPU usage plummeted. The WebSocket payload size decreased by 25%. We achieved what is essentially a "Zero-Copy" direct pipeline from the video file on the server straight to the pixels on the browser screen, unlocking pure 60 FPS performance without breaking a sweat.
|
||||
|
||||
## 3. Future Vision: Production Broadcaster Architecture (Rust / C++)
|
||||
**Current Limitation:** The Python prototype operates on a "Video-on-Demand" architecture. Each new user triggers a separate OpenCV video decoding pipeline. Python's single-core GIL architecture caps out around 3-4 users before the server freezes.
|
||||
**The Planned Evolution:** If ASCILINE is deployed as a massive public website, the backend will be rewritten from scratch in **Rust** or **C++**.
|
||||
- **Standardized Performance:** The engine will standardize on 30 FPS for perfect stability.
|
||||
- **True Broadcasting (Copy Cycles):** The server will decode the video exactly *once* in the background. The resulting byte array will be instantly duplicated (broadcasted) to thousands of connected WebSockets simultaneously.
|
||||
- **Load-Shedding & Auto-Shutdown:** Built-in safeguards will actively monitor concurrent connections. If a massive traffic spike (e.g., 5,000+ users) overwhelms network bandwidth or RAM, the engine will intelligently shed load or gracefully shut down the broadcast to prevent a hard server crash.
|
||||
- **Result:** A truly commercial-grade, real-time media server capable of handling thousands of concurrent viewers like a modern Twitch stream, built directly on top of the theoretical foundations proven in this Python prototype.
|
||||
17
README.md
17
README.md
|
|
@ -1,6 +1,6 @@
|
|||
# 🌌 ASCILINE Engine
|
||||
|
||||
**ASCILINE** is a high-performance, real-time ASCII video rendering engine. **Our core objective is to transform the web into a highly dynamic and interactive typographic canvas.** By moving away from traditional video players, ASCILINE streams visual data from a Python backend directly into the browser at **60 FPS** as raw, manipulable text.
|
||||
**ASCILINE** is a high-performance, cross-platform real-time ASCII video rendering engine. **Our core objective is to transform the web into a highly dynamic and interactive typographic canvas.** By moving away from traditional video players, ASCILINE streams visual data from a Python backend directly into the browser at **60 FPS** as raw, manipulable text.
|
||||
|
||||
| Output | Details |
|
||||
| :--- | :--- |
|
||||
|
|
@ -17,10 +17,12 @@
|
|||
|
||||
## 🚀 Technical Features
|
||||
|
||||
- **Cross-Platform**: Runs seamlessly on Windows, macOS, and Linux.
|
||||
- **Real-Time ASCII Streaming**: Low-latency video-to-ASCII conversion.
|
||||
- **Real-Time pixel Streaming**: Close visual quality to 360p video streaming it uses ▮ characters.
|
||||
- **High Performance**: Uses **HTML5 Canvas** for rendering instead of heavy DOM elements, enabling 60 FPS playback.
|
||||
- **Binary Protocol**: Frames are encoded into `Uint8Array` (binary) for efficient bandwidth usage.
|
||||
- **Real-Time Pixel Streaming**: Replaces characters with colored blocks, approaching 360p video quality.
|
||||
- **High Performance**: Uses **HTML5 Canvas** for rendering, optimized for cinematic 24-30 FPS playback. High-FPS sources are automatically decimated for stability.
|
||||
- **Master Clock Sync**: The audio track acts as the absolute master clock, guaranteeing perfect A/V synchronization.
|
||||
- **Zero-Copy Binary Protocol**: Frames are streamed as raw binary (`Uint8Array`) directly to the canvas, saving bandwidth and CPU.
|
||||
- **Multiple Color Modes**: Supports everything from classic B&W to 16M color ultra-fidelity.
|
||||
- **Flexible Video Management**: Supports JSON playlists (per-video mode & volume),
|
||||
folder-based auto-queuing (filesystem order), single-file mode, and infinite loop
|
||||
|
|
@ -79,13 +81,18 @@ python stream_server.py --playlist playlist.json --cols 220 --loop
|
|||
```
|
||||
Use `playlist.json` when you need different `--mode` or `--vol` settings for each video.
|
||||
|
||||
> 💡 **Windows Users:** You can use the included `serve.bat` shortcut for quicker typing: `.\serve video.mp4 --cols 240`
|
||||
|
||||
Open `http://localhost:8000` in your browser.
|
||||
|
||||
### 4. Run directly in Terminal (Standalone)
|
||||
If you prefer to bypass the web interface, you can render the video directly inside an ANSI-supported terminal (zero-flicker, true color):
|
||||
```bash
|
||||
python ascii_video_player2.py video.mp4 --cols 220 --quality 0
|
||||
python ascii_video_player2.py video.mp4 --cols 100 --quality 0
|
||||
```
|
||||
> 💡 **Windows Users:** Use the shortcut `.\play video.mp4 -c 100 -q 0`
|
||||
>
|
||||
> ⚠️ **Note:** Do not resize your terminal window during playback, as dynamic text wrapping will corrupt the ASCII layout.
|
||||
|
||||
## 🎨 Customization
|
||||
|
||||
|
|
|
|||
108
app.js
108
app.js
|
|
@ -41,6 +41,7 @@ let selectionBuffer = null;
|
|||
// Timing & Metrics
|
||||
let lastRenderTime = 0;
|
||||
let frameCount = 0, currentFps = 0, lastFpsUpdate = 0;
|
||||
let streamStartTime = 0;
|
||||
|
||||
const CHAR_LUT = new Array(128);
|
||||
for (let i = 0; i < 128; i++) CHAR_LUT[i] = String.fromCharCode(i);
|
||||
|
|
@ -140,10 +141,8 @@ function connectWebSocket() {
|
|||
frameCount = 0;
|
||||
currentFps = 0;
|
||||
|
||||
if (audioEl) {
|
||||
audioEl.src = '/audio?' + Date.now();
|
||||
audioEl.volume = volumeSlider ? volumeSlider.value : 1.0;
|
||||
}
|
||||
// Audio is loaded later in INIT handler (Audio Ready Gate).
|
||||
// Don't preload here — causes race conditions with vol=0 (204 response).
|
||||
|
||||
const protocol = location.protocol === 'https:' ? 'wss:' : 'ws:';
|
||||
ws = new WebSocket(`${protocol}//${location.host}/ws`);
|
||||
|
|
@ -166,30 +165,62 @@ function connectWebSocket() {
|
|||
pixelMode = (p.length > 5 && parseInt(p[5]) === 1);
|
||||
buildCanvas(parseInt(p[3]), parseInt(p[4]));
|
||||
|
||||
// Reload audio on every INIT so each video's audio plays correctly.
|
||||
// The server updates current_index BEFORE sending INIT, so /audio
|
||||
// will already serve the new video's audio when we request it here.
|
||||
// ── AUDIO READY GATE ──
|
||||
// Buffer video frames but don't render until audio is ready.
|
||||
// This prevents the 0.5s initial stutter.
|
||||
readyToRender = false;
|
||||
state = 'PLAYING';
|
||||
|
||||
const beginRendering = () => {
|
||||
readyToRender = true;
|
||||
streamStartTime = performance.now();
|
||||
lastRenderTime = performance.now();
|
||||
lastFpsUpdate = lastRenderTime;
|
||||
requestAnimationFrame(renderFrame);
|
||||
};
|
||||
|
||||
if (audioEl) {
|
||||
audioEl.pause();
|
||||
audioEl.src = '/audio?' + Date.now();
|
||||
audioEl.volume = volumeSlider ? volumeSlider.value : 1.0;
|
||||
audioEl.load();
|
||||
audioEl.play().catch(() => {});
|
||||
}
|
||||
|
||||
readyToRender = true;
|
||||
state = 'PLAYING';
|
||||
lastRenderTime = performance.now();
|
||||
lastFpsUpdate = lastRenderTime;
|
||||
requestAnimationFrame(renderFrame);
|
||||
// Wait for audio to actually start playing
|
||||
if (audioEl.readyState >= 3) {
|
||||
beginRendering();
|
||||
} else {
|
||||
audioEl.addEventListener('playing', beginRendering, { once: true });
|
||||
// Fallback: if audio fails to load (vol=0 / 204), start after 500ms
|
||||
setTimeout(() => {
|
||||
if (!readyToRender) beginRendering();
|
||||
}, 500);
|
||||
}
|
||||
} else {
|
||||
// No audio element at all → start immediately
|
||||
beginRendering();
|
||||
}
|
||||
return;
|
||||
}
|
||||
frameBuffer.push(event.data);
|
||||
|
||||
// Mode 1: Text Frame with Timestamp
|
||||
const text = event.data;
|
||||
const newlineIdx = text.indexOf('\n');
|
||||
const frameIndex = parseInt(text.substring(0, newlineIdx));
|
||||
const frameTime = frameIndex / targetFps;
|
||||
const frameData = text.substring(newlineIdx + 1);
|
||||
frameBuffer.push({ data: frameData, time: frameTime });
|
||||
} else {
|
||||
frameBuffer.push(event.data);
|
||||
// Binary Frames with 4-byte header
|
||||
const buffer = event.data;
|
||||
const view = new DataView(buffer);
|
||||
const frameIndex = view.getUint32(0, false); // Big-endian
|
||||
const frameTime = frameIndex / targetFps;
|
||||
const frameData = new Uint8Array(buffer, 4);
|
||||
frameBuffer.push({ data: frameData, time: frameTime });
|
||||
}
|
||||
|
||||
while (frameBuffer.length > BUFFER_SIZE * 3) frameBuffer.shift();
|
||||
while (frameBuffer.length > BUFFER_SIZE * 5) frameBuffer.shift();
|
||||
};
|
||||
|
||||
ws.onopen = () => { statusEl.textContent = 'Buffering...'; };
|
||||
|
|
@ -215,11 +246,31 @@ function connectWebSocket() {
|
|||
// ═══════════════════════════════════════
|
||||
|
||||
function renderFrame(now) {
|
||||
if (state !== 'PLAYING') return;
|
||||
if (state !== 'PLAYING' || !readyToRender) return;
|
||||
requestAnimationFrame(renderFrame);
|
||||
|
||||
const elapsed = now - lastRenderTime;
|
||||
if (elapsed < frameInterval) return;
|
||||
// ── MASTER CLOCK LOGIC ──
|
||||
let masterClock;
|
||||
if (audioEl && audioEl.readyState >= 1 && !audioEl.paused) {
|
||||
masterClock = audioEl.currentTime;
|
||||
} else {
|
||||
masterClock = (now - streamStartTime) / 1000.0;
|
||||
}
|
||||
|
||||
if (frameBuffer.length === 0) return;
|
||||
|
||||
// A/V Sync: Drop frames that are too far behind the master clock (catch up)
|
||||
while (frameBuffer.length > 1 && frameBuffer[0].time < masterClock - 0.1) {
|
||||
frameBuffer.shift();
|
||||
}
|
||||
|
||||
// A/V Sync: Wait if the frame is in the future
|
||||
if (frameBuffer[0].time > masterClock + 0.05) {
|
||||
return;
|
||||
}
|
||||
|
||||
const frameObj = frameBuffer.shift();
|
||||
const frame = frameObj.data;
|
||||
|
||||
frameCount++;
|
||||
if (now - lastFpsUpdate >= 1000) {
|
||||
|
|
@ -231,29 +282,28 @@ function renderFrame(now) {
|
|||
statusEl.textContent = `FPS: ${currentFps}/${Math.round(targetFps)} | Buf: ${frameBuffer.length} | ${label}`;
|
||||
}
|
||||
|
||||
if (frameBuffer.length === 0) return;
|
||||
lastRenderTime = now;
|
||||
const frame = frameBuffer.shift();
|
||||
|
||||
if (renderMode === 1) {
|
||||
player.style.display = 'block';
|
||||
player.style.color = '#fff';
|
||||
player.textContent = frame;
|
||||
} else if (pixelMode) {
|
||||
// ── DOT MODE: ImageData pixel blast (1 draw call) ──
|
||||
const view = new Uint8Array(frame);
|
||||
// ── ZERO-COPY PIXEL MODE ──
|
||||
// Server sends raw BGR (3 bytes/pixel). We swap B↔R here.
|
||||
const view = frame; // Already a Uint8Array
|
||||
const data = dotImageData.data;
|
||||
// view: [char,R,G,B, char,R,G,B, ...] → data: [R,G,B,A, R,G,B,A, ...]
|
||||
for (let src = 0, dst = 0; src < view.length; src += 4, dst += 4) {
|
||||
data[dst] = view[src + 1]; // R
|
||||
data[dst + 1] = view[src + 2]; // G
|
||||
data[dst + 2] = view[src + 3]; // B
|
||||
// view: [B,G,R, B,G,R, ...] → data: [R,G,B,A, R,G,B,A, ...]
|
||||
for (let src = 0, dst = 0; src < view.length; src += 3, dst += 4) {
|
||||
data[dst] = view[src + 2]; // R (from BGR)
|
||||
data[dst + 1] = view[src + 1]; // G
|
||||
data[dst + 2] = view[src]; // B
|
||||
// Alpha already set to 255 in buildCanvas
|
||||
}
|
||||
ctx.putImageData(dotImageData, 0, 0);
|
||||
} else {
|
||||
// ── STANDARD COLOR MODES (2-5): fillText per character ──
|
||||
const view = new Uint8Array(frame);
|
||||
const view = frame; // Already a Uint8Array
|
||||
|
||||
// 1. Draw Canvas (Background)
|
||||
ctx.fillStyle = '#050505';
|
||||
|
|
|
|||
|
|
@ -34,7 +34,7 @@ class VideoDecoder:
|
|||
Both undergo the same resize operation -> size consistency guaranteed.
|
||||
"""
|
||||
|
||||
def __init__(self, path: str, cols: int, rows: int) -> None:
|
||||
def __init__(self, path: str, cols: int, rows: int, skip_gray: bool = False) -> None:
|
||||
self._cap = cv2.VideoCapture(path)
|
||||
if not self._cap.isOpened():
|
||||
raise FileNotFoundError(f"Could not open video file: {path!r}")
|
||||
|
|
@ -44,6 +44,7 @@ class VideoDecoder:
|
|||
self.vid_w : int = int(self._cap.get(cv2.CAP_PROP_FRAME_WIDTH))
|
||||
self.vid_h : int = int(self._cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
|
||||
self._size : tuple = (cols, rows)
|
||||
self._skip_gray : bool = skip_gray
|
||||
|
||||
def __iter__(self):
|
||||
return self
|
||||
|
|
@ -51,18 +52,26 @@ class VideoDecoder:
|
|||
def __next__(self) -> tuple[np.ndarray, np.ndarray]:
|
||||
"""
|
||||
:return: (gray[H,W] uint8, bgr[H,W,3] uint8)
|
||||
gray is None when skip_gray=True (pixel mode optimization)
|
||||
"""
|
||||
ok, frame = self._cap.read()
|
||||
if not ok:
|
||||
raise StopIteration
|
||||
|
||||
small = cv2.resize(frame, self._size, interpolation=cv2.INTER_LINEAR)
|
||||
if self._skip_gray:
|
||||
return None, small
|
||||
gray = cv2.cvtColor(small, cv2.COLOR_BGR2GRAY)
|
||||
return gray, small # small = downscaled BGR frame
|
||||
|
||||
def release(self):
|
||||
self._cap.release()
|
||||
|
||||
def grab(self) -> bool:
|
||||
"""Advance the video by one frame WITHOUT decoding (nearly free).
|
||||
Used by stream_server for FPS decimation of high-FPS sources."""
|
||||
return self._cap.grab()
|
||||
|
||||
def __del__(self):
|
||||
self.release()
|
||||
|
||||
|
|
@ -183,6 +192,8 @@ class TerminalRenderer:
|
|||
_CURSOR_HOME = "\033[H"
|
||||
_HIDE_CURSOR = "\033[?25l"
|
||||
_SHOW_CURSOR = "\033[?25h"
|
||||
_DISABLE_WRAP = "\033[?7l" # prevent line wrapping
|
||||
_ENABLE_WRAP = "\033[?7h" # restore line wrapping
|
||||
_BLACK_BG = "\033[40m" # black background — for contrast
|
||||
_RESET_ALL = "\033[0m"
|
||||
_CLEAR_SCREEN = "\033[2J"
|
||||
|
|
@ -194,11 +205,13 @@ class TerminalRenderer:
|
|||
path : str,
|
||||
palette : list[str] | None = None,
|
||||
quantize_bits: int = 0,
|
||||
cols : int = 0,
|
||||
) -> None:
|
||||
"""
|
||||
:param path: Path to video file
|
||||
:param palette: Custom character palette (None -> 93 levels)
|
||||
:param quantize_bits: Color quantization (0=full quality, 2=fast)
|
||||
:param cols: Fixed columns. If 0, auto-fit to terminal.
|
||||
"""
|
||||
# ── Video metadata ────────────────────────────────────────────
|
||||
_probe = VideoDecoder(path, 2, 2)
|
||||
|
|
@ -215,18 +228,29 @@ class TerminalRenderer:
|
|||
orientation = "portrait" if vid_h > vid_w else "landscape"
|
||||
aspect = vid_h / vid_w
|
||||
|
||||
if orientation == "landscape":
|
||||
cols = t_cols
|
||||
if cols > 0:
|
||||
# User provided a fixed column width
|
||||
rows = max(1, int(cols * aspect * self.CHAR_RATIO))
|
||||
if rows > t_lines:
|
||||
else:
|
||||
# Auto-fit to terminal size (with a safe maximum to prevent lag/wrapping)
|
||||
safe_cols = min(t_cols, 160) # Windows terminal often struggles above 160 cols
|
||||
|
||||
if orientation == "landscape":
|
||||
cols = safe_cols
|
||||
rows = max(1, int(cols * aspect * self.CHAR_RATIO))
|
||||
if rows > t_lines:
|
||||
rows = t_lines
|
||||
cols = max(1, int(rows / (aspect * self.CHAR_RATIO)))
|
||||
else:
|
||||
rows = t_lines
|
||||
cols = max(1, int(rows / (aspect * self.CHAR_RATIO)))
|
||||
else:
|
||||
rows = t_lines
|
||||
cols = max(1, int(rows / (aspect * self.CHAR_RATIO)))
|
||||
if cols > t_cols:
|
||||
cols = t_cols
|
||||
rows = max(1, int(cols * aspect * self.CHAR_RATIO))
|
||||
if cols > safe_cols:
|
||||
cols = safe_cols
|
||||
rows = max(1, int(cols * aspect * self.CHAR_RATIO))
|
||||
|
||||
# ── Calculate Center Padding ──────────────────────────────────────────────
|
||||
self._pad_y = max(0, (t_lines - rows) // 2)
|
||||
self._pad_x = " " * max(0, (t_cols - cols) // 2)
|
||||
|
||||
# ── Info screen ──────────────────────────────────────────────────
|
||||
print(self._CLEAR_SCREEN)
|
||||
|
|
@ -250,7 +274,7 @@ class TerminalRenderer:
|
|||
"""Main playback loop."""
|
||||
stdout = sys.stdout
|
||||
|
||||
stdout.write(self._HIDE_CURSOR + self._BLACK_BG)
|
||||
stdout.write(self._DISABLE_WRAP + self._HIDE_CURSOR + self._BLACK_BG)
|
||||
stdout.flush()
|
||||
|
||||
try:
|
||||
|
|
@ -258,6 +282,12 @@ class TerminalRenderer:
|
|||
t0 = time.perf_counter()
|
||||
|
||||
ascii_frame = self._mapper.convert(gray_frame, bgr_frame)
|
||||
|
||||
# Apply padding for centering
|
||||
if self._pad_x:
|
||||
ascii_frame = self._pad_x + ascii_frame.replace('\n', '\n' + self._pad_x)
|
||||
if self._pad_y > 0:
|
||||
ascii_frame = ('\n' * self._pad_y) + ascii_frame
|
||||
|
||||
stdout.write(self._CURSOR_HOME + ascii_frame)
|
||||
stdout.flush()
|
||||
|
|
@ -270,7 +300,7 @@ class TerminalRenderer:
|
|||
pass
|
||||
|
||||
finally:
|
||||
stdout.write(self._SHOW_CURSOR + self._RESET_ALL + "\n")
|
||||
stdout.write(self._ENABLE_WRAP + self._SHOW_CURSOR + self._RESET_ALL + "\n")
|
||||
stdout.flush()
|
||||
self._decoder.release()
|
||||
|
||||
|
|
@ -288,8 +318,10 @@ if __name__ == "__main__":
|
|||
help="Path to video file (MP4, AVI, MKV ...)")
|
||||
parser.add_argument("--palette", default=None,
|
||||
help="Custom character palette, space-separated")
|
||||
parser.add_argument("--quality", type=int, choices=[0, 1, 2, 3], default=0,
|
||||
parser.add_argument("-q", "--quality", type=int, choices=[0, 1, 2, 3], default=0,
|
||||
help="Color quality: 0=max quality, 3=max speed (default: 0)")
|
||||
parser.add_argument("-c", "--cols", type=int, default=0,
|
||||
help="Fixed grid width. If 0, auto-fits to terminal (default: 0)")
|
||||
args = parser.parse_args()
|
||||
|
||||
custom_palette = args.palette.split() if args.palette else None
|
||||
|
|
@ -299,6 +331,7 @@ if __name__ == "__main__":
|
|||
path = args.video,
|
||||
palette = custom_palette,
|
||||
quantize_bits = args.quality,
|
||||
cols = args.cols,
|
||||
)
|
||||
renderer.play()
|
||||
except FileNotFoundError as e:
|
||||
|
|
|
|||
2
play.bat
Normal file
2
play.bat
Normal file
|
|
@ -0,0 +1,2 @@
|
|||
@echo off
|
||||
python ascii_video_player2.py %*
|
||||
2
serve.bat
Normal file
2
serve.bat
Normal file
|
|
@ -0,0 +1,2 @@
|
|||
@echo off
|
||||
python stream_server.py %*
|
||||
102
stream_server.py
102
stream_server.py
|
|
@ -262,7 +262,7 @@ async def websocket_endpoint(websocket: WebSocket):
|
|||
rows = rows_cfg
|
||||
|
||||
try:
|
||||
decoder = VideoDecoder(video_path, cols, rows)
|
||||
decoder = VideoDecoder(video_path, cols, rows, skip_gray=pixel_mode)
|
||||
except FileNotFoundError:
|
||||
await websocket.send_text(f"Error: '{video_path}' not found!")
|
||||
queue_index += 1
|
||||
|
|
@ -274,27 +274,63 @@ async def websocket_endpoint(websocket: WebSocket):
|
|||
continue
|
||||
|
||||
mapper = AsciiMapper()
|
||||
fps = decoder.fps
|
||||
frame_t = 1.0 / fps
|
||||
source_fps = decoder.fps
|
||||
MAX_FPS = 30
|
||||
char_byte_lut= np.array([ord(c) for c in mapper._lut], dtype=np.uint8)
|
||||
qb = {5: 0, 4: 2, 3: 3, 2: 5}.get(render_mode, 0)
|
||||
|
||||
await websocket.send_text(f"INIT:{fps}:{render_mode}:{cols}:{rows}:{int(pixel_mode)}")
|
||||
# ── FPS DECIMATION ──
|
||||
# If source > 30 FPS, skip every Nth frame using grab() (no decode).
|
||||
# This halves CPU load for 60 FPS sources.
|
||||
if source_fps > MAX_FPS:
|
||||
skip_n = round(source_fps / MAX_FPS) # e.g. 60/30 = 2
|
||||
effective_fps = source_fps / skip_n
|
||||
else:
|
||||
skip_n = 1
|
||||
effective_fps = source_fps
|
||||
frame_t = 1.0 / effective_fps
|
||||
|
||||
await websocket.send_text(f"INIT:{effective_fps}:{render_mode}:{cols}:{rows}:{int(pixel_mode)}")
|
||||
if skip_n > 1:
|
||||
print(f"[FPS CAP] {source_fps} FPS → {effective_fps} FPS (skip every {skip_n} frames)")
|
||||
|
||||
frame_buf = np.empty((rows, cols, 4), dtype=np.uint8) if render_mode > 1 else None
|
||||
|
||||
import struct
|
||||
start_time = asyncio.get_event_loop().time()
|
||||
frame_index = 0
|
||||
|
||||
# Pre-allocate send buffer WITH header space to avoid per-frame concat
|
||||
if pixel_mode:
|
||||
# Zero-Copy Pixel: 4-byte header + raw BGR (3 bytes per pixel)
|
||||
pixel_send_buf = bytearray(4 + rows * cols * 3)
|
||||
elif render_mode > 1:
|
||||
# ASCII Color: 4-byte header + [char,R,G,B] per pixel
|
||||
ascii_send_buf = bytearray(4 + rows * cols * 4)
|
||||
|
||||
raw_frame_num = 0
|
||||
try:
|
||||
for gray_frame, bgr_frame in decoder:
|
||||
t0 = asyncio.get_event_loop().time()
|
||||
while True:
|
||||
# ── FPS DECIMATION via grab() ──
|
||||
# For 60→30 fps: grab (skip) 1 frame, then decode 1 frame.
|
||||
# grab() is ~10x faster than read() because it skips decoding.
|
||||
for _ in range(skip_n - 1):
|
||||
if not decoder.grab():
|
||||
break # EOF reached during skip
|
||||
|
||||
try:
|
||||
gray_frame, bgr_frame = next(decoder)
|
||||
except StopIteration:
|
||||
break
|
||||
|
||||
if pixel_mode:
|
||||
# Pixel Mode: skip character mapping, send raw RGB
|
||||
rgb = bgr_frame[:, :, ::-1]
|
||||
if qb > 0:
|
||||
rgb = (rgb >> qb) << qb
|
||||
frame_buf[:, :, 0] = 0xDB
|
||||
frame_buf[:, :, 1:] = rgb
|
||||
await websocket.send_bytes(frame_buf.tobytes())
|
||||
# ── ZERO-COPY PIXEL MODE ──
|
||||
# Send raw BGR bytes directly. No RGB conversion,
|
||||
# no dummy 0xDB char, no intermediate numpy copies.
|
||||
bgr_bytes = bgr_frame.tobytes()
|
||||
struct.pack_into(">I", pixel_send_buf, 0, frame_index)
|
||||
pixel_send_buf[4:] = bgr_bytes
|
||||
await websocket.send_bytes(bytes(pixel_send_buf))
|
||||
else:
|
||||
indices = np.floor_divide(gray_frame, max(1, 256 // mapper._n))
|
||||
np.clip(indices, 0, mapper._n - 1, out=indices)
|
||||
|
|
@ -302,21 +338,24 @@ async def websocket_endpoint(websocket: WebSocket):
|
|||
if render_mode == 1:
|
||||
char_matrix = mapper._lut[indices]
|
||||
lines = [''.join(row) for row in char_matrix]
|
||||
await websocket.send_text('\n'.join(lines))
|
||||
await websocket.send_text(f"{frame_index}\n" + '\n'.join(lines))
|
||||
else:
|
||||
H, W = gray_frame.shape
|
||||
char_codes = char_byte_lut[indices]
|
||||
rgb = bgr_frame[:, :, ::-1]
|
||||
if qb > 0:
|
||||
rgb = (rgb >> qb) << qb
|
||||
frame_buf[:, :, 0] = char_codes
|
||||
frame_buf[:, :, 1:] = rgb
|
||||
await websocket.send_bytes(frame_buf.tobytes())
|
||||
struct.pack_into(">I", ascii_send_buf, 0, frame_index)
|
||||
ascii_send_buf[4:] = frame_buf.tobytes()
|
||||
await websocket.send_bytes(bytes(ascii_send_buf))
|
||||
|
||||
elapsed = asyncio.get_event_loop().time() - t0
|
||||
wait = frame_t - elapsed
|
||||
elapsed = asyncio.get_event_loop().time() - start_time
|
||||
wait = (frame_index * frame_t) - elapsed
|
||||
if wait > 0:
|
||||
await asyncio.sleep(wait)
|
||||
|
||||
frame_index += 1
|
||||
|
||||
finally:
|
||||
decoder.release()
|
||||
|
|
@ -505,6 +544,33 @@ if __name__ == "__main__":
|
|||
app.state.cols = args.cols
|
||||
app.state.rows = args.rows
|
||||
|
||||
# ── High FPS Warning ──
|
||||
high_fps_videos = []
|
||||
for entry in queue:
|
||||
cap = cv2.VideoCapture(entry['video'])
|
||||
if cap.isOpened():
|
||||
fps = cap.get(cv2.CAP_PROP_FPS)
|
||||
if fps > 35: # Consider > 35 as high FPS
|
||||
high_fps_videos.append((entry['video'], fps))
|
||||
cap.release()
|
||||
|
||||
if high_fps_videos:
|
||||
print("\n\033[1;33m[WARNING] High FPS Source(s) Detected:\033[0m")
|
||||
for vid, fps in high_fps_videos:
|
||||
print(f" - \033[36m{vid}\033[0m is \033[1;31m{fps:.1f} FPS\033[0m")
|
||||
print("\033[33mASCILINE is optimized for 24-30 FPS cinematic playback.")
|
||||
print("High FPS videos will automatically be decimated to ~30 FPS,")
|
||||
print("but performance may still drop depending on the system's CPU.")
|
||||
print("For optimal performance, we recommend using 30 FPS source videos.\033[0m\n")
|
||||
|
||||
while True:
|
||||
choice = input("\033[1mDo you want to continue anyway? (y/n): \033[0m").strip().lower()
|
||||
if choice == 'y':
|
||||
break
|
||||
elif choice == 'n':
|
||||
print("Exiting...")
|
||||
exit(0)
|
||||
|
||||
# ── Startup Banner ──
|
||||
print(ASCII_LOGO)
|
||||
print(f"\033[1;37m{'═'*55}\033[0m")
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue