Desktop windows: frames, input and the tray

A program with a window needs three things from the operating system: a rectangle on the screen, a way to put pixels into it, and a stream of events telling it what the user did (a key, a click, a resize, the close button). kola gives you exactly those three and nothing on top of them. There are no buttons, text boxes or fonts in the standard library. What you get is a framebuffer, a block of memory where every pixel of the window is a few bytes you write yourself, plus the events, plus a notification-area icon with a small menu. The sections below take them in turn: opening a window, drawing a frame, reading input, testing a window program without a screen, the tray, and the layer underneath that talks to the display.

The Window capability

On a desktop, a program does not draw on the screen directly. A separate program owns the screen: on Linux a compositor (Wayland) or an X server, on Windows and macOS a system component. Your program connects to it, hands it finished images of its window, and receives the user's input from it.

In kola, permission to do any of that is a capability, sys.window, of type Window. As with the clock or the filesystem, main receives it inside sys and nothing else can create one, so a function that never receives a &Window cannot open a window, and a reviewer can see that from its signature alone. Window is an authority token: it grants permission and owns nothing, so you may drop it on any path.

The window code itself lives in the standard library module std::gui, written in kola. A program that opens a window has to bring it in with use std::gui;. If you forget, the checker says so and names the repair:

fn main(sys: Sys) -> u8 !{io, block} {
    match sys.window.open(8u32, 8u32) {
        Ok(w) => { w.close(); 0u8 },
        Err(_) => 1u8,
    }
}

error[card §7]: `sys.window.open` is `std::gui::open`, and this program does not link `std::gui`
  --> e.kl:2:11
   |
 2 |     match sys.window.open(8u32, 8u32) {
   |           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
   = help: add `use std::gui;` — a window, its `Surface` and `WinErr` all live there

Opening a window and drawing a frame

sys.window.open(width, height) returns Result[WindowHandle, WinErr]. Each side must be between 1 and 1,000,000 pixels (a Wayland window accepts up to 16,384), otherwise the answer is WinErr::Invalid. WinErr has three variants: Unsupported (there is no display to open a window on), Invalid (the request itself is wrong) and Busy (try again later). WindowHandle is a resource: the checker makes sure every path either calls w.close() or passes the handle on.

The window's pixels are a frame of shared memory. You do not write into it whenever you like; you borrow it, write, and give it back. w.surface() moves the frame out of the window into a Surface, a linear value, and w.present(s) takes it back and shows it on the screen. Between the two calls the frame is yours and the window has no frame, which is what makes the arrangement safe: the compositor never reads a half-drawn image, and you never write into memory it is showing.

Each pixel is four bytes in the format called XRGB8888: a 32-bit word with blue in the low byte, then green, then red, then a byte that is ignored. On every machine kola targets, the word is stored low byte first, so in memory a pixel reads B, G, R, X. The bytes of one row of pixels are followed directly by the next row; the number of bytes per row is the stride, and s.stride() returns it (always width * 4).

The cheapest complete example is a test, because tests run against a simulated window that needs no screen (more on that below):

use std::gui;
use std::gui::WindowEvent;
use std::gui::WinErr;

// Paint every pixel of a lent frame one colour. A pixel is four bytes in
// memory order B, G, R, X (XRGB8888 read as a little-endian 32-bit word).
fn fill(px: &mut [u8], r: u8, g: u8, b: u8) !{} {
    let mut i = 0usize;
    while i + 3usize < px.len {
        px[i] = b;
        px[i + 1usize] = g;
        px[i + 2usize] = r;
        px[i + 3usize] = 0u8;
        i = i + 4usize;
    }
}

test first_frame(mut sim: Sim) {
    let mut w = sim.window.open(64u32, 48u32).unwrap();
    match w.next_event() {
        Some(WindowEvent::Resized) => {},
        _ => { assert false; },
    }
    let mut s = w.surface().unwrap();
    assert s.stride() == 256u32;
    let mut px = s.frame.bytes();
    assert px.len == 12288usize;
    fill(px, 18u8, 52u8, 86u8);
    w.present(s).unwrap();
    let (width, height) = w.size();
    assert width == 64u32;
    assert height == 48u32;
    w.close();
}

test bounds(mut sim: Sim) {
    match sim.window.open(0u32, 10u32) {
        Ok(w) => { w.close(); assert false; },
        Err(WinErr::Invalid) => {},
        Err(_) => { assert false; },
    }
}

Run it with kola test (add --oracle to compare the interpreter with the compiled build). A 64 by 48 window has a stride of 256 bytes and a frame of 12,288 bytes, which is what the two asserts check.

w.surface() can fail in two ways. Invalid means the frame is already lent to you. Busy means the compositor has not finished with the last frame you presented; it is still copying it to the screen. Busy is not an error in the usual sense, only a signal to draw a little later, and the event loop below handles it that way.

A window cannot be closed while its frame is lent

Because the Surface is linear and present is the only function that consumes it, closing the window while you still hold its frame is not a run-time error. It does not compile:

use std::gui;
test bad(mut sim: Sim) {
    let mut w = sim.window.open(10u32, 10u32).unwrap();
    let s = w.surface().unwrap();
    w.close();
}

error[card §2]: resource `s` is not consumed on this path
  --> d.kl:4:5
   |
 4 |     let s = w.surface().unwrap();
   |     ^^^
   = help: no implicit destructors: move it on, return it, destructure it, or pass it to a consuming fn

In C with a typical windowing library, the same mistake leaves a pointer into freed shared memory, and the crash, if any, comes later and somewhere else. Here the fix is visible at the point of the error: present the frame first, then close the window.

Events

w.next_event() returns Option[WindowEvent]. It never waits: if nothing has happened, it answers None straight away. A window program is therefore a loop that takes every pending event, redraws if something changed, and sleeps for a few milliseconds when there is nothing to do.

mods is a bit mask of the modifier keys held at the time: 1 Shift, 2 Ctrl, 4 Alt, 8 Super (the Windows or Command key), 16 Caps Lock. Test a bit with (mods & 2u32) != 0u32.

Keys are reported as Linux evdev codes, the numbers the Linux kernel's input layer gives each physical key, on Windows and macOS as well, so one table serves every platform: Escape is 1, the digits 1 to 0 are 2 to 11, A is 30. A key code names a position on the keyboard, not a letter. On a French keyboard the key that Linux calls code 16 is labelled A, while on a US keyboard it is Q. For the character, use the Text event. std::gui::us_char(key, mods) converts a code to the ASCII character it produces on a US layout (0 when it produces none or when Ctrl or Alt is held), which is fine for keyboard shortcuts and wrong for text entry.

An input method (IME) is the system component people use to type languages with more characters than keys, such as Chinese or Japanese: the user types several keys, sees a provisional string, and confirms it. ImePreedit(s, active) reports the provisional string while it is being composed; ImeCommit(s) delivers the confirmed text. A program that only needs ASCII can ignore both.

The text events carry their string in the heap region, so next_event declares alloc(heap) in its effect row, and so must the function that calls it.

Here is a complete program that opens a real window, fills it with one colour, darkens the blue on each left click, and exits on Escape or the close button:

use std::gui;
use std::gui::WindowEvent;
use std::gui::WinErr;
use std::log;

fn main(sys: Sys) -> u8 !{io, block, alloc(heap), log} {
    let mut w = match sys.window.open(320u32, 200u32) {
        Ok(w) => w,
        Err(WinErr::Unsupported) => { log::error(f"no display to open a window on"); return 1u8; },
        Err(_) => { return 2u8; },
    };
    let mut shade = 0u8;
    let mut dirty = true;
    loop {
        match w.next_event() {
            Some(WindowEvent::CloseRequested) => { w.close(); return 0u8; },
            Some(WindowEvent::Resized) => { dirty = true; },
            Some(WindowEvent::KeyDown(1u32, _)) => { w.close(); return 0u8; },
            Some(WindowEvent::PointerButton(1u8, true, _)) => { shade = shade +% 32u8; dirty = true; },
            Some(_) => {},
            None => {
                if dirty {
                    match w.surface() {
                        Ok(s) => {
                            let mut s = s;
                            let mut px = s.frame.bytes();
                            let mut i = 0usize;
                            while i + 3usize < px.len {
                                px[i] = shade;
                                px[i + 1usize] = 96u8;
                                px[i + 2usize] = 160u8;
                                px[i + 3usize] = 0u8;
                                i = i + 4usize;
                            }
                            match w.present(s) { Ok(_) => { dirty = false; }, Err(_) => {} }
                        },
                        Err(WinErr::Busy) => {},
                        Err(_) => { w.close(); return 3u8; },
                    }
                }
                sys.clock.sleep(10u64);
            },
        }
    }
}

A few details in it are worth a second look:

• shade +% 32u8 is wrapping addition. A plain + on a u8 traps on overflow, which here would stop the program on the eighth click.

• A match arm cannot bind a name as mut, so the arm binds s and rebinds it with let mut s = s;.

• Err(WinErr::Busy) does nothing: the frame stays dirty and the loop tries again after the next sleep.

• A redraw happens only when something changed. Redrawing on every pass would keep the CPU busy for nothing.

Build it with kola build --link win win.kl and run it on a desktop. With no display at all (a server, an SSH session), open answers Unsupported and the program logs the error and exits with 1. kola never pretends: outside a test there is no invisible window that accepts frames and reports success.

Testing a window program without a screen

Under kola test, and under kola run --sim, the window capability is a deterministic twin of a real window: it holds a frame in ordinary memory, reports one Resized when it opens, and after that reports only what the test puts into its queue. Two methods exist for that, and only on the twin (on a real window both return false):

• w.inject(e) queues an event e, which the next next_event returns.

• w.inject_size(width, height) resizes the window as a compositor would and queues a Resized (each side 1 to 16,384).

This makes the event handling of a window program testable on a build server that has no display:

use std::gui;
use std::gui::WindowEvent;

test keys_and_resize(mut sim: Sim) {
    let mut w = sim.window.open(8u32, 8u32).unwrap();
    let _ = w.next_event();
    let k = w.inject(WindowEvent::KeyDown(30u32, 1u32));
    let c = w.inject(WindowEvent::CloseRequested);
    assert k && c;
    match w.next_event() {
        Some(WindowEvent::KeyDown(key, mods)) => {
            assert key == 30u32;
            assert gui::us_char(key, mods) == 65u8;
        },
        _ => { assert false; },
    }
    match w.next_event() {
        Some(WindowEvent::CloseRequested) => {},
        _ => { assert false; },
    }
    let resized = w.inject_size(100u32, 50u32);
    assert resized;
    match w.next_event() {
        Some(WindowEvent::Resized) => {},
        _ => { assert false; },
    }
    let (nw, nh) = w.size();
    assert nw == 100u32;
    assert nh == 50u32;
    let s = w.surface().unwrap();
    assert s.stride() == 400u32;
    w.present(s).unwrap();
    w.close();
}

Key 30 with Shift (mods 1) is a capital A, character code 65. After the resize the next frame is 100 pixels wide, so its stride is 400 bytes.

Where the window comes from

std::gui hides the platform, but you should know what it picks, because that decides what works where. sys.window.open tries, in this order:

On Linux the protocols are plain messages on a socket, so the clients are ordinary kola code in std and run identically under the interpreter, the C backend and the own backend. On Windows and macOS the window system is a library, not a protocol, so the third backend calls that library. In all three cases the program sees the same events and the same frame format.

Some limits follow from that choice:

• On Wayland and X11 the Text event uses a built-in US layout; no keyboard layout description is read from the system. Ctrl and Alt chords produce no Text.

• An X display reached over TCP (DISPLAY=host:0) is Unsupported; only local sockets are used. A server that requires a cookie (the Xauthority file) is supported.

• On macOS the window must be served from the process's main thread, and a window server must be present (a plain SSH session has none). macOS has no input-method events here, and a dead key (one that adds an accent to the next letter) produces no Text.

• The first event on a native window is a Resized to the window's real client size, which may differ from what you asked for.

The tray

The tray, or notification area, is the strip of small icons in a corner of the desktop panel: a network indicator, a messenger, a clock. A program can put an icon there with a small menu, which suits background tools that have no main window at all.

sys.window.tray(icon, width, height, tip) returns Result[TrayHandle, WinErr]:

• icon holds width * height pixels of four bytes each, in the order B, G, R, A, where A is opacity (alpha). The colours must be premultiplied by the alpha: a half-transparent pixel stores half of its red, green and blue. Each side is 1 to 256.

• tip is the tooltip text, UTF-8, at most 127 bytes.

• t.item(id, label) appends a menu item. id is your number for it (1 or more), label is at most 255 bytes, and an empty label makes a separator line. A menu holds at most 64 items.

• t.next_event() returns Option[TrayEvent] without waiting: TrayEvent::Activated when the user clicks the icon itself, TrayEvent::Item(id) when they choose a menu item.

• t.close() removes the icon; it is the only way to dispose of the linear TrayHandle.

Under a test the tray is a twin as well, and t.inject(e) queues an event for it:

use std::gui;
use std::gui::TrayEvent;

test tray_menu(mut sim: Sim) {
    // a 2 x 2 icon: four opaque pixels, bytes B, G, R, A
    let icon: [u8; 16] = [86u8, 52u8, 18u8, 255u8, 86u8, 52u8, 18u8, 255u8,
                          86u8, 52u8, 18u8, 255u8, 86u8, 52u8, 18u8, 255u8];
    let mut t = sim.window.tray(&icon, 2u32, 2u32, &b"kola demo").unwrap();
    t.item(1u32, &b"Open").unwrap();
    t.item(2u32, &b"").unwrap();
    t.item(3u32, &b"Quit").unwrap();
    let q = t.inject(TrayEvent::Item(3u32));
    assert q;
    match t.next_event() {
        Some(TrayEvent::Item(id)) => { assert id == 3u32; },
        _ => { assert false; },
    }
    match t.next_event() {
        None => {},
        Some(_) => { assert false; },
    }
    t.close();
}

A real tray program is the same shape as the window loop: take events until the user picks Quit, sleep briefly when there are none. Each platform implements the tray differently, and the differences show:

The layer underneath

The Wayland and X11 clients are kola code, so they need a few primitives below them. These are methods on Window and are available to your own code too, for example if you want to write a client for a protocol std::gui does not speak. None of them speaks a protocol; they only open the channels:

• sys.window.connect_display() connects to the Wayland compositor's socket ($XDG_RUNTIME_DIR/$WAYLAND_DISPLAY) and returns a Conn, the same connection type networking uses. It needs the Window capability, not a network grant.

• sys.window.shm(len) creates a block of shared memory, a linear Shm. shm.bytes() lends it as &mut [u8], shm.close() releases it, and conn.write_fd(bytes, &shm) sends a message together with the memory's file descriptor, which is how a Wayland client hands its frame to the compositor.

• sys.window.connect_x11() connects to the X server named by $DISPLAY, and sys.window.x11_authority(out) reads the Xauthority file that holds its access cookie.

• sys.window.connect_bus() connects to the D-Bus session bus the Linux tray uses.

• sys.window.native_open(width, height) and sys.window.native_tray(...) are the native window and tray on Windows and macOS, under std::gui's own window and tray.

Under Sim, and on platforms where they do not apply, these answer Unsupported. Their exact signatures are in the GUI section of STD-CARD.md (kola help std-card).

What std::gui does not do

std::gui stops at pixels and events on purpose. It has no widgets, no layout, no text rendering and no image decoding; a program that needs a button draws a rectangle and checks whether a click landed inside it. It has no drag and drop, no clipboard and no multi-window coordination. Those are jobs for a library on top of it, and because the layer below is a plain frame and a plain event, such a library is ordinary kola code that is tested against the twin like the examples above.