Zig breaking change – initial Writergate

Previous Scandal

Summary

Deprecates all existing std.io readers and writers in favor of the newly provided std.io.Reader and std.io.Writer which are non-generic and have the buffer above the vtable - in other words the buffer is in the interface, not the implementation. This means that although Reader and Writer are no longer generic, they are still transparent to optimization; all of the interface functions have a concrete hot path operating on the buffer, and only make vtable calls when the buffer is full.

I have a lot more changes to upstream but it was taking too long to finish them so I decided to do it more piecemeal. Therefore, I opened this tiny baby PR to get things started.

These changes are extremely breaking. I am sorry for that, but I have carefully examined the situation and acquired confidence that this is the direction that Zig needs to go. I hope you will strap in your seatbelt and come along for the ride; it will be worth it.

The breakage in this first PR mainly has to do with formatted printing.

Upgrade Guide

Turn on -freference-trace to help you find all the format string breakage.

• std.fs.File.reader -> std.fs.File.deprecatedReader
• std.fs.File.writer -> std.fs.File.deprecatedWriter
• std.fmt.format -> std.fmt.deprecatedFormat
• std.fmt.fmtSliceEscapeLower -> std.ascii.hexEscape
• std.fmt.fmtSliceEscapeUpper -> std.ascii.hexEscape
• std.fmt.fmtSliceHexLower -> {x}
• std.fmt.fmtSliceHexUpper -> {X}
• std.fmt.fmtIntSizeDec -> {B}
• std.fmt.fmtIntSizeBin -> {Bi}
• std.fmt.fmtDuration -> {D}
• std.fmt.fmtDurationSigned -> {D}
• {} -> {f} when there is a format method. This prevents footguns when adding or deleting format methods.
• format method signature
  • anytype -> *std.io.Writer
  • inferred error set -> error{WriteFailed}
  • FormatOptions -> (deleted)
• std.fmt.Formatter
  • now takes context type explicitly
  • no fmt string

These are deprecated but not deleted yet:

• std.io.GenericReader -> std.io.Reader
• std.io.GenericWriter -> std.io.Writer
• std.io.AnyReader -> std.io.Reader
• std.io.AnyWriter -> std.io.Writer

If you have an old stream and you need a new one, you can use adaptToNewApi() like this:

fn foo(old_writer: anytype) !void {
    var adapter = old_writer.adaptToNewApi();
    const w = &adapter.new_interface;
    try w.print("{s}", .{"example"});
    // ...
}

New API

Formatted Printing

• {t} is shorthand for @tagName() and @errorName()
• {b64}: output string as standard base64

std.io.Writer and std.io.Reader

These have a bunch of handy new APIs that are more convenient, perform better, and are not generic. For instance look at how reading until a delimiter works now.

These streams also feature some unique concepts compared with other languages' stream implementations:

• The concept of discarding when reading: allows efficiently ignoring data. For instance a decompression stream, when asked to discard a large amount of data, can skip decompression of entire frames.
• The concept of splatting when writing: this allows a logical "memset" operation to pass through I/O pipelines without actually doing any memory copying, turning an O(M*N) operation into O(M) operation, where M is the number of streams in the pipeline and N is the number of repeated bytes. In some cases it can be even more efficient, such as when splatting a zero value that ends up being written to a file; this can be lowered as a seek forward.
• Sending a file when writing: this allows an I/O pipeline to do direct fd-to-fd copying when the operating system supports it.
• The stream user provides the buffer, but the stream implementation decides the minimum buffer size. This effectively moves state from the stream implementation into the user's buffer

std.fs.File.Reader

Memoizes key information about a file handle such as:

• The size from calling stat, or the error that occurred therein.
• The current seek position.
• The error that occurred when trying to seek.
• Whether reading should be done positionally or streaming.
• Whether reading should be done via fd-to-fd syscalls (e.g. sendfile)
• versus plain variants (e.g. read).

Fulfills the std.io.Reader interface.

This API turned out to be super handy in practice. Having a concrete type to pass around that memoizes file size is really nice.

std.fs.File.Writer

Same idea but for writing.

What's NOT Included in this Branch

This is part of a series of changes leading up to "I/O as an Interface" and Async/Await Resurrection. However, this branch does not do any of that. It also does not do any of these things:

• Rework tls
• Rework http
• Rework json
• Rework zon
• Rework zstd
• Rework flate
• Rework zip
• Rework package fetching
• Delete fifo.LinearFifo
• Delete the deprecated APIs mentioned above

I have done all the above in a separate branch and plan to upstream them one at a time in follow-up PRs, eliminating dependencies on the old streaming APIs like a game of pick-up-sticks.

Merge Checklist:

• • I'd rather not have this field but I don't see how to get rid of it