fq README

The project's README from github.com/wader/fq. As one markdown file: README.md.

fq

Tool, language and decoders for working with binary formats.

fq demo

Basic usage is fq . file, fq d file or fq 'some query' file ....

For details see the manual, also available as man fq and as AsciiDoc.

Background

fq is a tool, language, and decoders for working with binary formats and data. In most cases it behaves and feels similar to jq and it also uses the same expression language. To get the most out of fq it's recommended to learn more about jq.

It features a structural hex viewer, nested format decoding, slicing and concatenating binary data, bit-level decoding and an interactive REPL with auto-completion.

Goals

Hopes

Supported formats

aac_frame, adts, adts_frame, aiff, amf0, apev2, apple_bookmark, ar, asn1_ber, av1_ccr, av1_frame, av1_obu, avc_annexb, avc_au, avc_dcr, avc_nalu, avc_pps, avc_sei, avc_sps, avi, avro_ocf, bencode, bitcoin_blkdat, bitcoin_block, bitcoin_script, bitcoin_transaction, bits, bplist, bsd_loopback_frame, bson, bytes, bzip2, caff, cbor, csv, dns, dns_tcp, elf, ether8023_frame, exif, fairplay_spc, fit, flac, flac_frame, flac_metadatablock, flac_metadatablocks, flac_picture, flac_streaminfo, gif, gzip, heif, hevc_annexb, hevc_au, hevc_dcr, hevc_nalu, hevc_pps, hevc_sps, hevc_vps, html, icc_profile, icmp, icmpv6, id3v1, id3v11, id3v2, ipv4_packet, ipv6_packet, jp2c, jpeg, json, jsonl, leveldb_descriptor, leveldb_log, leveldb_table, luajit, macho, macho_fat, markdown, matroska, midi, moc3, mosaic, mp3, mp3_frame, mp3_frame_vbri, mp3_frame_xing, mp4, mpeg_asc, mpeg_es, mpeg_pes, mpeg_pes_packet, mpeg_spu, mpeg_ts, msgpack, negentropy, nes, ogg, ogg_page, opentimestamps, opus_packet, pcap, pcapng, pg_btree, pg_control, pg_heap, png, prores_frame, protobuf, protobuf_widevine, pssh_playready, rtmp, safetensors, sll2_packet, sll_packet, stl, tap, tar, tcp_segment, tiff, tls, toml, tzif, tzx, udp_datagram, vorbis_comment, vorbis_packet, vp8_frame, vp9_cfm, vp9_frame, vpx_ccr, wasm, wav, webp, xml, yaml, zip

It can also work with some common text formats like URLs, hex, base64, PEM etc and for some serialization formats like XML, YAML, etc. it can transform both from and to jq values.

For details see the manual.

Presentations and media

Install

Use one of the methods listed below or download a pre-built release for macOS, Linux or Windows. Unarchive it and move the executable to PATH etc.

On macOS if you don't install using one of the method below then you might have to manually allow the binary to run. This can be done by trying to run the binary, ignore the warning and then go into security preference and allow it. Same can be done with these commands:

xattr -d com.apple.quarantine fq
# for macOS version before Sequoia you might need this also
spctl --add fq

Homebrew (macOS)

brew install wader/tap/fq

MacPorts

On macOS, fq can also be installed via MacPorts. More details here.

sudo port install fq

Windows

fq can be installed via scoop.

scoop install fq

Arch Linux

fq can be installed from the extra repository using pacman:

pacman -S fq

You can also build and install the development (VCS) package using an AUR helper:

paru -S fq-git

Gentoo

fq can be installed from the Gentoo package repository using emerge:

emerge dev-util/fq

Guix

guix shell fq -- fq --help # in develompen shell
guix install fq            # into current profile

Nix

nix-shell -p fq

FreeBSD

Use the fq port.

Alpine

Currently in edge testing but should work fine in stable also.

apk add -X http://dl-cdn.alpinelinux.org/alpine/edge/testing fq

Build from source

Make sure you have go 1.22 or later installed.

To install directly from git repository (no git clone needed):

# build and install latest release
go install github.com/wader/fq@latest

# build and install latest master
go install github.com/wader/fq@master

# copy binary to $PATH if needed
cp "$(go env GOPATH)/bin/fq" /usr/local/bin

To build, run and test from source:

# build and run
go run .
# build and run with arguments
go run . -d mp3 . file.mp3
# just build
go build -o fq .
# run all tests and build binary
make test fq

Development and adding a new decoder

See dev.md

This project would not have been possible without itchyny's jq implementation gojq. I also want to thank HexFiend for inspiration and ideas and stedolan for inventing the jq language.

Tools

Projects and Standards

TODO and ideas

See TODO.md

License

fq is distributed under the terms of the MIT License.

See the LICENSE file for license details.

Licenses of direct dependencies:

fq (wader/fq)

xml
Searching Database…

10.6K ·GitHub
Fq
What is this?fq is a command-line tool, language and collection of decoders for working with binary formats - often described as jq for binary data. You can dump a file's structure, run jq-style queries against it and explore results in an interactive REPL.
So is it a GUI app, a code library, or something I run in the terminal?It is a standalone CLI tool written in Go, plus an interactive REPL with auto-completion. You install one executable and run queries against files straight from the terminal.
How do I install it and take a first look at a file?Install via Homebrew on macOS, scoop on Windows, pacman or AUR on Arch, packages on Gentoo, Guix, Nix, FreeBSD and Alpine, or grab a pre-built release binary for macOS, Linux or Windows. Then run fq . file to dump a file's structure or fq 'some query' file to filter.
Do I need to know jq to get anything out of it?It helps: fq uses the same expression language as jq, and the README recommends learning jq to get the most out of the tool. Building from source also requires Go 1.22 or later; the pre-built binaries have no such requirement.
Are there any install gotchas?On macOS, a manually downloaded binary may be blocked on first run - you need to allow it in Security preferences (the README shows the exact commands). Beyond that, installation is straightforward across macOS, Linux and Windows.
What kinds of files can it decode?Its decoder list covers media (mp4, mp3, flac, jpeg, png, gif, webp, wav), executables (elf, macho), archives (zip, tar), network captures and protocols (pcap, pcapng, DNS, TLS, IPv4/IPv6), serialization formats (protobuf, bson, cbor, msgpack) and more. It also works with text like URLs, hex, base64 and PEM, and can convert XML and YAML to and from jq values.
What can it do with a file once it is open?It shows a structural hex viewer, decodes nested formats, supports slicing and concatenating binary data and bit-level decoding, and offers an interactive REPL with auto-completion for exploring the results.
Why did the author build it?The stated goals are to make binaries more accessible, queryable and sliceable, with nested formats and bit-oriented decoding in a quick, comfortable CLI tool. The project builds on itchyny's gojq implementation and took inspiration from HexFiend and the jq language.
How does it compare to Wireshark, ImHex or HexFiend?The README lists them as related works with different focus: HexFiend is a macOS hex editor with format template support, ImHex a hex editor for reverse engineers, Wireshark decodes network traffic, and kaitai does declarative binary format parsing. fq aims to combine a jq-like query language, a REPL and a broad decoder set in one CLI tool.
What are similar projects?yq, brook, jq, scapy, grpcurl, liteparse, zenc, fx
Which topics is it in?packet-capture, repl, codec, xml, media, parsing, binary-analysis, parser, protocol, tls, reverse-engineering, json
Creator
wader
Size
53.2 MB
Stars
10.6K
Updated
CommitOct 16Releasev0.18.0Aug 25
Age
5y
Commits · 90 Days