audio/music

A music file describes a track, a stem, or a mix. Use it for any file where the music is the point. It records which recording the file holds or which song it covers, which part of that recording each stretch of the file plays, and the facts that an editor cuts to: tempo, key, beats, and an energy curve. Every field is measured.

envelope the same fields in every typemeasured computed from the file's bytes; the same value every timeobserved what a model saw or heard; never a recommendationmixed some parts measured and some observed; the description says whichderived computed from other fields; the writer regenerates it and never edits itinstant · t one point in timespan · s, e a start time and an end timespans inside applies to the whole file and contains a list of timed itemst, or t and e an instant, or a span if it has an end timeNo mark: the field has no time of its own and applies to the whole file.

low-tide-chorus.wav.analyzemediaone JSON object next to the original

From top to bottom: the envelope with the file's own tags, the levels, the music, and a flat timeline. Nothing in this type is observed.

The envelopealways present, in this order

format

"analyzemedia"

Always this string.

version

"0.1"

Within a major version, changes only add fields.

type

"audio/music"

The writer declares the type. Readers don't second-guess it.

generatedmeasured

{ at, by, models }

When the writer made the file, which writer, and which models it used. Nothing per item.

sourcethe file and its tags; the writer keeps the tags even when recognition disagrees with them

name, bytes, sha256, quickHashmeasured

The hashes bind the sidecar to the file. A renamed file still matches. A different file doesn't.

container, duration, audiomeasured

audio { codec, channels, sampleRate, bitRate }

What the container reports.

tagsmeasured

{ title, artist, album, albumArtist, track, disc, year, genre, composer, comment, artwork }

The container's own tags, present only when the file has them. artwork holds { mime, bytes, width, height }, not the image bytes.

recordedmeasured

{ utc, local, offset, zoneSource }

Present when the file carries a capture time. Audio has no location, so there is no time of day.

signalwhole-file levels; every value is a fact of the samples

loudnessmeasured

{ integrated, truePeak, range }

LUFS, dBTP, and LU, as broadcast meters report them.

noiseFloor · dynamicRangemeasured

dBFS · dB

noiseFloor is the level of the quietest sustained stretch. dynamicRange is the peak minus the noise floor.

clipping[]measuredspan · s, e

[ { s, e } ]

Spans at full scale. The list is empty when the recording is clean.

waveformmeasuredspans inside

{ step, peaks [] }

The peak level per step seconds, as fractions of full scale. This is enough to draw an overview without decoding the file.

musicalways present, and measured throughout

The whole file is music, so there is no list of where music plays. A single track has one song. A mix has many. The catalog says what the recording is. The samples say how it moves.

songs[]measuredspans inside

[ { label, pieces [ { s, e, at } ], seconds, tempo, key, keyConfidence, recognition } ]

One entry per distinct recording, lettered. pieces lists the spans of the file where the recording plays. Each piece has at: the position in the recognized recording that plays at s. tempo is in BPM. key reads like "F# minor", with a confidence from 0 to 1.

songs[].recognitionmeasured

{ match, title, artists [], album, releaseDate, label, duration, language, genres [], isrc, upc, score, confidence, matchedAt, spotify, youtube, deezer, musicbrainz } | null

match is recording when the file holds that exact recording, and cover when the file holds another performance of the same song. confidence is identified (a score of 70 or higher) or likely (a score from 40 to 69). A null value means that the catalog doesn't have the recording. This field names the recording. It says nothing about rights.

beats[]measuredinstant · t

[ t, … ]

Every beat in the whole file.

energymeasuredspans inside

{ step, values [] }

The short-term loudness per step seconds, from 0 to 1 relative to the file's own maximum. This is the curve that an editor cuts to.

events[]derivedt, or t and e

[ { t, e, kind, text } ]

One music event per piece, and silences as silence.

Which part of the song is playing

This is the one idea in this type that needs a drawing. A file rarely holds a whole recording. It might hold a chorus edit, a stem, or a clip. pieces[].at says where in the recognized recording each span of the file starts. recognition.duration says how long the recording is.

To find which part of the song plays at any moment, subtract. Second 30 of the file is second 150 of the recording.

The same file on its own timeline

Everything here is measured: the piece, the energy curve, every beat at 96 BPM, and the waveform. A music file has no summary and no observed field. Its recognition is its one-line description.

A music file answers the question "what is this, and which part of it is playing". A music file never holds speech, summary, people, sections, instruments, or lyrics. The writer keeps source.tags even when recognition disagrees with them. The tags are what the file says about itself. The recognition is what the sound is.

Open questions

energy and waveform These are two measured curves over the same file. The waveform is the peak level. The energy is the short-term loudness relative to the maximum. One curve is enough to cut to the music. The energy curve is more useful, and the waveform could stay only in speech.
recognition The catalog returns eighteen fields. A reader uses title, artists, match, duration, confidence, and the ids. The rest could be dropped without anyone noticing.
events[] With one piece and no speech, the events list has one entry. It exists for consistency with the other types.

Rules you can rely on

Time Times are seconds from the start, with two decimals. Spans have s and e. Instants have t. The format doesn't use timecodes.
Identity Ids start at 1 in time order. People and songs are letters in order of first appearance.
Absent versus empty A missing section means that the writer didn't analyze it. An empty section means that the writer analyzed it and found nothing.
Measured versus observed The grade is a property of the field, stated for every field. No field holds an opinion about quality, importance, or intent.
Vocabularies Each vocabulary is a closed list that includes other. Read an unknown value as other.
Types are declared A file has the type that it declares. The format defines no test to tell types apart.