YAML pitfalls: the Norway problem, tabs and anchors

YAML is pleasant to write because it has almost no punctuation. The same feature makes it easy to write something that means a different thing from what you see. Most YAML bugs are not syntax errors at all: the file parses, and a value quietly changes type. This article walks through the traps that cause real incidents, with the exact output of a parser for each, and ends with a short routine for converting YAML to JSON without surprises. Every example can be pasted into the YAML ⇄ JSON converter.

Two versions of YAML are still in use

YAML 1.1 (2005) and YAML 1.2 (2009, revised as 1.2.2 in 2021) disagree on how unquoted values are typed. YAML 1.2 uses a small "core schema": true/false are booleans, null and ~ are null, things that look like numbers are numbers, and everything else is a string. YAML 1.1 had many more implicit types. The trouble is that a large share of the software reading your YAML still follows 1.1 rules: PyYAML, for example, and older Ruby and Go libraries. The same file can therefore mean two different things depending on which program reads it.

The Norway problem

Here is a list of country codes:

countries: [GB, IE, NO, SE]

A YAML 1.2 parser reads four strings. A YAML 1.1 parser reads "GB", "IE", false, "SE", because in 1.1 the words y, yes, n, no, on and off (in any of three capitalisations) are booleans. The same happens to a feature flag set to on, an answer field set to no, or a setting called Y. The converter flags each of these with a warning naming the line, even though under 1.2 they are valid strings.

The fix is to quote: [GB, IE, "NO", SE] reads the same everywhere. When the tool writes YAML from JSON, it quotes every string that any version could misread, which is why "NO", "on" and "yes" come out in quotes.

Numbers that are not what they look like

• Leading zeros. mode: 0644 is the number 644 in YAML 1.2, and the octal number 0644, which is 420, in YAML 1.1. For a file permission that is the difference between rw-r--r-- and a nonsense mode. Write 0o644 (YAML 1.2 octal), or better, quote it and convert in code.
• Version strings. version: 1.10 is the float 1.1. Your release "1.10" becomes "1.1" the moment any program reads it as a number. Quote versions.
• Times. start: 12:30 was the base-60 number 750 in YAML 1.1 (12 × 60 + 30); in 1.2 it is a string. Quote it.
• Large integers. id: 12345678901234567890 is a valid integer, but JavaScript, and therefore most JSON tooling, cannot store it exactly: it becomes 12345678901234567000. The converter keeps the digits in its JSON text, but the program that reads the JSON may not. Send IDs as strings.
• Infinity and NaN. .inf and .nan are YAML floats with no JSON equivalent; the converter writes them as null and tells you how many it replaced.

Indentation and tabs

Indentation is the structure. Two rules catch people out:

1. Tabs are forbidden for indentation. A tab copied from an editor makes the file invalid, and the error appears on a line that looks fine. The converter reports A tab is used for indentation with the line number.
2. A line that is indented by one space too many changes meaning. In this file the c line belongs to nothing:

a:
  b: 1
   c: 2

A plain scalar can continue on the next line if that line is more indented, so the parser reads the value of b as 1 c, then meets a colon and gives up. The error points at line 3. The reverse mistake, one space too few, silently moves a key up a level, which no parser can detect. After converting, look at the JSON nesting: it shows the structure that was actually written.

Lists under a key may sit at the same indentation as the key or one level deeper; both are valid YAML, and the converter's "indent lists under their key" option chooses which one it writes.

Anchors, aliases and merge keys

Anchors (&name) and aliases (*name) let you reuse a block, and the << merge key copies the keys of a mapping into another:

defaults: &defaults
  image: shop-api:1.8.2
  replicas: 2
production:
  <<: *defaults
  replicas: 6

In JSON the result is two complete objects; production.replicas is 6, because a key written explicitly always wins over a merged one. Two cautions. First, << was defined for YAML 1.1 and is not part of the 1.2 core schema, so a strict 1.2 parser may treat it as an ordinary key named <<. Second, aliases can be nested: nine levels of ten aliases each, over ten strings, expand to a billion values from a file of a few hundred bytes. This "billion laughs" pattern is a real denial-of-service attack, so the converter counts expanded values and stops at a million.

Converting YAML to JSON safely

1. Convert, then read the JSON, not the YAML. Types are explicit in JSON: false versus "NO", 1.1 versus "1.10".
2. Read the warnings. A warning means a different program may read your file differently.
3. Quote anything that is not obviously a number or a boolean: country codes, versions, times, postcodes, IDs and file modes.
4. Check the nesting against what you intended, especially after editing indentation by hand.
5. Keep comments in the source. JSON has no comments, so converting and converting back drops them.

Common mistakes

• Trusting a YAML linter that uses the other version of the rules.
• Writing on: as a key in a CI file and assuming every tool treats it as the string "on". Some YAML 1.1 readers turn the key into true.
• Using yes/no for booleans because the file "looks friendlier"; use true/false.
• Quoting the key but not the value, or the other way round, for the same kind of data.
• Pasting YAML into a chat or issue tracker that converts spaces to tabs or non-breaking spaces.

Paste your own file into the YAML ⇄ JSON converter to see its exact data and any version-dependent values, and use the JSON formatter to inspect large JSON results.