Better nom Error Messages with nom_locate

UPDATE: The original version of this post only passed the name of the last parser in the chain. This version is updated to show all the parsers that lead to the error.

Parsing Parser Errors

I designed a note taking format called NeoDoc^nd. It currently exists only as an embedded feature of my personal static site generator. I'm in the process of extracting it to its own library. That means digging into the nom^nom parser/combinator and its cryptic as hell error messages.

"Just how cryptic?" you ask. Let me give you an example.

A First Look

This codes makes a tiny rust app that uses nom's alt feature to attempt to use two child parsers to process a piece of text.

[package]
name = "default_errors"
version = "0.1.0"
edition = "2024"

[dependencies]
nom = "8.0.0"

use nom::branch::alt;
use nom::bytes::complete::tag;
use nom::{IResult, Parser};
use std::fs;

fn main() {
  let input = "charlie delta";
  let result = alt((alfa, bravo)).parse(input);
  fs::write("output.txt", format!("{:#?}", result))
    .unwrap();
}

fn alfa(input: &str) -> IResult<&str, &str> {
  let (input, result) = tag("xxx").parse(input)?;
  Ok((input, result))
}

fn bravo(input: &str) -> IResult<&str, &str> {
  let (input, result) = tag("yyy").parse(input)?;
  Ok((input, result))
}

The parsers attempt to process the words "charlie delta" as input. They both fail since neither of their patterns (i.e. "xxx" or "yyy") match the word. The result is that nom throws this error:

Err(
    Error(
        Error {
            input: "charlie delta",
            code: Tag,
        },
    ),
)

• The input: "charlie delta" shows us what the parser attempted to process.

• The code: Tag lets us know that the error was generated because on of the attempts to use a .tag() failed.

  Critically, it doesn't tell us which one of them failed.

Figuring out the issue in this example is no big deal. It's quick to see that neither the alfa nor the bravo parsers match the "charlie delta" input passed to them. Things aren't so obvious in real world parsers. There are tons of different functions branching out and interacting with each other. It doesn't take long to cross a threshold where figuring out where an error occurred is a non-trivial task.

I used to use a crate called nom_supreme^ns to help with this. It provided errors with more details about where they came from. Unfortunately, it doesn't work with the most recent version of nom.

That's where the nom_locate crate enters the scene.

Locating Errors

nom_locate^nl works by adding a LocatedSpan struct to the mix. Your input goes inside and gets passed around as a passenger. For example:

[package]
name = "locate_basic"
version = "0.1.0"
edition = "2024"

[dependencies]
nom = "8.0.0"
nom_locate = "5.0.0"

use nom::bytes::complete::tag;
use nom::{IResult, Parser};
use nom_locate::LocatedSpan;
use std::fs;

type Content<'a> = LocatedSpan<&'a str, Vec<&'a str>>;

fn main() {
  let input = "alfa not bravo";
  let content = Content::new_extra(input, vec![]);
  let result = alfa_parser(content);
  fs::write("output.txt", format!("{:#?}", result))
    .unwrap();
}

fn alfa_parser(
  mut content: Content
) -> IResult<Content, ()> {
  content.extra.push("alfa_parser");
  let (content, _) = tag("alfa ").parse(content)?;
  let (content, _) = bravo_parser.parse(content)?;
  Ok((content, ()))
}

fn bravo_parser(
  mut content: Content
) -> IResult<Content, ()> {
  content.extra.push("bravo_parser");
  let (content, _) = tag("xxx").parse(content)?;
  Ok((content, ()))
}

That codes contains an intentional error where the bravo_parser fails. This is the error that gets produced:

Err(
    Error(
        Error {
            input: LocatedSpan {
                offset: 5,
                line: 1,
                fragment: "not bravo",
                extra: [
                    "alfa_parser",
                    "bravo_parser",
                ],
            },
            code: Tag,
        },
    ),
)

The input of the Error now contains a LocatedSpan with more details about where the error occurred. The extra field contains the values we appended to its Vec on lines 19 and 28. Collectively they give us the path to the parser that failed.

That's a huge improvement. And, it gets even better.

Showing Off Errors

We'll use a little more complicated example to demonstrate what nom_locate can do. It starts by defining two parser functions:

[package]
name = "locate_report"
version = "0.1.0"
edition = "2024"

[dependencies]
nom = "8.0.0"
nom_locate = "5.0.0"

use crate::Content;
use crate::second_parser;

use nom::bytes::complete::tag;
use nom::character::complete::line_ending;
use nom::{IResult, Parser};

pub fn first_parser(
  mut content: Content
) -> IResult<Content, Content> {
  content.extra.push("first_parser");
  let (content, _) = tag("alfa").parse(content)?;
  let (content, _) = line_ending.parse(content)?;
  let (content, _) = tag("bravo").parse(content)?;
  let (content, result) =
    second_parser.parse(content)?;
  Ok((content, result))
}

and

use crate::Content;

use nom::bytes::complete::tag;
use nom::{IResult, Parser};

pub fn second_parser(
  mut content: Content
) -> IResult<Content, Content> {
  content.extra.push("second_parser");
  let (content, result) =
    tag("fail here").parse(content)?;
  Ok((content, result))
}

These are the same basic type of parsers from the prior example. The big difference is that first_parser does a little more work to get us down to a second line before passing off to second_parser. That'll help demonstrate the error output of this report function:

use crate::Content;

use nom::error::Error;
use nom::{Err, Finish};
use nom_locate::LocatedSpan;
use std::fs;

type ResultHolder<'a> = Result<
  (Content<'a>, Content<'a>),
  Err<Error<LocatedSpan<&'a str, Vec<&'a str>>>>,
>;

pub fn report(result: ResultHolder) {
  match result.finish() {
    Ok(_) => println!("Parsing successful"),
    Err(e) => {
      let error_message = format!(
        "PARSING ERROR:\n-> {}\nfailed at:\n{}\non line {} column {}:",
        e.input.extra.join("\n-> "),
        e.input.fragment(),
        e.input.location_line(),
        e.input.get_utf8_column(),
      );
      let error_line = String::from_utf8(
        e.input.get_line_beginning().to_vec(),
      )
      .unwrap();
      let pointer_line = format!(
        "{}^",
        " ".repeat(e.input.get_utf8_column() - 1),
      );
      let parts = [
        error_message.to_string(),
        error_line.to_string(),
        pointer_line.to_string(),
      ];
      fs::write("output.txt", parts.join("\n"))
        .unwrap();
    }
  }
}

Here's the overview of how it works:

• The type ResultHolder<'a> lines just keep us from having to put the entire signature in the function definition.

• The result.finish() call on line 14 combines a couple of different nom error types so they can be matched in a single call.

• Line 15 prints a success message of everything parsed properly.

• The Err branch that starts on line 16 does a bunch of formatting to print out a nice error message.

• The e.input field from the error contains the LocatedSpan struct that was passed in to the report.

  e.input.extra on line 19 pulls the vec with the values of the names of the parsers we added with content.extra.push() in the parsers.

  e.input.fragment() on line 20 gets the text that was attempting to be parsed where the failure occurred.

  e.input.location_line() on line 21 gets the line number from the original input where the error occured (which will be 2 in our case since first_parser got us to the second line).

  e.input.get_utf8_column() on line 22 give us the column of the .location_line() where the error starts. (There's a similar .get_column() that can be used if all you've got is ASCII, but I don't mess with that.)

• Lines 24-27 contain e.input.get_line_beginning() that pulls in the full line that the error occured on. It comes in as &[u8] so the .to_vec() and String::from_utf8() calls are used to turn it into text.

• The rest of the funciton is the formatting that shows the line where the error occurred and puts a ^ under the column where the error started.

We tie all that toghter in this main.rs file:

mod first_parser;
mod report;
mod second_parser;

use first_parser::first_parser;
use nom::Parser;
use nom_locate::LocatedSpan;
use report::report;
use second_parser::second_parser;

type Content<'a> = LocatedSpan<&'a str, Vec<&'a str>>;

fn main() {
  let input = Content::new_extra(
    "alfa\nbravo charlie delta",
    vec![],
  );
  let result = first_parser.parse(input);
  report(result);
}

And, when we run it, we get this:

PARSING ERROR:
-> first_parser
-> second_parser
failed at:
 charlie delta
on line 2 column 6:
bravo charlie delta
     ^

Orders of magnititude more useful than the original.

Outro

It took an hour or two to figure out the approach and dial things in. It took the rest of the day to write this post. At 8pm I've done zero work on the parser itself. I'm cool with that. I'll be able to move so much faster with these upgraded error messages. I'll make up the time in nothing flat.

-a