Better nom Error Messages with nom_locate

Parsing Parser Errors

I designed a note taking format called NeoDoc^nd. It currently exists only as an embedded feature of my personal static site generator. I'm in the process of extracting it to its own library. That means digging into the nom^nom parser/combinator and its cryptic as hell error messages.

"Just how cryptic?" you ask. Let me give you an example.

A First Look

This codes makes a tiny rust app that uses nom's alt feature to attempt to use two child parsers to process a piece of text.

[package]
name = "default_errors"
version = "0.1.0"
edition = "2024"

[dependencies]
nom = "8.0.0"

use nom::branch::alt;
use nom::bytes::complete::tag;
use nom::{IResult, Parser};
use std::fs;

fn main() {
  let input = "target";
  let result = alt((alfa, bravo)).parse(input);
  fs::write("output.txt", format!("{:#?}", result))
    .unwrap();
}

fn alfa(input: &str) -> IResult<&str, &str> {
  let (input, result) = tag("xxx").parse(input)?;
  Ok((input, result))
}

fn bravo(input: &str) -> IResult<&str, &str> {
  let (input, result) = tag("yyy").parse(input)?;
  Ok((input, result))
}

The parsers attempt to process the word "target" as input. They both fail since neither of their patterns (i.e. "xxx" or "yyy") match the word. The result is that nom throws this error:

Err(
    Error(
        Error {
            input: "target",
            code: Tag,
        },
    ),
)

• The input: "target" shows us what the parser attempted to process.

• The code: Tag lets us know that the error was generated because on of the attempts to use a .tag() failed.

  Critically, it doesn't tell us which one of them failed.

Figuring out the issue in this example is no big deal. It's quick to see that neither the alfa nor the bravo parsers match the "target" input passed to them. Things aren't so obvious in real world parsers. There are tons of different functions branching out and interacting with each other. It doesn't take long to cross a threshold where figuring out where an error occurred is a non-trivial task.

I used to use a crate called nom_supreme^ns to help with this. It provided errors with more details about where they came from. Unfortunately, it doesn't work with the most recent version of nom.

That's where the nom_locate crate enters the scene.

Locating Errors

nom_locate^nl works by adding a LocatedSpan struct to the mix. Your input goes inside and gets passed around as a passenger. For example:

[package]
name = "locate_basic"
version = "0.1.0"
edition = "2024"

[dependencies]
nom = "8.0.0"
nom_locate = "5.0.0"

use nom::bytes::complete::tag;
use nom::{IResult, Parser};
use nom_locate::LocatedSpan;
use std::fs;

type Span<'a> = LocatedSpan<&'a str, &'a str>;

fn main() {
  let input = "target";
  let span = Span::new_extra(input, "");
  let result = some_parser(span);
  fs::write("output.txt", format!("{:#?}", result))
    .unwrap();
}

fn some_parser(mut span: Span) -> IResult<Span, Span> {
  span.extra = "some_parser";
  let (span, result) = tag("xxx").parse(span)?;
  Ok((span, result))
}

Err(
    Error(
        Error {
            input: LocatedSpan {
                offset: 0,
                line: 1,
                fragment: "target",
                extra: "some_parser",
            },
            code: Tag,
        },
    ),
)

The input now contains the LocatedSpan with more details about where the error occurred. The value of the extra field is what we defined on line 17. Using it as a reference, we know in which function the failure occurred.

That's a huge improvement. And, it gets even better.

Showing Off Errors

We'll use a little more complicated example to demonstrate what nom_locate can do. It starts by defining two parser functions:

[package]
name = "locate_report"
version = "0.1.0"
edition = "2024"

[dependencies]
nom = "8.0.0"
nom_locate = "5.0.0"

use crate::Span;
use crate::second_parser;

use nom::bytes::complete::tag;
use nom::character::complete::line_ending;
use nom::{IResult, Parser};

pub fn first_parser(
  mut span: Span
) -> IResult<Span, Span> {
  span.extra = "prelude_parser";
  let (span, _) = tag("prelude").parse(span)?;
  let (span, _) = line_ending.parse(span)?;
  let (span, _) = tag("spacer").parse(span)?;
  let (span, result) = second_parser.parse(span)?;
  Ok((span, result))
}

and

use crate::Span;

use nom::bytes::complete::tag;
use nom::{IResult, Parser};

pub fn second_parser(
  mut span: Span
) -> IResult<Span, Span> {
  span.extra = "second_parser";
  let (span, result) = tag("xxx").parse(span)?;
  Ok((span, result))
}

These are the same basic type of parsers from the prior examples. The big difference is that first_parser does a little more work to get us down to a second line before passing off to second_parser. That'll help demonstrate the error output of this report function:

use crate::Span;

use nom::error::Error;
use nom::{Err, Finish};
use nom_locate::LocatedSpan;
use std::cmp::max;
use std::fs;

type ResultHolder<'a> = Result<
  (Span<'a>, Span<'a>),
  Err<Error<LocatedSpan<&'a str, &'a str>>>,
>;

pub fn report(result: ResultHolder) {
  match result.finish() {
    Ok(_) => println!("Parsing successful"),
    Err(e) => {
      let error_message = format!(
        "ERROR: {} failed on line {} column {}",
        e.input.extra,
        e.input.location_line(),
        e.input.get_utf8_column(),
      );
      let error_line = String::from_utf8(
        e.input.get_line_beginning().to_vec(),
      )
      .unwrap();
      let divider_spaces = max(
        error_message.chars().collect::<Vec<_>>().len(),
        error_line.chars().collect::<Vec<_>>().len(),
      );
      let pointer_line = format!(
        "{}^",
        " ".repeat(e.input.get_utf8_column() - 1),
      );
      let parts = [
        error_message.to_string(),
        "-".repeat(divider_spaces).to_string(),
        error_line.to_string(),
        pointer_line.to_string(),
      ];
      fs::write("output.txt", parts.join("\n"))
        .unwrap();
    }
  }
}

Here's the overview of how it works:

• The type ResultHolder<'a> lines just keep us from having to put the entire signature in the function definition.

• The result.finish() call on line 14 combines a couple of different nom error types so they can be matched in a single call.

• Line 15 prints a success message of everything parsed properly.

• The Err branch that starts on line 16 does a bunch of formatting to print out a nice error message.

• The e.input field from the error contains the LocatedSpan struct that was passed in to the report.

  e.input.extra on line 19 pulls the value with the name of the parser we added in those functions.

  e.input.location_line() on line 20 gets the line number from the original input where the error occured (which will be 2 in our case since first_parser got us to the second line).

  e.input.get_utf8_column() on line 21 give us the column of the .location_line() where the error starts. (There's a similar .get_column() that can be used if all you've got is ASCII, but I don't mess with that.)

• Lines 23-25 contain e.input.get_line_beginning() that pulls in the full line that the error occured on. It comes in as &[u8] so the .to_vec() and String::from_utf8() calls are used to turn it into text.

• The rest of the funciton is the formatting that shows the line where the error occurred and puts a ^ under the column where the error started.

We tie all that toghter in this main.rs file:

mod first_parser;
mod report;
mod second_parser;

use first_parser::first_parser;
use nom::Parser;
use nom_locate::LocatedSpan;
use report::report;
use second_parser::second_parser;

type Span<'a> = LocatedSpan<&'a str, &'a str>;

fn main() {
  let input =
    Span::new_extra("prelude\nspacer target", "");
  let result = first_parser.parse(input);
  report(result);
}

And, when we run it, we get this:

ERROR: second_parser failed on line 2 column 7
----------------------------------------------
spacer target
      ^

Orders of magnititude more useful than the original.

Outro

It took an hour or two to figure out the approach and dial things in. It took the rest of the day to write this post. At 8pm I've done zero work on the parser itself. I'm cool with that. I'll be able to move so much faster with these upgraded error messages. I'll make up the time in nothing flat.

-a