Documenter fails with NPE and unhelpful message if Javadoc contains quotation marks

[dschulten]

I had a method with example code in the javadoc, where I forgot to escape ".

  /**
   * Findet Gemeinde, Gemarkung und Flur anhand ihrer Fachschluessel.
   *
   * <p>Example:
   *
   * <pre>
   * Tuple3<Gemeinde, Gemarkung, Flur> gemeindeGemarkungUndFlur =
   *     gemeindeRepository.findGemeindeGemarkungFlurByFachschluessel("08117026", "2050", "000") // <<=== double quotes
   *     
   * </pre>
   * /

IDEs no longer complain about that, but the Json Parser in Documenter was unhappy without giving very useful information:

java.lang.ExceptionInInitializerError
	at org.springframework.modulith.docs.Asciidoctor.getSpringModulithDocsSource(Asciidoctor.java:427)
	at org.springframework.modulith.docs.Asciidoctor.<clinit>(Asciidoctor.java:59)
	at org.springframework.modulith.docs.Documenter.toModuleCanvas(Documenter.java:432)
	at org.springframework.modulith.docs.Documenter.lambda$writeModuleCanvases$2(Documenter.java:392)
	at java.base/java.lang.Iterable.forEach(Iterable.java:75)
	at org.springframework.modulith.docs.Documenter.writeModuleCanvases(Documenter.java:388)
	at org.springframework.modulith.docs.Documenter.writeDocumentation(Documenter.java:201)
	at org.springframework.modulith.docs.Documenter.writeDocumentation(Documenter.java:180)
	at de.kommone.kps.vertrag.ApplicationModularityTests.verifyStructure(ApplicationModularityTests.java:13)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
Caused by: org.springframework.boot.json.JsonParseException: Cannot parse JSON
	at org.springframework.boot.json.AbstractJsonParser.tryParse(AbstractJsonParser.java:57)
	at org.springframework.boot.json.BasicJsonParser.parseList(BasicJsonParser.java:51)
	at org.springframework.modulith.docs.SpringModulithDocumentationSource.from(SpringModulithDocumentationSource.java:123)
	at org.springframework.modulith.docs.SpringModulithDocumentationSource.<init>(SpringModulithDocumentationSource.java:62)
	at java.base/java.util.Optional.map(Optional.java:260)
	at org.springframework.modulith.docs.SpringModulithDocumentationSource.<clinit>(SpringModulithDocumentationSource.java:49)
	... 12 more
Caused by: java.lang.NullPointerException: Cannot load from object array because "values" is null
	at org.springframework.boot.json.BasicJsonParser.parseMapInternal(BasicJsonParser.java:84)
	at org.springframework.boot.json.BasicJsonParser.parseInternal(BasicJsonParser.java:71)
	at org.springframework.boot.json.BasicJsonParser.parseListInternal(BasicJsonParser.java:58)
	at org.springframework.boot.json.BasicJsonParser.parseInternal(BasicJsonParser.java:68)
	at org.springframework.boot.json.BasicJsonParser.parseMapInternal(BasicJsonParser.java:87)
	at org.springframework.boot.json.BasicJsonParser.parseInternal(BasicJsonParser.java:71)
	at org.springframework.boot.json.BasicJsonParser.parseListInternal(BasicJsonParser.java:58)
	at org.springframework.boot.json.BasicJsonParser.lambda$parseList$2(BasicJsonParser.java:51)
	at org.springframework.boot.json.AbstractJsonParser.trimParse(AbstractJsonParser.java:46)
	at org.springframework.boot.json.AbstractJsonParser.parseList(AbstractJsonParser.java:40)
	at org.springframework.boot.json.BasicJsonParser.lambda$parseList$3(BasicJsonParser.java:51)
	at org.springframework.boot.json.AbstractJsonParser.tryParse(AbstractJsonParser.java:53)
	... 17 more

It happens in BasicJsonParser, and the stacktrace does make it easy to find the place:

	private Map<String, Object> parseMapInternal(int nesting, String json) {
		Map<String, Object> map = new LinkedHashMap<>();
		json = trimEdges(json, '{', '}').trim();
		for (String pair : tokenize(json)) {
			String[] values = StringUtils.trimArrayElements(StringUtils.split(pair, ":"));
			Assert.state(values[0].startsWith("\"") && values[0].endsWith("\""), // <<===== values is null here
					"Expecting double-quotes around field names");
			String key = trimEdges(values[0], '"', '"');
			Object value = parseInternal(nesting, values[1]);
			map.put(key, value);
		}
		return map;
	}

The json in tokenize(json) was invalid, because the json String contained double quotes around the strings "08117026" etc. so that the comment string contains offending nested double quotes:

{
  "name": "findGemeindeGemarkungFlurByFachschluessel",
  "signature": "findGemeindeGemarkungFlurByFachschluessel(java.lang.String, java.lang.String, java.lang.String)",
  "comment": "Findet Gemeinde, Gemarkung und Flur anhand ihrer Fachschluessel.  <p>Example:  <pre> Tuple3<Gemeinde, Gemarkung, Flur> gemeindeGemarkungUndFlur = gemeindeRepository.findGemeindeGemarkungFlurByFachschluessel("08117026", "2050", "000")</pre>"
}

The fix was to replace the double quotes in the example code of the javadoc by ".

Would it be possible to check if values is null and make sure the offending json becomes part of the error message? or maybe even ensure that Javadoc is escaped before it goes into the Documenter json?

[odrotbohm]

Do you have an example of the JSON that's to be read here? BasicJsonParser is a Boot component. If we wanted to fix this on the reading side, we'd have to fix it in Boot. That said, I wonder if we should replace the translation between Javadoc and JSON instead to make sure that BJP doesn't stumble over the problem.

[dschulten]

yes, the offending json is in the original report: note the String parameters such as "000" at the end of the comment attribute's value.