Commit Graph

11 Commits

Author SHA1 Message Date
Tom Thorogood e6df8867af
Fix Rdlength related parsing bug in RFC3597 records (#1214)
* Set Rdlength in fromRFC3597

This was a bug found by oss-fuzz. My bad (#1211).

* Limit maximum length of Rdata in (*RFC3597).parse

RDATA must be a 16-bit unsigned integer.

* Validate Rdlength and off in UnpackRRWithHeader

* Revert "Validate Rdlength and off in UnpackRRWithHeader"

This reverts commit 2f6a8811b944b100af7605e53a6fb164944a6d65.

* Use hex.DecodedLen in (*RFC3597).fromRFC3597

While this isn't done elsewhere, it is clearer and more obvious.
2021-02-01 09:10:38 +01:00
Tom Thorogood 13238cb6ad
Support parsing known RR types in RFC 3597 format (#1211)
* Support parsing known RR types in RFC 3597 format

This is the format used for "Unknown DNS Resource Records", but it's
also useful to support parsing known RR types in this way.

RFC 3597 says:

   An implementation MAY also choose to represent some RRs of known type
   using the above generic representations for the type, class and/or
   RDATA, which carries the benefit of making the resulting master file
   portable to servers where these types are unknown.  Using the generic
   representation for the RDATA of an RR of known type can also be
   useful in the case of an RR type where the text format varies
   depending on a version, protocol, or similar field (or several)
   embedded in the RDATA when such a field has a value for which no text
   format is known, e.g., a LOC RR [RFC1876] with a VERSION other than
   0.

   Even though an RR of known type represented in the \# format is
   effectively treated as an unknown type for the purpose of parsing the
   RDATA text representation, all further processing by the server MUST
   treat it as a known type and take into account any applicable type-
   specific rules regarding compression, canonicalization, etc.

* Correct mistakes in TestZoneParserAddressAAAA

This was spotted when writing TestParseKnownRRAsRFC3597.

* Eliminate canParseAsRR

This has the advantage that concrete types will now be returned for
parsed ANY, NULL, OPT and TSIG records.

* Expand TestDynamicUpdateParsing for RFC 3597

This ensures we're properly handling empty RDATA for RFC 3597 parsed
records.
2021-01-30 14:05:25 +01:00
taciomcosta d128d10d17
refactor: remove ParseZone and parseZone (#1099) 2020-04-28 09:24:18 +02:00
Tom Thorogood 25cacca8ca Prohibit newlines before record data in the ZoneParser (#979)
* Merge setRR into ZoneParser.Next

* Remove file argument from RR.parse

This was only used to fill in the ParseError file field. Instead we now
fill in that field in ZoneParser.Next.

* Move dynamic update check out of RR.parse

This consolidates all the dynamic update checks into one place.

* Check for unexpected newline before parsing RR data

* Move rr.parse call into if-statement

* Allow dynamic updates for TKEY and RFC3597 records

* Document that ParseError file field is unset from parse

* Inline allowDynamicUpdate into ZoneParser.Next

* Improve and simplify TestUnexpectedNewline
2019-06-10 07:38:54 +01:00
Tom F 487e4636d5 ZoneParser: error on parsing an IPv6 address in an A record (#923)
* ZoneParser: error on parsing an IPv6 address in an A record

And vice versa for IPv4 with AAAA.

The implementation of isIPv6 is inspired by e341bae08d/src/net/ip.go (L678-L681) .

* Fix benchmarks that try to use ::1 as A record.

* Test A/AAAA parsing via NewRR rather than zone parser.

* Document why we distinguish IPv4 vs IPv6 via existence of ":".
2019-03-09 09:02:18 +00:00
Tom Thorogood 274da7d3ef
Add new ZoneParser API (#794)
* Improve ParseZone tests

* Add new ZoneParser API

* Use the ZoneParser API directly in ReadRR

* Merge parseZoneHelper into ParseZone

* Make generate string building slightly more efficient

* Add SetDefaultTTL method to ZoneParser

This makes it possible for external consumers to implement ReadRR.

* Make $INCLUDE directive opt-in

The $INCLUDE directive opens a user controlled file and parses it as
a DNS zone file. The error messages may reveal portions of sensitive
files, such as:
	/etc/passwd: dns: not a TTL: "root0:0:root:/root:/bin/bash" at line: 1:31
	/etc/shadow: dns: not a TTL: "root:$6$<redacted>::0:99999:7:::" at line: 1:125

Both ParseZone and ReadRR are currently opt-in for backward
compatibility.

* Disable $INCLUDE support in ReadRR

ReadRR and NewRR are often passed untrusted input. At the same time,
$INCLUDE isn't really useful for ReadRR as it only ever returns the
first record.

This is a breaking change, but it currently represents a slight
security risk.

* Document the need to drain the ParseZone chan

* Cleanup the documentation of NewRR, ReadRR and ParseZone

* Document the ZoneParser API

* Deprecated the ParseZone function

* Add whitespace to ZoneParser.Next

* Remove prevName field from ZoneParser

This doesn't track anything meaningful as both zp.prevName and h.Name
are only ever set at the same point and to the same value.

* Use uint8 for ZoneParser.include field

It has a maximum value of 7 which easily fits within uint8.

This reduces the size of ZoneParser from 160 bytes to 152 bytes.

* Add setParseError helper to ZoneParser

* Surface $INCLUDE os.Open error in error message

* Rename ZoneParser.include field to includeDepth

* Make maximum $INCLUDE depth a const

* Add ParseZone and ZoneParser benchmarks

* Parse $GENERATE directive with a single ZoneParser

This should be more efficient than calling NewRR for each generated
record.

* Run go fmt on generate_test.go

* Add a benchmark for $GENERATE directives

* Use a custom reader for generate

This avoids the overhead and memory usage of building the zone string.

name         old time/op    new time/op    delta
Generate-12     165µs ± 4%     157µs ± 2%   -5.06%  (p=0.000 n=25+25)

name         old alloc/op   new alloc/op   delta
Generate-12    42.1kB ± 0%    31.8kB ± 0%  -24.42%  (p=0.000 n=20+23)

name         old allocs/op  new allocs/op  delta
Generate-12     1.56k ± 0%     1.55k ± 0%   -0.38%  (p=0.000 n=25+25)

* Return correct ParseError from generateReader

The last commit made these regular errors while they had been
ParseErrors before.

* Return error message as string from modToPrintf

This is slightly simpler and they don't need to be errors.

* Skip setting includeDepth in generate

This sub parser isn't allowed to use $INCLUDE directives anyway.

Note: If generate is ever changed to allow $INCLUDE directives, then
      this line must be added back. Without doing that, it would be
      be possible to exceed maxIncludeDepth.

* Make generateReader errors sticky

ReadByte should not be called after an error has been returned, but
this is cheap insurance.

* Move file and lex fields to end of generateReader

These are only used for creating a ParseError and so are unlikely to be
accessed.

* Don't return offset with error in modToPrintf

Along for the ride, are some whitespace and style changes.

* Add whitespace to generate and simplify step

* Use a for loop instead of goto in generate

* Support $INCLUDE directives inside $GENERATE directives

This was previously supported and may be useful. This is now more
rigorous as the maximum include depth is respected and relative
$INCLUDE directives are now supported from within $GENERATE.

* Don't return any lexer tokens after read error

Without this, read errors are likely to be lost and become parse errors
of the remaining str. The $GENERATE code relies on surfacing errors from
the reader.

* Support $INCLUDE in NewRR and ReadRR

Removing $INCLUDE support from these is a breaking change and should
not be included in this pull request.

* Add test to ensure $GENERATE respects $INCLUDE support

* Unify TestZoneParserIncludeDisallowed with other tests

* Remove stray whitespace from TestGenerateSurfacesErrors

* Move ZoneParser SetX methods above Err method

* $GENERATE should not accept step of 0

If step is allowed to be 0, then generateReader (and the code it
replaced) will get stuck in an infinite loop.

This is a potential DOS vulnerability.

* Fix ReadRR comment for file argument

I missed this previosuly. The file argument is also used to
resolve relative $INCLUDE directives.

* Prevent test panics on nil error

* Rework ZoneParser.subNext

This is slightly cleaner and will close the underlying *os.File even
if an error occurs.

* Make ZoneParser.generate call subNext

This also moves the calls to setParseError into generate.

* Report errors when parsing rest of $GENERATE directive

* Report proper error location in $GENERATE directive

This makes error messages much clearer.

* Simplify modToPrintf func

Note: When width is 0, the leading 0 of the fmt string is now excluded.
      This should not alter the formatting of numbers in anyway.

* Add comment explaining sub field

* Remove outdated error comment from generate
2018-10-20 11:47:56 +10:30
Tom Thorogood 17c1bc6792
Eliminate lexer goroutines (#792)
* Eliminate zlexer goroutine

This replaces the zlexer goroutine and channels with a zlexer struct
that maintains state and provides a channel-like API.

* Eliminate klexer goroutine

This replaces the klexer goroutine and channels with a klexer struct
that maintains state and provides a channel-like API.

* Merge scan into zlexer and klexer

This does result in tokenText existing twice, but it's pretty simple
and small so it's not that bad.

* Avoid using text/scanner.Position to track position

* Track escape within zlexer.Next

* Avoid zl.commt check on space and tab in zlexer

* Track stri within zlexer.Next

* Track comi within zlexer.Next

There is one special case at the start of a comment that needs to be
handled, otherwise this is as simple as stri was.

* Use a single token buffer in zlexer

This is safe as there is never both a non-empty string buffer and a
non-empty comment buffer.

* Don't hardcode length of zl.tok in zlexer

* Eliminate lex.length field

This is always set to len(l.token) and is only queried in a few places.

It was added in 47cc5b052d without any
obvious need.

* Add whitespace to klexer.Next

* Track lex within klexer.Next

* Use a strings.Builder in klexer.Next

* Simplify : case in klexer.Next

* Add whitespace to zlexer.Next

* Change for loop style in zlexer.Next and klexer.Next

* Surface read errors in zlexer

* Surface read errors from klexer

* Remove debug line from parseKey

* Rename tokenText to readByte

* Make readByte return ok bool

Also change the for loop style to match the Next for loops.

* Make readByte errors sticky

klexer.Next calls readByte separately from within the loop. Without
readByte being sticky, an error that occurs during that readByte call
may be lost.

* Panic in testRR if the error is non-nil

* Add whitespace and unify field setting in zlexer.Next

* Remove eof fields from zlexer and klexer

With readByte having sticky errors, this no longer needed. zl.eof = true
was also in the wrong place and could mask an unbalanced brace error.

* Merge zl.tok blocks in zlexer.Next

* Split the tok buffer into separate string and comment buffers

The invariant of stri > 0 && comi > 0 never being true was broken when
x == '\n' && !zl.quote && zl.commt && zl.brace != 0 (the
"If not in a brace this ends the comment AND the RR" block).

Split the buffer back out into two separate buffers to avoid clobbering.

* Replace token slices with arrays in zlexer

* Add a NewRR benchmark

* Move token buffers into zlexer.Next

These don't need to be retained across Next calls and can be stack
allocated inside Next. This drastically reduces memory consumption as
they accounted for nearly half of all the memory used.

name      old alloc/op   new alloc/op   delta
NewRR-12    9.72kB ± 0%    4.98kB ± 0%  -48.72%  (p=0.000 n=10+10)

* Add a ReadRR benchmark

Unlike NewRR, this will use an io.Reader that does not implement any
methods aside from Read. In particular it does not implement
io.ByteReader.

* Avoid using a bufio.Reader for io.ByteReader readers

At the same time use a smaller buffer size of 1KiB rather than the
bufio.NewReader default of 4KiB.

name       old time/op    new time/op    delta
NewRR-12     11.0µs ± 3%     9.5µs ± 2%  -13.77%  (p=0.000 n=9+10)
ReadRR-12    11.2µs ±16%     9.8µs ± 1%  -13.03%  (p=0.000 n=10+10)

name       old alloc/op   new alloc/op   delta
NewRR-12     4.98kB ± 0%    0.81kB ± 0%  -83.79%  (p=0.000 n=10+10)
ReadRR-12    4.87kB ± 0%    1.82kB ± 0%  -62.73%  (p=0.000 n=10+10)

name       old allocs/op  new allocs/op  delta
NewRR-12       19.0 ± 0%      17.0 ± 0%  -10.53%  (p=0.000 n=10+10)
ReadRR-12      19.0 ± 0%      19.0 ± 0%     ~     (all equal)

ReadRR-12    11.2µs ±16%     9.8µs ± 1%  -13.03%  (p=0.000 n=10+10)

* Surface any remaining comment from zlexer.Next

* Improve comment handling in zlexer.Next

This both fixes a regression where comments could be lost under certain
circumstances and now emits comments that occur within braces.

* Remove outdated comment from zlexer.Next and klexer.Next

* Delay converting LF to space in braced comment

* Fixup TestParseZoneComments

* Remove tokenUpper field from lex

Not computing this for every token, and instead only
when needed is a substantial performance improvement.

name       old time/op    new time/op    delta
NewRR-12     9.56µs ± 0%    6.30µs ± 1%  -34.08%  (p=0.000 n=9+10)
ReadRR-12    9.93µs ± 1%    6.67µs ± 1%  -32.77%  (p=0.000 n=10+10)

name       old alloc/op   new alloc/op   delta
NewRR-12       824B ± 0%      808B ± 0%   -1.94%  (p=0.000 n=10+10)
ReadRR-12    1.83kB ± 0%    1.82kB ± 0%   -0.87%  (p=0.000 n=10+10)

name       old allocs/op  new allocs/op  delta
NewRR-12       17.0 ± 0%      17.0 ± 0%     ~     (all equal)
ReadRR-12      19.0 ± 0%      19.0 ± 0%     ~     (all equal)

* Update ParseZone documentation to match comment changes

The zlexer code was changed to return comments more often, so update the
ParseZone documentation to match.
2018-10-15 17:42:31 +10:30
tr3e 501e858f67 Fix issue #742 (#745)
* Fix error comparison in SetTA

* Add testcase TestParseTA()
2018-09-22 18:36:01 +01:00
Joel Sing ed07089f3b Correctly handle $GENERATE modifiers (#703)
* Add a ParseZone test for $GENERATE.

* Add a test for modToPrintf used by $GENERATE.

* Correctly handle $GENERATE modifiers.

As per http://www.zytrax.com/books/dns/ch8/generate.html, the width and type (aka base)
components of a modifier are optional. This means that ${2,0,d}, ${2,0} and ${2} are
valid modifiers, however only the first format was previously permitted. Use default
values for the width and/or type if they are unspecified in the modifier.
2018-06-23 09:12:44 +01:00
Miek Gieben e420576857 scan: Fix $INCLUDE arguments to parseZone (#508)
When an $INCLUDE was seen the arguments to parseZone where in the wrong
order meaning the filename was used as the `neworigin` instead of the
actual origin we need.

Extend the testcase to check for the full name of the record.
2017-08-18 14:14:42 +01:00
Tim Esselens bbca4873b3 variable shadowing of token (#503)
* Added test for $INCLUDE statement parser in zone files

* FIX: localized l to switch statement, shadowed later call to os.Open(l.token)
2017-08-08 15:19:10 -07:00