When decoding a HIP resource record, 'base64.StdEncoding.DecodedLen' can return a length larger than the length of the decoded public key. This change decodes the public key and retrieves the correct length. In our tests, the public key length was being set to 33, instead of 32. Below is our offending resource record:
'23b5993f649c0827.a.b.c. 3600 IN HIP 5 200100100020001523B5993F649C0827 Cm6k4jhir9YYoKq9JDqD3Ob1hBfCuwbWam1igFPhkGg='
* Set Rdlength in fromRFC3597
This was a bug found by oss-fuzz. My bad (#1211).
* Limit maximum length of Rdata in (*RFC3597).parse
RDATA must be a 16-bit unsigned integer.
* Validate Rdlength and off in UnpackRRWithHeader
* Revert "Validate Rdlength and off in UnpackRRWithHeader"
This reverts commit 2f6a8811b944b100af7605e53a6fb164944a6d65.
* Use hex.DecodedLen in (*RFC3597).fromRFC3597
While this isn't done elsewhere, it is clearer and more obvious.
* Fixed the default values of HorizPre and VertPre
According to RFC-1876 those fields should be:
"a pair of four-bit unsigned
integers, each ranging from zero to nine, with the most
significant four bits representing the base and the second
number representing the power of ten by which to multiply
the base. This allows sizes from 0e0 (<1cm) to 9e9
(90,000km) to be expressed"
Current values for HorizPre and VertPre (165=0xA5 and 162=0xA2)
are incorrect because the first HEX digit is greater then 9
The default values should be:
HorizPre = 10000m = 10000 * 100 cm = 10^6 = 0x16
VertPre = 10m = 10 * 100 cm = 10^3 = 0x13
Size = 1m = 1 * 100 cm = 10^2 = 0x12
The value of Size was correct, but this PR changes it to HEX
representation to be more readable
* Informative comments
Made comments on LOC record default field values more informative
Co-Authored-By: Richard Gibson <richard.gibson@gmail.com>
Co-authored-by: Richard Gibson <richard.gibson@gmail.com>
* APL record: add structure and code point
* APL record: add wire format support
* APL record: add presentation format support
* APL record: add isDuplicate implementation
* APL record: add copy implementation
* APL record: add len implementation
* APL record: run go generate
* APL record: fix condition checking for equality
* APL record: use switches to map family to address length
* APL record: check bounds of individual fields rather than whole header
* APL record: stylistic changes
* APL record: remove APLPrefix methods from public interface
* APL record: update README
* APL record: additional cleanup for code review
* APL record: change return type from pointer to struct
* APL record: refactor of pack and unpack to eliminate extra variables
* Merge setRR into ZoneParser.Next
* Remove file argument from RR.parse
This was only used to fill in the ParseError file field. Instead we now
fill in that field in ZoneParser.Next.
* Move dynamic update check out of RR.parse
This consolidates all the dynamic update checks into one place.
* Check for unexpected newline before parsing RR data
* Move rr.parse call into if-statement
* Allow dynamic updates for TKEY and RFC3597 records
* Document that ParseError file field is unset from parse
* Inline allowDynamicUpdate into ZoneParser.Next
* Improve and simplify TestUnexpectedNewline
* ZoneParser: error on parsing an IPv6 address in an A record
And vice versa for IPv4 with AAAA.
The implementation of isIPv6 is inspired by e341bae08d/src/net/ip.go (L678-L681) .
* Fix benchmarks that try to use ::1 as A record.
* Test A/AAAA parsing via NewRR rather than zone parser.
* Document why we distinguish IPv4 vs IPv6 via existence of ":".
* Eliminate Variable bool from parserFunc
Instead we now check whether the last token read from the zlexer was
a zNewline or zEOF. The error check above should be tripped for any
record that ends prematurely.
* Use an interface method for parsing zone file records
* Prevent panic in TestOmittedTTL if no regexp match
* Move slurpRemainder into fixed length parse functions
This is consistent with the original logic in setRR and avoids potential
edge cases.
* Parse synthetic records according to RFC 3597
These records lack a presentation format and cannot be parsed otherwise.
This behaviour is consistent with how this previously operated.
* Add comment getter to zlexer
* Use zlexer.Comment instead of lex.comment
* Move comment handling out of setRR code
* Move comment field from lex to zlexer
* Eliminate ZoneParser.com field
* Return empty string from zlexer.Comment on error
* Only reset zlexer.comment field once per Next
* Remove zlexer merge TODO
I'm pretty sure these have to remain separate which is okay.
* Eliminate zlexer goroutine
This replaces the zlexer goroutine and channels with a zlexer struct
that maintains state and provides a channel-like API.
* Eliminate klexer goroutine
This replaces the klexer goroutine and channels with a klexer struct
that maintains state and provides a channel-like API.
* Merge scan into zlexer and klexer
This does result in tokenText existing twice, but it's pretty simple
and small so it's not that bad.
* Avoid using text/scanner.Position to track position
* Track escape within zlexer.Next
* Avoid zl.commt check on space and tab in zlexer
* Track stri within zlexer.Next
* Track comi within zlexer.Next
There is one special case at the start of a comment that needs to be
handled, otherwise this is as simple as stri was.
* Use a single token buffer in zlexer
This is safe as there is never both a non-empty string buffer and a
non-empty comment buffer.
* Don't hardcode length of zl.tok in zlexer
* Eliminate lex.length field
This is always set to len(l.token) and is only queried in a few places.
It was added in 47cc5b052d without any
obvious need.
* Add whitespace to klexer.Next
* Track lex within klexer.Next
* Use a strings.Builder in klexer.Next
* Simplify : case in klexer.Next
* Add whitespace to zlexer.Next
* Change for loop style in zlexer.Next and klexer.Next
* Surface read errors in zlexer
* Surface read errors from klexer
* Remove debug line from parseKey
* Rename tokenText to readByte
* Make readByte return ok bool
Also change the for loop style to match the Next for loops.
* Make readByte errors sticky
klexer.Next calls readByte separately from within the loop. Without
readByte being sticky, an error that occurs during that readByte call
may be lost.
* Panic in testRR if the error is non-nil
* Add whitespace and unify field setting in zlexer.Next
* Remove eof fields from zlexer and klexer
With readByte having sticky errors, this no longer needed. zl.eof = true
was also in the wrong place and could mask an unbalanced brace error.
* Merge zl.tok blocks in zlexer.Next
* Split the tok buffer into separate string and comment buffers
The invariant of stri > 0 && comi > 0 never being true was broken when
x == '\n' && !zl.quote && zl.commt && zl.brace != 0 (the
"If not in a brace this ends the comment AND the RR" block).
Split the buffer back out into two separate buffers to avoid clobbering.
* Replace token slices with arrays in zlexer
* Add a NewRR benchmark
* Move token buffers into zlexer.Next
These don't need to be retained across Next calls and can be stack
allocated inside Next. This drastically reduces memory consumption as
they accounted for nearly half of all the memory used.
name old alloc/op new alloc/op delta
NewRR-12 9.72kB ± 0% 4.98kB ± 0% -48.72% (p=0.000 n=10+10)
* Add a ReadRR benchmark
Unlike NewRR, this will use an io.Reader that does not implement any
methods aside from Read. In particular it does not implement
io.ByteReader.
* Avoid using a bufio.Reader for io.ByteReader readers
At the same time use a smaller buffer size of 1KiB rather than the
bufio.NewReader default of 4KiB.
name old time/op new time/op delta
NewRR-12 11.0µs ± 3% 9.5µs ± 2% -13.77% (p=0.000 n=9+10)
ReadRR-12 11.2µs ±16% 9.8µs ± 1% -13.03% (p=0.000 n=10+10)
name old alloc/op new alloc/op delta
NewRR-12 4.98kB ± 0% 0.81kB ± 0% -83.79% (p=0.000 n=10+10)
ReadRR-12 4.87kB ± 0% 1.82kB ± 0% -62.73% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
NewRR-12 19.0 ± 0% 17.0 ± 0% -10.53% (p=0.000 n=10+10)
ReadRR-12 19.0 ± 0% 19.0 ± 0% ~ (all equal)
ReadRR-12 11.2µs ±16% 9.8µs ± 1% -13.03% (p=0.000 n=10+10)
* Surface any remaining comment from zlexer.Next
* Improve comment handling in zlexer.Next
This both fixes a regression where comments could be lost under certain
circumstances and now emits comments that occur within braces.
* Remove outdated comment from zlexer.Next and klexer.Next
* Delay converting LF to space in braced comment
* Fixup TestParseZoneComments
* Remove tokenUpper field from lex
Not computing this for every token, and instead only
when needed is a substantial performance improvement.
name old time/op new time/op delta
NewRR-12 9.56µs ± 0% 6.30µs ± 1% -34.08% (p=0.000 n=9+10)
ReadRR-12 9.93µs ± 1% 6.67µs ± 1% -32.77% (p=0.000 n=10+10)
name old alloc/op new alloc/op delta
NewRR-12 824B ± 0% 808B ± 0% -1.94% (p=0.000 n=10+10)
ReadRR-12 1.83kB ± 0% 1.82kB ± 0% -0.87% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
NewRR-12 17.0 ± 0% 17.0 ± 0% ~ (all equal)
ReadRR-12 19.0 ± 0% 19.0 ± 0% ~ (all equal)
* Update ParseZone documentation to match comment changes
The zlexer code was changed to return comments more often, so update the
ParseZone documentation to match.
* Remove redundant parenthesis
These were caught with:
gofmt -r '(a) -> a' -w *.go
This commit only includes the changes where the formatting makes the
ordering of operations clear.
* Remove more redundant parenthesis
These were caught with:
gofmt -r '(a) -> a' -w *.go
This commit includes the remaining changes where the formatting does not
make the ordering of operations as clear as the previous commit.
* Add support for TKEY RRs
- make sure Key and Data fields are variable length hex fields
- checkin output from 'go generate'
- add a TKEY specific test to ensure this stays working
* go format changes
* address review comments
* add ability to parse TKEY via string
* handle review comments - change TKEY string output
Implement the CSYNC record.
Fixes#290
Long overdue, lets add this record. Similar in vain as NSEC/NSEC3, we
need to implement len() our selves. Presentation format parsing and
tests are done as well.
This is CoreDNS running with CSYNC support, `dig` doesn't support this
at the moment, so:
~~~
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 40323
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;csync.example.org. IN TYPE62
;; ANSWER SECTION:
csync.example.org. 10 IN TYPE62 \# 12 000335240042000460000008
;; AUTHORITY SECTION:
example.org. 10 IN NS a.iana-servers.net.
example.org. 10 IN NS b.iana-servers.net.
~~~
* Fix $TTL handling
* Error when there is no TTL for an RR
* Fix relative name handling
* Error when a relative name is used without an origin (cf. https://tools.ietf.org/html/rfc1035#section-5.1 )
Fixes#484
* Test for proper parsing of whitespace-separated (TXT) character-strings
* Properly parse whitespace-separated (TXT) character-strings
* Remove non-RFC treatment of backslash sequences in character-strings
Fixes gh-420
* For tests, remove non-RFC treatment of backslashes in domain names
1) Refactoring of tlsa.go
- moved routine to create the certificate rdata to its own go module
as this is shared between TLSA and SMIMEA records
2) Added support for creating an SMIMEA domain name
3) Developed in accordance with draft-ietf-dane-smime-12 RFC
Miek,
Submitting for your review. Happy to make any recommended changes or
address omissions.
Lightly tested against our internal DNS service which hosts DANE
SMIMEA records for our email certificates.
Parse tests are added.
Update the size-xxx-member tags to point to another field in the struct
that should be used for the length in that field. Fix NSEC3/HIP and TSIG
to use to this and generate the correct pack/unpack functions for them.
Remove IPSECKEY from the lib and handle it as an unknown record - it is
such a horrible RR, needed kludges before - now just handle it as an
unknown RR.
All types now use generated pack and unpack functions. The blacklist is
removed.
Remove the use of reflection when packing and unpacking, instead
generate all the pack and unpack functions using msg_generate.
This will generate zmsg.go which in turn calls the helper functions from
msg_helper.go.
This increases the speed by about ~30% while cutting back on memory
usage. Not all RRs are using it, but that will be rectified in upcoming
PR.
Most of the speed increase is in the header/question section parsing.
These functions *are* not generated, but straight forward enough. The
implementation can be found in msg.go.
The new code has been fuzzed by go-fuzz, which turned up some issues.
All files that started with 'z', and not autogenerated were renamed,
i.e. zscan.go is now scan.go.
Reflection is still used, in subsequent PRs it will be removed entirely.
Miek Gieben
joseph-stanton-ax
madestro
Shubhendra Singh Chauhan
Tom Thorogood
JINMEI Tatuya
Disconnect3d
Catena cyber
Manabu Sonoda
Pavel Rybintsev
Jan Včelák
Tom F
tr3e
spsholleman
Richard Gibson
Jon Nappi