mirror of
https://github.com/ralsina/tartrazine.git
synced 2024-11-10 13:32:24 +00:00
78d8f43a88
TestPlan: - go test -run TestTokenize ./internal/tokenizer - go test -tags flex -run TestTokenize ./internal/tokenizer (shold fail as default fixtures are from regex-based tokenizer) |
||
---|---|---|
.. | ||
flex | ||
common.go | ||
tokenize_c.go | ||
tokenize_test.go | ||
tokenize.go |