Alexander Bezzubov 78d8f43a88 tokenizer: hide flex-based impl, avoid build failures on win ...

TestPlan:
 - go test -run TestTokenize ./internal/tokenizer
 - go test -tags flex -run TestTokenize ./internal/tokenizer
   (shold fail as default fixtures are from regex-based tokenizer)

2020-03-19 19:58:48 +01:00

..

flex

tokenizer: hide flex-based impl, avoid build failures on win

2020-03-19 19:58:48 +01:00

common.go

address review feedback

2019-04-16 19:38:48 +02:00

tokenize_c.go

*: module rename to go-enry/go-enry/v4

2020-03-19 17:31:29 +01:00

tokenize_test.go

tokenizer: hide flex-based impl, avoid build failures on win

2020-03-19 19:58:48 +01:00

tokenize.go

tokenizer: fix a bug and regenerate the code \w latest Go

2020-03-19 19:08:21 +01:00