ralsina/tartrazine
1
0
Fork 0
mirror of https://github.com/ralsina/tartrazine.git synced 2025-06-19 14:43:05 -03:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

*: module rename to go-enry/go-enry/v4

Browse Source
This commit is contained in:
Máximo Cuadros
2020-03-19 17:31:29 +01:00
parent bc5e031cee
commit 84efad7693
50 changed files with 60 additions and 172 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
internal/tokenizer/tokenize_c.go
View File

@ -2,7 +2,7 @@
package tokenizer
import "github.com/bzz/enry/v2/internal/tokenizer/flex"
import "github.com/go-enry/go-enry/v2/internal/tokenizer/flex"
// Tokenize returns lexical tokens from content. The tokens returned match what
// the Linguist library returns. At most the first ByteLimit bytes of content are tokenized.