mirror of
https://github.com/ralsina/tartrazine.git
synced 2025-07-04 15:29:25 +00:00
Compare commits
4 Commits
Author | SHA1 | Date | |
---|---|---|---|
4dd2e925b0 | |||
7bda19cdea | |||
0e7dafe711 | |||
082241eb0f |
59
README.md
59
README.md
@ -2,36 +2,11 @@
|
|||||||
|
|
||||||
Tartrazine is a library to syntax-highlight code. It is
|
Tartrazine is a library to syntax-highlight code. It is
|
||||||
a port of [Pygments](https://pygments.org/) to
|
a port of [Pygments](https://pygments.org/) to
|
||||||
[Crystal](https://crystal-lang.org/). Kind of.
|
[Crystal](https://crystal-lang.org/).
|
||||||
|
|
||||||
The CLI tool can be used to highlight many things in many styles.
|
It also provides a CLI tool which can be used to highlight many things in many styles.
|
||||||
|
|
||||||
# A port of what? Why "kind of"?
|
Currently Tartrazine supports 247 languages. and it has 331 themes (63 from Chroma, the rest are base16 themes via
|
||||||
|
|
||||||
Pygments is a staple of the Python ecosystem, and it's great.
|
|
||||||
It lets you highlight code in many languages, and it has many
|
|
||||||
themes. Chroma is "Pygments for Go", it's actually a port of
|
|
||||||
Pygments to Go, and it's great too.
|
|
||||||
|
|
||||||
I wanted that in Crystal, so I started this project. But I did
|
|
||||||
not read much of the Pygments code. Or much of Chroma's.
|
|
||||||
|
|
||||||
Chroma has taken most of the Pygments lexers and turned them into
|
|
||||||
XML descriptions. What I did was take those XML files from Chroma
|
|
||||||
and a pile of test cases from Pygments, and I slapped them together
|
|
||||||
until the tests passed and my code produced the same output as
|
|
||||||
Chroma. Think of it as *extreme TDD*.
|
|
||||||
|
|
||||||
Currently the pass rate for tests in the supported languages
|
|
||||||
is `96.8%`, which is *not bad for a couple days hacking*.
|
|
||||||
|
|
||||||
This only covers the RegexLexers, which are the most common ones,
|
|
||||||
but it means the supported languages are a subset of Chroma's, which
|
|
||||||
is a subset of Pygments'.
|
|
||||||
|
|
||||||
Currently Tartrazine supports ... 247 languages.
|
|
||||||
|
|
||||||
It has 331 themes (63 from Chroma, the rest are base16 themes via
|
|
||||||
[Sixteen](https://github.com/ralsina/sixteen)
|
[Sixteen](https://github.com/ralsina/sixteen)
|
||||||
|
|
||||||
## Installation
|
## Installation
|
||||||
@ -58,7 +33,7 @@ $ tartrazine whatever.c -l c -t catppuccin-macchiato --line-numbers -f terminal
|
|||||||
Generate a standalone HTML file from a C source file with the syntax highlighted:
|
Generate a standalone HTML file from a C source file with the syntax highlighted:
|
||||||
|
|
||||||
```shell
|
```shell
|
||||||
$ tartrazine whatever.c -l c -t catppuccin-macchiato --line-numbers \
|
$ tartrazine whatever.c -t catppuccin-macchiato --line-numbers \
|
||||||
--standalone -f html -o whatever.html
|
--standalone -f html -o whatever.html
|
||||||
```
|
```
|
||||||
|
|
||||||
@ -87,3 +62,29 @@ puts formatter.format(File.read(ARGV[0]), lexer)
|
|||||||
## Contributors
|
## Contributors
|
||||||
|
|
||||||
- [Roberto Alsina](https://github.com/ralsina) - creator and maintainer
|
- [Roberto Alsina](https://github.com/ralsina) - creator and maintainer
|
||||||
|
|
||||||
|
## A port of what? Why "kind of"?
|
||||||
|
|
||||||
|
Pygments is a staple of the Python ecosystem, and it's great.
|
||||||
|
It lets you highlight code in many languages, and it has many
|
||||||
|
themes. Chroma is "Pygments for Go", it's actually a port of
|
||||||
|
Pygments to Go, and it's great too.
|
||||||
|
|
||||||
|
I wanted that in Crystal, so I started this project. But I did
|
||||||
|
not read much of the Pygments code. Or much of Chroma's.
|
||||||
|
|
||||||
|
Chroma has taken most of the Pygments lexers and turned them into
|
||||||
|
XML descriptions. What I did was take those XML files from Chroma
|
||||||
|
and a pile of test cases from Pygments, and I slapped them together
|
||||||
|
until the tests passed and my code produced the same output as
|
||||||
|
Chroma. Think of it as [*extreme TDD*](https://ralsina.me/weblog/posts/tartrazine-reimplementing-pygments.html)
|
||||||
|
|
||||||
|
Currently the pass rate for tests in the supported languages
|
||||||
|
is `96.8%`, which is *not bad for a couple days hacking*.
|
||||||
|
|
||||||
|
This only covers the RegexLexers, which are the most common ones,
|
||||||
|
but it means the supported languages are a subset of Chroma's, which
|
||||||
|
is a subset of Pygments' and DelegatingLexers (useful for things like template languages)
|
||||||
|
|
||||||
|
Then performance was bad, so I hacked and hacked and made it
|
||||||
|
significantly [faster than chroma](https://ralsina.me/weblog/posts/a-tale-of-optimization.html) which is fun.
|
4
TODO.md
4
TODO.md
@ -8,8 +8,8 @@
|
|||||||
* ✅ Implement lexer loader that respects aliases
|
* ✅ Implement lexer loader that respects aliases
|
||||||
* ✅ Implement lexer loader by file extension
|
* ✅ Implement lexer loader by file extension
|
||||||
* ✅ Add --line-numbers to terminal formatter
|
* ✅ Add --line-numbers to terminal formatter
|
||||||
* Implement lexer loader by mime type
|
* ✅ Implement lexer loader by mime type
|
||||||
* ✅ Implement Delegating lexers
|
* ✅ Implement Delegating lexers
|
||||||
* ✅ Add RstLexer
|
* ✅ Add RstLexer
|
||||||
* Add Mako template lexer
|
* Add Mako template lexer
|
||||||
* Implement heuristic lexer detection
|
* ✅ Implement heuristic lexer detection
|
||||||
|
@ -10,11 +10,13 @@ targets:
|
|||||||
|
|
||||||
dependencies:
|
dependencies:
|
||||||
baked_file_system:
|
baked_file_system:
|
||||||
github: schovi/baked_file_system
|
github: ralsina/baked_file_system
|
||||||
|
branch: master
|
||||||
base58:
|
base58:
|
||||||
github: crystal-china/base58.cr
|
github: crystal-china/base58.cr
|
||||||
sixteen:
|
sixteen:
|
||||||
github: ralsina/sixteen
|
github: ralsina/sixteen
|
||||||
|
branch: main
|
||||||
docopt:
|
docopt:
|
||||||
github: chenkovsky/docopt.cr
|
github: chenkovsky/docopt.cr
|
||||||
|
|
||||||
|
@ -11,7 +11,7 @@ module Tartrazine
|
|||||||
"#{i + 1}".rjust(4).ljust(5)
|
"#{i + 1}".rjust(4).ljust(5)
|
||||||
end
|
end
|
||||||
|
|
||||||
def format(text : String, lexer : Lexer) : String
|
def format(text : String, lexer : BaseLexer) : String
|
||||||
outp = String::Builder.new("")
|
outp = String::Builder.new("")
|
||||||
format(text, lexer, outp)
|
format(text, lexer, outp)
|
||||||
outp.to_s
|
outp.to_s
|
||||||
|
10
src/lexer.cr
10
src/lexer.cr
@ -9,13 +9,21 @@ module Tartrazine
|
|||||||
|
|
||||||
# Get the lexer object for a language name
|
# Get the lexer object for a language name
|
||||||
# FIXME: support mimetypes
|
# FIXME: support mimetypes
|
||||||
def self.lexer(name : String? = nil, filename : String? = nil) : BaseLexer
|
def self.lexer(name : String? = nil, filename : String? = nil, mimetype : String? = nil) : BaseLexer
|
||||||
return lexer_by_name(name) if name && name != "autodetect"
|
return lexer_by_name(name) if name && name != "autodetect"
|
||||||
return lexer_by_filename(filename) if filename
|
return lexer_by_filename(filename) if filename
|
||||||
|
return lexer_by_mimetype(mimetype) if mimetype
|
||||||
|
|
||||||
Lexer.from_xml(LexerFiles.get("/#{LEXERS_BY_NAME["plaintext"]}.xml").gets_to_end)
|
Lexer.from_xml(LexerFiles.get("/#{LEXERS_BY_NAME["plaintext"]}.xml").gets_to_end)
|
||||||
end
|
end
|
||||||
|
|
||||||
|
private def self.lexer_by_mimetype(mimetype : String) : BaseLexer
|
||||||
|
lexer_file_name = LEXERS_BY_MIMETYPE.fetch(mimetype, nil)
|
||||||
|
raise Exception.new("Unknown mimetype: #{mimetype}") if lexer_file_name.nil?
|
||||||
|
|
||||||
|
Lexer.from_xml(LexerFiles.get("/#{lexer_file_name}.xml").gets_to_end)
|
||||||
|
end
|
||||||
|
|
||||||
private def self.lexer_by_name(name : String) : BaseLexer
|
private def self.lexer_by_name(name : String) : BaseLexer
|
||||||
lexer_file_name = LEXERS_BY_NAME.fetch(name.downcase, nil)
|
lexer_file_name = LEXERS_BY_NAME.fetch(name.downcase, nil)
|
||||||
return create_delegating_lexer(name) if lexer_file_name.nil? && name.includes? "+"
|
return create_delegating_lexer(name) if lexer_file_name.nil? && name.includes? "+"
|
||||||
|
Reference in New Issue
Block a user