diff --git a/journal.md b/journal.md new file mode 100644 index 0000000..4137d1c --- /dev/null +++ b/journal.md @@ -0,0 +1,462 @@ +# Nicoletta On Crystal + +## What? + +A while back (10 YEARS???? WTH.) I wrote a static site generator. I mean, I wrote one that is large and somewhat popular, called [Nikola](https://github.com/getnikola/nikola) but I also wrote a tiny one called [Nicoletta](https://github.com/ralsina/nicoletta) + +Why? Because it's a nice little project and it shows the very basics of how to do a whole project. + +All it does is: + +* Find markdown files +* Build them +* Use templates to generate HTML files +* Put those in an output folder + +And that's it, that's a SSG. + +So, if I wanted a "toy" project to practice new (to me) programming languages, why not rewrite that? + +And why not write about how it goes while I do it? + +Hence this. + +## So, what's Crystal? + +It's (they say) "A language for humans and computers". In short: a compiled, statically typed language with a ruby flavoured syntax. + +And why? Again, why not? + +## Getting started + +I installed it using [curl](https://crystal-lang.org/install/on_ubuntu/) and that got me version 1.8.2 which is the latest at the time of writing this. + +You can get your project started by running a command: + +```shell +nicoletta/crystal +✦ > crystal init app nicoletta . + create /home/ralsina/zig/nicoletta/crystal/.gitignore + create /home/ralsina/zig/nicoletta/crystal/.editorconfig + create /home/ralsina/zig/nicoletta/crystal/LICENSE + create /home/ralsina/zig/nicoletta/crystal/README.md + create /home/ralsina/zig/nicoletta/crystal/shard.yml + create /home/ralsina/zig/nicoletta/crystal/src/nicoletta.cr + create /home/ralsina/zig/nicoletta/crystal/spec/spec_helper.cr + create /home/ralsina/zig/nicoletta/crystal/spec/nicoletta_spec.cr +Initialized empty Git repository in /home/ralsina/zig/nicoletta/crystal/.git/ +``` + +Some maybe interesting bits: + +* It inits a git repo, with a gitignore in it +* Sets you up with a MIT license +* Creates a reasonable README with nice placeholders +* We get a `shard.yml`with metadata +* Source code in `src/` +* `spec/` seems to be for tests? + +Mind you, I still have zero idea about the language :-) + +This apparently compiles into a do-nothing program, which is ok. Surprisied to see [starship](https://starship.rs/) seems to support crystal in the prompt! + +```shell +crystal on  main [?] is 📦 v0.1.0 via 🔮 v1.8.2 +> crystal build src/nicoletta.cr + +crystal on  main [?] is 📦 v0.1.0 via 🔮 v1.8.2 +> ls -l +total 1748 +-rw-rw-r-- 1 ralsina ralsina 2085 may 31 18:15 journal.md +-rw-r--r-- 1 ralsina ralsina 1098 may 31 18:08 LICENSE +-rwxrwxr-x 1 ralsina ralsina 1762896 may 31 18:15 nicoletta* +-rw-r--r-- 1 ralsina ralsina 604 may 31 18:08 README.md +-rw-r--r-- 1 ralsina ralsina 167 may 31 18:08 shard.yml +drwxrwxr-x 2 ralsina ralsina 4096 may 31 18:08 spec/ +drwxrwxr-x 2 ralsina ralsina 4096 may 31 18:08 src/ +``` + +Perhaps a bit surprising that the do-nothing binary is 1.7MB tho (1.2MB stripped) but it's just 380KB in "release mode" which is nice. + +## Learning a Bit of Crystal + +At this point I will stop and learn some syntax: + +* How to declare a variable / a literal / a constant +* How to do an if / loop +* How to define / call a function + +Because you know, one *has* to know at least that much 😁 + +There seems to be a decent set of [tutorials at this level.](https://crystal-lang.org/reference/1.8/tutorials/basics/index.html) let's see how it looks. + +Good thing: this is valid Crystal: + +```crystal +module Nicoletta + VERSION = "0.1.0" + + 😀 = "Hello world" + puts 😀 +end +``` + +Also nice that variables can change type. + +Having the docs say integers are `int32` and anything else is "for special use cases" is not great. `int32` is small. + +Also not a huge fan of separate unsigned types. + +I *hate* the "spaceship operator" `<==>` which "compares its operands and returns a value that is either zero (both operands are equal), a positive value (the first operand is bigger), or a negative value (the second operand is bigger)" ... *hate* it. + +Numbers have named methods, which is nice. However it randomly shows some weird syntax that has not been seen before. One of these is not like the others: + +```crystal +p! -5.abs, # absolute value + 4.3.round, # round to nearest integer + 5.even?, # odd/even check + 10.gcd(16) # greatest common divisor +``` + +Or maybe the `?` is just part of the method name? Who knows! Not me! + +Nice string interpolation thingie. + +```crystal +name = "Crystal" +puts "Hello #{name}" +``` + +Why would anyone add an `underscore` method to strings? That's just weird. + +Slices are reasonable, `whatever[x..y]` uses negative indexes for "from the right". + +We have truthy values, 0 is truthy, only nil, false and null pointers are falsy. Ok. + +I *strongly dislike* using `unless` as a keyword instead of `if` with a negated condition. I consider that to be keyword proliferation and cutesy. + +Methods support overloading. Ok. + +Ok, I know just enough Crystal to be slightly dangerous. Those feel like **good** tutorials. Short, to the point, give you enough rope to ... make something with rope, or whatever. + +## Learning a Bit More Crystal + +So: errors? Classes? Blocks? How? + +Classes are [pretty straightforward](https://crystal-lang.org/reference/1.8/syntax_and_semantics/new%2C_initialize_and_allocate.html) ... apparently they are a bit frowned upon for performance reasons because they are heap allocated, but whatevs. + +Inheritance with method overloading is not my cup of tea but 🤷 + +Exceptions are [pretty simple](https://crystal-lang.org/reference/1.8/syntax_and_semantics/exception_handling.html) but `begin / rescue / else / ensure / end`? Eek. + +Also, I find that variables have `nil` type in the `ensure` block confusing. + +[Requiring files](https://crystal-lang.org/reference/1.8/syntax_and_semantics/requiring_files.html) is not going to be a problem. + +[Blocks](https://crystal-lang.org/reference/1.8/syntax_and_semantics/blocks_and_procs.html) are interesting but I am not going to try to use them yet. + +## Dinner Break + +I will grab dinner, and then try to implement Nicoletta, somehow. I'll probably fail 😅 + + +## Implementing Nicoletta + +The code for nicoletta [is not long](https://github.com/ralsina/nicoletta/blob/master/nicoletta.py) so this should be a piece of cake. + +No need to have a `main` in Crystal. Things just are executed. + +First, I need a way to read the configuration. It looks like this: + +```yaml +TITLE: "Nicoletta Test Blog" +``` + +That is *technically YAML* so surely there is a crystal thing to read it. In fact, it's in the standard library! This fragment works: + +```crystal +require "yaml" + +VERSION = "0.1.0" + +tpl_data = File.open("conf") do |file| + YAML.parse(file) +end +p! tpl_data +``` + +And when executed does this, which is correct: + +```sh +crystal on  main [!?] is 📦 v0.1.0 via 🔮 v1.8.2 +> crystal run src/nicoletta.cr +tpl_data # => {"TITLE" => "Nicoletta Test Blog"} +``` + +Looks like what I want to store this sort of data is a [Hash](https://crystal-lang.org/reference/1.8/syntax_and_semantics/literals/hash.html) + +Next step: read templates and put them in a hash indexed by path. + +Templates are files in `templates/` which look like this: + +``` +

${title}

+date: ${date} +
+${text} +``` + +Of course the syntax will probably have to change, but for now I don't care. + +To find all files in `templates` I can apparently use [`Dir.glob`](https://crystal-lang.org/api/1.8.2/Dir.html#glob%28%2Apatterns%3APath%7CString%2Cmatch_hidden%3Dfalse%2Cfollow_symlinks%3Dfalse%29%3AArray%28String%29-class-method) + +And I swear I wrote this *almost* in the first attempt: + +```Crystal +# Load templates +templates = {} of String => String +Dir.glob("templates/*.tmpl").each do |path| + templates[path] = File.read(path) +end +``` + +Next is iterating over all files in `posts/` (which are meant to be markdown with YAML metadata on top) and do things with them. + +Iterating them is the same as before (hey, this is *nice*) + +```Crystal +Dir.glob("posts/*.md").each do |path| + # Stuff +end +``` + +But I will need a `Post` class and so on, so... + +Here is a `Post` class that is initialized by a path, parses metadata and keeps the text. + +```Crystal +class Post + def initialize(path) + contents = File.read(path) + metadata, @text = contents.split("\n\n", 2) + @metadata = YAML.parse(metadata) + end + @metadata : YAML::Any + @text : String +end +``` + +Next step is to give that class a method to parse the markdown and convert it to HTML. + +I am *not* implementing that so I googled for a Crystal markdown implementation and found [markd](https://github.com/icyleaf/markd) which is sadly abandoned but looks ok. + +Using it is surprisingly painless thanks to Crystal's [shards](https://crystal-lang.org/reference/1.8/man/shards/index.html) dependency manager. First, I added it to `shard.yml`: + +```yaml +dependencies: + markd: + github: icyleaf/markd +``` + +Ran `shards install`: + +```sh +crystal on  main [!+?] is 📦 v0.1.0 via 🔮 v1.8.2 +> shards install +Resolving dependencies +Fetching https://github.com/icyleaf/markd.git +Installing markd (0.5.0) +Writing shard.lock +``` + +Then added a `require "markd"`, slapped this code in the `Post` class and that's it: + +```Crystal + def html + Markd.to_html(@text) + end +``` + +Here is the code to parse all the posts and hold them in an array: + +``` +posts = [] of Post + +Dir.glob("posts/*.md").each do |path| + posts << Post.new(path) +end +``` + +Now I need a Crystal implementation of some template language, something like [handlebars](https://handlebarsjs.com/), I don't need much! + +The standard library has a template language called [ECR](https://crystal-lang.org/api/1.8.2/ECR.html) which is pretty nice but it's compile-time and I need this to be done in runtime. So googled and found ... [Kilt](https://github.com/jeromegn/kilt) + +I will use the [crustache](https://github.com/MakeNowJust/crustache) variant, which implements the [Mustache](https://mustache.github.io/) standard. + +Again, added the dependency to `shard.yml` and ran `shards install`: + +```yaml +dependencies: + markd: + github: icyleaf/markd + crustache: + github: MakeNowJust/crustache +``` + +After some refactoring of template code, the template loader now looks like this: + +```Crystal +class Template + @text : String + @compiled : Crustache::Syntax::Template + + def initialize(path) + @text = File.read(path) + @compiled = Crustache.parse(@text) + end +end + +# Load templates +templates = {} of String => Template + +Dir.glob("templates/*.tmpl").each do |path| + templates[path] = Template.new(path) +end +``` + +I changed the templates from whatever they were before to mustache: + +```html +

{{title}}

+date: {{date}} +
+{{text}} +``` + +I can now implement `Post.render`... except that top-level variables like `templates` are not accessible from inside classes and that messes up my code, so it needs refactoring. So. + +This sure as hell is not idiomatic Crystal, but bear with me, I am a beginner here. + +This scans for all posts, then prints them rendered with the `post.tmpl` template: + +```Crystal +class Post + @metadata = {} of YAML::Any => YAML::Any + @text : String + @link : String + @html : String + + def initialize(path) + contents = File.read(path) + metadata, @text = contents.split("\n\n", 2) + @metadata = YAML.parse(metadata).as_h + @link = path.split("/")[-1][0..-4] + ".html" + @html = Markd.to_html(@text) + end + + def render(template) + Crustache.render template.@compiled, @metadata.merge({"link" => @link, "text" => @html}) + end +end + +posts = [] of Post + +Dir.glob("posts/*.md").each do |path| + posts << Post.new(path) + p! p.render templates["templates/post.tmpl"] +end +``` + +Believe it or not, this is almost done. +Now I need to make it output that (passed through another template) into the right path in a `output/` folder. + + +This **almost** works: + +```Crystal +Dir.glob("posts/*.md").each do |path| + post = Post.new(path) + rendered_post = post.render templates["templates/post.tmpl"] + rendered_page = Crustache.render(templates["templates/page.tmpl"].@compiled, + tpl_data.merge({ + "content" => rendered_post, + })) + File.open("output/#{post.@link}", "w") do |io| + io.puts rendered_page + end +end +``` + +For some reason all my HTML is escaped, I think that's the template engine trying to be safe 😤 + +Turns out I had to use TRIPLE handlebars to print unescaped HTML, so after a small fix in the templates... + +![A small HTML page](https://i.imgur.com/HjvL0Y4.png) + +So, success! It has been fun, and I quite like the language! + +Here's the full source code, all 60 lines of it: + +```Crystal +# Nicoletta, a minimal static site generator. + +require "yaml" +require "markd" +require "crustache" + +VERSION = "0.1.0" + +# Load config file +tpl_data = File.open("conf") do |file| + YAML.parse(file).as_h +end + +class Template + @text : String + @compiled : Crustache::Syntax::Template + + def initialize(path) + @text = File.read(path) + @compiled = Crustache.parse(@text) + end +end + +# Load templates +templates = {} of String => Template + +Dir.glob("templates/*.tmpl").each do |path| + templates[path] = Template.new(path) +end + +class Post + @metadata = {} of YAML::Any => YAML::Any + @text : String + @link : String + @html : String + + def initialize(path) + contents = File.read(path) + metadata, @text = contents.split("\n\n", 2) + @metadata = YAML.parse(metadata).as_h + @link = path.split("/")[-1][0..-4] + ".html" + @html = Markd.to_html(@text) + end + + def render(template) + Crustache.render template.@compiled, @metadata.merge({"link" => @link, "text" => @html}) + end +end + +Dir.glob("posts/*.md").each do |path| + post = Post.new(path) + rendered_post = post.render templates["templates/post.tmpl"] + rendered_page = Crustache.render(templates["templates/page.tmpl"].@compiled, + tpl_data.merge({ + "content" => rendered_post, + })) + File.open("output/#{post.@link}", "w") do |io| + io.puts rendered_page + end +end +``` + diff --git a/output/welcome.html b/output/welcome.html new file mode 100644 index 0000000..71d724e --- /dev/null +++ b/output/welcome.html @@ -0,0 +1,57 @@ + + + + + + + + + + + Starter Template for Bootstrap + + + + + + +
+
+

Hi there, Welcome to Nicoletta!

+date: 2013-02-01 00:00:00 UTC +
+

If you want a real featureful static site generator, you may want to check Nicoletta's +big brother Nikola

+

This is just some random filler. It's markdown, so we can do this and this.

+ + +
+
+ + + + + + diff --git a/posts/welcome.md b/posts/welcome.md new file mode 100644 index 0000000..084ff2d --- /dev/null +++ b/posts/welcome.md @@ -0,0 +1,7 @@ +title: Hi there, Welcome to Nicoletta! +date: 2013-02-01 + +If you want a real featureful static site generator, you may want to check Nicoletta's +big brother [Nikola](http://getnikola.com) + +This is just some random filler. It's markdown, so we can do *this* and **this**. diff --git a/shard.yml b/shard.yml index 3b9414a..5692b4b 100644 --- a/shard.yml +++ b/shard.yml @@ -8,6 +8,12 @@ targets: nicoletta: main: src/nicoletta.cr -crystal: 1.8.1 +crystal: 1.8.2 license: MIT + +dependencies: + markd: + github: icyleaf/markd + crustache: + github: MakeNowJust/crustache diff --git a/src/nicoletta.cr b/src/nicoletta.cr index 4265176..9ba7c67 100644 --- a/src/nicoletta.cr +++ b/src/nicoletta.cr @@ -1,12 +1,60 @@ # Nicoletta, a minimal static site generator. -reuire "yaml" +require "yaml" +require "markd" +require "crustache" -module Nicoletta - VERSION = "0.1.0" +VERSION = "0.1.0" - tpl_data = File.open("conf") do |file| - YAML.parse(file) - end - p! tpl_data +# Load config file +tpl_data = File.open("conf") do |file| + YAML.parse(file).as_h +end + +class Template + @text : String + @compiled : Crustache::Syntax::Template + + def initialize(path) + @text = File.read(path) + @compiled = Crustache.parse(@text) + end +end + +# Load templates +templates = {} of String => Template + +Dir.glob("templates/*.tmpl").each do |path| + templates[path] = Template.new(path) +end + +class Post + @metadata = {} of YAML::Any => YAML::Any + @text : String + @link : String + @html : String + + def initialize(path) + contents = File.read(path) + metadata, @text = contents.split("\n\n", 2) + @metadata = YAML.parse(metadata).as_h + @link = path.split("/")[-1][0..-4] + ".html" + @html = Markd.to_html(@text) + end + + def render(template) + Crustache.render template.@compiled, @metadata.merge({"link" => @link, "text" => @html}) + end +end + +Dir.glob("posts/*.md").each do |path| + post = Post.new(path) + rendered_post = post.render templates["templates/post.tmpl"] + rendered_page = Crustache.render(templates["templates/page.tmpl"].@compiled, + tpl_data.merge({ + "content" => rendered_post, + })) + File.open("output/#{post.@link}", "w") do |io| + io.puts rendered_page + end end diff --git a/templates/page.tmpl b/templates/page.tmpl new file mode 100644 index 0000000..3835290 --- /dev/null +++ b/templates/page.tmpl @@ -0,0 +1,50 @@ + + + + + + + + + + + Starter Template for Bootstrap + + + + + + +
+
+ {{{content}}} +
+
+ + + + + + diff --git a/templates/post.tmpl b/templates/post.tmpl new file mode 100644 index 0000000..10cc6f6 --- /dev/null +++ b/templates/post.tmpl @@ -0,0 +1,4 @@ +

{{title}}

+date: {{date}} +
+{{{text}}}