Nicoletta-Crystal/journal.md

# Nicoletta On Crystal

## What?

A while back (10 YEARS???? WTH.) I wrote a static site generator. I mean, I wrote one that is large and somewhat popular, called [Nikola](https://github.com/getnikola/nikola) but I also wrote a tiny one called [Nicoletta](https://github.com/ralsina/nicoletta)

Why? Because it's a nice little project and it shows the very basics of how to do a whole project. 

All it does is:

* Find markdown files
* Build them
* Use templates to generate HTML files
* Put those in an output folder

And that's it, that's a SSG.

So, if I wanted a "toy" project to practice new (to me) programming languages, why not rewrite that?

And why not write about how it goes while I do it?

Hence this.

## So, what's Crystal?

It's (they say) "A language for humans and computers". In short: a compiled, statically typed language with a ruby flavoured syntax.

And why? Again, why not?

## Getting started

I installed it using [curl](https://crystal-lang.org/install/on_ubuntu/) and that got me version 1.8.2 which is the latest at the time of writing this.

You can get your project started by running a command:

```shell
nicoletta/crystal
✦ > crystal init app nicoletta .
    create  /home/ralsina/zig/nicoletta/crystal/.gitignore
    create  /home/ralsina/zig/nicoletta/crystal/.editorconfig
    create  /home/ralsina/zig/nicoletta/crystal/LICENSE
    create  /home/ralsina/zig/nicoletta/crystal/README.md
    create  /home/ralsina/zig/nicoletta/crystal/shard.yml
    create  /home/ralsina/zig/nicoletta/crystal/src/nicoletta.cr
    create  /home/ralsina/zig/nicoletta/crystal/spec/spec_helper.cr
    create  /home/ralsina/zig/nicoletta/crystal/spec/nicoletta_spec.cr
Initialized empty Git repository in /home/ralsina/zig/nicoletta/crystal/.git/
```

Some maybe interesting bits:

* It inits a git repo, with a gitignore in it
* Sets you up with a MIT license
* Creates a reasonable README with nice placeholders
* We get a `shard.yml`with metadata
* Source code in `src/`
* `spec/` seems to be for tests?

Mind you, I still have zero idea about the language :-)

This apparently compiles into a do-nothing program, which is ok. Surprisied to see [starship](https://starship.rs/) seems to support crystal in the prompt!

```shell
crystal on  main [?] is 📦 v0.1.0 via 🔮 v1.8.2 
> crystal build src/nicoletta.cr

crystal on  main [?] is 📦 v0.1.0 via 🔮 v1.8.2 
> ls -l
total 1748
-rw-rw-r-- 1 ralsina ralsina    2085 may 31 18:15 journal.md
-rw-r--r-- 1 ralsina ralsina    1098 may 31 18:08 LICENSE
-rwxrwxr-x 1 ralsina ralsina 1762896 may 31 18:15 nicoletta*
-rw-r--r-- 1 ralsina ralsina     604 may 31 18:08 README.md
-rw-r--r-- 1 ralsina ralsina     167 may 31 18:08 shard.yml
drwxrwxr-x 2 ralsina ralsina    4096 may 31 18:08 spec/
drwxrwxr-x 2 ralsina ralsina    4096 may 31 18:08 src/
```

Perhaps a bit surprising that the do-nothing binary is 1.7MB tho (1.2MB stripped) but it's just 380KB in "release mode" which is nice.

## Learning a Bit of Crystal

At this point I will stop and learn some syntax:

* How to declare a variable / a literal / a constant
* How to do an if / loop
* How to define / call a function

Because you know, one *has* to know at least that much 😁

There seems to be a decent set of [tutorials at this level.](https://crystal-lang.org/reference/1.8/tutorials/basics/index.html) let's see how it looks.

Good thing: this is valid Crystal:

```crystal
module Nicoletta
  VERSION = "0.1.0"

  😀 = "Hello world"
  puts 😀 
end
```

Also nice that variables can change type.

Having the docs say integers are `int32` and anything else is "for special use cases" is not great. `int32` is small.

Also not a huge fan of separate unsigned types.

I *hate* the "spaceship operator" `<==>` which "compares its operands and returns a value that is either zero (both operands are equal), a positive value (the first operand is bigger), or a negative value (the second operand is bigger)"  ... *hate* it.

Numbers have named methods, which is nice. However it randomly shows some weird syntax that has not been seen before. One of these is not like the others:

```crystal
p! -5.abs,   # absolute value
  4.3.round, # round to nearest integer
  5.even?,   # odd/even check
  10.gcd(16) # greatest common divisor
```

Or maybe the `?` is just part of the method name? Who knows! Not me!

Nice string interpolation thingie.

```crystal
name = "Crystal"
puts "Hello #{name}"
```

Why would anyone add an `underscore` method to strings? That's just weird.

Slices are reasonable, `whatever[x..y]` uses negative indexes for "from the right".

We have truthy values, 0 is truthy, only nil, false and null pointers are falsy. Ok.

I *strongly dislike* using `unless` as a keyword instead of `if` with a negated condition. I consider that to be keyword proliferation and cutesy.

Methods support overloading. Ok.

Ok, I know just enough Crystal to be slightly dangerous. Those feel like **good** tutorials. Short, to the point, give you enough rope to ... make something with rope, or whatever.

## Learning a Bit More Crystal

So: errors? Classes? Blocks? How?

Classes are [pretty straightforward](https://crystal-lang.org/reference/1.8/syntax_and_semantics/new%2C_initialize_and_allocate.html) ... apparently they are a bit frowned upon for performance reasons because they are heap allocated, but whatevs.

Inheritance with method overloading is not my cup of tea but 🤷

Exceptions are [pretty simple](https://crystal-lang.org/reference/1.8/syntax_and_semantics/exception_handling.html) but `begin / rescue / else / ensure / end`? Eek.

Also, I find that variables have `nil` type in the `ensure` block confusing.

[Requiring files](https://crystal-lang.org/reference/1.8/syntax_and_semantics/requiring_files.html) is not going to be a problem. 

[Blocks](https://crystal-lang.org/reference/1.8/syntax_and_semantics/blocks_and_procs.html) are interesting but I am not going to try to use them yet.

## Dinner Break

I will grab dinner, and then try to implement Nicoletta, somehow. I'll probably fail 😅


## Implementing Nicoletta

The code for nicoletta [is not long](https://github.com/ralsina/nicoletta/blob/master/nicoletta.py) so this should be a piece of cake.

No need to have a `main` in Crystal. Things just are executed.

First, I need a way to read the configuration. It looks like this:

```yaml
TITLE: "Nicoletta Test Blog"
```

That is *technically YAML* so surely there is a crystal thing to read it. In fact, it's in the standard library! This fragment works:

```crystal
require "yaml"

VERSION = "0.1.0"

tpl_data = File.open("conf") do |file|
  YAML.parse(file)
end
p! tpl_data
``` 

And when executed does this, which is correct:

```sh
crystal on  main [!?] is 📦 v0.1.0 via 🔮 v1.8.2 
> crystal run src/nicoletta.cr
tpl_data # => {"TITLE" => "Nicoletta Test Blog"}
```

Looks like what I want to store this sort of data is a [Hash](https://crystal-lang.org/reference/1.8/syntax_and_semantics/literals/hash.html)

Next step: read templates and put them in a hash indexed by path.

Templates are files in `templates/` which look like this:

```
<h2><a href="${link}">${title}</a></h2>
date: ${date}
<hr>
${text}
```

Of course the syntax will probably have to change, but for now I don't care.

To find all files in `templates` I can apparently use [`Dir.glob`](https://crystal-lang.org/api/1.8.2/Dir.html#glob%28%2Apatterns%3APath%7CString%2Cmatch_hidden%3Dfalse%2Cfollow_symlinks%3Dfalse%29%3AArray%28String%29-class-method)

And I swear I wrote this *almost* in the first attempt:

```Crystal
# Load templates
templates = {} of String => String
Dir.glob("templates/*.tmpl").each do |path|
  templates[path] = File.read(path)
end
```

Next is iterating over all files in `posts/` (which are meant to be markdown with YAML metadata on top) and do things with them.

Iterating them is the same as before (hey, this is *nice*)

```Crystal
Dir.glob("posts/*.md").each do |path|
  # Stuff
end
```

But I will need a `Post` class and so on, so...

Here is a `Post` class that is initialized by a path, parses metadata and keeps the text. 

```Crystal
class Post
  def initialize(path)
    contents = File.read(path)
    metadata, @text = contents.split("\n\n", 2)
    @metadata = YAML.parse(metadata)
  end
  @metadata : YAML::Any
  @text : String
end
```
 
Next step is to give that class a method to parse the markdown and convert it to HTML.

I am *not* implementing that so I googled for a Crystal markdown implementation and found [markd](https://github.com/icyleaf/markd) which is sadly abandoned but looks ok.

Using it is surprisingly painless thanks to Crystal's [shards](https://crystal-lang.org/reference/1.8/man/shards/index.html) dependency manager. First, I added it to `shard.yml`:

```yaml
dependencies:
  markd:
   github: icyleaf/markd
```

Ran `shards install`:

```sh
crystal on  main [!+?] is 📦 v0.1.0 via 🔮 v1.8.2 
> shards install
Resolving dependencies
Fetching https://github.com/icyleaf/markd.git
Installing markd (0.5.0)
Writing shard.lock
```

Then added a `require "markd"`, slapped this code in the `Post` class and that's it:

```Crystal
  def html
    Markd.to_html(@text)
  end
```

Here is the code to parse all the posts and hold them in an array:

```
posts = [] of Post

Dir.glob("posts/*.md").each do |path|
  posts << Post.new(path)
end
```

Now I need a Crystal implementation of some template language, something like [handlebars](https://handlebarsjs.com/), I don't need much!

The standard library has a template language called [ECR](https://crystal-lang.org/api/1.8.2/ECR.html) which is pretty nice but it's compile-time and I need this to be done in runtime. So googled and found ... [Kilt](https://github.com/jeromegn/kilt)

I will use the [crustache](https://github.com/MakeNowJust/crustache) variant, which implements the [Mustache](https://mustache.github.io/) standard.

Again, added the dependency to `shard.yml` and ran `shards install`:

```yaml
dependencies:
  markd:
   github: icyleaf/markd
  crustache:
   github: MakeNowJust/crustache
```

After some refactoring of template code, the template loader now looks like this:

```Crystal
class Template
  @text : String
  @compiled : Crustache::Syntax::Template

  def initialize(path)
    @text = File.read(path)
    @compiled = Crustache.parse(@text)
  end
end

# Load templates
templates = {} of String => Template

Dir.glob("templates/*.tmpl").each do |path|
  templates[path] = Template.new(path)
end
```

I changed the templates from whatever they were before to mustache:

```html
<h2><a href="{{link}}">{{title}}</a></h2>
date: {{date}}
<hr>
{{text}}
```

I can now implement `Post.render`... except that top-level variables like `templates` are not accessible from inside classes and that messes up my code, so it needs refactoring. So.

This sure as hell is not idiomatic Crystal, but bear with me, I am a beginner here.

This scans for all posts, then prints them rendered with the `post.tmpl` template:

```Crystal
class Post
  @metadata = {} of YAML::Any => YAML::Any
  @text : String
  @link : String
  @html : String

  def initialize(path)
    contents = File.read(path)
    metadata, @text = contents.split("\n\n", 2)
    @metadata = YAML.parse(metadata).as_h
    @link = path.split("/")[-1][0..-4] + ".html"
    @html = Markd.to_html(@text)
  end

  def render(template)
    Crustache.render template.@compiled, @metadata.merge({"link" => @link, "text" => @html})
  end
end

posts = [] of Post

Dir.glob("posts/*.md").each do |path|
  posts << Post.new(path)
  p! p.render templates["templates/post.tmpl"]
end
```

Believe it or not, this is almost done.
Now I need to make it output that (passed through another template) into the right path in a `output/` folder.


This **almost** works:

```Crystal
Dir.glob("posts/*.md").each do |path|
  post = Post.new(path)
  rendered_post = post.render templates["templates/post.tmpl"]
  rendered_page = Crustache.render(templates["templates/page.tmpl"].@compiled,
    tpl_data.merge({
      "content" => rendered_post,
    }))
  File.open("output/#{post.@link}", "w") do |io|
    io.puts rendered_page
  end
end
```

For some reason all my HTML is escaped, I think that's the template engine trying to be safe 😤

Turns out I had to use TRIPLE handlebars to print unescaped HTML, so after a small fix in the templates... 

![A small HTML page](https://i.imgur.com/HjvL0Y4.png)

So, success! It has been fun, and I quite like the language!

Here's the full source code, all 60 lines of it:

```Crystal
# Nicoletta, a minimal static site generator.

require "yaml"
require "markd"
require "crustache"

VERSION = "0.1.0"

# Load config file
tpl_data = File.open("conf") do |file|
  YAML.parse(file).as_h
end

class Template
  @text : String
  @compiled : Crustache::Syntax::Template

  def initialize(path)
    @text = File.read(path)
    @compiled = Crustache.parse(@text)
  end
end

# Load templates
templates = {} of String => Template

Dir.glob("templates/*.tmpl").each do |path|
  templates[path] = Template.new(path)
end

class Post
  @metadata = {} of YAML::Any => YAML::Any
  @text : String
  @link : String
  @html : String

  def initialize(path)
    contents = File.read(path)
    metadata, @text = contents.split("\n\n", 2)
    @metadata = YAML.parse(metadata).as_h
    @link = path.split("/")[-1][0..-4] + ".html"
    @html = Markd.to_html(@text)
  end

  def render(template)
    Crustache.render template.@compiled, @metadata.merge({"link" => @link, "text" => @html})
  end
end

Dir.glob("posts/*.md").each do |path|
  post = Post.new(path)
  rendered_post = post.render templates["templates/post.tmpl"]
  rendered_page = Crustache.render(templates["templates/page.tmpl"].@compiled,
    tpl_data.merge({
      "content" => rendered_post,
    }))
  File.open("output/#{post.@link}", "w") do |io|
    io.puts rendered_page
  end
end
```