Nicoletta-Crystal/journal.md

Nicoletta On Crystal

What?

A while back (10 YEARS???? WTH.) I wrote a static site generator. I mean, I wrote one that is large and somewhat popular, called Nikola but I also wrote a tiny one called Nicoletta

Why? Because it's a nice little project and it shows the very basics of how to do a whole project.

All it does is:

• Find markdown files
• Build them
• Use templates to generate HTML files
• Put those in an output folder

And that's it, that's a SSG.

So, if I wanted a "toy" project to practice new (to me) programming languages, why not rewrite that?

And why not write about how it goes while I do it?

Hence this.

So, what's Crystal?

It's (they say) "A language for humans and computers". In short: a compiled, statically typed language with a ruby flavoured syntax.

And why? Again, why not?

Getting started

I installed it using curl and that got me version 1.8.2 which is the latest at the time of writing this.

You can get your project started by running a command:

nicoletta/crystal
✦ > crystal init app nicoletta .
    create  /home/ralsina/zig/nicoletta/crystal/.gitignore
    create  /home/ralsina/zig/nicoletta/crystal/.editorconfig
    create  /home/ralsina/zig/nicoletta/crystal/LICENSE
    create  /home/ralsina/zig/nicoletta/crystal/README.md
    create  /home/ralsina/zig/nicoletta/crystal/shard.yml
    create  /home/ralsina/zig/nicoletta/crystal/src/nicoletta.cr
    create  /home/ralsina/zig/nicoletta/crystal/spec/spec_helper.cr
    create  /home/ralsina/zig/nicoletta/crystal/spec/nicoletta_spec.cr
Initialized empty Git repository in /home/ralsina/zig/nicoletta/crystal/.git/

Some maybe interesting bits:

• It inits a git repo, with a gitignore in it
• Sets you up with a MIT license
• Creates a reasonable README with nice placeholders
• We get a shard.ymlwith metadata
• Source code in src/
• spec/ seems to be for tests?

Mind you, I still have zero idea about the language :-)

This apparently compiles into a do-nothing program, which is ok. Surprisied to see starship seems to support crystal in the prompt!

crystal on  main [?] is 📦 v0.1.0 via 🔮 v1.8.2 
> crystal build src/nicoletta.cr

crystal on  main [?] is 📦 v0.1.0 via 🔮 v1.8.2 
> ls -l
total 1748
-rw-rw-r-- 1 ralsina ralsina    2085 may 31 18:15 journal.md
-rw-r--r-- 1 ralsina ralsina    1098 may 31 18:08 LICENSE
-rwxrwxr-x 1 ralsina ralsina 1762896 may 31 18:15 nicoletta*
-rw-r--r-- 1 ralsina ralsina     604 may 31 18:08 README.md
-rw-r--r-- 1 ralsina ralsina     167 may 31 18:08 shard.yml
drwxrwxr-x 2 ralsina ralsina    4096 may 31 18:08 spec/
drwxrwxr-x 2 ralsina ralsina    4096 may 31 18:08 src/

Perhaps a bit surprising that the do-nothing binary is 1.7MB tho (1.2MB stripped) but it's just 380KB in "release mode" which is nice.

Learning a Bit of Crystal

At this point I will stop and learn some syntax:

• How to declare a variable / a literal / a constant
• How to do an if / loop
• How to define / call a function

Because you know, one has to know at least that much 😁

There seems to be a decent set of tutorials at this level. let's see how it looks.

Good thing: this is valid Crystal:

module Nicoletta
  VERSION = "0.1.0"

  😀 = "Hello world"
  puts 😀 
end

Also nice that variables can change type.

Having the docs say integers are int32 and anything else is "for special use cases" is not great. int32 is small.

Also not a huge fan of separate unsigned types.

I hate the "spaceship operator" <==> which "compares its operands and returns a value that is either zero (both operands are equal), a positive value (the first operand is bigger), or a negative value (the second operand is bigger)" ... hate it.

Numbers have named methods, which is nice. However it randomly shows some weird syntax that has not been seen before. One of these is not like the others:

p! -5.abs,   # absolute value
  4.3.round, # round to nearest integer
  5.even?,   # odd/even check
  10.gcd(16) # greatest common divisor

Or maybe the ? is just part of the method name? Who knows! Not me!

Nice string interpolation thingie.

name = "Crystal"
puts "Hello #{name}"

Why would anyone add an underscore method to strings? That's just weird.

Slices are reasonable, whatever[x..y] uses negative indexes for "from the right".

We have truthy values, 0 is truthy, only nil, false and null pointers are falsy. Ok.

I strongly dislike using unless as a keyword instead of if with a negated condition. I consider that to be keyword proliferation and cutesy.

Methods support overloading. Ok.

Ok, I know just enough Crystal to be slightly dangerous. Those feel like good tutorials. Short, to the point, give you enough rope to ... make something with rope, or whatever.

Learning a Bit More Crystal

So: errors? Classes? Blocks? How?

Classes are pretty straightforward ... apparently they are a bit frowned upon for performance reasons because they are heap allocated, but whatevs.

Inheritance with method overloading is not my cup of tea but 🤷

Exceptions are pretty simple but begin / rescue / else / ensure / end? Eek.

Also, I find that variables have nil type in the ensure block confusing.

Requiring files is not going to be a problem.

Blocks are interesting but I am not going to try to use them yet.

Dinner Break

I will grab dinner, and then try to implement Nicoletta, somehow. I'll probably fail 😅

Implementing Nicoletta

The code for nicoletta is not long so this should be a piece of cake.

No need to have a main in Crystal. Things just are executed.

First, I need a way to read the configuration. It looks like this:

TITLE: "Nicoletta Test Blog"

That is technically YAML so surely there is a crystal thing to read it. In fact, it's in the standard library! This fragment works:

require "yaml"

VERSION = "0.1.0"

tpl_data = File.open("conf") do |file|
  YAML.parse(file)
end
p! tpl_data

And when executed does this, which is correct:

crystal on  main [!?] is 📦 v0.1.0 via 🔮 v1.8.2 
> crystal run src/nicoletta.cr
tpl_data # => {"TITLE" => "Nicoletta Test Blog"}

Looks like what I want to store this sort of data is a Hash

Next step: read templates and put them in a hash indexed by path.

Templates are files in templates/ which look like this:

<h2><a href="${link}">${title}</a></h2>
date: ${date}
<hr>
${text}

Of course the syntax will probably have to change, but for now I don't care.

To find all files in templates I can apparently use Dir.glob

And I swear I wrote this almost in the first attempt:

# Load templates
templates = {} of String => String
Dir.glob("templates/*.tmpl").each do |path|
  templates[path] = File.read(path)
end

Next is iterating over all files in posts/ (which are meant to be markdown with YAML metadata on top) and do things with them.

Iterating them is the same as before (hey, this is nice)

Dir.glob("posts/*.md").each do |path|
  # Stuff
end

But I will need a Post class and so on, so...

Here is a Post class that is initialized by a path, parses metadata and keeps the text.

class Post
  def initialize(path)
    contents = File.read(path)
    metadata, @text = contents.split("\n\n", 2)
    @metadata = YAML.parse(metadata)
  end
  @metadata : YAML::Any
  @text : String
end

Next step is to give that class a method to parse the markdown and convert it to HTML.

I am not implementing that so I googled for a Crystal markdown implementation and found markd which is sadly abandoned but looks ok.

Using it is surprisingly painless thanks to Crystal's shards dependency manager. First, I added it to shard.yml:

dependencies:
  markd:
   github: icyleaf/markd

Ran shards install:

crystal on  main [!+?] is 📦 v0.1.0 via 🔮 v1.8.2 
> shards install
Resolving dependencies
Fetching https://github.com/icyleaf/markd.git
Installing markd (0.5.0)
Writing shard.lock

Then added a require "markd", slapped this code in the Post class and that's it:

  def html
    Markd.to_html(@text)
  end

Here is the code to parse all the posts and hold them in an array:

posts = [] of Post

Dir.glob("posts/*.md").each do |path|
  posts << Post.new(path)
end

Now I need a Crystal implementation of some template language, something like handlebars, I don't need much!

The standard library has a template language called ECR which is pretty nice but it's compile-time and I need this to be done in runtime. So googled and found ... Kilt

I will use the crustache variant, which implements the Mustache standard.

Again, added the dependency to shard.yml and ran shards install:

dependencies:
  markd:
   github: icyleaf/markd
  crustache:
   github: MakeNowJust/crustache

After some refactoring of template code, the template loader now looks like this:

class Template
  @text : String
  @compiled : Crustache::Syntax::Template

  def initialize(path)
    @text = File.read(path)
    @compiled = Crustache.parse(@text)
  end
end

# Load templates
templates = {} of String => Template

Dir.glob("templates/*.tmpl").each do |path|
  templates[path] = Template.new(path)
end

I changed the templates from whatever they were before to mustache:

<h2><a href="{{link}}">{{title}}</a></h2>
date: {{date}}
<hr>
{{text}}

I can now implement Post.render... except that top-level variables like templates are not accessible from inside classes and that messes up my code, so it needs refactoring. So.

This sure as hell is not idiomatic Crystal, but bear with me, I am a beginner here.

This scans for all posts, then prints them rendered with the post.tmpl template:

class Post
  @metadata = {} of YAML::Any => YAML::Any
  @text : String
  @link : String
  @html : String

  def initialize(path)
    contents = File.read(path)
    metadata, @text = contents.split("\n\n", 2)
    @metadata = YAML.parse(metadata).as_h
    @link = path.split("/")[-1][0..-4] + ".html"
    @html = Markd.to_html(@text)
  end

  def render(template)
    Crustache.render template.@compiled, @metadata.merge({"link" => @link, "text" => @html})
  end
end

posts = [] of Post

Dir.glob("posts/*.md").each do |path|
  posts << Post.new(path)
  p! p.render templates["templates/post.tmpl"]
end

Believe it or not, this is almost done. Now I need to make it output that (passed through another template) into the right path in a output/ folder.

This almost works:

Dir.glob("posts/*.md").each do |path|
  post = Post.new(path)
  rendered_post = post.render templates["templates/post.tmpl"]
  rendered_page = Crustache.render(templates["templates/page.tmpl"].@compiled,
    tpl_data.merge({
      "content" => rendered_post,
    }))
  File.open("output/#{post.@link}", "w") do |io|
    io.puts rendered_page
  end
end

For some reason all my HTML is escaped, I think that's the template engine trying to be safe 😤

Turns out I had to use TRIPLE handlebars to print unescaped HTML, so after a small fix in the templates...

So, success! It has been fun, and I quite like the language!

Here's the full source code, all 60 lines of it:

# Nicoletta, a minimal static site generator.

require "yaml"
require "markd"
require "crustache"

VERSION = "0.1.0"

# Load config file
tpl_data = File.open("conf") do |file|
  YAML.parse(file).as_h
end

class Template
  @text : String
  @compiled : Crustache::Syntax::Template

  def initialize(path)
    @text = File.read(path)
    @compiled = Crustache.parse(@text)
  end
end

# Load templates
templates = {} of String => Template

Dir.glob("templates/*.tmpl").each do |path|
  templates[path] = Template.new(path)
end

class Post
  @metadata = {} of YAML::Any => YAML::Any
  @text : String
  @link : String
  @html : String

  def initialize(path)
    contents = File.read(path)
    metadata, @text = contents.split("\n\n", 2)
    @metadata = YAML.parse(metadata).as_h
    @link = path.split("/")[-1][0..-4] + ".html"
    @html = Markd.to_html(@text)
  end

  def render(template)
    Crustache.render template.@compiled, @metadata.merge({"link" => @link, "text" => @html})
  end
end

Dir.glob("posts/*.md").each do |path|
  post = Post.new(path)
  rendered_post = post.render templates["templates/post.tmpl"]
  rendered_page = Crustache.render(templates["templates/page.tmpl"].@compiled,
    tpl_data.merge({
      "content" => rendered_post,
    }))
  File.open("output/#{post.@link}", "w") do |io|
    io.puts rendered_page
  end
end