Compare commits

..

81 Commits

Author SHA1 Message Date
21893fe612 Beginnings of making frontend pretty, switched to pico css 2024-07-09 20:37:43 -03:00
665b4f9ab7 Fix proxy config confusion 2024-07-09 16:40:15 -03:00
3b2297e954 Housekeeping 2024-07-09 13:23:21 -03:00
4f4daf5943 Types tweak 2024-07-09 13:23:16 -03:00
2d333c3df1 * More robust stderr/stdout mixing when
running commandsserver side
* More robust streaming responses client side
2024-07-09 13:01:37 -03:00
f015afe7f0 Ensure network exists for start-proxy 2024-07-09 10:57:36 -03:00
81ec077928 Use base58 2024-07-08 22:34:38 -03:00
afb6e8df0c Dockerfile linter 2024-07-08 22:34:27 -03:00
50e8ff7e56 add check to ensure config/funkos is empty 2024-07-08 22:34:07 -03:00
6489ec0dc2 Use base58 for random strings 2024-07-08 22:33:52 -03:00
56ff326098 Lint 2024-07-08 22:02:47 -03:00
80ea5d4fde add check to ensure config/funkos is empty 2024-07-08 21:53:57 -03:00
67c37a49e1 Refactored log setup onto separate project 2024-07-08 21:27:40 -03:00
ef8f5c357f Some type 2024-07-08 17:42:33 -03:00
a1a141c77e ignore faaso config 2024-07-08 17:41:37 -03:00
77b1539776 Some types 2024-07-08 17:41:11 -03:00
eea98ff8f6 todo management 2024-07-08 15:20:12 -03:00
a2d65d4b5e Cleanup tmp directory after build 2024-07-08 15:18:06 -03:00
0358744b46 Remove template correctly in Runtime.runtime_files 2024-07-08 13:51:39 -03:00
3378899612 todo management 2024-07-08 13:39:37 -03:00
a896f2e032 Rethought login 2024-07-08 13:34:54 -03:00
fe52566872 Faaso deploy command 2024-07-08 11:46:40 -03:00
62d66a5286 Faaso deploy command 2024-07-08 11:46:33 -03:00
2face37b19 todo management 2024-07-08 11:24:55 -03:00
c59bf87bbd oops 2024-07-08 11:16:26 -03:00
8e6ec620aa Faaso deploy command 2024-07-08 11:16:03 -03:00
5e7764ca9f todo management 2024-07-08 07:41:06 -03:00
d93c8518da Minor refactor 2024-07-08 07:37:01 -03:00
379b4e2472 No need to filter containers here 2024-07-08 07:18:27 -03:00
7104403585 todo management 2024-07-08 07:16:51 -03:00
47fcdda2d4 todo management 2024-07-08 07:13:46 -03:00
68c9960a58 Fix log 2024-07-07 21:50:41 -03:00
52e461ec80 Save a little space in the final image 2024-07-07 21:50:10 -03:00
bcbf74bbaa Update example funko 2024-07-07 21:49:52 -03:00
bbe934f2b8 todo management 2024-07-07 21:49:21 -03:00
8228ea3233 todo management 2024-07-07 21:19:00 -03:00
f14a8d1c39 Use configurable auth everywhere (not tested) 2024-07-07 21:07:20 -03:00
4aa307c65c Implemented basic client config for auth 2024-07-07 20:48:32 -03:00
2ddbda5a4f Fixed terminal proxying/auth 2024-07-07 16:06:49 -03:00
10775aeb11 todo management 2024-07-07 14:02:01 -03:00
6bd98d6792 Fix forward_auth in caddy, keep proxy open when run locally 2024-07-07 13:59:54 -03:00
7dd5248a6e Use password from config in basic auth 2024-07-07 13:29:12 -03:00
31ab54478a Added config support for daemon, made password configurable 2024-07-07 11:51:14 -03:00
3e4c940479 No need for proxy.env anymore 2024-07-07 11:08:14 -03:00
30e447dc8a Do basic auth in Caddy, but delegating user/pass auth to kemal 2024-07-07 11:07:34 -03:00
3b45b6a28f todo management 2024-07-07 10:16:30 -03:00
495e5c350e todo management 2024-07-07 10:05:59 -03:00
c7188b87d7 Fix logging 2024-07-07 10:02:06 -03:00
824c94bebc Compare client/server versions and warn if different 2024-07-06 22:27:34 -03:00
889a5a2955 todo management 2024-07-06 22:15:17 -03:00
de46e9864b Make caddy reload reactive on modified file 2024-07-06 21:37:30 -03:00
46ff8fc584 Express runtime working 2024-07-06 21:21:02 -03:00
125870d0a8 Unify stderr/stdout of faaso_run 2024-07-06 21:20:20 -03:00
6ff67e0190 Added ping endpoint, lower resource usage 2024-07-06 20:48:56 -03:00
b611ed199b Streaming responses (WIP) 2024-07-06 20:28:58 -03:00
86e2db39fb todo management 2024-07-06 20:28:08 -03:00
31509df8f7 todo management 2024-07-06 15:30:43 -03:00
c28239295f Use new config envvar 2024-07-06 12:54:49 -03:00
eedad69f72 Fix bug 2024-07-06 12:21:45 -03:00
bf8d86de9b todo management 2024-07-06 12:13:20 -03:00
43e1bb44b1 todo management 2024-07-06 12:00:48 -03:00
53986899c1 todo management 2024-07-06 11:59:55 -03:00
283620f49a Remove random stuff from Caddyfile 2024-07-06 11:58:40 -03:00
2c513b34c8 Move Caddyfile into config/ 2024-07-06 11:57:30 -03:00
d6e8a6013a todo management 2024-07-06 11:57:18 -03:00
f828268cba todo management 2024-07-06 11:53:53 -03:00
30cfd2e5e6 Sort-of-working flask runtime 2024-07-05 17:33:05 -03:00
f0aa127eed Use debug build 2024-07-05 17:32:41 -03:00
8efab6b5f8 Cleanup, less brittle 2024-07-05 17:09:57 -03:00
fc55f19660 lint 2024-07-05 16:36:27 -03:00
1b2bddf2d2 Add 'delete' action for funkos that removes all trace 2024-07-05 16:36:02 -03:00
cd868b1c86 Use make, build always the same way 2024-07-05 16:35:42 -03:00
1d2b44a3ba make non-embedded runtimes work properly 2024-07-05 15:58:33 -03:00
1a48ffbf22 Refactor runtime code into reason 2024-07-05 15:27:27 -03:00
d5efd6b301 Make things work enough for video demo 2024-07-05 13:25:40 -03:00
0ffd562cca Fix 2024-07-05 12:05:51 -03:00
2644cd4b86 Fix rucksack and bugs 2024-07-05 12:04:36 -03:00
2eb20aa09b More templating, lint 2024-07-05 10:33:40 -03:00
42f43db2b1 Basic runtime template implementation 2024-07-05 10:23:46 -03:00
6eb9c49c22 Return success/failure in commands uniformly 2024-07-05 00:13:13 -03:00
33cd8be45b Some debug info 2024-07-04 23:38:44 -03:00
56 changed files with 1081 additions and 398 deletions

View File

@ -1,19 +1,17 @@
# This configuration file was generated by `ameba --gen-config`
# on 2024-07-03 18:18:32 UTC using Ameba version 1.6.1.
# on 2024-07-07 14:44:11 UTC using Ameba version 1.6.1.
# The point is for the user to remove these configuration records
# one by one as the reported problems are removed from the code base.
# Problems found: 6
# Problems found: 5
# Run `ameba --only Documentation/DocumentationAdmonition` for details
Documentation/DocumentationAdmonition:
Description: Reports documentation admonitions
Timezone: UTC
Excluded:
- src/secrets.cr
- src/daemon/main.cr
- src/daemon/secrets.cr
- src/daemon/funko.cr
- src/funko.cr
- spec/faaso_spec.cr
Admonitions:
- TODO
@ -22,6 +20,16 @@ Documentation/DocumentationAdmonition:
Enabled: true
Severity: Warning
# Problems found: 1
# Run `ameba --only Lint/UselessAssign` for details
Lint/UselessAssign:
Description: Disallows useless variable assignments
ExcludeTypeDeclarations: false
Excluded:
- src/daemon/config.cr
Enabled: true
Severity: Warning
# Problems found: 3
# Run `ameba --only Naming/BlockParameterName` for details
Naming/BlockParameterName:

1
.gitignore vendored
View File

@ -7,3 +7,4 @@ export/
secrets/
.rucksack
.rucksack.toc
.faaso.yml

3
.hadolint.yml Normal file
View File

@ -0,0 +1,3 @@
ignored:
- DL3018
- DL3059

38
.pre-commit-config.yaml Normal file
View File

@ -0,0 +1,38 @@
# See https://pre-commit.com for more information
# See https://pre-commit.com/hooks.html for more hooks
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.4.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-yaml
- id: check-added-large-files
- id: check-merge-conflict
- repo: https://github.com/jumanjihouse/pre-commit-hooks
rev: 3.0.0
hooks:
- id: shellcheck
- id: markdownlint
exclude: '^content'
- repo: https://github.com/mrtazz/checkmake
rev: 0.2.2
hooks:
- id: checkmake
- repo: https://github.com/python-jsonschema/check-jsonschema
rev: 0.23.2
hooks:
- id: check-github-workflows
- repo: local
hooks:
- id: empty-funkos
name: empty-funkos
entry: test ! -s config/funkos
language: system
pass_filenames: false
- repo: https://github.com/hadolint/hadolint
rev: v2.12.0
hooks:
- id: hadolint-docker
exclude: 'j2$'

View File

@ -1,27 +1,48 @@
FROM --platform=${TARGETPLATFORM:-linux/amd64} alpine as build
RUN apk update && apk add crystal shards yaml-dev openssl-dev zlib-dev libxml2-dev && apk cache clean
FROM --platform=${TARGETPLATFORM:-linux/amd64} alpine:3.20 AS build
RUN apk add --no-cache \
crystal \
shards \
yaml-dev \
openssl-dev \
zlib-dev \
libxml2-dev \
make
RUN rm -rf /var/cache/apk/*
RUN addgroup -S app && adduser app -S -G app
WORKDIR /home/app
COPY shard.yml ./
COPY shard.yml Makefile ./
RUN mkdir src/
COPY src/ src/
RUN shards install
RUN shards build -d --error-trace
RUN strip bin/*
COPY runtimes/ runtimes/
RUN make
# RUN strip bin/*
FROM --platform=${TARGETPLATFORM:-linux/amd64} alpine as ship
RUN apk update && apk add caddy nss-tools multirun docker openssl zlib yaml pcre2 gc libevent libgcc libxml2 ttyd && apk cache clean
FROM --platform=${TARGETPLATFORM:-linux/amd64} alpine:3.20 AS ship
RUN apk add --no-cache \
caddy \
nss-tools \
multirun \
docker \
openssl \
zlib \
yaml \
pcre2 \
gc \
libevent \
libgcc \
libxml2 \
ttyd
RUN rm -rf /var/cache/apk/*
# Unprivileged user
RUN addgroup -S app && adduser app -S -G app
WORKDIR /home/app
RUN mkdir /home/app/tmp && chown app /home/app/tmp
RUN mkdir runtimes public
COPY runtimes/ runtimes/
COPY public/ public/
COPY Caddyfile ./
COPY --from=build /home/app/bin/faaso-daemon /home/app/bin/faaso /usr/bin/
RUN mkdir /secrets
# Mount points for persistent data
RUN mkdir /secrets /config
CMD ["/usr/bin/multirun", "-v", "faaso-daemon", "caddy run --config Caddyfile"]
CMD ["/usr/bin/multirun", "-v", "faaso-daemon", "caddy run --config config/Caddyfile"]

View File

@ -1,9 +1,26 @@
build: shard.yml $(wildcard src/**/*cr)
shards build
proxy: build
build: shard.yml $(wildcard src/**/*) $(runtimes/**/*)
shards build -d --error-trace
cat .rucksack >> bin/faaso
cat .rucksack >> bin/faaso-daemon
proxy:
docker build . -t faaso-proxy
all: build proxy
start-proxy:
docker run --name faaso-proxy-one --rm --network=faaso-net --env-file=proxy.env -v /var/run/docker.sock:/var/run/docker.sock -v secrets:/home/app/secrets -p 8888:8888 faaso-proxy
docker network create faaso-net || true
docker run --name faaso-proxy-one \
--rm --network=faaso-net \
-e FAASO_SECRET_PATH=${PWD}/secrets \
-v /var/run/docker.sock:/var/run/docker.sock \
-v ${PWD}/secrets:/home/app/secrets \
-v ${PWD}/config:/home/app/config \
-p 8888:8888 faaso-proxy
test:
crystal spec
.PHONY: build proxy-image start-proxy
clean:
rm bin/*
.PHONY: all build proxy-image start-proxy test clean

View File

@ -50,12 +50,14 @@ This will give you:
You need a server, with docker. In that server, build it as explained above.
You can run the `faaso-proxy` with something like this:
```
docker run --network=faaso-net -v /var/run/docker.sock:/var/run/docker.sock -p 8888:8888 faaso-proxy
```shell
docker run --network=faaso-net \
-v /var/run/docker.sock:/var/run/docker.sock \
-p 8888:8888 faaso-proxy
```
That will give `faaso-proxy` access to your docker, and expose the functionality in
port 8888.
That will give `faaso-proxy` access to your docker, and expose the functionality
in port 8888.
## What it Does
@ -65,7 +67,8 @@ In FaaSO you (the user) can create Funkos. Funkos are the moral equivalent of AW
lambdas and whatever they are called in other systems. In short, they are simple
programs that handle web requests.
For example, here is a `hello world` level funko written using Crystal, a file called `funko.cr`:
For example, here is a `hello world` level funko written using Crystal,
a file called `funko.cr`:
```crystal
get "/" do
@ -82,7 +85,8 @@ runtime: crystal
```
If you have those two files in a folder, that folder is a funko, which is called
`hello` and FaaSO knows it's written in Crystal. In fact, it knows (because the crystal runtime explains that, don't worry about it yet) that it's part of an
`hello` and FaaSO knows it's written in Crystal. In fact, it knows (because the
crystal runtime explains that, don't worry about it yet) that it's part of an
application written in the [Kemal framework](https://kemalcr.com/) and it knows
how to create a whole container which runs the app, and how to check its health,
and so on.
@ -93,13 +97,14 @@ The full details of how to write funkos are still in flux, so not documenting
it for now. Eventually, you will be able to just write the parts you
need to write to create funkos in different languages. It's easy!
### So what can a funko do?
### So what can a funko do
Once you have a funko, you can *build* it, which will give you a docker image.
```faaso build --local myfunko/```
Or you can export it and get rid of all the mistery of how your funko **really** works:
Or you can export it and get rid of all the mistery of how your funko
**really** works:
```faaso export myfunko/ myfuko-exported```
@ -116,9 +121,9 @@ than one, although currently only one is used by the proxy.
The proxy has a few goals:
1) You can connect to it using `faaso` and have it build/run/etc your funkos.
* This builds the funko in your machine: `faaso build -l myfunko/`
* This builds the funko in the server pointed at by FAASO_SERVER: `faaso build myfunko/`
* This builds the funko in the server pointed at by FAASO_SERVER:
`faaso build myfunko/`
Yes, they are exactly the same thing. In fact, if you don't use the `-l` flag,
faaso just tells the proxy "hey proxy, run *your* copy of faaso over there and
@ -145,4 +150,4 @@ beyond bugfixes, since I am redesigning things all the time.
## Contributors
- [Roberto Alsina](https://github.com/ralsina) - creator and maintainer
* [Roberto Alsina](https://github.com/ralsina) - creator and maintainer

43
TODO.md Normal file
View File

@ -0,0 +1,43 @@
# TODO LIST
## Things that need doing before first release
* User flow for initial proxy setup
* ✅ Setting up password
* Setting up hostname for Caddy's automatic HTTPS
* Config UI in frontend?
* Polish frontend UI **A LOT**
* ✅ Version checks for consistency between client/server
* ✅ Have 3 runtimes:
* ✅ Crystal + Kemal
* ✅ Python + Flask
* ✅ Nodejs + Express
* Create a site
* Document
* FaaSO for app developers
* FaaSO for runtime developers
* FaaSO server setup
* APIs
* Sanitize all inputs
* ✅ Streaming responses in slow operations like scaling down
or building
* ✅ Make more things configurable / remove hardcoded stuff
* ✅ Make server take options from file
* ✅ Make server take options from environment
* ✅ Make server password configurable
* ✅ admin/admin auth client side
*`faaso login` is not working properly yet with proxy
* CD for binaries and images for at least arm64/x86
* Multi-container docker logs [faaso logs -f FUNKO]
* ✅ Configurable verbosity, support stderr/stdout split
* ✅ Fix proxy reload / Make it reload on file changes
* Implement `faaso help command`
* ✅ Fix `export examples/hello_crystal` it has a `template/`
* ✅ Implement zero-downtime rollout (`faaso deploy`)
* ✅ Cleanup `tmp/whatever` after use
*`faaso scale` remote is broken
* ✅ Setup linters/pre-commit/etc
## Things to do but not before release
* Propagate errors from `run_faaso` to the remote client

View File

@ -5,9 +5,11 @@
}
http://*:8888 {
basicauth /admin/* {
# admin $2a$14$C35905PxPzICAZKc/O9jYOS7ipZNPBtrndja8Yu3bvs/UujckryHS
admin {$HTTP_BASIC_AUTH_PASSWORD}
forward_auth /admin/* http://127.0.0.1:3000 {
uri /auth
copy_headers {
Authorization
}
}
handle_path /admin/terminal/* {
@ -16,4 +18,5 @@ http://*:8888 {
handle_path /admin/* {
reverse_proxy /* http://127.0.0.1:3000
}
import funkos
}

1
config/faaso.yml Normal file
View File

@ -0,0 +1 @@
password: adminfoo

0
config/funkos Normal file
View File

View File

@ -38,7 +38,7 @@ up/downscaling, no multiple versions routed by header.
Specifically: no downscaling to zero. It makes everything MUCH
more complicated.
# Function structure
## Function structure
Example using crystal, but it could be anything. Any function has
an associated runtime, for example "crystal" or "python".
@ -69,7 +69,9 @@ Probably some `metadata.yml` that is *not* in the template.
* Files that should be copied along the function
* Whatever
# Implementation Ideas
## Implementation Ideas
* caddy for proxy? It's simple, fast, API-configurable.
* Local docker registry for images? See https://www.docker.com/blog/how-to-use-your-own-registry-2/ (maybe later)
* Local docker registry for images? See
[use own registry](https://www.docker.com/blog/how-to-use-your-own-registry-2/)
(maybe later)

View File

@ -15,21 +15,24 @@ Solution: start the funko on the server. Done. It's implemented.
## Variant 3: Deploy to the server and it's already running
1. If it's already running and it's running the latest image, then nothing to be done.
2. It it's running and is not the latest, we can stop it and start with the latest image.
1. If it's already running and it's running the latest image, then nothing
to be done.
2. It it's running and is not the latest, we can stop it and start with the
latest image.
* Action 2 causes downtime. Usually it will not be significant, but it's there.
* In the future it may be important to have zero downtime.
* We need to figure out what is implied by doing "zero downtime" to see if
not doing it now would make it impossible.
For zero downtime, we want to have two instances running, switch the proxy to the new
one, then stop the old one.
For zero downtime, we want to have two instances running, switch the proxy
to the new one, then stop the old one.
Currently it's impossible to run two instances because the container name is
faaso-funkoname, and we can't have 2 of those.
So: we could instead have faaso-funkoname-1, faaso-funkoname-2, etc. with some sort of suffix
So: we could instead have faaso-funkoname-1, faaso-funkoname-2, etc.
with some sort of suffix
Changes implied in the faaso code:
@ -40,9 +43,3 @@ Changes implied in the faaso code:
* What happens if we have two instances with different images?
Answers coming up.

View File

@ -49,10 +49,12 @@ faaso-net.
The proxy is the only container exposed to the host network, everything
else needs to be accessed through it.
The faaso-proxy container will automatically proxy all requests so if you access the URL `http://faaso-proxy:8888/funko/hello/foo` that will be
The faaso-proxy container will automatically proxy all requests so if
you access the URL `http://faaso-proxy:8888/funko/hello/foo` that will be
proxied to `/foo` in the `hello` funko.
This is all done via naming conventions. You can create your own `faaso-whatever` container, add it to the `faaso-net` and faaso will
This is all done via naming conventions. You can create your own
`faaso-whatever` container, add it to the `faaso-net` and faaso will
happily consider it a funko.
In the same way all funkos are simply docker containers running in that
@ -78,4 +80,3 @@ faaso-proxy -- GET /bar --> faaso-funko1
The dynamic proxying is achieved by reading the current state of
Docker and just adapt to it using the naming conventions mentioned
above.

View File

@ -31,7 +31,7 @@ So, the proxy can periodically examine its secret store and populate a folder
If on starting a funko we always do a bind mount of `/secrets/foo` to `/secrets`
then it will always have its secrets in place.
## Problem 2: how can the proxy know the secrets without keeping them in the image?
## Problem 2: how can the proxy know the secrets without keeping them in the image
They can't be shipped via the image, so they need to be injected via the admin API.

View File

@ -0,0 +1,11 @@
# Readme for Hello_crystal
This is a funko using the Crystal runtime for [FaaSO](https://git.ralsina.me/ralsina/faaso)
## What is Hello_crystal
Write here what it is
## How to use Hello_crystal
And so on.

View File

@ -1,4 +1,4 @@
name: function
name: hello_crystal
version: 0.1.0
targets:
@ -12,4 +12,3 @@ dependencies:
# development_dependencies:
# webmock:
# github: manastech/webmock.cr

View File

@ -1 +0,0 @@
HTTP_BASIC_AUTH_PASSWORD=$2a$14$C35905PxPzICAZKc/O9jYOS7ipZNPBtrndja8Yu3bvs/UujckryHS

View File

@ -2,8 +2,8 @@
<head>
<meta charset="utf-8" />
<link rel="stylesheet" href="https://matcha.mizu.sh/matcha.css" />
<script src="https://unpkg.com/htmx.org@2.0.0"></script>
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/@picocss/pico@1/css/pico.min.css" />
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/@picocss/pico@2/css/pico.colors.min.css" /> <script src="https://unpkg.com/htmx.org@2.0.0"></script>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
</head>
@ -11,15 +11,20 @@
<header class="container">
<h1>FaaSO Admin Interface</h1>
</header>
<main class=container>
<h2>Your Funko Collection
<button id="update-funkos" style="float:right; display:inline;" hx-trigger="load, click, every 60s"
<article>
<nav>
<ul>
<li><strong style="font-size: 200%;">Your Funko Collection</strong></li>
</ul>
<ul>
<li><button id="update-funkos" style="float:right; display:inline;" hx-trigger="load, click, every 60s"
hx-get="funkos/?format=html" hx-target="#funko-list">
Refresh
</button>
</h2>
</ul>
</nav>
<span id="message"></span>
<table hx-target="#message">
<table hx-target="#message" class="striped">
<thead>
<tr>
<th>Name</th>
@ -31,23 +36,27 @@
</tbody>
</table>
<div id="terminal" style="resize: vertical; overflow: auto;"></div>
</main>
</article>
<script>
update_funkos = function () {
document.getElementById("update-funkos").click()
}
</script>
<main class=container>
<h2>
Your Secrets
<button id="update-secrets" style="float:right; display:inline;" hx-trigger="load, click, every 60s"
<article>
<nav>
<ul>
<li><strong style="font-size: 200%;">Your Secrets</strong>
</ul>
<ul>
<li><button id="update-secrets" style="float:right; display:inline;" hx-trigger="load, click, every 60s"
hx-get="secrets/?format=html" hx-target="#secret-list">
Refresh
</button>
<button style="float:right; display:inline;" onclick="show_new_secret()">
<li><button style="float:right; display:inline;" onclick="show_new_secret()">
Add
</button>
</h2>
</ul>
</nav>
<span id="message"></span>
<table hx-target="#message">
<thead>
@ -60,14 +69,20 @@
<tbody id="secret-list">
</tbody>
<dialog id="add-secret">
<topic>New Secret</topic>
<article>
<header>
New Secret
</header>
<form hx-post="secrets/">
<input placeholder="funko name" id="new-secret-funko" name="funko">
<input placeholder="secret name" id="new-secret-name" name="name">
<input placeholder="secret value" type="password" id="new-secret-password" name="value">
<button type="submit" hx-on:htmx:after-request="hide_new_secret()">CREATE</button>
<fieldset role="group" style="text-align: right;">
<button style="width:9em; display: inline;" type="submit" hx-on:htmx:after-request="hide_new_secret()">CREATE</button>
<button style="width:9em; display: inline;" onclick="hide_new_secret(); close();" aria-label="Close" rel="prev">CLOSE</button>
</fieldset>
</form>
<button onclick="hide_new_secret(); close();">CLOSE</button>
</article>
</dialog>
<script>
update_secrets = function() {
@ -87,5 +102,5 @@
update_secrets()
}
</script>
</main>
</article>
</body>

View File

@ -10,7 +10,7 @@ RUN shards build --release
RUN strip bin/*
FROM --platform=${TARGETPLATFORM:-linux/amd64} alpine as ship
RUN apk update && apk upgrade && apk add openssl pcre2 libgcc gc libevent curl {{ ship_packages | join " " }} && apk cache clean
RUN apk update && apk upgrade && apk add pcre2 libgcc gc libevent curl {{ ship_packages | join " " }} && apk cache clean
RUN addgroup -S app && adduser app -S -G app
WORKDIR /home/app

View File

@ -0,0 +1,4 @@
# README
This is the readme for people wanting to change this runtime,
not for people trying to use it

View File

@ -0,0 +1,11 @@
# Readme for {{name | title}}
This is a funko using the Crystal runtime for [FaaSO](https://git.ralsina.me/ralsina/faaso)
## What is {{name | title}}
Write here what it is
## How to use {{name | title}}
And so on.

View File

@ -0,0 +1,16 @@
require "kemal"
# This is a kemal app, you can add handlers, middleware, etc.
# A basic hello world get endpoint
get "/" do
"Hello World Crystal!"
end
# The `/ping/` endpoint is configured in the container as a healthcheck
# You can make it better by checking that your database is responding
# or whatever checks you think are important
#
get "/ping/" do
"OK"
end

View File

@ -0,0 +1,2 @@
name: {{ name }}
runtime: {{ runtime }}

View File

@ -0,0 +1,14 @@
name: {{ name }}
version: 0.1.0
targets:
funko:
main: main.cr
dependencies:
kemal:
github: kemalcr/kemal
# development_dependencies:
# webmock:
# github: manastech/webmock.cr

View File

@ -0,0 +1,20 @@
FROM --platform=${TARGETPLATFORM:-linux/amd64} alpine as build
RUN apk update && apk upgrade && apk add nodejs npm {{ ship_packages | join(" ") }} {{ devel_packages | join(" ") }} && apk cache clean
WORKDIR /home/app
COPY ./ ./
RUN npm i
FROM --platform=${TARGETPLATFORM:-linux/amd64} alpine as ship
RUN apk update && apk upgrade && apk add nodejs curl {{ ship_packages | join " " }} && apk cache clean
RUN addgroup -S app && adduser app -S -G app
WORKDIR /home/app
USER app
COPY --from=build /home/app/ .
CMD ["node", "funko.js"]
HEALTHCHECK {{ healthcheck_options }} CMD {{ healthcheck_command }}

View File

@ -0,0 +1,15 @@
const express = require('express')
const app = express()
const port = 3000
app.get('/', (req, res) => {
res.send('Hello World!')
})
app.get('/ping', (req, res) => {
res.send('OK')
})
app.listen(port, () => {
console.log(`Example funko listening on port ${port}`)
})

View File

@ -0,0 +1,2 @@
name: {{ name }}
runtime: {{ runtime }}

View File

@ -0,0 +1,11 @@
{
"name": "{{name}}",
"version": "1.0.0",
"main": "funko.js",
"author": "",
"license": "MIT",
"description": "Example Funko",
"dependencies": {
"express": "^4.19.2"
}
}

View File

@ -0,0 +1,22 @@
FROM --platform=${TARGETPLATFORM:-linux/amd64} alpine as build
RUN apk update && apk upgrade && apk add python3 gcc musl-dev linux-headers python3-dev {{ ship_packages | join(" ") }} {{ devel_packages | join(" ") }} && apk cache clean
WORKDIR /home/app
COPY requirements.txt *.py ./
RUN python3 -m venv venv
RUN venv/bin/pip install uwsgi
RUN venv/bin/pip install -r requirements.txt
FROM --platform=${TARGETPLATFORM:-linux/amd64} alpine as ship
RUN apk update && apk upgrade && apk add python3 uwsgi curl {{ ship_packages | join " " }} && apk cache clean
RUN addgroup -S app && adduser app -S -G app
WORKDIR /home/app
USER app
COPY --from=build /home/app/ .
CMD ["venv/bin/uwsgi", "--http", "0.0.0.0:3000", "--master", "-p", "1", "-w", "funko:app"]
HEALTHCHECK {{ healthcheck_options }} CMD {{ healthcheck_command }}

6
runtimes/flask/main.py Normal file
View File

@ -0,0 +1,6 @@
from flask import Flask
app = Flask({{name}})
if __name__ == '__main__':
serve(app, host='0.0.0.0', port=5000)

View File

@ -0,0 +1,11 @@
from flask import Flask
app = Flask("{{name}}")
@app.route('/')
def handle(req):
return "Hello World from Flask!"
@app.route('/ping')
def handle(req):
return "OK"

View File

@ -0,0 +1,2 @@
name: {{ name }}
runtime: {{ runtime }}

View File

@ -0,0 +1 @@
flask

View File

@ -4,6 +4,14 @@ shards:
git: https://github.com/sija/backtracer.cr.git
version: 1.2.2
base58:
git: https://github.com/crystal-china/base58.cr.git
version: 0.1.0+git.commit.d1150d4a6f086013a475640ad00e561a2fe1082a
cr-config:
git: https://github.com/crystal-community/cr-config.git
version: 5.1.0+git.commit.5eae3dfbf97da7dfa7c6e64a2a508069948518d3
crest:
git: https://github.com/mamantoha/crest.git
version: 1.3.13
@ -22,7 +30,7 @@ shards:
docr:
git: https://github.com/ralsina/docr.git
version: 0.1.1+git.commit.18f15cc7111b1d0c63347c7cca07aee9ec87a7a8
version: 0.1.1+git.commit.98a20178d5ae1391f1cd56e372530de6aa2b1ebc
exception_page:
git: https://github.com/crystal-loot/exception_page.git
@ -40,6 +48,14 @@ shards:
git: https://github.com/kemalcr/kemal.git
version: 1.5.0
kemal-basic-auth:
git: https://github.com/kemalcr/kemal-basic-auth.git
version: 1.0.0
oplog:
git: https://github.com/ralsina/oplog.git
version: 0.1.0+git.commit.70e3a7bbc2f1f4d75cf4e142244b263ee2844ba1
radix:
git: https://github.com/luislavena/radix.git
version: 0.4.1
@ -47,4 +63,3 @@ shards:
rucksack:
git: https://github.com/busyloop/rucksack.git
version: 2.0.0

View File

@ -15,10 +15,14 @@ crystal: ">= 1.12.2"
license: MIT
dependencies:
base58:
github: crystal-china/base58.cr
crest:
github: mamantoha/crest
crinja:
github: straight-shoota/crinja
cr-config:
github: crystal-community/cr-config
crystar:
github: naqvis/crystar
docopt:
@ -28,6 +32,10 @@ dependencies:
branch: add_exposed_ports
kemal:
github: kemalcr/kemal
kemal-basic-auth:
github: kemalcr/kemal-basic-auth
oplog:
github: ralsina/oplog
rucksack:
github: busyloop/rucksack

View File

@ -1,67 +1,71 @@
require "base58"
module Faaso
module Commands
# Build images for one or more funkos from source
struct Build
def run(options, folders : Array(String))
def run(options, folders : Array(String)) : Int32
funkos = Funko::Funko.from_paths(folders)
# Create temporary build location
if options["--local"]
funkos.each do |funko|
# Create temporary build location
tmp_dir = Path.new("tmp", UUID.random.to_s)
Dir.mkdir_p(tmp_dir) unless File.exists? tmp_dir
funko.prepare_build tmp_dir
funkos.each do |funko|
tmp_dir = Path.new("tmp", Random.base58(8))
Dir.mkdir_p(tmp_dir) unless File.exists? tmp_dir
funko.runtime = nil if options["--no-runtime"]
funko.prepare_build(path: tmp_dir)
if options["--local"]
Log.info { "Building function... #{funko.name} in #{tmp_dir}" }
funko.build tmp_dir
FileUtils.rm_rf(tmp_dir)
next
end
else # Running against a server
funkos.each do |funko|
# Create a tarball for the funko
buf = IO::Memory.new
Compress::Gzip::Writer.open(buf) do |gzip|
Crystar::Writer.open(gzip) do |tw|
Dir.glob("#{funko.path}/**/*").each do |path|
next unless File.file? path
rel_path = Path[path].relative_to funko.path
file_info = File.info(path)
hdr = Crystar::Header.new(
name: rel_path.to_s,
mode: file_info.permissions.to_u32,
size: file_info.size,
)
tw.write_header(hdr)
tw.write(File.read(path).to_slice)
end
Faaso.check_version
# Create a tarball for the funko
buf = IO::Memory.new
Compress::Gzip::Writer.open(buf) do |gzip|
Crystar::Writer.open(gzip) do |tw|
Log.debug { "Adding files to tarball" }
Dir.glob("#{tmp_dir}/**/*").each do |path|
next unless File.file? path
rel_path = Path[path].relative_to tmp_dir
Log.debug { "Adding #{rel_path}" }
file_info = File.info(path)
hdr = Crystar::Header.new(
name: rel_path.to_s,
mode: file_info.permissions.to_u32,
size: file_info.size,
)
tw.write_header(hdr)
tw.write(File.read(path).to_slice)
end
end
tmp = File.tempname
File.open(tmp, "w") do |outf|
outf << buf
end
url = "#{FAASO_SERVER}funkos/build/"
begin
Log.info { "Uploading funko to #{FAASO_SERVER}" }
response = Crest.post(
url,
{"funko.tgz" => File.open(tmp), "name" => "funko.tgz"},
user: "admin", password: "admin"
)
Log.info { "Build finished successfully." }
body = JSON.parse(response.body)
Log.info { body["stdout"] }
rescue ex : Crest::InternalServerError
Log.error { "Error building funko #{funko.name} from #{funko.path}" }
body = JSON.parse(ex.response.body)
Log.info { body["stdout"] }
Log.error { body["stderr"] }
exit 1
end
end
FileUtils.rm_rf(tmp_dir)
tmp = File.tempname
File.open(tmp, "w") do |outf|
outf << buf
end
url = "#{Config.server}funkos/build/"
user, password = Config.auth
Log.info { "Uploading funko to #{Config.server}" }
Log.info { "Starting remote build:" }
Crest.post(
url,
{"funko.tgz" => File.open(tmp), "name" => "funko.tgz"},
user: user, password: password
) do |response|
IO.copy(response.body_io, STDOUT)
end
Log.info { "Build finished successfully." }
rescue ex : Crest::InternalServerError
Log.error(exception: ex) { "Error building funko #{funko.name} from #{funko.path}" }
return 1
end
0
end
end
end

59
src/commands/deploy.cr Normal file
View File

@ -0,0 +1,59 @@
module Faaso
module Commands
struct Deploy
def local(options, funko_name : String) : Int32
funko = Funko::Funko.from_names([funko_name])[0]
# Get scale, check for out-of-date containers
current_scale = funko.scale
latest_image = funko.latest_image
containers = funko.containers
out_of_date = containers.count { |container| container.image_id != latest_image }
Log.info { "Need to update #{out_of_date} containers" }
Log.info { "Scaling from #{current_scale} to #{current_scale + out_of_date}" }
# Increase scale to get enough up-to-date containers
new_containers = funko.scale(current_scale + out_of_date)
# Wait for them to be healthy
begin
funko.wait_for(current_scale + out_of_date, 120, healthy: true)
rescue ex : Exception
# Failed to start, rollback
Log.error(exception: ex) { "Failed to scale, rolling back" }
docker_api = Docr::API.new(Docr::Client.new)
new_containers.each do |container_id|
docker_api.containers.stop(container_id)
docker_api.containers.delete(container_id)
end
return 1
end
Log.info { "Scaling down to #{current_scale}" }
# Decrease scale to the desired amount
funko.scale(current_scale)
funko.wait_for(current_scale, 30)
Log.info { "Deployed #{funko_name}" }
0
end
def remote(options, funko_name : String) : Int32
user, password = Config.auth
Faaso.check_version
Crest.get(
"#{Config.server}funkos/#{funko_name}/deploy/", \
user: user, password: password) do |response|
IO.copy(response.body_io, STDOUT)
end
0
end
def run(options, funko_name : String) : Int32
Log.info { "Deploying #{funko_name}" }
if options["--local"]
local(options, funko_name)
else
remote(options, funko_name)
end
end
end
end
end

View File

@ -1,7 +1,7 @@
module Faaso
module Commands
struct Export
def run(options, source : String, destination : String)
def run(options, source : String, destination : String) : Int32
funko = Funko::Funko.from_paths([source])[0]
# Create temporary build location
dst_path = destination
@ -12,6 +12,7 @@ module Faaso
Log.info { "Exporting #{funko.path} to #{dst_path}" }
Dir.mkdir_p(dst_path)
funko.prepare_build Path[dst_path]
0
end
end
end

47
src/commands/login.cr Normal file
View File

@ -0,0 +1,47 @@
module Faaso
module Commands
struct Login
def run(options) : Int32
server = Config.server
Log.info { "Enter password for #{server}" }
if STDIN.tty?
password = (STDIN.noecho &.gets.try &.chomp).to_s
else
password = STDIN.gets.to_s
end
# This is tricky. If the service is running behind a reverse proxy
# then /version is locked, but if it's not, only /auth is locked.
# So we try /version first without a password, and if it succeeds
# we try /auth with the password. If /version fails, we try /version
# with the password
#
begin
# Version without password.
Crest.get("#{server}version/")
# Auth with password
begin
Crest.get("#{server}auth/", user: "admin", password: password)
rescue ex : Crest::Unauthorized
# Failed with auth/
Log.error { "Wrong password" }
return 1
end
rescue ex : Crest::Unauthorized
# Version with password
Crest.get("#{server}version/", user: "admin", password: password)
end
# If we got here the password is ok
CONFIG.hosts[server] = {"admin", password}
Config.save
0
rescue ex : Crest::Unauthorized
Log.error { "Wrong password" }
1
rescue ex : Socket::ConnectError
Log.error { "Connection refused" }
1
end
end
end
end

View File

@ -1,26 +1,35 @@
require "rucksack"
require "../runtime.cr"
module Faaso
module Commands
# Creates a new empty funko out of a given runtime
struct New
@@known : Array(String) = {{`find ./runtimes -type d -mindepth 1`.split('\n').reject(&.empty?)}}
def run(options, folder)
if options["-r"].as(String) == "list"
Log.info { "Crystal has some included runtimes:\n" }
@@known.each do |runtime|
Log.info { " * #{Path[runtime].basename}" }
end
Log.info { "\nOr if you have your own, use a folder name" }
def run(options, folder) : Int32
runtime = options["-r"].as(String)
# Give a list of known runtimes
if runtime == "list"
Runtime.list
return 0
end
# Create new folder
if Dir.exists? folder
Log.error { "Folder #{folder} already exists" }
return 1
end
# Get runtime template files list
template_base, template_files = Runtime.template_files(runtime)
Runtime.copy_templated(
template_base,
template_files,
folder,
{"name" => Path[folder].basename,
"runtime" => runtime}
)
0
end
end
end
end
# Embed runtimes in the binary using rucksack
{% for name in `find ./runtimes -type f`.split('\n') %}
rucksack({{name}})
{% end %}

View File

@ -10,41 +10,46 @@ module Faaso
# In both cases stopped instances after the required
# scale is reached are deleted.
struct Scale
def local(options, name, scale)
def local(options, name : String, scale : Int | Nil) : Int32
funko = Funko::Funko.from_names([name])[0]
# Asked about scale
if !scale
if funko.image_history.empty?
Log.error { "Unknown funko #{funko.name}" }
return 1
end
if scale.nil?
Log.info { "Funko #{name} has a scale of #{funko.scale}" }
return 0
end
# Asked to set scale
if funko.image_history.empty?
Log.error { "Error: no images available for #{funko.name}:latest" }
exit 1
end
funko.scale(scale.as(String).to_i)
funko.scale(scale)
0
end
def remote(options, name, scale)
if !scale
response = Crest.get(
"#{FAASO_SERVER}funkos/#{name}/scale/", \
user: "admin", password: "admin")
else
response = Crest.post(
"#{FAASO_SERVER}funkos/#{name}/scale/",
{"scale" => scale}, user: "admin", password: "admin")
def remote(options, name : String, scale : Int | Nil) : Int32
user, password = Config.auth
Faaso.check_version
if scale.nil?
Crest.get(
"#{Config.server}funkos/#{name}/scale/", \
user: user, password: password) do |response|
IO.copy(response.body_io, STDOUT)
end
return 0
end
body = JSON.parse(response.body)
Log.info { body["output"] }
Crest.post(
"#{Config.server}funkos/#{name}/scale/",
{"scale" => scale}, user: user, password: password) do |response|
IO.copy(response.body_io, STDOUT)
end
0
rescue ex : Crest::InternalServerError
Log.error { "Error scaling funko #{name}" }
body = JSON.parse(ex.response.body)
Log.info { body["output"] }
exit 1
Log.error(exception: ex) { "Error scaling funko #{name}" }
1
end
def run(options, name, scale)
def run(options, name : String, scale) : Int32
scale = scale.try &.to_s.to_i
if options["--local"]
return local(options, name, scale)
end

View File

@ -1,7 +1,7 @@
module Faaso
module Commands
struct Secret
def local(options, funko, name, secret)
def local(options, funko, name, secret) : Int32
if options["--add"]
dst_dir = "secrets/#{funko}"
Dir.mkdir_p(dst_dir) unless Dir.exists?(dst_dir)
@ -9,29 +9,33 @@ module Faaso
elsif options["--delete"]
File.delete("secrets/#{funko}/#{name}")
end
0
end
def remote(options, funko, name, secret)
def remote(options, funko, name, secret) : Int32
Faaso.check_version
user, password = Config.auth
if options["--add"]
Crest.post(
"#{FAASO_SERVER}secrets/",
"#{Config.server}secrets/",
{
"funko" => funko,
"name" => name,
"value" => secret,
}, user: "admin", password: "admin")
}, user: user, password: password)
Log.info { "Secret created" }
elsif options["--delete"]
Crest.delete(
"#{FAASO_SERVER}secrets/#{funko}/#{name}",
user: "admin", password: "admin")
"#{Config.server}secrets/#{funko}/#{name}",
user: user, password: password)
end
0
rescue ex : Crest::RequestFailed
Log.error { "Error #{ex.response.status_code}" }
exit 1
1
end
def run(options, funko, name)
def run(options, funko, name) : Int32
if options["--add"]
Log.info { "Enter the secret, end with Ctrl-D" } if STDIN.tty?
secret = STDIN.gets_to_end

View File

@ -1,10 +1,15 @@
module Faaso
module Commands
struct Status
def local(options, name)
def local(options, name) : Int32
funko = Funko::Funko.from_names([name])[0]
status = funko.docker_status
if status.images.size == 0
Log.error { "Unkown funko: #{name}" }
return 1
end
Log.info { "Name: #{status.@name}" }
Log.info { "Scale: #{status.scale}" }
@ -17,22 +22,24 @@ module Faaso
status.images.each do |image|
Log.info { " #{image.repo_tags} #{Time.unix(image.created)}" }
end
0
end
def remote(options, name)
response = Crest.get(
"#{FAASO_SERVER}funkos/#{name}/status/", \
user: "admin", password: "admin")
body = JSON.parse(response.body)
Log.info { body["output"] }
def remote(options, name) : Int32
Faaso.check_version
user, password = Config.auth
Crest.get(
"#{Config.server}funkos/#{name}/status/", \
user: user, password: password) do |response|
IO.copy(response.body_io, STDOUT)
end
0
rescue ex : Crest::InternalServerError
Log.error { "Error scaling funko #{name}" }
body = JSON.parse(ex.response.body)
Log.info { body["output"] }
exit 1
Log.error(exception: ex) { "Error scaling funko #{name}" }
1
end
def run(options, name)
def run(options, name) : Int32
if options["--local"]
return local(options, name)
end

44
src/config.cr Normal file
View File

@ -0,0 +1,44 @@
require "yaml"
CONFIG = Config.load
class Config
include YAML::Serializable
property hosts : Hash(String, {String, String}) = Hash(String, {String, String}).new
def initialize
@hosts = {} of String => {String, String}
end
def self.load : Config
if File.file? ".faaso.yml"
return Config.from_yaml(File.read(".faaso.yml"))
end
Config.new
end
def self.save
File.open(".faaso.yml", "w") do |outf|
outf << CONFIG.to_yaml
end
end
@@already_warned = false
def self.server : String
@@already_warned = true
url = ENV.fetch("FAASO_SERVER", nil)
if url.nil?
Log.warn { "FAASO_SERVER not set" } unless @@already_warned
url = "http://localhost:3000/"
end
url += "/" unless url.ends_with? "/"
Log.info { "Using server #{url}" } unless @@already_warned
url
end
def self.auth : {String, String}
CONFIG.hosts.fetch(server, {"admin", ""})
end
end

47
src/daemon/config.cr Normal file
View File

@ -0,0 +1,47 @@
require "cr-config"
require "kemal-basic-auth"
class Config
include CrConfig
option password : String, default: "admin"
def self.load
builder = Config.new_builder
builder.providers do
[
CrConfig::Providers::SimpleFileProvider.new("config/faaso.yml"),
CrConfig::Providers::EnvVarProvider.new,
]
end
config = builder.build
Config.set_instance config
end
end
class ConfigAuthHandler < Kemal::BasicAuth::Handler
only ["/auth", "/auth/*"]
def call(context)
return call_next(context) unless only_match?(context)
super
end
def initialize
# Ignored, just make the compiler happy
@credentials = Kemal::BasicAuth::Credentials.new({"foo" => "bar"})
end
def authorize?(value) : String?
username, password = Base64.decode_string(value[BASIC.size + 1..-1]).split(":")
if username == "admin" && password == Config.instance.password
username
else
nil
end
end
end
# Tie auth to config
add_handler ConfigAuthHandler.new

View File

@ -1,3 +1,4 @@
require "base58"
require "docr"
require "kemal"
require "../funko.cr"
@ -8,39 +9,25 @@ module Funko
# Get the funko's status
get "/funkos/:name/status/" do |env|
name = env.params.url["name"]
response = run_faaso(["status", name])
if response["exit_code"] != 0
halt env, status_code: 500, response: response.to_json
else
response.to_json
end
run_faaso(["status", name], env)
end
# Get the funko's scale
get "/funkos/:name/scale/" do |env|
name = env.params.url["name"]
response = run_faaso(["scale", name])
if response["exit_code"] != 0
halt env, status_code: 500, response: response.to_json
else
response.to_json
end
run_faaso(["scale", name], env)
end
# Set the funko's scale
post "/funkos/:name/scale/" do |env|
name = env.params.url["name"]
scale = env.params.body["scale"].as(String)
response = run_faaso(["scale", name, scale])
if response["exit_code"] != 0
Log.error { response }
halt env, status_code: 500, response: response.to_json
else
Log.info { response }
response.to_json
end
run_faaso(["scale", name, scale], env)
end
get "/funkos/:name/deploy" do |env|
name = env.params.url["name"]
run_faaso(["deploy", name], env)
end
# Build image for funko received as "funko.tgz"
@ -48,7 +35,7 @@ module Funko
# mosquito-cr/mosquito to make it a job queue
post "/funkos/build/" do |env|
# Create place to build funko
tmp_dir = Path.new("tmp", UUID.random.to_s)
tmp_dir = Path.new("tmp", Random.base58(8))
Dir.mkdir_p(tmp_dir) unless File.exists? tmp_dir
# Expand tarball in there
@ -56,23 +43,20 @@ module Funko
Compress::Gzip::Reader.open(file) do |gzip|
Crystar::Reader.open(gzip) do |tar|
tar.each_entry do |entry|
File.open(Path.new(tmp_dir, entry.name), "w") do |dst|
IO.copy entry.io, dst
dst = Path.new(tmp_dir, entry.name)
Dir.mkdir_p dst.dirname
File.open(Path.new(tmp_dir, entry.name), "w") do |outf|
IO.copy entry.io, outf
end
end
end
end
# Build the thing
response = run_faaso(["build", tmp_dir.to_s])
if response["exit_code"] != 0
halt env, status_code: 500, response: response.to_json
else
response.to_json
end
run_faaso(["build", tmp_dir.to_s, "--no-runtime"], env)
ensure
FileUtils.rm_rf(tmp_dir) unless tmp_dir.nil?
end
# Endpoints for the web frontend
# General status for the front page
@ -124,6 +108,16 @@ module Funko
funko.wait_for(1, 1)
end
# Delete => scale to 0, remove all containers and images
delete "/funkos/:name/" do |env|
name = env.params.url["name"]
funko = Funko.from_names([name])[0]
funko.scale(0)
funko.wait_for(0, 1)
funko.remove_all_containers
funko.remove_all_images
end
# Return an iframe that shows the container's logs
get "/funkos/terminal/logs/:instance/" do |env|
instance = env.params.url["instance"]
@ -138,20 +132,24 @@ module Funko
"<iframe src='terminal/' width='100%' height='100%'></iframe>"
end
# Helper to run faaso locally and get a response back
def run_faaso(args : Array(String))
Log.info { "Running faaso [#{args.join(", ")}, -l]" }
output = IO::Memory.new
status = Process.run(
# Helper to run faaso locally and respond via env
def run_faaso(args : Array(String), env)
args << "-l" # Always local in the server
Log.info { "Running faaso [#{args}" }
Process.run(
command: "faaso",
args: args + ["-l"], # Always local in the server
output: output,
error: output,
)
result = {
"exit_code" => status.exit_code,
"output" => output.to_s,
}
result
args: args,
env: {"FAASO_SERVER_SIDE" => "true"},
) do |process|
loop do
data = process.output.gets(chomp: false)
env.response.print data
env.response.flush
Fiber.yield # Without this the process never ends
break if process.terminated?
end
end
# FIXME: find a way to raise an exception on failure
# of the faaso process
end
end

View File

@ -1,3 +1,4 @@
require "./config.cr"
require "./funko.cr"
require "./proxy.cr"
require "./secrets.cr"
@ -8,8 +9,27 @@ require "docr"
require "kemal"
require "uuid"
Config.load
macro version
"{{ `grep version shard.yml | cut -d: -f2` }}".strip()
end
get "/" do |env|
env.redirect "/index.html"
end
get "/version" do
"#{version}"
end
get "/auth" do
end
get "/reload" do
Log.info { "Reloading configuration" }
Config.load
"Config reloaded"
end
Kemal.run

View File

@ -1,9 +1,11 @@
require "docr"
require "./funko.cr"
require "docr"
require "kemal"
module Proxy
@@current_config = File.read("Caddyfile")
CADDY_CONFIG_PATH = "config/Caddyfile"
CADDY_CONFIG_FUNKOS = "config/funkos"
@@current_config = File.read(CADDY_CONFIG_FUNKOS)
# Get current proxy config
get "/proxy/" do
@ -18,31 +20,11 @@ module Proxy
update_proxy_config
end
def self.update_proxy_config
def self.update_proxy_config : Nil
docker_api = Docr::API.new(Docr::Client.new)
containers = docker_api.containers.list(all: true)
config = <<-CONFIG
{
http_port 8888
https_port 8887
local_certs
}
http://*:8888 {
basicauth /admin/* {
admin {$HTTP_BASIC_AUTH_PASSWORD}
}
handle_path /admin/terminal/* {
reverse_proxy /* http://127.0.0.1:7681
}
handle_path /admin/* {
reverse_proxy /* http://127.0.0.1:3000
}
CONFIG
config = ""
funkos = Funko::Funko.from_docker
funkos.each do |funko|
next if funko.name == "proxy"
@ -60,22 +42,20 @@ CONFIG
}
)
end
config += "\n}"
if @@current_config != config
Log.info { "Updating proxy config" }
File.open("Caddyfile", "w") do |file|
File.open(CADDY_CONFIG_FUNKOS, "w") do |file|
file << config
end
# Reload config
Process.run(command: "caddy", args: ["reload", "--config", "Caddyfile"])
@@current_config = config
Process.run(command: "caddy", args: ["reload", "--config", CADDY_CONFIG_PATH])
end
config
end
end
# Update proxy config once a second
# Update proxy config every 1 second (if changed)
spawn do
loop do
Proxy.update_proxy_config

View File

@ -4,7 +4,7 @@ module Terminal
@@terminal_process : Process | Nil = nil
def start_terminal(_args = ["sh"], readonly = true)
args = ["-p", "7681", "-c", "admin:admin", "-o"]
args = ["-p", "7681", "-o"]
args += ["-W"] unless readonly
args += _args
# We have a process there, kill it

View File

@ -1,5 +1,7 @@
require "./commands/build.cr"
require "./commands/deploy.cr"
require "./commands/export.cr"
require "./commands/login.cr"
require "./commands/new.cr"
require "./commands/scale.cr"
require "./commands/secret.cr"
@ -11,9 +13,6 @@ require "docr/utils.cr"
require "json"
require "uuid"
# API if you just ran faaso-daemon
FAASO_SERVER = ENV.fetch("FAASO_SERVER", "http://localhost:3000/")
# Functions as a Service, Ops!
module Faaso
VERSION = "0.1.0"
@ -28,9 +27,19 @@ module Faaso
))
rescue ex : Docr::Errors::DockerAPIError
raise ex if ex.status_code != 409 # Network already exists
end
module Commands
# Compare version with server's
def self.check_version
user, password = Config.auth
server_version = Crest.get(
"#{Config.server}version/", \
user: user, password: password).body
local_version = "#{version}"
if server_version != local_version
Log.warn { "Server is version #{server_version} and client is #{local_version}" }
end
end
end

View File

@ -1,3 +1,4 @@
require "./runtime.cr"
require "crinja"
require "file_utils"
require "yaml"
@ -80,34 +81,37 @@ module Funko
# Get the number of running instances of this funko
def scale
docker_api = Docr::API.new(Docr::Client.new)
docker_api.containers.list.select { |container|
container.@state == "running"
}.count { |container|
container.@names.any?(&.starts_with?("/faaso-#{name}-"))
}
containers.size
end
# Set the number of running instances of this funko
def scale(new_scale : Int)
# Returns the list of IDs started or stopped
def scale(new_scale : Int) : Array(String)
docker_api = Docr::API.new(Docr::Client.new)
current_scale = self.scale
return if current_scale == new_scale
result = [] of String
if current_scale == new_scale
Log.info { "Funko #{name} already at scale #{new_scale}" }
return result
end
Log.info { "Scaling #{name} from #{current_scale} to #{new_scale}" }
if new_scale > current_scale
(current_scale...new_scale).each {
Log.info { "Adding instance" }
id = create_container
result << (id = create_container)
start(id)
sleep 0.1.seconds
}
else
containers.select { |container| container.@state == "running" }.sort! { |i, j|
# Sort them older to newer, so we stop the oldest
containers.sort! { |i, j|
i.@created <=> j.@created
}.each { |container|
Log.info { "Removing instance" }
docker_api.containers.stop(container.@id)
result << container.@id
current_scale -= 1
break if current_scale == new_scale
sleep 0.1.seconds
@ -119,6 +123,8 @@ module Funko
Log.info { "Pruning dead instance" }
docker_api.containers.delete(container.@id)
}
result
end
# Setup the target directory `path` with all the files needed
@ -126,23 +132,18 @@ module Funko
def prepare_build(path : Path)
# Copy runtime if requested
if !runtime.nil?
runtime_dir = Path.new("runtimes", runtime.as(String))
raise Exception.new("Error: runtime #{runtime} not found for funko #{name} in #{path}") unless File.exists?(runtime_dir)
Dir.glob("#{runtime_dir}/*").each { |src|
FileUtils.cp_r(src, path)
}
# Replace templates with processed files
context = _to_context
Dir.glob("#{path}/**/*.j2").each { |template|
dst = template[..-4]
File.open(dst, "w") do |file|
file << Crinja.render(File.read(template), context)
end
File.delete template
}
# Get runtime files list
runtime_base, runtime_files = Runtime.runtime_files(runtime.as(String))
Runtime.copy_templated(
runtime_base,
runtime_files,
path.to_s,
_to_context
)
end
# Copy funko
# Copy funko on top of runtime
raise Exception.new("Internal error: empty funko path for #{name}") if self.path.empty?
Dir.glob("#{self.path}/*").each { |src|
FileUtils.cp_r(src, path)
@ -151,10 +152,14 @@ module Funko
# Build image using docker in path previously prepared using `prepare_build`
def build(path : Path)
Log.info { "Building image for #{name} in #{path}" }
docker_api = Docr::API.new(Docr::Client.new)
tags = ["faaso-#{name}:latest"]
Log.info { " Tags: #{tags}" }
docker_api.images.build(
context: path.to_s,
tags: ["faaso-#{name}:latest"]) { |x| Log.info { x } }
tags: tags,
no_cache: true) { |x| Log.info { x } }
end
def images
@ -188,7 +193,7 @@ module Funko
docker_api.containers.list(all: true).select { |container|
container.@names.any?(&.starts_with?("/faaso-#{name}-")) &&
container.@state == "running"
}
} || [] of Docr::Types::ContainerSummary
end
# A comprehensive status for the funko:
@ -211,12 +216,36 @@ module Funko
end
end
# Wait up to `t` seconds for the funko to reach the requested `state`
def wait_for(new_scale : Int, t)
# Wait up to `t` seconds for the funko to reach the desired scale
# If `healthy` is true, it will wait for the container to be declared
# healthy by the healthcheck
def wait_for(new_scale : Int, t : Int, healthy : Bool = false)
docker_api = Docr::API.new(Docr::Client.new)
channel = Channel(Nil).new
spawn do
loop do
channel.send(nil) if scale == new_scale
if healthy
channel.send(nil) if containers.select { |container|
begin
details = docker_api.containers.inspect(container.@id)
if details.nil?
false
elsif details.state.nil?
false
elsif details.state.as(Docr::Types::ContainerState).health.nil?
false
elsif details.state.as(Docr::Types::ContainerState).health.as(Docr::Types::Health).status == "healthy"
true
end
false
rescue ex : Docr::Errors::DockerAPIError
Log.error { "#{ex}" } unless ex.status_code == 304 # This just happens
false
end
} == new_scale
else
channel.send(nil) if scale == new_scale
end
sleep 0.2.seconds
end
end
@ -225,13 +254,47 @@ module Funko
when channel.receive
Log.info { "Funko #{name} reached scale #{new_scale}" }
when timeout(t.seconds)
Log.error { "Funko #{name} did not reach scale #{new_scale} in #{t} seconds" }
raise Exception.new("Funko #{name} did not reach scale #{new_scale} in #{t} seconds")
end
end
# Remove all containers related to this funko
def remove_all_containers
docker_api = Docr::API.new(Docr::Client.new)
docker_api.containers.list(all: true).select { |container|
container.@names.any?(&.starts_with?("/faaso-#{name}-"))
}.each { |container|
begin
docker_api.containers.stop(container.@id) if container.status != "exited"
rescue ex : Docr::Errors::DockerAPIError
Log.error { "#{ex}" } unless ex.status_code == 304 # This just happens
end
docker_api.containers.delete(container.@id)
}
end
# Remove all images related to this funko
def remove_all_images
docker_api = Docr::API.new(Docr::Client.new)
docker_api.images.list.select { |image|
return false if image.@repo_tags.nil?
true if image.@repo_tags.as(Array(String)).any?(&.starts_with?("faaso-#{name}:"))
}.each { |image|
docker_api.images.delete(image.@id)
}
end
# Create a container for this funko
def create_container(autostart : Bool = true) : String
secrets_mount = "#{Dir.current}/secrets/#{name}"
# The path to secrets is tricky. On the server it will be in
# ./secrets/ BUT when you call on the Docker API you need to
# pass the path in the HOST SYSTEM WHERE DOCKER IS RUNNING
# so allow for a FAASO_SECRET_PATH override which will
# be set for the proxy container
secrets_mount = ENV.fetch(
"FAASO_SECRET_PATH",
"#{Dir.current}/secrets/#{name}"
)
Dir.mkdir_p(secrets_mount)
conf = Docr::Types::CreateContainerConfig.new(
image: "faaso-#{name}:latest",
@ -250,7 +313,7 @@ module Funko
)
docker_api = Docr::API.new(Docr::Client.new)
response = docker_api.containers.create(name: "faaso-#{name}-#{randstr}", config: conf)
response = docker_api.containers.create(name: "faaso-#{name}-#{Random.base58(6)}", config: conf)
response.@warnings.each { |msg| Log.warn { msg } }
docker_api.containers.start(response.@id) if autostart
response.@id
@ -290,8 +353,3 @@ module Funko
end
end
end
def randstr(length = 6) : String
chars = "abcdefghijklmnopqrstuvwxyz0123456789"
String.new(Bytes.new(chars.to_slice.sample(length).to_unsafe, length))
end

View File

@ -1,83 +1,62 @@
require "./config.cr"
require "./faaso.cr"
require "colorize"
require "docopt"
require "oplog"
require "rucksack"
# Log formatter for
struct LogFormat < Log::StaticFormatter
@@colors = {
"FATAL" => :red,
"ERROR" => :red,
"WARN" => :yellow,
"INFO" => :green,
"DEBUG" => :blue,
"TRACE" => :light_blue,
}
def run
string "#{@entry.message}".colorize(@@colors[@entry.severity.label])
end
def self.setup(verbosity)
Colorize.on_tty_only!
if verbosity < 3
_verbosity = [
Log::Severity::Fatal,
Log::Severity::Error,
Log::Severity::Warn,
][[verbosity, 2].min]
Log.setup(
_verbosity,
Log::IOBackend.new(io: STDERR, formatter: LogFormat)
)
end
_verbosity = [Log::Severity::Info,
Log::Severity::Debug,
Log::Severity::Trace,
][[verbosity - 3, 3].min]
Log.setup(
_verbosity,
Log::IOBackend.new(io: STDOUT, formatter: LogFormat)
)
end
macro version
"{{ `grep version shard.yml | cut -d: -f2` }}".strip()
end
doc = <<-DOC
FaaSO CLI tool.
Usage:
faaso build FOLDER ... [-v <level>] [-l]
faaso build FOLDER ... [-v <level>] [-l] [--no-runtime]
faaso deploy FUNKO [-v <level>] [-l]
faaso export SOURCE DESTINATION [-v <level>]
faaso login [-v <level>]
faaso new -r runtime FOLDER [-v <level>]
faaso scale FUNKO [SCALE] [-v <level>] [-l]
faaso secret (-d|-a) FUNKO SECRET [-v <level>] [-l]
faaso status FUNKO [-v <level>] [-l]
faaso version
faaso help COMMAND
Options:
-a --add Add
-d --delete Delete
-h --help Show this screen
-l --local Run commands locally instead of against a FaaSO server
--no-runtime Don't merge a runtime into the funko
-r runtime Runtime for the new funko (use -r list for examples)
-v level Control the logging verbosity, 0 to 5 [default: 3]
-v level Control the logging verbosity, 0 to 6 [default: 4]
DOC
ans = Docopt.docopt(doc, ARGV)
LogFormat.setup(ans["-v"].to_s.to_i)
Oplog.setup(ans["-v"].to_s.to_i) unless ENV.fetch("FAASO_SERVER_SIDE", nil)
Log.debug { ans }
case ans
when .fetch("build", false)
Faaso::Commands::Build.new.run(ans, ans["FOLDER"].as(Array(String)))
exit Faaso::Commands::Build.new.run(ans, ans["FOLDER"].as(Array(String)))
when .fetch("deploy", false)
exit Faaso::Commands::Deploy.new.run(ans, ans["FUNKO"].as(String))
when .fetch("export", false)
Faaso::Commands::Export.new.run(ans, ans["SOURCE"].as(String), ans["DESTINATION"].as(String))
exit Faaso::Commands::Export.new.run(ans, ans["SOURCE"].as(String), ans["DESTINATION"].as(String))
when .fetch("login", false)
exit Faaso::Commands::Login.new.run(ans)
when .fetch("new", false)
Faaso::Commands::New.new.run(ans, ans["FOLDER"].as(Array(String))[0])
exit Faaso::Commands::New.new.run(ans, ans["FOLDER"].as(Array(String))[0])
when .fetch("scale", false)
Faaso::Commands::Scale.new.run(ans, ans["FUNKO"].as(String), ans["SCALE"])
exit Faaso::Commands::Scale.new.run(ans, ans["FUNKO"].as(String), ans["SCALE"])
when .fetch("secret", false)
Faaso::Commands::Secret.new.run(ans, ans["FUNKO"].as(String), ans["SECRET"].as(String))
exit Faaso::Commands::Secret.new.run(ans, ans["FUNKO"].as(String), ans["SECRET"].as(String))
when .fetch("status", false)
Faaso::Commands::Status.new.run(ans, ans["FUNKO"].as(String))
exit Faaso::Commands::Status.new.run(ans, ans["FUNKO"].as(String))
when .fetch("version", false)
Log.info { "#{version}" }
end
exit 0

104
src/runtime.cr Normal file
View File

@ -0,0 +1,104 @@
require "rucksack"
module Runtime
extend self
@@known : Array(String) = {{`find ./runtimes -type d -mindepth 1`.split('\n').reject(&.empty?)}}
@@filelist : Array(String) = {{`find ./runtimes -type f -mindepth 1`.split('\n').reject(&.empty?)}}
Log.debug { "@@known: #{@@known}" }
Log.debug { "@@filelist: #{@@filelist}" }
def list
Log.info { "Crystal has some included runtimes:\n" }
@@known.each do |i|
Log.info { " * #{Path[i].basename}" }
end
Log.info { "\nOr if you have your own, use a folder name" }
end
def self.runtime_files(runtime : String) : {String, Array(String)}
runtime_base = ""
runtime_files = [] of String
if @@known.includes? "./runtimes/#{runtime}"
Log.info { "Using known runtime #{runtime}" }
runtime_base = "./runtimes/#{runtime}/"
runtime_files = @@filelist.select(&.starts_with?(runtime_base)).map { |path|
Path[path].normalize.to_s
}
elsif File.exists? runtime
Log.info { "Using directory #{runtime} as runtime" }
runtime_base = "#{runtime}"
runtime_files = Dir.glob("#{runtime_base}/**/*").select { |file| File.file?(file) }
runtime_files = runtime_files.map { |file| Path[file].normalize.to_s }
else
raise Exception.new("Can't find runtime #{runtime}")
end
{runtime_base, runtime_files.reject(&.starts_with? Path[runtime_base, "template"].normalize.to_s)}
end
def self.template_files(runtime : String) : {String, Array(String)}
template_base = ""
template_files = [] of String
if @@known.includes? "./runtimes/#{runtime}"
Log.info { "Using known runtime #{runtime}" }
template_base = "./runtimes/#{runtime}/template"
template_files = @@filelist.select(&.starts_with?(template_base))
elsif File.exists? runtime
Log.info { "Using directory #{runtime} as runtime" }
template_base = "#{runtime}/template"
template_files = Dir.glob("#{template_base}/**/*").select { |file| File.file?(file) }
template_files = template_files.map { |file| Path[file].normalize.to_s }
else
raise Exception.new("Can't find runtime #{runtime}")
end
{template_base, template_files}
end
# Copyes files from a runtime to a destination folder.
# Files ending in .j2 are rendered as Jinja2 templates
# using the provided context
def copy_templated(
base_path : String,
files : Array(String),
dst_path : String,
context
)
Dir.mkdir_p dst_path
files.each do |file|
content = IO::Memory.new
# We need to use RUCKSACK_MODE=0 so it
# fallbacks to the filesystem. Read the
# file into a buffer
rucksack(file).read(content)
if content.nil?
raise Exception.new("Can't find file #{file}")
return 1
end
# file is like "#{base}/foo"
# dst is like #{dst_path}/foo
dst = Path[dst_path] / Path[file].relative_to(base_path)
# Make sure we have dest dir
Dir.mkdir_p dst.dirname unless File.directory? dst.dirname
# Render templated files
if file.ends_with? ".j2"
dst = dst.sibling(dst.stem)
Log.info { " Creating file #{dst} from #{file}" }
File.open(dst, "w") do |outf|
outf << Crinja.render(content.to_s, context)
end
else # Just copy the file
Log.info { " Creating file #{dst} from #{file}" }
File.open(dst, "w") do |outf|
outf << content.to_s
end
end
end
end
end
# Embed runtimes in the faaso binary using rucksack
{% for name in `find ./runtimes -type f`.split('\n') %}
rucksack({{name}})
{% end %}

View File

@ -1,50 +1,46 @@
<%- result.each do |f| -%>
<tr hx-indicator="#spinner-<%= f["name"] %>">
<td>
<td style="vertical-align: top;">
<%= f["name"] %>
<img id="spinner-<%= f["name"] %>" src="bars.svg" class="htmx-indicator">
</td>
<td>
<table>
<thead>
<th>ID</th>
<th>Current?</th>
<th>Actions</th>
</thead>
<tbody>
<%- f["containers"].as(Array(Docr::Types::ContainerSummary)).each do |c| -%>
<tr>
<td><tt><%= c.@names[0].split("-")[-1] %></tt></td>
<td>
<td style="vertical-align: top;">
<%- f["containers"].as(Array(Docr::Types::ContainerSummary)).each do |c| -%>
<div class="grid">
<div>
<tt><%= c.@names[0].split("-")[-1] %></tt>
</div>
<div>
<%- if c.image_id == f["latest_image"] -%>
<span style="color:green;""> 🟢</span>
<%- else -%>
<span style="color:red;""> 🟢</span>
<%- end -%>
</td>
<td>
</div>
<div role="group">
<button hx-target="#terminal" hx-get="funkos/terminal/logs/<%= c.@names[0].lstrip("/") %>/">Logs</button>
<button hx-target="#terminal" hx-get="funkos/terminal/shell/<%= c.@names[0].lstrip("/") %>/">Shell</button>
</td>
</tr>
<%- end -%>
</tbody>
</p>
</div>
</div>
<%- end -%>
</td>
</table>
<td>
<div role="group">
<%- if f["name"] == "proxy" -%>
<%- else -%>
<%- if f["scale"].as(String).to_i > 0 -%>
<button disabled hx-get="funkos/<%= f["name"] %>/start">Start</button>
<button hx-get="funkos/<%= f["name"] %>/stop" hx-on:htmx:after-request="update_funkos()">Stop</button>
<button disabled hx-delete="funkos/<%= f["name"] %>/" hx-on:htmx:after-request="update_funkos()">Delete</button>
<%- else -%>
<button hx-get="funkos/<%= f["name"] %>/start" hx-on:htmx:after-request="update_funkos()">Start</button>
<button disabled hx-get="funkos/<%= f["name"] %>/stop" hx-on:htmx:after-request="update_funkos()">Stop</button>
<button hx-delete="funkos/<%= f["name"] %>/" hx-on:htmx:after-request="update_funkos()">Delete</button>
<%- end -%>
<button hx-get="funkos/<%= f["name"] %>/restart" hx-on:htmx:after-request="update_funkos()">Restart</button>
<%- end -%>
</div>
</td>
</tr>
<%- end -%>

View File

@ -1,12 +0,0 @@
User nobody
Group nogroup
Port 8888
Listen 0.0.0.0
Timeout 600
Allow 0.0.0.0/0
ReverseOnly Yes
ReverseMagic Yes
ReversePath "/admin/" "http://127.0.0.1:3000/"
ReversePath "/admin/terminal" "http://127.0.0.1:7681"
ReversePath "/faaso/hello/" "http://hello-d89veq:3000/"