ralsina/faaso
1
0
Fork 0
Code Issues Pull Requests Actions Releases Activity

Compare commits

base: ralsina:81ec0779287292341d335cfe12e89f962e8f1046
Branches Tags
ralsina:main
..
compare: ralsina:main
Branches Tags
ralsina:main

6 Commits
81ec077928 ... main

Author SHA1 Message Date
Roberto Alsina
21893fe612 Beginnings of making frontend pretty, switched to pico css 2024-07-09 20:37:43 -03:00
Roberto Alsina
665b4f9ab7 Fix proxy config confusion 2024-07-09 16:40:15 -03:00
Roberto Alsina
3b2297e954 Housekeeping 2024-07-09 13:23:21 -03:00
Roberto Alsina
4f4daf5943 Types tweak 2024-07-09 13:23:16 -03:00
Roberto Alsina
2d333c3df1 * More robust stderr/stdout mixing when 
running commandsserver side
* More robust streaming responses client side
 2024-07-09 13:01:37 -03:00
Roberto Alsina
f015afe7f0 Ensure network exists for start-proxy 2024-07-09 10:57:36 -03:00
14 changed files with 105 additions and 106 deletions
Expand all files Collapse all files

1
Makefile
View File

@ -8,6 +8,7 @@ proxy:
all: build proxy all: build proxy
start-proxy: start-proxy:
docker network create faaso-net || true
docker run --name faaso-proxy-one \ docker run --name faaso-proxy-one \
--rm --network=faaso-net \ --rm --network=faaso-net \
-e FAASO_SECRET_PATH=${PWD}/secrets \ -e FAASO_SECRET_PATH=${PWD}/secrets \

14
TODO.md
View File

@ -12,11 +12,12 @@
* ✅ Crystal + Kemal * ✅ Crystal + Kemal
* ✅ Python + Flask * ✅ Python + Flask
* ✅ Nodejs + Express * ✅ Nodejs + Express
* Document * Create a site
* How to create a runtime * Document
* How to create a funko * FaaSO for app developers
* How to setup the proxy * FaaSO for runtime developers
* APIs * FaaSO server setup
* APIs
* Sanitize all inputs * Sanitize all inputs
* ✅ Streaming responses in slow operations like scaling down * ✅ Streaming responses in slow operations like scaling down
or building or building
@ -34,7 +35,8 @@
* ✅ Fix `export examples/hello_crystal` it has a `template/` * ✅ Fix `export examples/hello_crystal` it has a `template/`
* ✅ Implement zero-downtime rollout (`faaso deploy`) * ✅ Implement zero-downtime rollout (`faaso deploy`)
* ✅ Cleanup `tmp/whatever` after use * ✅ Cleanup `tmp/whatever` after use
* `faaso scale` remote is broken * ✅ `faaso scale` remote is broken
* ✅ Setup linters/pre-commit/etc
## Things to do but not before release ## Things to do but not before release

51
public/index.html
View File

@ -2,8 +2,8 @@
<head> <head>
<meta charset="utf-8" /> <meta charset="utf-8" />
<link rel="stylesheet" href="https://matcha.mizu.sh/matcha.css" /> <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/@picocss/pico@1/css/pico.min.css" />
<script src="https://unpkg.com/htmx.org@2.0.0"></script> <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/@picocss/pico@2/css/pico.colors.min.css" /> <script src="https://unpkg.com/htmx.org@2.0.0"></script>
<meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="viewport" content="width=device-width, initial-scale=1.0">
</head> </head>
@ -11,15 +11,20 @@
<header class="container"> <header class="container">
<h1>FaaSO Admin Interface</h1> <h1>FaaSO Admin Interface</h1>
</header> </header>
<main class=container> <article>
<h2>Your Funko Collection <nav>
<button id="update-funkos" style="float:right; display:inline;" hx-trigger="load, click, every 60s" <ul>
<li><strong style="font-size: 200%;">Your Funko Collection</strong></li>
</ul>
<ul>
<li><button id="update-funkos" style="float:right; display:inline;" hx-trigger="load, click, every 60s"
hx-get="funkos/?format=html" hx-target="#funko-list"> hx-get="funkos/?format=html" hx-target="#funko-list">
Refresh Refresh
</button> </button>
</h2> </ul>
</nav>
<span id="message"></span> <span id="message"></span>
<table hx-target="#message"> <table hx-target="#message" class="striped">
<thead> <thead>
<tr> <tr>
<th>Name</th> <th>Name</th>
@ -31,23 +36,27 @@
</tbody> </tbody>
</table> </table>
<div id="terminal" style="resize: vertical; overflow: auto;"></div> <div id="terminal" style="resize: vertical; overflow: auto;"></div>
</main> </article>
<script> <script>
update_funkos = function () { update_funkos = function () {
document.getElementById("update-funkos").click() document.getElementById("update-funkos").click()
} }
</script> </script>
<main class=container> <article>
<h2> <nav>
Your Secrets <ul>
<button id="update-secrets" style="float:right; display:inline;" hx-trigger="load, click, every 60s" <li><strong style="font-size: 200%;">Your Secrets</strong>
</ul>
<ul>
<li><button id="update-secrets" style="float:right; display:inline;" hx-trigger="load, click, every 60s"
hx-get="secrets/?format=html" hx-target="#secret-list"> hx-get="secrets/?format=html" hx-target="#secret-list">
Refresh Refresh
</button> </button>
<button style="float:right; display:inline;" onclick="show_new_secret()"> <li><button style="float:right; display:inline;" onclick="show_new_secret()">
Add Add
</button> </button>
</h2> </ul>
</nav>
<span id="message"></span> <span id="message"></span>
<table hx-target="#message"> <table hx-target="#message">
<thead> <thead>
@ -60,14 +69,20 @@
<tbody id="secret-list"> <tbody id="secret-list">
</tbody> </tbody>
<dialog id="add-secret"> <dialog id="add-secret">
<topic>New Secret</topic> <article>
<header>
New Secret
</header>
<form hx-post="secrets/"> <form hx-post="secrets/">
<input placeholder="funko name" id="new-secret-funko" name="funko"> <input placeholder="funko name" id="new-secret-funko" name="funko">
<input placeholder="secret name" id="new-secret-name" name="name"> <input placeholder="secret name" id="new-secret-name" name="name">
<input placeholder="secret value" type="password" id="new-secret-password" name="value"> <input placeholder="secret value" type="password" id="new-secret-password" name="value">
<button type="submit" hx-on:htmx:after-request="hide_new_secret()">CREATE</button> <fieldset role="group" style="text-align: right;">
<button style="width:9em; display: inline;" type="submit" hx-on:htmx:after-request="hide_new_secret()">CREATE</button>
<button style="width:9em; display: inline;" onclick="hide_new_secret(); close();" aria-label="Close" rel="prev">CLOSE</button>
</fieldset>
</form> </form>
<button onclick="hide_new_secret(); close();">CLOSE</button> </article>
</dialog> </dialog>
<script> <script>
update_secrets = function() { update_secrets = function() {
@ -87,5 +102,5 @@
update_secrets() update_secrets()
} }
</script> </script>
</main> </article>
</body> </body>

4
shard.lock
View File

@ -44,10 +44,6 @@ shards:
git: https://github.com/mamantoha/http_proxy.git git: https://github.com/mamantoha/http_proxy.git
version: 0.10.3 version: 0.10.3
inotify:
git: https://github.com/petoem/inotify.cr.git
version: 1.0.3
kemal: kemal:
git: https://github.com/kemalcr/kemal.git git: https://github.com/kemalcr/kemal.git
version: 1.5.0 version: 1.5.0

2
shard.yml
View File

@ -30,8 +30,6 @@ dependencies:
docr: docr:
github: ralsina/docr github: ralsina/docr
branch: add_exposed_ports branch: add_exposed_ports
inotify:
github: petoem/inotify.cr
kemal: kemal:
github: kemalcr/kemal github: kemalcr/kemal
kemal-basic-auth: kemal-basic-auth:

5
src/commands/build.cr
View File

@ -58,10 +58,7 @@ module Faaso
{"funko.tgz" => File.open(tmp), "name" => "funko.tgz"}, {"funko.tgz" => File.open(tmp), "name" => "funko.tgz"},
user: user, password: password user: user, password: password
) do |response| ) do |response|
loop do IO.copy(response.body_io, STDOUT)
Log.info { response.body_io.gets }
break if response.body_io.closed?
end
end end
Log.info { "Build finished successfully." } Log.info { "Build finished successfully." }
rescue ex : Crest::InternalServerError rescue ex : Crest::InternalServerError

5
src/commands/deploy.cr
View File

@ -41,10 +41,7 @@ module Faaso
Crest.get( Crest.get(
"#{Config.server}funkos/#{funko_name}/deploy/", \ "#{Config.server}funkos/#{funko_name}/deploy/", \
user: user, password: password) do |response| user: user, password: password) do |response|
loop do IO.copy(response.body_io, STDOUT)
Log.info { response.body_io.gets }
break if response.body_io.closed?
end
end end
0 0
end end

29
src/commands/scale.cr
View File

@ -10,14 +10,14 @@ module Faaso
# In both cases stopped instances after the required # In both cases stopped instances after the required
# scale is reached are deleted. # scale is reached are deleted.
struct Scale struct Scale
def local(options, name : String, scale : Int) : Int32 def local(options, name : String, scale : Int | Nil) : Int32
funko = Funko::Funko.from_names([name])[0] funko = Funko::Funko.from_names([name])[0]
# Asked about scale # Asked about scale
if funko.image_history.empty? if funko.image_history.empty?
Log.error { "Unknown funko #{funko.name}" } Log.error { "Unknown funko #{funko.name}" }
return 1 return 1
end end
if !scale if scale.nil?
Log.info { "Funko #{name} has a scale of #{funko.scale}" } Log.info { "Funko #{name} has a scale of #{funko.scale}" }
return 0 return 0
end end
@ -26,19 +26,21 @@ module Faaso
0 0
end end
def remote(options, name : String, scale : Int) : Int32 def remote(options, name : String, scale : Int | Nil) : Int32
user, password = Config.auth user, password = Config.auth
Faaso.check_version Faaso.check_version
if !scale if scale.nil?
response = Crest.get( Crest.get(
"#{Config.server}funkos/#{name}/scale/", \ "#{Config.server}funkos/#{name}/scale/", \
user: user, password: password) user: user, password: password) do |response|
Log.info { " => " + response.body } IO.copy(response.body_io, STDOUT)
else end
response = Crest.post( return 0
"#{Config.server}funkos/#{name}/scale/", end
{"scale" => scale}, user: user, password: password) Crest.post(
Log.info { " => " + response.body } "#{Config.server}funkos/#{name}/scale/",
{"scale" => scale}, user: user, password: password) do |response|
IO.copy(response.body_io, STDOUT)
end end
0 0
rescue ex : Crest::InternalServerError rescue ex : Crest::InternalServerError
@ -46,7 +48,8 @@ module Faaso
1 1
end end
def run(options, name : String, scale : Int) : Int32 def run(options, name : String, scale) : Int32
scale = scale.try &.to_s.to_i
if options["--local"] if options["--local"]
return local(options, name, scale) return local(options, name, scale)
end end

5
src/commands/status.cr
View File

@ -31,10 +31,7 @@ module Faaso
Crest.get( Crest.get(
"#{Config.server}funkos/#{name}/status/", \ "#{Config.server}funkos/#{name}/status/", \
user: user, password: password) do |response| user: user, password: password) do |response|
loop do IO.copy(response.body_io, STDOUT)
Log.info { response.body_io.gets }
break if response.body_io.closed?
end
end end
0 0
rescue ex : Crest::InternalServerError rescue ex : Crest::InternalServerError

15
src/daemon/funko.cr
View File

@ -133,20 +133,21 @@ module Funko
end end
# Helper to run faaso locally and respond via env # Helper to run faaso locally and respond via env
def run_faaso(args : Array(String), env) : Bool def run_faaso(args : Array(String), env)
Log.info { "Running faaso [#{args.join(", ")}, -l, 2>&1]" } args << "-l" # Always local in the server
Log.info { "Running faaso [#{args}" }
Process.run( Process.run(
command: "faaso", command: "faaso",
args: args + ["-l", "2>&1"], # Always local in the server args: args,
shell: true, env: {"FAASO_SERVER_SIDE" => "true"},
) do |process| ) do |process|
loop do loop do
env.response.print process.output.gets(chomp: false) data = process.output.gets(chomp: false)
env.response.print data
env.response.flush env.response.flush
Fiber.yield Fiber.yield # Without this the process never ends
break if process.terminated? break if process.terminated?
end end
true
end end
# FIXME: find a way to raise an exception on failure # FIXME: find a way to raise an exception on failure
# of the faaso process # of the faaso process

17
src/daemon/proxy.cr
View File

@ -1,16 +1,11 @@
require "./funko.cr" require "./funko.cr"
require "docr" require "docr"
require "inotify"
require "kemal" require "kemal"
module Proxy module Proxy
CADDY_CONFIG_PATH = "config/funkos" CADDY_CONFIG_PATH = "config/Caddyfile"
@@current_config = File.read(CADDY_CONFIG_PATH) CADDY_CONFIG_FUNKOS = "config/funkos"
@@current_config = File.read(CADDY_CONFIG_FUNKOS)
@@watcher = Inotify.watch(CADDY_CONFIG_PATH) do |_|
Log.info { "Reloading caddy config" }
Process.run(command: "caddy", args: ["reload", "--config", CADDY_CONFIG_PATH])
end
# Get current proxy config # Get current proxy config
get "/proxy/" do get "/proxy/" do
@ -25,7 +20,7 @@ module Proxy
update_proxy_config update_proxy_config
end end
def self.update_proxy_config def self.update_proxy_config : Nil
docker_api = Docr::API.new(Docr::Client.new) docker_api = Docr::API.new(Docr::Client.new)
containers = docker_api.containers.list(all: true) containers = docker_api.containers.list(all: true)
@ -50,13 +45,13 @@ module Proxy
if @@current_config != config if @@current_config != config
Log.info { "Updating proxy config" } Log.info { "Updating proxy config" }
File.open(CADDY_CONFIG_PATH, "w") do |file| File.open(CADDY_CONFIG_FUNKOS, "w") do |file|
file << config file << config
end end
# Reload config # Reload config
@@current_config = config @@current_config = config
Process.run(command: "caddy", args: ["reload", "--config", CADDY_CONFIG_PATH])
end end
config
end end
end end

6
src/funko.cr
View File

@ -90,7 +90,11 @@ module Funko
docker_api = Docr::API.new(Docr::Client.new) docker_api = Docr::API.new(Docr::Client.new)
current_scale = self.scale current_scale = self.scale
result = [] of String result = [] of String
return result if current_scale == new_scale
if current_scale == new_scale
Log.info { "Funko #{name} already at scale #{new_scale}" }
return result
end
Log.info { "Scaling #{name} from #{current_scale} to #{new_scale}" } Log.info { "Scaling #{name} from #{current_scale} to #{new_scale}" }
if new_scale > current_scale if new_scale > current_scale

21
src/main.cr
View File

@ -35,29 +35,28 @@ Options:
DOC DOC
ans = Docopt.docopt(doc, ARGV) ans = Docopt.docopt(doc, ARGV)
Oplog.setup(ans["-v"].to_s.to_i) Oplog.setup(ans["-v"].to_s.to_i) unless ENV.fetch("FAASO_SERVER_SIDE", nil)
Log.debug { ans } Log.debug { ans }
status : Int32 = 0
case ans case ans
when .fetch("build", false) when .fetch("build", false)
status = Faaso::Commands::Build.new.run(ans, ans["FOLDER"].as(Array(String))) exit Faaso::Commands::Build.new.run(ans, ans["FOLDER"].as(Array(String)))
when .fetch("deploy", false) when .fetch("deploy", false)
status = Faaso::Commands::Deploy.new.run(ans, ans["FUNKO"].as(String)) exit Faaso::Commands::Deploy.new.run(ans, ans["FUNKO"].as(String))
when .fetch("export", false) when .fetch("export", false)
status = Faaso::Commands::Export.new.run(ans, ans["SOURCE"].as(String), ans["DESTINATION"].as(String)) exit Faaso::Commands::Export.new.run(ans, ans["SOURCE"].as(String), ans["DESTINATION"].as(String))
when .fetch("login", false) when .fetch("login", false)
status = Faaso::Commands::Login.new.run(ans) exit Faaso::Commands::Login.new.run(ans)
when .fetch("new", false) when .fetch("new", false)
status = Faaso::Commands::New.new.run(ans, ans["FOLDER"].as(Array(String))[0]) exit Faaso::Commands::New.new.run(ans, ans["FOLDER"].as(Array(String))[0])
when .fetch("scale", false) when .fetch("scale", false)
status = Faaso::Commands::Scale.new.run(ans, ans["FUNKO"].as(String), ans["SCALE"].as(String).to_i) exit Faaso::Commands::Scale.new.run(ans, ans["FUNKO"].as(String), ans["SCALE"])
when .fetch("secret", false) when .fetch("secret", false)
status = Faaso::Commands::Secret.new.run(ans, ans["FUNKO"].as(String), ans["SECRET"].as(String)) exit Faaso::Commands::Secret.new.run(ans, ans["FUNKO"].as(String), ans["SECRET"].as(String))
when .fetch("status", false) when .fetch("status", false)
status = Faaso::Commands::Status.new.run(ans, ans["FUNKO"].as(String)) exit Faaso::Commands::Status.new.run(ans, ans["FUNKO"].as(String))
when .fetch("version", false) when .fetch("version", false)
Log.info { "#{version}" } Log.info { "#{version}" }
end end
exit(status) exit 0

36
src/views/funkos.ecr
View File

@ -1,39 +1,32 @@
<%- result.each do |f| -%> <%- result.each do |f| -%>
<tr hx-indicator="#spinner-<%= f["name"] %>"> <tr hx-indicator="#spinner-<%= f["name"] %>">
<td> <td style="vertical-align: top;">
<%= f["name"] %> <%= f["name"] %>
<img id="spinner-<%= f["name"] %>" src="bars.svg" class="htmx-indicator"> <img id="spinner-<%= f["name"] %>" src="bars.svg" class="htmx-indicator">
</td> </td>
<td> <td style="vertical-align: top;">
<table> <%- f["containers"].as(Array(Docr::Types::ContainerSummary)).each do |c| -%>
<thead> <div class="grid">
<th>ID</th> <div>
<th>Current?</th> <tt><%= c.@names[0].split("-")[-1] %></tt>
<th>Actions</th> </div>
</thead> <div>
<tbody>
<%- f["containers"].as(Array(Docr::Types::ContainerSummary)).each do |c| -%>
<tr>
<td><tt><%= c.@names[0].split("-")[-1] %></tt></td>
<td>
<%- if c.image_id == f["latest_image"] -%> <%- if c.image_id == f["latest_image"] -%>
<span style="color:green;""> 🟢</span> <span style="color:green;""> 🟢</span>
<%- else -%> <%- else -%>
<span style="color:red;""> 🟢</span> <span style="color:red;""> 🟢</span>
<%- end -%> <%- end -%>
</td> </div>
<td> <div role="group">
<button hx-target="#terminal" hx-get="funkos/terminal/logs/<%= c.@names[0].lstrip("/") %>/">Logs</button> <button hx-target="#terminal" hx-get="funkos/terminal/logs/<%= c.@names[0].lstrip("/") %>/">Logs</button>
<button hx-target="#terminal" hx-get="funkos/terminal/shell/<%= c.@names[0].lstrip("/") %>/">Shell</button> <button hx-target="#terminal" hx-get="funkos/terminal/shell/<%= c.@names[0].lstrip("/") %>/">Shell</button>
</td> </div>
</tr> </div>
<%- end -%> <%- end -%>
</tbody>
</p>
</td> </td>
</table>
<td> <td>
<div role="group">
<%- if f["name"] == "proxy" -%> <%- if f["name"] == "proxy" -%>
<%- else -%> <%- else -%>
<%- if f["scale"].as(String).to_i > 0 -%> <%- if f["scale"].as(String).to_i > 0 -%>
@ -47,6 +40,7 @@
<%- end -%> <%- end -%>
<button hx-get="funkos/<%= f["name"] %>/restart" hx-on:htmx:after-request="update_funkos()">Restart</button> <button hx-get="funkos/<%= f["name"] %>/restart" hx-on:htmx:after-request="update_funkos()">Restart</button>
<%- end -%> <%- end -%>
</div>
</td> </td>
</tr> </tr>
<%- end -%> <%- end -%>