Hayvok 2021-02-26 06:09 UTC link

Zero/near-zero dependencies, minimal configuration, portable, and trivial to run. Awesome.

Even if Redbean here doesn't end up taking over the world, I hope that more ecosystems and toolchains sit up and take notice. We need more tools that look like this.

mikewarot 2021-02-26 06:16 UTC link

I really like your view of the world, that programs should be portable, tiny, and just work. The slamming of so much functionality into a zip file is inspiring.

Would it be possible to do something similar with Free Pascal, allowing the recreation of something like Turbo Pascal, except really, really portable .com output?

ToJans 2021-02-26 06:37 UTC link

The short description is a bit too short; this is mind blowing!

Redbean is a portable, single file executable webserver which also acts as a zip-file for the static content it hosts, and runs on Linux + Mac + Windows + FreeBSD + OpenBSD + NetBSD + BIOS without any recompilation...

You can manage the static content using standard pkzip tooling! Now that's what I call thinking out of the box!

Edited: some rewording and breathing space

hprx 2021-02-26 06:53 UTC link

I just spent 30 minutes reading about this. I'm so shocked that I'm logging in to comment after 5 years of lurking.

Justine has built a c library that allows you compile a binary once and have it run it on any os or baremetal. The SAME binary. Quite frankly, that sentence doesn't even make sense to me.

Check out https://storage.googleapis.com/justine/cosmopolitan/index.ht...

As far as I'm concerned, this is literal magic. Look at the magic numbers: https://github.com/jart/cosmopolitan/blob/37a4c70c3634862d8d...

I could go on, but there's no binary portability comparison with any other language. And she has made some pretty neat optimizations.

Back in the day, I saw some pretty neat stuff with the ELF format, but this takes the cake.

Wow.

Edit: I'm editing because this is just so bloody absurd.

https://storage.googleapis.com/justine/printimage.html

$ ./printimage.com someimage.jpg

Like wow. And also video.

https://storage.googleapis.com/justine/printvideo.html

I'm struggling to put my shock into words. I've been around.

There's engineering. There's academia.

But this falls into straight-up wizardry.

brainless 2021-02-26 08:07 UTC link

OK I see we are going crazy, me too. Thanks @jart.

I have been thinking about portable web apps with embedded SQLite for some time. I do not have the technical chops at C level to pull this off. I am really inspired by this project. I hope Redbean leads a way to distribute self-hosted apps in today's era of the cloud.

mbgerring 2021-02-26 08:30 UTC link

I am floored. Not only is this just breathtaking sorcery, I also needed this exact thing for a project. I don’t understand probably 70% of the description of this software, but I understand what it does, and I’m more excited to carefully study and use this than I have been about anything in years.

safeaim 2021-02-26 09:08 UTC link

Am I the only one having issues trying to get this working? The webserver starts just fine, but once I add the index.html with zip as in the example, it stops working. This is on Mac 10.15, CentOS 8 and Ubuntu 18.04 LTS.

Centos: [centos@test ~]$ ./redbean.com -vv

  error: Uncaught SIGSEGV on test.novalocal
  ./redbean.com
  EINVAL/err=22/errno:2/GetLastError:0
  Linux test.novalocal 4.18.0-240.10.1.el8_3.x86_64 #1 SMP 
  Mon Jan 18 17:05:51 UTC 2021

On the mac: ~ my-Maccie$./redbean.com -vv Killed: 9

On Ubuntu: ubuntu@localhost:~$ ./redbean.com -vv

  error: Uncaught SIGSEGV on localhost
  ./redbean.com
  EINVAL/err=22/errno:2/GetLastError:0
  Linux localhost 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 
  28 05:20:47 UTC 2021

mhd 2021-02-26 09:27 UTC link

It's a neat coincidence that this is on at the same time as an article about CP/M, because I think there could be some overlap in the programs that would be a good fit for cosmopolitan.

Sure, there's server appliances like this, and I hope someone makes a neat services wrapper to abstract at least some platform's intricacies so you can do a "cosmo-service up redbean" on BSD/Linux/Windows.

But coming back to CP/M, there you had a lowest common denominator of terminal applications, too, but spread across different architectures. And you could still produce some quite intersting, if a bit business-like applications. These days you probably can even rely on more ANSI colors and maybe even unicode fonts (hopefull as an option, not mandatory).

The ZIP characteristics of the APE format make it even easier to distribute a whole application in a rather simple way. Yes, sure, you can do regular Unix-style servers and pipe-it-together CLI tools, but I wouldn't mind more self-contained "business" applications with a lo-fi aesthetic regarding interface and API usage. The PICO-8 of TUIs…

aembleton 2021-02-26 09:31 UTC link

Just make sure you don't have another server running on :8080.

I had syncthing running and kept getting some error whenever I tried to run redbean. The error didn't make much sense, but eventually I realised that this was the casue of the error.

I'm really impressed how portable this is. The only improvement I can see is if it auto-opened your browser to :8080. That way it would be easy to distribute the binary and have people run your application and just interact with it through their browser. No need to ship electron then!

chriszhang 2021-02-26 10:17 UTC link

Someone please be kind and explain to me how this works? Don't different executable formats need different headers?

Don't ELF need "0x7F ELF" header? Don't Windows executable need "MZ" header? How is it fulfilling both requirements simultaneously?

shaded-enmity 2021-02-26 12:01 UTC link

If you want to learn more how these things work I'd highly suggest going through the PoC||GTFO archive (https://github.com/angea/pocorgtfo/blob/master/README.md) and check out entries by Ange Albertini or entries named like "This ZIP is also a PDF".

raphaelj 2021-02-26 12:23 UTC link

The amount of work that went to this is insane. She made a (full ?) implementation of libc: https://github.com/jart/cosmopolitan/tree/master/libc

haolez 2021-02-26 12:49 UTC link

This is so powerful that it can be used as a "weapon": it's the perfect lib for making malware. I think this is a compliment!

TickleSteve 2021-02-26 13:04 UTC link

TLDR: This is a packed file format that can look to the OS as being executable (for Windows/Linux/OSX). The code is x86 and so offers native performance on those processors. On other platforms, an x86 emulator is built in so can't offer native execution speeds.

While neat, its not the "best of all worlds" due to the lack of native code on anything other than x86/x64. Also, claiming "bare metal" is supported is a stretch as you are limited to having no I/O of any form (as there is no platform code).

nwienert 2021-02-26 16:23 UTC link

Can someone marry this with QuickJS and make a micro node competitor with a web server, 0 deps and no install?

Wonder how far something like that could go, could be really interesting for replacing bash for me with something easier to write like JS.

aero- 2021-02-26 18:53 UTC link

Can someone explain this to me like I'm an 8 year old?

tgbugs 2021-02-26 19:04 UTC link

I looked into building SBCL using chibicc last weekend in the hope that one could build a truly portable lisp runtime using the ape toolchain (Yes, yes, I know you couldn't possibly pick two more different approaches to software portability). Turns out the GNUC extensions for __asm__ are the primary stumbling block (though I'm sure there are others). tcc supports the GNUC extensions needed, and chibicc supports the thread local storage needed for dynamic variables. Looking forward to future developments!

IggleSniggle 2021-02-26 21:56 UTC link

I feel like I’ve learned more from your website and your source code in a few hours than I otherwise would have from years of study.

You have a real talent for distilling the complex into something simple. It’s apparent in the purpose of your programs, in your writing, and in your source code. Thank you so much!

Basics of computing architecture that I didn’t especially think were too complex for me to learn, but we’re too abstracted for me to become interested in, are laid bare in each one of these programs and their source. This is all incredibly exciting.

jan6 2021-02-26 23:25 UTC link

I will say, I WOULD absolutely love it, if it weren't for the little problem...that...I can't "just add to the zip"...because for whatever reason, it ruins the special setup... on windows, 7zip and windows explorer's builtin zip stuff refuse to modify it, as if it was corrupted (understandable), but explorer can view it, on linux, zip (apparently info-zip 3.0 according to the version) can add to it just fine, but ruins the special properties, on linux it just spawns the debug gdb with "no symbol table loaded" and no register or assembly data, and on windows it's either wrong format, or `check failed: 0x1 == 0x0 (32)` so also broken... I definitely love the idea of this, and the APE in general, and it works as-is, but modification seems impossible to me...changing to .zip doesn't change the breakage

anyone else had luck with adding stuff to it? is it just me? or is it like "it works with openbsd zip command only"?

arcticbull 2021-02-27 03:13 UTC link

I absolutely love reading Justine's code. It comes up from time to time here and it just makes me happy. It reminds me of the passage from "Programming Sucks" by Still Drinking [1]

  Every programmer occasionally, when nobody’s home, turns off the lights, pours a glass of scotch, puts on some light German electronica, and opens up a file on their computer. It’s a different file for every programmer. Sometimes they wrote it, sometimes they found it and knew they had to save it. They read over the lines, and weep at their beauty, then the tears turn bitter as they remember the rest of the files and the inevitable collapse of all that is good and true in the world.

  This file is Good Code. It has sensible and consistent names for functions and variables. It’s concise. It doesn’t do anything obviously stupid. It has never had to live in the wild, or answer to a sales team. It does exactly one, mundane, specific thing, and it does it well. It was written by a single person, and never touched by another. It reads like poetry written by someone over thirty.

Her work is always just a little bit trippy in a good way haha.

[1] http://stilldrinking.org/programming-sucks

jart 2021-02-26 07:11 UTC link

Author here. Absolutely. I used to love Turbo Pascal and Delphi when I was younger. If Free Pascal uses GNU LD.BFD or LLVM LLD when it links programs, then all you'd need to do is is configure it to use cosmopolitan.a when linking system call functions like read(), write(), etc. See https://github.com/jart/cosmopolitan Another option is if Free Pascal wants to write all the system call support from scratch, then doing that now is going to be a whole lot easier since the Cosmopolitan codebase does a really good job documenting all the magic numbers you'll need. See files like https://github.com/jart/cosmopolitan/blob/master/libc/sysv/s... and https://github.com/jart/cosmopolitan/blob/master/libc/sysv/c... I've been working on a tiny C11 compiler called chibicc which has most GNU extensions and I managed to get it working as an actually portable executable with an integrated assembler: https://github.com/jart/cosmopolitan/blob/master/third_party... I also got Antirez's KILO text editor working as an APE binary. https://github.com/jart/cosmopolitan/blob/master/examples/ki... If we can build a linker too then we can get a freestanding single file toolchain + ide that's able to be a modern version of Turbo C.

malux85 2021-02-26 07:14 UTC link

I agree, I got to the page on actually portable executable, and had to read it a few times to make sure I was grokking it properly. This is just sheer cleverness, THIS is the stuff that should be on hacker news!

skrebbel 2021-02-26 08:09 UTC link

Everything she does has this level of jaw-dropping amazingness. Between her and Fabrice Bellard i don't know many people who consistently get my chin to hit the table.

I hope that Cosmopolitan becomes mainstream, so much more software could have that It Just Works quality.

alisonatwork 2021-02-26 08:14 UTC link

Agreed, this is the best programming-related thing I've seen on the internet in a long time. It reminds me of when I was a teenager, excited to become a computer programmer. Then I grew up and joined the real world workforce and it was all far less exciting than I imagined.

The coolest thing about this Actually Portable Executable is that once there's a compiler and linker built with it, I can play around with writing C on Windows without having to faff about with WSL or MinGW or learning what makes MSVC different from the C I learned in university and coded in my first job before I became a Java/JavaScript/Python/etc programmer. When I discovered Go I thought I had discovered the better C that just worked everywhere, but the idea of having plain old C that also just works everywhere is very appealing.

tutfbhuf 2021-02-26 08:22 UTC link

Is it that this binary does some tricks to run on Linux, Mac, Windows, FreeBSD, OpenBSD, NetBSD, BIOS or does it really work on all operating systems including something like Plan 9 or even TempleOS?

And does it run on different architectures like ARMv6?

jart 2021-02-26 09:49 UTC link

Author here. It works fine on RHEL5 / CentOs5 for me. I can also confirm RHEL7. Please file an issue. What will help in particular is if you can give me the faulting RIP address. That should be in the crash report or in your dmesg log.

tromp 2021-02-26 09:56 UTC link

Same here on MacOS Catalina 10.15.7. The original invocation runs fine

    $ bash -c './redbean.com -vv'
    W2021-02-26T01:50:12.665277:tool/net/redbean.c:1144:redbean:3302] setsockopt(server, IPPROTO_TCP, TCP_FASTOPEN, &yes, sizeof(yes)) → EINVAL/err=22/errno:22/GetLastError:0
    W--------------------000015:tool/net/redbean.c:1145:redbean:3302] setsockopt(server, IPPROTO_TCP, TCP_QUICKACK, &yes, sizeof(yes)) → ENOPROTOOPT/err=42/errno:42/GetLastError:0
    V--------------------000022:tool/net/redbean.c:1166:redbean:3302] 0.0.0.0:8080 listen

Visiting http://127.0.0.1:8080/ in my web browser shows a nice redbean page with lobster graphic at http://127.0.0.1:8080/tool/net/redbean.html but then following usage instructions in another shell:

    $ echo '<b>hello</b>' >index.html
    $ zip redbean.com index.html
      adding: index.html (stored 0%)
    $ ./redbean.com -vv
    Killed: 9

I blame my zip

    $ zip -v
    Copyright (c) 1990-2008 Info-ZIP - Type 'zip "-L"' for software license.
    This is Zip 3.0 (July 5th 2008), by Info-ZIP.
    Currently maintained by E. Gordon.

Addition of index.html shrinks(!) redbean.com from size 204800 to size 204348, with changes starting from char 172938. That clearly corrupts the portable executable format.

oneeyedpigeon 2021-02-26 10:10 UTC link

Yup, I get "Killed: 9" on macOS (Big Sur) too. I actually had some kind of problem (I think) with the original invocation:

    $ bash -c './redbean.com -vv'
    W2021-02-26T02:06:29.162953:tool/net/redbean.c:1144:redbean:83621] setsockopt(server, IPPROTO_TCP, TCP_FASTOPEN, &yes, sizeof(yes)) → EINVAL/err=22/errno:22/GetLastError:0
    W--------------------000026:tool/net/redbean.c:1145:redbean:83621] setsockopt(server, IPPROTO_TCP, TCP_QUICKACK, &yes, sizeof(yes)) → ENOPROTOOPT/err=42/errno:42/GetLastError:0
    error:./redbean.com: check failed: 0xffffffffffffffff != 0xffffffffffffffff (48)
    6fffffffff70 0000004034cf UNKNOWN
    6fffffffff80 000000403225 UNKNOWN
    6fffffffffc0 0000004027f3 UNKNOWN
    6fffffffffe0 000000402a21 UNKNOWN
    7ffeefbff870 00000040227b UNKNOWN

zimpenfish 2021-02-26 10:25 UTC link

IIRC, from vague memories of things flying past, Big Sur won't let a modified binary run once it's been checked by Gatekeeper. Which means the first run gets checked and notarised but when you add the `index.html`, the on-disk binary has changed and Gatekeeper won't allow it to run - I guess it's to prevent malicious code modifications, etc.

jart 2021-02-26 10:45 UTC link

Author here. We're living in the most exciting time for developing terminal applications. When Microsoft unexpectedly added support for VT100 and XTERM codes to CMD.EXE it totally changed the equation and ANSI became universal for the first time. Blinkenlights is an example of a TUI application I created using Cosmopolitan and it literally works everywhere. https://justine.lol/blinkenlights/index.html You don't need curses. All that's needed is an ioctl() call which flips a bit in termios. Cosmopolitan polyfills that across operating systems. Another cool example of a demo app is this conway's game of life tui gui: https://justine.lol/apelife/index.html

Gys 2021-02-26 10:55 UTC link

The OP explains https://justine.lol/ape.html

r0b05 2021-02-26 11:17 UTC link

Let's say we are able to do this. Would we have any security concerns to be addressed?

TeMPOraL 2021-02-26 12:31 UTC link

Agreed. This single sentence at the end of αcτµαlly pδrταblε εxεcµταblε page:

> I believe the best chance we have of doing that [writing software that stands the test of time with minimal toil], is by gluing together the binary interfaces that've already achieved a decades-long consensus, and ignoring the APIs.

...it's a kind of thought I don't think I could ever come up with. Mind blown.

cirowrc 2021-02-26 13:13 UTC link

scary scary scary

nabla9 2021-02-26 13:17 UTC link

She started her career with a very successful spam tool. She was 14 at the time.

donaldihunter 2021-02-26 14:03 UTC link

I came here for this. E.g. https://www.alchemistowl.org/pocorgtfo/pocorgtfo19.pdf

"This file, pocorgtfo19.pdf, is valid as a PDF document, a ZIP archive, and a HTML page. It is also available as a Windows PE executable, a PNG image and an MP4 video, all of which have the same MD5 as this PDF."

OOPMan 2021-02-26 14:29 UTC link

No I/O eh?

I'm sure the FP zealots will love it :-)

jart 2021-02-26 15:21 UTC link

I'm just one woman. What Cosmopolitan does so far, it does really well. It was only as recently as a few days ago that I got mmap() and malloc() polyfilled on bare metal. It has serial uart i/o. It's going to have e1000/virtio sockets soon. You can help me will that future into existence. I need people who know o/s dev and can help me write code that does things like correctly set up pic controller.

sva_ 2021-02-26 15:50 UTC link

I think you could trivially issue a system() call, you'd just have to use the corresponding command to open the default browser for each particular system. (Linux: xdg-open, MacOs: open, Windows: start)

BugWatch 2021-02-26 16:55 UTC link

I still use the classic version of Tiddlywiki that is a single self-contained self-editing html file, and I "app-ify" it on Windows by changing the extension to *.hta.

But having the similar solution with the SQLite, would be a perfect combination.

lambda_obrien 2021-02-26 17:44 UTC link

Strong agree; someone needs to give her like a million dollars a year salary and unlimited resources to explore whatever she wants for life, just to see what she comes up with.

anderspitman 2021-02-26 19:42 UTC link

Can someone explain this to me like I'm a 31 year old?

mikewarot 2021-02-26 19:46 UTC link

On most computer systems these days can select a set of files and folders and tell the computer to "Send to compressed file", the files that come out of that process usually end in ".zip", and are thus known as "zip files"

This is a zip file, like any other... you can add and remove things from it. However, this ZIP file has a superpower, it can display the contents on the web... it has a web server built into it.

You can take this file, run it on a Windows, Linux, Macintosh, and it will work. You don't need separate versions for each different system... this just works on all three systems, unlike almost every program ever written.

This is one of the most impressive feats of programming things I've seen in my 40+ years of programming. The web serving is clever, but the superpower that it can run on anything really took a huge amount of work, which the author built into a tool called "ape", and has shared with the world, so other people can use it, and help her make it better.

brabel 2021-02-26 20:16 UTC link

On Mac Big Sur it doesn't seem to work either:

    ▶ zsh -c './redbean.com -vv' 
    W2021-02-26T12:12:24.019387:tool/net/redbean.c:1144:redbean:56779] setsockopt(server, IPPROTO_TCP, TCP_FASTOPEN, &yes, sizeof(yes)) → EINVAL/err=22/errno:22/GetLastError:0

W--------------------000028:tool/net/redbean.c:1145:redbean:56779] setsockopt(server, IPPROTO_TCP, TCP_QUICKACK, &yes, sizeof(yes)) → ENOPROTOOPT/err=42/errno:42/GetLastError:0 V--------------------000023:tool/net/redbean.c:1166:redbean:56779] 0.0.0.0:8080 listen ^CV2021-02-26T12:12:27.012972:tool/net/redbean.c:1183:redbean:56779] 0.0.0.0:8080 terminated

EDIT: I am an idiot: it's working just fine... the messages looked like an error to me, but visiting https://localhost:8080/ does show a nice HTML page!

jart 2021-02-26 22:00 UTC link

You need __asm__ in chibicc for sbcl? I have good news for you. I did exactly that in cosmopolitan's vendored fork of chibicc here: https://github.com/jart/cosmopolitan/blob/master/third_party... I went on a coding rampage back in December, adding to chibicc pretty much every GNU extension under the moon: https://github.com/jart/cosmopolitan/blob/master/third_party... It should be a perfect fit for use cases needed by Cosmopolitan and SBCL. Right now it's unlikely to get merged back upstream because Rui is still focusing on using Chibicc for his book, and that requires being more conservative about feature inclusion right now.

I would highly recommend using chibicc. Bellard's work on tcc was fantastic when it came out. However it didn't age well. Its GNU extension support is roughly equivalent to GCC 2.x. The x86_64 support that got bolted on later isn't very good. It was much more elegant back when it was only doing i386. Hackability was also laid low by merging a lot of external contributions. The TCC design, while amazingly fast, also carries the tradeoff of making things like inline assembly hacks really hard because it generates the x86 binary content directly, rather than going through the intermediate step of generating an assembly file and running it through a proper assembler -- which chibicc now has!

dev-tacular 2021-02-27 02:59 UTC link

Right? It's amazing to see what some people can accomplish with the same 24 hours a day we all get.

erikpukinskis 2021-02-27 16:25 UTC link

This is too real. Except at this point, after 25 years I have probably ten of these.

Editorial Channel

What the content says

Structural Channel

What the site does