Indeed. There is no reason why CGI would need shells or scripting languages though, you can just write them in any programming language. It's not that hard; I wrote this pastebin clone in C: https://github.com/gsliepen/cbin/
It's not an issue with the actual CGI program. It's hard to make exec alone work the way people expect without doing something like exec('sh', '-c',...) so a lot of servers were doing that.
If you drop the requirement that the image has to be taken with wavelengths our eyes are sensitive to, you could image it using radio telescopes. We already have this capability, the problem though with radio interferometry is that while you can get an effectively huge aperture, the contrast level will be very low, and I am guessing that after subtracting the signal from the star, the signal from the planet will not be above the noise level. Note that optical interferometers would have the same problem.
> For example, instead of a power function that uses a loop, you could generate specialized code like x * x * x * x * x directly. This eliminates runtime overhead and creates highly optimized code.
This is misguided. For decennia now, there is no reason to assume that hand-unrolled code is faster than a for-loop. Compilers optimize this stuff, and they do this even better than mindlessly multiplying x by itself. For example, raising x to the power 6 only needs 3 multiplications, see for example: https://godbolt.org/z/Edz4jjqvv
While there are definitely use cases for meta-programming, optimization is not one of them.
Optimization is absolutely one of them, once you start dealing with higher order programs or programs with sophisticated control flow. Compiler optimizations are not enough to infer the Futamura projections in all cases, for instance.
Interesting, I never heard of Futamura projections before. Looking at the definition, it seems like the first projection (specializing an interpreter for given source code) is already handled pretty well by today's compilers, just by unrolling and inlining. And they can go even further and evaluate parts at compile time, see for example https://github.com/IoanThomas/constexpr-chip8. I can see how the second and third projections are not handled by compiler optimizations though.
The point is that compiler optimisations are a black box and not guaranteed. They can be very brittle wrt to seemingly harmless source changes (even something as simple as making an extra intermediate assignment). You are at the mercy of the 'fuel' of the optimisation passes. With staging you get to control exactly what gets inlined/partially evaluated. Of course, to get good results you need to know what to optimise for.
> With staging you get to control exactly what gets inlined/partially evaluated.
I want to stress that this is not true. Sure, sometimes it might work, but compilers can also uninline, as well as reorder the way things are evaluated. Compilers don't do a 1:1 mapping of lines of code to assembly instructions anymore; instead they are designed to take your program as input, and generate the best executable that has the same observable effect as your code. So whatever optimization you perform in the source code, it is going to be very brittle as well wrt to seemingly harmless compiler changes (like changing compiler flags, updating the compiler to a new version, and so on).
While indeed nothing is guaranteed, at this point in time the compiler is vastly better at optimizing code than humans are. If you want to make a point that multi-stage programming helps optimize code, you have to do much better than an example of raising x to some power.
I think you are missing the point a bit. With staging you can build up arbitrary levels of compile time abstractions and be sure that they will not appear in the final executable. Of course, an optimising compiler will reorder/rearrange code regardless. But it won't reintroduce all the abstraction layers that have been staged away. After enough abstraction layers, without staging even a compiler that optimises aggressively won't know to evaluate them away.
Let's put it another way: do you think there is utility in macros at all? And do you think that type safe code is better than untyped code? If you say yes to both, you must also think that staging is useful, since it basically gives you type safe macros. Now lots more things can be macros instead of runtime functions, and you don't need to deal with the ergonomic issues that macros have in other languages. For a more real world example, see Jeremy Yallop's work on fused lexing and parsing.
> Could anyone explain to me how this is different from templates or parameter pack expansion in C++?
I don't think it's any different.
> I can see the constexpr-ness here is encoded in the type system
I also see they introduce new constructs like let$, so it's not just a type system thing.
> I looked at the paper but I can't find anything related to C++.
I don't think the author needs to compare their code to C++. That said, it looks to me like it is similar to the upcoming C++26's reflection capabilities.
It's an interpreter though, not a JIT. This kind of programming language thing is a bit of a hobby horse of mine, so see the comment I just posted on this for full details:
I don't understand why you would code these explicit state machines when you can just write normal code that is much more readable. The state machine example they start with could be written as:
while (true) {
wait();
fill();
finish();
}
I don't think the approach from the article would have any benefits like less bugs or higher performance.
For a very simple example like this, your version will probably be okay, but it has its own set of problems:
* It's difficult to introspect the current state of the system. If I were to build an API that fetches the current state, I'd need to do something like add an extra state that the different functions could then update, which makes all the code more messy. By turning this into an explicit state machine, I can encode the states directly in the system and introspect them.
* Similarly it's often useful to be able to listen to state transitions in order to update other systems. I could include that as part of the functions themselves, but again, if I just encode this operation as an explicit state machine, the transition points fall out very nicely from that.
* Here there is no branching, but state machines can often have complicated branching logic as to which state should be called next. It's possible to write this directly as code, but in my experience, it often gets complicated more quickly than you'd think. This is just a simple example, but in practice a bottle filler probably has extra states to track whether the machine has been turned off, in which case if it's in the `fill` state it will switch to the `finish` or `wait` state to ensure the internal machinery gets reset before losing power. Adding this extra logic to a state machine is usually easier than adding it to imperative code.
* In your version, the different functions need to set up the state ready to be used by the next function (or need to rely on the expected internal state at the end of the previous function). This gives the functions an implicit order that is not really enforced anywhere. You can imagine in a more complicated state machine, easily switching up two functions and calling them in the wrong order. In OP's version, because the state is encoded in such a type-safe way, any state transitions must handle the data from the previous state(s) directly, and provide all the correct data for the next state. Even if you were to get some of the states the wrong way round, you'd still need to correctly handle that transition, which prevents anything from breaking even if the behaviour is incorrect.
If you try to implement an actual state machine (or even interlinked ones like TCP) in this style you will have a very bad time.
The FSM model is restrictive but this makes it much easier to exhaustively cover and validate all state combinations.
To be concrete, the kind of code you end up writing in your non-FSM approach (for a non-trivial example where you have, say, an RPC per state transition or other logic between the transitions) is
def finish():
if state == WAIT:
#error
elif state == FILL:
…
And worse, you need to validate your state in each transition, which ends up being duplicative and error-prone vs. just specifying your transitions once and then depending on them to enforce that the states are valid.
One very common reason is to make the code non-blocking. In fact Rust's async/await system works by converting the code into a state machine.
Unfortunately Rust doesn't have proper support for coroutines or generators yet so often you'll want to use a hand written state machine anyway.
Even if it did, sometimes the problem domain means that a state machine naturally fits the semantics better than using a control flow based approach anyway.
I tend to think of state machines as becoming important when you're forced to deal with the unpredictably of the real world, rather than just pummeling bits until they repent.
You've got some complicated Thing to control which you don't have full visibility into... Like, say, a Bluetooth implementation, or a robot. You have a task to complete, which goes through many steps, and requires some careful reset operations before you can try again when things inevitably don't go according to plan. What steps are needed for the reset depends on where you're at in the process. Maybe you only need to retry from there steps ago instead of going all the way back to the beginning... The states help you keep track of where things are at, and more rigorously define the paths available.
It's a formal model that we can opt into surfacing, or subsume into convenient pre-packaged idioms. For engineering purposes you want to be aware of both.
It's way easier to make sense of why it's relevant to write towards a formalism when you are working in assembly code and what is near at hand is load and store, push and pop, compare and jump.
Likewise, if the code you are writing is actually concurrent in nature(such as the state machines written for video games, where execution is being handed off cooperatively across the game's various entities to produce state changes over time) most prepackaged idioms are insufficient or address the wrong issue. Utilizing a while loop and function calls for this assumes you can hand off something to the compiler and it produces comparisons, jumps, and stack manipulations, and that that's what you want - but in a concurrent environment, your concerns shift towards how to "synchronize, pause and resume" computations and effects, which is a much different way of thinking about control flow that makes the formal model relevant again.
There's definitely a missing part of this which talks about when to use this sort of approach. The answer is often when there's non trivial amounts of stuff that happens between the end of one method and the start of the next which is in control of the external system. That said, I often argue that async/await solves the majority of that problem by implicit modeling of the state machine while keeping the code readable.
Because to reason about things becomes harder as the stakes are raised. We had to implement paxos for distributed systems in college and my partner I started over probably about three times trying to code it normally. Then we switched to just focusing on defining states and the conditions that transition between them and our solution became much easier to code.
One benefit is that if you persist the state on transition, you can create a program that survives restarts and crashes. The pattern is very useful if you have tasks with very long run times that need to be orchestrated somehow.
There are also some parsers that can be very rough to implement in an imperative fashion.
That's only because the diagram is completely wrong and your code is wrong accordingly, because it implements the incorrect diagram.
Usually a wait state has a self referential transition. You don't perform the wait once, you keep waiting until the condition for transitioning to the fill state is true.
The next problem is that you are doing polling. If you were to implement this on a microcontroller then your code cannot be called from an interrupt and let the microcontroller go to sleep to conserve power.
That’s true for everything. If you feel confident you can safely refactor and modify a state machine encoded in this way, go for it. Most of us who have seen the trenches don’t feel confident and gladly accept tool assistance.
That's just a minimalistic example. You might want to say, add a progress bar, or persist the state in an SQL database, which you can't do by letting the CPU ram through a huge code block.
While NixOS goes a bit further with it, most other distrubtions also compile everything from source, cryptographically verify that the sources they use are not tampered with, and have versioned dependencies between packages. Debian also has reproducible builds.
The problem is just that the build systems did not strip pre-compiled object files before building from source. Even with that fixed, if nobody checks the source code then you can add all the backdoors you want, and there is nothing in NixOS or any other distro that would protect against that.
"It's peer-to-peer, trust me bro!" The problem is that you are still using a website provided by a third-party to serve you the JavaScript program that initiates the transfer. It's easy to replace that JavaScript by something that just transfers a copy to the third-party itself. To be sure that the transfer is actually peer-to-peer, either the sender or receiver should run their own fillepizza server (and have verified that the source code does not contain any backdoors or phone-home code). But if you do that, you actually don't need a peer-to-peer solution anymore, it's turned into a client-server problem.
> Currently, packets are not being encrypted within the UDP tunnel so packet sniffing over the internet is possible. It is encouraged to use this over a protocol like SSH
No encryption takes the P out of VPN. Also, if you are going to need SSH to make it secure, then you can just use OpenSSH's built-in support for the tun device using the -w option.
Agreed. I will be working on eliminating that limitation soon. For now, I have removed that part from the blog post to reflect the current situation a bit more clearly.
Congrats on getting this self-hosted. However, the solution looks more complicated than necessary. You shouldn't have to set up a database server and Docker just to sync a little bit of data. Ideally, there would be a sync server implementation that uses SQLite and can just run stand-alone.
In theory you could set up a simple sync server that implements the necessary endpoints and nothing more, nothing less. Guides like these run the official, Mozilla-maintained sync server, which is obviously based on serving millions of users rather than being a minimal implementation for someone in a home lab.
One could fork https://github.com/mozilla-services/syncstorage-rs, take out the plumbing, and reimplement it all without a secondary database and the Google Cloud base architecture, I'm sure, but like with so many things, you'd first need to gather a group of people with enough interest and spare time to do the development.
reply