A word of warning guys, I've had a look through the code and unless I've missed the obvious, there's nothing in there to change user ID; which means this would either need to be run as root, or would need to listen on a port > 1024.
In the case of the former, that's a huge step backwards in terms of security.
In the case of the latter, that would mean you'd need another reverse proxy hooked up - which would negate the need for this web server to begin with.
This can easily be fixed within Go though:
import (
"log"
"syscall"
)
const (
user_id int = 1000
group_id int = 1000
)
func secureDaemon() {
// set group id first as you need to be root to change group
err := syscall.Setgid(group_id)
if err != nil {
log.Fatalln(err)
}
err = syscall.Setuid(user_id)
if err != nil {
log.Fatalln(err)
}
}
You can also add chroot to your code if you want to be ultra paranoid:
This will need to be done before you change your user ID (as you need root permissions to chroot) and you may need to compile the Go without CGO because some of the standard Go libraries will have SO dependencies (I found this to be the case with domain name lookups).
(the above code is adapted from my own Go web framework that I'm in the processes of building)
yeah I saw it, thanks so much. I like this kind of single-purpose small libraries. The only thing is, I hesitate to depend on other people's repositories since they may get moved or change the API.
WARNING: I implemented a tiny library for that in go a while ago. Go under Linux is not safe! Linux is not posix compliant and only sets the IDs for the current thread. Go almost always spawned more threads before initializing your code which are then still owned by root and are in the threadpool for your goroutines:
I'm guessing it might depend on the point in your Go code whrn you setuid as the tests that I did on my own project, setuid changed the user id as expacted then the processes were spawned.
However you have given me reason to test my own code again and more thoroughly :-)
Indeed, but it would make more sense (in my opinion) to have the web server natively 'de-elevate' it's permissions than to expect the sysadmin to be competent enough to set up authbind manually. Particularly when setuid/setgid is so easy in Go and that every other webserver out there natively drops down to a designated webuser after the daemons been launched and ports binded.
It's easy enough to run this as a non-root user and use your local software firewall port to forward port 80 inbound to whatever high port this server is listening on.
Indeed, but that doesn't change the fact that it makes more sense having that functionality in the web server itself given its a standard and expected function in all web servers.
I tend not to sign up to many cloud based solutions as I prefer to host my own solutions (partly out of paranoia and partly because I have the server capacity to run my own solutions so it seems to make sense to use my servers where I can).
With github, it's very rare when I have anything worth contributing to other peoples gits so I've never really missed it. But I'm sure I'll end up on there at some point in the future.
I remain not-entirely-convinced (1) by Go, but things like this are turning the tide. If it is indeed slower than nginx, it is not dramatically so. Considering the amount of time that went in to writing each, the Go version clearly "wins" from the point of view of anyone thinking of writing new code.
(1) I find error handling just too tedious to get right (Edit: "I find error handling _in Go_ just too tedious...")
Looking at the benchmark (https://gist.github.com/azer/5955772), which measure suggests that its slower than nginx? I was looking at the transaction rate and the longest transaction. For reverse proxying, it seems to be as fast as nginx. For static file serving, it appears to be slightly slower. Between the two use cases, it seems like it's a negligible difference.
I would like to know how the benchmarks are generated, but my point is that it wasn't slower-enough to ignore it, and that's amazing considering how much micro-optimization has gone into nginx and how much easier the Go version is to write.
The Go version may in fact be faster, but I don't think that's necessarily fair either: if the Go version adds features to be more comparable to nginx it would definitely slow down.
> The Go version may in fact be faster, but I don't think that's necessarily fair either: if the Go version adds features to be more comparable to nginx it would definitely slow down.
Good point. That is often how alternative parallel implementations are started. A small example is written. Benchmarks are published. The new project is hailed as better than predecessor. Many jump ship. Request for features comes in. New features starts to slow down the product. Someone new comes, writes a simple benchmark and so on.
With benchmarks like this it worth asking 3 questions:
* Are they relevant. Does this represent the expected production workload. You could find this is 10x better than nginx at some feature but that feature is just not interesting to you then benchmarks are not that interesting
* Does the speed difference matter? Does it matter if the page loads 1ms faster when it was only 20ms before.
* Does it scale. This kind of relates to first question but it looks at abnormal amounts of traffic (Slashdot effects). Either from an attack or just becoming popular too quickly. If simply adds a server with more CPUs would that let Go server scale better than nginx? It might even though nginx could probably process sequential requests faster.
From experience of running big production web apps, you need to get error handling right regardless of how tedious it might be.
A recent issue where one of our developers ate an exception cost us 12 days of developer time. Another historical issue where the error handling and return codes weren't understood and handled correctly threw 500mb of stack dumps a minute and took our logging system out.
I like not having exceptions (that are widely used). It's easier to reason about your code. It's, let's just say, "stressful", to know that any line of code you write could potentially blow up and start re-winding the whole stack. Especially in a language like Java that doesn't have stack objects with destructors. You have to indent everything 30 lines in, with finally blocks all over the place. It turns into a mess. Go has a bunch of "if err != nil"s, but for some reason those are less objectionable to me. I feel in control with those. And the code runs down the page, not to the right.
How do you propose to make exceptions work in the presence of large number of cooperating goroutines? As in, there's no implicit control flow the way you have it in "less-threaded" languages.
Go has exceptions already: panic and recover. These are equivalent to exceptions (formally so even, in that exceptions could be macro-expanded to panic and recover, and also the other way round).
I don't believe that's a problem; goroutines still have a stack, they just aren't always assigned to an OS thread. At least that's my understanding.
Go even has exceptions, it just doesn't usually use them.
Can you explain what you think the problem is with supporting exceptions?
You raise a good point though, in that there is a valid question of "how would you introduce exceptions into the language / runtime". I think the answer is that the err return value can become implicit. Throwing an exception is the equivalent of "return _, err". Invoking a function that can throw an exception is done by not checking the err return; if an exception is thrown/returned and not checked then it is immediately rethrown (return _,err) in an implicit method. try/catch should be easy to introduce, although the semantics of e.g. "defer" could be tricky.
> How do you propose to make exceptions work in the presence of large number of cooperating goroutines?
Exceptions propagate up the call stack within a goroutine, and then, if not recovered within the goroutine that raised them, crash the whole program.
(I cheated a bit -- that's how go already handles exceptions, which it calls "panics". The difference between go and, say, Java is that the core and standard library don't panic as much as Java's throws exceptions, leaving the decision to panic to user code more often.)
So it's a reverse HTTP proxy, which can also serve files locally? That's an interesting thing, no doubt, but it doesn't feel like a web-server.
I wrote a flexible reverse proxy[0] using node.js, a year or so ago, and haven't missed the ability to serve static files directly - so I'm wondering what the use-case for that is? I guess proxying to rails, or similar?
Honestly I never have. Because in practise it is fast enough that it just didn't come up.
I'm sure that if I were to pimp the project properly then that would be a good thing to do, but half the fun of my proxy is to allow "interesting" rewrites, dynamically.
To benchmark I'd be too tempted try to game the results by doing a straight pass-through comparison to mod_proxy, nginx, etc. That would lose half of the appeal of the project in the first place.
The point is that it's written in language-du-jour and therefore fascinating and novel.
In fairness though, json will probably wind up being a common configuration format anyhow. And the idea of programmatically configurable system software is nice. I liked the Mongrel2 approach of having it in SQLite.
Nothing in the json spec requires that numbers are limited to those internally representable in a javascript runtime. With the right tooling, you can encode/decode very accurate numeric values. I'm not familiar with the go libraries, but python's simplejson has a use_decimal flag that interprets floats using python's Decimal module.
That said, I can't think of any reason a web server needs super accurate numbers. All the config options I can think of require nice round ints. Maybe specifying weights to individual backends to proxy to (so all the weights sum up to 1.0)?
99% of sites are CRUD apps, but we're math heavy financial software. We store a lot of reference data in configuration for performance reasons. This needs to be decimal format.
JSON is a subset of the ECMA JavaScript standard which declares only floating point.
Do "100!" on wolfram alpha and find me a parser that will load that as an example.
I'm very interested in this because this is the kind of web server I was planning to write for my work, but from a quick glance I didn't find HTTPS support.
Code organization question: is this a common convention in Go, to nest the main package in a subdirectory and put all of the 'modules' in different files but with the same package name?
(I'm more used to the Clojure namespace/filesystem symmetry, so this is somewhat new to me. I like the more shallow project tree but it's not immediately apparent where things are defined.)
You have to have your main() in package "main" and all your files in one directory have to be in the same package. So when you want a main() and your logic in a "real" package, your have to seperate them.
json format is too limited for a web server configuration if you want to be the nginx alternative. Think about how you would represent conditional statements, how to include and reuse external configurations and how to have comments in json.
In the case of the former, that's a huge step backwards in terms of security.
In the case of the latter, that would mean you'd need another reverse proxy hooked up - which would negate the need for this web server to begin with.
This can easily be fixed within Go though:
You can also add chroot to your code if you want to be ultra paranoid: This will need to be done before you change your user ID (as you need root permissions to chroot) and you may need to compile the Go without CGO because some of the standard Go libraries will have SO dependencies (I found this to be the case with domain name lookups).(the above code is adapted from my own Go web framework that I'm in the processes of building)