Yeah, we did this at a previous employer.

One example, is we had them ssh, download & extract a tarball (the Linux source, but the content doesn't matter). Sometimes, they'd gunzip to stdout. The reaction tells you a lot "lol whoopsie" followed by a quick fix: person knows what they're doing. "uh… what is going on? did I break it?" followed with general cluelessness… maybe not.

That did occasionally break tmux, though.

Part of it was "what are the specs of this thing you're SSH'd into?" and we had one candidate who was adamant the numbers must be wrong: 2 GiB is too little RAM, no machine is that small! Yeah we didn't spin up 128 GiB VM for your interview…

I never cease to be amazed at how few people really realize just how little hardware is often required for getting real work done. You'd be surprised just how much that 2GB vm with a couple cores can handle!

I started with a single 1xx MHz core and 16MB of RAM. And I'm sure some with even less, lol.

Supporting your point: Hardware is awesome if you use it wisely.

My first Linux box was a 20mhz 386SX laptop with 3 megs of RAM (1 meg on the motherboard, 2 in an expansion.) I could barely run Linux 0.99.x. The distro was SLS, and it came on 12 or so floppy disks. I quickly upgraded to a 486 with 8 megs RAM, then 20... which seemed incredible at the time (1994-ish.)

It's amazing how bloated today's software is...

If you give the person you're interviewing access to the same tools they'd have in a regular day on the job (Google, manpages, etc.), I'd say that's a fair and probably relatively enjoyable interview.

Rejecting someone because they can't recall the correct netstat syntax doesn't seem like good hiring practice, but I assume in good faith that's not what you meant :)

Yeah, I google, tealdear, "--help", and manpage anything I don't use at least once a week, every time. Usually I don't remember them otherwise, and if I think I do, I don't trust my memory that well. Only exception is if I remember enough to be able to ctrl+r them out of shell history faster than I can do those things—and actually, for some of those, I do use them often, but couldn't possibly tell you how because I only run a couple commands 99% of the time and always pull them out of history unless it's one of the rare exceptional cases—I couldn't rsync for a particular outcome without consulting a reference, to save my life, even though I use it often.

And usually you only use a fairly small set of tools that often, in any job, and which set will depend on the employer, how things are set up, and what exactly you're doing.

Oh and somehow I get "-r" versus "-R" for "recursive" wrong almost every time, even for commands I type almost daily, unless I check first. It's weird. If tools could get on the same damn page about which means "recursive", that'd be great.

TL;DR I do have a pretty good idea what I'm doing, but look like an absolute idiot if anyone watches me do it. Much worse, even, if I know they're watching and we're not in some kind of relatively high-trust relationship (so, definitely not in an interview setting).

Exactly, all man pages and google is fair. We want to see how they think not rote memorization.

I love this point. Joke: are are you hiring?

I'm quite happy to try to demonstrate how I think, but I hate hate hate leet code because A) it's not relevant to showing how one thinks and B) I've read so much dunking on it on HN that I'm now stopping interviews when they pull out the hackerrank or live code to say 'without using the library, reverse this linked list'.

That sounds awesome! Wish I got the chance to do more hands-on interviews in the mobile dev space, most of my interviews just end up being run of the mill leetcoding.

People in higher up positions like yourself will rarely be subjected to testing with tools like this. You are basically trying to remove the human from equation and industrialize the whole process.

What we're trying to do is respect peoples' time. We can get more about someone's technical understanding in 30 minutes of hands on exercises than we can in a full day of panel interviews. It's better for us as we have a much better understanding of where you're at Linux wise and it's better for you because you only need to come to two hours of interviews, total. Seems like a win win to me.

In my experience this type of interview (and coding interviews in general) usually fall into one of two categories: 1) "I learned this neat trick and want to show candidates how smart I am" or 2) "I have this bug in prod and I want to see if you can fix it for me."

If the interview was along the lines of upgrading the packages on the system, debugging why nginx was crashing, figuring out the specs of the system, etc. that is totally fine with me and I believe respectful of a candidates time. Unfortunately it always turns into something else when people need to come up with new "challenges" for canidates.

Framing a question like “a system has a high load average, what commands would you use to begin diagnosing that?” and taking that conversation as deep as the candidate can go is neither time consuming nor requires a panel of people.

No, I'm trying to make sure the person who is interviewing for a job where they will deal with computers on a daily basis appears to have seen a computer at some prior point in their life.

I wouldn't feel the need to do this if so many candidates didn't fail rudimentary tests. A SWE candidate MUST be able to write the function min(), in the language and tooling of their choice. But in an interview, a sizable fraction cannot. (The actual bar is far higher than min(), ofc., but min() ought to be trivial.)

> Why are you afraid of this?

> My org has run a hands-on technical exam with a stack of linux admin basics ... they are based on real problems we've had and the feedback is overwhelmingly "this was one of the best technical interviews I've ever had."

You essentially answered your own question.

Putting thought into the interview process and working with candidates through real problems is valuable. I cannot say the same for outsourcing or "automating" this portion of an interview using 3rd party SaaS.

We do this in our org as well. 30 minutes of troubleshooting linux issues is a good way to evaluate a candidates experience. We run it as a team exercise with the candidate so that we also get the added bonus of how do they work in a team setting, how do they communicate, etc.