You've been arrogant for years and you know what? So have I. I and many, many people in this industry have been doing this for years.
What's different from you and many other people is that you admit and face your bias rather than justify it.
I'm seriously curious what google interview board members have to say about this study. People like Gayle Laakmaan have been saying things to justify the whole process for years; but now that there's actual science, what do they have to say in the face of science?
I interviewed a lot of people while I was at Google, and sometimes the people were clearly, clearly too anxious to interview. Like one fresh-grad who's hands were shaking. I thought about how much pressure the kid must have been under.
You try to help them relax, but you don't have much time, and the whole thing is just so unnatural.
Other places I've since been at, and interviewed with, give you a laptop, a problem to solve, and some time. You can focus on the problem instead of the situation much better, and I think that really helps. As an interviewer, it also helps keep some distance from the interviewer. It's too easy to try to micromanage the interview and keep the candidate uneasy. But there's a natural "don't interrupt someone when working" instinct that keeps interviewers more distant when the candidate's programming.
Coderpads on video conference, I think, are pretty good when the candidate has a good space to work in. They're in comfortable territory and you can see them by their keystrokes. The interviewer feels like they're getting more accurate data about how the candidate really operates. I just wish it could do keyboard shortcuts better -- emacs users like me hate seeing Ctrl-N bring up a new window.
The issue of IDE/compiler/keyboard/keybindings is also there, but that's easier to work with.
Microsoft interview. I was just graduating college, and was super anxious walking in to my first interview. The interviewer realized that writing on the whiteboard was going to be awkward for the problem he was asking me to solve so he let me just sit down at his computer while he hung out to to the side and watched me work in notepad. I think that was the best way to do that interview, because the second I started writing code my brain flipped in to code-writing mode and everything relaxed. One of the better interviews in my career.
Nowadays that's "favoritism" in an interview and it wouldn't fly. Everyone must pass through the same meatgrinder (that, surprise, fails people other than white men more often).
She will just say that the process is designed to minimize false positives and the expense of more false negatives. A bad hire is more expensive than a no hire.
The problem is this isn't what gets you the smartest people on the planet. A small number might be interested in DS&A to that level, but most are not. They will learn enough to know how to google what they need and move on to what they are interested in. Smart people are constantly getting job offers from coworkers and bosses who move to other companies if they don't start their own.
The few smart people who do put in the effort necessary to go to the FAANG companies always leave after getting the golden sticker on their resume. They leave behind them a residue of SAT preppers who have PhD's in inversing binary trees.
From my experience it won't get you the smartest people on the planet, but it will get you people who _think_ they are the smartest people on the planet.
At Google I've worked with some very humble but also super intelligent awesome engineers, people much smarter than I. Definitely the majority of my interactions. But I've also worked with people who clearly took the "Google hires the smartest people" company line, and their own success at it, a little too personally.
I don't think it's a good message to send. Although to be fair I haven't heard internally in a while.
Also I don't know if it's really a gold sticker anymmore. Nor do I think it's true that the smart people leave. Some do, but honestly, there are some damn brilliant people at Google who have found their brilliant corner and produce brilliance there.
Or some people here who are just brilliant at playing Big Company. Unfortunately there's more of that all the time.
> She will just say that the process is designed to minimize false positives and the expense of more false negatives. A bad hire is more expensive than a no hire.
That's what they keep saying. However, it remains to be proven that regurgitating answers on a whiteboard in 45 minutes implies that the candidate is not a false negative, or even a true positive to begin with.
Someone who able to immediately understand and write out the solution to 80% of hard DS&A questions is at least able to write code. The question is if this is honestly better than generating your own FizzBuzz. I don't think so. I have heard rumors that Google has studied employee performance predictions based on interview score and found no correlation. I would bet this is an open secret and no one knows what to replace it with that isn't enormously expensive.
> no one knows what to replace it with that isn't enormously expensive
That's the crazy thing about this. FAANG has near-infinite resources, relatively speaking. Google could be A/B testing their interviewing process, experimenting new approaches, experimenting (as the OP study did) with eye-trackers and other tech to try to gain insights on candidate behavior. They have the "engineering for the sake of engineering" and "innovation for the sake of innovation" culture that allows for moonshots and boondoggles. I don't think they're exactly known for following "if it ain't broke, don't fix it", given how many times they kill off products only to recreate them later on, especially their chat apps.
Yet they stick to the traditional whiteboarding method because-- why?
Maybe Google has studied all of those things and found whiteboarding to be the best predictor. The company seems to be doing well, so something must be working with their hiring. They dropped GPA/college requirements so they're clearly open to non-traditional backgrounds.
> The company seems to be doing well, so something must be working with their hiring.
People use this argument often, but is technical competence of engineers- specifically at passing their interview process -the explanation for their economic success? And not market penetration, near-monopoly positions, great UX, etc.?
Not to mention, as we've seen in the high-profile Facebook SDK crashes this past week, share price doesn't necessarily mean product quality.
It means you're someone willing to spend two to three months on your life sucking a little dick for a chance to work at FAANG... the companies so good they've recommended 15 vacuum cleaners after purchasing one. How many hands do they think I actually have.
Boy, am I the only one who thinks working at FAANG is pretty great and worth the time practicing the interview?
The compensation is much better. The management is much better. The work life balance is much better.
I moved from SEA. Everything is worse there.
I read this and feel like people have unrealistic standard. FAANG isn't good enough? Working there for a year probably put your wealth of life at 0.1% of the world. That's not good enough?
We don't care about reality, merely perceptions. "Well I humiliated the person for an hour in front of a whiteboard" is the new "Nobody gets fired for buying IBM."
intelligence is multi dimensional. Even more so at a working environment. I can't believe algorithm intelligence is a relevant skillset for 99% of the projects. Including projects inside google.
Gayle Laakman specifically has a business empire dedicated to help people get through the process. Regardless of the actual merits of the process her opinion is going to be biased.
Quite a racket, being part of setting up this gauntlet and then selling books on how to get through it. Some might call that a conflict of interest, but there's no such thing as ethics for the nobility so to hell with what the peasants think.
I don't think it's relevant. Her business is designed to get you to game and pass the interview regardless of whether or not the interview is effective.
Therefore her business interests are orthogonal to her opinions on "interviews." She may still be biased, but I don't think business interests color her bias.
I'm not a googler (but I've interviewed there) and I disagree somewhat with the original premise. I think whiteboarding does filter out bad candidates but also filters out good candidates who fail the anxiety test. Google, with their massive pipeline, probably doesn't care about the false negatives enough to change; they still find enough people to hire. It's the companies with smaller pipelines that need to scoop up these good candidates.
I used to be reasonably good at algorithmic stuff in university 20 years ago. Since then I hardly ever practice. It's not that I am bad at them, now, but its just so hit and miss whether an answer will come to me in time, you might as well toss a coin. If I was doing algorithmic stuff every day I have no doubt I would be better. Usually what happens is I get out of an interview and a an answer (or a better answer than I gave) pops into my head. Like I say, might as well toss a coin.
In general, hiring practices probably optimize for filtering out bad (or at least somewhat speculative) hires over passing on a potentially really good candidate (especially if there are any concerns to go along with the overall positives).
Of course, if the pipeline is massive (whether it's jobs, schools, etc.) this tendency gets amped up even more and anyone who doesn't come across as pretty much perfect on all dimensions--whether they are or not--is going to get dinged.
I've trained for interviewing twice at Google. I don't volunteer to give the interviews, though, because the interview I was trained to give, I would never pass.
And sadly, this is common and recognized at Google, was mentioned in interview training, and is just said to be part of the fact that the bar gets higher, or something, mumble mumble.
Every year I've noticed the questions get harder too. A hard question in 2012 (off the top of my head, 2 pointer solution to linked list cycle detection) might be considered an easy question today.
From what I have read from Gayle Laakmaan, they know this throws away many good engineers, but it is acceptable to them because it doesn’t let bad ones through.
I've definitely repeated that line in the past to justify tech industry interviewing practices - but sometimes it goes beyond "setting a high bar" to something more toxic: an interviewer using the interview as a chance to show off how smart _they_ are, rather than assess the candidate; deliberately creating high-stakes "pressure cooker" environments because, you know, that's just how we work here and they should get used to it; and so on. These things then get rolled into the same justification: after all, we're an exclusive club of ultra-smart 10x ninja pirate Jedi, which of course means we should have the default assumption that others don't belong in this club.
I wonder now: for every "bad one" not let through by this sort of process, how many other "good ones" look at it and say "no thanks"? How many hear about this sort of hazing and are dissuaded from applying in the first place? How many experience it one too many times and just quit the industry altogether? How many of those "bad ones" are even objectively bad, and not just having a bad day or intimidated by a process rife with both intentional and unintentional hostility? In other words: are we actually assessing what we claim to assess with _any_ predictive accuracy, and is the collateral damage to company reputation and the pool of available candidates - a damage often hidden to the individual companies that impose this process - even worth it?
I think it goes far beyond that. I'd bet dollars to donuts that you will find both protected and unprotected groups over represented in the false negative pile. Do women have more test anxiety then men? If so, guess what your process is selecting for. Do minorities have more self doubt than majorities. Same thing. And so on.
And that doesn't address the file drawer effect. I'm 54. I can code a binary tree, hash table or what have you, if I have to, but I am not as practiced as somebody fresh out of school, because that heavy lifting is done by libraries, and I'm busy contributing original mathematical algorithms that no one ever asks about because they don't have the background to understand the explanation. So I don't go on FAANG type interviews. I know I will fail, and if I don't my offer will be predicated on that apparently lower performance. At best I will have an utterly miserable experience. So I self select myself out, mostly on age.
> I can code a binary tree, hash table or what have you, if I have to, but I am not as practiced as somebody fresh out of school, because that heavy lifting is done by libraries, and I'm busy contributing original mathematical algorithms that no one ever asks about because they don't have the background to understand the explanation.
This made me chuckle, because it's absolutely how it is.
Some other comments here talk about how nobody creates new algorithms these days unless they are a researcher. But it's not true at all! Like you I'm creating new algorithms and other tricky techniques all the time, but I'm not fresh on the stuff I learned at school... or so I thought.
I have been really surprised to find some screening interviews recently asked me shockingly trivial questions. So simple that I stumbled over myself trying to simplify the answers, thinking "I know too much about this topic and need to keep it simple", and "can these questions really distinguish candidates?".
I did the TripleByte online test recently and found the questions much simpler than I expected, including those in languages I've never seen before. That is not TripleByte's reputation. I see people writing about how difficult they found the questions. (Admission: I didn't score all 5s, but I can't figure out why unless it's timing as I think I answered them all correctly.)
So I'm thinking, perhaps it's not so bad, people just make it sound bad because there's a wide variation in people's knowledge, abilities and expectations.
If you are thinking you might apply for something like a FAANG or other hard-reputation company, and feeling put off by the horror stories, I would say, just take a look at the old stuff a bit for a refresh, then give it a try and you might be pleasantly surprised to find their reputation is because people less skilled than yourself found it hard. Their "hard" might not be hard for you.
You'll probably still fail the interview if it's a FAANG because they are so selective, but I would bet my dollars to donuts that it would be more refreshing than miserable if you're not attached to passing.
I know your point isn't about yourself, it's about bias in hiring, including bias due to perception by the candidates, but I wanted to address that side point about feeling there's no point applying. That sounds like anxiety to me. (I have it too, I'm trying to get over it.)
>And that doesn't address the file drawer effect. I'm 54. I can code a binary tree, hash table or what have you, if I have to, but I am not as practiced as somebody fresh out of school, because that heavy lifting is done by libraries, and I'm busy contributing original mathematical algorithms that no one ever asks about because they don't have the background to understand the explanation.
Ooohhhh this is frustrating. Nothing is more frustrating than telling an interviewer about something cool you've done, and realizing they don't understand, but also don't care.
Yes, I remember one guy asking what I would when a page was going slow. I explained how I would work backwards checking the network tab in the browser, make sure it was the endpoint that was the bottleneck, blah blah blah to the database. He just wanted me to say "use explain" - which i would have got to if he had bothered listening a couple more minutes. Then he wanted me to do a "challenge" that was expected to take around 10 hours. No thanks, you didn't convince me I want to work for you.
But they're not aware that the filter is just anxiety.
Google is basically using anxiety to filter good candidates and eliminate false positives which works in a sense but is still highly illogical.
Why not use a technical filter to filter for technical candidates? Anxiety seems like a pointless filter.... how does that even eliminate false positives?
it would seem it might be 'letting in' a whole load of people who don't have appropriate anxiety responses. There are some situations where anxiety is perfectly normal, and perhaps even useful, but if you screen out people who have normal anxiety, what impact does that have on your company operations and culture?
I'm a pretty anxious person, so much so that I've pretty much retired instead of doing anymore technical interviews. I had a discussion with a very anxious, but brilliant software developer a few years back and we came to the conclusion that as long as we don't let our anxiety get too far out of hand it's actually kind of a superpower in the job setting. That's because that niggling anxiety you get in the back of your head as you're coding generally makes you more careful about how you're designing and testing your code. You're more careful about security, safety and correctness. Whenever I get anxious while coding I stop and ask myself if maybe it's a message that I need to listen to - maybe it's trying to tell me to tread carefully because I'm entering an area that's potentially problematic.
People without that niggling sense of anxiety always in the background, the folks who are easily going to ace the programming interview because their anxiety levels are so low, those folks, I theorize, are potentially not going to be so careful. Now you could argue that that's a plus in many situations - a startup that needs to get code out the door right away, for example. But it really depends on the domain. For critical systems in applications like healthcare, avionics or robotic control I think the anxious coder is the one you want.
Companies that completely weed out the anxious programmers, as you imply, will have a different culture with perhaps too much emphasis on risky behavior.
> But they're not aware that the filter is just anxiety.
Could this be deliberate? As working with people with anxiety problems is often a pain in the ass as they won't report problems for fear of seeming stupid or ask for help.
Confident people are far easier to work on a team with.
This is essentially the point of a lot of hiring and performance evaluation theory.
E.g. it's acceptable (but not ideal) not to promote talented commanders. It's catastrophic to promote someone to General who isn't ready or is unsuited.
Software engineers are not commanders or generals and it is NOT difficult to fire someone in the United States of America. Everyone in this industry has a at-will contract.
However, high rates of employees getting fired is bad for culture. People are always anxious about their jobs - if they see a few colleagues fired for nonperformance, a number of them will be driven to anxiety. (I am one of these people, and being anxious about job security ironically makes me much worse at my job)
One of the things I've liked about freelancing/contracting is that I know I'm going to get "fired" (we call it "successful completion of the project" but its the same thing, just that they would like to have me work for them again). And since it happens on a regular basis, I also am practiced in finding a new job and I know about how long it will take and what types of prospects to look for. It's really reduced job anxiety for me, although I never had too much.
Although as a counterpoint, it's also easier to get work, because part of the value I'm providing is that you "fire" me whenever you want and no hard feelings. So nobody does "leetcode interviews", because it's not like we're getting married, like an employee. So there's much lower risk to hiring someone, because a bad hire doesn't infect anything, because you expected it to be temporary in the first place.
Right but I also do think that FAANG is over-estimating how many of and how bad these "bad apples" aka. slighly above average developers would be and how terribly difficult all the work they do is.
When you have your pick of the bunch, of course you're going to look at slightly above average as "bad".
I'm not sure how much engineer skill matters though. One slightly above engineer might be fine. But if half your org is made up of slightly above average engineers, I feel like there will be a knock-on effect.
I don't know maybe my view of slightly above average is inflated but I think of that bunch (which I might be a part of) as being easily pulled in by good technical leadership. If only to make their life ultimately easier / less painful. You don't need 60-70% amazing engineers to do that. But I guess they can do whatever they want. They have infinite money and unfortunately a good reputation (in many cases unwarranted). They've become like the Harvard or Yale of tech. Oh you went to Google, great come on in and join us.
And yet, time and time again we see that Generals promoted during peace-time rarely have the skills necessary to lead men under intense combat conditions. The best generals are identified on the battlefield.
Coming from a military background, I can confidently say that promotions at that level are more the end of a political process than a true assessment of skills...
Over the past few years in tech, I have interacted with and observed (as a fly on the wall) a wide variety of recruiters, hiring managers, etc. from companies as small as a dozen (usually around the time a founder/CEO brings in external "support").
There exists an incredible amount of arrogance and ego among them by-in-large. The best are humble and curious while maintaining clear and direct communication. To be fair, they are often tasked with making important judgement calls sans complete information, though that's the fault of the leadership/culture as to a poor approach.
At the start of my third year as a student, I biked through a record-worthy rainfall to my first interview with a small local games studio. Showed up completely soaked, but sat down relaxed and feeling reasonably confident. (I had done a handful of unity and opengl projects.)
The interview started with a bit of small talk, I chatted with one of the guys about Brood War and Starcraft 2. Later during the technical interview they asked me about the difference between private, protected, and public and said I was the only student they had interviewed who had answered correctly which was wild and honestly stunned me for a moment.
They liked how in one of my examples of scripting something I had written some fun dialog and I mentioned I did writing as a hobby which seemed like a plus. I talked about which classes I enjoyed the most and how they were challenging/interesting.
I did not get a job offer and learned from a friend that I had come off as "depressed and disinterested" (I don't think they realized he was going to relay that info to me...) All I can do is guess that it's a combination from showing up completely drenched and sharing how I enjoyed those "challenging" classes made it seem like I would get quickly bored of scripting...
After that, I bombed an interview for a QA role because I went into it completely unprepared for just how much it would differ from a programming interview.
Point being those first two experiences got in my head and I basically had anxiety over anything job-hunt related for the next 2.5 years.
When I did get a job, I was extremely happy with the interview process (even though I felt I did poorly on it). Here's more or less how it was structured (TL;DR):
- Office tour/chat with a programmer who had referred me
- quick 5 minute introduction to senior programmers in charge of the technical interview
- 1 hour alone in a meeting room with a laptop and 4 written questions (a generous amount of time)
- ~15 minutes reviewing my answers with the senior programmers
- importantly, they gave me a chance to talk about my answers and when I got something wrong they would simply state that it had an error and see if I could spot it
- 15 minutes on C++/memory/performance/behavior quirks (important stuff in AAA games)
- ~30 minutes talking about stuff I had worked on
- occasionally they would mention something related I hadn't heard about and explain it while gauging how well I could follow along
Basically, based on my own interview experience/anxiety if I had to choose an interview method, it would be very similar to what I just mentioned. Seeing a familiar face and the introduction followed by the alone time did a lot to ease my anxiety. The time where I felt most comfortable was talking about the stuff I had done because I was very familiar with all of it/it was easy to talk about.
The process seemed less focused on where I had gaps in my knowledge and more focused if I had a decent amount of knowledge in general and if I had the ability to recognize and correct the gaps in my knowledge.
Sorry if it's a bit of an info dump, but it's something I think back to a lot.
What's different from you and many other people is that you admit and face your bias rather than justify it.
I'm seriously curious what google interview board members have to say about this study. People like Gayle Laakmaan have been saying things to justify the whole process for years; but now that there's actual science, what do they have to say in the face of science?