The case against employee engagement tools
If you’re working at a big organization you’ve probably heard or used an employee engagement tool. Here’s how it works:
- It regularly sends online surveys to the employees to measure their engagement.
- Then it aggregates that data and presents the management team with beautiful reports about different engagement categories.
The core idea is that the surveys will provide reliable data for spotting and reacting to organizational problems. The reports look something like this:
This post is about my experience using Officevibe at two teams for a period of 2 years and eventually quitting it (not truly but we’ll get to that). Even though this post is about Officevibe, you may have similar experience with any other survey-based employee engagement tool like Peakon, SurveyMonkey, Cultureiq or a bunch of others.
Lazy
One of the biggest selling point of employee survey software like Officevibe is the convenience of gathering the data for you with a standard set of questions. So as a manager you don’t constantly have to go around, inspect employees and most importantly quantify their feelings about working in your organization. That is, if you can trust those surveys on the first place.
I happen to have done my MSc thesis on measuring human emotion in an engaging game at Philips, Netherlands and published a scientific paper from our findings. As anyone who has worked in the field would tell you, it is impossible to measure human emotions through sensors (like Galvanic Skin Response, Heart rate, body tracking or EKG) let alone self reports. The truth is: we humans are notorious at identifying or reporting our emotions specially on retrospect.
In Thinking, Fast and Slow the Nobel prize winner Daniel Kahneman goes into details of talking about two systems that drive almost all our actions:
- System 1: fast, instinctive emotional, also called the elephant by Kevin Simler & Robin Hanso in the Elephant in the Brain (your subconscious)
- System 2: deliberate and logical (your conscious self)
The problem is: System 2 is often in a position of rationalizing the emotions that has been created on a subconscious level. I highly recommend The righteous mind by Jonathan Haidt on this matter because he has very interesting questions that he has asked his subjects from different cultures over the years and they reveal a lot about how helpless we are to understand or control System 1 (hence “elephant”).
There’s no reliable correlation between what people respond to a survey about their experience in the organization and what’s actually going on at ground zero.
My experience: one team’s management put “a high Officevibe score in category X” (X being one of the problematic diagrams) as a key metric for team in the OKR. Even though the team was told that they should not let this coincidence affect their response, at the end of the quarter the team flat out was the second best in that metric across the entire Officevibe ecosystem (not just the company). I suspect the first team was just a test account at Officevibe itself! ☺
Using employee engagement tools is just too lazy. If you are in a management or lead position, your best chance of get bullshit-free honest feedback is to get out of your chair and talk to your directs. Technology may advance to an extent that it’ll rule the humanity, but for now good old face to face communication is our best bet at seeking the truth when it comes to organizational behavior.
Treat each employee as an individual, not as a statistics generator. Officevibe is not Datadog for humans.
One way Officevibe could help without breaching the privacy, is to suggest people with low scores to contact their managers and even give them tip for how to approach the problem. But that’s the hard part that requires psychological expertise and not every “software company” is willing or has the resources to go that extra step.
Language
For an online survey to work across the borders, it should ensure that the respondents’ understanding of the question is exactly the same no matter what language they speak. Officevibe (based in Montreal, Canada) supports English, French, Spanish, Portuguese, and German none of which were the native language of the majority working at that company. Many Europeans can speak English but the levels vary quite a bit. As someone who has worked with international talented people in the past 11 years I can testify that often what I said is not what is heard and what’s said is not what I heard. There are many techniques to get it right and I’ll write about it in another article but it’s safe to say that you should leave a big error margin when relying on surveys that’s not done in one’s mother tongue.
In my case, it came up several times both for employees and managers that people didn’t fully understand what a certain question or statistical report even means despite the fact that Sweden is one of the top English speaking countries as a second language and there’s adept explanation about those stats on Officevibe’s UI and Support pages.
One way Officevibe can avoid this problem is to translate their questions to other languages. They may market their product in Sweden (a language that they don’t support) but there are sure people from India, China, Pakistan and all corners of the world working here. Their home country, Canada is no exception:
Meet bob! He has 1.8 hands!
Most people have 2 hands but have you ever seen a person with 1 hand? How about a person with 1.8 hands? It’s abstract, but as a thought experiment, can you pause and imagine such a creature? Is one hand the “normal” length and the other is like 20% shorter? Is one hand stronger than the other? Thicker? More “able”?
Whatever you imagined a person with 1.8 hands to be, there’s no such thing! But that’s exactly the kind of statistics that you get from such employee engagement tools. They go to great extent to anonymize the data. In case of Officevibe:
only aggregated results are presented in Reports to give managers a sense of what’s going on in their team (source). if a team answers 20 questions about happiness, a manager could see that the happiness score is low. However, they will never be able to see the 20 answers that made that score so low (source). Reports are not visible to managers if the group contains fewer than 3 participants. Regardless of the size of the group, a score for a Metric is only visible when there are at least 3 responses from 3 distinct users over the last 3 months (source).
And that’s exactly the point that works against the main reason of using the employee engagement tool: you want to increase engagement (for example happiness) because you want your employees to stay, but the average aggregation will blind you to the outliers and you’ll find out when they quit!
There’s no escape from the ineffectiveness of aggregated scores without ruining the privacy aspects of the product. The best reaction to a low happiness stat is to say to the team that you know someone is unhappy and they should feel free to come and talk to you, which in a healthy organization they would do anyway with or without employee engagement tools!
In my experience, we lost several prominent members of the team while this tool was being used. It felt like the managers were so reliant on the beautiful colorful Officevibe diagrams that they were blinded to how the individual team members were feeling. The managers were focused on the Officevibe’s “stress” score, a problem that in my experience wasn’t the most immediate issue because we had a pretty flexible working hours and a family friendly company culture.
Averages prevent spotting the outliers. The outliers are exactly the ones you want to focus on. Sometimes it’s better not to use a tool at all, in order to avoid the illusion of knowledge where there’s none.
There’s only one thing that Officevibe breaks all those privacy rules to put you on the spot: when you don’t use their platform!
If you don’t respond to Officevibe Surveys for more than a month, your status changes to ‘’inactive’’ in the platform. Because of this status (which is only visible to a restricted number of people inside your organization), you may get an email nudge to remind you to participate (source)
So much for anonymity
This is obvious but I have to spell it out. There’s a huge information asymmetry when using employee engagement tools like Officevibe: the employees can’t see each other’s feedback (even about each other) but the managers can. In fact your immediate manager gets an email whenever you submit an “anonymous” feedback (knowing that it is from their team) and in my experience, they guess who it is. At least they try to!
In fact at least on one occasion a colleague of mine casually talked about an awkward conversation they had with a manager who thought he’s following up with them about how they experiences our product while I know for a fact that: 1. they don’t use our product, 2. I wrote that very feedback! My colleague quit, not just because of that, but there was a whole class of reasons that could be avoided has the feedback was treated anonymously!
This sparked me to learn more about how Officevibe handles anonymity and to be fair, they’re doing everything they possibly can:
To protect your anonymity, Officevibe displays your anonymous feedback in groups of which you are a member if the group satisfies the “5 users rule” (source). In order to best protect anonymity, there is a time delay of up to 15 minutes between feedback being posted and the manager being notified. This is to prevent a scenario where a manager who is in the vicinity may see an employee on the Officevibe platform, and would not be able to connect any new feedback received with that employee. (source: my email correspondence with their support team)
The reporter might be anonymous but that doesn’t prevent them from mentioning someone by name.
One of my most bitter experiences with Officevibe comes from the time I shared the wrong link on social media. Like most people, I have added many of my former colleagues or professional connections on LinkedIn. One day, I shared a video that was going viral on another social media. It was a short video showing an immigrant doing something out of ordinary in Stockholm subway (Immigration is a very sensitive topic in Sweden. If you want to learn more about why it’s a taboo, have a look at this paper written by John Åberg Senior Lecturer at Malmö university). One of my “links” on LinkedIn found it distasteful so they used Officevibe to anonymously report me. 2 hours after sharing the video I was at a private meeting with my boss’ boss. After an initial interrogation about my intention of “misrepresenting the company” on social media, he gave me the benefit of the doubt. I apologized and removed the link. I had a guess for who reported me because according to LinkedIn a potential suspect visited my profile at those 2 hours, but I decided not to let a guess consume me. I gave them the benefit of the doubt too. After all, my action wasn’t correct, so there was no point in holding a grudge.
To be clear: it wasn’t smart of me to share that video. The video was more of a Facebook material (a social network I don’t use due to its lack of quality). I should also mention that the boss approached the situation very professionally and avoided drama. But I got the message. After that I’m extremely careful about what I share on LinkedIn and have put a disclaimer on the top of my profile saying:
I don’t represent any company or other people on my personal profile.
Did I have to be reported anonymously? Probably not the most effective way for giving feedback to your peers. But if the tool exists, why not use it? It’s more convenient than a frank face to face conversation. Even better if it reaches higher up in the career ladder right? 😒
That was when I had enough. I couldn’t use the platform because I didn’t see any value on using it. I didn’t see the point: if I were to give feedback to my peers, I would rather do it directly. If I wanted my team to take action, the retrospective would be a better forum. My manager? We had 1:1s! There was absolutely zero use for a fancy semi-anonymous survey-based “employee engagement” software (these things are not free by the way. Officevibe charges 4$/mo per user!). So I decided to quit it and close my account, but here’s when things got interesting!
One way Officevibe could reduce the problem (as I’ve suggested them) is to put a banner on top of the anonymous feedback mentioning that despite they do their best to respect people’s privacy, the managers can still guess who wrote the feedback. In fact they should mention the names of the people who have access to the report. If you see your manager chain over where you’re writing the anonymous feedback, you’ll be considerate.
Also when it shows the anonymous feedback to the managers they should have a banner saying that they should respect the choice of the employee to stay anonymous and instead of trying to put a face on the text, they should read it objectively. Their current policy is fair but not well communicated:
there is only so much that can be put in place to prevent misuse. It is the responsibility of every manager to ensure that the principles of anonymity are being upheld. Please note that under our terms of service, companies must respect the employee’s right to anonymity. For your reference, you can review our Terms of Service here: https://www.officevibe.com/terms (source: my email correspondence with their support)
Can’t quit?!
Fortunately I live in EU where GDPR gives the users complete control over their data (this only applies to businesses, the governments can bite more than they can swallow). Since the GDPR has come to effect about a year ago I have been happily using it to get rid of any digital footprint I don’t like and most importantly take control of my digital life. So I contacted Officevibe:
My company uses Officevibe. I no longer want to be part of this platform. I would like to request a data erasure in accordance to GDPR regulations (I live in Sweden).
“Data erasure” is the technical term for requesting to wipe out all your data at your request (if you want to learn more about GDPR terms, I have another article explaining them in plain English).
This was the response:
I have received confirmation from our legal team that the right to erase data is toward the data controller (which is your employer), not toward the data processor (Officevibe). As such, under our terms of service, we are obliged to seek confirmation from the data controller before deleting any data.
Essentially, the survey site refuses to delete your answers unless your employer asks them. Let’s make one point clear: thought my correspondence with Officevibe, they’ve been extremely professional and helpful. Like most companies they want happy customers and they did their best to navigate the issue within their possible boundaries. Allan, who I’ve been mainly talking to, said that he’ll take my feedback to their PMs and even erased a couple of my “anonymous” feedback on my request (but still no account removal).
That doesn’t mean that their product is the silver bullet for all organizational problems though. I got in touch with one of the internal managers who had account management access and explained basically what I wrote on this blog post.
He was very helpful, listened to my arguments and agreed that the platform is not being used as it was intended. However he asked me if it is OK to stay in the platform. I knew if I stay, I’ll be flagged as an inactive user and probably would be forced to participate, but I decided that if push comes to shove I’ll just do the bare minimum to get it over with. After all, I loved working there and didn’t want to loose my job because someone with power didn’t like my disobedience over a minor issue. Turned out they never asked me to fill in the surveys.
One way Officevibe could solve the problem is to allow the employees delete their account but inform the management team. After all, it was me (the employee) who created the account on their site so it’s just not fair business that they ask me to go through the hoops to remove my personal data from their site.
Conclusion
Not everyone who is not using Officevibe is as vocal as me or makes those mistakes, but mistakes are how we learn. If you find these employee engagement surveys non-productive, know that you’re not alone. If you’re a manager speculating about using one, maybe this little post gives you some feedback from the “employee” side of things.
In the end it’s about the choice. Do you prefer face to face conversation or asynchronous written form? Do you believe the employers should allow 3rd parties to gather, store and analyze information about their employee’s feedback and “emotions”? People are different and your mileage may vary but I believe one of the main reasons to have managers in an organization is to understand and lead people. Otherwise, you can replace the managers entirely with software!
Feedback
I received very interesting feedback about this post that would like to share.
One of my ex-colleagues offered a counter-argument from the management perspective:
A tool is a tool; and it can be used in many ways [...] I definitely had a different perspective than you [...]. You are only looking at the micro-perspective but not considering where [the company] was when we started this and the fact that the tool made people actually talk about engagement and motivation in ways unheard of before. But yeah; the tool does not replace trust and direct relationships; it is a compliment.
Another Engineering Manager said:
I found your article (as usual!) very thoughtful and interesting. At [another company name] we are using &frankly: it has all the flaws of Officevibe minus the sensible UI / data representation & browsing.
Personally I ended up not using it for my team and having a standard set of questions that I go through verbally during every single 1:1. It takes about 30–60 seconds to go through them (we use a numeric score system) and if a score stands out particularly (or the engineer wants to articulate more) we follow-up with a discussion.
I like to think that in my team we managed to build enough rapport to displace the need for anonymity . As a manager you anyhow want to work 100% with the the person who has an unattended need, not 8% with all your team members, of which only one has that need.
One thing that I particularly like about this way of doing things is that — for as much as I like to plot graphs over time and average the values in the team — the entire exercise feels highly unscientific, and more of a conversation starter or a very rough representation of reality. In other words: it’s not data that pretends to be actionable, it’s data that requires investigation and conversation to be even only understood.
That said, the problem with Officevibe and other tools is really how people use the them, rather than the tool themselves. Officevibe and others may suck to inform a manager actions towards any of their team members, but I hold that there is a lot of value for [company name] to be able to get — with minimal effort and in under one week — a general sense of stuff that works and stuff that doesn’t.
Tip: if you live in the EU, you can ask your employer and Officevibe for data erasure after you quit your job.
Liked what you read? Check out my 4P system for career search.
