OpenAI and Jony Ive: Shaping the Future of Human-Computer Interaction

Earlier today, OpenAI posted a video on social media announcing the $6.4 billion acquisition of io, Jony Ive’s “nascent hardware company” as well as a collaboration with his design studio LoveFrom. The video showcased a joint interview over coffee at Francis Ford Coppola’s Café Zoetrope about the future not just of AI, but of our overall relationship with technology.

Ive is famous for these sorts of “deep thinker” musings, challenging the fundamental nature of the human relationship with the physical world of product design. He couldn’t have picked a better person to go into business with than Altman, who co-founded OpenAI nine years ago when AI was more Skynet than helping write your term paper, and then landed center stage after the November 2022 public launch of ChatGPT. They are both futurists more interested in pushing boundaries than providing attractive quarterly earnings reports.

In the video, Altman mentions how the human-machine interface has remained relatively unchanged for almost fifty years. He described wanting to ask ChatGPT something from an earlier part of their conversation: “I would get on my laptop, I’d open it up, I’d launch a web browser, I’d start typing, and I’d have to, like, explain that thing. And I would hit enter, and I would wait, and I would get a response.” The launch of the iPhone nearly 18 years ago provided the most recent form factor alternative, but still relies on an app and (for most people) typing into a text box. Apple’s Siri provided another alternative (albeit a limited one) with its launch in 2011, but it wasn’t until over a decade later that OpenAI released Advanced Voice Mode which is the first real AI-powered voice assistant.

We’ve had to wait a long time for this. Star Trek had a voice interface with the U.S.S. Enterprise computer as far back as the first season in the 1960s, and it’s been a staple of the Enterprise computer ever since. The lack of such an interface provided a comic moment in the 1986 movie Star Trek IV: The Voyage Home when Scotty picks up a mouse like a microphone and attempts to verbally greet a Macintosh Plus at Plexicorp. When told to use the keyboard he retorts “a keyboard… how quaint”.

Indeed, four decades later, Scotty would still find our relationship with computers mostly confined to a “quaint” keyboard with voice assistants still rather a curiosity for setting cooking timers, asking for map directions or asking to call someone while driving. One modern alternative, the Rabbit r1, ditched the keyboard entirely, but received mixed reviews despite winning major design awards. OpenAI’s acquisition of io puts Silicon Valley heavyweights under the same well-funded roof. Altman and Ive bring both visionary ambition and the credibility to attract funding for a moonshot. 

So, what would this new human-machine interface look like? OpenAI will reportedly not be releasing any of its collaborations with Ive and LoveFrom until 2026, but we can guess. Traditional peripherals like keyboards, mice, and even touchscreens will likely become fallback options rather than primary interfaces. The replacement will likely be the most natural mode of communication for human beings: your voice. 

Humans have been speaking to one another for at least the last 50,000 years, and our bodies and brains have adapted to it over this time. It’s natural, and we’re predisposed to it. Babies do not need to be explicitly taught how to speak–it’s instinctual, like learning to walk. Provided they’re in a social environment and not isolated, and developmentally typical, babies pick up on speech through the process of language acquisition. 

Writing and reading are significantly more recent going back only about 5,000 years. While less natural than speaking and listening, it is in many ways much more efficient. Humans can read on average 2-3x faster than listening. It's also much better for things like skimming or scanning such as when reviewing a dashboard. With reading or observing, you can access information selectively versus listening which is a linear mode of communication. 

Despite this efficiency gain, inputting information via text requires learning to type, and navigate a graphical user interface. Storing information requires typing letters and numbers into forms, emails, documents, spreadsheets, ERP systems, search engines, URL bars and the list goes on and on. All these things need to be built and maintained and while originally optimized for one workflow or perhaps a handful of different ways of presenting data, one small desired change in how it’s used means weeks or months of development work for what might be a simple request to consume data in a unique way one time.

Voice + screen interfaces thus present the best of both worlds. Instead of building a program to input, store, process, and present information, you can just speak naturally. The applications for this are obvious when you think about it paired with agentic AI. Instead of writing a query, you might one day be able to ask AI to “Check the impact of the Shanghai port congestion on our Q3 delivery schedule for our high-margin SKUs, and show me which suppliers are at risk of missing their delivery windows.” Instead of listening to an AI voice rattle off a long list of suppliers, the user would be presented with an easily scannable table of suppliers that they could also ask to be sorted using their voice. 

None of this graphical display would need to be preprogrammed, designed in advance or require any input by the user. Rather, it could be created on request by AI. The user could also choose to drill down in our hypothetical example: “Show me the top three suppliers with the highest cumulative revenue at risk due to this delay, and break it down by affected SKUs, delivery method, and inventory levels at our North American distribution centers” and be presented with that information, instantly without a query, filtering, or even a prebuilt program. Provided your AI agent has access to a data lake containing this information, it can take your natural language “query”, process the data available to it, and then show you exactly what you asked for.

Voice does have some limitations such as needing to speak out loud potentially in a public place where a keyboard fallback would be needed. Eventually, Brain-Computer Interfaces (BCI) could eliminate the need to speak out loud, and instead the user would just think what they want to accomplish. Traditionally direct neurological connections involved brain implants or inserting a sensor close to the brain through a blood vessel, but several companies including Meta are working on non-invasive BCI including using magnetoencephalography (MEG) and electroencephalography (EEG). This technology already exists in limited form, but more powerful iterations are likely still several years away. In the meantime, traditional keyboard backup in noisy or noise sensitive areas will likely remain the go-to for technology companies pursuing voice AI.

What OpenAI’s acquisitions of io and collaboration with LoveFrom ultimately produce remains to be seen. Altman called one of their early prototypes “the coolest piece of technology the world will have ever seen” which could either be Steve Jobs-like hyperbole, or a serious assessment by one of the most influential technology leaders today. One thing is for sure though, the era of interacting with technology via keyboard and mouse as a primary interface is coming to an end. After over sixty years of waiting, it’s time to say “Hello Computer…”

AI: Finding our Feet Among Endless Possibilities

The following was originally published on May 17, 2021 in the now defunct trade publication The Habtic Standard. Accessible here at The Wayback Machine.

The messaging around AI tells us that it is essential in all areas of life and that the possibilities to support and improve life are ‘endless’. While many potential developments in AI are still firmly in the future, in the workplace AI is making inroads, from workplace wellness programs to the software platforms used every day. When appropriately implemented, AI promises to ‘connect the dots’ in ways heretofore unseen and improve the lives of employees. However, far too often, the potential benefits remain unrealized.

It is only recently that technical advances created the possibility of putting AI into the hands of ordinary people with digital assistants like Amazon’s Alexa, and onto the average employee’s work computer with tools like Microsoft Insights. Now that AI is being integrated into many of the daily driver software products used across wide swaths of industry, the potential benefits both to the employee and the employer are becoming more apparent.

Programs: Wellness for the Bottom Line

Far removed from the early days of skin-pinch BMI meters and belt clip pedometers, today’s workplace wellness programs go far beyond mere ‘health’ and focus on overall ‘wellness’ through smartphones, apps, and wearables that generate mountains of data.

AI has the potential to spin this ‘straw into gold’ by sifting through this data to make unseen connections and recommendations to not only the individual employee, but also if desired – and under strict privacy laws – to their physician and the company’s management.

It is this concern over what will happen to this data ‘gold’ that might be one of the greatest barriers to the general employee acceptance of AI in the workplace, even more than AI itself. Data privacy gaffes by IT companies in the early days of in-home AI assistants resulted in only half-hearted laughs and lingering questions about who exactly was listening 1. That said, nobody really goes to work worried that their computer will become sentient and lock them in the company data center à la HAL in 2001: A Space Odyssey.

For a generation, employers have relied on easily quantifiable metrics such as step counts or weight loss which evidence suggests may actually do more harm than good 2. Historically this makes sense, as some of the reasons for employer buy-in to workplace wellness programs circled around the financial – reduced absenteeism and healthcare premiums. While these metrics may improve certain aspects of physical health, the evolution from a ‘health’ program to a true ‘wellness’ program requires far more buy-in on the part of both employers and employees.

AI: Employee Support

 The true value of AI in the workplace is in data driven, dispassionate observations and recommendations in the pursuit of happier and more productive employees. Some of these changes could be truly revolutionary. For instance, AI could suggest adjusting working hours for each employee based on a productivity model. Likewise, a potential AI recommendation might be when to stop working and take breaks.

Employers reluctant to provide pro-employee accommodations to increase engagement often fail to see employees as individuals and the data suggests this strategy is outdated and not working. Recent studies show workplace engagement resulted in 41% lower absenteeism, 58% fewer safety incidents and 40% fewer quality defects, while disengaged employees cost American companies between USD 483 billion and USD 605 billion each year in lost productivity 3 4.

There’s an App for That: Mental Health and AI

Millennials are among the most stressed generations in recent memory, with nearly 1 in 3 suffering from a mental health condition 5. With many workplace discussions around mental health limited to the employee assistance plan folder during orientation, they aren’t often getting the help needed to manage workplace derived stress.

The United States National Council for Behavioral Health estimated in 2017 that the country lacked over 15,000 psychiatrists 6. This shortage means that access to mental health professionals is constrained and those who are lucky enough to get an appointment often do not have the time required to get treated effectively. Artificial Intelligence presents the brightest hope for not only identifying but preventing mental health issues in the workplace.

Surprisingly, 82% of people prefer the idea of interacting with a chat-bot versus a human as they feel it can provide a judgement-free experience with quick access to the answers they are looking for 7. The massive strides in natural language processing means AI can potentially pick up on clues that might otherwise be missed such as variables in speech that could indicate, for example, depression 8. AI chatbots can also check in with individuals far more often than a human clinician could and be available 24 hours a day. Studies have shown that regular check-ins – even via an AI-enabled chatbot – can help identify and even prevent suicidal behavior 8.

These advancements came not a moment too soon, as a joint study by the database giant Oracle and HR consultancy Workplace Intelligence found that due to the COVID-19 pandemic 2020 was the most stressful year in recent history with 70% of people reporting more workplace stress than any other year and 78% of people reporting a negative impact on their mental health 7. A forced shift to remote work almost overnight required herculean efforts and long hours of sustained work while disconnected from family, friends and social activities.

While 51% of respondents to the Oracle/Workplace Intelligence survey noted that their employers added mental health services during the COVID-19 pandemic, 76% of respondents felt that more needed to be done 7.

Drawing on the organization’s data generation, AI can automate tasks and improve access to data needed by employees during the workday. It can also make recommendations on how to work smarter instead of harder.

The AI industry has realized this and is working to integrate AI into its existing product lines. At its Ignite conference in March 2021 Microsoft announced a new employee experience platform which would integrate AI to ‘automatically [organize] content and expertise across [the] organization’ and provide ‘data-driven, privacy-protected insights and recommendations to improve productivity and wellbeing’ 9.

There is Hope for the Future 

While Artificial Intelligence promises to improve productivity, reduce workplace stress and make the job easier and more engaging, it can only do so with the consent, privacy protection and buy-in from both employers and employees. Proactive employers will embrace AI and seek to integrate it wherever possible and give its recommendations the careful consideration they deserve. An evolving, technically literate work force will want it to improve their work lives, mental and physical health.

REFERENCES

1. Lynskey D. ‘Alexa, are you invading my privacy?' – the dark side of our voice assistants. The Guardian. https://www.theguardian.com/technology/2019/oct/09/alexa-are-you-invading-my-privacy-the-dark-side-of-our-voice-assistants.

2. Johnson R. The Biggest Losers Might be Your Employees. Corporate Wellness Magazine. https://www.corporatewellnessmagazine.com/article/the-biggest-losers-might-be-your-employees.

3. Harter J, Schmidt F, Agrawal S, Plowman S, Blue A. The Relationship Between Engagement at Work and Organizational Outcomes, 2016 Q12® Meta Analysis: Ninth Edition. Washington, DC: Gallup; 2016.

4. State of the American Workplace. Washington DC: Gallup; 2017.

5. BlueCross BlueShield. Millennial Health Trends in Behavioral Health Conditions. Topeka, Kansas: Blue Health Intelligence; 2020.

6. National Council for Behavioral Health. The Psychiatric Shortage Causes and Solutions. Washington, DC; 2017.

7. Oracle, Workplace Intelligence. AI@Work Study 2020 As Uncertainty Remains, Anxiety and Stress Reach a Tipping Point at Work. Redwood Shores, CA; 2020.

8. Ducharme J. Artificial Intelligence Could Help Solve America's Impending Mental Health Crisis. Time. https://time.com/5727535/artificial-intelligence-psychiatry/.

9. Microsoft. Microsoft Viva. https://www.microsoft.com/en-us/microsoft-viva.

Lost in the Crowd: Impacts and Solutions for Bad Synergy

The following was originally published on March 15, 2021 in the now defunct trade publication The Habtic Standard. Accessible here at The Wayback Machine.

Among the most misunderstood words in corporate jargon today is the concept of “workplace synergy”. When judiciously used, synergies can dramatically improve efficiency and productivity. However, when misjudged, the result can be catastrophic – for both the employee as an individual, and the company.

A business acquaintance recently mentioned to me, that her entire company – worldwide – had participated in a full day of seminars put on by the US-based corporate headquarters. Although she agreed that there was probably some valuable information in these seminar sessions, she felt the loss of a whole working day probably wasn’t justified by the results. Everyone who has worked in a mid to large size company has likely stood in her shoes: frustrated and stressed by a lost workday in the name of promoting that most ethereal of corporate buzzwords: synergy.

Workplace Synergy in the 21st Century

The word synergy itself comes from the Greek synergia meaning “working together” 1. “Workplace synergy” often refers to the sharing of knowledge or resources and eliminating inefficiencies within one or more teams. In many companies this takes the form of exercises (retreats, seminars, meetings etc.) promoting unit cohesion within a single team and cooperation across two or more teams.

Eliminate inefficiency, promote shared use of resources and “go further with less” – at first glance, this seems a no-brainer. Although synergy might sound like a science, it’s not necessarily predictable, measurable or replicable. Poorly conceived plans with vague methodology and intangible benefits have the hidden potential to be a kind of corporate napalm.

While many factors go into the success or failure of a workplace synergy initiative, much of the blame for failure can be attributed to managers’ failure to really understand where and what type of synergy is appropriate, where it should not be pursued, or in many cases even what it is. Yet managers forge ahead, dismissing the concerns of naysayers as “inertia” due to deeply held misconceptions and bias 2.

Logistical vs Management Synergies

Workplace synergy can be broken down into two primary focal points within a company: logistical synergy and management synergy.

Logistical synergies often come with clear and specific goals. Logistical synergies can be about internal company development or external growth, where a company looks for synergy from a merger or acquisition. Internal initiatives might consider purchasing power, centralizing the advertising budget and coordinated marketing. External initiatives revolving around mergers and acquisitions are often about “revenue synergy”, such as new opportunities to cross-sell to existing customers, channel optimization and brand extension 3.

Management synergy, in contrast, can be thought of as a synonym for “teamwork” and the pursuit of the elimination of barriers to greater collaboration and esprit de corps. Empowering employees, promoting trust, removing barriers to communication and establishing unit cohesion are all goals of management synergy 4. Seminars, teambuilding exercises, company retreats and corporate propaganda are typical ways that managers attempt to promote this.

Because management synergies are more easily summarized, there is a tendency by managers to confuse concrete logistical synergies with more vague management synergies, taking the view that “any synergy is better than none”. Unfortunately, this couldn’t be further from the truth.

“I Don’t Really Know What It is, I Just Know It's Good for the Company”

High-level managers are often seduced by the word synergy itself. It sounds ethereal, intelligent, and is often encountered by management in trade publications and pitches from management consultants. Speakers at executive roundtables may give presentations on their company’s synergy initiatives. Accordingly, managers see synergy as a desirable within their organizations even if they may not know exactly where or when to implement it. This is what management consultants Michael Goold and Andrew Campbell refer to as “synergy bias” 2. This dogged pursuit of synergy at all costs often distracts executives from planning to reach a clear and measurable objective.

Papa Doesn’t Always Know Best

There is a tendency by upper management to fall victim to what Goold and Campbell refer to as “parenting bias” 2, where upper management assumes that “unit managers, overly focused on their own businesses and overly protective of their own authority, disregard or undervalue opportunities to collaborate with each other” and conclude that top-down involvement is needed to spur-on synergy 2.

Frequently, however, there are good reasons why lower-level managers have not pursued synergy on their own, and high-level managers are often too far removed to see the potential issues clearly. Instead, high-level managers resort to “sloganization” with ill-defined or misguided goals. To quote one famous corporate trainer: “vague goals produce vague results” and nobody is satisfied 5.

Synergy Without Stress

Nebulous workplace synergy initiatives that manifest themselves in a constant stream of meetings, standups, retreats, planning sessions, seminars and webinars are also caustic to employee wellbeing. In today's global information economy, the workplace tempo beats ever faster, and many employees are already stressed and overworked. A compilation of workplace stress surveys by the United States National Institute for Occupational Safety and Health (NIOSH) suggests between 25% and 40% of American workers experience a high level of stress in the workplace 6. The UK charity Mind found that 19% of employees called in sick to work due to stress (most cited other reasons for doing so) and a quarter of all employees reported considering quitting their company due to stress 7. A study conducted by researchers at Stanford University found that workplace stress kills 120,000 workers every year 8.

Losing a workday (or more) to a corporate buzzword or yet another trendy, vague “management initiative” while the day-to-day work piles up in your inbox runs the risk of exacerbating this problem and encouraging disengagement.

Is Your Orchestra Drowning Out the Singer?

Overly simplistic attempts at workplace synergy can also erode what little remains of ‘trust in the individual’ in today’s corporate world. Following the reductive and tired logic that a unit is the sum of its equal and identical parts is to ignore the individual talents and strengths team members bring to the table. This can lead to feelings of frustration by employees – in many cases the best performers. High-performance individual employees forced to participate in collaborative settings may be ostracized if they become a perceived threat to their coworkers. A study published in the Journal of Applied Psychology found that: “...cooperative contexts proved socially disadvantageous for high performers” 9. The same study found that “even stars can flounder without supportive peer relationships” 9 and a Gallup study found that employers lose between USD 483 and 600 billion due to employee disengagement, or “about a third of the disengaged worker's salary” 10.

Participation and Collegiality

While poorly implemented workplace synergy exercises such as worldwide seminars, or mountaintop matchmaking have the potential to impede productivity and frustrate employees, it remains true that properly implemented strategies can (and do) lead to greater efficiency and happier employees.

Employees forced to work at home by lockdowns, stay-at-home orders and closed offices in the era of COVID-19 often feel alienated from their colleagues. Optional teambuilding activities provide a good alternative for those who want to reconnect with their colleagues when meeting in person is not an option. My employer held an optional after-hours Christmas Trivia contest via Zoom for those employees who wanted to reconnect with their colleagues in a socially distanced but fun manner over the holidays, while allowing those employees who wanted more time away from work the flexibility to do so. This approach accommodated both preferences, and many chose to participate voluntarily.

Looking Down the Line

Senior managers looking for cross-team synergies need look no further than down their own chain of command. Low- or mid-level managers will (given sufficient autonomy) often implement small-scale synergies independently. This is what upper management should seek out and consider upscaling if appropriate. If a top-down approach – after much consideration – is determined to be necessary, then the initiative should be piloted on a small scale first, so that the sought benefits can be measured and potential problems resolved. By empowering mid-level managers who have greater experience in their area, companies can avoid the pitfalls of strategies that ultimately cause harm. These may be obvious to the mid-level manager but not to upper management – and may even be counterintuitive. Empowering mid-level managers also reassures them that their ‘on the ground’ insights are valued and it will encourage them to seek out other beneficial synergies.

A Wiser, More Skeptical Approach to Workplace Synergy

Managers should approach any proposal to implement a workplace synergy initiative with a healthy dose of skepticism. Implemented poorly, the potential downsides to both the employee and the firm far outweigh taking no action at all. Upper management should not be the driving force behind a synergy initiative but rather empower mid-level managers to pursue synergy independently. Employers should consider whether the potential loss of employee individualism is justified by concrete and defined goals. Most of all, employers must remember that their employees are far more than “human capital” and seek to preserve the dignity of the individual. Their employees – and their bottom line – will thank them for it.

REFERENCES

1. Synergy. Online Etymology Dictionary. https://www.etymonline.com/word/synergy.

2. Goold M, Campbell A. Desperately Seeking Synergy. Harvard Business Review. https://hbr.org/1998/09/desperately-seeking-synergy.

3. Chartier J, Liu A, Raberger N, Silva R. Seven rules to crack the code on revenue synergies in M&A. McKinsey & Co. https://www.mckinsey.com/business-functions/marketing-and-sales/our-insights/seven-rules-to-crack-the-code-on-revenue-synergies-in-ma.

4. High-Performance Teamwork. Center for Management & Organization Effectiveness. https://cmoe.com/products-and-services/high-performance-teamwork/.

5. Canfield J, Healy K. The Success Principles for Teens: How to Get From Where You Are to Where You Want to Be. Deerfield Beach, FL: Health Communications Inc.; 2010.

6. National Institute for Occupational Safety and Health. Stress At Work. Cincinnati, Ohio: National Institute for Occupational Safety and Health. 1999: p. 4.

7. Work is biggest cause of stress in people's lives. Mind. https://www.mind.org.uk/news-campaigns/news/work-is-biggest-cause-of-stress-in-peoples-lives/.

8. Goh J, Pfeffer J, Zenios S. The Relationship Between Workplace Stressors and Mortality and Health Costs in the United States. Management Science. 2016;62(2): 608-628. https://www.gsb.stanford.edu/faculty-research/publications/relationship-between-workplace-stressors-mortality-health-costs-united.

9. Campbell E, Liao H, Chuang A, Zhou J, Dong Y. Hot shots and cool reception? An expanded view of social consequences for high performers. Journal of Applied Psychology. 2017;102(5): 845-866. http://web.ba.ntu.edu.tw/professor/contents/aichia/CampbellLiaoChuangZhouDong2017JAP.pdf.

10. Hickman A. How to Help Your Managers Build Out, Not Burn Out. Gallup. https://www.gallup.com/workplace/249140/inspire-management-breakthrough-not-breakdown.aspx.