What AI Gets Wrong
At jobsites and in union halls, artificial intelligence appears to be no match for good old human labor
THIS AI-POWERED ROBOT is a product of SF-based Raise Robotics. It has been implemented by building trades workers in a union construction project for the SF Public Utilities Commission, in which the robot completed facade layout marking. | Photo courtesy Conley Oster
Building trades leaders throughout San Francisco are facing down how and why to use artificial-intelligence-powered software and the equipment connected to it. Their primary objectives include ensuring that work is completed to a high standard, that workers form solid relationships with both coworkers and clients, and that union members don’t lose jobs and benefits to AI-powered software and robotics.
The mood is tense. Many corporations appear intent on eliminating the human element from building and construction trades work.
Robert Collins is a member of Sign and Display Local 510 who also serves as deco general foreman for Freeman, an international trade show, exhibit, and event company headquartered in Dallas. He explained the situation bluntly.
“Despite all the concerns, right now robots and AI-powered software cannot do trades and construction jobs to the standard of excellence and the level of demand that customers want,” he said. “In addition, when it comes to custom jobs like the ones Local 510 does, clients prefer human interactions to clarify what they want.”
Robotic equipment like mechanical arms and hands simply aren’t dexterous, quick, or cost-effective enough to perform jobs such as putting up and taking down booths and signs, Collins said.
In an undeniable irony, Local 510 has for decades been the go-to force for setup and tear-down of tech and AI conferences in the City, many of them at the Moscone Center. Yet interestingly, AI companies that request and man event booths during these conferences haven’t reached out to union workers to ask how they get the work done.
“We — as in Local 510 members involved with crewing jobs — gather and store data in private spreadsheets,” Collins said. “It all relates to people: which workers are good at which tasks, who works well together, and what additional cooperation, communication, supplies, and equipment it took to get jobs done.
“We do not share that information,” he said.
Not that he’s ever been asked.
FIFTH-PERIOD APPRENTICE Jacob Welch learns offsets at the UA Local 38 Training Center. | Photo courtesy Rich Harlan
A Difference in Standards
One of the main issues that frustrates collaboration between AI companies and the trades is a basic culture difference regarding standards for outcomes.
“For a long time, the tech industry’s motto seemed to be ‘Move fast, break things.’ In the building trades, we say, ‘Move fast, and do it right the first time,’” said Rich Harlan, training specialist for UA Plumbers and Pipefitters Local 38.
Harlan said that an “80%-is-good-enough” standard for software does not translate to adequate results for buildings, government-mandated safety measures, and essential services such as water.
“Water needs to be clean and healthy 100% of the time,” he said. “You don’t want water coming out of the pipes that’s only 80% clean, or pipes that leak 20%.”
Harlan said that for trades workers, such margins of error would result in lost goodwill, canceled contracts, and a lack of confidence in union labor.
Harlan recalled teaching one class at Local 38’s training center in the City in which he watched apprentices ask the free version of ChatGPT to explain wet venting, a process by which a single pipe is used to both drain waste and vent the plumbing system.
“The AI-powered software didn’t know the difference between California’s and Washington’s standards,” Harlan said. “The apprentices said, ‘AI is celebrating that it can save us time, but it just wasted ours.’”
Lessons From Covid
Five years ago, Local 510’s jobs were coming back in spurts. Pandemic restrictions were easing up, and events were starting to appear on the work schedule once again.
“About this time, signatory employers [of Local 510] — specifically, Freeman — started using point-to-point inventory systems,” Collins said. “They [now] work with a network of warehouses and a network of tractor-trailers. They pull from one place to another with the [necessary] supplies and equipment.”
Local 510 couldn’t ignore AI because contractors hiring union workers wanted inventory control, which is essentially defined as maintaining the right amount of stock to meet customer demand in a timely manner while also keeping costs to a minimum.
Collins said, “The issue is that many contractors’ systems weren’t ready for AI-powered algorithms. The AI was not and still is not getting a lot of things right. They rolled it out anyway.”
For years, Local 510 workers have been working out the glitches in the AI-powered software that they’re expected to use on the jobsite.
“All of this data to centralize and standardize has to be manually entered,” Collins explained. “Unless every trailer is hit with radio frequency identifications via microchips, I don’t see any employer investing in that level of technology.”
Collins noted that AI-powered robots designed to survey terrain and program floor plans aren’t capable of doing these jobs as well as people. For starters, robots aren’t particularly adept at identifying mistakes or challenges. Humans tend to be much better at that.
“Further, the culture union members create is important,” Collins said. “Union members have each other’s backs. We go out of our way to train each other and mentor each other. So, when you have people doing a job, the people ensure that apprentices get the experience they need and build relationships to help them become journeymen.”
The more a contractor tries to insert a digital element into the equation, the less time workers get for face-to-face, supportive, career-advancing interaction.
“That makes final outcomes worse for the human workers as well as the customers,” Collins said.
Auditioning AI for Front-Office Work, Training, and More
Bill Olinger, director of communications for Local 38, has tried to use AI-powered software programs to research information for his job. It hasn’t gone well.
“As an example, once I wanted to look up the history of a former business manager,” he said. “I gave the free version of ChatGPT detailed directions. It spat out info on a totally different local labor leader. That made me frustrated. Now I don’t rely on AI for help.”
Over at Local 38’s training center, Harlan bought a paid subscription to Google Gemini. He uses the AI assistant to perform high-level searches that help generate ideas and starting points for lessons. He also uses it to research background information, which he verifies before sharing in a class.
“An instructor could teach you 20 different ways to handle an issue. An AI-powered program might give you three.”
“Even though I’m using AI for some things right now, I think AI would make a terrible teacher,” Harlan said. “I’ve been trying to get AI programs to create quizzes of 10 questions on topics we’ve covered. It has trouble coming up with good questions and answers. The questions can be random. The questions won’t match the scope of the material. For example, you could have too many questions on one topic and none on another.”
As an instructor, Harlan determines who to pair up on projects. He asks himself who should brush up on skills and determines which students would benefit most from working solo or collaboratively.
Harlan hasn’t seen any AI-powered software program that could do this part of the job.
Outside the classroom, training center instructors serve as a bridge between training center grads and contractors. An instructor could recommend their former student for a job based on the person’s attitude and achievements in class, for instance. It’s unlikely that a contractor would look to an AI agent to provide such references.
Training center instructors often attend educational programs to learn from one another. Every summer, Harlan teaches at the UA’s week-long Instructor Training Program at Washtenaw Community College in Ann Arbor, Mich.
Approximately 2,000 instructors from local unions in North America and Australia attend the course each year. It would be impossible for an AI-powered robot to go to such a conference and learn shared techniques and information in the same way that a person would.
Harlan and his students have also experimented with what AI can do during class exercises.
“I’ve tried to use AI-powered software for construction scheduling,” he said. “When you schedule a job, you want to avoid trade stacking. This is when you schedule too many subcontractors or subtrades on a project at the same time. People get in each other’s way. That slows everyone down and makes them frustrated. AI is not good at sequencing jobs for the trades.”
INSTRUCTORS sit for a lecture on mobile technology for construction at the UA’s annual week-long Instructor Training Program in Ann Arbor, Mich. | Photo courtesy Rich Harlan
Harlan said that there’s value in getting trades students familiar with resources such as the National Fire Protection Association’s fire protection manual rather than relying on AI to impart such knowledge.
Harlan said, “At a jobsite, a contractor or inspector will ask you a medical-gas-related question. If you want to keep your job, you need to know the material or know how to quickly look up an answer in the manual.
“We promote using the manual in class over and over again. That way students learn the material and get a sense of different risks.”
For context, medical gases are gases like oxygen and nitrous oxide that are manufactured to be administered to patients. It’s important that medical gas delivery and vacuum systems be installed properly to avoid flammability risks.
Timing is important, too. Local 38 apprentices learn that there’s typically a three-year code cycle for government regulations.
“Codes evolve with the built environment,” Harlan said. “Learning without looking up the answer in AI is better. The student gets to know what the current code looks like. AI-powered programs tend to find answers where there is a bulk of information.”
The majority of information synthesized by AI software and provided in answers usually comes from the past. Since AI programs tend not to explain why or how they retrieved certain information, a student won’t know that an AI agent pulled an answer based on outdated codes.
“A regulation that’s brand-new won’t have so many hits on the internet, but AI can’t handle that nuance,” Harlan explained. “It gives users what most people are searching for. That might be something very different than what the student needs.”
A final concern is that AI tends to offer cookie-cutter solutions.
Said Harlan: “An instructor could teach you 20 different ways to handle an issue. An AI-powered program might give you three. In San Francisco — and really, in construction — uniqueness sells. Customers want something special. Unions’ expertise in the internal architecture like plumbing is one of our selling points.”
The View From District Council 16
“You cannot learn our trade through AI. You have to learn it by hand from other workers.”
Trevor Long is a business representative with District Council 16 and a member of Glaziers Local 718. Long echoed Olinger’s concerns regarding the accuracy of AI-powered search results.
Long said that AI-powered search engines often hallucinate in a way that mischaracterizes projects.
“AI aims to please,” he said. “It wants to validate the requests and information you give it.
“Recently, I saw AI programs characterize the 22-story housing tower at 1111 Sutter Street as having a project labor agreement. There was a union agreement to build this tower — but the tower didn’t have a PLA.”
Long uses free accounts for several AI-powered programs, including Grok, ChatGPT, and Perplexity, to do writing and research.
In using the chatbots, he said, “you run the risk of having AI write everything the same way, so materials become predictable. They start to sound the same. You want to avoid that.”
Anthony Nuanes is also a business rep with DC 16 and is a member of Carpet, Linoleum, and Soft Tile Workers Local 12.
“You cannot learn our trade through AI,” he said. “You have to learn it by hand from other workers. Sometimes I use AI to assist with a staff report, but floor covering is skilled labor paid by the hour. We don’t do estimation, accounting, or logistics. We don’t use AI for any of those tasks.”
Nuanes said he’s seen videos from South Korea in which a robot spread an adhesive for floor covering.
“We don’t have that here yet,” he said.
WORKERS are trained on an AI-powered robot built by SF-based Raise Robotics, whose engineers consulted with building trades workers in order to create a product with their buy-in that would be useful to them. | Photo courtesy Conley Oster
Thoughts From a Robotics Developer
Conley Oster is the cofounder and chief operating officer of Raise Robotics, an SF-based robotics company that uses AI-powered software. The difference between him and thousands of other AI executives is that his firm has made a point of listening to union workers.
“We started engaging with members of the International Union of Painters and Allied Trades about three years ago,” Oster said. “Before Raise, I was working in crane and rigging. Prior to that, [I was] a manufacturing engineer.”
Oster wanted to learn from unions because he felt that his company needed their buy-in.
“Right now, any big project will be a union project,” he said. “If unions don’t want us there, we won’t be working on big projects.”
Oster came up with the idea of having an AI-powered robot assist with curtain-wall bracket installation — a difficult and repetitive task that needs to be done at height.
“I learned that you don’t have to make the AI the most technically complex product,” he said. “The robot doesn’t have to do everything.”
First, Raise developed an idea of what the robot could accomplish. Then it designed the robot and accompanying software to perform these tasks. During this process, Raise created a training program to teach union workers how to operate the robot.
When Raise eventually held the class, it went better than expected. Oster said the trainings were visible evidence that union workers are technically minded. He wasn’t surprised that they found working with the robot to be the most exciting part of the two-day class.
“Union workers taught us that usability is a huge piece of the puzzle,” Oster said. “As manufacturers, we can make AI-powered devices smaller, more mechanical, and more flexible. We also need to make them easier to diagnose and repair if they have a problem. All of this makes the robots easier for workers to use.”
Raise’s engagement resulted in DC 16 locals utilizing Raise robots in the recent work its members did on the new San Francisco Public Utilities Commission operations center on Marin Street. Oster said the experience showed him that AI companies that don’t listen to union members — from the leadership to the rank-and-file — are missing out.
“Unions give you that helpful, nuanced feedback,” he said. “If you get union members trained at the outset, you can certify them to use your robots right at the beginning of a project — and you get people who want to work with your machines.”