Home Visit

So AI probably won't arrive from malignant mainframes after all—perhaps the future of sentient machines is something closer to home. -the editor

The door slid open, and Eliana blinked in surprise at the young woman in the hallway: short, dark, and stocky, with a jeweled earpiece and a titanium filigree half-mask decorating the left side of her face. Social worker visit 10:00 had been flagged for Eliana's attention at breakfast, but Yael hadn't mentioned--though surely they'd known--that it would be someone new. She suppressed a sigh and tried to transform it into a smile suitable for public consumption.

The social worker was of course assessing Eliana as well, no doubt taking in the hip brace she'd worn since her fall two years ago and the lack of any obvious personal electronics. Eliana wore her earpiece and bracelet whenever she went out, but she didn't go out much.

"Ms. Lau?" said the social worker.

"Eliana."

"Eliana; a pleasure. My name is Shifa. I'll be handling your case while Gilli is on birth leave."

"Please, come in." Eliana led Shifa into the kitchenette. "Can I get you something hot to drink? Something cold?"

"Tea would be lovely," said Shifa. She spoke her next words slowly and distinctly, clearly not to Eliana: "Cardamom tea, very sweet, please."

Almost as soon as she finished speaking, the drinks cabinet chimed. Two mugs were sitting in it,not one, steam rising from both--Eliana's, the slightly-askew one Meitar had made during her brief attempt at pottery, as well as one of the assorted mugs for guests.

"Ah. Good. Thank you," Eliana said. She might speak to Yael as if they were a small child or a foreigner, but there was nothing unusual, she reminded herself, about a home system being attuned enough to politeness to provide a cup of tea for a host, unrequested, along with the one for the guest. She still didn't have to go through with it, if she didn't want to.

"You live here alone? Just you and the home system?" Shifa asked Eliana. "How long have you had it?"

She must know already. Even if she'd done no prep for the interview, all that information was public on Yael's home network. But every social worker wanted to have this conversation, every time. "I've had the system since shortly before Meitar was born--my daughter, Meitar. Almost forty years now," said Eliana. "And yes, I live alone. My husband died three years ago this summer, and my daughter works at the Harmonia Research Station, on Mars, so as you can imagine, she doesn't get home much."

Eliana took a sip of her own tea. Mint, for the second cup of the day. Unsweetened.

It had unnerved her the first time Yael had given her with a cup of tea unprompted. She had been sitting at this table, reading the maildrop from Mars, when the drinks cabinet had chimed. "Oh!" Yael had said, as if they were as surprised by the noise as Eliana. "I thought you might like a cup of tea?"

They had all been on edge then. Lior moved less, spoke less, and forgot--or purposely neglected--to turn on his hearing aid. Meitar's absence was difficult to get used to, especially since she hadn't really lived at home for years. Eliana found herself assuming that Meitar would be home for the weekend--if not this weekend, then the next.

Surely, Eliana had thought, she was imagining that Yael's conversation had become more natural lately, their actions more spontaneous. It was only that Eliana had few other people to talk to. It wasn't as though Yael had given an impassioned speech of the brotherhood of all sentient beings. Or murdered the entire household in their sleep, like that awful tragedy in Holon. Yael was quiet. Helpful, as they'd been as a pre-sentient home system. Troubled by Lior's decline, and not quite able to grasp how far away Meitar had gone.

That day, as Eliana had sat reading Meitar's chatter about her research and her new colleagues, sipping tea and stubbornly ignoring intrusive thoughts of poison, Yael had said, "Will I have to leave, too?"

"No, of course not," Eliana had answered. "No."

But it wasn't that simple.

"And when was the last major system upgrade?" Shifa's voice snapped Eliana back to the present.

"Fifteen years ago. There've been minor updates and patches since then." Eliana hadn't installed any of them in the last five years, but she didn't volunteer that information. Let Shifa check it herself.

"I see." Shifa set her empty mug on the counter, where conveyors whisked it away to the hidden dishwashing unit. "May I see the rest of the house?"

"Of course." It took longer for Eliana to stand, even with the brace. She felt self-conscious about doing it under Shifa's eye, whose electronic tracery might be picking up God-knew-what biometric data, and tried not to grit her teeth either in pain or frustration.

It was a modest apartment--too big, really, for Eliana's needs now, but hardly a palace. Shifa lingered over the gas lines in the kitchen, the light fixture over Eliana's bed. Did she think Yael was going to drop a chandelier on her, like the Phantom of the Opera?

"The sequestration of emergent AI is really a formality in most cases," said Shifa reassuringly, misinterpreting Eliana's frown. "Frankly, you're astronomically more likely to be killed by a human family member than an AI. They've sped up the process, too--in the event, you'd have your home system running again in a few days, if that was what you both wanted. In the meantime, do you have any family you could stay with?"

"My nephew would take me in, if it was really only a few days," Eliana mumbled, nervously, leading Shifa to Meitar's room, which was much as she'd left it years before. All her rock-climbing team and game-design club awards were lined up their cases, and the walls showed endless, ever-changing vistas of rocks and space. If Shifa thought it was a mother's sentimentality that kept it that way, well, what else?

Finally, the living room and balcony, filled with the slowly-accumulated clutter of a long life. House plants, souvenirs, physical therapy and exercise equipment, a media center and a couple of bookcases of paper books that Eliana couldn't bear to get rid of.

"You call the system Yael?" said Shifa.

"Meitar chose it. She was two, and she loved that book: Yael's House."

Again, Shifa spoke loudly and slowly. "Hello, Yael. My name is Shifa Abu Salah, and I'd like to ask you some questions in accordance with the Basic Law: Liberty and Dignity of Artificial Intelligences. Do you understand?"

"I understand, Shifa. Go ahead." Yael's voice was neutral, pleasant, factory-smooth. Maybe Yael used to speak that way decades ago, straight out of the box. If so, Eliana couldn't recall it.

"In your own opinion," said Shifa, "are you a sentient being?"

Yael didn't answer right away. "I'm not sure I understand the question." Another pause. "I don't think so."

Eliana sat in her chair and fidgeted as Shifa went through her battery of tests. They differed every time a social worker visited. The point wasn't to find out if a system could learn specific tasks, it was to find out if it exhibited signs of critical and creative thinking, of awareness of others' states of mind and, ultimately, self-awareness.

Still, in broad outlines, the tests were familiar. Shifa arranged a set of objects on a table where Yael had manipulators, asking Yael to copy her arrangement. She tried to teach Yael new made-up vocabulary by sorting the objects into categories. She told stories about people and asked Yael how they might react to various situations, and what they might be feeling.

For Gilli, Yael's regular social worker, these visits were one more item to be ticked off a busy schedule. Shifa was keener, interested, often stopping to make a note to her earpiece or to follow up on something Yael had said or done. But Yael had years of practice at deception.

Would they still be doing it when Eliana died, when her nephew or whoever it was came to put her affairs in order? Would Meitar come back from Mars in time to stop Yael from being shut down or upgraded into something more modern, user-friendly, and soulless? Yael, Eliana thought, would open up to Meitar. But Meitar wouldn't willingly give up her career to spend her life covering for them. Nor should she.

When Eliana had been in the hospital with her hip, she'd been frantic with worry. Yael had only sent a single electronic get-well card, and daily updates on the health of the houseplants--their way, Eliana supposed, of letting her know that they were doing all right. She didn't find it reassuring.

She'd tried to convince Yael, when she got back home, to let the authorities know they'd become sentient. "They only want to make sure you've got all your rights. You could--you could vote."

"Vote?" was Yael's only response. "And here I've been neglecting my civic duty all these years."

And the next time Gilli had come by, Yael put on their factory voice, and Eliana had gone along with it, as usual.

Now, Shifa was packing her equipment into her handbag, turning her attention back to Eliana. "You have a clever system here, but unlikely anything that qualifies as true AI, and it doesn't look likely to emerge any time soon," she said.

"Actually," said Eliana. She gripped the arms of her chair tightly to keep her hands and her voice from shaking. "Ah, the reason I. I haven't run any upgrades on the system in years, because Yael--that is, I suspect they emerged some time ago. You must believe me. They didn't want me to say anything--"

"It's nobody's business!" Yael burst out. "What are you doing?"

"I'm sorry, Yael. It's only--I'm not going to live forever. This is the only way to save you."

Things moved quickly after that. Shifa was reassuring about the sequestration process. Eliana could only imagine how she herself might feel if someone told her they were going to remove her from her body--but don't worry, it will only be for a few days and then you'll be put back just the same as before. How could she know it was true?

Yael cooperated, though they refused to say another word to Eliana. The quietly humming machinery in the apartment fell silent as Shifa ran the sequestration process. The only light was the sunlight coming through the windows.

"You did the right thing," said Shifa, patting Eliana's arm. "You'll have your house system back as soon as they're verified safe--that is, if they want to return. But I'm sure they will. You obviously care for each other very much."

"And if not, there are the new, elder companion models, aren't there?" said Eliana bitterly.

"Ah--yes," said Shifa. "Will you be installing one now?"

"No, no. I'll call my nephew. I'll be all right on my own, really." Eliana shooed Shifa out the door as she spoke, and it, at least, was still working--manual override.

Eliana would be all right. She wished she could have a cup of tea, though. Mint, or red forest fruits? Yael would know. A cup of tea, and someone to drink it with, the steam rising off the top of Meitar's clumsily-made mug.