Unfair ratings cost some Instacart shoppers hundreds a week

An Instcart shopper carries groceries in a file photograph.

Dec. 21, 2020 5 AM PT

Bags of groceries don’t just vanish into thin air. But in case the laws of physics ceased to exist, Loreen Zahara does her due diligence. The Instacart shopper keeps receipts for purchases and even photographs them upon delivery — on a customer’s stoop or in front of their garage.

Yet when one customer gave her a one-star rating over a missing bag of pineapples and another awarded her one star and claimed an entire order wasn’t delivered, it was Zahara who suffered the consequences: a loss of hundreds of dollars of potential earnings per week.

Instacart’s order-allocation system takes the “customer is always right” mantra to new extremes, some of its professional shoppers say. The grocery delivery company presents its workforce of independent contractors with orders based in part on their in-app ratings: Those with higher scores get first pick, often leaving behind fewer and less lucrative batches for everyone else. Interviews with more than 10 shoppers and receipts reviewed by The Times show a sharp decline in earnings for shoppers whose ratings drop just slightly below 4.95 out of 5 stars. Often, shoppers said, the negative reviews were beyond workers’ control.

Even though Zahara has evidence those two complete orders reached the customers’ homes, it was enough to drop her rating to a 4.94. She went from earning an average of more than $1,270 per week to $690 per week, while working the same total hours, screenshots and weekly earnings reports show.

When Zahara had a rating of 4.95, compensation for batches of deliveries available to her ranged from $15 to $45. At a 4.94, screenshots show orders dipped to $9 to $22, with those at the higher end in a different county from where she lived and typically worked.

“I just had to live with the bad ratings and bad batches and no money,” she said.

Instacart says the system was developed to ensure ratings are “fair and accurate,” and do not unfairly penalize shoppers.

To protect shoppers, Instacart automatically forgives a customer’s single lowest rating, Instacart spokeswoman Natalia Montalvo said. And “ratings that are outside of shoppers’ control” are also forgiven — such as when a customer complains that requested item is out of stock at a store, she said.

The system helps the San Francisco company with its effort to maintain high quality control and comply with labor laws in the U.S., three former employees said. It also has a benefit for workers, the company said.

When Instacart began testing the system in 2019, it billed it as a reward to shoppers “who offer an excellent experience for customers.”

“Shoppers significantly preferred being incentivized by quality, as opposed to any other factors such as time spent shopping or speed,” Montalvo said in a statement.

The dozen shoppers interviewed by The Times said they have seen low ratings when customers complain that a specific item — a certain brand of alfredo sauce, in one case — is unavailable. Screenshots of communications with Instacart’s support team show that even when presented with evidence that shoppers have done everything expected of them, poor ratings don’t always go away.

Customers are asked few questions when reporting items or even entire orders missing, which has allowed some shadowy businesses to take advantage of the system. A network of “refund brokers” on forums like Discord, Reddit and Telegram can be hired to help Instacart customers get their money back. Dozens of satisfied customers have posted receipts to show the refunds the brokers managed to secure for them, some receiving as much as $500 of groceries for free. It’s unclear if Zahara, or any of the shoppers in this story, was the victim of a refund broker or an ordinary customer making a false report.

Instacart says bad actors are few and far between, and brokers insist their process doesn’t harm delivery workers — “shoppers keep their tips and this will not [affect] them negatively,” one broker’s FAQ read.

But shoppers say they suffer when customers pair false reports with low ratings.

Donatus Okeke, a full-time shopper who began working for Instacart after he was laid off from his job during the pandemic, struggled to persuade Instacart to remove low ratings he says he doesn’t deserve.

In one case, Okeke said, he was dispatched to deliver an order to a hotel room. The customer didn’t answer the door, so Okeke said he spoke to the hotel receptionist, who agreed to hold the groceries for the customer. Okeke said he called the front desk to check whether the order had been retrieved; hotel staff confirmed it had. The customer later reported the order missing and gave Okeke a one-star rating. Though Okeke said the hotel receptionist confirmed the order had been delivered on a conference call with Instacart customer support, the company did not remove the rating.

Instacart policy forgives a shopper’s single lowest review, but Okeke was still saddled with another low rating he thought was unfair. A customer gave Okeke permission to purchase antibacterial wipes, but issued him a three-star review over his product selection, calling it a “poor replacement.”

In the month before he got the one-star review, screenshots show Donatus averaged close to $1,900 a week. After the hotel incident, he earned an average of $1,200 a week for the same number of hours, as the orders available to him were smaller or offered lower tips.

“I have 74 ratings that are above four stars,” he said in October. “I’m sitting at 4.92 on the power of two ratings that I don’t deserve. And if those two ratings are not being held against me, I’m at a 4.98. It’s an unbelievable thing.”

Ratings-based system

Instacart introduced the ratings-based order system just before the pandemic brought on packed supermarkets, empty shelves and an unprecedented spike in grocery delivery orders.

It was part of a broader effort to delineate more clearly between Instacart’s contractors and its part-time employees, who work in partnering grocery stores, brought on by mounting regulatory pressure, three former Instacart employees said.

In Instacart’s first few years, both employees and contractors worked shifts, as opposed to the on-demand model common in the gig economy. Orders were doled out based on a number of factors including a shopper’s average speed.

But that approach bumped against labor law, which limits how closely a company can govern the actions of contractors.

Among the legal challenges, for example, is a 2019 lawsuit filed by the city of San Diego that alleged Instacart had crossed the line by giving priority in assignments to contractors based on speed and efficiency. In doing so, the city argued that Instacart was dictating how contractors were expected to perform their jobs — a level of oversight that meant they should be classified as employees and provided with benefits.

In response to the city’s request for a preliminary injunction, Instacart filed a motion that pointed to the introduction of on-demand orders in July 2019, among other factors, as evidence of the flexibility granted to contractors. In granting the injunction, the judge said Instacart’s actions suggest the company “already took steps to bring itself into compliance” and would need to make just a few additional changes to ensure shoppers are truly “free agents.”

According to three former employees, including two senior managers, policy changes such as switching to an on-demand system and prioritizing ratings rather than speed and efficiency could bolster Instacart’s general argument that it doesn’t dictate exactly how its contractors should do their work.

Instacart did not respond to the specific question of whether it changed systems in response to regulatory pressure.

In rolling out the new system, Montalvo said, “we listened closely to feedback from the shopper community, who indicated that quality was the preferred way of being recognized with priority access to batches.”

After a key victory at the ballot box in November, legal pressure may prove lower in California. Instacart spent $30 million backing Proposition 22 to secure a carve-out from a state labor law that makes it harder to classify workers as contractors. Under Proposition 22, Instacart shoppers will remain contractors but are being offered a few additional benefits such as a healthcare subsidy if they qualify. Fights remain on the horizon, however, in other states, and the San Diego lawsuit, filed in San Diego County Superior Court, is ongoing.

“We will continue to vigorously prosecute our claims since nothing about Prop. 22 is retroactive,” Hilary Nemchik, the director of communications for City Atty. Mara Elliott, said.

The new system also helps Instacart manage one of the biggest challenges for gig economy firms: quality control.

“Customers rely on Instacart shoppers to pick their groceries as well or better than they would themselves,” read a blog post the company published when the ratings-based system was first launched. “As such, we want to recognize shoppers who offer an excellent experience for customers, giving the highest quality shoppers more prioritization and opportunities to shop.”

Under the new system, shoppers who have the highest ratings in a particular market get first pick of available orders. They usually snatch the most lucrative orders, leaving shoppers with lower ratings to choose from jobs that pay less, are farther away or have lower tips. Other variables, such as the number of available orders, when orders are expected to be delivered, and where the shoppers are located also play a role in what orders each shopper sees, according to the company.

“A shopper’s average rating is just one factor that determines which available batches they see,” Montalvo said.

Shortly after the company introduced the system in March, Instacart decided to pause the rollout as the pandemic spread.

“In the early days of the COVID-19 pandemic, with an increase of customer orders and limited available inventory at many retailers, we found that shopper ratings were fluctuating based on many factors that were outside of shoppers’ control,” Montalvo said.

An onslaught of bad ratings for out-of-stock items prompted the company to temporarily forgive any rating under five stars to prevent low ratings for things that shoppers could not control. In June, the company restarted the program and rolled back low-ratings forgiveness as the marketplace “came back into balance,” Montalvo said.

Instacart says shoppers support the new method.

“We’ve heard from shoppers time and again that they value providing a great experience for customers, and they want to be recognized for their efforts,” Montalvo said.

Refund brokers

Several times a week, screenshots of customers complaining of missing items or orders appear on one of the dozens of Facebook groups frequented by Instacart shoppers. “Woke up to this awesome email this morning of a customer claiming that I did not deliver their order,” read an Instacart shopper’s post. “Sick of lying customers,” read one. “How do customers get away with saying things were not delivered?” read another.

Zahara says it has cost her hundreds of dollars a month. She started working for Instacart in March after being furloughed as a flight attendant. At first, she made $4,800 a month, more than she expected and far more than she would have had she applied for unemployment. But on July 19, a customer complained about a missing bag of frozen pineapple chunks.

Zahara had taken photos of the pineapples in a bag at the customer’s door and uploaded it in the app at the end of her delivery, screenshots show. Still, the customer retracted her tip and gave Zahara a one-star rating. When Zahara reached out to customer support, an agent said that the company would investigate and that if a low rating was unsubstantiated, it would be removed. As of November, the rating had not been removed.

Montalvo said the company reviews any incident a shopper has reported as fraudulent behavior. If a customer is found to have behaved fraudulently, “the ratings they left their shopper will be automatically forgiven,” she said.

Zahara went from a five-star rating to a 4.94, and with that, her weekly earnings plummeted. Though she worked the same number of hours, Zahara’s weekly income fell roughly $650 a week, copies of her weekly earnings show.

“I always had a five-star rating,” said Zahara, who complains of two additional poor reviews that were accompanied by what she said were false claims of missing orders and items. “But now we have a few dishonest customers and they give you a low rating, I’m assuming, to back their claim.”

Zahara resorted to spending $200 on a GoPro camera, which she wears conspicuously around her neck as she completes her orders. She hopes recording every delivery could help her against any additional fraudulent claims or at least discourage customers from lying. She’s also circulated a petition, with 3,406 signatures, demanding Instacart change its system and create a team dedicated to unwarranted customer ratings and fraud.

“This system is flawed,” the petition reads. “INSTACART provides no restitution for the Shopper even with proof of their delivery, item, etc. They offer no department, resource, phone number or other, for us to dispute these issues. Yet they can resolve a customer complaint in an hour!”

“We always welcome shopper feedback and are constantly working to make sure we’re delivering the best offering for shoppers,” Montalvo said.

Shoppers have expressed their frustrations to the company in emails to customer support and in a forum in the shopper app called ShopTalk. Some complain that customers are able to leave low ratings without giving any feedback, despite Instacart’s insistence that this would not be allowed. Others complain about customers who report missing orders despite proof of delivery. Many vent about how one or two poor reviews have dragged down their ratings, and in turn their earnings. There are numerous complaints that customer support is unresponsive or unhelpful, screenshots show.

Montalvo said Instacart has added thousands of agents to its customer care team since March. She said there is a team within the company’s trust and safety organization “dedicated to addressing potential instances of fraud.” Montalvo declined to say how big that team was or how many people are assigned to handle specific instances of customer fraud.

She said it is company policy to deactivate any customer engaging in fraudulent activity.

Still, customers have discovered tricks for getting refunds from Instacart and many online delivery services, and have taken to sharing their best practices online. In a subreddit called Illegal Life Pro Tips, with half a million members, users recommend the “DNA” — or “did not arrive” — method to get refunds on Instacart orders.

Some hire refund brokers to make false claims. Refund brokers don’t reveal their methods, but experts at Dark Owl, a company that monitors the dark net for corporate clients, say they abuse Instacart’s refund policy by reporting either that something is wrong with the order or that it wasn’t delivered. Certain refund brokers require customers to fill out forms providing their Instacart username and password. Receipts posted online show refunds are processed by Instacart’s customer service team. If the ruse works, brokers take a cut.

Some, though not all, brokers limit the size of orders they will challenge. One lists a $150 maximum; another shows receipts for refunds that exceeded $400. Brokers say they have higher success rates complaining about orders within three days of receiving them — the sooner the better. To ensure accounts that use their services don’t get deactivated, they recommend at least three normal orders between each attempted refund. If a customer gets more than three refunds on one service, they recommend creating a new account, or getting groceries delivered to a neighbor.

Montalvo said refund brokers are not a widespread problem. “We have a dedicated team that closely monitors activity on the platform and will deactivate individuals for fraudulent behavior,” she said.

Shoppers, however, believe they’re being penalized due to unseemly behavior from customers.

One New York contractor, who asked not to be named after her account was recently reactivated, said she lost her job until she managed to prove that a complaining customer was actually a repeat offender.

The shopper said she had communicated with the customer in the app and was buzzed into the customer’s building. When she knocked on the customer’s door, however, no one answered, so she took a picture of the order at the door with a time stamp, according to screenshots reviewed by The Times.

The shopper said she received a call from Instacart 20 minutes later saying the customer did not receive the order. She said Instacart deactivated her account soon after.

Distraught, she messaged a friend, who is also an Instacart shopper. They soon realized they both had delivered orders to the same address that were later reported missing. After submitting screenshots of the messages with her friend to Instacart, she said, the company reactivated her account.

“The customers will continue to lie to get free things and the shoppers ultimately lose their income,” Zahara said. “There are so many people on the Facebook pages complaining about the same thing happening to them.

“I just can’t see how this benefits Instacart,” she said.