Wired previously discovered a Perplexity crawler that managed to bypass the Robot Exclusion Protocol.
Tl;dr
- Amazon is investigating possible non-compliance by Perplexity AI.
- Perplexity AI is suspected of ignoring the Robots Exclusion Protocol.
- Perplexity AI officials deny the allegations.
- Amazon’s cloud service is committed to investigating potential violations.
Amazon’s investigation into Perplexity AI
According to a report by Wired, Amazon Web Services (AWS) is taking a serious look at the activities of Perplexity AI. It is alleged that the company is violating AWS server usage rules by using a robot that does not comply with the Robots Exclusion Protocol, a web standard that allows developers to control the access of robots to certain pages on their domain.
Allegations against Perplexity AI
Indications suggest that the Perplexity AI service used a “crawler” hosted on AWS servers, which would override the guidelines established by the protocol. Indeed, Wired discovered a virtual machine, sneakily operated by Perplexity AI, bypassing the instructions in the robots.txt file on its site.
Between respecting the rules and denying allegations
It would be in “entering titles or short descriptions of their items into the company’s chatbot“, that Wired reportedly confirmed that Perplexity AI was indeed extracting its content. However, Perplexity spokesperson Sara Platnick denied the accusation, saying that their bots were following the protocol and were not violating AWS’s terms of service.
Amazon Web Services’ Position
In response to these allegations, AWS reaffirmed its commitment to reviewing any indications of potential abuse of its resources. The company confirmed that they regularly discuss these issues with their customers. AWS also pointed out that the latest allegations were brought to its attention by Wired. In this context, AWS highlighted, “We are reviewing the information the publication provided to us, as we do with all reports of possible violations.”
In short, the situation remains unclear, despite Perplexity AI’s denials and Amazon’s commitment to maintaining a rule-compliant environment. The ongoing investigation will therefore be decisive in establishing the truth of this matter.
