---
title: "Authenticated Scraping"
description: "Have your LLM access and scrape authenticated content with AnythingLLM's Authenticated Scraping tool"
---

import { Callout } from "nextra/components";
import Image from "next/image";

<Callout type="info" emoji="🖥️">
  **Desktop Only Feature (v1.8.3+)**<br/>
  The Authenticated Scraping tool is exclusively available in the AnythingLLM Desktop application!
</Callout>

# Authenticated Scraping

<Callout type="warning" emoji="⚠️">
  **Security Note**
  All credentials and session data are stored locally on your machine. AnythingLLM never transmits or stores your login information outside of your local machine.
</Callout>

The Authenticated Scraping tool enables you to access and scrape gated online content from websites or services that require authentication but hold critical contexual content you might want to use in your workflows, such as your personal LinkedIn feed or internal company portals that you have access to.

Your LLM can now access these websites and scrape and view content just like you would in a regular browser!

<Image
  src="/images/features/browser-tool/browser-tool.png"
  height={500}
  width={500}
  quality={100}
  style={{ borderRadius: "20px", marginBottom: 10, margin: "auto" }}
/>

## Key Features

- **Secure Session Storage**: Credentials are stored locally using isolated browser sessions
- **Session Persistence**: Login sessions persist between app restarts until explicitly cleared or the authentication expires for the associated service
- **Isolated Environment**: Separate from your actual web browser
- **Full User Control**: Clear stored data or sessions at any time with a single click

## Using the Authenticated Scraping Tool

### Accessing the Tool
1. Open AnythingLLM Desktop
2. Navigate to Settings > Tools > Browser Tool
3. Click "Open Private Browser" to launch the isolated browser window

<Image
  src="/images/features/browser-tool/manager.png"
  height={1000}
  width={1000}
  quality={100}
  style={{ borderRadius: "20px", marginBottom: 10, margin: "auto" }}
/>

### Authentication Process
1. Log into your desired service (e.g., LinkedIn, Gmail) through the Authenticated Scraping tool.
2. Your session will persist until you explicitly clear the browser data or the authentication expires for the associated service.
3. AnythingLLM can now access authenticated content from these services when scraping or via agentic workflow execution.
4. The returned content will be text only. No images, videos, or other media will be returned.

<Callout type="info" emoji="💡">
  **Heads up!**
  The Authenticated Scraping tool is not a magic bullet. It is a tool that allows you to access authenticated content from websites that require authentication. It cannot currently **interact** with the content of the page you are accessing (eg: browser automation, RPA, etc).
</Callout>

### Managing Browser Data

- **Clearing Data**: Use the "Clear Browser Data" button to remove all stored credentials and sessions
- **When should you clear the browser data?**:
  - When switching between different service accounts
  - If you encounter authentication issues when the LLM tries to access the site you want to scrape

## Common Use Cases

<Callout type="warning" emoji="⚠️">
  **Warning!**
  Some web services may detect and restrict automated access, even though this tool functions as a standard browser. Use this feature responsibly and at your own discretion, as certain services may suspend or block accounts that they perceive as engaging in automated activity.
</Callout>

- Scraping your personal linkedin profile or feed.
- Accessing internal company documentation that is behind a login or SSO portal.
- Collecting or accessing data from paid or authenticated web service you have access to normally.

## Troubleshooting

If you encounter issues:
1. Clear the browser data and try again
2. Ensure you're fully logged into the service by opening the private browser and navigating to the site you want the LLM to access.

Some services have very short lived sessions, those services may require you to log in again after a certain amount of time or might be a bad use-case for this tool. You can always re-authenticate with the service by opening the private browser and navigating to the site you want the LLM to access and logging in again to refresh the session.